2024-03-04T13:10:19.495 INFO:root:teuthology version: 0.0.1.dev260+gb1dac55 2024-03-04T13:10:19.496 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/vshankar-2024-03-04_08:26:39-fs-wip-vshankar-testing-20240304.042522-testing-default-smithi/7581129 --name vshankar-2024-03-04_08:26:39-fs-wip-vshankar-testing-20240304.042522-testing-default-smithi --description fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} --verbose --owner scheduled_vshankar@teuthology -- /home/teuthworker/archive/vshankar-2024-03-04_08:26:39-fs-wip-vshankar-testing-20240304.042522-testing-default-smithi/7581129/orig.config.yaml 2024-03-04T13:10:19.551 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-04T13:10:19.635 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/vshankar-2024-03-04_08:26:39-fs-wip-vshankar-testing-20240304.042522-testing-default-smithi/7581129 branch: wip-vshankar-testing-20240304.042522 description: fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} email: vshankar@redhat.com first_in_suite: false job_id: '7581129' kernel: flavor: default kdb: true sha1: 786939134d7eee4dc7a805245d3f595929af37d7 last_in_suite: false machine_type: smithi meta: - desc: 'setup ceph/quincy ' name: vshankar-2024-03-04_08:26:39-fs-wip-vshankar-testing-20240304.042522-testing-default-smithi no_nested_subset: false nuke-on-error: true os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-vshankar-testing-20240304.042522 ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: mon pg warn min per osd: 0 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - OSD_DOWN - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN sha1: 717ce3cce8d0166e4e577faa8004e6f9cb4128c0 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 717ce3cce8d0166e4e577faa8004e6f9cb4128c0 kclient: syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-vshankar-testing-20240304.042522 sha1: 717ce3cce8d0166e4e577faa8004e6f9cb4128c0 owner: scheduled_vshankar@teuthology priority: 80 repo: https://git.ceph.com/ceph-ci.git roles: - - host.a - client.0 - osd.0 - osd.1 - osd.2 - - host.b - client.1 - osd.3 - osd.4 - osd.5 seed: 3119 sha1: 717ce3cce8d0166e4e577faa8004e6f9cb4128c0 sleep_before_teardown: 0 subset: 270/1024 suite: fs suite_branch: wip-vshankar-testing-20240304.042522 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_717ce3cce8d0166e4e577faa8004e6f9cb4128c0/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 717ce3cce8d0166e4e577faa8004e6f9cb4128c0 targets: smithi107.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLhjGpMlJpo1+6FPuaLmsZgZjsL41nHgbU1AOw2k0B5VtQJsWVD+BhhlM4IU1X4sUzrXnFX7jEjskdK7KCnCAVQ= smithi169.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBI/j2+/BeHyPDZdYKrYfQgkYQy9OghXD5ru6c5b5rpDDegrS8tuTyP3aqfG96F6vVqlUYb2frSFtDHdntTYU6rw= tasks: - install: branch: quincy exclude_packages: - ceph-volume - print: '**** done install task...' - cephadm: cephadm_branch: quincy cephadm_git_url: https://github.com/ceph/ceph conf: osd: osd_class_default_list: '*' osd_class_load_list: '*' image: quay.ceph.io/ceph-ci/ceph:quincy roleless: true - print: '**** done end installing quincy cephadm ...' - cephadm.shell: host.a: - ceph config set mgr mgr/cephadm/use_repo_digest true --force - print: '**** done cephadm.shell ceph config set mgr...' - cephadm.shell: host.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: host.a: - ceph fs volume create cephfs --placement=4 - ceph fs dump - cephadm.shell: host.a: - ceph fs set cephfs max_mds 1 - cephadm.shell: host.a: - ceph fs set cephfs allow_standby_replay true - cephadm.shell: host.a: - ceph fs set cephfs inline_data true --yes-i-really-really-mean-it - cephadm.shell: host.a: - ceph fs dump - ceph --format=json fs dump | jq -e ".filesystems | length == 1" - while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done - fs.pre_upgrade_save: null - ceph-fuse: null - print: '**** done client' - parallel: - upgrade-tasks - workload-tasks - cephadm.shell: host.a: - ceph fs dump - fs.post_upgrade_checks: null teuthology: fragments_dropped: [] meta: {} postmerge: [] variables: fail_fs: true teuthology_branch: main teuthology_sha1: b1dac5519c57c269ea98a22fb7729016a1d0e4d2 timestamp: 2024-03-04_08:26:39 tube: smithi upgrade-tasks: sequential: - cephadm.shell: env: - sha1 host.a: - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 --daemon-types mgr - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph orch upgrade status ; sleep 30 ; done - ceph versions | jq -e '.mgr | length == 1' - ceph versions | jq -e '.mgr | keys' | grep $sha1 - ceph versions | jq -e '.overall | length == 2' - ceph orch upgrade check quay.ceph.io/ceph-ci/ceph:$sha1 | jq -e '.up_to_date | length == 2' - ceph orch ps - cephadm.shell: env: - sha1 host.a: - ceph config set mgr mgr/orchestrator/fail_fs true - cephadm.shell: env: - sha1 host.a: - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 - cephadm.shell: env: - sha1 host.a: - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done - ceph orch ps - ceph orch upgrade status - ceph health detail - ceph versions - echo "wait for servicemap items w/ changing names to refresh" - sleep 60 - ceph orch ps - ceph versions - ceph versions | jq -e '.overall | length == 1' - ceph versions | jq -e '.overall | keys' | grep $sha1 user: vshankar verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 workload-tasks: sequential: - workunit: clients: all: - suites/fsstress.sh 2024-03-04T13:10:19.635 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_717ce3cce8d0166e4e577faa8004e6f9cb4128c0/qa; will attempt to use it 2024-03-04T13:10:19.636 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_717ce3cce8d0166e4e577faa8004e6f9cb4128c0/qa/tasks 2024-03-04T13:10:19.636 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-04T13:10:19.638 INFO:teuthology.task.internal:Checking packages... 2024-03-04T13:10:19.661 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '717ce3cce8d0166e4e577faa8004e6f9cb4128c0' 2024-03-04T13:10:19.661 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-04T13:10:19.661 INFO:teuthology.packaging:ref: None 2024-03-04T13:10:19.661 INFO:teuthology.packaging:tag: None 2024-03-04T13:10:19.661 INFO:teuthology.packaging:branch: wip-vshankar-testing-20240304.042522 2024-03-04T13:10:19.661 INFO:teuthology.packaging:sha1: 717ce3cce8d0166e4e577faa8004e6f9cb4128c0 2024-03-04T13:10:19.661 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-vshankar-testing-20240304.042522 2024-03-04T13:10:19.884 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-1845.g717ce3cc 2024-03-04T13:10:19.885 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-04T13:10:19.892 INFO:teuthology.task.internal:no buildpackages task found 2024-03-04T13:10:19.892 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-04T13:10:19.906 INFO:teuthology.task.internal:Saving configuration 2024-03-04T13:10:19.928 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-04T13:10:19.971 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-04T13:10:19.996 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi107.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-03-04_08:26:39-fs-wip-vshankar-testing-20240304.042522-testing-default-smithi/7581129', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-04 13:05:02.532842', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLhjGpMlJpo1+6FPuaLmsZgZjsL41nHgbU1AOw2k0B5VtQJsWVD+BhhlM4IU1X4sUzrXnFX7jEjskdK7KCnCAVQ='} 2024-03-04T13:10:20.021 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi169.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-03-04_08:26:39-fs-wip-vshankar-testing-20240304.042522-testing-default-smithi/7581129', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-04 13:05:02.531091', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBI/j2+/BeHyPDZdYKrYfQgkYQy9OghXD5ru6c5b5rpDDegrS8tuTyP3aqfG96F6vVqlUYb2frSFtDHdntTYU6rw='} 2024-03-04T13:10:20.022 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-04T13:10:20.027 INFO:teuthology.task.internal:roles: ubuntu@smithi107.front.sepia.ceph.com - ['host.a', 'client.0', 'osd.0', 'osd.1', 'osd.2'] 2024-03-04T13:10:20.027 INFO:teuthology.task.internal:roles: ubuntu@smithi169.front.sepia.ceph.com - ['host.b', 'client.1', 'osd.3', 'osd.4', 'osd.5'] 2024-03-04T13:10:20.027 INFO:teuthology.run_tasks:Running task console_log... 2024-03-04T13:10:20.144 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f7d0cb00160>, signals=[15]) 2024-03-04T13:10:20.144 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-04T13:10:20.149 INFO:teuthology.task.internal:Opening connections... 2024-03-04T13:10:20.150 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi107.front.sepia.ceph.com 2024-03-04T13:10:20.151 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-04T13:10:20.241 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi169.front.sepia.ceph.com 2024-03-04T13:10:20.242 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-04T13:10:20.335 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-04T13:10:20.346 DEBUG:teuthology.orchestra.run.smithi107:> uname -m 2024-03-04T13:10:20.363 INFO:teuthology.orchestra.run.smithi107.stdout:x86_64 2024-03-04T13:10:20.364 DEBUG:teuthology.orchestra.run.smithi107:> cat /etc/os-release 2024-03-04T13:10:20.419 INFO:teuthology.orchestra.run.smithi107.stdout:NAME="CentOS Stream" 2024-03-04T13:10:20.420 INFO:teuthology.orchestra.run.smithi107.stdout:VERSION="9" 2024-03-04T13:10:20.420 INFO:teuthology.orchestra.run.smithi107.stdout:ID="centos" 2024-03-04T13:10:20.420 INFO:teuthology.orchestra.run.smithi107.stdout:ID_LIKE="rhel fedora" 2024-03-04T13:10:20.420 INFO:teuthology.orchestra.run.smithi107.stdout:VERSION_ID="9" 2024-03-04T13:10:20.420 INFO:teuthology.orchestra.run.smithi107.stdout:PLATFORM_ID="platform:el9" 2024-03-04T13:10:20.420 INFO:teuthology.orchestra.run.smithi107.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-04T13:10:20.420 INFO:teuthology.orchestra.run.smithi107.stdout:ANSI_COLOR="0;31" 2024-03-04T13:10:20.420 INFO:teuthology.orchestra.run.smithi107.stdout:LOGO="fedora-logo-icon" 2024-03-04T13:10:20.420 INFO:teuthology.orchestra.run.smithi107.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-04T13:10:20.420 INFO:teuthology.orchestra.run.smithi107.stdout:HOME_URL="https://centos.org/" 2024-03-04T13:10:20.420 INFO:teuthology.orchestra.run.smithi107.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-04T13:10:20.420 INFO:teuthology.orchestra.run.smithi107.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-04T13:10:20.421 INFO:teuthology.orchestra.run.smithi107.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-04T13:10:20.421 INFO:teuthology.lock.ops:Updating smithi107.front.sepia.ceph.com on lock server 2024-03-04T13:10:20.445 DEBUG:teuthology.orchestra.run.smithi169:> uname -m 2024-03-04T13:10:20.463 INFO:teuthology.orchestra.run.smithi169.stdout:x86_64 2024-03-04T13:10:20.463 DEBUG:teuthology.orchestra.run.smithi169:> cat /etc/os-release 2024-03-04T13:10:20.520 INFO:teuthology.orchestra.run.smithi169.stdout:NAME="CentOS Stream" 2024-03-04T13:10:20.520 INFO:teuthology.orchestra.run.smithi169.stdout:VERSION="9" 2024-03-04T13:10:20.521 INFO:teuthology.orchestra.run.smithi169.stdout:ID="centos" 2024-03-04T13:10:20.521 INFO:teuthology.orchestra.run.smithi169.stdout:ID_LIKE="rhel fedora" 2024-03-04T13:10:20.521 INFO:teuthology.orchestra.run.smithi169.stdout:VERSION_ID="9" 2024-03-04T13:10:20.521 INFO:teuthology.orchestra.run.smithi169.stdout:PLATFORM_ID="platform:el9" 2024-03-04T13:10:20.521 INFO:teuthology.orchestra.run.smithi169.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-04T13:10:20.521 INFO:teuthology.orchestra.run.smithi169.stdout:ANSI_COLOR="0;31" 2024-03-04T13:10:20.521 INFO:teuthology.orchestra.run.smithi169.stdout:LOGO="fedora-logo-icon" 2024-03-04T13:10:20.521 INFO:teuthology.orchestra.run.smithi169.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-04T13:10:20.521 INFO:teuthology.orchestra.run.smithi169.stdout:HOME_URL="https://centos.org/" 2024-03-04T13:10:20.522 INFO:teuthology.orchestra.run.smithi169.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-04T13:10:20.522 INFO:teuthology.orchestra.run.smithi169.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-04T13:10:20.522 INFO:teuthology.orchestra.run.smithi169.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-04T13:10:20.522 INFO:teuthology.lock.ops:Updating smithi169.front.sepia.ceph.com on lock server 2024-03-04T13:10:20.545 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-04T13:10:20.557 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-04T13:10:20.563 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-04T13:10:20.563 DEBUG:teuthology.orchestra.run.smithi107:> test '!' -e /home/ubuntu/cephtest 2024-03-04T13:10:20.566 DEBUG:teuthology.orchestra.run.smithi169:> test '!' -e /home/ubuntu/cephtest 2024-03-04T13:10:20.582 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-04T13:10:20.588 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-04T13:10:20.588 DEBUG:teuthology.orchestra.run.smithi107:> test -z $(ls -A /var/lib/ceph) 2024-03-04T13:10:20.623 DEBUG:teuthology.orchestra.run.smithi169:> test -z $(ls -A /var/lib/ceph) 2024-03-04T13:10:20.721 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-04T13:10:20.774 INFO:teuthology.run_tasks:Running task kernel... 2024-03-04T13:10:20.789 INFO:teuthology.task.kernel:normalize config orig: {'flavor': 'default', 'kdb': True, 'sha1': '786939134d7eee4dc7a805245d3f595929af37d7'} 2024-03-04T13:10:20.789 INFO:teuthology.task.kernel:config {'host.a': {'flavor': 'default', 'kdb': True, 'sha1': '786939134d7eee4dc7a805245d3f595929af37d7'}, 'host.b': {'flavor': 'default', 'kdb': True, 'sha1': '786939134d7eee4dc7a805245d3f595929af37d7'}}, timeout 300 2024-03-04T13:10:20.790 DEBUG:teuthology.orchestra.run.smithi107:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-04T13:10:20.790 DEBUG:teuthology.orchestra.run.smithi169:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-04T13:10:20.806 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-04T13:10:20.806 DEBUG:teuthology.task.kernel:sha1 for host.a is 786939134d7eee4dc7a805245d3f595929af37d7 2024-03-04T13:10:20.806 INFO:teuthology.task.kernel:Checking kernel version of host.a, want "786939134d7eee4dc7a805245d3f595929af37d7"... 2024-03-04T13:10:20.806 DEBUG:teuthology.orchestra.run.smithi107:> uname -r 2024-03-04T13:10:20.807 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-04T13:10:20.807 DEBUG:teuthology.task.kernel:sha1 for host.b is 786939134d7eee4dc7a805245d3f595929af37d7 2024-03-04T13:10:20.808 INFO:teuthology.task.kernel:Checking kernel version of host.b, want "786939134d7eee4dc7a805245d3f595929af37d7"... 2024-03-04T13:10:20.808 DEBUG:teuthology.orchestra.run.smithi169:> uname -r 2024-03-04T13:10:20.863 INFO:teuthology.orchestra.run.smithi107.stdout:5.14.0-361.el9.x86_64 2024-03-04T13:10:20.863 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs 786939134d7eee4dc7a805245d3f595929af37d7 2024-03-04T13:10:20.863 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-03-04T13:10:20.863 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=786939134d7eee4dc7a805245d3f595929af37d7 2024-03-04T13:10:20.867 INFO:teuthology.orchestra.run.smithi169.stdout:5.14.0-361.el9.x86_64 2024-03-04T13:10:20.867 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs 786939134d7eee4dc7a805245d3f595929af37d7 2024-03-04T13:10:20.867 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-03-04T13:10:20.867 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=786939134d7eee4dc7a805245d3f595929af37d7 2024-03-04T13:10:21.016 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum upgrade -y linux-firmware 2024-03-04T13:10:21.020 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum upgrade -y linux-firmware 2024-03-04T13:10:23.164 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS-9 - AppStream 13 MB/s | 6.7 MB 00:00 2024-03-04T13:10:23.261 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS-9 - AppStream 11 MB/s | 6.7 MB 00:00 2024-03-04T13:10:24.548 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS-9 - BaseOS 7.4 MB/s | 2.0 MB 00:00 2024-03-04T13:10:24.661 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS-9 - BaseOS 6.8 MB/s | 2.0 MB 00:00 2024-03-04T13:10:26.085 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - BaseOS 7.3 MB/s | 8.0 MB 00:01 2024-03-04T13:10:26.321 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS Stream 9 - BaseOS 5.6 MB/s | 8.0 MB 00:01 2024-03-04T13:10:28.357 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS Stream 9 - AppStream 20 MB/s | 19 MB 00:00 2024-03-04T13:10:36.376 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - AppStream 2.1 MB/s | 19 MB 00:09 2024-03-04T13:10:38.139 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS Stream 9 - CRB 1.0 MB/s | 6.1 MB 00:06 2024-03-04T13:10:39.723 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS Stream 9 - Extras packages 32 kB/s | 15 kB 00:00 2024-03-04T13:10:45.743 INFO:teuthology.orchestra.run.smithi169.stdout:Extra Packages for Enterprise Linux 3.5 MB/s | 20 MB 00:05 2024-03-04T13:10:49.991 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - CRB 623 kB/s | 6.1 MB 00:10 2024-03-04T13:10:50.648 INFO:teuthology.orchestra.run.smithi169.stdout:lab-extras 15 kB/s | 1.7 kB 00:00 2024-03-04T13:10:51.958 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - Extras packages 18 kB/s | 15 kB 00:00 2024-03-04T13:10:52.940 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2024-03-04T13:10:52.951 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-03-04T13:10:52.951 INFO:teuthology.orchestra.run.smithi169.stdout: Package Arch Version Repository Size 2024-03-04T13:10:52.951 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-03-04T13:10:52.951 INFO:teuthology.orchestra.run.smithi169.stdout:Upgrading: 2024-03-04T13:10:52.951 INFO:teuthology.orchestra.run.smithi169.stdout: linux-firmware noarch 20240219-143.el9 CentOS-BaseOS 375 M 2024-03-04T13:10:52.951 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-03-04T13:10:52.951 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction Summary 2024-03-04T13:10:52.952 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-03-04T13:10:52.952 INFO:teuthology.orchestra.run.smithi169.stdout:Upgrade 1 Package 2024-03-04T13:10:52.952 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-03-04T13:10:52.952 INFO:teuthology.orchestra.run.smithi169.stdout:Total download size: 375 M 2024-03-04T13:10:52.952 INFO:teuthology.orchestra.run.smithi169.stdout:Downloading Packages: 2024-03-04T13:11:10.098 INFO:teuthology.orchestra.run.smithi107.stdout:Extra Packages for Enterprise Linux 1.1 MB/s | 20 MB 00:18 2024-03-04T13:11:15.006 INFO:teuthology.orchestra.run.smithi107.stdout:lab-extras 11 kB/s | 1.7 kB 00:00 2024-03-04T13:11:17.386 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-03-04T13:11:17.401 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-03-04T13:11:17.402 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repository Size 2024-03-04T13:11:17.402 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-03-04T13:11:17.402 INFO:teuthology.orchestra.run.smithi107.stdout:Upgrading: 2024-03-04T13:11:17.402 INFO:teuthology.orchestra.run.smithi107.stdout: linux-firmware noarch 20240219-143.el9 CentOS-BaseOS 375 M 2024-03-04T13:11:17.402 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:11:17.402 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-03-04T13:11:17.402 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-03-04T13:11:17.402 INFO:teuthology.orchestra.run.smithi107.stdout:Upgrade 1 Package 2024-03-04T13:11:17.402 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:11:17.402 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 375 M 2024-03-04T13:11:17.402 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-03-04T13:11:20.888 INFO:teuthology.orchestra.run.smithi169.stdout:linux-firmware-20240219-143.el9.noarch.rpm 13 MB/s | 375 MB 00:27 2024-03-04T13:11:20.889 INFO:teuthology.orchestra.run.smithi169.stdout:-------------------------------------------------------------------------------- 2024-03-04T13:11:20.889 INFO:teuthology.orchestra.run.smithi169.stdout:Total 13 MB/s | 375 MB 00:27 2024-03-04T13:11:20.889 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction check 2024-03-04T13:11:20.989 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction check succeeded. 2024-03-04T13:11:20.990 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction test 2024-03-04T13:11:23.356 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction test succeeded. 2024-03-04T13:11:23.356 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction 2024-03-04T13:11:28.772 INFO:teuthology.orchestra.run.smithi107.stdout:linux-firmware-20240219-143.el9.noarch.rpm 33 MB/s | 375 MB 00:11 2024-03-04T13:11:28.773 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-03-04T13:11:28.773 INFO:teuthology.orchestra.run.smithi107.stdout:Total 33 MB/s | 375 MB 00:11 2024-03-04T13:11:28.773 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-03-04T13:11:28.870 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-03-04T13:11:28.870 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-03-04T13:11:29.626 INFO:teuthology.orchestra.run.smithi169.stdout: Preparing : 1/1 2024-03-04T13:11:29.857 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-04T13:11:29.946 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-04T13:11:30.945 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2024-03-04T13:11:30.945 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-04T13:11:31.338 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-03-04T13:11:31.338 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-03-04T13:11:31.371 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-04T13:11:31.371 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-03-04T13:11:31.371 INFO:teuthology.orchestra.run.smithi169.stdout:Upgraded: 2024-03-04T13:11:31.372 INFO:teuthology.orchestra.run.smithi169.stdout: linux-firmware-20240219-143.el9.noarch 2024-03-04T13:11:31.372 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-03-04T13:11:31.372 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2024-03-04T13:11:31.668 INFO:teuthology.task.kernel:Downloading kernel 786939134d7eee4dc7a805245d3f595929af37d7 on host.b... 2024-03-04T13:11:31.668 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=786939134d7eee4dc7a805245d3f595929af37d7 2024-03-04T13:11:31.823 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/786939134d7eee4dc7a805245d3f595929af37d7/centos/9/flavors/default/x86_64/ 2024-03-04T13:11:31.824 DEBUG:teuthology.orchestra.run.smithi169:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.8.0_rc6_g786939134d7e-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/786939134d7eee4dc7a805245d3f595929af37d7/centos/9/flavors/default/x86_64/ --input-file=- 2024-03-04T13:11:31.826 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi169.front.sepia.ceph.com to complete... 2024-03-04T13:11:37.760 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-03-04T13:11:37.954 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-04T13:11:38.053 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-04T13:11:39.289 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2024-03-04T13:11:39.289 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-04T13:11:39.691 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-04T13:11:39.692 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:11:39.692 INFO:teuthology.orchestra.run.smithi107.stdout:Upgraded: 2024-03-04T13:11:39.692 INFO:teuthology.orchestra.run.smithi107.stdout: linux-firmware-20240219-143.el9.noarch 2024-03-04T13:11:39.692 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:11:39.692 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-03-04T13:11:39.981 INFO:teuthology.task.kernel:Downloading kernel 786939134d7eee4dc7a805245d3f595929af37d7 on host.a... 2024-03-04T13:11:39.981 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=786939134d7eee4dc7a805245d3f595929af37d7 2024-03-04T13:11:40.127 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/786939134d7eee4dc7a805245d3f595929af37d7/centos/9/flavors/default/x86_64/ 2024-03-04T13:11:40.127 DEBUG:teuthology.orchestra.run.smithi107:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.8.0_rc6_g786939134d7e-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/786939134d7eee4dc7a805245d3f595929af37d7/centos/9/flavors/default/x86_64/ --input-file=- 2024-03-04T13:11:40.130 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi107.front.sepia.ceph.com to complete... 2024-03-04T13:11:45.058 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04 13:11:45 URL:https://3.chacra.ceph.com/r/kernel/testing/786939134d7eee4dc7a805245d3f595929af37d7/centos/9/flavors/default/x86_64/kernel-6.8.0_rc6_g786939134d7e-1.x86_64.rpm [336049686/336049686] -> "/tmp/kernel.x86_64.rpm" [1] 2024-03-04T13:11:45.059 INFO:teuthology.orchestra.run.smithi169.stderr:FINISHED --2024-03-04 13:11:45-- 2024-03-04T13:11:45.059 INFO:teuthology.orchestra.run.smithi169.stderr:Total wall clock time: 13s 2024-03-04T13:11:45.059 INFO:teuthology.orchestra.run.smithi169.stderr:Downloaded: 1 files, 320M in 13s (24.8 MB/s) 2024-03-04T13:11:45.061 INFO:teuthology.task.kernel:Installing kernel 786939134d7eee4dc7a805245d3f595929af37d7 on host.b... 2024-03-04T13:11:45.061 DEBUG:teuthology.orchestra.run.smithi169:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-03-04T13:11:46.991 INFO:teuthology.orchestra.run.smithi169.stdout:Verifying... ######################################## 2024-03-04T13:11:47.048 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing... ######################################## 2024-03-04T13:11:47.127 INFO:teuthology.orchestra.run.smithi169.stdout:Updating / installing... 2024-03-04T13:11:54.631 INFO:teuthology.orchestra.run.smithi169.stdout:kernel-6.8.0_rc6_g786939134d7e-1 ######################################## 2024-03-04T13:12:11.167 INFO:teuthology.orchestra.run.smithi169.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-04T13:12:33.550 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi169.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-03-04T13:12:33.550 DEBUG:teuthology.orchestra.run.smithi169:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-03-04T13:12:33.669 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e 2024-03-04T13:12:33.669 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/System.map 2024-03-04T13:12:33.669 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/config 2024-03-04T13:12:33.669 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel 2024-03-04T13:12:33.669 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/arch 2024-03-04T13:12:33.669 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/arch/x86 2024-03-04T13:12:33.669 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/arch/x86/crypto 2024-03-04T13:12:33.669 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-03-04T13:12:33.669 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-03-04T13:12:33.669 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-03-04T13:12:33.669 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-03-04T13:12:33.669 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-03-04T13:12:33.670 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-03-04T13:12:33.670 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-03-04T13:12:33.670 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/arch/x86/crypto/crc32c-intel.ko 2024-03-04T13:12:33.670 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-03-04T13:12:33.670 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-03-04T13:12:33.670 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-03-04T13:12:33.670 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-03-04T13:12:33.670 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/arch/x86/crypto/serpent-avx2.ko 2024-03-04T13:12:33.670 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-03-04T13:12:33.670 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-03-04T13:12:33.670 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-03-04T13:12:33.670 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-03-04T13:12:33.670 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-03-04T13:12:33.670 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-03-04T13:12:33.671 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-03-04T13:12:33.671 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/arch/x86/kernel 2024-03-04T13:12:33.671 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/arch/x86/kernel/cpu 2024-03-04T13:12:33.671 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/arch/x86/kernel/cpu/mce 2024-03-04T13:12:33.671 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-03-04T13:12:33.671 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/arch/x86/kvm 2024-03-04T13:12:33.671 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/arch/x86/kvm/kvm-amd.ko 2024-03-04T13:12:33.671 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/arch/x86/kvm/kvm-intel.ko 2024-03-04T13:12:33.671 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/arch/x86/kvm/kvm.ko 2024-03-04T13:12:33.671 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto 2024-03-04T13:12:33.671 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/ansi_cprng.ko 2024-03-04T13:12:33.671 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/anubis.ko 2024-03-04T13:12:33.671 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/arc4.ko 2024-03-04T13:12:33.671 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/async_tx 2024-03-04T13:12:33.672 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/async_tx/async_memcpy.ko 2024-03-04T13:12:33.672 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/async_tx/async_pq.ko 2024-03-04T13:12:33.672 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/async_tx/async_raid6_recov.ko 2024-03-04T13:12:33.672 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/async_tx/async_tx.ko 2024-03-04T13:12:33.672 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/async_tx/async_xor.ko 2024-03-04T13:12:33.672 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/async_tx/raid6test.ko 2024-03-04T13:12:33.672 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/authenc.ko 2024-03-04T13:12:33.672 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/authencesn.ko 2024-03-04T13:12:33.672 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/blake2b_generic.ko 2024-03-04T13:12:33.672 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/blowfish_common.ko 2024-03-04T13:12:33.672 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/blowfish_generic.ko 2024-03-04T13:12:33.672 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/camellia_generic.ko 2024-03-04T13:12:33.672 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/cast5_generic.ko 2024-03-04T13:12:33.672 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/cast6_generic.ko 2024-03-04T13:12:33.673 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/cast_common.ko 2024-03-04T13:12:33.673 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/ccm.ko 2024-03-04T13:12:33.673 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/cmac.ko 2024-03-04T13:12:33.673 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/crc32_generic.ko 2024-03-04T13:12:33.673 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/crypto_engine.ko 2024-03-04T13:12:33.673 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/crypto_user.ko 2024-03-04T13:12:33.673 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/cts.ko 2024-03-04T13:12:33.673 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/des_generic.ko 2024-03-04T13:12:33.673 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/dh_generic.ko 2024-03-04T13:12:33.673 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/echainiv.ko 2024-03-04T13:12:33.673 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/essiv.ko 2024-03-04T13:12:33.673 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/fcrypt.ko 2024-03-04T13:12:33.673 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/khazad.ko 2024-03-04T13:12:33.673 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/lz4.ko 2024-03-04T13:12:33.674 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/lz4hc.ko 2024-03-04T13:12:33.674 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/md4.ko 2024-03-04T13:12:33.674 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/michael_mic.ko 2024-03-04T13:12:33.674 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/pcbc.ko 2024-03-04T13:12:33.674 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/pcrypt.ko 2024-03-04T13:12:33.674 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/rmd160.ko 2024-03-04T13:12:33.674 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/seed.ko 2024-03-04T13:12:33.674 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/serpent_generic.ko 2024-03-04T13:12:33.674 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/tcrypt.ko 2024-03-04T13:12:33.674 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/tea.ko 2024-03-04T13:12:33.674 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/twofish_common.ko 2024-03-04T13:12:33.674 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/twofish_generic.ko 2024-03-04T13:12:33.674 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/vmac.ko 2024-03-04T13:12:33.674 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/wp512.ko 2024-03-04T13:12:33.675 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/xcbc.ko 2024-03-04T13:12:33.675 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/xor.ko 2024-03-04T13:12:33.675 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/xxhash_generic.ko 2024-03-04T13:12:33.675 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers 2024-03-04T13:12:33.675 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/acpi 2024-03-04T13:12:33.675 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/acpi/acpi_ipmi.ko 2024-03-04T13:12:33.675 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/acpi/acpi_pad.ko 2024-03-04T13:12:33.675 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/acpi/custom_method.ko 2024-03-04T13:12:33.675 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/acpi/ec_sys.ko 2024-03-04T13:12:33.675 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/acpi/platform_profile.ko 2024-03-04T13:12:33.675 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/acpi/sbs.ko 2024-03-04T13:12:33.675 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/acpi/sbshc.ko 2024-03-04T13:12:33.675 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/acpi/video.ko 2024-03-04T13:12:33.675 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata 2024-03-04T13:12:33.676 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/acard-ahci.ko 2024-03-04T13:12:33.676 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/ahci_platform.ko 2024-03-04T13:12:33.676 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/ata_generic.ko 2024-03-04T13:12:33.676 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/libahci_platform.ko 2024-03-04T13:12:33.676 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_acpi.ko 2024-03-04T13:12:33.676 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_ali.ko 2024-03-04T13:12:33.676 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_amd.ko 2024-03-04T13:12:33.676 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_artop.ko 2024-03-04T13:12:33.676 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_atiixp.ko 2024-03-04T13:12:33.676 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_atp867x.ko 2024-03-04T13:12:33.676 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_cmd640.ko 2024-03-04T13:12:33.676 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_cmd64x.ko 2024-03-04T13:12:33.676 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_cypress.ko 2024-03-04T13:12:33.676 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_efar.ko 2024-03-04T13:12:33.677 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_hpt366.ko 2024-03-04T13:12:33.677 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_hpt37x.ko 2024-03-04T13:12:33.677 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_hpt3x2n.ko 2024-03-04T13:12:33.677 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_hpt3x3.ko 2024-03-04T13:12:33.677 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_it8213.ko 2024-03-04T13:12:33.677 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_it821x.ko 2024-03-04T13:12:33.677 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_jmicron.ko 2024-03-04T13:12:33.677 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_marvell.ko 2024-03-04T13:12:33.677 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_mpiix.ko 2024-03-04T13:12:33.677 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_netcell.ko 2024-03-04T13:12:33.677 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_ninja32.ko 2024-03-04T13:12:33.677 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_ns87410.ko 2024-03-04T13:12:33.677 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_ns87415.ko 2024-03-04T13:12:33.677 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_oldpiix.ko 2024-03-04T13:12:33.678 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_opti.ko 2024-03-04T13:12:33.678 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_optidma.ko 2024-03-04T13:12:33.678 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_pcmcia.ko 2024-03-04T13:12:33.678 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_pdc2027x.ko 2024-03-04T13:12:33.678 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-03-04T13:12:33.678 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_piccolo.ko 2024-03-04T13:12:33.678 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_rdc.ko 2024-03-04T13:12:33.678 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_sch.ko 2024-03-04T13:12:33.678 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_serverworks.ko 2024-03-04T13:12:33.678 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_sil680.ko 2024-03-04T13:12:33.678 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_sis.ko 2024-03-04T13:12:33.678 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_sl82c105.ko 2024-03-04T13:12:33.678 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_triflex.ko 2024-03-04T13:12:33.678 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_via.ko 2024-03-04T13:12:33.678 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pdc_adma.ko 2024-03-04T13:12:33.679 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/sata_inic162x.ko 2024-03-04T13:12:33.679 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/sata_mv.ko 2024-03-04T13:12:33.679 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/sata_nv.ko 2024-03-04T13:12:33.679 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/sata_promise.ko 2024-03-04T13:12:33.679 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/sata_qstor.ko 2024-03-04T13:12:33.679 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/sata_sil.ko 2024-03-04T13:12:33.679 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/sata_sil24.ko 2024-03-04T13:12:33.679 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/sata_sis.ko 2024-03-04T13:12:33.679 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/sata_svw.ko 2024-03-04T13:12:33.679 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/sata_sx4.ko 2024-03-04T13:12:33.679 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/sata_uli.ko 2024-03-04T13:12:33.679 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/sata_via.ko 2024-03-04T13:12:33.679 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/sata_vsc.ko 2024-03-04T13:12:33.679 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/auxdisplay 2024-03-04T13:12:33.679 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/auxdisplay/ks0108.ko 2024-03-04T13:12:33.680 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/base 2024-03-04T13:12:33.680 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/base/regmap 2024-03-04T13:12:33.680 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/base/regmap/regmap-i2c.ko 2024-03-04T13:12:33.680 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/base/regmap/regmap-mmio.ko 2024-03-04T13:12:33.680 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/bcma 2024-03-04T13:12:33.680 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/bcma/bcma.ko 2024-03-04T13:12:33.680 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/block 2024-03-04T13:12:33.680 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/block/aoe 2024-03-04T13:12:33.680 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/block/aoe/aoe.ko 2024-03-04T13:12:33.680 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/block/brd.ko 2024-03-04T13:12:33.680 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/block/drbd 2024-03-04T13:12:33.680 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/block/drbd/drbd.ko 2024-03-04T13:12:33.680 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/block/floppy.ko 2024-03-04T13:12:33.680 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/block/loop.ko 2024-03-04T13:12:33.680 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/block/mtip32xx 2024-03-04T13:12:33.681 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-03-04T13:12:33.681 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/block/nbd.ko 2024-03-04T13:12:33.681 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/block/null_blk 2024-03-04T13:12:33.681 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/block/null_blk/null_blk.ko 2024-03-04T13:12:33.681 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/block/pktcdvd.ko 2024-03-04T13:12:33.681 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/block/rbd.ko 2024-03-04T13:12:33.681 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/block/virtio_blk.ko 2024-03-04T13:12:33.681 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/block/xen-blkback 2024-03-04T13:12:33.681 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-03-04T13:12:33.681 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/block/xen-blkfront.ko 2024-03-04T13:12:33.681 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/block/zram 2024-03-04T13:12:33.681 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/block/zram/zram.ko 2024-03-04T13:12:33.681 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/char 2024-03-04T13:12:33.681 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/char/hangcheck-timer.ko 2024-03-04T13:12:33.682 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/char/hw_random 2024-03-04T13:12:33.682 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/char/hw_random/amd-rng.ko 2024-03-04T13:12:33.682 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/char/hw_random/intel-rng.ko 2024-03-04T13:12:33.682 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-03-04T13:12:33.682 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/char/hw_random/via-rng.ko 2024-03-04T13:12:33.682 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/char/hw_random/virtio-rng.ko 2024-03-04T13:12:33.682 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/char/ipmi 2024-03-04T13:12:33.682 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-03-04T13:12:33.682 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-03-04T13:12:33.682 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-03-04T13:12:33.682 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/char/ipmi/ipmi_si.ko 2024-03-04T13:12:33.682 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-03-04T13:12:33.682 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-03-04T13:12:33.682 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/char/lp.ko 2024-03-04T13:12:33.682 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/char/mwave 2024-03-04T13:12:33.683 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/char/mwave/mwave.ko 2024-03-04T13:12:33.683 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/char/ppdev.ko 2024-03-04T13:12:33.683 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/char/tlclk.ko 2024-03-04T13:12:33.683 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/char/tpm 2024-03-04T13:12:33.683 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/char/tpm/tpm.ko 2024-03-04T13:12:33.683 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/char/tpm/tpm_atmel.ko 2024-03-04T13:12:33.683 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/char/tpm/tpm_infineon.ko 2024-03-04T13:12:33.683 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/char/tpm/tpm_nsc.ko 2024-03-04T13:12:33.683 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/char/tpm/tpm_tis.ko 2024-03-04T13:12:33.683 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-03-04T13:12:33.683 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/char/uv_mmtimer.ko 2024-03-04T13:12:33.683 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/char/virtio_console.ko 2024-03-04T13:12:33.683 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/cpufreq 2024-03-04T13:12:33.683 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-03-04T13:12:33.683 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-03-04T13:12:33.684 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/cpufreq/p4-clockmod.ko 2024-03-04T13:12:33.684 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-03-04T13:12:33.684 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/cpufreq/powernow-k8.ko 2024-03-04T13:12:33.684 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/cpufreq/speedstep-lib.ko 2024-03-04T13:12:33.684 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/crypto 2024-03-04T13:12:33.684 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/crypto/ccp 2024-03-04T13:12:33.684 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-03-04T13:12:33.684 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/crypto/ccp/ccp.ko 2024-03-04T13:12:33.684 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/crypto/intel 2024-03-04T13:12:33.684 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/crypto/intel/qat 2024-03-04T13:12:33.684 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/crypto/intel/qat/qat_common 2024-03-04T13:12:33.684 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-03-04T13:12:33.684 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-03-04T13:12:33.684 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-03-04T13:12:33.685 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/crypto/padlock-aes.ko 2024-03-04T13:12:33.685 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/crypto/padlock-sha.ko 2024-03-04T13:12:33.685 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/crypto/virtio 2024-03-04T13:12:33.685 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-03-04T13:12:33.685 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/dca 2024-03-04T13:12:33.685 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/dca/dca.ko 2024-03-04T13:12:33.685 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/devfreq 2024-03-04T13:12:33.685 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-03-04T13:12:33.685 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/dma 2024-03-04T13:12:33.685 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/dma/dw 2024-03-04T13:12:33.685 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/dma/dw/dw_dmac.ko 2024-03-04T13:12:33.685 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/dma/ioat 2024-03-04T13:12:33.685 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/dma/ioat/ioatdma.ko 2024-03-04T13:12:33.685 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/edac 2024-03-04T13:12:33.685 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/edac/amd64_edac.ko 2024-03-04T13:12:33.686 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/edac/e752x_edac.ko 2024-03-04T13:12:33.686 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/edac/edac_mce_amd.ko 2024-03-04T13:12:33.686 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/edac/i3000_edac.ko 2024-03-04T13:12:33.686 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/edac/i3200_edac.ko 2024-03-04T13:12:33.686 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/edac/i5100_edac.ko 2024-03-04T13:12:33.686 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/edac/i5400_edac.ko 2024-03-04T13:12:33.686 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/edac/i7300_edac.ko 2024-03-04T13:12:33.686 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/edac/i7core_edac.ko 2024-03-04T13:12:33.686 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/edac/i82975x_edac.ko 2024-03-04T13:12:33.686 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/edac/ie31200_edac.ko 2024-03-04T13:12:33.686 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/edac/sb_edac.ko 2024-03-04T13:12:33.686 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/edac/x38_edac.ko 2024-03-04T13:12:33.686 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/firmware 2024-03-04T13:12:33.686 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/firmware/edd.ko 2024-03-04T13:12:33.687 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/firmware/iscsi_ibft.ko 2024-03-04T13:12:33.687 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/gpio 2024-03-04T13:12:33.687 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/gpio/gpio-ich.ko 2024-03-04T13:12:33.687 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/gpio/gpio-viperboard.ko 2024-03-04T13:12:33.687 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid 2024-03-04T13:12:33.687 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-appleir.ko 2024-03-04T13:12:33.687 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-aureal.ko 2024-03-04T13:12:33.687 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-axff.ko 2024-03-04T13:12:33.687 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-dr.ko 2024-03-04T13:12:33.687 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-elecom.ko 2024-03-04T13:12:33.687 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-elo.ko 2024-03-04T13:12:33.687 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-emsff.ko 2024-03-04T13:12:33.687 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-gaff.ko 2024-03-04T13:12:33.687 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-gt683r.ko 2024-03-04T13:12:33.687 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-gyration.ko 2024-03-04T13:12:33.688 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-holtek-kbd.ko 2024-03-04T13:12:33.688 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-holtek-mouse.ko 2024-03-04T13:12:33.688 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-holtekff.ko 2024-03-04T13:12:33.688 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-hyperv.ko 2024-03-04T13:12:33.688 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-icade.ko 2024-03-04T13:12:33.688 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-keytouch.ko 2024-03-04T13:12:33.688 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-kye.ko 2024-03-04T13:12:33.688 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-lcpower.ko 2024-03-04T13:12:33.688 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-led.ko 2024-03-04T13:12:33.688 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-lenovo.ko 2024-03-04T13:12:33.688 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-logitech-dj.ko 2024-03-04T13:12:33.688 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-03-04T13:12:33.688 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-multitouch.ko 2024-03-04T13:12:33.688 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-ortek.ko 2024-03-04T13:12:33.688 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-penmount.ko 2024-03-04T13:12:33.689 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-petalynx.ko 2024-03-04T13:12:33.689 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-picolcd.ko 2024-03-04T13:12:33.689 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-pl.ko 2024-03-04T13:12:33.689 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-plantronics.ko 2024-03-04T13:12:33.689 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-primax.ko 2024-03-04T13:12:33.689 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-rmi.ko 2024-03-04T13:12:33.689 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-roccat-arvo.ko 2024-03-04T13:12:33.689 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-roccat-common.ko 2024-03-04T13:12:33.689 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-roccat-isku.ko 2024-03-04T13:12:33.689 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-roccat-kone.ko 2024-03-04T13:12:33.689 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-03-04T13:12:33.689 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-roccat-konepure.ko 2024-03-04T13:12:33.689 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-03-04T13:12:33.689 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-roccat-lua.ko 2024-03-04T13:12:33.690 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-roccat-pyra.ko 2024-03-04T13:12:33.690 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-roccat-ryos.ko 2024-03-04T13:12:33.690 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-roccat-savu.ko 2024-03-04T13:12:33.690 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-roccat.ko 2024-03-04T13:12:33.690 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-saitek.ko 2024-03-04T13:12:33.690 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-samsung.ko 2024-03-04T13:12:33.690 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-sensor-hub.ko 2024-03-04T13:12:33.690 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-sjoy.ko 2024-03-04T13:12:33.690 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-sony.ko 2024-03-04T13:12:33.690 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-speedlink.ko 2024-03-04T13:12:33.690 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-steelseries.ko 2024-03-04T13:12:33.690 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-sunplus.ko 2024-03-04T13:12:33.690 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-thrustmaster.ko 2024-03-04T13:12:33.690 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-tivo.ko 2024-03-04T13:12:33.691 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-tmff.ko 2024-03-04T13:12:33.691 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-topseed.ko 2024-03-04T13:12:33.691 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-twinhan.ko 2024-03-04T13:12:33.691 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-uclogic.ko 2024-03-04T13:12:33.691 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-waltop.ko 2024-03-04T13:12:33.691 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-wiimote.ko 2024-03-04T13:12:33.691 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-xinmo.ko 2024-03-04T13:12:33.691 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-zpff.ko 2024-03-04T13:12:33.691 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-zydacron.ko 2024-03-04T13:12:33.691 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/uhid.ko 2024-03-04T13:12:33.691 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/wacom.ko 2024-03-04T13:12:33.691 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hv 2024-03-04T13:12:33.691 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hv/hv_balloon.ko 2024-03-04T13:12:33.691 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hv/hv_utils.ko 2024-03-04T13:12:33.691 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hv/hv_vmbus.ko 2024-03-04T13:12:33.692 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon 2024-03-04T13:12:33.692 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/abituguru.ko 2024-03-04T13:12:33.692 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/abituguru3.ko 2024-03-04T13:12:33.692 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/acpi_power_meter.ko 2024-03-04T13:12:33.692 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/ad7414.ko 2024-03-04T13:12:33.692 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/ad7418.ko 2024-03-04T13:12:33.692 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/adc128d818.ko 2024-03-04T13:12:33.692 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/adm1025.ko 2024-03-04T13:12:33.692 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/adm1026.ko 2024-03-04T13:12:33.692 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/adm1029.ko 2024-03-04T13:12:33.692 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/adm1031.ko 2024-03-04T13:12:33.692 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/adm9240.ko 2024-03-04T13:12:33.692 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/ads7828.ko 2024-03-04T13:12:33.692 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/adt7410.ko 2024-03-04T13:12:33.693 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/adt7411.ko 2024-03-04T13:12:33.693 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/adt7462.ko 2024-03-04T13:12:33.693 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/adt7470.ko 2024-03-04T13:12:33.693 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/adt7475.ko 2024-03-04T13:12:33.693 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/adt7x10.ko 2024-03-04T13:12:33.693 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/amc6821.ko 2024-03-04T13:12:33.693 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/applesmc.ko 2024-03-04T13:12:33.693 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/asb100.ko 2024-03-04T13:12:33.693 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/asc7621.ko 2024-03-04T13:12:33.693 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/asus_atk0110.ko 2024-03-04T13:12:33.693 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/atxp1.ko 2024-03-04T13:12:33.693 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/coretemp.ko 2024-03-04T13:12:33.693 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-03-04T13:12:33.694 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/dme1737.ko 2024-03-04T13:12:33.694 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/ds1621.ko 2024-03-04T13:12:33.694 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/ds620.ko 2024-03-04T13:12:33.694 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/emc1403.ko 2024-03-04T13:12:33.694 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/emc6w201.ko 2024-03-04T13:12:33.694 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/f71805f.ko 2024-03-04T13:12:33.694 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/f71882fg.ko 2024-03-04T13:12:33.694 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/f75375s.ko 2024-03-04T13:12:33.694 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/fam15h_power.ko 2024-03-04T13:12:33.694 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/fschmd.ko 2024-03-04T13:12:33.694 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/g760a.ko 2024-03-04T13:12:33.694 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/g762.ko 2024-03-04T13:12:33.694 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/gl518sm.ko 2024-03-04T13:12:33.694 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/gl520sm.ko 2024-03-04T13:12:33.695 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/hwmon-vid.ko 2024-03-04T13:12:33.695 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/i5500_temp.ko 2024-03-04T13:12:33.695 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/i5k_amb.ko 2024-03-04T13:12:33.695 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/ibmaem.ko 2024-03-04T13:12:33.695 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/ibmpex.ko 2024-03-04T13:12:33.695 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/ina209.ko 2024-03-04T13:12:33.695 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/ina2xx.ko 2024-03-04T13:12:33.695 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/it87.ko 2024-03-04T13:12:33.695 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/k10temp.ko 2024-03-04T13:12:33.695 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/k8temp.ko 2024-03-04T13:12:33.695 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/lineage-pem.ko 2024-03-04T13:12:33.695 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/lm63.ko 2024-03-04T13:12:33.695 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/lm73.ko 2024-03-04T13:12:33.695 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/lm75.ko 2024-03-04T13:12:33.696 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/lm77.ko 2024-03-04T13:12:33.696 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/lm78.ko 2024-03-04T13:12:33.696 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/lm80.ko 2024-03-04T13:12:33.696 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/lm83.ko 2024-03-04T13:12:33.696 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/lm85.ko 2024-03-04T13:12:33.696 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/lm87.ko 2024-03-04T13:12:33.696 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/lm90.ko 2024-03-04T13:12:33.696 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/lm92.ko 2024-03-04T13:12:33.696 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/lm93.ko 2024-03-04T13:12:33.696 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/lm95234.ko 2024-03-04T13:12:33.696 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/lm95241.ko 2024-03-04T13:12:33.696 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/lm95245.ko 2024-03-04T13:12:33.696 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/ltc2945.ko 2024-03-04T13:12:33.696 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/ltc4151.ko 2024-03-04T13:12:33.697 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/ltc4215.ko 2024-03-04T13:12:33.697 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/ltc4222.ko 2024-03-04T13:12:33.697 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/ltc4245.ko 2024-03-04T13:12:33.697 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/ltc4260.ko 2024-03-04T13:12:33.697 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/ltc4261.ko 2024-03-04T13:12:33.697 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/max16065.ko 2024-03-04T13:12:33.697 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/max1619.ko 2024-03-04T13:12:33.697 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/max1668.ko 2024-03-04T13:12:33.697 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/max197.ko 2024-03-04T13:12:33.697 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/max6639.ko 2024-03-04T13:12:33.697 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/max6650.ko 2024-03-04T13:12:33.697 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/max6697.ko 2024-03-04T13:12:33.697 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/mcp3021.ko 2024-03-04T13:12:33.697 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/nct6683.ko 2024-03-04T13:12:33.698 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/nct6775-core.ko 2024-03-04T13:12:33.698 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/nct6775.ko 2024-03-04T13:12:33.698 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/nct7802.ko 2024-03-04T13:12:33.698 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/pc87360.ko 2024-03-04T13:12:33.698 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/pc87427.ko 2024-03-04T13:12:33.698 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/pcf8591.ko 2024-03-04T13:12:33.698 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/pmbus 2024-03-04T13:12:33.698 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-03-04T13:12:33.698 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-03-04T13:12:33.698 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-03-04T13:12:33.698 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/pmbus/max16064.ko 2024-03-04T13:12:33.698 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/pmbus/max34440.ko 2024-03-04T13:12:33.698 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/pmbus/max8688.ko 2024-03-04T13:12:33.698 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-03-04T13:12:33.699 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-03-04T13:12:33.699 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-03-04T13:12:33.699 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-03-04T13:12:33.699 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-03-04T13:12:33.699 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-03-04T13:12:33.699 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/powr1220.ko 2024-03-04T13:12:33.699 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/sch5627.ko 2024-03-04T13:12:33.699 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/sch5636.ko 2024-03-04T13:12:33.699 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/sch56xx-common.ko 2024-03-04T13:12:33.699 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/sht15.ko 2024-03-04T13:12:33.699 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/sht21.ko 2024-03-04T13:12:33.699 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/shtc1.ko 2024-03-04T13:12:33.699 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/sis5595.ko 2024-03-04T13:12:33.699 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/smsc47b397.ko 2024-03-04T13:12:33.699 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/smsc47m1.ko 2024-03-04T13:12:33.700 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/smsc47m192.ko 2024-03-04T13:12:33.700 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/thmc50.ko 2024-03-04T13:12:33.700 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/tmp102.ko 2024-03-04T13:12:33.700 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/tmp103.ko 2024-03-04T13:12:33.700 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/tmp401.ko 2024-03-04T13:12:33.700 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/tmp421.ko 2024-03-04T13:12:33.700 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/via-cputemp.ko 2024-03-04T13:12:33.700 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/via686a.ko 2024-03-04T13:12:33.700 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/vt1211.ko 2024-03-04T13:12:33.700 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/vt8231.ko 2024-03-04T13:12:33.700 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/w83627ehf.ko 2024-03-04T13:12:33.700 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/w83627hf.ko 2024-03-04T13:12:33.700 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/w83781d.ko 2024-03-04T13:12:33.700 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/w83791d.ko 2024-03-04T13:12:33.701 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/w83792d.ko 2024-03-04T13:12:33.701 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/w83793.ko 2024-03-04T13:12:33.701 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/w83795.ko 2024-03-04T13:12:33.701 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/w83l785ts.ko 2024-03-04T13:12:33.701 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/w83l786ng.ko 2024-03-04T13:12:33.701 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c 2024-03-04T13:12:33.701 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/algos 2024-03-04T13:12:33.701 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-03-04T13:12:33.701 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-03-04T13:12:33.701 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/busses 2024-03-04T13:12:33.701 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-03-04T13:12:33.701 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-03-04T13:12:33.701 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-03-04T13:12:33.701 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-03-04T13:12:33.701 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-03-04T13:12:33.702 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-03-04T13:12:33.702 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-03-04T13:12:33.702 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-03-04T13:12:33.702 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/busses/i2c-i801.ko 2024-03-04T13:12:33.702 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/busses/i2c-isch.ko 2024-03-04T13:12:33.702 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-03-04T13:12:33.702 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-03-04T13:12:33.702 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-03-04T13:12:33.702 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/busses/i2c-parport.ko 2024-03-04T13:12:33.702 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-03-04T13:12:33.702 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-03-04T13:12:33.702 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-03-04T13:12:33.702 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-03-04T13:12:33.702 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-03-04T13:12:33.703 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-03-04T13:12:33.703 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/busses/i2c-via.ko 2024-03-04T13:12:33.703 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-03-04T13:12:33.703 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-03-04T13:12:33.703 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/i2c-dev.ko 2024-03-04T13:12:33.703 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/i2c-mux.ko 2024-03-04T13:12:33.703 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-03-04T13:12:33.703 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/i2c-smbus.ko 2024-03-04T13:12:33.703 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/i2c-stub.ko 2024-03-04T13:12:33.703 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/infiniband 2024-03-04T13:12:33.703 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/infiniband/core 2024-03-04T13:12:33.703 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/infiniband/core/ib_cm.ko 2024-03-04T13:12:33.703 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/infiniband/core/ib_core.ko 2024-03-04T13:12:33.703 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/infiniband/core/ib_umad.ko 2024-03-04T13:12:33.703 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-03-04T13:12:33.704 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/infiniband/core/iw_cm.ko 2024-03-04T13:12:33.704 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/infiniband/core/rdma_cm.ko 2024-03-04T13:12:33.704 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-03-04T13:12:33.704 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/infiniband/hw 2024-03-04T13:12:33.704 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/infiniband/hw/cxgb4 2024-03-04T13:12:33.704 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-03-04T13:12:33.704 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/infiniband/hw/mlx4 2024-03-04T13:12:33.704 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-03-04T13:12:33.704 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/infiniband/hw/mlx5 2024-03-04T13:12:33.704 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-03-04T13:12:33.704 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/infiniband/hw/mthca 2024-03-04T13:12:33.704 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-03-04T13:12:33.704 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/infiniband/ulp 2024-03-04T13:12:33.704 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/infiniband/ulp/ipoib 2024-03-04T13:12:33.705 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-03-04T13:12:33.705 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/infiniband/ulp/iser 2024-03-04T13:12:33.705 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-03-04T13:12:33.705 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/infiniband/ulp/isert 2024-03-04T13:12:33.705 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-03-04T13:12:33.705 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/infiniband/ulp/srp 2024-03-04T13:12:33.705 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-03-04T13:12:33.705 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/infiniband/ulp/srpt 2024-03-04T13:12:33.705 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-03-04T13:12:33.705 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input 2024-03-04T13:12:33.705 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input/gameport 2024-03-04T13:12:33.705 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-03-04T13:12:33.705 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input/gameport/fm801-gp.ko 2024-03-04T13:12:33.705 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input/gameport/gameport.ko 2024-03-04T13:12:33.706 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input/joydev.ko 2024-03-04T13:12:33.706 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input/keyboard 2024-03-04T13:12:33.706 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input/keyboard/gpio_keys.ko 2024-03-04T13:12:33.706 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input/mouse 2024-03-04T13:12:33.706 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input/mouse/appletouch.ko 2024-03-04T13:12:33.706 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input/mouse/bcm5974.ko 2024-03-04T13:12:33.706 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input/mouse/cyapatp.ko 2024-03-04T13:12:33.706 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input/mouse/elan_i2c.ko 2024-03-04T13:12:33.706 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input/mouse/sermouse.ko 2024-03-04T13:12:33.706 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-03-04T13:12:33.706 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input/mouse/synaptics_usb.ko 2024-03-04T13:12:33.706 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input/mouse/vsxxxaa.ko 2024-03-04T13:12:33.706 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input/rmi4 2024-03-04T13:12:33.706 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input/rmi4/rmi_core.ko 2024-03-04T13:12:33.706 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input/serio 2024-03-04T13:12:33.707 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input/serio/altera_ps2.ko 2024-03-04T13:12:33.707 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input/serio/arc_ps2.ko 2024-03-04T13:12:33.707 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-03-04T13:12:33.707 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input/serio/serio_raw.ko 2024-03-04T13:12:33.707 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input/sparse-keymap.ko 2024-03-04T13:12:33.707 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/leds 2024-03-04T13:12:33.707 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/leds/leds-blinkm.ko 2024-03-04T13:12:33.707 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/leds/leds-lm3530.ko 2024-03-04T13:12:33.707 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/leds/leds-lp3944.ko 2024-03-04T13:12:33.707 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/leds/leds-ss4200.ko 2024-03-04T13:12:33.707 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/leds/trigger 2024-03-04T13:12:33.707 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-03-04T13:12:33.707 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-03-04T13:12:33.707 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-03-04T13:12:33.707 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-03-04T13:12:33.708 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-03-04T13:12:33.708 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-03-04T13:12:33.708 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-03-04T13:12:33.708 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-03-04T13:12:33.708 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-03-04T13:12:33.708 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/md 2024-03-04T13:12:33.708 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/md/bcache 2024-03-04T13:12:33.708 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/md/bcache/bcache.ko 2024-03-04T13:12:33.708 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/md/dm-bio-prison.ko 2024-03-04T13:12:33.708 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/md/dm-cache-smq.ko 2024-03-04T13:12:33.708 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/md/dm-cache.ko 2024-03-04T13:12:33.708 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/md/dm-crypt.ko 2024-03-04T13:12:33.708 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/md/dm-delay.ko 2024-03-04T13:12:33.709 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/md/dm-flakey.ko 2024-03-04T13:12:33.709 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/md/dm-log-userspace.ko 2024-03-04T13:12:33.709 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/md/dm-multipath.ko 2024-03-04T13:12:33.709 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/md/dm-queue-length.ko 2024-03-04T13:12:33.709 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/md/dm-raid.ko 2024-03-04T13:12:33.709 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/md/dm-round-robin.ko 2024-03-04T13:12:33.709 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/md/dm-service-time.ko 2024-03-04T13:12:33.709 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/md/dm-switch.ko 2024-03-04T13:12:33.709 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/md/dm-thin-pool.ko 2024-03-04T13:12:33.709 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/md/dm-verity.ko 2024-03-04T13:12:33.709 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/md/persistent-data 2024-03-04T13:12:33.709 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-03-04T13:12:33.709 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/md/raid0.ko 2024-03-04T13:12:33.709 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/md/raid1.ko 2024-03-04T13:12:33.709 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/md/raid10.ko 2024-03-04T13:12:33.710 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/md/raid456.ko 2024-03-04T13:12:33.710 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/message 2024-03-04T13:12:33.710 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/message/fusion 2024-03-04T13:12:33.710 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/message/fusion/mptbase.ko 2024-03-04T13:12:33.710 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/message/fusion/mptctl.ko 2024-03-04T13:12:33.710 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/message/fusion/mptfc.ko 2024-03-04T13:12:33.710 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/message/fusion/mptlan.ko 2024-03-04T13:12:33.710 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/message/fusion/mptsas.ko 2024-03-04T13:12:33.710 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/message/fusion/mptscsih.ko 2024-03-04T13:12:33.710 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/message/fusion/mptspi.ko 2024-03-04T13:12:33.710 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/mfd 2024-03-04T13:12:33.710 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/mfd/lpc_ich.ko 2024-03-04T13:12:33.710 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/mfd/lpc_sch.ko 2024-03-04T13:12:33.710 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/mfd/mfd-core.ko 2024-03-04T13:12:33.710 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/mfd/sm501.ko 2024-03-04T13:12:33.711 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/mfd/viperboard.ko 2024-03-04T13:12:33.711 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/mfd/vx855.ko 2024-03-04T13:12:33.711 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/mfd/wl1273-core.ko 2024-03-04T13:12:33.711 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc 2024-03-04T13:12:33.711 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/altera-stapl 2024-03-04T13:12:33.711 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-03-04T13:12:33.711 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/apds9802als.ko 2024-03-04T13:12:33.711 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/apds990x.ko 2024-03-04T13:12:33.711 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/bh1770glc.ko 2024-03-04T13:12:33.711 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/cb710 2024-03-04T13:12:33.711 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/cb710/cb710.ko 2024-03-04T13:12:33.711 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/eeprom 2024-03-04T13:12:33.711 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/eeprom/at24.ko 2024-03-04T13:12:33.711 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-03-04T13:12:33.711 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/eeprom/max6875.ko 2024-03-04T13:12:33.712 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/enclosure.ko 2024-03-04T13:12:33.712 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/hpilo.ko 2024-03-04T13:12:33.712 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/isl29003.ko 2024-03-04T13:12:33.712 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/isl29020.ko 2024-03-04T13:12:33.712 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/lis3lv02d 2024-03-04T13:12:33.712 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-03-04T13:12:33.712 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-03-04T13:12:33.712 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/mei 2024-03-04T13:12:33.712 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/mei/mei-me.ko 2024-03-04T13:12:33.712 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/mei/mei-txe.ko 2024-03-04T13:12:33.712 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/mei/mei.ko 2024-03-04T13:12:33.712 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/sgi-gru 2024-03-04T13:12:33.712 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/sgi-gru/gru.ko 2024-03-04T13:12:33.712 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/sgi-xp 2024-03-04T13:12:33.713 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/sgi-xp/xp.ko 2024-03-04T13:12:33.713 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/sgi-xp/xpc.ko 2024-03-04T13:12:33.713 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-03-04T13:12:33.713 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/tifm_7xx1.ko 2024-03-04T13:12:33.713 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/tifm_core.ko 2024-03-04T13:12:33.713 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/tsl2550.ko 2024-03-04T13:12:33.713 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/vmw_balloon.ko 2024-03-04T13:12:33.713 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/vmw_vmci 2024-03-04T13:12:33.713 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-03-04T13:12:33.713 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net 2024-03-04T13:12:33.713 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/bonding 2024-03-04T13:12:33.713 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/bonding/bonding.ko 2024-03-04T13:12:33.713 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/dsa 2024-03-04T13:12:33.713 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/dsa/b53 2024-03-04T13:12:33.713 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/dsa/b53/b53_common.ko 2024-03-04T13:12:33.714 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/dsa/bcm-sf2.ko 2024-03-04T13:12:33.714 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/dsa/mv88e6060.ko 2024-03-04T13:12:33.714 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/dsa/mv88e6xxx 2024-03-04T13:12:33.714 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-03-04T13:12:33.714 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/dummy.ko 2024-03-04T13:12:33.714 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/eql.ko 2024-03-04T13:12:33.714 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet 2024-03-04T13:12:33.714 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/3com 2024-03-04T13:12:33.714 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-03-04T13:12:33.714 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-03-04T13:12:33.714 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-03-04T13:12:33.714 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-03-04T13:12:33.714 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/8390 2024-03-04T13:12:33.714 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/8390/8390.ko 2024-03-04T13:12:33.715 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-03-04T13:12:33.715 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-03-04T13:12:33.715 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-03-04T13:12:33.715 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/adaptec 2024-03-04T13:12:33.715 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-03-04T13:12:33.715 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/agere 2024-03-04T13:12:33.715 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/agere/et131x.ko 2024-03-04T13:12:33.715 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/alteon 2024-03-04T13:12:33.715 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-03-04T13:12:33.715 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/altera 2024-03-04T13:12:33.715 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-03-04T13:12:33.715 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/amd 2024-03-04T13:12:33.715 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-03-04T13:12:33.715 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-03-04T13:12:33.716 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-03-04T13:12:33.716 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/atheros 2024-03-04T13:12:33.716 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/atheros/alx 2024-03-04T13:12:33.716 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-03-04T13:12:33.716 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/atheros/atl1c 2024-03-04T13:12:33.716 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-03-04T13:12:33.716 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/atheros/atl1e 2024-03-04T13:12:33.716 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-03-04T13:12:33.716 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/atheros/atlx 2024-03-04T13:12:33.716 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-03-04T13:12:33.716 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-03-04T13:12:33.716 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/broadcom 2024-03-04T13:12:33.716 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-03-04T13:12:33.716 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-03-04T13:12:33.717 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-03-04T13:12:33.717 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-03-04T13:12:33.717 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-03-04T13:12:33.717 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/broadcom/genet 2024-03-04T13:12:33.717 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-03-04T13:12:33.717 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-03-04T13:12:33.717 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/brocade 2024-03-04T13:12:33.717 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/brocade/bna 2024-03-04T13:12:33.717 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-03-04T13:12:33.717 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/chelsio 2024-03-04T13:12:33.717 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/chelsio/cxgb 2024-03-04T13:12:33.717 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-03-04T13:12:33.717 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-03-04T13:12:33.717 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-03-04T13:12:33.718 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-03-04T13:12:33.718 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-03-04T13:12:33.718 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-03-04T13:12:33.718 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-03-04T13:12:33.718 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-03-04T13:12:33.718 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-03-04T13:12:33.718 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/cisco 2024-03-04T13:12:33.718 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/cisco/enic 2024-03-04T13:12:33.718 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-03-04T13:12:33.718 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/dec 2024-03-04T13:12:33.718 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/dec/tulip 2024-03-04T13:12:33.718 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-03-04T13:12:33.718 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-03-04T13:12:33.718 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-03-04T13:12:33.719 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-03-04T13:12:33.719 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-03-04T13:12:33.719 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-03-04T13:12:33.719 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/dlink 2024-03-04T13:12:33.719 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-03-04T13:12:33.719 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-03-04T13:12:33.719 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/dnet.ko 2024-03-04T13:12:33.719 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/emulex 2024-03-04T13:12:33.719 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/emulex/benet 2024-03-04T13:12:33.719 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-03-04T13:12:33.719 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/ethoc.ko 2024-03-04T13:12:33.719 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/fealnx.ko 2024-03-04T13:12:33.719 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/intel 2024-03-04T13:12:33.719 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/intel/e100.ko 2024-03-04T13:12:33.720 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/intel/e1000 2024-03-04T13:12:33.720 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-03-04T13:12:33.720 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/intel/e1000e 2024-03-04T13:12:33.720 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-03-04T13:12:33.720 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/intel/fm10k 2024-03-04T13:12:33.720 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-03-04T13:12:33.720 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/intel/i40e 2024-03-04T13:12:33.720 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-03-04T13:12:33.720 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/intel/iavf 2024-03-04T13:12:33.720 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-03-04T13:12:33.720 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/intel/igb 2024-03-04T13:12:33.720 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-03-04T13:12:33.720 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/intel/igbvf 2024-03-04T13:12:33.720 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-03-04T13:12:33.721 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/intel/ixgbe 2024-03-04T13:12:33.721 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-03-04T13:12:33.721 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/intel/ixgbevf 2024-03-04T13:12:33.721 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-03-04T13:12:33.721 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/jme.ko 2024-03-04T13:12:33.721 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/marvell 2024-03-04T13:12:33.721 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-03-04T13:12:33.721 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/marvell/skge.ko 2024-03-04T13:12:33.721 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-03-04T13:12:33.721 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/mellanox 2024-03-04T13:12:33.721 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/mellanox/mlx4 2024-03-04T13:12:33.721 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-03-04T13:12:33.721 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-03-04T13:12:33.721 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/mellanox/mlx5 2024-03-04T13:12:33.722 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-03-04T13:12:33.722 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-03-04T13:12:33.722 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/micrel 2024-03-04T13:12:33.722 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-03-04T13:12:33.722 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/myricom 2024-03-04T13:12:33.722 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/myricom/myri10ge 2024-03-04T13:12:33.722 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-03-04T13:12:33.722 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/natsemi 2024-03-04T13:12:33.722 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-03-04T13:12:33.722 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-03-04T13:12:33.722 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/neterion 2024-03-04T13:12:33.722 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-03-04T13:12:33.722 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/nvidia 2024-03-04T13:12:33.722 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-03-04T13:12:33.722 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/packetengines 2024-03-04T13:12:33.723 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-03-04T13:12:33.723 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-03-04T13:12:33.723 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/qlogic 2024-03-04T13:12:33.723 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/qlogic/netxen 2024-03-04T13:12:33.723 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-03-04T13:12:33.723 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-03-04T13:12:33.723 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-03-04T13:12:33.723 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-03-04T13:12:33.723 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/rdc 2024-03-04T13:12:33.723 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-03-04T13:12:33.723 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/realtek 2024-03-04T13:12:33.723 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-03-04T13:12:33.723 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-03-04T13:12:33.723 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/realtek/atp.ko 2024-03-04T13:12:33.724 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-03-04T13:12:33.724 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/rocker 2024-03-04T13:12:33.724 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-03-04T13:12:33.724 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/sfc 2024-03-04T13:12:33.724 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-03-04T13:12:33.724 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/silan 2024-03-04T13:12:33.724 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-03-04T13:12:33.724 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/sis 2024-03-04T13:12:33.724 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/sis/sis190.ko 2024-03-04T13:12:33.724 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/sis/sis900.ko 2024-03-04T13:12:33.724 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/smsc 2024-03-04T13:12:33.724 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-03-04T13:12:33.724 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-03-04T13:12:33.724 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-03-04T13:12:33.725 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-03-04T13:12:33.725 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/stmicro 2024-03-04T13:12:33.725 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/stmicro/stmmac 2024-03-04T13:12:33.725 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-03-04T13:12:33.725 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-03-04T13:12:33.725 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/sun 2024-03-04T13:12:33.725 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/sun/cassini.ko 2024-03-04T13:12:33.725 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/sun/niu.ko 2024-03-04T13:12:33.725 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/sun/sungem.ko 2024-03-04T13:12:33.725 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-03-04T13:12:33.725 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/tehuti 2024-03-04T13:12:33.725 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-03-04T13:12:33.725 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/ti 2024-03-04T13:12:33.725 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/ti/tlan.ko 2024-03-04T13:12:33.726 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/via 2024-03-04T13:12:33.726 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-03-04T13:12:33.726 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-03-04T13:12:33.726 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/wiznet 2024-03-04T13:12:33.726 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-03-04T13:12:33.726 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-03-04T13:12:33.726 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/xircom 2024-03-04T13:12:33.726 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-03-04T13:12:33.726 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/geneve.ko 2024-03-04T13:12:33.726 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/hyperv 2024-03-04T13:12:33.726 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-03-04T13:12:33.726 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ieee802154 2024-03-04T13:12:33.726 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ieee802154/fakelb.ko 2024-03-04T13:12:33.726 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ifb.ko 2024-03-04T13:12:33.727 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ipvlan 2024-03-04T13:12:33.727 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ipvlan/ipvlan.ko 2024-03-04T13:12:33.727 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/macvlan.ko 2024-03-04T13:12:33.727 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/macvtap.ko 2024-03-04T13:12:33.727 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/mdio 2024-03-04T13:12:33.727 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/mdio.ko 2024-03-04T13:12:33.727 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-03-04T13:12:33.727 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-03-04T13:12:33.727 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/mdio/mdio-regmap.ko 2024-03-04T13:12:33.727 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/mii.ko 2024-03-04T13:12:33.727 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/net_failover.ko 2024-03-04T13:12:33.727 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/netconsole.ko 2024-03-04T13:12:33.727 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/nlmon.ko 2024-03-04T13:12:33.727 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/pcs 2024-03-04T13:12:33.727 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/pcs/pcs-lynx.ko 2024-03-04T13:12:33.728 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-03-04T13:12:33.728 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/phy 2024-03-04T13:12:33.728 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/phy/amd.ko 2024-03-04T13:12:33.728 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-03-04T13:12:33.728 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-03-04T13:12:33.728 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/phy/bcm7xxx.ko 2024-03-04T13:12:33.728 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/phy/bcm87xx.ko 2024-03-04T13:12:33.728 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/phy/broadcom.ko 2024-03-04T13:12:33.728 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/phy/cicada.ko 2024-03-04T13:12:33.728 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/phy/davicom.ko 2024-03-04T13:12:33.728 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/phy/dp83640.ko 2024-03-04T13:12:33.728 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/phy/et1011c.ko 2024-03-04T13:12:33.728 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/phy/icplus.ko 2024-03-04T13:12:33.728 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/phy/lxt.ko 2024-03-04T13:12:33.729 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/phy/marvell.ko 2024-03-04T13:12:33.729 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/phy/micrel.ko 2024-03-04T13:12:33.729 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/phy/national.ko 2024-03-04T13:12:33.729 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/phy/phylink.ko 2024-03-04T13:12:33.729 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/phy/qsemi.ko 2024-03-04T13:12:33.729 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/phy/realtek.ko 2024-03-04T13:12:33.729 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/phy/smsc.ko 2024-03-04T13:12:33.729 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/phy/ste10Xp.ko 2024-03-04T13:12:33.729 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/phy/vitesse.ko 2024-03-04T13:12:33.729 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ppp 2024-03-04T13:12:33.729 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ppp/bsd_comp.ko 2024-03-04T13:12:33.729 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ppp/ppp_async.ko 2024-03-04T13:12:33.729 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ppp/ppp_deflate.ko 2024-03-04T13:12:33.729 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ppp/ppp_generic.ko 2024-03-04T13:12:33.729 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ppp/ppp_mppe.ko 2024-03-04T13:12:33.730 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ppp/ppp_synctty.ko 2024-03-04T13:12:33.730 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ppp/pppoe.ko 2024-03-04T13:12:33.730 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ppp/pppox.ko 2024-03-04T13:12:33.730 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ppp/pptp.ko 2024-03-04T13:12:33.730 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/slip 2024-03-04T13:12:33.730 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/slip/slhc.ko 2024-03-04T13:12:33.730 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/slip/slip.ko 2024-03-04T13:12:33.730 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/sungem_phy.ko 2024-03-04T13:12:33.730 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/tap.ko 2024-03-04T13:12:33.730 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/team 2024-03-04T13:12:33.730 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/team/team.ko 2024-03-04T13:12:33.730 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/team/team_mode_activebackup.ko 2024-03-04T13:12:33.730 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/team/team_mode_broadcast.ko 2024-03-04T13:12:33.730 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-03-04T13:12:33.730 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/team/team_mode_random.ko 2024-03-04T13:12:33.730 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-03-04T13:12:33.731 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/tun.ko 2024-03-04T13:12:33.731 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/veth.ko 2024-03-04T13:12:33.731 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/virtio_net.ko 2024-03-04T13:12:33.731 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/vmxnet3 2024-03-04T13:12:33.731 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-03-04T13:12:33.731 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/vxlan 2024-03-04T13:12:33.731 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/vxlan/vxlan.ko 2024-03-04T13:12:33.731 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/xen-netback 2024-03-04T13:12:33.731 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/xen-netback/xen-netback.ko 2024-03-04T13:12:33.731 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/xen-netfront.ko 2024-03-04T13:12:33.731 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ntb 2024-03-04T13:12:33.731 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ntb/ntb.ko 2024-03-04T13:12:33.731 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/nvme 2024-03-04T13:12:33.731 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/nvme/host 2024-03-04T13:12:33.732 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/nvme/host/nvme-core.ko 2024-03-04T13:12:33.732 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/nvme/host/nvme.ko 2024-03-04T13:12:33.732 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/parport 2024-03-04T13:12:33.732 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/parport/parport.ko 2024-03-04T13:12:33.732 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/parport/parport_cs.ko 2024-03-04T13:12:33.732 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/parport/parport_pc.ko 2024-03-04T13:12:33.732 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/parport/parport_serial.ko 2024-03-04T13:12:33.732 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/pci 2024-03-04T13:12:33.732 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/pci/hotplug 2024-03-04T13:12:33.732 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-03-04T13:12:33.732 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/pci/pcie 2024-03-04T13:12:33.732 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/pci/pcie/aer_inject.ko 2024-03-04T13:12:33.732 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/pci/xen-pcifront.ko 2024-03-04T13:12:33.732 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/pcmcia 2024-03-04T13:12:33.733 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/pcmcia/i82092.ko 2024-03-04T13:12:33.733 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/pcmcia/pd6729.ko 2024-03-04T13:12:33.733 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/pcmcia/yenta_socket.ko 2024-03-04T13:12:33.733 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/pinctrl 2024-03-04T13:12:33.733 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/pinctrl/intel 2024-03-04T13:12:33.733 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-03-04T13:12:33.733 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/platform 2024-03-04T13:12:33.733 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/platform/x86 2024-03-04T13:12:33.733 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/platform/x86/acer-wmi.ko 2024-03-04T13:12:33.733 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/platform/x86/acerhdf.ko 2024-03-04T13:12:33.733 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/platform/x86/apple-gmux.ko 2024-03-04T13:12:33.733 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/platform/x86/asus-laptop.ko 2024-03-04T13:12:33.733 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-03-04T13:12:33.733 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/platform/x86/asus-wmi.ko 2024-03-04T13:12:33.734 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/platform/x86/classmate-laptop.ko 2024-03-04T13:12:33.734 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-03-04T13:12:33.734 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-03-04T13:12:33.734 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-03-04T13:12:33.734 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-03-04T13:12:33.734 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/platform/x86/hdaps.ko 2024-03-04T13:12:33.734 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/platform/x86/intel 2024-03-04T13:12:33.734 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-03-04T13:12:33.734 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/platform/x86/intel_ips.ko 2024-03-04T13:12:33.734 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/platform/x86/msi-wmi.ko 2024-03-04T13:12:33.734 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/platform/x86/mxm-wmi.ko 2024-03-04T13:12:33.734 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-03-04T13:12:33.734 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/platform/x86/samsung-laptop.ko 2024-03-04T13:12:33.734 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/platform/x86/samsung-q10.ko 2024-03-04T13:12:33.734 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/platform/x86/topstar-laptop.ko 2024-03-04T13:12:33.735 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-03-04T13:12:33.735 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/platform/x86/toshiba_haps.ko 2024-03-04T13:12:33.735 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/platform/x86/wmi-bmof.ko 2024-03-04T13:12:33.735 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/platform/x86/wmi.ko 2024-03-04T13:12:33.735 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/powercap 2024-03-04T13:12:33.735 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/powercap/intel_rapl_common.ko 2024-03-04T13:12:33.735 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/powercap/intel_rapl_msr.ko 2024-03-04T13:12:33.735 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/pps 2024-03-04T13:12:33.735 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/pps/clients 2024-03-04T13:12:33.735 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/pps/clients/pps-gpio.ko 2024-03-04T13:12:33.735 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/pps/clients/pps-ldisc.ko 2024-03-04T13:12:33.735 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/pps/clients/pps_parport.ko 2024-03-04T13:12:33.735 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/pps/pps_core.ko 2024-03-04T13:12:33.735 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ptp 2024-03-04T13:12:33.736 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ptp/ptp.ko 2024-03-04T13:12:33.736 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ptp/ptp_kvm.ko 2024-03-04T13:12:33.736 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc 2024-03-04T13:12:33.736 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-bq32k.ko 2024-03-04T13:12:33.736 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-ds1286.ko 2024-03-04T13:12:33.736 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-ds1307.ko 2024-03-04T13:12:33.736 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-ds1374.ko 2024-03-04T13:12:33.736 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-ds1511.ko 2024-03-04T13:12:33.736 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-ds1553.ko 2024-03-04T13:12:33.736 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-ds1672.ko 2024-03-04T13:12:33.736 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-ds1742.ko 2024-03-04T13:12:33.736 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-ds2404.ko 2024-03-04T13:12:33.736 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-ds3232.ko 2024-03-04T13:12:33.737 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-em3027.ko 2024-03-04T13:12:33.737 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-fm3130.ko 2024-03-04T13:12:33.737 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-isl12022.ko 2024-03-04T13:12:33.737 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-isl1208.ko 2024-03-04T13:12:33.737 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-m41t80.ko 2024-03-04T13:12:33.737 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-m48t35.ko 2024-03-04T13:12:33.737 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-m48t59.ko 2024-03-04T13:12:33.737 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-max6900.ko 2024-03-04T13:12:33.737 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-msm6242.ko 2024-03-04T13:12:33.737 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-pcf2127.ko 2024-03-04T13:12:33.737 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-pcf85063.ko 2024-03-04T13:12:33.737 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-pcf8523.ko 2024-03-04T13:12:33.737 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-pcf8563.ko 2024-03-04T13:12:33.737 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-pcf8583.ko 2024-03-04T13:12:33.738 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-rp5c01.ko 2024-03-04T13:12:33.738 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-rs5c372.ko 2024-03-04T13:12:33.738 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-03-04T13:12:33.738 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-rx8025.ko 2024-03-04T13:12:33.738 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-rx8581.ko 2024-03-04T13:12:33.738 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-03-04T13:12:33.738 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-x1205.ko 2024-03-04T13:12:33.738 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi 2024-03-04T13:12:33.738 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/3w-9xxx.ko 2024-03-04T13:12:33.738 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/3w-sas.ko 2024-03-04T13:12:33.738 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/3w-xxxx.ko 2024-03-04T13:12:33.738 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/BusLogic.ko 2024-03-04T13:12:33.738 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/a100u2w.ko 2024-03-04T13:12:33.738 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/aacraid 2024-03-04T13:12:33.738 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/aacraid/aacraid.ko 2024-03-04T13:12:33.738 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/advansys.ko 2024-03-04T13:12:33.739 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/aic7xxx 2024-03-04T13:12:33.739 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-03-04T13:12:33.739 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-03-04T13:12:33.739 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/am53c974.ko 2024-03-04T13:12:33.739 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/arcmsr 2024-03-04T13:12:33.739 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-03-04T13:12:33.739 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/atp870u.ko 2024-03-04T13:12:33.739 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/be2iscsi 2024-03-04T13:12:33.739 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-03-04T13:12:33.739 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/bfa 2024-03-04T13:12:33.739 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/bfa/bfa.ko 2024-03-04T13:12:33.739 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/bnx2fc 2024-03-04T13:12:33.739 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-03-04T13:12:33.739 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/bnx2i 2024-03-04T13:12:33.740 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-03-04T13:12:33.740 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/ch.ko 2024-03-04T13:12:33.740 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/csiostor 2024-03-04T13:12:33.740 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/csiostor/csiostor.ko 2024-03-04T13:12:33.740 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/cxgbi 2024-03-04T13:12:33.740 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/cxgbi/cxgb3i 2024-03-04T13:12:33.740 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-03-04T13:12:33.740 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/cxgbi/cxgb4i 2024-03-04T13:12:33.740 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-03-04T13:12:33.740 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-03-04T13:12:33.740 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/dc395x.ko 2024-03-04T13:12:33.740 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/device_handler 2024-03-04T13:12:33.740 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-03-04T13:12:33.740 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-03-04T13:12:33.741 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-03-04T13:12:33.741 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-03-04T13:12:33.741 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/dmx3191d.ko 2024-03-04T13:12:33.741 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/esas2r 2024-03-04T13:12:33.741 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/esas2r/esas2r.ko 2024-03-04T13:12:33.741 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/esp_scsi.ko 2024-03-04T13:12:33.741 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/fcoe 2024-03-04T13:12:33.741 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/fcoe/fcoe.ko 2024-03-04T13:12:33.741 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-03-04T13:12:33.741 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/fnic 2024-03-04T13:12:33.741 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/fnic/fnic.ko 2024-03-04T13:12:33.741 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/hpsa.ko 2024-03-04T13:12:33.741 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/hptiop.ko 2024-03-04T13:12:33.741 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/hv_storvsc.ko 2024-03-04T13:12:33.742 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/initio.ko 2024-03-04T13:12:33.742 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/ipr.ko 2024-03-04T13:12:33.742 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/ips.ko 2024-03-04T13:12:33.742 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/isci 2024-03-04T13:12:33.742 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/isci/isci.ko 2024-03-04T13:12:33.742 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-03-04T13:12:33.742 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/iscsi_tcp.ko 2024-03-04T13:12:33.742 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/libfc 2024-03-04T13:12:33.742 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/libfc/libfc.ko 2024-03-04T13:12:33.742 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/libiscsi.ko 2024-03-04T13:12:33.742 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/libiscsi_tcp.ko 2024-03-04T13:12:33.742 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/libsas 2024-03-04T13:12:33.742 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/libsas/libsas.ko 2024-03-04T13:12:33.742 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/lpfc 2024-03-04T13:12:33.742 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/lpfc/lpfc.ko 2024-03-04T13:12:33.742 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/megaraid 2024-03-04T13:12:33.743 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/megaraid.ko 2024-03-04T13:12:33.743 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-03-04T13:12:33.743 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-03-04T13:12:33.743 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-03-04T13:12:33.743 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/mpt3sas 2024-03-04T13:12:33.743 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-03-04T13:12:33.743 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/mvsas 2024-03-04T13:12:33.743 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/mvsas/mvsas.ko 2024-03-04T13:12:33.743 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/mvumi.ko 2024-03-04T13:12:33.743 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/pm8001 2024-03-04T13:12:33.743 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-03-04T13:12:33.743 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/pmcraid.ko 2024-03-04T13:12:33.743 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/qla1280.ko 2024-03-04T13:12:33.743 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/qla2xxx 2024-03-04T13:12:33.744 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-03-04T13:12:33.744 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-03-04T13:12:33.744 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/qla4xxx 2024-03-04T13:12:33.744 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-03-04T13:12:33.744 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/raid_class.ko 2024-03-04T13:12:33.744 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/scsi_debug.ko 2024-03-04T13:12:33.744 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/scsi_transport_fc.ko 2024-03-04T13:12:33.744 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-03-04T13:12:33.744 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/scsi_transport_sas.ko 2024-03-04T13:12:33.744 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/scsi_transport_spi.ko 2024-03-04T13:12:33.744 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/scsi_transport_srp.ko 2024-03-04T13:12:33.744 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/ses.ko 2024-03-04T13:12:33.744 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/st.ko 2024-03-04T13:12:33.744 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/stex.ko 2024-03-04T13:12:33.745 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/sym53c8xx_2 2024-03-04T13:12:33.745 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-03-04T13:12:33.745 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/virtio_scsi.ko 2024-03-04T13:12:33.745 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/vmw_pvscsi.ko 2024-03-04T13:12:33.745 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/wd719x.ko 2024-03-04T13:12:33.745 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ssb 2024-03-04T13:12:33.745 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ssb/ssb.ko 2024-03-04T13:12:33.745 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/target 2024-03-04T13:12:33.745 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/target/iscsi 2024-03-04T13:12:33.745 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-03-04T13:12:33.745 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/target/loopback 2024-03-04T13:12:33.745 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/target/loopback/tcm_loop.ko 2024-03-04T13:12:33.745 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/target/target_core_file.ko 2024-03-04T13:12:33.745 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/target/target_core_iblock.ko 2024-03-04T13:12:33.745 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/target/target_core_mod.ko 2024-03-04T13:12:33.746 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/target/target_core_pscsi.ko 2024-03-04T13:12:33.746 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/target/target_core_user.ko 2024-03-04T13:12:33.746 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/target/tcm_fc 2024-03-04T13:12:33.746 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-03-04T13:12:33.746 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/thermal 2024-03-04T13:12:33.746 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/thermal/intel 2024-03-04T13:12:33.746 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/thermal/intel/int340x_thermal 2024-03-04T13:12:33.746 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-03-04T13:12:33.746 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-03-04T13:12:33.746 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-03-04T13:12:33.746 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-03-04T13:12:33.746 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-03-04T13:12:33.746 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-03-04T13:12:33.746 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-03-04T13:12:33.746 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-03-04T13:12:33.747 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-03-04T13:12:33.747 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-03-04T13:12:33.747 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-03-04T13:12:33.747 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-03-04T13:12:33.747 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-03-04T13:12:33.747 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-03-04T13:12:33.747 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-03-04T13:12:33.747 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-03-04T13:12:33.747 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-03-04T13:12:33.747 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-03-04T13:12:33.747 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/tty 2024-03-04T13:12:33.747 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/tty/ipwireless 2024-03-04T13:12:33.747 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-03-04T13:12:33.747 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/tty/n_gsm.ko 2024-03-04T13:12:33.748 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/tty/n_hdlc.ko 2024-03-04T13:12:33.748 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/tty/nozomi.ko 2024-03-04T13:12:33.748 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/tty/serial 2024-03-04T13:12:33.748 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/tty/serial/8250 2024-03-04T13:12:33.748 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-03-04T13:12:33.748 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/tty/serial/arc_uart.ko 2024-03-04T13:12:33.748 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/tty/serial/jsm 2024-03-04T13:12:33.748 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/tty/serial/jsm/jsm.ko 2024-03-04T13:12:33.748 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ufs 2024-03-04T13:12:33.748 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ufs/core 2024-03-04T13:12:33.748 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ufs/core/ufshcd-core.ko 2024-03-04T13:12:33.748 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ufs/host 2024-03-04T13:12:33.748 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-03-04T13:12:33.748 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/uio 2024-03-04T13:12:33.748 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/uio/uio.ko 2024-03-04T13:12:33.749 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/uio/uio_aec.ko 2024-03-04T13:12:33.749 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/uio/uio_cif.ko 2024-03-04T13:12:33.749 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/uio/uio_pci_generic.ko 2024-03-04T13:12:33.749 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/uio/uio_sercos3.ko 2024-03-04T13:12:33.749 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb 2024-03-04T13:12:33.749 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/class 2024-03-04T13:12:33.749 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/class/cdc-acm.ko 2024-03-04T13:12:33.749 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/class/cdc-wdm.ko 2024-03-04T13:12:33.749 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/class/usblp.ko 2024-03-04T13:12:33.749 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/class/usbtmc.ko 2024-03-04T13:12:33.749 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/host 2024-03-04T13:12:33.749 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/host/sl811-hcd.ko 2024-03-04T13:12:33.749 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/image 2024-03-04T13:12:33.749 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/image/mdc800.ko 2024-03-04T13:12:33.750 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/image/microtek.ko 2024-03-04T13:12:33.750 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/misc 2024-03-04T13:12:33.750 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/misc/adutux.ko 2024-03-04T13:12:33.750 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/misc/appledisplay.ko 2024-03-04T13:12:33.750 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/misc/emi26.ko 2024-03-04T13:12:33.750 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/misc/emi62.ko 2024-03-04T13:12:33.750 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/misc/ezusb.ko 2024-03-04T13:12:33.750 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/misc/idmouse.ko 2024-03-04T13:12:33.750 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/misc/iowarrior.ko 2024-03-04T13:12:33.750 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/misc/isight_firmware.ko 2024-03-04T13:12:33.750 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/misc/ldusb.ko 2024-03-04T13:12:33.750 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/misc/legousbtower.ko 2024-03-04T13:12:33.750 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/misc/sisusbvga 2024-03-04T13:12:33.750 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-03-04T13:12:33.751 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/misc/trancevibrator.ko 2024-03-04T13:12:33.751 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/misc/usb3503.ko 2024-03-04T13:12:33.751 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/misc/usblcd.ko 2024-03-04T13:12:33.751 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/misc/usbsevseg.ko 2024-03-04T13:12:33.751 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/misc/uss720.ko 2024-03-04T13:12:33.751 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/misc/yurex.ko 2024-03-04T13:12:33.751 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/phy 2024-03-04T13:12:33.751 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/phy/phy-generic.ko 2024-03-04T13:12:33.751 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial 2024-03-04T13:12:33.751 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/aircable.ko 2024-03-04T13:12:33.751 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/ark3116.ko 2024-03-04T13:12:33.751 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/belkin_sa.ko 2024-03-04T13:12:33.751 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/ch341.ko 2024-03-04T13:12:33.751 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/cp210x.ko 2024-03-04T13:12:33.751 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/cyberjack.ko 2024-03-04T13:12:33.752 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/cypress_m8.ko 2024-03-04T13:12:33.752 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/digi_acceleport.ko 2024-03-04T13:12:33.752 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/empeg.ko 2024-03-04T13:12:33.752 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/ftdi_sio.ko 2024-03-04T13:12:33.752 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/garmin_gps.ko 2024-03-04T13:12:33.752 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/io_edgeport.ko 2024-03-04T13:12:33.752 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/io_ti.ko 2024-03-04T13:12:33.752 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/ipaq.ko 2024-03-04T13:12:33.752 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/ipw.ko 2024-03-04T13:12:33.752 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/ir-usb.ko 2024-03-04T13:12:33.752 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-03-04T13:12:33.752 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/keyspan.ko 2024-03-04T13:12:33.752 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/keyspan_pda.ko 2024-03-04T13:12:33.752 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/kl5kusb105.ko 2024-03-04T13:12:33.753 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/kobil_sct.ko 2024-03-04T13:12:33.753 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/mct_u232.ko 2024-03-04T13:12:33.753 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/mos7720.ko 2024-03-04T13:12:33.753 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/mos7840.ko 2024-03-04T13:12:33.753 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/navman.ko 2024-03-04T13:12:33.753 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/omninet.ko 2024-03-04T13:12:33.753 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/opticon.ko 2024-03-04T13:12:33.753 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/option.ko 2024-03-04T13:12:33.753 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/oti6858.ko 2024-03-04T13:12:33.753 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/pl2303.ko 2024-03-04T13:12:33.753 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/qcaux.ko 2024-03-04T13:12:33.753 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/qcserial.ko 2024-03-04T13:12:33.753 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/quatech2.ko 2024-03-04T13:12:33.753 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/safe_serial.ko 2024-03-04T13:12:33.753 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/sierra.ko 2024-03-04T13:12:33.754 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/spcp8x5.ko 2024-03-04T13:12:33.754 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/ssu100.ko 2024-03-04T13:12:33.754 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/symbolserial.ko 2024-03-04T13:12:33.754 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-03-04T13:12:33.754 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-03-04T13:12:33.754 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/usb_debug.ko 2024-03-04T13:12:33.754 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/usb_wwan.ko 2024-03-04T13:12:33.754 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/visor.ko 2024-03-04T13:12:33.754 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/whiteheat.ko 2024-03-04T13:12:33.754 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/xsens_mt.ko 2024-03-04T13:12:33.754 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/storage 2024-03-04T13:12:33.754 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/storage/uas.ko 2024-03-04T13:12:33.754 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/storage/ums-alauda.ko 2024-03-04T13:12:33.754 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/storage/ums-cypress.ko 2024-03-04T13:12:33.754 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/storage/ums-datafab.ko 2024-03-04T13:12:33.755 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-03-04T13:12:33.755 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/storage/ums-freecom.ko 2024-03-04T13:12:33.755 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/storage/ums-isd200.ko 2024-03-04T13:12:33.755 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-03-04T13:12:33.755 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/storage/ums-karma.ko 2024-03-04T13:12:33.755 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/storage/ums-onetouch.ko 2024-03-04T13:12:33.755 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/storage/ums-realtek.ko 2024-03-04T13:12:33.755 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/storage/ums-sddr09.ko 2024-03-04T13:12:33.755 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/storage/ums-sddr55.ko 2024-03-04T13:12:33.755 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/storage/ums-usbat.ko 2024-03-04T13:12:33.755 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/storage/usb-storage.ko 2024-03-04T13:12:33.755 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/usbip 2024-03-04T13:12:33.755 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/usbip/usbip-core.ko 2024-03-04T13:12:33.755 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/usbip/usbip-host.ko 2024-03-04T13:12:33.755 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-03-04T13:12:33.756 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/vfio 2024-03-04T13:12:33.756 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/vfio/pci 2024-03-04T13:12:33.756 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-03-04T13:12:33.756 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/vfio/pci/vfio-pci.ko 2024-03-04T13:12:33.756 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/vfio/vfio.ko 2024-03-04T13:12:33.756 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-03-04T13:12:33.756 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/vhost 2024-03-04T13:12:33.756 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/vhost/vhost.ko 2024-03-04T13:12:33.756 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/vhost/vhost_iotlb.ko 2024-03-04T13:12:33.756 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/vhost/vhost_net.ko 2024-03-04T13:12:33.756 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/vhost/vhost_scsi.ko 2024-03-04T13:12:33.756 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/video 2024-03-04T13:12:33.756 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/video/backlight 2024-03-04T13:12:33.756 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/video/backlight/apple_bl.ko 2024-03-04T13:12:33.757 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/video/backlight/lcd.ko 2024-03-04T13:12:33.757 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/video/backlight/platform_lcd.ko 2024-03-04T13:12:33.757 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/virtio 2024-03-04T13:12:33.757 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/virtio/virtio.ko 2024-03-04T13:12:33.757 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/virtio/virtio_balloon.ko 2024-03-04T13:12:33.757 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/virtio/virtio_mmio.ko 2024-03-04T13:12:33.757 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/virtio/virtio_pci.ko 2024-03-04T13:12:33.757 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-03-04T13:12:33.757 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-03-04T13:12:33.757 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/virtio/virtio_ring.ko 2024-03-04T13:12:33.757 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/w1 2024-03-04T13:12:33.757 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/w1/masters 2024-03-04T13:12:33.757 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/w1/masters/ds2482.ko 2024-03-04T13:12:33.757 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/w1/masters/ds2490.ko 2024-03-04T13:12:33.757 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/w1/slaves 2024-03-04T13:12:33.757 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-03-04T13:12:33.758 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-03-04T13:12:33.758 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-03-04T13:12:33.758 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-03-04T13:12:33.758 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-03-04T13:12:33.758 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-03-04T13:12:33.758 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-03-04T13:12:33.758 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-03-04T13:12:33.758 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-03-04T13:12:33.758 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/w1/slaves/w1_smem.ko 2024-03-04T13:12:33.758 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/w1/slaves/w1_therm.ko 2024-03-04T13:12:33.758 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/w1/wire.ko 2024-03-04T13:12:33.758 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/watchdog 2024-03-04T13:12:33.758 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/watchdog/alim1535_wdt.ko 2024-03-04T13:12:33.758 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/watchdog/alim7101_wdt.ko 2024-03-04T13:12:33.759 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/watchdog/f71808e_wdt.ko 2024-03-04T13:12:33.759 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/watchdog/hpwdt.ko 2024-03-04T13:12:33.759 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/watchdog/i6300esb.ko 2024-03-04T13:12:33.759 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-03-04T13:12:33.759 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/watchdog/iTCO_wdt.ko 2024-03-04T13:12:33.759 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/watchdog/ib700wdt.ko 2024-03-04T13:12:33.759 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/watchdog/ibmasr.ko 2024-03-04T13:12:33.759 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-03-04T13:12:33.759 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/watchdog/it8712f_wdt.ko 2024-03-04T13:12:33.759 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/watchdog/it87_wdt.ko 2024-03-04T13:12:33.759 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/watchdog/machzwd.ko 2024-03-04T13:12:33.759 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/watchdog/nv_tco.ko 2024-03-04T13:12:33.759 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/watchdog/pcwd_pci.ko 2024-03-04T13:12:33.759 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/watchdog/pcwd_usb.ko 2024-03-04T13:12:33.759 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-03-04T13:12:33.760 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/watchdog/sch311x_wdt.ko 2024-03-04T13:12:33.760 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/watchdog/softdog.ko 2024-03-04T13:12:33.760 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/watchdog/sp5100_tco.ko 2024-03-04T13:12:33.760 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/watchdog/via_wdt.ko 2024-03-04T13:12:33.760 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-03-04T13:12:33.760 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/watchdog/w83877f_wdt.ko 2024-03-04T13:12:33.760 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/watchdog/w83977f_wdt.ko 2024-03-04T13:12:33.760 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/watchdog/wdt_pci.ko 2024-03-04T13:12:33.760 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/watchdog/xen_wdt.ko 2024-03-04T13:12:33.760 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/xen 2024-03-04T13:12:33.760 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/xen/xen-acpi-processor.ko 2024-03-04T13:12:33.760 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/xen/xen-evtchn.ko 2024-03-04T13:12:33.760 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/xen/xen-gntalloc.ko 2024-03-04T13:12:33.760 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/xen/xen-gntdev.ko 2024-03-04T13:12:33.760 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/xen/xen-pciback 2024-03-04T13:12:33.761 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-03-04T13:12:33.761 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/xen/xen-privcmd.ko 2024-03-04T13:12:33.761 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/xen/xenfs 2024-03-04T13:12:33.761 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/xen/xenfs/xenfs.ko 2024-03-04T13:12:33.761 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs 2024-03-04T13:12:33.761 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/9p 2024-03-04T13:12:33.761 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/9p/9p.ko 2024-03-04T13:12:33.761 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/affs 2024-03-04T13:12:33.761 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/affs/affs.ko 2024-03-04T13:12:33.761 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/befs 2024-03-04T13:12:33.761 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/befs/befs.ko 2024-03-04T13:12:33.761 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/binfmt_misc.ko 2024-03-04T13:12:33.761 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/btrfs 2024-03-04T13:12:33.761 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/btrfs/btrfs.ko 2024-03-04T13:12:33.761 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/cachefiles 2024-03-04T13:12:33.762 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/cachefiles/cachefiles.ko 2024-03-04T13:12:33.762 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/ceph 2024-03-04T13:12:33.762 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/ceph/ceph.ko 2024-03-04T13:12:33.762 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/coda 2024-03-04T13:12:33.762 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/coda/coda.ko 2024-03-04T13:12:33.762 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/cramfs 2024-03-04T13:12:33.762 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/cramfs/cramfs.ko 2024-03-04T13:12:33.762 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/dlm 2024-03-04T13:12:33.762 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/dlm/dlm.ko 2024-03-04T13:12:33.762 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/ecryptfs 2024-03-04T13:12:33.762 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/ecryptfs/ecryptfs.ko 2024-03-04T13:12:33.762 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/f2fs 2024-03-04T13:12:33.762 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/f2fs/f2fs.ko 2024-03-04T13:12:33.762 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/fat 2024-03-04T13:12:33.763 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/fat/fat.ko 2024-03-04T13:12:33.763 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/fat/msdos.ko 2024-03-04T13:12:33.763 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/fat/vfat.ko 2024-03-04T13:12:33.763 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/fuse 2024-03-04T13:12:33.763 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/fuse/cuse.ko 2024-03-04T13:12:33.763 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/fuse/fuse.ko 2024-03-04T13:12:33.763 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/gfs2 2024-03-04T13:12:33.763 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/gfs2/gfs2.ko 2024-03-04T13:12:33.763 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/hfs 2024-03-04T13:12:33.763 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/hfs/hfs.ko 2024-03-04T13:12:33.763 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/hfsplus 2024-03-04T13:12:33.763 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/hfsplus/hfsplus.ko 2024-03-04T13:12:33.763 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/isofs 2024-03-04T13:12:33.763 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/isofs/isofs.ko 2024-03-04T13:12:33.763 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/jfs 2024-03-04T13:12:33.764 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/jfs/jfs.ko 2024-03-04T13:12:33.764 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/lockd 2024-03-04T13:12:33.764 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/lockd/lockd.ko 2024-03-04T13:12:33.764 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/minix 2024-03-04T13:12:33.764 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/minix/minix.ko 2024-03-04T13:12:33.764 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/netfs 2024-03-04T13:12:33.764 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/netfs/netfs.ko 2024-03-04T13:12:33.764 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nfs 2024-03-04T13:12:33.764 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nfs/blocklayout 2024-03-04T13:12:33.764 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-03-04T13:12:33.764 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nfs/filelayout 2024-03-04T13:12:33.764 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-03-04T13:12:33.764 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nfs/flexfilelayout 2024-03-04T13:12:33.764 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-03-04T13:12:33.764 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nfs/nfs.ko 2024-03-04T13:12:33.765 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nfs/nfsv3.ko 2024-03-04T13:12:33.765 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nfs/nfsv4.ko 2024-03-04T13:12:33.765 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nfs_common 2024-03-04T13:12:33.765 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nfs_common/grace.ko 2024-03-04T13:12:33.765 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nfs_common/nfs_acl.ko 2024-03-04T13:12:33.765 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nfsd 2024-03-04T13:12:33.765 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nfsd/nfsd.ko 2024-03-04T13:12:33.765 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nilfs2 2024-03-04T13:12:33.765 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nilfs2/nilfs2.ko 2024-03-04T13:12:33.765 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls 2024-03-04T13:12:33.765 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/mac-celtic.ko 2024-03-04T13:12:33.765 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/mac-centeuro.ko 2024-03-04T13:12:33.765 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/mac-croatian.ko 2024-03-04T13:12:33.765 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/mac-cyrillic.ko 2024-03-04T13:12:33.765 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/mac-gaelic.ko 2024-03-04T13:12:33.766 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/mac-greek.ko 2024-03-04T13:12:33.766 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/mac-iceland.ko 2024-03-04T13:12:33.766 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/mac-inuit.ko 2024-03-04T13:12:33.766 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/mac-roman.ko 2024-03-04T13:12:33.766 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/mac-romanian.ko 2024-03-04T13:12:33.766 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/mac-turkish.ko 2024-03-04T13:12:33.766 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_cp1250.ko 2024-03-04T13:12:33.766 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_cp1251.ko 2024-03-04T13:12:33.766 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_cp1255.ko 2024-03-04T13:12:33.766 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_cp737.ko 2024-03-04T13:12:33.766 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_cp775.ko 2024-03-04T13:12:33.766 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_cp850.ko 2024-03-04T13:12:33.766 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_cp852.ko 2024-03-04T13:12:33.766 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_cp855.ko 2024-03-04T13:12:33.767 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_cp857.ko 2024-03-04T13:12:33.767 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_cp860.ko 2024-03-04T13:12:33.767 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_cp861.ko 2024-03-04T13:12:33.767 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_cp862.ko 2024-03-04T13:12:33.767 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_cp863.ko 2024-03-04T13:12:33.767 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_cp864.ko 2024-03-04T13:12:33.767 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_cp865.ko 2024-03-04T13:12:33.767 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_cp866.ko 2024-03-04T13:12:33.767 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_cp869.ko 2024-03-04T13:12:33.767 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_cp874.ko 2024-03-04T13:12:33.767 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_cp932.ko 2024-03-04T13:12:33.767 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_cp936.ko 2024-03-04T13:12:33.767 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_cp949.ko 2024-03-04T13:12:33.767 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_cp950.ko 2024-03-04T13:12:33.767 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_euc-jp.ko 2024-03-04T13:12:33.768 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_iso8859-1.ko 2024-03-04T13:12:33.768 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_iso8859-13.ko 2024-03-04T13:12:33.768 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_iso8859-14.ko 2024-03-04T13:12:33.768 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_iso8859-15.ko 2024-03-04T13:12:33.768 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_iso8859-2.ko 2024-03-04T13:12:33.768 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_iso8859-3.ko 2024-03-04T13:12:33.768 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_iso8859-4.ko 2024-03-04T13:12:33.768 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_iso8859-5.ko 2024-03-04T13:12:33.768 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_iso8859-6.ko 2024-03-04T13:12:33.768 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_iso8859-7.ko 2024-03-04T13:12:33.768 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_iso8859-9.ko 2024-03-04T13:12:33.768 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_koi8-r.ko 2024-03-04T13:12:33.768 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_koi8-ru.ko 2024-03-04T13:12:33.768 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_koi8-u.ko 2024-03-04T13:12:33.769 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_ucs2_utils.ko 2024-03-04T13:12:33.769 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_utf8.ko 2024-03-04T13:12:33.769 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/ocfs2 2024-03-04T13:12:33.769 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/ocfs2/cluster 2024-03-04T13:12:33.769 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-03-04T13:12:33.769 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/ocfs2/dlm 2024-03-04T13:12:33.769 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-03-04T13:12:33.769 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/ocfs2/dlmfs 2024-03-04T13:12:33.769 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-03-04T13:12:33.769 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/ocfs2/ocfs2.ko 2024-03-04T13:12:33.769 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-03-04T13:12:33.769 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-03-04T13:12:33.769 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-03-04T13:12:33.769 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/overlayfs 2024-03-04T13:12:33.769 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/overlayfs/overlay.ko 2024-03-04T13:12:33.770 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/pstore 2024-03-04T13:12:33.770 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/pstore/ramoops.ko 2024-03-04T13:12:33.770 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/reiserfs 2024-03-04T13:12:33.770 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/reiserfs/reiserfs.ko 2024-03-04T13:12:33.770 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/romfs 2024-03-04T13:12:33.770 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/romfs/romfs.ko 2024-03-04T13:12:33.770 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/smb 2024-03-04T13:12:33.770 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/smb/client 2024-03-04T13:12:33.770 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/smb/client/cifs.ko 2024-03-04T13:12:33.770 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/smb/common 2024-03-04T13:12:33.770 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/smb/common/cifs_arc4.ko 2024-03-04T13:12:33.770 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/smb/common/cifs_md4.ko 2024-03-04T13:12:33.770 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/squashfs 2024-03-04T13:12:33.770 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/squashfs/squashfs.ko 2024-03-04T13:12:33.770 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/sysv 2024-03-04T13:12:33.771 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/sysv/sysv.ko 2024-03-04T13:12:33.771 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/udf 2024-03-04T13:12:33.771 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/udf/udf.ko 2024-03-04T13:12:33.771 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/ufs 2024-03-04T13:12:33.771 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/ufs/ufs.ko 2024-03-04T13:12:33.771 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/xfs 2024-03-04T13:12:33.771 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/xfs/xfs.ko 2024-03-04T13:12:33.771 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/kernel 2024-03-04T13:12:33.771 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/kernel/rcu 2024-03-04T13:12:33.771 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/kernel/rcu/rcutorture.ko 2024-03-04T13:12:33.771 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/kernel/torture.ko 2024-03-04T13:12:33.771 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/kernel/trace 2024-03-04T13:12:33.771 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-03-04T13:12:33.771 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/lib 2024-03-04T13:12:33.771 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/lib/asn1_encoder.ko 2024-03-04T13:12:33.772 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/lib/crc-itu-t.ko 2024-03-04T13:12:33.772 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/lib/crc8.ko 2024-03-04T13:12:33.772 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/lib/crypto 2024-03-04T13:12:33.772 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/lib/crypto/libarc4.ko 2024-03-04T13:12:33.772 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/lib/crypto/libdes.ko 2024-03-04T13:12:33.772 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/lib/libcrc32c.ko 2024-03-04T13:12:33.772 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/lib/lru_cache.ko 2024-03-04T13:12:33.772 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/lib/lz4 2024-03-04T13:12:33.772 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/lib/lz4/lz4_compress.ko 2024-03-04T13:12:33.772 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/lib/lz4/lz4hc_compress.ko 2024-03-04T13:12:33.772 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/lib/math 2024-03-04T13:12:33.772 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/lib/math/cordic.ko 2024-03-04T13:12:33.772 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/lib/raid6 2024-03-04T13:12:33.772 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/lib/raid6/raid6_pq.ko 2024-03-04T13:12:33.773 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/lib/reed_solomon 2024-03-04T13:12:33.773 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/lib/reed_solomon/reed_solomon.ko 2024-03-04T13:12:33.773 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/lib/ts_bm.ko 2024-03-04T13:12:33.773 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/lib/ts_fsm.ko 2024-03-04T13:12:33.773 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/lib/ts_kmp.ko 2024-03-04T13:12:33.773 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/lib/zstd 2024-03-04T13:12:33.773 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/lib/zstd/zstd_compress.ko 2024-03-04T13:12:33.773 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/mm 2024-03-04T13:12:33.773 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/mm/hwpoison-inject.ko 2024-03-04T13:12:33.773 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net 2024-03-04T13:12:33.773 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/6lowpan 2024-03-04T13:12:33.773 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/6lowpan/6lowpan.ko 2024-03-04T13:12:33.773 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/6lowpan/nhc_dest.ko 2024-03-04T13:12:33.773 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/6lowpan/nhc_fragment.ko 2024-03-04T13:12:33.773 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/6lowpan/nhc_hop.ko 2024-03-04T13:12:33.774 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/6lowpan/nhc_ipv6.ko 2024-03-04T13:12:33.774 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/6lowpan/nhc_mobility.ko 2024-03-04T13:12:33.774 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/6lowpan/nhc_routing.ko 2024-03-04T13:12:33.774 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/6lowpan/nhc_udp.ko 2024-03-04T13:12:33.774 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/802 2024-03-04T13:12:33.774 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/802/garp.ko 2024-03-04T13:12:33.774 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/802/mrp.ko 2024-03-04T13:12:33.774 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/802/p8022.ko 2024-03-04T13:12:33.774 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/802/psnap.ko 2024-03-04T13:12:33.774 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/802/stp.ko 2024-03-04T13:12:33.774 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/8021q 2024-03-04T13:12:33.774 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/8021q/8021q.ko 2024-03-04T13:12:33.774 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/9p 2024-03-04T13:12:33.774 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/9p/9pnet.ko 2024-03-04T13:12:33.775 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/9p/9pnet_fd.ko 2024-03-04T13:12:33.775 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/9p/9pnet_rdma.ko 2024-03-04T13:12:33.775 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/9p/9pnet_virtio.ko 2024-03-04T13:12:33.775 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/batman-adv 2024-03-04T13:12:33.775 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/batman-adv/batman-adv.ko 2024-03-04T13:12:33.775 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/bridge 2024-03-04T13:12:33.775 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/bridge/br_netfilter.ko 2024-03-04T13:12:33.775 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/bridge/bridge.ko 2024-03-04T13:12:33.775 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/bridge/netfilter 2024-03-04T13:12:33.775 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-03-04T13:12:33.775 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/bridge/netfilter/ebt_among.ko 2024-03-04T13:12:33.775 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/bridge/netfilter/ebt_arp.ko 2024-03-04T13:12:33.775 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-03-04T13:12:33.775 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-03-04T13:12:33.775 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/bridge/netfilter/ebt_ip.ko 2024-03-04T13:12:33.776 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-03-04T13:12:33.776 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/bridge/netfilter/ebt_limit.ko 2024-03-04T13:12:33.776 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/bridge/netfilter/ebt_log.ko 2024-03-04T13:12:33.776 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/bridge/netfilter/ebt_mark.ko 2024-03-04T13:12:33.776 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-03-04T13:12:33.776 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-03-04T13:12:33.776 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-03-04T13:12:33.776 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-03-04T13:12:33.776 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/bridge/netfilter/ebt_snat.ko 2024-03-04T13:12:33.776 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/bridge/netfilter/ebt_stp.ko 2024-03-04T13:12:33.776 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-03-04T13:12:33.776 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-03-04T13:12:33.776 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-03-04T13:12:33.776 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-03-04T13:12:33.777 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/bridge/netfilter/ebtables.ko 2024-03-04T13:12:33.777 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-03-04T13:12:33.777 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-03-04T13:12:33.777 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-03-04T13:12:33.777 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ceph 2024-03-04T13:12:33.777 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ceph/libceph.ko 2024-03-04T13:12:33.777 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/core 2024-03-04T13:12:33.777 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/core/failover.ko 2024-03-04T13:12:33.777 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/core/pktgen.ko 2024-03-04T13:12:33.777 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/dccp 2024-03-04T13:12:33.777 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/dccp/dccp.ko 2024-03-04T13:12:33.777 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/dccp/dccp_diag.ko 2024-03-04T13:12:33.777 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/dccp/dccp_ipv4.ko 2024-03-04T13:12:33.777 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/dccp/dccp_ipv6.ko 2024-03-04T13:12:33.778 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/dns_resolver 2024-03-04T13:12:33.778 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/dns_resolver/dns_resolver.ko 2024-03-04T13:12:33.778 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/dsa 2024-03-04T13:12:33.778 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/dsa/dsa_core.ko 2024-03-04T13:12:33.778 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/dsa/tag_brcm.ko 2024-03-04T13:12:33.778 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/dsa/tag_dsa.ko 2024-03-04T13:12:33.778 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/dsa/tag_none.ko 2024-03-04T13:12:33.778 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/dsa/tag_trailer.ko 2024-03-04T13:12:33.778 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ieee802154 2024-03-04T13:12:33.778 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ieee802154/6lowpan 2024-03-04T13:12:33.778 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-03-04T13:12:33.778 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ieee802154/ieee802154.ko 2024-03-04T13:12:33.778 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ieee802154/ieee802154_socket.ko 2024-03-04T13:12:33.778 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4 2024-03-04T13:12:33.778 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/ah4.ko 2024-03-04T13:12:33.778 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/esp4.ko 2024-03-04T13:12:33.779 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/fou.ko 2024-03-04T13:12:33.779 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/gre.ko 2024-03-04T13:12:33.779 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/inet_diag.ko 2024-03-04T13:12:33.779 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/ip_gre.ko 2024-03-04T13:12:33.779 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/ip_tunnel.ko 2024-03-04T13:12:33.779 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/ip_vti.ko 2024-03-04T13:12:33.779 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/ipcomp.ko 2024-03-04T13:12:33.779 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/ipip.ko 2024-03-04T13:12:33.779 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/netfilter 2024-03-04T13:12:33.779 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/netfilter/arp_tables.ko 2024-03-04T13:12:33.779 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-03-04T13:12:33.779 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-03-04T13:12:33.779 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-03-04T13:12:33.779 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-03-04T13:12:33.779 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-03-04T13:12:33.780 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-03-04T13:12:33.780 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-03-04T13:12:33.780 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-03-04T13:12:33.780 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-03-04T13:12:33.780 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/netfilter/iptable_security.ko 2024-03-04T13:12:33.780 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-03-04T13:12:33.780 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-03-04T13:12:33.780 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-03-04T13:12:33.780 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-03-04T13:12:33.780 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-03-04T13:12:33.780 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-03-04T13:12:33.780 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-03-04T13:12:33.780 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-03-04T13:12:33.780 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-03-04T13:12:33.781 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-03-04T13:12:33.781 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/tcp_bic.ko 2024-03-04T13:12:33.781 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/tcp_dctcp.ko 2024-03-04T13:12:33.781 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/tcp_diag.ko 2024-03-04T13:12:33.781 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/tcp_highspeed.ko 2024-03-04T13:12:33.781 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/tcp_htcp.ko 2024-03-04T13:12:33.781 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/tcp_hybla.ko 2024-03-04T13:12:33.781 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/tcp_illinois.ko 2024-03-04T13:12:33.781 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/tcp_lp.ko 2024-03-04T13:12:33.781 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/tcp_scalable.ko 2024-03-04T13:12:33.781 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/tcp_vegas.ko 2024-03-04T13:12:33.781 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/tcp_veno.ko 2024-03-04T13:12:33.781 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/tcp_westwood.ko 2024-03-04T13:12:33.781 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/tcp_yeah.ko 2024-03-04T13:12:33.781 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/tunnel4.ko 2024-03-04T13:12:33.782 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/udp_diag.ko 2024-03-04T13:12:33.782 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/udp_tunnel.ko 2024-03-04T13:12:33.782 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/xfrm4_tunnel.ko 2024-03-04T13:12:33.782 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6 2024-03-04T13:12:33.782 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/ah6.ko 2024-03-04T13:12:33.782 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/esp6.ko 2024-03-04T13:12:33.782 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/fou6.ko 2024-03-04T13:12:33.782 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/ip6_tunnel.ko 2024-03-04T13:12:33.782 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-03-04T13:12:33.782 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/ip6_vti.ko 2024-03-04T13:12:33.782 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/ipcomp6.ko 2024-03-04T13:12:33.782 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/netfilter 2024-03-04T13:12:33.782 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-03-04T13:12:33.782 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-03-04T13:12:33.782 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-03-04T13:12:33.783 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-03-04T13:12:33.783 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-03-04T13:12:33.783 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-03-04T13:12:33.783 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-03-04T13:12:33.783 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-03-04T13:12:33.783 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-03-04T13:12:33.783 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-03-04T13:12:33.783 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-03-04T13:12:33.783 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-03-04T13:12:33.783 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-03-04T13:12:33.783 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-03-04T13:12:33.783 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-03-04T13:12:33.783 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-03-04T13:12:33.783 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-03-04T13:12:33.784 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-03-04T13:12:33.784 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-03-04T13:12:33.784 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-03-04T13:12:33.784 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-03-04T13:12:33.784 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-03-04T13:12:33.784 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-03-04T13:12:33.784 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-03-04T13:12:33.784 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/sit.ko 2024-03-04T13:12:33.784 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/tunnel6.ko 2024-03-04T13:12:33.784 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/xfrm6_tunnel.ko 2024-03-04T13:12:33.784 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/key 2024-03-04T13:12:33.784 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/key/af_key.ko 2024-03-04T13:12:33.784 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/l2tp 2024-03-04T13:12:33.784 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/l2tp/l2tp_core.ko 2024-03-04T13:12:33.784 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/l2tp/l2tp_debugfs.ko 2024-03-04T13:12:33.785 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/l2tp/l2tp_eth.ko 2024-03-04T13:12:33.785 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/l2tp/l2tp_ip.ko 2024-03-04T13:12:33.785 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/l2tp/l2tp_ip6.ko 2024-03-04T13:12:33.785 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/l2tp/l2tp_netlink.ko 2024-03-04T13:12:33.785 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/l2tp/l2tp_ppp.ko 2024-03-04T13:12:33.785 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/llc 2024-03-04T13:12:33.785 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/llc/llc.ko 2024-03-04T13:12:33.785 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/mac802154 2024-03-04T13:12:33.785 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/mac802154/mac802154.ko 2024-03-04T13:12:33.785 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/mpls 2024-03-04T13:12:33.785 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/mpls/mpls_gso.ko 2024-03-04T13:12:33.785 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter 2024-03-04T13:12:33.785 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/ipset 2024-03-04T13:12:33.785 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/ipset/ip_set.ko 2024-03-04T13:12:33.785 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-03-04T13:12:33.786 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-03-04T13:12:33.786 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-03-04T13:12:33.786 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-03-04T13:12:33.786 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-03-04T13:12:33.786 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-03-04T13:12:33.786 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-03-04T13:12:33.786 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-03-04T13:12:33.786 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-03-04T13:12:33.786 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-03-04T13:12:33.786 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-03-04T13:12:33.786 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-03-04T13:12:33.786 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-03-04T13:12:33.786 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-03-04T13:12:33.786 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-03-04T13:12:33.787 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/ipvs 2024-03-04T13:12:33.787 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/ipvs/ip_vs.ko 2024-03-04T13:12:33.787 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-03-04T13:12:33.787 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-03-04T13:12:33.787 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-03-04T13:12:33.787 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-03-04T13:12:33.787 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-03-04T13:12:33.787 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-03-04T13:12:33.787 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-03-04T13:12:33.787 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-03-04T13:12:33.787 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-03-04T13:12:33.787 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-03-04T13:12:33.787 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-03-04T13:12:33.787 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-03-04T13:12:33.787 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-03-04T13:12:33.788 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nf_conncount.ko 2024-03-04T13:12:33.788 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nf_conntrack.ko 2024-03-04T13:12:33.788 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-03-04T13:12:33.788 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-03-04T13:12:33.788 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-03-04T13:12:33.788 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nf_conntrack_h323.ko 2024-03-04T13:12:33.788 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nf_conntrack_irc.ko 2024-03-04T13:12:33.788 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-03-04T13:12:33.788 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-03-04T13:12:33.788 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-03-04T13:12:33.788 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nf_conntrack_sane.ko 2024-03-04T13:12:33.788 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nf_conntrack_sip.ko 2024-03-04T13:12:33.788 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-03-04T13:12:33.788 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-03-04T13:12:33.789 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nf_dup_netdev.ko 2024-03-04T13:12:33.789 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nf_flow_table.ko 2024-03-04T13:12:33.789 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nf_flow_table_inet.ko 2024-03-04T13:12:33.789 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nf_log_syslog.ko 2024-03-04T13:12:33.789 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nf_nat.ko 2024-03-04T13:12:33.789 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nf_nat_amanda.ko 2024-03-04T13:12:33.789 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nf_nat_ftp.ko 2024-03-04T13:12:33.789 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nf_nat_irc.ko 2024-03-04T13:12:33.789 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nf_nat_sip.ko 2024-03-04T13:12:33.789 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nf_nat_tftp.ko 2024-03-04T13:12:33.789 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nf_synproxy_core.ko 2024-03-04T13:12:33.789 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nf_tables.ko 2024-03-04T13:12:33.789 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nfnetlink.ko 2024-03-04T13:12:33.789 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nfnetlink_acct.ko 2024-03-04T13:12:33.790 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nfnetlink_log.ko 2024-03-04T13:12:33.790 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nfnetlink_osf.ko 2024-03-04T13:12:33.790 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nfnetlink_queue.ko 2024-03-04T13:12:33.790 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nft_chain_nat.ko 2024-03-04T13:12:33.790 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nft_compat.ko 2024-03-04T13:12:33.790 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nft_ct.ko 2024-03-04T13:12:33.790 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nft_dup_netdev.ko 2024-03-04T13:12:33.790 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nft_fib.ko 2024-03-04T13:12:33.790 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nft_fwd_netdev.ko 2024-03-04T13:12:33.790 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nft_hash.ko 2024-03-04T13:12:33.790 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nft_limit.ko 2024-03-04T13:12:33.790 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nft_log.ko 2024-03-04T13:12:33.790 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nft_masq.ko 2024-03-04T13:12:33.790 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nft_nat.ko 2024-03-04T13:12:33.790 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nft_numgen.ko 2024-03-04T13:12:33.791 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nft_queue.ko 2024-03-04T13:12:33.791 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nft_quota.ko 2024-03-04T13:12:33.791 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nft_redir.ko 2024-03-04T13:12:33.791 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nft_reject.ko 2024-03-04T13:12:33.791 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nft_reject_inet.ko 2024-03-04T13:12:33.791 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nft_socket.ko 2024-03-04T13:12:33.791 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nft_tunnel.ko 2024-03-04T13:12:33.791 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_AUDIT.ko 2024-03-04T13:12:33.791 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_CHECKSUM.ko 2024-03-04T13:12:33.791 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_CLASSIFY.ko 2024-03-04T13:12:33.791 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-03-04T13:12:33.791 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_CT.ko 2024-03-04T13:12:33.791 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_DSCP.ko 2024-03-04T13:12:33.791 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_HL.ko 2024-03-04T13:12:33.792 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_HMARK.ko 2024-03-04T13:12:33.792 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_IDLETIMER.ko 2024-03-04T13:12:33.792 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_LED.ko 2024-03-04T13:12:33.792 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_LOG.ko 2024-03-04T13:12:33.792 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_MASQUERADE.ko 2024-03-04T13:12:33.792 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_NETMAP.ko 2024-03-04T13:12:33.792 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_NFLOG.ko 2024-03-04T13:12:33.792 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_NFQUEUE.ko 2024-03-04T13:12:33.792 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_RATEEST.ko 2024-03-04T13:12:33.792 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_REDIRECT.ko 2024-03-04T13:12:33.792 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_SECMARK.ko 2024-03-04T13:12:33.792 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_TCPMSS.ko 2024-03-04T13:12:33.792 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-03-04T13:12:33.792 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_TEE.ko 2024-03-04T13:12:33.793 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_TPROXY.ko 2024-03-04T13:12:33.793 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_TRACE.ko 2024-03-04T13:12:33.793 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_addrtype.ko 2024-03-04T13:12:33.793 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_bpf.ko 2024-03-04T13:12:33.793 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_cgroup.ko 2024-03-04T13:12:33.793 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_cluster.ko 2024-03-04T13:12:33.793 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_comment.ko 2024-03-04T13:12:33.793 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_connbytes.ko 2024-03-04T13:12:33.793 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_connlabel.ko 2024-03-04T13:12:33.793 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_connlimit.ko 2024-03-04T13:12:33.793 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_connmark.ko 2024-03-04T13:12:33.793 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_conntrack.ko 2024-03-04T13:12:33.793 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_cpu.ko 2024-03-04T13:12:33.793 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_dccp.ko 2024-03-04T13:12:33.793 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_devgroup.ko 2024-03-04T13:12:33.794 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_dscp.ko 2024-03-04T13:12:33.794 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_ecn.ko 2024-03-04T13:12:33.794 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_esp.ko 2024-03-04T13:12:33.794 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_hashlimit.ko 2024-03-04T13:12:33.794 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_helper.ko 2024-03-04T13:12:33.794 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_hl.ko 2024-03-04T13:12:33.794 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_ipcomp.ko 2024-03-04T13:12:33.794 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_iprange.ko 2024-03-04T13:12:33.794 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_ipvs.ko 2024-03-04T13:12:33.794 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_l2tp.ko 2024-03-04T13:12:33.794 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_length.ko 2024-03-04T13:12:33.794 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_limit.ko 2024-03-04T13:12:33.794 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_mac.ko 2024-03-04T13:12:33.794 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_mark.ko 2024-03-04T13:12:33.794 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_multiport.ko 2024-03-04T13:12:33.795 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_nat.ko 2024-03-04T13:12:33.795 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_nfacct.ko 2024-03-04T13:12:33.795 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_osf.ko 2024-03-04T13:12:33.795 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_owner.ko 2024-03-04T13:12:33.795 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_physdev.ko 2024-03-04T13:12:33.795 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_pkttype.ko 2024-03-04T13:12:33.795 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_policy.ko 2024-03-04T13:12:33.795 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_quota.ko 2024-03-04T13:12:33.795 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_rateest.ko 2024-03-04T13:12:33.795 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_realm.ko 2024-03-04T13:12:33.795 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_recent.ko 2024-03-04T13:12:33.795 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_sctp.ko 2024-03-04T13:12:33.795 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_set.ko 2024-03-04T13:12:33.795 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_socket.ko 2024-03-04T13:12:33.795 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_state.ko 2024-03-04T13:12:33.796 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_statistic.ko 2024-03-04T13:12:33.796 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_string.ko 2024-03-04T13:12:33.796 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_tcpmss.ko 2024-03-04T13:12:33.796 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_time.ko 2024-03-04T13:12:33.796 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_u32.ko 2024-03-04T13:12:33.796 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netlink 2024-03-04T13:12:33.796 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netlink/netlink_diag.ko 2024-03-04T13:12:33.796 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/nsh 2024-03-04T13:12:33.796 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/nsh/nsh.ko 2024-03-04T13:12:33.796 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/openvswitch 2024-03-04T13:12:33.796 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/openvswitch/openvswitch.ko 2024-03-04T13:12:33.796 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/openvswitch/vport-geneve.ko 2024-03-04T13:12:33.796 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/openvswitch/vport-gre.ko 2024-03-04T13:12:33.796 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/openvswitch/vport-vxlan.ko 2024-03-04T13:12:33.797 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/packet 2024-03-04T13:12:33.797 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/packet/af_packet_diag.ko 2024-03-04T13:12:33.797 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/rds 2024-03-04T13:12:33.797 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/rds/rds.ko 2024-03-04T13:12:33.797 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/rds/rds_rdma.ko 2024-03-04T13:12:33.797 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/rds/rds_tcp.ko 2024-03-04T13:12:33.797 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched 2024-03-04T13:12:33.797 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/act_csum.ko 2024-03-04T13:12:33.797 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/act_gact.ko 2024-03-04T13:12:33.797 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/act_mirred.ko 2024-03-04T13:12:33.797 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/act_nat.ko 2024-03-04T13:12:33.797 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/act_pedit.ko 2024-03-04T13:12:33.797 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/act_police.ko 2024-03-04T13:12:33.797 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/act_simple.ko 2024-03-04T13:12:33.797 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/act_skbedit.ko 2024-03-04T13:12:33.798 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/act_vlan.ko 2024-03-04T13:12:33.798 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/cls_basic.ko 2024-03-04T13:12:33.798 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/cls_bpf.ko 2024-03-04T13:12:33.798 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/cls_flow.ko 2024-03-04T13:12:33.798 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/cls_fw.ko 2024-03-04T13:12:33.798 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/cls_route.ko 2024-03-04T13:12:33.798 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/cls_u32.ko 2024-03-04T13:12:33.798 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/em_cmp.ko 2024-03-04T13:12:33.798 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/em_ipset.ko 2024-03-04T13:12:33.798 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/em_meta.ko 2024-03-04T13:12:33.798 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/em_nbyte.ko 2024-03-04T13:12:33.798 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/em_text.ko 2024-03-04T13:12:33.798 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/em_u32.ko 2024-03-04T13:12:33.798 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/sch_choke.ko 2024-03-04T13:12:33.798 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/sch_codel.ko 2024-03-04T13:12:33.798 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/sch_drr.ko 2024-03-04T13:12:33.799 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/sch_fq.ko 2024-03-04T13:12:33.799 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/sch_gred.ko 2024-03-04T13:12:33.799 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/sch_hfsc.ko 2024-03-04T13:12:33.799 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/sch_hhf.ko 2024-03-04T13:12:33.799 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/sch_htb.ko 2024-03-04T13:12:33.799 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/sch_ingress.ko 2024-03-04T13:12:33.799 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/sch_mqprio.ko 2024-03-04T13:12:33.799 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/sch_mqprio_lib.ko 2024-03-04T13:12:33.799 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/sch_multiq.ko 2024-03-04T13:12:33.799 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/sch_netem.ko 2024-03-04T13:12:33.799 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/sch_pie.ko 2024-03-04T13:12:33.799 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/sch_plug.ko 2024-03-04T13:12:33.799 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/sch_prio.ko 2024-03-04T13:12:33.799 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/sch_qfq.ko 2024-03-04T13:12:33.800 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/sch_red.ko 2024-03-04T13:12:33.800 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/sch_sfb.ko 2024-03-04T13:12:33.800 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/sch_sfq.ko 2024-03-04T13:12:33.800 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/sch_tbf.ko 2024-03-04T13:12:33.800 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/sch_teql.ko 2024-03-04T13:12:33.800 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sctp 2024-03-04T13:12:33.800 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sctp/sctp.ko 2024-03-04T13:12:33.800 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sctp/sctp_diag.ko 2024-03-04T13:12:33.800 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sunrpc 2024-03-04T13:12:33.800 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sunrpc/auth_gss 2024-03-04T13:12:33.800 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-03-04T13:12:33.800 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-03-04T13:12:33.800 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sunrpc/sunrpc.ko 2024-03-04T13:12:33.800 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sunrpc/xprtrdma 2024-03-04T13:12:33.800 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-03-04T13:12:33.801 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/tipc 2024-03-04T13:12:33.801 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/tipc/diag.ko 2024-03-04T13:12:33.801 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/tipc/tipc.ko 2024-03-04T13:12:33.801 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/unix 2024-03-04T13:12:33.801 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/unix/unix_diag.ko 2024-03-04T13:12:33.801 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/vmw_vsock 2024-03-04T13:12:33.801 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-03-04T13:12:33.801 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-03-04T13:12:33.801 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/vmw_vsock/vsock.ko 2024-03-04T13:12:33.801 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/vmw_vsock/vsock_diag.ko 2024-03-04T13:12:33.801 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/vmw_vsock/vsock_loopback.ko 2024-03-04T13:12:33.801 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/xfrm 2024-03-04T13:12:33.801 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/xfrm/xfrm_ipcomp.ko 2024-03-04T13:12:33.801 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/security 2024-03-04T13:12:33.801 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/security/keys 2024-03-04T13:12:33.802 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/security/keys/encrypted-keys 2024-03-04T13:12:33.802 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-03-04T13:12:33.802 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/security/keys/trusted-keys 2024-03-04T13:12:33.802 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/security/keys/trusted-keys/trusted.ko 2024-03-04T13:12:33.802 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/virt 2024-03-04T13:12:33.802 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/virt/lib 2024-03-04T13:12:33.802 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/virt/lib/irqbypass.ko 2024-03-04T13:12:33.802 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/modules.alias 2024-03-04T13:12:33.802 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/modules.alias.bin 2024-03-04T13:12:33.802 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/modules.builtin 2024-03-04T13:12:33.802 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/modules.builtin.alias.bin 2024-03-04T13:12:33.802 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/modules.builtin.bin 2024-03-04T13:12:33.802 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/modules.builtin.modinfo 2024-03-04T13:12:33.802 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/modules.dep 2024-03-04T13:12:33.802 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/modules.dep.bin 2024-03-04T13:12:33.803 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/modules.devname 2024-03-04T13:12:33.803 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/modules.order 2024-03-04T13:12:33.803 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/modules.softdep 2024-03-04T13:12:33.803 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/modules.symbols 2024-03-04T13:12:33.803 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/modules.symbols.bin 2024-03-04T13:12:33.803 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/vmlinuz 2024-03-04T13:12:33.807 DEBUG:teuthology.task.kernel:get_image_version: 6.8.0-rc6-g786939134d7e 2024-03-04T13:12:33.807 DEBUG:teuthology.orchestra.run.smithi169:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-03-04T13:12:33.829 INFO:teuthology.orchestra.run.smithi169.stdout:postinstall scriptlet (using /bin/sh): 2024-03-04T13:12:33.829 INFO:teuthology.orchestra.run.smithi169.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-03-04T13:12:33.829 INFO:teuthology.orchestra.run.smithi169.stdout: /usr/bin/kernel-install add 6.8.0-rc6-g786939134d7e /lib/modules/6.8.0-rc6-g786939134d7e/vmlinuz 2024-03-04T13:12:33.829 INFO:teuthology.orchestra.run.smithi169.stdout:fi 2024-03-04T13:12:33.829 INFO:teuthology.orchestra.run.smithi169.stdout:for file in vmlinuz System.map config; do 2024-03-04T13:12:33.829 INFO:teuthology.orchestra.run.smithi169.stdout: if ! cmp --silent "/lib/modules/6.8.0-rc6-g786939134d7e/${file}" "/boot/${file}-6.8.0-rc6-g786939134d7e"; then 2024-03-04T13:12:33.829 INFO:teuthology.orchestra.run.smithi169.stdout: cp "/lib/modules/6.8.0-rc6-g786939134d7e/${file}" "/boot/${file}-6.8.0-rc6-g786939134d7e" 2024-03-04T13:12:33.830 INFO:teuthology.orchestra.run.smithi169.stdout: fi 2024-03-04T13:12:33.830 INFO:teuthology.orchestra.run.smithi169.stdout:done 2024-03-04T13:12:33.830 INFO:teuthology.orchestra.run.smithi169.stdout:preuninstall scriptlet (using /bin/sh): 2024-03-04T13:12:33.830 INFO:teuthology.orchestra.run.smithi169.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2024-03-04T13:12:33.830 INFO:teuthology.orchestra.run.smithi169.stdout:new-kernel-pkg --remove 6.8.0-rc6-g786939134d7e --rminitrd --initrdfile=/boot/initramfs-6.8.0-rc6-g786939134d7e.img 2024-03-04T13:12:33.830 INFO:teuthology.orchestra.run.smithi169.stdout:elif [ -x /usr/bin/kernel-install ]; then 2024-03-04T13:12:33.830 INFO:teuthology.orchestra.run.smithi169.stdout:kernel-install remove 6.8.0-rc6-g786939134d7e 2024-03-04T13:12:33.830 INFO:teuthology.orchestra.run.smithi169.stdout:fi 2024-03-04T13:12:33.830 INFO:teuthology.orchestra.run.smithi169.stdout:postuninstall scriptlet (using /bin/sh): 2024-03-04T13:12:33.830 INFO:teuthology.orchestra.run.smithi169.stdout:if [ -x /sbin/update-bootloader ]; then 2024-03-04T13:12:33.830 INFO:teuthology.orchestra.run.smithi169.stdout:/sbin/update-bootloader --remove 6.8.0-rc6-g786939134d7e 2024-03-04T13:12:33.830 INFO:teuthology.orchestra.run.smithi169.stdout:fi 2024-03-04T13:12:33.831 DEBUG:teuthology.orchestra.run.smithi169:> sudo rpm -qi grub2-tools 2024-03-04T13:12:33.911 INFO:teuthology.orchestra.run.smithi169.stdout:Name : grub2-tools 2024-03-04T13:12:33.911 INFO:teuthology.orchestra.run.smithi169.stdout:Epoch : 1 2024-03-04T13:12:33.911 INFO:teuthology.orchestra.run.smithi169.stdout:Version : 2.06 2024-03-04T13:12:33.911 INFO:teuthology.orchestra.run.smithi169.stdout:Release : 68.el9 2024-03-04T13:12:33.911 INFO:teuthology.orchestra.run.smithi169.stdout:Architecture: x86_64 2024-03-04T13:12:33.911 INFO:teuthology.orchestra.run.smithi169.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-04T13:12:33.911 INFO:teuthology.orchestra.run.smithi169.stdout:Group : Unspecified 2024-03-04T13:12:33.912 INFO:teuthology.orchestra.run.smithi169.stdout:Size : 8264254 2024-03-04T13:12:33.912 INFO:teuthology.orchestra.run.smithi169.stdout:License : GPLv3+ 2024-03-04T13:12:33.912 INFO:teuthology.orchestra.run.smithi169.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-04T13:12:33.912 INFO:teuthology.orchestra.run.smithi169.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-04T13:12:33.912 INFO:teuthology.orchestra.run.smithi169.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-04T13:12:33.912 INFO:teuthology.orchestra.run.smithi169.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-04T13:12:33.912 INFO:teuthology.orchestra.run.smithi169.stdout:Packager : builder@centos.org 2024-03-04T13:12:33.912 INFO:teuthology.orchestra.run.smithi169.stdout:Vendor : CentOS 2024-03-04T13:12:33.912 INFO:teuthology.orchestra.run.smithi169.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-04T13:12:33.912 INFO:teuthology.orchestra.run.smithi169.stdout:Summary : Support tools for GRUB. 2024-03-04T13:12:33.912 INFO:teuthology.orchestra.run.smithi169.stdout:Description : 2024-03-04T13:12:33.912 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-03-04T13:12:33.912 INFO:teuthology.orchestra.run.smithi169.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-04T13:12:33.913 INFO:teuthology.orchestra.run.smithi169.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-04T13:12:33.913 INFO:teuthology.orchestra.run.smithi169.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-04T13:12:33.913 INFO:teuthology.orchestra.run.smithi169.stdout:hardware devices. 2024-03-04T13:12:33.913 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-03-04T13:12:33.913 INFO:teuthology.orchestra.run.smithi169.stdout:This subpackage provides tools for support of all platforms. 2024-03-04T13:12:33.914 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-04T13:12:33.914 INFO:teuthology.task.kernel:Updating grub on smithi169 to boot 6.8.0-rc6-g786939134d7e 2024-03-04T13:12:33.914 DEBUG:teuthology.orchestra.run.smithi169:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-04T13:12:34.547 INFO:teuthology.orchestra.run.smithi169.stderr:Generating grub configuration file ... 2024-03-04T13:12:35.947 INFO:teuthology.orchestra.run.smithi169.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-04T13:12:35.972 INFO:teuthology.orchestra.run.smithi169.stderr:done 2024-03-04T13:12:35.981 DEBUG:teuthology.orchestra.run.smithi169:> mktemp 2024-03-04T13:12:35.996 INFO:teuthology.orchestra.run.smithi169.stdout:/tmp/tmp.WOS6voRwZl 2024-03-04T13:12:35.996 DEBUG:teuthology.orchestra.run.smithi169:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.WOS6voRwZl 2024-03-04T13:12:36.064 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod 0666 /tmp/tmp.WOS6voRwZl 2024-03-04T13:12:36.197 DEBUG:teuthology.orchestra.remote:smithi169:/tmp/tmp.WOS6voRwZl is 6KB 2024-03-04T13:12:36.245 DEBUG:teuthology.orchestra.run.smithi169:> rm -fr /tmp/tmp.WOS6voRwZl 2024-03-04T13:12:36.259 DEBUG:teuthology.orchestra.run.smithi169:> sudo /bin/ls /boot/loader/entries || true 2024-03-04T13:12:36.324 INFO:teuthology.orchestra.run.smithi169.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-04T13:12:36.325 INFO:teuthology.orchestra.run.smithi169.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-04T13:12:36.325 INFO:teuthology.orchestra.run.smithi169.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-04T13:12:36.325 INFO:teuthology.orchestra.run.smithi169.stdout:15949e3c88704300882482eddfd7baae-6.8.0-rc6-g786939134d7e.conf 2024-03-04T13:12:36.326 DEBUG:teuthology.orchestra.run.smithi169:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.8.0-rc6-g786939134d7e 2024-03-04T13:12:36.479 DEBUG:teuthology.orchestra.run.smithi169:> sudo shutdown -r now 2024-03-04T13:13:06.482 INFO:teuthology.task.kernel:Checking client host.b for new kernel version... 2024-03-04T13:13:06.483 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi169.front.sepia.ceph.com' 2024-03-04T13:13:06.484 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-04T13:13:25.052 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.169 2024-03-04T13:13:34.053 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi169.front.sepia.ceph.com' 2024-03-04T13:13:34.054 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-04T13:13:37.120 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.169 2024-03-04T13:13:49.120 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi169.front.sepia.ceph.com' 2024-03-04T13:13:49.121 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-04T13:13:55.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04 13:13:55 URL:https://3.chacra.ceph.com/r/kernel/testing/786939134d7eee4dc7a805245d3f595929af37d7/centos/9/flavors/default/x86_64/kernel-6.8.0_rc6_g786939134d7e-1.x86_64.rpm [336049686/336049686] -> "/tmp/kernel.x86_64.rpm" [1] 2024-03-04T13:13:55.862 INFO:teuthology.orchestra.run.smithi107.stderr:FINISHED --2024-03-04 13:13:55-- 2024-03-04T13:13:55.863 INFO:teuthology.orchestra.run.smithi107.stderr:Total wall clock time: 2m 16s 2024-03-04T13:13:55.863 INFO:teuthology.orchestra.run.smithi107.stderr:Downloaded: 1 files, 320M in 2m 15s (2.37 MB/s) 2024-03-04T13:13:55.867 INFO:teuthology.task.kernel:Installing kernel 786939134d7eee4dc7a805245d3f595929af37d7 on host.a... 2024-03-04T13:13:55.867 DEBUG:teuthology.orchestra.run.smithi107:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-03-04T13:13:57.838 INFO:teuthology.orchestra.run.smithi107.stdout:Verifying... ######################################## 2024-03-04T13:13:57.896 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing... ######################################## 2024-03-04T13:13:57.975 INFO:teuthology.orchestra.run.smithi107.stdout:Updating / installing... 2024-03-04T13:14:05.484 INFO:teuthology.orchestra.run.smithi107.stdout:kernel-6.8.0_rc6_g786939134d7e-1 ######################################## 2024-03-04T13:14:07.548 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.169 2024-03-04T13:14:19.801 INFO:teuthology.orchestra.run.smithi107.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-04T13:14:22.549 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi169.front.sepia.ceph.com' 2024-03-04T13:14:22.550 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-04T13:14:23.716 DEBUG:teuthology.orchestra.run.smithi169:> true 2024-03-04T13:14:24.417 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi169.front.sepia.ceph.com' 2024-03-04T13:14:24.417 INFO:teuthology.task.kernel:Checking kernel version of host.b, want "6.8.0-rc6-g786939134d7e"... 2024-03-04T13:14:24.417 DEBUG:teuthology.orchestra.run.smithi169:> uname -r 2024-03-04T13:14:24.454 INFO:teuthology.orchestra.run.smithi169.stdout:6.8.0-rc6-g786939134d7e 2024-03-04T13:14:24.455 DEBUG:teuthology.task.kernel:current kernel version is 6.8.0-rc6-g786939134d7e vs 6.8.0-rc6-g786939134d7e 2024-03-04T13:14:24.455 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-04T13:14:24.455 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-04T13:14:25.456 INFO:teuthology.task.kernel:Enabling kdb on host.b... 2024-03-04T13:14:25.456 DEBUG:teuthology.orchestra.run.smithi169:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-04T13:14:25.593 INFO:teuthology.orchestra.run.smithi169.stdout:ttyS1 2024-03-04T13:14:25.643 DEBUG:teuthology.parallel:result is None 2024-03-04T13:14:42.306 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi107.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-03-04T13:14:42.306 DEBUG:teuthology.orchestra.run.smithi107:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-03-04T13:14:42.397 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e 2024-03-04T13:14:42.397 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/System.map 2024-03-04T13:14:42.397 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/config 2024-03-04T13:14:42.398 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel 2024-03-04T13:14:42.398 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/arch 2024-03-04T13:14:42.398 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/arch/x86 2024-03-04T13:14:42.398 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/arch/x86/crypto 2024-03-04T13:14:42.398 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-03-04T13:14:42.398 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-03-04T13:14:42.398 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-03-04T13:14:42.398 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-03-04T13:14:42.398 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-03-04T13:14:42.398 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-03-04T13:14:42.398 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-03-04T13:14:42.398 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/arch/x86/crypto/crc32c-intel.ko 2024-03-04T13:14:42.399 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-03-04T13:14:42.399 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-03-04T13:14:42.399 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-03-04T13:14:42.399 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-03-04T13:14:42.399 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/arch/x86/crypto/serpent-avx2.ko 2024-03-04T13:14:42.399 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-03-04T13:14:42.399 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-03-04T13:14:42.399 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-03-04T13:14:42.399 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-03-04T13:14:42.399 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-03-04T13:14:42.399 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-03-04T13:14:42.399 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-03-04T13:14:42.399 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/arch/x86/kernel 2024-03-04T13:14:42.400 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/arch/x86/kernel/cpu 2024-03-04T13:14:42.400 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/arch/x86/kernel/cpu/mce 2024-03-04T13:14:42.400 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-03-04T13:14:42.400 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/arch/x86/kvm 2024-03-04T13:14:42.400 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/arch/x86/kvm/kvm-amd.ko 2024-03-04T13:14:42.400 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/arch/x86/kvm/kvm-intel.ko 2024-03-04T13:14:42.400 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/arch/x86/kvm/kvm.ko 2024-03-04T13:14:42.400 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto 2024-03-04T13:14:42.400 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/ansi_cprng.ko 2024-03-04T13:14:42.400 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/anubis.ko 2024-03-04T13:14:42.400 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/arc4.ko 2024-03-04T13:14:42.400 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/async_tx 2024-03-04T13:14:42.401 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/async_tx/async_memcpy.ko 2024-03-04T13:14:42.401 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/async_tx/async_pq.ko 2024-03-04T13:14:42.401 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/async_tx/async_raid6_recov.ko 2024-03-04T13:14:42.401 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/async_tx/async_tx.ko 2024-03-04T13:14:42.401 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/async_tx/async_xor.ko 2024-03-04T13:14:42.401 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/async_tx/raid6test.ko 2024-03-04T13:14:42.401 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/authenc.ko 2024-03-04T13:14:42.401 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/authencesn.ko 2024-03-04T13:14:42.401 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/blake2b_generic.ko 2024-03-04T13:14:42.401 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/blowfish_common.ko 2024-03-04T13:14:42.401 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/blowfish_generic.ko 2024-03-04T13:14:42.401 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/camellia_generic.ko 2024-03-04T13:14:42.401 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/cast5_generic.ko 2024-03-04T13:14:42.402 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/cast6_generic.ko 2024-03-04T13:14:42.402 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/cast_common.ko 2024-03-04T13:14:42.402 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/ccm.ko 2024-03-04T13:14:42.402 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/cmac.ko 2024-03-04T13:14:42.402 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/crc32_generic.ko 2024-03-04T13:14:42.402 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/crypto_engine.ko 2024-03-04T13:14:42.402 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/crypto_user.ko 2024-03-04T13:14:42.402 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/cts.ko 2024-03-04T13:14:42.402 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/des_generic.ko 2024-03-04T13:14:42.402 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/dh_generic.ko 2024-03-04T13:14:42.402 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/echainiv.ko 2024-03-04T13:14:42.402 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/essiv.ko 2024-03-04T13:14:42.402 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/fcrypt.ko 2024-03-04T13:14:42.402 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/khazad.ko 2024-03-04T13:14:42.403 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/lz4.ko 2024-03-04T13:14:42.403 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/lz4hc.ko 2024-03-04T13:14:42.403 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/md4.ko 2024-03-04T13:14:42.403 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/michael_mic.ko 2024-03-04T13:14:42.403 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/pcbc.ko 2024-03-04T13:14:42.403 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/pcrypt.ko 2024-03-04T13:14:42.403 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/rmd160.ko 2024-03-04T13:14:42.403 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/seed.ko 2024-03-04T13:14:42.403 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/serpent_generic.ko 2024-03-04T13:14:42.403 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/tcrypt.ko 2024-03-04T13:14:42.403 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/tea.ko 2024-03-04T13:14:42.403 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/twofish_common.ko 2024-03-04T13:14:42.403 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/twofish_generic.ko 2024-03-04T13:14:42.403 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/vmac.ko 2024-03-04T13:14:42.404 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/wp512.ko 2024-03-04T13:14:42.404 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/xcbc.ko 2024-03-04T13:14:42.404 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/xor.ko 2024-03-04T13:14:42.404 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/xxhash_generic.ko 2024-03-04T13:14:42.404 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers 2024-03-04T13:14:42.404 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/acpi 2024-03-04T13:14:42.404 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/acpi/acpi_ipmi.ko 2024-03-04T13:14:42.404 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/acpi/acpi_pad.ko 2024-03-04T13:14:42.404 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/acpi/custom_method.ko 2024-03-04T13:14:42.404 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/acpi/ec_sys.ko 2024-03-04T13:14:42.404 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/acpi/platform_profile.ko 2024-03-04T13:14:42.404 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/acpi/sbs.ko 2024-03-04T13:14:42.404 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/acpi/sbshc.ko 2024-03-04T13:14:42.405 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/acpi/video.ko 2024-03-04T13:14:42.405 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata 2024-03-04T13:14:42.405 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/acard-ahci.ko 2024-03-04T13:14:42.405 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/ahci_platform.ko 2024-03-04T13:14:42.405 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/ata_generic.ko 2024-03-04T13:14:42.405 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/libahci_platform.ko 2024-03-04T13:14:42.405 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_acpi.ko 2024-03-04T13:14:42.405 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_ali.ko 2024-03-04T13:14:42.405 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_amd.ko 2024-03-04T13:14:42.405 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_artop.ko 2024-03-04T13:14:42.405 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_atiixp.ko 2024-03-04T13:14:42.405 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_atp867x.ko 2024-03-04T13:14:42.405 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_cmd640.ko 2024-03-04T13:14:42.405 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_cmd64x.ko 2024-03-04T13:14:42.405 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_cypress.ko 2024-03-04T13:14:42.406 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_efar.ko 2024-03-04T13:14:42.406 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_hpt366.ko 2024-03-04T13:14:42.406 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_hpt37x.ko 2024-03-04T13:14:42.406 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_hpt3x2n.ko 2024-03-04T13:14:42.406 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_hpt3x3.ko 2024-03-04T13:14:42.406 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_it8213.ko 2024-03-04T13:14:42.406 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_it821x.ko 2024-03-04T13:14:42.406 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_jmicron.ko 2024-03-04T13:14:42.407 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_marvell.ko 2024-03-04T13:14:42.407 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_mpiix.ko 2024-03-04T13:14:42.407 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_netcell.ko 2024-03-04T13:14:42.407 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_ninja32.ko 2024-03-04T13:14:42.407 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_ns87410.ko 2024-03-04T13:14:42.407 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_ns87415.ko 2024-03-04T13:14:42.407 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_oldpiix.ko 2024-03-04T13:14:42.407 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_opti.ko 2024-03-04T13:14:42.407 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_optidma.ko 2024-03-04T13:14:42.407 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_pcmcia.ko 2024-03-04T13:14:42.407 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_pdc2027x.ko 2024-03-04T13:14:42.407 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-03-04T13:14:42.408 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_piccolo.ko 2024-03-04T13:14:42.408 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_rdc.ko 2024-03-04T13:14:42.408 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_sch.ko 2024-03-04T13:14:42.408 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_serverworks.ko 2024-03-04T13:14:42.408 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_sil680.ko 2024-03-04T13:14:42.408 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_sis.ko 2024-03-04T13:14:42.408 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_sl82c105.ko 2024-03-04T13:14:42.408 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_triflex.ko 2024-03-04T13:14:42.408 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_via.ko 2024-03-04T13:14:42.408 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pdc_adma.ko 2024-03-04T13:14:42.408 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/sata_inic162x.ko 2024-03-04T13:14:42.408 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/sata_mv.ko 2024-03-04T13:14:42.408 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/sata_nv.ko 2024-03-04T13:14:42.409 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/sata_promise.ko 2024-03-04T13:14:42.409 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/sata_qstor.ko 2024-03-04T13:14:42.409 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/sata_sil.ko 2024-03-04T13:14:42.409 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/sata_sil24.ko 2024-03-04T13:14:42.409 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/sata_sis.ko 2024-03-04T13:14:42.409 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/sata_svw.ko 2024-03-04T13:14:42.409 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/sata_sx4.ko 2024-03-04T13:14:42.409 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/sata_uli.ko 2024-03-04T13:14:42.409 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/sata_via.ko 2024-03-04T13:14:42.410 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/sata_vsc.ko 2024-03-04T13:14:42.410 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/auxdisplay 2024-03-04T13:14:42.410 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/auxdisplay/ks0108.ko 2024-03-04T13:14:42.410 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/base 2024-03-04T13:14:42.410 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/base/regmap 2024-03-04T13:14:42.411 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/base/regmap/regmap-i2c.ko 2024-03-04T13:14:42.411 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/base/regmap/regmap-mmio.ko 2024-03-04T13:14:42.411 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/bcma 2024-03-04T13:14:42.411 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/bcma/bcma.ko 2024-03-04T13:14:42.411 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/block 2024-03-04T13:14:42.411 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/block/aoe 2024-03-04T13:14:42.412 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/block/aoe/aoe.ko 2024-03-04T13:14:42.412 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/block/brd.ko 2024-03-04T13:14:42.412 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/block/drbd 2024-03-04T13:14:42.412 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/block/drbd/drbd.ko 2024-03-04T13:14:42.412 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/block/floppy.ko 2024-03-04T13:14:42.412 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/block/loop.ko 2024-03-04T13:14:42.412 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/block/mtip32xx 2024-03-04T13:14:42.413 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-03-04T13:14:42.413 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/block/nbd.ko 2024-03-04T13:14:42.413 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/block/null_blk 2024-03-04T13:14:42.413 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/block/null_blk/null_blk.ko 2024-03-04T13:14:42.413 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/block/pktcdvd.ko 2024-03-04T13:14:42.413 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/block/rbd.ko 2024-03-04T13:14:42.413 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/block/virtio_blk.ko 2024-03-04T13:14:42.413 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/block/xen-blkback 2024-03-04T13:14:42.413 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-03-04T13:14:42.413 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/block/xen-blkfront.ko 2024-03-04T13:14:42.414 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/block/zram 2024-03-04T13:14:42.414 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/block/zram/zram.ko 2024-03-04T13:14:42.414 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/char 2024-03-04T13:14:42.414 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/char/hangcheck-timer.ko 2024-03-04T13:14:42.414 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/char/hw_random 2024-03-04T13:14:42.414 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/char/hw_random/amd-rng.ko 2024-03-04T13:14:42.414 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/char/hw_random/intel-rng.ko 2024-03-04T13:14:42.414 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-03-04T13:14:42.414 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/char/hw_random/via-rng.ko 2024-03-04T13:14:42.414 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/char/hw_random/virtio-rng.ko 2024-03-04T13:14:42.414 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/char/ipmi 2024-03-04T13:14:42.415 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-03-04T13:14:42.415 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-03-04T13:14:42.415 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-03-04T13:14:42.415 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/char/ipmi/ipmi_si.ko 2024-03-04T13:14:42.415 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-03-04T13:14:42.415 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-03-04T13:14:42.415 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/char/lp.ko 2024-03-04T13:14:42.416 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/char/mwave 2024-03-04T13:14:42.416 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/char/mwave/mwave.ko 2024-03-04T13:14:42.416 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/char/ppdev.ko 2024-03-04T13:14:42.416 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/char/tlclk.ko 2024-03-04T13:14:42.416 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/char/tpm 2024-03-04T13:14:42.416 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/char/tpm/tpm.ko 2024-03-04T13:14:42.416 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/char/tpm/tpm_atmel.ko 2024-03-04T13:14:42.416 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/char/tpm/tpm_infineon.ko 2024-03-04T13:14:42.416 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/char/tpm/tpm_nsc.ko 2024-03-04T13:14:42.416 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/char/tpm/tpm_tis.ko 2024-03-04T13:14:42.416 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-03-04T13:14:42.417 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/char/uv_mmtimer.ko 2024-03-04T13:14:42.417 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/char/virtio_console.ko 2024-03-04T13:14:42.417 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/cpufreq 2024-03-04T13:14:42.417 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-03-04T13:14:42.417 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-03-04T13:14:42.417 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/cpufreq/p4-clockmod.ko 2024-03-04T13:14:42.417 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-03-04T13:14:42.417 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/cpufreq/powernow-k8.ko 2024-03-04T13:14:42.417 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/cpufreq/speedstep-lib.ko 2024-03-04T13:14:42.417 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/crypto 2024-03-04T13:14:42.417 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/crypto/ccp 2024-03-04T13:14:42.417 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-03-04T13:14:42.417 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/crypto/ccp/ccp.ko 2024-03-04T13:14:42.417 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/crypto/intel 2024-03-04T13:14:42.417 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/crypto/intel/qat 2024-03-04T13:14:42.418 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/crypto/intel/qat/qat_common 2024-03-04T13:14:42.418 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-03-04T13:14:42.418 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-03-04T13:14:42.418 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-03-04T13:14:42.418 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/crypto/padlock-aes.ko 2024-03-04T13:14:42.418 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/crypto/padlock-sha.ko 2024-03-04T13:14:42.418 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/crypto/virtio 2024-03-04T13:14:42.418 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-03-04T13:14:42.418 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/dca 2024-03-04T13:14:42.418 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/dca/dca.ko 2024-03-04T13:14:42.418 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/devfreq 2024-03-04T13:14:42.418 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-03-04T13:14:42.418 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/dma 2024-03-04T13:14:42.419 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/dma/dw 2024-03-04T13:14:42.419 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/dma/dw/dw_dmac.ko 2024-03-04T13:14:42.419 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/dma/ioat 2024-03-04T13:14:42.419 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/dma/ioat/ioatdma.ko 2024-03-04T13:14:42.419 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/edac 2024-03-04T13:14:42.419 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/edac/amd64_edac.ko 2024-03-04T13:14:42.419 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/edac/e752x_edac.ko 2024-03-04T13:14:42.419 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/edac/edac_mce_amd.ko 2024-03-04T13:14:42.419 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/edac/i3000_edac.ko 2024-03-04T13:14:42.419 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/edac/i3200_edac.ko 2024-03-04T13:14:42.419 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/edac/i5100_edac.ko 2024-03-04T13:14:42.419 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/edac/i5400_edac.ko 2024-03-04T13:14:42.419 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/edac/i7300_edac.ko 2024-03-04T13:14:42.419 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/edac/i7core_edac.ko 2024-03-04T13:14:42.420 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/edac/i82975x_edac.ko 2024-03-04T13:14:42.420 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/edac/ie31200_edac.ko 2024-03-04T13:14:42.420 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/edac/sb_edac.ko 2024-03-04T13:14:42.420 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/edac/x38_edac.ko 2024-03-04T13:14:42.420 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/firmware 2024-03-04T13:14:42.420 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/firmware/edd.ko 2024-03-04T13:14:42.420 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/firmware/iscsi_ibft.ko 2024-03-04T13:14:42.420 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/gpio 2024-03-04T13:14:42.420 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/gpio/gpio-ich.ko 2024-03-04T13:14:42.420 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/gpio/gpio-viperboard.ko 2024-03-04T13:14:42.420 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid 2024-03-04T13:14:42.421 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-appleir.ko 2024-03-04T13:14:42.421 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-aureal.ko 2024-03-04T13:14:42.421 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-axff.ko 2024-03-04T13:14:42.421 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-dr.ko 2024-03-04T13:14:42.421 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-elecom.ko 2024-03-04T13:14:42.421 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-elo.ko 2024-03-04T13:14:42.421 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-emsff.ko 2024-03-04T13:14:42.421 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-gaff.ko 2024-03-04T13:14:42.421 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-gt683r.ko 2024-03-04T13:14:42.421 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-gyration.ko 2024-03-04T13:14:42.421 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-holtek-kbd.ko 2024-03-04T13:14:42.421 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-holtek-mouse.ko 2024-03-04T13:14:42.421 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-holtekff.ko 2024-03-04T13:14:42.421 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-hyperv.ko 2024-03-04T13:14:42.422 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-icade.ko 2024-03-04T13:14:42.422 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-keytouch.ko 2024-03-04T13:14:42.422 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-kye.ko 2024-03-04T13:14:42.422 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-lcpower.ko 2024-03-04T13:14:42.422 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-led.ko 2024-03-04T13:14:42.422 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-lenovo.ko 2024-03-04T13:14:42.422 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-logitech-dj.ko 2024-03-04T13:14:42.422 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-03-04T13:14:42.422 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-multitouch.ko 2024-03-04T13:14:42.422 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-ortek.ko 2024-03-04T13:14:42.422 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-penmount.ko 2024-03-04T13:14:42.422 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-petalynx.ko 2024-03-04T13:14:42.422 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-picolcd.ko 2024-03-04T13:14:42.423 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-pl.ko 2024-03-04T13:14:42.423 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-plantronics.ko 2024-03-04T13:14:42.423 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-primax.ko 2024-03-04T13:14:42.423 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-rmi.ko 2024-03-04T13:14:42.423 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-roccat-arvo.ko 2024-03-04T13:14:42.423 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-roccat-common.ko 2024-03-04T13:14:42.423 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-roccat-isku.ko 2024-03-04T13:14:42.423 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-roccat-kone.ko 2024-03-04T13:14:42.423 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-03-04T13:14:42.423 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-roccat-konepure.ko 2024-03-04T13:14:42.423 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-03-04T13:14:42.423 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-roccat-lua.ko 2024-03-04T13:14:42.423 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-roccat-pyra.ko 2024-03-04T13:14:42.423 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-roccat-ryos.ko 2024-03-04T13:14:42.424 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-roccat-savu.ko 2024-03-04T13:14:42.424 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-roccat.ko 2024-03-04T13:14:42.424 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-saitek.ko 2024-03-04T13:14:42.424 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-samsung.ko 2024-03-04T13:14:42.424 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-sensor-hub.ko 2024-03-04T13:14:42.424 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-sjoy.ko 2024-03-04T13:14:42.424 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-sony.ko 2024-03-04T13:14:42.424 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-speedlink.ko 2024-03-04T13:14:42.424 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-steelseries.ko 2024-03-04T13:14:42.424 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-sunplus.ko 2024-03-04T13:14:42.424 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-thrustmaster.ko 2024-03-04T13:14:42.424 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-tivo.ko 2024-03-04T13:14:42.424 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-tmff.ko 2024-03-04T13:14:42.425 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-topseed.ko 2024-03-04T13:14:42.425 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-twinhan.ko 2024-03-04T13:14:42.425 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-uclogic.ko 2024-03-04T13:14:42.425 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-waltop.ko 2024-03-04T13:14:42.425 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-wiimote.ko 2024-03-04T13:14:42.425 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-xinmo.ko 2024-03-04T13:14:42.425 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-zpff.ko 2024-03-04T13:14:42.425 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-zydacron.ko 2024-03-04T13:14:42.425 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/uhid.ko 2024-03-04T13:14:42.425 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/wacom.ko 2024-03-04T13:14:42.425 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hv 2024-03-04T13:14:42.425 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hv/hv_balloon.ko 2024-03-04T13:14:42.425 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hv/hv_utils.ko 2024-03-04T13:14:42.425 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hv/hv_vmbus.ko 2024-03-04T13:14:42.426 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon 2024-03-04T13:14:42.426 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/abituguru.ko 2024-03-04T13:14:42.426 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/abituguru3.ko 2024-03-04T13:14:42.426 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/acpi_power_meter.ko 2024-03-04T13:14:42.426 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/ad7414.ko 2024-03-04T13:14:42.426 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/ad7418.ko 2024-03-04T13:14:42.426 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/adc128d818.ko 2024-03-04T13:14:42.426 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/adm1025.ko 2024-03-04T13:14:42.426 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/adm1026.ko 2024-03-04T13:14:42.426 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/adm1029.ko 2024-03-04T13:14:42.426 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/adm1031.ko 2024-03-04T13:14:42.426 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/adm9240.ko 2024-03-04T13:14:42.426 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/ads7828.ko 2024-03-04T13:14:42.427 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/adt7410.ko 2024-03-04T13:14:42.427 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/adt7411.ko 2024-03-04T13:14:42.427 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/adt7462.ko 2024-03-04T13:14:42.427 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/adt7470.ko 2024-03-04T13:14:42.427 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/adt7475.ko 2024-03-04T13:14:42.427 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/adt7x10.ko 2024-03-04T13:14:42.427 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/amc6821.ko 2024-03-04T13:14:42.427 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/applesmc.ko 2024-03-04T13:14:42.427 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/asb100.ko 2024-03-04T13:14:42.427 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/asc7621.ko 2024-03-04T13:14:42.427 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/asus_atk0110.ko 2024-03-04T13:14:42.427 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/atxp1.ko 2024-03-04T13:14:42.427 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/coretemp.ko 2024-03-04T13:14:42.427 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-03-04T13:14:42.428 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/dme1737.ko 2024-03-04T13:14:42.428 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/ds1621.ko 2024-03-04T13:14:42.428 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/ds620.ko 2024-03-04T13:14:42.428 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/emc1403.ko 2024-03-04T13:14:42.428 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/emc6w201.ko 2024-03-04T13:14:42.428 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/f71805f.ko 2024-03-04T13:14:42.428 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/f71882fg.ko 2024-03-04T13:14:42.428 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/f75375s.ko 2024-03-04T13:14:42.428 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/fam15h_power.ko 2024-03-04T13:14:42.428 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/fschmd.ko 2024-03-04T13:14:42.428 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/g760a.ko 2024-03-04T13:14:42.428 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/g762.ko 2024-03-04T13:14:42.428 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/gl518sm.ko 2024-03-04T13:14:42.429 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/gl520sm.ko 2024-03-04T13:14:42.429 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/hwmon-vid.ko 2024-03-04T13:14:42.429 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/i5500_temp.ko 2024-03-04T13:14:42.429 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/i5k_amb.ko 2024-03-04T13:14:42.429 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/ibmaem.ko 2024-03-04T13:14:42.429 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/ibmpex.ko 2024-03-04T13:14:42.429 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/ina209.ko 2024-03-04T13:14:42.429 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/ina2xx.ko 2024-03-04T13:14:42.429 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/it87.ko 2024-03-04T13:14:42.429 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/k10temp.ko 2024-03-04T13:14:42.429 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/k8temp.ko 2024-03-04T13:14:42.429 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/lineage-pem.ko 2024-03-04T13:14:42.429 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/lm63.ko 2024-03-04T13:14:42.429 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/lm73.ko 2024-03-04T13:14:42.430 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/lm75.ko 2024-03-04T13:14:42.430 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/lm77.ko 2024-03-04T13:14:42.430 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/lm78.ko 2024-03-04T13:14:42.430 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/lm80.ko 2024-03-04T13:14:42.430 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/lm83.ko 2024-03-04T13:14:42.430 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/lm85.ko 2024-03-04T13:14:42.430 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/lm87.ko 2024-03-04T13:14:42.430 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/lm90.ko 2024-03-04T13:14:42.430 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/lm92.ko 2024-03-04T13:14:42.430 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/lm93.ko 2024-03-04T13:14:42.430 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/lm95234.ko 2024-03-04T13:14:42.430 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/lm95241.ko 2024-03-04T13:14:42.430 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/lm95245.ko 2024-03-04T13:14:42.431 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/ltc2945.ko 2024-03-04T13:14:42.431 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/ltc4151.ko 2024-03-04T13:14:42.431 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/ltc4215.ko 2024-03-04T13:14:42.431 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/ltc4222.ko 2024-03-04T13:14:42.431 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/ltc4245.ko 2024-03-04T13:14:42.431 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/ltc4260.ko 2024-03-04T13:14:42.431 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/ltc4261.ko 2024-03-04T13:14:42.431 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/max16065.ko 2024-03-04T13:14:42.431 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/max1619.ko 2024-03-04T13:14:42.431 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/max1668.ko 2024-03-04T13:14:42.431 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/max197.ko 2024-03-04T13:14:42.431 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/max6639.ko 2024-03-04T13:14:42.431 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/max6650.ko 2024-03-04T13:14:42.432 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/max6697.ko 2024-03-04T13:14:42.432 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/mcp3021.ko 2024-03-04T13:14:42.432 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/nct6683.ko 2024-03-04T13:14:42.432 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/nct6775-core.ko 2024-03-04T13:14:42.432 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/nct6775.ko 2024-03-04T13:14:42.432 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/nct7802.ko 2024-03-04T13:14:42.432 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/pc87360.ko 2024-03-04T13:14:42.432 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/pc87427.ko 2024-03-04T13:14:42.432 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/pcf8591.ko 2024-03-04T13:14:42.432 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/pmbus 2024-03-04T13:14:42.432 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-03-04T13:14:42.432 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-03-04T13:14:42.432 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-03-04T13:14:42.433 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/pmbus/max16064.ko 2024-03-04T13:14:42.433 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/pmbus/max34440.ko 2024-03-04T13:14:42.433 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/pmbus/max8688.ko 2024-03-04T13:14:42.433 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-03-04T13:14:42.433 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-03-04T13:14:42.433 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-03-04T13:14:42.433 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-03-04T13:14:42.433 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-03-04T13:14:42.433 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-03-04T13:14:42.433 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/powr1220.ko 2024-03-04T13:14:42.433 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/sch5627.ko 2024-03-04T13:14:42.433 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/sch5636.ko 2024-03-04T13:14:42.433 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/sch56xx-common.ko 2024-03-04T13:14:42.433 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/sht15.ko 2024-03-04T13:14:42.434 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/sht21.ko 2024-03-04T13:14:42.434 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/shtc1.ko 2024-03-04T13:14:42.434 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/sis5595.ko 2024-03-04T13:14:42.434 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/smsc47b397.ko 2024-03-04T13:14:42.434 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/smsc47m1.ko 2024-03-04T13:14:42.434 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/smsc47m192.ko 2024-03-04T13:14:42.434 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/thmc50.ko 2024-03-04T13:14:42.434 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/tmp102.ko 2024-03-04T13:14:42.434 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/tmp103.ko 2024-03-04T13:14:42.434 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/tmp401.ko 2024-03-04T13:14:42.434 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/tmp421.ko 2024-03-04T13:14:42.434 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/via-cputemp.ko 2024-03-04T13:14:42.434 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/via686a.ko 2024-03-04T13:14:42.435 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/vt1211.ko 2024-03-04T13:14:42.435 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/vt8231.ko 2024-03-04T13:14:42.435 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/w83627ehf.ko 2024-03-04T13:14:42.435 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/w83627hf.ko 2024-03-04T13:14:42.435 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/w83781d.ko 2024-03-04T13:14:42.435 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/w83791d.ko 2024-03-04T13:14:42.435 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/w83792d.ko 2024-03-04T13:14:42.435 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/w83793.ko 2024-03-04T13:14:42.435 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/w83795.ko 2024-03-04T13:14:42.435 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/w83l785ts.ko 2024-03-04T13:14:42.435 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/w83l786ng.ko 2024-03-04T13:14:42.435 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c 2024-03-04T13:14:42.435 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/algos 2024-03-04T13:14:42.435 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-03-04T13:14:42.435 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-03-04T13:14:42.436 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/busses 2024-03-04T13:14:42.436 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-03-04T13:14:42.436 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-03-04T13:14:42.436 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-03-04T13:14:42.436 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-03-04T13:14:42.436 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-03-04T13:14:42.436 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-03-04T13:14:42.436 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-03-04T13:14:42.436 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-03-04T13:14:42.436 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/busses/i2c-i801.ko 2024-03-04T13:14:42.436 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/busses/i2c-isch.ko 2024-03-04T13:14:42.436 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-03-04T13:14:42.436 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-03-04T13:14:42.437 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-03-04T13:14:42.437 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/busses/i2c-parport.ko 2024-03-04T13:14:42.437 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-03-04T13:14:42.437 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-03-04T13:14:42.437 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-03-04T13:14:42.437 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-03-04T13:14:42.437 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-03-04T13:14:42.437 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-03-04T13:14:42.437 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/busses/i2c-via.ko 2024-03-04T13:14:42.437 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-03-04T13:14:42.437 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-03-04T13:14:42.437 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/i2c-dev.ko 2024-03-04T13:14:42.437 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/i2c-mux.ko 2024-03-04T13:14:42.437 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-03-04T13:14:42.438 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/i2c-smbus.ko 2024-03-04T13:14:42.438 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/i2c-stub.ko 2024-03-04T13:14:42.438 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/infiniband 2024-03-04T13:14:42.438 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/infiniband/core 2024-03-04T13:14:42.438 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/infiniband/core/ib_cm.ko 2024-03-04T13:14:42.438 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/infiniband/core/ib_core.ko 2024-03-04T13:14:42.438 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/infiniband/core/ib_umad.ko 2024-03-04T13:14:42.438 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-03-04T13:14:42.438 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/infiniband/core/iw_cm.ko 2024-03-04T13:14:42.438 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/infiniband/core/rdma_cm.ko 2024-03-04T13:14:42.438 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-03-04T13:14:42.438 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/infiniband/hw 2024-03-04T13:14:42.438 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/infiniband/hw/cxgb4 2024-03-04T13:14:42.438 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-03-04T13:14:42.439 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/infiniband/hw/mlx4 2024-03-04T13:14:42.439 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-03-04T13:14:42.439 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/infiniband/hw/mlx5 2024-03-04T13:14:42.439 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-03-04T13:14:42.439 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/infiniband/hw/mthca 2024-03-04T13:14:42.439 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-03-04T13:14:42.439 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/infiniband/ulp 2024-03-04T13:14:42.439 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/infiniband/ulp/ipoib 2024-03-04T13:14:42.439 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-03-04T13:14:42.439 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/infiniband/ulp/iser 2024-03-04T13:14:42.439 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-03-04T13:14:42.439 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/infiniband/ulp/isert 2024-03-04T13:14:42.439 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-03-04T13:14:42.439 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/infiniband/ulp/srp 2024-03-04T13:14:42.440 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-03-04T13:14:42.440 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/infiniband/ulp/srpt 2024-03-04T13:14:42.440 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-03-04T13:14:42.440 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input 2024-03-04T13:14:42.440 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input/gameport 2024-03-04T13:14:42.440 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-03-04T13:14:42.440 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input/gameport/fm801-gp.ko 2024-03-04T13:14:42.440 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input/gameport/gameport.ko 2024-03-04T13:14:42.440 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input/joydev.ko 2024-03-04T13:14:42.440 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input/keyboard 2024-03-04T13:14:42.440 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input/keyboard/gpio_keys.ko 2024-03-04T13:14:42.440 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input/mouse 2024-03-04T13:14:42.440 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input/mouse/appletouch.ko 2024-03-04T13:14:42.441 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input/mouse/bcm5974.ko 2024-03-04T13:14:42.441 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input/mouse/cyapatp.ko 2024-03-04T13:14:42.441 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input/mouse/elan_i2c.ko 2024-03-04T13:14:42.441 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input/mouse/sermouse.ko 2024-03-04T13:14:42.441 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-03-04T13:14:42.441 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input/mouse/synaptics_usb.ko 2024-03-04T13:14:42.441 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input/mouse/vsxxxaa.ko 2024-03-04T13:14:42.441 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input/rmi4 2024-03-04T13:14:42.441 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input/rmi4/rmi_core.ko 2024-03-04T13:14:42.441 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input/serio 2024-03-04T13:14:42.441 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input/serio/altera_ps2.ko 2024-03-04T13:14:42.441 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input/serio/arc_ps2.ko 2024-03-04T13:14:42.441 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-03-04T13:14:42.441 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input/serio/serio_raw.ko 2024-03-04T13:14:42.442 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input/sparse-keymap.ko 2024-03-04T13:14:42.442 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/leds 2024-03-04T13:14:42.442 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/leds/leds-blinkm.ko 2024-03-04T13:14:42.442 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/leds/leds-lm3530.ko 2024-03-04T13:14:42.442 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/leds/leds-lp3944.ko 2024-03-04T13:14:42.442 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/leds/leds-ss4200.ko 2024-03-04T13:14:42.442 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/leds/trigger 2024-03-04T13:14:42.442 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-03-04T13:14:42.442 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-03-04T13:14:42.442 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-03-04T13:14:42.442 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-03-04T13:14:42.442 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-03-04T13:14:42.442 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-03-04T13:14:42.443 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-03-04T13:14:42.443 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-03-04T13:14:42.443 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-03-04T13:14:42.443 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/md 2024-03-04T13:14:42.443 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/md/bcache 2024-03-04T13:14:42.443 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/md/bcache/bcache.ko 2024-03-04T13:14:42.443 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/md/dm-bio-prison.ko 2024-03-04T13:14:42.443 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/md/dm-cache-smq.ko 2024-03-04T13:14:42.443 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/md/dm-cache.ko 2024-03-04T13:14:42.443 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/md/dm-crypt.ko 2024-03-04T13:14:42.443 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/md/dm-delay.ko 2024-03-04T13:14:42.443 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/md/dm-flakey.ko 2024-03-04T13:14:42.443 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/md/dm-log-userspace.ko 2024-03-04T13:14:42.443 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/md/dm-multipath.ko 2024-03-04T13:14:42.444 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/md/dm-queue-length.ko 2024-03-04T13:14:42.444 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/md/dm-raid.ko 2024-03-04T13:14:42.444 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/md/dm-round-robin.ko 2024-03-04T13:14:42.444 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/md/dm-service-time.ko 2024-03-04T13:14:42.444 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/md/dm-switch.ko 2024-03-04T13:14:42.444 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/md/dm-thin-pool.ko 2024-03-04T13:14:42.444 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/md/dm-verity.ko 2024-03-04T13:14:42.444 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/md/persistent-data 2024-03-04T13:14:42.444 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-03-04T13:14:42.444 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/md/raid0.ko 2024-03-04T13:14:42.444 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/md/raid1.ko 2024-03-04T13:14:42.445 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/md/raid10.ko 2024-03-04T13:14:42.445 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/md/raid456.ko 2024-03-04T13:14:42.445 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/message 2024-03-04T13:14:42.445 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/message/fusion 2024-03-04T13:14:42.445 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/message/fusion/mptbase.ko 2024-03-04T13:14:42.445 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/message/fusion/mptctl.ko 2024-03-04T13:14:42.445 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/message/fusion/mptfc.ko 2024-03-04T13:14:42.445 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/message/fusion/mptlan.ko 2024-03-04T13:14:42.445 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/message/fusion/mptsas.ko 2024-03-04T13:14:42.446 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/message/fusion/mptscsih.ko 2024-03-04T13:14:42.446 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/message/fusion/mptspi.ko 2024-03-04T13:14:42.446 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/mfd 2024-03-04T13:14:42.446 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/mfd/lpc_ich.ko 2024-03-04T13:14:42.446 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/mfd/lpc_sch.ko 2024-03-04T13:14:42.446 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/mfd/mfd-core.ko 2024-03-04T13:14:42.446 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/mfd/sm501.ko 2024-03-04T13:14:42.446 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/mfd/viperboard.ko 2024-03-04T13:14:42.446 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/mfd/vx855.ko 2024-03-04T13:14:42.446 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/mfd/wl1273-core.ko 2024-03-04T13:14:42.446 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc 2024-03-04T13:14:42.447 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/altera-stapl 2024-03-04T13:14:42.447 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-03-04T13:14:42.447 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/apds9802als.ko 2024-03-04T13:14:42.447 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/apds990x.ko 2024-03-04T13:14:42.447 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/bh1770glc.ko 2024-03-04T13:14:42.447 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/cb710 2024-03-04T13:14:42.447 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/cb710/cb710.ko 2024-03-04T13:14:42.447 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/eeprom 2024-03-04T13:14:42.447 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/eeprom/at24.ko 2024-03-04T13:14:42.447 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-03-04T13:14:42.447 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/eeprom/max6875.ko 2024-03-04T13:14:42.447 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/enclosure.ko 2024-03-04T13:14:42.447 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/hpilo.ko 2024-03-04T13:14:42.447 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/isl29003.ko 2024-03-04T13:14:42.448 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/isl29020.ko 2024-03-04T13:14:42.448 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/lis3lv02d 2024-03-04T13:14:42.448 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-03-04T13:14:42.448 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-03-04T13:14:42.448 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/mei 2024-03-04T13:14:42.448 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/mei/mei-me.ko 2024-03-04T13:14:42.448 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/mei/mei-txe.ko 2024-03-04T13:14:42.448 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/mei/mei.ko 2024-03-04T13:14:42.448 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/sgi-gru 2024-03-04T13:14:42.448 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/sgi-gru/gru.ko 2024-03-04T13:14:42.449 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/sgi-xp 2024-03-04T13:14:42.449 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/sgi-xp/xp.ko 2024-03-04T13:14:42.449 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/sgi-xp/xpc.ko 2024-03-04T13:14:42.449 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-03-04T13:14:42.449 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/tifm_7xx1.ko 2024-03-04T13:14:42.450 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/tifm_core.ko 2024-03-04T13:14:42.450 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/tsl2550.ko 2024-03-04T13:14:42.450 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/vmw_balloon.ko 2024-03-04T13:14:42.450 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/vmw_vmci 2024-03-04T13:14:42.450 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-03-04T13:14:42.450 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net 2024-03-04T13:14:42.451 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/bonding 2024-03-04T13:14:42.451 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/bonding/bonding.ko 2024-03-04T13:14:42.451 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/dsa 2024-03-04T13:14:42.451 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/dsa/b53 2024-03-04T13:14:42.451 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/dsa/b53/b53_common.ko 2024-03-04T13:14:42.452 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/dsa/bcm-sf2.ko 2024-03-04T13:14:42.452 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/dsa/mv88e6060.ko 2024-03-04T13:14:42.452 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/dsa/mv88e6xxx 2024-03-04T13:14:42.452 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-03-04T13:14:42.452 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/dummy.ko 2024-03-04T13:14:42.452 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/eql.ko 2024-03-04T13:14:42.452 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet 2024-03-04T13:14:42.453 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/3com 2024-03-04T13:14:42.453 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-03-04T13:14:42.453 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-03-04T13:14:42.453 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-03-04T13:14:42.453 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-03-04T13:14:42.453 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/8390 2024-03-04T13:14:42.453 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/8390/8390.ko 2024-03-04T13:14:42.453 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-03-04T13:14:42.453 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-03-04T13:14:42.453 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-03-04T13:14:42.453 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/adaptec 2024-03-04T13:14:42.453 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-03-04T13:14:42.453 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/agere 2024-03-04T13:14:42.454 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/agere/et131x.ko 2024-03-04T13:14:42.454 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/alteon 2024-03-04T13:14:42.454 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-03-04T13:14:42.454 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/altera 2024-03-04T13:14:42.454 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-03-04T13:14:42.454 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/amd 2024-03-04T13:14:42.454 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-03-04T13:14:42.454 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-03-04T13:14:42.454 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-03-04T13:14:42.454 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/atheros 2024-03-04T13:14:42.454 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/atheros/alx 2024-03-04T13:14:42.454 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-03-04T13:14:42.454 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/atheros/atl1c 2024-03-04T13:14:42.454 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-03-04T13:14:42.455 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/atheros/atl1e 2024-03-04T13:14:42.455 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-03-04T13:14:42.455 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/atheros/atlx 2024-03-04T13:14:42.455 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-03-04T13:14:42.455 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-03-04T13:14:42.455 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/broadcom 2024-03-04T13:14:42.455 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-03-04T13:14:42.455 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-03-04T13:14:42.455 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-03-04T13:14:42.455 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-03-04T13:14:42.455 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-03-04T13:14:42.455 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/broadcom/genet 2024-03-04T13:14:42.455 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-03-04T13:14:42.455 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-03-04T13:14:42.456 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/brocade 2024-03-04T13:14:42.456 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/brocade/bna 2024-03-04T13:14:42.456 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-03-04T13:14:42.456 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/chelsio 2024-03-04T13:14:42.456 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/chelsio/cxgb 2024-03-04T13:14:42.456 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-03-04T13:14:42.456 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-03-04T13:14:42.456 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-03-04T13:14:42.456 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-03-04T13:14:42.456 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-03-04T13:14:42.456 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-03-04T13:14:42.456 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-03-04T13:14:42.457 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-03-04T13:14:42.457 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-03-04T13:14:42.457 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/cisco 2024-03-04T13:14:42.457 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/cisco/enic 2024-03-04T13:14:42.457 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-03-04T13:14:42.457 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/dec 2024-03-04T13:14:42.457 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/dec/tulip 2024-03-04T13:14:42.457 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-03-04T13:14:42.457 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-03-04T13:14:42.457 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-03-04T13:14:42.457 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-03-04T13:14:42.457 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-03-04T13:14:42.457 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-03-04T13:14:42.457 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/dlink 2024-03-04T13:14:42.458 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-03-04T13:14:42.458 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-03-04T13:14:42.458 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/dnet.ko 2024-03-04T13:14:42.458 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/emulex 2024-03-04T13:14:42.458 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/emulex/benet 2024-03-04T13:14:42.458 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-03-04T13:14:42.458 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/ethoc.ko 2024-03-04T13:14:42.458 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/fealnx.ko 2024-03-04T13:14:42.458 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/intel 2024-03-04T13:14:42.458 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/intel/e100.ko 2024-03-04T13:14:42.458 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/intel/e1000 2024-03-04T13:14:42.458 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-03-04T13:14:42.458 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/intel/e1000e 2024-03-04T13:14:42.458 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-03-04T13:14:42.459 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/intel/fm10k 2024-03-04T13:14:42.459 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-03-04T13:14:42.459 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/intel/i40e 2024-03-04T13:14:42.459 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-03-04T13:14:42.459 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/intel/iavf 2024-03-04T13:14:42.459 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-03-04T13:14:42.459 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/intel/igb 2024-03-04T13:14:42.459 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-03-04T13:14:42.459 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/intel/igbvf 2024-03-04T13:14:42.459 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-03-04T13:14:42.459 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/intel/ixgbe 2024-03-04T13:14:42.459 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-03-04T13:14:42.459 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/intel/ixgbevf 2024-03-04T13:14:42.459 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-03-04T13:14:42.460 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/jme.ko 2024-03-04T13:14:42.460 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/marvell 2024-03-04T13:14:42.460 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-03-04T13:14:42.460 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/marvell/skge.ko 2024-03-04T13:14:42.460 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-03-04T13:14:42.460 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/mellanox 2024-03-04T13:14:42.460 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/mellanox/mlx4 2024-03-04T13:14:42.460 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-03-04T13:14:42.460 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-03-04T13:14:42.460 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/mellanox/mlx5 2024-03-04T13:14:42.460 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-03-04T13:14:42.460 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-03-04T13:14:42.460 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/micrel 2024-03-04T13:14:42.461 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-03-04T13:14:42.461 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/myricom 2024-03-04T13:14:42.461 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/myricom/myri10ge 2024-03-04T13:14:42.461 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-03-04T13:14:42.461 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/natsemi 2024-03-04T13:14:42.461 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-03-04T13:14:42.461 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-03-04T13:14:42.461 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/neterion 2024-03-04T13:14:42.461 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-03-04T13:14:42.461 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/nvidia 2024-03-04T13:14:42.461 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-03-04T13:14:42.461 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/packetengines 2024-03-04T13:14:42.461 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-03-04T13:14:42.462 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-03-04T13:14:42.462 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/qlogic 2024-03-04T13:14:42.462 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/qlogic/netxen 2024-03-04T13:14:42.462 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-03-04T13:14:42.462 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-03-04T13:14:42.462 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-03-04T13:14:42.462 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-03-04T13:14:42.462 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/rdc 2024-03-04T13:14:42.462 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-03-04T13:14:42.462 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/realtek 2024-03-04T13:14:42.462 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-03-04T13:14:42.462 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-03-04T13:14:42.462 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/realtek/atp.ko 2024-03-04T13:14:42.462 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-03-04T13:14:42.463 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/rocker 2024-03-04T13:14:42.463 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-03-04T13:14:42.463 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/sfc 2024-03-04T13:14:42.463 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-03-04T13:14:42.463 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/silan 2024-03-04T13:14:42.463 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-03-04T13:14:42.463 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/sis 2024-03-04T13:14:42.463 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/sis/sis190.ko 2024-03-04T13:14:42.463 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/sis/sis900.ko 2024-03-04T13:14:42.463 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/smsc 2024-03-04T13:14:42.463 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-03-04T13:14:42.463 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-03-04T13:14:42.463 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-03-04T13:14:42.464 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-03-04T13:14:42.464 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/stmicro 2024-03-04T13:14:42.464 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/stmicro/stmmac 2024-03-04T13:14:42.464 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-03-04T13:14:42.464 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-03-04T13:14:42.464 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/sun 2024-03-04T13:14:42.464 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/sun/cassini.ko 2024-03-04T13:14:42.464 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/sun/niu.ko 2024-03-04T13:14:42.464 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/sun/sungem.ko 2024-03-04T13:14:42.464 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-03-04T13:14:42.464 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/tehuti 2024-03-04T13:14:42.464 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-03-04T13:14:42.464 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/ti 2024-03-04T13:14:42.464 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/ti/tlan.ko 2024-03-04T13:14:42.465 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/via 2024-03-04T13:14:42.465 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-03-04T13:14:42.465 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-03-04T13:14:42.465 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/wiznet 2024-03-04T13:14:42.465 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-03-04T13:14:42.465 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-03-04T13:14:42.465 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/xircom 2024-03-04T13:14:42.465 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-03-04T13:14:42.465 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/geneve.ko 2024-03-04T13:14:42.465 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/hyperv 2024-03-04T13:14:42.465 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-03-04T13:14:42.465 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ieee802154 2024-03-04T13:14:42.465 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ieee802154/fakelb.ko 2024-03-04T13:14:42.465 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ifb.ko 2024-03-04T13:14:42.466 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ipvlan 2024-03-04T13:14:42.466 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ipvlan/ipvlan.ko 2024-03-04T13:14:42.466 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/macvlan.ko 2024-03-04T13:14:42.466 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/macvtap.ko 2024-03-04T13:14:42.466 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/mdio 2024-03-04T13:14:42.466 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/mdio.ko 2024-03-04T13:14:42.466 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-03-04T13:14:42.466 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-03-04T13:14:42.466 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/mdio/mdio-regmap.ko 2024-03-04T13:14:42.466 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/mii.ko 2024-03-04T13:14:42.466 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/net_failover.ko 2024-03-04T13:14:42.466 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/netconsole.ko 2024-03-04T13:14:42.466 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/nlmon.ko 2024-03-04T13:14:42.466 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/pcs 2024-03-04T13:14:42.467 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/pcs/pcs-lynx.ko 2024-03-04T13:14:42.467 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-03-04T13:14:42.467 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/phy 2024-03-04T13:14:42.467 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/phy/amd.ko 2024-03-04T13:14:42.467 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-03-04T13:14:42.467 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-03-04T13:14:42.467 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/phy/bcm7xxx.ko 2024-03-04T13:14:42.467 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/phy/bcm87xx.ko 2024-03-04T13:14:42.467 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/phy/broadcom.ko 2024-03-04T13:14:42.467 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/phy/cicada.ko 2024-03-04T13:14:42.467 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/phy/davicom.ko 2024-03-04T13:14:42.467 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/phy/dp83640.ko 2024-03-04T13:14:42.467 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/phy/et1011c.ko 2024-03-04T13:14:42.468 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/phy/icplus.ko 2024-03-04T13:14:42.468 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/phy/lxt.ko 2024-03-04T13:14:42.468 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/phy/marvell.ko 2024-03-04T13:14:42.468 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/phy/micrel.ko 2024-03-04T13:14:42.468 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/phy/national.ko 2024-03-04T13:14:42.468 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/phy/phylink.ko 2024-03-04T13:14:42.468 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/phy/qsemi.ko 2024-03-04T13:14:42.468 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/phy/realtek.ko 2024-03-04T13:14:42.468 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/phy/smsc.ko 2024-03-04T13:14:42.468 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/phy/ste10Xp.ko 2024-03-04T13:14:42.468 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/phy/vitesse.ko 2024-03-04T13:14:42.468 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ppp 2024-03-04T13:14:42.468 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ppp/bsd_comp.ko 2024-03-04T13:14:42.469 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ppp/ppp_async.ko 2024-03-04T13:14:42.469 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ppp/ppp_deflate.ko 2024-03-04T13:14:42.469 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ppp/ppp_generic.ko 2024-03-04T13:14:42.469 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ppp/ppp_mppe.ko 2024-03-04T13:14:42.469 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ppp/ppp_synctty.ko 2024-03-04T13:14:42.469 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ppp/pppoe.ko 2024-03-04T13:14:42.469 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ppp/pppox.ko 2024-03-04T13:14:42.469 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ppp/pptp.ko 2024-03-04T13:14:42.469 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/slip 2024-03-04T13:14:42.469 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/slip/slhc.ko 2024-03-04T13:14:42.469 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/slip/slip.ko 2024-03-04T13:14:42.469 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/sungem_phy.ko 2024-03-04T13:14:42.469 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/tap.ko 2024-03-04T13:14:42.469 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/team 2024-03-04T13:14:42.470 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/team/team.ko 2024-03-04T13:14:42.470 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/team/team_mode_activebackup.ko 2024-03-04T13:14:42.470 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/team/team_mode_broadcast.ko 2024-03-04T13:14:42.470 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-03-04T13:14:42.470 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/team/team_mode_random.ko 2024-03-04T13:14:42.470 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-03-04T13:14:42.470 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/tun.ko 2024-03-04T13:14:42.470 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/veth.ko 2024-03-04T13:14:42.470 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/virtio_net.ko 2024-03-04T13:14:42.470 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/vmxnet3 2024-03-04T13:14:42.470 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-03-04T13:14:42.471 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/vxlan 2024-03-04T13:14:42.471 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/vxlan/vxlan.ko 2024-03-04T13:14:42.471 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/xen-netback 2024-03-04T13:14:42.471 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/xen-netback/xen-netback.ko 2024-03-04T13:14:42.471 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/xen-netfront.ko 2024-03-04T13:14:42.471 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ntb 2024-03-04T13:14:42.471 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ntb/ntb.ko 2024-03-04T13:14:42.471 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/nvme 2024-03-04T13:14:42.471 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/nvme/host 2024-03-04T13:14:42.471 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/nvme/host/nvme-core.ko 2024-03-04T13:14:42.471 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/nvme/host/nvme.ko 2024-03-04T13:14:42.471 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/parport 2024-03-04T13:14:42.471 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/parport/parport.ko 2024-03-04T13:14:42.471 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/parport/parport_cs.ko 2024-03-04T13:14:42.472 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/parport/parport_pc.ko 2024-03-04T13:14:42.472 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/parport/parport_serial.ko 2024-03-04T13:14:42.472 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/pci 2024-03-04T13:14:42.472 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/pci/hotplug 2024-03-04T13:14:42.472 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-03-04T13:14:42.472 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/pci/pcie 2024-03-04T13:14:42.472 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/pci/pcie/aer_inject.ko 2024-03-04T13:14:42.472 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/pci/xen-pcifront.ko 2024-03-04T13:14:42.472 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/pcmcia 2024-03-04T13:14:42.472 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/pcmcia/i82092.ko 2024-03-04T13:14:42.472 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/pcmcia/pd6729.ko 2024-03-04T13:14:42.472 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/pcmcia/yenta_socket.ko 2024-03-04T13:14:42.472 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/pinctrl 2024-03-04T13:14:42.473 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/pinctrl/intel 2024-03-04T13:14:42.473 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-03-04T13:14:42.473 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/platform 2024-03-04T13:14:42.473 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/platform/x86 2024-03-04T13:14:42.473 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/platform/x86/acer-wmi.ko 2024-03-04T13:14:42.473 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/platform/x86/acerhdf.ko 2024-03-04T13:14:42.473 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/platform/x86/apple-gmux.ko 2024-03-04T13:14:42.473 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/platform/x86/asus-laptop.ko 2024-03-04T13:14:42.473 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-03-04T13:14:42.473 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/platform/x86/asus-wmi.ko 2024-03-04T13:14:42.473 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/platform/x86/classmate-laptop.ko 2024-03-04T13:14:42.473 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-03-04T13:14:42.474 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-03-04T13:14:42.474 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-03-04T13:14:42.474 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-03-04T13:14:42.474 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/platform/x86/hdaps.ko 2024-03-04T13:14:42.474 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/platform/x86/intel 2024-03-04T13:14:42.474 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-03-04T13:14:42.474 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/platform/x86/intel_ips.ko 2024-03-04T13:14:42.474 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/platform/x86/msi-wmi.ko 2024-03-04T13:14:42.474 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/platform/x86/mxm-wmi.ko 2024-03-04T13:14:42.475 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-03-04T13:14:42.475 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/platform/x86/samsung-laptop.ko 2024-03-04T13:14:42.475 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/platform/x86/samsung-q10.ko 2024-03-04T13:14:42.475 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/platform/x86/topstar-laptop.ko 2024-03-04T13:14:42.475 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-03-04T13:14:42.475 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/platform/x86/toshiba_haps.ko 2024-03-04T13:14:42.475 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/platform/x86/wmi-bmof.ko 2024-03-04T13:14:42.476 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/platform/x86/wmi.ko 2024-03-04T13:14:42.476 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/powercap 2024-03-04T13:14:42.476 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/powercap/intel_rapl_common.ko 2024-03-04T13:14:42.476 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/powercap/intel_rapl_msr.ko 2024-03-04T13:14:42.476 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/pps 2024-03-04T13:14:42.476 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/pps/clients 2024-03-04T13:14:42.476 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/pps/clients/pps-gpio.ko 2024-03-04T13:14:42.476 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/pps/clients/pps-ldisc.ko 2024-03-04T13:14:42.476 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/pps/clients/pps_parport.ko 2024-03-04T13:14:42.476 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/pps/pps_core.ko 2024-03-04T13:14:42.477 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ptp 2024-03-04T13:14:42.477 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ptp/ptp.ko 2024-03-04T13:14:42.477 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ptp/ptp_kvm.ko 2024-03-04T13:14:42.477 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc 2024-03-04T13:14:42.477 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-bq32k.ko 2024-03-04T13:14:42.477 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-ds1286.ko 2024-03-04T13:14:42.477 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-ds1307.ko 2024-03-04T13:14:42.477 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-ds1374.ko 2024-03-04T13:14:42.477 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-ds1511.ko 2024-03-04T13:14:42.477 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-ds1553.ko 2024-03-04T13:14:42.477 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-ds1672.ko 2024-03-04T13:14:42.477 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-ds1742.ko 2024-03-04T13:14:42.477 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-ds2404.ko 2024-03-04T13:14:42.477 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-ds3232.ko 2024-03-04T13:14:42.478 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-em3027.ko 2024-03-04T13:14:42.478 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-fm3130.ko 2024-03-04T13:14:42.478 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-isl12022.ko 2024-03-04T13:14:42.478 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-isl1208.ko 2024-03-04T13:14:42.478 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-m41t80.ko 2024-03-04T13:14:42.478 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-m48t35.ko 2024-03-04T13:14:42.478 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-m48t59.ko 2024-03-04T13:14:42.478 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-max6900.ko 2024-03-04T13:14:42.478 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-msm6242.ko 2024-03-04T13:14:42.478 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-pcf2127.ko 2024-03-04T13:14:42.478 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-pcf85063.ko 2024-03-04T13:14:42.478 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-pcf8523.ko 2024-03-04T13:14:42.478 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-pcf8563.ko 2024-03-04T13:14:42.478 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-pcf8583.ko 2024-03-04T13:14:42.479 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-rp5c01.ko 2024-03-04T13:14:42.479 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-rs5c372.ko 2024-03-04T13:14:42.479 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-03-04T13:14:42.479 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-rx8025.ko 2024-03-04T13:14:42.479 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-rx8581.ko 2024-03-04T13:14:42.479 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-03-04T13:14:42.479 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-x1205.ko 2024-03-04T13:14:42.479 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi 2024-03-04T13:14:42.479 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/3w-9xxx.ko 2024-03-04T13:14:42.479 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/3w-sas.ko 2024-03-04T13:14:42.479 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/3w-xxxx.ko 2024-03-04T13:14:42.479 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/BusLogic.ko 2024-03-04T13:14:42.479 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/a100u2w.ko 2024-03-04T13:14:42.480 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/aacraid 2024-03-04T13:14:42.480 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/aacraid/aacraid.ko 2024-03-04T13:14:42.480 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/advansys.ko 2024-03-04T13:14:42.480 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/aic7xxx 2024-03-04T13:14:42.480 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-03-04T13:14:42.480 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-03-04T13:14:42.480 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/am53c974.ko 2024-03-04T13:14:42.480 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/arcmsr 2024-03-04T13:14:42.480 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-03-04T13:14:42.480 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/atp870u.ko 2024-03-04T13:14:42.480 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/be2iscsi 2024-03-04T13:14:42.480 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-03-04T13:14:42.480 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/bfa 2024-03-04T13:14:42.480 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/bfa/bfa.ko 2024-03-04T13:14:42.481 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/bnx2fc 2024-03-04T13:14:42.481 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-03-04T13:14:42.481 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/bnx2i 2024-03-04T13:14:42.481 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-03-04T13:14:42.481 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/ch.ko 2024-03-04T13:14:42.481 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/csiostor 2024-03-04T13:14:42.481 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/csiostor/csiostor.ko 2024-03-04T13:14:42.481 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/cxgbi 2024-03-04T13:14:42.481 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/cxgbi/cxgb3i 2024-03-04T13:14:42.481 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-03-04T13:14:42.481 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/cxgbi/cxgb4i 2024-03-04T13:14:42.481 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-03-04T13:14:42.481 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-03-04T13:14:42.482 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/dc395x.ko 2024-03-04T13:14:42.482 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/device_handler 2024-03-04T13:14:42.482 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-03-04T13:14:42.482 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-03-04T13:14:42.482 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-03-04T13:14:42.482 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-03-04T13:14:42.482 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/dmx3191d.ko 2024-03-04T13:14:42.482 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/esas2r 2024-03-04T13:14:42.482 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/esas2r/esas2r.ko 2024-03-04T13:14:42.482 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/esp_scsi.ko 2024-03-04T13:14:42.482 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/fcoe 2024-03-04T13:14:42.482 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/fcoe/fcoe.ko 2024-03-04T13:14:42.482 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-03-04T13:14:42.483 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/fnic 2024-03-04T13:14:42.483 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/fnic/fnic.ko 2024-03-04T13:14:42.483 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/hpsa.ko 2024-03-04T13:14:42.483 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/hptiop.ko 2024-03-04T13:14:42.483 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/hv_storvsc.ko 2024-03-04T13:14:42.483 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/initio.ko 2024-03-04T13:14:42.483 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/ipr.ko 2024-03-04T13:14:42.483 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/ips.ko 2024-03-04T13:14:42.483 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/isci 2024-03-04T13:14:42.483 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/isci/isci.ko 2024-03-04T13:14:42.483 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-03-04T13:14:42.483 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/iscsi_tcp.ko 2024-03-04T13:14:42.483 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/libfc 2024-03-04T13:14:42.483 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/libfc/libfc.ko 2024-03-04T13:14:42.484 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/libiscsi.ko 2024-03-04T13:14:42.484 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/libiscsi_tcp.ko 2024-03-04T13:14:42.484 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/libsas 2024-03-04T13:14:42.484 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/libsas/libsas.ko 2024-03-04T13:14:42.484 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/lpfc 2024-03-04T13:14:42.484 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/lpfc/lpfc.ko 2024-03-04T13:14:42.484 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/megaraid 2024-03-04T13:14:42.484 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/megaraid.ko 2024-03-04T13:14:42.484 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-03-04T13:14:42.484 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-03-04T13:14:42.484 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-03-04T13:14:42.484 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/mpt3sas 2024-03-04T13:14:42.484 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-03-04T13:14:42.484 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/mvsas 2024-03-04T13:14:42.485 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/mvsas/mvsas.ko 2024-03-04T13:14:42.485 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/mvumi.ko 2024-03-04T13:14:42.485 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/pm8001 2024-03-04T13:14:42.485 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-03-04T13:14:42.485 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/pmcraid.ko 2024-03-04T13:14:42.485 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/qla1280.ko 2024-03-04T13:14:42.485 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/qla2xxx 2024-03-04T13:14:42.485 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-03-04T13:14:42.485 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-03-04T13:14:42.485 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/qla4xxx 2024-03-04T13:14:42.485 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-03-04T13:14:42.485 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/raid_class.ko 2024-03-04T13:14:42.485 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/scsi_debug.ko 2024-03-04T13:14:42.485 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/scsi_transport_fc.ko 2024-03-04T13:14:42.486 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-03-04T13:14:42.486 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/scsi_transport_sas.ko 2024-03-04T13:14:42.486 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/scsi_transport_spi.ko 2024-03-04T13:14:42.486 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/scsi_transport_srp.ko 2024-03-04T13:14:42.486 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/ses.ko 2024-03-04T13:14:42.486 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/st.ko 2024-03-04T13:14:42.486 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/stex.ko 2024-03-04T13:14:42.486 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/sym53c8xx_2 2024-03-04T13:14:42.486 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-03-04T13:14:42.486 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/virtio_scsi.ko 2024-03-04T13:14:42.486 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/vmw_pvscsi.ko 2024-03-04T13:14:42.486 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/wd719x.ko 2024-03-04T13:14:42.486 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ssb 2024-03-04T13:14:42.487 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ssb/ssb.ko 2024-03-04T13:14:42.487 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/target 2024-03-04T13:14:42.487 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/target/iscsi 2024-03-04T13:14:42.487 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-03-04T13:14:42.487 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/target/loopback 2024-03-04T13:14:42.487 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/target/loopback/tcm_loop.ko 2024-03-04T13:14:42.487 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/target/target_core_file.ko 2024-03-04T13:14:42.488 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/target/target_core_iblock.ko 2024-03-04T13:14:42.488 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/target/target_core_mod.ko 2024-03-04T13:14:42.488 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/target/target_core_pscsi.ko 2024-03-04T13:14:42.488 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/target/target_core_user.ko 2024-03-04T13:14:42.488 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/target/tcm_fc 2024-03-04T13:14:42.488 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-03-04T13:14:42.488 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/thermal 2024-03-04T13:14:42.488 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/thermal/intel 2024-03-04T13:14:42.488 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/thermal/intel/int340x_thermal 2024-03-04T13:14:42.488 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-03-04T13:14:42.488 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-03-04T13:14:42.488 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-03-04T13:14:42.488 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-03-04T13:14:42.488 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-03-04T13:14:42.489 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-03-04T13:14:42.489 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-03-04T13:14:42.489 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-03-04T13:14:42.489 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-03-04T13:14:42.489 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-03-04T13:14:42.489 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-03-04T13:14:42.489 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-03-04T13:14:42.489 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-03-04T13:14:42.489 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-03-04T13:14:42.489 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-03-04T13:14:42.489 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-03-04T13:14:42.489 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-03-04T13:14:42.489 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-03-04T13:14:42.489 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/tty 2024-03-04T13:14:42.490 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/tty/ipwireless 2024-03-04T13:14:42.490 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-03-04T13:14:42.490 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/tty/n_gsm.ko 2024-03-04T13:14:42.490 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/tty/n_hdlc.ko 2024-03-04T13:14:42.490 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/tty/nozomi.ko 2024-03-04T13:14:42.490 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/tty/serial 2024-03-04T13:14:42.490 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/tty/serial/8250 2024-03-04T13:14:42.490 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-03-04T13:14:42.490 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/tty/serial/arc_uart.ko 2024-03-04T13:14:42.490 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/tty/serial/jsm 2024-03-04T13:14:42.490 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/tty/serial/jsm/jsm.ko 2024-03-04T13:14:42.490 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ufs 2024-03-04T13:14:42.490 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ufs/core 2024-03-04T13:14:42.490 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ufs/core/ufshcd-core.ko 2024-03-04T13:14:42.491 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ufs/host 2024-03-04T13:14:42.491 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-03-04T13:14:42.491 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/uio 2024-03-04T13:14:42.491 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/uio/uio.ko 2024-03-04T13:14:42.491 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/uio/uio_aec.ko 2024-03-04T13:14:42.491 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/uio/uio_cif.ko 2024-03-04T13:14:42.491 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/uio/uio_pci_generic.ko 2024-03-04T13:14:42.492 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/uio/uio_sercos3.ko 2024-03-04T13:14:42.492 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb 2024-03-04T13:14:42.492 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/class 2024-03-04T13:14:42.492 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/class/cdc-acm.ko 2024-03-04T13:14:42.492 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/class/cdc-wdm.ko 2024-03-04T13:14:42.493 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/class/usblp.ko 2024-03-04T13:14:42.493 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/class/usbtmc.ko 2024-03-04T13:14:42.493 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/host 2024-03-04T13:14:42.493 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/host/sl811-hcd.ko 2024-03-04T13:14:42.493 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/image 2024-03-04T13:14:42.493 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/image/mdc800.ko 2024-03-04T13:14:42.494 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/image/microtek.ko 2024-03-04T13:14:42.494 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/misc 2024-03-04T13:14:42.494 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/misc/adutux.ko 2024-03-04T13:14:42.494 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/misc/appledisplay.ko 2024-03-04T13:14:42.494 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/misc/emi26.ko 2024-03-04T13:14:42.494 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/misc/emi62.ko 2024-03-04T13:14:42.494 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/misc/ezusb.ko 2024-03-04T13:14:42.494 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/misc/idmouse.ko 2024-03-04T13:14:42.495 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/misc/iowarrior.ko 2024-03-04T13:14:42.495 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/misc/isight_firmware.ko 2024-03-04T13:14:42.495 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/misc/ldusb.ko 2024-03-04T13:14:42.495 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/misc/legousbtower.ko 2024-03-04T13:14:42.495 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/misc/sisusbvga 2024-03-04T13:14:42.495 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-03-04T13:14:42.495 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/misc/trancevibrator.ko 2024-03-04T13:14:42.496 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/misc/usb3503.ko 2024-03-04T13:14:42.496 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/misc/usblcd.ko 2024-03-04T13:14:42.496 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/misc/usbsevseg.ko 2024-03-04T13:14:42.496 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/misc/uss720.ko 2024-03-04T13:14:42.496 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/misc/yurex.ko 2024-03-04T13:14:42.496 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/phy 2024-03-04T13:14:42.496 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/phy/phy-generic.ko 2024-03-04T13:14:42.497 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial 2024-03-04T13:14:42.497 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/aircable.ko 2024-03-04T13:14:42.497 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/ark3116.ko 2024-03-04T13:14:42.497 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/belkin_sa.ko 2024-03-04T13:14:42.497 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/ch341.ko 2024-03-04T13:14:42.497 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/cp210x.ko 2024-03-04T13:14:42.497 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/cyberjack.ko 2024-03-04T13:14:42.497 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/cypress_m8.ko 2024-03-04T13:14:42.497 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/digi_acceleport.ko 2024-03-04T13:14:42.497 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/empeg.ko 2024-03-04T13:14:42.497 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/ftdi_sio.ko 2024-03-04T13:14:42.497 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/garmin_gps.ko 2024-03-04T13:14:42.497 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/io_edgeport.ko 2024-03-04T13:14:42.498 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/io_ti.ko 2024-03-04T13:14:42.498 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/ipaq.ko 2024-03-04T13:14:42.498 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/ipw.ko 2024-03-04T13:14:42.498 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/ir-usb.ko 2024-03-04T13:14:42.498 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-03-04T13:14:42.498 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/keyspan.ko 2024-03-04T13:14:42.498 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/keyspan_pda.ko 2024-03-04T13:14:42.498 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/kl5kusb105.ko 2024-03-04T13:14:42.498 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/kobil_sct.ko 2024-03-04T13:14:42.498 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/mct_u232.ko 2024-03-04T13:14:42.498 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/mos7720.ko 2024-03-04T13:14:42.499 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/mos7840.ko 2024-03-04T13:14:42.499 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/navman.ko 2024-03-04T13:14:42.499 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/omninet.ko 2024-03-04T13:14:42.499 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/opticon.ko 2024-03-04T13:14:42.499 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/option.ko 2024-03-04T13:14:42.499 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/oti6858.ko 2024-03-04T13:14:42.499 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/pl2303.ko 2024-03-04T13:14:42.499 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/qcaux.ko 2024-03-04T13:14:42.499 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/qcserial.ko 2024-03-04T13:14:42.499 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/quatech2.ko 2024-03-04T13:14:42.499 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/safe_serial.ko 2024-03-04T13:14:42.499 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/sierra.ko 2024-03-04T13:14:42.499 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/spcp8x5.ko 2024-03-04T13:14:42.499 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/ssu100.ko 2024-03-04T13:14:42.500 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/symbolserial.ko 2024-03-04T13:14:42.500 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-03-04T13:14:42.500 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-03-04T13:14:42.500 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/usb_debug.ko 2024-03-04T13:14:42.500 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/usb_wwan.ko 2024-03-04T13:14:42.500 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/visor.ko 2024-03-04T13:14:42.500 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/whiteheat.ko 2024-03-04T13:14:42.500 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/xsens_mt.ko 2024-03-04T13:14:42.500 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/storage 2024-03-04T13:14:42.500 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/storage/uas.ko 2024-03-04T13:14:42.500 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/storage/ums-alauda.ko 2024-03-04T13:14:42.500 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/storage/ums-cypress.ko 2024-03-04T13:14:42.500 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/storage/ums-datafab.ko 2024-03-04T13:14:42.501 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-03-04T13:14:42.501 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/storage/ums-freecom.ko 2024-03-04T13:14:42.501 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/storage/ums-isd200.ko 2024-03-04T13:14:42.501 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-03-04T13:14:42.501 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/storage/ums-karma.ko 2024-03-04T13:14:42.501 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/storage/ums-onetouch.ko 2024-03-04T13:14:42.501 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/storage/ums-realtek.ko 2024-03-04T13:14:42.501 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/storage/ums-sddr09.ko 2024-03-04T13:14:42.501 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/storage/ums-sddr55.ko 2024-03-04T13:14:42.501 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/storage/ums-usbat.ko 2024-03-04T13:14:42.501 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/storage/usb-storage.ko 2024-03-04T13:14:42.501 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/usbip 2024-03-04T13:14:42.501 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/usbip/usbip-core.ko 2024-03-04T13:14:42.502 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/usbip/usbip-host.ko 2024-03-04T13:14:42.502 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-03-04T13:14:42.502 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/vfio 2024-03-04T13:14:42.502 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/vfio/pci 2024-03-04T13:14:42.502 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-03-04T13:14:42.502 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/vfio/pci/vfio-pci.ko 2024-03-04T13:14:42.502 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/vfio/vfio.ko 2024-03-04T13:14:42.502 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-03-04T13:14:42.502 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/vhost 2024-03-04T13:14:42.502 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/vhost/vhost.ko 2024-03-04T13:14:42.502 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/vhost/vhost_iotlb.ko 2024-03-04T13:14:42.502 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/vhost/vhost_net.ko 2024-03-04T13:14:42.502 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/vhost/vhost_scsi.ko 2024-03-04T13:14:42.502 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/video 2024-03-04T13:14:42.503 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/video/backlight 2024-03-04T13:14:42.503 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/video/backlight/apple_bl.ko 2024-03-04T13:14:42.503 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/video/backlight/lcd.ko 2024-03-04T13:14:42.503 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/video/backlight/platform_lcd.ko 2024-03-04T13:14:42.503 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/virtio 2024-03-04T13:14:42.503 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/virtio/virtio.ko 2024-03-04T13:14:42.503 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/virtio/virtio_balloon.ko 2024-03-04T13:14:42.503 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/virtio/virtio_mmio.ko 2024-03-04T13:14:42.503 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/virtio/virtio_pci.ko 2024-03-04T13:14:42.503 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-03-04T13:14:42.503 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-03-04T13:14:42.504 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/virtio/virtio_ring.ko 2024-03-04T13:14:42.504 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/w1 2024-03-04T13:14:42.504 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/w1/masters 2024-03-04T13:14:42.504 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/w1/masters/ds2482.ko 2024-03-04T13:14:42.504 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/w1/masters/ds2490.ko 2024-03-04T13:14:42.504 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/w1/slaves 2024-03-04T13:14:42.504 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-03-04T13:14:42.504 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-03-04T13:14:42.504 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-03-04T13:14:42.504 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-03-04T13:14:42.504 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-03-04T13:14:42.504 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-03-04T13:14:42.504 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-03-04T13:14:42.504 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-03-04T13:14:42.505 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-03-04T13:14:42.505 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/w1/slaves/w1_smem.ko 2024-03-04T13:14:42.505 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/w1/slaves/w1_therm.ko 2024-03-04T13:14:42.505 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/w1/wire.ko 2024-03-04T13:14:42.505 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/watchdog 2024-03-04T13:14:42.505 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/watchdog/alim1535_wdt.ko 2024-03-04T13:14:42.505 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/watchdog/alim7101_wdt.ko 2024-03-04T13:14:42.505 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/watchdog/f71808e_wdt.ko 2024-03-04T13:14:42.505 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/watchdog/hpwdt.ko 2024-03-04T13:14:42.505 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/watchdog/i6300esb.ko 2024-03-04T13:14:42.505 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-03-04T13:14:42.505 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/watchdog/iTCO_wdt.ko 2024-03-04T13:14:42.505 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/watchdog/ib700wdt.ko 2024-03-04T13:14:42.505 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/watchdog/ibmasr.ko 2024-03-04T13:14:42.506 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-03-04T13:14:42.506 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/watchdog/it8712f_wdt.ko 2024-03-04T13:14:42.506 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/watchdog/it87_wdt.ko 2024-03-04T13:14:42.506 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/watchdog/machzwd.ko 2024-03-04T13:14:42.506 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/watchdog/nv_tco.ko 2024-03-04T13:14:42.506 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/watchdog/pcwd_pci.ko 2024-03-04T13:14:42.506 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/watchdog/pcwd_usb.ko 2024-03-04T13:14:42.506 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-03-04T13:14:42.506 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/watchdog/sch311x_wdt.ko 2024-03-04T13:14:42.506 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/watchdog/softdog.ko 2024-03-04T13:14:42.506 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/watchdog/sp5100_tco.ko 2024-03-04T13:14:42.506 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/watchdog/via_wdt.ko 2024-03-04T13:14:42.506 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-03-04T13:14:42.507 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/watchdog/w83877f_wdt.ko 2024-03-04T13:14:42.507 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/watchdog/w83977f_wdt.ko 2024-03-04T13:14:42.507 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/watchdog/wdt_pci.ko 2024-03-04T13:14:42.507 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/watchdog/xen_wdt.ko 2024-03-04T13:14:42.507 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/xen 2024-03-04T13:14:42.507 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/xen/xen-acpi-processor.ko 2024-03-04T13:14:42.507 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/xen/xen-evtchn.ko 2024-03-04T13:14:42.507 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/xen/xen-gntalloc.ko 2024-03-04T13:14:42.507 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/xen/xen-gntdev.ko 2024-03-04T13:14:42.507 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/xen/xen-pciback 2024-03-04T13:14:42.507 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-03-04T13:14:42.507 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/xen/xen-privcmd.ko 2024-03-04T13:14:42.507 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/xen/xenfs 2024-03-04T13:14:42.507 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/xen/xenfs/xenfs.ko 2024-03-04T13:14:42.508 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs 2024-03-04T13:14:42.508 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/9p 2024-03-04T13:14:42.508 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/9p/9p.ko 2024-03-04T13:14:42.508 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/affs 2024-03-04T13:14:42.508 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/affs/affs.ko 2024-03-04T13:14:42.508 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/befs 2024-03-04T13:14:42.508 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/befs/befs.ko 2024-03-04T13:14:42.508 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/binfmt_misc.ko 2024-03-04T13:14:42.508 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/btrfs 2024-03-04T13:14:42.508 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/btrfs/btrfs.ko 2024-03-04T13:14:42.508 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/cachefiles 2024-03-04T13:14:42.508 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/cachefiles/cachefiles.ko 2024-03-04T13:14:42.508 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/ceph 2024-03-04T13:14:42.509 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/ceph/ceph.ko 2024-03-04T13:14:42.509 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/coda 2024-03-04T13:14:42.509 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/coda/coda.ko 2024-03-04T13:14:42.509 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/cramfs 2024-03-04T13:14:42.509 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/cramfs/cramfs.ko 2024-03-04T13:14:42.509 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/dlm 2024-03-04T13:14:42.509 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/dlm/dlm.ko 2024-03-04T13:14:42.509 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/ecryptfs 2024-03-04T13:14:42.509 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/ecryptfs/ecryptfs.ko 2024-03-04T13:14:42.509 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/f2fs 2024-03-04T13:14:42.509 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/f2fs/f2fs.ko 2024-03-04T13:14:42.509 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/fat 2024-03-04T13:14:42.509 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/fat/fat.ko 2024-03-04T13:14:42.509 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/fat/msdos.ko 2024-03-04T13:14:42.510 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/fat/vfat.ko 2024-03-04T13:14:42.510 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/fuse 2024-03-04T13:14:42.510 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/fuse/cuse.ko 2024-03-04T13:14:42.510 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/fuse/fuse.ko 2024-03-04T13:14:42.510 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/gfs2 2024-03-04T13:14:42.510 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/gfs2/gfs2.ko 2024-03-04T13:14:42.510 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/hfs 2024-03-04T13:14:42.510 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/hfs/hfs.ko 2024-03-04T13:14:42.510 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/hfsplus 2024-03-04T13:14:42.510 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/hfsplus/hfsplus.ko 2024-03-04T13:14:42.510 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/isofs 2024-03-04T13:14:42.510 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/isofs/isofs.ko 2024-03-04T13:14:42.510 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/jfs 2024-03-04T13:14:42.510 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/jfs/jfs.ko 2024-03-04T13:14:42.511 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/lockd 2024-03-04T13:14:42.511 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/lockd/lockd.ko 2024-03-04T13:14:42.511 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/minix 2024-03-04T13:14:42.511 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/minix/minix.ko 2024-03-04T13:14:42.511 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/netfs 2024-03-04T13:14:42.511 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/netfs/netfs.ko 2024-03-04T13:14:42.511 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nfs 2024-03-04T13:14:42.511 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nfs/blocklayout 2024-03-04T13:14:42.511 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-03-04T13:14:42.511 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nfs/filelayout 2024-03-04T13:14:42.511 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-03-04T13:14:42.511 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nfs/flexfilelayout 2024-03-04T13:14:42.511 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-03-04T13:14:42.512 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nfs/nfs.ko 2024-03-04T13:14:42.512 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nfs/nfsv3.ko 2024-03-04T13:14:42.512 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nfs/nfsv4.ko 2024-03-04T13:14:42.512 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nfs_common 2024-03-04T13:14:42.512 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nfs_common/grace.ko 2024-03-04T13:14:42.512 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nfs_common/nfs_acl.ko 2024-03-04T13:14:42.512 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nfsd 2024-03-04T13:14:42.512 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nfsd/nfsd.ko 2024-03-04T13:14:42.512 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nilfs2 2024-03-04T13:14:42.512 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nilfs2/nilfs2.ko 2024-03-04T13:14:42.512 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls 2024-03-04T13:14:42.512 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/mac-celtic.ko 2024-03-04T13:14:42.512 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/mac-centeuro.ko 2024-03-04T13:14:42.513 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/mac-croatian.ko 2024-03-04T13:14:42.513 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/mac-cyrillic.ko 2024-03-04T13:14:42.513 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/mac-gaelic.ko 2024-03-04T13:14:42.513 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/mac-greek.ko 2024-03-04T13:14:42.513 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/mac-iceland.ko 2024-03-04T13:14:42.513 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/mac-inuit.ko 2024-03-04T13:14:42.513 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/mac-roman.ko 2024-03-04T13:14:42.513 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/mac-romanian.ko 2024-03-04T13:14:42.513 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/mac-turkish.ko 2024-03-04T13:14:42.513 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_cp1250.ko 2024-03-04T13:14:42.513 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_cp1251.ko 2024-03-04T13:14:42.513 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_cp1255.ko 2024-03-04T13:14:42.513 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_cp737.ko 2024-03-04T13:14:42.513 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_cp775.ko 2024-03-04T13:14:42.514 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_cp850.ko 2024-03-04T13:14:42.514 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_cp852.ko 2024-03-04T13:14:42.514 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_cp855.ko 2024-03-04T13:14:42.514 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_cp857.ko 2024-03-04T13:14:42.514 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_cp860.ko 2024-03-04T13:14:42.514 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_cp861.ko 2024-03-04T13:14:42.514 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_cp862.ko 2024-03-04T13:14:42.514 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_cp863.ko 2024-03-04T13:14:42.515 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_cp864.ko 2024-03-04T13:14:42.515 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_cp865.ko 2024-03-04T13:14:42.515 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_cp866.ko 2024-03-04T13:14:42.515 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_cp869.ko 2024-03-04T13:14:42.515 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_cp874.ko 2024-03-04T13:14:42.515 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_cp932.ko 2024-03-04T13:14:42.515 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_cp936.ko 2024-03-04T13:14:42.516 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_cp949.ko 2024-03-04T13:14:42.516 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_cp950.ko 2024-03-04T13:14:42.516 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_euc-jp.ko 2024-03-04T13:14:42.516 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_iso8859-1.ko 2024-03-04T13:14:42.516 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_iso8859-13.ko 2024-03-04T13:14:42.516 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_iso8859-14.ko 2024-03-04T13:14:42.516 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_iso8859-15.ko 2024-03-04T13:14:42.516 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_iso8859-2.ko 2024-03-04T13:14:42.516 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_iso8859-3.ko 2024-03-04T13:14:42.516 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_iso8859-4.ko 2024-03-04T13:14:42.516 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_iso8859-5.ko 2024-03-04T13:14:42.517 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_iso8859-6.ko 2024-03-04T13:14:42.517 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_iso8859-7.ko 2024-03-04T13:14:42.517 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_iso8859-9.ko 2024-03-04T13:14:42.517 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_koi8-r.ko 2024-03-04T13:14:42.517 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_koi8-ru.ko 2024-03-04T13:14:42.517 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_koi8-u.ko 2024-03-04T13:14:42.517 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_ucs2_utils.ko 2024-03-04T13:14:42.517 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_utf8.ko 2024-03-04T13:14:42.517 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/ocfs2 2024-03-04T13:14:42.517 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/ocfs2/cluster 2024-03-04T13:14:42.517 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-03-04T13:14:42.517 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/ocfs2/dlm 2024-03-04T13:14:42.517 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-03-04T13:14:42.518 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/ocfs2/dlmfs 2024-03-04T13:14:42.518 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-03-04T13:14:42.518 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/ocfs2/ocfs2.ko 2024-03-04T13:14:42.518 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-03-04T13:14:42.518 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-03-04T13:14:42.518 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-03-04T13:14:42.518 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/overlayfs 2024-03-04T13:14:42.518 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/overlayfs/overlay.ko 2024-03-04T13:14:42.518 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/pstore 2024-03-04T13:14:42.518 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/pstore/ramoops.ko 2024-03-04T13:14:42.518 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/reiserfs 2024-03-04T13:14:42.518 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/reiserfs/reiserfs.ko 2024-03-04T13:14:42.518 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/romfs 2024-03-04T13:14:42.518 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/romfs/romfs.ko 2024-03-04T13:14:42.518 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/smb 2024-03-04T13:14:42.519 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/smb/client 2024-03-04T13:14:42.519 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/smb/client/cifs.ko 2024-03-04T13:14:42.519 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/smb/common 2024-03-04T13:14:42.519 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/smb/common/cifs_arc4.ko 2024-03-04T13:14:42.519 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/smb/common/cifs_md4.ko 2024-03-04T13:14:42.519 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/squashfs 2024-03-04T13:14:42.519 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/squashfs/squashfs.ko 2024-03-04T13:14:42.519 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/sysv 2024-03-04T13:14:42.519 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/sysv/sysv.ko 2024-03-04T13:14:42.519 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/udf 2024-03-04T13:14:42.519 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/udf/udf.ko 2024-03-04T13:14:42.519 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/ufs 2024-03-04T13:14:42.519 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/ufs/ufs.ko 2024-03-04T13:14:42.519 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/xfs 2024-03-04T13:14:42.520 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/xfs/xfs.ko 2024-03-04T13:14:42.520 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/kernel 2024-03-04T13:14:42.520 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/kernel/rcu 2024-03-04T13:14:42.520 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/kernel/rcu/rcutorture.ko 2024-03-04T13:14:42.520 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/kernel/torture.ko 2024-03-04T13:14:42.520 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/kernel/trace 2024-03-04T13:14:42.520 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-03-04T13:14:42.520 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/lib 2024-03-04T13:14:42.520 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/lib/asn1_encoder.ko 2024-03-04T13:14:42.520 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/lib/crc-itu-t.ko 2024-03-04T13:14:42.520 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/lib/crc8.ko 2024-03-04T13:14:42.520 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/lib/crypto 2024-03-04T13:14:42.520 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/lib/crypto/libarc4.ko 2024-03-04T13:14:42.521 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/lib/crypto/libdes.ko 2024-03-04T13:14:42.521 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/lib/libcrc32c.ko 2024-03-04T13:14:42.521 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/lib/lru_cache.ko 2024-03-04T13:14:42.521 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/lib/lz4 2024-03-04T13:14:42.521 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/lib/lz4/lz4_compress.ko 2024-03-04T13:14:42.521 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/lib/lz4/lz4hc_compress.ko 2024-03-04T13:14:42.521 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/lib/math 2024-03-04T13:14:42.521 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/lib/math/cordic.ko 2024-03-04T13:14:42.522 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/lib/raid6 2024-03-04T13:14:42.522 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/lib/raid6/raid6_pq.ko 2024-03-04T13:14:42.522 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/lib/reed_solomon 2024-03-04T13:14:42.522 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/lib/reed_solomon/reed_solomon.ko 2024-03-04T13:14:42.522 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/lib/ts_bm.ko 2024-03-04T13:14:42.522 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/lib/ts_fsm.ko 2024-03-04T13:14:42.522 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/lib/ts_kmp.ko 2024-03-04T13:14:42.522 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/lib/zstd 2024-03-04T13:14:42.522 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/lib/zstd/zstd_compress.ko 2024-03-04T13:14:42.522 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/mm 2024-03-04T13:14:42.522 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/mm/hwpoison-inject.ko 2024-03-04T13:14:42.522 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net 2024-03-04T13:14:42.522 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/6lowpan 2024-03-04T13:14:42.522 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/6lowpan/6lowpan.ko 2024-03-04T13:14:42.523 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/6lowpan/nhc_dest.ko 2024-03-04T13:14:42.523 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/6lowpan/nhc_fragment.ko 2024-03-04T13:14:42.523 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/6lowpan/nhc_hop.ko 2024-03-04T13:14:42.523 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/6lowpan/nhc_ipv6.ko 2024-03-04T13:14:42.523 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/6lowpan/nhc_mobility.ko 2024-03-04T13:14:42.523 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/6lowpan/nhc_routing.ko 2024-03-04T13:14:42.523 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/6lowpan/nhc_udp.ko 2024-03-04T13:14:42.523 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/802 2024-03-04T13:14:42.523 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/802/garp.ko 2024-03-04T13:14:42.523 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/802/mrp.ko 2024-03-04T13:14:42.523 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/802/p8022.ko 2024-03-04T13:14:42.523 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/802/psnap.ko 2024-03-04T13:14:42.523 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/802/stp.ko 2024-03-04T13:14:42.523 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/8021q 2024-03-04T13:14:42.524 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/8021q/8021q.ko 2024-03-04T13:14:42.524 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/9p 2024-03-04T13:14:42.524 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/9p/9pnet.ko 2024-03-04T13:14:42.524 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/9p/9pnet_fd.ko 2024-03-04T13:14:42.524 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/9p/9pnet_rdma.ko 2024-03-04T13:14:42.524 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/9p/9pnet_virtio.ko 2024-03-04T13:14:42.524 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/batman-adv 2024-03-04T13:14:42.524 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/batman-adv/batman-adv.ko 2024-03-04T13:14:42.524 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/bridge 2024-03-04T13:14:42.524 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/bridge/br_netfilter.ko 2024-03-04T13:14:42.524 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/bridge/bridge.ko 2024-03-04T13:14:42.525 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/bridge/netfilter 2024-03-04T13:14:42.525 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-03-04T13:14:42.525 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/bridge/netfilter/ebt_among.ko 2024-03-04T13:14:42.525 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/bridge/netfilter/ebt_arp.ko 2024-03-04T13:14:42.525 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-03-04T13:14:42.525 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-03-04T13:14:42.526 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/bridge/netfilter/ebt_ip.ko 2024-03-04T13:14:42.526 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-03-04T13:14:42.526 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/bridge/netfilter/ebt_limit.ko 2024-03-04T13:14:42.526 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/bridge/netfilter/ebt_log.ko 2024-03-04T13:14:42.526 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/bridge/netfilter/ebt_mark.ko 2024-03-04T13:14:42.527 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-03-04T13:14:42.527 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-03-04T13:14:42.527 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-03-04T13:14:42.527 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-03-04T13:14:42.527 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/bridge/netfilter/ebt_snat.ko 2024-03-04T13:14:42.528 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/bridge/netfilter/ebt_stp.ko 2024-03-04T13:14:42.528 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-03-04T13:14:42.528 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-03-04T13:14:42.528 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-03-04T13:14:42.528 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-03-04T13:14:42.528 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/bridge/netfilter/ebtables.ko 2024-03-04T13:14:42.528 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-03-04T13:14:42.528 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-03-04T13:14:42.529 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-03-04T13:14:42.529 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ceph 2024-03-04T13:14:42.529 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ceph/libceph.ko 2024-03-04T13:14:42.529 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/core 2024-03-04T13:14:42.529 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/core/failover.ko 2024-03-04T13:14:42.529 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/core/pktgen.ko 2024-03-04T13:14:42.529 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/dccp 2024-03-04T13:14:42.529 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/dccp/dccp.ko 2024-03-04T13:14:42.529 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/dccp/dccp_diag.ko 2024-03-04T13:14:42.530 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/dccp/dccp_ipv4.ko 2024-03-04T13:14:42.530 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/dccp/dccp_ipv6.ko 2024-03-04T13:14:42.530 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/dns_resolver 2024-03-04T13:14:42.530 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/dns_resolver/dns_resolver.ko 2024-03-04T13:14:42.530 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/dsa 2024-03-04T13:14:42.530 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/dsa/dsa_core.ko 2024-03-04T13:14:42.530 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/dsa/tag_brcm.ko 2024-03-04T13:14:42.530 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/dsa/tag_dsa.ko 2024-03-04T13:14:42.530 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/dsa/tag_none.ko 2024-03-04T13:14:42.530 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/dsa/tag_trailer.ko 2024-03-04T13:14:42.530 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ieee802154 2024-03-04T13:14:42.530 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ieee802154/6lowpan 2024-03-04T13:14:42.531 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-03-04T13:14:42.531 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ieee802154/ieee802154.ko 2024-03-04T13:14:42.531 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ieee802154/ieee802154_socket.ko 2024-03-04T13:14:42.531 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4 2024-03-04T13:14:42.531 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/ah4.ko 2024-03-04T13:14:42.531 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/esp4.ko 2024-03-04T13:14:42.531 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/fou.ko 2024-03-04T13:14:42.531 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/gre.ko 2024-03-04T13:14:42.531 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/inet_diag.ko 2024-03-04T13:14:42.531 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/ip_gre.ko 2024-03-04T13:14:42.531 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/ip_tunnel.ko 2024-03-04T13:14:42.531 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/ip_vti.ko 2024-03-04T13:14:42.531 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/ipcomp.ko 2024-03-04T13:14:42.531 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/ipip.ko 2024-03-04T13:14:42.532 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/netfilter 2024-03-04T13:14:42.532 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/netfilter/arp_tables.ko 2024-03-04T13:14:42.532 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-03-04T13:14:42.532 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-03-04T13:14:42.532 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-03-04T13:14:42.532 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-03-04T13:14:42.532 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-03-04T13:14:42.532 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-03-04T13:14:42.532 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-03-04T13:14:42.533 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-03-04T13:14:42.533 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-03-04T13:14:42.533 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/netfilter/iptable_security.ko 2024-03-04T13:14:42.533 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-03-04T13:14:42.533 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-03-04T13:14:42.534 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-03-04T13:14:42.534 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-03-04T13:14:42.534 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-03-04T13:14:42.534 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-03-04T13:14:42.534 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-03-04T13:14:42.534 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-03-04T13:14:42.534 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-03-04T13:14:42.534 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-03-04T13:14:42.534 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/tcp_bic.ko 2024-03-04T13:14:42.535 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/tcp_dctcp.ko 2024-03-04T13:14:42.535 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/tcp_diag.ko 2024-03-04T13:14:42.535 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/tcp_highspeed.ko 2024-03-04T13:14:42.535 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/tcp_htcp.ko 2024-03-04T13:14:42.535 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/tcp_hybla.ko 2024-03-04T13:14:42.535 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/tcp_illinois.ko 2024-03-04T13:14:42.535 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/tcp_lp.ko 2024-03-04T13:14:42.535 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/tcp_scalable.ko 2024-03-04T13:14:42.535 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/tcp_vegas.ko 2024-03-04T13:14:42.535 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/tcp_veno.ko 2024-03-04T13:14:42.535 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/tcp_westwood.ko 2024-03-04T13:14:42.535 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/tcp_yeah.ko 2024-03-04T13:14:42.535 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/tunnel4.ko 2024-03-04T13:14:42.536 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/udp_diag.ko 2024-03-04T13:14:42.536 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/udp_tunnel.ko 2024-03-04T13:14:42.536 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/xfrm4_tunnel.ko 2024-03-04T13:14:42.536 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6 2024-03-04T13:14:42.536 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/ah6.ko 2024-03-04T13:14:42.536 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/esp6.ko 2024-03-04T13:14:42.536 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/fou6.ko 2024-03-04T13:14:42.536 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/ip6_tunnel.ko 2024-03-04T13:14:42.536 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-03-04T13:14:42.536 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/ip6_vti.ko 2024-03-04T13:14:42.536 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/ipcomp6.ko 2024-03-04T13:14:42.536 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/netfilter 2024-03-04T13:14:42.536 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-03-04T13:14:42.537 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-03-04T13:14:42.537 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-03-04T13:14:42.537 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-03-04T13:14:42.537 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-03-04T13:14:42.537 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-03-04T13:14:42.537 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-03-04T13:14:42.537 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-03-04T13:14:42.537 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-03-04T13:14:42.537 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-03-04T13:14:42.537 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-03-04T13:14:42.537 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-03-04T13:14:42.537 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-03-04T13:14:42.537 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-03-04T13:14:42.537 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-03-04T13:14:42.537 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-03-04T13:14:42.538 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-03-04T13:14:42.538 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-03-04T13:14:42.538 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-03-04T13:14:42.538 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-03-04T13:14:42.538 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-03-04T13:14:42.538 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-03-04T13:14:42.538 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-03-04T13:14:42.538 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-03-04T13:14:42.538 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/sit.ko 2024-03-04T13:14:42.538 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/tunnel6.ko 2024-03-04T13:14:42.538 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/xfrm6_tunnel.ko 2024-03-04T13:14:42.538 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/key 2024-03-04T13:14:42.538 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/key/af_key.ko 2024-03-04T13:14:42.538 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/l2tp 2024-03-04T13:14:42.539 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/l2tp/l2tp_core.ko 2024-03-04T13:14:42.539 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/l2tp/l2tp_debugfs.ko 2024-03-04T13:14:42.539 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/l2tp/l2tp_eth.ko 2024-03-04T13:14:42.539 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/l2tp/l2tp_ip.ko 2024-03-04T13:14:42.539 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/l2tp/l2tp_ip6.ko 2024-03-04T13:14:42.539 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/l2tp/l2tp_netlink.ko 2024-03-04T13:14:42.539 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/l2tp/l2tp_ppp.ko 2024-03-04T13:14:42.539 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/llc 2024-03-04T13:14:42.539 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/llc/llc.ko 2024-03-04T13:14:42.539 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/mac802154 2024-03-04T13:14:42.539 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/mac802154/mac802154.ko 2024-03-04T13:14:42.539 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/mpls 2024-03-04T13:14:42.539 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/mpls/mpls_gso.ko 2024-03-04T13:14:42.539 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter 2024-03-04T13:14:42.540 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/ipset 2024-03-04T13:14:42.540 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/ipset/ip_set.ko 2024-03-04T13:14:42.540 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-03-04T13:14:42.540 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-03-04T13:14:42.540 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-03-04T13:14:42.540 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-03-04T13:14:42.540 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-03-04T13:14:42.540 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-03-04T13:14:42.540 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-03-04T13:14:42.540 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-03-04T13:14:42.540 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-03-04T13:14:42.540 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-03-04T13:14:42.540 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-03-04T13:14:42.540 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-03-04T13:14:42.541 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-03-04T13:14:42.541 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-03-04T13:14:42.541 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-03-04T13:14:42.541 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/ipvs 2024-03-04T13:14:42.541 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/ipvs/ip_vs.ko 2024-03-04T13:14:42.541 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-03-04T13:14:42.541 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-03-04T13:14:42.541 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-03-04T13:14:42.541 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-03-04T13:14:42.541 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-03-04T13:14:42.541 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-03-04T13:14:42.541 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-03-04T13:14:42.541 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-03-04T13:14:42.541 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-03-04T13:14:42.542 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-03-04T13:14:42.542 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-03-04T13:14:42.542 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-03-04T13:14:42.542 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-03-04T13:14:42.542 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nf_conncount.ko 2024-03-04T13:14:42.542 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nf_conntrack.ko 2024-03-04T13:14:42.542 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-03-04T13:14:42.542 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-03-04T13:14:42.542 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-03-04T13:14:42.542 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nf_conntrack_h323.ko 2024-03-04T13:14:42.542 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nf_conntrack_irc.ko 2024-03-04T13:14:42.542 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-03-04T13:14:42.542 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-03-04T13:14:42.543 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-03-04T13:14:42.543 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nf_conntrack_sane.ko 2024-03-04T13:14:42.543 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nf_conntrack_sip.ko 2024-03-04T13:14:42.543 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-03-04T13:14:42.543 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-03-04T13:14:42.543 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nf_dup_netdev.ko 2024-03-04T13:14:42.543 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nf_flow_table.ko 2024-03-04T13:14:42.543 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nf_flow_table_inet.ko 2024-03-04T13:14:42.543 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nf_log_syslog.ko 2024-03-04T13:14:42.543 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nf_nat.ko 2024-03-04T13:14:42.543 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nf_nat_amanda.ko 2024-03-04T13:14:42.543 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nf_nat_ftp.ko 2024-03-04T13:14:42.543 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nf_nat_irc.ko 2024-03-04T13:14:42.544 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nf_nat_sip.ko 2024-03-04T13:14:42.544 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nf_nat_tftp.ko 2024-03-04T13:14:42.544 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nf_synproxy_core.ko 2024-03-04T13:14:42.544 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nf_tables.ko 2024-03-04T13:14:42.544 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nfnetlink.ko 2024-03-04T13:14:42.544 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nfnetlink_acct.ko 2024-03-04T13:14:42.544 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nfnetlink_log.ko 2024-03-04T13:14:42.544 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nfnetlink_osf.ko 2024-03-04T13:14:42.544 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nfnetlink_queue.ko 2024-03-04T13:14:42.544 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nft_chain_nat.ko 2024-03-04T13:14:42.544 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nft_compat.ko 2024-03-04T13:14:42.544 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nft_ct.ko 2024-03-04T13:14:42.544 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nft_dup_netdev.ko 2024-03-04T13:14:42.545 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nft_fib.ko 2024-03-04T13:14:42.545 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nft_fwd_netdev.ko 2024-03-04T13:14:42.545 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nft_hash.ko 2024-03-04T13:14:42.545 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nft_limit.ko 2024-03-04T13:14:42.545 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nft_log.ko 2024-03-04T13:14:42.545 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nft_masq.ko 2024-03-04T13:14:42.545 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nft_nat.ko 2024-03-04T13:14:42.545 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nft_numgen.ko 2024-03-04T13:14:42.545 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nft_queue.ko 2024-03-04T13:14:42.545 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nft_quota.ko 2024-03-04T13:14:42.545 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nft_redir.ko 2024-03-04T13:14:42.545 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nft_reject.ko 2024-03-04T13:14:42.545 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nft_reject_inet.ko 2024-03-04T13:14:42.545 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nft_socket.ko 2024-03-04T13:14:42.546 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nft_tunnel.ko 2024-03-04T13:14:42.546 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_AUDIT.ko 2024-03-04T13:14:42.546 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_CHECKSUM.ko 2024-03-04T13:14:42.546 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_CLASSIFY.ko 2024-03-04T13:14:42.546 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-03-04T13:14:42.547 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_CT.ko 2024-03-04T13:14:42.547 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_DSCP.ko 2024-03-04T13:14:42.547 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_HL.ko 2024-03-04T13:14:42.547 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_HMARK.ko 2024-03-04T13:14:42.547 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_IDLETIMER.ko 2024-03-04T13:14:42.547 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_LED.ko 2024-03-04T13:14:42.548 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_LOG.ko 2024-03-04T13:14:42.548 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_MASQUERADE.ko 2024-03-04T13:14:42.548 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_NETMAP.ko 2024-03-04T13:14:42.548 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_NFLOG.ko 2024-03-04T13:14:42.548 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_NFQUEUE.ko 2024-03-04T13:14:42.549 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_RATEEST.ko 2024-03-04T13:14:42.549 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_REDIRECT.ko 2024-03-04T13:14:42.549 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_SECMARK.ko 2024-03-04T13:14:42.549 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_TCPMSS.ko 2024-03-04T13:14:42.549 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-03-04T13:14:42.549 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_TEE.ko 2024-03-04T13:14:42.550 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_TPROXY.ko 2024-03-04T13:14:42.550 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_TRACE.ko 2024-03-04T13:14:42.550 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_addrtype.ko 2024-03-04T13:14:42.550 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_bpf.ko 2024-03-04T13:14:42.550 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_cgroup.ko 2024-03-04T13:14:42.551 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_cluster.ko 2024-03-04T13:14:42.551 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_comment.ko 2024-03-04T13:14:42.551 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_connbytes.ko 2024-03-04T13:14:42.551 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_connlabel.ko 2024-03-04T13:14:42.551 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_connlimit.ko 2024-03-04T13:14:42.551 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_connmark.ko 2024-03-04T13:14:42.552 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_conntrack.ko 2024-03-04T13:14:42.552 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_cpu.ko 2024-03-04T13:14:42.552 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_dccp.ko 2024-03-04T13:14:42.552 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_devgroup.ko 2024-03-04T13:14:42.552 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_dscp.ko 2024-03-04T13:14:42.553 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_ecn.ko 2024-03-04T13:14:42.553 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_esp.ko 2024-03-04T13:14:42.553 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_hashlimit.ko 2024-03-04T13:14:42.553 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_helper.ko 2024-03-04T13:14:42.553 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_hl.ko 2024-03-04T13:14:42.553 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_ipcomp.ko 2024-03-04T13:14:42.554 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_iprange.ko 2024-03-04T13:14:42.554 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_ipvs.ko 2024-03-04T13:14:42.554 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_l2tp.ko 2024-03-04T13:14:42.554 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_length.ko 2024-03-04T13:14:42.554 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_limit.ko 2024-03-04T13:14:42.554 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_mac.ko 2024-03-04T13:14:42.555 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_mark.ko 2024-03-04T13:14:42.555 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_multiport.ko 2024-03-04T13:14:42.555 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_nat.ko 2024-03-04T13:14:42.555 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_nfacct.ko 2024-03-04T13:14:42.555 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_osf.ko 2024-03-04T13:14:42.556 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_owner.ko 2024-03-04T13:14:42.556 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_physdev.ko 2024-03-04T13:14:42.556 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_pkttype.ko 2024-03-04T13:14:42.556 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_policy.ko 2024-03-04T13:14:42.556 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_quota.ko 2024-03-04T13:14:42.556 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_rateest.ko 2024-03-04T13:14:42.557 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_realm.ko 2024-03-04T13:14:42.557 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_recent.ko 2024-03-04T13:14:42.557 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_sctp.ko 2024-03-04T13:14:42.557 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_set.ko 2024-03-04T13:14:42.557 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_socket.ko 2024-03-04T13:14:42.557 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_state.ko 2024-03-04T13:14:42.558 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_statistic.ko 2024-03-04T13:14:42.558 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_string.ko 2024-03-04T13:14:42.558 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_tcpmss.ko 2024-03-04T13:14:42.558 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_time.ko 2024-03-04T13:14:42.558 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_u32.ko 2024-03-04T13:14:42.558 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netlink 2024-03-04T13:14:42.559 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netlink/netlink_diag.ko 2024-03-04T13:14:42.559 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/nsh 2024-03-04T13:14:42.559 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/nsh/nsh.ko 2024-03-04T13:14:42.559 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/openvswitch 2024-03-04T13:14:42.559 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/openvswitch/openvswitch.ko 2024-03-04T13:14:42.560 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/openvswitch/vport-geneve.ko 2024-03-04T13:14:42.560 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/openvswitch/vport-gre.ko 2024-03-04T13:14:42.560 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/openvswitch/vport-vxlan.ko 2024-03-04T13:14:42.560 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/packet 2024-03-04T13:14:42.560 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/packet/af_packet_diag.ko 2024-03-04T13:14:42.561 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/rds 2024-03-04T13:14:42.561 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/rds/rds.ko 2024-03-04T13:14:42.561 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/rds/rds_rdma.ko 2024-03-04T13:14:42.561 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/rds/rds_tcp.ko 2024-03-04T13:14:42.561 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched 2024-03-04T13:14:42.561 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/act_csum.ko 2024-03-04T13:14:42.562 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/act_gact.ko 2024-03-04T13:14:42.562 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/act_mirred.ko 2024-03-04T13:14:42.562 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/act_nat.ko 2024-03-04T13:14:42.562 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/act_pedit.ko 2024-03-04T13:14:42.562 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/act_police.ko 2024-03-04T13:14:42.563 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/act_simple.ko 2024-03-04T13:14:42.563 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/act_skbedit.ko 2024-03-04T13:14:42.563 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/act_vlan.ko 2024-03-04T13:14:42.563 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/cls_basic.ko 2024-03-04T13:14:42.563 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/cls_bpf.ko 2024-03-04T13:14:42.563 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/cls_flow.ko 2024-03-04T13:14:42.564 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/cls_fw.ko 2024-03-04T13:14:42.564 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/cls_route.ko 2024-03-04T13:14:42.564 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/cls_u32.ko 2024-03-04T13:14:42.564 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/em_cmp.ko 2024-03-04T13:14:42.564 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/em_ipset.ko 2024-03-04T13:14:42.565 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/em_meta.ko 2024-03-04T13:14:42.565 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/em_nbyte.ko 2024-03-04T13:14:42.565 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/em_text.ko 2024-03-04T13:14:42.565 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/em_u32.ko 2024-03-04T13:14:42.565 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/sch_choke.ko 2024-03-04T13:14:42.565 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/sch_codel.ko 2024-03-04T13:14:42.565 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/sch_drr.ko 2024-03-04T13:14:42.565 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/sch_fq.ko 2024-03-04T13:14:42.566 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/sch_gred.ko 2024-03-04T13:14:42.566 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/sch_hfsc.ko 2024-03-04T13:14:42.566 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/sch_hhf.ko 2024-03-04T13:14:42.566 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/sch_htb.ko 2024-03-04T13:14:42.566 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/sch_ingress.ko 2024-03-04T13:14:42.567 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/sch_mqprio.ko 2024-03-04T13:14:42.567 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/sch_mqprio_lib.ko 2024-03-04T13:14:42.567 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/sch_multiq.ko 2024-03-04T13:14:42.567 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/sch_netem.ko 2024-03-04T13:14:42.567 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/sch_pie.ko 2024-03-04T13:14:42.567 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/sch_plug.ko 2024-03-04T13:14:42.567 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/sch_prio.ko 2024-03-04T13:14:42.567 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/sch_qfq.ko 2024-03-04T13:14:42.567 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/sch_red.ko 2024-03-04T13:14:42.567 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/sch_sfb.ko 2024-03-04T13:14:42.568 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/sch_sfq.ko 2024-03-04T13:14:42.568 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/sch_tbf.ko 2024-03-04T13:14:42.568 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/sch_teql.ko 2024-03-04T13:14:42.568 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sctp 2024-03-04T13:14:42.568 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sctp/sctp.ko 2024-03-04T13:14:42.568 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sctp/sctp_diag.ko 2024-03-04T13:14:42.568 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sunrpc 2024-03-04T13:14:42.568 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sunrpc/auth_gss 2024-03-04T13:14:42.568 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-03-04T13:14:42.568 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-03-04T13:14:42.568 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sunrpc/sunrpc.ko 2024-03-04T13:14:42.568 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sunrpc/xprtrdma 2024-03-04T13:14:42.569 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-03-04T13:14:42.569 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/tipc 2024-03-04T13:14:42.569 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/tipc/diag.ko 2024-03-04T13:14:42.569 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/tipc/tipc.ko 2024-03-04T13:14:42.569 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/unix 2024-03-04T13:14:42.569 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/unix/unix_diag.ko 2024-03-04T13:14:42.569 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/vmw_vsock 2024-03-04T13:14:42.569 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-03-04T13:14:42.569 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-03-04T13:14:42.569 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/vmw_vsock/vsock.ko 2024-03-04T13:14:42.569 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/vmw_vsock/vsock_diag.ko 2024-03-04T13:14:42.569 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/vmw_vsock/vsock_loopback.ko 2024-03-04T13:14:42.569 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/xfrm 2024-03-04T13:14:42.569 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/xfrm/xfrm_ipcomp.ko 2024-03-04T13:14:42.570 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/security 2024-03-04T13:14:42.570 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/security/keys 2024-03-04T13:14:42.570 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/security/keys/encrypted-keys 2024-03-04T13:14:42.570 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-03-04T13:14:42.570 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/security/keys/trusted-keys 2024-03-04T13:14:42.570 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/security/keys/trusted-keys/trusted.ko 2024-03-04T13:14:42.570 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/virt 2024-03-04T13:14:42.570 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/virt/lib 2024-03-04T13:14:42.570 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/kernel/virt/lib/irqbypass.ko 2024-03-04T13:14:42.570 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/modules.alias 2024-03-04T13:14:42.570 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/modules.alias.bin 2024-03-04T13:14:42.570 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/modules.builtin 2024-03-04T13:14:42.570 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/modules.builtin.alias.bin 2024-03-04T13:14:42.571 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/modules.builtin.bin 2024-03-04T13:14:42.571 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/modules.builtin.modinfo 2024-03-04T13:14:42.571 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/modules.dep 2024-03-04T13:14:42.571 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/modules.dep.bin 2024-03-04T13:14:42.571 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/modules.devname 2024-03-04T13:14:42.571 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/modules.order 2024-03-04T13:14:42.571 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/modules.softdep 2024-03-04T13:14:42.571 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/modules.symbols 2024-03-04T13:14:42.571 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/modules.symbols.bin 2024-03-04T13:14:42.571 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc6-g786939134d7e/vmlinuz 2024-03-04T13:14:42.575 DEBUG:teuthology.task.kernel:get_image_version: 6.8.0-rc6-g786939134d7e 2024-03-04T13:14:42.576 DEBUG:teuthology.orchestra.run.smithi107:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-03-04T13:14:42.598 INFO:teuthology.orchestra.run.smithi107.stdout:postinstall scriptlet (using /bin/sh): 2024-03-04T13:14:42.598 INFO:teuthology.orchestra.run.smithi107.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-03-04T13:14:42.598 INFO:teuthology.orchestra.run.smithi107.stdout: /usr/bin/kernel-install add 6.8.0-rc6-g786939134d7e /lib/modules/6.8.0-rc6-g786939134d7e/vmlinuz 2024-03-04T13:14:42.598 INFO:teuthology.orchestra.run.smithi107.stdout:fi 2024-03-04T13:14:42.598 INFO:teuthology.orchestra.run.smithi107.stdout:for file in vmlinuz System.map config; do 2024-03-04T13:14:42.599 INFO:teuthology.orchestra.run.smithi107.stdout: if ! cmp --silent "/lib/modules/6.8.0-rc6-g786939134d7e/${file}" "/boot/${file}-6.8.0-rc6-g786939134d7e"; then 2024-03-04T13:14:42.599 INFO:teuthology.orchestra.run.smithi107.stdout: cp "/lib/modules/6.8.0-rc6-g786939134d7e/${file}" "/boot/${file}-6.8.0-rc6-g786939134d7e" 2024-03-04T13:14:42.599 INFO:teuthology.orchestra.run.smithi107.stdout: fi 2024-03-04T13:14:42.599 INFO:teuthology.orchestra.run.smithi107.stdout:done 2024-03-04T13:14:42.599 INFO:teuthology.orchestra.run.smithi107.stdout:preuninstall scriptlet (using /bin/sh): 2024-03-04T13:14:42.599 INFO:teuthology.orchestra.run.smithi107.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2024-03-04T13:14:42.599 INFO:teuthology.orchestra.run.smithi107.stdout:new-kernel-pkg --remove 6.8.0-rc6-g786939134d7e --rminitrd --initrdfile=/boot/initramfs-6.8.0-rc6-g786939134d7e.img 2024-03-04T13:14:42.599 INFO:teuthology.orchestra.run.smithi107.stdout:elif [ -x /usr/bin/kernel-install ]; then 2024-03-04T13:14:42.599 INFO:teuthology.orchestra.run.smithi107.stdout:kernel-install remove 6.8.0-rc6-g786939134d7e 2024-03-04T13:14:42.599 INFO:teuthology.orchestra.run.smithi107.stdout:fi 2024-03-04T13:14:42.599 INFO:teuthology.orchestra.run.smithi107.stdout:postuninstall scriptlet (using /bin/sh): 2024-03-04T13:14:42.599 INFO:teuthology.orchestra.run.smithi107.stdout:if [ -x /sbin/update-bootloader ]; then 2024-03-04T13:14:42.599 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/update-bootloader --remove 6.8.0-rc6-g786939134d7e 2024-03-04T13:14:42.599 INFO:teuthology.orchestra.run.smithi107.stdout:fi 2024-03-04T13:14:42.600 DEBUG:teuthology.orchestra.run.smithi107:> sudo rpm -qi grub2-tools 2024-03-04T13:14:42.675 INFO:teuthology.orchestra.run.smithi107.stdout:Name : grub2-tools 2024-03-04T13:14:42.675 INFO:teuthology.orchestra.run.smithi107.stdout:Epoch : 1 2024-03-04T13:14:42.676 INFO:teuthology.orchestra.run.smithi107.stdout:Version : 2.06 2024-03-04T13:14:42.676 INFO:teuthology.orchestra.run.smithi107.stdout:Release : 68.el9 2024-03-04T13:14:42.676 INFO:teuthology.orchestra.run.smithi107.stdout:Architecture: x86_64 2024-03-04T13:14:42.676 INFO:teuthology.orchestra.run.smithi107.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-04T13:14:42.676 INFO:teuthology.orchestra.run.smithi107.stdout:Group : Unspecified 2024-03-04T13:14:42.676 INFO:teuthology.orchestra.run.smithi107.stdout:Size : 8264254 2024-03-04T13:14:42.676 INFO:teuthology.orchestra.run.smithi107.stdout:License : GPLv3+ 2024-03-04T13:14:42.676 INFO:teuthology.orchestra.run.smithi107.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-04T13:14:42.677 INFO:teuthology.orchestra.run.smithi107.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-04T13:14:42.677 INFO:teuthology.orchestra.run.smithi107.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-04T13:14:42.677 INFO:teuthology.orchestra.run.smithi107.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-04T13:14:42.677 INFO:teuthology.orchestra.run.smithi107.stdout:Packager : builder@centos.org 2024-03-04T13:14:42.677 INFO:teuthology.orchestra.run.smithi107.stdout:Vendor : CentOS 2024-03-04T13:14:42.677 INFO:teuthology.orchestra.run.smithi107.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-04T13:14:42.677 INFO:teuthology.orchestra.run.smithi107.stdout:Summary : Support tools for GRUB. 2024-03-04T13:14:42.677 INFO:teuthology.orchestra.run.smithi107.stdout:Description : 2024-03-04T13:14:42.678 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:14:42.678 INFO:teuthology.orchestra.run.smithi107.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-04T13:14:42.678 INFO:teuthology.orchestra.run.smithi107.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-04T13:14:42.678 INFO:teuthology.orchestra.run.smithi107.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-04T13:14:42.678 INFO:teuthology.orchestra.run.smithi107.stdout:hardware devices. 2024-03-04T13:14:42.678 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:14:42.679 INFO:teuthology.orchestra.run.smithi107.stdout:This subpackage provides tools for support of all platforms. 2024-03-04T13:14:42.680 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-04T13:14:42.680 INFO:teuthology.task.kernel:Updating grub on smithi107 to boot 6.8.0-rc6-g786939134d7e 2024-03-04T13:14:42.680 DEBUG:teuthology.orchestra.run.smithi107:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-04T13:14:43.273 INFO:teuthology.orchestra.run.smithi107.stderr:Generating grub configuration file ... 2024-03-04T13:14:44.721 INFO:teuthology.orchestra.run.smithi107.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-04T13:14:44.751 INFO:teuthology.orchestra.run.smithi107.stderr:done 2024-03-04T13:14:44.761 DEBUG:teuthology.orchestra.run.smithi107:> mktemp 2024-03-04T13:14:44.776 INFO:teuthology.orchestra.run.smithi107.stdout:/tmp/tmp.pVzm23LGFW 2024-03-04T13:14:44.776 DEBUG:teuthology.orchestra.run.smithi107:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.pVzm23LGFW 2024-03-04T13:14:44.845 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod 0666 /tmp/tmp.pVzm23LGFW 2024-03-04T13:14:44.989 DEBUG:teuthology.orchestra.remote:smithi107:/tmp/tmp.pVzm23LGFW is 6KB 2024-03-04T13:14:45.038 DEBUG:teuthology.orchestra.run.smithi107:> rm -fr /tmp/tmp.pVzm23LGFW 2024-03-04T13:14:45.052 DEBUG:teuthology.orchestra.run.smithi107:> sudo /bin/ls /boot/loader/entries || true 2024-03-04T13:14:45.126 INFO:teuthology.orchestra.run.smithi107.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-04T13:14:45.126 INFO:teuthology.orchestra.run.smithi107.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-04T13:14:45.126 INFO:teuthology.orchestra.run.smithi107.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-04T13:14:45.126 INFO:teuthology.orchestra.run.smithi107.stdout:15949e3c88704300882482eddfd7baae-6.8.0-rc6-g786939134d7e.conf 2024-03-04T13:14:45.127 DEBUG:teuthology.orchestra.run.smithi107:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.8.0-rc6-g786939134d7e 2024-03-04T13:14:45.282 DEBUG:teuthology.orchestra.run.smithi107:> sudo shutdown -r now 2024-03-04T13:15:15.285 INFO:teuthology.task.kernel:Checking client host.a for new kernel version... 2024-03-04T13:15:15.286 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-03-04T13:15:15.287 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-04T13:15:49.948 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.107 2024-03-04T13:15:58.949 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-03-04T13:15:58.951 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-04T13:16:58.953 DEBUG:teuthology.orchestra.remote:timed out 2024-03-04T13:17:10.954 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-03-04T13:17:10.956 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-04T13:17:11.400 DEBUG:teuthology.orchestra.run.smithi107:> true 2024-03-04T13:17:12.001 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-03-04T13:17:12.001 INFO:teuthology.task.kernel:Checking kernel version of host.a, want "6.8.0-rc6-g786939134d7e"... 2024-03-04T13:17:12.001 DEBUG:teuthology.orchestra.run.smithi107:> uname -r 2024-03-04T13:17:12.052 INFO:teuthology.orchestra.run.smithi107.stdout:6.8.0-rc6-g786939134d7e 2024-03-04T13:17:12.053 DEBUG:teuthology.task.kernel:current kernel version is 6.8.0-rc6-g786939134d7e vs 6.8.0-rc6-g786939134d7e 2024-03-04T13:17:12.053 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-04T13:17:12.053 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-04T13:17:13.053 INFO:teuthology.task.kernel:Enabling kdb on host.a... 2024-03-04T13:17:13.054 DEBUG:teuthology.orchestra.run.smithi107:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-04T13:17:13.205 INFO:teuthology.orchestra.run.smithi107.stdout:ttyS1 2024-03-04T13:17:13.273 DEBUG:teuthology.parallel:result is None 2024-03-04T13:17:13.274 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-04T13:17:13.283 INFO:teuthology.task.internal:Creating test directory... 2024-03-04T13:17:13.283 DEBUG:teuthology.orchestra.run.smithi107:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-04T13:17:13.290 DEBUG:teuthology.orchestra.run.smithi169:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-04T13:17:13.349 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-04T13:17:13.362 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-04T13:17:13.370 INFO:teuthology.task.internal:Creating archive directory... 2024-03-04T13:17:13.370 DEBUG:teuthology.orchestra.run.smithi107:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-04T13:17:13.377 DEBUG:teuthology.orchestra.run.smithi169:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-04T13:17:13.443 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-04T13:17:13.451 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-04T13:17:13.451 DEBUG:teuthology.orchestra.run.smithi107:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-04T13:17:13.458 DEBUG:teuthology.orchestra.run.smithi169:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-04T13:17:13.551 INFO:teuthology.orchestra.run.smithi107.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-04T13:17:13.565 INFO:teuthology.orchestra.run.smithi169.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-04T13:17:13.590 INFO:teuthology.orchestra.run.smithi107.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-04T13:17:13.606 INFO:teuthology.orchestra.run.smithi169.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-04T13:17:13.611 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-04T13:17:13.619 INFO:teuthology.task.internal:Configuring sudo... 2024-03-04T13:17:13.619 DEBUG:teuthology.orchestra.run.smithi107:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-04T13:17:13.626 DEBUG:teuthology.orchestra.run.smithi169:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-04T13:17:13.723 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-04T13:17:13.733 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-04T13:17:13.733 DEBUG:teuthology.orchestra.run.smithi107:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-04T13:17:13.740 DEBUG:teuthology.orchestra.run.smithi169:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-04T13:17:13.787 DEBUG:teuthology.orchestra.run.smithi107:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-04T13:17:13.866 DEBUG:teuthology.orchestra.run.smithi107:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-04T13:17:13.986 DEBUG:teuthology.orchestra.run.smithi107:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-04T13:17:14.069 DEBUG:teuthology.orchestra.run.smithi107:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-04T13:17:14.162 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-03-04T13:17:14.162 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-04T13:17:14.249 DEBUG:teuthology.orchestra.run.smithi169:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-04T13:17:14.337 DEBUG:teuthology.orchestra.run.smithi169:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-04T13:17:14.438 DEBUG:teuthology.orchestra.run.smithi169:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-04T13:17:14.524 DEBUG:teuthology.orchestra.run.smithi169:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-04T13:17:14.623 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-03-04T13:17:14.623 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-04T13:17:14.717 DEBUG:teuthology.orchestra.run.smithi107:> sudo service rsyslog restart 2024-03-04T13:17:14.724 DEBUG:teuthology.orchestra.run.smithi169:> sudo service rsyslog restart 2024-03-04T13:17:14.842 INFO:teuthology.orchestra.run.smithi107.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-04T13:17:14.885 INFO:teuthology.orchestra.run.smithi169.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-04T13:17:15.179 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-04T13:17:15.186 INFO:teuthology.task.internal:Starting timer... 2024-03-04T13:17:15.186 INFO:teuthology.run_tasks:Running task pcp... 2024-03-04T13:17:15.206 INFO:teuthology.run_tasks:Running task selinux... 2024-03-04T13:17:15.217 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-03-04T13:17:15.217 DEBUG:teuthology.orchestra.run.smithi107:> sudo service auditd rotate 2024-03-04T13:17:15.360 INFO:teuthology.orchestra.run.smithi107.stdout:Rotating logs: 2024-03-04T13:17:15.365 DEBUG:teuthology.orchestra.run.smithi169:> sudo service auditd rotate 2024-03-04T13:17:15.493 INFO:teuthology.orchestra.run.smithi169.stdout:Rotating logs: 2024-03-04T13:17:15.497 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-04T13:17:15.497 DEBUG:teuthology.orchestra.run.smithi107:> /usr/sbin/getenforce 2024-03-04T13:17:15.568 INFO:teuthology.orchestra.run.smithi107.stdout:Permissive 2024-03-04T13:17:15.568 DEBUG:teuthology.orchestra.run.smithi169:> /usr/sbin/getenforce 2024-03-04T13:17:15.621 INFO:teuthology.orchestra.run.smithi169.stdout:Permissive 2024-03-04T13:17:15.621 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi107.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi169.front.sepia.ceph.com': 'permissive'} 2024-03-04T13:17:15.622 DEBUG:teuthology.orchestra.run.smithi107:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-04T13:17:15.705 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-04T13:17:15.706 DEBUG:teuthology.orchestra.run.smithi169:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-04T13:17:15.787 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-04T13:17:15.788 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-04T13:17:15.788 DEBUG:teuthology.orchestra.run.smithi107:> sudo /usr/sbin/setenforce permissive 2024-03-04T13:17:15.882 DEBUG:teuthology.orchestra.run.smithi169:> sudo /usr/sbin/setenforce permissive 2024-03-04T13:17:15.967 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-04T13:17:15.976 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-03-04T13:17:15.991 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-03-04T13:17:16.148 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-04T13:17:16.183 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-04T13:17:16.184 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi107.front.sepia.ceph.com,smithi169.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-04T13:23:14.751 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi107.front.sepia.ceph.com'), Remote(name='ubuntu@smithi169.front.sepia.ceph.com')] 2024-03-04T13:23:14.752 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-03-04T13:23:14.753 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-04T13:23:14.903 DEBUG:teuthology.orchestra.run.smithi107:> true 2024-03-04T13:23:15.051 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-03-04T13:23:15.052 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi169.front.sepia.ceph.com' 2024-03-04T13:23:15.052 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-04T13:23:15.198 DEBUG:teuthology.orchestra.run.smithi169:> true 2024-03-04T13:23:15.349 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi169.front.sepia.ceph.com' 2024-03-04T13:23:15.350 INFO:teuthology.run_tasks:Running task clock... 2024-03-04T13:23:15.361 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-04T13:23:15.361 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-04T13:23:15.362 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-04T13:23:15.369 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-04T13:23:15.370 DEBUG:teuthology.orchestra.run.smithi169:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-04T13:23:15.471 INFO:teuthology.orchestra.run.smithi107.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-04T13:23:15.472 INFO:teuthology.orchestra.run.smithi169.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-04T13:23:15.523 INFO:teuthology.orchestra.run.smithi169.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-04T13:23:15.523 INFO:teuthology.orchestra.run.smithi107.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-04T13:23:15.618 INFO:teuthology.orchestra.run.smithi107.stderr:sudo: ntpd: command not found 2024-03-04T13:23:15.620 INFO:teuthology.orchestra.run.smithi169.stderr:sudo: ntpd: command not found 2024-03-04T13:23:15.661 INFO:teuthology.orchestra.run.smithi107.stdout:506 Cannot talk to daemon 2024-03-04T13:23:15.662 INFO:teuthology.orchestra.run.smithi169.stdout:506 Cannot talk to daemon 2024-03-04T13:23:15.711 INFO:teuthology.orchestra.run.smithi169.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-04T13:23:15.716 INFO:teuthology.orchestra.run.smithi107.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-04T13:23:15.763 INFO:teuthology.orchestra.run.smithi169.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-04T13:23:15.771 INFO:teuthology.orchestra.run.smithi107.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-04T13:23:15.894 INFO:teuthology.orchestra.run.smithi169.stderr:bash: line 1: ntpq: command not found 2024-03-04T13:23:15.900 INFO:teuthology.orchestra.run.smithi107.stderr:bash: line 1: ntpq: command not found 2024-03-04T13:23:15.903 INFO:teuthology.orchestra.run.smithi169.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-04T13:23:15.903 INFO:teuthology.orchestra.run.smithi169.stdout:=============================================================================== 2024-03-04T13:23:15.904 INFO:teuthology.orchestra.run.smithi169.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-04T13:23:15.904 INFO:teuthology.orchestra.run.smithi169.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-04T13:23:15.904 INFO:teuthology.orchestra.run.smithi169.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-04T13:23:15.904 INFO:teuthology.orchestra.run.smithi169.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-04T13:23:15.907 INFO:teuthology.orchestra.run.smithi107.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-04T13:23:15.908 INFO:teuthology.orchestra.run.smithi107.stdout:=============================================================================== 2024-03-04T13:23:15.908 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-04T13:23:15.908 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-04T13:23:15.908 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-04T13:23:15.908 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-04T13:23:15.909 INFO:teuthology.run_tasks:Running task install... 2024-03-04T13:23:15.918 DEBUG:teuthology.task.install:project ceph 2024-03-04T13:23:15.918 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '717ce3cce8d0166e4e577faa8004e6f9cb4128c0'}} 2024-03-04T13:23:15.918 DEBUG:teuthology.task.install:config {'branch': 'quincy', 'exclude_packages': ['ceph-volume'], 'flavor': 'default', 'sha1': '717ce3cce8d0166e4e577faa8004e6f9cb4128c0'} 2024-03-04T13:23:15.918 INFO:teuthology.task.install:Using flavor: default 2024-03-04T13:23:15.926 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-04T13:23:15.927 INFO:teuthology.task.install:extra packages: [] 2024-03-04T13:23:15.927 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'quincy', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '717ce3cce8d0166e4e577faa8004e6f9cb4128c0', 'tag': None, 'wait_for_package': False} 2024-03-04T13:23:15.928 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-04T13:23:15.928 INFO:teuthology.packaging:ref: None 2024-03-04T13:23:15.928 INFO:teuthology.packaging:tag: None 2024-03-04T13:23:15.928 INFO:teuthology.packaging:branch: quincy 2024-03-04T13:23:15.928 INFO:teuthology.packaging:sha1: 717ce3cce8d0166e4e577faa8004e6f9cb4128c0 2024-03-04T13:23:15.928 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-03-04T13:23:15.930 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'quincy', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '717ce3cce8d0166e4e577faa8004e6f9cb4128c0', 'tag': None, 'wait_for_package': False} 2024-03-04T13:23:15.931 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-04T13:23:15.931 INFO:teuthology.packaging:ref: None 2024-03-04T13:23:15.931 INFO:teuthology.packaging:tag: None 2024-03-04T13:23:15.931 INFO:teuthology.packaging:branch: quincy 2024-03-04T13:23:15.931 INFO:teuthology.packaging:sha1: 717ce3cce8d0166e4e577faa8004e6f9cb4128c0 2024-03-04T13:23:15.931 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-03-04T13:23:17.105 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/quincy/42608bb191117be10dba720e6bf8a02f3171cd34/centos/9/flavors/default/ 2024-03-04T13:23:17.105 INFO:teuthology.task.install.rpm:Package version is 17.2.7-545.g42608bb1 2024-03-04T13:23:17.213 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/quincy/42608bb191117be10dba720e6bf8a02f3171cd34/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/quincy/42608bb191117be10dba720e6bf8a02f3171cd34/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/quincy/42608bb191117be10dba720e6bf8a02f3171cd34/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-04T13:23:17.213 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-03-04T13:23:17.214 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-04T13:23:17.314 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-03-04T13:23:17.314 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-04T13:23:17.314 INFO:teuthology.packaging:ref: None 2024-03-04T13:23:17.314 INFO:teuthology.packaging:tag: None 2024-03-04T13:23:17.314 INFO:teuthology.packaging:branch: quincy 2024-03-04T13:23:17.314 INFO:teuthology.packaging:sha1: 717ce3cce8d0166e4e577faa8004e6f9cb4128c0 2024-03-04T13:23:17.315 DEBUG:teuthology.orchestra.run.smithi169:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/quincy/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-04T13:23:17.449 DEBUG:teuthology.orchestra.run.smithi169:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-04T13:23:17.580 DEBUG:teuthology.orchestra.run.smithi169:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-04T13:23:17.666 INFO:teuthology.orchestra.run.smithi169.stdout:check_obsoletes = 1 2024-03-04T13:23:17.671 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum clean all 2024-03-04T13:23:18.027 INFO:teuthology.orchestra.run.smithi169.stdout:74 files removed 2024-03-04T13:23:18.065 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install ceph-radosgw 2024-03-04T13:23:18.454 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/quincy/42608bb191117be10dba720e6bf8a02f3171cd34/centos/9/flavors/default/ 2024-03-04T13:23:18.454 INFO:teuthology.task.install.rpm:Package version is 17.2.7-545.g42608bb1 2024-03-04T13:23:18.796 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/quincy/42608bb191117be10dba720e6bf8a02f3171cd34/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/quincy/42608bb191117be10dba720e6bf8a02f3171cd34/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/quincy/42608bb191117be10dba720e6bf8a02f3171cd34/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-04T13:23:18.796 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-03-04T13:23:18.796 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-04T13:23:18.896 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-03-04T13:23:18.896 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-04T13:23:18.896 INFO:teuthology.packaging:ref: None 2024-03-04T13:23:18.896 INFO:teuthology.packaging:tag: None 2024-03-04T13:23:18.896 INFO:teuthology.packaging:branch: quincy 2024-03-04T13:23:18.896 INFO:teuthology.packaging:sha1: 717ce3cce8d0166e4e577faa8004e6f9cb4128c0 2024-03-04T13:23:18.896 DEBUG:teuthology.orchestra.run.smithi107:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/quincy/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-04T13:23:19.035 DEBUG:teuthology.orchestra.run.smithi107:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-04T13:23:19.166 DEBUG:teuthology.orchestra.run.smithi107:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-04T13:23:19.256 INFO:teuthology.orchestra.run.smithi107.stdout:check_obsoletes = 1 2024-03-04T13:23:19.260 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum clean all 2024-03-04T13:23:19.615 INFO:teuthology.orchestra.run.smithi107.stdout:74 files removed 2024-03-04T13:23:19.654 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install ceph-radosgw 2024-03-04T13:23:20.582 INFO:teuthology.orchestra.run.smithi169.stdout:ceph packages for x86_64 38 kB/s | 77 kB 00:02 2024-03-04T13:23:22.192 INFO:teuthology.orchestra.run.smithi107.stdout:ceph packages for x86_64 38 kB/s | 77 kB 00:02 2024-03-04T13:23:22.199 INFO:teuthology.orchestra.run.smithi169.stdout:ceph noarch packages 6.0 kB/s | 9.1 kB 00:01 2024-03-04T13:23:23.026 INFO:teuthology.orchestra.run.smithi107.stdout:ceph noarch packages 12 kB/s | 9.1 kB 00:00 2024-03-04T13:23:23.053 INFO:teuthology.orchestra.run.smithi169.stdout:ceph source packages 2.3 kB/s | 1.8 kB 00:00 2024-03-04T13:23:24.295 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS-9 - AppStream 5.8 MB/s | 6.7 MB 00:01 2024-03-04T13:23:25.134 INFO:teuthology.orchestra.run.smithi107.stdout:ceph source packages 901 B/s | 1.8 kB 00:02 2024-03-04T13:23:26.332 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS-9 - AppStream 5.8 MB/s | 6.7 MB 00:01 2024-03-04T13:23:26.337 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS-9 - BaseOS 2.6 MB/s | 2.0 MB 00:00 2024-03-04T13:23:28.071 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS Stream 9 - BaseOS 6.1 MB/s | 8.0 MB 00:01 2024-03-04T13:23:28.857 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS-9 - BaseOS 1.6 MB/s | 2.0 MB 00:01 2024-03-04T13:23:30.541 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - BaseOS 6.4 MB/s | 8.0 MB 00:01 2024-03-04T13:24:13.662 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS Stream 9 - AppStream 443 kB/s | 19 MB 00:44 2024-03-04T13:24:15.887 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - AppStream 445 kB/s | 19 MB 00:44 2024-03-04T13:24:18.851 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS Stream 9 - CRB 4.7 MB/s | 6.1 MB 00:01 2024-03-04T13:24:20.688 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS Stream 9 - Extras packages 33 kB/s | 15 kB 00:00 2024-03-04T13:24:21.459 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - CRB 3.5 MB/s | 6.1 MB 00:01 2024-03-04T13:24:22.937 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - Extras packages 105 kB/s | 15 kB 00:00 2024-03-04T13:24:23.931 INFO:teuthology.orchestra.run.smithi169.stdout:Extra Packages for Enterprise Linux 6.7 MB/s | 20 MB 00:03 2024-03-04T13:24:29.698 INFO:teuthology.orchestra.run.smithi169.stdout:lab-extras 22 kB/s | 1.7 kB 00:00 2024-03-04T13:24:32.459 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2024-03-04T13:24:32.471 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-03-04T13:24:32.471 INFO:teuthology.orchestra.run.smithi169.stdout: Package Arch Version Repository Size 2024-03-04T13:24:32.472 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-03-04T13:24:32.472 INFO:teuthology.orchestra.run.smithi169.stdout:Installing: 2024-03-04T13:24:32.472 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-radosgw x86_64 2:17.2.7-545.g42608bb1.el9 ceph 14 M 2024-03-04T13:24:32.472 INFO:teuthology.orchestra.run.smithi169.stdout:Upgrading: 2024-03-04T13:24:32.472 INFO:teuthology.orchestra.run.smithi169.stdout: librados2 x86_64 2:17.2.7-545.g42608bb1.el9 ceph 3.1 M 2024-03-04T13:24:32.472 INFO:teuthology.orchestra.run.smithi169.stdout: librbd1 x86_64 2:17.2.7-545.g42608bb1.el9 ceph 2.9 M 2024-03-04T13:24:32.472 INFO:teuthology.orchestra.run.smithi169.stdout:Installing dependencies: 2024-03-04T13:24:32.472 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-base x86_64 2:17.2.7-545.g42608bb1.el9 ceph 5.0 M 2024-03-04T13:24:32.472 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-common x86_64 2:17.2.7-545.g42608bb1.el9 ceph 23 M 2024-03-04T13:24:32.473 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-selinux x86_64 2:17.2.7-545.g42608bb1.el9 ceph 25 k 2024-03-04T13:24:32.473 INFO:teuthology.orchestra.run.smithi169.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-04T13:24:32.473 INFO:teuthology.orchestra.run.smithi169.stdout: libcephfs2 x86_64 2:17.2.7-545.g42608bb1.el9 ceph 635 k 2024-03-04T13:24:32.473 INFO:teuthology.orchestra.run.smithi169.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-04T13:24:32.473 INFO:teuthology.orchestra.run.smithi169.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-04T13:24:32.473 INFO:teuthology.orchestra.run.smithi169.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-04T13:24:32.473 INFO:teuthology.orchestra.run.smithi169.stdout: libradosstriper1 x86_64 2:17.2.7-545.g42608bb1.el9 ceph 457 k 2024-03-04T13:24:32.473 INFO:teuthology.orchestra.run.smithi169.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-04T13:24:32.473 INFO:teuthology.orchestra.run.smithi169.stdout: librgw2 x86_64 2:17.2.7-545.g42608bb1.el9 ceph 4.9 M 2024-03-04T13:24:32.473 INFO:teuthology.orchestra.run.smithi169.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-04T13:24:32.473 INFO:teuthology.orchestra.run.smithi169.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-04T13:24:32.473 INFO:teuthology.orchestra.run.smithi169.stdout: python3-ceph-argparse x86_64 2:17.2.7-545.g42608bb1.el9 ceph 45 k 2024-03-04T13:24:32.473 INFO:teuthology.orchestra.run.smithi169.stdout: python3-ceph-common x86_64 2:17.2.7-545.g42608bb1.el9 ceph 116 k 2024-03-04T13:24:32.474 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cephfs x86_64 2:17.2.7-545.g42608bb1.el9 ceph 156 k 2024-03-04T13:24:32.474 INFO:teuthology.orchestra.run.smithi169.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-04T13:24:32.474 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rados x86_64 2:17.2.7-545.g42608bb1.el9 ceph 320 k 2024-03-04T13:24:32.474 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rbd x86_64 2:17.2.7-545.g42608bb1.el9 ceph 292 k 2024-03-04T13:24:32.474 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rgw x86_64 2:17.2.7-545.g42608bb1.el9 ceph 99 k 2024-03-04T13:24:32.474 INFO:teuthology.orchestra.run.smithi169.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-04T13:24:32.474 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-03-04T13:24:32.474 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction Summary 2024-03-04T13:24:32.474 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-03-04T13:24:32.474 INFO:teuthology.orchestra.run.smithi169.stdout:Install 22 Packages 2024-03-04T13:24:32.474 INFO:teuthology.orchestra.run.smithi169.stdout:Upgrade 2 Packages 2024-03-04T13:24:32.474 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-03-04T13:24:32.475 INFO:teuthology.orchestra.run.smithi169.stdout:Total download size: 58 M 2024-03-04T13:24:32.475 INFO:teuthology.orchestra.run.smithi169.stdout:Downloading Packages: 2024-03-04T13:24:35.106 INFO:teuthology.orchestra.run.smithi169.stdout:(1/24): ceph-common-17.2.7-545.g42608bb1.el9.x8 8.9 MB/s | 23 MB 00:02 2024-03-04T13:24:35.157 INFO:teuthology.orchestra.run.smithi169.stdout:(2/24): ceph-selinux-17.2.7-545.g42608bb1.el9.x 508 kB/s | 25 kB 00:00 2024-03-04T13:24:35.241 INFO:teuthology.orchestra.run.smithi169.stdout:(3/24): libcephfs2-17.2.7-545.g42608bb1.el9.x86 7.5 MB/s | 635 kB 00:00 2024-03-04T13:24:35.299 INFO:teuthology.orchestra.run.smithi169.stdout:(4/24): libradosstriper1-17.2.7-545.g42608bb1.e 7.7 MB/s | 457 kB 00:00 2024-03-04T13:24:35.583 INFO:teuthology.orchestra.run.smithi169.stdout:(5/24): librgw2-17.2.7-545.g42608bb1.el9.x86_64 17 MB/s | 4.9 MB 00:00 2024-03-04T13:24:35.725 INFO:teuthology.orchestra.run.smithi169.stdout:(6/24): python3-ceph-argparse-17.2.7-545.g42608 320 kB/s | 45 kB 00:00 2024-03-04T13:24:35.775 INFO:teuthology.orchestra.run.smithi169.stdout:(7/24): python3-ceph-common-17.2.7-545.g42608bb 2.3 MB/s | 116 kB 00:00 2024-03-04T13:24:35.825 INFO:teuthology.orchestra.run.smithi169.stdout:(8/24): python3-cephfs-17.2.7-545.g42608bb1.el9 3.1 MB/s | 156 kB 00:00 2024-03-04T13:24:35.901 INFO:teuthology.orchestra.run.smithi169.stdout:(9/24): python3-rados-17.2.7-545.g42608bb1.el9. 4.2 MB/s | 320 kB 00:00 2024-03-04T13:24:35.959 INFO:teuthology.orchestra.run.smithi169.stdout:(10/24): python3-rbd-17.2.7-545.g42608bb1.el9.x 5.0 MB/s | 292 kB 00:00 2024-03-04T13:24:36.009 INFO:teuthology.orchestra.run.smithi169.stdout:(11/24): python3-rgw-17.2.7-545.g42608bb1.el9.x 2.0 MB/s | 99 kB 00:00 2024-03-04T13:24:38.218 INFO:teuthology.orchestra.run.smithi169.stdout:(12/24): libpmemobj-1.12.1-1.el9.x86_64.rpm 72 kB/s | 159 kB 00:02 2024-03-04T13:24:39.051 INFO:teuthology.orchestra.run.smithi169.stdout:(13/24): librabbitmq-0.11.0-7.el9.x86_64.rpm 53 kB/s | 44 kB 00:00 2024-03-04T13:24:40.818 INFO:teuthology.orchestra.run.smithi169.stdout:(14/24): librdkafka-1.6.1-102.el9.x86_64.rpm 374 kB/s | 661 kB 00:01 2024-03-04T13:24:41.427 INFO:teuthology.orchestra.run.smithi169.stdout:(15/24): lttng-ust-2.12.0-6.el9.x86_64.rpm 465 kB/s | 282 kB 00:00 2024-03-04T13:24:41.637 INFO:teuthology.orchestra.run.smithi169.stdout:(16/24): python3-prettytable-0.7.2-27.el9.noarc 196 kB/s | 41 kB 00:00 2024-03-04T13:24:42.460 INFO:teuthology.orchestra.run.smithi169.stdout:(17/24): gperftools-libs-2.9.1-2.el9.x86_64.rpm 375 kB/s | 309 kB 00:00 2024-03-04T13:24:42.727 INFO:teuthology.orchestra.run.smithi169.stdout:(18/24): liboath-2.6.7-2.el9.x86_64.rpm 182 kB/s | 49 kB 00:00 2024-03-04T13:24:43.244 INFO:teuthology.orchestra.run.smithi169.stdout:(19/24): libunwind-1.6.2-1.el9.x86_64.rpm 130 kB/s | 67 kB 00:00 2024-03-04T13:24:44.428 INFO:teuthology.orchestra.run.smithi169.stdout:(20/24): thrift-0.15.0-2.el9.x86_64.rpm 1.3 MB/s | 1.6 MB 00:01 2024-03-04T13:24:44.711 INFO:teuthology.orchestra.run.smithi169.stdout:(21/24): librados2-17.2.7-545.g42608bb1.el9.x86 11 MB/s | 3.1 MB 00:00 2024-03-04T13:24:44.928 INFO:teuthology.orchestra.run.smithi169.stdout:(22/24): librbd1-17.2.7-545.g42608bb1.el9.x86_6 14 MB/s | 2.9 MB 00:00 2024-03-04T13:24:52.003 INFO:teuthology.orchestra.run.smithi169.stdout:(23/24): ceph-base-17.2.7-545.g42608bb1.el9.x86 266 kB/s | 5.0 MB 00:19 2024-03-04T13:24:53.983 INFO:teuthology.orchestra.run.smithi107.stdout:Extra Packages for Enterprise Linux 680 kB/s | 20 MB 00:30 2024-03-04T13:24:59.691 INFO:teuthology.orchestra.run.smithi107.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-03-04T13:25:02.562 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-03-04T13:25:02.574 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-03-04T13:25:02.574 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repository Size 2024-03-04T13:25:02.575 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-03-04T13:25:02.575 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-03-04T13:25:02.575 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-radosgw x86_64 2:17.2.7-545.g42608bb1.el9 ceph 14 M 2024-03-04T13:25:02.575 INFO:teuthology.orchestra.run.smithi107.stdout:Upgrading: 2024-03-04T13:25:02.575 INFO:teuthology.orchestra.run.smithi107.stdout: librados2 x86_64 2:17.2.7-545.g42608bb1.el9 ceph 3.1 M 2024-03-04T13:25:02.575 INFO:teuthology.orchestra.run.smithi107.stdout: librbd1 x86_64 2:17.2.7-545.g42608bb1.el9 ceph 2.9 M 2024-03-04T13:25:02.575 INFO:teuthology.orchestra.run.smithi107.stdout:Installing dependencies: 2024-03-04T13:25:02.575 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-base x86_64 2:17.2.7-545.g42608bb1.el9 ceph 5.0 M 2024-03-04T13:25:02.575 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-common x86_64 2:17.2.7-545.g42608bb1.el9 ceph 23 M 2024-03-04T13:25:02.575 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-selinux x86_64 2:17.2.7-545.g42608bb1.el9 ceph 25 k 2024-03-04T13:25:02.576 INFO:teuthology.orchestra.run.smithi107.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-04T13:25:02.576 INFO:teuthology.orchestra.run.smithi107.stdout: libcephfs2 x86_64 2:17.2.7-545.g42608bb1.el9 ceph 635 k 2024-03-04T13:25:02.576 INFO:teuthology.orchestra.run.smithi107.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-04T13:25:02.576 INFO:teuthology.orchestra.run.smithi107.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-04T13:25:02.576 INFO:teuthology.orchestra.run.smithi107.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-04T13:25:02.576 INFO:teuthology.orchestra.run.smithi107.stdout: libradosstriper1 x86_64 2:17.2.7-545.g42608bb1.el9 ceph 457 k 2024-03-04T13:25:02.576 INFO:teuthology.orchestra.run.smithi107.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-04T13:25:02.576 INFO:teuthology.orchestra.run.smithi107.stdout: librgw2 x86_64 2:17.2.7-545.g42608bb1.el9 ceph 4.9 M 2024-03-04T13:25:02.576 INFO:teuthology.orchestra.run.smithi107.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-04T13:25:02.576 INFO:teuthology.orchestra.run.smithi107.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-04T13:25:02.576 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ceph-argparse x86_64 2:17.2.7-545.g42608bb1.el9 ceph 45 k 2024-03-04T13:25:02.577 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ceph-common x86_64 2:17.2.7-545.g42608bb1.el9 ceph 116 k 2024-03-04T13:25:02.577 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cephfs x86_64 2:17.2.7-545.g42608bb1.el9 ceph 156 k 2024-03-04T13:25:02.577 INFO:teuthology.orchestra.run.smithi107.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-04T13:25:02.577 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rados x86_64 2:17.2.7-545.g42608bb1.el9 ceph 320 k 2024-03-04T13:25:02.577 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rbd x86_64 2:17.2.7-545.g42608bb1.el9 ceph 292 k 2024-03-04T13:25:02.577 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rgw x86_64 2:17.2.7-545.g42608bb1.el9 ceph 99 k 2024-03-04T13:25:02.577 INFO:teuthology.orchestra.run.smithi107.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-04T13:25:02.577 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:25:02.577 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-03-04T13:25:02.577 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-03-04T13:25:02.577 INFO:teuthology.orchestra.run.smithi107.stdout:Install 22 Packages 2024-03-04T13:25:02.577 INFO:teuthology.orchestra.run.smithi107.stdout:Upgrade 2 Packages 2024-03-04T13:25:02.577 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:25:02.578 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 58 M 2024-03-04T13:25:02.578 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-03-04T13:25:05.250 INFO:teuthology.orchestra.run.smithi107.stdout:(1/24): ceph-common-17.2.7-545.g42608bb1.el9.x8 9.0 MB/s | 23 MB 00:02 2024-03-04T13:25:05.300 INFO:teuthology.orchestra.run.smithi107.stdout:(2/24): ceph-selinux-17.2.7-545.g42608bb1.el9.x 514 kB/s | 25 kB 00:00 2024-03-04T13:25:05.375 INFO:teuthology.orchestra.run.smithi107.stdout:(3/24): libcephfs2-17.2.7-545.g42608bb1.el9.x86 8.4 MB/s | 635 kB 00:00 2024-03-04T13:25:05.433 INFO:teuthology.orchestra.run.smithi107.stdout:(4/24): libradosstriper1-17.2.7-545.g42608bb1.e 7.7 MB/s | 457 kB 00:00 2024-03-04T13:25:05.709 INFO:teuthology.orchestra.run.smithi107.stdout:(5/24): librgw2-17.2.7-545.g42608bb1.el9.x86_64 18 MB/s | 4.9 MB 00:00 2024-03-04T13:25:05.759 INFO:teuthology.orchestra.run.smithi107.stdout:(6/24): python3-ceph-argparse-17.2.7-545.g42608 918 kB/s | 45 kB 00:00 2024-03-04T13:25:05.810 INFO:teuthology.orchestra.run.smithi107.stdout:(7/24): python3-ceph-common-17.2.7-545.g42608bb 2.3 MB/s | 116 kB 00:00 2024-03-04T13:25:05.860 INFO:teuthology.orchestra.run.smithi107.stdout:(8/24): python3-cephfs-17.2.7-545.g42608bb1.el9 3.1 MB/s | 156 kB 00:00 2024-03-04T13:25:05.918 INFO:teuthology.orchestra.run.smithi107.stdout:(9/24): python3-rados-17.2.7-545.g42608bb1.el9. 5.4 MB/s | 320 kB 00:00 2024-03-04T13:25:05.977 INFO:teuthology.orchestra.run.smithi107.stdout:(10/24): python3-rbd-17.2.7-545.g42608bb1.el9.x 4.9 MB/s | 292 kB 00:00 2024-03-04T13:25:06.036 INFO:teuthology.orchestra.run.smithi107.stdout:(11/24): python3-rgw-17.2.7-545.g42608bb1.el9.x 1.7 MB/s | 99 kB 00:00 2024-03-04T13:25:06.394 INFO:teuthology.orchestra.run.smithi107.stdout:(12/24): libpmemobj-1.12.1-1.el9.x86_64.rpm 445 kB/s | 159 kB 00:00 2024-03-04T13:25:06.470 INFO:teuthology.orchestra.run.smithi107.stdout:(13/24): librabbitmq-0.11.0-7.el9.x86_64.rpm 594 kB/s | 44 kB 00:00 2024-03-04T13:25:06.645 INFO:teuthology.orchestra.run.smithi107.stdout:(14/24): librdkafka-1.6.1-102.el9.x86_64.rpm 3.7 MB/s | 661 kB 00:00 2024-03-04T13:25:06.737 INFO:teuthology.orchestra.run.smithi107.stdout:(15/24): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.0 MB/s | 282 kB 00:00 2024-03-04T13:25:06.812 INFO:teuthology.orchestra.run.smithi107.stdout:(16/24): python3-prettytable-0.7.2-27.el9.noarc 555 kB/s | 41 kB 00:00 2024-03-04T13:25:07.021 INFO:teuthology.orchestra.run.smithi107.stdout:(17/24): gperftools-libs-2.9.1-2.el9.x86_64.rpm 1.5 MB/s | 309 kB 00:00 2024-03-04T13:25:07.071 INFO:teuthology.orchestra.run.smithi107.stdout:(18/24): liboath-2.6.7-2.el9.x86_64.rpm 987 kB/s | 49 kB 00:00 2024-03-04T13:25:07.121 INFO:teuthology.orchestra.run.smithi107.stdout:(19/24): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-03-04T13:25:07.262 INFO:teuthology.orchestra.run.smithi107.stdout:(20/24): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-03-04T13:25:07.513 INFO:teuthology.orchestra.run.smithi107.stdout:(21/24): librados2-17.2.7-545.g42608bb1.el9.x86 12 MB/s | 3.1 MB 00:00 2024-03-04T13:25:07.714 INFO:teuthology.orchestra.run.smithi107.stdout:(22/24): librbd1-17.2.7-545.g42608bb1.el9.x86_6 15 MB/s | 2.9 MB 00:00 2024-03-04T13:25:13.384 INFO:teuthology.orchestra.run.smithi169.stdout:(24/24): ceph-radosgw-17.2.7-545.g42608bb1.el9. 353 kB/s | 14 MB 00:40 2024-03-04T13:25:13.390 INFO:teuthology.orchestra.run.smithi169.stdout:-------------------------------------------------------------------------------- 2024-03-04T13:25:13.391 INFO:teuthology.orchestra.run.smithi169.stdout:Total 1.4 MB/s | 58 MB 00:40 2024-03-04T13:25:13.392 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction check 2024-03-04T13:25:13.464 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction check succeeded. 2024-03-04T13:25:13.464 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction test 2024-03-04T13:25:13.986 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction test succeeded. 2024-03-04T13:25:13.986 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction 2024-03-04T13:25:14.054 INFO:teuthology.orchestra.run.smithi107.stdout:(23/24): ceph-base-17.2.7-545.g42608bb1.el9.x86 456 kB/s | 5.0 MB 00:11 2024-03-04T13:25:15.371 INFO:teuthology.orchestra.run.smithi169.stdout: Preparing : 1/1 2024-03-04T13:25:15.828 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/26 2024-03-04T13:25:15.886 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : librados2-2:17.2.7-545.g42608bb1.el9.x86_64 2/26 2024-03-04T13:25:16.051 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: librados2-2:17.2.7-545.g42608bb1.el9.x86_64 2/26 2024-03-04T13:25:16.147 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-rados-2:17.2.7-545.g42608bb1.el9.x86_64 3/26 2024-03-04T13:25:16.179 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libcephfs2-2:17.2.7-545.g42608bb1.el9.x86_64 4/26 2024-03-04T13:25:16.307 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: libcephfs2-2:17.2.7-545.g42608bb1.el9.x86_64 4/26 2024-03-04T13:25:16.368 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : thrift-0.15.0-2.el9.x86_64 5/26 2024-03-04T13:25:16.403 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/26 2024-03-04T13:25:16.937 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/26 2024-03-04T13:25:16.982 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : librgw2-2:17.2.7-545.g42608bb1.el9.x86_64 8/26 2024-03-04T13:25:17.065 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: librgw2-2:17.2.7-545.g42608bb1.el9.x86_64 8/26 2024-03-04T13:25:17.105 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : liboath-2.6.7-2.el9.x86_64 9/26 2024-03-04T13:25:17.170 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-ceph-argparse-2:17.2.7-545.g42608bb1.el9.x 10/26 2024-03-04T13:25:17.227 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cephfs-2:17.2.7-545.g42608bb1.el9.x86_64 11/26 2024-03-04T13:25:17.326 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-rgw-2:17.2.7-545.g42608bb1.el9.x86_64 12/26 2024-03-04T13:25:17.352 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libradosstriper1-2:17.2.7-545.g42608bb1.el9.x86_64 13/26 2024-03-04T13:25:17.461 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: libradosstriper1-2:17.2.7-545.g42608bb1.el9.x86_64 13/26 2024-03-04T13:25:17.525 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/26 2024-03-04T13:25:17.570 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/26 2024-03-04T13:25:17.611 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 16/26 2024-03-04T13:25:17.988 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/26 2024-03-04T13:25:18.014 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : librbd1-2:17.2.7-545.g42608bb1.el9.x86_64 18/26 2024-03-04T13:25:18.131 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: librbd1-2:17.2.7-545.g42608bb1.el9.x86_64 18/26 2024-03-04T13:25:18.220 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-rbd-2:17.2.7-545.g42608bb1.el9.x86_64 19/26 2024-03-04T13:25:18.250 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-ceph-common-2:17.2.7-545.g42608bb1.el9.x86 20/26 2024-03-04T13:25:21.049 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-common-2:17.2.7-545.g42608bb1.el9.x86_64 21/26 2024-03-04T13:25:21.092 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-common-2:17.2.7-545.g42608bb1.el9.x86_64 21/26 2024-03-04T13:25:21.694 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-common-2:17.2.7-545.g42608bb1.el9.x86_64 21/26 2024-03-04T13:25:21.742 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-base-2:17.2.7-545.g42608bb1.el9.x86_64 22/26 2024-03-04T13:25:21.894 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-base-2:17.2.7-545.g42608bb1.el9.x86_64 22/26 2024-03-04T13:25:21.894 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-04T13:25:21.894 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-04T13:25:21.894 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-03-04T13:25:21.935 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-selinux-2:17.2.7-545.g42608bb1.el9.x86_64 23/26 2024-03-04T13:25:30.095 INFO:teuthology.orchestra.run.smithi107.stdout:(24/24): ceph-radosgw-17.2.7-545.g42608bb1.el9. 527 kB/s | 14 MB 00:27 2024-03-04T13:25:30.100 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-03-04T13:25:30.101 INFO:teuthology.orchestra.run.smithi107.stdout:Total 2.1 MB/s | 58 MB 00:27 2024-03-04T13:25:30.102 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-03-04T13:25:30.172 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-03-04T13:25:30.172 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-03-04T13:25:30.708 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-03-04T13:25:30.708 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-03-04T13:25:32.103 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-03-04T13:25:32.510 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/26 2024-03-04T13:25:32.543 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-selinux-2:17.2.7-545.g42608bb1.el9.x86_64 23/26 2024-03-04T13:25:32.544 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /sys 2024-03-04T13:25:32.544 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /proc 2024-03-04T13:25:32.544 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /mnt 2024-03-04T13:25:32.544 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /var/tmp 2024-03-04T13:25:32.544 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /home 2024-03-04T13:25:32.545 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /root 2024-03-04T13:25:32.545 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /tmp 2024-03-04T13:25:32.545 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-03-04T13:25:32.553 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : librados2-2:17.2.7-545.g42608bb1.el9.x86_64 2/26 2024-03-04T13:25:32.713 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librados2-2:17.2.7-545.g42608bb1.el9.x86_64 2/26 2024-03-04T13:25:32.824 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-rados-2:17.2.7-545.g42608bb1.el9.x86_64 3/26 2024-03-04T13:25:32.855 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libcephfs2-2:17.2.7-545.g42608bb1.el9.x86_64 4/26 2024-03-04T13:25:32.981 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: libcephfs2-2:17.2.7-545.g42608bb1.el9.x86_64 4/26 2024-03-04T13:25:33.054 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : thrift-0.15.0-2.el9.x86_64 5/26 2024-03-04T13:25:33.104 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/26 2024-03-04T13:25:33.643 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/26 2024-03-04T13:25:33.674 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : librgw2-2:17.2.7-545.g42608bb1.el9.x86_64 8/26 2024-03-04T13:25:33.756 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librgw2-2:17.2.7-545.g42608bb1.el9.x86_64 8/26 2024-03-04T13:25:33.792 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : liboath-2.6.7-2.el9.x86_64 9/26 2024-03-04T13:25:33.854 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-ceph-argparse-2:17.2.7-545.g42608bb1.el9.x 10/26 2024-03-04T13:25:33.927 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cephfs-2:17.2.7-545.g42608bb1.el9.x86_64 11/26 2024-03-04T13:25:33.982 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-radosgw-2:17.2.7-545.g42608bb1.el9.x86_64 24/26 2024-03-04T13:25:34.037 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-rgw-2:17.2.7-545.g42608bb1.el9.x86_64 12/26 2024-03-04T13:25:34.060 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libradosstriper1-2:17.2.7-545.g42608bb1.el9.x86_64 13/26 2024-03-04T13:25:34.112 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-545.g42608bb1.el9.x86_64 24/26 2024-03-04T13:25:34.113 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-04T13:25:34.113 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-04T13:25:34.113 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-04T13:25:34.113 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-04T13:25:34.113 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-03-04T13:25:34.117 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-03-04T13:25:34.152 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: libradosstriper1-2:17.2.7-545.g42608bb1.el9.x86_64 13/26 2024-03-04T13:25:34.220 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/26 2024-03-04T13:25:34.259 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-03-04T13:25:34.259 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 26/26 2024-03-04T13:25:34.300 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/26 2024-03-04T13:25:34.348 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 16/26 2024-03-04T13:25:34.736 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/26 2024-03-04T13:25:34.772 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : librbd1-2:17.2.7-545.g42608bb1.el9.x86_64 18/26 2024-03-04T13:25:34.890 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librbd1-2:17.2.7-545.g42608bb1.el9.x86_64 18/26 2024-03-04T13:25:34.999 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-rbd-2:17.2.7-545.g42608bb1.el9.x86_64 19/26 2024-03-04T13:25:35.033 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-ceph-common-2:17.2.7-545.g42608bb1.el9.x86 20/26 2024-03-04T13:25:35.983 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 26/26 2024-03-04T13:25:35.983 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-base-2:17.2.7-545.g42608bb1.el9.x86_64 1/26 2024-03-04T13:25:35.983 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-common-2:17.2.7-545.g42608bb1.el9.x86_64 2/26 2024-03-04T13:25:35.983 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-radosgw-2:17.2.7-545.g42608bb1.el9.x86_64 3/26 2024-03-04T13:25:35.983 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-selinux-2:17.2.7-545.g42608bb1.el9.x86_64 4/26 2024-03-04T13:25:35.983 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libcephfs2-2:17.2.7-545.g42608bb1.el9.x86_64 5/26 2024-03-04T13:25:35.983 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libradosstriper1-2:17.2.7-545.g42608bb1.el9.x86_64 6/26 2024-03-04T13:25:35.983 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librgw2-2:17.2.7-545.g42608bb1.el9.x86_64 7/26 2024-03-04T13:25:35.983 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-ceph-argparse-2:17.2.7-545.g42608bb1.el9.x 8/26 2024-03-04T13:25:35.983 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-ceph-common-2:17.2.7-545.g42608bb1.el9.x86 9/26 2024-03-04T13:25:35.983 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cephfs-2:17.2.7-545.g42608bb1.el9.x86_64 10/26 2024-03-04T13:25:35.984 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-rados-2:17.2.7-545.g42608bb1.el9.x86_64 11/26 2024-03-04T13:25:35.984 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-rbd-2:17.2.7-545.g42608bb1.el9.x86_64 12/26 2024-03-04T13:25:35.984 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-rgw-2:17.2.7-545.g42608bb1.el9.x86_64 13/26 2024-03-04T13:25:35.984 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 14/26 2024-03-04T13:25:35.984 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 15/26 2024-03-04T13:25:35.984 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 16/26 2024-03-04T13:25:35.984 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 17/26 2024-03-04T13:25:35.984 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 18/26 2024-03-04T13:25:35.984 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 19/26 2024-03-04T13:25:35.984 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 20/26 2024-03-04T13:25:35.984 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 21/26 2024-03-04T13:25:35.984 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 22/26 2024-03-04T13:25:35.984 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librados2-2:17.2.7-545.g42608bb1.el9.x86_64 23/26 2024-03-04T13:25:35.985 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 24/26 2024-03-04T13:25:35.985 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librbd1-2:17.2.7-545.g42608bb1.el9.x86_64 25/26 2024-03-04T13:25:36.740 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 26/26 2024-03-04T13:25:36.741 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-03-04T13:25:36.741 INFO:teuthology.orchestra.run.smithi169.stdout:Upgraded: 2024-03-04T13:25:36.741 INFO:teuthology.orchestra.run.smithi169.stdout: librados2-2:17.2.7-545.g42608bb1.el9.x86_64 2024-03-04T13:25:36.741 INFO:teuthology.orchestra.run.smithi169.stdout: librbd1-2:17.2.7-545.g42608bb1.el9.x86_64 2024-03-04T13:25:36.741 INFO:teuthology.orchestra.run.smithi169.stdout:Installed: 2024-03-04T13:25:36.741 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-base-2:17.2.7-545.g42608bb1.el9.x86_64 2024-03-04T13:25:36.741 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-common-2:17.2.7-545.g42608bb1.el9.x86_64 2024-03-04T13:25:36.741 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-radosgw-2:17.2.7-545.g42608bb1.el9.x86_64 2024-03-04T13:25:36.741 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-selinux-2:17.2.7-545.g42608bb1.el9.x86_64 2024-03-04T13:25:36.741 INFO:teuthology.orchestra.run.smithi169.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-04T13:25:36.741 INFO:teuthology.orchestra.run.smithi169.stdout: libcephfs2-2:17.2.7-545.g42608bb1.el9.x86_64 2024-03-04T13:25:36.741 INFO:teuthology.orchestra.run.smithi169.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-04T13:25:36.742 INFO:teuthology.orchestra.run.smithi169.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-04T13:25:36.742 INFO:teuthology.orchestra.run.smithi169.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-04T13:25:36.742 INFO:teuthology.orchestra.run.smithi169.stdout: libradosstriper1-2:17.2.7-545.g42608bb1.el9.x86_64 2024-03-04T13:25:36.742 INFO:teuthology.orchestra.run.smithi169.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-04T13:25:36.742 INFO:teuthology.orchestra.run.smithi169.stdout: librgw2-2:17.2.7-545.g42608bb1.el9.x86_64 2024-03-04T13:25:36.742 INFO:teuthology.orchestra.run.smithi169.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-04T13:25:36.742 INFO:teuthology.orchestra.run.smithi169.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-04T13:25:36.742 INFO:teuthology.orchestra.run.smithi169.stdout: python3-ceph-argparse-2:17.2.7-545.g42608bb1.el9.x86_64 2024-03-04T13:25:36.742 INFO:teuthology.orchestra.run.smithi169.stdout: python3-ceph-common-2:17.2.7-545.g42608bb1.el9.x86_64 2024-03-04T13:25:36.742 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cephfs-2:17.2.7-545.g42608bb1.el9.x86_64 2024-03-04T13:25:36.742 INFO:teuthology.orchestra.run.smithi169.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-04T13:25:36.742 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rados-2:17.2.7-545.g42608bb1.el9.x86_64 2024-03-04T13:25:36.742 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rbd-2:17.2.7-545.g42608bb1.el9.x86_64 2024-03-04T13:25:36.742 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rgw-2:17.2.7-545.g42608bb1.el9.x86_64 2024-03-04T13:25:36.743 INFO:teuthology.orchestra.run.smithi169.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-04T13:25:36.743 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-03-04T13:25:36.743 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2024-03-04T13:25:37.185 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install ceph-test 2024-03-04T13:25:37.885 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-common-2:17.2.7-545.g42608bb1.el9.x86_64 21/26 2024-03-04T13:25:37.930 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-common-2:17.2.7-545.g42608bb1.el9.x86_64 21/26 2024-03-04T13:25:38.020 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:01:09 ago on Mon 04 Mar 2024 01:24:29 PM UTC. 2024-03-04T13:25:38.195 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2024-03-04T13:25:38.196 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-03-04T13:25:38.196 INFO:teuthology.orchestra.run.smithi169.stdout: Package Arch Version Repository Size 2024-03-04T13:25:38.196 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-03-04T13:25:38.196 INFO:teuthology.orchestra.run.smithi169.stdout:Installing: 2024-03-04T13:25:38.196 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-test x86_64 2:17.2.7-545.g42608bb1.el9 ceph 48 M 2024-03-04T13:25:38.196 INFO:teuthology.orchestra.run.smithi169.stdout:Installing dependencies: 2024-03-04T13:25:38.196 INFO:teuthology.orchestra.run.smithi169.stdout: jq x86_64 1.6-16.el9 CentOS-BaseOS 186 k 2024-03-04T13:25:38.196 INFO:teuthology.orchestra.run.smithi169.stdout: libcephsqlite x86_64 2:17.2.7-545.g42608bb1.el9 ceph 162 k 2024-03-04T13:25:38.197 INFO:teuthology.orchestra.run.smithi169.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-BaseOS 217 k 2024-03-04T13:25:38.197 INFO:teuthology.orchestra.run.smithi169.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-04T13:25:38.197 INFO:teuthology.orchestra.run.smithi169.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-04T13:25:38.197 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-03-04T13:25:38.197 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction Summary 2024-03-04T13:25:38.197 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-03-04T13:25:38.197 INFO:teuthology.orchestra.run.smithi169.stdout:Install 6 Packages 2024-03-04T13:25:38.197 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-03-04T13:25:38.198 INFO:teuthology.orchestra.run.smithi169.stdout:Total download size: 49 M 2024-03-04T13:25:38.198 INFO:teuthology.orchestra.run.smithi169.stdout:Installed size: 196 M 2024-03-04T13:25:38.198 INFO:teuthology.orchestra.run.smithi169.stdout:Downloading Packages: 2024-03-04T13:25:38.534 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-common-2:17.2.7-545.g42608bb1.el9.x86_64 21/26 2024-03-04T13:25:38.536 INFO:teuthology.orchestra.run.smithi169.stdout:(1/6): libcephsqlite-17.2.7-545.g42608bb1.el9.x 482 kB/s | 162 kB 00:00 2024-03-04T13:25:38.583 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-base-2:17.2.7-545.g42608bb1.el9.x86_64 22/26 2024-03-04T13:25:38.627 INFO:teuthology.orchestra.run.smithi169.stdout:(2/6): socat-1.7.4.1-5.el9.x86_64.rpm 700 kB/s | 300 kB 00:00 2024-03-04T13:25:38.653 INFO:teuthology.orchestra.run.smithi169.stdout:(3/6): xmlstarlet-1.6.1-20.el9.x86_64.rpm 545 kB/s | 63 kB 00:00 2024-03-04T13:25:38.772 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-base-2:17.2.7-545.g42608bb1.el9.x86_64 22/26 2024-03-04T13:25:38.772 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-04T13:25:38.772 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-04T13:25:38.773 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:25:38.811 INFO:teuthology.orchestra.run.smithi169.stdout:(4/6): jq-1.6-16.el9.x86_64.rpm 1.0 MB/s | 186 kB 00:00 2024-03-04T13:25:38.825 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-selinux-2:17.2.7-545.g42608bb1.el9.x86_64 23/26 2024-03-04T13:25:38.845 INFO:teuthology.orchestra.run.smithi169.stdout:(5/6): oniguruma-6.9.6-1.el9.5.x86_64.rpm 1.1 MB/s | 217 kB 00:00 2024-03-04T13:25:42.270 INFO:teuthology.orchestra.run.smithi169.stdout:(6/6): ceph-test-17.2.7-545.g42608bb1.el9.x86_6 12 MB/s | 48 MB 00:04 2024-03-04T13:25:42.276 INFO:teuthology.orchestra.run.smithi169.stdout:-------------------------------------------------------------------------------- 2024-03-04T13:25:42.276 INFO:teuthology.orchestra.run.smithi169.stdout:Total 12 MB/s | 49 MB 00:04 2024-03-04T13:25:42.278 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction check 2024-03-04T13:25:42.328 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction check succeeded. 2024-03-04T13:25:42.329 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction test 2024-03-04T13:25:42.796 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction test succeeded. 2024-03-04T13:25:42.797 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction 2024-03-04T13:25:43.495 INFO:teuthology.orchestra.run.smithi169.stdout: Preparing : 1/1 2024-03-04T13:25:43.587 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 1/6 2024-03-04T13:25:43.632 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : jq-1.6-16.el9.x86_64 2/6 2024-03-04T13:25:43.707 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 3/6 2024-03-04T13:25:43.767 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 4/6 2024-03-04T13:25:43.798 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libcephsqlite-2:17.2.7-545.g42608bb1.el9.x86_64 5/6 2024-03-04T13:25:49.070 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: libcephsqlite-2:17.2.7-545.g42608bb1.el9.x86_64 5/6 2024-03-04T13:25:49.195 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-test-2:17.2.7-545.g42608bb1.el9.x86_64 6/6 2024-03-04T13:25:49.313 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-selinux-2:17.2.7-545.g42608bb1.el9.x86_64 23/26 2024-03-04T13:25:49.313 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /sys 2024-03-04T13:25:49.313 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /proc 2024-03-04T13:25:49.313 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /mnt 2024-03-04T13:25:49.313 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /var/tmp 2024-03-04T13:25:49.313 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /home 2024-03-04T13:25:49.314 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /root 2024-03-04T13:25:49.314 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /tmp 2024-03-04T13:25:49.314 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:25:50.706 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-radosgw-2:17.2.7-545.g42608bb1.el9.x86_64 24/26 2024-03-04T13:25:50.865 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-545.g42608bb1.el9.x86_64 24/26 2024-03-04T13:25:50.865 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-04T13:25:50.865 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-04T13:25:50.866 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-04T13:25:50.866 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-04T13:25:50.866 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:25:50.870 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-03-04T13:25:50.977 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-03-04T13:25:50.977 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 26/26 2024-03-04T13:25:51.432 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-test-2:17.2.7-545.g42608bb1.el9.x86_64 6/6 2024-03-04T13:25:51.432 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-test-2:17.2.7-545.g42608bb1.el9.x86_64 1/6 2024-03-04T13:25:51.432 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libcephsqlite-2:17.2.7-545.g42608bb1.el9.x86_64 2/6 2024-03-04T13:25:51.432 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 3/6 2024-03-04T13:25:51.432 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/6 2024-03-04T13:25:51.433 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : jq-1.6-16.el9.x86_64 5/6 2024-03-04T13:25:51.759 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 6/6 2024-03-04T13:25:51.759 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-03-04T13:25:51.760 INFO:teuthology.orchestra.run.smithi169.stdout:Installed: 2024-03-04T13:25:51.760 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-test-2:17.2.7-545.g42608bb1.el9.x86_64 2024-03-04T13:25:51.760 INFO:teuthology.orchestra.run.smithi169.stdout: jq-1.6-16.el9.x86_64 2024-03-04T13:25:51.760 INFO:teuthology.orchestra.run.smithi169.stdout: libcephsqlite-2:17.2.7-545.g42608bb1.el9.x86_64 2024-03-04T13:25:51.760 INFO:teuthology.orchestra.run.smithi169.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-04T13:25:51.760 INFO:teuthology.orchestra.run.smithi169.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-04T13:25:51.760 INFO:teuthology.orchestra.run.smithi169.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-04T13:25:51.760 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-03-04T13:25:51.760 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2024-03-04T13:25:52.085 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install ceph 2024-03-04T13:25:52.614 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 26/26 2024-03-04T13:25:52.614 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-base-2:17.2.7-545.g42608bb1.el9.x86_64 1/26 2024-03-04T13:25:52.614 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-common-2:17.2.7-545.g42608bb1.el9.x86_64 2/26 2024-03-04T13:25:52.614 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-radosgw-2:17.2.7-545.g42608bb1.el9.x86_64 3/26 2024-03-04T13:25:52.615 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-selinux-2:17.2.7-545.g42608bb1.el9.x86_64 4/26 2024-03-04T13:25:52.615 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libcephfs2-2:17.2.7-545.g42608bb1.el9.x86_64 5/26 2024-03-04T13:25:52.615 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libradosstriper1-2:17.2.7-545.g42608bb1.el9.x86_64 6/26 2024-03-04T13:25:52.615 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librgw2-2:17.2.7-545.g42608bb1.el9.x86_64 7/26 2024-03-04T13:25:52.615 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-ceph-argparse-2:17.2.7-545.g42608bb1.el9.x 8/26 2024-03-04T13:25:52.615 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-ceph-common-2:17.2.7-545.g42608bb1.el9.x86 9/26 2024-03-04T13:25:52.615 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cephfs-2:17.2.7-545.g42608bb1.el9.x86_64 10/26 2024-03-04T13:25:52.615 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-rados-2:17.2.7-545.g42608bb1.el9.x86_64 11/26 2024-03-04T13:25:52.615 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-rbd-2:17.2.7-545.g42608bb1.el9.x86_64 12/26 2024-03-04T13:25:52.615 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-rgw-2:17.2.7-545.g42608bb1.el9.x86_64 13/26 2024-03-04T13:25:52.615 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 14/26 2024-03-04T13:25:52.615 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 15/26 2024-03-04T13:25:52.615 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 16/26 2024-03-04T13:25:52.615 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 17/26 2024-03-04T13:25:52.616 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 18/26 2024-03-04T13:25:52.616 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 19/26 2024-03-04T13:25:52.616 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 20/26 2024-03-04T13:25:52.616 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 21/26 2024-03-04T13:25:52.616 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 22/26 2024-03-04T13:25:52.616 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librados2-2:17.2.7-545.g42608bb1.el9.x86_64 23/26 2024-03-04T13:25:52.616 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 24/26 2024-03-04T13:25:52.616 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librbd1-2:17.2.7-545.g42608bb1.el9.x86_64 25/26 2024-03-04T13:25:52.881 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:01:23 ago on Mon 04 Mar 2024 01:24:29 PM UTC. 2024-03-04T13:25:53.061 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2024-03-04T13:25:53.065 INFO:teuthology.orchestra.run.smithi169.stdout:======================================================================================== 2024-03-04T13:25:53.065 INFO:teuthology.orchestra.run.smithi169.stdout: Package Arch Version Repository Size 2024-03-04T13:25:53.065 INFO:teuthology.orchestra.run.smithi169.stdout:======================================================================================== 2024-03-04T13:25:53.065 INFO:teuthology.orchestra.run.smithi169.stdout:Installing: 2024-03-04T13:25:53.065 INFO:teuthology.orchestra.run.smithi169.stdout: ceph x86_64 2:17.2.7-545.g42608bb1.el9 ceph 6.5 k 2024-03-04T13:25:53.065 INFO:teuthology.orchestra.run.smithi169.stdout:Installing dependencies: 2024-03-04T13:25:53.065 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mds x86_64 2:17.2.7-545.g42608bb1.el9 ceph 2.1 M 2024-03-04T13:25:53.065 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr x86_64 2:17.2.7-545.g42608bb1.el9 ceph 1.4 M 2024-03-04T13:25:53.065 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-modules-core noarch 2:17.2.7-545.g42608bb1.el9 ceph-noarch 239 k 2024-03-04T13:25:53.065 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mon x86_64 2:17.2.7-545.g42608bb1.el9 ceph 3.7 M 2024-03-04T13:25:53.065 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-osd x86_64 2:17.2.7-545.g42608bb1.el9 ceph 15 M 2024-03-04T13:25:53.066 INFO:teuthology.orchestra.run.smithi169.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-04T13:25:53.066 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-04T13:25:53.066 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-04T13:25:53.066 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-04T13:25:53.066 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-04T13:25:53.066 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-04T13:25:53.066 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-04T13:25:53.066 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-04T13:25:53.066 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-04T13:25:53.066 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-04T13:25:53.066 INFO:teuthology.orchestra.run.smithi169.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-04T13:25:53.066 INFO:teuthology.orchestra.run.smithi169.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-04T13:25:53.066 INFO:teuthology.orchestra.run.smithi169.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-04T13:25:53.066 INFO:teuthology.orchestra.run.smithi169.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-04T13:25:53.066 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-04T13:25:53.067 INFO:teuthology.orchestra.run.smithi169.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-04T13:25:53.067 INFO:teuthology.orchestra.run.smithi169.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-04T13:25:53.067 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-04T13:25:53.067 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-04T13:25:53.067 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-04T13:25:53.067 INFO:teuthology.orchestra.run.smithi169.stdout: python3-requests noarch 2.25.1-8.el9 CentOS-BaseOS 113 k 2024-03-04T13:25:53.067 INFO:teuthology.orchestra.run.smithi169.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-04T13:25:53.067 INFO:teuthology.orchestra.run.smithi169.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-04T13:25:53.067 INFO:teuthology.orchestra.run.smithi169.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-04T13:25:53.067 INFO:teuthology.orchestra.run.smithi169.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-04T13:25:53.067 INFO:teuthology.orchestra.run.smithi169.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-04T13:25:53.067 INFO:teuthology.orchestra.run.smithi169.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-04T13:25:53.067 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-03-04T13:25:53.067 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction Summary 2024-03-04T13:25:53.068 INFO:teuthology.orchestra.run.smithi169.stdout:======================================================================================== 2024-03-04T13:25:53.068 INFO:teuthology.orchestra.run.smithi169.stdout:Install 33 Packages 2024-03-04T13:25:53.068 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-03-04T13:25:53.068 INFO:teuthology.orchestra.run.smithi169.stdout:Total download size: 27 M 2024-03-04T13:25:53.068 INFO:teuthology.orchestra.run.smithi169.stdout:Installed size: 99 M 2024-03-04T13:25:53.068 INFO:teuthology.orchestra.run.smithi169.stdout:Downloading Packages: 2024-03-04T13:25:53.573 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 26/26 2024-03-04T13:25:53.573 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:25:53.573 INFO:teuthology.orchestra.run.smithi107.stdout:Upgraded: 2024-03-04T13:25:53.573 INFO:teuthology.orchestra.run.smithi107.stdout: librados2-2:17.2.7-545.g42608bb1.el9.x86_64 2024-03-04T13:25:53.573 INFO:teuthology.orchestra.run.smithi107.stdout: librbd1-2:17.2.7-545.g42608bb1.el9.x86_64 2024-03-04T13:25:53.573 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-03-04T13:25:53.573 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-base-2:17.2.7-545.g42608bb1.el9.x86_64 2024-03-04T13:25:53.573 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-common-2:17.2.7-545.g42608bb1.el9.x86_64 2024-03-04T13:25:53.573 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-radosgw-2:17.2.7-545.g42608bb1.el9.x86_64 2024-03-04T13:25:53.574 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-selinux-2:17.2.7-545.g42608bb1.el9.x86_64 2024-03-04T13:25:53.574 INFO:teuthology.orchestra.run.smithi107.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-04T13:25:53.574 INFO:teuthology.orchestra.run.smithi107.stdout: libcephfs2-2:17.2.7-545.g42608bb1.el9.x86_64 2024-03-04T13:25:53.574 INFO:teuthology.orchestra.run.smithi107.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-04T13:25:53.574 INFO:teuthology.orchestra.run.smithi107.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-04T13:25:53.574 INFO:teuthology.orchestra.run.smithi107.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-04T13:25:53.574 INFO:teuthology.orchestra.run.smithi107.stdout: libradosstriper1-2:17.2.7-545.g42608bb1.el9.x86_64 2024-03-04T13:25:53.574 INFO:teuthology.orchestra.run.smithi107.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-04T13:25:53.574 INFO:teuthology.orchestra.run.smithi107.stdout: librgw2-2:17.2.7-545.g42608bb1.el9.x86_64 2024-03-04T13:25:53.574 INFO:teuthology.orchestra.run.smithi107.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-04T13:25:53.574 INFO:teuthology.orchestra.run.smithi107.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-04T13:25:53.574 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ceph-argparse-2:17.2.7-545.g42608bb1.el9.x86_64 2024-03-04T13:25:53.574 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ceph-common-2:17.2.7-545.g42608bb1.el9.x86_64 2024-03-04T13:25:53.574 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cephfs-2:17.2.7-545.g42608bb1.el9.x86_64 2024-03-04T13:25:53.575 INFO:teuthology.orchestra.run.smithi107.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-04T13:25:53.575 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rados-2:17.2.7-545.g42608bb1.el9.x86_64 2024-03-04T13:25:53.575 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rbd-2:17.2.7-545.g42608bb1.el9.x86_64 2024-03-04T13:25:53.575 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rgw-2:17.2.7-545.g42608bb1.el9.x86_64 2024-03-04T13:25:53.575 INFO:teuthology.orchestra.run.smithi107.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-04T13:25:53.575 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:25:53.575 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-03-04T13:25:54.084 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install ceph-test 2024-03-04T13:25:54.519 INFO:teuthology.orchestra.run.smithi169.stdout:(1/33): ceph-17.2.7-545.g42608bb1.el9.x86_64.rp 14 kB/s | 6.5 kB 00:00 2024-03-04T13:25:54.589 INFO:teuthology.orchestra.run.smithi169.stdout:(2/33): ceph-mds-17.2.7-545.g42608bb1.el9.x86_6 3.8 MB/s | 2.1 MB 00:00 2024-03-04T13:25:54.912 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:00:55 ago on Mon 04 Mar 2024 01:24:59 PM UTC. 2024-03-04T13:25:55.086 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-03-04T13:25:55.087 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-03-04T13:25:55.087 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repository Size 2024-03-04T13:25:55.087 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-03-04T13:25:55.088 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-03-04T13:25:55.088 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-test x86_64 2:17.2.7-545.g42608bb1.el9 ceph 48 M 2024-03-04T13:25:55.088 INFO:teuthology.orchestra.run.smithi107.stdout:Installing dependencies: 2024-03-04T13:25:55.088 INFO:teuthology.orchestra.run.smithi107.stdout: jq x86_64 1.6-16.el9 CentOS-BaseOS 186 k 2024-03-04T13:25:55.088 INFO:teuthology.orchestra.run.smithi107.stdout: libcephsqlite x86_64 2:17.2.7-545.g42608bb1.el9 ceph 162 k 2024-03-04T13:25:55.088 INFO:teuthology.orchestra.run.smithi107.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-BaseOS 217 k 2024-03-04T13:25:55.088 INFO:teuthology.orchestra.run.smithi107.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-04T13:25:55.088 INFO:teuthology.orchestra.run.smithi107.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-04T13:25:55.088 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:25:55.088 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-03-04T13:25:55.088 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-03-04T13:25:55.088 INFO:teuthology.orchestra.run.smithi107.stdout:Install 6 Packages 2024-03-04T13:25:55.089 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:25:55.089 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 49 M 2024-03-04T13:25:55.089 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 196 M 2024-03-04T13:25:55.089 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-03-04T13:25:55.381 INFO:teuthology.orchestra.run.smithi169.stdout:(3/33): ceph-osd-17.2.7-545.g42608bb1.el9.x86_6 19 MB/s | 15 MB 00:00 2024-03-04T13:25:55.429 INFO:teuthology.orchestra.run.smithi107.stdout:(1/6): libcephsqlite-17.2.7-545.g42608bb1.el9.x 481 kB/s | 162 kB 00:00 2024-03-04T13:25:55.440 INFO:teuthology.orchestra.run.smithi169.stdout:(4/33): ceph-mgr-modules-core-17.2.7-545.g42608 4.1 MB/s | 239 kB 00:00 2024-03-04T13:25:55.836 INFO:teuthology.orchestra.run.smithi107.stdout:(2/6): socat-1.7.4.1-5.el9.x86_64.rpm 402 kB/s | 300 kB 00:00 2024-03-04T13:25:55.887 INFO:teuthology.orchestra.run.smithi107.stdout:(3/6): xmlstarlet-1.6.1-20.el9.x86_64.rpm 137 kB/s | 63 kB 00:00 2024-03-04T13:25:55.998 INFO:teuthology.orchestra.run.smithi169.stdout:(5/33): python3-mako-1.1.4-6.el9.noarch.rpm 259 kB/s | 144 kB 00:00 2024-03-04T13:25:56.053 INFO:teuthology.orchestra.run.smithi107.stdout:(4/6): jq-1.6-16.el9.x86_64.rpm 860 kB/s | 186 kB 00:00 2024-03-04T13:25:56.140 INFO:teuthology.orchestra.run.smithi169.stdout:(6/33): python3-markupsafe-1.1.1-12.el9.x86_64. 224 kB/s | 32 kB 00:00 2024-03-04T13:25:56.254 INFO:teuthology.orchestra.run.smithi107.stdout:(5/6): oniguruma-6.9.6-1.el9.5.x86_64.rpm 592 kB/s | 217 kB 00:00 2024-03-04T13:25:56.303 INFO:teuthology.orchestra.run.smithi169.stdout:(7/33): python3-pytz-2021.1-5.el9.noarch.rpm 299 kB/s | 47 kB 00:00 2024-03-04T13:25:56.466 INFO:teuthology.orchestra.run.smithi169.stdout:(8/33): python3-toml-0.10.2-6.el9.noarch.rpm 232 kB/s | 39 kB 00:00 2024-03-04T13:25:56.925 INFO:teuthology.orchestra.run.smithi169.stdout:(9/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 526 kB/s | 241 kB 00:00 2024-03-04T13:25:58.417 INFO:teuthology.orchestra.run.smithi169.stdout:(10/33): python3-cryptography-36.0.1-4.el9.x86_ 785 kB/s | 1.1 MB 00:01 2024-03-04T13:25:58.609 INFO:teuthology.orchestra.run.smithi169.stdout:(11/33): python3-ply-3.11-14.el9.noarch.rpm 539 kB/s | 103 kB 00:00 2024-03-04T13:25:59.603 INFO:teuthology.orchestra.run.smithi169.stdout:(12/33): python3-pycparser-2.20-6.el9.noarch.rp 598 kB/s | 124 kB 00:00 2024-03-04T13:25:59.603 INFO:teuthology.orchestra.run.smithi169.stdout:(13/33): python3-requests-2.25.1-8.el9.noarch.r 468 kB/s | 113 kB 00:00 2024-03-04T13:25:59.604 INFO:teuthology.orchestra.run.smithi169.stdout:(14/33): python3-urllib3-1.26.5-5.el9.noarch.rp 609 kB/s | 187 kB 00:00 2024-03-04T13:25:59.604 INFO:teuthology.orchestra.run.smithi169.stdout:(15/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 659 kB/s | 43 kB 00:00 2024-03-04T13:25:59.604 INFO:teuthology.orchestra.run.smithi169.stdout:(16/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-03-04T13:25:59.604 INFO:teuthology.orchestra.run.smithi169.stdout:(17/33): python3-cherrypy-18.6.1-2.el9.noarch.r 6.1 MB/s | 358 kB 00:00 2024-03-04T13:25:59.604 INFO:teuthology.orchestra.run.smithi169.stdout:(18/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 328 kB/s | 11 kB 00:00 2024-03-04T13:25:59.619 INFO:teuthology.orchestra.run.smithi169.stdout:(19/33): python3-jaraco-classes-3.2.1-5.el9.noa 541 kB/s | 18 kB 00:00 2024-03-04T13:25:59.657 INFO:teuthology.orchestra.run.smithi169.stdout:(20/33): python3-jaraco-collections-3.0.0-8.el9 707 kB/s | 23 kB 00:00 2024-03-04T13:25:59.686 INFO:teuthology.orchestra.run.smithi169.stdout:(21/33): python3-jaraco-functools-3.5.0-2.el9.n 594 kB/s | 19 kB 00:00 2024-03-04T13:25:59.720 INFO:teuthology.orchestra.run.smithi169.stdout:(22/33): python3-jaraco-text-3.2.0-6.el9.noarch 598 kB/s | 20 kB 00:00 2024-03-04T13:25:59.762 INFO:teuthology.orchestra.run.smithi169.stdout:(23/33): python3-logutils-0.3.5-21.el9.noarch.r 1.1 MB/s | 46 kB 00:00 2024-03-04T13:25:59.796 INFO:teuthology.orchestra.run.smithi169.stdout:(24/33): python3-more-itertools-8.12.0-2.el9.no 2.4 MB/s | 79 kB 00:00 2024-03-04T13:25:59.841 INFO:teuthology.orchestra.run.smithi169.stdout:(25/33): python3-pecan-1.4.2-3.el9.noarch.rpm 6.5 MB/s | 272 kB 00:00 2024-03-04T13:25:59.872 INFO:teuthology.orchestra.run.smithi169.stdout:(26/33): python3-portend-3.1.0-2.el9.noarch.rpm 506 kB/s | 16 kB 00:00 2024-03-04T13:25:59.913 INFO:teuthology.orchestra.run.smithi169.stdout:(27/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 2.1 MB/s | 90 kB 00:00 2024-03-04T13:25:59.974 INFO:teuthology.orchestra.run.smithi169.stdout:(28/33): python3-tempora-5.0.0-2.el9.noarch.rpm 619 kB/s | 36 kB 00:00 2024-03-04T13:26:00.014 INFO:teuthology.orchestra.run.smithi169.stdout:(29/33): python3-webob-1.8.7-6.el9.noarch.rpm 5.5 MB/s | 230 kB 00:00 2024-03-04T13:26:00.064 INFO:teuthology.orchestra.run.smithi169.stdout:(30/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 8.4 MB/s | 427 kB 00:00 2024-03-04T13:26:00.097 INFO:teuthology.orchestra.run.smithi169.stdout:(31/33): python3-zc-lockfile-2.0-10.el9.noarch. 616 kB/s | 20 kB 00:00 2024-03-04T13:26:01.922 INFO:teuthology.orchestra.run.smithi169.stdout:(32/33): ceph-mgr-17.2.7-545.g42608bb1.el9.x86_ 183 kB/s | 1.4 MB 00:07 2024-03-04T13:26:04.039 INFO:teuthology.orchestra.run.smithi169.stdout:(33/33): ceph-mon-17.2.7-545.g42608bb1.el9.x86_ 398 kB/s | 3.7 MB 00:09 2024-03-04T13:26:04.046 INFO:teuthology.orchestra.run.smithi169.stdout:-------------------------------------------------------------------------------- 2024-03-04T13:26:04.046 INFO:teuthology.orchestra.run.smithi169.stdout:Total 2.4 MB/s | 27 MB 00:10 2024-03-04T13:26:04.048 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction check 2024-03-04T13:26:04.131 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction check succeeded. 2024-03-04T13:26:04.131 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction test 2024-03-04T13:26:04.591 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction test succeeded. 2024-03-04T13:26:04.592 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction 2024-03-04T13:26:05.957 INFO:teuthology.orchestra.run.smithi169.stdout: Preparing : 1/1 2024-03-04T13:26:06.034 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-03-04T13:26:06.074 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-03-04T13:26:06.117 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-03-04T13:26:06.255 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-03-04T13:26:06.358 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-03-04T13:26:06.416 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/33 2024-03-04T13:26:06.511 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-03-04T13:26:06.577 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 8/33 2024-03-04T13:26:06.662 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 9/33 2024-03-04T13:26:06.733 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-ply-3.11-14.el9.noarch 10/33 2024-03-04T13:26:06.823 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 11/33 2024-03-04T13:26:07.079 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 12/33 2024-03-04T13:26:07.141 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 13/33 2024-03-04T13:26:07.186 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 14/33 2024-03-04T13:26:07.232 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 15/33 2024-03-04T13:26:07.270 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 16/33 2024-03-04T13:26:07.377 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 17/33 2024-03-04T13:26:07.411 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-03-04T13:26:07.446 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 19/33 2024-03-04T13:26:07.493 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 20/33 2024-03-04T13:26:07.557 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/33 2024-03-04T13:26:07.608 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 22/33 2024-03-04T13:26:07.761 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 23/33 2024-03-04T13:26:07.812 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 24/33 2024-03-04T13:26:07.924 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/33 2024-03-04T13:26:08.109 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 26/33 2024-03-04T13:26:08.262 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-03-04T13:26:08.441 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-545.g42608bb1.el9.n 28/33 2024-03-04T13:26:08.470 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mgr-2:17.2.7-545.g42608bb1.el9.x86_64 29/33 2024-03-04T13:26:08.544 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mgr-2:17.2.7-545.g42608bb1.el9.x86_64 29/33 2024-03-04T13:26:08.544 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-04T13:26:08.544 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-04T13:26:08.544 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-04T13:26:08.544 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-04T13:26:08.544 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-03-04T13:26:09.971 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-osd-2:17.2.7-545.g42608bb1.el9.x86_64 30/33 2024-03-04T13:26:10.050 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-osd-2:17.2.7-545.g42608bb1.el9.x86_64 30/33 2024-03-04T13:26:10.050 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-04T13:26:10.050 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-04T13:26:10.050 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-04T13:26:10.051 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-04T13:26:10.051 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-03-04T13:26:10.472 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mon-2:17.2.7-545.g42608bb1.el9.x86_64 31/33 2024-03-04T13:26:10.540 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mon-2:17.2.7-545.g42608bb1.el9.x86_64 31/33 2024-03-04T13:26:10.540 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-04T13:26:10.540 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-04T13:26:10.540 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-04T13:26:10.540 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-04T13:26:10.540 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-03-04T13:26:10.765 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mds-2:17.2.7-545.g42608bb1.el9.x86_64 32/33 2024-03-04T13:26:10.832 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mds-2:17.2.7-545.g42608bb1.el9.x86_64 32/33 2024-03-04T13:26:10.832 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-04T13:26:10.832 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-04T13:26:10.832 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-04T13:26:10.832 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-04T13:26:10.832 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-03-04T13:26:10.923 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-2:17.2.7-545.g42608bb1.el9.x86_64 33/33 2024-03-04T13:26:11.026 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-2:17.2.7-545.g42608bb1.el9.x86_64 33/33 2024-03-04T13:26:11.026 INFO:teuthology.orchestra.run.smithi169.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-03-04T13:26:11.026 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-03-04T13:26:12.907 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-2:17.2.7-545.g42608bb1.el9.x86_64 1/33 2024-03-04T13:26:12.907 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mds-2:17.2.7-545.g42608bb1.el9.x86_64 2/33 2024-03-04T13:26:12.907 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mgr-2:17.2.7-545.g42608bb1.el9.x86_64 3/33 2024-03-04T13:26:12.908 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mon-2:17.2.7-545.g42608bb1.el9.x86_64 4/33 2024-03-04T13:26:12.908 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-osd-2:17.2.7-545.g42608bb1.el9.x86_64 5/33 2024-03-04T13:26:12.908 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-545.g42608bb1.el9.n 6/33 2024-03-04T13:26:12.908 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 7/33 2024-03-04T13:26:12.908 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 8/33 2024-03-04T13:26:12.908 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 9/33 2024-03-04T13:26:12.908 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 10/33 2024-03-04T13:26:12.908 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 11/33 2024-03-04T13:26:12.908 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 12/33 2024-03-04T13:26:12.908 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-ply-3.11-14.el9.noarch 13/33 2024-03-04T13:26:12.909 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 14/33 2024-03-04T13:26:12.909 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 15/33 2024-03-04T13:26:12.909 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 16/33 2024-03-04T13:26:12.909 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-03-04T13:26:12.909 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-03-04T13:26:12.909 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-03-04T13:26:12.909 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-03-04T13:26:12.909 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-03-04T13:26:12.909 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-03-04T13:26:12.909 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-03-04T13:26:12.909 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-03-04T13:26:12.909 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-03-04T13:26:12.909 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-03-04T13:26:12.910 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-03-04T13:26:12.910 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-03-04T13:26:12.910 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-03-04T13:26:12.910 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-03-04T13:26:12.910 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 31/33 2024-03-04T13:26:12.910 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-03-04T13:26:14.003 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-03-04T13:26:14.004 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-03-04T13:26:14.004 INFO:teuthology.orchestra.run.smithi169.stdout:Installed: 2024-03-04T13:26:14.004 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-2:17.2.7-545.g42608bb1.el9.x86_64 2024-03-04T13:26:14.004 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mds-2:17.2.7-545.g42608bb1.el9.x86_64 2024-03-04T13:26:14.004 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-2:17.2.7-545.g42608bb1.el9.x86_64 2024-03-04T13:26:14.004 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-modules-core-2:17.2.7-545.g42608bb1.el9.noarch 2024-03-04T13:26:14.004 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mon-2:17.2.7-545.g42608bb1.el9.x86_64 2024-03-04T13:26:14.004 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-osd-2:17.2.7-545.g42608bb1.el9.x86_64 2024-03-04T13:26:14.004 INFO:teuthology.orchestra.run.smithi169.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-04T13:26:14.004 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-04T13:26:14.004 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-04T13:26:14.005 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-04T13:26:14.005 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-04T13:26:14.005 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-04T13:26:14.005 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-04T13:26:14.005 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-04T13:26:14.005 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-04T13:26:14.005 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-04T13:26:14.005 INFO:teuthology.orchestra.run.smithi169.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-04T13:26:14.005 INFO:teuthology.orchestra.run.smithi169.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-04T13:26:14.005 INFO:teuthology.orchestra.run.smithi169.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-04T13:26:14.005 INFO:teuthology.orchestra.run.smithi169.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-04T13:26:14.005 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-04T13:26:14.005 INFO:teuthology.orchestra.run.smithi169.stdout: python3-ply-3.11-14.el9.noarch 2024-03-04T13:26:14.005 INFO:teuthology.orchestra.run.smithi169.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-04T13:26:14.006 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-04T13:26:14.006 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-04T13:26:14.006 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-04T13:26:14.006 INFO:teuthology.orchestra.run.smithi169.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-04T13:26:14.006 INFO:teuthology.orchestra.run.smithi169.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-04T13:26:14.006 INFO:teuthology.orchestra.run.smithi169.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-04T13:26:14.006 INFO:teuthology.orchestra.run.smithi169.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-04T13:26:14.006 INFO:teuthology.orchestra.run.smithi169.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-04T13:26:14.006 INFO:teuthology.orchestra.run.smithi169.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-04T13:26:14.006 INFO:teuthology.orchestra.run.smithi169.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-04T13:26:14.006 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-03-04T13:26:14.006 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2024-03-04T13:26:14.382 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install ceph-base 2024-03-04T13:26:15.199 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:01:46 ago on Mon 04 Mar 2024 01:24:29 PM UTC. 2024-03-04T13:26:15.319 INFO:teuthology.orchestra.run.smithi169.stdout:Package ceph-base-2:17.2.7-545.g42608bb1.el9.x86_64 is already installed. 2024-03-04T13:26:15.375 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2024-03-04T13:26:15.376 INFO:teuthology.orchestra.run.smithi169.stdout:Nothing to do. 2024-03-04T13:26:15.376 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2024-03-04T13:26:15.447 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install cephadm 2024-03-04T13:26:16.241 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:01:47 ago on Mon 04 Mar 2024 01:24:29 PM UTC. 2024-03-04T13:26:16.417 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2024-03-04T13:26:16.418 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-03-04T13:26:16.418 INFO:teuthology.orchestra.run.smithi169.stdout: Package Arch Version Repository Size 2024-03-04T13:26:16.418 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-03-04T13:26:16.418 INFO:teuthology.orchestra.run.smithi169.stdout:Installing: 2024-03-04T13:26:16.418 INFO:teuthology.orchestra.run.smithi169.stdout: cephadm noarch 2:17.2.7-545.g42608bb1.el9 ceph-noarch 80 k 2024-03-04T13:26:16.418 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-03-04T13:26:16.419 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction Summary 2024-03-04T13:26:16.419 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-03-04T13:26:16.419 INFO:teuthology.orchestra.run.smithi169.stdout:Install 1 Package 2024-03-04T13:26:16.419 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-03-04T13:26:16.419 INFO:teuthology.orchestra.run.smithi169.stdout:Total download size: 80 k 2024-03-04T13:26:16.419 INFO:teuthology.orchestra.run.smithi169.stdout:Installed size: 364 k 2024-03-04T13:26:16.419 INFO:teuthology.orchestra.run.smithi169.stdout:Downloading Packages: 2024-03-04T13:26:18.796 INFO:teuthology.orchestra.run.smithi169.stdout:cephadm-17.2.7-545.g42608bb1.el9.noarch.rpm 34 kB/s | 80 kB 00:02 2024-03-04T13:26:18.797 INFO:teuthology.orchestra.run.smithi169.stdout:-------------------------------------------------------------------------------- 2024-03-04T13:26:18.797 INFO:teuthology.orchestra.run.smithi169.stdout:Total 34 kB/s | 80 kB 00:02 2024-03-04T13:26:18.798 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction check 2024-03-04T13:26:18.816 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction check succeeded. 2024-03-04T13:26:18.816 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction test 2024-03-04T13:26:18.823 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction test succeeded. 2024-03-04T13:26:18.824 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction 2024-03-04T13:26:18.939 INFO:teuthology.orchestra.run.smithi169.stdout: Preparing : 1/1 2024-03-04T13:26:19.533 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: cephadm-2:17.2.7-545.g42608bb1.el9.noarch 1/1 2024-03-04T13:26:19.650 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : cephadm-2:17.2.7-545.g42608bb1.el9.noarch 1/1 2024-03-04T13:26:20.160 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: cephadm-2:17.2.7-545.g42608bb1.el9.noarch 1/1 2024-03-04T13:26:20.534 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : cephadm-2:17.2.7-545.g42608bb1.el9.noarch 1/1 2024-03-04T13:26:20.534 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-03-04T13:26:20.534 INFO:teuthology.orchestra.run.smithi169.stdout:Installed: 2024-03-04T13:26:20.534 INFO:teuthology.orchestra.run.smithi169.stdout: cephadm-2:17.2.7-545.g42608bb1.el9.noarch 2024-03-04T13:26:20.534 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-03-04T13:26:20.534 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2024-03-04T13:26:20.777 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install ceph-immutable-object-cache 2024-03-04T13:26:21.631 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:01:52 ago on Mon 04 Mar 2024 01:24:29 PM UTC. 2024-03-04T13:26:21.801 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2024-03-04T13:26:21.801 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-03-04T13:26:21.801 INFO:teuthology.orchestra.run.smithi169.stdout: Package Arch Version Repo Size 2024-03-04T13:26:21.802 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-03-04T13:26:21.802 INFO:teuthology.orchestra.run.smithi169.stdout:Installing: 2024-03-04T13:26:21.802 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-545.g42608bb1.el9 ceph 137 k 2024-03-04T13:26:21.802 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-03-04T13:26:21.802 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction Summary 2024-03-04T13:26:21.802 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-03-04T13:26:21.802 INFO:teuthology.orchestra.run.smithi169.stdout:Install 1 Package 2024-03-04T13:26:21.802 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-03-04T13:26:21.804 INFO:teuthology.orchestra.run.smithi169.stdout:Total download size: 137 k 2024-03-04T13:26:21.804 INFO:teuthology.orchestra.run.smithi169.stdout:Installed size: 379 k 2024-03-04T13:26:21.804 INFO:teuthology.orchestra.run.smithi169.stdout:Downloading Packages: 2024-03-04T13:26:22.545 INFO:teuthology.orchestra.run.smithi169.stdout:ceph-immutable-object-cache-17.2.7-545.g42608bb 184 kB/s | 137 kB 00:00 2024-03-04T13:26:22.548 INFO:teuthology.orchestra.run.smithi169.stdout:-------------------------------------------------------------------------------- 2024-03-04T13:26:22.548 INFO:teuthology.orchestra.run.smithi169.stdout:Total 183 kB/s | 137 kB 00:00 2024-03-04T13:26:22.548 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction check 2024-03-04T13:26:22.575 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction check succeeded. 2024-03-04T13:26:22.575 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction test 2024-03-04T13:26:22.670 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction test succeeded. 2024-03-04T13:26:22.671 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction 2024-03-04T13:26:22.894 INFO:teuthology.orchestra.run.smithi169.stdout: Preparing : 1/1 2024-03-04T13:26:22.954 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-545.g42608bb1.e 1/1 2024-03-04T13:26:23.026 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-545.g42608bb1.e 1/1 2024-03-04T13:26:23.026 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-04T13:26:23.026 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-04T13:26:23.026 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-03-04T13:26:24.334 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-545.g42608bb1.e 1/1 2024-03-04T13:26:24.334 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-03-04T13:26:24.334 INFO:teuthology.orchestra.run.smithi169.stdout:Installed: 2024-03-04T13:26:24.334 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-immutable-object-cache-2:17.2.7-545.g42608bb1.el9.x86_64 2024-03-04T13:26:24.334 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-03-04T13:26:24.334 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2024-03-04T13:26:24.644 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install ceph-mgr 2024-03-04T13:26:25.498 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:01:56 ago on Mon 04 Mar 2024 01:24:29 PM UTC. 2024-03-04T13:26:25.621 INFO:teuthology.orchestra.run.smithi169.stdout:Package ceph-mgr-2:17.2.7-545.g42608bb1.el9.x86_64 is already installed. 2024-03-04T13:26:25.677 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2024-03-04T13:26:25.678 INFO:teuthology.orchestra.run.smithi169.stdout:Nothing to do. 2024-03-04T13:26:25.678 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2024-03-04T13:26:25.746 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install ceph-mgr-dashboard 2024-03-04T13:26:26.543 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:01:57 ago on Mon 04 Mar 2024 01:24:29 PM UTC. 2024-03-04T13:26:26.718 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2024-03-04T13:26:26.719 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-03-04T13:26:26.719 INFO:teuthology.orchestra.run.smithi169.stdout: Package Arch Version Repository Size 2024-03-04T13:26:26.719 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-03-04T13:26:26.719 INFO:teuthology.orchestra.run.smithi169.stdout:Installing: 2024-03-04T13:26:26.719 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-dashboard noarch 2:17.2.7-545.g42608bb1.el9 ceph-noarch 1.7 M 2024-03-04T13:26:26.719 INFO:teuthology.orchestra.run.smithi169.stdout:Installing dependencies: 2024-03-04T13:26:26.719 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-grafana-dashboards noarch 2:17.2.7-545.g42608bb1.el9 ceph-noarch 23 k 2024-03-04T13:26:26.719 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-prometheus-alerts noarch 2:17.2.7-545.g42608bb1.el9 ceph-noarch 15 k 2024-03-04T13:26:26.719 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-03-04T13:26:26.719 INFO:teuthology.orchestra.run.smithi169.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-04T13:26:26.719 INFO:teuthology.orchestra.run.smithi169.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-04T13:26:26.719 INFO:teuthology.orchestra.run.smithi169.stdout:Installing weak dependencies: 2024-03-04T13:26:26.720 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-03-04T13:26:26.720 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-03-04T13:26:26.720 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction Summary 2024-03-04T13:26:26.720 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-03-04T13:26:26.720 INFO:teuthology.orchestra.run.smithi169.stdout:Install 7 Packages 2024-03-04T13:26:26.720 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-03-04T13:26:26.720 INFO:teuthology.orchestra.run.smithi169.stdout:Total download size: 2.0 M 2024-03-04T13:26:26.721 INFO:teuthology.orchestra.run.smithi169.stdout:Installed size: 8.1 M 2024-03-04T13:26:26.721 INFO:teuthology.orchestra.run.smithi169.stdout:Downloading Packages: 2024-03-04T13:26:27.273 INFO:teuthology.orchestra.run.smithi169.stdout:(1/7): ceph-grafana-dashboards-17.2.7-545.g4260 88 kB/s | 23 kB 00:00 2024-03-04T13:26:27.357 INFO:teuthology.orchestra.run.smithi169.stdout:(2/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 110 kB/s | 9.0 kB 00:00 2024-03-04T13:26:27.440 INFO:teuthology.orchestra.run.smithi169.stdout:(3/7): ceph-prometheus-alerts-17.2.7-545.g42608 34 kB/s | 15 kB 00:00 2024-03-04T13:26:27.457 INFO:teuthology.orchestra.run.smithi169.stdout:(4/7): python3-jwt-2.4.0-1.el9.noarch.rpm 411 kB/s | 41 kB 00:00 2024-03-04T13:26:27.599 INFO:teuthology.orchestra.run.smithi169.stdout:(5/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 195 kB/s | 31 kB 00:00 2024-03-04T13:26:27.658 INFO:teuthology.orchestra.run.smithi169.stdout:(6/7): python3-routes-2.5.1-5.el9.noarch.rpm 945 kB/s | 188 kB 00:00 2024-03-04T13:26:28.149 INFO:teuthology.orchestra.run.smithi169.stdout:(7/7): ceph-mgr-dashboard-17.2.7-545.g42608bb1. 1.5 MB/s | 1.7 MB 00:01 2024-03-04T13:26:28.154 INFO:teuthology.orchestra.run.smithi169.stdout:-------------------------------------------------------------------------------- 2024-03-04T13:26:28.155 INFO:teuthology.orchestra.run.smithi169.stdout:Total 1.4 MB/s | 2.0 MB 00:01 2024-03-04T13:26:28.155 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction check 2024-03-04T13:26:28.181 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction check succeeded. 2024-03-04T13:26:28.181 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction test 2024-03-04T13:26:28.310 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction test succeeded. 2024-03-04T13:26:28.310 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction 2024-03-04T13:26:28.700 INFO:teuthology.orchestra.run.smithi169.stdout: Preparing : 1/1 2024-03-04T13:26:28.759 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-03-04T13:26:28.818 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-03-04T13:26:28.922 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-03-04T13:26:28.966 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-03-04T13:26:29.012 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-545.g42608bb1.el9.no 5/7 2024-03-04T13:26:29.314 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-545.g42608bb1.el9.n 6/7 2024-03-04T13:26:29.353 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-545.g42608bb1.el9.noarch 7/7 2024-03-04T13:26:30.050 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-545.g42608bb1.el9.noarch 7/7 2024-03-04T13:26:30.050 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-545.g42608bb1.el9.n 1/7 2024-03-04T13:26:30.050 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-545.g42608bb1.el9.noarch 2/7 2024-03-04T13:26:30.050 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-545.g42608bb1.el9.no 3/7 2024-03-04T13:26:30.051 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-03-04T13:26:30.051 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-03-04T13:26:30.051 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-03-04T13:26:30.422 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-03-04T13:26:30.422 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-03-04T13:26:30.423 INFO:teuthology.orchestra.run.smithi169.stdout:Installed: 2024-03-04T13:26:30.423 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-grafana-dashboards-2:17.2.7-545.g42608bb1.el9.noarch 2024-03-04T13:26:30.423 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-dashboard-2:17.2.7-545.g42608bb1.el9.noarch 2024-03-04T13:26:30.423 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-prometheus-alerts-2:17.2.7-545.g42608bb1.el9.noarch 2024-03-04T13:26:30.423 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-03-04T13:26:30.423 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-03-04T13:26:30.423 INFO:teuthology.orchestra.run.smithi169.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-04T13:26:30.423 INFO:teuthology.orchestra.run.smithi169.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-04T13:26:30.423 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-03-04T13:26:30.423 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2024-03-04T13:26:30.693 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install ceph-mgr-diskprediction-local 2024-03-04T13:26:31.539 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:02:02 ago on Mon 04 Mar 2024 01:24:29 PM UTC. 2024-03-04T13:26:31.718 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2024-03-04T13:26:31.719 INFO:teuthology.orchestra.run.smithi169.stdout:=========================================================================================== 2024-03-04T13:26:31.719 INFO:teuthology.orchestra.run.smithi169.stdout: Package Arch Version Repository Size 2024-03-04T13:26:31.719 INFO:teuthology.orchestra.run.smithi169.stdout:=========================================================================================== 2024-03-04T13:26:31.719 INFO:teuthology.orchestra.run.smithi169.stdout:Installing: 2024-03-04T13:26:31.719 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-545.g42608bb1.el9 ceph-noarch 7.4 M 2024-03-04T13:26:31.720 INFO:teuthology.orchestra.run.smithi169.stdout:Installing dependencies: 2024-03-04T13:26:31.720 INFO:teuthology.orchestra.run.smithi169.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-04T13:26:31.720 INFO:teuthology.orchestra.run.smithi169.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-04T13:26:31.720 INFO:teuthology.orchestra.run.smithi169.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-04T13:26:31.720 INFO:teuthology.orchestra.run.smithi169.stdout: libgfortran x86_64 11.4.1-3.el9 CentOS-BaseOS 802 k 2024-03-04T13:26:31.720 INFO:teuthology.orchestra.run.smithi169.stdout: libquadmath x86_64 11.4.1-3.el9 CentOS-BaseOS 194 k 2024-03-04T13:26:31.720 INFO:teuthology.orchestra.run.smithi169.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-04T13:26:31.720 INFO:teuthology.orchestra.run.smithi169.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-04T13:26:31.720 INFO:teuthology.orchestra.run.smithi169.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-04T13:26:31.720 INFO:teuthology.orchestra.run.smithi169.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-04T13:26:31.720 INFO:teuthology.orchestra.run.smithi169.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-04T13:26:31.720 INFO:teuthology.orchestra.run.smithi169.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-04T13:26:31.720 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-03-04T13:26:31.720 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction Summary 2024-03-04T13:26:31.721 INFO:teuthology.orchestra.run.smithi169.stdout:=========================================================================================== 2024-03-04T13:26:31.721 INFO:teuthology.orchestra.run.smithi169.stdout:Install 12 Packages 2024-03-04T13:26:31.721 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-03-04T13:26:31.721 INFO:teuthology.orchestra.run.smithi169.stdout:Total download size: 37 M 2024-03-04T13:26:31.721 INFO:teuthology.orchestra.run.smithi169.stdout:Installed size: 209 M 2024-03-04T13:26:31.721 INFO:teuthology.orchestra.run.smithi169.stdout:Downloading Packages: 2024-03-04T13:26:33.631 INFO:teuthology.orchestra.run.smithi169.stdout:(1/12): flexiblas-3.0.4-8.el9.x86_64.rpm 93 kB/s | 32 kB 00:00 2024-03-04T13:26:33.723 INFO:teuthology.orchestra.run.smithi169.stdout:(2/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 189 kB/s | 17 kB 00:00 2024-03-04T13:26:33.823 INFO:teuthology.orchestra.run.smithi169.stdout:(3/12): openblas-0.3.21-2.el9.x86_64.rpm 345 kB/s | 34 kB 00:00 2024-03-04T13:26:34.574 INFO:teuthology.orchestra.run.smithi169.stdout:(4/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 2.3 MB/s | 3.0 MB 00:01 2024-03-04T13:26:34.682 INFO:teuthology.orchestra.run.smithi169.stdout:(5/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 5.5 MB/s | 4.7 MB 00:00 2024-03-04T13:26:35.241 INFO:teuthology.orchestra.run.smithi169.stdout:(6/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 7.2 MB/s | 4.8 MB 00:00 2024-03-04T13:26:35.298 INFO:teuthology.orchestra.run.smithi169.stdout:(7/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 355 kB/s | 217 kB 00:00 2024-03-04T13:26:35.741 INFO:teuthology.orchestra.run.smithi169.stdout:(8/12): libgfortran-11.4.1-3.el9.x86_64.rpm 1.8 MB/s | 802 kB 00:00 2024-03-04T13:26:36.592 INFO:teuthology.orchestra.run.smithi169.stdout:(9/12): libquadmath-11.4.1-3.el9.x86_64.rpm 231 kB/s | 194 kB 00:00 2024-03-04T13:26:37.408 INFO:teuthology.orchestra.run.smithi169.stdout:(10/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 7.4 MB/s | 16 MB 00:02 2024-03-04T13:26:37.433 INFO:teuthology.orchestra.run.smithi169.stdout:(11/12): python3-devel-3.9.17-2.el9.x86_64.rpm 295 kB/s | 245 kB 00:00 2024-03-04T13:26:50.907 INFO:teuthology.orchestra.run.smithi169.stdout:(12/12): ceph-mgr-diskprediction-local-17.2.7-5 430 kB/s | 7.4 MB 00:17 2024-03-04T13:26:50.913 INFO:teuthology.orchestra.run.smithi169.stdout:-------------------------------------------------------------------------------- 2024-03-04T13:26:50.913 INFO:teuthology.orchestra.run.smithi169.stdout:Total 1.9 MB/s | 37 MB 00:19 2024-03-04T13:26:50.958 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction check 2024-03-04T13:26:51.021 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction check succeeded. 2024-03-04T13:26:51.022 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction test 2024-03-04T13:26:51.565 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction test succeeded. 2024-03-04T13:26:51.565 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction 2024-03-04T13:26:52.410 INFO:teuthology.orchestra.run.smithi169.stdout: Preparing : 1/1 2024-03-04T13:26:52.474 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 1/12 2024-03-04T13:26:52.512 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-03-04T13:26:52.659 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 3/12 2024-03-04T13:26:52.741 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 4/12 2024-03-04T13:26:53.000 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-03-04T13:26:53.030 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-03-04T13:26:53.129 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-03-04T13:26:53.918 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-03-04T13:26:54.047 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-03-04T13:26:55.566 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-03-04T13:26:56.717 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-03-04T13:26:56.754 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-545.g42608b 12/12 2024-03-04T13:26:59.314 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-545.g42608b 12/12 2024-03-04T13:26:59.315 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-545.g42608b 1/12 2024-03-04T13:26:59.315 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 2/12 2024-03-04T13:26:59.315 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 3/12 2024-03-04T13:26:59.315 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 4/12 2024-03-04T13:26:59.315 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 5/12 2024-03-04T13:26:59.315 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-03-04T13:26:59.315 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 7/12 2024-03-04T13:26:59.315 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 8/12 2024-03-04T13:26:59.315 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 9/12 2024-03-04T13:26:59.315 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 10/12 2024-03-04T13:26:59.315 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 11/12 2024-03-04T13:26:59.737 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 12/12 2024-03-04T13:26:59.737 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-03-04T13:26:59.738 INFO:teuthology.orchestra.run.smithi169.stdout:Installed: 2024-03-04T13:26:59.738 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-diskprediction-local-2:17.2.7-545.g42608bb1.el9.noarch 2024-03-04T13:26:59.738 INFO:teuthology.orchestra.run.smithi169.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-04T13:26:59.738 INFO:teuthology.orchestra.run.smithi169.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-04T13:26:59.738 INFO:teuthology.orchestra.run.smithi169.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-04T13:26:59.738 INFO:teuthology.orchestra.run.smithi169.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-04T13:26:59.738 INFO:teuthology.orchestra.run.smithi169.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-04T13:26:59.738 INFO:teuthology.orchestra.run.smithi169.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-04T13:26:59.738 INFO:teuthology.orchestra.run.smithi169.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-04T13:26:59.738 INFO:teuthology.orchestra.run.smithi169.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-04T13:26:59.738 INFO:teuthology.orchestra.run.smithi169.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-04T13:26:59.738 INFO:teuthology.orchestra.run.smithi169.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-04T13:26:59.739 INFO:teuthology.orchestra.run.smithi169.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-04T13:26:59.739 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-03-04T13:26:59.739 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2024-03-04T13:27:00.122 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install ceph-mgr-rook 2024-03-04T13:27:00.963 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:02:31 ago on Mon 04 Mar 2024 01:24:29 PM UTC. 2024-03-04T13:27:01.145 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2024-03-04T13:27:01.146 INFO:teuthology.orchestra.run.smithi169.stdout:======================================================================================= 2024-03-04T13:27:01.146 INFO:teuthology.orchestra.run.smithi169.stdout: Package Arch Version Repository Size 2024-03-04T13:27:01.146 INFO:teuthology.orchestra.run.smithi169.stdout:======================================================================================= 2024-03-04T13:27:01.147 INFO:teuthology.orchestra.run.smithi169.stdout:Installing: 2024-03-04T13:27:01.147 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-rook noarch 2:17.2.7-545.g42608bb1.el9 ceph-noarch 47 k 2024-03-04T13:27:01.147 INFO:teuthology.orchestra.run.smithi169.stdout:Installing dependencies: 2024-03-04T13:27:01.147 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-04T13:27:01.147 INFO:teuthology.orchestra.run.smithi169.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-04T13:27:01.147 INFO:teuthology.orchestra.run.smithi169.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-03-04T13:27:01.147 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-04T13:27:01.147 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-04T13:27:01.147 INFO:teuthology.orchestra.run.smithi169.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-04T13:27:01.147 INFO:teuthology.orchestra.run.smithi169.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-04T13:27:01.147 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-04T13:27:01.147 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-04T13:27:01.147 INFO:teuthology.orchestra.run.smithi169.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-04T13:27:01.148 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-04T13:27:01.148 INFO:teuthology.orchestra.run.smithi169.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-04T13:27:01.148 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-03-04T13:27:01.148 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction Summary 2024-03-04T13:27:01.148 INFO:teuthology.orchestra.run.smithi169.stdout:======================================================================================= 2024-03-04T13:27:01.148 INFO:teuthology.orchestra.run.smithi169.stdout:Install 13 Packages 2024-03-04T13:27:01.148 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-03-04T13:27:01.149 INFO:teuthology.orchestra.run.smithi169.stdout:Total download size: 2.0 M 2024-03-04T13:27:01.149 INFO:teuthology.orchestra.run.smithi169.stdout:Installed size: 26 M 2024-03-04T13:27:01.149 INFO:teuthology.orchestra.run.smithi169.stdout:Downloading Packages: 2024-03-04T13:27:01.571 INFO:teuthology.orchestra.run.smithi169.stdout:(1/13): python3-jsonpointer-2.0-4.el9.noarch.rp 70 kB/s | 18 kB 00:00 2024-03-04T13:27:01.596 INFO:teuthology.orchestra.run.smithi169.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 86 kB/s | 24 kB 00:00 2024-03-04T13:27:01.664 INFO:teuthology.orchestra.run.smithi169.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.9 MB/s | 174 kB 00:00 2024-03-04T13:27:01.689 INFO:teuthology.orchestra.run.smithi169.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.4 MB/s | 132 kB 00:00 2024-03-04T13:27:01.722 INFO:teuthology.orchestra.run.smithi169.stdout:(5/13): python3-pyasn1-modules-0.4.8-6.el9.noar 3.6 MB/s | 211 kB 00:00 2024-03-04T13:27:01.748 INFO:teuthology.orchestra.run.smithi169.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 754 kB/s | 43 kB 00:00 2024-03-04T13:27:01.781 INFO:teuthology.orchestra.run.smithi169.stdout:(7/13): ceph-mgr-rook-17.2.7-545.g42608bb1.el9. 101 kB/s | 47 kB 00:00 2024-03-04T13:27:01.881 INFO:teuthology.orchestra.run.smithi169.stdout:(8/13): python3-certifi-2021.10.8-2.el9.noarch. 113 kB/s | 15 kB 00:00 2024-03-04T13:27:01.907 INFO:teuthology.orchestra.run.smithi169.stdout:(9/13): python3-cachetools-4.2.4-1.el9.noarch.r 176 kB/s | 32 kB 00:00 2024-03-04T13:27:02.032 INFO:teuthology.orchestra.run.smithi169.stdout:(10/13): python3-google-auth-2.28.0-1.el9.noarc 811 kB/s | 203 kB 00:00 2024-03-04T13:27:02.061 INFO:teuthology.orchestra.run.smithi169.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 395 kB/s | 59 kB 00:00 2024-03-04T13:27:02.116 INFO:teuthology.orchestra.run.smithi107.stdout:(6/6): ceph-test-17.2.7-545.g42608bb1.el9.x86_6 733 kB/s | 48 MB 01:07 2024-03-04T13:27:02.119 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-03-04T13:27:02.119 INFO:teuthology.orchestra.run.smithi107.stdout:Total 747 kB/s | 49 MB 01:07 2024-03-04T13:27:02.119 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-03-04T13:27:02.124 INFO:teuthology.orchestra.run.smithi169.stdout:(12/13): python3-kubernetes-26.1.0-2.el9.noarch 4.2 MB/s | 1.0 MB 00:00 2024-03-04T13:27:02.143 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-03-04T13:27:02.143 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-03-04T13:27:02.149 INFO:teuthology.orchestra.run.smithi169.stdout:(13/13): python3-websocket-client-1.2.3-2.el9.n 774 kB/s | 90 kB 00:00 2024-03-04T13:27:02.154 INFO:teuthology.orchestra.run.smithi169.stdout:-------------------------------------------------------------------------------- 2024-03-04T13:27:02.155 INFO:teuthology.orchestra.run.smithi169.stdout:Total 2.0 MB/s | 2.0 MB 00:01 2024-03-04T13:27:02.155 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction check 2024-03-04T13:27:02.197 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction check succeeded. 2024-03-04T13:27:02.197 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction test 2024-03-04T13:27:02.350 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction test succeeded. 2024-03-04T13:27:02.352 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction 2024-03-04T13:27:02.597 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-03-04T13:27:02.598 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-03-04T13:27:03.026 INFO:teuthology.orchestra.run.smithi169.stdout: Preparing : 1/1 2024-03-04T13:27:03.252 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-03-04T13:27:03.317 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-03-04T13:27:03.324 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-03-04T13:27:03.382 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-03-04T13:27:03.420 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 1/6 2024-03-04T13:27:03.429 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-03-04T13:27:03.455 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : jq-1.6-16.el9.x86_64 2/6 2024-03-04T13:27:03.477 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-03-04T13:27:03.528 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 3/6 2024-03-04T13:27:03.604 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-03-04T13:27:03.606 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 4/6 2024-03-04T13:27:03.636 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libcephsqlite-2:17.2.7-545.g42608bb1.el9.x86_64 5/6 2024-03-04T13:27:03.782 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 7/13 2024-03-04T13:27:03.852 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-03-04T13:27:04.784 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-03-04T13:27:04.843 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-03-04T13:27:04.876 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-03-04T13:27:04.929 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-03-04T13:27:04.953 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mgr-rook-2:17.2.7-545.g42608bb1.el9.noarch 13/13 2024-03-04T13:27:05.850 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-545.g42608bb1.el9.noarch 13/13 2024-03-04T13:27:05.850 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mgr-rook-2:17.2.7-545.g42608bb1.el9.noarch 1/13 2024-03-04T13:27:05.850 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-03-04T13:27:05.850 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-03-04T13:27:05.851 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-03-04T13:27:05.851 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-03-04T13:27:05.851 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-03-04T13:27:05.851 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-03-04T13:27:05.851 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-03-04T13:27:05.851 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-03-04T13:27:05.851 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 10/13 2024-03-04T13:27:05.851 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-03-04T13:27:05.851 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-03-04T13:27:06.310 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-03-04T13:27:06.310 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-03-04T13:27:06.310 INFO:teuthology.orchestra.run.smithi169.stdout:Installed: 2024-03-04T13:27:06.310 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-rook-2:17.2.7-545.g42608bb1.el9.noarch 2024-03-04T13:27:06.311 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-04T13:27:06.311 INFO:teuthology.orchestra.run.smithi169.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-04T13:27:06.311 INFO:teuthology.orchestra.run.smithi169.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-03-04T13:27:06.311 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-04T13:27:06.311 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-04T13:27:06.311 INFO:teuthology.orchestra.run.smithi169.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-04T13:27:06.311 INFO:teuthology.orchestra.run.smithi169.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-04T13:27:06.311 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-04T13:27:06.311 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-04T13:27:06.311 INFO:teuthology.orchestra.run.smithi169.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-04T13:27:06.311 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-04T13:27:06.311 INFO:teuthology.orchestra.run.smithi169.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-04T13:27:06.311 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-03-04T13:27:06.311 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2024-03-04T13:27:06.609 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install ceph-mgr-cephadm 2024-03-04T13:27:07.455 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:02:38 ago on Mon 04 Mar 2024 01:24:29 PM UTC. 2024-03-04T13:27:07.631 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2024-03-04T13:27:07.632 INFO:teuthology.orchestra.run.smithi169.stdout:======================================================================================= 2024-03-04T13:27:07.632 INFO:teuthology.orchestra.run.smithi169.stdout: Package Arch Version Repository Size 2024-03-04T13:27:07.632 INFO:teuthology.orchestra.run.smithi169.stdout:======================================================================================= 2024-03-04T13:27:07.632 INFO:teuthology.orchestra.run.smithi169.stdout:Installing: 2024-03-04T13:27:07.632 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-cephadm noarch 2:17.2.7-545.g42608bb1.el9 ceph-noarch 121 k 2024-03-04T13:27:07.632 INFO:teuthology.orchestra.run.smithi169.stdout:Installing dependencies: 2024-03-04T13:27:07.632 INFO:teuthology.orchestra.run.smithi169.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-03-04T13:27:07.632 INFO:teuthology.orchestra.run.smithi169.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-04T13:27:07.632 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jinja2 noarch 2.11.3-5.el9 CentOS-AppStream 227 k 2024-03-04T13:27:07.633 INFO:teuthology.orchestra.run.smithi169.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-04T13:27:07.633 INFO:teuthology.orchestra.run.smithi169.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-04T13:27:07.633 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-03-04T13:27:07.633 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction Summary 2024-03-04T13:27:07.633 INFO:teuthology.orchestra.run.smithi169.stdout:======================================================================================= 2024-03-04T13:27:07.633 INFO:teuthology.orchestra.run.smithi169.stdout:Install 6 Packages 2024-03-04T13:27:07.633 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-03-04T13:27:07.634 INFO:teuthology.orchestra.run.smithi169.stdout:Total download size: 6.8 M 2024-03-04T13:27:07.634 INFO:teuthology.orchestra.run.smithi169.stdout:Installed size: 33 M 2024-03-04T13:27:07.634 INFO:teuthology.orchestra.run.smithi169.stdout:Downloading Packages: 2024-03-04T13:27:08.351 INFO:teuthology.orchestra.run.smithi169.stdout:(1/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 728 kB/s | 227 kB 00:00 2024-03-04T13:27:08.505 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: libcephsqlite-2:17.2.7-545.g42608bb1.el9.x86_64 5/6 2024-03-04T13:27:08.626 INFO:teuthology.orchestra.run.smithi169.stdout:(2/6): python3-babel-2.9.1-2.el9.noarch.rpm 9.8 MB/s | 5.8 MB 00:00 2024-03-04T13:27:08.642 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-test-2:17.2.7-545.g42608bb1.el9.x86_64 6/6 2024-03-04T13:27:08.677 INFO:teuthology.orchestra.run.smithi169.stdout:(3/6): ceph-mgr-cephadm-17.2.7-545.g42608bb1.el 190 kB/s | 121 kB 00:00 2024-03-04T13:27:09.052 INFO:teuthology.orchestra.run.smithi169.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 137 kB/s | 58 kB 00:00 2024-03-04T13:27:09.102 INFO:teuthology.orchestra.run.smithi169.stdout:(5/6): python3-asyncssh-2.13.2-4.el9.noarch.rpm 735 kB/s | 548 kB 00:00 2024-03-04T13:27:09.128 INFO:teuthology.orchestra.run.smithi169.stdout:(6/6): python3-typing-extensions-4.4.0-2.el9.no 113 kB/s | 51 kB 00:00 2024-03-04T13:27:09.138 INFO:teuthology.orchestra.run.smithi169.stdout:-------------------------------------------------------------------------------- 2024-03-04T13:27:09.139 INFO:teuthology.orchestra.run.smithi169.stdout:Total 4.5 MB/s | 6.8 MB 00:01 2024-03-04T13:27:09.139 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction check 2024-03-04T13:27:09.169 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction check succeeded. 2024-03-04T13:27:09.169 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction test 2024-03-04T13:27:09.312 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction test succeeded. 2024-03-04T13:27:09.313 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction 2024-03-04T13:27:09.689 INFO:teuthology.orchestra.run.smithi169.stdout: Preparing : 1/1 2024-03-04T13:27:09.919 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-03-04T13:27:09.977 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 2/6 2024-03-04T13:27:10.682 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-03-04T13:27:10.789 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-03-04T13:27:10.861 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-03-04T13:27:10.891 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-545.g42608bb1.el9.noarch 6/6 2024-03-04T13:27:11.077 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-test-2:17.2.7-545.g42608bb1.el9.x86_64 6/6 2024-03-04T13:27:11.077 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-test-2:17.2.7-545.g42608bb1.el9.x86_64 1/6 2024-03-04T13:27:11.077 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libcephsqlite-2:17.2.7-545.g42608bb1.el9.x86_64 2/6 2024-03-04T13:27:11.077 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 3/6 2024-03-04T13:27:11.078 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/6 2024-03-04T13:27:11.078 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : jq-1.6-16.el9.x86_64 5/6 2024-03-04T13:27:11.452 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 6/6 2024-03-04T13:27:11.452 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:27:11.452 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-03-04T13:27:11.453 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-test-2:17.2.7-545.g42608bb1.el9.x86_64 2024-03-04T13:27:11.453 INFO:teuthology.orchestra.run.smithi107.stdout: jq-1.6-16.el9.x86_64 2024-03-04T13:27:11.453 INFO:teuthology.orchestra.run.smithi107.stdout: libcephsqlite-2:17.2.7-545.g42608bb1.el9.x86_64 2024-03-04T13:27:11.453 INFO:teuthology.orchestra.run.smithi107.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-04T13:27:11.453 INFO:teuthology.orchestra.run.smithi107.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-04T13:27:11.453 INFO:teuthology.orchestra.run.smithi107.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-04T13:27:11.453 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:27:11.453 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-03-04T13:27:11.849 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-545.g42608bb1.el9.noarch 6/6 2024-03-04T13:27:11.849 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-545.g42608bb1.el9.noarch 1/6 2024-03-04T13:27:11.849 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-03-04T13:27:11.849 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-03-04T13:27:11.849 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 4/6 2024-03-04T13:27:11.849 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-03-04T13:27:11.902 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install ceph 2024-03-04T13:27:12.238 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-03-04T13:27:12.238 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-03-04T13:27:12.238 INFO:teuthology.orchestra.run.smithi169.stdout:Installed: 2024-03-04T13:27:12.238 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-cephadm-2:17.2.7-545.g42608bb1.el9.noarch 2024-03-04T13:27:12.238 INFO:teuthology.orchestra.run.smithi169.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-03-04T13:27:12.238 INFO:teuthology.orchestra.run.smithi169.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-04T13:27:12.238 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-04T13:27:12.238 INFO:teuthology.orchestra.run.smithi169.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-04T13:27:12.238 INFO:teuthology.orchestra.run.smithi169.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-04T13:27:12.238 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-03-04T13:27:12.239 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2024-03-04T13:27:12.553 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install ceph-fuse 2024-03-04T13:27:12.745 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:02:13 ago on Mon 04 Mar 2024 01:24:59 PM UTC. 2024-03-04T13:27:12.936 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-03-04T13:27:12.938 INFO:teuthology.orchestra.run.smithi107.stdout:======================================================================================== 2024-03-04T13:27:12.938 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repository Size 2024-03-04T13:27:12.938 INFO:teuthology.orchestra.run.smithi107.stdout:======================================================================================== 2024-03-04T13:27:12.939 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-03-04T13:27:12.939 INFO:teuthology.orchestra.run.smithi107.stdout: ceph x86_64 2:17.2.7-545.g42608bb1.el9 ceph 6.5 k 2024-03-04T13:27:12.939 INFO:teuthology.orchestra.run.smithi107.stdout:Installing dependencies: 2024-03-04T13:27:12.939 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mds x86_64 2:17.2.7-545.g42608bb1.el9 ceph 2.1 M 2024-03-04T13:27:12.939 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr x86_64 2:17.2.7-545.g42608bb1.el9 ceph 1.4 M 2024-03-04T13:27:12.939 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-modules-core noarch 2:17.2.7-545.g42608bb1.el9 ceph-noarch 239 k 2024-03-04T13:27:12.940 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mon x86_64 2:17.2.7-545.g42608bb1.el9 ceph 3.7 M 2024-03-04T13:27:12.940 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-osd x86_64 2:17.2.7-545.g42608bb1.el9 ceph 15 M 2024-03-04T13:27:12.940 INFO:teuthology.orchestra.run.smithi107.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-04T13:27:12.940 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-04T13:27:12.940 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-04T13:27:12.940 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-04T13:27:12.941 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-04T13:27:12.941 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-04T13:27:12.941 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-04T13:27:12.941 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-04T13:27:12.941 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-04T13:27:12.942 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-04T13:27:12.942 INFO:teuthology.orchestra.run.smithi107.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-04T13:27:12.942 INFO:teuthology.orchestra.run.smithi107.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-04T13:27:12.942 INFO:teuthology.orchestra.run.smithi107.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-04T13:27:12.942 INFO:teuthology.orchestra.run.smithi107.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-04T13:27:12.943 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-04T13:27:12.943 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-04T13:27:12.944 INFO:teuthology.orchestra.run.smithi107.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-04T13:27:12.944 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-04T13:27:12.944 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-04T13:27:12.944 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-04T13:27:12.944 INFO:teuthology.orchestra.run.smithi107.stdout: python3-requests noarch 2.25.1-8.el9 CentOS-BaseOS 113 k 2024-03-04T13:27:12.944 INFO:teuthology.orchestra.run.smithi107.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-04T13:27:12.944 INFO:teuthology.orchestra.run.smithi107.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-04T13:27:12.944 INFO:teuthology.orchestra.run.smithi107.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-04T13:27:12.944 INFO:teuthology.orchestra.run.smithi107.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-04T13:27:12.944 INFO:teuthology.orchestra.run.smithi107.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-04T13:27:12.945 INFO:teuthology.orchestra.run.smithi107.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-04T13:27:12.945 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:27:12.945 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-03-04T13:27:12.945 INFO:teuthology.orchestra.run.smithi107.stdout:======================================================================================== 2024-03-04T13:27:12.945 INFO:teuthology.orchestra.run.smithi107.stdout:Install 33 Packages 2024-03-04T13:27:12.945 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:27:12.946 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 27 M 2024-03-04T13:27:12.946 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 99 M 2024-03-04T13:27:12.946 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-03-04T13:27:13.383 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:02:44 ago on Mon 04 Mar 2024 01:24:29 PM UTC. 2024-03-04T13:27:13.552 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2024-03-04T13:27:13.553 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-03-04T13:27:13.553 INFO:teuthology.orchestra.run.smithi169.stdout: Package Arch Version Repository Size 2024-03-04T13:27:13.553 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-03-04T13:27:13.553 INFO:teuthology.orchestra.run.smithi169.stdout:Installing: 2024-03-04T13:27:13.553 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-fuse x86_64 2:17.2.7-545.g42608bb1.el9 ceph 812 k 2024-03-04T13:27:13.553 INFO:teuthology.orchestra.run.smithi169.stdout:Installing dependencies: 2024-03-04T13:27:13.553 INFO:teuthology.orchestra.run.smithi169.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-04T13:27:13.553 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-03-04T13:27:13.553 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction Summary 2024-03-04T13:27:13.554 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-03-04T13:27:13.554 INFO:teuthology.orchestra.run.smithi169.stdout:Install 2 Packages 2024-03-04T13:27:13.554 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-03-04T13:27:13.554 INFO:teuthology.orchestra.run.smithi169.stdout:Total download size: 891 k 2024-03-04T13:27:13.554 INFO:teuthology.orchestra.run.smithi169.stdout:Installed size: 2.6 M 2024-03-04T13:27:13.554 INFO:teuthology.orchestra.run.smithi169.stdout:Downloading Packages: 2024-03-04T13:27:13.568 INFO:teuthology.orchestra.run.smithi107.stdout:(1/33): ceph-17.2.7-545.g42608bb1.el9.x86_64.rp 26 kB/s | 6.5 kB 00:00 2024-03-04T13:27:13.804 INFO:teuthology.orchestra.run.smithi169.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 316 kB/s | 79 kB 00:00 2024-03-04T13:27:13.846 INFO:teuthology.orchestra.run.smithi107.stdout:(2/33): ceph-mgr-17.2.7-545.g42608bb1.el9.x86_6 2.6 MB/s | 1.4 MB 00:00 2024-03-04T13:27:13.935 INFO:teuthology.orchestra.run.smithi107.stdout:(3/33): ceph-mds-17.2.7-545.g42608bb1.el9.x86_6 3.3 MB/s | 2.1 MB 00:00 2024-03-04T13:27:14.010 INFO:teuthology.orchestra.run.smithi107.stdout:(4/33): ceph-mgr-modules-core-17.2.7-545.g42608 3.2 MB/s | 239 kB 00:00 2024-03-04T13:27:14.102 INFO:teuthology.orchestra.run.smithi107.stdout:(5/33): ceph-mon-17.2.7-545.g42608bb1.el9.x86_6 6.9 MB/s | 3.7 MB 00:00 2024-03-04T13:27:14.386 INFO:teuthology.orchestra.run.smithi107.stdout:(6/33): python3-markupsafe-1.1.1-12.el9.x86_64. 147 kB/s | 32 kB 00:00 2024-03-04T13:27:14.411 INFO:teuthology.orchestra.run.smithi107.stdout:(7/33): python3-mako-1.1.4-6.el9.noarch.rpm 361 kB/s | 144 kB 00:00 2024-03-04T13:27:14.461 INFO:teuthology.orchestra.run.smithi107.stdout:(8/33): python3-pytz-2021.1-5.el9.noarch.rpm 642 kB/s | 47 kB 00:00 2024-03-04T13:27:14.495 INFO:teuthology.orchestra.run.smithi107.stdout:(9/33): python3-toml-0.10.2-6.el9.noarch.rpm 466 kB/s | 39 kB 00:00 2024-03-04T13:27:14.703 INFO:teuthology.orchestra.run.smithi169.stdout:(2/2): ceph-fuse-17.2.7-545.g42608bb1.el9.x86_6 707 kB/s | 812 kB 00:01 2024-03-04T13:27:14.707 INFO:teuthology.orchestra.run.smithi169.stdout:-------------------------------------------------------------------------------- 2024-03-04T13:27:14.708 INFO:teuthology.orchestra.run.smithi169.stdout:Total 772 kB/s | 891 kB 00:01 2024-03-04T13:27:14.709 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction check 2024-03-04T13:27:14.762 INFO:teuthology.orchestra.run.smithi107.stdout:(10/33): ceph-osd-17.2.7-545.g42608bb1.el9.x86_ 17 MB/s | 15 MB 00:00 2024-03-04T13:27:14.776 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction check succeeded. 2024-03-04T13:27:14.776 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction test 2024-03-04T13:27:14.795 INFO:teuthology.orchestra.run.smithi107.stdout:(11/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 724 kB/s | 241 kB 00:00 2024-03-04T13:27:14.874 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction test succeeded. 2024-03-04T13:27:14.874 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction 2024-03-04T13:27:14.970 INFO:teuthology.orchestra.run.smithi107.stdout:(12/33): python3-cryptography-36.0.1-4.el9.x86_ 2.4 MB/s | 1.1 MB 00:00 2024-03-04T13:27:14.997 INFO:teuthology.orchestra.run.smithi107.stdout:(13/33): python3-ply-3.11-14.el9.noarch.rpm 454 kB/s | 103 kB 00:00 2024-03-04T13:27:15.021 INFO:teuthology.orchestra.run.smithi107.stdout:(14/33): python3-pycparser-2.20-6.el9.noarch.rp 551 kB/s | 124 kB 00:00 2024-03-04T13:27:15.046 INFO:teuthology.orchestra.run.smithi107.stdout:(15/33): python3-requests-2.25.1-8.el9.noarch.r 1.5 MB/s | 113 kB 00:00 2024-03-04T13:27:15.113 INFO:teuthology.orchestra.run.smithi107.stdout:(16/33): python3-urllib3-1.26.5-5.el9.noarch.rp 2.0 MB/s | 187 kB 00:00 2024-03-04T13:27:15.126 INFO:teuthology.orchestra.run.smithi169.stdout: Preparing : 1/1 2024-03-04T13:27:15.301 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-03-04T13:27:15.374 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-fuse-2:17.2.7-545.g42608bb1.el9.x86_64 2/2 2024-03-04T13:27:15.614 INFO:teuthology.orchestra.run.smithi107.stdout:(17/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 74 kB/s | 43 kB 00:00 2024-03-04T13:27:15.772 INFO:teuthology.orchestra.run.smithi107.stdout:(18/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 68 kB/s | 11 kB 00:00 2024-03-04T13:27:15.889 INFO:teuthology.orchestra.run.smithi107.stdout:(19/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 204 kB/s | 172 kB 00:00 2024-03-04T13:27:15.914 INFO:teuthology.orchestra.run.smithi107.stdout:(20/33): python3-jaraco-classes-3.2.1-5.el9.noa 125 kB/s | 18 kB 00:00 2024-03-04T13:27:16.156 INFO:teuthology.orchestra.run.smithi107.stdout:(21/33): python3-cherrypy-18.6.1-2.el9.noarch.r 344 kB/s | 358 kB 00:01 2024-03-04T13:27:16.190 INFO:teuthology.orchestra.run.smithi107.stdout:(22/33): python3-jaraco-collections-3.0.0-8.el9 77 kB/s | 23 kB 00:00 2024-03-04T13:27:16.216 INFO:teuthology.orchestra.run.smithi107.stdout:(23/33): python3-jaraco-functools-3.5.0-2.el9.n 65 kB/s | 19 kB 00:00 2024-03-04T13:27:16.307 INFO:teuthology.orchestra.run.smithi107.stdout:(24/33): python3-jaraco-text-3.2.0-6.el9.noarch 131 kB/s | 20 kB 00:00 2024-03-04T13:27:16.340 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-fuse-2:17.2.7-545.g42608bb1.el9.x86_64 2/2 2024-03-04T13:27:16.340 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-fuse-2:17.2.7-545.g42608bb1.el9.x86_64 1/2 2024-03-04T13:27:16.399 INFO:teuthology.orchestra.run.smithi107.stdout:(25/33): python3-more-itertools-8.12.0-2.el9.no 430 kB/s | 79 kB 00:00 2024-03-04T13:27:16.424 INFO:teuthology.orchestra.run.smithi107.stdout:(26/33): python3-logutils-0.3.5-21.el9.noarch.r 198 kB/s | 46 kB 00:00 2024-03-04T13:27:16.541 INFO:teuthology.orchestra.run.smithi107.stdout:(27/33): python3-portend-3.1.0-2.el9.noarch.rpm 116 kB/s | 16 kB 00:00 2024-03-04T13:27:16.583 INFO:teuthology.orchestra.run.smithi107.stdout:(28/33): python3-pecan-1.4.2-3.el9.noarch.rpm 989 kB/s | 272 kB 00:00 2024-03-04T13:27:16.609 INFO:teuthology.orchestra.run.smithi107.stdout:(29/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 491 kB/s | 90 kB 00:00 2024-03-04T13:27:16.665 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-03-04T13:27:16.665 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-03-04T13:27:16.665 INFO:teuthology.orchestra.run.smithi169.stdout:Installed: 2024-03-04T13:27:16.665 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-fuse-2:17.2.7-545.g42608bb1.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-03-04T13:27:16.665 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-03-04T13:27:16.665 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2024-03-04T13:27:16.692 INFO:teuthology.orchestra.run.smithi107.stdout:(30/33): python3-tempora-5.0.0-2.el9.noarch.rpm 239 kB/s | 36 kB 00:00 2024-03-04T13:27:16.789 INFO:teuthology.orchestra.run.smithi107.stdout:(31/33): python3-webob-1.8.7-6.el9.noarch.rpm 1.1 MB/s | 230 kB 00:00 2024-03-04T13:27:16.859 INFO:teuthology.orchestra.run.smithi107.stdout:(32/33): python3-zc-lockfile-2.0-10.el9.noarch. 120 kB/s | 20 kB 00:00 2024-03-04T13:27:16.976 INFO:teuthology.orchestra.run.smithi107.stdout:(33/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 1.1 MB/s | 427 kB 00:00 2024-03-04T13:27:16.990 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-03-04T13:27:16.990 INFO:teuthology.orchestra.run.smithi107.stdout:Total 6.6 MB/s | 27 MB 00:04 2024-03-04T13:27:16.991 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-03-04T13:27:16.993 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install librados-devel 2024-03-04T13:27:17.069 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-03-04T13:27:17.070 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-03-04T13:27:17.483 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-03-04T13:27:17.484 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-03-04T13:27:17.852 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:02:48 ago on Mon 04 Mar 2024 01:24:29 PM UTC. 2024-03-04T13:27:18.028 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2024-03-04T13:27:18.029 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-03-04T13:27:18.029 INFO:teuthology.orchestra.run.smithi169.stdout: Package Arch Version Repo Size 2024-03-04T13:27:18.029 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-03-04T13:27:18.029 INFO:teuthology.orchestra.run.smithi169.stdout:Installing: 2024-03-04T13:27:18.030 INFO:teuthology.orchestra.run.smithi169.stdout: librados-devel x86_64 2:17.2.7-545.g42608bb1.el9 ceph 126 k 2024-03-04T13:27:18.030 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-03-04T13:27:18.030 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction Summary 2024-03-04T13:27:18.030 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-03-04T13:27:18.030 INFO:teuthology.orchestra.run.smithi169.stdout:Install 1 Package 2024-03-04T13:27:18.030 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-03-04T13:27:18.030 INFO:teuthology.orchestra.run.smithi169.stdout:Total download size: 126 k 2024-03-04T13:27:18.030 INFO:teuthology.orchestra.run.smithi169.stdout:Installed size: 448 k 2024-03-04T13:27:18.030 INFO:teuthology.orchestra.run.smithi169.stdout:Downloading Packages: 2024-03-04T13:27:18.485 INFO:teuthology.orchestra.run.smithi169.stdout:librados-devel-17.2.7-545.g42608bb1.el9.x86_64. 278 kB/s | 126 kB 00:00 2024-03-04T13:27:18.486 INFO:teuthology.orchestra.run.smithi169.stdout:-------------------------------------------------------------------------------- 2024-03-04T13:27:18.487 INFO:teuthology.orchestra.run.smithi169.stdout:Total 276 kB/s | 126 kB 00:00 2024-03-04T13:27:18.487 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction check 2024-03-04T13:27:18.507 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction check succeeded. 2024-03-04T13:27:18.507 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction test 2024-03-04T13:27:18.612 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction test succeeded. 2024-03-04T13:27:18.613 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction 2024-03-04T13:27:18.818 INFO:teuthology.orchestra.run.smithi169.stdout: Preparing : 1/1 2024-03-04T13:27:18.894 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-03-04T13:27:18.932 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : librados-devel-2:17.2.7-545.g42608bb1.el9.x86_64 1/1 2024-03-04T13:27:18.963 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-03-04T13:27:19.001 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-03-04T13:27:19.053 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-03-04T13:27:19.194 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-03-04T13:27:19.315 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-03-04T13:27:19.369 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/33 2024-03-04T13:27:19.473 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-03-04T13:27:19.488 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: librados-devel-2:17.2.7-545.g42608bb1.el9.x86_64 1/1 2024-03-04T13:27:19.538 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 8/33 2024-03-04T13:27:19.582 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 9/33 2024-03-04T13:27:19.647 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-ply-3.11-14.el9.noarch 10/33 2024-03-04T13:27:19.761 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 11/33 2024-03-04T13:27:19.813 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librados-devel-2:17.2.7-545.g42608bb1.el9.x86_64 1/1 2024-03-04T13:27:19.814 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-03-04T13:27:19.814 INFO:teuthology.orchestra.run.smithi169.stdout:Installed: 2024-03-04T13:27:19.814 INFO:teuthology.orchestra.run.smithi169.stdout: librados-devel-2:17.2.7-545.g42608bb1.el9.x86_64 2024-03-04T13:27:19.814 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-03-04T13:27:19.814 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2024-03-04T13:27:20.017 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 12/33 2024-03-04T13:27:20.099 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 13/33 2024-03-04T13:27:20.127 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install libcephfs2 2024-03-04T13:27:20.149 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 14/33 2024-03-04T13:27:20.197 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 15/33 2024-03-04T13:27:20.233 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 16/33 2024-03-04T13:27:20.353 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 17/33 2024-03-04T13:27:20.389 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-03-04T13:27:20.424 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 19/33 2024-03-04T13:27:20.474 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 20/33 2024-03-04T13:27:20.555 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/33 2024-03-04T13:27:20.598 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 22/33 2024-03-04T13:27:20.742 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 23/33 2024-03-04T13:27:20.784 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 24/33 2024-03-04T13:27:20.900 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/33 2024-03-04T13:27:20.988 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:02:51 ago on Mon 04 Mar 2024 01:24:29 PM UTC. 2024-03-04T13:27:21.085 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 26/33 2024-03-04T13:27:21.106 INFO:teuthology.orchestra.run.smithi169.stdout:Package libcephfs2-2:17.2.7-545.g42608bb1.el9.x86_64 is already installed. 2024-03-04T13:27:21.162 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2024-03-04T13:27:21.163 INFO:teuthology.orchestra.run.smithi169.stdout:Nothing to do. 2024-03-04T13:27:21.163 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2024-03-04T13:27:21.226 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install libcephfs-devel 2024-03-04T13:27:21.242 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-03-04T13:27:21.413 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-545.g42608bb1.el9.n 28/33 2024-03-04T13:27:21.443 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-2:17.2.7-545.g42608bb1.el9.x86_64 29/33 2024-03-04T13:27:21.513 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-2:17.2.7-545.g42608bb1.el9.x86_64 29/33 2024-03-04T13:27:21.513 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-04T13:27:21.513 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-04T13:27:21.513 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-04T13:27:21.514 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-04T13:27:21.514 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:27:22.027 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:02:53 ago on Mon 04 Mar 2024 01:24:29 PM UTC. 2024-03-04T13:27:22.203 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2024-03-04T13:27:22.203 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-03-04T13:27:22.203 INFO:teuthology.orchestra.run.smithi169.stdout: Package Arch Version Repo Size 2024-03-04T13:27:22.203 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-03-04T13:27:22.203 INFO:teuthology.orchestra.run.smithi169.stdout:Installing: 2024-03-04T13:27:22.203 INFO:teuthology.orchestra.run.smithi169.stdout: libcephfs-devel x86_64 2:17.2.7-545.g42608bb1.el9 ceph 25 k 2024-03-04T13:27:22.203 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-03-04T13:27:22.203 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction Summary 2024-03-04T13:27:22.204 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-03-04T13:27:22.204 INFO:teuthology.orchestra.run.smithi169.stdout:Install 1 Package 2024-03-04T13:27:22.204 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-03-04T13:27:22.204 INFO:teuthology.orchestra.run.smithi169.stdout:Total download size: 25 k 2024-03-04T13:27:22.204 INFO:teuthology.orchestra.run.smithi169.stdout:Installed size: 110 k 2024-03-04T13:27:22.205 INFO:teuthology.orchestra.run.smithi169.stdout:Downloading Packages: 2024-03-04T13:27:22.380 INFO:teuthology.orchestra.run.smithi169.stdout:libcephfs-devel-17.2.7-545.g42608bb1.el9.x86_64 142 kB/s | 25 kB 00:00 2024-03-04T13:27:22.381 INFO:teuthology.orchestra.run.smithi169.stdout:-------------------------------------------------------------------------------- 2024-03-04T13:27:22.382 INFO:teuthology.orchestra.run.smithi169.stdout:Total 140 kB/s | 25 kB 00:00 2024-03-04T13:27:22.382 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction check 2024-03-04T13:27:22.390 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction check succeeded. 2024-03-04T13:27:22.390 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction test 2024-03-04T13:27:22.394 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction test succeeded. 2024-03-04T13:27:22.395 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction 2024-03-04T13:27:22.535 INFO:teuthology.orchestra.run.smithi169.stdout: Preparing : 1/1 2024-03-04T13:27:22.703 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libcephfs-devel-2:17.2.7-545.g42608bb1.el9.x86_64 1/1 2024-03-04T13:27:22.970 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-osd-2:17.2.7-545.g42608bb1.el9.x86_64 30/33 2024-03-04T13:27:23.044 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-osd-2:17.2.7-545.g42608bb1.el9.x86_64 30/33 2024-03-04T13:27:23.044 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-04T13:27:23.044 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-04T13:27:23.044 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-04T13:27:23.044 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-04T13:27:23.045 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:27:23.160 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: libcephfs-devel-2:17.2.7-545.g42608bb1.el9.x86_64 1/1 2024-03-04T13:27:23.451 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libcephfs-devel-2:17.2.7-545.g42608bb1.el9.x86_64 1/1 2024-03-04T13:27:23.451 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-03-04T13:27:23.451 INFO:teuthology.orchestra.run.smithi169.stdout:Installed: 2024-03-04T13:27:23.451 INFO:teuthology.orchestra.run.smithi169.stdout: libcephfs-devel-2:17.2.7-545.g42608bb1.el9.x86_64 2024-03-04T13:27:23.451 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-03-04T13:27:23.452 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2024-03-04T13:27:23.462 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mon-2:17.2.7-545.g42608bb1.el9.x86_64 31/33 2024-03-04T13:27:23.527 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mon-2:17.2.7-545.g42608bb1.el9.x86_64 31/33 2024-03-04T13:27:23.528 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-04T13:27:23.528 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-04T13:27:23.528 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-04T13:27:23.528 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-04T13:27:23.528 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:27:23.664 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install librados2 2024-03-04T13:27:23.737 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mds-2:17.2.7-545.g42608bb1.el9.x86_64 32/33 2024-03-04T13:27:23.801 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mds-2:17.2.7-545.g42608bb1.el9.x86_64 32/33 2024-03-04T13:27:23.801 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-04T13:27:23.802 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-04T13:27:23.802 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-04T13:27:23.802 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-04T13:27:23.802 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:27:23.867 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-2:17.2.7-545.g42608bb1.el9.x86_64 33/33 2024-03-04T13:27:23.973 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-2:17.2.7-545.g42608bb1.el9.x86_64 33/33 2024-03-04T13:27:23.973 INFO:teuthology.orchestra.run.smithi107.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-03-04T13:27:23.974 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:27:24.494 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:02:55 ago on Mon 04 Mar 2024 01:24:29 PM UTC. 2024-03-04T13:27:24.606 INFO:teuthology.orchestra.run.smithi169.stdout:Package librados2-2:17.2.7-545.g42608bb1.el9.x86_64 is already installed. 2024-03-04T13:27:24.664 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2024-03-04T13:27:24.665 INFO:teuthology.orchestra.run.smithi169.stdout:Nothing to do. 2024-03-04T13:27:24.665 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2024-03-04T13:27:24.732 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install librbd1 2024-03-04T13:27:25.534 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:02:56 ago on Mon 04 Mar 2024 01:24:29 PM UTC. 2024-03-04T13:27:25.642 INFO:teuthology.orchestra.run.smithi169.stdout:Package librbd1-2:17.2.7-545.g42608bb1.el9.x86_64 is already installed. 2024-03-04T13:27:25.698 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2024-03-04T13:27:25.699 INFO:teuthology.orchestra.run.smithi169.stdout:Nothing to do. 2024-03-04T13:27:25.699 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2024-03-04T13:27:25.764 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install python3-rados 2024-03-04T13:27:25.849 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-2:17.2.7-545.g42608bb1.el9.x86_64 1/33 2024-03-04T13:27:25.849 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mds-2:17.2.7-545.g42608bb1.el9.x86_64 2/33 2024-03-04T13:27:25.849 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-2:17.2.7-545.g42608bb1.el9.x86_64 3/33 2024-03-04T13:27:25.850 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mon-2:17.2.7-545.g42608bb1.el9.x86_64 4/33 2024-03-04T13:27:25.850 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-osd-2:17.2.7-545.g42608bb1.el9.x86_64 5/33 2024-03-04T13:27:25.850 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-545.g42608bb1.el9.n 6/33 2024-03-04T13:27:25.850 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 7/33 2024-03-04T13:27:25.851 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 8/33 2024-03-04T13:27:25.851 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 9/33 2024-03-04T13:27:25.851 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 10/33 2024-03-04T13:27:25.851 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 11/33 2024-03-04T13:27:25.851 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 12/33 2024-03-04T13:27:25.852 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-ply-3.11-14.el9.noarch 13/33 2024-03-04T13:27:25.852 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 14/33 2024-03-04T13:27:25.852 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 15/33 2024-03-04T13:27:25.852 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 16/33 2024-03-04T13:27:25.853 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-03-04T13:27:25.853 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-03-04T13:27:25.853 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-03-04T13:27:25.853 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-03-04T13:27:25.854 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-03-04T13:27:25.854 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-03-04T13:27:25.854 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-03-04T13:27:25.854 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-03-04T13:27:25.854 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-03-04T13:27:25.855 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-03-04T13:27:25.855 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-03-04T13:27:25.855 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-03-04T13:27:25.855 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-03-04T13:27:25.855 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-03-04T13:27:25.856 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 31/33 2024-03-04T13:27:25.856 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-03-04T13:27:26.580 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:02:57 ago on Mon 04 Mar 2024 01:24:29 PM UTC. 2024-03-04T13:27:26.704 INFO:teuthology.orchestra.run.smithi169.stdout:Package python3-rados-2:17.2.7-545.g42608bb1.el9.x86_64 is already installed. 2024-03-04T13:27:26.763 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2024-03-04T13:27:26.765 INFO:teuthology.orchestra.run.smithi169.stdout:Nothing to do. 2024-03-04T13:27:26.765 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2024-03-04T13:27:26.791 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-03-04T13:27:26.791 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:27:26.792 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-03-04T13:27:26.792 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-2:17.2.7-545.g42608bb1.el9.x86_64 2024-03-04T13:27:26.792 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mds-2:17.2.7-545.g42608bb1.el9.x86_64 2024-03-04T13:27:26.792 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-2:17.2.7-545.g42608bb1.el9.x86_64 2024-03-04T13:27:26.792 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-modules-core-2:17.2.7-545.g42608bb1.el9.noarch 2024-03-04T13:27:26.792 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mon-2:17.2.7-545.g42608bb1.el9.x86_64 2024-03-04T13:27:26.792 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-osd-2:17.2.7-545.g42608bb1.el9.x86_64 2024-03-04T13:27:26.792 INFO:teuthology.orchestra.run.smithi107.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-04T13:27:26.792 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-04T13:27:26.792 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-04T13:27:26.792 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-04T13:27:26.792 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-04T13:27:26.792 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-04T13:27:26.793 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-04T13:27:26.793 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-04T13:27:26.793 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-04T13:27:26.793 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-04T13:27:26.793 INFO:teuthology.orchestra.run.smithi107.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-04T13:27:26.793 INFO:teuthology.orchestra.run.smithi107.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-04T13:27:26.793 INFO:teuthology.orchestra.run.smithi107.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-04T13:27:26.793 INFO:teuthology.orchestra.run.smithi107.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-04T13:27:26.793 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-04T13:27:26.793 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ply-3.11-14.el9.noarch 2024-03-04T13:27:26.793 INFO:teuthology.orchestra.run.smithi107.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-04T13:27:26.793 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-04T13:27:26.793 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-04T13:27:26.793 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-04T13:27:26.794 INFO:teuthology.orchestra.run.smithi107.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-04T13:27:26.794 INFO:teuthology.orchestra.run.smithi107.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-04T13:27:26.794 INFO:teuthology.orchestra.run.smithi107.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-04T13:27:26.794 INFO:teuthology.orchestra.run.smithi107.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-04T13:27:26.794 INFO:teuthology.orchestra.run.smithi107.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-04T13:27:26.794 INFO:teuthology.orchestra.run.smithi107.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-04T13:27:26.794 INFO:teuthology.orchestra.run.smithi107.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-04T13:27:26.794 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:27:26.794 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-03-04T13:27:26.834 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install python3-rgw 2024-03-04T13:27:27.295 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install ceph-base 2024-03-04T13:27:27.655 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:02:58 ago on Mon 04 Mar 2024 01:24:29 PM UTC. 2024-03-04T13:27:27.771 INFO:teuthology.orchestra.run.smithi169.stdout:Package python3-rgw-2:17.2.7-545.g42608bb1.el9.x86_64 is already installed. 2024-03-04T13:27:27.827 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2024-03-04T13:27:27.829 INFO:teuthology.orchestra.run.smithi169.stdout:Nothing to do. 2024-03-04T13:27:27.829 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2024-03-04T13:27:27.901 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install python3-cephfs 2024-03-04T13:27:28.126 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:02:29 ago on Mon 04 Mar 2024 01:24:59 PM UTC. 2024-03-04T13:27:28.247 INFO:teuthology.orchestra.run.smithi107.stdout:Package ceph-base-2:17.2.7-545.g42608bb1.el9.x86_64 is already installed. 2024-03-04T13:27:28.303 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-03-04T13:27:28.304 INFO:teuthology.orchestra.run.smithi107.stdout:Nothing to do. 2024-03-04T13:27:28.304 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-03-04T13:27:28.377 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install cephadm 2024-03-04T13:27:28.724 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:02:59 ago on Mon 04 Mar 2024 01:24:29 PM UTC. 2024-03-04T13:27:28.844 INFO:teuthology.orchestra.run.smithi169.stdout:Package python3-cephfs-2:17.2.7-545.g42608bb1.el9.x86_64 is already installed. 2024-03-04T13:27:28.900 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2024-03-04T13:27:28.901 INFO:teuthology.orchestra.run.smithi169.stdout:Nothing to do. 2024-03-04T13:27:28.902 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2024-03-04T13:27:28.966 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install python3-rbd 2024-03-04T13:27:29.199 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:02:30 ago on Mon 04 Mar 2024 01:24:59 PM UTC. 2024-03-04T13:27:29.375 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-03-04T13:27:29.375 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-03-04T13:27:29.375 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repository Size 2024-03-04T13:27:29.376 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-03-04T13:27:29.376 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-03-04T13:27:29.376 INFO:teuthology.orchestra.run.smithi107.stdout: cephadm noarch 2:17.2.7-545.g42608bb1.el9 ceph-noarch 80 k 2024-03-04T13:27:29.376 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:27:29.376 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-03-04T13:27:29.376 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-03-04T13:27:29.376 INFO:teuthology.orchestra.run.smithi107.stdout:Install 1 Package 2024-03-04T13:27:29.376 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:27:29.377 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 80 k 2024-03-04T13:27:29.377 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 364 k 2024-03-04T13:27:29.377 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-03-04T13:27:29.785 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:03:00 ago on Mon 04 Mar 2024 01:24:29 PM UTC. 2024-03-04T13:27:29.794 INFO:teuthology.orchestra.run.smithi107.stdout:cephadm-17.2.7-545.g42608bb1.el9.noarch.rpm 193 kB/s | 80 kB 00:00 2024-03-04T13:27:29.795 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-03-04T13:27:29.796 INFO:teuthology.orchestra.run.smithi107.stdout:Total 192 kB/s | 80 kB 00:00 2024-03-04T13:27:29.797 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-03-04T13:27:29.813 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-03-04T13:27:29.813 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-03-04T13:27:29.820 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-03-04T13:27:29.821 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-03-04T13:27:29.903 INFO:teuthology.orchestra.run.smithi169.stdout:Package python3-rbd-2:17.2.7-545.g42608bb1.el9.x86_64 is already installed. 2024-03-04T13:27:29.949 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-03-04T13:27:29.959 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2024-03-04T13:27:29.960 INFO:teuthology.orchestra.run.smithi169.stdout:Nothing to do. 2024-03-04T13:27:29.960 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2024-03-04T13:27:30.024 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install rbd-fuse 2024-03-04T13:27:30.601 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: cephadm-2:17.2.7-545.g42608bb1.el9.noarch 1/1 2024-03-04T13:27:30.752 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : cephadm-2:17.2.7-545.g42608bb1.el9.noarch 1/1 2024-03-04T13:27:30.818 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:03:01 ago on Mon 04 Mar 2024 01:24:29 PM UTC. 2024-03-04T13:27:30.986 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2024-03-04T13:27:30.987 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-03-04T13:27:30.987 INFO:teuthology.orchestra.run.smithi169.stdout: Package Architecture Version Repository Size 2024-03-04T13:27:30.987 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-03-04T13:27:30.987 INFO:teuthology.orchestra.run.smithi169.stdout:Installing: 2024-03-04T13:27:30.987 INFO:teuthology.orchestra.run.smithi169.stdout: rbd-fuse x86_64 2:17.2.7-545.g42608bb1.el9 ceph 81 k 2024-03-04T13:27:30.988 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-03-04T13:27:30.988 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction Summary 2024-03-04T13:27:30.988 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-03-04T13:27:30.988 INFO:teuthology.orchestra.run.smithi169.stdout:Install 1 Package 2024-03-04T13:27:30.988 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-03-04T13:27:30.988 INFO:teuthology.orchestra.run.smithi169.stdout:Total download size: 81 k 2024-03-04T13:27:30.988 INFO:teuthology.orchestra.run.smithi169.stdout:Installed size: 222 k 2024-03-04T13:27:30.988 INFO:teuthology.orchestra.run.smithi169.stdout:Downloading Packages: 2024-03-04T13:27:31.258 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: cephadm-2:17.2.7-545.g42608bb1.el9.noarch 1/1 2024-03-04T13:27:31.425 INFO:teuthology.orchestra.run.smithi169.stdout:rbd-fuse-17.2.7-545.g42608bb1.el9.x86_64.rpm 187 kB/s | 81 kB 00:00 2024-03-04T13:27:31.426 INFO:teuthology.orchestra.run.smithi169.stdout:-------------------------------------------------------------------------------- 2024-03-04T13:27:31.427 INFO:teuthology.orchestra.run.smithi169.stdout:Total 185 kB/s | 81 kB 00:00 2024-03-04T13:27:31.428 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction check 2024-03-04T13:27:31.450 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction check succeeded. 2024-03-04T13:27:31.450 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction test 2024-03-04T13:27:31.552 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction test succeeded. 2024-03-04T13:27:31.552 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction 2024-03-04T13:27:31.561 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : cephadm-2:17.2.7-545.g42608bb1.el9.noarch 1/1 2024-03-04T13:27:31.561 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:27:31.561 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-03-04T13:27:31.561 INFO:teuthology.orchestra.run.smithi107.stdout: cephadm-2:17.2.7-545.g42608bb1.el9.noarch 2024-03-04T13:27:31.561 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:27:31.561 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-03-04T13:27:31.737 INFO:teuthology.orchestra.run.smithi169.stdout: Preparing : 1/1 2024-03-04T13:27:31.825 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install ceph-immutable-object-cache 2024-03-04T13:27:31.858 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : rbd-fuse-2:17.2.7-545.g42608bb1.el9.x86_64 1/1 2024-03-04T13:27:32.411 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: rbd-fuse-2:17.2.7-545.g42608bb1.el9.x86_64 1/1 2024-03-04T13:27:32.671 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:02:33 ago on Mon 04 Mar 2024 01:24:59 PM UTC. 2024-03-04T13:27:32.764 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : rbd-fuse-2:17.2.7-545.g42608bb1.el9.x86_64 1/1 2024-03-04T13:27:32.764 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-03-04T13:27:32.764 INFO:teuthology.orchestra.run.smithi169.stdout:Installed: 2024-03-04T13:27:32.764 INFO:teuthology.orchestra.run.smithi169.stdout: rbd-fuse-2:17.2.7-545.g42608bb1.el9.x86_64 2024-03-04T13:27:32.764 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-03-04T13:27:32.764 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2024-03-04T13:27:32.851 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-03-04T13:27:32.852 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-03-04T13:27:32.852 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repo Size 2024-03-04T13:27:32.852 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-03-04T13:27:32.852 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-03-04T13:27:32.852 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-545.g42608bb1.el9 ceph 137 k 2024-03-04T13:27:32.852 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:27:32.852 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-03-04T13:27:32.852 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-03-04T13:27:32.852 INFO:teuthology.orchestra.run.smithi107.stdout:Install 1 Package 2024-03-04T13:27:32.852 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:27:32.853 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 137 k 2024-03-04T13:27:32.853 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 379 k 2024-03-04T13:27:32.853 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-03-04T13:27:33.076 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install rbd-mirror 2024-03-04T13:27:33.110 INFO:teuthology.orchestra.run.smithi107.stdout:ceph-immutable-object-cache-17.2.7-545.g42608bb 533 kB/s | 137 kB 00:00 2024-03-04T13:27:33.111 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-03-04T13:27:33.112 INFO:teuthology.orchestra.run.smithi107.stdout:Total 526 kB/s | 137 kB 00:00 2024-03-04T13:27:33.112 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-03-04T13:27:33.138 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-03-04T13:27:33.138 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-03-04T13:27:33.233 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-03-04T13:27:33.233 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-03-04T13:27:33.435 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-03-04T13:27:33.536 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-545.g42608bb1.e 1/1 2024-03-04T13:27:33.620 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-545.g42608bb1.e 1/1 2024-03-04T13:27:33.620 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-04T13:27:33.620 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-04T13:27:33.620 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:27:33.934 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:03:04 ago on Mon 04 Mar 2024 01:24:29 PM UTC. 2024-03-04T13:27:34.107 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2024-03-04T13:27:34.109 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-03-04T13:27:34.109 INFO:teuthology.orchestra.run.smithi169.stdout: Package Arch Version Repo Size 2024-03-04T13:27:34.109 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-03-04T13:27:34.109 INFO:teuthology.orchestra.run.smithi169.stdout:Installing: 2024-03-04T13:27:34.109 INFO:teuthology.orchestra.run.smithi169.stdout: rbd-mirror x86_64 2:17.2.7-545.g42608bb1.el9 ceph 2.9 M 2024-03-04T13:27:34.110 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-03-04T13:27:34.110 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction Summary 2024-03-04T13:27:34.110 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-03-04T13:27:34.110 INFO:teuthology.orchestra.run.smithi169.stdout:Install 1 Package 2024-03-04T13:27:34.110 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-03-04T13:27:34.110 INFO:teuthology.orchestra.run.smithi169.stdout:Total download size: 2.9 M 2024-03-04T13:27:34.110 INFO:teuthology.orchestra.run.smithi169.stdout:Installed size: 12 M 2024-03-04T13:27:34.110 INFO:teuthology.orchestra.run.smithi169.stdout:Downloading Packages: 2024-03-04T13:27:34.953 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-545.g42608bb1.e 1/1 2024-03-04T13:27:34.953 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:27:34.953 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-03-04T13:27:34.953 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-immutable-object-cache-2:17.2.7-545.g42608bb1.el9.x86_64 2024-03-04T13:27:34.953 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:27:34.953 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-03-04T13:27:35.232 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install ceph-mgr 2024-03-04T13:27:35.653 INFO:teuthology.orchestra.run.smithi169.stdout:rbd-mirror-17.2.7-545.g42608bb1.el9.x86_64.rpm 1.9 MB/s | 2.9 MB 00:01 2024-03-04T13:27:35.655 INFO:teuthology.orchestra.run.smithi169.stdout:-------------------------------------------------------------------------------- 2024-03-04T13:27:35.655 INFO:teuthology.orchestra.run.smithi169.stdout:Total 1.9 MB/s | 2.9 MB 00:01 2024-03-04T13:27:35.656 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction check 2024-03-04T13:27:35.687 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction check succeeded. 2024-03-04T13:27:35.687 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction test 2024-03-04T13:27:35.800 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction test succeeded. 2024-03-04T13:27:35.800 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction 2024-03-04T13:27:36.072 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:02:37 ago on Mon 04 Mar 2024 01:24:59 PM UTC. 2024-03-04T13:27:36.192 INFO:teuthology.orchestra.run.smithi107.stdout:Package ceph-mgr-2:17.2.7-545.g42608bb1.el9.x86_64 is already installed. 2024-03-04T13:27:36.247 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-03-04T13:27:36.248 INFO:teuthology.orchestra.run.smithi107.stdout:Nothing to do. 2024-03-04T13:27:36.248 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-03-04T13:27:36.309 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install ceph-mgr-dashboard 2024-03-04T13:27:36.312 INFO:teuthology.orchestra.run.smithi169.stdout: Preparing : 1/1 2024-03-04T13:27:36.387 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : rbd-mirror-2:17.2.7-545.g42608bb1.el9.x86_64 1/1 2024-03-04T13:27:36.463 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: rbd-mirror-2:17.2.7-545.g42608bb1.el9.x86_64 1/1 2024-03-04T13:27:36.463 INFO:teuthology.orchestra.run.smithi169.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-04T13:27:36.463 INFO:teuthology.orchestra.run.smithi169.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-04T13:27:36.464 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-04T13:27:36.464 INFO:teuthology.orchestra.run.smithi169.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-04T13:27:36.464 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-03-04T13:27:37.116 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:02:38 ago on Mon 04 Mar 2024 01:24:59 PM UTC. 2024-03-04T13:27:37.290 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-03-04T13:27:37.291 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-03-04T13:27:37.291 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repository Size 2024-03-04T13:27:37.291 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-03-04T13:27:37.291 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-03-04T13:27:37.291 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-dashboard noarch 2:17.2.7-545.g42608bb1.el9 ceph-noarch 1.7 M 2024-03-04T13:27:37.291 INFO:teuthology.orchestra.run.smithi107.stdout:Installing dependencies: 2024-03-04T13:27:37.291 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-grafana-dashboards noarch 2:17.2.7-545.g42608bb1.el9 ceph-noarch 23 k 2024-03-04T13:27:37.291 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-prometheus-alerts noarch 2:17.2.7-545.g42608bb1.el9 ceph-noarch 15 k 2024-03-04T13:27:37.291 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-03-04T13:27:37.291 INFO:teuthology.orchestra.run.smithi107.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-04T13:27:37.291 INFO:teuthology.orchestra.run.smithi107.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-04T13:27:37.292 INFO:teuthology.orchestra.run.smithi107.stdout:Installing weak dependencies: 2024-03-04T13:27:37.292 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-03-04T13:27:37.292 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:27:37.292 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-03-04T13:27:37.292 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-03-04T13:27:37.292 INFO:teuthology.orchestra.run.smithi107.stdout:Install 7 Packages 2024-03-04T13:27:37.292 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:27:37.293 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 2.0 M 2024-03-04T13:27:37.293 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 8.1 M 2024-03-04T13:27:37.293 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-03-04T13:27:37.706 INFO:teuthology.orchestra.run.smithi107.stdout:(1/7): ceph-grafana-dashboards-17.2.7-545.g4260 66 kB/s | 23 kB 00:00 2024-03-04T13:27:37.731 INFO:teuthology.orchestra.run.smithi107.stdout:(2/7): ceph-prometheus-alerts-17.2.7-545.g42608 39 kB/s | 15 kB 00:00 2024-03-04T13:27:37.823 INFO:teuthology.orchestra.run.smithi107.stdout:(3/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 78 kB/s | 9.0 kB 00:00 2024-03-04T13:27:37.875 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : rbd-mirror-2:17.2.7-545.g42608bb1.el9.x86_64 1/1 2024-03-04T13:27:37.875 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-03-04T13:27:37.875 INFO:teuthology.orchestra.run.smithi169.stdout:Installed: 2024-03-04T13:27:37.875 INFO:teuthology.orchestra.run.smithi169.stdout: rbd-mirror-2:17.2.7-545.g42608bb1.el9.x86_64 2024-03-04T13:27:37.875 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-03-04T13:27:37.876 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2024-03-04T13:27:37.908 INFO:teuthology.orchestra.run.smithi107.stdout:(4/7): ceph-mgr-dashboard-17.2.7-545.g42608bb1. 3.1 MB/s | 1.7 MB 00:00 2024-03-04T13:27:37.932 INFO:teuthology.orchestra.run.smithi107.stdout:(5/7): python3-jwt-2.4.0-1.el9.noarch.rpm 205 kB/s | 41 kB 00:00 2024-03-04T13:27:37.957 INFO:teuthology.orchestra.run.smithi107.stdout:(6/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 231 kB/s | 31 kB 00:00 2024-03-04T13:27:38.070 INFO:teuthology.orchestra.run.smithi107.stdout:(7/7): python3-routes-2.5.1-5.el9.noarch.rpm 1.1 MB/s | 188 kB 00:00 2024-03-04T13:27:38.074 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-03-04T13:27:38.075 INFO:teuthology.orchestra.run.smithi107.stdout:Total 2.5 MB/s | 2.0 MB 00:00 2024-03-04T13:27:38.075 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-03-04T13:27:38.101 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-03-04T13:27:38.102 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-03-04T13:27:38.173 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install rbd-nbd 2024-03-04T13:27:38.240 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-03-04T13:27:38.240 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-03-04T13:27:38.637 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-03-04T13:27:38.693 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-03-04T13:27:38.741 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-03-04T13:27:38.827 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-03-04T13:27:38.858 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-03-04T13:27:38.929 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-545.g42608bb1.el9.no 5/7 2024-03-04T13:27:39.021 INFO:teuthology.orchestra.run.smithi169.stdout:Last metadata expiration check: 0:03:10 ago on Mon 04 Mar 2024 01:24:29 PM UTC. 2024-03-04T13:27:39.192 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2024-03-04T13:27:39.192 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-03-04T13:27:39.192 INFO:teuthology.orchestra.run.smithi169.stdout: Package Architecture Version Repository Size 2024-03-04T13:27:39.192 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-03-04T13:27:39.192 INFO:teuthology.orchestra.run.smithi169.stdout:Installing: 2024-03-04T13:27:39.193 INFO:teuthology.orchestra.run.smithi169.stdout: rbd-nbd x86_64 2:17.2.7-545.g42608bb1.el9 ceph 164 k 2024-03-04T13:27:39.193 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-03-04T13:27:39.193 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction Summary 2024-03-04T13:27:39.193 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2024-03-04T13:27:39.193 INFO:teuthology.orchestra.run.smithi169.stdout:Install 1 Package 2024-03-04T13:27:39.193 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-03-04T13:27:39.193 INFO:teuthology.orchestra.run.smithi169.stdout:Total download size: 164 k 2024-03-04T13:27:39.194 INFO:teuthology.orchestra.run.smithi169.stdout:Installed size: 477 k 2024-03-04T13:27:39.194 INFO:teuthology.orchestra.run.smithi169.stdout:Downloading Packages: 2024-03-04T13:27:39.223 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-545.g42608bb1.el9.n 6/7 2024-03-04T13:27:39.262 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-545.g42608bb1.el9.noarch 7/7 2024-03-04T13:27:39.440 INFO:teuthology.orchestra.run.smithi169.stdout:rbd-nbd-17.2.7-545.g42608bb1.el9.x86_64.rpm 669 kB/s | 164 kB 00:00 2024-03-04T13:27:39.442 INFO:teuthology.orchestra.run.smithi169.stdout:-------------------------------------------------------------------------------- 2024-03-04T13:27:39.442 INFO:teuthology.orchestra.run.smithi169.stdout:Total 660 kB/s | 164 kB 00:00 2024-03-04T13:27:39.443 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction check 2024-03-04T13:27:39.469 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction check succeeded. 2024-03-04T13:27:39.469 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction test 2024-03-04T13:27:39.566 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction test succeeded. 2024-03-04T13:27:39.566 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction 2024-03-04T13:27:39.732 INFO:teuthology.orchestra.run.smithi169.stdout: Preparing : 1/1 2024-03-04T13:27:39.882 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : rbd-nbd-2:17.2.7-545.g42608bb1.el9.x86_64 1/1 2024-03-04T13:27:39.897 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-545.g42608bb1.el9.noarch 7/7 2024-03-04T13:27:39.897 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-545.g42608bb1.el9.n 1/7 2024-03-04T13:27:39.897 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-545.g42608bb1.el9.noarch 2/7 2024-03-04T13:27:39.897 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-545.g42608bb1.el9.no 3/7 2024-03-04T13:27:39.897 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-03-04T13:27:39.897 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-03-04T13:27:39.898 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-03-04T13:27:40.290 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-03-04T13:27:40.290 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:27:40.290 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-03-04T13:27:40.290 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-grafana-dashboards-2:17.2.7-545.g42608bb1.el9.noarch 2024-03-04T13:27:40.290 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-dashboard-2:17.2.7-545.g42608bb1.el9.noarch 2024-03-04T13:27:40.290 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-prometheus-alerts-2:17.2.7-545.g42608bb1.el9.noarch 2024-03-04T13:27:40.290 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-03-04T13:27:40.290 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-03-04T13:27:40.290 INFO:teuthology.orchestra.run.smithi107.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-04T13:27:40.291 INFO:teuthology.orchestra.run.smithi107.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-04T13:27:40.291 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:27:40.291 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-03-04T13:27:40.400 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: rbd-nbd-2:17.2.7-545.g42608bb1.el9.x86_64 1/1 2024-03-04T13:27:40.697 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install ceph-mgr-diskprediction-local 2024-03-04T13:27:40.720 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : rbd-nbd-2:17.2.7-545.g42608bb1.el9.x86_64 1/1 2024-03-04T13:27:40.720 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-03-04T13:27:40.720 INFO:teuthology.orchestra.run.smithi169.stdout:Installed: 2024-03-04T13:27:40.720 INFO:teuthology.orchestra.run.smithi169.stdout: rbd-nbd-2:17.2.7-545.g42608bb1.el9.x86_64 2024-03-04T13:27:40.720 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-03-04T13:27:40.720 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2024-03-04T13:27:41.032 DEBUG:teuthology.parallel:result is None 2024-03-04T13:27:41.543 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:02:42 ago on Mon 04 Mar 2024 01:24:59 PM UTC. 2024-03-04T13:27:41.731 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-03-04T13:27:41.732 INFO:teuthology.orchestra.run.smithi107.stdout:=========================================================================================== 2024-03-04T13:27:41.735 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repository Size 2024-03-04T13:27:41.736 INFO:teuthology.orchestra.run.smithi107.stdout:=========================================================================================== 2024-03-04T13:27:41.736 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-03-04T13:27:41.736 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-545.g42608bb1.el9 ceph-noarch 7.4 M 2024-03-04T13:27:41.736 INFO:teuthology.orchestra.run.smithi107.stdout:Installing dependencies: 2024-03-04T13:27:41.736 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-04T13:27:41.736 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-04T13:27:41.736 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-04T13:27:41.736 INFO:teuthology.orchestra.run.smithi107.stdout: libgfortran x86_64 11.4.1-3.el9 CentOS-BaseOS 802 k 2024-03-04T13:27:41.736 INFO:teuthology.orchestra.run.smithi107.stdout: libquadmath x86_64 11.4.1-3.el9 CentOS-BaseOS 194 k 2024-03-04T13:27:41.736 INFO:teuthology.orchestra.run.smithi107.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-04T13:27:41.736 INFO:teuthology.orchestra.run.smithi107.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-04T13:27:41.737 INFO:teuthology.orchestra.run.smithi107.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-04T13:27:41.737 INFO:teuthology.orchestra.run.smithi107.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-04T13:27:41.737 INFO:teuthology.orchestra.run.smithi107.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-04T13:27:41.737 INFO:teuthology.orchestra.run.smithi107.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-04T13:27:41.737 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:27:41.737 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-03-04T13:27:41.737 INFO:teuthology.orchestra.run.smithi107.stdout:=========================================================================================== 2024-03-04T13:27:41.737 INFO:teuthology.orchestra.run.smithi107.stdout:Install 12 Packages 2024-03-04T13:27:41.737 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:27:41.738 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 37 M 2024-03-04T13:27:41.738 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 209 M 2024-03-04T13:27:41.738 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-03-04T13:27:42.049 INFO:teuthology.orchestra.run.smithi107.stdout:(1/12): flexiblas-3.0.4-8.el9.x86_64.rpm 129 kB/s | 32 kB 00:00 2024-03-04T13:27:42.116 INFO:teuthology.orchestra.run.smithi107.stdout:(2/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 260 kB/s | 17 kB 00:00 2024-03-04T13:27:42.231 INFO:teuthology.orchestra.run.smithi107.stdout:(3/12): openblas-0.3.21-2.el9.x86_64.rpm 305 kB/s | 34 kB 00:00 2024-03-04T13:27:42.317 INFO:teuthology.orchestra.run.smithi107.stdout:(4/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 5.8 MB/s | 3.0 MB 00:00 2024-03-04T13:27:42.852 INFO:teuthology.orchestra.run.smithi107.stdout:(5/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 7.8 MB/s | 4.7 MB 00:00 2024-03-04T13:27:42.951 INFO:teuthology.orchestra.run.smithi107.stdout:(6/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 7.6 MB/s | 4.8 MB 00:00 2024-03-04T13:27:43.018 INFO:teuthology.orchestra.run.smithi107.stdout:(7/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 3.2 MB/s | 217 kB 00:00 2024-03-04T13:27:43.593 INFO:teuthology.orchestra.run.smithi107.stdout:(8/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 24 MB/s | 16 MB 00:00 2024-03-04T13:27:43.635 INFO:teuthology.orchestra.run.smithi107.stdout:(9/12): libgfortran-11.4.1-3.el9.x86_64.rpm 1.3 MB/s | 802 kB 00:00 2024-03-04T13:27:43.669 INFO:teuthology.orchestra.run.smithi107.stdout:(10/12): libquadmath-11.4.1-3.el9.x86_64.rpm 2.6 MB/s | 194 kB 00:00 2024-03-04T13:27:43.888 INFO:teuthology.orchestra.run.smithi107.stdout:(11/12): python3-devel-3.9.17-2.el9.x86_64.rpm 992 kB/s | 245 kB 00:00 2024-03-04T13:27:52.443 INFO:teuthology.orchestra.run.smithi107.stdout:(12/12): ceph-mgr-diskprediction-local-17.2.7-5 713 kB/s | 7.4 MB 00:10 2024-03-04T13:27:52.449 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-03-04T13:27:52.450 INFO:teuthology.orchestra.run.smithi107.stdout:Total 3.5 MB/s | 37 MB 00:10 2024-03-04T13:27:52.498 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-03-04T13:27:52.562 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-03-04T13:27:52.562 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-03-04T13:27:53.126 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-03-04T13:27:53.127 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-03-04T13:27:53.964 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-03-04T13:27:54.047 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 1/12 2024-03-04T13:27:54.082 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-03-04T13:27:54.226 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 3/12 2024-03-04T13:27:54.277 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 4/12 2024-03-04T13:27:54.539 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-03-04T13:27:54.574 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-03-04T13:27:54.697 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-03-04T13:27:55.523 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-03-04T13:27:55.660 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-03-04T13:27:57.185 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-03-04T13:27:58.362 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-03-04T13:27:58.399 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-545.g42608b 12/12 2024-03-04T13:28:01.055 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-545.g42608b 12/12 2024-03-04T13:28:01.055 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-545.g42608b 1/12 2024-03-04T13:28:01.055 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 2/12 2024-03-04T13:28:01.055 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 3/12 2024-03-04T13:28:01.055 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 4/12 2024-03-04T13:28:01.055 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 5/12 2024-03-04T13:28:01.055 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-03-04T13:28:01.055 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 7/12 2024-03-04T13:28:01.055 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 8/12 2024-03-04T13:28:01.055 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 9/12 2024-03-04T13:28:01.056 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 10/12 2024-03-04T13:28:01.056 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 11/12 2024-03-04T13:28:01.523 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 12/12 2024-03-04T13:28:01.524 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:28:01.524 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-03-04T13:28:01.524 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-diskprediction-local-2:17.2.7-545.g42608bb1.el9.noarch 2024-03-04T13:28:01.524 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-04T13:28:01.524 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-04T13:28:01.524 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-04T13:28:01.524 INFO:teuthology.orchestra.run.smithi107.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-04T13:28:01.525 INFO:teuthology.orchestra.run.smithi107.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-04T13:28:01.525 INFO:teuthology.orchestra.run.smithi107.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-04T13:28:01.525 INFO:teuthology.orchestra.run.smithi107.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-04T13:28:01.525 INFO:teuthology.orchestra.run.smithi107.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-04T13:28:01.525 INFO:teuthology.orchestra.run.smithi107.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-04T13:28:01.525 INFO:teuthology.orchestra.run.smithi107.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-04T13:28:01.525 INFO:teuthology.orchestra.run.smithi107.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-04T13:28:01.525 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:28:01.525 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-03-04T13:28:02.038 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install ceph-mgr-rook 2024-03-04T13:28:02.898 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:03:03 ago on Mon 04 Mar 2024 01:24:59 PM UTC. 2024-03-04T13:28:03.078 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-03-04T13:28:03.080 INFO:teuthology.orchestra.run.smithi107.stdout:======================================================================================= 2024-03-04T13:28:03.080 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repository Size 2024-03-04T13:28:03.080 INFO:teuthology.orchestra.run.smithi107.stdout:======================================================================================= 2024-03-04T13:28:03.080 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-03-04T13:28:03.080 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-rook noarch 2:17.2.7-545.g42608bb1.el9 ceph-noarch 47 k 2024-03-04T13:28:03.080 INFO:teuthology.orchestra.run.smithi107.stdout:Installing dependencies: 2024-03-04T13:28:03.080 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-04T13:28:03.080 INFO:teuthology.orchestra.run.smithi107.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-04T13:28:03.080 INFO:teuthology.orchestra.run.smithi107.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-03-04T13:28:03.081 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-04T13:28:03.081 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-04T13:28:03.081 INFO:teuthology.orchestra.run.smithi107.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-04T13:28:03.081 INFO:teuthology.orchestra.run.smithi107.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-04T13:28:03.081 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-04T13:28:03.081 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-04T13:28:03.081 INFO:teuthology.orchestra.run.smithi107.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-04T13:28:03.081 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-04T13:28:03.081 INFO:teuthology.orchestra.run.smithi107.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-04T13:28:03.081 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:28:03.081 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-03-04T13:28:03.081 INFO:teuthology.orchestra.run.smithi107.stdout:======================================================================================= 2024-03-04T13:28:03.081 INFO:teuthology.orchestra.run.smithi107.stdout:Install 13 Packages 2024-03-04T13:28:03.081 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:28:03.082 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 2.0 M 2024-03-04T13:28:03.082 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 26 M 2024-03-04T13:28:03.082 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-03-04T13:28:03.343 INFO:teuthology.orchestra.run.smithi107.stdout:(1/13): python3-jsonpointer-2.0-4.el9.noarch.rp 90 kB/s | 18 kB 00:00 2024-03-04T13:28:03.360 INFO:teuthology.orchestra.run.smithi107.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 112 kB/s | 24 kB 00:00 2024-03-04T13:28:03.494 INFO:teuthology.orchestra.run.smithi107.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.2 MB/s | 174 kB 00:00 2024-03-04T13:28:03.519 INFO:teuthology.orchestra.run.smithi107.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 835 kB/s | 132 kB 00:00 2024-03-04T13:28:03.569 INFO:teuthology.orchestra.run.smithi107.stdout:(5/13): python3-pyasn1-modules-0.4.8-6.el9.noar 2.8 MB/s | 211 kB 00:00 2024-03-04T13:28:03.594 INFO:teuthology.orchestra.run.smithi107.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 572 kB/s | 43 kB 00:00 2024-03-04T13:28:03.620 INFO:teuthology.orchestra.run.smithi107.stdout:(7/13): ceph-mgr-rook-17.2.7-545.g42608bb1.el9. 99 kB/s | 47 kB 00:00 2024-03-04T13:28:03.970 INFO:teuthology.orchestra.run.smithi107.stdout:(8/13): python3-certifi-2021.10.8-2.el9.noarch. 40 kB/s | 15 kB 00:00 2024-03-04T13:28:03.995 INFO:teuthology.orchestra.run.smithi107.stdout:(9/13): python3-cachetools-4.2.4-1.el9.noarch.r 76 kB/s | 32 kB 00:00 2024-03-04T13:28:04.129 INFO:teuthology.orchestra.run.smithi107.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 445 kB/s | 59 kB 00:00 2024-03-04T13:28:04.162 INFO:teuthology.orchestra.run.smithi107.stdout:(11/13): python3-google-auth-2.28.0-1.el9.noarc 374 kB/s | 203 kB 00:00 2024-03-04T13:28:04.238 INFO:teuthology.orchestra.run.smithi107.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 831 kB/s | 90 kB 00:00 2024-03-04T13:28:04.347 INFO:teuthology.orchestra.run.smithi107.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 2.7 MB/s | 1.0 MB 00:00 2024-03-04T13:28:04.357 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-03-04T13:28:04.357 INFO:teuthology.orchestra.run.smithi107.stdout:Total 1.6 MB/s | 2.0 MB 00:01 2024-03-04T13:28:04.358 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-03-04T13:28:04.397 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-03-04T13:28:04.397 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-03-04T13:28:04.549 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-03-04T13:28:04.550 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-03-04T13:28:05.222 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-03-04T13:28:05.433 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-03-04T13:28:05.514 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-03-04T13:28:05.579 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-03-04T13:28:05.618 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-03-04T13:28:05.665 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-03-04T13:28:05.791 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-03-04T13:28:05.918 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 7/13 2024-03-04T13:28:05.982 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-03-04T13:28:06.903 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-03-04T13:28:06.949 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-03-04T13:28:06.999 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-03-04T13:28:07.052 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-03-04T13:28:07.076 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-rook-2:17.2.7-545.g42608bb1.el9.noarch 13/13 2024-03-04T13:28:08.014 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-545.g42608bb1.el9.noarch 13/13 2024-03-04T13:28:08.014 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-rook-2:17.2.7-545.g42608bb1.el9.noarch 1/13 2024-03-04T13:28:08.014 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-03-04T13:28:08.014 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-03-04T13:28:08.015 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-03-04T13:28:08.015 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-03-04T13:28:08.015 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-03-04T13:28:08.015 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-03-04T13:28:08.015 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-03-04T13:28:08.015 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-03-04T13:28:08.016 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 10/13 2024-03-04T13:28:08.016 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-03-04T13:28:08.016 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-03-04T13:28:08.472 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-03-04T13:28:08.472 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:28:08.472 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-03-04T13:28:08.472 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-rook-2:17.2.7-545.g42608bb1.el9.noarch 2024-03-04T13:28:08.473 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-04T13:28:08.473 INFO:teuthology.orchestra.run.smithi107.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-04T13:28:08.473 INFO:teuthology.orchestra.run.smithi107.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-03-04T13:28:08.473 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-04T13:28:08.473 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-04T13:28:08.473 INFO:teuthology.orchestra.run.smithi107.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-04T13:28:08.473 INFO:teuthology.orchestra.run.smithi107.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-04T13:28:08.473 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-04T13:28:08.473 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-04T13:28:08.473 INFO:teuthology.orchestra.run.smithi107.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-04T13:28:08.473 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-04T13:28:08.473 INFO:teuthology.orchestra.run.smithi107.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-04T13:28:08.473 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:28:08.473 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-03-04T13:28:08.853 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install ceph-mgr-cephadm 2024-03-04T13:28:09.680 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:03:10 ago on Mon 04 Mar 2024 01:24:59 PM UTC. 2024-03-04T13:28:09.859 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-03-04T13:28:09.859 INFO:teuthology.orchestra.run.smithi107.stdout:======================================================================================= 2024-03-04T13:28:09.859 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repository Size 2024-03-04T13:28:09.859 INFO:teuthology.orchestra.run.smithi107.stdout:======================================================================================= 2024-03-04T13:28:09.860 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-03-04T13:28:09.860 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-cephadm noarch 2:17.2.7-545.g42608bb1.el9 ceph-noarch 121 k 2024-03-04T13:28:09.860 INFO:teuthology.orchestra.run.smithi107.stdout:Installing dependencies: 2024-03-04T13:28:09.860 INFO:teuthology.orchestra.run.smithi107.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-03-04T13:28:09.860 INFO:teuthology.orchestra.run.smithi107.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-04T13:28:09.860 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jinja2 noarch 2.11.3-5.el9 CentOS-AppStream 227 k 2024-03-04T13:28:09.860 INFO:teuthology.orchestra.run.smithi107.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-04T13:28:09.860 INFO:teuthology.orchestra.run.smithi107.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-04T13:28:09.860 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:28:09.860 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-03-04T13:28:09.860 INFO:teuthology.orchestra.run.smithi107.stdout:======================================================================================= 2024-03-04T13:28:09.860 INFO:teuthology.orchestra.run.smithi107.stdout:Install 6 Packages 2024-03-04T13:28:09.860 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:28:09.861 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 6.8 M 2024-03-04T13:28:09.861 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 33 M 2024-03-04T13:28:09.861 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-03-04T13:28:10.508 INFO:teuthology.orchestra.run.smithi107.stdout:(1/6): ceph-mgr-cephadm-17.2.7-545.g42608bb1.el 314 kB/s | 121 kB 00:00 2024-03-04T13:28:10.542 INFO:teuthology.orchestra.run.smithi107.stdout:(2/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 540 kB/s | 227 kB 00:00 2024-03-04T13:28:10.659 INFO:teuthology.orchestra.run.smithi107.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 509 kB/s | 58 kB 00:00 2024-03-04T13:28:10.743 INFO:teuthology.orchestra.run.smithi107.stdout:(4/6): python3-typing-extensions-4.4.0-2.el9.no 821 kB/s | 51 kB 00:00 2024-03-04T13:28:10.876 INFO:teuthology.orchestra.run.smithi107.stdout:(5/6): python3-babel-2.9.1-2.el9.noarch.rpm 7.7 MB/s | 5.8 MB 00:00 2024-03-04T13:28:10.936 INFO:teuthology.orchestra.run.smithi107.stdout:(6/6): python3-asyncssh-2.13.2-4.el9.noarch.rpm 1.3 MB/s | 548 kB 00:00 2024-03-04T13:28:10.942 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-03-04T13:28:10.943 INFO:teuthology.orchestra.run.smithi107.stdout:Total 6.2 MB/s | 6.8 MB 00:01 2024-03-04T13:28:10.943 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-03-04T13:28:10.973 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-03-04T13:28:10.973 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-03-04T13:28:11.117 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-03-04T13:28:11.118 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-03-04T13:28:11.502 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-03-04T13:28:11.732 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-03-04T13:28:11.809 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 2/6 2024-03-04T13:28:12.512 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-03-04T13:28:12.647 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-03-04T13:28:12.718 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-03-04T13:28:12.757 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-545.g42608bb1.el9.noarch 6/6 2024-03-04T13:28:13.679 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-545.g42608bb1.el9.noarch 6/6 2024-03-04T13:28:13.679 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-545.g42608bb1.el9.noarch 1/6 2024-03-04T13:28:13.679 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-03-04T13:28:13.679 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-03-04T13:28:13.680 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 4/6 2024-03-04T13:28:13.680 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-03-04T13:28:14.042 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-03-04T13:28:14.042 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:28:14.042 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-03-04T13:28:14.042 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-cephadm-2:17.2.7-545.g42608bb1.el9.noarch 2024-03-04T13:28:14.042 INFO:teuthology.orchestra.run.smithi107.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-03-04T13:28:14.042 INFO:teuthology.orchestra.run.smithi107.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-04T13:28:14.042 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-04T13:28:14.042 INFO:teuthology.orchestra.run.smithi107.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-04T13:28:14.042 INFO:teuthology.orchestra.run.smithi107.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-04T13:28:14.042 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:28:14.042 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-03-04T13:28:14.431 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install ceph-fuse 2024-03-04T13:28:15.256 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:03:16 ago on Mon 04 Mar 2024 01:24:59 PM UTC. 2024-03-04T13:28:15.425 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-03-04T13:28:15.425 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-03-04T13:28:15.425 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repository Size 2024-03-04T13:28:15.426 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-03-04T13:28:15.426 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-03-04T13:28:15.426 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-fuse x86_64 2:17.2.7-545.g42608bb1.el9 ceph 812 k 2024-03-04T13:28:15.426 INFO:teuthology.orchestra.run.smithi107.stdout:Installing dependencies: 2024-03-04T13:28:15.426 INFO:teuthology.orchestra.run.smithi107.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-04T13:28:15.426 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:28:15.426 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-03-04T13:28:15.426 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-03-04T13:28:15.426 INFO:teuthology.orchestra.run.smithi107.stdout:Install 2 Packages 2024-03-04T13:28:15.426 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:28:15.427 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 891 k 2024-03-04T13:28:15.427 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 2.6 M 2024-03-04T13:28:15.427 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-03-04T13:28:15.688 INFO:teuthology.orchestra.run.smithi107.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 303 kB/s | 79 kB 00:00 2024-03-04T13:28:16.763 INFO:teuthology.orchestra.run.smithi107.stdout:(2/2): ceph-fuse-17.2.7-545.g42608bb1.el9.x86_6 608 kB/s | 812 kB 00:01 2024-03-04T13:28:16.766 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-03-04T13:28:16.767 INFO:teuthology.orchestra.run.smithi107.stdout:Total 665 kB/s | 891 kB 00:01 2024-03-04T13:28:16.768 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-03-04T13:28:16.835 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-03-04T13:28:16.835 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-03-04T13:28:16.933 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-03-04T13:28:16.933 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-03-04T13:28:17.200 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-03-04T13:28:17.390 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-03-04T13:28:17.471 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-fuse-2:17.2.7-545.g42608bb1.el9.x86_64 2/2 2024-03-04T13:28:18.449 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-fuse-2:17.2.7-545.g42608bb1.el9.x86_64 2/2 2024-03-04T13:28:18.449 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-fuse-2:17.2.7-545.g42608bb1.el9.x86_64 1/2 2024-03-04T13:28:18.820 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-03-04T13:28:18.820 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:28:18.820 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-03-04T13:28:18.820 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-fuse-2:17.2.7-545.g42608bb1.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-03-04T13:28:18.820 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:28:18.820 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-03-04T13:28:19.155 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install librados-devel 2024-03-04T13:28:20.004 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:03:21 ago on Mon 04 Mar 2024 01:24:59 PM UTC. 2024-03-04T13:28:20.178 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-03-04T13:28:20.179 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-03-04T13:28:20.179 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repo Size 2024-03-04T13:28:20.179 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-03-04T13:28:20.179 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-03-04T13:28:20.179 INFO:teuthology.orchestra.run.smithi107.stdout: librados-devel x86_64 2:17.2.7-545.g42608bb1.el9 ceph 126 k 2024-03-04T13:28:20.180 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:28:20.180 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-03-04T13:28:20.180 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-03-04T13:28:20.180 INFO:teuthology.orchestra.run.smithi107.stdout:Install 1 Package 2024-03-04T13:28:20.180 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:28:20.180 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 126 k 2024-03-04T13:28:20.181 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 448 k 2024-03-04T13:28:20.181 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-03-04T13:28:20.412 INFO:teuthology.orchestra.run.smithi107.stdout:librados-devel-17.2.7-545.g42608bb1.el9.x86_64. 552 kB/s | 126 kB 00:00 2024-03-04T13:28:20.412 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-03-04T13:28:20.412 INFO:teuthology.orchestra.run.smithi107.stdout:Total 543 kB/s | 126 kB 00:00 2024-03-04T13:28:20.413 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-03-04T13:28:20.430 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-03-04T13:28:20.431 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-03-04T13:28:20.532 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-03-04T13:28:20.532 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-03-04T13:28:20.713 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-03-04T13:28:20.843 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : librados-devel-2:17.2.7-545.g42608bb1.el9.x86_64 1/1 2024-03-04T13:28:21.480 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librados-devel-2:17.2.7-545.g42608bb1.el9.x86_64 1/1 2024-03-04T13:28:21.813 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librados-devel-2:17.2.7-545.g42608bb1.el9.x86_64 1/1 2024-03-04T13:28:21.814 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:28:21.814 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-03-04T13:28:21.814 INFO:teuthology.orchestra.run.smithi107.stdout: librados-devel-2:17.2.7-545.g42608bb1.el9.x86_64 2024-03-04T13:28:21.814 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:28:21.814 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-03-04T13:28:22.194 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install libcephfs2 2024-03-04T13:28:23.038 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:03:24 ago on Mon 04 Mar 2024 01:24:59 PM UTC. 2024-03-04T13:28:23.158 INFO:teuthology.orchestra.run.smithi107.stdout:Package libcephfs2-2:17.2.7-545.g42608bb1.el9.x86_64 is already installed. 2024-03-04T13:28:23.214 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-03-04T13:28:23.215 INFO:teuthology.orchestra.run.smithi107.stdout:Nothing to do. 2024-03-04T13:28:23.215 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-03-04T13:28:23.293 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install libcephfs-devel 2024-03-04T13:28:24.090 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:03:25 ago on Mon 04 Mar 2024 01:24:59 PM UTC. 2024-03-04T13:28:24.267 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-03-04T13:28:24.268 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-03-04T13:28:24.268 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repo Size 2024-03-04T13:28:24.268 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-03-04T13:28:24.268 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-03-04T13:28:24.268 INFO:teuthology.orchestra.run.smithi107.stdout: libcephfs-devel x86_64 2:17.2.7-545.g42608bb1.el9 ceph 25 k 2024-03-04T13:28:24.268 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:28:24.268 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-03-04T13:28:24.268 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-03-04T13:28:24.268 INFO:teuthology.orchestra.run.smithi107.stdout:Install 1 Package 2024-03-04T13:28:24.268 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:28:24.269 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 25 k 2024-03-04T13:28:24.269 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 110 k 2024-03-04T13:28:24.269 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-03-04T13:28:24.590 INFO:teuthology.orchestra.run.smithi107.stdout:libcephfs-devel-17.2.7-545.g42608bb1.el9.x86_64 78 kB/s | 25 kB 00:00 2024-03-04T13:28:24.591 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-03-04T13:28:24.592 INFO:teuthology.orchestra.run.smithi107.stdout:Total 77 kB/s | 25 kB 00:00 2024-03-04T13:28:24.592 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-03-04T13:28:24.604 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-03-04T13:28:24.604 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-03-04T13:28:24.612 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-03-04T13:28:24.612 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-03-04T13:28:24.753 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-03-04T13:28:24.933 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libcephfs-devel-2:17.2.7-545.g42608bb1.el9.x86_64 1/1 2024-03-04T13:28:25.360 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: libcephfs-devel-2:17.2.7-545.g42608bb1.el9.x86_64 1/1 2024-03-04T13:28:25.662 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libcephfs-devel-2:17.2.7-545.g42608bb1.el9.x86_64 1/1 2024-03-04T13:28:25.663 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:28:25.663 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-03-04T13:28:25.663 INFO:teuthology.orchestra.run.smithi107.stdout: libcephfs-devel-2:17.2.7-545.g42608bb1.el9.x86_64 2024-03-04T13:28:25.663 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:28:25.663 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-03-04T13:28:25.920 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install librados2 2024-03-04T13:28:26.746 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:03:27 ago on Mon 04 Mar 2024 01:24:59 PM UTC. 2024-03-04T13:28:26.858 INFO:teuthology.orchestra.run.smithi107.stdout:Package librados2-2:17.2.7-545.g42608bb1.el9.x86_64 is already installed. 2024-03-04T13:28:26.914 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-03-04T13:28:26.915 INFO:teuthology.orchestra.run.smithi107.stdout:Nothing to do. 2024-03-04T13:28:26.915 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-03-04T13:28:26.992 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install librbd1 2024-03-04T13:28:27.813 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:03:28 ago on Mon 04 Mar 2024 01:24:59 PM UTC. 2024-03-04T13:28:27.927 INFO:teuthology.orchestra.run.smithi107.stdout:Package librbd1-2:17.2.7-545.g42608bb1.el9.x86_64 is already installed. 2024-03-04T13:28:27.983 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-03-04T13:28:27.984 INFO:teuthology.orchestra.run.smithi107.stdout:Nothing to do. 2024-03-04T13:28:27.984 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-03-04T13:28:28.065 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install python3-rados 2024-03-04T13:28:28.853 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:03:29 ago on Mon 04 Mar 2024 01:24:59 PM UTC. 2024-03-04T13:28:28.970 INFO:teuthology.orchestra.run.smithi107.stdout:Package python3-rados-2:17.2.7-545.g42608bb1.el9.x86_64 is already installed. 2024-03-04T13:28:29.026 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-03-04T13:28:29.027 INFO:teuthology.orchestra.run.smithi107.stdout:Nothing to do. 2024-03-04T13:28:29.027 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-03-04T13:28:29.097 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install python3-rgw 2024-03-04T13:28:29.891 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:03:30 ago on Mon 04 Mar 2024 01:24:59 PM UTC. 2024-03-04T13:28:30.008 INFO:teuthology.orchestra.run.smithi107.stdout:Package python3-rgw-2:17.2.7-545.g42608bb1.el9.x86_64 is already installed. 2024-03-04T13:28:30.065 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-03-04T13:28:30.066 INFO:teuthology.orchestra.run.smithi107.stdout:Nothing to do. 2024-03-04T13:28:30.066 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-03-04T13:28:30.140 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install python3-cephfs 2024-03-04T13:28:30.949 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:03:31 ago on Mon 04 Mar 2024 01:24:59 PM UTC. 2024-03-04T13:28:31.072 INFO:teuthology.orchestra.run.smithi107.stdout:Package python3-cephfs-2:17.2.7-545.g42608bb1.el9.x86_64 is already installed. 2024-03-04T13:28:31.129 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-03-04T13:28:31.130 INFO:teuthology.orchestra.run.smithi107.stdout:Nothing to do. 2024-03-04T13:28:31.130 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-03-04T13:28:31.195 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install python3-rbd 2024-03-04T13:28:32.010 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:03:33 ago on Mon 04 Mar 2024 01:24:59 PM UTC. 2024-03-04T13:28:32.127 INFO:teuthology.orchestra.run.smithi107.stdout:Package python3-rbd-2:17.2.7-545.g42608bb1.el9.x86_64 is already installed. 2024-03-04T13:28:32.183 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-03-04T13:28:32.184 INFO:teuthology.orchestra.run.smithi107.stdout:Nothing to do. 2024-03-04T13:28:32.184 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-03-04T13:28:32.246 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install rbd-fuse 2024-03-04T13:28:33.042 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:03:34 ago on Mon 04 Mar 2024 01:24:59 PM UTC. 2024-03-04T13:28:33.211 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-03-04T13:28:33.212 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-03-04T13:28:33.212 INFO:teuthology.orchestra.run.smithi107.stdout: Package Architecture Version Repository Size 2024-03-04T13:28:33.212 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-03-04T13:28:33.212 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-03-04T13:28:33.213 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-fuse x86_64 2:17.2.7-545.g42608bb1.el9 ceph 81 k 2024-03-04T13:28:33.213 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:28:33.213 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-03-04T13:28:33.213 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-03-04T13:28:33.213 INFO:teuthology.orchestra.run.smithi107.stdout:Install 1 Package 2024-03-04T13:28:33.213 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:28:33.213 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 81 k 2024-03-04T13:28:33.213 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 222 k 2024-03-04T13:28:33.213 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-03-04T13:28:33.443 INFO:teuthology.orchestra.run.smithi107.stdout:rbd-fuse-17.2.7-545.g42608bb1.el9.x86_64.rpm 355 kB/s | 81 kB 00:00 2024-03-04T13:28:33.444 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-03-04T13:28:33.445 INFO:teuthology.orchestra.run.smithi107.stdout:Total 350 kB/s | 81 kB 00:00 2024-03-04T13:28:33.446 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-03-04T13:28:33.468 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-03-04T13:28:33.468 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-03-04T13:28:33.567 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-03-04T13:28:33.567 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-03-04T13:28:33.756 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-03-04T13:28:33.882 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : rbd-fuse-2:17.2.7-545.g42608bb1.el9.x86_64 1/1 2024-03-04T13:28:34.412 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: rbd-fuse-2:17.2.7-545.g42608bb1.el9.x86_64 1/1 2024-03-04T13:28:34.727 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : rbd-fuse-2:17.2.7-545.g42608bb1.el9.x86_64 1/1 2024-03-04T13:28:34.728 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:28:34.728 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-03-04T13:28:34.728 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-fuse-2:17.2.7-545.g42608bb1.el9.x86_64 2024-03-04T13:28:34.728 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:28:34.728 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-03-04T13:28:35.066 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install rbd-mirror 2024-03-04T13:28:35.905 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:03:36 ago on Mon 04 Mar 2024 01:24:59 PM UTC. 2024-03-04T13:28:36.077 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-03-04T13:28:36.077 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-03-04T13:28:36.078 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repo Size 2024-03-04T13:28:36.078 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-03-04T13:28:36.078 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-03-04T13:28:36.078 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-mirror x86_64 2:17.2.7-545.g42608bb1.el9 ceph 2.9 M 2024-03-04T13:28:36.078 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:28:36.078 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-03-04T13:28:36.078 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-03-04T13:28:36.078 INFO:teuthology.orchestra.run.smithi107.stdout:Install 1 Package 2024-03-04T13:28:36.078 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:28:36.078 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 2.9 M 2024-03-04T13:28:36.078 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 12 M 2024-03-04T13:28:36.078 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-03-04T13:28:36.741 INFO:teuthology.orchestra.run.smithi107.stdout:rbd-mirror-17.2.7-545.g42608bb1.el9.x86_64.rpm 4.4 MB/s | 2.9 MB 00:00 2024-03-04T13:28:36.742 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-03-04T13:28:36.743 INFO:teuthology.orchestra.run.smithi107.stdout:Total 4.4 MB/s | 2.9 MB 00:00 2024-03-04T13:28:36.744 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-03-04T13:28:36.774 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-03-04T13:28:36.774 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-03-04T13:28:36.883 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-03-04T13:28:36.884 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-03-04T13:28:37.371 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-03-04T13:28:37.475 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : rbd-mirror-2:17.2.7-545.g42608bb1.el9.x86_64 1/1 2024-03-04T13:28:37.547 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: rbd-mirror-2:17.2.7-545.g42608bb1.el9.x86_64 1/1 2024-03-04T13:28:37.547 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-04T13:28:37.547 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-04T13:28:37.547 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-04T13:28:37.547 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-04T13:28:37.547 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:28:38.971 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : rbd-mirror-2:17.2.7-545.g42608bb1.el9.x86_64 1/1 2024-03-04T13:28:38.971 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:28:38.971 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-03-04T13:28:38.971 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-mirror-2:17.2.7-545.g42608bb1.el9.x86_64 2024-03-04T13:28:38.971 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:28:38.971 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-03-04T13:28:39.258 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install rbd-nbd 2024-03-04T13:28:40.096 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:03:41 ago on Mon 04 Mar 2024 01:24:59 PM UTC. 2024-03-04T13:28:40.267 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-03-04T13:28:40.268 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-03-04T13:28:40.268 INFO:teuthology.orchestra.run.smithi107.stdout: Package Architecture Version Repository Size 2024-03-04T13:28:40.268 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-03-04T13:28:40.268 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-03-04T13:28:40.268 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-nbd x86_64 2:17.2.7-545.g42608bb1.el9 ceph 164 k 2024-03-04T13:28:40.268 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:28:40.268 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-03-04T13:28:40.268 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-03-04T13:28:40.268 INFO:teuthology.orchestra.run.smithi107.stdout:Install 1 Package 2024-03-04T13:28:40.268 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:28:40.268 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 164 k 2024-03-04T13:28:40.268 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 477 k 2024-03-04T13:28:40.269 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-03-04T13:28:40.512 INFO:teuthology.orchestra.run.smithi107.stdout:rbd-nbd-17.2.7-545.g42608bb1.el9.x86_64.rpm 679 kB/s | 164 kB 00:00 2024-03-04T13:28:40.513 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-03-04T13:28:40.514 INFO:teuthology.orchestra.run.smithi107.stdout:Total 669 kB/s | 164 kB 00:00 2024-03-04T13:28:40.515 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-03-04T13:28:40.540 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-03-04T13:28:40.540 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-03-04T13:28:40.637 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-03-04T13:28:40.638 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-03-04T13:28:40.819 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-03-04T13:28:40.950 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : rbd-nbd-2:17.2.7-545.g42608bb1.el9.x86_64 1/1 2024-03-04T13:28:41.523 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: rbd-nbd-2:17.2.7-545.g42608bb1.el9.x86_64 1/1 2024-03-04T13:28:41.868 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : rbd-nbd-2:17.2.7-545.g42608bb1.el9.x86_64 1/1 2024-03-04T13:28:41.868 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:28:41.868 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-03-04T13:28:41.868 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-nbd-2:17.2.7-545.g42608bb1.el9.x86_64 2024-03-04T13:28:41.868 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:28:41.868 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-03-04T13:28:42.174 DEBUG:teuthology.parallel:result is None 2024-03-04T13:28:42.175 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-04T13:28:42.175 INFO:teuthology.packaging:ref: None 2024-03-04T13:28:42.175 INFO:teuthology.packaging:tag: None 2024-03-04T13:28:42.175 INFO:teuthology.packaging:branch: quincy 2024-03-04T13:28:42.175 INFO:teuthology.packaging:sha1: 717ce3cce8d0166e4e577faa8004e6f9cb4128c0 2024-03-04T13:28:42.175 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-03-04T13:28:42.383 DEBUG:teuthology.orchestra.run.smithi107:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-04T13:28:42.444 INFO:teuthology.orchestra.run.smithi107.stdout:17.2.7-545.g42608bb1.el9 2024-03-04T13:28:42.445 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-545.g42608bb1.el9 2024-03-04T13:28:42.445 INFO:teuthology.task.install:The correct ceph version 17.2.7-545.g42608bb1 is installed. 2024-03-04T13:28:42.447 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-04T13:28:42.447 INFO:teuthology.packaging:ref: None 2024-03-04T13:28:42.447 INFO:teuthology.packaging:tag: None 2024-03-04T13:28:42.447 INFO:teuthology.packaging:branch: quincy 2024-03-04T13:28:42.447 INFO:teuthology.packaging:sha1: 717ce3cce8d0166e4e577faa8004e6f9cb4128c0 2024-03-04T13:28:42.448 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-03-04T13:28:42.798 DEBUG:teuthology.orchestra.run.smithi169:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-04T13:28:42.872 INFO:teuthology.orchestra.run.smithi169.stdout:17.2.7-545.g42608bb1.el9 2024-03-04T13:28:42.872 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-545.g42608bb1.el9 2024-03-04T13:28:42.873 INFO:teuthology.task.install:The correct ceph version 17.2.7-545.g42608bb1 is installed. 2024-03-04T13:28:42.874 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-04T13:28:42.875 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-03-04T13:28:42.875 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-04T13:28:42.962 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-03-04T13:28:42.962 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-04T13:28:43.063 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-04T13:28:43.063 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-03-04T13:28:43.063 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/usr/bin/daemon-helper 2024-03-04T13:28:43.160 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-04T13:28:43.293 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-03-04T13:28:43.293 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/usr/bin/daemon-helper 2024-03-04T13:28:43.395 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-04T13:28:43.512 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-04T13:28:43.512 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-03-04T13:28:43.512 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-04T13:28:43.607 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-04T13:28:43.744 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-03-04T13:28:43.744 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-04T13:28:43.831 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-04T13:28:43.953 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-04T13:28:43.953 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-03-04T13:28:43.953 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/usr/bin/stdin-killer 2024-03-04T13:28:44.050 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-04T13:28:44.146 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-03-04T13:28:44.146 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/usr/bin/stdin-killer 2024-03-04T13:28:44.233 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-04T13:28:44.353 INFO:teuthology.run_tasks:Running task print... 2024-03-04T13:28:44.362 INFO:teuthology.task.print:**** done install task... 2024-03-04T13:28:44.362 INFO:teuthology.run_tasks:Running task cephadm... 2024-03-04T13:28:44.622 INFO:tasks.cephadm:Config: {'cephadm_branch': 'quincy', 'cephadm_git_url': 'https://github.com/ceph/ceph', 'conf': {'osd': {'osd_class_default_list': '*', 'osd_class_load_list': '*', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'mon pg warn min per osd': 0}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}}, 'image': 'quay.ceph.io/ceph-ci/ceph:quincy', 'roleless': True, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'OSD_DOWN', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN'], 'sha1': '717ce3cce8d0166e4e577faa8004e6f9cb4128c0'} 2024-03-04T13:28:44.622 INFO:tasks.cephadm:Cluster image is quay.ceph.io/ceph-ci/ceph:quincy 2024-03-04T13:28:44.623 INFO:tasks.cephadm:Cluster fsid is 1fc09a58-da2b-11ee-95c6-87774f69a715 2024-03-04T13:28:44.623 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-03-04T13:28:44.623 INFO:tasks.cephadm:No mon roles; fabricating mons 2024-03-04T13:28:44.623 INFO:tasks.cephadm:Monitor IPs: {'mon.smithi107': '172.21.15.107', 'mon.smithi169': '172.21.15.169'} 2024-03-04T13:28:44.623 INFO:tasks.cephadm:Normalizing hostnames... 2024-03-04T13:28:44.623 DEBUG:teuthology.orchestra.run.smithi107:> sudo hostname $(hostname -s) 2024-03-04T13:28:44.727 DEBUG:teuthology.orchestra.run.smithi169:> sudo hostname $(hostname -s) 2024-03-04T13:28:44.827 INFO:tasks.cephadm:Downloading cephadm (repo https://github.com/ceph/ceph ref quincy)... 2024-03-04T13:28:44.828 DEBUG:teuthology.orchestra.run.smithi107:> curl --silent https://raw.githubusercontent.com/ceph/ceph/quincy/src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-04T13:28:45.160 INFO:teuthology.orchestra.run.smithi107.stdout:-rw-r--r--. 1 ubuntu ubuntu 367183 Mar 4 13:28 /home/ubuntu/cephtest/cephadm 2024-03-04T13:28:45.160 DEBUG:teuthology.orchestra.run.smithi169:> curl --silent https://raw.githubusercontent.com/ceph/ceph/quincy/src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-04T13:28:45.494 INFO:teuthology.orchestra.run.smithi169.stdout:-rw-r--r--. 1 ubuntu ubuntu 367183 Mar 4 13:28 /home/ubuntu/cephtest/cephadm 2024-03-04T13:28:45.495 DEBUG:teuthology.orchestra.run.smithi107:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-04T13:28:45.554 DEBUG:teuthology.orchestra.run.smithi169:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-04T13:28:45.678 INFO:tasks.cephadm:Pulling image quay.ceph.io/ceph-ci/ceph:quincy on all hosts... 2024-03-04T13:28:45.679 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy pull 2024-03-04T13:28:45.686 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy pull 2024-03-04T13:28:46.758 INFO:teuthology.orchestra.run.smithi169.stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:quincy... 2024-03-04T13:28:46.969 INFO:teuthology.orchestra.run.smithi107.stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:quincy... 2024-03-04T13:29:20.227 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2024-03-04T13:29:20.228 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_version": "ceph version 17.2.7-545-g42608bb1 (42608bb191117be10dba720e6bf8a02f3171cd34) quincy (stable)", 2024-03-04T13:29:20.228 INFO:teuthology.orchestra.run.smithi169.stdout: "image_id": "9eec3f22859d068e18b5466d2f5dd398da33643a965bdd7e9132169cd1636b22", 2024-03-04T13:29:20.228 INFO:teuthology.orchestra.run.smithi169.stdout: "repo_digests": [ 2024-03-04T13:29:20.228 INFO:teuthology.orchestra.run.smithi169.stdout: "quay.ceph.io/ceph-ci/ceph@sha256:ee85afea73f504aaae524393a00a3b140e2a1cfd81d76da4525eaa6e73286a7d" 2024-03-04T13:29:20.228 INFO:teuthology.orchestra.run.smithi169.stdout: ] 2024-03-04T13:29:20.228 INFO:teuthology.orchestra.run.smithi169.stdout:} 2024-03-04T13:29:20.617 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-03-04T13:29:20.617 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_version": "ceph version 17.2.7-545-g42608bb1 (42608bb191117be10dba720e6bf8a02f3171cd34) quincy (stable)", 2024-03-04T13:29:20.617 INFO:teuthology.orchestra.run.smithi107.stdout: "image_id": "9eec3f22859d068e18b5466d2f5dd398da33643a965bdd7e9132169cd1636b22", 2024-03-04T13:29:20.618 INFO:teuthology.orchestra.run.smithi107.stdout: "repo_digests": [ 2024-03-04T13:29:20.618 INFO:teuthology.orchestra.run.smithi107.stdout: "quay.ceph.io/ceph-ci/ceph@sha256:ee85afea73f504aaae524393a00a3b140e2a1cfd81d76da4525eaa6e73286a7d" 2024-03-04T13:29:20.618 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-03-04T13:29:20.618 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-03-04T13:29:20.670 DEBUG:teuthology.orchestra.run.smithi107:> sudo mkdir -p /etc/ceph 2024-03-04T13:29:20.766 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /etc/ceph 2024-03-04T13:29:20.869 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod 777 /etc/ceph 2024-03-04T13:29:20.966 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod 777 /etc/ceph 2024-03-04T13:29:21.064 INFO:tasks.cephadm:Writing seed config... 2024-03-04T13:29:21.065 INFO:tasks.cephadm: override: [osd] osd_class_default_list = * 2024-03-04T13:29:21.065 INFO:tasks.cephadm: override: [osd] osd_class_load_list = * 2024-03-04T13:29:21.065 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-03-04T13:29:21.065 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-03-04T13:29:21.065 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-03-04T13:29:21.066 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-03-04T13:29:21.066 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-03-04T13:29:21.066 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-03-04T13:29:21.066 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-03-04T13:29:21.066 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-03-04T13:29:21.066 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-03-04T13:29:21.066 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-03-04T13:29:21.066 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-03-04T13:29:21.066 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-03-04T13:29:21.066 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-03-04T13:29:21.066 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-03-04T13:29:21.066 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-03-04T13:29:21.066 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-03-04T13:29:21.066 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-03-04T13:29:21.067 INFO:tasks.cephadm: override: [client] debug client = 20 2024-03-04T13:29:21.067 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-03-04T13:29:21.067 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-03-04T13:29:21.067 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-03-04T13:29:21.067 INFO:tasks.cephadm: override: [global] mon pg warn min per osd = 0 2024-03-04T13:29:21.067 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-03-04T13:29:21.067 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-03-04T13:29:21.067 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-03-04T13:29:21.067 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-03-04T13:29:21.067 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-03-04T13:29:21.067 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-03-04T13:29:21.067 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-03-04T13:29:21.067 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-03-04T13:29:21.067 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-03-04T13:29:21.067 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-03-04T13:29:21.067 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-03-04T13:29:21.068 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-03-04T13:29:21.068 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-03-04T13:29:21.068 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-03-04T13:29:21.068 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-03-04T13:29:21.068 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-03-04T13:29:21.069 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-03-04T13:29:21.069 DEBUG:teuthology.orchestra.run.smithi107:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-03-04T13:29:21.124 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 1fc09a58-da2b-11ee-95c6-87774f69a715 mon pg warn min per osd = 0 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = true bdev debug aio = true osd sloppy crc = true osd_class_default_list = * osd_class_load_list = * bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-03-04T13:29:21.125 DEBUG:teuthology.orchestra.run.smithi107:mon.smithi107> sudo journalctl -f -n 0 -u ceph-1fc09a58-da2b-11ee-95c6-87774f69a715@mon.smithi107.service 2024-03-04T13:29:21.131 INFO:tasks.cephadm:Bootstrapping... 2024-03-04T13:29:21.132 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy -v bootstrap --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-ip 172.21.15.107 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-03-04T13:29:21.434 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-03-04T13:29:21.434 INFO:teuthology.orchestra.run.smithi107.stdout:cephadm ['--image', 'quay.ceph.io/ceph-ci/ceph:quincy', '-v', 'bootstrap', '--fsid', '1fc09a58-da2b-11ee-95c6-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-ip', '172.21.15.107', '--skip-admin-label'] 2024-03-04T13:29:21.489 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-04T13:29:21.490 INFO:teuthology.orchestra.run.smithi107.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-03-04T13:29:21.490 INFO:teuthology.orchestra.run.smithi107.stdout:Verifying podman|docker is present... 2024-03-04T13:29:21.541 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-04T13:29:21.541 INFO:teuthology.orchestra.run.smithi107.stdout:Verifying lvm2 is present... 2024-03-04T13:29:21.541 INFO:teuthology.orchestra.run.smithi107.stdout:Verifying time synchronization is in place... 2024-03-04T13:29:21.556 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-04T13:29:21.556 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-04T13:29:21.571 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-04T13:29:21.571 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout inactive 2024-03-04T13:29:21.590 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout enabled 2024-03-04T13:29:21.606 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout active 2024-03-04T13:29:21.606 INFO:teuthology.orchestra.run.smithi107.stdout:Unit chronyd.service is enabled and running 2024-03-04T13:29:21.606 INFO:teuthology.orchestra.run.smithi107.stdout:Repeating the final host check... 2024-03-04T13:29:21.655 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-04T13:29:21.655 INFO:teuthology.orchestra.run.smithi107.stdout:podman (/bin/podman) version 4.9.3 is present 2024-03-04T13:29:21.655 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl is present 2024-03-04T13:29:21.655 INFO:teuthology.orchestra.run.smithi107.stdout:lvcreate is present 2024-03-04T13:29:21.672 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-04T13:29:21.672 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-04T13:29:21.693 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-04T13:29:21.693 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout inactive 2024-03-04T13:29:21.716 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout enabled 2024-03-04T13:29:21.736 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout active 2024-03-04T13:29:21.736 INFO:teuthology.orchestra.run.smithi107.stdout:Unit chronyd.service is enabled and running 2024-03-04T13:29:21.736 INFO:teuthology.orchestra.run.smithi107.stdout:Host looks OK 2024-03-04T13:29:21.737 INFO:teuthology.orchestra.run.smithi107.stdout:Cluster fsid: 1fc09a58-da2b-11ee-95c6-87774f69a715 2024-03-04T13:29:21.737 INFO:teuthology.orchestra.run.smithi107.stdout:Acquiring lock 140367731234320 on /run/cephadm/1fc09a58-da2b-11ee-95c6-87774f69a715.lock 2024-03-04T13:29:21.737 INFO:teuthology.orchestra.run.smithi107.stdout:Lock 140367731234320 acquired on /run/cephadm/1fc09a58-da2b-11ee-95c6-87774f69a715.lock 2024-03-04T13:29:21.739 INFO:teuthology.orchestra.run.smithi107.stdout:Verifying IP 172.21.15.107 port 3300 ... 2024-03-04T13:29:21.740 INFO:teuthology.orchestra.run.smithi107.stdout:Verifying IP 172.21.15.107 port 6789 ... 2024-03-04T13:29:21.740 INFO:teuthology.orchestra.run.smithi107.stdout:Base mon IP(s) is [172.21.15.107:3300, 172.21.15.107:6789], mon addrv is [v2:172.21.15.107:3300,v1:172.21.15.107:6789] 2024-03-04T13:29:21.754 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.107 metric 100 2024-03-04T13:29:21.755 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.107 metric 100 2024-03-04T13:29:21.767 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-03-04T13:29:21.767 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-03-04T13:29:21.767 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1759sec hoplimit 64 pref medium 2024-03-04T13:29:21.779 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-03-04T13:29:21.779 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-03-04T13:29:21.779 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-04T13:29:21.779 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-03-04T13:29:21.779 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:47eb/64 scope link 2024-03-04T13:29:21.779 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-04T13:29:21.781 INFO:teuthology.orchestra.run.smithi107.stdout:Mon IP `172.21.15.107` is in CIDR network `172.21.0.0/20` 2024-03-04T13:29:21.782 INFO:teuthology.orchestra.run.smithi107.stdout:Mon IP `172.21.15.107` is in CIDR network `172.21.0.0/20` 2024-03-04T13:29:21.782 INFO:teuthology.orchestra.run.smithi107.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-03-04T13:29:21.785 INFO:teuthology.orchestra.run.smithi107.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-03-04T13:29:21.787 INFO:teuthology.orchestra.run.smithi107.stdout:Pulling container image quay.ceph.io/ceph-ci/ceph:quincy... 2024-03-04T13:29:22.500 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stdout 9eec3f22859d068e18b5466d2f5dd398da33643a965bdd7e9132169cd1636b22 2024-03-04T13:29:22.501 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stderr Trying to pull quay.ceph.io/ceph-ci/ceph:quincy... 2024-03-04T13:29:22.501 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stderr Getting image source signatures 2024-03-04T13:29:22.502 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stderr Copying blob sha256:e2add13e83f06afbaf799fc1053235e7e5b00d410b4b726de64e68e97b73fa42 2024-03-04T13:29:22.502 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stderr Copying blob sha256:7feca07754707458c3945cf0062cf4dabc512f6d90fe1a9a1370b362b6011124 2024-03-04T13:29:22.502 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stderr Copying config sha256:9eec3f22859d068e18b5466d2f5dd398da33643a965bdd7e9132169cd1636b22 2024-03-04T13:29:22.502 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stderr Writing manifest to image destination 2024-03-04T13:29:23.723 INFO:teuthology.orchestra.run.smithi107.stdout:ceph: stdout ceph version 17.2.7-545-g42608bb1 (42608bb191117be10dba720e6bf8a02f3171cd34) quincy (stable) 2024-03-04T13:29:23.724 INFO:teuthology.orchestra.run.smithi107.stdout:Ceph version: ceph version 17.2.7-545-g42608bb1 (42608bb191117be10dba720e6bf8a02f3171cd34) quincy (stable) 2024-03-04T13:29:23.724 INFO:teuthology.orchestra.run.smithi107.stdout:Extracting ceph user uid/gid from container image... 2024-03-04T13:29:24.858 INFO:teuthology.orchestra.run.smithi107.stdout:stat: stdout 167 167 2024-03-04T13:29:24.859 INFO:teuthology.orchestra.run.smithi107.stdout:Creating initial keys... 2024-03-04T13:29:25.912 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph-authtool: stdout AQC1zOVlM9VBERAAV+g/sV4gtDgkA0G/JQPmlQ== 2024-03-04T13:29:34.318 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph-authtool: stdout AQC2zOVlV9hzGBAAgIQ1J5bNRXxbEuUHFI/gVg== 2024-03-04T13:29:37.633 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph-authtool: stdout AQDBzOVlxNQfABAAz+Qzfnd3huzQnzyhh+9WtQ== 2024-03-04T13:29:37.636 INFO:teuthology.orchestra.run.smithi107.stdout:Creating initial monmap... 2024-03-04T13:29:38.850 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-04T13:29:38.850 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = octopus 2024-03-04T13:29:38.850 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 1fc09a58-da2b-11ee-95c6-87774f69a715 2024-03-04T13:29:38.850 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-04T13:29:38.850 INFO:teuthology.orchestra.run.smithi107.stdout:monmaptool for smithi107 [v2:172.21.15.107:3300,v1:172.21.15.107:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-04T13:29:38.850 INFO:teuthology.orchestra.run.smithi107.stdout:setting min_mon_release = octopus 2024-03-04T13:29:38.851 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/monmaptool: set fsid to 1fc09a58-da2b-11ee-95c6-87774f69a715 2024-03-04T13:29:38.851 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-04T13:29:38.851 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:29:38.851 INFO:teuthology.orchestra.run.smithi107.stdout:Creating mon... 2024-03-04T13:29:40.435 INFO:teuthology.orchestra.run.smithi107.stdout:create mon.smithi107 on 2024-03-04T13:29:40.809 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-03-04T13:29:41.053 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-03-04T13:29:41.313 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-1fc09a58-da2b-11ee-95c6-87774f69a715.target → /etc/systemd/system/ceph-1fc09a58-da2b-11ee-95c6-87774f69a715.target. 2024-03-04T13:29:41.313 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-1fc09a58-da2b-11ee-95c6-87774f69a715.target → /etc/systemd/system/ceph-1fc09a58-da2b-11ee-95c6-87774f69a715.target. 2024-03-04T13:29:41.615 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-1fc09a58-da2b-11ee-95c6-87774f69a715@mon.smithi107 2024-03-04T13:29:41.615 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Failed to reset failed state of unit ceph-1fc09a58-da2b-11ee-95c6-87774f69a715@mon.smithi107.service: Unit ceph-1fc09a58-da2b-11ee-95c6-87774f69a715@mon.smithi107.service not loaded. 2024-03-04T13:29:41.995 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-1fc09a58-da2b-11ee-95c6-87774f69a715.target.wants/ceph-1fc09a58-da2b-11ee-95c6-87774f69a715@mon.smithi107.service → /etc/systemd/system/ceph-1fc09a58-da2b-11ee-95c6-87774f69a715@.service. 2024-03-04T13:29:42.646 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:42 smithi107 podman[28349]: 2024-03-04 13:29:42.297196528 +0000 UTC m=+0.030875981 image pull 9eec3f22859d068e18b5466d2f5dd398da33643a965bdd7e9132169cd1636b22 quay.ceph.io/ceph-ci/ceph:quincy 2024-03-04T13:29:42.646 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:42 smithi107 podman[28349]: 2024-03-04 13:29:42.650210226 +0000 UTC m=+0.383889714 container create 395f9e7a9086c6e1e9fb2e6a682df21dc1a4ea893c32eedb5aaca5d2a7a0e58c (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-mon-smithi107, RELEASE=quincy-42608bb, io.buildah.version=1.33.5, org.label-schema.build-date=20240222, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.schema-version=1.0) 2024-03-04T13:29:43.097 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-04T13:29:43.097 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout disabled 2024-03-04T13:29:43.119 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-04T13:29:43.120 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout inactive 2024-03-04T13:29:43.120 INFO:teuthology.orchestra.run.smithi107.stdout:firewalld.service is not enabled 2024-03-04T13:29:43.120 INFO:teuthology.orchestra.run.smithi107.stdout:Not possible to enable service . firewalld.service is not available 2024-03-04T13:29:43.121 INFO:teuthology.orchestra.run.smithi107.stdout:Waiting for mon to start... 2024-03-04T13:29:43.121 INFO:teuthology.orchestra.run.smithi107.stdout:Waiting for mon... 2024-03-04T13:29:43.276 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:42 smithi107 podman[28349]: 2024-03-04 13:29:42.906387271 +0000 UTC m=+0.640066769 container init 395f9e7a9086c6e1e9fb2e6a682df21dc1a4ea893c32eedb5aaca5d2a7a0e58c (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-mon-smithi107, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240222, RELEASE=quincy-42608bb, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8) 2024-03-04T13:29:43.276 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:42 smithi107 podman[28349]: 2024-03-04 13:29:42.915920813 +0000 UTC m=+0.649600308 container start 395f9e7a9086c6e1e9fb2e6a682df21dc1a4ea893c32eedb5aaca5d2a7a0e58c (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-mon-smithi107, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, ceph=True, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=quincy-42608bb, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-03-04T13:29:43.276 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:42 smithi107 ceph-mon[28367]: mkfs 1fc09a58-da2b-11ee-95c6-87774f69a715 2024-03-04T13:29:43.276 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:42 smithi107 ceph-mon[28367]: mon.smithi107 is new leader, mons smithi107 in quorum (ranks 0) 2024-03-04T13:29:43.277 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:43 smithi107 bash[28349]: 395f9e7a9086c6e1e9fb2e6a682df21dc1a4ea893c32eedb5aaca5d2a7a0e58c 2024-03-04T13:29:43.277 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:43 smithi107 systemd[1]: Started Ceph mon.smithi107 for 1fc09a58-da2b-11ee-95c6-87774f69a715. 2024-03-04T13:29:44.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:43 smithi107 ceph-mon[28367]: mon.smithi107 is new leader, mons smithi107 in quorum (ranks 0) 2024-03-04T13:29:44.088 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:43 smithi107 ceph-mon[28367]: monmap e1: 1 mons at {smithi107=[v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-04T13:29:44.088 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:43 smithi107 ceph-mon[28367]: fsmap 2024-03-04T13:29:44.088 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:43 smithi107 ceph-mon[28367]: osdmap e1: 0 total, 0 up, 0 in 2024-03-04T13:29:44.088 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:43 smithi107 ceph-mon[28367]: mgrmap e1: no daemons active 2024-03-04T13:29:44.088 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:43 smithi107 ceph-mon[28367]: from='client.? 172.21.15.107:0/1719564962' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-03-04T13:29:44.471 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout cluster: 2024-03-04T13:29:44.471 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout id: 1fc09a58-da2b-11ee-95c6-87774f69a715 2024-03-04T13:29:44.472 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-03-04T13:29:44.472 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-03-04T13:29:44.478 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout services: 2024-03-04T13:29:44.478 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum smithi107 (age 0.816568s) 2024-03-04T13:29:44.478 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-03-04T13:29:44.478 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-03-04T13:29:44.478 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-03-04T13:29:44.478 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout data: 2024-03-04T13:29:44.478 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-03-04T13:29:44.479 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-03-04T13:29:44.479 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-03-04T13:29:44.479 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout pgs: 2024-03-04T13:29:44.479 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-03-04T13:29:44.479 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:43.711+0000 7fe8d1000700 1 Processor -- start 2024-03-04T13:29:44.479 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:43.712+0000 7fe8d1000700 1 -- start start 2024-03-04T13:29:44.479 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:43.712+0000 7fe8d1000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8cc07c7a0 0x7fe8cc07cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:29:44.479 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:43.712+0000 7fe8d1000700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8cc07d0f0 con 0x7fe8cc07c7a0 2024-03-04T13:29:44.479 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:43.713+0000 7fe8c9e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8cc07c7a0 0x7fe8cc07cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:29:44.479 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:43.714+0000 7fe8c9e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8cc07c7a0 0x7fe8cc07cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49062/0 (socket says 172.21.15.107:49062) 2024-03-04T13:29:44.479 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:43.714+0000 7fe8c9e00700 1 -- 172.21.15.107:0/3703072204 learned_addr learned my addr 172.21.15.107:0/3703072204 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:29:44.480 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:43.714+0000 7fe8c9e00700 1 -- 172.21.15.107:0/3703072204 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8cc07d230 con 0x7fe8cc07c7a0 2024-03-04T13:29:44.480 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:43.715+0000 7fe8c9e00700 1 --2- 172.21.15.107:0/3703072204 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8cc07c7a0 0x7fe8cc07cbb0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc0098e0 tx=0x7fe8bc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=893634f566c747c1 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:29:44.480 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:43.715+0000 7fe8c9400700 1 -- 172.21.15.107:0/3703072204 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe8bc00b860 con 0x7fe8cc07c7a0 2024-03-04T13:29:44.480 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:43.715+0000 7fe8c9400700 1 -- 172.21.15.107:0/3703072204 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8bc00b9c0 con 0x7fe8cc07c7a0 2024-03-04T13:29:44.480 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:43.715+0000 7fe8c9400700 1 -- 172.21.15.107:0/3703072204 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe8bc00bc90 con 0x7fe8cc07c7a0 2024-03-04T13:29:44.480 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:43.716+0000 7fe8d1000700 1 -- 172.21.15.107:0/3703072204 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8cc07c7a0 msgr2=0x7fe8cc07cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:29:44.480 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:43.716+0000 7fe8d1000700 1 --2- 172.21.15.107:0/3703072204 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8cc07c7a0 0x7fe8cc07cbb0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc0098e0 tx=0x7fe8bc009250 comp rx=0 tx=0).stop 2024-03-04T13:29:44.480 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:43.716+0000 7fe8d1000700 1 -- 172.21.15.107:0/3703072204 shutdown_connections 2024-03-04T13:29:44.480 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:43.716+0000 7fe8d1000700 1 --2- 172.21.15.107:0/3703072204 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8cc07c7a0 0x7fe8cc07cbb0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:29:44.480 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:43.716+0000 7fe8d1000700 1 -- 172.21.15.107:0/3703072204 >> 172.21.15.107:0/3703072204 conn(0x7fe8cc078030 msgr2=0x7fe8cc07a460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:29:44.480 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:43.716+0000 7fe8d1000700 1 -- 172.21.15.107:0/3703072204 shutdown_connections 2024-03-04T13:29:44.480 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:43.716+0000 7fe8d1000700 1 -- 172.21.15.107:0/3703072204 wait complete. 2024-03-04T13:29:44.480 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:43.717+0000 7fe8d1000700 1 Processor -- start 2024-03-04T13:29:44.481 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:43.717+0000 7fe8d1000700 1 -- start start 2024-03-04T13:29:44.481 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:43.717+0000 7fe8d1000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8cc07c7a0 0x7fe8cc117350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:29:44.481 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:43.717+0000 7fe8d1000700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8cc117890 con 0x7fe8cc07c7a0 2024-03-04T13:29:44.481 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:43.718+0000 7fe8c9e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8cc07c7a0 0x7fe8cc117350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:29:44.481 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:43.718+0000 7fe8c9e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8cc07c7a0 0x7fe8cc117350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49068/0 (socket says 172.21.15.107:49068) 2024-03-04T13:29:44.481 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:43.718+0000 7fe8c9e00700 1 -- 172.21.15.107:0/1719564962 learned_addr learned my addr 172.21.15.107:0/1719564962 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:29:44.481 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:43.718+0000 7fe8c9e00700 1 -- 172.21.15.107:0/1719564962 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8bc0094f0 con 0x7fe8cc07c7a0 2024-03-04T13:29:44.481 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:43.718+0000 7fe8c9e00700 1 --2- 172.21.15.107:0/1719564962 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8cc07c7a0 0x7fe8cc117350 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc009290 tx=0x7fe8bc016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:29:44.481 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:43.719+0000 7fe8c2a00700 1 -- 172.21.15.107:0/1719564962 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe8bc01c5c0 con 0x7fe8cc07c7a0 2024-03-04T13:29:44.481 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:43.719+0000 7fe8c2a00700 1 -- 172.21.15.107:0/1719564962 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8bc01c740 con 0x7fe8cc07c7a0 2024-03-04T13:29:44.481 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:43.719+0000 7fe8c2a00700 1 -- 172.21.15.107:0/1719564962 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe8bc01ca50 con 0x7fe8cc07c7a0 2024-03-04T13:29:44.481 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:43.719+0000 7fe8d1000700 1 -- 172.21.15.107:0/1719564962 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8cc117a90 con 0x7fe8cc07c7a0 2024-03-04T13:29:44.481 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:43.719+0000 7fe8d1000700 1 -- 172.21.15.107:0/1719564962 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8cc118730 con 0x7fe8cc07c7a0 2024-03-04T13:29:44.482 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:43.719+0000 7fe8c2a00700 1 -- 172.21.15.107:0/1719564962 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fe8bc01a450 con 0x7fe8cc07c7a0 2024-03-04T13:29:44.482 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:43.719+0000 7fe8c2a00700 1 -- 172.21.15.107:0/1719564962 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fe8bc02e790 con 0x7fe8cc07c7a0 2024-03-04T13:29:44.482 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:43.720+0000 7fe8d1000700 1 -- 172.21.15.107:0/1719564962 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8cc05f5c0 con 0x7fe8cc07c7a0 2024-03-04T13:29:44.482 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:43.723+0000 7fe8c2a00700 1 -- 172.21.15.107:0/1719564962 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fe8bc01ce30 con 0x7fe8cc07c7a0 2024-03-04T13:29:44.482 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:43.800+0000 7fe8d1000700 1 -- 172.21.15.107:0/1719564962 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7fe8cc119200 con 0x7fe8cc07c7a0 2024-03-04T13:29:44.482 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:43.801+0000 7fe8c2a00700 1 -- 172.21.15.107:0/1719564962 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+325 (secure 0 0 0) 0x7fe8bc01a890 con 0x7fe8cc07c7a0 2024-03-04T13:29:44.482 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:43.802+0000 7fe8d1000700 1 -- 172.21.15.107:0/1719564962 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8cc07c7a0 msgr2=0x7fe8cc117350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:29:44.482 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:43.802+0000 7fe8d1000700 1 --2- 172.21.15.107:0/1719564962 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8cc07c7a0 0x7fe8cc117350 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc009290 tx=0x7fe8bc016000 comp rx=0 tx=0).stop 2024-03-04T13:29:44.482 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:43.802+0000 7fe8d1000700 1 -- 172.21.15.107:0/1719564962 shutdown_connections 2024-03-04T13:29:44.482 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:43.802+0000 7fe8d1000700 1 --2- 172.21.15.107:0/1719564962 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8cc07c7a0 0x7fe8cc117350 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:29:44.482 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:43.802+0000 7fe8d1000700 1 -- 172.21.15.107:0/1719564962 >> 172.21.15.107:0/1719564962 conn(0x7fe8cc078030 msgr2=0x7fe8cc10bad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:29:44.482 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:43.802+0000 7fe8d1000700 1 -- 172.21.15.107:0/1719564962 shutdown_connections 2024-03-04T13:29:44.482 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:43.803+0000 7fe8d1000700 1 -- 172.21.15.107:0/1719564962 wait complete. 2024-03-04T13:29:44.483 INFO:teuthology.orchestra.run.smithi107.stdout:mon is available 2024-03-04T13:29:44.483 INFO:teuthology.orchestra.run.smithi107.stdout:Assimilating anything we can from ceph.conf... 2024-03-04T13:29:45.337 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:45 smithi107 ceph-mon[28367]: from='client.? 172.21.15.107:0/3279424433' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-04T13:29:45.337 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:45 smithi107 ceph-mon[28367]: from='client.? 172.21.15.107:0/3279424433' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-03-04T13:29:46.206 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-03-04T13:29:46.206 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout [global] 2024-03-04T13:29:46.206 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout fsid = 1fc09a58-da2b-11ee-95c6-87774f69a715 2024-03-04T13:29:46.206 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.107:3300,v1:172.21.15.107:6789] 2024-03-04T13:29:46.206 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-04T13:29:46.206 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-04T13:29:46.206 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-04T13:29:46.207 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-04T13:29:46.207 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-03-04T13:29:46.207 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-04T13:29:46.207 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-04T13:29:46.207 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-03-04T13:29:46.207 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout [osd] 2024-03-04T13:29:46.207 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-04T13:29:46.207 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-04T13:29:46.207 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-04T13:29:46.207 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-04T13:29:46.207 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-04T13:29:46.207 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:45.085+0000 7f15b0200700 1 Processor -- start 2024-03-04T13:29:46.207 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:45.085+0000 7f15b0200700 1 -- start start 2024-03-04T13:29:46.208 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:45.085+0000 7f15b0200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15a8102770 0x7f15a8102b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:29:46.208 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:45.085+0000 7f15b0200700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a81030c0 con 0x7f15a8102770 2024-03-04T13:29:46.208 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:45.086+0000 7f15ad800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15a8102770 0x7f15a8102b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:29:46.208 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:45.086+0000 7f15ad800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15a8102770 0x7f15a8102b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49076/0 (socket says 172.21.15.107:49076) 2024-03-04T13:29:46.208 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:45.086+0000 7f15ad800700 1 -- 172.21.15.107:0/1356712383 learned_addr learned my addr 172.21.15.107:0/1356712383 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:29:46.208 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:45.086+0000 7f15ad800700 1 -- 172.21.15.107:0/1356712383 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15a8103200 con 0x7f15a8102770 2024-03-04T13:29:46.208 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:45.087+0000 7f15ad800700 1 --2- 172.21.15.107:0/1356712383 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15a8102770 0x7f15a8102b80 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f159c0098e0 tx=0x7f159c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e27c13f2f62d272 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:29:46.208 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:45.087+0000 7f15a7e00700 1 -- 172.21.15.107:0/1356712383 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f159c00b860 con 0x7f15a8102770 2024-03-04T13:29:46.208 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:45.087+0000 7f15a7e00700 1 -- 172.21.15.107:0/1356712383 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f159c00b9c0 con 0x7f15a8102770 2024-03-04T13:29:46.208 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:45.087+0000 7f15a7e00700 1 -- 172.21.15.107:0/1356712383 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f159c00bc90 con 0x7f15a8102770 2024-03-04T13:29:46.208 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:45.088+0000 7f15b0200700 1 -- 172.21.15.107:0/1356712383 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15a8102770 msgr2=0x7f15a8102b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:29:46.208 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:45.088+0000 7f15b0200700 1 --2- 172.21.15.107:0/1356712383 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15a8102770 0x7f15a8102b80 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f159c0098e0 tx=0x7f159c009250 comp rx=0 tx=0).stop 2024-03-04T13:29:46.208 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:45.088+0000 7f15b0200700 1 -- 172.21.15.107:0/1356712383 shutdown_connections 2024-03-04T13:29:46.208 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:45.088+0000 7f15b0200700 1 --2- 172.21.15.107:0/1356712383 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15a8102770 0x7f15a8102b80 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:29:46.209 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:45.088+0000 7f15b0200700 1 -- 172.21.15.107:0/1356712383 >> 172.21.15.107:0/1356712383 conn(0x7f15a80fded0 msgr2=0x7f15a8100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:29:46.209 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:45.088+0000 7f15b0200700 1 -- 172.21.15.107:0/1356712383 shutdown_connections 2024-03-04T13:29:46.209 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:45.088+0000 7f15b0200700 1 -- 172.21.15.107:0/1356712383 wait complete. 2024-03-04T13:29:46.209 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:45.089+0000 7f15b0200700 1 Processor -- start 2024-03-04T13:29:46.209 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:45.089+0000 7f15b0200700 1 -- start start 2024-03-04T13:29:46.209 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:45.089+0000 7f15b0200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15a8102770 0x7f15a8077c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:29:46.209 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:45.089+0000 7f15b0200700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a8076080 con 0x7f15a8102770 2024-03-04T13:29:46.209 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:45.090+0000 7f15ad800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15a8102770 0x7f15a8077c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:29:46.209 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:45.090+0000 7f15ad800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15a8102770 0x7f15a8077c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49078/0 (socket says 172.21.15.107:49078) 2024-03-04T13:29:46.209 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:45.090+0000 7f15ad800700 1 -- 172.21.15.107:0/3279424433 learned_addr learned my addr 172.21.15.107:0/3279424433 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:29:46.209 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:45.090+0000 7f15ad800700 1 -- 172.21.15.107:0/3279424433 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f159c0094f0 con 0x7f15a8102770 2024-03-04T13:29:46.209 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:45.090+0000 7f15ad800700 1 --2- 172.21.15.107:0/3279424433 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15a8102770 0x7f15a8077c00 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f159c003080 tx=0x7f159c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:29:46.209 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:45.090+0000 7f15a5600700 1 -- 172.21.15.107:0/3279424433 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f159c01c5c0 con 0x7f15a8102770 2024-03-04T13:29:46.209 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:45.091+0000 7f15a5600700 1 -- 172.21.15.107:0/3279424433 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f159c01c720 con 0x7f15a8102770 2024-03-04T13:29:46.209 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:45.091+0000 7f15a5600700 1 -- 172.21.15.107:0/3279424433 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f159c01ca30 con 0x7f15a8102770 2024-03-04T13:29:46.210 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:45.091+0000 7f15b0200700 1 -- 172.21.15.107:0/3279424433 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15a8076280 con 0x7f15a8102770 2024-03-04T13:29:46.210 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:45.091+0000 7f15b0200700 1 -- 172.21.15.107:0/3279424433 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15a8076710 con 0x7f15a8102770 2024-03-04T13:29:46.210 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:45.091+0000 7f15a5600700 1 -- 172.21.15.107:0/3279424433 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f159c01a450 con 0x7f15a8102770 2024-03-04T13:29:46.210 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:45.091+0000 7f15a5600700 1 -- 172.21.15.107:0/3279424433 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f159c02e790 con 0x7f15a8102770 2024-03-04T13:29:46.210 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:45.092+0000 7f15b0200700 1 -- 172.21.15.107:0/3279424433 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15a810aca0 con 0x7f15a8102770 2024-03-04T13:29:46.210 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:45.094+0000 7f15a5600700 1 -- 172.21.15.107:0/3279424433 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f159c01a890 con 0x7f15a8102770 2024-03-04T13:29:46.210 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:45.168+0000 7f15b0200700 1 -- 172.21.15.107:0/3279424433 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f15a8064b00 con 0x7f15a8102770 2024-03-04T13:29:46.210 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:45.176+0000 7f15a5600700 1 -- 172.21.15.107:0/3279424433 <== mon.0 v2:172.21.15.107:3300/0 7 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f159c02ede0 con 0x7f15a8102770 2024-03-04T13:29:46.210 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:45.176+0000 7f15a5600700 1 -- 172.21.15.107:0/3279424433 <== mon.0 v2:172.21.15.107:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+431 (secure 0 0 0) 0x7f159c037590 con 0x7f15a8102770 2024-03-04T13:29:46.210 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:45.178+0000 7f15b0200700 1 -- 172.21.15.107:0/3279424433 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15a8102770 msgr2=0x7f15a8077c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:29:46.210 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:45.178+0000 7f15b0200700 1 --2- 172.21.15.107:0/3279424433 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15a8102770 0x7f15a8077c00 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f159c003080 tx=0x7f159c016000 comp rx=0 tx=0).stop 2024-03-04T13:29:46.210 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:45.178+0000 7f15b0200700 1 -- 172.21.15.107:0/3279424433 shutdown_connections 2024-03-04T13:29:46.210 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:45.178+0000 7f15b0200700 1 --2- 172.21.15.107:0/3279424433 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15a8102770 0x7f15a8077c00 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:29:46.210 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:45.178+0000 7f15b0200700 1 -- 172.21.15.107:0/3279424433 >> 172.21.15.107:0/3279424433 conn(0x7f15a80fded0 msgr2=0x7f15a8107e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:29:46.210 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:45.178+0000 7f15b0200700 1 -- 172.21.15.107:0/3279424433 shutdown_connections 2024-03-04T13:29:46.211 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:45.178+0000 7f15b0200700 1 -- 172.21.15.107:0/3279424433 wait complete. 2024-03-04T13:29:46.211 INFO:teuthology.orchestra.run.smithi107.stdout:Generating new minimal ceph.conf... 2024-03-04T13:29:47.337 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:46 smithi107 ceph-mon[28367]: from='client.? 172.21.15.107:0/2438094217' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:29:47.833 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:46.859+0000 7f5829a00700 1 Processor -- start 2024-03-04T13:29:47.833 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:46.859+0000 7f5829a00700 1 -- start start 2024-03-04T13:29:47.833 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:46.859+0000 7f5829a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f582407c7a0 0x7f582407cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:29:47.833 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:46.859+0000 7f5829a00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f582407d0f0 con 0x7f582407c7a0 2024-03-04T13:29:47.833 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:46.860+0000 7f5822a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f582407c7a0 0x7f582407cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:29:47.833 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:46.860+0000 7f5822a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f582407c7a0 0x7f582407cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49084/0 (socket says 172.21.15.107:49084) 2024-03-04T13:29:47.833 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:46.860+0000 7f5822a00700 1 -- 172.21.15.107:0/2026410181 learned_addr learned my addr 172.21.15.107:0/2026410181 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:29:47.833 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:46.860+0000 7f5822a00700 1 -- 172.21.15.107:0/2026410181 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f582407d230 con 0x7f582407c7a0 2024-03-04T13:29:47.833 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:46.860+0000 7f5822a00700 1 --2- 172.21.15.107:0/2026410181 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f582407c7a0 0x7f582407cbb0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f58140098e0 tx=0x7f5814009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9988bd2b5a93627 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:29:47.834 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:46.861+0000 7f5821600700 1 -- 172.21.15.107:0/2026410181 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f581400b860 con 0x7f582407c7a0 2024-03-04T13:29:47.834 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:46.861+0000 7f5821600700 1 -- 172.21.15.107:0/2026410181 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f581400be50 con 0x7f582407c7a0 2024-03-04T13:29:47.834 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:46.861+0000 7f5821600700 1 -- 172.21.15.107:0/2026410181 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5814010380 con 0x7f582407c7a0 2024-03-04T13:29:47.834 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:46.861+0000 7f5829a00700 1 -- 172.21.15.107:0/2026410181 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f582407c7a0 msgr2=0x7f582407cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:29:47.834 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:46.861+0000 7f5829a00700 1 --2- 172.21.15.107:0/2026410181 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f582407c7a0 0x7f582407cbb0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f58140098e0 tx=0x7f5814009250 comp rx=0 tx=0).stop 2024-03-04T13:29:47.834 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:46.861+0000 7f5829a00700 1 -- 172.21.15.107:0/2026410181 shutdown_connections 2024-03-04T13:29:47.834 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:46.861+0000 7f5829a00700 1 --2- 172.21.15.107:0/2026410181 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f582407c7a0 0x7f582407cbb0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:29:47.834 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:46.861+0000 7f5829a00700 1 -- 172.21.15.107:0/2026410181 >> 172.21.15.107:0/2026410181 conn(0x7f5824078030 msgr2=0x7f582407a460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:29:47.834 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:46.862+0000 7f5829a00700 1 -- 172.21.15.107:0/2026410181 shutdown_connections 2024-03-04T13:29:47.834 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:46.862+0000 7f5829a00700 1 -- 172.21.15.107:0/2026410181 wait complete. 2024-03-04T13:29:47.834 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:46.862+0000 7f5829a00700 1 Processor -- start 2024-03-04T13:29:47.834 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:46.863+0000 7f5829a00700 1 -- start start 2024-03-04T13:29:47.834 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:46.863+0000 7f5829a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f582407c7a0 0x7f5824117bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:29:47.835 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:46.863+0000 7f5829a00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58241180f0 con 0x7f582407c7a0 2024-03-04T13:29:47.835 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:46.863+0000 7f5822a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f582407c7a0 0x7f5824117bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:29:47.835 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:46.863+0000 7f5822a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f582407c7a0 0x7f5824117bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49094/0 (socket says 172.21.15.107:49094) 2024-03-04T13:29:47.835 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:46.863+0000 7f5822a00700 1 -- 172.21.15.107:0/2438094217 learned_addr learned my addr 172.21.15.107:0/2438094217 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:29:47.835 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:46.863+0000 7f5822a00700 1 -- 172.21.15.107:0/2438094217 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58140094f0 con 0x7f582407c7a0 2024-03-04T13:29:47.835 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:46.864+0000 7f5822a00700 1 --2- 172.21.15.107:0/2438094217 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f582407c7a0 0x7f5824117bb0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5814006b00 tx=0x7f5814009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:29:47.835 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:46.864+0000 7f581aa00700 1 -- 172.21.15.107:0/2438094217 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f58140107c0 con 0x7f582407c7a0 2024-03-04T13:29:47.835 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:46.864+0000 7f581aa00700 1 -- 172.21.15.107:0/2438094217 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f5814010db0 con 0x7f582407c7a0 2024-03-04T13:29:47.835 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:46.864+0000 7f5829a00700 1 -- 172.21.15.107:0/2438094217 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58241182f0 con 0x7f582407c7a0 2024-03-04T13:29:47.835 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:46.864+0000 7f581aa00700 1 -- 172.21.15.107:0/2438094217 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f581401acf0 con 0x7f582407c7a0 2024-03-04T13:29:47.835 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:46.864+0000 7f5829a00700 1 -- 172.21.15.107:0/2438094217 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5824118f90 con 0x7f582407c7a0 2024-03-04T13:29:47.835 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:46.864+0000 7f581aa00700 1 -- 172.21.15.107:0/2438094217 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f5814024440 con 0x7f582407c7a0 2024-03-04T13:29:47.835 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:46.864+0000 7f581aa00700 1 -- 172.21.15.107:0/2438094217 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f58140197a0 con 0x7f582407c7a0 2024-03-04T13:29:47.835 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:46.865+0000 7f5829a00700 1 -- 172.21.15.107:0/2438094217 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f582405f5c0 con 0x7f582407c7a0 2024-03-04T13:29:47.836 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:46.868+0000 7f581aa00700 1 -- 172.21.15.107:0/2438094217 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f58140426d0 con 0x7f582407c7a0 2024-03-04T13:29:47.836 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:46.943+0000 7f5829a00700 1 -- 172.21.15.107:0/2438094217 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f58241191d0 con 0x7f582407c7a0 2024-03-04T13:29:47.836 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:46.943+0000 7f581aa00700 1 -- 172.21.15.107:0/2438094217 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+177 (secure 0 0 0) 0x7f5814031020 con 0x7f582407c7a0 2024-03-04T13:29:47.836 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:46.944+0000 7f5829a00700 1 -- 172.21.15.107:0/2438094217 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f582407c7a0 msgr2=0x7f5824117bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:29:47.836 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:46.944+0000 7f5829a00700 1 --2- 172.21.15.107:0/2438094217 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f582407c7a0 0x7f5824117bb0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5814006b00 tx=0x7f5814009250 comp rx=0 tx=0).stop 2024-03-04T13:29:47.836 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:46.945+0000 7f5829a00700 1 -- 172.21.15.107:0/2438094217 shutdown_connections 2024-03-04T13:29:47.836 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:46.945+0000 7f5829a00700 1 --2- 172.21.15.107:0/2438094217 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f582407c7a0 0x7f5824117bb0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:29:47.836 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:46.945+0000 7f5829a00700 1 -- 172.21.15.107:0/2438094217 >> 172.21.15.107:0/2438094217 conn(0x7f5824078030 msgr2=0x7f582407a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:29:47.836 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:46.945+0000 7f5829a00700 1 -- 172.21.15.107:0/2438094217 shutdown_connections 2024-03-04T13:29:47.836 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:46.945+0000 7f5829a00700 1 -- 172.21.15.107:0/2438094217 wait complete. 2024-03-04T13:29:47.837 INFO:teuthology.orchestra.run.smithi107.stdout:Restarting the monitor... 2024-03-04T13:29:48.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:47 smithi107 systemd[1]: Stopping Ceph mon.smithi107 for 1fc09a58-da2b-11ee-95c6-87774f69a715... 2024-03-04T13:29:48.724 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:48 smithi107 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-mon-smithi107[28363]: 2024-03-04T13:29:48.344+0000 7fdb1fc00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi107 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-04T13:29:48.724 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:48 smithi107 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-mon-smithi107[28363]: 2024-03-04T13:29:48.344+0000 7fdb1fc00700 -1 mon.smithi107@0(leader) e1 *** Got Signal Terminated *** 2024-03-04T13:29:48.724 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:48 smithi107 podman[28611]: 2024-03-04 13:29:48.416693261 +0000 UTC m=+0.214955372 container died 395f9e7a9086c6e1e9fb2e6a682df21dc1a4ea893c32eedb5aaca5d2a7a0e58c (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-mon-smithi107, CEPH_POINT_RELEASE=, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=quincy-42608bb) 2024-03-04T13:29:49.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:48 smithi107 podman[28611]: 2024-03-04 13:29:48.839765609 +0000 UTC m=+0.638027727 container cleanup 395f9e7a9086c6e1e9fb2e6a682df21dc1a4ea893c32eedb5aaca5d2a7a0e58c (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-mon-smithi107, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240222, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=quincy-42608bb, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux ) 2024-03-04T13:29:49.088 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:48 smithi107 bash[28611]: ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-mon-smithi107 2024-03-04T13:29:49.382 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:49 smithi107 podman[28626]: 2024-03-04 13:29:49.136348285 +0000 UTC m=+0.701441764 container remove 395f9e7a9086c6e1e9fb2e6a682df21dc1a4ea893c32eedb5aaca5d2a7a0e58c (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-mon-smithi107, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=quincy-42608bb, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-04T13:29:50.049 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:49 smithi107 systemd[1]: ceph-1fc09a58-da2b-11ee-95c6-87774f69a715@mon.smithi107.service: Deactivated successfully. 2024-03-04T13:29:50.049 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:49 smithi107 systemd[1]: Stopped Ceph mon.smithi107 for 1fc09a58-da2b-11ee-95c6-87774f69a715. 2024-03-04T13:29:50.049 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:49 smithi107 systemd[1]: ceph-1fc09a58-da2b-11ee-95c6-87774f69a715@mon.smithi107.service: Consumed 1.953s CPU time. 2024-03-04T13:29:50.049 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:49 smithi107 systemd[1]: Starting Ceph mon.smithi107 for 1fc09a58-da2b-11ee-95c6-87774f69a715... 2024-03-04T13:29:50.305 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 podman[28719]: 2024-03-04 13:29:49.95999807 +0000 UTC m=+0.035220347 image pull 9eec3f22859d068e18b5466d2f5dd398da33643a965bdd7e9132169cd1636b22 quay.ceph.io/ceph-ci/ceph:quincy 2024-03-04T13:29:50.305 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 podman[28719]: 2024-03-04 13:29:50.077724712 +0000 UTC m=+0.152947044 container create 49f5da990d351f8884dee805009c3a572c63450c8a57344d930aedde1c6b4e6e (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-mon-smithi107, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=quincy-42608bb, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240222) 2024-03-04T13:29:50.305 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 podman[28719]: 2024-03-04 13:29:50.302498975 +0000 UTC m=+0.377721337 container init 49f5da990d351f8884dee805009c3a572c63450c8a57344d930aedde1c6b4e6e (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-mon-smithi107, RELEASE=quincy-42608bb, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-03-04T13:29:50.403 INFO:teuthology.orchestra.run.smithi107.stdout:Setting mon public_network to 172.21.0.0/20 2024-03-04T13:29:50.563 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 podman[28719]: 2024-03-04 13:29:50.311306533 +0000 UTC m=+0.386528812 container start 49f5da990d351f8884dee805009c3a572c63450c8a57344d930aedde1c6b4e6e (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-mon-smithi107, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=quincy-42608bb, GIT_CLEAN=True, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.license=GPLv2) 2024-03-04T13:29:50.564 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: set uid:gid to 167:167 (ceph:ceph) 2024-03-04T13:29:50.564 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: ceph version 17.2.7-545-g42608bb1 (42608bb191117be10dba720e6bf8a02f3171cd34) quincy (stable), process ceph-mon, pid 2 2024-03-04T13:29:50.564 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: pidfile_write: ignore empty --pid-file 2024-03-04T13:29:50.564 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: load: jerasure load: lrc 2024-03-04T13:29:50.564 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: RocksDB version: 6.15.5 2024-03-04T13:29:50.565 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-03-04T13:29:50.565 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Compile date Mar 3 2024 2024-03-04T13:29:50.565 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: DB SUMMARY 2024-03-04T13:29:50.565 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: DB Session ID: 7YVQ6TSZNCFZBQN8XFKI 2024-03-04T13:29:50.565 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: CURRENT file: CURRENT 2024-03-04T13:29:50.565 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: IDENTITY file: IDENTITY 2024-03-04T13:29:50.565 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: MANIFEST file: MANIFEST-000009 size: 131 Bytes 2024-03-04T13:29:50.566 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi107/store.db dir, Total Num: 1, files: 000008.sst 2024-03-04T13:29:50.566 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi107/store.db: 000010.log size: 97815 ; 2024-03-04T13:29:50.566 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.error_if_exists: 0 2024-03-04T13:29:50.566 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.create_if_missing: 0 2024-03-04T13:29:50.566 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.paranoid_checks: 1 2024-03-04T13:29:50.566 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-04T13:29:50.566 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.env: 0x561b65f59a80 2024-03-04T13:29:50.567 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.fs: Posix File System 2024-03-04T13:29:50.567 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.info_log: 0x561b67537cc0 2024-03-04T13:29:50.567 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.max_file_opening_threads: 16 2024-03-04T13:29:50.567 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.statistics: (nil) 2024-03-04T13:29:50.567 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.use_fsync: 0 2024-03-04T13:29:50.567 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.max_log_file_size: 0 2024-03-04T13:29:50.567 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-04T13:29:50.568 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-04T13:29:50.568 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.keep_log_file_num: 1000 2024-03-04T13:29:50.568 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.recycle_log_file_num: 0 2024-03-04T13:29:50.568 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.allow_fallocate: 1 2024-03-04T13:29:50.568 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.allow_mmap_reads: 0 2024-03-04T13:29:50.568 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.allow_mmap_writes: 0 2024-03-04T13:29:50.571 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.use_direct_reads: 0 2024-03-04T13:29:50.571 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-04T13:29:50.571 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.create_missing_column_families: 0 2024-03-04T13:29:50.571 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.db_log_dir: 2024-03-04T13:29:50.571 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-smithi107/store.db 2024-03-04T13:29:50.571 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-04T13:29:50.572 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-04T13:29:50.572 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-04T13:29:50.572 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-04T13:29:50.572 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-04T13:29:50.572 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-04T13:29:50.573 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.advise_random_on_open: 1 2024-03-04T13:29:50.573 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.db_write_buffer_size: 0 2024-03-04T13:29:50.573 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.write_buffer_manager: 0x561b682acae0 2024-03-04T13:29:50.573 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-04T13:29:50.573 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-03-04T13:29:50.573 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-04T13:29:50.573 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-04T13:29:50.574 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.rate_limiter: (nil) 2024-03-04T13:29:50.574 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-04T13:29:50.574 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.wal_recovery_mode: 2 2024-03-04T13:29:50.574 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.enable_thread_tracking: 0 2024-03-04T13:29:50.574 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.enable_pipelined_write: 0 2024-03-04T13:29:50.574 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.unordered_write: 0 2024-03-04T13:29:50.574 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-04T13:29:50.575 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-04T13:29:50.575 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-04T13:29:50.575 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-04T13:29:50.575 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.row_cache: None 2024-03-04T13:29:50.575 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.wal_filter: None 2024-03-04T13:29:50.575 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-04T13:29:50.575 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.allow_ingest_behind: 0 2024-03-04T13:29:50.576 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.preserve_deletes: 0 2024-03-04T13:29:50.576 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.two_write_queues: 0 2024-03-04T13:29:50.576 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.manual_wal_flush: 0 2024-03-04T13:29:50.576 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.atomic_flush: 0 2024-03-04T13:29:50.576 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-04T13:29:50.576 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-04T13:29:50.576 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-04T13:29:50.577 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.log_readahead_size: 0 2024-03-04T13:29:50.577 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-04T13:29:50.577 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.best_efforts_recovery: 0 2024-03-04T13:29:50.577 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-04T13:29:50.577 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-04T13:29:50.577 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.allow_data_in_errors: 0 2024-03-04T13:29:50.577 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.db_host_id: __hostname__ 2024-03-04T13:29:50.578 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.max_background_jobs: 2 2024-03-04T13:29:50.578 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.max_background_compactions: -1 2024-03-04T13:29:50.578 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.max_subcompactions: 1 2024-03-04T13:29:50.578 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-04T13:29:50.578 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-04T13:29:50.578 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-04T13:29:50.578 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.max_total_wal_size: 0 2024-03-04T13:29:50.579 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-04T13:29:50.579 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-04T13:29:50.579 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-04T13:29:50.579 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-04T13:29:50.580 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.max_open_files: -1 2024-03-04T13:29:50.580 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.bytes_per_sync: 0 2024-03-04T13:29:50.580 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-04T13:29:50.580 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-04T13:29:50.580 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.compaction_readahead_size: 0 2024-03-04T13:29:50.580 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.max_background_flushes: -1 2024-03-04T13:29:50.580 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Compression algorithms supported: 2024-03-04T13:29:50.581 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-04T13:29:50.581 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: kZSTD supported: 0 2024-03-04T13:29:50.581 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: kXpressCompression supported: 0 2024-03-04T13:29:50.581 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: kLZ4HCCompression supported: 1 2024-03-04T13:29:50.581 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: kZlibCompression supported: 1 2024-03-04T13:29:50.581 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: kSnappyCompression supported: 1 2024-03-04T13:29:50.581 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: kLZ4Compression supported: 1 2024-03-04T13:29:50.582 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: kBZip2Compression supported: 0 2024-03-04T13:29:50.582 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-04T13:29:50.582 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: [db/version_set.cc:4724] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi107/store.db/MANIFEST-000009 2024-03-04T13:29:50.582 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: [db/column_family.cc:595] --------------- Options for column family [default]: 2024-03-04T13:29:50.582 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-04T13:29:50.582 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.merge_operator: 2024-03-04T13:29:50.582 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.compaction_filter: None 2024-03-04T13:29:50.583 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.compaction_filter_factory: None 2024-03-04T13:29:50.583 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.sst_partitioner_factory: None 2024-03-04T13:29:50.583 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-04T13:29:50.583 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-04T13:29:50.583 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x561b6758cf58) 2024-03-04T13:29:50.583 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: cache_index_and_filter_blocks: 1 2024-03-04T13:29:50.583 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-04T13:29:50.584 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-04T13:29:50.584 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: pin_top_level_index_and_filter: 1 2024-03-04T13:29:50.584 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: index_type: 0 2024-03-04T13:29:50.584 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: data_block_index_type: 0 2024-03-04T13:29:50.584 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: index_shortening: 1 2024-03-04T13:29:50.584 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-04T13:29:50.584 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: hash_index_allow_collision: 1 2024-03-04T13:29:50.585 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: checksum: 1 2024-03-04T13:29:50.585 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: no_block_cache: 0 2024-03-04T13:29:50.585 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: block_cache: 0x561b675bf090 2024-03-04T13:29:50.585 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: block_cache_name: BinnedLRUCache 2024-03-04T13:29:50.585 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: block_cache_options: 2024-03-04T13:29:50.585 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: capacity : 536870912 2024-03-04T13:29:50.585 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: num_shard_bits : 4 2024-03-04T13:29:50.586 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: strict_capacity_limit : 0 2024-03-04T13:29:50.586 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: high_pri_pool_ratio: 0.000 2024-03-04T13:29:50.586 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: block_cache_compressed: (nil) 2024-03-04T13:29:50.586 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: persistent_cache: (nil) 2024-03-04T13:29:50.586 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: block_size: 4096 2024-03-04T13:29:50.586 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: block_size_deviation: 10 2024-03-04T13:29:50.586 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: block_restart_interval: 16 2024-03-04T13:29:50.587 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: index_block_restart_interval: 1 2024-03-04T13:29:50.587 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: metadata_block_size: 4096 2024-03-04T13:29:50.587 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: partition_filters: 0 2024-03-04T13:29:50.587 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: use_delta_encoding: 1 2024-03-04T13:29:50.587 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2024-03-04T13:29:50.587 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: whole_key_filtering: 1 2024-03-04T13:29:50.587 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: verify_compression: 0 2024-03-04T13:29:50.587 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: read_amp_bytes_per_bit: 0 2024-03-04T13:29:50.588 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: format_version: 4 2024-03-04T13:29:50.588 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: enable_index_compression: 1 2024-03-04T13:29:50.588 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: block_align: 0 2024-03-04T13:29:50.588 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.write_buffer_size: 33554432 2024-03-04T13:29:50.588 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.max_write_buffer_number: 2 2024-03-04T13:29:50.588 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.compression: NoCompression 2024-03-04T13:29:50.588 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.bottommost_compression: Disabled 2024-03-04T13:29:50.589 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.prefix_extractor: nullptr 2024-03-04T13:29:50.589 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-04T13:29:50.589 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.num_levels: 7 2024-03-04T13:29:50.589 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-04T13:29:50.589 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-04T13:29:50.589 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-04T13:29:50.589 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-04T13:29:50.590 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-04T13:29:50.590 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-04T13:29:50.590 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-04T13:29:50.590 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-04T13:29:50.590 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-04T13:29:50.590 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-04T13:29:50.590 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-04T13:29:50.591 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.compression_opts.level: 32767 2024-03-04T13:29:50.591 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.compression_opts.strategy: 0 2024-03-04T13:29:50.591 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-04T13:29:50.591 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-04T13:29:50.591 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-04T13:29:50.591 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.compression_opts.enabled: false 2024-03-04T13:29:50.591 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-04T13:29:50.592 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-04T13:29:50.592 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-04T13:29:50.592 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.target_file_size_base: 67108864 2024-03-04T13:29:50.592 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-04T13:29:50.592 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-04T13:29:50.592 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-04T13:29:50.592 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-04T13:29:50.593 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-04T13:29:50.593 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-04T13:29:50.593 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-04T13:29:50.593 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-04T13:29:50.593 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-04T13:29:50.593 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-04T13:29:50.593 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-04T13:29:50.594 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-04T13:29:50.594 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-04T13:29:50.594 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.arena_block_size: 4194304 2024-03-04T13:29:50.594 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-04T13:29:50.594 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-04T13:29:50.594 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-03-04T13:29:50.594 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.disable_auto_compactions: 0 2024-03-04T13:29:50.594 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-04T13:29:50.595 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-04T13:29:50.595 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-04T13:29:50.595 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-04T13:29:50.595 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-04T13:29:50.595 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-04T13:29:50.595 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-04T13:29:50.595 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-04T13:29:50.596 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-04T13:29:50.596 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-04T13:29:50.596 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.table_properties_collectors: 2024-03-04T13:29:50.596 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.inplace_update_support: 0 2024-03-04T13:29:50.596 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-04T13:29:50.596 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-04T13:29:50.596 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-04T13:29:50.597 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-04T13:29:50.597 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.bloom_locality: 0 2024-03-04T13:29:50.597 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.max_successive_merges: 0 2024-03-04T13:29:50.597 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-04T13:29:50.597 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.paranoid_file_checks: 0 2024-03-04T13:29:50.597 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.force_consistency_checks: 1 2024-03-04T13:29:50.598 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.report_bg_io_stats: 0 2024-03-04T13:29:50.598 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.ttl: 2592000 2024-03-04T13:29:50.598 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-04T13:29:50.598 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.enable_blob_files: false 2024-03-04T13:29:50.598 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.min_blob_size: 0 2024-03-04T13:29:50.598 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.blob_file_size: 268435456 2024-03-04T13:29:50.598 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-04T13:29:50.598 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-04T13:29:50.599 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-04T13:29:50.599 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: [db/version_set.cc:4764] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi107/store.db/MANIFEST-000009 succeeded,manifest_file_number is 9, next_file_number is 11, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-03-04T13:29:50.599 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: [db/version_set.cc:4779] Column family [default] (ID 0), log number is 5 2024-03-04T13:29:50.599 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: [db/version_set.cc:4082] Creating manifest 13 2024-03-04T13:29:50.599 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: EVENT_LOG_v1 {"time_micros": 1709558990360622, "job": 1, "event": "recovery_started", "wal_files": [10]} 2024-03-04T13:29:50.599 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: [db/db_impl/db_impl_open.cc:845] Recovering log #10 mode 2 2024-03-04T13:29:50.599 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: [table/block_based/filter_policy.cc:991] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-03-04T13:29:50.600 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: EVENT_LOG_v1 {"time_micros": 1709558990363362, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 14, "file_size": 92844, "file_checksum": "", "file_checksum_func_name": "Unknown", "table_properties": {"data_size": 90939, "index_size": 265, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13372, "raw_average_key_size": 49, "raw_value_size": 83640, "raw_average_value_size": 309, "num_data_blocks": 11, "num_entries": 270, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1709558990, "oldest_key_time": 0, "file_creation_time": 0, "db_id": "dcfaf532-0a93-46a9-8105-df3a14b94c0c", "db_session_id": "7YVQ6TSZNCFZBQN8XFKI"}} 2024-03-04T13:29:50.600 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: [db/version_set.cc:4082] Creating manifest 15 2024-03-04T13:29:50.600 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: EVENT_LOG_v1 {"time_micros": 1709558990364328, "job": 1, "event": "recovery_finished"} 2024-03-04T13:29:50.600 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: [file/delete_scheduler.cc:69] Deleted file /var/lib/ceph/mon/ceph-smithi107/store.db/000010.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-04T13:29:50.600 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: [db/db_impl/db_impl_open.cc:1700] SstFileManager instance 0x561b67553180 2024-03-04T13:29:50.600 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: DB pointer 0x561b67628000 2024-03-04T13:29:50.600 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: [db/db_impl/db_impl.cc:901] ------- DUMPING STATS ------- 2024-03-04T13:29:50.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: rocksdb: [db/db_impl/db_impl.cc:903] 2024-03-04T13:29:50.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: ** DB Stats ** 2024-03-04T13:29:50.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-04T13:29:50.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-04T13:29:50.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-04T13:29:50.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-04T13:29:50.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-04T13:29:50.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-03-04T13:29:50.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-04T13:29:50.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: 2024-03-04T13:29:50.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: ** Compaction Stats [default] ** 2024-03-04T13:29:50.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-03-04T13:29:50.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-04T13:29:50.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: L0 2/0 92.47 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 47.9 0.00 0.00 1 0.002 0 0 2024-03-04T13:29:50.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Sum 2/0 92.47 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 47.9 0.00 0.00 1 0.002 0 0 2024-03-04T13:29:50.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 47.9 0.00 0.00 1 0.002 0 0 2024-03-04T13:29:50.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: 2024-03-04T13:29:50.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: ** Compaction Stats [default] ** 2024-03-04T13:29:50.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-03-04T13:29:50.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-04T13:29:50.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 47.9 0.00 0.00 1 0.002 0 0 2024-03-04T13:29:50.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-04T13:29:50.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-04T13:29:50.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-04T13:29:50.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-04T13:29:50.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-04T13:29:50.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-04T13:29:50.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Cumulative compaction: 0.00 GB write, 8.90 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-04T13:29:50.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Interval compaction: 0.00 GB write, 8.90 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-04T13:29:50.605 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-04T13:29:50.605 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: 2024-03-04T13:29:50.605 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-04T13:29:50.605 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: 2024-03-04T13:29:50.605 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: ** Compaction Stats [default] ** 2024-03-04T13:29:50.605 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-03-04T13:29:50.605 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-04T13:29:50.606 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: L0 2/0 92.47 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 47.9 0.00 0.00 1 0.002 0 0 2024-03-04T13:29:50.606 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Sum 2/0 92.47 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 47.9 0.00 0.00 1 0.002 0 0 2024-03-04T13:29:50.606 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-03-04T13:29:50.606 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: 2024-03-04T13:29:50.606 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: ** Compaction Stats [default] ** 2024-03-04T13:29:50.606 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-03-04T13:29:50.606 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-04T13:29:50.607 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 47.9 0.00 0.00 1 0.002 0 0 2024-03-04T13:29:50.607 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-04T13:29:50.607 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-04T13:29:50.607 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-04T13:29:50.607 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-04T13:29:50.607 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-04T13:29:50.607 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-04T13:29:50.608 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Cumulative compaction: 0.00 GB write, 8.86 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-04T13:29:50.608 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-04T13:29:50.608 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-04T13:29:50.608 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: 2024-03-04T13:29:50.608 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-04T13:29:50.608 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: starting mon.smithi107 rank 0 at public addrs [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] at bind addrs [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi107 fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 2024-03-04T13:29:50.608 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: mon.smithi107@-1(???) e1 preinit fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 2024-03-04T13:29:50.609 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: mon.smithi107@-1(???).mds e1 new map 2024-03-04T13:29:50.609 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: mon.smithi107@-1(???).mds e1 print_map 2024-03-04T13:29:50.609 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: e1 2024-03-04T13:29:50.609 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-04T13:29:50.609 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-04T13:29:50.609 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: legacy client fscid: -1 2024-03-04T13:29:50.609 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: 2024-03-04T13:29:50.610 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: No filesystems configured 2024-03-04T13:29:50.610 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: mon.smithi107@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-04T13:29:50.610 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: mon.smithi107@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-04T13:29:50.610 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: mon.smithi107@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-04T13:29:50.610 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: mon.smithi107@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-04T13:29:50.610 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: mon.smithi107@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-03-04T13:29:50.610 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 bash[28719]: 49f5da990d351f8884dee805009c3a572c63450c8a57344d930aedde1c6b4e6e 2024-03-04T13:29:50.610 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 systemd[1]: Started Ceph mon.smithi107 for 1fc09a58-da2b-11ee-95c6-87774f69a715. 2024-03-04T13:29:50.611 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: mon.smithi107 is new leader, mons smithi107 in quorum (ranks 0) 2024-03-04T13:29:50.611 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: monmap e1: 1 mons at {smithi107=[v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-04T13:29:50.611 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: fsmap 2024-03-04T13:29:50.611 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: osdmap e1: 0 total, 0 up, 0 in 2024-03-04T13:29:50.611 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:50 smithi107 ceph-mon[28739]: mgrmap e1: no daemons active 2024-03-04T13:29:51.861 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:51.054+0000 7fd188000700 1 Processor -- start 2024-03-04T13:29:51.861 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:51.055+0000 7fd188000700 1 -- start start 2024-03-04T13:29:51.861 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:51.055+0000 7fd188000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd180071420 0x7fd1801053e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:29:51.862 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:51.055+0000 7fd188000700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd180105920 con 0x7fd180071420 2024-03-04T13:29:51.862 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:51.056+0000 7fd185600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd180071420 0x7fd1801053e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:29:51.862 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:51.056+0000 7fd185600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd180071420 0x7fd1801053e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:58392/0 (socket says 172.21.15.107:58392) 2024-03-04T13:29:51.862 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:51.056+0000 7fd185600700 1 -- 172.21.15.107:0/305300844 learned_addr learned my addr 172.21.15.107:0/305300844 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:29:51.862 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:51.056+0000 7fd185600700 1 -- 172.21.15.107:0/305300844 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd180105a60 con 0x7fd180071420 2024-03-04T13:29:51.862 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:51.057+0000 7fd185600700 1 --2- 172.21.15.107:0/305300844 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd180071420 0x7fd1801053e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fd1740098e0 tx=0x7fd174009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4491d64772cfd41a server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:29:51.863 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:51.057+0000 7fd17fe00700 1 -- 172.21.15.107:0/305300844 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd17400b860 con 0x7fd180071420 2024-03-04T13:29:51.863 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:51.057+0000 7fd17fe00700 1 -- 172.21.15.107:0/305300844 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fd17400be50 con 0x7fd180071420 2024-03-04T13:29:51.863 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:51.057+0000 7fd17fe00700 1 -- 172.21.15.107:0/305300844 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd174010380 con 0x7fd180071420 2024-03-04T13:29:51.863 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:51.058+0000 7fd188000700 1 -- 172.21.15.107:0/305300844 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd180071420 msgr2=0x7fd1801053e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:29:51.863 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:51.058+0000 7fd188000700 1 --2- 172.21.15.107:0/305300844 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd180071420 0x7fd1801053e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fd1740098e0 tx=0x7fd174009250 comp rx=0 tx=0).stop 2024-03-04T13:29:51.863 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:51.058+0000 7fd188000700 1 -- 172.21.15.107:0/305300844 shutdown_connections 2024-03-04T13:29:51.863 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:51.058+0000 7fd188000700 1 --2- 172.21.15.107:0/305300844 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd180071420 0x7fd1801053e0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:29:51.863 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:51.058+0000 7fd188000700 1 -- 172.21.15.107:0/305300844 >> 172.21.15.107:0/305300844 conn(0x7fd1800fcdd0 msgr2=0x7fd1800ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:29:51.864 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:51.058+0000 7fd188000700 1 -- 172.21.15.107:0/305300844 shutdown_connections 2024-03-04T13:29:51.864 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:51.058+0000 7fd188000700 1 -- 172.21.15.107:0/305300844 wait complete. 2024-03-04T13:29:51.864 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:51.059+0000 7fd188000700 1 Processor -- start 2024-03-04T13:29:51.864 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:51.059+0000 7fd188000700 1 -- start start 2024-03-04T13:29:51.864 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:51.059+0000 7fd188000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd180071420 0x7fd180102e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:29:51.864 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:51.059+0000 7fd188000700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1801012e0 con 0x7fd180071420 2024-03-04T13:29:51.865 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:51.060+0000 7fd185600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd180071420 0x7fd180102e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:29:51.865 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:51.060+0000 7fd185600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd180071420 0x7fd180102e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:58404/0 (socket says 172.21.15.107:58404) 2024-03-04T13:29:51.865 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:51.060+0000 7fd185600700 1 -- 172.21.15.107:0/2283641942 learned_addr learned my addr 172.21.15.107:0/2283641942 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:29:51.866 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:51.060+0000 7fd185600700 1 -- 172.21.15.107:0/2283641942 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1740094f0 con 0x7fd180071420 2024-03-04T13:29:51.866 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:51.060+0000 7fd185600700 1 --2- 172.21.15.107:0/2283641942 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd180071420 0x7fd180102e60 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fd174003050 tx=0x7fd174009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:29:51.866 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:51.060+0000 7fd17d600700 1 -- 172.21.15.107:0/2283641942 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd174010760 con 0x7fd180071420 2024-03-04T13:29:51.866 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:51.061+0000 7fd17d600700 1 -- 172.21.15.107:0/2283641942 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fd174010d50 con 0x7fd180071420 2024-03-04T13:29:51.866 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:51.061+0000 7fd17d600700 1 -- 172.21.15.107:0/2283641942 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd17401ae20 con 0x7fd180071420 2024-03-04T13:29:51.866 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:51.061+0000 7fd188000700 1 -- 172.21.15.107:0/2283641942 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1801014e0 con 0x7fd180071420 2024-03-04T13:29:51.866 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:51.061+0000 7fd188000700 1 -- 172.21.15.107:0/2283641942 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd180101970 con 0x7fd180071420 2024-03-04T13:29:51.866 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:51.061+0000 7fd17d600700 1 -- 172.21.15.107:0/2283641942 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fd17402e6c0 con 0x7fd180071420 2024-03-04T13:29:51.866 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:51.061+0000 7fd17d600700 1 -- 172.21.15.107:0/2283641942 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fd1740248b0 con 0x7fd180071420 2024-03-04T13:29:51.866 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:51.062+0000 7fd188000700 1 -- 172.21.15.107:0/2283641942 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd18004bed0 con 0x7fd180071420 2024-03-04T13:29:51.866 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:51.067+0000 7fd17d600700 1 -- 172.21.15.107:0/2283641942 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fd174029070 con 0x7fd180071420 2024-03-04T13:29:51.866 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:51.139+0000 7fd188000700 1 -- 172.21.15.107:0/2283641942 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7fd180106810 con 0x7fd180071420 2024-03-04T13:29:51.866 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:51.142+0000 7fd17d600700 1 -- 172.21.15.107:0/2283641942 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7fd180106810 con 0x7fd180071420 2024-03-04T13:29:51.866 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:51.143+0000 7fd188000700 1 -- 172.21.15.107:0/2283641942 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd180071420 msgr2=0x7fd180102e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:29:51.867 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:51.143+0000 7fd188000700 1 --2- 172.21.15.107:0/2283641942 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd180071420 0x7fd180102e60 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fd174003050 tx=0x7fd174009250 comp rx=0 tx=0).stop 2024-03-04T13:29:51.867 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:51.144+0000 7fd188000700 1 -- 172.21.15.107:0/2283641942 shutdown_connections 2024-03-04T13:29:51.867 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:51.144+0000 7fd188000700 1 --2- 172.21.15.107:0/2283641942 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd180071420 0x7fd180102e60 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:29:51.867 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:51.144+0000 7fd188000700 1 -- 172.21.15.107:0/2283641942 >> 172.21.15.107:0/2283641942 conn(0x7fd1800fcdd0 msgr2=0x7fd1800fe4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:29:51.867 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:51.144+0000 7fd188000700 1 -- 172.21.15.107:0/2283641942 shutdown_connections 2024-03-04T13:29:51.867 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:51.144+0000 7fd188000700 1 -- 172.21.15.107:0/2283641942 wait complete. 2024-03-04T13:29:51.867 INFO:teuthology.orchestra.run.smithi107.stdout:Wrote config to /etc/ceph/ceph.conf 2024-03-04T13:29:51.867 INFO:teuthology.orchestra.run.smithi107.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-03-04T13:29:51.867 INFO:teuthology.orchestra.run.smithi107.stdout:Creating mgr... 2024-03-04T13:29:51.867 INFO:teuthology.orchestra.run.smithi107.stdout:Verifying port 9283 ... 2024-03-04T13:29:52.172 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-1fc09a58-da2b-11ee-95c6-87774f69a715@mgr.smithi107.wpzgwl 2024-03-04T13:29:52.172 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Failed to reset failed state of unit ceph-1fc09a58-da2b-11ee-95c6-87774f69a715@mgr.smithi107.wpzgwl.service: Unit ceph-1fc09a58-da2b-11ee-95c6-87774f69a715@mgr.smithi107.wpzgwl.service not loaded. 2024-03-04T13:29:52.434 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-1fc09a58-da2b-11ee-95c6-87774f69a715.target.wants/ceph-1fc09a58-da2b-11ee-95c6-87774f69a715@mgr.smithi107.wpzgwl.service → /etc/systemd/system/ceph-1fc09a58-da2b-11ee-95c6-87774f69a715@.service. 2024-03-04T13:29:52.464 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:52 smithi107 ceph-mon[28739]: from='client.? 172.21.15.107:0/2283641942' entity='client.admin' 2024-03-04T13:29:53.232 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-04T13:29:53.232 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout disabled 2024-03-04T13:29:53.256 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-04T13:29:53.257 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout inactive 2024-03-04T13:29:53.257 INFO:teuthology.orchestra.run.smithi107.stdout:firewalld.service is not enabled 2024-03-04T13:29:53.257 INFO:teuthology.orchestra.run.smithi107.stdout:Not possible to enable service . firewalld.service is not available 2024-03-04T13:29:53.290 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-04T13:29:53.290 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout disabled 2024-03-04T13:29:53.316 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-04T13:29:53.317 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout inactive 2024-03-04T13:29:53.317 INFO:teuthology.orchestra.run.smithi107.stdout:firewalld.service is not enabled 2024-03-04T13:29:53.317 INFO:teuthology.orchestra.run.smithi107.stdout:Not possible to open ports <[9283]>. firewalld.service is not available 2024-03-04T13:29:53.317 INFO:teuthology.orchestra.run.smithi107.stdout:Waiting for mgr to start... 2024-03-04T13:29:53.317 INFO:teuthology.orchestra.run.smithi107.stdout:Waiting for mgr... 2024-03-04T13:29:54.337 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:54 smithi107 ceph-mon[28739]: from='client.? 172.21.15.107:0/175531710' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-04T13:29:54.781 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-03-04T13:29:54.781 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout { 2024-03-04T13:29:54.781 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "fsid": "1fc09a58-da2b-11ee-95c6-87774f69a715", 2024-03-04T13:29:54.781 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "health": { 2024-03-04T13:29:54.781 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-04T13:29:54.781 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-04T13:29:54.781 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-04T13:29:54.781 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-03-04T13:29:54.781 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-04T13:29:54.781 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-04T13:29:54.782 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 0 2024-03-04T13:29:54.782 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ], 2024-03-04T13:29:54.782 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-04T13:29:54.782 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "smithi107" 2024-03-04T13:29:54.782 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ], 2024-03-04T13:29:54.782 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-03-04T13:29:54.782 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-04T13:29:54.782 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-04T13:29:54.782 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-03-04T13:29:54.782 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-04T13:29:54.782 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-03-04T13:29:54.782 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-04T13:29:54.782 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-04T13:29:54.782 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-04T13:29:54.783 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-04T13:29:54.783 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-04T13:29:54.783 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-04T13:29:54.783 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-04T13:29:54.783 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-04T13:29:54.783 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-03-04T13:29:54.783 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-04T13:29:54.783 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-04T13:29:54.783 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-04T13:29:54.783 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-04T13:29:54.790 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-04T13:29:54.790 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-04T13:29:54.790 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-04T13:29:54.790 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-04T13:29:54.790 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-04T13:29:54.790 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-03-04T13:29:54.790 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-04T13:29:54.791 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-04T13:29:54.791 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-04T13:29:54.791 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-04T13:29:54.791 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-03-04T13:29:54.791 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-04T13:29:54.791 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-04T13:29:54.791 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-04T13:29:54.791 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-04T13:29:54.791 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-04T13:29:54.791 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-04T13:29:54.791 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "restful" 2024-03-04T13:29:54.791 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ], 2024-03-04T13:29:54.791 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-04T13:29:54.791 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-03-04T13:29:54.792 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-04T13:29:54.792 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-04T13:29:54.792 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "modified": "2024-03-04T13:29:42.986437+0000", 2024-03-04T13:29:54.792 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-04T13:29:54.792 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-03-04T13:29:54.792 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-04T13:29:54.792 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout } 2024-03-04T13:29:54.792 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:53.969+0000 7f840d600700 1 Processor -- start 2024-03-04T13:29:54.792 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:53.969+0000 7f840d600700 1 -- start start 2024-03-04T13:29:54.792 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:53.970+0000 7f840d600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8408077c10 0x7f8408076120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:29:54.792 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:53.970+0000 7f840d600700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8408078020 con 0x7f8408077c10 2024-03-04T13:29:54.792 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:53.970+0000 7f8406a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8408077c10 0x7f8408076120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:29:54.792 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:53.970+0000 7f8406a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8408077c10 0x7f8408076120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:58436/0 (socket says 172.21.15.107:58436) 2024-03-04T13:29:54.793 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:53.970+0000 7f8406a00700 1 -- 172.21.15.107:0/1216969936 learned_addr learned my addr 172.21.15.107:0/1216969936 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:29:54.793 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:53.971+0000 7f8406a00700 1 -- 172.21.15.107:0/1216969936 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8408076660 con 0x7f8408077c10 2024-03-04T13:29:54.793 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:53.971+0000 7f8406a00700 1 --2- 172.21.15.107:0/1216969936 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8408077c10 0x7f8408076120 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f83f80098e0 tx=0x7f83f8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f50ba191c90e37b2 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:29:54.793 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:53.971+0000 7f8405600700 1 -- 172.21.15.107:0/1216969936 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f83f800b860 con 0x7f8408077c10 2024-03-04T13:29:54.793 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:53.971+0000 7f8405600700 1 -- 172.21.15.107:0/1216969936 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f83f800be50 con 0x7f8408077c10 2024-03-04T13:29:54.793 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:53.971+0000 7f8405600700 1 -- 172.21.15.107:0/1216969936 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f83f8010380 con 0x7f8408077c10 2024-03-04T13:29:54.793 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:53.972+0000 7f840d600700 1 -- 172.21.15.107:0/1216969936 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8408077c10 msgr2=0x7f8408076120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:29:54.793 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:53.972+0000 7f840d600700 1 --2- 172.21.15.107:0/1216969936 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8408077c10 0x7f8408076120 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f83f80098e0 tx=0x7f83f8009250 comp rx=0 tx=0).stop 2024-03-04T13:29:54.793 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:53.972+0000 7f840d600700 1 -- 172.21.15.107:0/1216969936 shutdown_connections 2024-03-04T13:29:54.793 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:53.972+0000 7f840d600700 1 --2- 172.21.15.107:0/1216969936 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8408077c10 0x7f8408076120 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:29:54.793 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:53.972+0000 7f840d600700 1 -- 172.21.15.107:0/1216969936 >> 172.21.15.107:0/1216969936 conn(0x7f84080fded0 msgr2=0x7f8408100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:29:54.793 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:53.972+0000 7f840d600700 1 -- 172.21.15.107:0/1216969936 shutdown_connections 2024-03-04T13:29:54.793 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:53.973+0000 7f840d600700 1 -- 172.21.15.107:0/1216969936 wait complete. 2024-03-04T13:29:54.794 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:53.973+0000 7f840d600700 1 Processor -- start 2024-03-04T13:29:54.794 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:53.973+0000 7f840d600700 1 -- start start 2024-03-04T13:29:54.794 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:53.974+0000 7f840d600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8408077c10 0x7f8408117e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:29:54.794 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:53.974+0000 7f840d600700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84081183c0 con 0x7f8408077c10 2024-03-04T13:29:54.794 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:53.974+0000 7f8406a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8408077c10 0x7f8408117e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:29:54.794 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:53.974+0000 7f8406a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8408077c10 0x7f8408117e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:58438/0 (socket says 172.21.15.107:58438) 2024-03-04T13:29:54.794 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:53.974+0000 7f8406a00700 1 -- 172.21.15.107:0/175531710 learned_addr learned my addr 172.21.15.107:0/175531710 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:29:54.794 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:53.974+0000 7f8406a00700 1 -- 172.21.15.107:0/175531710 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83f80094f0 con 0x7f8408077c10 2024-03-04T13:29:54.794 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:53.975+0000 7f8406a00700 1 --2- 172.21.15.107:0/175531710 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8408077c10 0x7f8408117e80 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f83f8013030 tx=0x7f83f8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:29:54.794 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:53.975+0000 7f83fea00700 1 -- 172.21.15.107:0/175531710 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f83f8010760 con 0x7f8408077c10 2024-03-04T13:29:54.794 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:53.975+0000 7f83fea00700 1 -- 172.21.15.107:0/175531710 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f83f8010d50 con 0x7f8408077c10 2024-03-04T13:29:54.794 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:53.975+0000 7f83fea00700 1 -- 172.21.15.107:0/175531710 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f83f801ae60 con 0x7f8408077c10 2024-03-04T13:29:54.794 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:53.975+0000 7f840d600700 1 -- 172.21.15.107:0/175531710 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84081185c0 con 0x7f8408077c10 2024-03-04T13:29:54.794 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:53.975+0000 7f840d600700 1 -- 172.21.15.107:0/175531710 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8408119260 con 0x7f8408077c10 2024-03-04T13:29:54.795 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:53.975+0000 7f83fea00700 1 -- 172.21.15.107:0/175531710 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f83f80196d0 con 0x7f8408077c10 2024-03-04T13:29:54.795 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:53.976+0000 7f83fea00700 1 -- 172.21.15.107:0/175531710 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f83f8024910 con 0x7f8408077c10 2024-03-04T13:29:54.795 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:53.976+0000 7f840d600700 1 -- 172.21.15.107:0/175531710 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f840805f5c0 con 0x7f8408077c10 2024-03-04T13:29:54.795 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:53.979+0000 7f83fea00700 1 -- 172.21.15.107:0/175531710 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f83f8029060 con 0x7f8408077c10 2024-03-04T13:29:54.795 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:54.057+0000 7f840d600700 1 -- 172.21.15.107:0/175531710 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f8408064ad0 con 0x7f8408077c10 2024-03-04T13:29:54.795 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:54.057+0000 7f83fea00700 1 -- 172.21.15.107:0/175531710 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1248 (secure 0 0 0) 0x7f83f801a460 con 0x7f8408077c10 2024-03-04T13:29:54.795 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:54.059+0000 7f840d600700 1 -- 172.21.15.107:0/175531710 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8408077c10 msgr2=0x7f8408117e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:29:54.795 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:54.059+0000 7f840d600700 1 --2- 172.21.15.107:0/175531710 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8408077c10 0x7f8408117e80 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f83f8013030 tx=0x7f83f8009250 comp rx=0 tx=0).stop 2024-03-04T13:29:54.795 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:54.059+0000 7f840d600700 1 -- 172.21.15.107:0/175531710 shutdown_connections 2024-03-04T13:29:54.795 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:54.059+0000 7f840d600700 1 --2- 172.21.15.107:0/175531710 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8408077c10 0x7f8408117e80 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:29:54.795 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:54.059+0000 7f840d600700 1 -- 172.21.15.107:0/175531710 >> 172.21.15.107:0/175531710 conn(0x7f84080fded0 msgr2=0x7f8408100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:29:54.795 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:54.060+0000 7f840d600700 1 -- 172.21.15.107:0/175531710 shutdown_connections 2024-03-04T13:29:54.795 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:54.060+0000 7f840d600700 1 -- 172.21.15.107:0/175531710 wait complete. 2024-03-04T13:29:54.795 INFO:teuthology.orchestra.run.smithi107.stdout:mgr not available, waiting (1/15)... 2024-03-04T13:29:57.794 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:29:57 smithi107 ceph-mon[28739]: from='client.? 172.21.15.107:0/3755645321' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-04T13:29:58.193 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-03-04T13:29:58.193 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout { 2024-03-04T13:29:58.193 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "fsid": "1fc09a58-da2b-11ee-95c6-87774f69a715", 2024-03-04T13:29:58.193 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "health": { 2024-03-04T13:29:58.193 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-04T13:29:58.193 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-04T13:29:58.193 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-04T13:29:58.194 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-03-04T13:29:58.194 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-04T13:29:58.194 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-04T13:29:58.194 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 0 2024-03-04T13:29:58.194 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ], 2024-03-04T13:29:58.194 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-04T13:29:58.194 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "smithi107" 2024-03-04T13:29:58.194 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ], 2024-03-04T13:29:58.194 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "quorum_age": 7, 2024-03-04T13:29:58.194 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-04T13:29:58.194 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-04T13:29:58.194 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-03-04T13:29:58.194 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-04T13:29:58.195 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-03-04T13:29:58.195 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-04T13:29:58.195 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-04T13:29:58.195 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-04T13:29:58.195 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-04T13:29:58.195 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-04T13:29:58.195 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-04T13:29:58.195 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-04T13:29:58.195 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-04T13:29:58.195 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-03-04T13:29:58.195 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-04T13:29:58.195 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-04T13:29:58.196 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-04T13:29:58.196 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-04T13:29:58.196 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-04T13:29:58.196 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-04T13:29:58.196 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-04T13:29:58.196 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-04T13:29:58.196 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-04T13:29:58.196 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-03-04T13:29:58.196 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-04T13:29:58.196 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-04T13:29:58.196 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-04T13:29:58.196 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-04T13:29:58.196 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-03-04T13:29:58.197 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-04T13:29:58.197 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-04T13:29:58.197 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-04T13:29:58.197 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-04T13:29:58.197 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-04T13:29:58.197 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-04T13:29:58.197 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "restful" 2024-03-04T13:29:58.197 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ], 2024-03-04T13:29:58.197 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-04T13:29:58.197 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-03-04T13:29:58.197 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-04T13:29:58.197 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-04T13:29:58.197 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "modified": "2024-03-04T13:29:42.986437+0000", 2024-03-04T13:29:58.197 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-04T13:29:58.198 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-03-04T13:29:58.198 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-04T13:29:58.198 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout } 2024-03-04T13:29:58.198 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:57.370+0000 7fc19ec00700 1 Processor -- start 2024-03-04T13:29:58.198 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:57.370+0000 7fc19ec00700 1 -- start start 2024-03-04T13:29:58.198 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:57.370+0000 7fc19ec00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc198102770 0x7fc198102b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:29:58.198 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:57.370+0000 7fc19ec00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1981030c0 con 0x7fc198102770 2024-03-04T13:29:58.198 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:57.371+0000 7fc197e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc198102770 0x7fc198102b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:29:58.198 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:57.371+0000 7fc197e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc198102770 0x7fc198102b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:58442/0 (socket says 172.21.15.107:58442) 2024-03-04T13:29:58.198 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:57.371+0000 7fc197e00700 1 -- 172.21.15.107:0/4255843688 learned_addr learned my addr 172.21.15.107:0/4255843688 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:29:58.198 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:57.371+0000 7fc197e00700 1 -- 172.21.15.107:0/4255843688 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc198103200 con 0x7fc198102770 2024-03-04T13:29:58.198 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:57.372+0000 7fc197e00700 1 --2- 172.21.15.107:0/4255843688 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc198102770 0x7fc198102b80 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc1880098e0 tx=0x7fc188009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=299105a839d1487f server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:29:58.198 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:57.372+0000 7fc196a00700 1 -- 172.21.15.107:0/4255843688 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc18800b860 con 0x7fc198102770 2024-03-04T13:29:58.199 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:57.372+0000 7fc196a00700 1 -- 172.21.15.107:0/4255843688 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fc18800be50 con 0x7fc198102770 2024-03-04T13:29:58.199 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:57.372+0000 7fc196a00700 1 -- 172.21.15.107:0/4255843688 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc188010380 con 0x7fc198102770 2024-03-04T13:29:58.199 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:57.373+0000 7fc19ec00700 1 -- 172.21.15.107:0/4255843688 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc198102770 msgr2=0x7fc198102b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:29:58.199 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:57.373+0000 7fc19ec00700 1 --2- 172.21.15.107:0/4255843688 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc198102770 0x7fc198102b80 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc1880098e0 tx=0x7fc188009250 comp rx=0 tx=0).stop 2024-03-04T13:29:58.199 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:57.373+0000 7fc19ec00700 1 -- 172.21.15.107:0/4255843688 shutdown_connections 2024-03-04T13:29:58.199 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:57.373+0000 7fc19ec00700 1 --2- 172.21.15.107:0/4255843688 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc198102770 0x7fc198102b80 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:29:58.199 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:57.373+0000 7fc19ec00700 1 -- 172.21.15.107:0/4255843688 >> 172.21.15.107:0/4255843688 conn(0x7fc1980fded0 msgr2=0x7fc198100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:29:58.199 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:57.373+0000 7fc19ec00700 1 -- 172.21.15.107:0/4255843688 shutdown_connections 2024-03-04T13:29:58.199 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:57.373+0000 7fc19ec00700 1 -- 172.21.15.107:0/4255843688 wait complete. 2024-03-04T13:29:58.199 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:57.374+0000 7fc19ec00700 1 Processor -- start 2024-03-04T13:29:58.199 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:57.374+0000 7fc19ec00700 1 -- start start 2024-03-04T13:29:58.199 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:57.374+0000 7fc19ec00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc198102770 0x7fc198077c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:29:58.199 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:57.374+0000 7fc19ec00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc198076080 con 0x7fc198102770 2024-03-04T13:29:58.200 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:57.374+0000 7fc197e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc198102770 0x7fc198077c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:29:58.200 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:57.374+0000 7fc197e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc198102770 0x7fc198077c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:58450/0 (socket says 172.21.15.107:58450) 2024-03-04T13:29:58.200 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:57.374+0000 7fc197e00700 1 -- 172.21.15.107:0/3755645321 learned_addr learned my addr 172.21.15.107:0/3755645321 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:29:58.200 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:57.375+0000 7fc197e00700 1 -- 172.21.15.107:0/3755645321 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1880094f0 con 0x7fc198102770 2024-03-04T13:29:58.200 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:57.375+0000 7fc197e00700 1 --2- 172.21.15.107:0/3755645321 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc198102770 0x7fc198077c00 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc18800b440 tx=0x7fc188009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:29:58.200 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:57.375+0000 7fc18fe00700 1 -- 172.21.15.107:0/3755645321 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc1880107c0 con 0x7fc198102770 2024-03-04T13:29:58.200 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:57.375+0000 7fc18fe00700 1 -- 172.21.15.107:0/3755645321 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fc188010db0 con 0x7fc198102770 2024-03-04T13:29:58.200 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:57.375+0000 7fc18fe00700 1 -- 172.21.15.107:0/3755645321 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc18801ada0 con 0x7fc198102770 2024-03-04T13:29:58.200 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:57.375+0000 7fc19ec00700 1 -- 172.21.15.107:0/3755645321 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc198076280 con 0x7fc198102770 2024-03-04T13:29:58.200 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:57.376+0000 7fc19ec00700 1 -- 172.21.15.107:0/3755645321 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc198076710 con 0x7fc198102770 2024-03-04T13:29:58.200 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:57.376+0000 7fc18fe00700 1 -- 172.21.15.107:0/3755645321 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fc18802e6c0 con 0x7fc198102770 2024-03-04T13:29:58.200 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:57.376+0000 7fc18fe00700 1 -- 172.21.15.107:0/3755645321 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fc1880249a0 con 0x7fc198102770 2024-03-04T13:29:58.200 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:57.377+0000 7fc19ec00700 1 -- 172.21.15.107:0/3755645321 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc19810b670 con 0x7fc198102770 2024-03-04T13:29:58.201 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:57.379+0000 7fc18fe00700 1 -- 172.21.15.107:0/3755645321 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fc188010920 con 0x7fc198102770 2024-03-04T13:29:58.201 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:57.457+0000 7fc19ec00700 1 -- 172.21.15.107:0/3755645321 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fc1981045e0 con 0x7fc198102770 2024-03-04T13:29:58.201 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:57.457+0000 7fc18fe00700 1 -- 172.21.15.107:0/3755645321 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1248 (secure 0 0 0) 0x7fc188010ac0 con 0x7fc198102770 2024-03-04T13:29:58.201 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:57.458+0000 7fc19ec00700 1 -- 172.21.15.107:0/3755645321 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc198102770 msgr2=0x7fc198077c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:29:58.201 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:57.458+0000 7fc19ec00700 1 --2- 172.21.15.107:0/3755645321 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc198102770 0x7fc198077c00 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc18800b440 tx=0x7fc188009250 comp rx=0 tx=0).stop 2024-03-04T13:29:58.201 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:57.458+0000 7fc19ec00700 1 -- 172.21.15.107:0/3755645321 shutdown_connections 2024-03-04T13:29:58.201 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:57.458+0000 7fc19ec00700 1 --2- 172.21.15.107:0/3755645321 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc198102770 0x7fc198077c00 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:29:58.201 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:57.458+0000 7fc19ec00700 1 -- 172.21.15.107:0/3755645321 >> 172.21.15.107:0/3755645321 conn(0x7fc1980fded0 msgr2=0x7fc1980fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:29:58.202 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:57.459+0000 7fc19ec00700 1 -- 172.21.15.107:0/3755645321 shutdown_connections 2024-03-04T13:29:58.202 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:29:57.459+0000 7fc19ec00700 1 -- 172.21.15.107:0/3755645321 wait complete. 2024-03-04T13:29:58.202 INFO:teuthology.orchestra.run.smithi107.stdout:mgr not available, waiting (2/15)... 2024-03-04T13:30:01.782 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:01 smithi107 ceph-mon[28739]: from='client.? 172.21.15.107:0/1123465644' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-04T13:30:01.801 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-03-04T13:30:01.801 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout { 2024-03-04T13:30:01.801 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "fsid": "1fc09a58-da2b-11ee-95c6-87774f69a715", 2024-03-04T13:30:01.801 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "health": { 2024-03-04T13:30:01.801 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-04T13:30:01.801 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-04T13:30:01.802 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-04T13:30:01.802 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-03-04T13:30:01.802 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-04T13:30:01.802 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-04T13:30:01.802 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 0 2024-03-04T13:30:01.802 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ], 2024-03-04T13:30:01.802 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-04T13:30:01.802 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "smithi107" 2024-03-04T13:30:01.802 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ], 2024-03-04T13:30:01.802 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2024-03-04T13:30:01.802 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-04T13:30:01.802 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-04T13:30:01.802 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-03-04T13:30:01.802 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-04T13:30:01.802 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-03-04T13:30:01.803 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-04T13:30:01.803 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-04T13:30:01.803 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-04T13:30:01.803 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-04T13:30:01.803 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-04T13:30:01.803 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-04T13:30:01.803 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-04T13:30:01.803 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-04T13:30:01.803 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-03-04T13:30:01.803 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-04T13:30:01.803 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-04T13:30:01.803 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-04T13:30:01.803 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-04T13:30:01.803 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-04T13:30:01.803 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-04T13:30:01.803 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-04T13:30:01.804 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-04T13:30:01.804 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-04T13:30:01.804 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-03-04T13:30:01.804 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-04T13:30:01.804 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-04T13:30:01.804 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-04T13:30:01.804 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-04T13:30:01.804 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-03-04T13:30:01.804 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-04T13:30:01.804 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-04T13:30:01.804 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-04T13:30:01.804 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-04T13:30:01.804 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-04T13:30:01.804 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-04T13:30:01.805 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "restful" 2024-03-04T13:30:01.805 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ], 2024-03-04T13:30:01.805 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-04T13:30:01.805 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-03-04T13:30:01.805 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-04T13:30:01.805 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-04T13:30:01.805 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "modified": "2024-03-04T13:29:42.986437+0000", 2024-03-04T13:30:01.805 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-04T13:30:01.805 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-03-04T13:30:01.805 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-04T13:30:01.805 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout } 2024-03-04T13:30:01.805 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:01.002+0000 7f41ce200700 1 Processor -- start 2024-03-04T13:30:01.805 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:01.002+0000 7f41ce200700 1 -- start start 2024-03-04T13:30:01.805 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:01.002+0000 7f41ce200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f41c8077bb0 0x7f41c80760c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:30:01.805 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:01.002+0000 7f41ce200700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41c8077fc0 con 0x7f41c8077bb0 2024-03-04T13:30:01.806 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:01.003+0000 7f41c7400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f41c8077bb0 0x7f41c80760c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:30:01.806 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:01.003+0000 7f41c7400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f41c8077bb0 0x7f41c80760c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39330/0 (socket says 172.21.15.107:39330) 2024-03-04T13:30:01.806 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:01.003+0000 7f41c7400700 1 -- 172.21.15.107:0/2889039559 learned_addr learned my addr 172.21.15.107:0/2889039559 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:30:01.806 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:01.003+0000 7f41c7400700 1 -- 172.21.15.107:0/2889039559 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41c8076600 con 0x7f41c8077bb0 2024-03-04T13:30:01.806 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:01.003+0000 7f41c7400700 1 --2- 172.21.15.107:0/2889039559 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f41c8077bb0 0x7f41c80760c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f41b80098e0 tx=0x7f41b8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcd7ceba052dbea9 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:30:01.806 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:01.004+0000 7f41c6000700 1 -- 172.21.15.107:0/2889039559 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f41b800b860 con 0x7f41c8077bb0 2024-03-04T13:30:01.806 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:01.004+0000 7f41c6000700 1 -- 172.21.15.107:0/2889039559 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f41b800be50 con 0x7f41c8077bb0 2024-03-04T13:30:01.806 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:01.004+0000 7f41c6000700 1 -- 172.21.15.107:0/2889039559 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f41b8010380 con 0x7f41c8077bb0 2024-03-04T13:30:01.806 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:01.005+0000 7f41ce200700 1 -- 172.21.15.107:0/2889039559 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f41c8077bb0 msgr2=0x7f41c80760c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:30:01.806 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:01.005+0000 7f41ce200700 1 --2- 172.21.15.107:0/2889039559 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f41c8077bb0 0x7f41c80760c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f41b80098e0 tx=0x7f41b8009250 comp rx=0 tx=0).stop 2024-03-04T13:30:01.806 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:01.005+0000 7f41ce200700 1 -- 172.21.15.107:0/2889039559 shutdown_connections 2024-03-04T13:30:01.806 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:01.005+0000 7f41ce200700 1 --2- 172.21.15.107:0/2889039559 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f41c8077bb0 0x7f41c80760c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:30:01.806 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:01.005+0000 7f41ce200700 1 -- 172.21.15.107:0/2889039559 >> 172.21.15.107:0/2889039559 conn(0x7f41c80fded0 msgr2=0x7f41c8100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:30:01.806 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:01.005+0000 7f41ce200700 1 -- 172.21.15.107:0/2889039559 shutdown_connections 2024-03-04T13:30:01.806 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:01.005+0000 7f41ce200700 1 -- 172.21.15.107:0/2889039559 wait complete. 2024-03-04T13:30:01.807 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:01.006+0000 7f41ce200700 1 Processor -- start 2024-03-04T13:30:01.807 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:01.006+0000 7f41ce200700 1 -- start start 2024-03-04T13:30:01.807 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:01.006+0000 7f41ce200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f41c8077bb0 0x7f41c8113a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:30:01.807 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:01.006+0000 7f41ce200700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41c8113fc0 con 0x7f41c8077bb0 2024-03-04T13:30:01.807 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:01.006+0000 7f41c7400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f41c8077bb0 0x7f41c8113a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:30:01.807 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:01.006+0000 7f41c7400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f41c8077bb0 0x7f41c8113a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39332/0 (socket says 172.21.15.107:39332) 2024-03-04T13:30:01.807 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:01.006+0000 7f41c7400700 1 -- 172.21.15.107:0/1123465644 learned_addr learned my addr 172.21.15.107:0/1123465644 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:30:01.807 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:01.007+0000 7f41c7400700 1 -- 172.21.15.107:0/1123465644 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41b80094f0 con 0x7f41c8077bb0 2024-03-04T13:30:01.807 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:01.007+0000 7f41c7400700 1 --2- 172.21.15.107:0/1123465644 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f41c8077bb0 0x7f41c8113a80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f41b8006fd0 tx=0x7f41b8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:30:01.807 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:01.007+0000 7f41bf400700 1 -- 172.21.15.107:0/1123465644 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f41b80107c0 con 0x7f41c8077bb0 2024-03-04T13:30:01.807 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:01.007+0000 7f41bf400700 1 -- 172.21.15.107:0/1123465644 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f41b8010db0 con 0x7f41c8077bb0 2024-03-04T13:30:01.807 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:01.007+0000 7f41ce200700 1 -- 172.21.15.107:0/1123465644 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41c81141c0 con 0x7f41c8077bb0 2024-03-04T13:30:01.807 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:01.007+0000 7f41bf400700 1 -- 172.21.15.107:0/1123465644 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f41b801add0 con 0x7f41c8077bb0 2024-03-04T13:30:01.807 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:01.008+0000 7f41ce200700 1 -- 172.21.15.107:0/1123465644 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41c8114e60 con 0x7f41c8077bb0 2024-03-04T13:30:01.808 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:01.008+0000 7f41bf400700 1 -- 172.21.15.107:0/1123465644 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f41b80226d0 con 0x7f41c8077bb0 2024-03-04T13:30:01.808 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:01.008+0000 7f41bf400700 1 -- 172.21.15.107:0/1123465644 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f41b801e070 con 0x7f41c8077bb0 2024-03-04T13:30:01.808 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:01.009+0000 7f41ce200700 1 -- 172.21.15.107:0/1123465644 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41c805f5c0 con 0x7f41c8077bb0 2024-03-04T13:30:01.808 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:01.011+0000 7f41bf400700 1 -- 172.21.15.107:0/1123465644 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f41b801a5c0 con 0x7f41c8077bb0 2024-03-04T13:30:01.808 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:01.089+0000 7f41ce200700 1 -- 172.21.15.107:0/1123465644 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f41c8064ad0 con 0x7f41c8077bb0 2024-03-04T13:30:01.808 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:01.090+0000 7f41bf400700 1 -- 172.21.15.107:0/1123465644 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1249 (secure 0 0 0) 0x7f41b8010920 con 0x7f41c8077bb0 2024-03-04T13:30:01.808 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:01.092+0000 7f41ce200700 1 -- 172.21.15.107:0/1123465644 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f41c8077bb0 msgr2=0x7f41c8113a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:30:01.808 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:01.092+0000 7f41ce200700 1 --2- 172.21.15.107:0/1123465644 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f41c8077bb0 0x7f41c8113a80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f41b8006fd0 tx=0x7f41b8009250 comp rx=0 tx=0).stop 2024-03-04T13:30:01.808 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:01.092+0000 7f41ce200700 1 -- 172.21.15.107:0/1123465644 shutdown_connections 2024-03-04T13:30:01.808 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:01.092+0000 7f41ce200700 1 --2- 172.21.15.107:0/1123465644 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f41c8077bb0 0x7f41c8113a80 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:30:01.808 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:01.092+0000 7f41ce200700 1 -- 172.21.15.107:0/1123465644 >> 172.21.15.107:0/1123465644 conn(0x7f41c80fded0 msgr2=0x7f41c8100190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:30:01.808 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:01.092+0000 7f41ce200700 1 -- 172.21.15.107:0/1123465644 shutdown_connections 2024-03-04T13:30:01.808 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:01.092+0000 7f41ce200700 1 -- 172.21.15.107:0/1123465644 wait complete. 2024-03-04T13:30:01.808 INFO:teuthology.orchestra.run.smithi107.stdout:mgr not available, waiting (3/15)... 2024-03-04T13:30:02.838 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:02 smithi107 ceph-mon[28739]: Activating manager daemon smithi107.wpzgwl 2024-03-04T13:30:02.838 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:02 smithi107 ceph-mon[28739]: mgrmap e2: smithi107.wpzgwl(active, starting, since 0.00816177s) 2024-03-04T13:30:02.838 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:02 smithi107 ceph-mon[28739]: from='mgr.14100 172.21.15.107:0/1298465622' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-04T13:30:02.838 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:02 smithi107 ceph-mon[28739]: from='mgr.14100 172.21.15.107:0/1298465622' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-04T13:30:02.838 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:02 smithi107 ceph-mon[28739]: from='mgr.14100 172.21.15.107:0/1298465622' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-04T13:30:02.838 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:02 smithi107 ceph-mon[28739]: from='mgr.14100 172.21.15.107:0/1298465622' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-04T13:30:02.838 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:02 smithi107 ceph-mon[28739]: from='mgr.14100 172.21.15.107:0/1298465622' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-04T13:30:02.838 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:02 smithi107 ceph-mon[28739]: from='mgr.14100 172.21.15.107:0/1298465622' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-04T13:30:02.838 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:02 smithi107 ceph-mon[28739]: from='mgr.14100 172.21.15.107:0/1298465622' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mon metadata", "id": "smithi107"}]: dispatch 2024-03-04T13:30:02.838 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:02 smithi107 ceph-mon[28739]: from='mgr.14100 172.21.15.107:0/1298465622' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mgr metadata", "who": "smithi107.wpzgwl", "id": "smithi107.wpzgwl"}]: dispatch 2024-03-04T13:30:02.839 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:02 smithi107 ceph-mon[28739]: Manager daemon smithi107.wpzgwl is now available 2024-03-04T13:30:02.839 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:02 smithi107 ceph-mon[28739]: from='mgr.14100 172.21.15.107:0/1298465622' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi107.wpzgwl/mirror_snapshot_schedule"}]: dispatch 2024-03-04T13:30:02.839 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:02 smithi107 ceph-mon[28739]: from='mgr.14100 172.21.15.107:0/1298465622' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi107.wpzgwl/trash_purge_schedule"}]: dispatch 2024-03-04T13:30:02.839 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:02 smithi107 ceph-mon[28739]: from='mgr.14100 172.21.15.107:0/1298465622' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:30:02.839 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:02 smithi107 ceph-mon[28739]: from='mgr.14100 172.21.15.107:0/1298465622' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:30:02.839 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:02 smithi107 ceph-mon[28739]: from='mgr.14100 172.21.15.107:0/1298465622' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:30:03.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:03 smithi107 ceph-mon[28739]: mgrmap e3: smithi107.wpzgwl(active, since 1.01918s) 2024-03-04T13:30:04.587 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:04 smithi107 ceph-mon[28739]: mgrmap e4: smithi107.wpzgwl(active, since 2s) 2024-03-04T13:30:05.300 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-03-04T13:30:05.301 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout { 2024-03-04T13:30:05.301 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "fsid": "1fc09a58-da2b-11ee-95c6-87774f69a715", 2024-03-04T13:30:05.301 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "health": { 2024-03-04T13:30:05.301 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-04T13:30:05.302 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-04T13:30:05.309 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-04T13:30:05.309 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-03-04T13:30:05.309 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-04T13:30:05.309 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-04T13:30:05.309 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 0 2024-03-04T13:30:05.309 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ], 2024-03-04T13:30:05.309 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-04T13:30:05.309 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "smithi107" 2024-03-04T13:30:05.309 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ], 2024-03-04T13:30:05.310 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "quorum_age": 14, 2024-03-04T13:30:05.310 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-04T13:30:05.310 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-04T13:30:05.310 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-03-04T13:30:05.310 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-04T13:30:05.310 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-03-04T13:30:05.310 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-04T13:30:05.310 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-04T13:30:05.310 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-04T13:30:05.310 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-04T13:30:05.311 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-04T13:30:05.311 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-04T13:30:05.311 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-04T13:30:05.311 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-04T13:30:05.311 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-03-04T13:30:05.311 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-04T13:30:05.311 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-04T13:30:05.311 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-04T13:30:05.311 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-04T13:30:05.311 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-04T13:30:05.311 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-04T13:30:05.311 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-04T13:30:05.311 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-04T13:30:05.311 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-04T13:30:05.312 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-03-04T13:30:05.312 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-04T13:30:05.312 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-04T13:30:05.312 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-04T13:30:05.312 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-04T13:30:05.312 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-03-04T13:30:05.312 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-04T13:30:05.312 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-04T13:30:05.312 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-04T13:30:05.312 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-04T13:30:05.312 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-04T13:30:05.312 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-04T13:30:05.312 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "restful" 2024-03-04T13:30:05.313 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ], 2024-03-04T13:30:05.313 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-04T13:30:05.313 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-03-04T13:30:05.313 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-04T13:30:05.313 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-04T13:30:05.313 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "modified": "2024-03-04T13:29:42.986437+0000", 2024-03-04T13:30:05.313 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-04T13:30:05.313 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-03-04T13:30:05.313 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-04T13:30:05.313 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout } 2024-03-04T13:30:05.313 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:04.400+0000 7fb3f3e00700 1 Processor -- start 2024-03-04T13:30:05.313 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:04.400+0000 7fb3f3e00700 1 -- start start 2024-03-04T13:30:05.313 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:04.401+0000 7fb3f3e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb3f40714c0 0x7fb3f4102f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:30:05.319 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:04.401+0000 7fb3f3e00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f4103490 con 0x7fb3f40714c0 2024-03-04T13:30:05.319 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:04.401+0000 7fb3f2a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb3f40714c0 0x7fb3f4102f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:30:05.319 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:04.401+0000 7fb3f2a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb3f40714c0 0x7fb3f4102f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39400/0 (socket says 172.21.15.107:39400) 2024-03-04T13:30:05.319 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:04.401+0000 7fb3f2a00700 1 -- 172.21.15.107:0/1188557368 learned_addr learned my addr 172.21.15.107:0/1188557368 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:30:05.319 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:04.402+0000 7fb3f2a00700 1 -- 172.21.15.107:0/1188557368 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3f41035d0 con 0x7fb3f40714c0 2024-03-04T13:30:05.319 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:04.402+0000 7fb3f2a00700 1 --2- 172.21.15.107:0/1188557368 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb3f40714c0 0x7fb3f4102f50 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb3e40098e0 tx=0x7fb3e4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da2a889cd43744a8 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:30:05.319 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:04.402+0000 7fb3f1600700 1 -- 172.21.15.107:0/1188557368 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb3e400b860 con 0x7fb3f40714c0 2024-03-04T13:30:05.319 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:04.403+0000 7fb3f1600700 1 -- 172.21.15.107:0/1188557368 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fb3e400be50 con 0x7fb3f40714c0 2024-03-04T13:30:05.320 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:04.403+0000 7fb3f1600700 1 -- 172.21.15.107:0/1188557368 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb3e4010380 con 0x7fb3f40714c0 2024-03-04T13:30:05.320 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:04.403+0000 7fb3f3e00700 1 -- 172.21.15.107:0/1188557368 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb3f40714c0 msgr2=0x7fb3f4102f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:30:05.320 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:04.403+0000 7fb3f3e00700 1 --2- 172.21.15.107:0/1188557368 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb3f40714c0 0x7fb3f4102f50 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb3e40098e0 tx=0x7fb3e4009250 comp rx=0 tx=0).stop 2024-03-04T13:30:05.320 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:04.403+0000 7fb3f3e00700 1 -- 172.21.15.107:0/1188557368 shutdown_connections 2024-03-04T13:30:05.320 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:04.403+0000 7fb3f3e00700 1 --2- 172.21.15.107:0/1188557368 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb3f40714c0 0x7fb3f4102f50 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:30:05.320 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:04.403+0000 7fb3f3e00700 1 -- 172.21.15.107:0/1188557368 >> 172.21.15.107:0/1188557368 conn(0x7fb3f40fcc10 msgr2=0x7fb3f40ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:30:05.320 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:04.404+0000 7fb3f3e00700 1 -- 172.21.15.107:0/1188557368 shutdown_connections 2024-03-04T13:30:05.320 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:04.404+0000 7fb3f3e00700 1 -- 172.21.15.107:0/1188557368 wait complete. 2024-03-04T13:30:05.320 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:04.404+0000 7fb3f3e00700 1 Processor -- start 2024-03-04T13:30:05.320 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:04.405+0000 7fb3f3e00700 1 -- start start 2024-03-04T13:30:05.320 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:04.405+0000 7fb3f3e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb3f40714c0 0x7fb3f4117960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:30:05.320 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:04.405+0000 7fb3f3e00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f4117ea0 con 0x7fb3f40714c0 2024-03-04T13:30:05.320 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:04.405+0000 7fb3f2a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb3f40714c0 0x7fb3f4117960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:30:05.321 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:04.405+0000 7fb3f2a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb3f40714c0 0x7fb3f4117960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39404/0 (socket says 172.21.15.107:39404) 2024-03-04T13:30:05.321 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:04.405+0000 7fb3f2a00700 1 -- 172.21.15.107:0/3900993340 learned_addr learned my addr 172.21.15.107:0/3900993340 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:30:05.321 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:04.405+0000 7fb3f2a00700 1 -- 172.21.15.107:0/3900993340 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3e40094f0 con 0x7fb3f40714c0 2024-03-04T13:30:05.321 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:04.406+0000 7fb3f2a00700 1 --2- 172.21.15.107:0/3900993340 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb3f40714c0 0x7fb3f4117960 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb3e400b440 tx=0x7fb3e4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:30:05.321 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:04.406+0000 7fb3eaa00700 1 -- 172.21.15.107:0/3900993340 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb3e40107c0 con 0x7fb3f40714c0 2024-03-04T13:30:05.321 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:04.406+0000 7fb3eaa00700 1 -- 172.21.15.107:0/3900993340 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fb3e4010db0 con 0x7fb3f40714c0 2024-03-04T13:30:05.321 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:04.406+0000 7fb3eaa00700 1 -- 172.21.15.107:0/3900993340 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb3e401ad10 con 0x7fb3f40714c0 2024-03-04T13:30:05.321 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:04.406+0000 7fb3f3e00700 1 -- 172.21.15.107:0/3900993340 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3f41180a0 con 0x7fb3f40714c0 2024-03-04T13:30:05.321 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:04.406+0000 7fb3f3e00700 1 -- 172.21.15.107:0/3900993340 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3f4118d40 con 0x7fb3f40714c0 2024-03-04T13:30:05.321 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:04.407+0000 7fb3f3e00700 1 -- 172.21.15.107:0/3900993340 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3f405f5c0 con 0x7fb3f40714c0 2024-03-04T13:30:05.321 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:04.408+0000 7fb3eaa00700 1 -- 172.21.15.107:0/3900993340 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 4) v1 ==== 43357+0+0 (secure 0 0 0) 0x7fb3e4010920 con 0x7fb3f40714c0 2024-03-04T13:30:05.321 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:04.408+0000 7fb3eaa00700 1 --2- 172.21.15.107:0/3900993340 >> [v2:172.21.15.107:6800/1723390229,v1:172.21.15.107:6801/1723390229] conn(0x7fb3d8036790 0x7fb3d8038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:30:05.321 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:04.408+0000 7fb3eaa00700 1 -- 172.21.15.107:0/3900993340 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fb3e404a790 con 0x7fb3f40714c0 2024-03-04T13:30:05.321 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:04.409+0000 7fb3f2000700 1 --2- 172.21.15.107:0/3900993340 >> [v2:172.21.15.107:6800/1723390229,v1:172.21.15.107:6801/1723390229] conn(0x7fb3d8036790 0x7fb3d8038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:30:05.322 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:04.410+0000 7fb3f2000700 1 --2- 172.21.15.107:0/3900993340 >> [v2:172.21.15.107:6800/1723390229,v1:172.21.15.107:6801/1723390229] conn(0x7fb3d8036790 0x7fb3d8038c40 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb3e0009990 tx=0x7fb3e0006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:30:05.322 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:04.414+0000 7fb3eaa00700 1 -- 172.21.15.107:0/3900993340 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fb3e401a4c0 con 0x7fb3f40714c0 2024-03-04T13:30:05.322 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:04.702+0000 7fb3f3e00700 1 -- 172.21.15.107:0/3900993340 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fb3f404bed0 con 0x7fb3f40714c0 2024-03-04T13:30:05.322 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:04.703+0000 7fb3eaa00700 1 -- 172.21.15.107:0/3900993340 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1248 (secure 0 0 0) 0x7fb3e4023b80 con 0x7fb3f40714c0 2024-03-04T13:30:05.322 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:04.706+0000 7fb3f3e00700 1 -- 172.21.15.107:0/3900993340 >> [v2:172.21.15.107:6800/1723390229,v1:172.21.15.107:6801/1723390229] conn(0x7fb3d8036790 msgr2=0x7fb3d8038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:30:05.322 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:04.706+0000 7fb3f3e00700 1 --2- 172.21.15.107:0/3900993340 >> [v2:172.21.15.107:6800/1723390229,v1:172.21.15.107:6801/1723390229] conn(0x7fb3d8036790 0x7fb3d8038c40 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb3e0009990 tx=0x7fb3e0006e40 comp rx=0 tx=0).stop 2024-03-04T13:30:05.322 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:04.706+0000 7fb3f3e00700 1 -- 172.21.15.107:0/3900993340 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb3f40714c0 msgr2=0x7fb3f4117960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:30:05.322 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:04.706+0000 7fb3f3e00700 1 --2- 172.21.15.107:0/3900993340 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb3f40714c0 0x7fb3f4117960 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb3e400b440 tx=0x7fb3e4009250 comp rx=0 tx=0).stop 2024-03-04T13:30:05.322 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:04.706+0000 7fb3f3e00700 1 -- 172.21.15.107:0/3900993340 shutdown_connections 2024-03-04T13:30:05.322 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:04.706+0000 7fb3f3e00700 1 --2- 172.21.15.107:0/3900993340 >> [v2:172.21.15.107:6800/1723390229,v1:172.21.15.107:6801/1723390229] conn(0x7fb3d8036790 0x7fb3d8038c40 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:30:05.322 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:04.706+0000 7fb3f3e00700 1 --2- 172.21.15.107:0/3900993340 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb3f40714c0 0x7fb3f4117960 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:30:05.322 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:04.706+0000 7fb3f3e00700 1 -- 172.21.15.107:0/3900993340 >> 172.21.15.107:0/3900993340 conn(0x7fb3f40fcc10 msgr2=0x7fb3f40fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:30:05.323 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:04.707+0000 7fb3f3e00700 1 -- 172.21.15.107:0/3900993340 shutdown_connections 2024-03-04T13:30:05.323 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:04.707+0000 7fb3f3e00700 1 -- 172.21.15.107:0/3900993340 wait complete. 2024-03-04T13:30:05.323 INFO:teuthology.orchestra.run.smithi107.stdout:mgr is available 2024-03-04T13:30:05.587 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:05 smithi107 ceph-mon[28739]: from='client.? 172.21.15.107:0/3900993340' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-04T13:30:06.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:06 smithi107 ceph-mon[28739]: from='client.? 172.21.15.107:0/1694110927' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-04T13:30:06.986 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-03-04T13:30:06.986 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout [global] 2024-03-04T13:30:06.986 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout fsid = 1fc09a58-da2b-11ee-95c6-87774f69a715 2024-03-04T13:30:06.986 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-04T13:30:06.986 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-04T13:30:06.986 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-04T13:30:06.986 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-04T13:30:06.986 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-03-04T13:30:06.986 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-04T13:30:06.986 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-04T13:30:06.986 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-03-04T13:30:06.986 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout [osd] 2024-03-04T13:30:06.987 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-04T13:30:06.987 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-04T13:30:06.987 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-04T13:30:06.987 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-04T13:30:06.987 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-04T13:30:06.987 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:05.957+0000 7fc712200700 1 Processor -- start 2024-03-04T13:30:06.987 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:05.957+0000 7fc712200700 1 -- start start 2024-03-04T13:30:06.987 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:05.957+0000 7fc712200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc70c104980 0x7fc70c104d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:30:06.987 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:05.957+0000 7fc712200700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc70c1052d0 con 0x7fc70c104980 2024-03-04T13:30:06.987 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:05.958+0000 7fc70b400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc70c104980 0x7fc70c104d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:30:06.987 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:05.958+0000 7fc70b400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc70c104980 0x7fc70c104d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39416/0 (socket says 172.21.15.107:39416) 2024-03-04T13:30:06.987 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:05.958+0000 7fc70b400700 1 -- 172.21.15.107:0/601921968 learned_addr learned my addr 172.21.15.107:0/601921968 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:30:06.987 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:05.959+0000 7fc70b400700 1 -- 172.21.15.107:0/601921968 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc70c105410 con 0x7fc70c104980 2024-03-04T13:30:06.988 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:05.959+0000 7fc70b400700 1 --2- 172.21.15.107:0/601921968 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc70c104980 0x7fc70c104d90 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc0098e0 tx=0x7fc6fc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=345d07febdfc85b9 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:30:06.988 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:05.959+0000 7fc70a000700 1 -- 172.21.15.107:0/601921968 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc6fc00b860 con 0x7fc70c104980 2024-03-04T13:30:06.988 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:05.960+0000 7fc70a000700 1 -- 172.21.15.107:0/601921968 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fc6fc00be50 con 0x7fc70c104980 2024-03-04T13:30:06.988 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:05.960+0000 7fc70a000700 1 -- 172.21.15.107:0/601921968 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc6fc010380 con 0x7fc70c104980 2024-03-04T13:30:06.988 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:05.960+0000 7fc712200700 1 -- 172.21.15.107:0/601921968 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc70c104980 msgr2=0x7fc70c104d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:30:06.988 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:05.960+0000 7fc712200700 1 --2- 172.21.15.107:0/601921968 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc70c104980 0x7fc70c104d90 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc0098e0 tx=0x7fc6fc009250 comp rx=0 tx=0).stop 2024-03-04T13:30:06.988 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:05.960+0000 7fc712200700 1 -- 172.21.15.107:0/601921968 shutdown_connections 2024-03-04T13:30:06.988 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:05.960+0000 7fc712200700 1 --2- 172.21.15.107:0/601921968 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc70c104980 0x7fc70c104d90 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:30:06.988 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:05.960+0000 7fc712200700 1 -- 172.21.15.107:0/601921968 >> 172.21.15.107:0/601921968 conn(0x7fc70c078030 msgr2=0x7fc70c078450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:30:06.988 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:05.960+0000 7fc712200700 1 -- 172.21.15.107:0/601921968 shutdown_connections 2024-03-04T13:30:06.988 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:05.960+0000 7fc712200700 1 -- 172.21.15.107:0/601921968 wait complete. 2024-03-04T13:30:06.988 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:05.961+0000 7fc712200700 1 Processor -- start 2024-03-04T13:30:06.988 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:05.962+0000 7fc712200700 1 -- start start 2024-03-04T13:30:06.988 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:05.962+0000 7fc712200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc70c104980 0x7fc70c113760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:30:06.989 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:05.962+0000 7fc712200700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc70c113ca0 con 0x7fc70c104980 2024-03-04T13:30:06.989 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:05.962+0000 7fc70b400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc70c104980 0x7fc70c113760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:30:06.989 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:05.962+0000 7fc70b400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc70c104980 0x7fc70c113760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39426/0 (socket says 172.21.15.107:39426) 2024-03-04T13:30:06.989 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:05.962+0000 7fc70b400700 1 -- 172.21.15.107:0/1694110927 learned_addr learned my addr 172.21.15.107:0/1694110927 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:30:06.989 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:05.963+0000 7fc70b400700 1 -- 172.21.15.107:0/1694110927 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6fc0094f0 con 0x7fc70c104980 2024-03-04T13:30:06.989 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:05.963+0000 7fc70b400700 1 --2- 172.21.15.107:0/1694110927 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc70c104980 0x7fc70c113760 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc000c00 tx=0x7fc6fc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:30:06.989 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:05.963+0000 7fc703400700 1 -- 172.21.15.107:0/1694110927 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc6fc0107c0 con 0x7fc70c104980 2024-03-04T13:30:06.989 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:05.963+0000 7fc703400700 1 -- 172.21.15.107:0/1694110927 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fc6fc010db0 con 0x7fc70c104980 2024-03-04T13:30:06.989 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:05.963+0000 7fc712200700 1 -- 172.21.15.107:0/1694110927 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc70c113ea0 con 0x7fc70c104980 2024-03-04T13:30:06.989 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:05.963+0000 7fc703400700 1 -- 172.21.15.107:0/1694110927 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc6fc01ad00 con 0x7fc70c104980 2024-03-04T13:30:06.989 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:05.963+0000 7fc712200700 1 -- 172.21.15.107:0/1694110927 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc70c114b40 con 0x7fc70c104980 2024-03-04T13:30:06.989 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:05.964+0000 7fc703400700 1 -- 172.21.15.107:0/1694110927 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 4) v1 ==== 43357+0+0 (secure 0 0 0) 0x7fc6fc0193c0 con 0x7fc70c104980 2024-03-04T13:30:06.989 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:05.965+0000 7fc712200700 1 -- 172.21.15.107:0/1694110927 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc70c05f5c0 con 0x7fc70c104980 2024-03-04T13:30:06.989 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:05.965+0000 7fc703400700 1 --2- 172.21.15.107:0/1694110927 >> [v2:172.21.15.107:6800/1723390229,v1:172.21.15.107:6801/1723390229] conn(0x7fc6f0036790 0x7fc6f0038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:30:06.990 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:05.965+0000 7fc703400700 1 -- 172.21.15.107:0/1694110927 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fc6fc04a5d0 con 0x7fc70c104980 2024-03-04T13:30:06.990 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:05.965+0000 7fc70aa00700 1 --2- 172.21.15.107:0/1694110927 >> [v2:172.21.15.107:6800/1723390229,v1:172.21.15.107:6801/1723390229] conn(0x7fc6f0036790 0x7fc6f0038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:30:06.990 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:05.966+0000 7fc70aa00700 1 --2- 172.21.15.107:0/1694110927 >> [v2:172.21.15.107:6800/1723390229,v1:172.21.15.107:6801/1723390229] conn(0x7fc6f0036790 0x7fc6f0038c40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc6f8009990 tx=0x7fc6f8006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:30:06.990 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:05.970+0000 7fc703400700 1 -- 172.21.15.107:0/1694110927 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fc6fc019670 con 0x7fc70c104980 2024-03-04T13:30:06.990 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:06.177+0000 7fc712200700 1 -- 172.21.15.107:0/1694110927 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fc70c04bed0 con 0x7fc70c104980 2024-03-04T13:30:06.990 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:06.179+0000 7fc703400700 1 -- 172.21.15.107:0/1694110927 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7fc6fc022340 con 0x7fc70c104980 2024-03-04T13:30:06.990 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:06.183+0000 7fc712200700 1 -- 172.21.15.107:0/1694110927 >> [v2:172.21.15.107:6800/1723390229,v1:172.21.15.107:6801/1723390229] conn(0x7fc6f0036790 msgr2=0x7fc6f0038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:30:06.990 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:06.183+0000 7fc712200700 1 --2- 172.21.15.107:0/1694110927 >> [v2:172.21.15.107:6800/1723390229,v1:172.21.15.107:6801/1723390229] conn(0x7fc6f0036790 0x7fc6f0038c40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc6f8009990 tx=0x7fc6f8006e40 comp rx=0 tx=0).stop 2024-03-04T13:30:06.990 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:06.183+0000 7fc712200700 1 -- 172.21.15.107:0/1694110927 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc70c104980 msgr2=0x7fc70c113760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:30:06.990 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:06.183+0000 7fc712200700 1 --2- 172.21.15.107:0/1694110927 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc70c104980 0x7fc70c113760 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc000c00 tx=0x7fc6fc009250 comp rx=0 tx=0).stop 2024-03-04T13:30:06.990 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:06.184+0000 7fc712200700 1 -- 172.21.15.107:0/1694110927 shutdown_connections 2024-03-04T13:30:06.990 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:06.184+0000 7fc712200700 1 --2- 172.21.15.107:0/1694110927 >> [v2:172.21.15.107:6800/1723390229,v1:172.21.15.107:6801/1723390229] conn(0x7fc6f0036790 0x7fc6f0038c40 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:30:06.990 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:06.184+0000 7fc712200700 1 --2- 172.21.15.107:0/1694110927 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc70c104980 0x7fc70c113760 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:30:06.990 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:06.184+0000 7fc712200700 1 -- 172.21.15.107:0/1694110927 >> 172.21.15.107:0/1694110927 conn(0x7fc70c078030 msgr2=0x7fc70c102820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:30:06.990 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:06.184+0000 7fc712200700 1 -- 172.21.15.107:0/1694110927 shutdown_connections 2024-03-04T13:30:06.991 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:06.184+0000 7fc712200700 1 -- 172.21.15.107:0/1694110927 wait complete. 2024-03-04T13:30:06.991 INFO:teuthology.orchestra.run.smithi107.stdout:Enabling cephadm module... 2024-03-04T13:30:08.250 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:07 smithi107 ceph-mon[28739]: from='client.? 172.21.15.107:0/900376436' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-03-04T13:30:08.649 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:07.574+0000 7f2c2c400700 1 Processor -- start 2024-03-04T13:30:08.649 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:07.574+0000 7f2c2c400700 1 -- start start 2024-03-04T13:30:08.649 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:07.574+0000 7f2c2c400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c24108f20 0x7f2c24109330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:30:08.649 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:07.575+0000 7f2c2c400700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c24109870 con 0x7f2c24108f20 2024-03-04T13:30:08.649 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:07.575+0000 7f2c29a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c24108f20 0x7f2c24109330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:30:08.649 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:07.575+0000 7f2c29a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c24108f20 0x7f2c24109330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39440/0 (socket says 172.21.15.107:39440) 2024-03-04T13:30:08.649 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:07.575+0000 7f2c29a00700 1 -- 172.21.15.107:0/3290595631 learned_addr learned my addr 172.21.15.107:0/3290595631 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:30:08.650 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:07.576+0000 7f2c29a00700 1 -- 172.21.15.107:0/3290595631 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c241099b0 con 0x7f2c24108f20 2024-03-04T13:30:08.650 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:07.576+0000 7f2c29a00700 1 --2- 172.21.15.107:0/3290595631 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c24108f20 0x7f2c24109330 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f2c18009e50 tx=0x7f2c18009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85b268edf146737b server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:30:08.650 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:07.576+0000 7f2c23e00700 1 -- 172.21.15.107:0/3290595631 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2c180092e0 con 0x7f2c24108f20 2024-03-04T13:30:08.650 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:07.576+0000 7f2c23e00700 1 -- 172.21.15.107:0/3290595631 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f2c18012430 con 0x7f2c24108f20 2024-03-04T13:30:08.650 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:07.576+0000 7f2c23e00700 1 -- 172.21.15.107:0/3290595631 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2c180075c0 con 0x7f2c24108f20 2024-03-04T13:30:08.650 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:07.577+0000 7f2c2c400700 1 -- 172.21.15.107:0/3290595631 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c24108f20 msgr2=0x7f2c24109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:30:08.650 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:07.577+0000 7f2c2c400700 1 --2- 172.21.15.107:0/3290595631 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c24108f20 0x7f2c24109330 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f2c18009e50 tx=0x7f2c18009840 comp rx=0 tx=0).stop 2024-03-04T13:30:08.650 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:07.577+0000 7f2c2c400700 1 -- 172.21.15.107:0/3290595631 shutdown_connections 2024-03-04T13:30:08.650 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:07.577+0000 7f2c2c400700 1 --2- 172.21.15.107:0/3290595631 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c24108f20 0x7f2c24109330 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:30:08.650 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:07.577+0000 7f2c2c400700 1 -- 172.21.15.107:0/3290595631 >> 172.21.15.107:0/3290595631 conn(0x7f2c24078030 msgr2=0x7f2c2407a460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:30:08.650 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:07.577+0000 7f2c2c400700 1 -- 172.21.15.107:0/3290595631 shutdown_connections 2024-03-04T13:30:08.650 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:07.577+0000 7f2c2c400700 1 -- 172.21.15.107:0/3290595631 wait complete. 2024-03-04T13:30:08.650 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:07.578+0000 7f2c2c400700 1 Processor -- start 2024-03-04T13:30:08.650 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:07.578+0000 7f2c2c400700 1 -- start start 2024-03-04T13:30:08.651 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:07.579+0000 7f2c2c400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c24108f20 0x7f2c24117bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:30:08.651 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:07.579+0000 7f2c2c400700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c24118100 con 0x7f2c24108f20 2024-03-04T13:30:08.651 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:07.579+0000 7f2c29a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c24108f20 0x7f2c24117bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:30:08.651 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:07.579+0000 7f2c29a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c24108f20 0x7f2c24117bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39452/0 (socket says 172.21.15.107:39452) 2024-03-04T13:30:08.651 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:07.579+0000 7f2c29a00700 1 -- 172.21.15.107:0/900376436 learned_addr learned my addr 172.21.15.107:0/900376436 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:30:08.651 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:07.579+0000 7f2c29a00700 1 -- 172.21.15.107:0/900376436 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c18009ae0 con 0x7f2c24108f20 2024-03-04T13:30:08.651 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:07.580+0000 7f2c29a00700 1 --2- 172.21.15.107:0/900376436 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c24108f20 0x7f2c24117bc0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2c18009fd0 tx=0x7f2c18009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:30:08.651 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:07.580+0000 7f2c21600700 1 -- 172.21.15.107:0/900376436 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2c180092e0 con 0x7f2c24108f20 2024-03-04T13:30:08.651 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:07.580+0000 7f2c21600700 1 -- 172.21.15.107:0/900376436 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f2c18007cc0 con 0x7f2c24108f20 2024-03-04T13:30:08.652 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:07.580+0000 7f2c21600700 1 -- 172.21.15.107:0/900376436 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2c1801c8c0 con 0x7f2c24108f20 2024-03-04T13:30:08.652 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:07.580+0000 7f2c2c400700 1 -- 172.21.15.107:0/900376436 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c24118300 con 0x7f2c24108f20 2024-03-04T13:30:08.652 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:07.580+0000 7f2c2c400700 1 -- 172.21.15.107:0/900376436 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c24118fa0 con 0x7f2c24108f20 2024-03-04T13:30:08.652 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:07.581+0000 7f2c21600700 1 -- 172.21.15.107:0/900376436 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 4) v1 ==== 43357+0+0 (secure 0 0 0) 0x7f2c18019070 con 0x7f2c24108f20 2024-03-04T13:30:08.652 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:07.581+0000 7f2c2c400700 1 -- 172.21.15.107:0/900376436 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c2405f5c0 con 0x7f2c24108f20 2024-03-04T13:30:08.652 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:07.582+0000 7f2c21600700 1 --2- 172.21.15.107:0/900376436 >> [v2:172.21.15.107:6800/1723390229,v1:172.21.15.107:6801/1723390229] conn(0x7f2c0c0367e0 0x7f2c0c038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:30:08.652 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:07.582+0000 7f2c21600700 1 -- 172.21.15.107:0/900376436 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f2c18049b50 con 0x7f2c24108f20 2024-03-04T13:30:08.652 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:07.582+0000 7f2c29000700 1 --2- 172.21.15.107:0/900376436 >> [v2:172.21.15.107:6800/1723390229,v1:172.21.15.107:6801/1723390229] conn(0x7f2c0c0367e0 0x7f2c0c038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:30:08.652 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:07.583+0000 7f2c29000700 1 --2- 172.21.15.107:0/900376436 >> [v2:172.21.15.107:6800/1723390229,v1:172.21.15.107:6801/1723390229] conn(0x7f2c0c0367e0 0x7f2c0c038c90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2c10009990 tx=0x7f2c10006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:30:08.652 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:07.587+0000 7f2c21600700 1 -- 172.21.15.107:0/900376436 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f2c18011390 con 0x7f2c24108f20 2024-03-04T13:30:08.652 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:07.848+0000 7f2c2c400700 1 -- 172.21.15.107:0/900376436 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f2c24119340 con 0x7f2c24108f20 2024-03-04T13:30:08.652 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:07.905+0000 7f2c21600700 1 -- 172.21.15.107:0/900376436 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mgrmap(e 5) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f2c18019320 con 0x7f2c24108f20 2024-03-04T13:30:08.652 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:07.907+0000 7f2c21600700 1 -- 172.21.15.107:0/900376436 <== mon.0 v2:172.21.15.107:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f2c18002c90 con 0x7f2c24108f20 2024-03-04T13:30:08.653 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:07.910+0000 7f2c2c400700 1 -- 172.21.15.107:0/900376436 >> [v2:172.21.15.107:6800/1723390229,v1:172.21.15.107:6801/1723390229] conn(0x7f2c0c0367e0 msgr2=0x7f2c0c038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:30:08.653 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:07.910+0000 7f2c2c400700 1 --2- 172.21.15.107:0/900376436 >> [v2:172.21.15.107:6800/1723390229,v1:172.21.15.107:6801/1723390229] conn(0x7f2c0c0367e0 0x7f2c0c038c90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2c10009990 tx=0x7f2c10006e40 comp rx=0 tx=0).stop 2024-03-04T13:30:08.653 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:07.910+0000 7f2c2c400700 1 -- 172.21.15.107:0/900376436 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c24108f20 msgr2=0x7f2c24117bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:30:08.653 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:07.910+0000 7f2c2c400700 1 --2- 172.21.15.107:0/900376436 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c24108f20 0x7f2c24117bc0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2c18009fd0 tx=0x7f2c18009840 comp rx=0 tx=0).stop 2024-03-04T13:30:08.653 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:07.910+0000 7f2c2c400700 1 -- 172.21.15.107:0/900376436 shutdown_connections 2024-03-04T13:30:08.653 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:07.910+0000 7f2c2c400700 1 --2- 172.21.15.107:0/900376436 >> [v2:172.21.15.107:6800/1723390229,v1:172.21.15.107:6801/1723390229] conn(0x7f2c0c0367e0 0x7f2c0c038c90 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:30:08.653 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:07.910+0000 7f2c2c400700 1 --2- 172.21.15.107:0/900376436 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c24108f20 0x7f2c24117bc0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:30:08.653 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:07.910+0000 7f2c2c400700 1 -- 172.21.15.107:0/900376436 >> 172.21.15.107:0/900376436 conn(0x7f2c24078030 msgr2=0x7f2c2407a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:30:08.653 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:07.910+0000 7f2c2c400700 1 -- 172.21.15.107:0/900376436 shutdown_connections 2024-03-04T13:30:08.653 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:07.910+0000 7f2c2c400700 1 -- 172.21.15.107:0/900376436 wait complete. 2024-03-04T13:30:09.222 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:08 smithi107 ceph-mon[28739]: from='client.? 172.21.15.107:0/900376436' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-03-04T13:30:09.222 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:08 smithi107 ceph-mon[28739]: mgrmap e5: smithi107.wpzgwl(active, since 6s) 2024-03-04T13:30:10.081 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:09 smithi107 ceph-mon[28739]: from='client.? 172.21.15.107:0/2122601814' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-04T13:30:10.242 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout { 2024-03-04T13:30:10.242 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-03-04T13:30:10.242 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-04T13:30:10.242 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "active_name": "smithi107.wpzgwl", 2024-03-04T13:30:10.242 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-04T13:30:10.242 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout } 2024-03-04T13:30:10.242 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:09.227+0000 7fee27400700 1 Processor -- start 2024-03-04T13:30:10.242 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:09.227+0000 7fee27400700 1 -- start start 2024-03-04T13:30:10.243 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:09.227+0000 7fee27400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee20077b90 0x7fee200760a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:30:10.243 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:09.227+0000 7fee27400700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee20077fa0 con 0x7fee20077b90 2024-03-04T13:30:10.243 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:09.228+0000 7fee24a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee20077b90 0x7fee200760a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:30:10.243 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:09.228+0000 7fee24a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee20077b90 0x7fee200760a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40094/0 (socket says 172.21.15.107:40094) 2024-03-04T13:30:10.243 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:09.228+0000 7fee24a00700 1 -- 172.21.15.107:0/3499154138 learned_addr learned my addr 172.21.15.107:0/3499154138 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:30:10.243 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:09.228+0000 7fee24a00700 1 -- 172.21.15.107:0/3499154138 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee200765e0 con 0x7fee20077b90 2024-03-04T13:30:10.243 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:09.228+0000 7fee24a00700 1 --2- 172.21.15.107:0/3499154138 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee20077b90 0x7fee200760a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fee14009e50 tx=0x7fee14009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40ae02fcbdd55b6e server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:30:10.243 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:09.229+0000 7fee1f400700 1 -- 172.21.15.107:0/3499154138 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fee140092e0 con 0x7fee20077b90 2024-03-04T13:30:10.243 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:09.229+0000 7fee1f400700 1 -- 172.21.15.107:0/3499154138 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fee14012430 con 0x7fee20077b90 2024-03-04T13:30:10.243 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:09.229+0000 7fee1f400700 1 -- 172.21.15.107:0/3499154138 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fee140075c0 con 0x7fee20077b90 2024-03-04T13:30:10.243 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:09.229+0000 7fee27400700 1 -- 172.21.15.107:0/3499154138 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee20077b90 msgr2=0x7fee200760a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:30:10.243 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:09.229+0000 7fee27400700 1 --2- 172.21.15.107:0/3499154138 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee20077b90 0x7fee200760a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fee14009e50 tx=0x7fee14009840 comp rx=0 tx=0).stop 2024-03-04T13:30:10.243 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:09.229+0000 7fee27400700 1 -- 172.21.15.107:0/3499154138 shutdown_connections 2024-03-04T13:30:10.244 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:09.229+0000 7fee27400700 1 --2- 172.21.15.107:0/3499154138 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee20077b90 0x7fee200760a0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:30:10.244 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:09.229+0000 7fee27400700 1 -- 172.21.15.107:0/3499154138 >> 172.21.15.107:0/3499154138 conn(0x7fee200fded0 msgr2=0x7fee20100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:30:10.244 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:09.230+0000 7fee27400700 1 -- 172.21.15.107:0/3499154138 shutdown_connections 2024-03-04T13:30:10.244 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:09.230+0000 7fee27400700 1 -- 172.21.15.107:0/3499154138 wait complete. 2024-03-04T13:30:10.244 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:09.230+0000 7fee27400700 1 Processor -- start 2024-03-04T13:30:10.244 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:09.230+0000 7fee27400700 1 -- start start 2024-03-04T13:30:10.244 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:09.231+0000 7fee27400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee20077b90 0x7fee20113a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:30:10.244 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:09.231+0000 7fee27400700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee20113fd0 con 0x7fee20077b90 2024-03-04T13:30:10.244 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:09.231+0000 7fee24a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee20077b90 0x7fee20113a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:30:10.244 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:09.231+0000 7fee24a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee20077b90 0x7fee20113a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40100/0 (socket says 172.21.15.107:40100) 2024-03-04T13:30:10.244 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:09.231+0000 7fee24a00700 1 -- 172.21.15.107:0/2122601814 learned_addr learned my addr 172.21.15.107:0/2122601814 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:30:10.244 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:09.231+0000 7fee24a00700 1 -- 172.21.15.107:0/2122601814 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee14009ae0 con 0x7fee20077b90 2024-03-04T13:30:10.245 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:09.232+0000 7fee24a00700 1 --2- 172.21.15.107:0/2122601814 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee20077b90 0x7fee20113a90 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fee140127d0 tx=0x7fee14009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:30:10.245 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:09.232+0000 7fee1cc00700 1 -- 172.21.15.107:0/2122601814 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fee140092e0 con 0x7fee20077b90 2024-03-04T13:30:10.245 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:09.232+0000 7fee1cc00700 1 -- 172.21.15.107:0/2122601814 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fee14007cc0 con 0x7fee20077b90 2024-03-04T13:30:10.245 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:09.232+0000 7fee1cc00700 1 -- 172.21.15.107:0/2122601814 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fee1401c970 con 0x7fee20077b90 2024-03-04T13:30:10.245 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:09.232+0000 7fee27400700 1 -- 172.21.15.107:0/2122601814 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee201141d0 con 0x7fee20077b90 2024-03-04T13:30:10.245 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:09.232+0000 7fee27400700 1 -- 172.21.15.107:0/2122601814 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee20114e70 con 0x7fee20077b90 2024-03-04T13:30:10.245 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:09.233+0000 7fee1cc00700 1 -- 172.21.15.107:0/2122601814 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 5) v1 ==== 43368+0+0 (secure 0 0 0) 0x7fee14019070 con 0x7fee20077b90 2024-03-04T13:30:10.245 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:09.233+0000 7fee27400700 1 -- 172.21.15.107:0/2122601814 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee2005f5c0 con 0x7fee20077b90 2024-03-04T13:30:10.245 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:09.233+0000 7fee1cc00700 1 --2- 172.21.15.107:0/2122601814 >> [v2:172.21.15.107:6800/1723390229,v1:172.21.15.107:6801/1723390229] conn(0x7fee080367e0 0x7fee08038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:30:10.245 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:09.233+0000 7fee1cc00700 1 -- 172.21.15.107:0/2122601814 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fee14049d50 con 0x7fee20077b90 2024-03-04T13:30:10.245 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:09.234+0000 7fee1fe00700 1 -- 172.21.15.107:0/2122601814 >> [v2:172.21.15.107:6800/1723390229,v1:172.21.15.107:6801/1723390229] conn(0x7fee080367e0 msgr2=0x7fee08038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/1723390229 2024-03-04T13:30:10.245 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:09.234+0000 7fee1fe00700 1 --2- 172.21.15.107:0/2122601814 >> [v2:172.21.15.107:6800/1723390229,v1:172.21.15.107:6801/1723390229] conn(0x7fee080367e0 0x7fee08038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T13:30:10.245 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:09.239+0000 7fee1cc00700 1 -- 172.21.15.107:0/2122601814 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fee1400bdc0 con 0x7fee20077b90 2024-03-04T13:30:10.246 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:09.435+0000 7fee1fe00700 1 -- 172.21.15.107:0/2122601814 >> [v2:172.21.15.107:6800/1723390229,v1:172.21.15.107:6801/1723390229] conn(0x7fee080367e0 msgr2=0x7fee08038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/1723390229 2024-03-04T13:30:10.246 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:09.435+0000 7fee1fe00700 1 --2- 172.21.15.107:0/2122601814 >> [v2:172.21.15.107:6800/1723390229,v1:172.21.15.107:6801/1723390229] conn(0x7fee080367e0 0x7fee08038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T13:30:10.246 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:09.499+0000 7fee27400700 1 -- 172.21.15.107:0/2122601814 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7fee20108bb0 con 0x7fee20077b90 2024-03-04T13:30:10.246 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:09.500+0000 7fee1cc00700 1 -- 172.21.15.107:0/2122601814 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+103 (secure 0 0 0) 0x7fee14020070 con 0x7fee20077b90 2024-03-04T13:30:10.246 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:09.502+0000 7fee27400700 1 -- 172.21.15.107:0/2122601814 >> [v2:172.21.15.107:6800/1723390229,v1:172.21.15.107:6801/1723390229] conn(0x7fee080367e0 msgr2=0x7fee08038c90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-04T13:30:10.246 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:09.502+0000 7fee27400700 1 --2- 172.21.15.107:0/2122601814 >> [v2:172.21.15.107:6800/1723390229,v1:172.21.15.107:6801/1723390229] conn(0x7fee080367e0 0x7fee08038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:30:10.246 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:09.502+0000 7fee27400700 1 -- 172.21.15.107:0/2122601814 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee20077b90 msgr2=0x7fee20113a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:30:10.246 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:09.502+0000 7fee27400700 1 --2- 172.21.15.107:0/2122601814 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee20077b90 0x7fee20113a90 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fee140127d0 tx=0x7fee14009840 comp rx=0 tx=0).stop 2024-03-04T13:30:10.246 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:09.503+0000 7fee27400700 1 -- 172.21.15.107:0/2122601814 shutdown_connections 2024-03-04T13:30:10.246 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:09.503+0000 7fee27400700 1 --2- 172.21.15.107:0/2122601814 >> [v2:172.21.15.107:6800/1723390229,v1:172.21.15.107:6801/1723390229] conn(0x7fee080367e0 0x7fee08038c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:30:10.246 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:09.503+0000 7fee27400700 1 --2- 172.21.15.107:0/2122601814 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee20077b90 0x7fee20113a90 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:30:10.246 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:09.503+0000 7fee27400700 1 -- 172.21.15.107:0/2122601814 >> 172.21.15.107:0/2122601814 conn(0x7fee200fded0 msgr2=0x7fee20100210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:30:10.246 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:09.503+0000 7fee27400700 1 -- 172.21.15.107:0/2122601814 shutdown_connections 2024-03-04T13:30:10.246 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:09.503+0000 7fee27400700 1 -- 172.21.15.107:0/2122601814 wait complete. 2024-03-04T13:30:10.247 INFO:teuthology.orchestra.run.smithi107.stdout:Waiting for the mgr to restart... 2024-03-04T13:30:10.247 INFO:teuthology.orchestra.run.smithi107.stdout:Waiting for mgr epoch 5... 2024-03-04T13:30:16.588 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:16 smithi107 ceph-mon[28739]: Active manager daemon smithi107.wpzgwl restarted 2024-03-04T13:30:16.588 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:16 smithi107 ceph-mon[28739]: Activating manager daemon smithi107.wpzgwl 2024-03-04T13:30:16.588 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:16 smithi107 ceph-mon[28739]: osdmap e2: 0 total, 0 up, 0 in 2024-03-04T13:30:16.588 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:16 smithi107 ceph-mon[28739]: mgrmap e6: smithi107.wpzgwl(active, starting, since 0.00565163s) 2024-03-04T13:30:16.588 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:16 smithi107 ceph-mon[28739]: from='mgr.14120 172.21.15.107:0/1095418985' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mon metadata", "id": "smithi107"}]: dispatch 2024-03-04T13:30:16.588 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:16 smithi107 ceph-mon[28739]: from='mgr.14120 172.21.15.107:0/1095418985' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mgr metadata", "who": "smithi107.wpzgwl", "id": "smithi107.wpzgwl"}]: dispatch 2024-03-04T13:30:16.588 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:16 smithi107 ceph-mon[28739]: from='mgr.14120 172.21.15.107:0/1095418985' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-04T13:30:16.588 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:16 smithi107 ceph-mon[28739]: from='mgr.14120 172.21.15.107:0/1095418985' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-04T13:30:16.589 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:16 smithi107 ceph-mon[28739]: from='mgr.14120 172.21.15.107:0/1095418985' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-04T13:30:16.589 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:16 smithi107 ceph-mon[28739]: Manager daemon smithi107.wpzgwl is now available 2024-03-04T13:30:16.589 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:16 smithi107 ceph-mon[28739]: from='mgr.14120 172.21.15.107:0/1095418985' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:30:16.589 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:16 smithi107 ceph-mon[28739]: from='mgr.14120 172.21.15.107:0/1095418985' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:30:16.589 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:16 smithi107 ceph-mon[28739]: from='mgr.14120 172.21.15.107:0/1095418985' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:30:16.589 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:16 smithi107 ceph-mon[28739]: from='mgr.14120 172.21.15.107:0/1095418985' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:30:16.589 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:16 smithi107 ceph-mon[28739]: from='mgr.14120 172.21.15.107:0/1095418985' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:30:16.589 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:16 smithi107 ceph-mon[28739]: from='mgr.14120 172.21.15.107:0/1095418985' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi107.wpzgwl/mirror_snapshot_schedule"}]: dispatch 2024-03-04T13:30:16.589 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:16 smithi107 ceph-mon[28739]: from='mgr.14120 172.21.15.107:0/1095418985' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi107.wpzgwl/trash_purge_schedule"}]: dispatch 2024-03-04T13:30:17.587 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:17 smithi107 ceph-mon[28739]: Found migration_current of "None". Setting to last migration. 2024-03-04T13:30:17.587 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:17 smithi107 ceph-mon[28739]: [04/Mar/2024:13:30:16] ENGINE Bus STARTING 2024-03-04T13:30:17.588 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:17 smithi107 ceph-mon[28739]: from='mgr.14120 172.21.15.107:0/1095418985' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:30:17.588 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:17 smithi107 ceph-mon[28739]: from='mgr.14120 172.21.15.107:0/1095418985' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:30:17.588 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:17 smithi107 ceph-mon[28739]: mgrmap e7: smithi107.wpzgwl(active, since 1.01679s) 2024-03-04T13:30:17.946 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout { 2024-03-04T13:30:17.947 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-03-04T13:30:17.947 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-04T13:30:17.947 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout } 2024-03-04T13:30:17.947 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:10.844+0000 7f1b94e00700 1 Processor -- start 2024-03-04T13:30:17.947 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:10.844+0000 7f1b94e00700 1 -- start start 2024-03-04T13:30:17.947 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:10.844+0000 7f1b94e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b90077bd0 0x7f1b900760e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:30:17.948 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:10.844+0000 7f1b94e00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b90077fe0 con 0x7f1b90077bd0 2024-03-04T13:30:17.948 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:10.845+0000 7f1b8de00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b90077bd0 0x7f1b900760e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:30:17.948 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:10.845+0000 7f1b8de00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b90077bd0 0x7f1b900760e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40116/0 (socket says 172.21.15.107:40116) 2024-03-04T13:30:17.948 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:10.845+0000 7f1b8de00700 1 -- 172.21.15.107:0/3863378774 learned_addr learned my addr 172.21.15.107:0/3863378774 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:30:17.948 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:10.845+0000 7f1b8de00700 1 -- 172.21.15.107:0/3863378774 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b90076620 con 0x7f1b90077bd0 2024-03-04T13:30:17.948 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:10.846+0000 7f1b8de00700 1 --2- 172.21.15.107:0/3863378774 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b90077bd0 0x7f1b900760e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f1b800098e0 tx=0x7f1b80009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc235338b5cfabc6 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:30:17.949 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:10.846+0000 7f1b8ca00700 1 -- 172.21.15.107:0/3863378774 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1b8000b860 con 0x7f1b90077bd0 2024-03-04T13:30:17.949 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:10.846+0000 7f1b8ca00700 1 -- 172.21.15.107:0/3863378774 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f1b8000be50 con 0x7f1b90077bd0 2024-03-04T13:30:17.949 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:10.846+0000 7f1b8ca00700 1 -- 172.21.15.107:0/3863378774 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1b80010380 con 0x7f1b90077bd0 2024-03-04T13:30:17.949 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:10.847+0000 7f1b94e00700 1 -- 172.21.15.107:0/3863378774 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b90077bd0 msgr2=0x7f1b900760e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:30:17.949 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:10.847+0000 7f1b94e00700 1 --2- 172.21.15.107:0/3863378774 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b90077bd0 0x7f1b900760e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f1b800098e0 tx=0x7f1b80009250 comp rx=0 tx=0).stop 2024-03-04T13:30:17.949 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:10.847+0000 7f1b94e00700 1 -- 172.21.15.107:0/3863378774 shutdown_connections 2024-03-04T13:30:17.950 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:10.847+0000 7f1b94e00700 1 --2- 172.21.15.107:0/3863378774 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b90077bd0 0x7f1b900760e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:30:17.950 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:10.847+0000 7f1b94e00700 1 -- 172.21.15.107:0/3863378774 >> 172.21.15.107:0/3863378774 conn(0x7f1b900fded0 msgr2=0x7f1b90100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:30:17.950 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:10.847+0000 7f1b94e00700 1 -- 172.21.15.107:0/3863378774 shutdown_connections 2024-03-04T13:30:17.950 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:10.847+0000 7f1b94e00700 1 -- 172.21.15.107:0/3863378774 wait complete. 2024-03-04T13:30:17.950 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:10.848+0000 7f1b94e00700 1 Processor -- start 2024-03-04T13:30:17.950 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:10.848+0000 7f1b94e00700 1 -- start start 2024-03-04T13:30:17.951 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:10.848+0000 7f1b94e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b90077bd0 0x7f1b90113a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:30:17.951 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:10.848+0000 7f1b94e00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b90113fb0 con 0x7f1b90077bd0 2024-03-04T13:30:17.951 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:10.848+0000 7f1b8de00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b90077bd0 0x7f1b90113a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:30:17.951 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:10.849+0000 7f1b8de00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b90077bd0 0x7f1b90113a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40130/0 (socket says 172.21.15.107:40130) 2024-03-04T13:30:17.951 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:10.849+0000 7f1b8de00700 1 -- 172.21.15.107:0/2052226913 learned_addr learned my addr 172.21.15.107:0/2052226913 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:30:17.951 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:10.849+0000 7f1b8de00700 1 -- 172.21.15.107:0/2052226913 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b800094f0 con 0x7f1b90077bd0 2024-03-04T13:30:17.952 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:10.849+0000 7f1b8de00700 1 --2- 172.21.15.107:0/2052226913 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b90077bd0 0x7f1b90113a70 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1b80006fd0 tx=0x7f1b80009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:30:17.952 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:10.849+0000 7f1b86000700 1 -- 172.21.15.107:0/2052226913 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1b800107c0 con 0x7f1b90077bd0 2024-03-04T13:30:17.952 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:10.849+0000 7f1b86000700 1 -- 172.21.15.107:0/2052226913 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f1b80010db0 con 0x7f1b90077bd0 2024-03-04T13:30:17.952 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:10.849+0000 7f1b86000700 1 -- 172.21.15.107:0/2052226913 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1b8001add0 con 0x7f1b90077bd0 2024-03-04T13:30:17.952 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:10.849+0000 7f1b94e00700 1 -- 172.21.15.107:0/2052226913 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b901141b0 con 0x7f1b90077bd0 2024-03-04T13:30:17.952 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:10.850+0000 7f1b94e00700 1 -- 172.21.15.107:0/2052226913 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b90114e50 con 0x7f1b90077bd0 2024-03-04T13:30:17.953 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:10.851+0000 7f1b86000700 1 -- 172.21.15.107:0/2052226913 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 5) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f1b80010920 con 0x7f1b90077bd0 2024-03-04T13:30:17.953 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:10.851+0000 7f1b86000700 1 --2- 172.21.15.107:0/2052226913 >> [v2:172.21.15.107:6800/1723390229,v1:172.21.15.107:6801/1723390229] conn(0x7f1b74036790 0x7f1b74038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:30:17.953 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:10.851+0000 7f1b94e00700 1 -- 172.21.15.107:0/2052226913 --> [v2:172.21.15.107:6800/1723390229,v1:172.21.15.107:6801/1723390229] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1b9005f5c0 con 0x7f1b74036790 2024-03-04T13:30:17.953 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:10.851+0000 7f1b86000700 1 -- 172.21.15.107:0/2052226913 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f1b8001e070 con 0x7f1b90077bd0 2024-03-04T13:30:17.953 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:10.851+0000 7f1b8d400700 1 -- 172.21.15.107:0/2052226913 >> [v2:172.21.15.107:6800/1723390229,v1:172.21.15.107:6801/1723390229] conn(0x7f1b74036790 msgr2=0x7f1b74038c40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/1723390229 2024-03-04T13:30:17.953 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:10.851+0000 7f1b8d400700 1 --2- 172.21.15.107:0/2052226913 >> [v2:172.21.15.107:6800/1723390229,v1:172.21.15.107:6801/1723390229] conn(0x7f1b74036790 0x7f1b74038c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T13:30:17.954 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:11.052+0000 7f1b8d400700 1 -- 172.21.15.107:0/2052226913 >> [v2:172.21.15.107:6800/1723390229,v1:172.21.15.107:6801/1723390229] conn(0x7f1b74036790 msgr2=0x7f1b74038c40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/1723390229 2024-03-04T13:30:17.954 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:11.052+0000 7f1b8d400700 1 --2- 172.21.15.107:0/2052226913 >> [v2:172.21.15.107:6800/1723390229,v1:172.21.15.107:6801/1723390229] conn(0x7f1b74036790 0x7f1b74038c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T13:30:17.954 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:11.453+0000 7f1b8d400700 1 -- 172.21.15.107:0/2052226913 >> [v2:172.21.15.107:6800/1723390229,v1:172.21.15.107:6801/1723390229] conn(0x7f1b74036790 msgr2=0x7f1b74038c40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/1723390229 2024-03-04T13:30:17.954 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:11.453+0000 7f1b8d400700 1 --2- 172.21.15.107:0/2052226913 >> [v2:172.21.15.107:6800/1723390229,v1:172.21.15.107:6801/1723390229] conn(0x7f1b74036790 0x7f1b74038c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T13:30:17.954 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:12.254+0000 7f1b8d400700 1 -- 172.21.15.107:0/2052226913 >> [v2:172.21.15.107:6800/1723390229,v1:172.21.15.107:6801/1723390229] conn(0x7f1b74036790 msgr2=0x7f1b74038c40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/1723390229 2024-03-04T13:30:17.954 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:12.254+0000 7f1b8d400700 1 --2- 172.21.15.107:0/2052226913 >> [v2:172.21.15.107:6800/1723390229,v1:172.21.15.107:6801/1723390229] conn(0x7f1b74036790 0x7f1b74038c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T13:30:17.954 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:13.856+0000 7f1b8d400700 1 -- 172.21.15.107:0/2052226913 >> [v2:172.21.15.107:6800/1723390229,v1:172.21.15.107:6801/1723390229] conn(0x7f1b74036790 msgr2=0x7f1b74038c40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/1723390229 2024-03-04T13:30:17.955 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:13.856+0000 7f1b8d400700 1 --2- 172.21.15.107:0/2052226913 >> [v2:172.21.15.107:6800/1723390229,v1:172.21.15.107:6801/1723390229] conn(0x7f1b74036790 0x7f1b74038c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-04T13:30:17.955 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:16.235+0000 7f1b86000700 1 -- 172.21.15.107:0/2052226913 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mgrmap(e 6) v1 ==== 43135+0+0 (secure 0 0 0) 0x7f1b8002bc90 con 0x7f1b90077bd0 2024-03-04T13:30:17.955 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:16.235+0000 7f1b86000700 1 -- 172.21.15.107:0/2052226913 >> [v2:172.21.15.107:6800/1723390229,v1:172.21.15.107:6801/1723390229] conn(0x7f1b74036790 msgr2=0x7f1b74038c40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-04T13:30:17.955 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:16.235+0000 7f1b86000700 1 --2- 172.21.15.107:0/2052226913 >> [v2:172.21.15.107:6800/1723390229,v1:172.21.15.107:6801/1723390229] conn(0x7f1b74036790 0x7f1b74038c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:30:17.955 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:17.240+0000 7f1b86000700 1 -- 172.21.15.107:0/2052226913 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mgrmap(e 7) v1 ==== 43262+0+0 (secure 0 0 0) 0x7f1b80049bd0 con 0x7f1b90077bd0 2024-03-04T13:30:17.955 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:17.240+0000 7f1b86000700 1 --2- 172.21.15.107:0/2052226913 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7f1b74039f50 0x7f1b7403c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:30:17.955 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:17.240+0000 7f1b86000700 1 -- 172.21.15.107:0/2052226913 --> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1b9005f5c0 con 0x7f1b74039f50 2024-03-04T13:30:17.955 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:17.242+0000 7f1b8d400700 1 --2- 172.21.15.107:0/2052226913 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7f1b74039f50 0x7f1b7403c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:30:17.956 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:17.245+0000 7f1b8d400700 1 --2- 172.21.15.107:0/2052226913 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7f1b74039f50 0x7f1b7403c330 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c0039d0 tx=0x7f1b7c009350 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:30:17.956 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:17.247+0000 7f1b86000700 1 -- 172.21.15.107:0/2052226913 <== mgr.14120 v2:172.21.15.107:6800/2371648646 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6549 (secure 0 0 0) 0x7f1b9005f5c0 con 0x7f1b74039f50 2024-03-04T13:30:17.956 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:17.254+0000 7f1b94e00700 1 -- 172.21.15.107:0/2052226913 --> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f1b90114640 con 0x7f1b74039f50 2024-03-04T13:30:17.956 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:17.255+0000 7f1b86000700 1 -- 172.21.15.107:0/2052226913 <== mgr.14120 v2:172.21.15.107:6800/2371648646 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f1b90114640 con 0x7f1b74039f50 2024-03-04T13:30:17.956 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:17.255+0000 7f1b94e00700 1 -- 172.21.15.107:0/2052226913 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7f1b74039f50 msgr2=0x7f1b7403c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:30:17.956 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:17.255+0000 7f1b94e00700 1 --2- 172.21.15.107:0/2052226913 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7f1b74039f50 0x7f1b7403c330 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c0039d0 tx=0x7f1b7c009350 comp rx=0 tx=0).stop 2024-03-04T13:30:17.956 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:17.256+0000 7f1b94e00700 1 -- 172.21.15.107:0/2052226913 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b90077bd0 msgr2=0x7f1b90113a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:30:17.956 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:17.256+0000 7f1b94e00700 1 --2- 172.21.15.107:0/2052226913 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b90077bd0 0x7f1b90113a70 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1b80006fd0 tx=0x7f1b80009250 comp rx=0 tx=0).stop 2024-03-04T13:30:17.957 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:17.256+0000 7f1b94e00700 1 -- 172.21.15.107:0/2052226913 shutdown_connections 2024-03-04T13:30:17.957 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:17.256+0000 7f1b94e00700 1 --2- 172.21.15.107:0/2052226913 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7f1b74039f50 0x7f1b7403c330 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:30:17.957 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:17.256+0000 7f1b94e00700 1 --2- 172.21.15.107:0/2052226913 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b90077bd0 0x7f1b90113a70 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:30:17.957 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:17.256+0000 7f1b94e00700 1 --2- 172.21.15.107:0/2052226913 >> [v2:172.21.15.107:6800/1723390229,v1:172.21.15.107:6801/1723390229] conn(0x7f1b74036790 0x7f1b74038c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:30:17.957 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:17.256+0000 7f1b94e00700 1 -- 172.21.15.107:0/2052226913 >> 172.21.15.107:0/2052226913 conn(0x7f1b900fded0 msgr2=0x7f1b901001d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:30:17.957 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:17.256+0000 7f1b94e00700 1 -- 172.21.15.107:0/2052226913 shutdown_connections 2024-03-04T13:30:17.957 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:17.256+0000 7f1b94e00700 1 -- 172.21.15.107:0/2052226913 wait complete. 2024-03-04T13:30:17.957 INFO:teuthology.orchestra.run.smithi107.stdout:mgr epoch 5 is available 2024-03-04T13:30:17.957 INFO:teuthology.orchestra.run.smithi107.stdout:Setting orchestrator backend to cephadm... 2024-03-04T13:30:18.494 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:18 smithi107 ceph-mon[28739]: [04/Mar/2024:13:30:17] ENGINE Serving on https://172.21.15.107:7150 2024-03-04T13:30:18.494 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:18 smithi107 ceph-mon[28739]: [04/Mar/2024:13:30:17] ENGINE Bus STARTED 2024-03-04T13:30:18.494 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:18 smithi107 ceph-mon[28739]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-04T13:30:18.494 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:18 smithi107 ceph-mon[28739]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-04T13:30:19.604 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:18.555+0000 7f91c6a00700 1 Processor -- start 2024-03-04T13:30:19.604 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:18.555+0000 7f91c6a00700 1 -- start start 2024-03-04T13:30:19.605 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:18.556+0000 7f91c6a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91c0078cc0 0x7f91c00771d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:30:19.605 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:18.556+0000 7f91c6a00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91c00790d0 con 0x7f91c0078cc0 2024-03-04T13:30:19.605 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:18.556+0000 7f91bfe00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91c0078cc0 0x7f91c00771d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:30:19.605 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:18.556+0000 7f91bfe00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91c0078cc0 0x7f91c00771d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52492/0 (socket says 172.21.15.107:52492) 2024-03-04T13:30:19.605 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:18.556+0000 7f91bfe00700 1 -- 172.21.15.107:0/1978631396 learned_addr learned my addr 172.21.15.107:0/1978631396 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:30:19.605 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:18.557+0000 7f91bfe00700 1 -- 172.21.15.107:0/1978631396 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91c0077710 con 0x7f91c0078cc0 2024-03-04T13:30:19.605 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:18.557+0000 7f91bfe00700 1 --2- 172.21.15.107:0/1978631396 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91c0078cc0 0x7f91c00771d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f91b00098e0 tx=0x7f91b0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f16adb3d7717a20c server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:30:19.605 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:18.557+0000 7f91bea00700 1 -- 172.21.15.107:0/1978631396 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f91b000b860 con 0x7f91c0078cc0 2024-03-04T13:30:19.605 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:18.557+0000 7f91bea00700 1 -- 172.21.15.107:0/1978631396 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f91b000be50 con 0x7f91c0078cc0 2024-03-04T13:30:19.605 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:18.557+0000 7f91bea00700 1 -- 172.21.15.107:0/1978631396 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f91b0010380 con 0x7f91c0078cc0 2024-03-04T13:30:19.605 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:18.558+0000 7f91c6a00700 1 -- 172.21.15.107:0/1978631396 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91c0078cc0 msgr2=0x7f91c00771d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:30:19.605 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:18.558+0000 7f91c6a00700 1 --2- 172.21.15.107:0/1978631396 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91c0078cc0 0x7f91c00771d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f91b00098e0 tx=0x7f91b0009250 comp rx=0 tx=0).stop 2024-03-04T13:30:19.605 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:18.558+0000 7f91c6a00700 1 -- 172.21.15.107:0/1978631396 shutdown_connections 2024-03-04T13:30:19.606 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:18.558+0000 7f91c6a00700 1 --2- 172.21.15.107:0/1978631396 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91c0078cc0 0x7f91c00771d0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:30:19.606 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:18.558+0000 7f91c6a00700 1 -- 172.21.15.107:0/1978631396 >> 172.21.15.107:0/1978631396 conn(0x7f91c00fdf00 msgr2=0x7f91c0100310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:30:19.606 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:18.558+0000 7f91c6a00700 1 -- 172.21.15.107:0/1978631396 shutdown_connections 2024-03-04T13:30:19.606 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:18.558+0000 7f91c6a00700 1 -- 172.21.15.107:0/1978631396 wait complete. 2024-03-04T13:30:19.606 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:18.559+0000 7f91c6a00700 1 Processor -- start 2024-03-04T13:30:19.606 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:18.559+0000 7f91c6a00700 1 -- start start 2024-03-04T13:30:19.606 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:18.559+0000 7f91c6a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91c0078cc0 0x7f91c01137e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:30:19.606 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:18.559+0000 7f91c6a00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91c0113d20 con 0x7f91c0078cc0 2024-03-04T13:30:19.606 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:18.560+0000 7f91bfe00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91c0078cc0 0x7f91c01137e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:30:19.606 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:18.560+0000 7f91bfe00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91c0078cc0 0x7f91c01137e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52502/0 (socket says 172.21.15.107:52502) 2024-03-04T13:30:19.606 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:18.560+0000 7f91bfe00700 1 -- 172.21.15.107:0/1983706145 learned_addr learned my addr 172.21.15.107:0/1983706145 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:30:19.606 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:18.560+0000 7f91bfe00700 1 -- 172.21.15.107:0/1983706145 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91b00094f0 con 0x7f91c0078cc0 2024-03-04T13:30:19.606 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:18.560+0000 7f91bfe00700 1 --2- 172.21.15.107:0/1983706145 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91c0078cc0 0x7f91c01137e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f91b0006fd0 tx=0x7f91b0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:30:19.607 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:18.561+0000 7f91b7e00700 1 -- 172.21.15.107:0/1983706145 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f91b00107c0 con 0x7f91c0078cc0 2024-03-04T13:30:19.607 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:18.561+0000 7f91b7e00700 1 -- 172.21.15.107:0/1983706145 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f91b0010db0 con 0x7f91c0078cc0 2024-03-04T13:30:19.607 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:18.561+0000 7f91c6a00700 1 -- 172.21.15.107:0/1983706145 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91c0113f20 con 0x7f91c0078cc0 2024-03-04T13:30:19.607 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:18.561+0000 7f91c6a00700 1 -- 172.21.15.107:0/1983706145 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91c0114bc0 con 0x7f91c0078cc0 2024-03-04T13:30:19.607 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:18.561+0000 7f91b7e00700 1 -- 172.21.15.107:0/1983706145 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f91b001ada0 con 0x7f91c0078cc0 2024-03-04T13:30:19.607 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:18.562+0000 7f91c6a00700 1 -- 172.21.15.107:0/1983706145 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91c005f5c0 con 0x7f91c0078cc0 2024-03-04T13:30:19.607 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:18.562+0000 7f91b7e00700 1 -- 172.21.15.107:0/1983706145 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f91b0010920 con 0x7f91c0078cc0 2024-03-04T13:30:19.607 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:18.562+0000 7f91b7e00700 1 --2- 172.21.15.107:0/1983706145 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7f91a4036790 0x7f91a4038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:30:19.607 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:18.563+0000 7f91b7e00700 1 -- 172.21.15.107:0/1983706145 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f91b0049db0 con 0x7f91c0078cc0 2024-03-04T13:30:19.607 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:18.563+0000 7f91bf400700 1 --2- 172.21.15.107:0/1983706145 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7f91a4036790 0x7f91a4038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:30:19.607 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:18.564+0000 7f91bf400700 1 --2- 172.21.15.107:0/1983706145 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7f91a4036790 0x7f91a4038c40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f91a8009990 tx=0x7f91a8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:30:19.609 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:18.568+0000 7f91b7e00700 1 -- 172.21.15.107:0/1983706145 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f91b00238d0 con 0x7f91c0078cc0 2024-03-04T13:30:19.609 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:18.791+0000 7f91c6a00700 1 -- 172.21.15.107:0/1983706145 --> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f91c0100230 con 0x7f91a4036790 2024-03-04T13:30:19.610 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:18.804+0000 7f91b7e00700 1 -- 172.21.15.107:0/1983706145 <== mgr.14120 v2:172.21.15.107:6800/2371648646 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f91c0100230 con 0x7f91a4036790 2024-03-04T13:30:19.610 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:18.807+0000 7f91c6a00700 1 -- 172.21.15.107:0/1983706145 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7f91a4036790 msgr2=0x7f91a4038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:30:19.610 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:18.807+0000 7f91c6a00700 1 --2- 172.21.15.107:0/1983706145 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7f91a4036790 0x7f91a4038c40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f91a8009990 tx=0x7f91a8006e40 comp rx=0 tx=0).stop 2024-03-04T13:30:19.610 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:18.807+0000 7f91c6a00700 1 -- 172.21.15.107:0/1983706145 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91c0078cc0 msgr2=0x7f91c01137e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:30:19.610 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:18.807+0000 7f91c6a00700 1 --2- 172.21.15.107:0/1983706145 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91c0078cc0 0x7f91c01137e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f91b0006fd0 tx=0x7f91b0009250 comp rx=0 tx=0).stop 2024-03-04T13:30:19.610 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:18.808+0000 7f91c6a00700 1 -- 172.21.15.107:0/1983706145 shutdown_connections 2024-03-04T13:30:19.610 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:18.808+0000 7f91c6a00700 1 --2- 172.21.15.107:0/1983706145 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7f91a4036790 0x7f91a4038c40 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:30:19.610 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:18.808+0000 7f91c6a00700 1 --2- 172.21.15.107:0/1983706145 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91c0078cc0 0x7f91c01137e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:30:19.610 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:18.808+0000 7f91c6a00700 1 -- 172.21.15.107:0/1983706145 >> 172.21.15.107:0/1983706145 conn(0x7f91c00fdf00 msgr2=0x7f91c00ff610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:30:19.610 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:18.808+0000 7f91c6a00700 1 -- 172.21.15.107:0/1983706145 shutdown_connections 2024-03-04T13:30:19.610 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:18.808+0000 7f91c6a00700 1 -- 172.21.15.107:0/1983706145 wait complete. 2024-03-04T13:30:19.760 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:19 smithi107 ceph-mon[28739]: mgrmap e8: smithi107.wpzgwl(active, since 2s) 2024-03-04T13:30:19.760 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:19 smithi107 ceph-mon[28739]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:30:19.761 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:19 smithi107 ceph-mon[28739]: from='mgr.14120 172.21.15.107:0/1095418985' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:30:19.761 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:19 smithi107 ceph-mon[28739]: from='mgr.14120 172.21.15.107:0/1095418985' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:30:21.235 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout value unchanged 2024-03-04T13:30:21.236 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:20.229+0000 7ffabc200700 1 Processor -- start 2024-03-04T13:30:21.236 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:20.230+0000 7ffabc200700 1 -- start start 2024-03-04T13:30:21.236 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:20.230+0000 7ffabc200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffab4076c40 0x7ffab4103150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:30:21.236 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:20.230+0000 7ffabc200700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffab4103690 con 0x7ffab4076c40 2024-03-04T13:30:21.236 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:20.230+0000 7ffab9800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffab4076c40 0x7ffab4103150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:30:21.236 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:20.230+0000 7ffab9800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffab4076c40 0x7ffab4103150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52508/0 (socket says 172.21.15.107:52508) 2024-03-04T13:30:21.236 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:20.230+0000 7ffab9800700 1 -- 172.21.15.107:0/3156419023 learned_addr learned my addr 172.21.15.107:0/3156419023 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:30:21.236 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:20.231+0000 7ffab9800700 1 -- 172.21.15.107:0/3156419023 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffab41037d0 con 0x7ffab4076c40 2024-03-04T13:30:21.236 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:20.231+0000 7ffab9800700 1 --2- 172.21.15.107:0/3156419023 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffab4076c40 0x7ffab4103150 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ffaa80098e0 tx=0x7ffaa8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57fa81f29eea0870 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:30:21.236 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:20.231+0000 7ffab3e00700 1 -- 172.21.15.107:0/3156419023 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffaa800b860 con 0x7ffab4076c40 2024-03-04T13:30:21.237 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:20.231+0000 7ffab3e00700 1 -- 172.21.15.107:0/3156419023 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7ffaa800be50 con 0x7ffab4076c40 2024-03-04T13:30:21.237 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:20.232+0000 7ffab3e00700 1 -- 172.21.15.107:0/3156419023 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffaa8010380 con 0x7ffab4076c40 2024-03-04T13:30:21.237 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:20.232+0000 7ffabc200700 1 -- 172.21.15.107:0/3156419023 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffab4076c40 msgr2=0x7ffab4103150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:30:21.237 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:20.232+0000 7ffabc200700 1 --2- 172.21.15.107:0/3156419023 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffab4076c40 0x7ffab4103150 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ffaa80098e0 tx=0x7ffaa8009250 comp rx=0 tx=0).stop 2024-03-04T13:30:21.237 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:20.232+0000 7ffabc200700 1 -- 172.21.15.107:0/3156419023 shutdown_connections 2024-03-04T13:30:21.237 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:20.232+0000 7ffabc200700 1 --2- 172.21.15.107:0/3156419023 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffab4076c40 0x7ffab4103150 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:30:21.237 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:20.232+0000 7ffabc200700 1 -- 172.21.15.107:0/3156419023 >> 172.21.15.107:0/3156419023 conn(0x7ffab40fcdd0 msgr2=0x7ffab40ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:30:21.237 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:20.233+0000 7ffabc200700 1 -- 172.21.15.107:0/3156419023 shutdown_connections 2024-03-04T13:30:21.237 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:20.233+0000 7ffabc200700 1 -- 172.21.15.107:0/3156419023 wait complete. 2024-03-04T13:30:21.237 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:20.233+0000 7ffabc200700 1 Processor -- start 2024-03-04T13:30:21.237 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:20.234+0000 7ffabc200700 1 -- start start 2024-03-04T13:30:21.237 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:20.234+0000 7ffabc200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffab4076c40 0x7ffab4117b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:30:21.237 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:20.234+0000 7ffabc200700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffab41180b0 con 0x7ffab4076c40 2024-03-04T13:30:21.238 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:20.234+0000 7ffab9800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffab4076c40 0x7ffab4117b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:30:21.238 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:20.234+0000 7ffab9800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffab4076c40 0x7ffab4117b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52524/0 (socket says 172.21.15.107:52524) 2024-03-04T13:30:21.238 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:20.234+0000 7ffab9800700 1 -- 172.21.15.107:0/1791056574 learned_addr learned my addr 172.21.15.107:0/1791056574 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:30:21.238 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:20.235+0000 7ffab9800700 1 -- 172.21.15.107:0/1791056574 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaa80094f0 con 0x7ffab4076c40 2024-03-04T13:30:21.238 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:20.235+0000 7ffab9800700 1 --2- 172.21.15.107:0/1791056574 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffab4076c40 0x7ffab4117b70 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ffaa8006b00 tx=0x7ffaa8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:30:21.238 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:20.235+0000 7ffab1600700 1 -- 172.21.15.107:0/1791056574 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffaa80107c0 con 0x7ffab4076c40 2024-03-04T13:30:21.238 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:20.235+0000 7ffab1600700 1 -- 172.21.15.107:0/1791056574 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7ffaa8010db0 con 0x7ffab4076c40 2024-03-04T13:30:21.238 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:20.235+0000 7ffabc200700 1 -- 172.21.15.107:0/1791056574 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffab41182b0 con 0x7ffab4076c40 2024-03-04T13:30:21.238 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:20.235+0000 7ffab1600700 1 -- 172.21.15.107:0/1791056574 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffaa801acf0 con 0x7ffab4076c40 2024-03-04T13:30:21.239 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:20.235+0000 7ffabc200700 1 -- 172.21.15.107:0/1791056574 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffab4118f50 con 0x7ffab4076c40 2024-03-04T13:30:21.239 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:20.236+0000 7ffab1600700 1 -- 172.21.15.107:0/1791056574 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7ffaa8010920 con 0x7ffab4076c40 2024-03-04T13:30:21.239 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:20.237+0000 7ffabc200700 1 -- 172.21.15.107:0/1791056574 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffab405f5c0 con 0x7ffab4076c40 2024-03-04T13:30:21.239 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:20.237+0000 7ffab1600700 1 --2- 172.21.15.107:0/1791056574 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7ffa9c0367e0 0x7ffa9c038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:30:21.239 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:20.237+0000 7ffab1600700 1 -- 172.21.15.107:0/1791056574 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7ffaa804a690 con 0x7ffab4076c40 2024-03-04T13:30:21.239 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:20.237+0000 7ffab8e00700 1 --2- 172.21.15.107:0/1791056574 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7ffa9c0367e0 0x7ffa9c038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:30:21.239 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:20.238+0000 7ffab8e00700 1 --2- 172.21.15.107:0/1791056574 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7ffa9c0367e0 0x7ffa9c038c90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ffaa0009990 tx=0x7ffaa0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:30:21.239 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:20.242+0000 7ffab1600700 1 -- 172.21.15.107:0/1791056574 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7ffaa801a530 con 0x7ffab4076c40 2024-03-04T13:30:21.239 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:20.452+0000 7ffabc200700 1 -- 172.21.15.107:0/1791056574 --> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7ffab4106cc0 con 0x7ffa9c0367e0 2024-03-04T13:30:21.239 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:20.454+0000 7ffab1600700 1 -- 172.21.15.107:0/1791056574 <== mgr.14120 v2:172.21.15.107:6800/2371648646 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7ffab4106cc0 con 0x7ffa9c0367e0 2024-03-04T13:30:21.239 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:20.456+0000 7ffabc200700 1 -- 172.21.15.107:0/1791056574 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7ffa9c0367e0 msgr2=0x7ffa9c038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:30:21.239 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:20.456+0000 7ffabc200700 1 --2- 172.21.15.107:0/1791056574 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7ffa9c0367e0 0x7ffa9c038c90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ffaa0009990 tx=0x7ffaa0006e40 comp rx=0 tx=0).stop 2024-03-04T13:30:21.239 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:20.457+0000 7ffabc200700 1 -- 172.21.15.107:0/1791056574 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffab4076c40 msgr2=0x7ffab4117b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:30:21.240 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:20.457+0000 7ffabc200700 1 --2- 172.21.15.107:0/1791056574 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffab4076c40 0x7ffab4117b70 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ffaa8006b00 tx=0x7ffaa8009250 comp rx=0 tx=0).stop 2024-03-04T13:30:21.240 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:20.457+0000 7ffabc200700 1 -- 172.21.15.107:0/1791056574 shutdown_connections 2024-03-04T13:30:21.240 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:20.457+0000 7ffabc200700 1 --2- 172.21.15.107:0/1791056574 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7ffa9c0367e0 0x7ffa9c038c90 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:30:21.240 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:20.457+0000 7ffabc200700 1 --2- 172.21.15.107:0/1791056574 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffab4076c40 0x7ffab4117b70 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:30:21.240 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:20.457+0000 7ffabc200700 1 -- 172.21.15.107:0/1791056574 >> 172.21.15.107:0/1791056574 conn(0x7ffab40fcdd0 msgr2=0x7ffab40fe530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:30:21.240 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:20.457+0000 7ffabc200700 1 -- 172.21.15.107:0/1791056574 shutdown_connections 2024-03-04T13:30:21.240 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:20.457+0000 7ffabc200700 1 -- 172.21.15.107:0/1791056574 wait complete. 2024-03-04T13:30:21.240 INFO:teuthology.orchestra.run.smithi107.stdout:Generating ssh key... 2024-03-04T13:30:21.666 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:21 smithi107 ceph-mon[28739]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:30:22.837 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:21.824+0000 7f525f200700 1 Processor -- start 2024-03-04T13:30:22.837 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:21.824+0000 7f525f200700 1 -- start start 2024-03-04T13:30:22.837 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:21.825+0000 7f525f200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5258102770 0x7f5258102b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:30:22.837 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:21.825+0000 7f525f200700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52581030c0 con 0x7f5258102770 2024-03-04T13:30:22.837 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:21.825+0000 7f5257e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5258102770 0x7f5258102b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:30:22.837 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:21.825+0000 7f5257e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5258102770 0x7f5258102b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52538/0 (socket says 172.21.15.107:52538) 2024-03-04T13:30:22.838 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:21.825+0000 7f5257e00700 1 -- 172.21.15.107:0/3874252120 learned_addr learned my addr 172.21.15.107:0/3874252120 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:30:22.838 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:21.826+0000 7f5257e00700 1 -- 172.21.15.107:0/3874252120 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5258103200 con 0x7f5258102770 2024-03-04T13:30:22.838 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:21.826+0000 7f5257e00700 1 --2- 172.21.15.107:0/3874252120 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5258102770 0x7f5258102b80 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f52480098e0 tx=0x7f5248009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e65f7183eeb11a99 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:30:22.838 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:21.827+0000 7f5256a00700 1 -- 172.21.15.107:0/3874252120 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f524800b860 con 0x7f5258102770 2024-03-04T13:30:22.838 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:21.827+0000 7f5256a00700 1 -- 172.21.15.107:0/3874252120 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f524800be50 con 0x7f5258102770 2024-03-04T13:30:22.838 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:21.827+0000 7f5256a00700 1 -- 172.21.15.107:0/3874252120 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5248010380 con 0x7f5258102770 2024-03-04T13:30:22.838 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:21.827+0000 7f525f200700 1 -- 172.21.15.107:0/3874252120 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5258102770 msgr2=0x7f5258102b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:30:22.838 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:21.827+0000 7f525f200700 1 --2- 172.21.15.107:0/3874252120 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5258102770 0x7f5258102b80 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f52480098e0 tx=0x7f5248009250 comp rx=0 tx=0).stop 2024-03-04T13:30:22.838 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:21.827+0000 7f525f200700 1 -- 172.21.15.107:0/3874252120 shutdown_connections 2024-03-04T13:30:22.838 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:21.827+0000 7f525f200700 1 --2- 172.21.15.107:0/3874252120 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5258102770 0x7f5258102b80 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:30:22.838 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:21.828+0000 7f525f200700 1 -- 172.21.15.107:0/3874252120 >> 172.21.15.107:0/3874252120 conn(0x7f52580fded0 msgr2=0x7f5258100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:30:22.839 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:21.828+0000 7f525f200700 1 -- 172.21.15.107:0/3874252120 shutdown_connections 2024-03-04T13:30:22.839 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:21.828+0000 7f525f200700 1 -- 172.21.15.107:0/3874252120 wait complete. 2024-03-04T13:30:22.839 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:21.829+0000 7f525f200700 1 Processor -- start 2024-03-04T13:30:22.839 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:21.829+0000 7f525f200700 1 -- start start 2024-03-04T13:30:22.839 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:21.829+0000 7f525f200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5258102770 0x7f5258113a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:30:22.839 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:21.829+0000 7f525f200700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5258113f40 con 0x7f5258102770 2024-03-04T13:30:22.839 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:21.829+0000 7f5257e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5258102770 0x7f5258113a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:30:22.839 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:21.829+0000 7f5257e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5258102770 0x7f5258113a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52546/0 (socket says 172.21.15.107:52546) 2024-03-04T13:30:22.839 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:21.829+0000 7f5257e00700 1 -- 172.21.15.107:0/3954241769 learned_addr learned my addr 172.21.15.107:0/3954241769 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:30:22.839 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:21.830+0000 7f5257e00700 1 -- 172.21.15.107:0/3954241769 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52480094f0 con 0x7f5258102770 2024-03-04T13:30:22.839 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:21.830+0000 7f5257e00700 1 --2- 172.21.15.107:0/3954241769 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5258102770 0x7f5258113a00 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f524800b440 tx=0x7f5248009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:30:22.840 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:21.830+0000 7f524fe00700 1 -- 172.21.15.107:0/3954241769 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f52480107c0 con 0x7f5258102770 2024-03-04T13:30:22.840 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:21.830+0000 7f524fe00700 1 -- 172.21.15.107:0/3954241769 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f5248010db0 con 0x7f5258102770 2024-03-04T13:30:22.840 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:21.830+0000 7f524fe00700 1 -- 172.21.15.107:0/3954241769 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f524801ada0 con 0x7f5258102770 2024-03-04T13:30:22.840 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:21.830+0000 7f525f200700 1 -- 172.21.15.107:0/3954241769 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5258114140 con 0x7f5258102770 2024-03-04T13:30:22.840 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:21.831+0000 7f525f200700 1 -- 172.21.15.107:0/3954241769 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5258114de0 con 0x7f5258102770 2024-03-04T13:30:22.840 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:21.832+0000 7f524fe00700 1 -- 172.21.15.107:0/3954241769 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f5248010920 con 0x7f5258102770 2024-03-04T13:30:22.840 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:21.832+0000 7f525f200700 1 -- 172.21.15.107:0/3954241769 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f525805f5c0 con 0x7f5258102770 2024-03-04T13:30:22.840 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:21.832+0000 7f524fe00700 1 --2- 172.21.15.107:0/3954241769 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7f523c0367e0 0x7f523c038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:30:22.840 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:21.832+0000 7f524fe00700 1 -- 172.21.15.107:0/3954241769 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f524804acf0 con 0x7f5258102770 2024-03-04T13:30:22.840 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:21.832+0000 7f5257400700 1 --2- 172.21.15.107:0/3954241769 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7f523c0367e0 0x7f523c038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:30:22.841 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:21.833+0000 7f5257400700 1 --2- 172.21.15.107:0/3954241769 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7f523c0367e0 0x7f523c038c90 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5244009990 tx=0x7f5244006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:30:22.841 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:21.841+0000 7f524fe00700 1 -- 172.21.15.107:0/3954241769 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f5248018940 con 0x7f5258102770 2024-03-04T13:30:22.841 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:22.040+0000 7f525f200700 1 -- 172.21.15.107:0/3954241769 --> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f52581046f0 con 0x7f523c0367e0 2024-03-04T13:30:22.841 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:22.124+0000 7f524fe00700 1 -- 172.21.15.107:0/3954241769 <== mgr.14120 v2:172.21.15.107:6800/2371648646 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f52581046f0 con 0x7f523c0367e0 2024-03-04T13:30:22.841 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:22.126+0000 7f525f200700 1 -- 172.21.15.107:0/3954241769 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7f523c0367e0 msgr2=0x7f523c038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:30:22.841 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:22.126+0000 7f525f200700 1 --2- 172.21.15.107:0/3954241769 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7f523c0367e0 0x7f523c038c90 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5244009990 tx=0x7f5244006e40 comp rx=0 tx=0).stop 2024-03-04T13:30:22.841 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:22.126+0000 7f525f200700 1 -- 172.21.15.107:0/3954241769 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5258102770 msgr2=0x7f5258113a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:30:22.841 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:22.126+0000 7f525f200700 1 --2- 172.21.15.107:0/3954241769 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5258102770 0x7f5258113a00 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f524800b440 tx=0x7f5248009250 comp rx=0 tx=0).stop 2024-03-04T13:30:22.841 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:22.127+0000 7f525f200700 1 -- 172.21.15.107:0/3954241769 shutdown_connections 2024-03-04T13:30:22.842 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:22.127+0000 7f525f200700 1 --2- 172.21.15.107:0/3954241769 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7f523c0367e0 0x7f523c038c90 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:30:22.842 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:22.127+0000 7f525f200700 1 --2- 172.21.15.107:0/3954241769 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5258102770 0x7f5258113a00 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:30:22.842 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:22.127+0000 7f525f200700 1 -- 172.21.15.107:0/3954241769 >> 172.21.15.107:0/3954241769 conn(0x7f52580fded0 msgr2=0x7f52580fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:30:22.842 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:22.127+0000 7f525f200700 1 -- 172.21.15.107:0/3954241769 shutdown_connections 2024-03-04T13:30:22.842 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:22.127+0000 7f525f200700 1 -- 172.21.15.107:0/3954241769 wait complete. 2024-03-04T13:30:23.388 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:23 smithi107 ceph-mon[28739]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:30:23.388 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:23 smithi107 ceph-mon[28739]: Generating ssh key... 2024-03-04T13:30:23.388 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:23 smithi107 ceph-mon[28739]: from='mgr.14120 172.21.15.107:0/1095418985' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:30:23.388 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:23 smithi107 ceph-mon[28739]: from='mgr.14120 172.21.15.107:0/1095418985' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:30:24.338 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:24 smithi107 ceph-mon[28739]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:30:24.374 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDI+M5RQcJ/GKF1O9RXkU5IR3uNr3kDgSfxLe+lzPUt27gelf8X8+Ba2tMylJBojMP8Jxi/F/JUeyPgI7apHDP0tayB6KXygj1sFxYRXXUI11e9pxC09R1iWBP5YKbtMaQQ+/H0lBJRoit/LokSeMmCQYEmf0T4GbbOqoFWQw97XYCbzDFkWvCupcKI60V3T9RIw+4xJUB4auZ7vzYN1Fo7E8/ehwsm1TQqNL8i+ZiGsb56lLUaAYjcZot7ALut1xdhT2MCY7vMs+6MjqmGB6zac3j6KdyM9lEvhQ3NfKEQ59AKhVBq3cEEWBf2X5zb+oKWKwoSJhDvYEfBoGWIq8eEHBTPcp249yztp6bC6r5xSlK+Vxv7x+I0CQ5QL5OYuelgx/tGmesAbBaPGihJs+nUEeY/z5mW/8aYYMKMIQ0eEa22ef5G93R4NquZi9Hi7z1VpjgEUcHSXtpf0ILzKuf8TaTAObVyI/9BhNmy3MA783Q4zHgbjQ8qpqK/hC0gPEE= ceph-1fc09a58-da2b-11ee-95c6-87774f69a715 2024-03-04T13:30:24.375 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:23.447+0000 7f27dea00700 1 Processor -- start 2024-03-04T13:30:24.375 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:23.447+0000 7f27dea00700 1 -- start start 2024-03-04T13:30:24.375 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:23.448+0000 7f27dea00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f27d807c7a0 0x7f27d807cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:30:24.375 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:23.448+0000 7f27dea00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27d807d0f0 con 0x7f27d807c7a0 2024-03-04T13:30:24.375 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:23.448+0000 7f27d7e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f27d807c7a0 0x7f27d807cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:30:24.375 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:23.448+0000 7f27d7e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f27d807c7a0 0x7f27d807cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52558/0 (socket says 172.21.15.107:52558) 2024-03-04T13:30:24.375 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:23.448+0000 7f27d7e00700 1 -- 172.21.15.107:0/518411428 learned_addr learned my addr 172.21.15.107:0/518411428 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:30:24.375 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:23.449+0000 7f27d7e00700 1 -- 172.21.15.107:0/518411428 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27d807d230 con 0x7f27d807c7a0 2024-03-04T13:30:24.375 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:23.449+0000 7f27d7e00700 1 --2- 172.21.15.107:0/518411428 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f27d807c7a0 0x7f27d807cbb0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f27c80098e0 tx=0x7f27c8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=126070a7a9869b6b server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:30:24.375 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:23.450+0000 7f27d7400700 1 -- 172.21.15.107:0/518411428 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f27c800b860 con 0x7f27d807c7a0 2024-03-04T13:30:24.375 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:23.450+0000 7f27d7400700 1 -- 172.21.15.107:0/518411428 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f27c800be50 con 0x7f27d807c7a0 2024-03-04T13:30:24.375 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:23.450+0000 7f27d7400700 1 -- 172.21.15.107:0/518411428 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f27c8010380 con 0x7f27d807c7a0 2024-03-04T13:30:24.376 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:23.450+0000 7f27dea00700 1 -- 172.21.15.107:0/518411428 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f27d807c7a0 msgr2=0x7f27d807cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:30:24.376 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:23.450+0000 7f27dea00700 1 --2- 172.21.15.107:0/518411428 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f27d807c7a0 0x7f27d807cbb0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f27c80098e0 tx=0x7f27c8009250 comp rx=0 tx=0).stop 2024-03-04T13:30:24.376 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:23.451+0000 7f27dea00700 1 -- 172.21.15.107:0/518411428 shutdown_connections 2024-03-04T13:30:24.376 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:23.451+0000 7f27dea00700 1 --2- 172.21.15.107:0/518411428 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f27d807c7a0 0x7f27d807cbb0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:30:24.376 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:23.451+0000 7f27dea00700 1 -- 172.21.15.107:0/518411428 >> 172.21.15.107:0/518411428 conn(0x7f27d8078030 msgr2=0x7f27d807a460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:30:24.376 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:23.451+0000 7f27dea00700 1 -- 172.21.15.107:0/518411428 shutdown_connections 2024-03-04T13:30:24.376 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:23.451+0000 7f27dea00700 1 -- 172.21.15.107:0/518411428 wait complete. 2024-03-04T13:30:24.376 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:23.452+0000 7f27dea00700 1 Processor -- start 2024-03-04T13:30:24.376 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:23.452+0000 7f27dea00700 1 -- start start 2024-03-04T13:30:24.376 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:23.452+0000 7f27dea00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f27d807c7a0 0x7f27d8117bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:30:24.376 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:23.452+0000 7f27dea00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27d81180f0 con 0x7f27d807c7a0 2024-03-04T13:30:24.376 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:23.452+0000 7f27d7e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f27d807c7a0 0x7f27d8117bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:30:24.376 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:23.452+0000 7f27d7e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f27d807c7a0 0x7f27d8117bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52566/0 (socket says 172.21.15.107:52566) 2024-03-04T13:30:24.377 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:23.452+0000 7f27d7e00700 1 -- 172.21.15.107:0/586098100 learned_addr learned my addr 172.21.15.107:0/586098100 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:30:24.377 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:23.453+0000 7f27d7e00700 1 -- 172.21.15.107:0/586098100 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27c80094f0 con 0x7f27d807c7a0 2024-03-04T13:30:24.377 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:23.453+0000 7f27d7e00700 1 --2- 172.21.15.107:0/586098100 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f27d807c7a0 0x7f27d8117bb0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f27c8006b00 tx=0x7f27c8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:30:24.377 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:23.453+0000 7f27d4c00700 1 -- 172.21.15.107:0/586098100 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f27c80107c0 con 0x7f27d807c7a0 2024-03-04T13:30:24.377 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:23.453+0000 7f27d4c00700 1 -- 172.21.15.107:0/586098100 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f27c8010db0 con 0x7f27d807c7a0 2024-03-04T13:30:24.377 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:23.453+0000 7f27d4c00700 1 -- 172.21.15.107:0/586098100 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f27c801acf0 con 0x7f27d807c7a0 2024-03-04T13:30:24.377 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:23.453+0000 7f27dea00700 1 -- 172.21.15.107:0/586098100 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27d81182f0 con 0x7f27d807c7a0 2024-03-04T13:30:24.377 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:23.453+0000 7f27dea00700 1 -- 172.21.15.107:0/586098100 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27d8118f90 con 0x7f27d807c7a0 2024-03-04T13:30:24.377 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:23.454+0000 7f27d4c00700 1 -- 172.21.15.107:0/586098100 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f27c8010920 con 0x7f27d807c7a0 2024-03-04T13:30:24.377 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:23.455+0000 7f27dea00700 1 -- 172.21.15.107:0/586098100 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27d805f5c0 con 0x7f27d807c7a0 2024-03-04T13:30:24.377 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:23.455+0000 7f27d4c00700 1 --2- 172.21.15.107:0/586098100 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7f27bc036790 0x7f27bc038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:30:24.377 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:23.455+0000 7f27d4c00700 1 -- 172.21.15.107:0/586098100 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f27c804a6b0 con 0x7f27d807c7a0 2024-03-04T13:30:24.377 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:23.455+0000 7f27cfe00700 1 --2- 172.21.15.107:0/586098100 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7f27bc036790 0x7f27bc038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:30:24.377 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:23.456+0000 7f27cfe00700 1 --2- 172.21.15.107:0/586098100 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7f27bc036790 0x7f27bc038c40 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f27c0009990 tx=0x7f27c0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:30:24.378 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:23.460+0000 7f27d4c00700 1 -- 172.21.15.107:0/586098100 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f27c801a530 con 0x7f27d807c7a0 2024-03-04T13:30:24.378 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:23.670+0000 7f27dea00700 1 -- 172.21.15.107:0/586098100 --> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f27d807b390 con 0x7f27bc036790 2024-03-04T13:30:24.378 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:23.671+0000 7f27d4c00700 1 -- 172.21.15.107:0/586098100 <== mgr.14120 v2:172.21.15.107:6800/2371648646 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f27d807b390 con 0x7f27bc036790 2024-03-04T13:30:24.378 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:23.673+0000 7f27dea00700 1 -- 172.21.15.107:0/586098100 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7f27bc036790 msgr2=0x7f27bc038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:30:24.378 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:23.673+0000 7f27dea00700 1 --2- 172.21.15.107:0/586098100 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7f27bc036790 0x7f27bc038c40 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f27c0009990 tx=0x7f27c0006e40 comp rx=0 tx=0).stop 2024-03-04T13:30:24.378 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:23.673+0000 7f27dea00700 1 -- 172.21.15.107:0/586098100 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f27d807c7a0 msgr2=0x7f27d8117bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:30:24.378 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:23.673+0000 7f27dea00700 1 --2- 172.21.15.107:0/586098100 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f27d807c7a0 0x7f27d8117bb0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f27c8006b00 tx=0x7f27c8009250 comp rx=0 tx=0).stop 2024-03-04T13:30:24.378 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:23.674+0000 7f27dea00700 1 -- 172.21.15.107:0/586098100 shutdown_connections 2024-03-04T13:30:24.378 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:23.674+0000 7f27dea00700 1 --2- 172.21.15.107:0/586098100 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7f27bc036790 0x7f27bc038c40 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:30:24.378 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:23.674+0000 7f27dea00700 1 --2- 172.21.15.107:0/586098100 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f27d807c7a0 0x7f27d8117bb0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:30:24.378 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:23.674+0000 7f27dea00700 1 -- 172.21.15.107:0/586098100 >> 172.21.15.107:0/586098100 conn(0x7f27d8078030 msgr2=0x7f27d807a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:30:24.378 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:23.674+0000 7f27dea00700 1 -- 172.21.15.107:0/586098100 shutdown_connections 2024-03-04T13:30:24.378 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:23.674+0000 7f27dea00700 1 -- 172.21.15.107:0/586098100 wait complete. 2024-03-04T13:30:24.378 INFO:teuthology.orchestra.run.smithi107.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-03-04T13:30:24.379 INFO:teuthology.orchestra.run.smithi107.stdout:Adding key to root@localhost authorized_keys... 2024-03-04T13:30:24.379 INFO:teuthology.orchestra.run.smithi107.stdout:Adding host smithi107... 2024-03-04T13:30:26.587 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:26 smithi107 ceph-mon[28739]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi107", "addr": "172.21.15.107", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:30:27.587 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:27 smithi107 ceph-mon[28739]: Deploying cephadm binary to smithi107 2024-03-04T13:30:30.056 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout Added host 'smithi107' with addr '172.21.15.107' 2024-03-04T13:30:30.056 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:24.976+0000 7fc4a3e00700 1 Processor -- start 2024-03-04T13:30:30.056 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:24.976+0000 7fc4a3e00700 1 -- start start 2024-03-04T13:30:30.057 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:24.977+0000 7fc4a3e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc49c104980 0x7fc49c104d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:30:30.057 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:24.977+0000 7fc4a3e00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc49c1052d0 con 0x7fc49c104980 2024-03-04T13:30:30.057 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:24.977+0000 7fc4a1400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc49c104980 0x7fc49c104d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:30:30.057 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:24.977+0000 7fc4a1400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc49c104980 0x7fc49c104d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52582/0 (socket says 172.21.15.107:52582) 2024-03-04T13:30:30.058 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:24.977+0000 7fc4a1400700 1 -- 172.21.15.107:0/1196331578 learned_addr learned my addr 172.21.15.107:0/1196331578 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:30:30.058 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:24.978+0000 7fc4a1400700 1 -- 172.21.15.107:0/1196331578 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc49c105410 con 0x7fc49c104980 2024-03-04T13:30:30.058 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:24.978+0000 7fc4a1400700 1 --2- 172.21.15.107:0/1196331578 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc49c104980 0x7fc49c104d90 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc4900098e0 tx=0x7fc490009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53262ec512f9afd5 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:30:30.058 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:24.978+0000 7fc49be00700 1 -- 172.21.15.107:0/1196331578 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc49000b860 con 0x7fc49c104980 2024-03-04T13:30:30.058 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:24.978+0000 7fc49be00700 1 -- 172.21.15.107:0/1196331578 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fc49000be50 con 0x7fc49c104980 2024-03-04T13:30:30.060 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:24.978+0000 7fc49be00700 1 -- 172.21.15.107:0/1196331578 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc490010380 con 0x7fc49c104980 2024-03-04T13:30:30.060 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:24.979+0000 7fc4a3e00700 1 -- 172.21.15.107:0/1196331578 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc49c104980 msgr2=0x7fc49c104d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:30:30.060 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:24.979+0000 7fc4a3e00700 1 --2- 172.21.15.107:0/1196331578 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc49c104980 0x7fc49c104d90 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc4900098e0 tx=0x7fc490009250 comp rx=0 tx=0).stop 2024-03-04T13:30:30.060 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:24.979+0000 7fc4a3e00700 1 -- 172.21.15.107:0/1196331578 shutdown_connections 2024-03-04T13:30:30.060 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:24.979+0000 7fc4a3e00700 1 --2- 172.21.15.107:0/1196331578 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc49c104980 0x7fc49c104d90 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:30:30.060 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:24.979+0000 7fc4a3e00700 1 -- 172.21.15.107:0/1196331578 >> 172.21.15.107:0/1196331578 conn(0x7fc49c078070 msgr2=0x7fc49c078490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:30:30.060 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:24.979+0000 7fc4a3e00700 1 -- 172.21.15.107:0/1196331578 shutdown_connections 2024-03-04T13:30:30.060 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:24.979+0000 7fc4a3e00700 1 -- 172.21.15.107:0/1196331578 wait complete. 2024-03-04T13:30:30.060 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:24.980+0000 7fc4a3e00700 1 Processor -- start 2024-03-04T13:30:30.061 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:24.980+0000 7fc4a3e00700 1 -- start start 2024-03-04T13:30:30.061 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:24.981+0000 7fc4a3e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc49c104980 0x7fc49c1136d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:30:30.061 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:24.981+0000 7fc4a3e00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc49c113c10 con 0x7fc49c104980 2024-03-04T13:30:30.061 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:24.981+0000 7fc4a1400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc49c104980 0x7fc49c1136d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:30:30.061 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:24.981+0000 7fc4a1400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc49c104980 0x7fc49c1136d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52584/0 (socket says 172.21.15.107:52584) 2024-03-04T13:30:30.061 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:24.981+0000 7fc4a1400700 1 -- 172.21.15.107:0/2103592156 learned_addr learned my addr 172.21.15.107:0/2103592156 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:30:30.063 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:24.981+0000 7fc4a1400700 1 -- 172.21.15.107:0/2103592156 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4900094f0 con 0x7fc49c104980 2024-03-04T13:30:30.063 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:24.982+0000 7fc4a1400700 1 --2- 172.21.15.107:0/2103592156 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc49c104980 0x7fc49c1136d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc49000b440 tx=0x7fc490009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:30:30.064 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:24.982+0000 7fc499600700 1 -- 172.21.15.107:0/2103592156 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc4900107c0 con 0x7fc49c104980 2024-03-04T13:30:30.064 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:24.982+0000 7fc499600700 1 -- 172.21.15.107:0/2103592156 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fc490010db0 con 0x7fc49c104980 2024-03-04T13:30:30.064 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:24.982+0000 7fc4a3e00700 1 -- 172.21.15.107:0/2103592156 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc49c113e10 con 0x7fc49c104980 2024-03-04T13:30:30.064 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:24.982+0000 7fc499600700 1 -- 172.21.15.107:0/2103592156 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc49001ada0 con 0x7fc49c104980 2024-03-04T13:30:30.064 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:24.982+0000 7fc4a3e00700 1 -- 172.21.15.107:0/2103592156 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc49c114ab0 con 0x7fc49c104980 2024-03-04T13:30:30.064 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:24.983+0000 7fc499600700 1 -- 172.21.15.107:0/2103592156 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7fc49002e440 con 0x7fc49c104980 2024-03-04T13:30:30.064 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:24.984+0000 7fc4a3e00700 1 -- 172.21.15.107:0/2103592156 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc49c05f5c0 con 0x7fc49c104980 2024-03-04T13:30:30.064 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:24.984+0000 7fc499600700 1 --2- 172.21.15.107:0/2103592156 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7fc4840367e0 0x7fc484038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:30:30.064 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:24.984+0000 7fc499600700 1 -- 172.21.15.107:0/2103592156 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fc49004ac60 con 0x7fc49c104980 2024-03-04T13:30:30.064 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:24.984+0000 7fc4a0a00700 1 --2- 172.21.15.107:0/2103592156 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7fc4840367e0 0x7fc484038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:30:30.064 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:24.985+0000 7fc4a0a00700 1 --2- 172.21.15.107:0/2103592156 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7fc4840367e0 0x7fc484038c90 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc488009990 tx=0x7fc488006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:30:30.064 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:24.989+0000 7fc499600700 1 -- 172.21.15.107:0/2103592156 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fc490018350 con 0x7fc49c104980 2024-03-04T13:30:30.064 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:25.201+0000 7fc4a3e00700 1 -- 172.21.15.107:0/2103592156 --> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi107", "addr": "172.21.15.107", "target": ["mon-mgr", ""]}) v1 -- 0x7fc49c102eb0 con 0x7fc4840367e0 2024-03-04T13:30:30.065 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:29.218+0000 7fc499600700 1 -- 172.21.15.107:0/2103592156 <== mgr.14120 v2:172.21.15.107:6800/2371648646 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fc49c102eb0 con 0x7fc4840367e0 2024-03-04T13:30:30.065 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:29.222+0000 7fc4a3e00700 1 -- 172.21.15.107:0/2103592156 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7fc4840367e0 msgr2=0x7fc484038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:30:30.065 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:29.222+0000 7fc4a3e00700 1 --2- 172.21.15.107:0/2103592156 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7fc4840367e0 0x7fc484038c90 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc488009990 tx=0x7fc488006e40 comp rx=0 tx=0).stop 2024-03-04T13:30:30.065 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:29.222+0000 7fc4a3e00700 1 -- 172.21.15.107:0/2103592156 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc49c104980 msgr2=0x7fc49c1136d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:30:30.065 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:29.222+0000 7fc4a3e00700 1 --2- 172.21.15.107:0/2103592156 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc49c104980 0x7fc49c1136d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc49000b440 tx=0x7fc490009250 comp rx=0 tx=0).stop 2024-03-04T13:30:30.065 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:29.222+0000 7fc4a3e00700 1 -- 172.21.15.107:0/2103592156 shutdown_connections 2024-03-04T13:30:30.065 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:29.222+0000 7fc4a3e00700 1 --2- 172.21.15.107:0/2103592156 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7fc4840367e0 0x7fc484038c90 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:30:30.066 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:29.222+0000 7fc4a3e00700 1 --2- 172.21.15.107:0/2103592156 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc49c104980 0x7fc49c1136d0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:30:30.066 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:29.222+0000 7fc4a3e00700 1 -- 172.21.15.107:0/2103592156 >> 172.21.15.107:0/2103592156 conn(0x7fc49c078070 msgr2=0x7fc49c1027a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:30:30.066 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:29.223+0000 7fc4a3e00700 1 -- 172.21.15.107:0/2103592156 shutdown_connections 2024-03-04T13:30:30.066 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:29.223+0000 7fc4a3e00700 1 -- 172.21.15.107:0/2103592156 wait complete. 2024-03-04T13:30:30.066 INFO:teuthology.orchestra.run.smithi107.stdout:Deploying mon service with default placement... 2024-03-04T13:30:30.340 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:30 smithi107 ceph-mon[28739]: from='mgr.14120 172.21.15.107:0/1095418985' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:30:30.341 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:30 smithi107 ceph-mon[28739]: Added host smithi107 2024-03-04T13:30:30.341 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:30 smithi107 ceph-mon[28739]: from='mgr.14120 172.21.15.107:0/1095418985' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:30:32.200 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-03-04T13:30:32.200 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:30.758+0000 7fd041a00700 1 Processor -- start 2024-03-04T13:30:32.200 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:30.758+0000 7fd041a00700 1 -- start start 2024-03-04T13:30:32.202 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:30.759+0000 7fd041a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd03c07c7a0 0x7fd03c07cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:30:32.202 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:30.759+0000 7fd041a00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd03c07d0f0 con 0x7fd03c07c7a0 2024-03-04T13:30:32.202 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:30.759+0000 7fd03aa00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd03c07c7a0 0x7fd03c07cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:30:32.202 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:30.759+0000 7fd03aa00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd03c07c7a0 0x7fd03c07cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59098/0 (socket says 172.21.15.107:59098) 2024-03-04T13:30:32.202 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:30.759+0000 7fd03aa00700 1 -- 172.21.15.107:0/421453787 learned_addr learned my addr 172.21.15.107:0/421453787 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:30:32.202 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:30.760+0000 7fd03aa00700 1 -- 172.21.15.107:0/421453787 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd03c07d230 con 0x7fd03c07c7a0 2024-03-04T13:30:32.202 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:30.760+0000 7fd03aa00700 1 --2- 172.21.15.107:0/421453787 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd03c07c7a0 0x7fd03c07cbb0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd02c0098e0 tx=0x7fd02c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59a94c3d0cf4a333 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:30:32.202 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:30.760+0000 7fd039600700 1 -- 172.21.15.107:0/421453787 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd02c00b860 con 0x7fd03c07c7a0 2024-03-04T13:30:32.204 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:30.761+0000 7fd039600700 1 -- 172.21.15.107:0/421453787 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fd02c00be50 con 0x7fd03c07c7a0 2024-03-04T13:30:32.205 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:30.761+0000 7fd039600700 1 -- 172.21.15.107:0/421453787 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd02c010380 con 0x7fd03c07c7a0 2024-03-04T13:30:32.205 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:30.761+0000 7fd041a00700 1 -- 172.21.15.107:0/421453787 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd03c07c7a0 msgr2=0x7fd03c07cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:30:32.205 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:30.761+0000 7fd041a00700 1 --2- 172.21.15.107:0/421453787 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd03c07c7a0 0x7fd03c07cbb0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd02c0098e0 tx=0x7fd02c009250 comp rx=0 tx=0).stop 2024-03-04T13:30:32.205 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:30.761+0000 7fd041a00700 1 -- 172.21.15.107:0/421453787 shutdown_connections 2024-03-04T13:30:32.205 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:30.761+0000 7fd041a00700 1 --2- 172.21.15.107:0/421453787 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd03c07c7a0 0x7fd03c07cbb0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:30:32.205 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:30.761+0000 7fd041a00700 1 -- 172.21.15.107:0/421453787 >> 172.21.15.107:0/421453787 conn(0x7fd03c078030 msgr2=0x7fd03c07a460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:30:32.205 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:30.761+0000 7fd041a00700 1 -- 172.21.15.107:0/421453787 shutdown_connections 2024-03-04T13:30:32.205 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:30.761+0000 7fd041a00700 1 -- 172.21.15.107:0/421453787 wait complete. 2024-03-04T13:30:32.205 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:30.762+0000 7fd041a00700 1 Processor -- start 2024-03-04T13:30:32.205 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:30.762+0000 7fd041a00700 1 -- start start 2024-03-04T13:30:32.205 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:30.762+0000 7fd041a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd03c07c7a0 0x7fd03c117bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:30:32.205 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:30.762+0000 7fd041a00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd03c1180f0 con 0x7fd03c07c7a0 2024-03-04T13:30:32.205 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:30.763+0000 7fd03aa00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd03c07c7a0 0x7fd03c117bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:30:32.205 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:30.763+0000 7fd03aa00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd03c07c7a0 0x7fd03c117bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59102/0 (socket says 172.21.15.107:59102) 2024-03-04T13:30:32.207 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:30.763+0000 7fd03aa00700 1 -- 172.21.15.107:0/3781389251 learned_addr learned my addr 172.21.15.107:0/3781389251 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:30:32.207 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:30.763+0000 7fd03aa00700 1 -- 172.21.15.107:0/3781389251 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd02c0094f0 con 0x7fd03c07c7a0 2024-03-04T13:30:32.207 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:30.763+0000 7fd03aa00700 1 --2- 172.21.15.107:0/3781389251 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd03c07c7a0 0x7fd03c117bb0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd02c003050 tx=0x7fd02c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:30:32.208 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:30.764+0000 7fd032a00700 1 -- 172.21.15.107:0/3781389251 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd02c010760 con 0x7fd03c07c7a0 2024-03-04T13:30:32.208 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:30.764+0000 7fd041a00700 1 -- 172.21.15.107:0/3781389251 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd03c1182f0 con 0x7fd03c07c7a0 2024-03-04T13:30:32.208 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:30.764+0000 7fd032a00700 1 -- 172.21.15.107:0/3781389251 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fd02c010d50 con 0x7fd03c07c7a0 2024-03-04T13:30:32.208 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:30.764+0000 7fd032a00700 1 -- 172.21.15.107:0/3781389251 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd02c01ae20 con 0x7fd03c07c7a0 2024-03-04T13:30:32.208 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:30.764+0000 7fd041a00700 1 -- 172.21.15.107:0/3781389251 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd03c118f90 con 0x7fd03c07c7a0 2024-03-04T13:30:32.208 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:30.765+0000 7fd032a00700 1 -- 172.21.15.107:0/3781389251 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7fd02c0108c0 con 0x7fd03c07c7a0 2024-03-04T13:30:32.208 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:30.765+0000 7fd041a00700 1 -- 172.21.15.107:0/3781389251 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd03c05f5c0 con 0x7fd03c07c7a0 2024-03-04T13:30:32.208 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:30.765+0000 7fd032a00700 1 --2- 172.21.15.107:0/3781389251 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7fd020036770 0x7fd020038c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:30:32.208 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:30.765+0000 7fd032a00700 1 -- 172.21.15.107:0/3781389251 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fd02c04a600 con 0x7fd03c07c7a0 2024-03-04T13:30:32.208 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:30.766+0000 7fd03a000700 1 --2- 172.21.15.107:0/3781389251 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7fd020036770 0x7fd020038c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:30:32.208 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:30.766+0000 7fd03a000700 1 --2- 172.21.15.107:0/3781389251 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7fd020036770 0x7fd020038c20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd028009990 tx=0x7fd028006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:30:32.208 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:30.771+0000 7fd032a00700 1 -- 172.21.15.107:0/3781389251 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fd02c01a6a0 con 0x7fd03c07c7a0 2024-03-04T13:30:32.210 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:30.996+0000 7fd041a00700 1 -- 172.21.15.107:0/3781389251 --> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}) v1 -- 0x7fd03c07b390 con 0x7fd020036770 2024-03-04T13:30:32.210 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:31.003+0000 7fd032a00700 1 -- 172.21.15.107:0/3781389251 <== mgr.14120 v2:172.21.15.107:6800/2371648646 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fd03c07b390 con 0x7fd020036770 2024-03-04T13:30:32.210 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:31.006+0000 7fd041a00700 1 -- 172.21.15.107:0/3781389251 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7fd020036770 msgr2=0x7fd020038c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:30:32.210 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:31.006+0000 7fd041a00700 1 --2- 172.21.15.107:0/3781389251 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7fd020036770 0x7fd020038c20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd028009990 tx=0x7fd028006e40 comp rx=0 tx=0).stop 2024-03-04T13:30:32.210 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:31.006+0000 7fd041a00700 1 -- 172.21.15.107:0/3781389251 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd03c07c7a0 msgr2=0x7fd03c117bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:30:32.210 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:31.006+0000 7fd041a00700 1 --2- 172.21.15.107:0/3781389251 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd03c07c7a0 0x7fd03c117bb0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd02c003050 tx=0x7fd02c009250 comp rx=0 tx=0).stop 2024-03-04T13:30:32.210 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:31.006+0000 7fd041a00700 1 -- 172.21.15.107:0/3781389251 shutdown_connections 2024-03-04T13:30:32.210 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:31.006+0000 7fd041a00700 1 --2- 172.21.15.107:0/3781389251 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7fd020036770 0x7fd020038c20 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:30:32.210 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:31.006+0000 7fd041a00700 1 --2- 172.21.15.107:0/3781389251 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd03c07c7a0 0x7fd03c117bb0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:30:32.210 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:31.006+0000 7fd041a00700 1 -- 172.21.15.107:0/3781389251 >> 172.21.15.107:0/3781389251 conn(0x7fd03c078030 msgr2=0x7fd03c07a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:30:32.211 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:31.006+0000 7fd041a00700 1 -- 172.21.15.107:0/3781389251 shutdown_connections 2024-03-04T13:30:32.211 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:31.006+0000 7fd041a00700 1 -- 172.21.15.107:0/3781389251 wait complete. 2024-03-04T13:30:32.211 INFO:teuthology.orchestra.run.smithi107.stdout:Deploying mgr service with default placement... 2024-03-04T13:30:32.337 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:32 smithi107 ceph-mon[28739]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:30:32.337 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:32 smithi107 ceph-mon[28739]: Saving service mon spec with placement count:5 2024-03-04T13:30:32.338 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:32 smithi107 ceph-mon[28739]: from='mgr.14120 172.21.15.107:0/1095418985' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:30:34.337 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:33 smithi107 ceph-mon[28739]: from='mgr.14120 172.21.15.107:0/1095418985' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:30:34.337 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:33 smithi107 ceph-mon[28739]: from='mgr.14120 172.21.15.107:0/1095418985' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:30:34.338 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:33 smithi107 ceph-mon[28739]: from='mgr.14120 172.21.15.107:0/1095418985' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:30:34.507 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-03-04T13:30:34.507 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:33.462+0000 7f08ab600700 1 Processor -- start 2024-03-04T13:30:34.507 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:33.462+0000 7f08ab600700 1 -- start start 2024-03-04T13:30:34.507 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:33.463+0000 7f08ab600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f08a4102770 0x7f08a4102b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:30:34.507 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:33.463+0000 7f08ab600700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08a41030c0 con 0x7f08a4102770 2024-03-04T13:30:34.507 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:33.463+0000 7f08a8c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f08a4102770 0x7f08a4102b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:30:34.507 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:33.463+0000 7f08a8c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f08a4102770 0x7f08a4102b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59118/0 (socket says 172.21.15.107:59118) 2024-03-04T13:30:34.507 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:33.463+0000 7f08a8c00700 1 -- 172.21.15.107:0/1938426516 learned_addr learned my addr 172.21.15.107:0/1938426516 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:30:34.507 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:33.464+0000 7f08a8c00700 1 -- 172.21.15.107:0/1938426516 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08a4103200 con 0x7f08a4102770 2024-03-04T13:30:34.507 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:33.464+0000 7f08a8c00700 1 --2- 172.21.15.107:0/1938426516 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f08a4102770 0x7f08a4102b80 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f08980098e0 tx=0x7f0898009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d4dfaf53ff8464e server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:30:34.508 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:33.464+0000 7f08a3400700 1 -- 172.21.15.107:0/1938426516 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f089800b860 con 0x7f08a4102770 2024-03-04T13:30:34.508 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:33.464+0000 7f08a3400700 1 -- 172.21.15.107:0/1938426516 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f089800be90 con 0x7f08a4102770 2024-03-04T13:30:34.508 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:33.464+0000 7f08a3400700 1 -- 172.21.15.107:0/1938426516 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f08980103c0 con 0x7f08a4102770 2024-03-04T13:30:34.508 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:33.465+0000 7f08ab600700 1 -- 172.21.15.107:0/1938426516 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f08a4102770 msgr2=0x7f08a4102b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:30:34.508 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:33.465+0000 7f08ab600700 1 --2- 172.21.15.107:0/1938426516 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f08a4102770 0x7f08a4102b80 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f08980098e0 tx=0x7f0898009250 comp rx=0 tx=0).stop 2024-03-04T13:30:34.508 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:33.465+0000 7f08ab600700 1 -- 172.21.15.107:0/1938426516 shutdown_connections 2024-03-04T13:30:34.508 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:33.465+0000 7f08ab600700 1 --2- 172.21.15.107:0/1938426516 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f08a4102770 0x7f08a4102b80 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:30:34.508 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:33.465+0000 7f08ab600700 1 -- 172.21.15.107:0/1938426516 >> 172.21.15.107:0/1938426516 conn(0x7f08a40fded0 msgr2=0x7f08a4100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:30:34.508 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:33.465+0000 7f08ab600700 1 -- 172.21.15.107:0/1938426516 shutdown_connections 2024-03-04T13:30:34.508 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:33.465+0000 7f08ab600700 1 -- 172.21.15.107:0/1938426516 wait complete. 2024-03-04T13:30:34.508 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:33.466+0000 7f08ab600700 1 Processor -- start 2024-03-04T13:30:34.508 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:33.466+0000 7f08ab600700 1 -- start start 2024-03-04T13:30:34.508 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:33.467+0000 7f08ab600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f08a4102770 0x7f08a4113a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:30:34.508 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:33.467+0000 7f08ab600700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08a4113fb0 con 0x7f08a4102770 2024-03-04T13:30:34.509 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:33.467+0000 7f08a8c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f08a4102770 0x7f08a4113a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:30:34.509 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:33.467+0000 7f08a8c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f08a4102770 0x7f08a4113a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59124/0 (socket says 172.21.15.107:59124) 2024-03-04T13:30:34.509 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:33.467+0000 7f08a8c00700 1 -- 172.21.15.107:0/3473603147 learned_addr learned my addr 172.21.15.107:0/3473603147 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:30:34.509 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:33.467+0000 7f08a8c00700 1 -- 172.21.15.107:0/3473603147 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08980094f0 con 0x7f08a4102770 2024-03-04T13:30:34.509 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:33.468+0000 7f08a8c00700 1 --2- 172.21.15.107:0/3473603147 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f08a4102770 0x7f08a4113a70 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0898006b00 tx=0x7f0898009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:30:34.509 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:33.468+0000 7f08a0c00700 1 -- 172.21.15.107:0/3473603147 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f08980109a0 con 0x7f08a4102770 2024-03-04T13:30:34.509 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:33.468+0000 7f08a0c00700 1 -- 172.21.15.107:0/3473603147 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f089801a430 con 0x7f08a4102770 2024-03-04T13:30:34.509 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:33.468+0000 7f08ab600700 1 -- 172.21.15.107:0/3473603147 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08a41141b0 con 0x7f08a4102770 2024-03-04T13:30:34.509 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:33.468+0000 7f08ab600700 1 -- 172.21.15.107:0/3473603147 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08a4114e50 con 0x7f08a4102770 2024-03-04T13:30:34.509 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:33.468+0000 7f08a0c00700 1 -- 172.21.15.107:0/3473603147 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f08980245b0 con 0x7f08a4102770 2024-03-04T13:30:34.509 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:33.469+0000 7f08a0c00700 1 -- 172.21.15.107:0/3473603147 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f0898010b00 con 0x7f08a4102770 2024-03-04T13:30:34.509 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:33.470+0000 7f08ab600700 1 -- 172.21.15.107:0/3473603147 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08a405f5c0 con 0x7f08a4102770 2024-03-04T13:30:34.509 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:33.470+0000 7f08a0c00700 1 --2- 172.21.15.107:0/3473603147 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7f088c0367e0 0x7f088c038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:30:34.510 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:33.470+0000 7f08a0c00700 1 -- 172.21.15.107:0/3473603147 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f089804a6f0 con 0x7f08a4102770 2024-03-04T13:30:34.510 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:33.470+0000 7f08a3e00700 1 --2- 172.21.15.107:0/3473603147 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7f088c0367e0 0x7f088c038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:30:34.510 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:33.471+0000 7f08a3e00700 1 --2- 172.21.15.107:0/3473603147 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7f088c0367e0 0x7f088c038c90 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f0890009990 tx=0x7f0890006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:30:34.510 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:33.475+0000 7f08a0c00700 1 -- 172.21.15.107:0/3473603147 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f089802c340 con 0x7f08a4102770 2024-03-04T13:30:34.510 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:33.702+0000 7f08ab600700 1 -- 172.21.15.107:0/3473603147 --> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}) v1 -- 0x7f08a4104720 con 0x7f088c0367e0 2024-03-04T13:30:34.510 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:33.708+0000 7f08a0c00700 1 -- 172.21.15.107:0/3473603147 <== mgr.14120 v2:172.21.15.107:6800/2371648646 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f08a4104720 con 0x7f088c0367e0 2024-03-04T13:30:34.510 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:33.710+0000 7f08ab600700 1 -- 172.21.15.107:0/3473603147 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7f088c0367e0 msgr2=0x7f088c038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:30:34.510 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:33.710+0000 7f08ab600700 1 --2- 172.21.15.107:0/3473603147 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7f088c0367e0 0x7f088c038c90 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f0890009990 tx=0x7f0890006e40 comp rx=0 tx=0).stop 2024-03-04T13:30:34.510 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:33.710+0000 7f08ab600700 1 -- 172.21.15.107:0/3473603147 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f08a4102770 msgr2=0x7f08a4113a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:30:34.510 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:33.710+0000 7f08ab600700 1 --2- 172.21.15.107:0/3473603147 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f08a4102770 0x7f08a4113a70 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0898006b00 tx=0x7f0898009250 comp rx=0 tx=0).stop 2024-03-04T13:30:34.510 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:33.710+0000 7f08ab600700 1 -- 172.21.15.107:0/3473603147 shutdown_connections 2024-03-04T13:30:34.510 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:33.710+0000 7f08ab600700 1 --2- 172.21.15.107:0/3473603147 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7f088c0367e0 0x7f088c038c90 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:30:34.510 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:33.710+0000 7f08ab600700 1 --2- 172.21.15.107:0/3473603147 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f08a4102770 0x7f08a4113a70 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:30:34.510 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:33.710+0000 7f08ab600700 1 -- 172.21.15.107:0/3473603147 >> 172.21.15.107:0/3473603147 conn(0x7f08a40fded0 msgr2=0x7f08a40fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:30:34.510 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:33.710+0000 7f08ab600700 1 -- 172.21.15.107:0/3473603147 shutdown_connections 2024-03-04T13:30:34.511 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:33.710+0000 7f08ab600700 1 -- 172.21.15.107:0/3473603147 wait complete. 2024-03-04T13:30:34.511 INFO:teuthology.orchestra.run.smithi107.stdout:Deploying crash service with default placement... 2024-03-04T13:30:35.081 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:34 smithi107 ceph-mon[28739]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:30:35.081 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:34 smithi107 ceph-mon[28739]: Saving service mgr spec with placement count:2 2024-03-04T13:30:36.525 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:36 smithi107 ceph-mon[28739]: from='client.14146 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:30:36.526 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:36 smithi107 ceph-mon[28739]: Saving service crash spec with placement * 2024-03-04T13:30:36.526 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:36 smithi107 ceph-mon[28739]: from='mgr.14120 172.21.15.107:0/1095418985' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:30:36.536 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout Scheduled crash update... 2024-03-04T13:30:36.541 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:35.087+0000 7f919b200700 1 Processor -- start 2024-03-04T13:30:36.541 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:35.087+0000 7f919b200700 1 -- start start 2024-03-04T13:30:36.541 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:35.088+0000 7f919b200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9194102770 0x7f9194102b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:30:36.541 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:35.088+0000 7f919b200700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91941030c0 con 0x7f9194102770 2024-03-04T13:30:36.541 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:35.088+0000 7f9193e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9194102770 0x7f9194102b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:30:36.542 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:35.088+0000 7f9193e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9194102770 0x7f9194102b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59134/0 (socket says 172.21.15.107:59134) 2024-03-04T13:30:36.542 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:35.088+0000 7f9193e00700 1 -- 172.21.15.107:0/1549943700 learned_addr learned my addr 172.21.15.107:0/1549943700 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:30:36.542 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:35.089+0000 7f9193e00700 1 -- 172.21.15.107:0/1549943700 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9194103200 con 0x7f9194102770 2024-03-04T13:30:36.542 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:35.089+0000 7f9193e00700 1 --2- 172.21.15.107:0/1549943700 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9194102770 0x7f9194102b80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f91840098e0 tx=0x7f9184009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f314076c039a34b1 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:30:36.542 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:35.089+0000 7f9192a00700 1 -- 172.21.15.107:0/1549943700 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f918400b860 con 0x7f9194102770 2024-03-04T13:30:36.542 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:35.090+0000 7f9192a00700 1 -- 172.21.15.107:0/1549943700 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f918400be90 con 0x7f9194102770 2024-03-04T13:30:36.542 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:35.090+0000 7f9192a00700 1 -- 172.21.15.107:0/1549943700 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f91840103c0 con 0x7f9194102770 2024-03-04T13:30:36.542 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:35.090+0000 7f919b200700 1 -- 172.21.15.107:0/1549943700 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9194102770 msgr2=0x7f9194102b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:30:36.542 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:35.090+0000 7f919b200700 1 --2- 172.21.15.107:0/1549943700 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9194102770 0x7f9194102b80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f91840098e0 tx=0x7f9184009250 comp rx=0 tx=0).stop 2024-03-04T13:30:36.542 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:35.091+0000 7f919b200700 1 -- 172.21.15.107:0/1549943700 shutdown_connections 2024-03-04T13:30:36.542 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:35.091+0000 7f919b200700 1 --2- 172.21.15.107:0/1549943700 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9194102770 0x7f9194102b80 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:30:36.542 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:35.091+0000 7f919b200700 1 -- 172.21.15.107:0/1549943700 >> 172.21.15.107:0/1549943700 conn(0x7f91940fded0 msgr2=0x7f9194100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:30:36.542 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:35.091+0000 7f919b200700 1 -- 172.21.15.107:0/1549943700 shutdown_connections 2024-03-04T13:30:36.543 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:35.091+0000 7f919b200700 1 -- 172.21.15.107:0/1549943700 wait complete. 2024-03-04T13:30:36.543 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:35.092+0000 7f919b200700 1 Processor -- start 2024-03-04T13:30:36.543 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:35.092+0000 7f919b200700 1 -- start start 2024-03-04T13:30:36.543 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:35.093+0000 7f919b200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9194102770 0x7f9194117b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:30:36.543 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:35.093+0000 7f919b200700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9194118090 con 0x7f9194102770 2024-03-04T13:30:36.543 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:35.093+0000 7f9193e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9194102770 0x7f9194117b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:30:36.543 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:35.093+0000 7f9193e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9194102770 0x7f9194117b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59136/0 (socket says 172.21.15.107:59136) 2024-03-04T13:30:36.543 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:35.093+0000 7f9193e00700 1 -- 172.21.15.107:0/4288079386 learned_addr learned my addr 172.21.15.107:0/4288079386 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:30:36.543 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:35.094+0000 7f9193e00700 1 -- 172.21.15.107:0/4288079386 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91840094f0 con 0x7f9194102770 2024-03-04T13:30:36.543 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:35.094+0000 7f9193e00700 1 --2- 172.21.15.107:0/4288079386 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9194102770 0x7f9194117b50 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9184006b00 tx=0x7f9184009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:30:36.543 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:35.094+0000 7f918be00700 1 -- 172.21.15.107:0/4288079386 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f91840109a0 con 0x7f9194102770 2024-03-04T13:30:36.544 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:35.094+0000 7f918be00700 1 -- 172.21.15.107:0/4288079386 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f918401a430 con 0x7f9194102770 2024-03-04T13:30:36.544 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:35.094+0000 7f918be00700 1 -- 172.21.15.107:0/4288079386 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f91840245b0 con 0x7f9194102770 2024-03-04T13:30:36.544 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:35.094+0000 7f919b200700 1 -- 172.21.15.107:0/4288079386 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9194118290 con 0x7f9194102770 2024-03-04T13:30:36.544 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:35.095+0000 7f919b200700 1 -- 172.21.15.107:0/4288079386 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9194118f30 con 0x7f9194102770 2024-03-04T13:30:36.544 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:35.096+0000 7f919b200700 1 -- 172.21.15.107:0/4288079386 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f919405f5c0 con 0x7f9194102770 2024-03-04T13:30:36.544 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:35.096+0000 7f918be00700 1 -- 172.21.15.107:0/4288079386 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f918401aaa0 con 0x7f9194102770 2024-03-04T13:30:36.544 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:35.097+0000 7f918be00700 1 --2- 172.21.15.107:0/4288079386 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7f91780367e0 0x7f9178038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:30:36.544 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:35.097+0000 7f918be00700 1 -- 172.21.15.107:0/4288079386 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f918404a680 con 0x7f9194102770 2024-03-04T13:30:36.544 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:35.097+0000 7f9193400700 1 --2- 172.21.15.107:0/4288079386 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7f91780367e0 0x7f9178038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:30:36.544 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:35.098+0000 7f9193400700 1 --2- 172.21.15.107:0/4288079386 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7f91780367e0 0x7f9178038c90 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9180009990 tx=0x7f9180006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:30:36.544 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:35.102+0000 7f918be00700 1 -- 172.21.15.107:0/4288079386 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f9184018940 con 0x7f9194102770 2024-03-04T13:30:36.544 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:35.325+0000 7f919b200700 1 -- 172.21.15.107:0/4288079386 --> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}) v1 -- 0x7f91940ff3a0 con 0x7f91780367e0 2024-03-04T13:30:36.544 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:35.331+0000 7f918be00700 1 -- 172.21.15.107:0/4288079386 <== mgr.14120 v2:172.21.15.107:6800/2371648646 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7f91940ff3a0 con 0x7f91780367e0 2024-03-04T13:30:36.545 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:35.333+0000 7f919b200700 1 -- 172.21.15.107:0/4288079386 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7f91780367e0 msgr2=0x7f9178038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:30:36.545 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:35.333+0000 7f919b200700 1 --2- 172.21.15.107:0/4288079386 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7f91780367e0 0x7f9178038c90 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9180009990 tx=0x7f9180006e40 comp rx=0 tx=0).stop 2024-03-04T13:30:36.545 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:35.333+0000 7f919b200700 1 -- 172.21.15.107:0/4288079386 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9194102770 msgr2=0x7f9194117b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:30:36.545 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:35.333+0000 7f919b200700 1 --2- 172.21.15.107:0/4288079386 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9194102770 0x7f9194117b50 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9184006b00 tx=0x7f9184009250 comp rx=0 tx=0).stop 2024-03-04T13:30:36.545 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:35.334+0000 7f919b200700 1 -- 172.21.15.107:0/4288079386 shutdown_connections 2024-03-04T13:30:36.545 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:35.334+0000 7f919b200700 1 --2- 172.21.15.107:0/4288079386 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7f91780367e0 0x7f9178038c90 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:30:36.545 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:35.334+0000 7f919b200700 1 --2- 172.21.15.107:0/4288079386 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9194102770 0x7f9194117b50 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:30:36.545 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:35.334+0000 7f919b200700 1 -- 172.21.15.107:0/4288079386 >> 172.21.15.107:0/4288079386 conn(0x7f91940fded0 msgr2=0x7f91940fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:30:36.545 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:35.334+0000 7f919b200700 1 -- 172.21.15.107:0/4288079386 shutdown_connections 2024-03-04T13:30:36.545 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:35.334+0000 7f919b200700 1 -- 172.21.15.107:0/4288079386 wait complete. 2024-03-04T13:30:36.545 INFO:teuthology.orchestra.run.smithi107.stdout:Deploying prometheus service with default placement... 2024-03-04T13:30:38.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:37 smithi107 ceph-mon[28739]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:30:38.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:37 smithi107 ceph-mon[28739]: from='mgr.14120 172.21.15.107:0/1095418985' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:30:38.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:37 smithi107 ceph-mon[28739]: from='mgr.14120 172.21.15.107:0/1095418985' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:30:38.128 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout Scheduled prometheus update... 2024-03-04T13:30:38.128 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:37.164+0000 7f82e0c00700 1 Processor -- start 2024-03-04T13:30:38.128 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:37.164+0000 7f82e0c00700 1 -- start start 2024-03-04T13:30:38.129 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:37.164+0000 7f82e0c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82dc102770 0x7f82dc102b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:30:38.129 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:37.164+0000 7f82e0c00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82dc1030c0 con 0x7f82dc102770 2024-03-04T13:30:38.129 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:37.165+0000 7f82d9e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82dc102770 0x7f82dc102b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:30:38.129 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:37.165+0000 7f82d9e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82dc102770 0x7f82dc102b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59142/0 (socket says 172.21.15.107:59142) 2024-03-04T13:30:38.129 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:37.165+0000 7f82d9e00700 1 -- 172.21.15.107:0/4228189388 learned_addr learned my addr 172.21.15.107:0/4228189388 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:30:38.129 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:37.165+0000 7f82d9e00700 1 -- 172.21.15.107:0/4228189388 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82dc103200 con 0x7f82dc102770 2024-03-04T13:30:38.129 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:37.165+0000 7f82d9e00700 1 --2- 172.21.15.107:0/4228189388 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82dc102770 0x7f82dc102b80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f82cc0098e0 tx=0x7f82cc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5170e3044ece46f0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:30:38.129 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:37.165+0000 7f82d9400700 1 -- 172.21.15.107:0/4228189388 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f82cc00b860 con 0x7f82dc102770 2024-03-04T13:30:38.129 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:37.166+0000 7f82d9400700 1 -- 172.21.15.107:0/4228189388 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f82cc00be90 con 0x7f82dc102770 2024-03-04T13:30:38.129 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:37.166+0000 7f82d9400700 1 -- 172.21.15.107:0/4228189388 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f82cc0103c0 con 0x7f82dc102770 2024-03-04T13:30:38.129 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:37.166+0000 7f82e0c00700 1 -- 172.21.15.107:0/4228189388 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82dc102770 msgr2=0x7f82dc102b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:30:38.129 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:37.166+0000 7f82e0c00700 1 --2- 172.21.15.107:0/4228189388 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82dc102770 0x7f82dc102b80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f82cc0098e0 tx=0x7f82cc009250 comp rx=0 tx=0).stop 2024-03-04T13:30:38.129 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:37.166+0000 7f82e0c00700 1 -- 172.21.15.107:0/4228189388 shutdown_connections 2024-03-04T13:30:38.130 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:37.166+0000 7f82e0c00700 1 --2- 172.21.15.107:0/4228189388 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82dc102770 0x7f82dc102b80 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:30:38.130 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:37.166+0000 7f82e0c00700 1 -- 172.21.15.107:0/4228189388 >> 172.21.15.107:0/4228189388 conn(0x7f82dc0fded0 msgr2=0x7f82dc100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:30:38.130 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:37.166+0000 7f82e0c00700 1 -- 172.21.15.107:0/4228189388 shutdown_connections 2024-03-04T13:30:38.130 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:37.166+0000 7f82e0c00700 1 -- 172.21.15.107:0/4228189388 wait complete. 2024-03-04T13:30:38.130 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:37.167+0000 7f82e0c00700 1 Processor -- start 2024-03-04T13:30:38.130 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:37.168+0000 7f82e0c00700 1 -- start start 2024-03-04T13:30:38.130 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:37.168+0000 7f82e0c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82dc102770 0x7f82dc113a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:30:38.130 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:37.168+0000 7f82e0c00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82dc113fb0 con 0x7f82dc102770 2024-03-04T13:30:38.130 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:37.168+0000 7f82d9e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82dc102770 0x7f82dc113a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:30:38.130 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:37.168+0000 7f82d9e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82dc102770 0x7f82dc113a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59148/0 (socket says 172.21.15.107:59148) 2024-03-04T13:30:38.130 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:37.168+0000 7f82d9e00700 1 -- 172.21.15.107:0/3863192248 learned_addr learned my addr 172.21.15.107:0/3863192248 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:30:38.130 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:37.169+0000 7f82d9e00700 1 -- 172.21.15.107:0/3863192248 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82cc0094f0 con 0x7f82dc102770 2024-03-04T13:30:38.130 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:37.169+0000 7f82d9e00700 1 --2- 172.21.15.107:0/3863192248 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82dc102770 0x7f82dc113a70 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f82cc006b00 tx=0x7f82cc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:30:38.130 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:37.169+0000 7f82d2a00700 1 -- 172.21.15.107:0/3863192248 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f82cc0109a0 con 0x7f82dc102770 2024-03-04T13:30:38.131 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:37.169+0000 7f82d2a00700 1 -- 172.21.15.107:0/3863192248 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f82cc01a430 con 0x7f82dc102770 2024-03-04T13:30:38.131 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:37.169+0000 7f82d2a00700 1 -- 172.21.15.107:0/3863192248 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f82cc0245b0 con 0x7f82dc102770 2024-03-04T13:30:38.131 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:37.169+0000 7f82e0c00700 1 -- 172.21.15.107:0/3863192248 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82dc1141b0 con 0x7f82dc102770 2024-03-04T13:30:38.131 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:37.169+0000 7f82e0c00700 1 -- 172.21.15.107:0/3863192248 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82dc114e50 con 0x7f82dc102770 2024-03-04T13:30:38.131 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:37.170+0000 7f82e0c00700 1 -- 172.21.15.107:0/3863192248 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82dc05f5c0 con 0x7f82dc102770 2024-03-04T13:30:38.131 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:37.170+0000 7f82d2a00700 1 -- 172.21.15.107:0/3863192248 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f82cc01aaa0 con 0x7f82dc102770 2024-03-04T13:30:38.131 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:37.171+0000 7f82d2a00700 1 --2- 172.21.15.107:0/3863192248 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7f82c00367e0 0x7f82c0038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:30:38.131 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:37.171+0000 7f82d2a00700 1 -- 172.21.15.107:0/3863192248 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f82cc04a750 con 0x7f82dc102770 2024-03-04T13:30:38.131 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:37.171+0000 7f82d1400700 1 --2- 172.21.15.107:0/3863192248 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7f82c00367e0 0x7f82c0038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:30:38.131 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:37.172+0000 7f82d1400700 1 --2- 172.21.15.107:0/3863192248 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7f82c00367e0 0x7f82c0038c90 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f82c4009990 tx=0x7f82c4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:30:38.131 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:37.177+0000 7f82d2a00700 1 -- 172.21.15.107:0/3863192248 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f82cc018940 con 0x7f82dc102770 2024-03-04T13:30:38.131 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:37.399+0000 7f82e0c00700 1 -- 172.21.15.107:0/3863192248 --> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}) v1 -- 0x7f82dc1046c0 con 0x7f82c00367e0 2024-03-04T13:30:38.131 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:37.406+0000 7f82d2a00700 1 -- 172.21.15.107:0/3863192248 <== mgr.14120 v2:172.21.15.107:6800/2371648646 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+31 (secure 0 0 0) 0x7f82dc1046c0 con 0x7f82c00367e0 2024-03-04T13:30:38.131 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:37.409+0000 7f82e0c00700 1 -- 172.21.15.107:0/3863192248 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7f82c00367e0 msgr2=0x7f82c0038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:30:38.131 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:37.409+0000 7f82e0c00700 1 --2- 172.21.15.107:0/3863192248 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7f82c00367e0 0x7f82c0038c90 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f82c4009990 tx=0x7f82c4006e40 comp rx=0 tx=0).stop 2024-03-04T13:30:38.132 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:37.409+0000 7f82e0c00700 1 -- 172.21.15.107:0/3863192248 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82dc102770 msgr2=0x7f82dc113a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:30:38.132 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:37.409+0000 7f82e0c00700 1 --2- 172.21.15.107:0/3863192248 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82dc102770 0x7f82dc113a70 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f82cc006b00 tx=0x7f82cc009250 comp rx=0 tx=0).stop 2024-03-04T13:30:38.132 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:37.409+0000 7f82e0c00700 1 -- 172.21.15.107:0/3863192248 shutdown_connections 2024-03-04T13:30:38.132 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:37.409+0000 7f82e0c00700 1 --2- 172.21.15.107:0/3863192248 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7f82c00367e0 0x7f82c0038c90 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:30:38.132 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:37.409+0000 7f82e0c00700 1 --2- 172.21.15.107:0/3863192248 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82dc102770 0x7f82dc113a70 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:30:38.132 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:37.409+0000 7f82e0c00700 1 -- 172.21.15.107:0/3863192248 >> 172.21.15.107:0/3863192248 conn(0x7f82dc0fded0 msgr2=0x7f82dc0fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:30:38.132 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:37.409+0000 7f82e0c00700 1 -- 172.21.15.107:0/3863192248 shutdown_connections 2024-03-04T13:30:38.132 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:37.410+0000 7f82e0c00700 1 -- 172.21.15.107:0/3863192248 wait complete. 2024-03-04T13:30:38.132 INFO:teuthology.orchestra.run.smithi107.stdout:Deploying grafana service with default placement... 2024-03-04T13:30:38.720 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:38 smithi107 ceph-mon[28739]: from='client.14148 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:30:38.720 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:38 smithi107 ceph-mon[28739]: Saving service prometheus spec with placement count:1 2024-03-04T13:30:38.720 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:38 smithi107 ceph-mon[28739]: from='mgr.14120 172.21.15.107:0/1095418985' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:30:39.751 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:39 smithi107 ceph-mon[28739]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:30:39.751 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:39 smithi107 ceph-mon[28739]: from='mgr.14120 172.21.15.107:0/1095418985' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:30:39.762 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout Scheduled grafana update... 2024-03-04T13:30:39.762 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:38.786+0000 7fb086600700 1 Processor -- start 2024-03-04T13:30:39.763 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:38.786+0000 7fb086600700 1 -- start start 2024-03-04T13:30:39.763 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:38.786+0000 7fb086600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb080101670 0x7fb080101a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:30:39.763 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:38.786+0000 7fb086600700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb080101fc0 con 0x7fb080101670 2024-03-04T13:30:39.763 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:38.787+0000 7fb07f400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb080101670 0x7fb080101a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:30:39.763 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:38.787+0000 7fb07f400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb080101670 0x7fb080101a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57226/0 (socket says 172.21.15.107:57226) 2024-03-04T13:30:39.763 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:38.787+0000 7fb07f400700 1 -- 172.21.15.107:0/1862219722 learned_addr learned my addr 172.21.15.107:0/1862219722 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:30:39.763 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:38.787+0000 7fb07f400700 1 -- 172.21.15.107:0/1862219722 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb080102100 con 0x7fb080101670 2024-03-04T13:30:39.763 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:38.788+0000 7fb07f400700 1 --2- 172.21.15.107:0/1862219722 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb080101670 0x7fb080101a80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb0700098e0 tx=0x7fb070009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80a35a01c649186d server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:30:39.763 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:38.788+0000 7fb07e000700 1 -- 172.21.15.107:0/1862219722 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb07000b860 con 0x7fb080101670 2024-03-04T13:30:39.763 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:38.788+0000 7fb07e000700 1 -- 172.21.15.107:0/1862219722 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb07000be90 con 0x7fb080101670 2024-03-04T13:30:39.763 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:38.788+0000 7fb07e000700 1 -- 172.21.15.107:0/1862219722 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb0700103c0 con 0x7fb080101670 2024-03-04T13:30:39.763 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:38.789+0000 7fb086600700 1 -- 172.21.15.107:0/1862219722 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb080101670 msgr2=0x7fb080101a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:30:39.763 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:38.789+0000 7fb086600700 1 --2- 172.21.15.107:0/1862219722 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb080101670 0x7fb080101a80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb0700098e0 tx=0x7fb070009250 comp rx=0 tx=0).stop 2024-03-04T13:30:39.764 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:38.789+0000 7fb086600700 1 -- 172.21.15.107:0/1862219722 shutdown_connections 2024-03-04T13:30:39.764 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:38.789+0000 7fb086600700 1 --2- 172.21.15.107:0/1862219722 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb080101670 0x7fb080101a80 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:30:39.764 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:38.789+0000 7fb086600700 1 -- 172.21.15.107:0/1862219722 >> 172.21.15.107:0/1862219722 conn(0x7fb0800fcdd0 msgr2=0x7fb0800ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:30:39.764 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:38.789+0000 7fb086600700 1 -- 172.21.15.107:0/1862219722 shutdown_connections 2024-03-04T13:30:39.764 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:38.789+0000 7fb086600700 1 -- 172.21.15.107:0/1862219722 wait complete. 2024-03-04T13:30:39.764 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:38.790+0000 7fb086600700 1 Processor -- start 2024-03-04T13:30:39.764 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:38.791+0000 7fb086600700 1 -- start start 2024-03-04T13:30:39.764 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:38.791+0000 7fb086600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb080101670 0x7fb080113730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:30:39.764 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:38.791+0000 7fb086600700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb080113c70 con 0x7fb080101670 2024-03-04T13:30:39.764 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:38.791+0000 7fb07f400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb080101670 0x7fb080113730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:30:39.764 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:38.791+0000 7fb07f400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb080101670 0x7fb080113730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57234/0 (socket says 172.21.15.107:57234) 2024-03-04T13:30:39.764 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:38.791+0000 7fb07f400700 1 -- 172.21.15.107:0/1038742107 learned_addr learned my addr 172.21.15.107:0/1038742107 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:30:39.764 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:38.792+0000 7fb07f400700 1 -- 172.21.15.107:0/1038742107 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0700094f0 con 0x7fb080101670 2024-03-04T13:30:39.765 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:38.792+0000 7fb07f400700 1 --2- 172.21.15.107:0/1038742107 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb080101670 0x7fb080113730 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb070006b00 tx=0x7fb070009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:30:39.765 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:38.792+0000 7fb077400700 1 -- 172.21.15.107:0/1038742107 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb0700109a0 con 0x7fb080101670 2024-03-04T13:30:39.765 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:38.792+0000 7fb077400700 1 -- 172.21.15.107:0/1038742107 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb07001a430 con 0x7fb080101670 2024-03-04T13:30:39.765 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:38.792+0000 7fb077400700 1 -- 172.21.15.107:0/1038742107 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb0700245b0 con 0x7fb080101670 2024-03-04T13:30:39.765 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:38.792+0000 7fb086600700 1 -- 172.21.15.107:0/1038742107 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb080113e70 con 0x7fb080101670 2024-03-04T13:30:39.765 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:38.792+0000 7fb086600700 1 -- 172.21.15.107:0/1038742107 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb080114b10 con 0x7fb080101670 2024-03-04T13:30:39.765 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:38.794+0000 7fb086600700 1 -- 172.21.15.107:0/1038742107 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb08005f5c0 con 0x7fb080101670 2024-03-04T13:30:39.765 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:38.794+0000 7fb077400700 1 -- 172.21.15.107:0/1038742107 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7fb07001aaa0 con 0x7fb080101670 2024-03-04T13:30:39.765 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:38.794+0000 7fb077400700 1 --2- 172.21.15.107:0/1038742107 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7fb064036780 0x7fb064038c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:30:39.765 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:38.794+0000 7fb077400700 1 -- 172.21.15.107:0/1038742107 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fb07004a790 con 0x7fb080101670 2024-03-04T13:30:39.765 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:38.795+0000 7fb07ea00700 1 --2- 172.21.15.107:0/1038742107 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7fb064036780 0x7fb064038c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:30:39.765 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:38.795+0000 7fb07ea00700 1 --2- 172.21.15.107:0/1038742107 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7fb064036780 0x7fb064038c30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb06c009990 tx=0x7fb06c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:30:39.765 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:38.799+0000 7fb077400700 1 -- 172.21.15.107:0/1038742107 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fb070018940 con 0x7fb080101670 2024-03-04T13:30:39.765 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:39.025+0000 7fb086600700 1 -- 172.21.15.107:0/1038742107 --> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}) v1 -- 0x7fb0801156f0 con 0x7fb064036780 2024-03-04T13:30:39.766 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:39.034+0000 7fb077400700 1 -- 172.21.15.107:0/1038742107 <== mgr.14120 v2:172.21.15.107:6800/2371648646 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7fb0801156f0 con 0x7fb064036780 2024-03-04T13:30:39.766 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:39.036+0000 7fb086600700 1 -- 172.21.15.107:0/1038742107 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7fb064036780 msgr2=0x7fb064038c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:30:39.766 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:39.036+0000 7fb086600700 1 --2- 172.21.15.107:0/1038742107 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7fb064036780 0x7fb064038c30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb06c009990 tx=0x7fb06c006e40 comp rx=0 tx=0).stop 2024-03-04T13:30:39.766 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:39.036+0000 7fb086600700 1 -- 172.21.15.107:0/1038742107 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb080101670 msgr2=0x7fb080113730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:30:39.766 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:39.036+0000 7fb086600700 1 --2- 172.21.15.107:0/1038742107 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb080101670 0x7fb080113730 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb070006b00 tx=0x7fb070009250 comp rx=0 tx=0).stop 2024-03-04T13:30:39.766 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:39.037+0000 7fb086600700 1 -- 172.21.15.107:0/1038742107 shutdown_connections 2024-03-04T13:30:39.766 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:39.037+0000 7fb086600700 1 --2- 172.21.15.107:0/1038742107 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7fb064036780 0x7fb064038c30 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:30:39.766 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:39.037+0000 7fb086600700 1 --2- 172.21.15.107:0/1038742107 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb080101670 0x7fb080113730 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:30:39.766 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:39.037+0000 7fb086600700 1 -- 172.21.15.107:0/1038742107 >> 172.21.15.107:0/1038742107 conn(0x7fb0800fcdd0 msgr2=0x7fb0800ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:30:39.766 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:39.037+0000 7fb086600700 1 -- 172.21.15.107:0/1038742107 shutdown_connections 2024-03-04T13:30:39.766 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:39.037+0000 7fb086600700 1 -- 172.21.15.107:0/1038742107 wait complete. 2024-03-04T13:30:39.766 INFO:teuthology.orchestra.run.smithi107.stdout:Deploying node-exporter service with default placement... 2024-03-04T13:30:40.939 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:40 smithi107 ceph-mon[28739]: from='client.14150 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:30:40.939 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:40 smithi107 ceph-mon[28739]: Saving service grafana spec with placement count:1 2024-03-04T13:30:41.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:41 smithi107 ceph-mon[28739]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:30:41.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:41 smithi107 ceph-mon[28739]: from='mgr.14120 172.21.15.107:0/1095418985' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:30:41.889 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout Scheduled node-exporter update... 2024-03-04T13:30:41.889 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:40.945+0000 7f8270e00700 1 Processor -- start 2024-03-04T13:30:41.889 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:40.945+0000 7f8270e00700 1 -- start start 2024-03-04T13:30:41.889 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:40.945+0000 7f8270e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f826c076c40 0x7f826c105360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:30:41.889 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:40.945+0000 7f8270e00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f826c1058a0 con 0x7f826c076c40 2024-03-04T13:30:41.889 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:40.946+0000 7f8269e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f826c076c40 0x7f826c105360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:30:41.889 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:40.946+0000 7f8269e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f826c076c40 0x7f826c105360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57248/0 (socket says 172.21.15.107:57248) 2024-03-04T13:30:41.889 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:40.946+0000 7f8269e00700 1 -- 172.21.15.107:0/2680745903 learned_addr learned my addr 172.21.15.107:0/2680745903 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:30:41.890 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:40.946+0000 7f8269e00700 1 -- 172.21.15.107:0/2680745903 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f826c1059e0 con 0x7f826c076c40 2024-03-04T13:30:41.890 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:40.947+0000 7f8269e00700 1 --2- 172.21.15.107:0/2680745903 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f826c076c40 0x7f826c105360 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f825c009e50 tx=0x7f825c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8e49d2fce887f41 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:30:41.890 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:40.947+0000 7f8269400700 1 -- 172.21.15.107:0/2680745903 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f825c0092e0 con 0x7f826c076c40 2024-03-04T13:30:41.890 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:40.947+0000 7f8269400700 1 -- 172.21.15.107:0/2680745903 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f825c012430 con 0x7f826c076c40 2024-03-04T13:30:41.890 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:40.947+0000 7f8269400700 1 -- 172.21.15.107:0/2680745903 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f825c0075a0 con 0x7f826c076c40 2024-03-04T13:30:41.890 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:40.948+0000 7f8270e00700 1 -- 172.21.15.107:0/2680745903 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f826c076c40 msgr2=0x7f826c105360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:30:41.890 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:40.948+0000 7f8270e00700 1 --2- 172.21.15.107:0/2680745903 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f826c076c40 0x7f826c105360 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f825c009e50 tx=0x7f825c009840 comp rx=0 tx=0).stop 2024-03-04T13:30:41.890 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:40.948+0000 7f8270e00700 1 -- 172.21.15.107:0/2680745903 shutdown_connections 2024-03-04T13:30:41.890 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:40.948+0000 7f8270e00700 1 --2- 172.21.15.107:0/2680745903 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f826c076c40 0x7f826c105360 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:30:41.890 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:40.948+0000 7f8270e00700 1 -- 172.21.15.107:0/2680745903 >> 172.21.15.107:0/2680745903 conn(0x7f826c0fcdd0 msgr2=0x7f826c0ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:30:41.890 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:40.948+0000 7f8270e00700 1 -- 172.21.15.107:0/2680745903 shutdown_connections 2024-03-04T13:30:41.891 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:40.948+0000 7f8270e00700 1 -- 172.21.15.107:0/2680745903 wait complete. 2024-03-04T13:30:41.891 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:40.949+0000 7f8270e00700 1 Processor -- start 2024-03-04T13:30:41.891 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:40.949+0000 7f8270e00700 1 -- start start 2024-03-04T13:30:41.892 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:40.949+0000 7f8270e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f826c076c40 0x7f826c102de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:30:41.892 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:40.949+0000 7f8270e00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f826c101260 con 0x7f826c076c40 2024-03-04T13:30:41.892 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:40.949+0000 7f8269e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f826c076c40 0x7f826c102de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:30:41.892 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:40.950+0000 7f8269e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f826c076c40 0x7f826c102de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57264/0 (socket says 172.21.15.107:57264) 2024-03-04T13:30:41.892 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:40.950+0000 7f8269e00700 1 -- 172.21.15.107:0/1057666466 learned_addr learned my addr 172.21.15.107:0/1057666466 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:30:41.892 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:40.950+0000 7f8269e00700 1 -- 172.21.15.107:0/1057666466 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f825c009ae0 con 0x7f826c076c40 2024-03-04T13:30:41.892 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:40.950+0000 7f8269e00700 1 --2- 172.21.15.107:0/1057666466 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f826c076c40 0x7f826c102de0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f825c0127d0 tx=0x7f825c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:30:41.892 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:40.950+0000 7f8262a00700 1 -- 172.21.15.107:0/1057666466 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f825c0092e0 con 0x7f826c076c40 2024-03-04T13:30:41.892 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:40.950+0000 7f8262a00700 1 -- 172.21.15.107:0/1057666466 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f825c007ce0 con 0x7f826c076c40 2024-03-04T13:30:41.893 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:40.950+0000 7f8270e00700 1 -- 172.21.15.107:0/1057666466 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f826c101460 con 0x7f826c076c40 2024-03-04T13:30:41.893 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:40.950+0000 7f8262a00700 1 -- 172.21.15.107:0/1057666466 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f825c0258e0 con 0x7f826c076c40 2024-03-04T13:30:41.893 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:40.951+0000 7f8270e00700 1 -- 172.21.15.107:0/1057666466 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f826c1018f0 con 0x7f826c076c40 2024-03-04T13:30:41.893 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:40.952+0000 7f8262a00700 1 -- 172.21.15.107:0/1057666466 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f825c019050 con 0x7f826c076c40 2024-03-04T13:30:41.893 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:40.952+0000 7f8270e00700 1 -- 172.21.15.107:0/1057666466 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f826c04bed0 con 0x7f826c076c40 2024-03-04T13:30:41.893 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:40.952+0000 7f8262a00700 1 --2- 172.21.15.107:0/1057666466 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7f8250036790 0x7f8250038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:30:41.893 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:40.952+0000 7f8262a00700 1 -- 172.21.15.107:0/1057666466 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f825c04a250 con 0x7f826c076c40 2024-03-04T13:30:41.893 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:40.952+0000 7f8261400700 1 --2- 172.21.15.107:0/1057666466 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7f8250036790 0x7f8250038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:30:41.893 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:40.953+0000 7f8261400700 1 --2- 172.21.15.107:0/1057666466 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7f8250036790 0x7f8250038c40 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f8254009990 tx=0x7f8254006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:30:41.893 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:40.957+0000 7f8262a00700 1 -- 172.21.15.107:0/1057666466 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f825c011570 con 0x7f826c076c40 2024-03-04T13:30:41.893 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:41.175+0000 7f8270e00700 1 -- 172.21.15.107:0/1057666466 --> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}) v1 -- 0x7f826c102130 con 0x7f8250036790 2024-03-04T13:30:41.893 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:41.183+0000 7f8262a00700 1 -- 172.21.15.107:0/1057666466 <== mgr.14120 v2:172.21.15.107:6800/2371648646 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+34 (secure 0 0 0) 0x7f826c102130 con 0x7f8250036790 2024-03-04T13:30:41.894 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:41.185+0000 7f8270e00700 1 -- 172.21.15.107:0/1057666466 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7f8250036790 msgr2=0x7f8250038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:30:41.894 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:41.185+0000 7f8270e00700 1 --2- 172.21.15.107:0/1057666466 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7f8250036790 0x7f8250038c40 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f8254009990 tx=0x7f8254006e40 comp rx=0 tx=0).stop 2024-03-04T13:30:41.894 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:41.185+0000 7f8270e00700 1 -- 172.21.15.107:0/1057666466 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f826c076c40 msgr2=0x7f826c102de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:30:41.894 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:41.185+0000 7f8270e00700 1 --2- 172.21.15.107:0/1057666466 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f826c076c40 0x7f826c102de0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f825c0127d0 tx=0x7f825c009840 comp rx=0 tx=0).stop 2024-03-04T13:30:41.894 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:41.185+0000 7f8270e00700 1 -- 172.21.15.107:0/1057666466 shutdown_connections 2024-03-04T13:30:41.894 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:41.185+0000 7f8270e00700 1 --2- 172.21.15.107:0/1057666466 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7f8250036790 0x7f8250038c40 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:30:41.894 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:41.185+0000 7f8270e00700 1 --2- 172.21.15.107:0/1057666466 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f826c076c40 0x7f826c102de0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:30:41.894 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:41.185+0000 7f8270e00700 1 -- 172.21.15.107:0/1057666466 >> 172.21.15.107:0/1057666466 conn(0x7f826c0fcdd0 msgr2=0x7f826c0fe4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:30:41.894 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:41.186+0000 7f8270e00700 1 -- 172.21.15.107:0/1057666466 shutdown_connections 2024-03-04T13:30:41.894 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:41.186+0000 7f8270e00700 1 -- 172.21.15.107:0/1057666466 wait complete. 2024-03-04T13:30:41.894 INFO:teuthology.orchestra.run.smithi107.stdout:Deploying alertmanager service with default placement... 2024-03-04T13:30:42.715 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:42 smithi107 ceph-mon[28739]: from='client.14152 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:30:42.716 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:42 smithi107 ceph-mon[28739]: Saving service node-exporter spec with placement * 2024-03-04T13:30:43.437 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout Scheduled alertmanager update... 2024-03-04T13:30:43.437 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:42.460+0000 7fd569000700 1 Processor -- start 2024-03-04T13:30:43.437 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:42.460+0000 7fd569000700 1 -- start start 2024-03-04T13:30:43.437 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:42.460+0000 7fd569000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd564102780 0x7fd564102b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:30:43.437 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:42.461+0000 7fd569000700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5641030d0 con 0x7fd564102780 2024-03-04T13:30:43.437 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:42.461+0000 7fd561e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd564102780 0x7fd564102b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:30:43.437 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:42.461+0000 7fd561e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd564102780 0x7fd564102b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57274/0 (socket says 172.21.15.107:57274) 2024-03-04T13:30:43.437 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:42.461+0000 7fd561e00700 1 -- 172.21.15.107:0/1905754819 learned_addr learned my addr 172.21.15.107:0/1905754819 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:30:43.437 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:42.461+0000 7fd561e00700 1 -- 172.21.15.107:0/1905754819 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd564103210 con 0x7fd564102780 2024-03-04T13:30:43.442 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:42.462+0000 7fd561e00700 1 --2- 172.21.15.107:0/1905754819 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd564102780 0x7fd564102b90 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd5540098e0 tx=0x7fd554009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a0adc5f15e8acd3 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:30:43.442 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:42.462+0000 7fd560a00700 1 -- 172.21.15.107:0/1905754819 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd55400b860 con 0x7fd564102780 2024-03-04T13:30:43.442 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:42.462+0000 7fd560a00700 1 -- 172.21.15.107:0/1905754819 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd55400be90 con 0x7fd564102780 2024-03-04T13:30:43.442 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:42.462+0000 7fd560a00700 1 -- 172.21.15.107:0/1905754819 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd5540103c0 con 0x7fd564102780 2024-03-04T13:30:43.442 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:42.463+0000 7fd569000700 1 -- 172.21.15.107:0/1905754819 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd564102780 msgr2=0x7fd564102b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:30:43.442 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:42.463+0000 7fd569000700 1 --2- 172.21.15.107:0/1905754819 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd564102780 0x7fd564102b90 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd5540098e0 tx=0x7fd554009250 comp rx=0 tx=0).stop 2024-03-04T13:30:43.443 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:42.463+0000 7fd569000700 1 -- 172.21.15.107:0/1905754819 shutdown_connections 2024-03-04T13:30:43.443 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:42.463+0000 7fd569000700 1 --2- 172.21.15.107:0/1905754819 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd564102780 0x7fd564102b90 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:30:43.443 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:42.463+0000 7fd569000700 1 -- 172.21.15.107:0/1905754819 >> 172.21.15.107:0/1905754819 conn(0x7fd5640fdf00 msgr2=0x7fd564100310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:30:43.443 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:42.463+0000 7fd569000700 1 -- 172.21.15.107:0/1905754819 shutdown_connections 2024-03-04T13:30:43.443 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:42.463+0000 7fd569000700 1 -- 172.21.15.107:0/1905754819 wait complete. 2024-03-04T13:30:43.443 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:42.464+0000 7fd569000700 1 Processor -- start 2024-03-04T13:30:43.443 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:42.464+0000 7fd569000700 1 -- start start 2024-03-04T13:30:43.443 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:42.464+0000 7fd569000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd564102780 0x7fd564078d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:30:43.443 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:42.464+0000 7fd569000700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd564077190 con 0x7fd564102780 2024-03-04T13:30:43.444 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:42.465+0000 7fd561e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd564102780 0x7fd564078d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:30:43.444 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:42.465+0000 7fd561e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd564102780 0x7fd564078d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57282/0 (socket says 172.21.15.107:57282) 2024-03-04T13:30:43.444 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:42.465+0000 7fd561e00700 1 -- 172.21.15.107:0/1505713314 learned_addr learned my addr 172.21.15.107:0/1505713314 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:30:43.444 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:42.465+0000 7fd561e00700 1 -- 172.21.15.107:0/1505713314 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5540094f0 con 0x7fd564102780 2024-03-04T13:30:43.444 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:42.465+0000 7fd561e00700 1 --2- 172.21.15.107:0/1505713314 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd564102780 0x7fd564078d10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd5540090b0 tx=0x7fd554009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:30:43.444 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:42.466+0000 7fd55a000700 1 -- 172.21.15.107:0/1505713314 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd554010840 con 0x7fd564102780 2024-03-04T13:30:43.444 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:42.466+0000 7fd55a000700 1 -- 172.21.15.107:0/1505713314 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd554010e70 con 0x7fd564102780 2024-03-04T13:30:43.444 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:42.466+0000 7fd55a000700 1 -- 172.21.15.107:0/1505713314 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd554024e90 con 0x7fd564102780 2024-03-04T13:30:43.444 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:42.466+0000 7fd569000700 1 -- 172.21.15.107:0/1505713314 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd564077390 con 0x7fd564102780 2024-03-04T13:30:43.444 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:42.466+0000 7fd569000700 1 -- 172.21.15.107:0/1505713314 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd564077820 con 0x7fd564102780 2024-03-04T13:30:43.444 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:42.467+0000 7fd55a000700 1 -- 172.21.15.107:0/1505713314 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7fd554024430 con 0x7fd564102780 2024-03-04T13:30:43.444 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:42.467+0000 7fd569000700 1 -- 172.21.15.107:0/1505713314 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd56404bed0 con 0x7fd564102780 2024-03-04T13:30:43.444 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:42.467+0000 7fd55a000700 1 --2- 172.21.15.107:0/1505713314 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7fd548036790 0x7fd548038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:30:43.444 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:42.467+0000 7fd55a000700 1 -- 172.21.15.107:0/1505713314 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fd55404a7d0 con 0x7fd564102780 2024-03-04T13:30:43.445 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:42.468+0000 7fd561400700 1 --2- 172.21.15.107:0/1505713314 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7fd548036790 0x7fd548038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:30:43.445 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:42.468+0000 7fd561400700 1 --2- 172.21.15.107:0/1505713314 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7fd548036790 0x7fd548038c40 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd550009990 tx=0x7fd550006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:30:43.445 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:42.472+0000 7fd55a000700 1 -- 172.21.15.107:0/1505713314 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fd55402c930 con 0x7fd564102780 2024-03-04T13:30:43.445 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:42.692+0000 7fd569000700 1 -- 172.21.15.107:0/1505713314 --> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}) v1 -- 0x7fd564077fa0 con 0x7fd548036790 2024-03-04T13:30:43.445 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:42.701+0000 7fd55a000700 1 -- 172.21.15.107:0/1505713314 <== mgr.14120 v2:172.21.15.107:6800/2371648646 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+33 (secure 0 0 0) 0x7fd564077fa0 con 0x7fd548036790 2024-03-04T13:30:43.445 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:42.704+0000 7fd569000700 1 -- 172.21.15.107:0/1505713314 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7fd548036790 msgr2=0x7fd548038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:30:43.445 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:42.704+0000 7fd569000700 1 --2- 172.21.15.107:0/1505713314 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7fd548036790 0x7fd548038c40 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd550009990 tx=0x7fd550006e40 comp rx=0 tx=0).stop 2024-03-04T13:30:43.445 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:42.704+0000 7fd569000700 1 -- 172.21.15.107:0/1505713314 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd564102780 msgr2=0x7fd564078d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:30:43.445 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:42.704+0000 7fd569000700 1 --2- 172.21.15.107:0/1505713314 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd564102780 0x7fd564078d10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd5540090b0 tx=0x7fd554009250 comp rx=0 tx=0).stop 2024-03-04T13:30:43.445 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:42.704+0000 7fd569000700 1 -- 172.21.15.107:0/1505713314 shutdown_connections 2024-03-04T13:30:43.445 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:42.704+0000 7fd569000700 1 --2- 172.21.15.107:0/1505713314 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7fd548036790 0x7fd548038c40 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:30:43.445 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:42.704+0000 7fd569000700 1 --2- 172.21.15.107:0/1505713314 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd564102780 0x7fd564078d10 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:30:43.445 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:42.704+0000 7fd569000700 1 -- 172.21.15.107:0/1505713314 >> 172.21.15.107:0/1505713314 conn(0x7fd5640fdf00 msgr2=0x7fd5640ff590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:30:43.445 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:42.705+0000 7fd569000700 1 -- 172.21.15.107:0/1505713314 shutdown_connections 2024-03-04T13:30:43.446 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:42.705+0000 7fd569000700 1 -- 172.21.15.107:0/1505713314 wait complete. 2024-03-04T13:30:44.061 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:43 smithi107 ceph-mon[28739]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:30:44.061 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:43 smithi107 ceph-mon[28739]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:30:44.061 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:43 smithi107 ceph-mon[28739]: Saving service alertmanager spec with placement count:1 2024-03-04T13:30:44.061 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:43 smithi107 ceph-mon[28739]: from='mgr.14120 172.21.15.107:0/1095418985' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:30:45.053 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:44.096+0000 7fa945c00700 1 Processor -- start 2024-03-04T13:30:45.053 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:44.096+0000 7fa945c00700 1 -- start start 2024-03-04T13:30:45.054 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:44.096+0000 7fa945c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa940104960 0x7fa940104d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:30:45.054 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:44.096+0000 7fa945c00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9401052b0 con 0x7fa940104960 2024-03-04T13:30:45.054 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:44.097+0000 7fa93ea00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa940104960 0x7fa940104d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:30:45.054 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:44.097+0000 7fa93ea00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa940104960 0x7fa940104d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57292/0 (socket says 172.21.15.107:57292) 2024-03-04T13:30:45.054 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:44.097+0000 7fa93ea00700 1 -- 172.21.15.107:0/4196749425 learned_addr learned my addr 172.21.15.107:0/4196749425 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:30:45.054 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:44.097+0000 7fa93ea00700 1 -- 172.21.15.107:0/4196749425 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9401053f0 con 0x7fa940104960 2024-03-04T13:30:45.054 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:44.098+0000 7fa93ea00700 1 --2- 172.21.15.107:0/4196749425 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa940104960 0x7fa940104d70 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa9300098e0 tx=0x7fa930009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36eafd1c1f7339b9 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:30:45.054 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:44.098+0000 7fa93d600700 1 -- 172.21.15.107:0/4196749425 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa93000b860 con 0x7fa940104960 2024-03-04T13:30:45.059 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:44.098+0000 7fa93d600700 1 -- 172.21.15.107:0/4196749425 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa93000be90 con 0x7fa940104960 2024-03-04T13:30:45.059 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:44.098+0000 7fa93d600700 1 -- 172.21.15.107:0/4196749425 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa9300103c0 con 0x7fa940104960 2024-03-04T13:30:45.059 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:44.099+0000 7fa945c00700 1 -- 172.21.15.107:0/4196749425 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa940104960 msgr2=0x7fa940104d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:30:45.059 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:44.099+0000 7fa945c00700 1 --2- 172.21.15.107:0/4196749425 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa940104960 0x7fa940104d70 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa9300098e0 tx=0x7fa930009250 comp rx=0 tx=0).stop 2024-03-04T13:30:45.059 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:44.099+0000 7fa945c00700 1 -- 172.21.15.107:0/4196749425 shutdown_connections 2024-03-04T13:30:45.059 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:44.099+0000 7fa945c00700 1 --2- 172.21.15.107:0/4196749425 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa940104960 0x7fa940104d70 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:30:45.059 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:44.099+0000 7fa945c00700 1 -- 172.21.15.107:0/4196749425 >> 172.21.15.107:0/4196749425 conn(0x7fa940100340 msgr2=0x7fa940102720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:30:45.059 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:44.099+0000 7fa945c00700 1 -- 172.21.15.107:0/4196749425 shutdown_connections 2024-03-04T13:30:45.059 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:44.099+0000 7fa945c00700 1 -- 172.21.15.107:0/4196749425 wait complete. 2024-03-04T13:30:45.059 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:44.100+0000 7fa945c00700 1 Processor -- start 2024-03-04T13:30:45.060 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:44.100+0000 7fa945c00700 1 -- start start 2024-03-04T13:30:45.060 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:44.100+0000 7fa945c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa940104960 0x7fa940113a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:30:45.060 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:44.100+0000 7fa945c00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa940113fa0 con 0x7fa940104960 2024-03-04T13:30:45.060 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:44.100+0000 7fa93ea00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa940104960 0x7fa940113a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:30:45.060 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:44.100+0000 7fa93ea00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa940104960 0x7fa940113a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57298/0 (socket says 172.21.15.107:57298) 2024-03-04T13:30:45.060 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:44.100+0000 7fa93ea00700 1 -- 172.21.15.107:0/2394155654 learned_addr learned my addr 172.21.15.107:0/2394155654 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:30:45.060 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:44.101+0000 7fa93ea00700 1 -- 172.21.15.107:0/2394155654 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9300094f0 con 0x7fa940104960 2024-03-04T13:30:45.060 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:44.101+0000 7fa93ea00700 1 --2- 172.21.15.107:0/2394155654 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa940104960 0x7fa940113a60 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa930006b00 tx=0x7fa930009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:30:45.060 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:44.101+0000 7fa936a00700 1 -- 172.21.15.107:0/2394155654 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa9300109a0 con 0x7fa940104960 2024-03-04T13:30:45.060 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:44.101+0000 7fa936a00700 1 -- 172.21.15.107:0/2394155654 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa93001a430 con 0x7fa940104960 2024-03-04T13:30:45.060 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:44.101+0000 7fa936a00700 1 -- 172.21.15.107:0/2394155654 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa9300245b0 con 0x7fa940104960 2024-03-04T13:30:45.060 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:44.101+0000 7fa945c00700 1 -- 172.21.15.107:0/2394155654 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9401141a0 con 0x7fa940104960 2024-03-04T13:30:45.060 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:44.101+0000 7fa945c00700 1 -- 172.21.15.107:0/2394155654 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa940114e40 con 0x7fa940104960 2024-03-04T13:30:45.060 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:44.102+0000 7fa936a00700 1 -- 172.21.15.107:0/2394155654 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7fa93001aaa0 con 0x7fa940104960 2024-03-04T13:30:45.061 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:44.102+0000 7fa945c00700 1 -- 172.21.15.107:0/2394155654 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa94005f5c0 con 0x7fa940104960 2024-03-04T13:30:45.061 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:44.103+0000 7fa936a00700 1 --2- 172.21.15.107:0/2394155654 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7fa9240367e0 0x7fa924038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:30:45.061 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:44.103+0000 7fa936a00700 1 -- 172.21.15.107:0/2394155654 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fa93004a740 con 0x7fa940104960 2024-03-04T13:30:45.061 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:44.103+0000 7fa93e000700 1 --2- 172.21.15.107:0/2394155654 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7fa9240367e0 0x7fa924038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:30:45.061 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:44.104+0000 7fa93e000700 1 --2- 172.21.15.107:0/2394155654 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7fa9240367e0 0x7fa924038c90 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa92c009990 tx=0x7fa92c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:30:45.061 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:44.108+0000 7fa936a00700 1 -- 172.21.15.107:0/2394155654 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fa930018940 con 0x7fa940104960 2024-03-04T13:30:45.061 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:44.315+0000 7fa945c00700 1 -- 172.21.15.107:0/2394155654 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7fa940106840 con 0x7fa940104960 2024-03-04T13:30:45.061 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:44.322+0000 7fa936a00700 1 -- 172.21.15.107:0/2394155654 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7fa930018350 con 0x7fa940104960 2024-03-04T13:30:45.061 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:44.326+0000 7fa945c00700 1 -- 172.21.15.107:0/2394155654 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7fa9240367e0 msgr2=0x7fa924038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:30:45.061 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:44.326+0000 7fa945c00700 1 --2- 172.21.15.107:0/2394155654 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7fa9240367e0 0x7fa924038c90 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa92c009990 tx=0x7fa92c006e40 comp rx=0 tx=0).stop 2024-03-04T13:30:45.061 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:44.326+0000 7fa945c00700 1 -- 172.21.15.107:0/2394155654 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa940104960 msgr2=0x7fa940113a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:30:45.061 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:44.326+0000 7fa945c00700 1 --2- 172.21.15.107:0/2394155654 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa940104960 0x7fa940113a60 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa930006b00 tx=0x7fa930009250 comp rx=0 tx=0).stop 2024-03-04T13:30:45.061 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:44.327+0000 7fa945c00700 1 -- 172.21.15.107:0/2394155654 shutdown_connections 2024-03-04T13:30:45.061 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:44.327+0000 7fa945c00700 1 --2- 172.21.15.107:0/2394155654 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7fa9240367e0 0x7fa924038c90 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:30:45.062 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:44.327+0000 7fa945c00700 1 --2- 172.21.15.107:0/2394155654 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa940104960 0x7fa940113a60 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:30:45.062 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:44.327+0000 7fa945c00700 1 -- 172.21.15.107:0/2394155654 >> 172.21.15.107:0/2394155654 conn(0x7fa940100340 msgr2=0x7fa9401035e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:30:45.062 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:44.327+0000 7fa945c00700 1 -- 172.21.15.107:0/2394155654 shutdown_connections 2024-03-04T13:30:45.062 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:44.327+0000 7fa945c00700 1 -- 172.21.15.107:0/2394155654 wait complete. 2024-03-04T13:30:45.587 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:45 smithi107 ceph-mon[28739]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:30:45.587 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:45 smithi107 ceph-mon[28739]: from='client.? 172.21.15.107:0/2394155654' entity='client.admin' 2024-03-04T13:30:46.650 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:45.709+0000 7fb421e00700 1 Processor -- start 2024-03-04T13:30:46.651 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:45.709+0000 7fb421e00700 1 -- start start 2024-03-04T13:30:46.651 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:45.710+0000 7fb421e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb41c071440 0x7fb41c1077c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:30:46.651 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:45.710+0000 7fb421e00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb41c1031b0 con 0x7fb41c071440 2024-03-04T13:30:46.654 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:45.710+0000 7fb41aa00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb41c071440 0x7fb41c1077c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:30:46.655 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:45.710+0000 7fb41aa00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb41c071440 0x7fb41c1077c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57312/0 (socket says 172.21.15.107:57312) 2024-03-04T13:30:46.655 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:45.710+0000 7fb41aa00700 1 -- 172.21.15.107:0/963750841 learned_addr learned my addr 172.21.15.107:0/963750841 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:30:46.655 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:45.711+0000 7fb41aa00700 1 -- 172.21.15.107:0/963750841 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb41c107d00 con 0x7fb41c071440 2024-03-04T13:30:46.655 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:45.711+0000 7fb41aa00700 1 --2- 172.21.15.107:0/963750841 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb41c071440 0x7fb41c1077c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb40c0098e0 tx=0x7fb40c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=312895887a40c8ae server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:30:46.655 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:45.711+0000 7fb419600700 1 -- 172.21.15.107:0/963750841 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb40c00b860 con 0x7fb41c071440 2024-03-04T13:30:46.655 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:45.711+0000 7fb419600700 1 -- 172.21.15.107:0/963750841 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb40c00be90 con 0x7fb41c071440 2024-03-04T13:30:46.655 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:45.711+0000 7fb419600700 1 -- 172.21.15.107:0/963750841 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb40c0103c0 con 0x7fb41c071440 2024-03-04T13:30:46.655 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:45.712+0000 7fb421e00700 1 -- 172.21.15.107:0/963750841 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb41c071440 msgr2=0x7fb41c1077c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:30:46.655 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:45.712+0000 7fb421e00700 1 --2- 172.21.15.107:0/963750841 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb41c071440 0x7fb41c1077c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb40c0098e0 tx=0x7fb40c009250 comp rx=0 tx=0).stop 2024-03-04T13:30:46.655 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:45.712+0000 7fb421e00700 1 -- 172.21.15.107:0/963750841 shutdown_connections 2024-03-04T13:30:46.655 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:45.712+0000 7fb421e00700 1 --2- 172.21.15.107:0/963750841 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb41c071440 0x7fb41c1077c0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:30:46.655 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:45.712+0000 7fb421e00700 1 -- 172.21.15.107:0/963750841 >> 172.21.15.107:0/963750841 conn(0x7fb41c0fcdd0 msgr2=0x7fb41c0ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:30:46.656 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:45.712+0000 7fb421e00700 1 -- 172.21.15.107:0/963750841 shutdown_connections 2024-03-04T13:30:46.656 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:45.712+0000 7fb421e00700 1 -- 172.21.15.107:0/963750841 wait complete. 2024-03-04T13:30:46.656 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:45.713+0000 7fb421e00700 1 Processor -- start 2024-03-04T13:30:46.656 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:45.713+0000 7fb421e00700 1 -- start start 2024-03-04T13:30:46.656 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:45.713+0000 7fb421e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb41c071440 0x7fb41c1137e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:30:46.656 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:45.713+0000 7fb421e00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb41c113d20 con 0x7fb41c071440 2024-03-04T13:30:46.656 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:45.714+0000 7fb41aa00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb41c071440 0x7fb41c1137e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:30:46.656 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:45.714+0000 7fb41aa00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb41c071440 0x7fb41c1137e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57324/0 (socket says 172.21.15.107:57324) 2024-03-04T13:30:46.656 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:45.714+0000 7fb41aa00700 1 -- 172.21.15.107:0/1576287123 learned_addr learned my addr 172.21.15.107:0/1576287123 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:30:46.656 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:45.714+0000 7fb41aa00700 1 -- 172.21.15.107:0/1576287123 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb40c0094f0 con 0x7fb41c071440 2024-03-04T13:30:46.656 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:45.714+0000 7fb41aa00700 1 --2- 172.21.15.107:0/1576287123 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb41c071440 0x7fb41c1137e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb40c006d40 tx=0x7fb40c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:30:46.656 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:45.715+0000 7fb412a00700 1 -- 172.21.15.107:0/1576287123 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb40c010840 con 0x7fb41c071440 2024-03-04T13:30:46.656 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:45.715+0000 7fb412a00700 1 -- 172.21.15.107:0/1576287123 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb40c010e70 con 0x7fb41c071440 2024-03-04T13:30:46.657 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:45.715+0000 7fb412a00700 1 -- 172.21.15.107:0/1576287123 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb40c024ea0 con 0x7fb41c071440 2024-03-04T13:30:46.657 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:45.715+0000 7fb421e00700 1 -- 172.21.15.107:0/1576287123 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb41c113f20 con 0x7fb41c071440 2024-03-04T13:30:46.657 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:45.715+0000 7fb421e00700 1 -- 172.21.15.107:0/1576287123 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb41c114bc0 con 0x7fb41c071440 2024-03-04T13:30:46.657 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:45.716+0000 7fb412a00700 1 -- 172.21.15.107:0/1576287123 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7fb40c024780 con 0x7fb41c071440 2024-03-04T13:30:46.657 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:45.716+0000 7fb421e00700 1 -- 172.21.15.107:0/1576287123 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb41c05f5c0 con 0x7fb41c071440 2024-03-04T13:30:46.657 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:45.716+0000 7fb412a00700 1 --2- 172.21.15.107:0/1576287123 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7fb400036740 0x7fb400038bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:30:46.657 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:45.716+0000 7fb412a00700 1 -- 172.21.15.107:0/1576287123 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fb40c04a6d0 con 0x7fb41c071440 2024-03-04T13:30:46.659 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:45.717+0000 7fb41a000700 1 --2- 172.21.15.107:0/1576287123 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7fb400036740 0x7fb400038bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:30:46.659 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:45.717+0000 7fb41a000700 1 --2- 172.21.15.107:0/1576287123 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7fb400036740 0x7fb400038bf0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb408009990 tx=0x7fb408006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:30:46.659 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:45.722+0000 7fb412a00700 1 -- 172.21.15.107:0/1576287123 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fb40c02c930 con 0x7fb41c071440 2024-03-04T13:30:46.659 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:45.930+0000 7fb421e00700 1 -- 172.21.15.107:0/1576287123 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7fb41c04bed0 con 0x7fb41c071440 2024-03-04T13:30:46.659 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:45.935+0000 7fb412a00700 1 -- 172.21.15.107:0/1576287123 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7fb40c02c340 con 0x7fb41c071440 2024-03-04T13:30:46.659 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:45.938+0000 7fb421e00700 1 -- 172.21.15.107:0/1576287123 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7fb400036740 msgr2=0x7fb400038bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:30:46.659 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:45.938+0000 7fb421e00700 1 --2- 172.21.15.107:0/1576287123 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7fb400036740 0x7fb400038bf0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb408009990 tx=0x7fb408006e40 comp rx=0 tx=0).stop 2024-03-04T13:30:46.659 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:45.938+0000 7fb421e00700 1 -- 172.21.15.107:0/1576287123 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb41c071440 msgr2=0x7fb41c1137e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:30:46.659 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:45.938+0000 7fb421e00700 1 --2- 172.21.15.107:0/1576287123 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb41c071440 0x7fb41c1137e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb40c006d40 tx=0x7fb40c009250 comp rx=0 tx=0).stop 2024-03-04T13:30:46.659 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:45.939+0000 7fb421e00700 1 -- 172.21.15.107:0/1576287123 shutdown_connections 2024-03-04T13:30:46.659 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:45.939+0000 7fb421e00700 1 --2- 172.21.15.107:0/1576287123 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7fb400036740 0x7fb400038bf0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:30:46.660 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:45.939+0000 7fb421e00700 1 --2- 172.21.15.107:0/1576287123 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb41c071440 0x7fb41c1137e0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:30:46.660 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:45.939+0000 7fb421e00700 1 -- 172.21.15.107:0/1576287123 >> 172.21.15.107:0/1576287123 conn(0x7fb41c0fcdd0 msgr2=0x7fb41c0fe4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:30:46.660 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:45.939+0000 7fb421e00700 1 -- 172.21.15.107:0/1576287123 shutdown_connections 2024-03-04T13:30:46.660 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:45.939+0000 7fb421e00700 1 -- 172.21.15.107:0/1576287123 wait complete. 2024-03-04T13:30:46.660 INFO:teuthology.orchestra.run.smithi107.stdout:Enabling the dashboard module... 2024-03-04T13:30:47.198 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:46 smithi107 ceph-mon[28739]: from='client.? 172.21.15.107:0/1576287123' entity='client.admin' 2024-03-04T13:30:48.337 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:47 smithi107 ceph-mon[28739]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:30:48.337 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:47 smithi107 ceph-mon[28739]: from='client.? 172.21.15.107:0/897736170' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-03-04T13:30:48.705 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:47.270+0000 7f9ac9c00700 1 Processor -- start 2024-03-04T13:30:48.706 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:47.270+0000 7f9ac9c00700 1 -- start start 2024-03-04T13:30:48.706 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:47.270+0000 7f9ac9c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ac407c7a0 0x7f9ac407cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:30:48.706 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:47.270+0000 7f9ac9c00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac407d0f0 con 0x7f9ac407c7a0 2024-03-04T13:30:48.706 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:47.271+0000 7f9ac2a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ac407c7a0 0x7f9ac407cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:30:48.706 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:47.271+0000 7f9ac2a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ac407c7a0 0x7f9ac407cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57338/0 (socket says 172.21.15.107:57338) 2024-03-04T13:30:48.707 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:47.271+0000 7f9ac2a00700 1 -- 172.21.15.107:0/3595091413 learned_addr learned my addr 172.21.15.107:0/3595091413 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:30:48.707 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:47.271+0000 7f9ac2a00700 1 -- 172.21.15.107:0/3595091413 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ac407d230 con 0x7f9ac407c7a0 2024-03-04T13:30:48.707 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:47.272+0000 7f9ac2a00700 1 --2- 172.21.15.107:0/3595091413 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ac407c7a0 0x7f9ac407cbb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9ab40098e0 tx=0x7f9ab4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c745c321cd07e913 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:30:48.707 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:47.272+0000 7f9ac1600700 1 -- 172.21.15.107:0/3595091413 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9ab400b860 con 0x7f9ac407c7a0 2024-03-04T13:30:48.707 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:47.272+0000 7f9ac1600700 1 -- 172.21.15.107:0/3595091413 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9ab400be90 con 0x7f9ac407c7a0 2024-03-04T13:30:48.707 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:47.272+0000 7f9ac1600700 1 -- 172.21.15.107:0/3595091413 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9ab40103c0 con 0x7f9ac407c7a0 2024-03-04T13:30:48.707 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:47.272+0000 7f9ac9c00700 1 -- 172.21.15.107:0/3595091413 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ac407c7a0 msgr2=0x7f9ac407cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:30:48.707 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:47.272+0000 7f9ac9c00700 1 --2- 172.21.15.107:0/3595091413 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ac407c7a0 0x7f9ac407cbb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9ab40098e0 tx=0x7f9ab4009250 comp rx=0 tx=0).stop 2024-03-04T13:30:48.707 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:47.272+0000 7f9ac9c00700 1 -- 172.21.15.107:0/3595091413 shutdown_connections 2024-03-04T13:30:48.707 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:47.273+0000 7f9ac9c00700 1 --2- 172.21.15.107:0/3595091413 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ac407c7a0 0x7f9ac407cbb0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:30:48.708 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:47.273+0000 7f9ac9c00700 1 -- 172.21.15.107:0/3595091413 >> 172.21.15.107:0/3595091413 conn(0x7f9ac4078030 msgr2=0x7f9ac407a460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:30:48.708 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:47.273+0000 7f9ac9c00700 1 -- 172.21.15.107:0/3595091413 shutdown_connections 2024-03-04T13:30:48.708 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:47.273+0000 7f9ac9c00700 1 -- 172.21.15.107:0/3595091413 wait complete. 2024-03-04T13:30:48.708 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:47.273+0000 7f9ac9c00700 1 Processor -- start 2024-03-04T13:30:48.708 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:47.274+0000 7f9ac9c00700 1 -- start start 2024-03-04T13:30:48.708 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:47.274+0000 7f9ac9c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ac407c7a0 0x7f9ac4117bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:30:48.708 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:47.274+0000 7f9ac9c00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac4118110 con 0x7f9ac407c7a0 2024-03-04T13:30:48.708 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:47.274+0000 7f9ac2a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ac407c7a0 0x7f9ac4117bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:30:48.708 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:47.274+0000 7f9ac2a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ac407c7a0 0x7f9ac4117bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57344/0 (socket says 172.21.15.107:57344) 2024-03-04T13:30:48.708 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:47.274+0000 7f9ac2a00700 1 -- 172.21.15.107:0/897736170 learned_addr learned my addr 172.21.15.107:0/897736170 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:30:48.708 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:47.275+0000 7f9ac2a00700 1 -- 172.21.15.107:0/897736170 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ab40094f0 con 0x7f9ac407c7a0 2024-03-04T13:30:48.708 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:47.275+0000 7f9ac2a00700 1 --2- 172.21.15.107:0/897736170 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ac407c7a0 0x7f9ac4117bd0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9ab4006d40 tx=0x7f9ab4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:30:48.708 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:47.275+0000 7f9abaa00700 1 -- 172.21.15.107:0/897736170 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9ab4010840 con 0x7f9ac407c7a0 2024-03-04T13:30:48.708 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:47.275+0000 7f9abaa00700 1 -- 172.21.15.107:0/897736170 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9ab4010e70 con 0x7f9ac407c7a0 2024-03-04T13:30:48.709 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:47.275+0000 7f9abaa00700 1 -- 172.21.15.107:0/897736170 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9ab4024ea0 con 0x7f9ac407c7a0 2024-03-04T13:30:48.709 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:47.275+0000 7f9ac9c00700 1 -- 172.21.15.107:0/897736170 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ac4118310 con 0x7f9ac407c7a0 2024-03-04T13:30:48.709 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:47.275+0000 7f9ac9c00700 1 -- 172.21.15.107:0/897736170 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ac4118fb0 con 0x7f9ac407c7a0 2024-03-04T13:30:48.709 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:47.276+0000 7f9abaa00700 1 -- 172.21.15.107:0/897736170 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f9ab401a4c0 con 0x7f9ac407c7a0 2024-03-04T13:30:48.709 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:47.276+0000 7f9ac9c00700 1 -- 172.21.15.107:0/897736170 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ac405f5c0 con 0x7f9ac407c7a0 2024-03-04T13:30:48.709 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:47.277+0000 7f9abaa00700 1 --2- 172.21.15.107:0/897736170 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7f9aa8036790 0x7f9aa8038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:30:48.710 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:47.277+0000 7f9abaa00700 1 -- 172.21.15.107:0/897736170 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f9ab404a8b0 con 0x7f9ac407c7a0 2024-03-04T13:30:48.710 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:47.277+0000 7f9ac2000700 1 --2- 172.21.15.107:0/897736170 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7f9aa8036790 0x7f9aa8038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:30:48.710 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:47.278+0000 7f9ac2000700 1 --2- 172.21.15.107:0/897736170 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7f9aa8036790 0x7f9aa8038c40 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f9ab0009990 tx=0x7f9ab0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:30:48.710 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:47.282+0000 7f9abaa00700 1 -- 172.21.15.107:0/897736170 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f9ab4023b80 con 0x7f9ac407c7a0 2024-03-04T13:30:48.710 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:47.543+0000 7f9ac9c00700 1 -- 172.21.15.107:0/897736170 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f9ac404bed0 con 0x7f9ac407c7a0 2024-03-04T13:30:48.710 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:47.946+0000 7f9abaa00700 1 -- 172.21.15.107:0/897736170 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mgrmap(e 9) v1 ==== 43381+0+0 (secure 0 0 0) 0x7f9ab40109a0 con 0x7f9ac407c7a0 2024-03-04T13:30:48.710 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:47.949+0000 7f9abaa00700 1 -- 172.21.15.107:0/897736170 <== mon.0 v2:172.21.15.107:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f9ab40193c0 con 0x7f9ac407c7a0 2024-03-04T13:30:48.710 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:47.953+0000 7f9ac9c00700 1 -- 172.21.15.107:0/897736170 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7f9aa8036790 msgr2=0x7f9aa8038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:30:48.710 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:47.953+0000 7f9ac9c00700 1 --2- 172.21.15.107:0/897736170 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7f9aa8036790 0x7f9aa8038c40 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f9ab0009990 tx=0x7f9ab0006e40 comp rx=0 tx=0).stop 2024-03-04T13:30:48.710 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:47.953+0000 7f9ac9c00700 1 -- 172.21.15.107:0/897736170 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ac407c7a0 msgr2=0x7f9ac4117bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:30:48.711 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:47.953+0000 7f9ac9c00700 1 --2- 172.21.15.107:0/897736170 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ac407c7a0 0x7f9ac4117bd0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9ab4006d40 tx=0x7f9ab4009250 comp rx=0 tx=0).stop 2024-03-04T13:30:48.711 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:47.953+0000 7f9ac9c00700 1 -- 172.21.15.107:0/897736170 shutdown_connections 2024-03-04T13:30:48.711 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:47.953+0000 7f9ac9c00700 1 --2- 172.21.15.107:0/897736170 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7f9aa8036790 0x7f9aa8038c40 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:30:48.711 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:47.953+0000 7f9ac9c00700 1 --2- 172.21.15.107:0/897736170 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ac407c7a0 0x7f9ac4117bd0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:30:48.711 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:47.953+0000 7f9ac9c00700 1 -- 172.21.15.107:0/897736170 >> 172.21.15.107:0/897736170 conn(0x7f9ac4078030 msgr2=0x7f9ac407a750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:30:48.711 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:47.953+0000 7f9ac9c00700 1 -- 172.21.15.107:0/897736170 shutdown_connections 2024-03-04T13:30:48.711 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:47.953+0000 7f9ac9c00700 1 -- 172.21.15.107:0/897736170 wait complete. 2024-03-04T13:30:49.212 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:48 smithi107 ceph-mon[28739]: from='client.? 172.21.15.107:0/897736170' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-03-04T13:30:49.212 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:48 smithi107 ceph-mon[28739]: mgrmap e9: smithi107.wpzgwl(active, since 31s) 2024-03-04T13:30:50.055 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:49 smithi107 ceph-mon[28739]: from='client.? 172.21.15.107:0/2713355974' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-04T13:30:50.311 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout { 2024-03-04T13:30:50.317 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-03-04T13:30:50.317 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-04T13:30:50.317 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "active_name": "smithi107.wpzgwl", 2024-03-04T13:30:50.318 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-04T13:30:50.318 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout } 2024-03-04T13:30:50.318 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:49.274+0000 7f88bf200700 1 Processor -- start 2024-03-04T13:30:50.318 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:49.275+0000 7f88bf200700 1 -- start start 2024-03-04T13:30:50.318 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:49.275+0000 7f88bf200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88b8101670 0x7f88b8101a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:30:50.318 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:49.275+0000 7f88bf200700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88b8101fc0 con 0x7f88b8101670 2024-03-04T13:30:50.318 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:49.275+0000 7f88b7e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88b8101670 0x7f88b8101a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:30:50.318 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:49.275+0000 7f88b7e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88b8101670 0x7f88b8101a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:58488/0 (socket says 172.21.15.107:58488) 2024-03-04T13:30:50.318 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:49.276+0000 7f88b7e00700 1 -- 172.21.15.107:0/2569804884 learned_addr learned my addr 172.21.15.107:0/2569804884 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:30:50.318 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:49.276+0000 7f88b7e00700 1 -- 172.21.15.107:0/2569804884 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88b8102100 con 0x7f88b8101670 2024-03-04T13:30:50.319 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:49.276+0000 7f88b7e00700 1 --2- 172.21.15.107:0/2569804884 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88b8101670 0x7f88b8101a80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f88a80098e0 tx=0x7f88a8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d260f8606f2495ae server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:30:50.319 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:49.276+0000 7f88b6a00700 1 -- 172.21.15.107:0/2569804884 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f88a800b860 con 0x7f88b8101670 2024-03-04T13:30:50.319 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:49.277+0000 7f88b6a00700 1 -- 172.21.15.107:0/2569804884 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f88a800be90 con 0x7f88b8101670 2024-03-04T13:30:50.319 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:49.277+0000 7f88b6a00700 1 -- 172.21.15.107:0/2569804884 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f88a80103c0 con 0x7f88b8101670 2024-03-04T13:30:50.319 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:49.277+0000 7f88bf200700 1 -- 172.21.15.107:0/2569804884 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88b8101670 msgr2=0x7f88b8101a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:30:50.319 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:49.277+0000 7f88bf200700 1 --2- 172.21.15.107:0/2569804884 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88b8101670 0x7f88b8101a80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f88a80098e0 tx=0x7f88a8009250 comp rx=0 tx=0).stop 2024-03-04T13:30:50.319 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:49.277+0000 7f88bf200700 1 -- 172.21.15.107:0/2569804884 shutdown_connections 2024-03-04T13:30:50.319 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:49.277+0000 7f88bf200700 1 --2- 172.21.15.107:0/2569804884 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88b8101670 0x7f88b8101a80 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:30:50.319 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:49.277+0000 7f88bf200700 1 -- 172.21.15.107:0/2569804884 >> 172.21.15.107:0/2569804884 conn(0x7f88b80fcdd0 msgr2=0x7f88b80ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:30:50.319 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:49.278+0000 7f88bf200700 1 -- 172.21.15.107:0/2569804884 shutdown_connections 2024-03-04T13:30:50.319 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:49.278+0000 7f88bf200700 1 -- 172.21.15.107:0/2569804884 wait complete. 2024-03-04T13:30:50.319 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:49.279+0000 7f88bf200700 1 Processor -- start 2024-03-04T13:30:50.319 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:49.279+0000 7f88bf200700 1 -- start start 2024-03-04T13:30:50.319 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:49.279+0000 7f88bf200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88b8101670 0x7f88b81166d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:30:50.320 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:49.279+0000 7f88bf200700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88b81120c0 con 0x7f88b8101670 2024-03-04T13:30:50.320 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:49.280+0000 7f88b7e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88b8101670 0x7f88b81166d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:30:50.320 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:49.280+0000 7f88b7e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88b8101670 0x7f88b81166d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:58502/0 (socket says 172.21.15.107:58502) 2024-03-04T13:30:50.320 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:49.280+0000 7f88b7e00700 1 -- 172.21.15.107:0/2713355974 learned_addr learned my addr 172.21.15.107:0/2713355974 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:30:50.320 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:49.280+0000 7f88b7e00700 1 -- 172.21.15.107:0/2713355974 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88a80094f0 con 0x7f88b8101670 2024-03-04T13:30:50.320 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:49.280+0000 7f88b7e00700 1 --2- 172.21.15.107:0/2713355974 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88b8101670 0x7f88b81166d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f88a80090b0 tx=0x7f88a8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:30:50.320 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:49.280+0000 7f88afe00700 1 -- 172.21.15.107:0/2713355974 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f88a8010840 con 0x7f88b8101670 2024-03-04T13:30:50.320 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:49.281+0000 7f88afe00700 1 -- 172.21.15.107:0/2713355974 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f88a8010e70 con 0x7f88b8101670 2024-03-04T13:30:50.320 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:49.281+0000 7f88bf200700 1 -- 172.21.15.107:0/2713355974 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88b8071450 con 0x7f88b8101670 2024-03-04T13:30:50.320 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:49.281+0000 7f88afe00700 1 -- 172.21.15.107:0/2713355974 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f88a8024e40 con 0x7f88b8101670 2024-03-04T13:30:50.320 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:49.281+0000 7f88bf200700 1 -- 172.21.15.107:0/2713355974 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88b8116c10 con 0x7f88b8101670 2024-03-04T13:30:50.320 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:49.282+0000 7f88afe00700 1 -- 172.21.15.107:0/2713355974 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 9) v1 ==== 43381+0+0 (secure 0 0 0) 0x7f88a80109a0 con 0x7f88b8101670 2024-03-04T13:30:50.320 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:49.282+0000 7f88bf200700 1 -- 172.21.15.107:0/2713355974 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88b804bed0 con 0x7f88b8101670 2024-03-04T13:30:50.320 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:49.283+0000 7f88afe00700 1 --2- 172.21.15.107:0/2713355974 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7f889c0367e0 0x7f889c038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:30:50.321 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:49.283+0000 7f88afe00700 1 -- 172.21.15.107:0/2713355974 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f88a804a840 con 0x7f88b8101670 2024-03-04T13:30:50.321 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:49.283+0000 7f88b7400700 1 -- 172.21.15.107:0/2713355974 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7f889c0367e0 msgr2=0x7f889c038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/2371648646 2024-03-04T13:30:50.321 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:49.283+0000 7f88b7400700 1 --2- 172.21.15.107:0/2713355974 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7f889c0367e0 0x7f889c038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T13:30:50.321 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:49.287+0000 7f88afe00700 1 -- 172.21.15.107:0/2713355974 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f88a802c520 con 0x7f88b8101670 2024-03-04T13:30:50.321 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:49.484+0000 7f88b7400700 1 -- 172.21.15.107:0/2713355974 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7f889c0367e0 msgr2=0x7f889c038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/2371648646 2024-03-04T13:30:50.321 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:49.484+0000 7f88b7400700 1 --2- 172.21.15.107:0/2713355974 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7f889c0367e0 0x7f889c038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T13:30:50.321 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:49.552+0000 7f88bf200700 1 -- 172.21.15.107:0/2713355974 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f88b8117120 con 0x7f88b8101670 2024-03-04T13:30:50.321 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:49.553+0000 7f88afe00700 1 -- 172.21.15.107:0/2713355974 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+103 (secure 0 0 0) 0x7f88a802c340 con 0x7f88b8101670 2024-03-04T13:30:50.321 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:49.555+0000 7f88bf200700 1 -- 172.21.15.107:0/2713355974 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7f889c0367e0 msgr2=0x7f889c038c90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-04T13:30:50.321 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:49.555+0000 7f88bf200700 1 --2- 172.21.15.107:0/2713355974 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7f889c0367e0 0x7f889c038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:30:50.321 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:49.555+0000 7f88bf200700 1 -- 172.21.15.107:0/2713355974 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88b8101670 msgr2=0x7f88b81166d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:30:50.321 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:49.555+0000 7f88bf200700 1 --2- 172.21.15.107:0/2713355974 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88b8101670 0x7f88b81166d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f88a80090b0 tx=0x7f88a8009250 comp rx=0 tx=0).stop 2024-03-04T13:30:50.321 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:49.555+0000 7f88bf200700 1 -- 172.21.15.107:0/2713355974 shutdown_connections 2024-03-04T13:30:50.322 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:49.555+0000 7f88bf200700 1 --2- 172.21.15.107:0/2713355974 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7f889c0367e0 0x7f889c038c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:30:50.322 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:49.555+0000 7f88bf200700 1 --2- 172.21.15.107:0/2713355974 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88b8101670 0x7f88b81166d0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:30:50.322 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:49.555+0000 7f88bf200700 1 -- 172.21.15.107:0/2713355974 >> 172.21.15.107:0/2713355974 conn(0x7f88b80fcdd0 msgr2=0x7f88b80fe490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:30:50.322 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:49.555+0000 7f88bf200700 1 -- 172.21.15.107:0/2713355974 shutdown_connections 2024-03-04T13:30:50.322 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:49.555+0000 7f88bf200700 1 -- 172.21.15.107:0/2713355974 wait complete. 2024-03-04T13:30:50.322 INFO:teuthology.orchestra.run.smithi107.stdout:Waiting for the mgr to restart... 2024-03-04T13:30:50.322 INFO:teuthology.orchestra.run.smithi107.stdout:Waiting for mgr epoch 9... 2024-03-04T13:30:56.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:56 smithi107 ceph-mon[28739]: Active manager daemon smithi107.wpzgwl restarted 2024-03-04T13:30:56.838 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:56 smithi107 ceph-mon[28739]: Activating manager daemon smithi107.wpzgwl 2024-03-04T13:30:56.838 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:56 smithi107 ceph-mon[28739]: osdmap e3: 0 total, 0 up, 0 in 2024-03-04T13:30:56.838 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:56 smithi107 ceph-mon[28739]: mgrmap e10: smithi107.wpzgwl(active, starting, since 0.0100836s) 2024-03-04T13:30:56.838 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:56 smithi107 ceph-mon[28739]: from='mgr.14162 172.21.15.107:0/1450595538' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mon metadata", "id": "smithi107"}]: dispatch 2024-03-04T13:30:56.838 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:56 smithi107 ceph-mon[28739]: from='mgr.14162 172.21.15.107:0/1450595538' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mgr metadata", "who": "smithi107.wpzgwl", "id": "smithi107.wpzgwl"}]: dispatch 2024-03-04T13:30:56.838 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:56 smithi107 ceph-mon[28739]: from='mgr.14162 172.21.15.107:0/1450595538' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-04T13:30:56.838 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:56 smithi107 ceph-mon[28739]: from='mgr.14162 172.21.15.107:0/1450595538' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-04T13:30:56.838 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:56 smithi107 ceph-mon[28739]: from='mgr.14162 172.21.15.107:0/1450595538' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-04T13:30:56.838 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:56 smithi107 ceph-mon[28739]: Manager daemon smithi107.wpzgwl is now available 2024-03-04T13:30:57.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:57 smithi107 ceph-mon[28739]: from='mgr.14162 172.21.15.107:0/1450595538' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:30:57.838 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:57 smithi107 ceph-mon[28739]: from='mgr.14162 172.21.15.107:0/1450595538' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:30:57.838 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:57 smithi107 ceph-mon[28739]: from='mgr.14162 172.21.15.107:0/1450595538' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi107.wpzgwl/mirror_snapshot_schedule"}]: dispatch 2024-03-04T13:30:57.838 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:57 smithi107 ceph-mon[28739]: from='mgr.14162 172.21.15.107:0/1450595538' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi107.wpzgwl/trash_purge_schedule"}]: dispatch 2024-03-04T13:30:57.838 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:57 smithi107 ceph-mon[28739]: from='mgr.14162 172.21.15.107:0/1450595538' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:30:57.838 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:57 smithi107 ceph-mon[28739]: mgrmap e11: smithi107.wpzgwl(active, since 1.01595s) 2024-03-04T13:30:58.596 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout { 2024-03-04T13:30:58.608 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-03-04T13:30:58.608 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-04T13:30:58.608 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout } 2024-03-04T13:30:58.608 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:50.884+0000 7f3626c00700 1 Processor -- start 2024-03-04T13:30:58.608 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:50.884+0000 7f3626c00700 1 -- start start 2024-03-04T13:30:58.609 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:50.884+0000 7f3626c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3620101670 0x7f3620101a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:30:58.609 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:50.884+0000 7f3626c00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3620101fc0 con 0x7f3620101670 2024-03-04T13:30:58.609 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:50.885+0000 7f361fe00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3620101670 0x7f3620101a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:30:58.609 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:50.885+0000 7f361fe00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3620101670 0x7f3620101a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:58506/0 (socket says 172.21.15.107:58506) 2024-03-04T13:30:58.609 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:50.885+0000 7f361fe00700 1 -- 172.21.15.107:0/199547762 learned_addr learned my addr 172.21.15.107:0/199547762 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:30:58.609 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:50.886+0000 7f361fe00700 1 -- 172.21.15.107:0/199547762 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3620102100 con 0x7f3620101670 2024-03-04T13:30:58.609 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:50.886+0000 7f361fe00700 1 --2- 172.21.15.107:0/199547762 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3620101670 0x7f3620101a80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f361000bb80 tx=0x7f361000b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4eea959bda5f6aa4 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:30:58.609 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:50.886+0000 7f361ea00700 1 -- 172.21.15.107:0/199547762 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f361000dd00 con 0x7f3620101670 2024-03-04T13:30:58.609 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:50.886+0000 7f361ea00700 1 -- 172.21.15.107:0/199547762 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f361000de60 con 0x7f3620101670 2024-03-04T13:30:58.609 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:50.886+0000 7f361ea00700 1 -- 172.21.15.107:0/199547762 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3610018920 con 0x7f3620101670 2024-03-04T13:30:58.609 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:50.887+0000 7f3626c00700 1 -- 172.21.15.107:0/199547762 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3620101670 msgr2=0x7f3620101a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:30:58.610 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:50.887+0000 7f3626c00700 1 --2- 172.21.15.107:0/199547762 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3620101670 0x7f3620101a80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f361000bb80 tx=0x7f361000b4f0 comp rx=0 tx=0).stop 2024-03-04T13:30:58.611 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:50.887+0000 7f3626c00700 1 -- 172.21.15.107:0/199547762 shutdown_connections 2024-03-04T13:30:58.611 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:50.887+0000 7f3626c00700 1 --2- 172.21.15.107:0/199547762 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3620101670 0x7f3620101a80 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:30:58.611 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:50.887+0000 7f3626c00700 1 -- 172.21.15.107:0/199547762 >> 172.21.15.107:0/199547762 conn(0x7f36200fcdd0 msgr2=0x7f36200ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:30:58.611 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:50.887+0000 7f3626c00700 1 -- 172.21.15.107:0/199547762 shutdown_connections 2024-03-04T13:30:58.611 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:50.887+0000 7f3626c00700 1 -- 172.21.15.107:0/199547762 wait complete. 2024-03-04T13:30:58.611 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:50.888+0000 7f3626c00700 1 Processor -- start 2024-03-04T13:30:58.611 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:50.888+0000 7f3626c00700 1 -- start start 2024-03-04T13:30:58.611 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:50.889+0000 7f3626c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3620101670 0x7f362010f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:30:58.612 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:50.889+0000 7f3626c00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f362010f880 con 0x7f3620101670 2024-03-04T13:30:58.612 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:50.889+0000 7f361fe00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3620101670 0x7f362010f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:30:58.612 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:50.889+0000 7f361fe00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3620101670 0x7f362010f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:58522/0 (socket says 172.21.15.107:58522) 2024-03-04T13:30:58.612 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:50.889+0000 7f361fe00700 1 -- 172.21.15.107:0/4223127439 learned_addr learned my addr 172.21.15.107:0/4223127439 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:30:58.612 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:50.889+0000 7f361fe00700 1 -- 172.21.15.107:0/4223127439 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f361000b790 con 0x7f3620101670 2024-03-04T13:30:58.612 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:50.890+0000 7f361fe00700 1 --2- 172.21.15.107:0/4223127439 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3620101670 0x7f362010f340 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f36100106a0 tx=0x7f361000b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:30:58.612 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:50.890+0000 7f3617e00700 1 -- 172.21.15.107:0/4223127439 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3610010bf0 con 0x7f3620101670 2024-03-04T13:30:58.612 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:50.890+0000 7f3617e00700 1 -- 172.21.15.107:0/4223127439 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f361001d050 con 0x7f3620101670 2024-03-04T13:30:58.612 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:50.890+0000 7f3617e00700 1 -- 172.21.15.107:0/4223127439 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3610022620 con 0x7f3620101670 2024-03-04T13:30:58.612 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:50.890+0000 7f3626c00700 1 -- 172.21.15.107:0/4223127439 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f362010fa80 con 0x7f3620101670 2024-03-04T13:30:58.613 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:50.890+0000 7f3626c00700 1 -- 172.21.15.107:0/4223127439 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3620110720 con 0x7f3620101670 2024-03-04T13:30:58.613 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:50.891+0000 7f3617e00700 1 -- 172.21.15.107:0/4223127439 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 9) v1 ==== 43381+0+0 (secure 0 0 0) 0x7f3610023470 con 0x7f3620101670 2024-03-04T13:30:58.613 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:50.892+0000 7f3617e00700 1 --2- 172.21.15.107:0/4223127439 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7f3604036830 0x7f3604038ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:30:58.613 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:50.892+0000 7f3617e00700 1 -- 172.21.15.107:0/4223127439 --> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f36040393f0 con 0x7f3604036830 2024-03-04T13:30:58.613 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:50.892+0000 7f3617e00700 1 -- 172.21.15.107:0/4223127439 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f3610014070 con 0x7f3620101670 2024-03-04T13:30:58.613 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:50.892+0000 7f361f400700 1 -- 172.21.15.107:0/4223127439 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7f3604036830 msgr2=0x7f3604038ce0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/2371648646 2024-03-04T13:30:58.613 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:50.892+0000 7f361f400700 1 --2- 172.21.15.107:0/4223127439 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7f3604036830 0x7f3604038ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T13:30:58.613 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:51.093+0000 7f361f400700 1 -- 172.21.15.107:0/4223127439 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7f3604036830 msgr2=0x7f3604038ce0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/2371648646 2024-03-04T13:30:58.613 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:51.093+0000 7f361f400700 1 --2- 172.21.15.107:0/4223127439 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7f3604036830 0x7f3604038ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T13:30:58.613 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:51.494+0000 7f361f400700 1 -- 172.21.15.107:0/4223127439 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7f3604036830 msgr2=0x7f3604038ce0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/2371648646 2024-03-04T13:30:58.613 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:51.494+0000 7f361f400700 1 --2- 172.21.15.107:0/4223127439 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7f3604036830 0x7f3604038ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T13:30:58.613 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:52.295+0000 7f361f400700 1 -- 172.21.15.107:0/4223127439 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7f3604036830 msgr2=0x7f3604038ce0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/2371648646 2024-03-04T13:30:58.613 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:52.295+0000 7f361f400700 1 --2- 172.21.15.107:0/4223127439 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7f3604036830 0x7f3604038ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T13:30:58.613 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:53.897+0000 7f361f400700 1 -- 172.21.15.107:0/4223127439 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7f3604036830 msgr2=0x7f3604038ce0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/2371648646 2024-03-04T13:30:58.614 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:53.897+0000 7f361f400700 1 --2- 172.21.15.107:0/4223127439 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7f3604036830 0x7f3604038ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-04T13:30:58.614 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:56.360+0000 7f3617e00700 1 -- 172.21.15.107:0/4223127439 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mgrmap(e 10) v1 ==== 43148+0+0 (secure 0 0 0) 0x7f3610023ad0 con 0x7f3620101670 2024-03-04T13:30:58.614 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:56.360+0000 7f3617e00700 1 -- 172.21.15.107:0/4223127439 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7f3604036830 msgr2=0x7f3604038ce0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-04T13:30:58.614 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:56.360+0000 7f3617e00700 1 --2- 172.21.15.107:0/4223127439 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7f3604036830 0x7f3604038ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:30:58.614 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:57.363+0000 7f3617e00700 1 -- 172.21.15.107:0/4223127439 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mgrmap(e 11) v1 ==== 43275+0+0 (secure 0 0 0) 0x7f3610022b00 con 0x7f3620101670 2024-03-04T13:30:58.614 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:57.364+0000 7f3617e00700 1 --2- 172.21.15.107:0/4223127439 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7f360403a070 0x7f360403c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:30:58.614 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:57.364+0000 7f3617e00700 1 -- 172.21.15.107:0/4223127439 --> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f36040393f0 con 0x7f360403a070 2024-03-04T13:30:58.614 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:57.365+0000 7f361f400700 1 --2- 172.21.15.107:0/4223127439 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7f360403a070 0x7f360403c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:30:58.614 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:57.367+0000 7f361f400700 1 --2- 172.21.15.107:0/4223127439 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7f360403a070 0x7f360403c450 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f360c0039d0 tx=0x7f360c0092f0 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:30:58.614 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:57.369+0000 7f3617e00700 1 -- 172.21.15.107:0/4223127439 <== mgr.14162 v2:172.21.15.107:6800/2040330534 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6549 (secure 0 0 0) 0x7f36040393f0 con 0x7f360403a070 2024-03-04T13:30:58.614 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:57.377+0000 7f3626c00700 1 -- 172.21.15.107:0/4223127439 --> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f362010ff10 con 0x7f360403a070 2024-03-04T13:30:58.615 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:57.377+0000 7f3617e00700 1 -- 172.21.15.107:0/4223127439 <== mgr.14162 v2:172.21.15.107:6800/2040330534 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7f362010ff10 con 0x7f360403a070 2024-03-04T13:30:58.615 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:57.378+0000 7f3626c00700 1 -- 172.21.15.107:0/4223127439 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7f360403a070 msgr2=0x7f360403c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:30:58.615 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:57.378+0000 7f3626c00700 1 --2- 172.21.15.107:0/4223127439 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7f360403a070 0x7f360403c450 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f360c0039d0 tx=0x7f360c0092f0 comp rx=0 tx=0).stop 2024-03-04T13:30:58.615 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:57.378+0000 7f3626c00700 1 -- 172.21.15.107:0/4223127439 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3620101670 msgr2=0x7f362010f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:30:58.615 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:57.378+0000 7f3626c00700 1 --2- 172.21.15.107:0/4223127439 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3620101670 0x7f362010f340 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f36100106a0 tx=0x7f361000b4f0 comp rx=0 tx=0).stop 2024-03-04T13:30:58.615 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:57.378+0000 7f3626c00700 1 -- 172.21.15.107:0/4223127439 shutdown_connections 2024-03-04T13:30:58.615 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:57.378+0000 7f3626c00700 1 --2- 172.21.15.107:0/4223127439 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7f360403a070 0x7f360403c450 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:30:58.615 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:57.378+0000 7f3626c00700 1 --2- 172.21.15.107:0/4223127439 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3620101670 0x7f362010f340 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:30:58.616 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:57.378+0000 7f3626c00700 1 --2- 172.21.15.107:0/4223127439 >> [v2:172.21.15.107:6800/2371648646,v1:172.21.15.107:6801/2371648646] conn(0x7f3604036830 0x7f3604038ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:30:58.616 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:57.378+0000 7f3626c00700 1 -- 172.21.15.107:0/4223127439 >> 172.21.15.107:0/4223127439 conn(0x7f36200fcdd0 msgr2=0x7f36200ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:30:58.616 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:57.379+0000 7f3626c00700 1 -- 172.21.15.107:0/4223127439 shutdown_connections 2024-03-04T13:30:58.616 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:57.379+0000 7f3626c00700 1 -- 172.21.15.107:0/4223127439 wait complete. 2024-03-04T13:30:58.616 INFO:teuthology.orchestra.run.smithi107.stdout:mgr epoch 9 is available 2024-03-04T13:30:58.616 INFO:teuthology.orchestra.run.smithi107.stdout:Generating a dashboard self-signed certificate... 2024-03-04T13:30:58.734 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:58 smithi107 ceph-mon[28739]: [04/Mar/2024:13:30:57] ENGINE Bus STARTING 2024-03-04T13:30:58.734 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:58 smithi107 ceph-mon[28739]: [04/Mar/2024:13:30:57] ENGINE Serving on https://172.21.15.107:7150 2024-03-04T13:30:58.734 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:58 smithi107 ceph-mon[28739]: [04/Mar/2024:13:30:57] ENGINE Bus STARTED 2024-03-04T13:30:58.735 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:58 smithi107 ceph-mon[28739]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-04T13:30:58.735 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:58 smithi107 ceph-mon[28739]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-04T13:30:59.839 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:59 smithi107 ceph-mon[28739]: mgrmap e12: smithi107.wpzgwl(active, since 2s) 2024-03-04T13:30:59.840 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:30:59 smithi107 ceph-mon[28739]: from='mgr.14162 172.21.15.107:0/1450595538' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:31:00.250 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-03-04T13:31:00.250 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:59.170+0000 7fbc68200700 1 Processor -- start 2024-03-04T13:31:00.250 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:59.170+0000 7fbc68200700 1 -- start start 2024-03-04T13:31:00.250 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:59.170+0000 7fbc68200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc60077b90 0x7fbc600760a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:31:00.250 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:59.170+0000 7fbc68200700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc60077fa0 con 0x7fbc60077b90 2024-03-04T13:31:00.250 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:59.171+0000 7fbc65800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc60077b90 0x7fbc600760a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:31:00.251 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:59.171+0000 7fbc65800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc60077b90 0x7fbc600760a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:35898/0 (socket says 172.21.15.107:35898) 2024-03-04T13:31:00.251 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:59.171+0000 7fbc65800700 1 -- 172.21.15.107:0/2093902330 learned_addr learned my addr 172.21.15.107:0/2093902330 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:31:00.251 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:59.171+0000 7fbc65800700 1 -- 172.21.15.107:0/2093902330 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc600765e0 con 0x7fbc60077b90 2024-03-04T13:31:00.251 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:59.172+0000 7fbc65800700 1 --2- 172.21.15.107:0/2093902330 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc60077b90 0x7fbc600760a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fbc540098e0 tx=0x7fbc54009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95e11b820427dedc server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:31:00.251 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:59.172+0000 7fbc5fe00700 1 -- 172.21.15.107:0/2093902330 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbc5400b860 con 0x7fbc60077b90 2024-03-04T13:31:00.251 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:59.172+0000 7fbc5fe00700 1 -- 172.21.15.107:0/2093902330 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbc5400be90 con 0x7fbc60077b90 2024-03-04T13:31:00.251 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:59.172+0000 7fbc5fe00700 1 -- 172.21.15.107:0/2093902330 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbc540103c0 con 0x7fbc60077b90 2024-03-04T13:31:00.251 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:59.173+0000 7fbc68200700 1 -- 172.21.15.107:0/2093902330 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc60077b90 msgr2=0x7fbc600760a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:31:00.251 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:59.173+0000 7fbc68200700 1 --2- 172.21.15.107:0/2093902330 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc60077b90 0x7fbc600760a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fbc540098e0 tx=0x7fbc54009250 comp rx=0 tx=0).stop 2024-03-04T13:31:00.251 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:59.173+0000 7fbc68200700 1 -- 172.21.15.107:0/2093902330 shutdown_connections 2024-03-04T13:31:00.251 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:59.173+0000 7fbc68200700 1 --2- 172.21.15.107:0/2093902330 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc60077b90 0x7fbc600760a0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:31:00.251 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:59.173+0000 7fbc68200700 1 -- 172.21.15.107:0/2093902330 >> 172.21.15.107:0/2093902330 conn(0x7fbc600fded0 msgr2=0x7fbc60100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:31:00.251 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:59.173+0000 7fbc68200700 1 -- 172.21.15.107:0/2093902330 shutdown_connections 2024-03-04T13:31:00.251 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:59.173+0000 7fbc68200700 1 -- 172.21.15.107:0/2093902330 wait complete. 2024-03-04T13:31:00.252 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:59.174+0000 7fbc68200700 1 Processor -- start 2024-03-04T13:31:00.252 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:59.174+0000 7fbc68200700 1 -- start start 2024-03-04T13:31:00.252 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:59.174+0000 7fbc68200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc60077b90 0x7fbc60113a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:31:00.252 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:59.174+0000 7fbc68200700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc60113fb0 con 0x7fbc60077b90 2024-03-04T13:31:00.252 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:59.175+0000 7fbc65800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc60077b90 0x7fbc60113a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:31:00.252 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:59.175+0000 7fbc65800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc60077b90 0x7fbc60113a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:35910/0 (socket says 172.21.15.107:35910) 2024-03-04T13:31:00.252 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:59.175+0000 7fbc65800700 1 -- 172.21.15.107:0/1818582904 learned_addr learned my addr 172.21.15.107:0/1818582904 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:31:00.252 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:59.175+0000 7fbc65800700 1 -- 172.21.15.107:0/1818582904 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc540094f0 con 0x7fbc60077b90 2024-03-04T13:31:00.252 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:59.175+0000 7fbc65800700 1 --2- 172.21.15.107:0/1818582904 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc60077b90 0x7fbc60113a70 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbc540090b0 tx=0x7fbc54009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:31:00.252 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:59.176+0000 7fbc5d600700 1 -- 172.21.15.107:0/1818582904 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbc540118f0 con 0x7fbc60077b90 2024-03-04T13:31:00.252 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:59.176+0000 7fbc5d600700 1 -- 172.21.15.107:0/1818582904 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbc54010d10 con 0x7fbc60077b90 2024-03-04T13:31:00.252 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:59.176+0000 7fbc68200700 1 -- 172.21.15.107:0/1818582904 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc601141b0 con 0x7fbc60077b90 2024-03-04T13:31:00.252 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:59.176+0000 7fbc5d600700 1 -- 172.21.15.107:0/1818582904 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbc54024e70 con 0x7fbc60077b90 2024-03-04T13:31:00.252 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:59.176+0000 7fbc68200700 1 -- 172.21.15.107:0/1818582904 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc60114e50 con 0x7fbc60077b90 2024-03-04T13:31:00.253 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:59.177+0000 7fbc5d600700 1 -- 172.21.15.107:0/1818582904 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 12) v1 ==== 43381+0+0 (secure 0 0 0) 0x7fbc5401a550 con 0x7fbc60077b90 2024-03-04T13:31:00.253 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:59.177+0000 7fbc68200700 1 -- 172.21.15.107:0/1818582904 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc6005f5c0 con 0x7fbc60077b90 2024-03-04T13:31:00.253 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:59.177+0000 7fbc5d600700 1 --2- 172.21.15.107:0/1818582904 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7fbc48036790 0x7fbc48038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:31:00.253 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:59.177+0000 7fbc5d600700 1 -- 172.21.15.107:0/1818582904 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fbc5404a750 con 0x7fbc60077b90 2024-03-04T13:31:00.253 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:59.178+0000 7fbc64e00700 1 --2- 172.21.15.107:0/1818582904 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7fbc48036790 0x7fbc48038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:31:00.253 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:59.178+0000 7fbc64e00700 1 --2- 172.21.15.107:0/1818582904 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7fbc48036790 0x7fbc48038c40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c009990 tx=0x7fbc4c006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:31:00.253 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:59.183+0000 7fbc5d600700 1 -- 172.21.15.107:0/1818582904 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fbc54018db0 con 0x7fbc60077b90 2024-03-04T13:31:00.253 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:59.397+0000 7fbc68200700 1 -- 172.21.15.107:0/1818582904 --> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7fbc60108be0 con 0x7fbc48036790 2024-03-04T13:31:00.253 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:59.469+0000 7fbc5d600700 1 -- 172.21.15.107:0/1818582904 <== mgr.14162 v2:172.21.15.107:6800/2040330534 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7fbc60108be0 con 0x7fbc48036790 2024-03-04T13:31:00.253 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:59.472+0000 7fbc68200700 1 -- 172.21.15.107:0/1818582904 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7fbc48036790 msgr2=0x7fbc48038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:31:00.253 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:59.472+0000 7fbc68200700 1 --2- 172.21.15.107:0/1818582904 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7fbc48036790 0x7fbc48038c40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c009990 tx=0x7fbc4c006e40 comp rx=0 tx=0).stop 2024-03-04T13:31:00.253 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:59.472+0000 7fbc68200700 1 -- 172.21.15.107:0/1818582904 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc60077b90 msgr2=0x7fbc60113a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:31:00.253 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:59.472+0000 7fbc68200700 1 --2- 172.21.15.107:0/1818582904 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc60077b90 0x7fbc60113a70 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbc540090b0 tx=0x7fbc54009250 comp rx=0 tx=0).stop 2024-03-04T13:31:00.253 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:59.473+0000 7fbc68200700 1 -- 172.21.15.107:0/1818582904 shutdown_connections 2024-03-04T13:31:00.254 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:59.473+0000 7fbc68200700 1 --2- 172.21.15.107:0/1818582904 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7fbc48036790 0x7fbc48038c40 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:31:00.254 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:59.473+0000 7fbc68200700 1 --2- 172.21.15.107:0/1818582904 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc60077b90 0x7fbc60113a70 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:31:00.254 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:59.473+0000 7fbc68200700 1 -- 172.21.15.107:0/1818582904 >> 172.21.15.107:0/1818582904 conn(0x7fbc600fded0 msgr2=0x7fbc601001d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:31:00.254 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:59.473+0000 7fbc68200700 1 -- 172.21.15.107:0/1818582904 shutdown_connections 2024-03-04T13:31:00.254 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:30:59.473+0000 7fbc68200700 1 -- 172.21.15.107:0/1818582904 wait complete. 2024-03-04T13:31:00.254 INFO:teuthology.orchestra.run.smithi107.stdout:Creating initial admin user... 2024-03-04T13:31:00.500 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:31:00 smithi107 ceph-mon[28739]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:31:00.501 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:31:00 smithi107 ceph-mon[28739]: from='mgr.14162 172.21.15.107:0/1450595538' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:31:00.501 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:31:00 smithi107 ceph-mon[28739]: from='mgr.14162 172.21.15.107:0/1450595538' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:31:01.843 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:31:01 smithi107 ceph-mon[28739]: from='mgr.14162 172.21.15.107:0/1450595538' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:31:01.843 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:31:01 smithi107 ceph-mon[28739]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:31:01.843 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:31:01 smithi107 ceph-mon[28739]: from='mgr.14162 172.21.15.107:0/1450595538' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:31:02.224 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$440ew9lECn.7mickrwT8KOKusl39drsxtT.wxvoI0VvShiUViH7SG", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1709559061, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-03-04T13:31:02.225 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:00.813+0000 7f329fa00700 1 Processor -- start 2024-03-04T13:31:02.225 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:00.813+0000 7f329fa00700 1 -- start start 2024-03-04T13:31:02.225 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:00.814+0000 7f329fa00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3298077bb0 0x7f32980760c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:31:02.226 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:00.814+0000 7f329fa00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3298077fc0 con 0x7f3298077bb0 2024-03-04T13:31:02.226 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:00.814+0000 7f329d000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3298077bb0 0x7f32980760c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:31:02.226 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:00.815+0000 7f329d000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3298077bb0 0x7f32980760c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:35922/0 (socket says 172.21.15.107:35922) 2024-03-04T13:31:02.226 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:00.815+0000 7f329d000700 1 -- 172.21.15.107:0/1951559918 learned_addr learned my addr 172.21.15.107:0/1951559918 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:31:02.226 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:00.815+0000 7f329d000700 1 -- 172.21.15.107:0/1951559918 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3298076600 con 0x7f3298077bb0 2024-03-04T13:31:02.226 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:00.815+0000 7f329d000700 1 --2- 172.21.15.107:0/1951559918 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3298077bb0 0x7f32980760c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f328c0098e0 tx=0x7f328c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a8c2aa6c659e149 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:31:02.228 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:00.816+0000 7f3297400700 1 -- 172.21.15.107:0/1951559918 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f328c00b860 con 0x7f3298077bb0 2024-03-04T13:31:02.229 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:00.816+0000 7f3297400700 1 -- 172.21.15.107:0/1951559918 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f328c00be90 con 0x7f3298077bb0 2024-03-04T13:31:02.229 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:00.816+0000 7f3297400700 1 -- 172.21.15.107:0/1951559918 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f328c0103c0 con 0x7f3298077bb0 2024-03-04T13:31:02.229 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:00.817+0000 7f329fa00700 1 -- 172.21.15.107:0/1951559918 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3298077bb0 msgr2=0x7f32980760c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:31:02.229 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:00.817+0000 7f329fa00700 1 --2- 172.21.15.107:0/1951559918 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3298077bb0 0x7f32980760c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f328c0098e0 tx=0x7f328c009250 comp rx=0 tx=0).stop 2024-03-04T13:31:02.229 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:00.817+0000 7f329fa00700 1 -- 172.21.15.107:0/1951559918 shutdown_connections 2024-03-04T13:31:02.229 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:00.817+0000 7f329fa00700 1 --2- 172.21.15.107:0/1951559918 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3298077bb0 0x7f32980760c0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:31:02.229 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:00.817+0000 7f329fa00700 1 -- 172.21.15.107:0/1951559918 >> 172.21.15.107:0/1951559918 conn(0x7f32980fded0 msgr2=0x7f3298100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:31:02.229 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:00.817+0000 7f329fa00700 1 -- 172.21.15.107:0/1951559918 shutdown_connections 2024-03-04T13:31:02.229 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:00.817+0000 7f329fa00700 1 -- 172.21.15.107:0/1951559918 wait complete. 2024-03-04T13:31:02.229 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:00.818+0000 7f329fa00700 1 Processor -- start 2024-03-04T13:31:02.229 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:00.818+0000 7f329fa00700 1 -- start start 2024-03-04T13:31:02.229 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:00.818+0000 7f329fa00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3298077bb0 0x7f3298113aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:31:02.229 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:00.818+0000 7f329fa00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3298113fe0 con 0x7f3298077bb0 2024-03-04T13:31:02.230 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:00.819+0000 7f329d000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3298077bb0 0x7f3298113aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:31:02.230 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:00.819+0000 7f329d000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3298077bb0 0x7f3298113aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:35930/0 (socket says 172.21.15.107:35930) 2024-03-04T13:31:02.230 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:00.819+0000 7f329d000700 1 -- 172.21.15.107:0/240715697 learned_addr learned my addr 172.21.15.107:0/240715697 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:31:02.230 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:00.819+0000 7f329d000700 1 -- 172.21.15.107:0/240715697 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f328c0094f0 con 0x7f3298077bb0 2024-03-04T13:31:02.230 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:00.819+0000 7f329d000700 1 --2- 172.21.15.107:0/240715697 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3298077bb0 0x7f3298113aa0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f328c003080 tx=0x7f328c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:31:02.230 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:00.820+0000 7f3294c00700 1 -- 172.21.15.107:0/240715697 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f328c011920 con 0x7f3298077bb0 2024-03-04T13:31:02.230 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:00.820+0000 7f329fa00700 1 -- 172.21.15.107:0/240715697 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32981141e0 con 0x7f3298077bb0 2024-03-04T13:31:02.230 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:00.820+0000 7f3294c00700 1 -- 172.21.15.107:0/240715697 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f328c010d60 con 0x7f3298077bb0 2024-03-04T13:31:02.232 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:00.820+0000 7f3294c00700 1 -- 172.21.15.107:0/240715697 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f328c0243f0 con 0x7f3298077bb0 2024-03-04T13:31:02.233 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:00.820+0000 7f329fa00700 1 -- 172.21.15.107:0/240715697 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3298114e80 con 0x7f3298077bb0 2024-03-04T13:31:02.233 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:00.821+0000 7f329fa00700 1 -- 172.21.15.107:0/240715697 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f329805f5c0 con 0x7f3298077bb0 2024-03-04T13:31:02.233 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:00.821+0000 7f3294c00700 1 -- 172.21.15.107:0/240715697 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 12) v1 ==== 43381+0+0 (secure 0 0 0) 0x7f328c02d3f0 con 0x7f3298077bb0 2024-03-04T13:31:02.233 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:00.821+0000 7f3294c00700 1 --2- 172.21.15.107:0/240715697 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7f32800367e0 0x7f3280038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:31:02.233 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:00.822+0000 7f3294c00700 1 -- 172.21.15.107:0/240715697 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f328c04a6b0 con 0x7f3298077bb0 2024-03-04T13:31:02.233 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:00.822+0000 7f3297e00700 1 --2- 172.21.15.107:0/240715697 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7f32800367e0 0x7f3280038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:31:02.233 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:00.823+0000 7f3297e00700 1 --2- 172.21.15.107:0/240715697 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7f32800367e0 0x7f3280038c90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3284009990 tx=0x7f3284006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:31:02.233 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:00.827+0000 7f3294c00700 1 -- 172.21.15.107:0/240715697 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f328c02b8e0 con 0x7f3298077bb0 2024-03-04T13:31:02.233 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:01.079+0000 7f329fa00700 1 -- 172.21.15.107:0/240715697 --> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f329805f3e0 con 0x7f32800367e0 2024-03-04T13:31:02.233 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:01.316+0000 7f3294c00700 1 -- 172.21.15.107:0/240715697 <== mgr.14162 v2:172.21.15.107:6800/2040330534 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f329805f3e0 con 0x7f32800367e0 2024-03-04T13:31:02.233 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:01.319+0000 7f329fa00700 1 -- 172.21.15.107:0/240715697 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7f32800367e0 msgr2=0x7f3280038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:31:02.233 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:01.319+0000 7f329fa00700 1 --2- 172.21.15.107:0/240715697 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7f32800367e0 0x7f3280038c90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3284009990 tx=0x7f3284006e40 comp rx=0 tx=0).stop 2024-03-04T13:31:02.233 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:01.319+0000 7f329fa00700 1 -- 172.21.15.107:0/240715697 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3298077bb0 msgr2=0x7f3298113aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:31:02.233 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:01.319+0000 7f329fa00700 1 --2- 172.21.15.107:0/240715697 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3298077bb0 0x7f3298113aa0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f328c003080 tx=0x7f328c009250 comp rx=0 tx=0).stop 2024-03-04T13:31:02.234 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:01.319+0000 7f329fa00700 1 -- 172.21.15.107:0/240715697 shutdown_connections 2024-03-04T13:31:02.234 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:01.319+0000 7f329fa00700 1 --2- 172.21.15.107:0/240715697 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7f32800367e0 0x7f3280038c90 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:31:02.234 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:01.319+0000 7f329fa00700 1 --2- 172.21.15.107:0/240715697 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3298077bb0 0x7f3298113aa0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:31:02.234 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:01.319+0000 7f329fa00700 1 -- 172.21.15.107:0/240715697 >> 172.21.15.107:0/240715697 conn(0x7f32980fded0 msgr2=0x7f32981001a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:31:02.234 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:01.320+0000 7f329fa00700 1 -- 172.21.15.107:0/240715697 shutdown_connections 2024-03-04T13:31:02.234 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:01.320+0000 7f329fa00700 1 -- 172.21.15.107:0/240715697 wait complete. 2024-03-04T13:31:02.234 INFO:teuthology.orchestra.run.smithi107.stdout:Fetching dashboard port number... 2024-03-04T13:31:03.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:31:03 smithi107 ceph-mon[28739]: mgrmap e13: smithi107.wpzgwl(active, since 6s) 2024-03-04T13:31:04.467 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 8443 2024-03-04T13:31:04.467 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:03.577+0000 7fb942400700 1 Processor -- start 2024-03-04T13:31:04.467 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:03.577+0000 7fb942400700 1 -- start start 2024-03-04T13:31:04.467 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:03.578+0000 7fb942400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb93c102770 0x7fb93c102b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:31:04.467 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:03.578+0000 7fb942400700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb93c1030c0 con 0x7fb93c102770 2024-03-04T13:31:04.467 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:03.578+0000 7fb93b400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb93c102770 0x7fb93c102b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:31:04.467 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:03.578+0000 7fb93b400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb93c102770 0x7fb93c102b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:35944/0 (socket says 172.21.15.107:35944) 2024-03-04T13:31:04.468 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:03.578+0000 7fb93b400700 1 -- 172.21.15.107:0/3062253163 learned_addr learned my addr 172.21.15.107:0/3062253163 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:31:04.468 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:03.579+0000 7fb93b400700 1 -- 172.21.15.107:0/3062253163 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb93c103200 con 0x7fb93c102770 2024-03-04T13:31:04.468 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:03.579+0000 7fb93b400700 1 --2- 172.21.15.107:0/3062253163 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb93c102770 0x7fb93c102b80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb92c0098e0 tx=0x7fb92c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4133ee066c0dae4 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:31:04.468 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:03.579+0000 7fb93a000700 1 -- 172.21.15.107:0/3062253163 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb92c00b860 con 0x7fb93c102770 2024-03-04T13:31:04.468 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:03.579+0000 7fb93a000700 1 -- 172.21.15.107:0/3062253163 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb92c00be90 con 0x7fb93c102770 2024-03-04T13:31:04.468 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:03.579+0000 7fb93a000700 1 -- 172.21.15.107:0/3062253163 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb92c0103c0 con 0x7fb93c102770 2024-03-04T13:31:04.468 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:03.580+0000 7fb942400700 1 -- 172.21.15.107:0/3062253163 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb93c102770 msgr2=0x7fb93c102b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:31:04.468 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:03.580+0000 7fb942400700 1 --2- 172.21.15.107:0/3062253163 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb93c102770 0x7fb93c102b80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb92c0098e0 tx=0x7fb92c009250 comp rx=0 tx=0).stop 2024-03-04T13:31:04.468 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:03.580+0000 7fb942400700 1 -- 172.21.15.107:0/3062253163 shutdown_connections 2024-03-04T13:31:04.468 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:03.580+0000 7fb942400700 1 --2- 172.21.15.107:0/3062253163 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb93c102770 0x7fb93c102b80 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:31:04.468 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:03.580+0000 7fb942400700 1 -- 172.21.15.107:0/3062253163 >> 172.21.15.107:0/3062253163 conn(0x7fb93c0fded0 msgr2=0x7fb93c100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:31:04.469 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:03.580+0000 7fb942400700 1 -- 172.21.15.107:0/3062253163 shutdown_connections 2024-03-04T13:31:04.469 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:03.580+0000 7fb942400700 1 -- 172.21.15.107:0/3062253163 wait complete. 2024-03-04T13:31:04.469 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:03.581+0000 7fb942400700 1 Processor -- start 2024-03-04T13:31:04.469 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:03.581+0000 7fb942400700 1 -- start start 2024-03-04T13:31:04.469 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:03.581+0000 7fb942400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb93c102770 0x7fb93c1137c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:31:04.469 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:03.581+0000 7fb942400700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb93c113d00 con 0x7fb93c102770 2024-03-04T13:31:04.469 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:03.582+0000 7fb93b400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb93c102770 0x7fb93c1137c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:31:04.469 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:03.582+0000 7fb93b400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb93c102770 0x7fb93c1137c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:35946/0 (socket says 172.21.15.107:35946) 2024-03-04T13:31:04.469 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:03.582+0000 7fb93b400700 1 -- 172.21.15.107:0/677147519 learned_addr learned my addr 172.21.15.107:0/677147519 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:31:04.469 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:03.582+0000 7fb93b400700 1 -- 172.21.15.107:0/677147519 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb92c0094f0 con 0x7fb93c102770 2024-03-04T13:31:04.469 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:03.582+0000 7fb93b400700 1 --2- 172.21.15.107:0/677147519 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb93c102770 0x7fb93c1137c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb92c0090b0 tx=0x7fb92c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:31:04.470 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:03.582+0000 7fb933400700 1 -- 172.21.15.107:0/677147519 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb92c010840 con 0x7fb93c102770 2024-03-04T13:31:04.470 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:03.583+0000 7fb933400700 1 -- 172.21.15.107:0/677147519 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb92c010e70 con 0x7fb93c102770 2024-03-04T13:31:04.470 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:03.583+0000 7fb933400700 1 -- 172.21.15.107:0/677147519 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb92c024e40 con 0x7fb93c102770 2024-03-04T13:31:04.470 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:03.583+0000 7fb942400700 1 -- 172.21.15.107:0/677147519 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb93c113f00 con 0x7fb93c102770 2024-03-04T13:31:04.470 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:03.583+0000 7fb942400700 1 -- 172.21.15.107:0/677147519 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb93c114ba0 con 0x7fb93c102770 2024-03-04T13:31:04.470 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:03.584+0000 7fb933400700 1 -- 172.21.15.107:0/677147519 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) v1 ==== 43425+0+0 (secure 0 0 0) 0x7fb92c024430 con 0x7fb93c102770 2024-03-04T13:31:04.470 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:03.584+0000 7fb942400700 1 -- 172.21.15.107:0/677147519 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb93c05f5c0 con 0x7fb93c102770 2024-03-04T13:31:04.470 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:03.584+0000 7fb933400700 1 --2- 172.21.15.107:0/677147519 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7fb920036880 0x7fb920038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:31:04.470 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:03.584+0000 7fb933400700 1 -- 172.21.15.107:0/677147519 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fb92c04aa50 con 0x7fb93c102770 2024-03-04T13:31:04.470 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:03.584+0000 7fb93aa00700 1 --2- 172.21.15.107:0/677147519 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7fb920036880 0x7fb920038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:31:04.471 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:03.585+0000 7fb93aa00700 1 --2- 172.21.15.107:0/677147519 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7fb920036880 0x7fb920038d30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb924009990 tx=0x7fb924006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:31:04.471 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:03.589+0000 7fb933400700 1 -- 172.21.15.107:0/677147519 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fb92c02c930 con 0x7fb93c102770 2024-03-04T13:31:04.471 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:03.795+0000 7fb942400700 1 -- 172.21.15.107:0/677147519 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7fb93c04bed0 con 0x7fb93c102770 2024-03-04T13:31:04.471 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:03.796+0000 7fb933400700 1 -- 172.21.15.107:0/677147519 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7fb92c02c340 con 0x7fb93c102770 2024-03-04T13:31:04.471 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:03.798+0000 7fb942400700 1 -- 172.21.15.107:0/677147519 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7fb920036880 msgr2=0x7fb920038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:31:04.471 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:03.798+0000 7fb942400700 1 --2- 172.21.15.107:0/677147519 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7fb920036880 0x7fb920038d30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb924009990 tx=0x7fb924006e40 comp rx=0 tx=0).stop 2024-03-04T13:31:04.471 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:03.798+0000 7fb942400700 1 -- 172.21.15.107:0/677147519 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb93c102770 msgr2=0x7fb93c1137c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:31:04.471 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:03.798+0000 7fb942400700 1 --2- 172.21.15.107:0/677147519 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb93c102770 0x7fb93c1137c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb92c0090b0 tx=0x7fb92c009250 comp rx=0 tx=0).stop 2024-03-04T13:31:04.472 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:03.799+0000 7fb942400700 1 -- 172.21.15.107:0/677147519 shutdown_connections 2024-03-04T13:31:04.472 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:03.799+0000 7fb942400700 1 --2- 172.21.15.107:0/677147519 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7fb920036880 0x7fb920038d30 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:31:04.472 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:03.799+0000 7fb942400700 1 --2- 172.21.15.107:0/677147519 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb93c102770 0x7fb93c1137c0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:31:04.472 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:03.799+0000 7fb942400700 1 -- 172.21.15.107:0/677147519 >> 172.21.15.107:0/677147519 conn(0x7fb93c0fded0 msgr2=0x7fb93c0ff570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:31:04.472 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:03.799+0000 7fb942400700 1 -- 172.21.15.107:0/677147519 shutdown_connections 2024-03-04T13:31:04.472 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:03.799+0000 7fb942400700 1 -- 172.21.15.107:0/677147519 wait complete. 2024-03-04T13:31:04.492 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-04T13:31:04.492 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout disabled 2024-03-04T13:31:04.512 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-04T13:31:04.513 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout inactive 2024-03-04T13:31:04.513 INFO:teuthology.orchestra.run.smithi107.stdout:firewalld.service is not enabled 2024-03-04T13:31:04.513 INFO:teuthology.orchestra.run.smithi107.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-03-04T13:31:04.520 INFO:teuthology.orchestra.run.smithi107.stdout:Ceph Dashboard is now available at: 2024-03-04T13:31:04.520 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:31:04.520 INFO:teuthology.orchestra.run.smithi107.stdout: URL: https://smithi107.front.sepia.ceph.com:8443/ 2024-03-04T13:31:04.520 INFO:teuthology.orchestra.run.smithi107.stdout: User: admin 2024-03-04T13:31:04.520 INFO:teuthology.orchestra.run.smithi107.stdout: Password: j8zguhx8jh 2024-03-04T13:31:04.520 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:31:04.521 INFO:teuthology.orchestra.run.smithi107.stdout:Saving cluster configuration to /var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/config directory 2024-03-04T13:31:04.523 INFO:teuthology.orchestra.run.smithi107.stdout:Enabling autotune for osd_memory_target 2024-03-04T13:31:04.829 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:31:04 smithi107 ceph-mon[28739]: from='client.? 172.21.15.107:0/677147519' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-03-04T13:31:06.065 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:05.120+0000 7f9beca00700 1 Processor -- start 2024-03-04T13:31:06.065 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:05.120+0000 7f9beca00700 1 -- start start 2024-03-04T13:31:06.066 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:05.121+0000 7f9beca00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9be8078590 0x7f9be80789a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:31:06.066 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:05.121+0000 7f9beca00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9be8078ee0 con 0x7f9be8078590 2024-03-04T13:31:06.066 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:05.121+0000 7f9be5e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9be8078590 0x7f9be80789a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:31:06.066 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:05.121+0000 7f9be5e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9be8078590 0x7f9be80789a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:35948/0 (socket says 172.21.15.107:35948) 2024-03-04T13:31:06.066 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:05.121+0000 7f9be5e00700 1 -- 172.21.15.107:0/750829349 learned_addr learned my addr 172.21.15.107:0/750829349 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:31:06.066 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:05.122+0000 7f9be5e00700 1 -- 172.21.15.107:0/750829349 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9be8079020 con 0x7f9be8078590 2024-03-04T13:31:06.066 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:05.122+0000 7f9be5e00700 1 --2- 172.21.15.107:0/750829349 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9be8078590 0x7f9be80789a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9bd8009e50 tx=0x7f9bd8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b71d3104de888377 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:31:06.066 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:05.122+0000 7f9be4a00700 1 -- 172.21.15.107:0/750829349 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9bd80092e0 con 0x7f9be8078590 2024-03-04T13:31:06.066 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:05.123+0000 7f9be4a00700 1 -- 172.21.15.107:0/750829349 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9bd8012430 con 0x7f9be8078590 2024-03-04T13:31:06.066 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:05.123+0000 7f9be4a00700 1 -- 172.21.15.107:0/750829349 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9bd80075a0 con 0x7f9be8078590 2024-03-04T13:31:06.066 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:05.123+0000 7f9beca00700 1 -- 172.21.15.107:0/750829349 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9be8078590 msgr2=0x7f9be80789a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:31:06.066 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:05.123+0000 7f9beca00700 1 --2- 172.21.15.107:0/750829349 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9be8078590 0x7f9be80789a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9bd8009e50 tx=0x7f9bd8009840 comp rx=0 tx=0).stop 2024-03-04T13:31:06.066 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:05.123+0000 7f9beca00700 1 -- 172.21.15.107:0/750829349 shutdown_connections 2024-03-04T13:31:06.067 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:05.123+0000 7f9beca00700 1 --2- 172.21.15.107:0/750829349 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9be8078590 0x7f9be80789a0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:31:06.067 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:05.123+0000 7f9beca00700 1 -- 172.21.15.107:0/750829349 >> 172.21.15.107:0/750829349 conn(0x7f9be8100340 msgr2=0x7f9be8102720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:31:06.067 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:05.123+0000 7f9beca00700 1 -- 172.21.15.107:0/750829349 shutdown_connections 2024-03-04T13:31:06.067 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:05.124+0000 7f9beca00700 1 -- 172.21.15.107:0/750829349 wait complete. 2024-03-04T13:31:06.067 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:05.124+0000 7f9beca00700 1 Processor -- start 2024-03-04T13:31:06.067 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:05.124+0000 7f9beca00700 1 -- start start 2024-03-04T13:31:06.067 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:05.125+0000 7f9beca00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9be8078590 0x7f9be8117be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:31:06.067 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:05.125+0000 7f9beca00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9be8118120 con 0x7f9be8078590 2024-03-04T13:31:06.067 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:05.125+0000 7f9be5e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9be8078590 0x7f9be8117be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:31:06.067 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:05.125+0000 7f9be5e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9be8078590 0x7f9be8117be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:35960/0 (socket says 172.21.15.107:35960) 2024-03-04T13:31:06.067 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:05.125+0000 7f9be5e00700 1 -- 172.21.15.107:0/3979230311 learned_addr learned my addr 172.21.15.107:0/3979230311 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:31:06.067 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:05.125+0000 7f9be5e00700 1 -- 172.21.15.107:0/3979230311 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bd8009ae0 con 0x7f9be8078590 2024-03-04T13:31:06.067 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:05.126+0000 7f9be5e00700 1 --2- 172.21.15.107:0/3979230311 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9be8078590 0x7f9be8117be0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9bd8012de0 tx=0x7f9bd8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:31:06.067 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:05.126+0000 7f9bde000700 1 -- 172.21.15.107:0/3979230311 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9bd80092e0 con 0x7f9be8078590 2024-03-04T13:31:06.068 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:05.126+0000 7f9bde000700 1 -- 172.21.15.107:0/3979230311 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9bd8007d30 con 0x7f9be8078590 2024-03-04T13:31:06.068 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:05.126+0000 7f9bde000700 1 -- 172.21.15.107:0/3979230311 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9bd8025930 con 0x7f9be8078590 2024-03-04T13:31:06.068 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:05.126+0000 7f9beca00700 1 -- 172.21.15.107:0/3979230311 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9be8118320 con 0x7f9be8078590 2024-03-04T13:31:06.068 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:05.126+0000 7f9beca00700 1 -- 172.21.15.107:0/3979230311 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9be8118fc0 con 0x7f9be8078590 2024-03-04T13:31:06.068 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:05.127+0000 7f9bde000700 1 -- 172.21.15.107:0/3979230311 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) v1 ==== 43425+0+0 (secure 0 0 0) 0x7f9bd8019050 con 0x7f9be8078590 2024-03-04T13:31:06.068 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:05.127+0000 7f9beca00700 1 -- 172.21.15.107:0/3979230311 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9be805f5c0 con 0x7f9be8078590 2024-03-04T13:31:06.068 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:05.128+0000 7f9bde000700 1 --2- 172.21.15.107:0/3979230311 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7f9bcc036880 0x7f9bcc038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:31:06.068 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:05.128+0000 7f9bde000700 1 -- 172.21.15.107:0/3979230311 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f9bd804a300 con 0x7f9be8078590 2024-03-04T13:31:06.068 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:05.128+0000 7f9be5400700 1 --2- 172.21.15.107:0/3979230311 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7f9bcc036880 0x7f9bcc038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:31:06.068 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:05.129+0000 7f9be5400700 1 --2- 172.21.15.107:0/3979230311 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7f9bcc036880 0x7f9bcc038d30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9bd4009990 tx=0x7f9bd4006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:31:06.068 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:05.133+0000 7f9bde000700 1 -- 172.21.15.107:0/3979230311 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f9bd8011570 con 0x7f9be8078590 2024-03-04T13:31:06.068 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:05.337+0000 7f9beca00700 1 -- 172.21.15.107:0/3979230311 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f9be804bed0 con 0x7f9be8078590 2024-03-04T13:31:06.069 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:05.338+0000 7f9bde000700 1 -- 172.21.15.107:0/3979230311 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f9bd80200c0 con 0x7f9be8078590 2024-03-04T13:31:06.069 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:05.340+0000 7f9beca00700 1 -- 172.21.15.107:0/3979230311 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7f9bcc036880 msgr2=0x7f9bcc038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:31:06.069 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:05.340+0000 7f9beca00700 1 --2- 172.21.15.107:0/3979230311 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7f9bcc036880 0x7f9bcc038d30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9bd4009990 tx=0x7f9bd4006e40 comp rx=0 tx=0).stop 2024-03-04T13:31:06.069 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:05.340+0000 7f9beca00700 1 -- 172.21.15.107:0/3979230311 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9be8078590 msgr2=0x7f9be8117be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:31:06.069 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:05.340+0000 7f9beca00700 1 --2- 172.21.15.107:0/3979230311 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9be8078590 0x7f9be8117be0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9bd8012de0 tx=0x7f9bd8009840 comp rx=0 tx=0).stop 2024-03-04T13:31:06.069 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:05.341+0000 7f9beca00700 1 -- 172.21.15.107:0/3979230311 shutdown_connections 2024-03-04T13:31:06.069 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:05.341+0000 7f9beca00700 1 --2- 172.21.15.107:0/3979230311 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7f9bcc036880 0x7f9bcc038d30 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:31:06.069 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:05.341+0000 7f9beca00700 1 --2- 172.21.15.107:0/3979230311 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9be8078590 0x7f9be8117be0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:31:06.069 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:05.341+0000 7f9beca00700 1 -- 172.21.15.107:0/3979230311 >> 172.21.15.107:0/3979230311 conn(0x7f9be8100340 msgr2=0x7f9be8103250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:31:06.069 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:05.341+0000 7f9beca00700 1 -- 172.21.15.107:0/3979230311 shutdown_connections 2024-03-04T13:31:06.069 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:05.341+0000 7f9beca00700 1 -- 172.21.15.107:0/3979230311 wait complete. 2024-03-04T13:31:07.687 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:06.682+0000 7fd9a5800700 1 Processor -- start 2024-03-04T13:31:07.687 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:06.682+0000 7fd9a5800700 1 -- start start 2024-03-04T13:31:07.687 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:06.683+0000 7fd9a5800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9a0102770 0x7fd9a0102b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:31:07.687 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:06.683+0000 7fd9a5800700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a01030c0 con 0x7fd9a0102770 2024-03-04T13:31:07.687 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:06.683+0000 7fd99ea00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9a0102770 0x7fd9a0102b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:31:07.688 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:06.684+0000 7fd99ea00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9a0102770 0x7fd9a0102b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:35974/0 (socket says 172.21.15.107:35974) 2024-03-04T13:31:07.688 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:06.684+0000 7fd99ea00700 1 -- 172.21.15.107:0/2161340257 learned_addr learned my addr 172.21.15.107:0/2161340257 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:31:07.688 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:06.684+0000 7fd99ea00700 1 -- 172.21.15.107:0/2161340257 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9a0103200 con 0x7fd9a0102770 2024-03-04T13:31:07.688 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:06.684+0000 7fd99ea00700 1 --2- 172.21.15.107:0/2161340257 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9a0102770 0x7fd9a0102b80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd9900098e0 tx=0x7fd990009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39f9778b6a4b7b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:31:07.688 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:06.685+0000 7fd99e000700 1 -- 172.21.15.107:0/2161340257 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd99000b860 con 0x7fd9a0102770 2024-03-04T13:31:07.688 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:06.685+0000 7fd99e000700 1 -- 172.21.15.107:0/2161340257 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd99000be90 con 0x7fd9a0102770 2024-03-04T13:31:07.688 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:06.685+0000 7fd99e000700 1 -- 172.21.15.107:0/2161340257 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd9900103c0 con 0x7fd9a0102770 2024-03-04T13:31:07.688 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:06.685+0000 7fd9a5800700 1 -- 172.21.15.107:0/2161340257 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9a0102770 msgr2=0x7fd9a0102b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:31:07.688 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:06.685+0000 7fd9a5800700 1 --2- 172.21.15.107:0/2161340257 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9a0102770 0x7fd9a0102b80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd9900098e0 tx=0x7fd990009250 comp rx=0 tx=0).stop 2024-03-04T13:31:07.688 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:06.685+0000 7fd9a5800700 1 -- 172.21.15.107:0/2161340257 shutdown_connections 2024-03-04T13:31:07.688 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:06.685+0000 7fd9a5800700 1 --2- 172.21.15.107:0/2161340257 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9a0102770 0x7fd9a0102b80 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:31:07.688 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:06.685+0000 7fd9a5800700 1 -- 172.21.15.107:0/2161340257 >> 172.21.15.107:0/2161340257 conn(0x7fd9a00fded0 msgr2=0x7fd9a0100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:31:07.689 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:06.685+0000 7fd9a5800700 1 -- 172.21.15.107:0/2161340257 shutdown_connections 2024-03-04T13:31:07.689 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:06.686+0000 7fd9a5800700 1 -- 172.21.15.107:0/2161340257 wait complete. 2024-03-04T13:31:07.689 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:06.686+0000 7fd9a5800700 1 Processor -- start 2024-03-04T13:31:07.689 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:06.686+0000 7fd9a5800700 1 -- start start 2024-03-04T13:31:07.689 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:06.687+0000 7fd9a5800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9a0102770 0x7fd9a01115c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:31:07.689 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:06.687+0000 7fd9a5800700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a0111b00 con 0x7fd9a0102770 2024-03-04T13:31:07.689 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:06.687+0000 7fd99ea00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9a0102770 0x7fd9a01115c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:31:07.689 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:06.687+0000 7fd99ea00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9a0102770 0x7fd9a01115c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:35988/0 (socket says 172.21.15.107:35988) 2024-03-04T13:31:07.689 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:06.687+0000 7fd99ea00700 1 -- 172.21.15.107:0/1615702868 learned_addr learned my addr 172.21.15.107:0/1615702868 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:31:07.689 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:06.687+0000 7fd99ea00700 1 -- 172.21.15.107:0/1615702868 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9900094f0 con 0x7fd9a0102770 2024-03-04T13:31:07.689 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:06.688+0000 7fd99ea00700 1 --2- 172.21.15.107:0/1615702868 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9a0102770 0x7fd9a01115c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd9900090b0 tx=0x7fd990009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:31:07.689 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:06.688+0000 7fd997400700 1 -- 172.21.15.107:0/1615702868 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd990010840 con 0x7fd9a0102770 2024-03-04T13:31:07.689 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:06.688+0000 7fd997400700 1 -- 172.21.15.107:0/1615702868 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd990010e70 con 0x7fd9a0102770 2024-03-04T13:31:07.690 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:06.688+0000 7fd9a5800700 1 -- 172.21.15.107:0/1615702868 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9a0111d00 con 0x7fd9a0102770 2024-03-04T13:31:07.690 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:06.688+0000 7fd997400700 1 -- 172.21.15.107:0/1615702868 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd990024e40 con 0x7fd9a0102770 2024-03-04T13:31:07.690 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:06.688+0000 7fd9a5800700 1 -- 172.21.15.107:0/1615702868 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9a01129a0 con 0x7fd9a0102770 2024-03-04T13:31:07.690 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:06.689+0000 7fd997400700 1 -- 172.21.15.107:0/1615702868 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) v1 ==== 43425+0+0 (secure 0 0 0) 0x7fd99001a4c0 con 0x7fd9a0102770 2024-03-04T13:31:07.690 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:06.689+0000 7fd9a5800700 1 -- 172.21.15.107:0/1615702868 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9a005f5c0 con 0x7fd9a0102770 2024-03-04T13:31:07.690 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:06.690+0000 7fd997400700 1 --2- 172.21.15.107:0/1615702868 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7fd984036880 0x7fd984038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:31:07.690 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:06.690+0000 7fd997400700 1 -- 172.21.15.107:0/1615702868 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fd99004ac10 con 0x7fd9a0102770 2024-03-04T13:31:07.690 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:06.690+0000 7fd996000700 1 --2- 172.21.15.107:0/1615702868 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7fd984036880 0x7fd984038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:31:07.690 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:06.691+0000 7fd996000700 1 --2- 172.21.15.107:0/1615702868 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7fd984036880 0x7fd984038d30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd988009990 tx=0x7fd988006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:31:07.690 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:06.695+0000 7fd997400700 1 -- 172.21.15.107:0/1615702868 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fd990018940 con 0x7fd9a0102770 2024-03-04T13:31:07.690 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:07.007+0000 7fd9a5800700 1 -- 172.21.15.107:0/1615702868 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7fd9a004bed0 con 0x7fd9a0102770 2024-03-04T13:31:07.690 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:07.011+0000 7fd997400700 1 -- 172.21.15.107:0/1615702868 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v33)=0 set mgr/dashboard/cluster/status v33) v1 ==== 153+0+0 (secure 0 0 0) 0x7fd99002d3a0 con 0x7fd9a0102770 2024-03-04T13:31:07.690 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:07.013+0000 7fd9a5800700 1 -- 172.21.15.107:0/1615702868 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7fd984036880 msgr2=0x7fd984038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:31:07.691 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:07.013+0000 7fd9a5800700 1 --2- 172.21.15.107:0/1615702868 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7fd984036880 0x7fd984038d30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd988009990 tx=0x7fd988006e40 comp rx=0 tx=0).stop 2024-03-04T13:31:07.691 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:07.013+0000 7fd9a5800700 1 -- 172.21.15.107:0/1615702868 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9a0102770 msgr2=0x7fd9a01115c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:31:07.691 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:07.013+0000 7fd9a5800700 1 --2- 172.21.15.107:0/1615702868 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9a0102770 0x7fd9a01115c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd9900090b0 tx=0x7fd990009250 comp rx=0 tx=0).stop 2024-03-04T13:31:07.691 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:07.014+0000 7fd9a5800700 1 -- 172.21.15.107:0/1615702868 shutdown_connections 2024-03-04T13:31:07.691 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:07.014+0000 7fd9a5800700 1 --2- 172.21.15.107:0/1615702868 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7fd984036880 0x7fd984038d30 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:31:07.691 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:07.014+0000 7fd9a5800700 1 --2- 172.21.15.107:0/1615702868 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9a0102770 0x7fd9a01115c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:31:07.691 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:07.014+0000 7fd9a5800700 1 -- 172.21.15.107:0/1615702868 >> 172.21.15.107:0/1615702868 conn(0x7fd9a00fded0 msgr2=0x7fd9a00ff5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:31:07.691 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:07.014+0000 7fd9a5800700 1 -- 172.21.15.107:0/1615702868 shutdown_connections 2024-03-04T13:31:07.691 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-04T13:31:07.014+0000 7fd9a5800700 1 -- 172.21.15.107:0/1615702868 wait complete. 2024-03-04T13:31:07.691 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-03-04T13:31:07.691 INFO:teuthology.orchestra.run.smithi107.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-03-04T13:31:07.691 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:31:07.691 INFO:teuthology.orchestra.run.smithi107.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-03-04T13:31:07.691 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:31:07.692 INFO:teuthology.orchestra.run.smithi107.stdout:Or, if you are only running a single cluster on this host: 2024-03-04T13:31:07.692 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:31:07.692 INFO:teuthology.orchestra.run.smithi107.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-03-04T13:31:07.692 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:31:07.692 INFO:teuthology.orchestra.run.smithi107.stdout:Please consider enabling telemetry to help improve Ceph: 2024-03-04T13:31:07.692 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:31:07.692 INFO:teuthology.orchestra.run.smithi107.stdout: ceph telemetry on 2024-03-04T13:31:07.692 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:31:07.692 INFO:teuthology.orchestra.run.smithi107.stdout:For more information see: 2024-03-04T13:31:07.692 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:31:07.692 INFO:teuthology.orchestra.run.smithi107.stdout: https://docs.ceph.com/docs/master/mgr/telemetry/ 2024-03-04T13:31:07.692 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:31:07.692 INFO:teuthology.orchestra.run.smithi107.stdout:Bootstrap complete. 2024-03-04T13:31:07.782 INFO:tasks.cephadm:Fetching config... 2024-03-04T13:31:07.783 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-03-04T13:31:07.783 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-03-04T13:31:07.830 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-03-04T13:31:07.830 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-03-04T13:31:07.830 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-03-04T13:31:07.910 INFO:tasks.cephadm:Fetching mon keyring... 2024-03-04T13:31:07.910 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-03-04T13:31:07.910 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/mon.smithi107/keyring of=/dev/stdout 2024-03-04T13:31:08.047 INFO:tasks.cephadm:Fetching pub ssh key... 2024-03-04T13:31:08.048 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-03-04T13:31:08.048 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-03-04T13:31:08.097 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-03-04T13:31:08.097 DEBUG:teuthology.orchestra.run.smithi107:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDI+M5RQcJ/GKF1O9RXkU5IR3uNr3kDgSfxLe+lzPUt27gelf8X8+Ba2tMylJBojMP8Jxi/F/JUeyPgI7apHDP0tayB6KXygj1sFxYRXXUI11e9pxC09R1iWBP5YKbtMaQQ+/H0lBJRoit/LokSeMmCQYEmf0T4GbbOqoFWQw97XYCbzDFkWvCupcKI60V3T9RIw+4xJUB4auZ7vzYN1Fo7E8/ehwsm1TQqNL8i+ZiGsb56lLUaAYjcZot7ALut1xdhT2MCY7vMs+6MjqmGB6zac3j6KdyM9lEvhQ3NfKEQ59AKhVBq3cEEWBf2X5zb+oKWKwoSJhDvYEfBoGWIq8eEHBTPcp249yztp6bC6r5xSlK+Vxv7x+I0CQ5QL5OYuelgx/tGmesAbBaPGihJs+nUEeY/z5mW/8aYYMKMIQ0eEa22ef5G93R4NquZi9Hi7z1VpjgEUcHSXtpf0ILzKuf8TaTAObVyI/9BhNmy3MA783Q4zHgbjQ8qpqK/hC0gPEE= ceph-1fc09a58-da2b-11ee-95c6-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-04T13:31:08.233 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:31:08 smithi107 ceph-mon[28739]: from='client.? 172.21.15.107:0/1615702868' entity='client.admin' 2024-03-04T13:31:08.270 INFO:teuthology.orchestra.run.smithi107.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDI+M5RQcJ/GKF1O9RXkU5IR3uNr3kDgSfxLe+lzPUt27gelf8X8+Ba2tMylJBojMP8Jxi/F/JUeyPgI7apHDP0tayB6KXygj1sFxYRXXUI11e9pxC09R1iWBP5YKbtMaQQ+/H0lBJRoit/LokSeMmCQYEmf0T4GbbOqoFWQw97XYCbzDFkWvCupcKI60V3T9RIw+4xJUB4auZ7vzYN1Fo7E8/ehwsm1TQqNL8i+ZiGsb56lLUaAYjcZot7ALut1xdhT2MCY7vMs+6MjqmGB6zac3j6KdyM9lEvhQ3NfKEQ59AKhVBq3cEEWBf2X5zb+oKWKwoSJhDvYEfBoGWIq8eEHBTPcp249yztp6bC6r5xSlK+Vxv7x+I0CQ5QL5OYuelgx/tGmesAbBaPGihJs+nUEeY/z5mW/8aYYMKMIQ0eEa22ef5G93R4NquZi9Hi7z1VpjgEUcHSXtpf0ILzKuf8TaTAObVyI/9BhNmy3MA783Q4zHgbjQ8qpqK/hC0gPEE= ceph-1fc09a58-da2b-11ee-95c6-87774f69a715 2024-03-04T13:31:08.313 DEBUG:teuthology.orchestra.run.smithi169:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDI+M5RQcJ/GKF1O9RXkU5IR3uNr3kDgSfxLe+lzPUt27gelf8X8+Ba2tMylJBojMP8Jxi/F/JUeyPgI7apHDP0tayB6KXygj1sFxYRXXUI11e9pxC09R1iWBP5YKbtMaQQ+/H0lBJRoit/LokSeMmCQYEmf0T4GbbOqoFWQw97XYCbzDFkWvCupcKI60V3T9RIw+4xJUB4auZ7vzYN1Fo7E8/ehwsm1TQqNL8i+ZiGsb56lLUaAYjcZot7ALut1xdhT2MCY7vMs+6MjqmGB6zac3j6KdyM9lEvhQ3NfKEQ59AKhVBq3cEEWBf2X5zb+oKWKwoSJhDvYEfBoGWIq8eEHBTPcp249yztp6bC6r5xSlK+Vxv7x+I0CQ5QL5OYuelgx/tGmesAbBaPGihJs+nUEeY/z5mW/8aYYMKMIQ0eEa22ef5G93R4NquZi9Hi7z1VpjgEUcHSXtpf0ILzKuf8TaTAObVyI/9BhNmy3MA783Q4zHgbjQ8qpqK/hC0gPEE= ceph-1fc09a58-da2b-11ee-95c6-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-04T13:31:08.450 INFO:teuthology.orchestra.run.smithi169.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDI+M5RQcJ/GKF1O9RXkU5IR3uNr3kDgSfxLe+lzPUt27gelf8X8+Ba2tMylJBojMP8Jxi/F/JUeyPgI7apHDP0tayB6KXygj1sFxYRXXUI11e9pxC09R1iWBP5YKbtMaQQ+/H0lBJRoit/LokSeMmCQYEmf0T4GbbOqoFWQw97XYCbzDFkWvCupcKI60V3T9RIw+4xJUB4auZ7vzYN1Fo7E8/ehwsm1TQqNL8i+ZiGsb56lLUaAYjcZot7ALut1xdhT2MCY7vMs+6MjqmGB6zac3j6KdyM9lEvhQ3NfKEQ59AKhVBq3cEEWBf2X5zb+oKWKwoSJhDvYEfBoGWIq8eEHBTPcp249yztp6bC6r5xSlK+Vxv7x+I0CQ5QL5OYuelgx/tGmesAbBaPGihJs+nUEeY/z5mW/8aYYMKMIQ0eEa22ef5G93R4NquZi9Hi7z1VpjgEUcHSXtpf0ILzKuf8TaTAObVyI/9BhNmy3MA783Q4zHgbjQ8qpqK/hC0gPEE= ceph-1fc09a58-da2b-11ee-95c6-87774f69a715 2024-03-04T13:31:08.519 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-03-04T13:31:08.964 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/mon.smithi107/config 2024-03-04T13:31:11.951 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:31:11 smithi107 ceph-mon[28739]: from='mgr.14162 172.21.15.107:0/1450595538' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:31:11.951 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:31:11 smithi107 ceph-mon[28739]: from='mgr.14162 172.21.15.107:0/1450595538' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:31:11.951 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:31:11 smithi107 ceph-mon[28739]: from='mgr.14162 172.21.15.107:0/1450595538' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "who": "osd/host:smithi107", "name": "osd_memory_target"}]: dispatch 2024-03-04T13:31:11.951 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:31:11 smithi107 ceph-mon[28739]: from='mgr.14162 172.21.15.107:0/1450595538' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:31:11.951 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:31:11 smithi107 ceph-mon[28739]: from='mgr.14162 172.21.15.107:0/1450595538' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi107", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-04T13:31:11.951 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:31:11 smithi107 ceph-mon[28739]: from='mgr.14162 172.21.15.107:0/1450595538' entity='mgr.smithi107.wpzgwl' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi107", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-03-04T13:31:11.951 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:31:11 smithi107 ceph-mon[28739]: from='mgr.14162 172.21.15.107:0/1450595538' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:31:11.951 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:31:11 smithi107 ceph-mon[28739]: Deploying daemon crash.smithi107 on smithi107 2024-03-04T13:31:12.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:12.016+0000 7fc782800700 1 -- 172.21.15.107:0/1135109185 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc77c071f10 msgr2=0x7fc77c070380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:31:12.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:12.016+0000 7fc782800700 1 --2- 172.21.15.107:0/1135109185 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc77c071f10 0x7fc77c070380 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc76c0048c0 tx=0x7fc76c009e30 comp rx=0 tx=0).stop 2024-03-04T13:31:12.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:12.016+0000 7fc782800700 1 -- 172.21.15.107:0/1135109185 shutdown_connections 2024-03-04T13:31:12.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:12.016+0000 7fc782800700 1 --2- 172.21.15.107:0/1135109185 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc77c071f10 0x7fc77c070380 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:31:12.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:12.016+0000 7fc782800700 1 -- 172.21.15.107:0/1135109185 >> 172.21.15.107:0/1135109185 conn(0x7fc77c0f82a0 msgr2=0x7fc77c0fa6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:31:12.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:12.016+0000 7fc782800700 1 -- 172.21.15.107:0/1135109185 shutdown_connections 2024-03-04T13:31:12.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:12.017+0000 7fc782800700 1 -- 172.21.15.107:0/1135109185 wait complete. 2024-03-04T13:31:12.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:12.017+0000 7fc782800700 1 Processor -- start 2024-03-04T13:31:12.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:12.018+0000 7fc782800700 1 -- start start 2024-03-04T13:31:12.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:12.018+0000 7fc782800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc77c071f10 0x7fc77c10f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:31:12.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:12.018+0000 7fc782800700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc77c10f5c0 con 0x7fc77c071f10 2024-03-04T13:31:12.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:12.018+0000 7fc77b400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc77c071f10 0x7fc77c10f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:31:12.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:12.018+0000 7fc77b400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc77c071f10 0x7fc77c10f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57620/0 (socket says 172.21.15.107:57620) 2024-03-04T13:31:12.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:12.018+0000 7fc77b400700 1 -- 172.21.15.107:0/411566554 learned_addr learned my addr 172.21.15.107:0/411566554 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:31:12.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:12.019+0000 7fc77b400700 1 -- 172.21.15.107:0/411566554 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc76c005040 con 0x7fc77c071f10 2024-03-04T13:31:12.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:12.019+0000 7fc77b400700 1 --2- 172.21.15.107:0/411566554 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc77c071f10 0x7fc77c10f080 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc76c009b80 tx=0x7fc76c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:31:12.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:12.019+0000 7fc773400700 1 -- 172.21.15.107:0/411566554 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc76c00b5c0 con 0x7fc77c071f10 2024-03-04T13:31:12.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:12.019+0000 7fc773400700 1 -- 172.21.15.107:0/411566554 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc76c00be70 con 0x7fc77c071f10 2024-03-04T13:31:12.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:12.019+0000 7fc782800700 1 -- 172.21.15.107:0/411566554 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc77c10f7c0 con 0x7fc77c071f10 2024-03-04T13:31:12.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:12.019+0000 7fc773400700 1 -- 172.21.15.107:0/411566554 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc76c01c660 con 0x7fc77c071f10 2024-03-04T13:31:12.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:12.020+0000 7fc782800700 1 -- 172.21.15.107:0/411566554 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc77c110460 con 0x7fc77c071f10 2024-03-04T13:31:12.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:12.021+0000 7fc773400700 1 -- 172.21.15.107:0/411566554 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) v1 ==== 43425+0+0 (secure 0 0 0) 0x7fc76c013420 con 0x7fc77c071f10 2024-03-04T13:31:12.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:12.021+0000 7fc782800700 1 -- 172.21.15.107:0/411566554 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc77c05f5c0 con 0x7fc77c071f10 2024-03-04T13:31:12.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:12.021+0000 7fc773400700 1 --2- 172.21.15.107:0/411566554 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7fc7600368c0 0x7fc760038d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:31:12.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:12.022+0000 7fc773400700 1 -- 172.21.15.107:0/411566554 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fc76c049b10 con 0x7fc77c071f10 2024-03-04T13:31:12.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:12.022+0000 7fc77aa00700 1 --2- 172.21.15.107:0/411566554 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7fc7600368c0 0x7fc760038d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:31:12.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:12.022+0000 7fc77aa00700 1 --2- 172.21.15.107:0/411566554 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7fc7600368c0 0x7fc760038d70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc768009990 tx=0x7fc768006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:31:12.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:12.027+0000 7fc773400700 1 -- 172.21.15.107:0/411566554 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fc76c0119c0 con 0x7fc77c071f10 2024-03-04T13:31:12.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:12.236+0000 7fc782800700 1 -- 172.21.15.107:0/411566554 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7fc77c04bed0 con 0x7fc77c071f10 2024-03-04T13:31:12.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:12.241+0000 7fc773400700 1 -- 172.21.15.107:0/411566554 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7fc76c011370 con 0x7fc77c071f10 2024-03-04T13:31:12.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:12.244+0000 7fc782800700 1 -- 172.21.15.107:0/411566554 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7fc7600368c0 msgr2=0x7fc760038d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:31:12.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:12.244+0000 7fc782800700 1 --2- 172.21.15.107:0/411566554 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7fc7600368c0 0x7fc760038d70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc768009990 tx=0x7fc768006e40 comp rx=0 tx=0).stop 2024-03-04T13:31:12.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:12.244+0000 7fc782800700 1 -- 172.21.15.107:0/411566554 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc77c071f10 msgr2=0x7fc77c10f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:31:12.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:12.244+0000 7fc782800700 1 --2- 172.21.15.107:0/411566554 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc77c071f10 0x7fc77c10f080 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc76c009b80 tx=0x7fc76c009e30 comp rx=0 tx=0).stop 2024-03-04T13:31:12.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:12.244+0000 7fc782800700 1 -- 172.21.15.107:0/411566554 shutdown_connections 2024-03-04T13:31:12.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:12.244+0000 7fc782800700 1 --2- 172.21.15.107:0/411566554 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7fc7600368c0 0x7fc760038d70 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:31:12.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:12.244+0000 7fc782800700 1 --2- 172.21.15.107:0/411566554 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc77c071f10 0x7fc77c10f080 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:31:12.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:12.244+0000 7fc782800700 1 -- 172.21.15.107:0/411566554 >> 172.21.15.107:0/411566554 conn(0x7fc77c0f82a0 msgr2=0x7fc77c0f8f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:31:12.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:12.245+0000 7fc782800700 1 -- 172.21.15.107:0/411566554 shutdown_connections 2024-03-04T13:31:12.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:12.245+0000 7fc782800700 1 -- 172.21.15.107:0/411566554 wait complete. 2024-03-04T13:31:13.048 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-03-04T13:31:13.048 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-03-04T13:31:13.337 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:31:13 smithi107 ceph-mon[28739]: from='client.? 172.21.15.107:0/411566554' entity='client.admin' 2024-03-04T13:31:13.983 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/mon.smithi107/config 2024-03-04T13:31:16.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:16.470+0000 7f8cb2400700 1 -- 172.21.15.107:0/3530497804 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8cac0fca00 msgr2=0x7f8cac0fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:31:16.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:16.470+0000 7f8cb2400700 1 --2- 172.21.15.107:0/3530497804 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8cac0fca00 0x7f8cac0fce10 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c0048c0 tx=0x7f8c9c009e30 comp rx=0 tx=0).stop 2024-03-04T13:31:16.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:16.471+0000 7f8cb2400700 1 -- 172.21.15.107:0/3530497804 shutdown_connections 2024-03-04T13:31:16.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:16.471+0000 7f8cb2400700 1 --2- 172.21.15.107:0/3530497804 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8cac0fca00 0x7f8cac0fce10 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:31:16.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:16.471+0000 7f8cb2400700 1 -- 172.21.15.107:0/3530497804 >> 172.21.15.107:0/3530497804 conn(0x7f8cac0f80c0 msgr2=0x7f8cac0fa4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:31:16.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:16.471+0000 7f8cb2400700 1 -- 172.21.15.107:0/3530497804 shutdown_connections 2024-03-04T13:31:16.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:16.471+0000 7f8cb2400700 1 -- 172.21.15.107:0/3530497804 wait complete. 2024-03-04T13:31:16.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:16.472+0000 7f8cb2400700 1 Processor -- start 2024-03-04T13:31:16.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:16.472+0000 7f8cb2400700 1 -- start start 2024-03-04T13:31:16.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:16.472+0000 7f8cb2400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8cac0fca00 0x7f8cac10efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:31:16.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:16.472+0000 7f8cb2400700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cac10f4e0 con 0x7f8cac0fca00 2024-03-04T13:31:16.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:16.472+0000 7f8cb1000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8cac0fca00 0x7f8cac10efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:31:16.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:16.472+0000 7f8cb1000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8cac0fca00 0x7f8cac10efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57636/0 (socket says 172.21.15.107:57636) 2024-03-04T13:31:16.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:16.472+0000 7f8cb1000700 1 -- 172.21.15.107:0/3458720227 learned_addr learned my addr 172.21.15.107:0/3458720227 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:31:16.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:16.473+0000 7f8cb1000700 1 -- 172.21.15.107:0/3458720227 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c9c005040 con 0x7f8cac0fca00 2024-03-04T13:31:16.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:16.473+0000 7f8cb1000700 1 --2- 172.21.15.107:0/3458720227 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8cac0fca00 0x7f8cac10efa0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c009b80 tx=0x7f8c9c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:31:16.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:16.473+0000 7f8ca9600700 1 -- 172.21.15.107:0/3458720227 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8c9c00b5c0 con 0x7f8cac0fca00 2024-03-04T13:31:16.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:16.473+0000 7f8ca9600700 1 -- 172.21.15.107:0/3458720227 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8c9c00be70 con 0x7f8cac0fca00 2024-03-04T13:31:16.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:16.473+0000 7f8ca9600700 1 -- 172.21.15.107:0/3458720227 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8c9c01c660 con 0x7f8cac0fca00 2024-03-04T13:31:16.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:16.473+0000 7f8cb2400700 1 -- 172.21.15.107:0/3458720227 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cac10f6e0 con 0x7f8cac0fca00 2024-03-04T13:31:16.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:16.474+0000 7f8cb2400700 1 -- 172.21.15.107:0/3458720227 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cac110380 con 0x7f8cac0fca00 2024-03-04T13:31:16.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:16.474+0000 7f8ca9600700 1 -- 172.21.15.107:0/3458720227 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) v1 ==== 43425+0+0 (secure 0 0 0) 0x7f8c9c013420 con 0x7f8cac0fca00 2024-03-04T13:31:16.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:16.475+0000 7f8cb2400700 1 -- 172.21.15.107:0/3458720227 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cac05f5c0 con 0x7f8cac0fca00 2024-03-04T13:31:16.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:16.475+0000 7f8ca9600700 1 --2- 172.21.15.107:0/3458720227 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7f8c90036870 0x7f8c90038d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:31:16.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:16.475+0000 7f8ca9600700 1 -- 172.21.15.107:0/3458720227 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f8c9c0499e0 con 0x7f8cac0fca00 2024-03-04T13:31:16.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:16.476+0000 7f8ca3e00700 1 --2- 172.21.15.107:0/3458720227 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7f8c90036870 0x7f8c90038d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:31:16.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:16.476+0000 7f8ca3e00700 1 --2- 172.21.15.107:0/3458720227 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7f8c90036870 0x7f8c90038d20 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8c98009990 tx=0x7f8c98006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:31:16.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:16.480+0000 7f8ca9600700 1 -- 172.21.15.107:0/3458720227 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f8c9c013b10 con 0x7f8cac0fca00 2024-03-04T13:31:16.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:16.692+0000 7f8cb2400700 1 -- 172.21.15.107:0/3458720227 --> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f8cac110f00 con 0x7f8c90036870 2024-03-04T13:31:16.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:16.698+0000 7f8ca9600700 1 -- 172.21.15.107:0/3458720227 <== mgr.14162 v2:172.21.15.107:6800/2040330534 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f8cac110f00 con 0x7f8c90036870 2024-03-04T13:31:16.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:16.700+0000 7f8cb2400700 1 -- 172.21.15.107:0/3458720227 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7f8c90036870 msgr2=0x7f8c90038d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:31:16.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:16.700+0000 7f8cb2400700 1 --2- 172.21.15.107:0/3458720227 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7f8c90036870 0x7f8c90038d20 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8c98009990 tx=0x7f8c98006e40 comp rx=0 tx=0).stop 2024-03-04T13:31:16.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:16.700+0000 7f8cb2400700 1 -- 172.21.15.107:0/3458720227 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8cac0fca00 msgr2=0x7f8cac10efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:31:16.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:16.700+0000 7f8cb2400700 1 --2- 172.21.15.107:0/3458720227 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8cac0fca00 0x7f8cac10efa0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c009b80 tx=0x7f8c9c009e30 comp rx=0 tx=0).stop 2024-03-04T13:31:16.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:16.700+0000 7f8cb2400700 1 -- 172.21.15.107:0/3458720227 shutdown_connections 2024-03-04T13:31:16.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:16.700+0000 7f8cb2400700 1 --2- 172.21.15.107:0/3458720227 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7f8c90036870 0x7f8c90038d20 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:31:16.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:16.700+0000 7f8cb2400700 1 --2- 172.21.15.107:0/3458720227 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8cac0fca00 0x7f8cac10efa0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:31:16.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:16.700+0000 7f8cb2400700 1 -- 172.21.15.107:0/3458720227 >> 172.21.15.107:0/3458720227 conn(0x7f8cac0f80c0 msgr2=0x7f8cac0f8f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:31:16.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:16.700+0000 7f8cb2400700 1 -- 172.21.15.107:0/3458720227 shutdown_connections 2024-03-04T13:31:16.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:16.700+0000 7f8cb2400700 1 -- 172.21.15.107:0/3458720227 wait complete. 2024-03-04T13:31:17.822 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi169 2024-03-04T13:31:17.822 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-03-04T13:31:17.822 DEBUG:teuthology.orchestra.run.smithi169:> dd of=/etc/ceph/ceph.conf 2024-03-04T13:31:17.880 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-03-04T13:31:17.880 DEBUG:teuthology.orchestra.run.smithi169:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-04T13:31:17.974 INFO:tasks.cephadm:Adding host smithi169 to orchestrator... 2024-03-04T13:31:17.974 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 -- ceph orch host add smithi169 2024-03-04T13:31:18.035 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:31:17 smithi107 ceph-mon[28739]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:31:18.035 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:31:17 smithi107 ceph-mon[28739]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:31:18.035 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:31:17 smithi107 ceph-mon[28739]: from='mgr.14162 172.21.15.107:0/1450595538' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:31:18.599 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/mon.smithi107/config 2024-03-04T13:31:19.633 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:31:19 smithi107 ceph-mon[28739]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:31:19.633 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:31:19 smithi107 ceph-mon[28739]: from='mgr.14162 172.21.15.107:0/1450595538' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:31:19.633 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:31:19 smithi107 ceph-mon[28739]: from='mgr.14162 172.21.15.107:0/1450595538' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:31:19.633 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:31:19 smithi107 ceph-mon[28739]: from='mgr.14162 172.21.15.107:0/1450595538' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:31:19.634 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:31:19 smithi107 ceph-mon[28739]: from='mgr.14162 172.21.15.107:0/1450595538' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:31:19.634 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:31:19 smithi107 ceph-mon[28739]: Deploying daemon node-exporter.smithi107 on smithi107 2024-03-04T13:31:20.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:20.674+0000 7fcdc5a00700 1 -- 172.21.15.107:0/3418858347 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcdc00fd710 msgr2=0x7fcdc00ffaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:31:20.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:20.674+0000 7fcdc5a00700 1 --2- 172.21.15.107:0/3418858347 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcdc00fd710 0x7fcdc00ffaf0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fcdb00048c0 tx=0x7fcdb0009e30 comp rx=0 tx=0).stop 2024-03-04T13:31:20.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:20.674+0000 7fcdc5a00700 1 -- 172.21.15.107:0/3418858347 shutdown_connections 2024-03-04T13:31:20.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:20.674+0000 7fcdc5a00700 1 --2- 172.21.15.107:0/3418858347 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcdc00fd710 0x7fcdc00ffaf0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:31:20.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:20.674+0000 7fcdc5a00700 1 -- 172.21.15.107:0/3418858347 >> 172.21.15.107:0/3418858347 conn(0x7fcdc00f7190 msgr2=0x7fcdc00f95c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:31:20.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:20.674+0000 7fcdc5a00700 1 -- 172.21.15.107:0/3418858347 shutdown_connections 2024-03-04T13:31:20.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:20.674+0000 7fcdc5a00700 1 -- 172.21.15.107:0/3418858347 wait complete. 2024-03-04T13:31:20.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:20.675+0000 7fcdc5a00700 1 Processor -- start 2024-03-04T13:31:20.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:20.675+0000 7fcdc5a00700 1 -- start start 2024-03-04T13:31:20.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:20.676+0000 7fcdc5a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcdc00fd710 0x7fcdc010f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:31:20.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:20.676+0000 7fcdc5a00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc010f5e0 con 0x7fcdc00fd710 2024-03-04T13:31:20.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:20.676+0000 7fcdbea00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcdc00fd710 0x7fcdc010f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:31:20.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:20.676+0000 7fcdbea00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcdc00fd710 0x7fcdc010f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39032/0 (socket says 172.21.15.107:39032) 2024-03-04T13:31:20.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:20.676+0000 7fcdbea00700 1 -- 172.21.15.107:0/3199823396 learned_addr learned my addr 172.21.15.107:0/3199823396 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:31:20.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:20.676+0000 7fcdbea00700 1 -- 172.21.15.107:0/3199823396 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdb0005040 con 0x7fcdc00fd710 2024-03-04T13:31:20.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:20.677+0000 7fcdbea00700 1 --2- 172.21.15.107:0/3199823396 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcdc00fd710 0x7fcdc010f0a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fcdb0009b80 tx=0x7fcdb0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:31:20.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:20.677+0000 7fcdb6a00700 1 -- 172.21.15.107:0/3199823396 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcdb000b5c0 con 0x7fcdc00fd710 2024-03-04T13:31:20.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:20.677+0000 7fcdb6a00700 1 -- 172.21.15.107:0/3199823396 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcdb000be70 con 0x7fcdc00fd710 2024-03-04T13:31:20.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:20.677+0000 7fcdb6a00700 1 -- 172.21.15.107:0/3199823396 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcdb001c660 con 0x7fcdc00fd710 2024-03-04T13:31:20.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:20.677+0000 7fcdc5a00700 1 -- 172.21.15.107:0/3199823396 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdc010f7e0 con 0x7fcdc00fd710 2024-03-04T13:31:20.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:20.677+0000 7fcdc5a00700 1 -- 172.21.15.107:0/3199823396 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdc0110480 con 0x7fcdc00fd710 2024-03-04T13:31:20.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:20.678+0000 7fcdc5a00700 1 -- 172.21.15.107:0/3199823396 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdc005f5c0 con 0x7fcdc00fd710 2024-03-04T13:31:20.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:20.679+0000 7fcdb6a00700 1 -- 172.21.15.107:0/3199823396 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) v1 ==== 43425+0+0 (secure 0 0 0) 0x7fcdb0004d30 con 0x7fcdc00fd710 2024-03-04T13:31:20.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:20.680+0000 7fcdb6a00700 1 --2- 172.21.15.107:0/3199823396 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7fcda40368c0 0x7fcda4038d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:31:20.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:20.680+0000 7fcdb6a00700 1 -- 172.21.15.107:0/3199823396 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fcdb0049b40 con 0x7fcdc00fd710 2024-03-04T13:31:20.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:20.680+0000 7fcdbe000700 1 --2- 172.21.15.107:0/3199823396 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7fcda40368c0 0x7fcda4038d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:31:20.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:20.681+0000 7fcdbe000700 1 --2- 172.21.15.107:0/3199823396 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7fcda40368c0 0x7fcda4038d70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fcdac009990 tx=0x7fcdac006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:31:20.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:20.685+0000 7fcdb6a00700 1 -- 172.21.15.107:0/3199823396 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fcdb0025400 con 0x7fcdc00fd710 2024-03-04T13:31:20.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:20.913+0000 7fcdc5a00700 1 -- 172.21.15.107:0/3199823396 --> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi169", "target": ["mon-mgr", ""]}) v1 -- 0x7fcdc0111010 con 0x7fcda40368c0 2024-03-04T13:31:21.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:31:21 smithi107 ceph-mon[28739]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:31:21.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:31:21 smithi107 ceph-mon[28739]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi169", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:31:21.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:31:21 smithi107 ceph-mon[28739]: from='mgr.14162 172.21.15.107:0/1450595538' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:31:23.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:31:23 smithi107 ceph-mon[28739]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:31:23.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:31:23 smithi107 ceph-mon[28739]: Deploying cephadm binary to smithi169 2024-03-04T13:31:25.368 INFO:teuthology.orchestra.run.smithi107.stdout:Added host 'smithi169' with addr '172.21.15.169' 2024-03-04T13:31:25.369 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:25.373+0000 7fcdb6a00700 1 -- 172.21.15.107:0/3199823396 <== mgr.14162 v2:172.21.15.107:6800/2040330534 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fcdc0111010 con 0x7fcda40368c0 2024-03-04T13:31:25.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:25.378+0000 7fcdc5a00700 1 -- 172.21.15.107:0/3199823396 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7fcda40368c0 msgr2=0x7fcda4038d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:31:25.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:25.378+0000 7fcdc5a00700 1 --2- 172.21.15.107:0/3199823396 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7fcda40368c0 0x7fcda4038d70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fcdac009990 tx=0x7fcdac006e40 comp rx=0 tx=0).stop 2024-03-04T13:31:25.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:25.378+0000 7fcdc5a00700 1 -- 172.21.15.107:0/3199823396 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcdc00fd710 msgr2=0x7fcdc010f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:31:25.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:25.378+0000 7fcdc5a00700 1 --2- 172.21.15.107:0/3199823396 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcdc00fd710 0x7fcdc010f0a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fcdb0009b80 tx=0x7fcdb0009e30 comp rx=0 tx=0).stop 2024-03-04T13:31:25.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:25.378+0000 7fcdc5a00700 1 -- 172.21.15.107:0/3199823396 shutdown_connections 2024-03-04T13:31:25.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:25.378+0000 7fcdc5a00700 1 --2- 172.21.15.107:0/3199823396 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7fcda40368c0 0x7fcda4038d70 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:31:25.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:25.378+0000 7fcdc5a00700 1 --2- 172.21.15.107:0/3199823396 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcdc00fd710 0x7fcdc010f0a0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:31:25.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:25.378+0000 7fcdc5a00700 1 -- 172.21.15.107:0/3199823396 >> 172.21.15.107:0/3199823396 conn(0x7fcdc00f7190 msgr2=0x7fcdc00f7e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:31:25.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:25.379+0000 7fcdc5a00700 1 -- 172.21.15.107:0/3199823396 shutdown_connections 2024-03-04T13:31:25.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:25.379+0000 7fcdc5a00700 1 -- 172.21.15.107:0/3199823396 wait complete. 2024-03-04T13:31:25.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:31:25 smithi107 ceph-mon[28739]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:31:25.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:31:25 smithi107 ceph-mon[28739]: from='mgr.14162 172.21.15.107:0/1450595538' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:31:25.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:31:25 smithi107 ceph-mon[28739]: from='mgr.14162 172.21.15.107:0/1450595538' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:31:25.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:31:25 smithi107 ceph-mon[28739]: from='mgr.14162 172.21.15.107:0/1450595538' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:31:25.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:31:25 smithi107 ceph-mon[28739]: from='mgr.14162 172.21.15.107:0/1450595538' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:31:25.838 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:31:25 smithi107 ceph-mon[28739]: from='mgr.14162 172.21.15.107:0/1450595538' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:31:26.463 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 -- ceph orch host ls --format=json 2024-03-04T13:31:26.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:31:26 smithi107 ceph-mon[28739]: Added host smithi169 2024-03-04T13:31:26.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:31:26 smithi107 ceph-mon[28739]: Deploying daemon alertmanager.smithi107 on smithi107 2024-03-04T13:31:26.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:31:26 smithi107 ceph-mon[28739]: from='mgr.14162 172.21.15.107:0/1450595538' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:31:26.930 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/mon.smithi107/config 2024-03-04T13:31:27.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:31:27 smithi107 ceph-mon[28739]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:31:29.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:29.678+0000 7f245be00700 1 -- 172.21.15.107:0/963128245 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f245c0fca00 msgr2=0x7f245c0fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:31:29.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:29.678+0000 7f245be00700 1 --2- 172.21.15.107:0/963128245 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f245c0fca00 0x7f245c0fce10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f244c0048c0 tx=0x7f244c009e30 comp rx=0 tx=0).stop 2024-03-04T13:31:29.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:29.678+0000 7f245be00700 1 -- 172.21.15.107:0/963128245 shutdown_connections 2024-03-04T13:31:29.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:29.678+0000 7f245be00700 1 --2- 172.21.15.107:0/963128245 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f245c0fca00 0x7f245c0fce10 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:31:29.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:29.678+0000 7f245be00700 1 -- 172.21.15.107:0/963128245 >> 172.21.15.107:0/963128245 conn(0x7f245c0f80c0 msgr2=0x7f245c0fa4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:31:29.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:29.678+0000 7f245be00700 1 -- 172.21.15.107:0/963128245 shutdown_connections 2024-03-04T13:31:29.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:29.678+0000 7f245be00700 1 -- 172.21.15.107:0/963128245 wait complete. 2024-03-04T13:31:29.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:29.679+0000 7f245be00700 1 Processor -- start 2024-03-04T13:31:29.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:29.680+0000 7f245be00700 1 -- start start 2024-03-04T13:31:29.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:29.680+0000 7f245be00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f245c0fca00 0x7f245c071e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:31:29.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:29.680+0000 7f245be00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f245c0702e0 con 0x7f245c0fca00 2024-03-04T13:31:29.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:29.680+0000 7f245aa00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f245c0fca00 0x7f245c071e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:31:29.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:29.680+0000 7f245aa00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f245c0fca00 0x7f245c071e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:53086/0 (socket says 172.21.15.107:53086) 2024-03-04T13:31:29.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:29.680+0000 7f245aa00700 1 -- 172.21.15.107:0/355789646 learned_addr learned my addr 172.21.15.107:0/355789646 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:31:29.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:29.681+0000 7f245aa00700 1 -- 172.21.15.107:0/355789646 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f244c005040 con 0x7f245c0fca00 2024-03-04T13:31:29.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:29.681+0000 7f245aa00700 1 --2- 172.21.15.107:0/355789646 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f245c0fca00 0x7f245c071e60 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f244c009b80 tx=0x7f244c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:31:29.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:29.681+0000 7f2452a00700 1 -- 172.21.15.107:0/355789646 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f244c00b5c0 con 0x7f245c0fca00 2024-03-04T13:31:29.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:29.681+0000 7f2452a00700 1 -- 172.21.15.107:0/355789646 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f244c00be70 con 0x7f245c0fca00 2024-03-04T13:31:29.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:29.681+0000 7f245be00700 1 -- 172.21.15.107:0/355789646 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f245c0704e0 con 0x7f245c0fca00 2024-03-04T13:31:29.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:29.681+0000 7f2452a00700 1 -- 172.21.15.107:0/355789646 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f244c01c660 con 0x7f245c0fca00 2024-03-04T13:31:29.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:29.682+0000 7f245be00700 1 -- 172.21.15.107:0/355789646 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f245c070950 con 0x7f245c0fca00 2024-03-04T13:31:29.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:29.683+0000 7f2452a00700 1 -- 172.21.15.107:0/355789646 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) v1 ==== 43425+0+0 (secure 0 0 0) 0x7f244c004d30 con 0x7f245c0fca00 2024-03-04T13:31:29.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:29.683+0000 7f245be00700 1 -- 172.21.15.107:0/355789646 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f245c106700 con 0x7f245c0fca00 2024-03-04T13:31:29.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:29.683+0000 7f2452a00700 1 --2- 172.21.15.107:0/355789646 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7f2440036870 0x7f2440038d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:31:29.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:29.683+0000 7f2452a00700 1 -- 172.21.15.107:0/355789646 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f244c049af0 con 0x7f245c0fca00 2024-03-04T13:31:29.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:29.684+0000 7f245a000700 1 --2- 172.21.15.107:0/355789646 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7f2440036870 0x7f2440038d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:31:29.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:29.685+0000 7f245a000700 1 --2- 172.21.15.107:0/355789646 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7f2440036870 0x7f2440038d20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2448009990 tx=0x7f2448006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:31:29.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:29.690+0000 7f2452a00700 1 -- 172.21.15.107:0/355789646 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f244c025400 con 0x7f245c0fca00 2024-03-04T13:31:29.838 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:31:29 smithi107 ceph-mon[28739]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:31:29.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:29.952+0000 7f245be00700 1 -- 172.21.15.107:0/355789646 --> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f245c02b410 con 0x7f2440036870 2024-03-04T13:31:29.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:29.955+0000 7f2452a00700 1 -- 172.21.15.107:0/355789646 <== mgr.14162 v2:172.21.15.107:6800/2040330534 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+161 (secure 0 0 0) 0x7f245c02b410 con 0x7f2440036870 2024-03-04T13:31:29.950 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:31:29.950 INFO:teuthology.orchestra.run.smithi107.stdout:[{"addr": "172.21.15.107", "hostname": "smithi107", "labels": [], "status": ""}, {"addr": "172.21.15.169", "hostname": "smithi169", "labels": [], "status": ""}] 2024-03-04T13:31:29.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:29.958+0000 7f245be00700 1 -- 172.21.15.107:0/355789646 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7f2440036870 msgr2=0x7f2440038d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:31:29.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:29.958+0000 7f245be00700 1 --2- 172.21.15.107:0/355789646 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7f2440036870 0x7f2440038d20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2448009990 tx=0x7f2448006e40 comp rx=0 tx=0).stop 2024-03-04T13:31:29.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:29.958+0000 7f245be00700 1 -- 172.21.15.107:0/355789646 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f245c0fca00 msgr2=0x7f245c071e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:31:29.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:29.958+0000 7f245be00700 1 --2- 172.21.15.107:0/355789646 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f245c0fca00 0x7f245c071e60 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f244c009b80 tx=0x7f244c009e30 comp rx=0 tx=0).stop 2024-03-04T13:31:29.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:29.958+0000 7f245be00700 1 -- 172.21.15.107:0/355789646 shutdown_connections 2024-03-04T13:31:29.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:29.958+0000 7f245be00700 1 --2- 172.21.15.107:0/355789646 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7f2440036870 0x7f2440038d20 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:31:29.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:29.958+0000 7f245be00700 1 --2- 172.21.15.107:0/355789646 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f245c0fca00 0x7f245c071e60 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:31:29.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:29.958+0000 7f245be00700 1 -- 172.21.15.107:0/355789646 >> 172.21.15.107:0/355789646 conn(0x7f245c0f80c0 msgr2=0x7f245c0f8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:31:29.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:29.958+0000 7f245be00700 1 -- 172.21.15.107:0/355789646 shutdown_connections 2024-03-04T13:31:29.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:29.959+0000 7f245be00700 1 -- 172.21.15.107:0/355789646 wait complete. 2024-03-04T13:31:30.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:31:30 smithi107 ceph-mon[28739]: from='client.14190 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-04T13:31:31.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:31:31 smithi107 ceph-mon[28739]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:31:31.981 INFO:tasks.cephadm:Setting crush tunables to default 2024-03-04T13:31:31.982 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 -- ceph osd crush tunables default 2024-03-04T13:31:33.780 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/mon.smithi107/config 2024-03-04T13:31:33.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:31:33 smithi107 ceph-mon[28739]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:31:35.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:31:35 smithi107 ceph-mon[28739]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:31:37.838 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:31:37 smithi107 ceph-mon[28739]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:31:38.248 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:38.244+0000 7fbddfe00700 1 -- 172.21.15.107:0/2776655716 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbdd80fcbc0 msgr2=0x7fbdd80fcfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:31:38.248 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:38.244+0000 7fbddfe00700 1 --2- 172.21.15.107:0/2776655716 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbdd80fcbc0 0x7fbdd80fcfd0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc0048c0 tx=0x7fbdcc009e30 comp rx=0 tx=0).stop 2024-03-04T13:31:38.248 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:38.244+0000 7fbddfe00700 1 -- 172.21.15.107:0/2776655716 shutdown_connections 2024-03-04T13:31:38.248 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:38.244+0000 7fbddfe00700 1 --2- 172.21.15.107:0/2776655716 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbdd80fcbc0 0x7fbdd80fcfd0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:31:38.248 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:38.244+0000 7fbddfe00700 1 -- 172.21.15.107:0/2776655716 >> 172.21.15.107:0/2776655716 conn(0x7fbdd80f82a0 msgr2=0x7fbdd80fa6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:31:38.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:38.244+0000 7fbddfe00700 1 -- 172.21.15.107:0/2776655716 shutdown_connections 2024-03-04T13:31:38.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:38.244+0000 7fbddfe00700 1 -- 172.21.15.107:0/2776655716 wait complete. 2024-03-04T13:31:38.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:38.245+0000 7fbddfe00700 1 Processor -- start 2024-03-04T13:31:38.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:38.245+0000 7fbddfe00700 1 -- start start 2024-03-04T13:31:38.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:38.246+0000 7fbddfe00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbdd80fcbc0 0x7fbdd8071e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:31:38.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:38.246+0000 7fbddfe00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd80702e0 con 0x7fbdd80fcbc0 2024-03-04T13:31:38.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:38.246+0000 7fbddd400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbdd80fcbc0 0x7fbdd8071e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:31:38.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:38.246+0000 7fbddd400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbdd80fcbc0 0x7fbdd8071e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48892/0 (socket says 172.21.15.107:48892) 2024-03-04T13:31:38.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:38.246+0000 7fbddd400700 1 -- 172.21.15.107:0/1291071750 learned_addr learned my addr 172.21.15.107:0/1291071750 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:31:38.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:38.246+0000 7fbddd400700 1 -- 172.21.15.107:0/1291071750 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdcc005040 con 0x7fbdd80fcbc0 2024-03-04T13:31:38.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:38.246+0000 7fbddd400700 1 --2- 172.21.15.107:0/1291071750 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbdd80fcbc0 0x7fbdd8071e60 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc009b80 tx=0x7fbdcc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:31:38.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:38.247+0000 7fbdd5600700 1 -- 172.21.15.107:0/1291071750 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbdcc00b5c0 con 0x7fbdd80fcbc0 2024-03-04T13:31:38.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:38.247+0000 7fbdd5600700 1 -- 172.21.15.107:0/1291071750 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbdcc00be70 con 0x7fbdd80fcbc0 2024-03-04T13:31:38.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:38.247+0000 7fbdd5600700 1 -- 172.21.15.107:0/1291071750 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbdcc01c660 con 0x7fbdd80fcbc0 2024-03-04T13:31:38.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:38.247+0000 7fbddfe00700 1 -- 172.21.15.107:0/1291071750 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdd80704e0 con 0x7fbdd80fcbc0 2024-03-04T13:31:38.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:38.247+0000 7fbddfe00700 1 -- 172.21.15.107:0/1291071750 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdd80709d0 con 0x7fbdd80fcbc0 2024-03-04T13:31:38.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:38.248+0000 7fbdd5600700 1 -- 172.21.15.107:0/1291071750 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) v1 ==== 43425+0+0 (secure 0 0 0) 0x7fbdcc013420 con 0x7fbdd80fcbc0 2024-03-04T13:31:38.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:38.248+0000 7fbddfe00700 1 -- 172.21.15.107:0/1291071750 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdd8106990 con 0x7fbdd80fcbc0 2024-03-04T13:31:38.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:38.248+0000 7fbdd5600700 1 --2- 172.21.15.107:0/1291071750 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7fbdc00368c0 0x7fbdc0038d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:31:38.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:38.248+0000 7fbdd5600700 1 -- 172.21.15.107:0/1291071750 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fbdcc0499a0 con 0x7fbdd80fcbc0 2024-03-04T13:31:38.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:38.250+0000 7fbddca00700 1 --2- 172.21.15.107:0/1291071750 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7fbdc00368c0 0x7fbdc0038d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:31:38.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:38.250+0000 7fbddca00700 1 --2- 172.21.15.107:0/1291071750 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7fbdc00368c0 0x7fbdc0038d70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fbdc4009990 tx=0x7fbdc4006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:31:38.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:38.255+0000 7fbdd5600700 1 -- 172.21.15.107:0/1291071750 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fbdcc013b10 con 0x7fbdd80fcbc0 2024-03-04T13:31:38.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:38.475+0000 7fbddfe00700 1 -- 172.21.15.107:0/1291071750 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fbdd804bed0 con 0x7fbdd80fcbc0 2024-03-04T13:31:38.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:38.605+0000 7fbdd5600700 1 -- 172.21.15.107:0/1291071750 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7fbdcc011370 con 0x7fbdd80fcbc0 2024-03-04T13:31:38.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:38.610+0000 7fbddfe00700 1 -- 172.21.15.107:0/1291071750 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7fbdc00368c0 msgr2=0x7fbdc0038d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:31:38.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:38.610+0000 7fbddfe00700 1 --2- 172.21.15.107:0/1291071750 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7fbdc00368c0 0x7fbdc0038d70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fbdc4009990 tx=0x7fbdc4006e40 comp rx=0 tx=0).stop 2024-03-04T13:31:38.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:38.610+0000 7fbddfe00700 1 -- 172.21.15.107:0/1291071750 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbdd80fcbc0 msgr2=0x7fbdd8071e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:31:38.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:38.610+0000 7fbddfe00700 1 --2- 172.21.15.107:0/1291071750 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbdd80fcbc0 0x7fbdd8071e60 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc009b80 tx=0x7fbdcc009e30 comp rx=0 tx=0).stop 2024-03-04T13:31:38.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:38.610+0000 7fbddfe00700 1 -- 172.21.15.107:0/1291071750 shutdown_connections 2024-03-04T13:31:38.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:38.610+0000 7fbddfe00700 1 --2- 172.21.15.107:0/1291071750 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7fbdc00368c0 0x7fbdc0038d70 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:31:38.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:38.610+0000 7fbddfe00700 1 --2- 172.21.15.107:0/1291071750 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbdd80fcbc0 0x7fbdd8071e60 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:31:38.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:38.610+0000 7fbddfe00700 1 -- 172.21.15.107:0/1291071750 >> 172.21.15.107:0/1291071750 conn(0x7fbdd80f82a0 msgr2=0x7fbdd80f8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:31:38.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:38.610+0000 7fbddfe00700 1 -- 172.21.15.107:0/1291071750 shutdown_connections 2024-03-04T13:31:38.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:31:38.610+0000 7fbddfe00700 1 -- 172.21.15.107:0/1291071750 wait complete. 2024-03-04T13:31:38.606 INFO:teuthology.orchestra.run.smithi107.stderr:adjusted tunables profile to default 2024-03-04T13:31:38.629 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:31:38 smithi107 ceph-mon[28739]: from='client.? 172.21.15.107:0/1291071750' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-04T13:31:38.629 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:31:38 smithi107 ceph-mon[28739]: from='mgr.14162 172.21.15.107:0/1450595538' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:31:38.629 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:31:38 smithi107 ceph-mon[28739]: from='mgr.14162 172.21.15.107:0/1450595538' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:31:38.629 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:31:38 smithi107 ceph-mon[28739]: from='mgr.14162 172.21.15.107:0/1450595538' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:31:38.629 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:31:38 smithi107 ceph-mon[28739]: from='mgr.14162 172.21.15.107:0/1450595538' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:31:39.433 INFO:tasks.cephadm:Adding mon.smithi107 on smithi107 2024-03-04T13:31:39.433 INFO:tasks.cephadm:Adding mon.smithi169 on smithi169 2024-03-04T13:31:39.433 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 -- ceph orch apply mon '2;smithi107:172.21.15.107=smithi107;smithi169:172.21.15.169=smithi169' 2024-03-04T13:31:39.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:31:39 smithi107 ceph-mon[28739]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:31:39.838 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:31:39 smithi107 ceph-mon[28739]: from='client.? 172.21.15.107:0/1291071750' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-04T13:31:39.838 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:31:39 smithi107 ceph-mon[28739]: osdmap e4: 0 total, 0 up, 0 in 2024-03-04T13:31:39.838 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:31:39 smithi107 ceph-mon[28739]: from='mgr.14162 172.21.15.107:0/1450595538' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:31:39.838 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:31:39 smithi107 ceph-mon[28739]: from='mgr.14162 172.21.15.107:0/1450595538' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:31:39.838 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:31:39 smithi107 ceph-mon[28739]: from='mgr.14162 172.21.15.107:0/1450595538' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-03-04T13:31:39.838 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:31:39 smithi107 ceph-mon[28739]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-03-04T13:31:39.838 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:31:39 smithi107 ceph-mon[28739]: from='mgr.14162 172.21.15.107:0/1450595538' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:31:39.838 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:31:39 smithi107 ceph-mon[28739]: Deploying daemon grafana.smithi107 on smithi107 2024-03-04T13:31:39.902 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-04T13:31:40.021 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-04T13:31:42.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:31:41 smithi107 ceph-mon[28739]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:31:42.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:31:41 smithi107 ceph-mon[28739]: from='mgr.14162 172.21.15.107:0/1450595538' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:31:42.661 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:42.666+0000 7f601a200700 1 -- 172.21.15.169:0/3195015405 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6014071f10 msgr2=0x7f6014070380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:31:42.661 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:42.666+0000 7f601a200700 1 --2- 172.21.15.169:0/3195015405 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6014071f10 0x7f6014070380 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f60040048c0 tx=0x7f6004009e30 comp rx=0 tx=0).stop 2024-03-04T13:31:42.661 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:42.667+0000 7f601a200700 1 -- 172.21.15.169:0/3195015405 shutdown_connections 2024-03-04T13:31:42.661 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:42.667+0000 7f601a200700 1 --2- 172.21.15.169:0/3195015405 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6014071f10 0x7f6014070380 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:31:42.661 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:42.667+0000 7f601a200700 1 -- 172.21.15.169:0/3195015405 >> 172.21.15.169:0/3195015405 conn(0x7f60140f82a0 msgr2=0x7f60140fa6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:31:42.662 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:42.667+0000 7f601a200700 1 -- 172.21.15.169:0/3195015405 shutdown_connections 2024-03-04T13:31:42.662 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:42.667+0000 7f601a200700 1 -- 172.21.15.169:0/3195015405 wait complete. 2024-03-04T13:31:42.662 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:42.668+0000 7f601a200700 1 Processor -- start 2024-03-04T13:31:42.663 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:42.668+0000 7f601a200700 1 -- start start 2024-03-04T13:31:42.663 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:42.668+0000 7f601a200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6014071f10 0x7f601410f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:31:42.663 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:42.668+0000 7f601a200700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f601410f5c0 con 0x7f6014071f10 2024-03-04T13:31:42.663 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:42.669+0000 7f6013400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6014071f10 0x7f601410f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:31:42.663 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:42.669+0000 7f6013400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6014071f10 0x7f601410f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.169:41496/0 (socket says 172.21.15.169:41496) 2024-03-04T13:31:42.663 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:42.669+0000 7f6013400700 1 -- 172.21.15.169:0/616107046 learned_addr learned my addr 172.21.15.169:0/616107046 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-03-04T13:31:42.663 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:42.670+0000 7f6013400700 1 -- 172.21.15.169:0/616107046 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6004005040 con 0x7f6014071f10 2024-03-04T13:31:42.664 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:42.670+0000 7f6013400700 1 --2- 172.21.15.169:0/616107046 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6014071f10 0x7f601410f080 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6004009680 tx=0x7f6004009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:31:42.664 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:42.670+0000 7f600b400700 1 -- 172.21.15.169:0/616107046 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f600400b5c0 con 0x7f6014071f10 2024-03-04T13:31:42.664 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:42.670+0000 7f601a200700 1 -- 172.21.15.169:0/616107046 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f601410f7c0 con 0x7f6014071f10 2024-03-04T13:31:42.664 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:42.671+0000 7f600b400700 1 -- 172.21.15.169:0/616107046 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6004004e60 con 0x7f6014071f10 2024-03-04T13:31:42.665 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:42.671+0000 7f600b400700 1 -- 172.21.15.169:0/616107046 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f600401c590 con 0x7f6014071f10 2024-03-04T13:31:42.665 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:42.671+0000 7f601a200700 1 -- 172.21.15.169:0/616107046 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6014110460 con 0x7f6014071f10 2024-03-04T13:31:42.665 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:42.672+0000 7f601a200700 1 -- 172.21.15.169:0/616107046 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f601405f5c0 con 0x7f6014071f10 2024-03-04T13:31:42.666 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:42.672+0000 7f600b400700 1 -- 172.21.15.169:0/616107046 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) v1 ==== 43425+0+0 (secure 0 0 0) 0x7f6004013a80 con 0x7f6014071f10 2024-03-04T13:31:42.666 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:42.673+0000 7f600b400700 1 --2- 172.21.15.169:0/616107046 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7f5ff80368c0 0x7f5ff8038d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:31:42.666 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:42.673+0000 7f600b400700 1 -- 172.21.15.169:0/616107046 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f600404a080 con 0x7f6014071f10 2024-03-04T13:31:42.667 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:42.673+0000 7f6012a00700 1 --2- 172.21.15.169:0/616107046 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7f5ff80368c0 0x7f5ff8038d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:31:42.668 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:42.675+0000 7f6012a00700 1 --2- 172.21.15.169:0/616107046 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7f5ff80368c0 0x7f5ff8038d70 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f6000009990 tx=0x7f6000006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:31:42.686 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:42.692+0000 7f600b400700 1 -- 172.21.15.169:0/616107046 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f6004011b20 con 0x7f6014071f10 2024-03-04T13:31:42.906 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:42.912+0000 7f601a200700 1 -- 172.21.15.169:0/616107046 --> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi107:172.21.15.107=smithi107;smithi169:172.21.15.169=smithi169", "target": ["mon-mgr", ""]}) v1 -- 0x7f60141110f0 con 0x7f5ff80368c0 2024-03-04T13:31:42.923 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:42.929+0000 7f600b400700 1 -- 172.21.15.169:0/616107046 <== mgr.14162 v2:172.21.15.107:6800/2040330534 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f60141110f0 con 0x7f5ff80368c0 2024-03-04T13:31:42.923 INFO:teuthology.orchestra.run.smithi169.stdout:Scheduled mon update... 2024-03-04T13:31:42.926 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:42.931+0000 7f601a200700 1 -- 172.21.15.169:0/616107046 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7f5ff80368c0 msgr2=0x7f5ff8038d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:31:42.926 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:42.931+0000 7f601a200700 1 --2- 172.21.15.169:0/616107046 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7f5ff80368c0 0x7f5ff8038d70 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f6000009990 tx=0x7f6000006e40 comp rx=0 tx=0).stop 2024-03-04T13:31:42.926 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:42.931+0000 7f601a200700 1 -- 172.21.15.169:0/616107046 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6014071f10 msgr2=0x7f601410f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:31:42.926 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:42.931+0000 7f601a200700 1 --2- 172.21.15.169:0/616107046 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6014071f10 0x7f601410f080 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6004009680 tx=0x7f6004009e30 comp rx=0 tx=0).stop 2024-03-04T13:31:42.926 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:42.932+0000 7f601a200700 1 -- 172.21.15.169:0/616107046 shutdown_connections 2024-03-04T13:31:42.926 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:42.932+0000 7f601a200700 1 --2- 172.21.15.169:0/616107046 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7f5ff80368c0 0x7f5ff8038d70 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:31:42.926 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:42.932+0000 7f601a200700 1 --2- 172.21.15.169:0/616107046 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6014071f10 0x7f601410f080 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:31:42.926 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:42.932+0000 7f601a200700 1 -- 172.21.15.169:0/616107046 >> 172.21.15.169:0/616107046 conn(0x7f60140f82a0 msgr2=0x7f60140f8f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:31:42.926 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:42.932+0000 7f601a200700 1 -- 172.21.15.169:0/616107046 shutdown_connections 2024-03-04T13:31:42.926 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:42.932+0000 7f601a200700 1 -- 172.21.15.169:0/616107046 wait complete. 2024-03-04T13:31:43.629 DEBUG:teuthology.orchestra.run.smithi169:mon.smithi169> sudo journalctl -f -n 0 -u ceph-1fc09a58-da2b-11ee-95c6-87774f69a715@mon.smithi169.service 2024-03-04T13:31:43.635 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-04T13:31:43.635 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 -- ceph mon dump -f json 2024-03-04T13:31:44.089 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-04T13:31:44.194 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-04T13:31:44.337 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:31:43 smithi107 ceph-mon[28739]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:31:44.337 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:31:43 smithi107 ceph-mon[28739]: from='client.14194 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi107:172.21.15.107=smithi107;smithi169:172.21.15.169=smithi169", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:31:44.338 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:31:43 smithi107 ceph-mon[28739]: Saving service mon spec with placement smithi107:172.21.15.107=smithi107;smithi169:172.21.15.169=smithi169;count:2 2024-03-04T13:31:44.338 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:31:43 smithi107 ceph-mon[28739]: from='mgr.14162 172.21.15.107:0/1450595538' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:31:46.019 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:46.025+0000 7f9705000700 1 -- 172.21.15.169:0/2698247447 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97000fedd0 msgr2=0x7f97000ff1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:31:46.020 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:46.025+0000 7f9705000700 1 --2- 172.21.15.169:0/2698247447 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97000fedd0 0x7f97000ff1e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f96f00048c0 tx=0x7f96f0009e30 comp rx=0 tx=0).stop 2024-03-04T13:31:46.020 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:46.025+0000 7f9705000700 1 -- 172.21.15.169:0/2698247447 shutdown_connections 2024-03-04T13:31:46.020 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:46.025+0000 7f9705000700 1 --2- 172.21.15.169:0/2698247447 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97000fedd0 0x7f97000ff1e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:31:46.020 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:46.025+0000 7f9705000700 1 -- 172.21.15.169:0/2698247447 >> 172.21.15.169:0/2698247447 conn(0x7f97000fa5a0 msgr2=0x7f97000fc980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:31:46.020 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:46.025+0000 7f9705000700 1 -- 172.21.15.169:0/2698247447 shutdown_connections 2024-03-04T13:31:46.020 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:46.025+0000 7f9705000700 1 -- 172.21.15.169:0/2698247447 wait complete. 2024-03-04T13:31:46.023 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:46.026+0000 7f9705000700 1 Processor -- start 2024-03-04T13:31:46.023 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:46.026+0000 7f9705000700 1 -- start start 2024-03-04T13:31:46.023 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:46.026+0000 7f9705000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97000fedd0 0x7f970010f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:31:46.023 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:46.027+0000 7f9705000700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f970010f650 con 0x7f97000fedd0 2024-03-04T13:31:46.023 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:46.027+0000 7f96fde00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97000fedd0 0x7f970010f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:31:46.023 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:46.027+0000 7f96fde00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97000fedd0 0x7f970010f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.169:41624/0 (socket says 172.21.15.169:41624) 2024-03-04T13:31:46.023 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:46.027+0000 7f96fde00700 1 -- 172.21.15.169:0/1041737836 learned_addr learned my addr 172.21.15.169:0/1041737836 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-03-04T13:31:46.023 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:46.028+0000 7f96fde00700 1 -- 172.21.15.169:0/1041737836 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96f0005040 con 0x7f97000fedd0 2024-03-04T13:31:46.023 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:46.028+0000 7f96fde00700 1 --2- 172.21.15.169:0/1041737836 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97000fedd0 0x7f970010f110 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f96f0009680 tx=0x7f96f0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:31:46.024 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:46.029+0000 7f96f6000700 1 -- 172.21.15.169:0/1041737836 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f96f000bed0 con 0x7f97000fedd0 2024-03-04T13:31:46.024 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:46.029+0000 7f9705000700 1 -- 172.21.15.169:0/1041737836 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f970010f850 con 0x7f97000fedd0 2024-03-04T13:31:46.024 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:46.029+0000 7f96f6000700 1 -- 172.21.15.169:0/1041737836 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f96f000b5c0 con 0x7f97000fedd0 2024-03-04T13:31:46.024 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:46.029+0000 7f9705000700 1 -- 172.21.15.169:0/1041737836 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97001104f0 con 0x7f97000fedd0 2024-03-04T13:31:46.024 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:46.029+0000 7f96f6000700 1 -- 172.21.15.169:0/1041737836 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f96f00255c0 con 0x7f97000fedd0 2024-03-04T13:31:46.024 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:46.030+0000 7f9705000700 1 -- 172.21.15.169:0/1041737836 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f970005f5c0 con 0x7f97000fedd0 2024-03-04T13:31:46.024 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:46.030+0000 7f96f6000700 1 -- 172.21.15.169:0/1041737836 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) v1 ==== 43425+0+0 (secure 0 0 0) 0x7f96f0004d30 con 0x7f97000fedd0 2024-03-04T13:31:46.025 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:46.031+0000 7f96f6000700 1 --2- 172.21.15.169:0/1041737836 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7f96e40368c0 0x7f96e4038d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:31:46.025 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:46.031+0000 7f96f6000700 1 -- 172.21.15.169:0/1041737836 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f96f004a1e0 con 0x7f97000fedd0 2024-03-04T13:31:46.025 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:46.031+0000 7f96fd400700 1 --2- 172.21.15.169:0/1041737836 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7f96e40368c0 0x7f96e4038d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:31:46.026 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:46.032+0000 7f96fd400700 1 --2- 172.21.15.169:0/1041737836 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7f96e40368c0 0x7f96e4038d70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f96ec009990 tx=0x7f96ec006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:31:46.052 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:46.058+0000 7f96f6000700 1 -- 172.21.15.169:0/1041737836 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f96f002d920 con 0x7f97000fedd0 2024-03-04T13:31:46.337 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:31:45 smithi107 ceph-mon[28739]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:31:46.338 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:46.345+0000 7f9705000700 1 -- 172.21.15.169:0/1041737836 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f9700104b90 con 0x7f97000fedd0 2024-03-04T13:31:46.340 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:46.346+0000 7f96f6000700 1 -- 172.21.15.169:0/1041737836 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7f96f001b390 con 0x7f97000fedd0 2024-03-04T13:31:46.340 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-03-04T13:31:46.340 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":1,"fsid":"1fc09a58-da2b-11ee-95c6-87774f69a715","modified":"2024-03-04T13:29:38.149007Z","created":"2024-03-04T13:29:38.149007Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi107","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:3300","nonce":0},{"type":"v1","addr":"172.21.15.107:6789","nonce":0}]},"addr":"172.21.15.107:6789/0","public_addr":"172.21.15.107:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-04T13:31:46.343 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:46.348+0000 7f9705000700 1 -- 172.21.15.169:0/1041737836 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7f96e40368c0 msgr2=0x7f96e4038d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:31:46.343 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:46.348+0000 7f9705000700 1 --2- 172.21.15.169:0/1041737836 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7f96e40368c0 0x7f96e4038d70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f96ec009990 tx=0x7f96ec006e40 comp rx=0 tx=0).stop 2024-03-04T13:31:46.343 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:46.348+0000 7f9705000700 1 -- 172.21.15.169:0/1041737836 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97000fedd0 msgr2=0x7f970010f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:31:46.343 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:46.348+0000 7f9705000700 1 --2- 172.21.15.169:0/1041737836 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97000fedd0 0x7f970010f110 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f96f0009680 tx=0x7f96f0009e30 comp rx=0 tx=0).stop 2024-03-04T13:31:46.343 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:46.349+0000 7f9705000700 1 -- 172.21.15.169:0/1041737836 shutdown_connections 2024-03-04T13:31:46.343 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:46.349+0000 7f9705000700 1 --2- 172.21.15.169:0/1041737836 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7f96e40368c0 0x7f96e4038d70 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:31:46.343 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:46.349+0000 7f9705000700 1 --2- 172.21.15.169:0/1041737836 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97000fedd0 0x7f970010f110 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:31:46.343 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:46.349+0000 7f9705000700 1 -- 172.21.15.169:0/1041737836 >> 172.21.15.169:0/1041737836 conn(0x7f97000fa5a0 msgr2=0x7f97000fc330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:31:46.343 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:46.349+0000 7f9705000700 1 -- 172.21.15.169:0/1041737836 shutdown_connections 2024-03-04T13:31:46.344 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:46.349+0000 7f9705000700 1 -- 172.21.15.169:0/1041737836 wait complete. 2024-03-04T13:31:46.345 INFO:teuthology.orchestra.run.smithi169.stderr:dumped monmap epoch 1 2024-03-04T13:31:47.337 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:31:46 smithi107 ceph-mon[28739]: from='client.? 172.21.15.169:0/1041737836' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-04T13:31:48.034 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-04T13:31:48.034 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 -- ceph mon dump -f json 2024-03-04T13:31:48.229 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:31:47 smithi107 ceph-mon[28739]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:31:48.485 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-04T13:31:48.586 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-04T13:31:50.337 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:31:49 smithi107 ceph-mon[28739]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:31:50.437 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:50.442+0000 7f05c9600700 1 -- 172.21.15.169:0/1364842941 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05c4065df0 msgr2=0x7f05c40ff780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:31:50.438 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:50.442+0000 7f05c9600700 1 --2- 172.21.15.169:0/1364842941 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05c4065df0 0x7f05c40ff780 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f05b40048c0 tx=0x7f05b4009e30 comp rx=0 tx=0).stop 2024-03-04T13:31:50.438 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:50.443+0000 7f05c9600700 1 -- 172.21.15.169:0/1364842941 shutdown_connections 2024-03-04T13:31:50.438 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:50.443+0000 7f05c9600700 1 --2- 172.21.15.169:0/1364842941 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05c4065df0 0x7f05c40ff780 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:31:50.438 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:50.443+0000 7f05c9600700 1 -- 172.21.15.169:0/1364842941 >> 172.21.15.169:0/1364842941 conn(0x7f05c40f7190 msgr2=0x7f05c40f95c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:31:50.438 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:50.443+0000 7f05c9600700 1 -- 172.21.15.169:0/1364842941 shutdown_connections 2024-03-04T13:31:50.438 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:50.443+0000 7f05c9600700 1 -- 172.21.15.169:0/1364842941 wait complete. 2024-03-04T13:31:50.438 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:50.444+0000 7f05c9600700 1 Processor -- start 2024-03-04T13:31:50.441 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:50.444+0000 7f05c9600700 1 -- start start 2024-03-04T13:31:50.441 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:50.444+0000 7f05c9600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05c4065df0 0x7f05c410f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:31:50.441 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:50.444+0000 7f05c9600700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05c410f5d0 con 0x7f05c4065df0 2024-03-04T13:31:50.441 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:50.445+0000 7f05c2a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05c4065df0 0x7f05c410f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:31:50.441 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:50.445+0000 7f05c2a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05c4065df0 0x7f05c410f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.169:41644/0 (socket says 172.21.15.169:41644) 2024-03-04T13:31:50.441 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:50.445+0000 7f05c2a00700 1 -- 172.21.15.169:0/1339747416 learned_addr learned my addr 172.21.15.169:0/1339747416 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-03-04T13:31:50.441 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:50.445+0000 7f05c2a00700 1 -- 172.21.15.169:0/1339747416 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05b4005040 con 0x7f05c4065df0 2024-03-04T13:31:50.441 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:50.446+0000 7f05c2a00700 1 --2- 172.21.15.169:0/1339747416 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05c4065df0 0x7f05c410f090 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f05b4009b80 tx=0x7f05b4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:31:50.442 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:50.446+0000 7f05baa00700 1 -- 172.21.15.169:0/1339747416 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f05b400b5c0 con 0x7f05c4065df0 2024-03-04T13:31:50.442 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:50.446+0000 7f05baa00700 1 -- 172.21.15.169:0/1339747416 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f05b400be70 con 0x7f05c4065df0 2024-03-04T13:31:50.442 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:50.446+0000 7f05c9600700 1 -- 172.21.15.169:0/1339747416 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05c410f7d0 con 0x7f05c4065df0 2024-03-04T13:31:50.442 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:50.446+0000 7f05baa00700 1 -- 172.21.15.169:0/1339747416 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f05b401c660 con 0x7f05c4065df0 2024-03-04T13:31:50.442 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:50.446+0000 7f05c9600700 1 -- 172.21.15.169:0/1339747416 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05c4110470 con 0x7f05c4065df0 2024-03-04T13:31:50.443 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:50.447+0000 7f05c9600700 1 -- 172.21.15.169:0/1339747416 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05c405f5c0 con 0x7f05c4065df0 2024-03-04T13:31:50.443 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:50.448+0000 7f05baa00700 1 -- 172.21.15.169:0/1339747416 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) v1 ==== 43425+0+0 (secure 0 0 0) 0x7f05b4004d30 con 0x7f05c4065df0 2024-03-04T13:31:50.444 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:50.448+0000 7f05baa00700 1 --2- 172.21.15.169:0/1339747416 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7f05a80368c0 0x7f05a8038d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:31:50.444 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:50.448+0000 7f05baa00700 1 -- 172.21.15.169:0/1339747416 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f05b4049a00 con 0x7f05c4065df0 2024-03-04T13:31:50.444 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:50.449+0000 7f05c2000700 1 --2- 172.21.15.169:0/1339747416 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7f05a80368c0 0x7f05a8038d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:31:50.444 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:50.450+0000 7f05c2000700 1 --2- 172.21.15.169:0/1339747416 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7f05a80368c0 0x7f05a8038d70 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f05b0009990 tx=0x7f05b0006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:31:50.452 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:50.458+0000 7f05baa00700 1 -- 172.21.15.169:0/1339747416 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f05b4025400 con 0x7f05c4065df0 2024-03-04T13:31:50.741 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:50.747+0000 7f05c9600700 1 -- 172.21.15.169:0/1339747416 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f05c4110fc0 con 0x7f05c4065df0 2024-03-04T13:31:50.741 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:50.748+0000 7f05baa00700 1 -- 172.21.15.169:0/1339747416 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7f05b4011370 con 0x7f05c4065df0 2024-03-04T13:31:50.742 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-03-04T13:31:50.742 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":1,"fsid":"1fc09a58-da2b-11ee-95c6-87774f69a715","modified":"2024-03-04T13:29:38.149007Z","created":"2024-03-04T13:29:38.149007Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi107","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:3300","nonce":0},{"type":"v1","addr":"172.21.15.107:6789","nonce":0}]},"addr":"172.21.15.107:6789/0","public_addr":"172.21.15.107:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-04T13:31:50.744 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:50.750+0000 7f05c9600700 1 -- 172.21.15.169:0/1339747416 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7f05a80368c0 msgr2=0x7f05a8038d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:31:50.744 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:50.750+0000 7f05c9600700 1 --2- 172.21.15.169:0/1339747416 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7f05a80368c0 0x7f05a8038d70 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f05b0009990 tx=0x7f05b0006e40 comp rx=0 tx=0).stop 2024-03-04T13:31:50.744 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:50.750+0000 7f05c9600700 1 -- 172.21.15.169:0/1339747416 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05c4065df0 msgr2=0x7f05c410f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:31:50.744 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:50.750+0000 7f05c9600700 1 --2- 172.21.15.169:0/1339747416 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05c4065df0 0x7f05c410f090 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f05b4009b80 tx=0x7f05b4009e30 comp rx=0 tx=0).stop 2024-03-04T13:31:50.744 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:50.750+0000 7f05c9600700 1 -- 172.21.15.169:0/1339747416 shutdown_connections 2024-03-04T13:31:50.745 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:50.750+0000 7f05c9600700 1 --2- 172.21.15.169:0/1339747416 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7f05a80368c0 0x7f05a8038d70 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:31:50.745 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:50.750+0000 7f05c9600700 1 --2- 172.21.15.169:0/1339747416 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05c4065df0 0x7f05c410f090 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:31:50.745 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:50.750+0000 7f05c9600700 1 -- 172.21.15.169:0/1339747416 >> 172.21.15.169:0/1339747416 conn(0x7f05c40f7190 msgr2=0x7f05c40f95c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:31:50.745 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:50.750+0000 7f05c9600700 1 -- 172.21.15.169:0/1339747416 shutdown_connections 2024-03-04T13:31:50.745 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:50.750+0000 7f05c9600700 1 -- 172.21.15.169:0/1339747416 wait complete. 2024-03-04T13:31:50.745 INFO:teuthology.orchestra.run.smithi169.stderr:dumped monmap epoch 1 2024-03-04T13:31:51.755 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:31:51 smithi107 ceph-mon[28739]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:31:51.756 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:31:51 smithi107 ceph-mon[28739]: from='client.? 172.21.15.169:0/1339747416' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-04T13:31:52.395 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-04T13:31:52.395 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 -- ceph mon dump -f json 2024-03-04T13:31:52.830 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-04T13:31:52.926 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-04T13:31:53.586 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:31:53 smithi107 ceph-mon[28739]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:31:54.647 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:54.652+0000 7fee61400700 1 -- 172.21.15.169:0/1941894083 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee5c1031d0 msgr2=0x7fee5c1035e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:31:54.647 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:54.652+0000 7fee61400700 1 --2- 172.21.15.169:0/1941894083 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee5c1031d0 0x7fee5c1035e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fee4c0048c0 tx=0x7fee4c009e30 comp rx=0 tx=0).stop 2024-03-04T13:31:54.647 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:54.652+0000 7fee61400700 1 -- 172.21.15.169:0/1941894083 shutdown_connections 2024-03-04T13:31:54.647 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:54.652+0000 7fee61400700 1 --2- 172.21.15.169:0/1941894083 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee5c1031d0 0x7fee5c1035e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:31:54.647 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:54.652+0000 7fee61400700 1 -- 172.21.15.169:0/1941894083 >> 172.21.15.169:0/1941894083 conn(0x7fee5c0722d0 msgr2=0x7fee5c0746e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:31:54.647 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:54.653+0000 7fee61400700 1 -- 172.21.15.169:0/1941894083 shutdown_connections 2024-03-04T13:31:54.647 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:54.653+0000 7fee61400700 1 -- 172.21.15.169:0/1941894083 wait complete. 2024-03-04T13:31:54.648 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:54.654+0000 7fee61400700 1 Processor -- start 2024-03-04T13:31:54.648 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:54.654+0000 7fee61400700 1 -- start start 2024-03-04T13:31:54.648 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:54.655+0000 7fee61400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee5c1031d0 0x7fee5c1134e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:31:54.648 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:54.655+0000 7fee61400700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee5c113a20 con 0x7fee5c1031d0 2024-03-04T13:31:54.649 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:54.655+0000 7fee5a800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee5c1031d0 0x7fee5c1134e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:31:54.649 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:54.655+0000 7fee5a800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee5c1031d0 0x7fee5c1134e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.169:33256/0 (socket says 172.21.15.169:33256) 2024-03-04T13:31:54.649 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:54.655+0000 7fee5a800700 1 -- 172.21.15.169:0/3645977079 learned_addr learned my addr 172.21.15.169:0/3645977079 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-03-04T13:31:54.649 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:54.656+0000 7fee5a800700 1 -- 172.21.15.169:0/3645977079 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee4c005040 con 0x7fee5c1031d0 2024-03-04T13:31:54.651 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:54.656+0000 7fee5a800700 1 --2- 172.21.15.169:0/3645977079 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee5c1031d0 0x7fee5c1134e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fee4c009b80 tx=0x7fee4c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:31:54.651 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:54.657+0000 7fee52a00700 1 -- 172.21.15.169:0/3645977079 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fee4c00b5c0 con 0x7fee5c1031d0 2024-03-04T13:31:54.651 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:54.657+0000 7fee61400700 1 -- 172.21.15.169:0/3645977079 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee5c113c20 con 0x7fee5c1031d0 2024-03-04T13:31:54.651 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:54.657+0000 7fee52a00700 1 -- 172.21.15.169:0/3645977079 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fee4c00be70 con 0x7fee5c1031d0 2024-03-04T13:31:54.651 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:54.657+0000 7fee52a00700 1 -- 172.21.15.169:0/3645977079 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fee4c01c660 con 0x7fee5c1031d0 2024-03-04T13:31:54.651 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:54.657+0000 7fee61400700 1 -- 172.21.15.169:0/3645977079 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee5c1148c0 con 0x7fee5c1031d0 2024-03-04T13:31:54.652 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:54.658+0000 7fee61400700 1 -- 172.21.15.169:0/3645977079 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee5c05f5c0 con 0x7fee5c1031d0 2024-03-04T13:31:54.652 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:54.658+0000 7fee52a00700 1 -- 172.21.15.169:0/3645977079 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) v1 ==== 43425+0+0 (secure 0 0 0) 0x7fee4c004d30 con 0x7fee5c1031d0 2024-03-04T13:31:54.653 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:54.659+0000 7fee52a00700 1 --2- 172.21.15.169:0/3645977079 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7fee40036870 0x7fee40038d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:31:54.653 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:54.659+0000 7fee52a00700 1 -- 172.21.15.169:0/3645977079 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fee4c049a30 con 0x7fee5c1031d0 2024-03-04T13:31:54.653 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:54.660+0000 7fee53e00700 1 --2- 172.21.15.169:0/3645977079 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7fee40036870 0x7fee40038d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:31:54.654 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:54.661+0000 7fee53e00700 1 --2- 172.21.15.169:0/3645977079 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7fee40036870 0x7fee40038d20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fee44009990 tx=0x7fee44006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:31:54.668 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:54.674+0000 7fee52a00700 1 -- 172.21.15.169:0/3645977079 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fee4c025400 con 0x7fee5c1031d0 2024-03-04T13:31:54.957 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:54.963+0000 7fee61400700 1 -- 172.21.15.169:0/3645977079 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fee5c115410 con 0x7fee5c1031d0 2024-03-04T13:31:54.958 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:54.964+0000 7fee52a00700 1 -- 172.21.15.169:0/3645977079 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7fee4c011370 con 0x7fee5c1031d0 2024-03-04T13:31:54.958 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-03-04T13:31:54.958 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":1,"fsid":"1fc09a58-da2b-11ee-95c6-87774f69a715","modified":"2024-03-04T13:29:38.149007Z","created":"2024-03-04T13:29:38.149007Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi107","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:3300","nonce":0},{"type":"v1","addr":"172.21.15.107:6789","nonce":0}]},"addr":"172.21.15.107:6789/0","public_addr":"172.21.15.107:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-04T13:31:54.961 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:54.967+0000 7fee61400700 1 -- 172.21.15.169:0/3645977079 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7fee40036870 msgr2=0x7fee40038d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:31:54.961 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:54.967+0000 7fee61400700 1 --2- 172.21.15.169:0/3645977079 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7fee40036870 0x7fee40038d20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fee44009990 tx=0x7fee44006e40 comp rx=0 tx=0).stop 2024-03-04T13:31:54.961 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:54.967+0000 7fee61400700 1 -- 172.21.15.169:0/3645977079 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee5c1031d0 msgr2=0x7fee5c1134e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:31:54.961 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:54.967+0000 7fee61400700 1 --2- 172.21.15.169:0/3645977079 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee5c1031d0 0x7fee5c1134e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fee4c009b80 tx=0x7fee4c009e30 comp rx=0 tx=0).stop 2024-03-04T13:31:54.961 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:54.967+0000 7fee61400700 1 -- 172.21.15.169:0/3645977079 shutdown_connections 2024-03-04T13:31:54.961 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:54.967+0000 7fee61400700 1 --2- 172.21.15.169:0/3645977079 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7fee40036870 0x7fee40038d20 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:31:54.962 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:54.967+0000 7fee61400700 1 --2- 172.21.15.169:0/3645977079 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee5c1031d0 0x7fee5c1134e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:31:54.962 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:54.967+0000 7fee61400700 1 -- 172.21.15.169:0/3645977079 >> 172.21.15.169:0/3645977079 conn(0x7fee5c0722d0 msgr2=0x7fee5c073030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:31:54.962 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:54.967+0000 7fee61400700 1 -- 172.21.15.169:0/3645977079 shutdown_connections 2024-03-04T13:31:54.962 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:54.967+0000 7fee61400700 1 -- 172.21.15.169:0/3645977079 wait complete. 2024-03-04T13:31:54.962 INFO:teuthology.orchestra.run.smithi169.stderr:dumped monmap epoch 1 2024-03-04T13:31:55.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:31:55 smithi107 ceph-mon[28739]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:31:55.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:31:55 smithi107 ceph-mon[28739]: from='client.? 172.21.15.169:0/3645977079' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-04T13:31:56.715 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-04T13:31:56.716 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 -- ceph mon dump -f json 2024-03-04T13:31:57.163 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-04T13:31:57.264 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-04T13:31:57.587 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:31:57 smithi107 ceph-mon[28739]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:31:59.078 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:59.083+0000 7fd3ae400700 1 -- 172.21.15.169:0/2040495252 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd3a81031d0 msgr2=0x7fd3a81035e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:31:59.078 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:59.083+0000 7fd3ae400700 1 --2- 172.21.15.169:0/2040495252 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd3a81031d0 0x7fd3a81035e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd3980048c0 tx=0x7fd398009e30 comp rx=0 tx=0).stop 2024-03-04T13:31:59.078 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:59.083+0000 7fd3ae400700 1 -- 172.21.15.169:0/2040495252 shutdown_connections 2024-03-04T13:31:59.078 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:59.083+0000 7fd3ae400700 1 --2- 172.21.15.169:0/2040495252 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd3a81031d0 0x7fd3a81035e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:31:59.078 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:59.083+0000 7fd3ae400700 1 -- 172.21.15.169:0/2040495252 >> 172.21.15.169:0/2040495252 conn(0x7fd3a80722d0 msgr2=0x7fd3a80746e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:31:59.078 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:59.084+0000 7fd3ae400700 1 -- 172.21.15.169:0/2040495252 shutdown_connections 2024-03-04T13:31:59.078 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:59.084+0000 7fd3ae400700 1 -- 172.21.15.169:0/2040495252 wait complete. 2024-03-04T13:31:59.079 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:59.084+0000 7fd3ae400700 1 Processor -- start 2024-03-04T13:31:59.079 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:59.085+0000 7fd3ae400700 1 -- start start 2024-03-04T13:31:59.079 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:59.085+0000 7fd3ae400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd3a81031d0 0x7fd3a81134e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:31:59.079 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:59.085+0000 7fd3ae400700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a8113a20 con 0x7fd3a81031d0 2024-03-04T13:31:59.080 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:59.085+0000 7fd3a7400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd3a81031d0 0x7fd3a81134e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:31:59.080 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:59.086+0000 7fd3a7400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd3a81031d0 0x7fd3a81134e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.169:33274/0 (socket says 172.21.15.169:33274) 2024-03-04T13:31:59.080 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:59.086+0000 7fd3a7400700 1 -- 172.21.15.169:0/3798559915 learned_addr learned my addr 172.21.15.169:0/3798559915 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-03-04T13:31:59.080 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:59.086+0000 7fd3a7400700 1 -- 172.21.15.169:0/3798559915 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd398005040 con 0x7fd3a81031d0 2024-03-04T13:31:59.080 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:59.087+0000 7fd3a7400700 1 --2- 172.21.15.169:0/3798559915 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd3a81031d0 0x7fd3a81134e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd398009b80 tx=0x7fd398009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:31:59.081 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:59.087+0000 7fd39f400700 1 -- 172.21.15.169:0/3798559915 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd39800b5c0 con 0x7fd3a81031d0 2024-03-04T13:31:59.081 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:59.087+0000 7fd39f400700 1 -- 172.21.15.169:0/3798559915 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd39800be70 con 0x7fd3a81031d0 2024-03-04T13:31:59.081 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:59.087+0000 7fd39f400700 1 -- 172.21.15.169:0/3798559915 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd39801c660 con 0x7fd3a81031d0 2024-03-04T13:31:59.081 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:59.087+0000 7fd3ae400700 1 -- 172.21.15.169:0/3798559915 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3a8113c20 con 0x7fd3a81031d0 2024-03-04T13:31:59.081 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:59.087+0000 7fd3ae400700 1 -- 172.21.15.169:0/3798559915 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3a81148c0 con 0x7fd3a81031d0 2024-03-04T13:31:59.082 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:59.088+0000 7fd3ae400700 1 -- 172.21.15.169:0/3798559915 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3a805f5c0 con 0x7fd3a81031d0 2024-03-04T13:31:59.082 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:59.089+0000 7fd39f400700 1 -- 172.21.15.169:0/3798559915 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) v1 ==== 43425+0+0 (secure 0 0 0) 0x7fd398004d30 con 0x7fd3a81031d0 2024-03-04T13:31:59.083 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:59.089+0000 7fd39f400700 1 --2- 172.21.15.169:0/3798559915 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7fd388036870 0x7fd388038d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:31:59.083 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:59.089+0000 7fd39f400700 1 -- 172.21.15.169:0/3798559915 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fd398049a30 con 0x7fd3a81031d0 2024-03-04T13:31:59.083 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:59.090+0000 7fd39fe00700 1 --2- 172.21.15.169:0/3798559915 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7fd388036870 0x7fd388038d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:31:59.084 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:59.090+0000 7fd39fe00700 1 --2- 172.21.15.169:0/3798559915 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7fd388036870 0x7fd388038d20 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd390009990 tx=0x7fd390006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:31:59.091 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:59.097+0000 7fd39f400700 1 -- 172.21.15.169:0/3798559915 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fd398025400 con 0x7fd3a81031d0 2024-03-04T13:31:59.376 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:59.382+0000 7fd3ae400700 1 -- 172.21.15.169:0/3798559915 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fd3a8115410 con 0x7fd3a81031d0 2024-03-04T13:31:59.378 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:59.384+0000 7fd39f400700 1 -- 172.21.15.169:0/3798559915 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7fd398011370 con 0x7fd3a81031d0 2024-03-04T13:31:59.378 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-03-04T13:31:59.378 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":1,"fsid":"1fc09a58-da2b-11ee-95c6-87774f69a715","modified":"2024-03-04T13:29:38.149007Z","created":"2024-03-04T13:29:38.149007Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi107","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:3300","nonce":0},{"type":"v1","addr":"172.21.15.107:6789","nonce":0}]},"addr":"172.21.15.107:6789/0","public_addr":"172.21.15.107:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-04T13:31:59.381 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:59.386+0000 7fd3ae400700 1 -- 172.21.15.169:0/3798559915 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7fd388036870 msgr2=0x7fd388038d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:31:59.381 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:59.386+0000 7fd3ae400700 1 --2- 172.21.15.169:0/3798559915 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7fd388036870 0x7fd388038d20 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd390009990 tx=0x7fd390006e40 comp rx=0 tx=0).stop 2024-03-04T13:31:59.381 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:59.387+0000 7fd3ae400700 1 -- 172.21.15.169:0/3798559915 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd3a81031d0 msgr2=0x7fd3a81134e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:31:59.381 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:59.387+0000 7fd3ae400700 1 --2- 172.21.15.169:0/3798559915 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd3a81031d0 0x7fd3a81134e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd398009b80 tx=0x7fd398009e30 comp rx=0 tx=0).stop 2024-03-04T13:31:59.381 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:59.387+0000 7fd3ae400700 1 -- 172.21.15.169:0/3798559915 shutdown_connections 2024-03-04T13:31:59.381 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:59.387+0000 7fd3ae400700 1 --2- 172.21.15.169:0/3798559915 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7fd388036870 0x7fd388038d20 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:31:59.381 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:59.387+0000 7fd3ae400700 1 --2- 172.21.15.169:0/3798559915 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd3a81031d0 0x7fd3a81134e0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:31:59.381 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:59.387+0000 7fd3ae400700 1 -- 172.21.15.169:0/3798559915 >> 172.21.15.169:0/3798559915 conn(0x7fd3a80722d0 msgr2=0x7fd3a8073030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:31:59.382 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:59.387+0000 7fd3ae400700 1 -- 172.21.15.169:0/3798559915 shutdown_connections 2024-03-04T13:31:59.382 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:31:59.387+0000 7fd3ae400700 1 -- 172.21.15.169:0/3798559915 wait complete. 2024-03-04T13:31:59.382 INFO:teuthology.orchestra.run.smithi169.stderr:dumped monmap epoch 1 2024-03-04T13:31:59.677 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:31:59 smithi107 ceph-mon[28739]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:31:59.677 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:31:59 smithi107 ceph-mon[28739]: from='client.? 172.21.15.169:0/3798559915' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-04T13:32:00.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:00 smithi107 ceph-mon[28739]: from='mgr.14162 172.21.15.107:0/1450595538' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:32:00.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:00 smithi107 ceph-mon[28739]: from='mgr.14162 172.21.15.107:0/1450595538' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:32:00.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:00 smithi107 ceph-mon[28739]: from='mgr.14162 172.21.15.107:0/1450595538' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:32:00.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:00 smithi107 ceph-mon[28739]: from='mgr.14162 172.21.15.107:0/1450595538' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:32:00.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:00 smithi107 ceph-mon[28739]: from='mgr.14162 172.21.15.107:0/1450595538' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:32:00.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:00 smithi107 ceph-mon[28739]: from='mgr.14162 172.21.15.107:0/1450595538' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:32:00.838 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:00 smithi107 ceph-mon[28739]: Deploying daemon prometheus.smithi107 on smithi107 2024-03-04T13:32:01.038 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-04T13:32:01.038 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 -- ceph mon dump -f json 2024-03-04T13:32:01.475 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-04T13:32:01.569 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-04T13:32:02.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:01 smithi107 ceph-mon[28739]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:32:02.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:01 smithi107 ceph-mon[28739]: from='mgr.14162 172.21.15.107:0/1450595538' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:32:03.402 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:03.407+0000 7f1f44800700 1 -- 172.21.15.169:0/1799794094 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f3c0fedd0 msgr2=0x7f1f3c0ff1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:32:03.403 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:03.407+0000 7f1f44800700 1 --2- 172.21.15.169:0/1799794094 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f3c0fedd0 0x7f1f3c0ff1e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1f300048c0 tx=0x7f1f30009e30 comp rx=0 tx=0).stop 2024-03-04T13:32:03.403 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:03.407+0000 7f1f44800700 1 -- 172.21.15.169:0/1799794094 shutdown_connections 2024-03-04T13:32:03.403 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:03.407+0000 7f1f44800700 1 --2- 172.21.15.169:0/1799794094 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f3c0fedd0 0x7f1f3c0ff1e0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:32:03.403 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:03.407+0000 7f1f44800700 1 -- 172.21.15.169:0/1799794094 >> 172.21.15.169:0/1799794094 conn(0x7f1f3c0fa5a0 msgr2=0x7f1f3c0fc980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:32:03.403 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:03.407+0000 7f1f44800700 1 -- 172.21.15.169:0/1799794094 shutdown_connections 2024-03-04T13:32:03.403 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:03.407+0000 7f1f44800700 1 -- 172.21.15.169:0/1799794094 wait complete. 2024-03-04T13:32:03.404 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:03.408+0000 7f1f44800700 1 Processor -- start 2024-03-04T13:32:03.404 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:03.408+0000 7f1f44800700 1 -- start start 2024-03-04T13:32:03.405 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:03.409+0000 7f1f44800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f3c0fedd0 0x7f1f3c10f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:32:03.405 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:03.409+0000 7f1f44800700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f3c10f650 con 0x7f1f3c0fedd0 2024-03-04T13:32:03.405 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:03.409+0000 7f1f41e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f3c0fedd0 0x7f1f3c10f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:32:03.405 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:03.409+0000 7f1f41e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f3c0fedd0 0x7f1f3c10f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.169:33302/0 (socket says 172.21.15.169:33302) 2024-03-04T13:32:03.405 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:03.409+0000 7f1f41e00700 1 -- 172.21.15.169:0/3244157153 learned_addr learned my addr 172.21.15.169:0/3244157153 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-03-04T13:32:03.405 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:03.410+0000 7f1f41e00700 1 -- 172.21.15.169:0/3244157153 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f30005040 con 0x7f1f3c0fedd0 2024-03-04T13:32:03.406 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:03.411+0000 7f1f41e00700 1 --2- 172.21.15.169:0/3244157153 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f3c0fedd0 0x7f1f3c10f110 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1f30009680 tx=0x7f1f30009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:32:03.406 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:03.411+0000 7f1f36000700 1 -- 172.21.15.169:0/3244157153 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1f3000bed0 con 0x7f1f3c0fedd0 2024-03-04T13:32:03.406 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:03.411+0000 7f1f44800700 1 -- 172.21.15.169:0/3244157153 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f3c10f850 con 0x7f1f3c0fedd0 2024-03-04T13:32:03.406 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:03.411+0000 7f1f36000700 1 -- 172.21.15.169:0/3244157153 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1f3000b5c0 con 0x7f1f3c0fedd0 2024-03-04T13:32:03.407 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:03.411+0000 7f1f36000700 1 -- 172.21.15.169:0/3244157153 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1f3001c5b0 con 0x7f1f3c0fedd0 2024-03-04T13:32:03.407 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:03.411+0000 7f1f44800700 1 -- 172.21.15.169:0/3244157153 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f3c1104f0 con 0x7f1f3c0fedd0 2024-03-04T13:32:03.407 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:03.412+0000 7f1f44800700 1 -- 172.21.15.169:0/3244157153 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f3c05f5c0 con 0x7f1f3c0fedd0 2024-03-04T13:32:03.408 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:03.413+0000 7f1f36000700 1 -- 172.21.15.169:0/3244157153 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) v1 ==== 43425+0+0 (secure 0 0 0) 0x7f1f30013a80 con 0x7f1f3c0fedd0 2024-03-04T13:32:03.408 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:03.413+0000 7f1f36000700 1 --2- 172.21.15.169:0/3244157153 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7f1f24036870 0x7f1f24038d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:32:03.408 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:03.413+0000 7f1f36000700 1 -- 172.21.15.169:0/3244157153 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f1f3004a060 con 0x7f1f3c0fedd0 2024-03-04T13:32:03.409 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:03.414+0000 7f1f41400700 1 --2- 172.21.15.169:0/3244157153 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7f1f24036870 0x7f1f24038d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:32:03.410 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:03.415+0000 7f1f41400700 1 --2- 172.21.15.169:0/3244157153 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7f1f24036870 0x7f1f24038d20 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c009990 tx=0x7f1f2c006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:32:03.423 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:03.428+0000 7f1f36000700 1 -- 172.21.15.169:0/3244157153 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f1f30011960 con 0x7f1f3c0fedd0 2024-03-04T13:32:03.709 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:03.714+0000 7f1f44800700 1 -- 172.21.15.169:0/3244157153 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f1f3c111060 con 0x7f1f3c0fedd0 2024-03-04T13:32:03.710 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:03.715+0000 7f1f36000700 1 -- 172.21.15.169:0/3244157153 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7f1f30011370 con 0x7f1f3c0fedd0 2024-03-04T13:32:03.710 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-03-04T13:32:03.711 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":1,"fsid":"1fc09a58-da2b-11ee-95c6-87774f69a715","modified":"2024-03-04T13:29:38.149007Z","created":"2024-03-04T13:29:38.149007Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi107","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:3300","nonce":0},{"type":"v1","addr":"172.21.15.107:6789","nonce":0}]},"addr":"172.21.15.107:6789/0","public_addr":"172.21.15.107:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-04T13:32:03.713 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:03.718+0000 7f1f44800700 1 -- 172.21.15.169:0/3244157153 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7f1f24036870 msgr2=0x7f1f24038d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:32:03.713 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:03.718+0000 7f1f44800700 1 --2- 172.21.15.169:0/3244157153 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7f1f24036870 0x7f1f24038d20 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c009990 tx=0x7f1f2c006e40 comp rx=0 tx=0).stop 2024-03-04T13:32:03.713 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:03.718+0000 7f1f44800700 1 -- 172.21.15.169:0/3244157153 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f3c0fedd0 msgr2=0x7f1f3c10f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:32:03.713 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:03.718+0000 7f1f44800700 1 --2- 172.21.15.169:0/3244157153 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f3c0fedd0 0x7f1f3c10f110 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1f30009680 tx=0x7f1f30009e30 comp rx=0 tx=0).stop 2024-03-04T13:32:03.713 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:03.718+0000 7f1f44800700 1 -- 172.21.15.169:0/3244157153 shutdown_connections 2024-03-04T13:32:03.713 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:03.718+0000 7f1f44800700 1 --2- 172.21.15.169:0/3244157153 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7f1f24036870 0x7f1f24038d20 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:32:03.713 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:03.718+0000 7f1f44800700 1 --2- 172.21.15.169:0/3244157153 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f3c0fedd0 0x7f1f3c10f110 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:32:03.713 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:03.718+0000 7f1f44800700 1 -- 172.21.15.169:0/3244157153 >> 172.21.15.169:0/3244157153 conn(0x7f1f3c0fa5a0 msgr2=0x7f1f3c0fc330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:32:03.713 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:03.718+0000 7f1f44800700 1 -- 172.21.15.169:0/3244157153 shutdown_connections 2024-03-04T13:32:03.713 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:03.718+0000 7f1f44800700 1 -- 172.21.15.169:0/3244157153 wait complete. 2024-03-04T13:32:03.714 INFO:teuthology.orchestra.run.smithi169.stderr:dumped monmap epoch 1 2024-03-04T13:32:04.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:03 smithi107 ceph-mon[28739]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:32:04.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:03 smithi107 ceph-mon[28739]: from='client.? 172.21.15.169:0/3244157153' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-04T13:32:05.400 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-04T13:32:05.400 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 -- ceph mon dump -f json 2024-03-04T13:32:05.839 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-04T13:32:05.938 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-04T13:32:06.086 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:05 smithi107 ceph-mon[28739]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:32:07.741 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:07.747+0000 7f3386600700 1 -- 172.21.15.169:0/1222233143 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33800fec00 msgr2=0x7f33800ff010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:32:07.741 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:07.747+0000 7f3386600700 1 --2- 172.21.15.169:0/1222233143 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33800fec00 0x7f33800ff010 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f33740048c0 tx=0x7f3374009e30 comp rx=0 tx=0).stop 2024-03-04T13:32:07.742 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:07.747+0000 7f3386600700 1 -- 172.21.15.169:0/1222233143 shutdown_connections 2024-03-04T13:32:07.742 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:07.747+0000 7f3386600700 1 --2- 172.21.15.169:0/1222233143 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33800fec00 0x7f33800ff010 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:32:07.742 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:07.747+0000 7f3386600700 1 -- 172.21.15.169:0/1222233143 >> 172.21.15.169:0/1222233143 conn(0x7f33800fa5a0 msgr2=0x7f33800fc980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:32:07.742 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:07.747+0000 7f3386600700 1 -- 172.21.15.169:0/1222233143 shutdown_connections 2024-03-04T13:32:07.742 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:07.747+0000 7f3386600700 1 -- 172.21.15.169:0/1222233143 wait complete. 2024-03-04T13:32:07.743 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:07.748+0000 7f3386600700 1 Processor -- start 2024-03-04T13:32:07.743 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:07.749+0000 7f3386600700 1 -- start start 2024-03-04T13:32:07.743 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:07.749+0000 7f3386600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33800fec00 0x7f338010ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:32:07.743 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:07.749+0000 7f3386600700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f338010f440 con 0x7f33800fec00 2024-03-04T13:32:07.743 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:07.749+0000 7f3385200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33800fec00 0x7f338010ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:32:07.744 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:07.750+0000 7f3385200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33800fec00 0x7f338010ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.169:43744/0 (socket says 172.21.15.169:43744) 2024-03-04T13:32:07.744 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:07.750+0000 7f3385200700 1 -- 172.21.15.169:0/650156005 learned_addr learned my addr 172.21.15.169:0/650156005 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-03-04T13:32:07.744 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:07.750+0000 7f3385200700 1 -- 172.21.15.169:0/650156005 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3374005040 con 0x7f33800fec00 2024-03-04T13:32:07.744 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:07.751+0000 7f3385200700 1 --2- 172.21.15.169:0/650156005 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33800fec00 0x7f338010ef00 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3374009b80 tx=0x7f3374009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:32:07.750 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:07.755+0000 7f337cc00700 1 -- 172.21.15.169:0/650156005 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f337400b5c0 con 0x7f33800fec00 2024-03-04T13:32:07.750 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:07.756+0000 7f337cc00700 1 -- 172.21.15.169:0/650156005 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f337400be70 con 0x7f33800fec00 2024-03-04T13:32:07.750 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:07.756+0000 7f337cc00700 1 -- 172.21.15.169:0/650156005 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f337401c660 con 0x7f33800fec00 2024-03-04T13:32:07.750 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:07.756+0000 7f3386600700 1 -- 172.21.15.169:0/650156005 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f338010f640 con 0x7f33800fec00 2024-03-04T13:32:07.750 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:07.756+0000 7f3386600700 1 -- 172.21.15.169:0/650156005 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33801102e0 con 0x7f33800fec00 2024-03-04T13:32:07.750 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:07.757+0000 7f3386600700 1 -- 172.21.15.169:0/650156005 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f338005f5c0 con 0x7f33800fec00 2024-03-04T13:32:07.751 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:07.757+0000 7f337cc00700 1 -- 172.21.15.169:0/650156005 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) v1 ==== 43425+0+0 (secure 0 0 0) 0x7f3374004d30 con 0x7f33800fec00 2024-03-04T13:32:07.751 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:07.758+0000 7f337cc00700 1 --2- 172.21.15.169:0/650156005 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7f3368036900 0x7f3368038db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:32:07.751 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:07.758+0000 7f337cc00700 1 -- 172.21.15.169:0/650156005 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f3374049f20 con 0x7f33800fec00 2024-03-04T13:32:07.752 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:07.758+0000 7f337fe00700 1 --2- 172.21.15.169:0/650156005 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7f3368036900 0x7f3368038db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:32:07.753 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:07.759+0000 7f337fe00700 1 --2- 172.21.15.169:0/650156005 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7f3368036900 0x7f3368038db0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f336c009990 tx=0x7f336c006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:32:07.764 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:07.770+0000 7f337cc00700 1 -- 172.21.15.169:0/650156005 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f33740247d0 con 0x7f33800fec00 2024-03-04T13:32:08.051 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:08.056+0000 7f3386600700 1 -- 172.21.15.169:0/650156005 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f3380110e30 con 0x7f33800fec00 2024-03-04T13:32:08.052 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:08.058+0000 7f337cc00700 1 -- 172.21.15.169:0/650156005 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7f33740247d0 con 0x7f33800fec00 2024-03-04T13:32:08.052 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-03-04T13:32:08.052 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":1,"fsid":"1fc09a58-da2b-11ee-95c6-87774f69a715","modified":"2024-03-04T13:29:38.149007Z","created":"2024-03-04T13:29:38.149007Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi107","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:3300","nonce":0},{"type":"v1","addr":"172.21.15.107:6789","nonce":0}]},"addr":"172.21.15.107:6789/0","public_addr":"172.21.15.107:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-04T13:32:08.055 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:08.060+0000 7f3386600700 1 -- 172.21.15.169:0/650156005 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7f3368036900 msgr2=0x7f3368038db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:32:08.055 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:08.060+0000 7f3386600700 1 --2- 172.21.15.169:0/650156005 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7f3368036900 0x7f3368038db0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f336c009990 tx=0x7f336c006e40 comp rx=0 tx=0).stop 2024-03-04T13:32:08.055 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:08.060+0000 7f3386600700 1 -- 172.21.15.169:0/650156005 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33800fec00 msgr2=0x7f338010ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:32:08.055 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:08.060+0000 7f3386600700 1 --2- 172.21.15.169:0/650156005 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33800fec00 0x7f338010ef00 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3374009b80 tx=0x7f3374009e30 comp rx=0 tx=0).stop 2024-03-04T13:32:08.055 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:08.061+0000 7f3386600700 1 -- 172.21.15.169:0/650156005 shutdown_connections 2024-03-04T13:32:08.055 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:08.061+0000 7f3386600700 1 --2- 172.21.15.169:0/650156005 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7f3368036900 0x7f3368038db0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:32:08.055 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:08.061+0000 7f3386600700 1 --2- 172.21.15.169:0/650156005 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33800fec00 0x7f338010ef00 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:32:08.055 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:08.061+0000 7f3386600700 1 -- 172.21.15.169:0/650156005 >> 172.21.15.169:0/650156005 conn(0x7f33800fa5a0 msgr2=0x7f33800fc310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:32:08.055 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:08.061+0000 7f3386600700 1 -- 172.21.15.169:0/650156005 shutdown_connections 2024-03-04T13:32:08.055 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:08.061+0000 7f3386600700 1 -- 172.21.15.169:0/650156005 wait complete. 2024-03-04T13:32:08.056 INFO:teuthology.orchestra.run.smithi169.stderr:dumped monmap epoch 1 2024-03-04T13:32:08.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:07 smithi107 ceph-mon[28739]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:32:09.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:08 smithi107 ceph-mon[28739]: from='client.? 172.21.15.169:0/650156005' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-04T13:32:09.759 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-04T13:32:09.759 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 -- ceph mon dump -f json 2024-03-04T13:32:10.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:09 smithi107 ceph-mon[28739]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:32:10.200 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-04T13:32:10.299 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-04T13:32:12.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:11 smithi107 ceph-mon[28739]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:32:12.200 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:12.206+0000 7f8bbe200700 1 -- 172.21.15.169:0/2792853828 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8bb80fedd0 msgr2=0x7f8bb80ff1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:32:12.200 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:12.206+0000 7f8bbe200700 1 --2- 172.21.15.169:0/2792853828 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8bb80fedd0 0x7f8bb80ff1e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8ba80048c0 tx=0x7f8ba8009e30 comp rx=0 tx=0).stop 2024-03-04T13:32:12.200 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:12.206+0000 7f8bbe200700 1 -- 172.21.15.169:0/2792853828 shutdown_connections 2024-03-04T13:32:12.200 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:12.206+0000 7f8bbe200700 1 --2- 172.21.15.169:0/2792853828 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8bb80fedd0 0x7f8bb80ff1e0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:32:12.201 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:12.206+0000 7f8bbe200700 1 -- 172.21.15.169:0/2792853828 >> 172.21.15.169:0/2792853828 conn(0x7f8bb80fa5a0 msgr2=0x7f8bb80fc980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:32:12.201 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:12.206+0000 7f8bbe200700 1 -- 172.21.15.169:0/2792853828 shutdown_connections 2024-03-04T13:32:12.201 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:12.207+0000 7f8bbe200700 1 -- 172.21.15.169:0/2792853828 wait complete. 2024-03-04T13:32:12.201 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:12.207+0000 7f8bbe200700 1 Processor -- start 2024-03-04T13:32:12.201 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:12.207+0000 7f8bbe200700 1 -- start start 2024-03-04T13:32:12.201 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:12.208+0000 7f8bbe200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8bb80fedd0 0x7f8bb810f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:32:12.201 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:12.208+0000 7f8bbe200700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb810f630 con 0x7f8bb80fedd0 2024-03-04T13:32:12.202 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:12.208+0000 7f8bb7400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8bb80fedd0 0x7f8bb810f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:32:12.202 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:12.208+0000 7f8bb7400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8bb80fedd0 0x7f8bb810f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.169:43764/0 (socket says 172.21.15.169:43764) 2024-03-04T13:32:12.202 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:12.208+0000 7f8bb7400700 1 -- 172.21.15.169:0/521748758 learned_addr learned my addr 172.21.15.169:0/521748758 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-03-04T13:32:12.203 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:12.209+0000 7f8bb7400700 1 -- 172.21.15.169:0/521748758 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ba8005040 con 0x7f8bb80fedd0 2024-03-04T13:32:12.203 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:12.210+0000 7f8bb7400700 1 --2- 172.21.15.169:0/521748758 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8bb80fedd0 0x7f8bb810f0f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8ba8009680 tx=0x7f8ba8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:32:12.204 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:12.210+0000 7f8baf400700 1 -- 172.21.15.169:0/521748758 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8ba800bed0 con 0x7f8bb80fedd0 2024-03-04T13:32:12.204 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:12.210+0000 7f8bbe200700 1 -- 172.21.15.169:0/521748758 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bb810f830 con 0x7f8bb80fedd0 2024-03-04T13:32:12.204 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:12.210+0000 7f8baf400700 1 -- 172.21.15.169:0/521748758 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8ba800b5c0 con 0x7f8bb80fedd0 2024-03-04T13:32:12.204 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:12.210+0000 7f8bbe200700 1 -- 172.21.15.169:0/521748758 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bb81104d0 con 0x7f8bb80fedd0 2024-03-04T13:32:12.204 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:12.210+0000 7f8baf400700 1 -- 172.21.15.169:0/521748758 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8ba8025560 con 0x7f8bb80fedd0 2024-03-04T13:32:12.205 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:12.211+0000 7f8bbe200700 1 -- 172.21.15.169:0/521748758 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bb805f5c0 con 0x7f8bb80fedd0 2024-03-04T13:32:12.205 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:12.212+0000 7f8baf400700 1 -- 172.21.15.169:0/521748758 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) v1 ==== 43425+0+0 (secure 0 0 0) 0x7f8ba8004d30 con 0x7f8bb80fedd0 2024-03-04T13:32:12.206 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:12.212+0000 7f8baf400700 1 --2- 172.21.15.169:0/521748758 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7f8b9c036820 0x7f8b9c038cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:32:12.206 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:12.212+0000 7f8baf400700 1 -- 172.21.15.169:0/521748758 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f8ba8049b60 con 0x7f8bb80fedd0 2024-03-04T13:32:12.206 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:12.213+0000 7f8bb6a00700 1 --2- 172.21.15.169:0/521748758 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7f8b9c036820 0x7f8b9c038cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:32:12.207 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:12.214+0000 7f8bb6a00700 1 --2- 172.21.15.169:0/521748758 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7f8b9c036820 0x7f8b9c038cd0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f8ba4009990 tx=0x7f8ba4006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:32:12.225 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:12.231+0000 7f8baf400700 1 -- 172.21.15.169:0/521748758 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f8ba8011960 con 0x7f8bb80fedd0 2024-03-04T13:32:12.489 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:12.495+0000 7f8bbe200700 1 -- 172.21.15.169:0/521748758 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f8bb8111020 con 0x7f8bb80fedd0 2024-03-04T13:32:12.490 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:12.496+0000 7f8baf400700 1 -- 172.21.15.169:0/521748758 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7f8ba8011370 con 0x7f8bb80fedd0 2024-03-04T13:32:12.490 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-03-04T13:32:12.490 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":1,"fsid":"1fc09a58-da2b-11ee-95c6-87774f69a715","modified":"2024-03-04T13:29:38.149007Z","created":"2024-03-04T13:29:38.149007Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi107","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:3300","nonce":0},{"type":"v1","addr":"172.21.15.107:6789","nonce":0}]},"addr":"172.21.15.107:6789/0","public_addr":"172.21.15.107:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-04T13:32:12.492 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:12.498+0000 7f8bbe200700 1 -- 172.21.15.169:0/521748758 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7f8b9c036820 msgr2=0x7f8b9c038cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:32:12.492 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:12.498+0000 7f8bbe200700 1 --2- 172.21.15.169:0/521748758 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7f8b9c036820 0x7f8b9c038cd0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f8ba4009990 tx=0x7f8ba4006e40 comp rx=0 tx=0).stop 2024-03-04T13:32:12.493 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:12.498+0000 7f8bbe200700 1 -- 172.21.15.169:0/521748758 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8bb80fedd0 msgr2=0x7f8bb810f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:32:12.493 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:12.498+0000 7f8bbe200700 1 --2- 172.21.15.169:0/521748758 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8bb80fedd0 0x7f8bb810f0f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8ba8009680 tx=0x7f8ba8009e30 comp rx=0 tx=0).stop 2024-03-04T13:32:12.493 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:12.499+0000 7f8bbe200700 1 -- 172.21.15.169:0/521748758 shutdown_connections 2024-03-04T13:32:12.493 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:12.499+0000 7f8bbe200700 1 --2- 172.21.15.169:0/521748758 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7f8b9c036820 0x7f8b9c038cd0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:32:12.493 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:12.499+0000 7f8bbe200700 1 --2- 172.21.15.169:0/521748758 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8bb80fedd0 0x7f8bb810f0f0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:32:12.493 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:12.499+0000 7f8bbe200700 1 -- 172.21.15.169:0/521748758 >> 172.21.15.169:0/521748758 conn(0x7f8bb80fa5a0 msgr2=0x7f8bb80fc350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:32:12.493 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:12.499+0000 7f8bbe200700 1 -- 172.21.15.169:0/521748758 shutdown_connections 2024-03-04T13:32:12.493 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:12.499+0000 7f8bbe200700 1 -- 172.21.15.169:0/521748758 wait complete. 2024-03-04T13:32:12.494 INFO:teuthology.orchestra.run.smithi169.stderr:dumped monmap epoch 1 2024-03-04T13:32:13.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:12 smithi107 ceph-mon[28739]: from='client.? 172.21.15.169:0/521748758' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-04T13:32:13.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:13 smithi107 ceph-mon[28739]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:32:14.199 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-04T13:32:14.200 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 -- ceph mon dump -f json 2024-03-04T13:32:14.646 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-04T13:32:14.749 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-04T13:32:16.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:15 smithi107 ceph-mon[28739]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:32:16.591 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:16.596+0000 7f9e25e00700 1 -- 172.21.15.169:0/2541890051 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e20066760 msgr2=0x7f9e20101c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:32:16.591 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:16.596+0000 7f9e25e00700 1 --2- 172.21.15.169:0/2541890051 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e20066760 0x7f9e20101c20 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9e100048c0 tx=0x7f9e10009e30 comp rx=0 tx=0).stop 2024-03-04T13:32:16.591 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:16.596+0000 7f9e25e00700 1 -- 172.21.15.169:0/2541890051 shutdown_connections 2024-03-04T13:32:16.591 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:16.596+0000 7f9e25e00700 1 --2- 172.21.15.169:0/2541890051 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e20066760 0x7f9e20101c20 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:32:16.591 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:16.596+0000 7f9e25e00700 1 -- 172.21.15.169:0/2541890051 >> 172.21.15.169:0/2541890051 conn(0x7f9e200f7210 msgr2=0x7f9e200f9640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:32:16.592 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:16.597+0000 7f9e25e00700 1 -- 172.21.15.169:0/2541890051 shutdown_connections 2024-03-04T13:32:16.592 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:16.597+0000 7f9e25e00700 1 -- 172.21.15.169:0/2541890051 wait complete. 2024-03-04T13:32:16.592 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:16.598+0000 7f9e25e00700 1 Processor -- start 2024-03-04T13:32:16.593 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:16.598+0000 7f9e25e00700 1 -- start start 2024-03-04T13:32:16.593 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:16.598+0000 7f9e25e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e20066760 0x7f9e2010f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:32:16.593 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:16.598+0000 7f9e25e00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e2010f5e0 con 0x7f9e20066760 2024-03-04T13:32:16.593 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:16.598+0000 7f9e1ea00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e20066760 0x7f9e2010f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:32:16.593 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:16.598+0000 7f9e1ea00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e20066760 0x7f9e2010f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.169:47872/0 (socket says 172.21.15.169:47872) 2024-03-04T13:32:16.593 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:16.598+0000 7f9e1ea00700 1 -- 172.21.15.169:0/172109016 learned_addr learned my addr 172.21.15.169:0/172109016 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-03-04T13:32:16.593 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:16.599+0000 7f9e1ea00700 1 -- 172.21.15.169:0/172109016 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e10005040 con 0x7f9e20066760 2024-03-04T13:32:16.593 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:16.599+0000 7f9e1ea00700 1 --2- 172.21.15.169:0/172109016 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e20066760 0x7f9e2010f0a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9e10009b80 tx=0x7f9e10009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:32:16.594 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:16.599+0000 7f9e16a00700 1 -- 172.21.15.169:0/172109016 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9e1000b5c0 con 0x7f9e20066760 2024-03-04T13:32:16.594 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:16.600+0000 7f9e25e00700 1 -- 172.21.15.169:0/172109016 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e2010f7e0 con 0x7f9e20066760 2024-03-04T13:32:16.594 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:16.600+0000 7f9e25e00700 1 -- 172.21.15.169:0/172109016 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e20110480 con 0x7f9e20066760 2024-03-04T13:32:16.594 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:16.600+0000 7f9e16a00700 1 -- 172.21.15.169:0/172109016 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9e1000be70 con 0x7f9e20066760 2024-03-04T13:32:16.594 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:16.600+0000 7f9e16a00700 1 -- 172.21.15.169:0/172109016 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9e100255c0 con 0x7f9e20066760 2024-03-04T13:32:16.595 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:16.601+0000 7f9e25e00700 1 -- 172.21.15.169:0/172109016 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e2005f5c0 con 0x7f9e20066760 2024-03-04T13:32:16.595 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:16.601+0000 7f9e16a00700 1 -- 172.21.15.169:0/172109016 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) v1 ==== 43425+0+0 (secure 0 0 0) 0x7f9e10004d30 con 0x7f9e20066760 2024-03-04T13:32:16.596 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:16.602+0000 7f9e16a00700 1 --2- 172.21.15.169:0/172109016 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7f9e04036870 0x7f9e04038d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:32:16.596 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:16.602+0000 7f9e16a00700 1 -- 172.21.15.169:0/172109016 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f9e10049ae0 con 0x7f9e20066760 2024-03-04T13:32:16.596 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:16.602+0000 7f9e1e000700 1 --2- 172.21.15.169:0/172109016 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7f9e04036870 0x7f9e04038d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:32:16.597 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:16.603+0000 7f9e1e000700 1 --2- 172.21.15.169:0/172109016 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7f9e04036870 0x7f9e04038d20 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c009990 tx=0x7f9e0c006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:32:16.605 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:16.612+0000 7f9e16a00700 1 -- 172.21.15.169:0/172109016 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f9e100238f0 con 0x7f9e20066760 2024-03-04T13:32:16.892 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:16.898+0000 7f9e25e00700 1 -- 172.21.15.169:0/172109016 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f9e2004bed0 con 0x7f9e20066760 2024-03-04T13:32:16.893 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:16.899+0000 7f9e16a00700 1 -- 172.21.15.169:0/172109016 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7f9e10023300 con 0x7f9e20066760 2024-03-04T13:32:16.893 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-03-04T13:32:16.893 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":1,"fsid":"1fc09a58-da2b-11ee-95c6-87774f69a715","modified":"2024-03-04T13:29:38.149007Z","created":"2024-03-04T13:29:38.149007Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi107","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:3300","nonce":0},{"type":"v1","addr":"172.21.15.107:6789","nonce":0}]},"addr":"172.21.15.107:6789/0","public_addr":"172.21.15.107:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-04T13:32:16.896 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:16.902+0000 7f9e25e00700 1 -- 172.21.15.169:0/172109016 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7f9e04036870 msgr2=0x7f9e04038d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:32:16.896 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:16.902+0000 7f9e25e00700 1 --2- 172.21.15.169:0/172109016 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7f9e04036870 0x7f9e04038d20 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c009990 tx=0x7f9e0c006e40 comp rx=0 tx=0).stop 2024-03-04T13:32:16.896 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:16.902+0000 7f9e25e00700 1 -- 172.21.15.169:0/172109016 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e20066760 msgr2=0x7f9e2010f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:32:16.896 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:16.902+0000 7f9e25e00700 1 --2- 172.21.15.169:0/172109016 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e20066760 0x7f9e2010f0a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9e10009b80 tx=0x7f9e10009e30 comp rx=0 tx=0).stop 2024-03-04T13:32:16.896 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:16.902+0000 7f9e25e00700 1 -- 172.21.15.169:0/172109016 shutdown_connections 2024-03-04T13:32:16.896 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:16.902+0000 7f9e25e00700 1 --2- 172.21.15.169:0/172109016 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7f9e04036870 0x7f9e04038d20 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:32:16.896 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:16.902+0000 7f9e25e00700 1 --2- 172.21.15.169:0/172109016 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e20066760 0x7f9e2010f0a0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:32:16.896 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:16.902+0000 7f9e25e00700 1 -- 172.21.15.169:0/172109016 >> 172.21.15.169:0/172109016 conn(0x7f9e200f7210 msgr2=0x7f9e200f7f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:32:16.896 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:16.902+0000 7f9e25e00700 1 -- 172.21.15.169:0/172109016 shutdown_connections 2024-03-04T13:32:16.896 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:16.902+0000 7f9e25e00700 1 -- 172.21.15.169:0/172109016 wait complete. 2024-03-04T13:32:16.897 INFO:teuthology.orchestra.run.smithi169.stderr:dumped monmap epoch 1 2024-03-04T13:32:17.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:17 smithi107 ceph-mon[28739]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:32:17.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:17 smithi107 ceph-mon[28739]: from='client.? 172.21.15.169:0/172109016' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-04T13:32:19.074 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-04T13:32:19.075 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 -- ceph mon dump -f json 2024-03-04T13:32:19.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:18 smithi107 ceph-mon[28739]: from='mgr.14162 172.21.15.107:0/1450595538' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:32:19.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:18 smithi107 ceph-mon[28739]: from='mgr.14162 172.21.15.107:0/1450595538' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:32:19.088 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:18 smithi107 ceph-mon[28739]: from='mgr.14162 172.21.15.107:0/1450595538' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:32:19.088 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:18 smithi107 ceph-mon[28739]: from='mgr.14162 172.21.15.107:0/1450595538' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-03-04T13:32:19.525 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-04T13:32:19.621 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-04T13:32:20.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:19 smithi107 ceph-mon[28739]: from='mgr.14162 172.21.15.107:0/1450595538' entity='mgr.smithi107.wpzgwl' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-03-04T13:32:20.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:19 smithi107 ceph-mon[28739]: mgrmap e14: smithi107.wpzgwl(active, since 82s) 2024-03-04T13:32:21.586 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:21.591+0000 7f3cc0600700 1 -- 172.21.15.169:0/3754195702 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3cb80fcbc0 msgr2=0x7f3cb80fcfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:32:21.586 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:21.591+0000 7f3cc0600700 1 --2- 172.21.15.169:0/3754195702 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3cb80fcbc0 0x7f3cb80fcfd0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3cac0048c0 tx=0x7f3cac009e30 comp rx=0 tx=0).stop 2024-03-04T13:32:21.586 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:21.591+0000 7f3cc0600700 1 -- 172.21.15.169:0/3754195702 shutdown_connections 2024-03-04T13:32:21.586 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:21.591+0000 7f3cc0600700 1 --2- 172.21.15.169:0/3754195702 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3cb80fcbc0 0x7f3cb80fcfd0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:32:21.587 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:21.591+0000 7f3cc0600700 1 -- 172.21.15.169:0/3754195702 >> 172.21.15.169:0/3754195702 conn(0x7f3cb80f82a0 msgr2=0x7f3cb80fa6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:32:21.587 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:21.592+0000 7f3cc0600700 1 -- 172.21.15.169:0/3754195702 shutdown_connections 2024-03-04T13:32:21.587 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:21.592+0000 7f3cc0600700 1 -- 172.21.15.169:0/3754195702 wait complete. 2024-03-04T13:32:21.587 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:21.593+0000 7f3cc0600700 1 Processor -- start 2024-03-04T13:32:21.588 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:21.593+0000 7f3cc0600700 1 -- start start 2024-03-04T13:32:21.588 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:21.593+0000 7f3cc0600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3cb80fcbc0 0x7f3cb8071e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:32:21.588 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:21.593+0000 7f3cc0600700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb80702e0 con 0x7f3cb80fcbc0 2024-03-04T13:32:21.588 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:21.594+0000 7f3cbdc00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3cb80fcbc0 0x7f3cb8071e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:32:21.588 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:21.594+0000 7f3cbdc00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3cb80fcbc0 0x7f3cb8071e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.169:47880/0 (socket says 172.21.15.169:47880) 2024-03-04T13:32:21.588 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:21.594+0000 7f3cbdc00700 1 -- 172.21.15.169:0/4183462159 learned_addr learned my addr 172.21.15.169:0/4183462159 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-03-04T13:32:21.588 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:21.594+0000 7f3cbdc00700 1 -- 172.21.15.169:0/4183462159 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cac005040 con 0x7f3cb80fcbc0 2024-03-04T13:32:21.589 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:21.594+0000 7f3cbdc00700 1 --2- 172.21.15.169:0/4183462159 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3cb80fcbc0 0x7f3cb8071e60 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3cac009b80 tx=0x7f3cac009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:32:21.589 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:21.595+0000 7f3cb5600700 1 -- 172.21.15.169:0/4183462159 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3cac00b5c0 con 0x7f3cb80fcbc0 2024-03-04T13:32:21.589 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:21.595+0000 7f3cc0600700 1 -- 172.21.15.169:0/4183462159 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cb80704e0 con 0x7f3cb80fcbc0 2024-03-04T13:32:21.589 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:21.595+0000 7f3cb5600700 1 -- 172.21.15.169:0/4183462159 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3cac00be70 con 0x7f3cb80fcbc0 2024-03-04T13:32:21.589 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:21.595+0000 7f3cb5600700 1 -- 172.21.15.169:0/4183462159 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3cac01c660 con 0x7f3cb80fcbc0 2024-03-04T13:32:21.589 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:21.595+0000 7f3cc0600700 1 -- 172.21.15.169:0/4183462159 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cb80709d0 con 0x7f3cb80fcbc0 2024-03-04T13:32:21.590 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:21.596+0000 7f3cc0600700 1 -- 172.21.15.169:0/4183462159 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cb8106990 con 0x7f3cb80fcbc0 2024-03-04T13:32:21.590 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:21.596+0000 7f3cb5600700 1 -- 172.21.15.169:0/4183462159 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 14) v1 ==== 43439+0+0 (secure 0 0 0) 0x7f3cac004d30 con 0x7f3cb80fcbc0 2024-03-04T13:32:21.591 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:21.597+0000 7f3cb5600700 1 --2- 172.21.15.169:0/4183462159 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7f3ca0036910 0x7f3ca0038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:32:21.591 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:21.597+0000 7f3cb5600700 1 -- 172.21.15.169:0/4183462159 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f3cac049ab0 con 0x7f3cb80fcbc0 2024-03-04T13:32:21.591 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:21.597+0000 7f3cbd200700 1 -- 172.21.15.169:0/4183462159 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7f3ca0036910 msgr2=0x7f3ca0038dc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/2040330534 2024-03-04T13:32:21.591 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:21.597+0000 7f3cbd200700 1 --2- 172.21.15.169:0/4183462159 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7f3ca0036910 0x7f3ca0038dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T13:32:21.602 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:21.608+0000 7f3cb5600700 1 -- 172.21.15.169:0/4183462159 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f3cac025400 con 0x7f3cb80fcbc0 2024-03-04T13:32:21.792 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:21.798+0000 7f3cbd200700 1 -- 172.21.15.169:0/4183462159 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7f3ca0036910 msgr2=0x7f3ca0038dc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/2040330534 2024-03-04T13:32:21.793 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:21.798+0000 7f3cbd200700 1 --2- 172.21.15.169:0/4183462159 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7f3ca0036910 0x7f3ca0038dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T13:32:21.892 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:21.898+0000 7f3cc0600700 1 -- 172.21.15.169:0/4183462159 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f3cb804bed0 con 0x7f3cb80fcbc0 2024-03-04T13:32:21.893 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:21.899+0000 7f3cb5600700 1 -- 172.21.15.169:0/4183462159 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7f3cac023300 con 0x7f3cb80fcbc0 2024-03-04T13:32:21.894 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-03-04T13:32:21.894 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":1,"fsid":"1fc09a58-da2b-11ee-95c6-87774f69a715","modified":"2024-03-04T13:29:38.149007Z","created":"2024-03-04T13:29:38.149007Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi107","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:3300","nonce":0},{"type":"v1","addr":"172.21.15.107:6789","nonce":0}]},"addr":"172.21.15.107:6789/0","public_addr":"172.21.15.107:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-04T13:32:21.896 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:21.902+0000 7f3cc0600700 1 -- 172.21.15.169:0/4183462159 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7f3ca0036910 msgr2=0x7f3ca0038dc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-04T13:32:21.896 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:21.902+0000 7f3cc0600700 1 --2- 172.21.15.169:0/4183462159 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7f3ca0036910 0x7f3ca0038dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:32:21.896 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:21.902+0000 7f3cc0600700 1 -- 172.21.15.169:0/4183462159 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3cb80fcbc0 msgr2=0x7f3cb8071e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:32:21.896 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:21.902+0000 7f3cc0600700 1 --2- 172.21.15.169:0/4183462159 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3cb80fcbc0 0x7f3cb8071e60 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3cac009b80 tx=0x7f3cac009e30 comp rx=0 tx=0).stop 2024-03-04T13:32:21.896 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:21.902+0000 7f3cc0600700 1 -- 172.21.15.169:0/4183462159 shutdown_connections 2024-03-04T13:32:21.897 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:21.902+0000 7f3cc0600700 1 --2- 172.21.15.169:0/4183462159 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7f3ca0036910 0x7f3ca0038dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:32:21.897 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:21.902+0000 7f3cc0600700 1 --2- 172.21.15.169:0/4183462159 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3cb80fcbc0 0x7f3cb8071e60 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:32:21.897 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:21.902+0000 7f3cc0600700 1 -- 172.21.15.169:0/4183462159 >> 172.21.15.169:0/4183462159 conn(0x7f3cb80f82a0 msgr2=0x7f3cb80f8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:32:21.897 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:21.902+0000 7f3cc0600700 1 -- 172.21.15.169:0/4183462159 shutdown_connections 2024-03-04T13:32:21.897 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:21.902+0000 7f3cc0600700 1 -- 172.21.15.169:0/4183462159 wait complete. 2024-03-04T13:32:21.897 INFO:teuthology.orchestra.run.smithi169.stderr:dumped monmap epoch 1 2024-03-04T13:32:22.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:21 smithi107 ceph-mon[28739]: from='client.? 172.21.15.169:0/4183462159' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-04T13:32:23.614 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-04T13:32:23.615 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 -- ceph mon dump -f json 2024-03-04T13:32:24.071 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-04T13:32:24.178 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-04T13:32:26.077 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:26.082+0000 7ffb44200700 1 -- 172.21.15.169:0/1436606131 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb3c071f10 msgr2=0x7ffb3c070380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:32:26.077 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:26.082+0000 7ffb44200700 1 --2- 172.21.15.169:0/1436606131 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb3c071f10 0x7ffb3c070380 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ffb300048c0 tx=0x7ffb30009e30 comp rx=0 tx=0).stop 2024-03-04T13:32:26.077 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:26.082+0000 7ffb44200700 1 -- 172.21.15.169:0/1436606131 shutdown_connections 2024-03-04T13:32:26.077 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:26.082+0000 7ffb44200700 1 --2- 172.21.15.169:0/1436606131 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb3c071f10 0x7ffb3c070380 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:32:26.077 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:26.082+0000 7ffb44200700 1 -- 172.21.15.169:0/1436606131 >> 172.21.15.169:0/1436606131 conn(0x7ffb3c0f82a0 msgr2=0x7ffb3c0fa6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:32:26.077 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:26.082+0000 7ffb44200700 1 -- 172.21.15.169:0/1436606131 shutdown_connections 2024-03-04T13:32:26.077 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:26.082+0000 7ffb44200700 1 -- 172.21.15.169:0/1436606131 wait complete. 2024-03-04T13:32:26.078 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:26.083+0000 7ffb44200700 1 Processor -- start 2024-03-04T13:32:26.078 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:26.084+0000 7ffb44200700 1 -- start start 2024-03-04T13:32:26.078 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:26.084+0000 7ffb44200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb3c071f10 0x7ffb3c113480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:32:26.078 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:26.084+0000 7ffb44200700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb3c1139c0 con 0x7ffb3c071f10 2024-03-04T13:32:26.079 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:26.085+0000 7ffb41800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb3c071f10 0x7ffb3c113480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:32:26.079 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:26.085+0000 7ffb41800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb3c071f10 0x7ffb3c113480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.169:58786/0 (socket says 172.21.15.169:58786) 2024-03-04T13:32:26.079 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:26.085+0000 7ffb41800700 1 -- 172.21.15.169:0/1721885184 learned_addr learned my addr 172.21.15.169:0/1721885184 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-03-04T13:32:26.079 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:26.086+0000 7ffb41800700 1 -- 172.21.15.169:0/1721885184 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb30005040 con 0x7ffb3c071f10 2024-03-04T13:32:26.080 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:26.086+0000 7ffb41800700 1 --2- 172.21.15.169:0/1721885184 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb3c071f10 0x7ffb3c113480 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ffb30009b80 tx=0x7ffb30009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:32:26.080 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:26.087+0000 7ffb39600700 1 -- 172.21.15.169:0/1721885184 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffb3000b5c0 con 0x7ffb3c071f10 2024-03-04T13:32:26.080 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:26.087+0000 7ffb39600700 1 -- 172.21.15.169:0/1721885184 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffb3000be70 con 0x7ffb3c071f10 2024-03-04T13:32:26.081 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:26.087+0000 7ffb44200700 1 -- 172.21.15.169:0/1721885184 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb3c113bc0 con 0x7ffb3c071f10 2024-03-04T13:32:26.081 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:26.087+0000 7ffb39600700 1 -- 172.21.15.169:0/1721885184 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffb3001c660 con 0x7ffb3c071f10 2024-03-04T13:32:26.081 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:26.087+0000 7ffb44200700 1 -- 172.21.15.169:0/1721885184 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb3c114860 con 0x7ffb3c071f10 2024-03-04T13:32:26.081 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:26.088+0000 7ffb44200700 1 -- 172.21.15.169:0/1721885184 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb3c05f5c0 con 0x7ffb3c071f10 2024-03-04T13:32:26.082 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:26.088+0000 7ffb39600700 1 -- 172.21.15.169:0/1721885184 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 14) v1 ==== 43439+0+0 (secure 0 0 0) 0x7ffb30004d30 con 0x7ffb3c071f10 2024-03-04T13:32:26.083 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:26.089+0000 7ffb39600700 1 --2- 172.21.15.169:0/1721885184 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7ffb24036910 0x7ffb24038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:32:26.083 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:26.089+0000 7ffb39600700 1 -- 172.21.15.169:0/1721885184 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7ffb30049fc0 con 0x7ffb3c071f10 2024-03-04T13:32:26.083 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:26.089+0000 7ffb40e00700 1 -- 172.21.15.169:0/1721885184 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7ffb24036910 msgr2=0x7ffb24038dc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/2040330534 2024-03-04T13:32:26.083 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:26.089+0000 7ffb40e00700 1 --2- 172.21.15.169:0/1721885184 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7ffb24036910 0x7ffb24038dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T13:32:26.096 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:26.103+0000 7ffb39600700 1 -- 172.21.15.169:0/1721885184 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7ffb300118c0 con 0x7ffb3c071f10 2024-03-04T13:32:26.284 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:26.290+0000 7ffb40e00700 1 -- 172.21.15.169:0/1721885184 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7ffb24036910 msgr2=0x7ffb24038dc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/2040330534 2024-03-04T13:32:26.284 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:26.290+0000 7ffb40e00700 1 --2- 172.21.15.169:0/1721885184 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7ffb24036910 0x7ffb24038dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T13:32:26.385 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:26.391+0000 7ffb44200700 1 -- 172.21.15.169:0/1721885184 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7ffb3c1153d0 con 0x7ffb3c071f10 2024-03-04T13:32:26.386 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:26.393+0000 7ffb39600700 1 -- 172.21.15.169:0/1721885184 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7ffb300118c0 con 0x7ffb3c071f10 2024-03-04T13:32:26.387 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-03-04T13:32:26.387 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":1,"fsid":"1fc09a58-da2b-11ee-95c6-87774f69a715","modified":"2024-03-04T13:29:38.149007Z","created":"2024-03-04T13:29:38.149007Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi107","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:3300","nonce":0},{"type":"v1","addr":"172.21.15.107:6789","nonce":0}]},"addr":"172.21.15.107:6789/0","public_addr":"172.21.15.107:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-04T13:32:26.389 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:26.395+0000 7ffb44200700 1 -- 172.21.15.169:0/1721885184 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7ffb24036910 msgr2=0x7ffb24038dc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-04T13:32:26.389 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:26.395+0000 7ffb44200700 1 --2- 172.21.15.169:0/1721885184 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7ffb24036910 0x7ffb24038dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:32:26.389 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:26.395+0000 7ffb44200700 1 -- 172.21.15.169:0/1721885184 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb3c071f10 msgr2=0x7ffb3c113480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:32:26.390 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:26.395+0000 7ffb44200700 1 --2- 172.21.15.169:0/1721885184 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb3c071f10 0x7ffb3c113480 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ffb30009b80 tx=0x7ffb30009e30 comp rx=0 tx=0).stop 2024-03-04T13:32:26.390 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:26.395+0000 7ffb44200700 1 -- 172.21.15.169:0/1721885184 shutdown_connections 2024-03-04T13:32:26.390 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:26.395+0000 7ffb44200700 1 --2- 172.21.15.169:0/1721885184 >> [v2:172.21.15.107:6800/2040330534,v1:172.21.15.107:6801/2040330534] conn(0x7ffb24036910 0x7ffb24038dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:32:26.390 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:26.395+0000 7ffb44200700 1 --2- 172.21.15.169:0/1721885184 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb3c071f10 0x7ffb3c113480 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:32:26.390 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:26.395+0000 7ffb44200700 1 -- 172.21.15.169:0/1721885184 >> 172.21.15.169:0/1721885184 conn(0x7ffb3c0f82a0 msgr2=0x7ffb3c0f8f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:32:26.390 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:26.395+0000 7ffb44200700 1 -- 172.21.15.169:0/1721885184 shutdown_connections 2024-03-04T13:32:26.390 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:26.395+0000 7ffb44200700 1 -- 172.21.15.169:0/1721885184 wait complete. 2024-03-04T13:32:26.390 INFO:teuthology.orchestra.run.smithi169.stderr:dumped monmap epoch 1 2024-03-04T13:32:26.587 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:26 smithi107 ceph-mon[28739]: from='client.? 172.21.15.169:0/1721885184' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-04T13:32:27.479 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:27 smithi107 ceph-mon[28739]: Active manager daemon smithi107.wpzgwl restarted 2024-03-04T13:32:27.479 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:27 smithi107 ceph-mon[28739]: Activating manager daemon smithi107.wpzgwl 2024-03-04T13:32:27.479 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:27 smithi107 ceph-mon[28739]: osdmap e5: 0 total, 0 up, 0 in 2024-03-04T13:32:27.479 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:27 smithi107 ceph-mon[28739]: mgrmap e15: smithi107.wpzgwl(active, starting, since 0.00858989s) 2024-03-04T13:32:27.479 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:27 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mon metadata", "id": "smithi107"}]: dispatch 2024-03-04T13:32:27.479 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:27 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mgr metadata", "who": "smithi107.wpzgwl", "id": "smithi107.wpzgwl"}]: dispatch 2024-03-04T13:32:27.479 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:27 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-04T13:32:27.479 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:27 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-04T13:32:27.480 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:27 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-04T13:32:27.480 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:27 smithi107 ceph-mon[28739]: Manager daemon smithi107.wpzgwl is now available 2024-03-04T13:32:27.480 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:27 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:32:27.480 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:27 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:32:27.480 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:27 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:32:27.480 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:27 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:32:27.480 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:27 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi107.wpzgwl/mirror_snapshot_schedule"}]: dispatch 2024-03-04T13:32:27.480 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:27 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi107.wpzgwl/trash_purge_schedule"}]: dispatch 2024-03-04T13:32:28.050 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-04T13:32:28.051 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 -- ceph mon dump -f json 2024-03-04T13:32:28.470 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-04T13:32:28.568 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-04T13:32:29.546 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:29 smithi107 ceph-mon[28739]: [04/Mar/2024:13:32:28] ENGINE Bus STARTING 2024-03-04T13:32:29.546 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:29 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:32:29.546 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:29 smithi107 ceph-mon[28739]: [04/Mar/2024:13:32:28] ENGINE Serving on https://172.21.15.107:7150 2024-03-04T13:32:29.546 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:29 smithi107 ceph-mon[28739]: [04/Mar/2024:13:32:28] ENGINE Bus STARTED 2024-03-04T13:32:29.546 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:29 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:32:29.546 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:29 smithi107 ceph-mon[28739]: mgrmap e16: smithi107.wpzgwl(active, since 1.00843s) 2024-03-04T13:32:30.502 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:30.508+0000 7fa96de00700 1 -- 172.21.15.169:0/1865707407 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9680fcbc0 msgr2=0x7fa9680fcfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:32:30.503 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:30.508+0000 7fa96de00700 1 --2- 172.21.15.169:0/1865707407 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9680fcbc0 0x7fa9680fcfd0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa9580048c0 tx=0x7fa958009e30 comp rx=0 tx=0).stop 2024-03-04T13:32:30.503 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:30.508+0000 7fa96de00700 1 -- 172.21.15.169:0/1865707407 shutdown_connections 2024-03-04T13:32:30.503 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:30.508+0000 7fa96de00700 1 --2- 172.21.15.169:0/1865707407 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9680fcbc0 0x7fa9680fcfd0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:32:30.503 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:30.508+0000 7fa96de00700 1 -- 172.21.15.169:0/1865707407 >> 172.21.15.169:0/1865707407 conn(0x7fa9680f82a0 msgr2=0x7fa9680fa6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:32:30.503 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:30.508+0000 7fa96de00700 1 -- 172.21.15.169:0/1865707407 shutdown_connections 2024-03-04T13:32:30.503 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:30.509+0000 7fa96de00700 1 -- 172.21.15.169:0/1865707407 wait complete. 2024-03-04T13:32:30.503 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:30.510+0000 7fa96de00700 1 Processor -- start 2024-03-04T13:32:30.503 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:30.510+0000 7fa96de00700 1 -- start start 2024-03-04T13:32:30.504 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:30.510+0000 7fa96de00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9680fcbc0 0x7fa96810f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:32:30.504 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:30.510+0000 7fa96de00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa96810f5c0 con 0x7fa9680fcbc0 2024-03-04T13:32:30.504 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:30.511+0000 7fa966a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9680fcbc0 0x7fa96810f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:32:30.505 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:30.511+0000 7fa966a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9680fcbc0 0x7fa96810f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.169:58808/0 (socket says 172.21.15.169:58808) 2024-03-04T13:32:30.505 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:30.511+0000 7fa966a00700 1 -- 172.21.15.169:0/211865878 learned_addr learned my addr 172.21.15.169:0/211865878 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-03-04T13:32:30.505 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:30.512+0000 7fa966a00700 1 -- 172.21.15.169:0/211865878 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa958005040 con 0x7fa9680fcbc0 2024-03-04T13:32:30.506 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:30.512+0000 7fa966a00700 1 --2- 172.21.15.169:0/211865878 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9680fcbc0 0x7fa96810f080 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa958009b80 tx=0x7fa958009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:32:30.506 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:30.513+0000 7fa95ea00700 1 -- 172.21.15.169:0/211865878 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa95800b5c0 con 0x7fa9680fcbc0 2024-03-04T13:32:30.506 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:30.513+0000 7fa95ea00700 1 -- 172.21.15.169:0/211865878 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa95800be70 con 0x7fa9680fcbc0 2024-03-04T13:32:30.506 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:30.513+0000 7fa96de00700 1 -- 172.21.15.169:0/211865878 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa96810f7c0 con 0x7fa9680fcbc0 2024-03-04T13:32:30.507 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:30.513+0000 7fa95ea00700 1 -- 172.21.15.169:0/211865878 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa95801c660 con 0x7fa9680fcbc0 2024-03-04T13:32:30.508 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:30.513+0000 7fa96de00700 1 -- 172.21.15.169:0/211865878 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa968110460 con 0x7fa9680fcbc0 2024-03-04T13:32:30.508 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:30.514+0000 7fa95ea00700 1 -- 172.21.15.169:0/211865878 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 43483+0+0 (secure 0 0 0) 0x7fa958013420 con 0x7fa9680fcbc0 2024-03-04T13:32:30.508 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:30.514+0000 7fa96de00700 1 -- 172.21.15.169:0/211865878 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa96805f5c0 con 0x7fa9680fcbc0 2024-03-04T13:32:30.508 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:30.514+0000 7fa95ea00700 1 --2- 172.21.15.169:0/211865878 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fa94c0369b0 0x7fa94c038e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:32:30.508 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:30.515+0000 7fa95ea00700 1 -- 172.21.15.169:0/211865878 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7fa958049be0 con 0x7fa9680fcbc0 2024-03-04T13:32:30.509 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:30.515+0000 7fa966000700 1 --2- 172.21.15.169:0/211865878 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fa94c0369b0 0x7fa94c038e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:32:30.509 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:30.516+0000 7fa966000700 1 --2- 172.21.15.169:0/211865878 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fa94c0369b0 0x7fa94c038e60 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa954009990 tx=0x7fa954006e40 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:32:30.514 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:30.521+0000 7fa95ea00700 1 -- 172.21.15.169:0/211865878 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fa958013b10 con 0x7fa9680fcbc0 2024-03-04T13:32:30.587 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:30 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:32:30.587 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:30 smithi107 ceph-mon[28739]: mgrmap e17: smithi107.wpzgwl(active, since 2s) 2024-03-04T13:32:30.792 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:30.798+0000 7fa96de00700 1 -- 172.21.15.169:0/211865878 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fa968110800 con 0x7fa9680fcbc0 2024-03-04T13:32:30.793 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:30.799+0000 7fa95ea00700 1 -- 172.21.15.169:0/211865878 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7fa9580493f0 con 0x7fa9680fcbc0 2024-03-04T13:32:30.793 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-03-04T13:32:30.793 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":1,"fsid":"1fc09a58-da2b-11ee-95c6-87774f69a715","modified":"2024-03-04T13:29:38.149007Z","created":"2024-03-04T13:29:38.149007Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi107","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:3300","nonce":0},{"type":"v1","addr":"172.21.15.107:6789","nonce":0}]},"addr":"172.21.15.107:6789/0","public_addr":"172.21.15.107:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-04T13:32:30.796 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:30.802+0000 7fa96de00700 1 -- 172.21.15.169:0/211865878 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fa94c0369b0 msgr2=0x7fa94c038e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:32:30.796 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:30.802+0000 7fa96de00700 1 --2- 172.21.15.169:0/211865878 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fa94c0369b0 0x7fa94c038e60 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa954009990 tx=0x7fa954006e40 comp rx=0 tx=0).stop 2024-03-04T13:32:30.796 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:30.802+0000 7fa96de00700 1 -- 172.21.15.169:0/211865878 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9680fcbc0 msgr2=0x7fa96810f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:32:30.796 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:30.802+0000 7fa96de00700 1 --2- 172.21.15.169:0/211865878 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9680fcbc0 0x7fa96810f080 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa958009b80 tx=0x7fa958009e30 comp rx=0 tx=0).stop 2024-03-04T13:32:30.797 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:30.802+0000 7fa96de00700 1 -- 172.21.15.169:0/211865878 shutdown_connections 2024-03-04T13:32:30.797 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:30.802+0000 7fa96de00700 1 --2- 172.21.15.169:0/211865878 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fa94c0369b0 0x7fa94c038e60 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:32:30.797 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:30.802+0000 7fa96de00700 1 --2- 172.21.15.169:0/211865878 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9680fcbc0 0x7fa96810f080 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:32:30.797 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:30.802+0000 7fa96de00700 1 -- 172.21.15.169:0/211865878 >> 172.21.15.169:0/211865878 conn(0x7fa9680f82a0 msgr2=0x7fa9680701c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:32:30.797 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:30.802+0000 7fa96de00700 1 -- 172.21.15.169:0/211865878 shutdown_connections 2024-03-04T13:32:30.797 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:30.803+0000 7fa96de00700 1 -- 172.21.15.169:0/211865878 wait complete. 2024-03-04T13:32:30.798 INFO:teuthology.orchestra.run.smithi169.stderr:dumped monmap epoch 1 2024-03-04T13:32:31.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:31 smithi107 ceph-mon[28739]: from='client.? 172.21.15.169:0/211865878' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-04T13:32:31.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:31 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:32:32.476 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-04T13:32:32.476 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 -- ceph mon dump -f json 2024-03-04T13:32:33.376 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-04T13:32:33.481 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-04T13:32:34.337 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:33 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:32:34.337 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:33 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:32:35.648 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:35.650+0000 7f78d3000700 1 -- 172.21.15.169:0/792428460 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f78cc0fcbc0 msgr2=0x7f78cc0fcfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:32:35.649 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:35.650+0000 7f78d3000700 1 --2- 172.21.15.169:0/792428460 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f78cc0fcbc0 0x7f78cc0fcfd0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f78bc0048c0 tx=0x7f78bc009e30 comp rx=0 tx=0).stop 2024-03-04T13:32:35.649 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:35.650+0000 7f78d3000700 1 -- 172.21.15.169:0/792428460 shutdown_connections 2024-03-04T13:32:35.649 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:35.650+0000 7f78d3000700 1 --2- 172.21.15.169:0/792428460 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f78cc0fcbc0 0x7f78cc0fcfd0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:32:35.649 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:35.650+0000 7f78d3000700 1 -- 172.21.15.169:0/792428460 >> 172.21.15.169:0/792428460 conn(0x7f78cc0f82a0 msgr2=0x7f78cc0fa6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:32:35.649 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:35.650+0000 7f78d3000700 1 -- 172.21.15.169:0/792428460 shutdown_connections 2024-03-04T13:32:35.649 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:35.650+0000 7f78d3000700 1 -- 172.21.15.169:0/792428460 wait complete. 2024-03-04T13:32:35.649 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:35.651+0000 7f78d3000700 1 Processor -- start 2024-03-04T13:32:35.649 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:35.651+0000 7f78d3000700 1 -- start start 2024-03-04T13:32:35.649 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:35.652+0000 7f78d3000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f78cc0fcbc0 0x7f78cc10f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:32:35.649 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:35.652+0000 7f78d3000700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78cc10f5e0 con 0x7f78cc0fcbc0 2024-03-04T13:32:35.649 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:35.652+0000 7f78cbe00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f78cc0fcbc0 0x7f78cc10f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:32:35.649 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:35.652+0000 7f78cbe00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f78cc0fcbc0 0x7f78cc10f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.169:45572/0 (socket says 172.21.15.169:45572) 2024-03-04T13:32:35.650 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:35.652+0000 7f78cbe00700 1 -- 172.21.15.169:0/170099343 learned_addr learned my addr 172.21.15.169:0/170099343 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-03-04T13:32:35.650 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:35.653+0000 7f78cbe00700 1 -- 172.21.15.169:0/170099343 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78bc005040 con 0x7f78cc0fcbc0 2024-03-04T13:32:35.650 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:35.653+0000 7f78cbe00700 1 --2- 172.21.15.169:0/170099343 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f78cc0fcbc0 0x7f78cc10f0a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f78bc009b80 tx=0x7f78bc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:32:35.650 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:35.654+0000 7f78c8c00700 1 -- 172.21.15.169:0/170099343 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f78bc00b5c0 con 0x7f78cc0fcbc0 2024-03-04T13:32:35.650 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:35.654+0000 7f78d3000700 1 -- 172.21.15.169:0/170099343 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78cc10f7e0 con 0x7f78cc0fcbc0 2024-03-04T13:32:35.650 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:35.654+0000 7f78c8c00700 1 -- 172.21.15.169:0/170099343 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f78bc00be70 con 0x7f78cc0fcbc0 2024-03-04T13:32:35.650 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:35.654+0000 7f78c8c00700 1 -- 172.21.15.169:0/170099343 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f78bc01c660 con 0x7f78cc0fcbc0 2024-03-04T13:32:35.650 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:35.654+0000 7f78d3000700 1 -- 172.21.15.169:0/170099343 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78cc110480 con 0x7f78cc0fcbc0 2024-03-04T13:32:35.651 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:35.655+0000 7f78d3000700 1 -- 172.21.15.169:0/170099343 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78cc05f5c0 con 0x7f78cc0fcbc0 2024-03-04T13:32:35.651 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:35.656+0000 7f78c8c00700 1 -- 172.21.15.169:0/170099343 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 43483+0+0 (secure 0 0 0) 0x7f78bc004d30 con 0x7f78cc0fcbc0 2024-03-04T13:32:35.651 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:35.656+0000 7f78c8c00700 1 --2- 172.21.15.169:0/170099343 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f78b0036960 0x7f78b0038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:32:35.651 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:35.656+0000 7f78c8c00700 1 -- 172.21.15.169:0/170099343 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f78bc04a0e0 con 0x7f78cc0fcbc0 2024-03-04T13:32:35.651 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:35.657+0000 7f78c3e00700 1 --2- 172.21.15.169:0/170099343 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f78b0036960 0x7f78b0038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:32:35.652 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:35.658+0000 7f78c3e00700 1 --2- 172.21.15.169:0/170099343 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f78b0036960 0x7f78b0038e10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f78b4009990 tx=0x7f78b4006e40 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:32:35.664 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:35.671+0000 7f78c8c00700 1 -- 172.21.15.169:0/170099343 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f78bc0249b0 con 0x7f78cc0fcbc0 2024-03-04T13:32:35.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:35 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:32:35.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:35 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:32:35.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:35 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "who": "osd/host:smithi107", "name": "osd_memory_target"}]: dispatch 2024-03-04T13:32:35.967 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:35.973+0000 7f78d3000700 1 -- 172.21.15.169:0/170099343 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f78cc110fd0 con 0x7f78cc0fcbc0 2024-03-04T13:32:35.968 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:35.974+0000 7f78c8c00700 1 -- 172.21.15.169:0/170099343 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7f78bc024360 con 0x7f78cc0fcbc0 2024-03-04T13:32:35.968 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-03-04T13:32:35.968 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":1,"fsid":"1fc09a58-da2b-11ee-95c6-87774f69a715","modified":"2024-03-04T13:29:38.149007Z","created":"2024-03-04T13:29:38.149007Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi107","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:3300","nonce":0},{"type":"v1","addr":"172.21.15.107:6789","nonce":0}]},"addr":"172.21.15.107:6789/0","public_addr":"172.21.15.107:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-04T13:32:35.971 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:35.977+0000 7f78d3000700 1 -- 172.21.15.169:0/170099343 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f78b0036960 msgr2=0x7f78b0038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:32:35.971 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:35.977+0000 7f78d3000700 1 --2- 172.21.15.169:0/170099343 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f78b0036960 0x7f78b0038e10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f78b4009990 tx=0x7f78b4006e40 comp rx=0 tx=0).stop 2024-03-04T13:32:35.971 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:35.977+0000 7f78d3000700 1 -- 172.21.15.169:0/170099343 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f78cc0fcbc0 msgr2=0x7f78cc10f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:32:35.972 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:35.977+0000 7f78d3000700 1 --2- 172.21.15.169:0/170099343 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f78cc0fcbc0 0x7f78cc10f0a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f78bc009b80 tx=0x7f78bc009e30 comp rx=0 tx=0).stop 2024-03-04T13:32:35.972 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:35.977+0000 7f78d3000700 1 -- 172.21.15.169:0/170099343 shutdown_connections 2024-03-04T13:32:35.972 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:35.977+0000 7f78d3000700 1 --2- 172.21.15.169:0/170099343 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f78b0036960 0x7f78b0038e10 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:32:35.972 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:35.977+0000 7f78d3000700 1 --2- 172.21.15.169:0/170099343 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f78cc0fcbc0 0x7f78cc10f0a0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:32:35.972 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:35.977+0000 7f78d3000700 1 -- 172.21.15.169:0/170099343 >> 172.21.15.169:0/170099343 conn(0x7f78cc0f82a0 msgr2=0x7f78cc0f8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:32:35.972 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:35.978+0000 7f78d3000700 1 -- 172.21.15.169:0/170099343 shutdown_connections 2024-03-04T13:32:35.972 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:35.978+0000 7f78d3000700 1 -- 172.21.15.169:0/170099343 wait complete. 2024-03-04T13:32:35.973 INFO:teuthology.orchestra.run.smithi169.stderr:dumped monmap epoch 1 2024-03-04T13:32:37.337 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:36 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:32:37.337 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:36 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:32:37.337 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:36 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:32:37.337 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:36 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:32:37.337 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:36 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "who": "osd/host:smithi169", "name": "osd_memory_target"}]: dispatch 2024-03-04T13:32:37.338 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:36 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:32:37.338 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:36 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:32:37.338 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:36 smithi107 ceph-mon[28739]: from='client.? 172.21.15.169:0/170099343' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-04T13:32:37.758 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-04T13:32:37.758 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 -- ceph mon dump -f json 2024-03-04T13:32:38.180 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-04T13:32:38.337 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:37 smithi107 ceph-mon[28739]: Updating smithi107:/etc/ceph/ceph.conf 2024-03-04T13:32:38.337 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:37 smithi107 ceph-mon[28739]: Updating smithi169:/etc/ceph/ceph.conf 2024-03-04T13:32:38.337 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:37 smithi107 ceph-mon[28739]: Updating smithi169:/var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-04T13:32:38.337 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:37 smithi107 ceph-mon[28739]: Updating smithi107:/var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-04T13:32:39.337 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:38 smithi107 ceph-mon[28739]: Updating smithi169:/etc/ceph/ceph.client.admin.keyring 2024-03-04T13:32:39.337 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:38 smithi107 ceph-mon[28739]: Updating smithi107:/etc/ceph/ceph.client.admin.keyring 2024-03-04T13:32:39.337 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:38 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:32:39.338 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:38 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:32:39.338 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:38 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:32:39.338 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:38 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:32:39.338 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:38 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:32:39.338 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:38 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi169", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-04T13:32:39.338 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:38 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi169", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-03-04T13:32:39.338 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:38 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:32:40.120 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:40.126+0000 7fef91e00700 1 -- 172.21.15.169:0/820475199 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef8c104de0 msgr2=0x7fef8c1051b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:32:40.120 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:40.126+0000 7fef91e00700 1 --2- 172.21.15.169:0/820475199 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef8c104de0 0x7fef8c1051b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fef7c0048c0 tx=0x7fef7c009e30 comp rx=0 tx=0).stop 2024-03-04T13:32:40.120 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:40.126+0000 7fef91e00700 1 -- 172.21.15.169:0/820475199 shutdown_connections 2024-03-04T13:32:40.120 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:40.126+0000 7fef91e00700 1 --2- 172.21.15.169:0/820475199 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef8c104de0 0x7fef8c1051b0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:32:40.120 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:40.126+0000 7fef91e00700 1 -- 172.21.15.169:0/820475199 >> 172.21.15.169:0/820475199 conn(0x7fef8c0fa950 msgr2=0x7fef8c0fcd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:32:40.120 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:40.126+0000 7fef91e00700 1 -- 172.21.15.169:0/820475199 shutdown_connections 2024-03-04T13:32:40.121 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:40.126+0000 7fef91e00700 1 -- 172.21.15.169:0/820475199 wait complete. 2024-03-04T13:32:40.121 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:40.127+0000 7fef91e00700 1 Processor -- start 2024-03-04T13:32:40.121 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:40.127+0000 7fef91e00700 1 -- start start 2024-03-04T13:32:40.121 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:40.127+0000 7fef91e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef8c104de0 0x7fef8c115150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:32:40.121 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:40.127+0000 7fef91e00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef8c115690 con 0x7fef8c104de0 2024-03-04T13:32:40.122 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:40.128+0000 7fef8aa00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef8c104de0 0x7fef8c115150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:32:40.122 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:40.128+0000 7fef8aa00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef8c104de0 0x7fef8c115150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.169:45590/0 (socket says 172.21.15.169:45590) 2024-03-04T13:32:40.122 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:40.128+0000 7fef8aa00700 1 -- 172.21.15.169:0/1455324897 learned_addr learned my addr 172.21.15.169:0/1455324897 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-03-04T13:32:40.122 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:40.129+0000 7fef8aa00700 1 -- 172.21.15.169:0/1455324897 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef7c005040 con 0x7fef8c104de0 2024-03-04T13:32:40.122 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:40.129+0000 7fef8aa00700 1 --2- 172.21.15.169:0/1455324897 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef8c104de0 0x7fef8c115150 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fef7c009330 tx=0x7fef7c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:32:40.124 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:40.129+0000 7fef82a00700 1 -- 172.21.15.169:0/1455324897 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fef7c00be50 con 0x7fef8c104de0 2024-03-04T13:32:40.124 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:40.129+0000 7fef82a00700 1 -- 172.21.15.169:0/1455324897 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fef7c009630 con 0x7fef8c104de0 2024-03-04T13:32:40.124 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:40.129+0000 7fef91e00700 1 -- 172.21.15.169:0/1455324897 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef8c115890 con 0x7fef8c104de0 2024-03-04T13:32:40.124 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:40.130+0000 7fef82a00700 1 -- 172.21.15.169:0/1455324897 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fef7c01c3f0 con 0x7fef8c104de0 2024-03-04T13:32:40.124 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:40.130+0000 7fef91e00700 1 -- 172.21.15.169:0/1455324897 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef8c115d80 con 0x7fef8c104de0 2024-03-04T13:32:40.124 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:40.131+0000 7fef91e00700 1 -- 172.21.15.169:0/1455324897 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef8c05f5c0 con 0x7fef8c104de0 2024-03-04T13:32:40.125 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:40.131+0000 7fef82a00700 1 -- 172.21.15.169:0/1455324897 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 43483+0+0 (secure 0 0 0) 0x7fef7c0097a0 con 0x7fef8c104de0 2024-03-04T13:32:40.126 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:40.132+0000 7fef82a00700 1 --2- 172.21.15.169:0/1455324897 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fef70036960 0x7fef70038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:32:40.126 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:40.132+0000 7fef82a00700 1 -- 172.21.15.169:0/1455324897 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7fef7c049c50 con 0x7fef8c104de0 2024-03-04T13:32:40.126 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:40.133+0000 7fef8a000700 1 --2- 172.21.15.169:0/1455324897 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fef70036960 0x7fef70038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:32:40.127 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:40.134+0000 7fef8a000700 1 --2- 172.21.15.169:0/1455324897 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fef70036960 0x7fef70038e10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fef78009990 tx=0x7fef78006e40 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:32:40.140 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:40.147+0000 7fef82a00700 1 -- 172.21.15.169:0/1455324897 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fef7c011960 con 0x7fef8c104de0 2024-03-04T13:32:40.337 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:39 smithi107 ceph-mon[28739]: Updating smithi169:/var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/config/ceph.client.admin.keyring 2024-03-04T13:32:40.337 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:39 smithi107 ceph-mon[28739]: Updating smithi107:/var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/config/ceph.client.admin.keyring 2024-03-04T13:32:40.337 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:39 smithi107 ceph-mon[28739]: Deploying daemon crash.smithi169 on smithi169 2024-03-04T13:32:40.443 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:40.449+0000 7fef91e00700 1 -- 172.21.15.169:0/1455324897 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fef8c110900 con 0x7fef8c104de0 2024-03-04T13:32:40.443 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:40.450+0000 7fef82a00700 1 -- 172.21.15.169:0/1455324897 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7fef7c011370 con 0x7fef8c104de0 2024-03-04T13:32:40.444 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-03-04T13:32:40.444 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":1,"fsid":"1fc09a58-da2b-11ee-95c6-87774f69a715","modified":"2024-03-04T13:29:38.149007Z","created":"2024-03-04T13:29:38.149007Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi107","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:3300","nonce":0},{"type":"v1","addr":"172.21.15.107:6789","nonce":0}]},"addr":"172.21.15.107:6789/0","public_addr":"172.21.15.107:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-04T13:32:40.446 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:40.452+0000 7fef91e00700 1 -- 172.21.15.169:0/1455324897 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fef70036960 msgr2=0x7fef70038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:32:40.446 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:40.452+0000 7fef91e00700 1 --2- 172.21.15.169:0/1455324897 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fef70036960 0x7fef70038e10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fef78009990 tx=0x7fef78006e40 comp rx=0 tx=0).stop 2024-03-04T13:32:40.446 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:40.452+0000 7fef91e00700 1 -- 172.21.15.169:0/1455324897 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef8c104de0 msgr2=0x7fef8c115150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:32:40.446 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:40.452+0000 7fef91e00700 1 --2- 172.21.15.169:0/1455324897 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef8c104de0 0x7fef8c115150 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fef7c009330 tx=0x7fef7c009e30 comp rx=0 tx=0).stop 2024-03-04T13:32:40.447 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:40.452+0000 7fef91e00700 1 -- 172.21.15.169:0/1455324897 shutdown_connections 2024-03-04T13:32:40.447 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:40.452+0000 7fef91e00700 1 --2- 172.21.15.169:0/1455324897 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fef70036960 0x7fef70038e10 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:32:40.447 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:40.452+0000 7fef91e00700 1 --2- 172.21.15.169:0/1455324897 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef8c104de0 0x7fef8c115150 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:32:40.447 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:40.452+0000 7fef91e00700 1 -- 172.21.15.169:0/1455324897 >> 172.21.15.169:0/1455324897 conn(0x7fef8c0fa950 msgr2=0x7fef8c0fc860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:32:40.447 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:40.452+0000 7fef91e00700 1 -- 172.21.15.169:0/1455324897 shutdown_connections 2024-03-04T13:32:40.447 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:40.453+0000 7fef91e00700 1 -- 172.21.15.169:0/1455324897 wait complete. 2024-03-04T13:32:40.447 INFO:teuthology.orchestra.run.smithi169.stderr:dumped monmap epoch 1 2024-03-04T13:32:41.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:41 smithi107 ceph-mon[28739]: from='client.? 172.21.15.169:0/1455324897' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-04T13:32:42.772 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-04T13:32:42.772 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 -- ceph mon dump -f json 2024-03-04T13:32:42.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:42 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:32:43.536 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-04T13:32:44.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:44 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:32:44.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:44 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:32:44.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:44 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:32:44.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:44 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:32:44.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:44 smithi107 ceph-mon[28739]: Deploying daemon node-exporter.smithi169 on smithi169 2024-03-04T13:32:45.392 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:45.397+0000 7ff5d5800700 1 -- 172.21.15.169:0/630808638 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5d0070330 msgr2=0x7ff5d0070700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:32:45.392 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:45.397+0000 7ff5d5800700 1 --2- 172.21.15.169:0/630808638 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5d0070330 0x7ff5d0070700 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff5c00048c0 tx=0x7ff5c0009e30 comp rx=0 tx=0).stop 2024-03-04T13:32:45.392 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:45.397+0000 7ff5d5800700 1 -- 172.21.15.169:0/630808638 shutdown_connections 2024-03-04T13:32:45.392 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:45.397+0000 7ff5d5800700 1 --2- 172.21.15.169:0/630808638 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5d0070330 0x7ff5d0070700 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:32:45.392 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:45.397+0000 7ff5d5800700 1 -- 172.21.15.169:0/630808638 >> 172.21.15.169:0/630808638 conn(0x7ff5d00f8760 msgr2=0x7ff5d00fab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:32:45.392 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:45.397+0000 7ff5d5800700 1 -- 172.21.15.169:0/630808638 shutdown_connections 2024-03-04T13:32:45.392 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:45.398+0000 7ff5d5800700 1 -- 172.21.15.169:0/630808638 wait complete. 2024-03-04T13:32:45.393 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:45.399+0000 7ff5d5800700 1 Processor -- start 2024-03-04T13:32:45.393 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:45.399+0000 7ff5d5800700 1 -- start start 2024-03-04T13:32:45.393 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:45.399+0000 7ff5d5800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5d0070330 0x7ff5d0115190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:32:45.393 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:45.399+0000 7ff5d5800700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5d01156d0 con 0x7ff5d0070330 2024-03-04T13:32:45.394 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:45.400+0000 7ff5cea00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5d0070330 0x7ff5d0115190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:32:45.394 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:45.400+0000 7ff5cea00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5d0070330 0x7ff5d0115190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.169:40380/0 (socket says 172.21.15.169:40380) 2024-03-04T13:32:45.394 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:45.400+0000 7ff5cea00700 1 -- 172.21.15.169:0/3491310518 learned_addr learned my addr 172.21.15.169:0/3491310518 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-03-04T13:32:45.394 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:45.401+0000 7ff5cea00700 1 -- 172.21.15.169:0/3491310518 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5c0005040 con 0x7ff5d0070330 2024-03-04T13:32:45.395 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:45.401+0000 7ff5cea00700 1 --2- 172.21.15.169:0/3491310518 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5d0070330 0x7ff5d0115190 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff5c0009330 tx=0x7ff5c0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:32:45.395 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:45.402+0000 7ff5c6a00700 1 -- 172.21.15.169:0/3491310518 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff5c000be50 con 0x7ff5d0070330 2024-03-04T13:32:45.395 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:45.402+0000 7ff5d5800700 1 -- 172.21.15.169:0/3491310518 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5d01158d0 con 0x7ff5d0070330 2024-03-04T13:32:45.395 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:45.402+0000 7ff5c6a00700 1 -- 172.21.15.169:0/3491310518 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff5c0009630 con 0x7ff5d0070330 2024-03-04T13:32:45.396 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:45.402+0000 7ff5c6a00700 1 -- 172.21.15.169:0/3491310518 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff5c001c3f0 con 0x7ff5d0070330 2024-03-04T13:32:45.396 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:45.402+0000 7ff5d5800700 1 -- 172.21.15.169:0/3491310518 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5d0115d40 con 0x7ff5d0070330 2024-03-04T13:32:45.396 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:45.403+0000 7ff5d5800700 1 -- 172.21.15.169:0/3491310518 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5d005f5c0 con 0x7ff5d0070330 2024-03-04T13:32:45.397 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:45.404+0000 7ff5c6a00700 1 -- 172.21.15.169:0/3491310518 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 43483+0+0 (secure 0 0 0) 0x7ff5c00097a0 con 0x7ff5d0070330 2024-03-04T13:32:45.398 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:45.404+0000 7ff5c6a00700 1 --2- 172.21.15.169:0/3491310518 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7ff5b40369b0 0x7ff5b4038e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:32:45.398 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:45.404+0000 7ff5c6a00700 1 -- 172.21.15.169:0/3491310518 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7ff5c0049b60 con 0x7ff5d0070330 2024-03-04T13:32:45.398 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:45.405+0000 7ff5ce000700 1 --2- 172.21.15.169:0/3491310518 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7ff5b40369b0 0x7ff5b4038e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:32:45.399 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:45.406+0000 7ff5ce000700 1 --2- 172.21.15.169:0/3491310518 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7ff5b40369b0 0x7ff5b4038e60 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc009990 tx=0x7ff5bc006e40 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:32:45.413 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:45.419+0000 7ff5c6a00700 1 -- 172.21.15.169:0/3491310518 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7ff5c0011940 con 0x7ff5d0070330 2024-03-04T13:32:45.731 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:45.736+0000 7ff5d5800700 1 -- 172.21.15.169:0/3491310518 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7ff5d0109110 con 0x7ff5d0070330 2024-03-04T13:32:45.731 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:45.737+0000 7ff5c6a00700 1 -- 172.21.15.169:0/3491310518 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7ff5c0011370 con 0x7ff5d0070330 2024-03-04T13:32:45.732 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-03-04T13:32:45.732 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":1,"fsid":"1fc09a58-da2b-11ee-95c6-87774f69a715","modified":"2024-03-04T13:29:38.149007Z","created":"2024-03-04T13:29:38.149007Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi107","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:3300","nonce":0},{"type":"v1","addr":"172.21.15.107:6789","nonce":0}]},"addr":"172.21.15.107:6789/0","public_addr":"172.21.15.107:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-04T13:32:45.735 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:45.741+0000 7ff5d5800700 1 -- 172.21.15.169:0/3491310518 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7ff5b40369b0 msgr2=0x7ff5b4038e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:32:45.735 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:45.741+0000 7ff5d5800700 1 --2- 172.21.15.169:0/3491310518 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7ff5b40369b0 0x7ff5b4038e60 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc009990 tx=0x7ff5bc006e40 comp rx=0 tx=0).stop 2024-03-04T13:32:45.735 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:45.741+0000 7ff5d5800700 1 -- 172.21.15.169:0/3491310518 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5d0070330 msgr2=0x7ff5d0115190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:32:45.735 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:45.741+0000 7ff5d5800700 1 --2- 172.21.15.169:0/3491310518 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5d0070330 0x7ff5d0115190 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff5c0009330 tx=0x7ff5c0009e30 comp rx=0 tx=0).stop 2024-03-04T13:32:45.735 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:45.741+0000 7ff5d5800700 1 -- 172.21.15.169:0/3491310518 shutdown_connections 2024-03-04T13:32:45.735 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:45.741+0000 7ff5d5800700 1 --2- 172.21.15.169:0/3491310518 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7ff5b40369b0 0x7ff5b4038e60 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:32:45.736 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:45.741+0000 7ff5d5800700 1 --2- 172.21.15.169:0/3491310518 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5d0070330 0x7ff5d0115190 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:32:45.736 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:45.741+0000 7ff5d5800700 1 -- 172.21.15.169:0/3491310518 >> 172.21.15.169:0/3491310518 conn(0x7ff5d00f8760 msgr2=0x7ff5d0103720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:32:45.736 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:45.742+0000 7ff5d5800700 1 -- 172.21.15.169:0/3491310518 shutdown_connections 2024-03-04T13:32:45.736 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:45.742+0000 7ff5d5800700 1 -- 172.21.15.169:0/3491310518 wait complete. 2024-03-04T13:32:45.736 INFO:teuthology.orchestra.run.smithi169.stderr:dumped monmap epoch 1 2024-03-04T13:32:46.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:46 smithi107 ceph-mon[28739]: from='client.? 172.21.15.169:0/3491310518' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-04T13:32:47.388 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-04T13:32:47.389 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 -- ceph mon dump -f json 2024-03-04T13:32:48.119 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-04T13:32:48.587 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:48 smithi107 ceph-mon[28739]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:32:48.587 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:48 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:32:50.508 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:50.513+0000 7f07e2000700 1 -- 172.21.15.169:0/679116615 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07dc070330 msgr2=0x7f07dc070700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:32:50.508 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:50.513+0000 7f07e2000700 1 --2- 172.21.15.169:0/679116615 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07dc070330 0x7f07dc070700 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f07cc0048c0 tx=0x7f07cc009e30 comp rx=0 tx=0).stop 2024-03-04T13:32:50.509 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:50.513+0000 7f07e2000700 1 -- 172.21.15.169:0/679116615 shutdown_connections 2024-03-04T13:32:50.509 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:50.513+0000 7f07e2000700 1 --2- 172.21.15.169:0/679116615 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07dc070330 0x7f07dc070700 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:32:50.509 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:50.513+0000 7f07e2000700 1 -- 172.21.15.169:0/679116615 >> 172.21.15.169:0/679116615 conn(0x7f07dc0f8760 msgr2=0x7f07dc0fab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:32:50.509 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:50.513+0000 7f07e2000700 1 -- 172.21.15.169:0/679116615 shutdown_connections 2024-03-04T13:32:50.509 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:50.513+0000 7f07e2000700 1 -- 172.21.15.169:0/679116615 wait complete. 2024-03-04T13:32:50.509 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:50.514+0000 7f07e2000700 1 Processor -- start 2024-03-04T13:32:50.509 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:50.514+0000 7f07e2000700 1 -- start start 2024-03-04T13:32:50.510 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:50.515+0000 7f07e2000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07dc070330 0x7f07dc115190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:32:50.510 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:50.515+0000 7f07e2000700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07dc1156d0 con 0x7f07dc070330 2024-03-04T13:32:50.510 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:50.515+0000 7f07db400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07dc070330 0x7f07dc115190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:32:50.510 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:50.515+0000 7f07db400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07dc070330 0x7f07dc115190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.169:40406/0 (socket says 172.21.15.169:40406) 2024-03-04T13:32:50.510 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:50.516+0000 7f07db400700 1 -- 172.21.15.169:0/2472705882 learned_addr learned my addr 172.21.15.169:0/2472705882 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-03-04T13:32:50.511 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:50.516+0000 7f07db400700 1 -- 172.21.15.169:0/2472705882 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07cc005040 con 0x7f07dc070330 2024-03-04T13:32:50.511 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:50.517+0000 7f07db400700 1 --2- 172.21.15.169:0/2472705882 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07dc070330 0x7f07dc115190 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f07cc009300 tx=0x7f07cc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:32:50.512 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:50.517+0000 7f07d3400700 1 -- 172.21.15.169:0/2472705882 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f07cc004e60 con 0x7f07dc070330 2024-03-04T13:32:50.512 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:50.517+0000 7f07e2000700 1 -- 172.21.15.169:0/2472705882 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07dc1158d0 con 0x7f07dc070330 2024-03-04T13:32:50.512 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:50.517+0000 7f07e2000700 1 -- 172.21.15.169:0/2472705882 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07dc115d40 con 0x7f07dc070330 2024-03-04T13:32:50.512 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:50.517+0000 7f07d3400700 1 -- 172.21.15.169:0/2472705882 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f07cc013420 con 0x7f07dc070330 2024-03-04T13:32:50.512 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:50.517+0000 7f07d3400700 1 -- 172.21.15.169:0/2472705882 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f07cc01c5b0 con 0x7f07dc070330 2024-03-04T13:32:50.513 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:50.518+0000 7f07e2000700 1 -- 172.21.15.169:0/2472705882 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07dc05f5c0 con 0x7f07dc070330 2024-03-04T13:32:50.513 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:50.519+0000 7f07d3400700 1 -- 172.21.15.169:0/2472705882 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 43483+0+0 (secure 0 0 0) 0x7f07cc013a90 con 0x7f07dc070330 2024-03-04T13:32:50.514 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:50.519+0000 7f07d3400700 1 --2- 172.21.15.169:0/2472705882 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f07c00369b0 0x7f07c0038e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:32:50.514 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:50.519+0000 7f07d3400700 1 -- 172.21.15.169:0/2472705882 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f07cc049ac0 con 0x7f07dc070330 2024-03-04T13:32:50.514 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:50.520+0000 7f07daa00700 1 --2- 172.21.15.169:0/2472705882 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f07c00369b0 0x7f07c0038e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:32:50.515 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:50.521+0000 7f07daa00700 1 --2- 172.21.15.169:0/2472705882 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f07c00369b0 0x7f07c0038e60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f07c8009990 tx=0x7f07c8006e40 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:32:50.528 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:50.534+0000 7f07d3400700 1 -- 172.21.15.169:0/2472705882 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f07cc04be60 con 0x7f07dc070330 2024-03-04T13:32:50.834 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:50.840+0000 7f07e2000700 1 -- 172.21.15.169:0/2472705882 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f07dc110850 con 0x7f07dc070330 2024-03-04T13:32:50.835 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:50.841+0000 7f07d3400700 1 -- 172.21.15.169:0/2472705882 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7f07cc011960 con 0x7f07dc070330 2024-03-04T13:32:50.836 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-03-04T13:32:50.836 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":1,"fsid":"1fc09a58-da2b-11ee-95c6-87774f69a715","modified":"2024-03-04T13:29:38.149007Z","created":"2024-03-04T13:29:38.149007Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi107","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:3300","nonce":0},{"type":"v1","addr":"172.21.15.107:6789","nonce":0}]},"addr":"172.21.15.107:6789/0","public_addr":"172.21.15.107:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-04T13:32:50.839 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:50.844+0000 7f07e2000700 1 -- 172.21.15.169:0/2472705882 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f07c00369b0 msgr2=0x7f07c0038e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:32:50.839 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:50.844+0000 7f07e2000700 1 --2- 172.21.15.169:0/2472705882 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f07c00369b0 0x7f07c0038e60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f07c8009990 tx=0x7f07c8006e40 comp rx=0 tx=0).stop 2024-03-04T13:32:50.839 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:50.844+0000 7f07e2000700 1 -- 172.21.15.169:0/2472705882 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07dc070330 msgr2=0x7f07dc115190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:32:50.839 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:50.844+0000 7f07e2000700 1 --2- 172.21.15.169:0/2472705882 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07dc070330 0x7f07dc115190 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f07cc009300 tx=0x7f07cc009e30 comp rx=0 tx=0).stop 2024-03-04T13:32:50.839 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:50.844+0000 7f07e2000700 1 -- 172.21.15.169:0/2472705882 shutdown_connections 2024-03-04T13:32:50.839 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:50.844+0000 7f07e2000700 1 --2- 172.21.15.169:0/2472705882 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f07c00369b0 0x7f07c0038e60 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:32:50.839 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:50.844+0000 7f07e2000700 1 --2- 172.21.15.169:0/2472705882 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07dc070330 0x7f07dc115190 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:32:50.839 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:50.844+0000 7f07e2000700 1 -- 172.21.15.169:0/2472705882 >> 172.21.15.169:0/2472705882 conn(0x7f07dc0f8760 msgr2=0x7f07dc103720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:32:50.839 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:50.844+0000 7f07e2000700 1 -- 172.21.15.169:0/2472705882 shutdown_connections 2024-03-04T13:32:50.839 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:50.844+0000 7f07e2000700 1 -- 172.21.15.169:0/2472705882 wait complete. 2024-03-04T13:32:50.840 INFO:teuthology.orchestra.run.smithi169.stderr:dumped monmap epoch 1 2024-03-04T13:32:51.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:50 smithi107 ceph-mon[28739]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:32:51.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:50 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:32:51.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:50 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:32:51.088 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:50 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:32:51.088 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:50 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:32:51.088 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:50 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi169.eihczr", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-04T13:32:51.088 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:50 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.smithi169.eihczr", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-04T13:32:51.088 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:50 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-04T13:32:51.088 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:50 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:32:51.088 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:50 smithi107 ceph-mon[28739]: Deploying daemon mgr.smithi169.eihczr on smithi169 2024-03-04T13:32:52.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:51 smithi107 ceph-mon[28739]: from='client.? 172.21.15.169:0/2472705882' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-04T13:32:53.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:52 smithi107 ceph-mon[28739]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:32:53.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:52 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:32:53.429 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-04T13:32:53.430 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 -- ceph mon dump -f json 2024-03-04T13:32:53.896 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-04T13:32:55.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:54 smithi107 ceph-mon[28739]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:32:55.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:54 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:32:55.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:54 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:32:55.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:54 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:32:55.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:54 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:32:55.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:54 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-04T13:32:55.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:54 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:32:55.088 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:54 smithi107 ceph-mon[28739]: Deploying daemon mon.smithi169 on smithi169 2024-03-04T13:32:56.049 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:56.055+0000 7f903ee00700 1 -- 172.21.15.169:0/1316313742 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9038102bd0 msgr2=0x7f9038102fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:32:56.050 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:56.055+0000 7f903ee00700 1 --2- 172.21.15.169:0/1316313742 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9038102bd0 0x7f9038102fa0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f90280048c0 tx=0x7f9028009e30 comp rx=0 tx=0).stop 2024-03-04T13:32:56.050 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:56.055+0000 7f903ee00700 1 -- 172.21.15.169:0/1316313742 shutdown_connections 2024-03-04T13:32:56.050 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:56.055+0000 7f903ee00700 1 --2- 172.21.15.169:0/1316313742 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9038102bd0 0x7f9038102fa0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:32:56.050 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:56.055+0000 7f903ee00700 1 -- 172.21.15.169:0/1316313742 >> 172.21.15.169:0/1316313742 conn(0x7f90380f8760 msgr2=0x7f90380fab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:32:56.050 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:56.055+0000 7f903ee00700 1 -- 172.21.15.169:0/1316313742 shutdown_connections 2024-03-04T13:32:56.050 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:56.056+0000 7f903ee00700 1 -- 172.21.15.169:0/1316313742 wait complete. 2024-03-04T13:32:56.051 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:56.057+0000 7f903ee00700 1 Processor -- start 2024-03-04T13:32:56.051 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:56.057+0000 7f903ee00700 1 -- start start 2024-03-04T13:32:56.051 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:56.058+0000 7f903ee00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9038102bd0 0x7f9038112f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:32:56.051 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:56.058+0000 7f903ee00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9038113460 con 0x7f9038102bd0 2024-03-04T13:32:56.052 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:56.059+0000 7f9037e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9038102bd0 0x7f9038112f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:32:56.052 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:56.059+0000 7f9037e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9038102bd0 0x7f9038112f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.169:50616/0 (socket says 172.21.15.169:50616) 2024-03-04T13:32:56.052 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:56.059+0000 7f9037e00700 1 -- 172.21.15.169:0/3864568642 learned_addr learned my addr 172.21.15.169:0/3864568642 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-03-04T13:32:56.053 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:56.059+0000 7f9037e00700 1 -- 172.21.15.169:0/3864568642 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9028005040 con 0x7f9038102bd0 2024-03-04T13:32:56.053 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:56.060+0000 7f9037e00700 1 --2- 172.21.15.169:0/3864568642 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9038102bd0 0x7f9038112f20 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9028009300 tx=0x7f9028009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:32:56.054 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:56.060+0000 7f902fe00700 1 -- 172.21.15.169:0/3864568642 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9028004e60 con 0x7f9038102bd0 2024-03-04T13:32:56.054 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:56.060+0000 7f903ee00700 1 -- 172.21.15.169:0/3864568642 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9038113660 con 0x7f9038102bd0 2024-03-04T13:32:56.054 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:56.060+0000 7f902fe00700 1 -- 172.21.15.169:0/3864568642 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9028013420 con 0x7f9038102bd0 2024-03-04T13:32:56.054 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:56.060+0000 7f902fe00700 1 -- 172.21.15.169:0/3864568642 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9028025560 con 0x7f9038102bd0 2024-03-04T13:32:56.054 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:56.060+0000 7f903ee00700 1 -- 172.21.15.169:0/3864568642 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9038113b50 con 0x7f9038102bd0 2024-03-04T13:32:56.055 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:56.061+0000 7f903ee00700 1 -- 172.21.15.169:0/3864568642 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f903805f5c0 con 0x7f9038102bd0 2024-03-04T13:32:56.055 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:56.062+0000 7f902fe00700 1 -- 172.21.15.169:0/3864568642 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 43483+0+0 (secure 0 0 0) 0x7f9028013a90 con 0x7f9038102bd0 2024-03-04T13:32:56.056 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:56.062+0000 7f902fe00700 1 --2- 172.21.15.169:0/3864568642 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f901c036960 0x7f901c038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:32:56.056 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:56.062+0000 7f902fe00700 1 -- 172.21.15.169:0/3864568642 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f9028049b30 con 0x7f9038102bd0 2024-03-04T13:32:56.056 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:56.063+0000 7f9037400700 1 --2- 172.21.15.169:0/3864568642 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f901c036960 0x7f901c038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:32:56.057 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:56.064+0000 7f9037400700 1 --2- 172.21.15.169:0/3864568642 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f901c036960 0x7f901c038e10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9024009990 tx=0x7f9024006e40 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:32:56.071 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:56.077+0000 7f902fe00700 1 -- 172.21.15.169:0/3864568642 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f9028023300 con 0x7f9038102bd0 2024-03-04T13:32:56.360 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:56.366+0000 7f903ee00700 1 -- 172.21.15.169:0/3864568642 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f903810e6d0 con 0x7f9038102bd0 2024-03-04T13:32:56.361 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:56.367+0000 7f902fe00700 1 -- 172.21.15.169:0/3864568642 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7f9028011b60 con 0x7f9038102bd0 2024-03-04T13:32:56.361 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-03-04T13:32:56.361 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":1,"fsid":"1fc09a58-da2b-11ee-95c6-87774f69a715","modified":"2024-03-04T13:29:38.149007Z","created":"2024-03-04T13:29:38.149007Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi107","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:3300","nonce":0},{"type":"v1","addr":"172.21.15.107:6789","nonce":0}]},"addr":"172.21.15.107:6789/0","public_addr":"172.21.15.107:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-04T13:32:56.363 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:56.370+0000 7f903ee00700 1 -- 172.21.15.169:0/3864568642 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f901c036960 msgr2=0x7f901c038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:32:56.364 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:56.370+0000 7f903ee00700 1 --2- 172.21.15.169:0/3864568642 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f901c036960 0x7f901c038e10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9024009990 tx=0x7f9024006e40 comp rx=0 tx=0).stop 2024-03-04T13:32:56.364 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:56.370+0000 7f903ee00700 1 -- 172.21.15.169:0/3864568642 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9038102bd0 msgr2=0x7f9038112f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:32:56.364 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:56.370+0000 7f903ee00700 1 --2- 172.21.15.169:0/3864568642 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9038102bd0 0x7f9038112f20 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9028009300 tx=0x7f9028009e30 comp rx=0 tx=0).stop 2024-03-04T13:32:56.364 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:56.370+0000 7f903ee00700 1 -- 172.21.15.169:0/3864568642 shutdown_connections 2024-03-04T13:32:56.364 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:56.370+0000 7f903ee00700 1 --2- 172.21.15.169:0/3864568642 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f901c036960 0x7f901c038e10 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:32:56.365 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:56.370+0000 7f903ee00700 1 --2- 172.21.15.169:0/3864568642 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9038102bd0 0x7f9038112f20 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:32:56.365 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:56.370+0000 7f903ee00700 1 -- 172.21.15.169:0/3864568642 >> 172.21.15.169:0/3864568642 conn(0x7f90380f8760 msgr2=0x7f90380fa600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:32:56.365 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:56.370+0000 7f903ee00700 1 -- 172.21.15.169:0/3864568642 shutdown_connections 2024-03-04T13:32:56.365 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:32:56.371+0000 7f903ee00700 1 -- 172.21.15.169:0/3864568642 wait complete. 2024-03-04T13:32:56.365 INFO:teuthology.orchestra.run.smithi169.stderr:dumped monmap epoch 1 2024-03-04T13:32:57.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:56 smithi107 ceph-mon[28739]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:32:57.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:56 smithi107 ceph-mon[28739]: from='client.? 172.21.15.169:0/3864568642' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-04T13:32:58.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:57 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:32:58.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:57 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:32:58.309 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-04T13:32:58.309 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 -- ceph mon dump -f json 2024-03-04T13:32:59.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:32:58 smithi107 ceph-mon[28739]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:32:59.237 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: load: jerasure load: lrc 2024-03-04T13:32:59.237 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: RocksDB version: 6.15.5 2024-03-04T13:32:59.237 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-03-04T13:32:59.237 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Compile date Mar 3 2024 2024-03-04T13:32:59.237 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: DB SUMMARY 2024-03-04T13:32:59.237 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: DB Session ID: QDISSZH10D24XTLE386V 2024-03-04T13:32:59.237 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: CURRENT file: CURRENT 2024-03-04T13:32:59.293 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/mon.smithi169/config 2024-03-04T13:32:59.552 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: IDENTITY file: IDENTITY 2024-03-04T13:32:59.552 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: MANIFEST file: MANIFEST-000003 size: 57 Bytes 2024-03-04T13:32:59.552 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi169/store.db dir, Total Num: 0, files: 2024-03-04T13:32:59.553 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi169/store.db: 000004.log size: 511 ; 2024-03-04T13:32:59.553 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.error_if_exists: 0 2024-03-04T13:32:59.553 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.create_if_missing: 0 2024-03-04T13:32:59.553 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.paranoid_checks: 1 2024-03-04T13:32:59.553 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-04T13:32:59.553 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.env: 0x55847f41da80 2024-03-04T13:32:59.553 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.fs: Posix File System 2024-03-04T13:32:59.553 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.info_log: 0x558480c13cc0 2024-03-04T13:32:59.553 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.max_file_opening_threads: 16 2024-03-04T13:32:59.553 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.statistics: (nil) 2024-03-04T13:32:59.553 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.use_fsync: 0 2024-03-04T13:32:59.554 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.max_log_file_size: 0 2024-03-04T13:32:59.554 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-04T13:32:59.554 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-04T13:32:59.554 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.keep_log_file_num: 1000 2024-03-04T13:32:59.554 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.recycle_log_file_num: 0 2024-03-04T13:32:59.554 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.allow_fallocate: 1 2024-03-04T13:32:59.554 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.allow_mmap_reads: 0 2024-03-04T13:32:59.554 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.allow_mmap_writes: 0 2024-03-04T13:32:59.554 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.use_direct_reads: 0 2024-03-04T13:32:59.554 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-04T13:32:59.554 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.create_missing_column_families: 0 2024-03-04T13:32:59.554 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.db_log_dir: 2024-03-04T13:32:59.554 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-smithi169/store.db 2024-03-04T13:32:59.554 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-04T13:32:59.555 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-04T13:32:59.555 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-04T13:32:59.555 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-04T13:32:59.555 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-04T13:32:59.555 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-04T13:32:59.555 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.advise_random_on_open: 1 2024-03-04T13:32:59.555 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.db_write_buffer_size: 0 2024-03-04T13:32:59.555 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.write_buffer_manager: 0x558481988ae0 2024-03-04T13:32:59.555 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-04T13:32:59.555 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-03-04T13:32:59.556 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-04T13:32:59.556 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-04T13:32:59.556 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.rate_limiter: (nil) 2024-03-04T13:32:59.559 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-04T13:32:59.559 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.wal_recovery_mode: 2 2024-03-04T13:32:59.559 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.enable_thread_tracking: 0 2024-03-04T13:32:59.559 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.enable_pipelined_write: 0 2024-03-04T13:32:59.559 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.unordered_write: 0 2024-03-04T13:32:59.559 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-04T13:32:59.560 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-04T13:32:59.560 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-04T13:32:59.560 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-04T13:32:59.560 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.row_cache: None 2024-03-04T13:32:59.560 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.wal_filter: None 2024-03-04T13:32:59.560 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-04T13:32:59.560 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.allow_ingest_behind: 0 2024-03-04T13:32:59.560 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.preserve_deletes: 0 2024-03-04T13:32:59.560 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.two_write_queues: 0 2024-03-04T13:32:59.560 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.manual_wal_flush: 0 2024-03-04T13:32:59.560 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.atomic_flush: 0 2024-03-04T13:32:59.560 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-04T13:32:59.560 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-04T13:32:59.561 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-04T13:32:59.561 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.log_readahead_size: 0 2024-03-04T13:32:59.561 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-04T13:32:59.561 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.best_efforts_recovery: 0 2024-03-04T13:32:59.561 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-04T13:32:59.561 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-04T13:32:59.561 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.allow_data_in_errors: 0 2024-03-04T13:32:59.561 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.db_host_id: __hostname__ 2024-03-04T13:32:59.561 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.max_background_jobs: 2 2024-03-04T13:32:59.561 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.max_background_compactions: -1 2024-03-04T13:32:59.561 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.max_subcompactions: 1 2024-03-04T13:32:59.561 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-04T13:32:59.561 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-04T13:32:59.562 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-04T13:32:59.562 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.max_total_wal_size: 0 2024-03-04T13:32:59.562 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-04T13:32:59.562 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-04T13:32:59.563 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-04T13:32:59.563 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-04T13:32:59.563 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.max_open_files: -1 2024-03-04T13:32:59.564 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.bytes_per_sync: 0 2024-03-04T13:32:59.564 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-04T13:32:59.564 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-04T13:32:59.564 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.compaction_readahead_size: 0 2024-03-04T13:32:59.564 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.max_background_flushes: -1 2024-03-04T13:32:59.564 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Compression algorithms supported: 2024-03-04T13:32:59.564 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-04T13:32:59.564 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: kZSTD supported: 0 2024-03-04T13:32:59.564 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: kXpressCompression supported: 0 2024-03-04T13:32:59.564 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: kLZ4HCCompression supported: 1 2024-03-04T13:32:59.564 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: kZlibCompression supported: 1 2024-03-04T13:32:59.564 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: kSnappyCompression supported: 1 2024-03-04T13:32:59.565 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: kLZ4Compression supported: 1 2024-03-04T13:32:59.565 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: kBZip2Compression supported: 0 2024-03-04T13:32:59.565 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-04T13:32:59.565 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: [db/version_set.cc:4724] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi169/store.db/MANIFEST-000003 2024-03-04T13:32:59.565 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: [db/column_family.cc:595] --------------- Options for column family [default]: 2024-03-04T13:32:59.565 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-04T13:32:59.565 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.merge_operator: 2024-03-04T13:32:59.565 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.compaction_filter: None 2024-03-04T13:32:59.565 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.compaction_filter_factory: None 2024-03-04T13:32:59.565 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.sst_partitioner_factory: None 2024-03-04T13:32:59.565 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-04T13:32:59.565 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-04T13:32:59.565 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x558480c68f58) 2024-03-04T13:32:59.566 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: cache_index_and_filter_blocks: 1 2024-03-04T13:32:59.566 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-04T13:32:59.566 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-04T13:32:59.566 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: pin_top_level_index_and_filter: 1 2024-03-04T13:32:59.566 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: index_type: 0 2024-03-04T13:32:59.566 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: data_block_index_type: 0 2024-03-04T13:32:59.566 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: index_shortening: 1 2024-03-04T13:32:59.566 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-04T13:32:59.566 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: hash_index_allow_collision: 1 2024-03-04T13:32:59.566 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: checksum: 1 2024-03-04T13:32:59.566 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: no_block_cache: 0 2024-03-04T13:32:59.566 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: block_cache: 0x558480c9b090 2024-03-04T13:32:59.567 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: block_cache_name: BinnedLRUCache 2024-03-04T13:32:59.567 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: block_cache_options: 2024-03-04T13:32:59.567 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: capacity : 536870912 2024-03-04T13:32:59.567 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: num_shard_bits : 4 2024-03-04T13:32:59.567 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: strict_capacity_limit : 0 2024-03-04T13:32:59.567 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: high_pri_pool_ratio: 0.000 2024-03-04T13:32:59.567 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: block_cache_compressed: (nil) 2024-03-04T13:32:59.567 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: persistent_cache: (nil) 2024-03-04T13:32:59.567 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: block_size: 4096 2024-03-04T13:32:59.567 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: block_size_deviation: 10 2024-03-04T13:32:59.567 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: block_restart_interval: 16 2024-03-04T13:32:59.567 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: index_block_restart_interval: 1 2024-03-04T13:32:59.567 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: metadata_block_size: 4096 2024-03-04T13:32:59.567 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: partition_filters: 0 2024-03-04T13:32:59.568 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: use_delta_encoding: 1 2024-03-04T13:32:59.568 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2024-03-04T13:32:59.568 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: whole_key_filtering: 1 2024-03-04T13:32:59.568 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: verify_compression: 0 2024-03-04T13:32:59.568 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: read_amp_bytes_per_bit: 0 2024-03-04T13:32:59.568 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: format_version: 4 2024-03-04T13:32:59.568 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: enable_index_compression: 1 2024-03-04T13:32:59.568 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: block_align: 0 2024-03-04T13:32:59.568 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.write_buffer_size: 33554432 2024-03-04T13:32:59.568 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.max_write_buffer_number: 2 2024-03-04T13:32:59.568 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.compression: NoCompression 2024-03-04T13:32:59.568 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.bottommost_compression: Disabled 2024-03-04T13:32:59.568 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.prefix_extractor: nullptr 2024-03-04T13:32:59.568 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-04T13:32:59.569 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.num_levels: 7 2024-03-04T13:32:59.569 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-04T13:32:59.569 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-04T13:32:59.569 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-04T13:32:59.569 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-04T13:32:59.569 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-04T13:32:59.569 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-04T13:32:59.569 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-04T13:32:59.569 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-04T13:32:59.569 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-04T13:32:59.569 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-04T13:32:59.569 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-04T13:32:59.569 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.compression_opts.level: 32767 2024-03-04T13:32:59.569 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.compression_opts.strategy: 0 2024-03-04T13:32:59.570 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-04T13:32:59.570 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-04T13:32:59.570 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-04T13:32:59.570 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.compression_opts.enabled: false 2024-03-04T13:32:59.570 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-04T13:32:59.570 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-04T13:32:59.570 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-04T13:32:59.570 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.target_file_size_base: 67108864 2024-03-04T13:32:59.570 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-04T13:32:59.570 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-04T13:32:59.570 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-04T13:32:59.570 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-04T13:32:59.570 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-04T13:32:59.571 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-04T13:32:59.571 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-04T13:32:59.571 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-04T13:32:59.571 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-04T13:32:59.571 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-04T13:32:59.571 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-04T13:32:59.571 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-04T13:32:59.571 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-04T13:32:59.571 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.arena_block_size: 4194304 2024-03-04T13:32:59.571 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-04T13:32:59.571 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-04T13:32:59.571 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-03-04T13:32:59.571 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.disable_auto_compactions: 0 2024-03-04T13:32:59.571 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-04T13:32:59.572 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-04T13:32:59.572 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-04T13:32:59.572 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-04T13:32:59.572 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-04T13:32:59.572 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-04T13:32:59.572 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-04T13:32:59.572 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-04T13:32:59.572 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-04T13:32:59.572 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-04T13:32:59.572 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.table_properties_collectors: 2024-03-04T13:32:59.572 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.inplace_update_support: 0 2024-03-04T13:32:59.572 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-04T13:32:59.572 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-04T13:32:59.572 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-04T13:32:59.573 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-04T13:32:59.573 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.bloom_locality: 0 2024-03-04T13:32:59.573 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.max_successive_merges: 0 2024-03-04T13:32:59.573 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-04T13:32:59.573 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.paranoid_file_checks: 0 2024-03-04T13:32:59.573 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.force_consistency_checks: 1 2024-03-04T13:32:59.573 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.report_bg_io_stats: 0 2024-03-04T13:32:59.573 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.ttl: 2592000 2024-03-04T13:32:59.573 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-04T13:32:59.573 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.enable_blob_files: false 2024-03-04T13:32:59.573 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.min_blob_size: 0 2024-03-04T13:32:59.573 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.blob_file_size: 268435456 2024-03-04T13:32:59.574 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-04T13:32:59.574 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-04T13:32:59.574 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-04T13:32:59.574 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: [db/version_set.cc:4764] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi169/store.db/MANIFEST-000003 succeeded,manifest_file_number is 3, next_file_number is 5, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-03-04T13:32:59.574 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: [db/version_set.cc:4779] Column family [default] (ID 0), log number is 0 2024-03-04T13:32:59.574 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: [db/version_set.cc:4082] Creating manifest 7 2024-03-04T13:32:59.574 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: EVENT_LOG_v1 {"time_micros": 1709559179247673, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-03-04T13:32:59.574 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: [db/db_impl/db_impl_open.cc:845] Recovering log #4 mode 2 2024-03-04T13:32:59.574 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: [table/block_based/filter_policy.cc:991] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-03-04T13:32:59.574 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: EVENT_LOG_v1 {"time_micros": 1709559179248433, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1545, "file_checksum": "", "file_checksum_func_name": "Unknown", "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1709559179, "oldest_key_time": 0, "file_creation_time": 0, "db_id": "04792177-651b-4587-83fe-f963a16a352a", "db_session_id": "QDISSZH10D24XTLE386V"}} 2024-03-04T13:32:59.574 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: [db/version_set.cc:4082] Creating manifest 9 2024-03-04T13:32:59.574 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: EVENT_LOG_v1 {"time_micros": 1709559179249023, "job": 1, "event": "recovery_finished"} 2024-03-04T13:32:59.574 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: [file/delete_scheduler.cc:69] Deleted file /var/lib/ceph/mon/ceph-smithi169/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-04T13:32:59.574 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: [db/db_impl/db_impl_open.cc:1700] SstFileManager instance 0x558480c2f180 2024-03-04T13:32:59.575 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: DB pointer 0x558480d04000 2024-03-04T13:32:59.575 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: mon.smithi169 does not exist in monmap, will attempt to join an existing cluster 2024-03-04T13:32:59.575 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: [db/db_impl/db_impl.cc:901] ------- DUMPING STATS ------- 2024-03-04T13:32:59.575 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: rocksdb: [db/db_impl/db_impl.cc:903] 2024-03-04T13:32:59.575 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: ** DB Stats ** 2024-03-04T13:32:59.575 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-04T13:32:59.575 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-04T13:32:59.575 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-04T13:32:59.575 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-04T13:32:59.575 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-04T13:32:59.575 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-03-04T13:32:59.575 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-04T13:32:59.575 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: 2024-03-04T13:32:59.576 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: ** Compaction Stats [default] ** 2024-03-04T13:32:59.576 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-03-04T13:32:59.576 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-04T13:32:59.576 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: L0 1/0 1.51 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.0 0.00 0.00 1 0.001 0 0 2024-03-04T13:32:59.576 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: Sum 1/0 1.51 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.0 0.00 0.00 1 0.001 0 0 2024-03-04T13:32:59.576 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.0 0.00 0.00 1 0.001 0 0 2024-03-04T13:32:59.576 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: 2024-03-04T13:32:59.576 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: ** Compaction Stats [default] ** 2024-03-04T13:32:59.576 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-03-04T13:32:59.576 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-04T13:32:59.576 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2.0 0.00 0.00 1 0.001 0 0 2024-03-04T13:32:59.576 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-04T13:32:59.576 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-04T13:32:59.577 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-04T13:32:59.577 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-04T13:32:59.577 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-04T13:32:59.577 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-04T13:32:59.577 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: Cumulative compaction: 0.00 GB write, 0.24 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-04T13:32:59.577 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: Interval compaction: 0.00 GB write, 0.24 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-04T13:32:59.577 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-04T13:32:59.577 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: 2024-03-04T13:32:59.577 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-04T13:32:59.577 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: 2024-03-04T13:32:59.577 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: ** Compaction Stats [default] ** 2024-03-04T13:32:59.577 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-03-04T13:32:59.577 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-04T13:32:59.577 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: L0 1/0 1.51 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.0 0.00 0.00 1 0.001 0 0 2024-03-04T13:32:59.577 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: Sum 1/0 1.51 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.0 0.00 0.00 1 0.001 0 0 2024-03-04T13:32:59.578 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-03-04T13:32:59.578 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: 2024-03-04T13:32:59.578 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: ** Compaction Stats [default] ** 2024-03-04T13:32:59.578 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-03-04T13:32:59.578 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-04T13:32:59.578 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2.0 0.00 0.00 1 0.001 0 0 2024-03-04T13:32:59.578 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-04T13:32:59.578 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-04T13:32:59.578 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-04T13:32:59.578 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-04T13:32:59.578 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-04T13:32:59.578 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-04T13:32:59.578 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: Cumulative compaction: 0.00 GB write, 0.23 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-04T13:32:59.578 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-04T13:32:59.578 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-04T13:32:59.579 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: 2024-03-04T13:32:59.579 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-04T13:32:59.579 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: using public_addr v2:172.21.15.169:0/0 -> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] 2024-03-04T13:32:59.579 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: starting mon.smithi169 rank -1 at public addrs [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] at bind addrs [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi169 fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 2024-03-04T13:32:59.579 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: mon.smithi169@-1(???) e0 preinit fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 2024-03-04T13:32:59.579 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 bash[36263]: ba4ee916297fdd8fc62969689728b91425293dceddd7179ef90a69f022b94a0e 2024-03-04T13:32:59.580 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 systemd[1]: Started Ceph mon.smithi169 for 1fc09a58-da2b-11ee-95c6-87774f69a715. 2024-03-04T13:32:59.580 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: mon.smithi169@-1(synchronizing).mds e1 new map 2024-03-04T13:32:59.580 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: mon.smithi169@-1(synchronizing).mds e1 print_map 2024-03-04T13:32:59.580 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: e1 2024-03-04T13:32:59.580 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-04T13:32:59.580 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-04T13:32:59.580 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: legacy client fscid: -1 2024-03-04T13:32:59.580 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: 2024-03-04T13:32:59.580 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: No filesystems configured 2024-03-04T13:32:59.580 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: mon.smithi169@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-03-04T13:32:59.580 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: mon.smithi169@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-03-04T13:32:59.581 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: mon.smithi169@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-03-04T13:32:59.581 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: mon.smithi169@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-03-04T13:32:59.581 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: mon.smithi169@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-03-04T13:32:59.581 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: mon.smithi169@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-03-04T13:32:59.581 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: mon.smithi169@-1(synchronizing).osd e5 e5: 0 total, 0 up, 0 in 2024-03-04T13:32:59.581 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: mon.smithi169@-1(synchronizing).osd e5 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-04T13:32:59.581 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: mon.smithi169@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-03-04T13:32:59.581 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: mon.smithi169@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-03-04T13:32:59.581 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: mon.smithi169@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-03-04T13:32:59.581 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:32:59.581 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:32:59.581 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "who": "osd/host:smithi107", "name": "osd_memory_target"}]: dispatch 2024-03-04T13:32:59.581 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:32:59.581 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:32:59.582 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:32:59.582 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:32:59.582 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "who": "osd/host:smithi169", "name": "osd_memory_target"}]: dispatch 2024-03-04T13:32:59.582 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:32:59.582 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:32:59.582 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: from='client.? 172.21.15.169:0/170099343' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-04T13:32:59.582 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: Updating smithi107:/etc/ceph/ceph.conf 2024-03-04T13:32:59.582 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: Updating smithi169:/etc/ceph/ceph.conf 2024-03-04T13:32:59.582 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: Updating smithi169:/var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-04T13:32:59.582 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: Updating smithi107:/var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-04T13:32:59.582 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: Updating smithi169:/etc/ceph/ceph.client.admin.keyring 2024-03-04T13:32:59.582 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: Updating smithi107:/etc/ceph/ceph.client.admin.keyring 2024-03-04T13:32:59.582 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:32:59.583 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:32:59.583 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:32:59.583 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:32:59.583 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:32:59.583 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi169", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-04T13:32:59.583 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi169", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-03-04T13:32:59.583 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:32:59.583 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: Updating smithi169:/var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/config/ceph.client.admin.keyring 2024-03-04T13:32:59.583 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: Updating smithi107:/var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/config/ceph.client.admin.keyring 2024-03-04T13:32:59.583 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: Deploying daemon crash.smithi169 on smithi169 2024-03-04T13:32:59.583 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: from='client.? 172.21.15.169:0/1455324897' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-04T13:32:59.584 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:32:59.584 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:32:59.584 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:32:59.584 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:32:59.584 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:32:59.584 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: Deploying daemon node-exporter.smithi169 on smithi169 2024-03-04T13:32:59.584 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: from='client.? 172.21.15.169:0/3491310518' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-04T13:32:59.584 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:32:59.584 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:32:59.584 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:32:59.584 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:32:59.584 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:32:59.584 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:32:59.585 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:32:59.585 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi169.eihczr", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-04T13:32:59.585 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.smithi169.eihczr", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-04T13:32:59.585 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-04T13:32:59.585 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:32:59.585 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: Deploying daemon mgr.smithi169.eihczr on smithi169 2024-03-04T13:32:59.585 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: from='client.? 172.21.15.169:0/2472705882' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-04T13:32:59.585 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:32:59.585 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:32:59.585 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:32:59.585 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:32:59.585 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:32:59.585 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:32:59.585 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:32:59.586 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-04T13:32:59.586 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:32:59.586 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: Deploying daemon mon.smithi169 on smithi169 2024-03-04T13:32:59.586 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:32:59.586 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: from='client.? 172.21.15.169:0/3864568642' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-04T13:32:59.586 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:32:59.586 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:32:59.586 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:32:59.586 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:32:59 smithi169 ceph-mon[36313]: mon.smithi169@-1(synchronizing).paxosservice(auth 1..6) refresh upgraded, format 0 -> 3 2024-03-04T13:33:04.697 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:33:04.702+0000 7f1194a00700 1 -- 172.21.15.169:0/1669227130 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1174003580 msgr2=0x7f1174005a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:33:04.697 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:33:04.702+0000 7f1194a00700 1 --2- 172.21.15.169:0/1669227130 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1174003580 0x7f1174005a00 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1190070260 tx=0x7f118000b040 comp rx=0 tx=0).stop 2024-03-04T13:33:04.697 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:33:04.702+0000 7f1194a00700 1 -- 172.21.15.169:0/1669227130 shutdown_connections 2024-03-04T13:33:04.697 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:33:04.702+0000 7f1194a00700 1 --2- 172.21.15.169:0/1669227130 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1174003580 0x7f1174005a00 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:33:04.697 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:33:04.702+0000 7f1194a00700 1 -- 172.21.15.169:0/1669227130 >> 172.21.15.169:0/1669227130 conn(0x7f11900f8980 msgr2=0x7f11900fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:33:04.697 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:33:04.703+0000 7f1194a00700 1 -- 172.21.15.169:0/1669227130 shutdown_connections 2024-03-04T13:33:04.698 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:33:04.703+0000 7f1194a00700 1 -- 172.21.15.169:0/1669227130 wait complete. 2024-03-04T13:33:04.698 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:04 smithi169 ceph-mon[36313]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:33:04.698 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:04 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mon metadata", "id": "smithi107"}]: dispatch 2024-03-04T13:33:04.698 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:04 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mon metadata", "id": "smithi169"}]: dispatch 2024-03-04T13:33:04.698 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:04 smithi169 ceph-mon[36313]: mon.smithi107 calling monitor election 2024-03-04T13:33:04.698 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:04 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mon metadata", "id": "smithi169"}]: dispatch 2024-03-04T13:33:04.698 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:04 smithi169 ceph-mon[36313]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:33:04.698 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:04 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mon metadata", "id": "smithi169"}]: dispatch 2024-03-04T13:33:04.698 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:04 smithi169 ceph-mon[36313]: mon.smithi169 calling monitor election 2024-03-04T13:33:04.698 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:04 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mon metadata", "id": "smithi169"}]: dispatch 2024-03-04T13:33:04.699 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:04 smithi169 ceph-mon[36313]: from='mgr.? 172.21.15.169:0/1897822577' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi169.eihczr/crt"}]: dispatch 2024-03-04T13:33:04.699 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:04 smithi169 ceph-mon[36313]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:33:04.699 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:04 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mon metadata", "id": "smithi169"}]: dispatch 2024-03-04T13:33:04.699 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:04 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mon metadata", "id": "smithi169"}]: dispatch 2024-03-04T13:33:04.699 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:04 smithi169 ceph-mon[36313]: mon.smithi107 is new leader, mons smithi107,smithi169 in quorum (ranks 0,1) 2024-03-04T13:33:04.699 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:04 smithi169 ceph-mon[36313]: monmap e2: 2 mons at {smithi107=[v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0],smithi169=[v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-04T13:33:04.699 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:04 smithi169 ceph-mon[36313]: fsmap 2024-03-04T13:33:04.699 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:04 smithi169 ceph-mon[36313]: osdmap e5: 0 total, 0 up, 0 in 2024-03-04T13:33:04.699 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:04 smithi169 ceph-mon[36313]: mgrmap e17: smithi107.wpzgwl(active, since 37s) 2024-03-04T13:33:04.699 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:04 smithi169 ceph-mon[36313]: Standby manager daemon smithi169.eihczr started 2024-03-04T13:33:04.699 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:04 smithi169 ceph-mon[36313]: from='mgr.? 172.21.15.169:0/1897822577' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-04T13:33:04.699 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:04 smithi169 ceph-mon[36313]: overall HEALTH_OK 2024-03-04T13:33:04.699 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:04 smithi169 ceph-mon[36313]: from='mgr.? 172.21.15.169:0/1897822577' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi169.eihczr/key"}]: dispatch 2024-03-04T13:33:04.699 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:04 smithi169 ceph-mon[36313]: from='mgr.? 172.21.15.169:0/1897822577' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-04T13:33:04.700 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:04 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:33:04.700 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:04 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:33:04.700 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:33:04.704+0000 7f1194a00700 1 Processor -- start 2024-03-04T13:33:04.700 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:33:04.705+0000 7f1194a00700 1 -- start start 2024-03-04T13:33:04.700 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:33:04.706+0000 7f1194a00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1174003580 0x7f1190119790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:33:04.701 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:33:04.706+0000 7f1194a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1190119cd0 0x7f1190113650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:33:04.701 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:33:04.707+0000 7f1194a00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f119011a290 con 0x7f1190119cd0 2024-03-04T13:33:04.701 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:33:04.707+0000 7f1194a00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1190113b90 con 0x7f1174003580 2024-03-04T13:33:04.702 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:33:04.707+0000 7f118de00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1174003580 0x7f1190119790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:33:04.702 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:33:04.707+0000 7f118de00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1174003580 0x7f1190119790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:50514/0 (socket says 172.21.15.169:50514) 2024-03-04T13:33:04.702 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:33:04.707+0000 7f118de00700 1 -- 172.21.15.169:0/1818293048 learned_addr learned my addr 172.21.15.169:0/1818293048 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-03-04T13:33:04.702 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:33:04.707+0000 7f118d400700 1 --2- 172.21.15.169:0/1818293048 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1190119cd0 0x7f1190113650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:33:04.702 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:33:04.708+0000 7f118de00700 1 -- 172.21.15.169:0/1818293048 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1174003580 msgr2=0x7f1190119790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-03-04T13:33:04.702 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:33:04.708+0000 7f118de00700 1 -- 172.21.15.169:0/1818293048 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1174003580 msgr2=0x7f1190119790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-04T13:33:04.702 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:33:04.708+0000 7f118de00700 1 --2- 172.21.15.169:0/1818293048 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1174003580 0x7f1190119790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-03-04T13:33:04.702 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:33:04.708+0000 7f118de00700 1 --2- 172.21.15.169:0/1818293048 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1174003580 0x7f1190119790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T13:33:04.703 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:33:04.708+0000 7f118d400700 1 -- 172.21.15.169:0/1818293048 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1174003580 msgr2=0x7f1190119790 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-04T13:33:04.703 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:33:04.708+0000 7f118d400700 1 --2- 172.21.15.169:0/1818293048 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1174003580 0x7f1190119790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:33:04.703 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:33:04.708+0000 7f118d400700 1 -- 172.21.15.169:0/1818293048 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1180005040 con 0x7f1190119cd0 2024-03-04T13:33:04.703 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:33:04.708+0000 7f118d400700 1 --2- 172.21.15.169:0/1818293048 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1190119cd0 0x7f1190113650 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f117c00b790 tx=0x7f117c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:33:04.703 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:33:04.709+0000 7f1186000700 1 -- 172.21.15.169:0/1818293048 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f117c011810 con 0x7f1190119cd0 2024-03-04T13:33:04.704 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:33:04.709+0000 7f1186000700 1 -- 172.21.15.169:0/1818293048 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f117c011e40 con 0x7f1190119cd0 2024-03-04T13:33:04.704 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:33:04.709+0000 7f1186000700 1 -- 172.21.15.169:0/1818293048 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f117c00f2e0 con 0x7f1190119cd0 2024-03-04T13:33:04.704 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:33:04.709+0000 7f1194a00700 1 -- 172.21.15.169:0/1818293048 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1190113e70 con 0x7f1190119cd0 2024-03-04T13:33:04.704 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:33:04.709+0000 7f1194a00700 1 -- 172.21.15.169:0/1818293048 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11901143b0 con 0x7f1190119cd0 2024-03-04T13:33:04.705 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:33:04.711+0000 7f1194a00700 1 -- 172.21.15.169:0/1818293048 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f119005f5c0 con 0x7f1190119cd0 2024-03-04T13:33:04.706 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:33:04.712+0000 7f1186000700 1 -- 172.21.15.169:0/1818293048 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 86567+0+0 (secure 0 0 0) 0x7f117c010380 con 0x7f1190119cd0 2024-03-04T13:33:04.706 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:33:04.713+0000 7f1186000700 1 --2- 172.21.15.169:0/1818293048 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f1170069440 0x7f117006b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:33:04.707 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:33:04.713+0000 7f1186000700 1 -- 172.21.15.169:0/1818293048 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f117c086370 con 0x7f1190119cd0 2024-03-04T13:33:04.707 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:33:04.714+0000 7f118de00700 1 --2- 172.21.15.169:0/1818293048 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f1170069440 0x7f117006b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:33:04.708 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:33:04.715+0000 7f118de00700 1 --2- 172.21.15.169:0/1818293048 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f1170069440 0x7f117006b8f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f1180004fd0 tx=0x7f118000b040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:33:04.721 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:33:04.727+0000 7f1186000700 1 -- 172.21.15.169:0/1818293048 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f117c0570f0 con 0x7f1190119cd0 2024-03-04T13:33:04.838 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:04 smithi107 ceph-mon[28739]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:33:04.838 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:04 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mon metadata", "id": "smithi107"}]: dispatch 2024-03-04T13:33:04.838 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:04 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mon metadata", "id": "smithi169"}]: dispatch 2024-03-04T13:33:04.838 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:04 smithi107 ceph-mon[28739]: mon.smithi107 calling monitor election 2024-03-04T13:33:04.838 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:04 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mon metadata", "id": "smithi169"}]: dispatch 2024-03-04T13:33:04.838 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:04 smithi107 ceph-mon[28739]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:33:04.838 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:04 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mon metadata", "id": "smithi169"}]: dispatch 2024-03-04T13:33:04.839 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:04 smithi107 ceph-mon[28739]: mon.smithi169 calling monitor election 2024-03-04T13:33:04.839 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:04 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mon metadata", "id": "smithi169"}]: dispatch 2024-03-04T13:33:04.839 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:04 smithi107 ceph-mon[28739]: from='mgr.? 172.21.15.169:0/1897822577' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi169.eihczr/crt"}]: dispatch 2024-03-04T13:33:04.839 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:04 smithi107 ceph-mon[28739]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:33:04.839 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:04 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mon metadata", "id": "smithi169"}]: dispatch 2024-03-04T13:33:04.839 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:04 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mon metadata", "id": "smithi169"}]: dispatch 2024-03-04T13:33:04.839 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:04 smithi107 ceph-mon[28739]: mon.smithi107 is new leader, mons smithi107,smithi169 in quorum (ranks 0,1) 2024-03-04T13:33:04.839 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:04 smithi107 ceph-mon[28739]: monmap e2: 2 mons at {smithi107=[v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0],smithi169=[v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-04T13:33:04.839 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:04 smithi107 ceph-mon[28739]: fsmap 2024-03-04T13:33:04.839 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:04 smithi107 ceph-mon[28739]: osdmap e5: 0 total, 0 up, 0 in 2024-03-04T13:33:04.839 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:04 smithi107 ceph-mon[28739]: mgrmap e17: smithi107.wpzgwl(active, since 37s) 2024-03-04T13:33:04.839 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:04 smithi107 ceph-mon[28739]: Standby manager daemon smithi169.eihczr started 2024-03-04T13:33:04.839 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:04 smithi107 ceph-mon[28739]: from='mgr.? 172.21.15.169:0/1897822577' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-04T13:33:04.840 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:04 smithi107 ceph-mon[28739]: overall HEALTH_OK 2024-03-04T13:33:04.840 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:04 smithi107 ceph-mon[28739]: from='mgr.? 172.21.15.169:0/1897822577' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi169.eihczr/key"}]: dispatch 2024-03-04T13:33:04.840 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:04 smithi107 ceph-mon[28739]: from='mgr.? 172.21.15.169:0/1897822577' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-04T13:33:04.840 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:04 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:33:04.840 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:04 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:33:05.032 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:33:05.039+0000 7f1194a00700 1 -- 172.21.15.169:0/1818293048 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f1190114d70 con 0x7f1190119cd0 2024-03-04T13:33:05.034 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:33:05.040+0000 7f1186000700 1 -- 172.21.15.169:0/1818293048 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1021 (secure 0 0 0) 0x7f117c015410 con 0x7f1190119cd0 2024-03-04T13:33:05.034 INFO:teuthology.orchestra.run.smithi169.stdout: 2024-03-04T13:33:05.034 INFO:teuthology.orchestra.run.smithi169.stdout:{"epoch":2,"fsid":"1fc09a58-da2b-11ee-95c6-87774f69a715","modified":"2024-03-04T13:32:59.306877Z","created":"2024-03-04T13:29:38.149007Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi107","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:3300","nonce":0},{"type":"v1","addr":"172.21.15.107:6789","nonce":0}]},"addr":"172.21.15.107:6789/0","public_addr":"172.21.15.107:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"smithi169","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:3300","nonce":0},{"type":"v1","addr":"172.21.15.169:6789","nonce":0}]},"addr":"172.21.15.169:6789/0","public_addr":"172.21.15.169:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-04T13:33:05.037 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:33:05.043+0000 7f1194a00700 1 -- 172.21.15.169:0/1818293048 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f1170069440 msgr2=0x7f117006b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:33:05.037 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:33:05.043+0000 7f1194a00700 1 --2- 172.21.15.169:0/1818293048 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f1170069440 0x7f117006b8f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f1180004fd0 tx=0x7f118000b040 comp rx=0 tx=0).stop 2024-03-04T13:33:05.037 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:33:05.043+0000 7f1194a00700 1 -- 172.21.15.169:0/1818293048 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1190119cd0 msgr2=0x7f1190113650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:33:05.037 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:33:05.043+0000 7f1194a00700 1 --2- 172.21.15.169:0/1818293048 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1190119cd0 0x7f1190113650 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f117c00b790 tx=0x7f117c0091a0 comp rx=0 tx=0).stop 2024-03-04T13:33:05.037 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:33:05.043+0000 7f1194a00700 1 -- 172.21.15.169:0/1818293048 shutdown_connections 2024-03-04T13:33:05.037 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:33:05.043+0000 7f1194a00700 1 --2- 172.21.15.169:0/1818293048 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f1170069440 0x7f117006b8f0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:33:05.037 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:33:05.043+0000 7f1194a00700 1 --2- 172.21.15.169:0/1818293048 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1174003580 0x7f1190119790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:33:05.037 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:33:05.043+0000 7f1194a00700 1 --2- 172.21.15.169:0/1818293048 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1190119cd0 0x7f1190113650 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:33:05.037 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:33:05.043+0000 7f1194a00700 1 -- 172.21.15.169:0/1818293048 >> 172.21.15.169:0/1818293048 conn(0x7f11900f8980 msgr2=0x7f11900fcb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:33:05.037 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:33:05.043+0000 7f1194a00700 1 -- 172.21.15.169:0/1818293048 shutdown_connections 2024-03-04T13:33:05.037 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:33:05.043+0000 7f1194a00700 1 -- 172.21.15.169:0/1818293048 wait complete. 2024-03-04T13:33:05.038 INFO:teuthology.orchestra.run.smithi169.stderr:dumped monmap epoch 2 2024-03-04T13:33:05.604 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:05 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:33:05.605 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:05 smithi169 ceph-mon[36313]: mgrmap e18: smithi107.wpzgwl(active, since 37s), standbys: smithi169.eihczr 2024-03-04T13:33:05.605 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:05 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mgr metadata", "who": "smithi169.eihczr", "id": "smithi169.eihczr"}]: dispatch 2024-03-04T13:33:05.605 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:05 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:33:05.605 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:05 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:33:05.605 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:05 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:33:05.605 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:05 smithi169 ceph-mon[36313]: from='client.? 172.21.15.169:0/1818293048' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-04T13:33:05.605 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:05 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mon metadata", "id": "smithi169"}]: dispatch 2024-03-04T13:33:05.668 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-03-04T13:33:05.669 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 -- ceph config generate-minimal-conf 2024-03-04T13:33:05.759 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:05 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:33:05.759 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:05 smithi107 ceph-mon[28739]: mgrmap e18: smithi107.wpzgwl(active, since 37s), standbys: smithi169.eihczr 2024-03-04T13:33:05.759 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:05 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mgr metadata", "who": "smithi169.eihczr", "id": "smithi169.eihczr"}]: dispatch 2024-03-04T13:33:05.759 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:05 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:33:05.759 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:05 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:33:05.760 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:05 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:33:05.760 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:05 smithi107 ceph-mon[28739]: from='client.? 172.21.15.169:0/1818293048' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-04T13:33:05.760 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:05 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mon metadata", "id": "smithi169"}]: dispatch 2024-03-04T13:33:06.150 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/mon.smithi107/config 2024-03-04T13:33:06.587 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:06 smithi107 ceph-mon[28739]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:33:06.854 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:06 smithi169 ceph-mon[36313]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:33:08.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:33:08.100+0000 7f3f15a00700 1 -- 172.21.15.107:0/3587748444 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f10104dd0 msgr2=0x7f3f101051a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:33:08.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:33:08.100+0000 7f3f15a00700 1 --2- 172.21.15.107:0/3587748444 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f10104dd0 0x7f3f101051a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3f000048c0 tx=0x7f3f00009e30 comp rx=0 tx=0).stop 2024-03-04T13:33:08.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:33:08.100+0000 7f3f15a00700 1 -- 172.21.15.107:0/3587748444 shutdown_connections 2024-03-04T13:33:08.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:33:08.100+0000 7f3f15a00700 1 --2- 172.21.15.107:0/3587748444 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f10104dd0 0x7f3f101051a0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:33:08.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:33:08.100+0000 7f3f15a00700 1 -- 172.21.15.107:0/3587748444 >> 172.21.15.107:0/3587748444 conn(0x7f3f100faae0 msgr2=0x7f3f100fcec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:33:08.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:33:08.100+0000 7f3f15a00700 1 -- 172.21.15.107:0/3587748444 shutdown_connections 2024-03-04T13:33:08.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:33:08.100+0000 7f3f15a00700 1 -- 172.21.15.107:0/3587748444 wait complete. 2024-03-04T13:33:08.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:33:08.101+0000 7f3f15a00700 1 Processor -- start 2024-03-04T13:33:08.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:33:08.101+0000 7f3f15a00700 1 -- start start 2024-03-04T13:33:08.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:33:08.101+0000 7f3f15a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f10104dd0 0x7f3f10115310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:33:08.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:33:08.101+0000 7f3f15a00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3f10115850 0x7f3f1010f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:33:08.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:33:08.101+0000 7f3f15a00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f10115e60 con 0x7f3f10104dd0 2024-03-04T13:33:08.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:33:08.101+0000 7f3f15a00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f10115fd0 con 0x7f3f10115850 2024-03-04T13:33:08.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:33:08.102+0000 7f3f0ea00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f10104dd0 0x7f3f10115310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:33:08.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:33:08.102+0000 7f3f0ea00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f10104dd0 0x7f3f10115310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50984/0 (socket says 172.21.15.107:50984) 2024-03-04T13:33:08.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:33:08.102+0000 7f3f0ea00700 1 -- 172.21.15.107:0/869461625 learned_addr learned my addr 172.21.15.107:0/869461625 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:33:08.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:33:08.102+0000 7f3f0e000700 1 --2- 172.21.15.107:0/869461625 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3f10115850 0x7f3f1010f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:33:08.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:33:08.102+0000 7f3f0e000700 1 -- 172.21.15.107:0/869461625 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3f10115850 msgr2=0x7f3f1010f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-03-04T13:33:08.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:33:08.102+0000 7f3f0e000700 1 -- 172.21.15.107:0/869461625 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3f10115850 msgr2=0x7f3f1010f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-04T13:33:08.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:33:08.102+0000 7f3f0e000700 1 --2- 172.21.15.107:0/869461625 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3f10115850 0x7f3f1010f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-03-04T13:33:08.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:33:08.102+0000 7f3f0e000700 1 --2- 172.21.15.107:0/869461625 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3f10115850 0x7f3f1010f1f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T13:33:08.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:33:08.103+0000 7f3f0ea00700 1 -- 172.21.15.107:0/869461625 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3f10115850 msgr2=0x7f3f1010f1f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-04T13:33:08.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:33:08.103+0000 7f3f0ea00700 1 --2- 172.21.15.107:0/869461625 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3f10115850 0x7f3f1010f1f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:33:08.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:33:08.103+0000 7f3f0ea00700 1 -- 172.21.15.107:0/869461625 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f00005040 con 0x7f3f10104dd0 2024-03-04T13:33:08.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:33:08.103+0000 7f3f0ea00700 1 --2- 172.21.15.107:0/869461625 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f10104dd0 0x7f3f10115310 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3f0000b870 tx=0x7f3f00009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:33:08.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:33:08.103+0000 7f3f06a00700 1 -- 172.21.15.107:0/869461625 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f0000e040 con 0x7f3f10104dd0 2024-03-04T13:33:08.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:33:08.103+0000 7f3f06a00700 1 -- 172.21.15.107:0/869461625 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3f00013420 con 0x7f3f10104dd0 2024-03-04T13:33:08.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:33:08.104+0000 7f3f06a00700 1 -- 172.21.15.107:0/869461625 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f0001c630 con 0x7f3f10104dd0 2024-03-04T13:33:08.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:33:08.104+0000 7f3f15a00700 1 -- 172.21.15.107:0/869461625 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f1010f7f0 con 0x7f3f10104dd0 2024-03-04T13:33:08.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:33:08.104+0000 7f3f15a00700 1 -- 172.21.15.107:0/869461625 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f1010fd30 con 0x7f3f10104dd0 2024-03-04T13:33:08.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:33:08.105+0000 7f3f15a00700 1 -- 172.21.15.107:0/869461625 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f10110060 con 0x7f3f10104dd0 2024-03-04T13:33:08.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:33:08.106+0000 7f3f06a00700 1 -- 172.21.15.107:0/869461625 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 86567+0+0 (secure 0 0 0) 0x7f3f00013a20 con 0x7f3f10104dd0 2024-03-04T13:33:08.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:33:08.107+0000 7f3f06a00700 1 --2- 172.21.15.107:0/869461625 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f3ef4069380 0x7f3ef406b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:33:08.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:33:08.107+0000 7f3f06a00700 1 -- 172.21.15.107:0/869461625 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f3f00087350 con 0x7f3f10104dd0 2024-03-04T13:33:08.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:33:08.107+0000 7f3f0e000700 1 --2- 172.21.15.107:0/869461625 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f3ef4069380 0x7f3ef406b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:33:08.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:33:08.108+0000 7f3f0e000700 1 --2- 172.21.15.107:0/869461625 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f3ef4069380 0x7f3ef406b830 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f3efc006f30 tx=0x7f3efc008040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:33:08.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:33:08.112+0000 7f3f06a00700 1 -- 172.21.15.107:0/869461625 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f3f10110060 con 0x7f3f10104dd0 2024-03-04T13:33:08.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:33:08.363+0000 7f3f15a00700 1 -- 172.21.15.107:0/869461625 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f3f1005f5c0 con 0x7f3f10104dd0 2024-03-04T13:33:08.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:33:08.364+0000 7f3f06a00700 1 -- 172.21.15.107:0/869461625 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v10) v1 ==== 76+0+227 (secure 0 0 0) 0x7f3f00054800 con 0x7f3f10104dd0 2024-03-04T13:33:08.360 INFO:teuthology.orchestra.run.smithi107.stdout:# minimal ceph.conf for 1fc09a58-da2b-11ee-95c6-87774f69a715 2024-03-04T13:33:08.360 INFO:teuthology.orchestra.run.smithi107.stdout:[global] 2024-03-04T13:33:08.360 INFO:teuthology.orchestra.run.smithi107.stdout: fsid = 1fc09a58-da2b-11ee-95c6-87774f69a715 2024-03-04T13:33:08.360 INFO:teuthology.orchestra.run.smithi107.stdout: mon_host = [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] 2024-03-04T13:33:08.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:33:08.367+0000 7f3f15a00700 1 -- 172.21.15.107:0/869461625 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f3ef4069380 msgr2=0x7f3ef406b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:33:08.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:33:08.367+0000 7f3f15a00700 1 --2- 172.21.15.107:0/869461625 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f3ef4069380 0x7f3ef406b830 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f3efc006f30 tx=0x7f3efc008040 comp rx=0 tx=0).stop 2024-03-04T13:33:08.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:33:08.367+0000 7f3f15a00700 1 -- 172.21.15.107:0/869461625 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f10104dd0 msgr2=0x7f3f10115310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:33:08.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:33:08.367+0000 7f3f15a00700 1 --2- 172.21.15.107:0/869461625 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f10104dd0 0x7f3f10115310 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3f0000b870 tx=0x7f3f00009e30 comp rx=0 tx=0).stop 2024-03-04T13:33:08.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:33:08.367+0000 7f3f15a00700 1 -- 172.21.15.107:0/869461625 shutdown_connections 2024-03-04T13:33:08.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:33:08.367+0000 7f3f15a00700 1 --2- 172.21.15.107:0/869461625 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f3ef4069380 0x7f3ef406b830 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:33:08.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:33:08.367+0000 7f3f15a00700 1 --2- 172.21.15.107:0/869461625 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f10104dd0 0x7f3f10115310 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:33:08.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:33:08.367+0000 7f3f15a00700 1 --2- 172.21.15.107:0/869461625 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3f10115850 0x7f3f1010f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:33:08.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:33:08.367+0000 7f3f15a00700 1 -- 172.21.15.107:0/869461625 >> 172.21.15.107:0/869461625 conn(0x7f3f100faae0 msgr2=0x7f3f100fd0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:33:08.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:33:08.368+0000 7f3f15a00700 1 -- 172.21.15.107:0/869461625 shutdown_connections 2024-03-04T13:33:08.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:33:08.368+0000 7f3f15a00700 1 -- 172.21.15.107:0/869461625 wait complete. 2024-03-04T13:33:08.586 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:08 smithi107 ceph-mon[28739]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:33:08.587 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:08 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:33:08.587 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:08 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:33:08.587 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:08 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:33:08.587 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:08 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:33:08.587 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:08 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:33:08.587 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:08 smithi107 ceph-mon[28739]: Updating smithi107:/etc/ceph/ceph.conf 2024-03-04T13:33:08.587 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:08 smithi107 ceph-mon[28739]: Updating smithi169:/etc/ceph/ceph.conf 2024-03-04T13:33:08.604 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:08 smithi169 ceph-mon[36313]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:33:08.604 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:08 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:33:08.604 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:08 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:33:08.604 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:08 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:33:08.604 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:08 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:33:08.604 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:08 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:33:08.604 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:08 smithi169 ceph-mon[36313]: Updating smithi107:/etc/ceph/ceph.conf 2024-03-04T13:33:08.604 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:08 smithi169 ceph-mon[36313]: Updating smithi169:/etc/ceph/ceph.conf 2024-03-04T13:33:09.115 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-03-04T13:33:09.115 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-03-04T13:33:09.115 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/ceph/ceph.conf 2024-03-04T13:33:09.201 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-03-04T13:33:09.201 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-04T13:33:09.309 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-03-04T13:33:09.309 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/etc/ceph/ceph.conf 2024-03-04T13:33:09.394 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-03-04T13:33:09.394 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-04T13:33:09.528 INFO:tasks.cephadm:Deploying OSDs... 2024-03-04T13:33:09.529 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-03-04T13:33:09.529 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/scratch_devs of=/dev/stdout 2024-03-04T13:33:09.569 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-04T13:33:09.570 DEBUG:teuthology.orchestra.run.smithi107:> stat /dev/vg_nvme/lv_1 2024-03-04T13:33:09.587 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:09 smithi107 ceph-mon[28739]: Updating smithi107:/var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-04T13:33:09.587 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:09 smithi107 ceph-mon[28739]: from='client.? 172.21.15.107:0/869461625' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:33:09.587 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:09 smithi107 ceph-mon[28739]: Updating smithi169:/var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-04T13:33:09.587 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:09 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:33:09.587 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:09 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:33:09.588 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:09 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:33:09.588 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:09 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:33:09.588 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:09 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:33:09.604 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:09 smithi169 ceph-mon[36313]: Updating smithi107:/var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-04T13:33:09.604 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:09 smithi169 ceph-mon[36313]: from='client.? 172.21.15.107:0/869461625' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:33:09.605 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:09 smithi169 ceph-mon[36313]: Updating smithi169:/var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-04T13:33:09.605 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:09 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:33:09.605 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:09 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:33:09.605 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:09 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:33:09.605 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:09 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:33:09.605 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:09 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:33:09.623 INFO:teuthology.orchestra.run.smithi107.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-04T13:33:09.623 INFO:teuthology.orchestra.run.smithi107.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-04T13:33:09.623 INFO:teuthology.orchestra.run.smithi107.stdout:Device: 5h/5d Inode: 631 Links: 1 2024-03-04T13:33:09.623 INFO:teuthology.orchestra.run.smithi107.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-04T13:33:09.623 INFO:teuthology.orchestra.run.smithi107.stdout:Context: system_u:object_r:device_t:s0 2024-03-04T13:33:09.623 INFO:teuthology.orchestra.run.smithi107.stdout:Access: 2024-03-04 13:32:16.803519008 +0000 2024-03-04T13:33:09.624 INFO:teuthology.orchestra.run.smithi107.stdout:Modify: 2024-03-04 13:22:39.957783714 +0000 2024-03-04T13:33:09.624 INFO:teuthology.orchestra.run.smithi107.stdout:Change: 2024-03-04 13:22:39.957783714 +0000 2024-03-04T13:33:09.624 INFO:teuthology.orchestra.run.smithi107.stdout: Birth: 2024-03-04 13:22:39.957783714 +0000 2024-03-04T13:33:09.624 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-04T13:33:09.749 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records in 2024-03-04T13:33:09.750 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records out 2024-03-04T13:33:09.750 INFO:teuthology.orchestra.run.smithi107.stderr:512 bytes copied, 0.000343406 s, 1.5 MB/s 2024-03-04T13:33:09.754 DEBUG:teuthology.orchestra.run.smithi107:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-04T13:33:09.805 DEBUG:teuthology.orchestra.run.smithi107:> stat /dev/vg_nvme/lv_2 2024-03-04T13:33:09.844 INFO:teuthology.orchestra.run.smithi107.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-04T13:33:09.844 INFO:teuthology.orchestra.run.smithi107.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-04T13:33:09.844 INFO:teuthology.orchestra.run.smithi107.stdout:Device: 5h/5d Inode: 653 Links: 1 2024-03-04T13:33:09.845 INFO:teuthology.orchestra.run.smithi107.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-04T13:33:09.845 INFO:teuthology.orchestra.run.smithi107.stdout:Context: system_u:object_r:device_t:s0 2024-03-04T13:33:09.845 INFO:teuthology.orchestra.run.smithi107.stdout:Access: 2024-03-04 13:32:16.804518991 +0000 2024-03-04T13:33:09.845 INFO:teuthology.orchestra.run.smithi107.stdout:Modify: 2024-03-04 13:22:40.535773729 +0000 2024-03-04T13:33:09.845 INFO:teuthology.orchestra.run.smithi107.stdout:Change: 2024-03-04 13:22:40.535773729 +0000 2024-03-04T13:33:09.845 INFO:teuthology.orchestra.run.smithi107.stdout: Birth: 2024-03-04 13:22:40.535773729 +0000 2024-03-04T13:33:09.845 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-04T13:33:09.953 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records in 2024-03-04T13:33:09.953 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records out 2024-03-04T13:33:09.953 INFO:teuthology.orchestra.run.smithi107.stderr:512 bytes copied, 0.000283894 s, 1.8 MB/s 2024-03-04T13:33:09.955 DEBUG:teuthology.orchestra.run.smithi107:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-04T13:33:09.997 DEBUG:teuthology.orchestra.run.smithi107:> stat /dev/vg_nvme/lv_3 2024-03-04T13:33:10.094 INFO:teuthology.orchestra.run.smithi107.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-04T13:33:10.094 INFO:teuthology.orchestra.run.smithi107.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-04T13:33:10.094 INFO:teuthology.orchestra.run.smithi107.stdout:Device: 5h/5d Inode: 671 Links: 1 2024-03-04T13:33:10.094 INFO:teuthology.orchestra.run.smithi107.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-04T13:33:10.094 INFO:teuthology.orchestra.run.smithi107.stdout:Context: system_u:object_r:device_t:s0 2024-03-04T13:33:10.094 INFO:teuthology.orchestra.run.smithi107.stdout:Access: 2024-03-04 13:32:16.805518974 +0000 2024-03-04T13:33:10.094 INFO:teuthology.orchestra.run.smithi107.stdout:Modify: 2024-03-04 13:22:41.082764281 +0000 2024-03-04T13:33:10.095 INFO:teuthology.orchestra.run.smithi107.stdout:Change: 2024-03-04 13:22:41.082764281 +0000 2024-03-04T13:33:10.095 INFO:teuthology.orchestra.run.smithi107.stdout: Birth: 2024-03-04 13:22:41.082764281 +0000 2024-03-04T13:33:10.095 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-04T13:33:10.173 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records in 2024-03-04T13:33:10.173 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records out 2024-03-04T13:33:10.173 INFO:teuthology.orchestra.run.smithi107.stderr:512 bytes copied, 0.00030568 s, 1.7 MB/s 2024-03-04T13:33:10.177 DEBUG:teuthology.orchestra.run.smithi107:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-04T13:33:10.220 DEBUG:teuthology.orchestra.run.smithi107:> stat /dev/vg_nvme/lv_4 2024-03-04T13:33:10.312 INFO:teuthology.orchestra.run.smithi107.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-04T13:33:10.312 INFO:teuthology.orchestra.run.smithi107.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-04T13:33:10.313 INFO:teuthology.orchestra.run.smithi107.stdout:Device: 5h/5d Inode: 689 Links: 1 2024-03-04T13:33:10.313 INFO:teuthology.orchestra.run.smithi107.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-04T13:33:10.313 INFO:teuthology.orchestra.run.smithi107.stdout:Context: system_u:object_r:device_t:s0 2024-03-04T13:33:10.313 INFO:teuthology.orchestra.run.smithi107.stdout:Access: 2024-03-04 13:32:16.805518974 +0000 2024-03-04T13:33:10.313 INFO:teuthology.orchestra.run.smithi107.stdout:Modify: 2024-03-04 13:22:41.658754332 +0000 2024-03-04T13:33:10.313 INFO:teuthology.orchestra.run.smithi107.stdout:Change: 2024-03-04 13:22:41.658754332 +0000 2024-03-04T13:33:10.313 INFO:teuthology.orchestra.run.smithi107.stdout: Birth: 2024-03-04 13:22:41.658754332 +0000 2024-03-04T13:33:10.313 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-04T13:33:10.391 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records in 2024-03-04T13:33:10.392 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records out 2024-03-04T13:33:10.392 INFO:teuthology.orchestra.run.smithi107.stderr:512 bytes copied, 0.000325817 s, 1.6 MB/s 2024-03-04T13:33:10.395 DEBUG:teuthology.orchestra.run.smithi107:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-04T13:33:10.436 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-03-04T13:33:10.437 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/scratch_devs of=/dev/stdout 2024-03-04T13:33:10.450 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:10 smithi107 ceph-mon[28739]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:33:10.450 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:10 smithi107 ceph-mon[28739]: Reconfiguring mon.smithi107 (unknown last config time)... 2024-03-04T13:33:10.450 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:10 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-04T13:33:10.450 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:10 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-04T13:33:10.451 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:10 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:33:10.451 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:10 smithi107 ceph-mon[28739]: Reconfiguring daemon mon.smithi107 on smithi107 2024-03-04T13:33:10.488 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-04T13:33:10.489 DEBUG:teuthology.orchestra.run.smithi169:> stat /dev/vg_nvme/lv_1 2024-03-04T13:33:10.579 INFO:teuthology.orchestra.run.smithi169.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-04T13:33:10.579 INFO:teuthology.orchestra.run.smithi169.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-04T13:33:10.579 INFO:teuthology.orchestra.run.smithi169.stdout:Device: 5h/5d Inode: 633 Links: 1 2024-03-04T13:33:10.579 INFO:teuthology.orchestra.run.smithi169.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-04T13:33:10.579 INFO:teuthology.orchestra.run.smithi169.stdout:Context: system_u:object_r:device_t:s0 2024-03-04T13:33:10.579 INFO:teuthology.orchestra.run.smithi169.stdout:Access: 2024-03-04 13:32:58.510715215 +0000 2024-03-04T13:33:10.579 INFO:teuthology.orchestra.run.smithi169.stdout:Modify: 2024-03-04 13:22:41.014039415 +0000 2024-03-04T13:33:10.579 INFO:teuthology.orchestra.run.smithi169.stdout:Change: 2024-03-04 13:22:41.014039415 +0000 2024-03-04T13:33:10.579 INFO:teuthology.orchestra.run.smithi169.stdout: Birth: 2024-03-04 13:22:41.014039415 +0000 2024-03-04T13:33:10.580 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-04T13:33:10.604 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:10 smithi169 ceph-mon[36313]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:33:10.604 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:10 smithi169 ceph-mon[36313]: Reconfiguring mon.smithi107 (unknown last config time)... 2024-03-04T13:33:10.604 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:10 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-04T13:33:10.604 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:10 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-04T13:33:10.604 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:10 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:33:10.604 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:10 smithi169 ceph-mon[36313]: Reconfiguring daemon mon.smithi107 on smithi107 2024-03-04T13:33:10.663 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records in 2024-03-04T13:33:10.663 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records out 2024-03-04T13:33:10.663 INFO:teuthology.orchestra.run.smithi169.stderr:512 bytes copied, 0.000391339 s, 1.3 MB/s 2024-03-04T13:33:10.667 DEBUG:teuthology.orchestra.run.smithi169:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-04T13:33:10.760 DEBUG:teuthology.orchestra.run.smithi169:> stat /dev/vg_nvme/lv_2 2024-03-04T13:33:10.802 INFO:teuthology.orchestra.run.smithi169.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-04T13:33:10.802 INFO:teuthology.orchestra.run.smithi169.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-04T13:33:10.802 INFO:teuthology.orchestra.run.smithi169.stdout:Device: 5h/5d Inode: 651 Links: 1 2024-03-04T13:33:10.802 INFO:teuthology.orchestra.run.smithi169.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-04T13:33:10.802 INFO:teuthology.orchestra.run.smithi169.stdout:Context: system_u:object_r:device_t:s0 2024-03-04T13:33:10.802 INFO:teuthology.orchestra.run.smithi169.stdout:Access: 2024-03-04 13:32:58.511715193 +0000 2024-03-04T13:33:10.802 INFO:teuthology.orchestra.run.smithi169.stdout:Modify: 2024-03-04 13:22:41.559027061 +0000 2024-03-04T13:33:10.802 INFO:teuthology.orchestra.run.smithi169.stdout:Change: 2024-03-04 13:22:41.559027061 +0000 2024-03-04T13:33:10.802 INFO:teuthology.orchestra.run.smithi169.stdout: Birth: 2024-03-04 13:22:41.559027061 +0000 2024-03-04T13:33:10.803 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-04T13:33:10.924 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records in 2024-03-04T13:33:10.924 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records out 2024-03-04T13:33:10.924 INFO:teuthology.orchestra.run.smithi169.stderr:512 bytes copied, 0.000339883 s, 1.5 MB/s 2024-03-04T13:33:10.928 DEBUG:teuthology.orchestra.run.smithi169:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-04T13:33:10.977 DEBUG:teuthology.orchestra.run.smithi169:> stat /dev/vg_nvme/lv_3 2024-03-04T13:33:11.020 INFO:teuthology.orchestra.run.smithi169.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-04T13:33:11.020 INFO:teuthology.orchestra.run.smithi169.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-04T13:33:11.020 INFO:teuthology.orchestra.run.smithi169.stdout:Device: 5h/5d Inode: 672 Links: 1 2024-03-04T13:33:11.020 INFO:teuthology.orchestra.run.smithi169.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-04T13:33:11.020 INFO:teuthology.orchestra.run.smithi169.stdout:Context: system_u:object_r:device_t:s0 2024-03-04T13:33:11.020 INFO:teuthology.orchestra.run.smithi169.stdout:Access: 2024-03-04 13:32:58.512715170 +0000 2024-03-04T13:33:11.020 INFO:teuthology.orchestra.run.smithi169.stdout:Modify: 2024-03-04 13:22:42.144013801 +0000 2024-03-04T13:33:11.020 INFO:teuthology.orchestra.run.smithi169.stdout:Change: 2024-03-04 13:22:42.144013801 +0000 2024-03-04T13:33:11.020 INFO:teuthology.orchestra.run.smithi169.stdout: Birth: 2024-03-04 13:22:42.144013801 +0000 2024-03-04T13:33:11.021 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-04T13:33:11.130 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records in 2024-03-04T13:33:11.131 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records out 2024-03-04T13:33:11.131 INFO:teuthology.orchestra.run.smithi169.stderr:512 bytes copied, 0.00029874 s, 1.7 MB/s 2024-03-04T13:33:11.133 DEBUG:teuthology.orchestra.run.smithi169:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-04T13:33:11.177 DEBUG:teuthology.orchestra.run.smithi169:> stat /dev/vg_nvme/lv_4 2024-03-04T13:33:11.270 INFO:teuthology.orchestra.run.smithi169.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-04T13:33:11.270 INFO:teuthology.orchestra.run.smithi169.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-04T13:33:11.270 INFO:teuthology.orchestra.run.smithi169.stdout:Device: 5h/5d Inode: 688 Links: 1 2024-03-04T13:33:11.270 INFO:teuthology.orchestra.run.smithi169.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-04T13:33:11.270 INFO:teuthology.orchestra.run.smithi169.stdout:Context: system_u:object_r:device_t:s0 2024-03-04T13:33:11.270 INFO:teuthology.orchestra.run.smithi169.stdout:Access: 2024-03-04 13:32:58.513715148 +0000 2024-03-04T13:33:11.270 INFO:teuthology.orchestra.run.smithi169.stdout:Modify: 2024-03-04 13:22:42.714000881 +0000 2024-03-04T13:33:11.270 INFO:teuthology.orchestra.run.smithi169.stdout:Change: 2024-03-04 13:22:42.714000881 +0000 2024-03-04T13:33:11.271 INFO:teuthology.orchestra.run.smithi169.stdout: Birth: 2024-03-04 13:22:42.714000881 +0000 2024-03-04T13:33:11.271 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-04T13:33:11.358 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records in 2024-03-04T13:33:11.358 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records out 2024-03-04T13:33:11.358 INFO:teuthology.orchestra.run.smithi169.stderr:512 bytes copied, 0.000345506 s, 1.5 MB/s 2024-03-04T13:33:11.363 DEBUG:teuthology.orchestra.run.smithi169:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-04T13:33:11.421 INFO:tasks.cephadm:Deploying osd.0 on smithi107 with /dev/vg_nvme/lv_4... 2024-03-04T13:33:11.421 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-04T13:33:12.543 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/mon.smithi107/config 2024-03-04T13:33:12.587 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:12 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:33:12.587 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:12 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:33:12.587 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:12 smithi107 ceph-mon[28739]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:33:12.587 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:12 smithi107 ceph-mon[28739]: Reconfiguring mgr.smithi107.wpzgwl (unknown last config time)... 2024-03-04T13:33:12.587 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:12 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi107.wpzgwl", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-04T13:33:12.587 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:12 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-04T13:33:12.587 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:12 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:33:12.587 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:12 smithi107 ceph-mon[28739]: Reconfiguring daemon mgr.smithi107.wpzgwl on smithi107 2024-03-04T13:33:12.604 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:12 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:33:12.604 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:12 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:33:12.604 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:12 smithi169 ceph-mon[36313]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:33:12.604 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:12 smithi169 ceph-mon[36313]: Reconfiguring mgr.smithi107.wpzgwl (unknown last config time)... 2024-03-04T13:33:12.605 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:12 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi107.wpzgwl", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-04T13:33:12.605 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:12 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-04T13:33:12.605 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:12 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:33:12.605 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:12 smithi169 ceph-mon[36313]: Reconfiguring daemon mgr.smithi107.wpzgwl on smithi107 2024-03-04T13:33:13.411 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:13 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:33:13.604 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:13 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:33:14.400 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:14 smithi107 ceph-mon[28739]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:33:14.400 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:14 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:33:14.400 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:14 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:33:14.400 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:14 smithi107 ceph-mon[28739]: Reconfiguring crash.smithi107 (monmap changed)... 2024-03-04T13:33:14.400 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:14 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi107", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-04T13:33:14.400 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:14 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:33:14.400 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:14 smithi107 ceph-mon[28739]: Reconfiguring daemon crash.smithi107 on smithi107 2024-03-04T13:33:14.604 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:14 smithi169 ceph-mon[36313]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:33:14.604 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:14 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:33:14.605 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:14 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:33:14.605 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:14 smithi169 ceph-mon[36313]: Reconfiguring crash.smithi107 (monmap changed)... 2024-03-04T13:33:14.605 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:14 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi107", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-04T13:33:14.605 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:14 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:33:14.605 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:14 smithi169 ceph-mon[36313]: Reconfiguring daemon crash.smithi107 on smithi107 2024-03-04T13:33:16.037 INFO:teuthology.orchestra.run.smithi107.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-04T13:33:16.037 INFO:teuthology.orchestra.run.smithi107.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-04T13:33:16.037 INFO:teuthology.orchestra.run.smithi107.stderr: stderr: 10+0 records in 2024-03-04T13:33:16.037 INFO:teuthology.orchestra.run.smithi107.stderr:10+0 records out 2024-03-04T13:33:16.037 INFO:teuthology.orchestra.run.smithi107.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0357956 s, 293 MB/s 2024-03-04T13:33:16.037 INFO:teuthology.orchestra.run.smithi107.stderr: stderr: 2024-03-04T13:33:16.037 INFO:teuthology.orchestra.run.smithi107.stderr:--> Zapping successful for: 2024-03-04T13:33:16.763 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:16 smithi107 ceph-mon[28739]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:33:16.809 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 -- ceph orch daemon add osd smithi107:vg_nvme/lv_4 2024-03-04T13:33:16.854 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:16 smithi169 ceph-mon[36313]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:33:17.767 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:17 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:33:17.767 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:17 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:33:17.767 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:17 smithi107 ceph-mon[28739]: Reconfiguring alertmanager.smithi107 (dependencies changed)... 2024-03-04T13:33:17.767 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:17 smithi107 ceph-mon[28739]: Reconfiguring daemon alertmanager.smithi107 on smithi107 2024-03-04T13:33:17.854 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:17 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:33:17.854 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:17 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:33:17.854 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:17 smithi169 ceph-mon[36313]: Reconfiguring alertmanager.smithi107 (dependencies changed)... 2024-03-04T13:33:17.854 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:17 smithi169 ceph-mon[36313]: Reconfiguring daemon alertmanager.smithi107 on smithi107 2024-03-04T13:33:18.738 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:18 smithi107 ceph-mon[28739]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:33:18.747 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/mon.smithi107/config 2024-03-04T13:33:18.855 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:18 smithi169 ceph-mon[36313]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:33:20.814 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:20 smithi107 ceph-mon[28739]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:33:20.854 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:20 smithi169 ceph-mon[36313]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:33:22.649 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:22 smithi107 ceph-mon[28739]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:33:22.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:33:22.716+0000 7fa768c00700 1 -- 172.21.15.107:0/1136310900 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa7640fda80 msgr2=0x7fa7640ffe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:33:22.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:33:22.716+0000 7fa768c00700 1 --2- 172.21.15.107:0/1136310900 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa7640fda80 0x7fa7640ffe60 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa754004940 tx=0x7fa754009e30 comp rx=0 tx=0).stop 2024-03-04T13:33:22.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:33:22.716+0000 7fa768c00700 1 -- 172.21.15.107:0/1136310900 shutdown_connections 2024-03-04T13:33:22.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:33:22.716+0000 7fa768c00700 1 --2- 172.21.15.107:0/1136310900 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa7641003a0 0x7fa764102780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:33:22.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:33:22.716+0000 7fa768c00700 1 --2- 172.21.15.107:0/1136310900 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa7640fda80 0x7fa7640ffe60 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:33:22.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:33:22.716+0000 7fa768c00700 1 -- 172.21.15.107:0/1136310900 >> 172.21.15.107:0/1136310900 conn(0x7fa7640f74e0 msgr2=0x7fa7640f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:33:22.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:33:22.716+0000 7fa768c00700 1 -- 172.21.15.107:0/1136310900 shutdown_connections 2024-03-04T13:33:22.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:33:22.717+0000 7fa768c00700 1 -- 172.21.15.107:0/1136310900 wait complete. 2024-03-04T13:33:22.713 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:33:22.717+0000 7fa768c00700 1 Processor -- start 2024-03-04T13:33:22.713 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:33:22.718+0000 7fa768c00700 1 -- start start 2024-03-04T13:33:22.713 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:33:22.718+0000 7fa768c00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa7640fda80 0x7fa7640fd510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:33:22.713 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:33:22.718+0000 7fa768c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa7641003a0 0x7fa7640fb990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:33:22.713 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:33:22.718+0000 7fa768c00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7640fc020 con 0x7fa7641003a0 2024-03-04T13:33:22.713 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:33:22.718+0000 7fa768c00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7640fc190 con 0x7fa7640fda80 2024-03-04T13:33:22.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:33:22.718+0000 7fa761400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa7641003a0 0x7fa7640fb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:33:22.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:33:22.718+0000 7fa761400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa7641003a0 0x7fa7640fb990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39880/0 (socket says 172.21.15.107:39880) 2024-03-04T13:33:22.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:33:22.718+0000 7fa761400700 1 -- 172.21.15.107:0/278643892 learned_addr learned my addr 172.21.15.107:0/278643892 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:33:22.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:33:22.719+0000 7fa761e00700 1 --2- 172.21.15.107:0/278643892 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa7640fda80 0x7fa7640fd510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:33:22.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:33:22.719+0000 7fa761400700 1 -- 172.21.15.107:0/278643892 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa7640fda80 msgr2=0x7fa7640fd510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:33:22.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:33:22.719+0000 7fa761400700 1 --2- 172.21.15.107:0/278643892 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa7640fda80 0x7fa7640fd510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:33:22.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:33:22.719+0000 7fa761400700 1 -- 172.21.15.107:0/278643892 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa754005040 con 0x7fa7641003a0 2024-03-04T13:33:22.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:33:22.719+0000 7fa761400700 1 --2- 172.21.15.107:0/278643892 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa7641003a0 0x7fa7640fb990 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa75000b790 tx=0x7fa750009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:33:22.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:33:22.720+0000 7fa75a000700 1 -- 172.21.15.107:0/278643892 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa75000f770 con 0x7fa7641003a0 2024-03-04T13:33:22.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:33:22.720+0000 7fa75a000700 1 -- 172.21.15.107:0/278643892 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa75000fda0 con 0x7fa7641003a0 2024-03-04T13:33:22.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:33:22.720+0000 7fa768c00700 1 -- 172.21.15.107:0/278643892 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7640fc470 con 0x7fa7641003a0 2024-03-04T13:33:22.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:33:22.720+0000 7fa75a000700 1 -- 172.21.15.107:0/278643892 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa75000d310 con 0x7fa7641003a0 2024-03-04T13:33:22.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:33:22.720+0000 7fa768c00700 1 -- 172.21.15.107:0/278643892 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7640fc7f0 con 0x7fa7641003a0 2024-03-04T13:33:22.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:33:22.721+0000 7fa768c00700 1 -- 172.21.15.107:0/278643892 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa764107ad0 con 0x7fa7641003a0 2024-03-04T13:33:22.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:33:22.722+0000 7fa75a000700 1 -- 172.21.15.107:0/278643892 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 86567+0+0 (secure 0 0 0) 0x7fa750010420 con 0x7fa7641003a0 2024-03-04T13:33:22.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:33:22.723+0000 7fa75a000700 1 --2- 172.21.15.107:0/278643892 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fa748069210 0x7fa74806b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:33:22.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:33:22.723+0000 7fa75a000700 1 -- 172.21.15.107:0/278643892 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7fa750086130 con 0x7fa7641003a0 2024-03-04T13:33:22.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:33:22.724+0000 7fa761e00700 1 --2- 172.21.15.107:0/278643892 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fa748069210 0x7fa74806b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:33:22.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:33:22.724+0000 7fa761e00700 1 --2- 172.21.15.107:0/278643892 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fa748069210 0x7fa74806b6c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa75400b3b0 tx=0x7fa754009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:33:22.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:33:22.729+0000 7fa75a000700 1 -- 172.21.15.107:0/278643892 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fa750056f30 con 0x7fa7641003a0 2024-03-04T13:33:22.854 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:22 smithi169 ceph-mon[36313]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:33:22.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:33:22.947+0000 7fa768c00700 1 -- 172.21.15.107:0/278643892 --> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fa76402b310 con 0x7fa748069210 2024-03-04T13:33:23.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:23 smithi107 ceph-mon[28739]: from='client.14248 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:33:23.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:23 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T13:33:23.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:23 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T13:33:23.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:23 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:33:23.854 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:23 smithi169 ceph-mon[36313]: from='client.14248 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:33:23.854 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:23 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T13:33:23.854 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:23 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T13:33:23.855 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:23 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:33:24.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:24 smithi107 ceph-mon[28739]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:33:24.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:24 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:33:24.838 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:24 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:33:24.854 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:24 smithi169 ceph-mon[36313]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:33:24.854 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:24 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:33:24.854 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:24 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:33:25.587 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:25 smithi107 ceph-mon[28739]: Reconfiguring grafana.smithi107 (dependencies changed)... 2024-03-04T13:33:25.587 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:25 smithi107 ceph-mon[28739]: Reconfiguring daemon grafana.smithi107 on smithi107 2024-03-04T13:33:25.853 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:25 smithi169 ceph-mon[36313]: Reconfiguring grafana.smithi107 (dependencies changed)... 2024-03-04T13:33:25.854 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:25 smithi169 ceph-mon[36313]: Reconfiguring daemon grafana.smithi107 on smithi107 2024-03-04T13:33:26.742 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:26 smithi107 ceph-mon[28739]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:33:26.854 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:26 smithi169 ceph-mon[36313]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:33:27.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:27 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:33:27.853 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:27 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:33:28.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:28 smithi107 ceph-mon[28739]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:33:28.853 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:28 smithi169 ceph-mon[36313]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:33:30.786 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:30 smithi107 ceph-mon[28739]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:33:30.854 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:30 smithi169 ceph-mon[36313]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:33:32.056 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:31 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:33:32.056 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:31 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:33:32.057 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:31 smithi107 ceph-mon[28739]: Reconfiguring prometheus.smithi107 (dependencies changed)... 2024-03-04T13:33:32.057 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:31 smithi107 ceph-mon[28739]: Reconfiguring daemon prometheus.smithi107 on smithi107 2024-03-04T13:33:32.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:31 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:33:32.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:31 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:33:32.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:31 smithi169 ceph-mon[36313]: Reconfiguring prometheus.smithi107 (dependencies changed)... 2024-03-04T13:33:32.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:31 smithi169 ceph-mon[36313]: Reconfiguring daemon prometheus.smithi107 on smithi107 2024-03-04T13:33:33.019 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:32 smithi107 ceph-mon[28739]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:33:33.019 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:32 smithi107 ceph-mon[28739]: from='client.? 172.21.15.107:0/1758695189' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c857e387-3d2b-478d-9807-24ea8b9fe02d"}]: dispatch 2024-03-04T13:33:33.019 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:32 smithi107 ceph-mon[28739]: from='client.? 172.21.15.107:0/1758695189' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c857e387-3d2b-478d-9807-24ea8b9fe02d"}]': finished 2024-03-04T13:33:33.019 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:32 smithi107 ceph-mon[28739]: osdmap e6: 1 total, 0 up, 1 in 2024-03-04T13:33:33.019 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:32 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-04T13:33:33.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:32 smithi169 ceph-mon[36313]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:33:33.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:32 smithi169 ceph-mon[36313]: from='client.? 172.21.15.107:0/1758695189' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c857e387-3d2b-478d-9807-24ea8b9fe02d"}]: dispatch 2024-03-04T13:33:33.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:32 smithi169 ceph-mon[36313]: from='client.? 172.21.15.107:0/1758695189' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c857e387-3d2b-478d-9807-24ea8b9fe02d"}]': finished 2024-03-04T13:33:33.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:32 smithi169 ceph-mon[36313]: osdmap e6: 1 total, 0 up, 1 in 2024-03-04T13:33:33.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:32 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-04T13:33:34.077 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:33 smithi107 ceph-mon[28739]: from='client.? 172.21.15.107:0/4068192137' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T13:33:34.105 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:33 smithi169 ceph-mon[36313]: from='client.? 172.21.15.107:0/4068192137' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T13:33:35.086 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:34 smithi107 ceph-mon[28739]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:33:35.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:34 smithi169 ceph-mon[36313]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:33:37.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:36 smithi107 ceph-mon[28739]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:33:37.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:36 smithi169 ceph-mon[36313]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:33:39.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:38 smithi107 ceph-mon[28739]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:33:39.105 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:38 smithi169 ceph-mon[36313]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:33:40.902 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:40 smithi107 ceph-mon[28739]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:33:41.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:40 smithi169 ceph-mon[36313]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:33:42.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:42 smithi107 ceph-mon[28739]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:33:42.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:42 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:33:43.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:42 smithi169 ceph-mon[36313]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:33:43.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:42 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:33:45.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:44 smithi107 ceph-mon[28739]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:33:45.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:44 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:33:45.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:44 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:33:45.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:44 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi169", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-04T13:33:45.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:44 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:33:45.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:44 smithi169 ceph-mon[36313]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:33:45.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:44 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:33:45.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:44 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:33:45.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:44 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi169", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-04T13:33:45.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:44 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:33:45.892 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:45 smithi169 ceph-mon[36313]: Reconfiguring crash.smithi169 (monmap changed)... 2024-03-04T13:33:45.892 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:45 smithi169 ceph-mon[36313]: Reconfiguring daemon crash.smithi169 on smithi169 2024-03-04T13:33:45.957 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:45 smithi107 ceph-mon[28739]: Reconfiguring crash.smithi169 (monmap changed)... 2024-03-04T13:33:45.957 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:45 smithi107 ceph-mon[28739]: Reconfiguring daemon crash.smithi169 on smithi169 2024-03-04T13:33:47.005 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:46 smithi169 ceph-mon[36313]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:33:47.006 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:46 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-04T13:33:47.006 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:46 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:33:47.006 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:46 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:33:47.006 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:46 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:33:47.006 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:46 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi169.eihczr", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-04T13:33:47.006 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:46 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-04T13:33:47.006 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:46 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:33:47.061 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:46 smithi107 ceph-mon[28739]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:33:47.061 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:46 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-04T13:33:47.062 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:46 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:33:47.062 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:46 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:33:47.062 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:46 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:33:47.062 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:46 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi169.eihczr", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-04T13:33:47.062 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:46 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-04T13:33:47.062 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:46 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:33:47.921 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:47 smithi107 ceph-mon[28739]: Deploying daemon osd.0 on smithi107 2024-03-04T13:33:47.921 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:47 smithi107 ceph-mon[28739]: Reconfiguring mgr.smithi169.eihczr (monmap changed)... 2024-03-04T13:33:47.922 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:47 smithi107 ceph-mon[28739]: Reconfiguring daemon mgr.smithi169.eihczr on smithi169 2024-03-04T13:33:48.103 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:47 smithi169 ceph-mon[36313]: Deploying daemon osd.0 on smithi107 2024-03-04T13:33:48.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:47 smithi169 ceph-mon[36313]: Reconfiguring mgr.smithi169.eihczr (monmap changed)... 2024-03-04T13:33:48.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:47 smithi169 ceph-mon[36313]: Reconfiguring daemon mgr.smithi169.eihczr on smithi169 2024-03-04T13:33:49.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:48 smithi169 ceph-mon[36313]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:33:49.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:48 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:33:49.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:48 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:33:49.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:48 smithi169 ceph-mon[36313]: Reconfiguring mon.smithi169 (monmap changed)... 2024-03-04T13:33:49.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:48 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-04T13:33:49.105 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:48 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-04T13:33:49.105 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:48 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:33:49.105 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:48 smithi169 ceph-mon[36313]: Reconfiguring daemon mon.smithi169 on smithi169 2024-03-04T13:33:49.248 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:48 smithi107 ceph-mon[28739]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:33:49.248 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:48 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:33:49.248 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:48 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:33:49.248 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:48 smithi107 ceph-mon[28739]: Reconfiguring mon.smithi169 (monmap changed)... 2024-03-04T13:33:49.248 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:48 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-04T13:33:49.248 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:48 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-04T13:33:49.249 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:48 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:33:49.249 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:48 smithi107 ceph-mon[28739]: Reconfiguring daemon mon.smithi169 on smithi169 2024-03-04T13:33:51.005 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:50 smithi107 ceph-mon[28739]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:33:51.006 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:50 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:33:51.006 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:50 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:33:51.006 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:50 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-04T13:33:51.006 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:50 smithi107 ceph-mon[28739]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-04T13:33:51.006 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:50 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi107.front.sepia.ceph.com:9093"}]: dispatch 2024-03-04T13:33:51.006 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:50 smithi107 ceph-mon[28739]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi107.front.sepia.ceph.com:9093"}]: dispatch 2024-03-04T13:33:51.007 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:50 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:33:51.007 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:50 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-04T13:33:51.007 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:50 smithi107 ceph-mon[28739]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-04T13:33:51.007 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:50 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi107.front.sepia.ceph.com:3000"}]: dispatch 2024-03-04T13:33:51.007 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:50 smithi107 ceph-mon[28739]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi107.front.sepia.ceph.com:3000"}]: dispatch 2024-03-04T13:33:51.007 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:50 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:33:51.007 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:50 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-04T13:33:51.008 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:50 smithi107 ceph-mon[28739]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-04T13:33:51.008 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:50 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi107.front.sepia.ceph.com:9095"}]: dispatch 2024-03-04T13:33:51.008 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:50 smithi107 ceph-mon[28739]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi107.front.sepia.ceph.com:9095"}]: dispatch 2024-03-04T13:33:51.008 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:50 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:33:51.008 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:50 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:33:51.018 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:50 smithi169 ceph-mon[36313]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:33:51.018 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:50 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:33:51.018 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:50 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:33:51.018 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:50 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-04T13:33:51.018 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:50 smithi169 ceph-mon[36313]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-04T13:33:51.018 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:50 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi107.front.sepia.ceph.com:9093"}]: dispatch 2024-03-04T13:33:51.019 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:50 smithi169 ceph-mon[36313]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi107.front.sepia.ceph.com:9093"}]: dispatch 2024-03-04T13:33:51.019 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:50 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:33:51.019 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:50 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-04T13:33:51.019 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:50 smithi169 ceph-mon[36313]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-04T13:33:51.019 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:50 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi107.front.sepia.ceph.com:3000"}]: dispatch 2024-03-04T13:33:51.019 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:50 smithi169 ceph-mon[36313]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi107.front.sepia.ceph.com:3000"}]: dispatch 2024-03-04T13:33:51.019 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:50 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:33:51.019 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:50 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-04T13:33:51.019 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:50 smithi169 ceph-mon[36313]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-04T13:33:51.020 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:50 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi107.front.sepia.ceph.com:9095"}]: dispatch 2024-03-04T13:33:51.020 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:50 smithi169 ceph-mon[36313]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi107.front.sepia.ceph.com:9095"}]: dispatch 2024-03-04T13:33:51.020 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:50 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:33:51.020 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:50 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:33:53.001 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:52 smithi107 ceph-mon[28739]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:33:53.001 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:52 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:33:53.001 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:52 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:33:53.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:52 smithi169 ceph-mon[36313]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:33:53.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:52 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:33:53.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:52 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:33:54.836 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:54 smithi107 ceph-mon[28739]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:33:55.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:54 smithi169 ceph-mon[36313]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:33:56.077 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:55 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:33:56.077 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:55 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:33:56.103 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:55 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:33:56.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:55 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:33:57.086 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:56 smithi107 ceph-mon[28739]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:33:57.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:56 smithi169 ceph-mon[36313]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:33:58.056 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:57 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:33:58.057 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:57 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:33:58.057 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:57 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:33:58.076 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:57 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:33:58.076 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:57 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:33:58.076 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:57 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:33:59.028 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:58 smithi107 ceph-mon[28739]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:33:59.028 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:58 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:33:59.028 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:58 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:33:59.028 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:58 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:33:59.028 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:58 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:33:59.029 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:33:58 smithi107 ceph-mon[28739]: from='osd.0 [v2:172.21.15.107:6802/3721776817,v1:172.21.15.107:6803/3721776817]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-04T13:33:59.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:58 smithi169 ceph-mon[36313]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:33:59.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:58 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:33:59.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:58 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:33:59.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:58 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:33:59.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:58 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:33:59.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:33:58 smithi169 ceph-mon[36313]: from='osd.0 [v2:172.21.15.107:6802/3721776817,v1:172.21.15.107:6803/3721776817]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-04T13:33:59.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:33:59.925+0000 7fa75a000700 1 -- 172.21.15.107:0/278643892 <== mgr.14212 v2:172.21.15.107:6800/3848495603 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fa76402b310 con 0x7fa748069210 2024-03-04T13:33:59.920 INFO:teuthology.orchestra.run.smithi107.stdout:Created osd(s) 0 on host 'smithi107' 2024-03-04T13:33:59.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:33:59.928+0000 7fa768c00700 1 -- 172.21.15.107:0/278643892 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fa748069210 msgr2=0x7fa74806b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:33:59.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:33:59.928+0000 7fa768c00700 1 --2- 172.21.15.107:0/278643892 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fa748069210 0x7fa74806b6c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa75400b3b0 tx=0x7fa754009e30 comp rx=0 tx=0).stop 2024-03-04T13:33:59.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:33:59.928+0000 7fa768c00700 1 -- 172.21.15.107:0/278643892 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa7641003a0 msgr2=0x7fa7640fb990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:33:59.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:33:59.928+0000 7fa768c00700 1 --2- 172.21.15.107:0/278643892 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa7641003a0 0x7fa7640fb990 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa75000b790 tx=0x7fa750009250 comp rx=0 tx=0).stop 2024-03-04T13:33:59.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:33:59.928+0000 7fa768c00700 1 -- 172.21.15.107:0/278643892 shutdown_connections 2024-03-04T13:33:59.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:33:59.928+0000 7fa768c00700 1 --2- 172.21.15.107:0/278643892 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fa748069210 0x7fa74806b6c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:33:59.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:33:59.928+0000 7fa768c00700 1 --2- 172.21.15.107:0/278643892 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa7640fda80 0x7fa7640fd510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:33:59.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:33:59.928+0000 7fa768c00700 1 --2- 172.21.15.107:0/278643892 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa7641003a0 0x7fa7640fb990 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:33:59.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:33:59.928+0000 7fa768c00700 1 -- 172.21.15.107:0/278643892 >> 172.21.15.107:0/278643892 conn(0x7fa7640f74e0 msgr2=0x7fa7640f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:33:59.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:33:59.929+0000 7fa768c00700 1 -- 172.21.15.107:0/278643892 shutdown_connections 2024-03-04T13:33:59.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:33:59.929+0000 7fa768c00700 1 -- 172.21.15.107:0/278643892 wait complete. 2024-03-04T13:34:00.575 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:00 smithi169 ceph-mon[36313]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:34:00.575 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:00 smithi169 ceph-mon[36313]: from='osd.0 [v2:172.21.15.107:6802/3721776817,v1:172.21.15.107:6803/3721776817]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-04T13:34:00.575 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:00 smithi169 ceph-mon[36313]: osdmap e7: 1 total, 0 up, 1 in 2024-03-04T13:34:00.575 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:00 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-04T13:34:00.575 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:00 smithi169 ceph-mon[36313]: from='osd.0 [v2:172.21.15.107:6802/3721776817,v1:172.21.15.107:6803/3721776817]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-03-04T13:34:00.575 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:00 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:34:00.575 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:00 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:34:00.586 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:00 smithi107 ceph-mon[28739]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:34:00.586 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:00 smithi107 ceph-mon[28739]: from='osd.0 [v2:172.21.15.107:6802/3721776817,v1:172.21.15.107:6803/3721776817]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-04T13:34:00.587 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:00 smithi107 ceph-mon[28739]: osdmap e7: 1 total, 0 up, 1 in 2024-03-04T13:34:00.587 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:00 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-04T13:34:00.587 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:00 smithi107 ceph-mon[28739]: from='osd.0 [v2:172.21.15.107:6802/3721776817,v1:172.21.15.107:6803/3721776817]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-03-04T13:34:00.587 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:00 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:34:00.587 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:00 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:34:01.551 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:01 smithi107 ceph-mon[28739]: from='osd.0 [v2:172.21.15.107:6802/3721776817,v1:172.21.15.107:6803/3721776817]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-03-04T13:34:01.551 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:01 smithi107 ceph-mon[28739]: osdmap e8: 1 total, 0 up, 1 in 2024-03-04T13:34:01.551 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:01 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-04T13:34:01.551 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:01 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-04T13:34:01.551 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:01 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:34:01.551 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:01 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:34:01.554 DEBUG:teuthology.orchestra.run.smithi107:osd.0> sudo journalctl -f -n 0 -u ceph-1fc09a58-da2b-11ee-95c6-87774f69a715@osd.0.service 2024-03-04T13:34:01.561 INFO:tasks.cephadm:Deploying osd.1 on smithi107 with /dev/vg_nvme/lv_3... 2024-03-04T13:34:01.561 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-04T13:34:01.604 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:01 smithi169 ceph-mon[36313]: from='osd.0 [v2:172.21.15.107:6802/3721776817,v1:172.21.15.107:6803/3721776817]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-03-04T13:34:01.604 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:01 smithi169 ceph-mon[36313]: osdmap e8: 1 total, 0 up, 1 in 2024-03-04T13:34:01.604 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:01 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-04T13:34:01.604 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:01 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-04T13:34:01.604 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:01 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:34:01.604 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:01 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:34:02.337 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:02 smithi107 ceph-mon[28739]: purged_snaps scrub starts 2024-03-04T13:34:02.337 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:02 smithi107 ceph-mon[28739]: purged_snaps scrub ok 2024-03-04T13:34:02.337 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:02 smithi107 ceph-mon[28739]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:34:02.337 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:02 smithi107 ceph-mon[28739]: osd.0 [v2:172.21.15.107:6802/3721776817,v1:172.21.15.107:6803/3721776817] boot 2024-03-04T13:34:02.337 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:02 smithi107 ceph-mon[28739]: osdmap e9: 1 total, 1 up, 1 in 2024-03-04T13:34:02.337 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:02 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-04T13:34:02.584 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/mon.smithi107/config 2024-03-04T13:34:02.604 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:02 smithi169 ceph-mon[36313]: purged_snaps scrub starts 2024-03-04T13:34:02.604 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:02 smithi169 ceph-mon[36313]: purged_snaps scrub ok 2024-03-04T13:34:02.604 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:02 smithi169 ceph-mon[36313]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T13:34:02.604 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:02 smithi169 ceph-mon[36313]: osd.0 [v2:172.21.15.107:6802/3721776817,v1:172.21.15.107:6803/3721776817] boot 2024-03-04T13:34:02.604 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:02 smithi169 ceph-mon[36313]: osdmap e9: 1 total, 1 up, 1 in 2024-03-04T13:34:02.604 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:02 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-04T13:34:03.587 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:03 smithi107 ceph-mon[28739]: osdmap e10: 1 total, 1 up, 1 in 2024-03-04T13:34:03.603 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:03 smithi169 ceph-mon[36313]: osdmap e10: 1 total, 1 up, 1 in 2024-03-04T13:34:04.531 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:04 smithi107 ceph-mon[28739]: pgmap v46: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-04T13:34:04.604 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:04 smithi169 ceph-mon[36313]: pgmap v46: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-04T13:34:05.870 INFO:teuthology.orchestra.run.smithi107.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-04T13:34:05.870 INFO:teuthology.orchestra.run.smithi107.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-04T13:34:05.870 INFO:teuthology.orchestra.run.smithi107.stderr: stderr: 10+0 records in 2024-03-04T13:34:05.870 INFO:teuthology.orchestra.run.smithi107.stderr:10+0 records out 2024-03-04T13:34:05.870 INFO:teuthology.orchestra.run.smithi107.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0323223 s, 324 MB/s 2024-03-04T13:34:05.870 INFO:teuthology.orchestra.run.smithi107.stderr:--> Zapping successful for: 2024-03-04T13:34:06.573 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:06 smithi107 ceph-mon[28739]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-04T13:34:06.650 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 -- ceph orch daemon add osd smithi107:vg_nvme/lv_3 2024-03-04T13:34:06.853 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:06 smithi169 ceph-mon[36313]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-04T13:34:07.471 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/mon.smithi107/config 2024-03-04T13:34:08.707 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:08 smithi107 ceph-mon[28739]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-04T13:34:08.708 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:08 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:34:08.708 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:08 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:34:08.854 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:08 smithi169 ceph-mon[36313]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-04T13:34:08.854 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:08 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:34:08.854 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:08 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:34:10.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:34:10.146+0000 7f26d9c00700 1 -- 172.21.15.107:0/378953774 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f26d40708c0 msgr2=0x7f26d4070d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:34:10.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:34:10.146+0000 7f26d9c00700 1 --2- 172.21.15.107:0/378953774 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f26d40708c0 0x7f26d4070d10 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f26c0004940 tx=0x7f26c0009e30 comp rx=0 tx=0).stop 2024-03-04T13:34:10.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:34:10.146+0000 7f26d9c00700 1 -- 172.21.15.107:0/378953774 shutdown_connections 2024-03-04T13:34:10.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:34:10.146+0000 7f26d9c00700 1 --2- 172.21.15.107:0/378953774 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f26d40708c0 0x7f26d4070d10 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:34:10.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:34:10.146+0000 7f26d9c00700 1 --2- 172.21.15.107:0/378953774 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26d4071f10 0x7f26d4070380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:34:10.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:34:10.146+0000 7f26d9c00700 1 -- 172.21.15.107:0/378953774 >> 172.21.15.107:0/378953774 conn(0x7f26d40f8430 msgr2=0x7f26d40fa840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:34:10.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:34:10.146+0000 7f26d9c00700 1 -- 172.21.15.107:0/378953774 shutdown_connections 2024-03-04T13:34:10.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:34:10.146+0000 7f26d9c00700 1 -- 172.21.15.107:0/378953774 wait complete. 2024-03-04T13:34:10.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:34:10.147+0000 7f26d9c00700 1 Processor -- start 2024-03-04T13:34:10.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:34:10.147+0000 7f26d9c00700 1 -- start start 2024-03-04T13:34:10.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:34:10.147+0000 7f26d9c00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26d40708c0 0x7f26d410f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:34:10.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:34:10.147+0000 7f26d9c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f26d4071f10 0x7f26d410fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:34:10.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:34:10.147+0000 7f26d9c00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26d4110470 con 0x7f26d4071f10 2024-03-04T13:34:10.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:34:10.147+0000 7f26d9c00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26d4110d10 con 0x7f26d40708c0 2024-03-04T13:34:10.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:34:10.147+0000 7f26d3400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f26d4071f10 0x7f26d410fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:34:10.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:34:10.147+0000 7f26d3400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f26d4071f10 0x7f26d410fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40402/0 (socket says 172.21.15.107:40402) 2024-03-04T13:34:10.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:34:10.147+0000 7f26d3400700 1 -- 172.21.15.107:0/1502907652 learned_addr learned my addr 172.21.15.107:0/1502907652 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:34:10.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:34:10.148+0000 7f26d3e00700 1 --2- 172.21.15.107:0/1502907652 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26d40708c0 0x7f26d410f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:34:10.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:34:10.148+0000 7f26d3400700 1 -- 172.21.15.107:0/1502907652 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26d40708c0 msgr2=0x7f26d410f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:34:10.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:34:10.148+0000 7f26d3400700 1 --2- 172.21.15.107:0/1502907652 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26d40708c0 0x7f26d410f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:34:10.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:34:10.148+0000 7f26d3400700 1 -- 172.21.15.107:0/1502907652 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26c0005040 con 0x7f26d4071f10 2024-03-04T13:34:10.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:34:10.148+0000 7f26d3400700 1 --2- 172.21.15.107:0/1502907652 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f26d4071f10 0x7f26d410fe70 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f26c0000c00 tx=0x7f26c0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:34:10.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:34:10.148+0000 7f26cbe00700 1 -- 172.21.15.107:0/1502907652 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f26c000e040 con 0x7f26d4071f10 2024-03-04T13:34:10.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:34:10.148+0000 7f26d9c00700 1 -- 172.21.15.107:0/1502907652 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26d4110f90 con 0x7f26d4071f10 2024-03-04T13:34:10.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:34:10.148+0000 7f26cbe00700 1 -- 172.21.15.107:0/1502907652 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f26c0004290 con 0x7f26d4071f10 2024-03-04T13:34:10.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:34:10.148+0000 7f26cbe00700 1 -- 172.21.15.107:0/1502907652 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f26c001c630 con 0x7f26d4071f10 2024-03-04T13:34:10.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:34:10.149+0000 7f26d9c00700 1 -- 172.21.15.107:0/1502907652 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26d41114d0 con 0x7f26d4071f10 2024-03-04T13:34:10.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:34:10.150+0000 7f26d9c00700 1 -- 172.21.15.107:0/1502907652 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26d404bed0 con 0x7f26d4071f10 2024-03-04T13:34:10.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:34:10.151+0000 7f26cbe00700 1 -- 172.21.15.107:0/1502907652 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 86567+0+0 (secure 0 0 0) 0x7f26c000bc60 con 0x7f26d4071f10 2024-03-04T13:34:10.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:34:10.151+0000 7f26cbe00700 1 --2- 172.21.15.107:0/1502907652 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f26b8069210 0x7f26b806b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:34:10.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:34:10.152+0000 7f26cbe00700 1 -- 172.21.15.107:0/1502907652 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1978+0+0 (secure 0 0 0) 0x7f26c0087760 con 0x7f26d4071f10 2024-03-04T13:34:10.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:34:10.152+0000 7f26d3e00700 1 --2- 172.21.15.107:0/1502907652 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f26b8069210 0x7f26b806b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:34:10.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:34:10.152+0000 7f26d3e00700 1 --2- 172.21.15.107:0/1502907652 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f26b8069210 0x7f26b806b6c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f26c4007fd0 tx=0x7f26c4009040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:34:10.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:34:10.161+0000 7f26cbe00700 1 -- 172.21.15.107:0/1502907652 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f26c0058300 con 0x7f26d4071f10 2024-03-04T13:34:10.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:34:10.377+0000 7f26d9c00700 1 -- 172.21.15.107:0/1502907652 --> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f26d41119b0 con 0x7f26b8069210 2024-03-04T13:34:10.717 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:10 smithi107 ceph-mon[28739]: pgmap v49: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-04T13:34:10.717 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:10 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T13:34:10.717 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:10 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T13:34:10.717 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:10 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:34:10.853 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:10 smithi169 ceph-mon[36313]: pgmap v49: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-04T13:34:10.854 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:10 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T13:34:10.854 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:10 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T13:34:10.854 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:10 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:34:11.604 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:11 smithi169 ceph-mon[36313]: from='client.14266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:34:11.836 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:11 smithi107 ceph-mon[28739]: from='client.14266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:34:12.735 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:12 smithi107 ceph-mon[28739]: pgmap v50: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-04T13:34:12.735 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:12 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:34:12.854 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:12 smithi169 ceph-mon[36313]: pgmap v50: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-04T13:34:12.854 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:12 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:34:14.502 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:14 smithi169 ceph-mon[36313]: Detected new or changed devices on smithi107 2024-03-04T13:34:14.503 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:14 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:34:14.503 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:14 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:34:14.503 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:14 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-04T13:34:14.503 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:14 smithi169 ceph-mon[36313]: Adjusting osd_memory_target on smithi107 to 12866M 2024-03-04T13:34:14.503 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:14 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:34:14.503 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:14 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:34:14.503 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:14 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:34:14.503 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:14 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:34:14.503 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:14 smithi169 ceph-mon[36313]: pgmap v51: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-04T13:34:14.503 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:14 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:34:14.533 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:14 smithi107 ceph-mon[28739]: Detected new or changed devices on smithi107 2024-03-04T13:34:14.533 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:14 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:34:14.533 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:14 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:34:14.533 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:14 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-04T13:34:14.533 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:14 smithi107 ceph-mon[28739]: Adjusting osd_memory_target on smithi107 to 12866M 2024-03-04T13:34:14.533 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:14 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:34:14.533 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:14 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:34:14.534 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:14 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:34:14.534 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:14 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:34:14.534 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:14 smithi107 ceph-mon[28739]: pgmap v51: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-04T13:34:14.534 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:14 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:34:15.547 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:15 smithi107 ceph-mon[28739]: from='client.? 172.21.15.107:0/613307710' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fa29dc8f-419b-4318-b43e-e13fffd87204"}]: dispatch 2024-03-04T13:34:15.548 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:15 smithi107 ceph-mon[28739]: from='client.? 172.21.15.107:0/613307710' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fa29dc8f-419b-4318-b43e-e13fffd87204"}]': finished 2024-03-04T13:34:15.548 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:15 smithi107 ceph-mon[28739]: osdmap e11: 2 total, 1 up, 2 in 2024-03-04T13:34:15.548 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:15 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-04T13:34:15.604 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:15 smithi169 ceph-mon[36313]: from='client.? 172.21.15.107:0/613307710' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fa29dc8f-419b-4318-b43e-e13fffd87204"}]: dispatch 2024-03-04T13:34:15.604 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:15 smithi169 ceph-mon[36313]: from='client.? 172.21.15.107:0/613307710' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fa29dc8f-419b-4318-b43e-e13fffd87204"}]': finished 2024-03-04T13:34:15.604 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:15 smithi169 ceph-mon[36313]: osdmap e11: 2 total, 1 up, 2 in 2024-03-04T13:34:15.604 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:15 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-04T13:34:16.836 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:16 smithi107 ceph-mon[28739]: pgmap v53: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-04T13:34:16.836 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:16 smithi107 ceph-mon[28739]: from='client.? 172.21.15.107:0/3257627729' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T13:34:16.836 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:16 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:34:16.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:16 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:34:16.853 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:16 smithi169 ceph-mon[36313]: pgmap v53: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-04T13:34:16.854 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:16 smithi169 ceph-mon[36313]: from='client.? 172.21.15.107:0/3257627729' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T13:34:16.854 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:16 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:34:16.854 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:16 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:34:18.836 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:18 smithi107 ceph-mon[28739]: pgmap v54: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-04T13:34:18.853 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:18 smithi169 ceph-mon[36313]: pgmap v54: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-04T13:34:20.086 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:20 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:34:20.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:20 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:34:20.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:20 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:34:20.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:20 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:34:20.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:20 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:34:20.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:20 smithi107 ceph-mon[28739]: pgmap v55: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-04T13:34:20.354 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:20 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:34:20.354 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:20 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:34:20.354 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:20 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:34:20.354 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:20 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:34:20.354 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:20 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:34:20.354 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:20 smithi169 ceph-mon[36313]: pgmap v55: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-04T13:34:22.587 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:22 smithi107 ceph-mon[28739]: pgmap v56: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-04T13:34:22.603 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:22 smithi169 ceph-mon[36313]: pgmap v56: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-04T13:34:24.788 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:24 smithi107 ceph-mon[28739]: pgmap v57: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-04T13:34:24.789 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:24 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-04T13:34:24.789 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:24 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:34:24.789 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:24 smithi107 ceph-mon[28739]: Deploying daemon osd.1 on smithi107 2024-03-04T13:34:24.854 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:24 smithi169 ceph-mon[36313]: pgmap v57: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-04T13:34:24.854 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:24 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-04T13:34:24.854 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:24 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:34:24.854 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:24 smithi169 ceph-mon[36313]: Deploying daemon osd.1 on smithi107 2024-03-04T13:34:26.795 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:26 smithi107 ceph-mon[28739]: pgmap v58: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-04T13:34:26.853 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:26 smithi169 ceph-mon[36313]: pgmap v58: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-04T13:34:27.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:27 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:34:27.853 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:27 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:34:28.836 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:28 smithi107 ceph-mon[28739]: pgmap v59: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-04T13:34:28.853 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:28 smithi169 ceph-mon[36313]: pgmap v59: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-04T13:34:30.755 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:30 smithi107 ceph-mon[28739]: pgmap v60: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-04T13:34:30.853 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:30 smithi169 ceph-mon[36313]: pgmap v60: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-04T13:34:32.944 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:32 smithi107 ceph-mon[28739]: pgmap v61: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-04T13:34:32.944 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:32 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:34:32.944 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:32 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:34:32.945 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:32 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:34:33.103 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:32 smithi169 ceph-mon[36313]: pgmap v61: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-04T13:34:33.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:32 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:34:33.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:32 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:34:33.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:32 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:34:34.836 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:34 smithi107 ceph-mon[28739]: pgmap v62: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-04T13:34:35.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:34 smithi169 ceph-mon[36313]: pgmap v62: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-04T13:34:36.587 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:36 smithi107 ceph-mon[28739]: pgmap v63: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-04T13:34:36.587 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:36 smithi107 ceph-mon[28739]: from='osd.1 [v2:172.21.15.107:6810/4224626475,v1:172.21.15.107:6811/4224626475]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-04T13:34:36.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:34:36.711+0000 7f26cbe00700 1 -- 172.21.15.107:0/1502907652 <== mgr.14212 v2:172.21.15.107:6800/3848495603 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f26d41119b0 con 0x7f26b8069210 2024-03-04T13:34:36.706 INFO:teuthology.orchestra.run.smithi107.stdout:Created osd(s) 1 on host 'smithi107' 2024-03-04T13:34:36.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:34:36.714+0000 7f26d9c00700 1 -- 172.21.15.107:0/1502907652 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f26b8069210 msgr2=0x7f26b806b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:34:36.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:34:36.714+0000 7f26d9c00700 1 --2- 172.21.15.107:0/1502907652 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f26b8069210 0x7f26b806b6c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f26c4007fd0 tx=0x7f26c4009040 comp rx=0 tx=0).stop 2024-03-04T13:34:36.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:34:36.714+0000 7f26d9c00700 1 -- 172.21.15.107:0/1502907652 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f26d4071f10 msgr2=0x7f26d410fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:34:36.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:34:36.714+0000 7f26d9c00700 1 --2- 172.21.15.107:0/1502907652 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f26d4071f10 0x7f26d410fe70 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f26c0000c00 tx=0x7f26c0009e30 comp rx=0 tx=0).stop 2024-03-04T13:34:36.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:34:36.715+0000 7f26d9c00700 1 -- 172.21.15.107:0/1502907652 shutdown_connections 2024-03-04T13:34:36.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:34:36.715+0000 7f26d9c00700 1 --2- 172.21.15.107:0/1502907652 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f26b8069210 0x7f26b806b6c0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:34:36.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:34:36.715+0000 7f26d9c00700 1 --2- 172.21.15.107:0/1502907652 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f26d40708c0 0x7f26d410f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:34:36.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:34:36.715+0000 7f26d9c00700 1 --2- 172.21.15.107:0/1502907652 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f26d4071f10 0x7f26d410fe70 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:34:36.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:34:36.715+0000 7f26d9c00700 1 -- 172.21.15.107:0/1502907652 >> 172.21.15.107:0/1502907652 conn(0x7f26d40f8430 msgr2=0x7f26d4102be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:34:36.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:34:36.715+0000 7f26d9c00700 1 -- 172.21.15.107:0/1502907652 shutdown_connections 2024-03-04T13:34:36.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:34:36.715+0000 7f26d9c00700 1 -- 172.21.15.107:0/1502907652 wait complete. 2024-03-04T13:34:36.853 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:36 smithi169 ceph-mon[36313]: pgmap v63: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-04T13:34:36.854 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:36 smithi169 ceph-mon[36313]: from='osd.1 [v2:172.21.15.107:6810/4224626475,v1:172.21.15.107:6811/4224626475]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-04T13:34:37.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:37 smithi107 ceph-mon[28739]: from='osd.1 [v2:172.21.15.107:6810/4224626475,v1:172.21.15.107:6811/4224626475]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-04T13:34:37.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:37 smithi107 ceph-mon[28739]: osdmap e12: 2 total, 1 up, 2 in 2024-03-04T13:34:37.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:37 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-04T13:34:37.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:37 smithi107 ceph-mon[28739]: from='osd.1 [v2:172.21.15.107:6810/4224626475,v1:172.21.15.107:6811/4224626475]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-03-04T13:34:37.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:37 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:34:37.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:37 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:34:37.854 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:37 smithi169 ceph-mon[36313]: from='osd.1 [v2:172.21.15.107:6810/4224626475,v1:172.21.15.107:6811/4224626475]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-04T13:34:37.854 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:37 smithi169 ceph-mon[36313]: osdmap e12: 2 total, 1 up, 2 in 2024-03-04T13:34:37.854 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:37 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-04T13:34:37.854 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:37 smithi169 ceph-mon[36313]: from='osd.1 [v2:172.21.15.107:6810/4224626475,v1:172.21.15.107:6811/4224626475]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-03-04T13:34:37.854 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:37 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:34:37.854 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:37 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:34:38.219 DEBUG:teuthology.orchestra.run.smithi107:osd.1> sudo journalctl -f -n 0 -u ceph-1fc09a58-da2b-11ee-95c6-87774f69a715@osd.1.service 2024-03-04T13:34:38.226 INFO:tasks.cephadm:Deploying osd.2 on smithi107 with /dev/vg_nvme/lv_2... 2024-03-04T13:34:38.226 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-04T13:34:38.787 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:38 smithi107 ceph-mon[28739]: pgmap v65: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-04T13:34:38.788 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:38 smithi107 ceph-mon[28739]: from='osd.1 [v2:172.21.15.107:6810/4224626475,v1:172.21.15.107:6811/4224626475]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-03-04T13:34:38.788 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:38 smithi107 ceph-mon[28739]: osdmap e13: 2 total, 1 up, 2 in 2024-03-04T13:34:38.788 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:38 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-04T13:34:38.788 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:38 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-04T13:34:38.854 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:38 smithi169 ceph-mon[36313]: pgmap v65: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-04T13:34:38.854 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:38 smithi169 ceph-mon[36313]: from='osd.1 [v2:172.21.15.107:6810/4224626475,v1:172.21.15.107:6811/4224626475]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-03-04T13:34:38.854 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:38 smithi169 ceph-mon[36313]: osdmap e13: 2 total, 1 up, 2 in 2024-03-04T13:34:38.854 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:38 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-04T13:34:38.854 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:38 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-04T13:34:39.165 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/mon.smithi107/config 2024-03-04T13:34:39.521 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:39 smithi107 ceph-mon[28739]: purged_snaps scrub starts 2024-03-04T13:34:39.521 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:39 smithi107 ceph-mon[28739]: purged_snaps scrub ok 2024-03-04T13:34:39.521 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:39 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-04T13:34:39.521 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:39 smithi107 ceph-mon[28739]: osd.1 [v2:172.21.15.107:6810/4224626475,v1:172.21.15.107:6811/4224626475] boot 2024-03-04T13:34:39.522 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:39 smithi107 ceph-mon[28739]: osdmap e14: 2 total, 2 up, 2 in 2024-03-04T13:34:39.522 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:39 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-04T13:34:39.854 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:39 smithi169 ceph-mon[36313]: purged_snaps scrub starts 2024-03-04T13:34:39.854 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:39 smithi169 ceph-mon[36313]: purged_snaps scrub ok 2024-03-04T13:34:39.854 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:39 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-04T13:34:39.854 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:39 smithi169 ceph-mon[36313]: osd.1 [v2:172.21.15.107:6810/4224626475,v1:172.21.15.107:6811/4224626475] boot 2024-03-04T13:34:39.855 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:39 smithi169 ceph-mon[36313]: osdmap e14: 2 total, 2 up, 2 in 2024-03-04T13:34:39.855 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:39 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-04T13:34:40.776 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:40 smithi107 ceph-mon[28739]: pgmap v68: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-04T13:34:40.776 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:40 smithi107 ceph-mon[28739]: osdmap e15: 2 total, 2 up, 2 in 2024-03-04T13:34:40.781 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:40 smithi169 ceph-mon[36313]: pgmap v68: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-04T13:34:40.781 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:40 smithi169 ceph-mon[36313]: osdmap e15: 2 total, 2 up, 2 in 2024-03-04T13:34:42.586 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:42 smithi107 ceph-mon[28739]: pgmap v70: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-04T13:34:42.586 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:42 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:34:42.587 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:42 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:34:42.587 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:42 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:34:42.604 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:42 smithi169 ceph-mon[36313]: pgmap v70: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-04T13:34:42.604 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:42 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:34:42.604 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:42 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:34:42.604 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:42 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:34:42.711 INFO:teuthology.orchestra.run.smithi107.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-04T13:34:42.711 INFO:teuthology.orchestra.run.smithi107.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-04T13:34:42.711 INFO:teuthology.orchestra.run.smithi107.stderr: stderr: 10+0 records in 2024-03-04T13:34:42.712 INFO:teuthology.orchestra.run.smithi107.stderr:10+0 records out 2024-03-04T13:34:42.712 INFO:teuthology.orchestra.run.smithi107.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0388513 s, 270 MB/s 2024-03-04T13:34:42.712 INFO:teuthology.orchestra.run.smithi107.stderr:--> Zapping successful for: 2024-03-04T13:34:43.784 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 -- ceph orch daemon add osd smithi107:vg_nvme/lv_2 2024-03-04T13:34:44.343 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/mon.smithi107/config 2024-03-04T13:34:44.440 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:44 smithi107 ceph-mon[28739]: pgmap v71: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-04T13:34:44.853 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:44 smithi169 ceph-mon[36313]: pgmap v71: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-04T13:34:46.566 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:46 smithi107 ceph-mon[28739]: pgmap v72: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-04T13:34:46.853 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:46 smithi169 ceph-mon[36313]: pgmap v72: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-04T13:34:47.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:34:47.102+0000 7fc6ba800700 1 -- 172.21.15.107:0/1944000101 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6b40ff150 msgr2=0x7fc6b40ff560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:34:47.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:34:47.102+0000 7fc6ba800700 1 --2- 172.21.15.107:0/1944000101 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6b40ff150 0x7fc6b40ff560 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc6a4004940 tx=0x7fc6a4009e30 comp rx=0 tx=0).stop 2024-03-04T13:34:47.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:34:47.102+0000 7fc6ba800700 1 -- 172.21.15.107:0/1944000101 shutdown_connections 2024-03-04T13:34:47.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:34:47.102+0000 7fc6ba800700 1 --2- 172.21.15.107:0/1944000101 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc6b4100360 0x7fc6b41007b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:34:47.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:34:47.102+0000 7fc6ba800700 1 --2- 172.21.15.107:0/1944000101 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6b40ff150 0x7fc6b40ff560 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:34:47.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:34:47.102+0000 7fc6ba800700 1 -- 172.21.15.107:0/1944000101 >> 172.21.15.107:0/1944000101 conn(0x7fc6b40fa920 msgr2=0x7fc6b40fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:34:47.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:34:47.102+0000 7fc6ba800700 1 -- 172.21.15.107:0/1944000101 shutdown_connections 2024-03-04T13:34:47.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:34:47.102+0000 7fc6ba800700 1 -- 172.21.15.107:0/1944000101 wait complete. 2024-03-04T13:34:47.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:34:47.103+0000 7fc6ba800700 1 Processor -- start 2024-03-04T13:34:47.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:34:47.103+0000 7fc6ba800700 1 -- start start 2024-03-04T13:34:47.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:34:47.104+0000 7fc6ba800700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc6b40ff150 0x7fc6b410fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:34:47.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:34:47.104+0000 7fc6ba800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6b4100360 0x7fc6b41100d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:34:47.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:34:47.104+0000 7fc6ba800700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b41106d0 con 0x7fc6b4100360 2024-03-04T13:34:47.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:34:47.104+0000 7fc6ba800700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b4110f70 con 0x7fc6b40ff150 2024-03-04T13:34:47.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:34:47.104+0000 7fc6b2a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6b4100360 0x7fc6b41100d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:34:47.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:34:47.104+0000 7fc6b2a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6b4100360 0x7fc6b41100d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47144/0 (socket says 172.21.15.107:47144) 2024-03-04T13:34:47.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:34:47.104+0000 7fc6b2a00700 1 -- 172.21.15.107:0/745690901 learned_addr learned my addr 172.21.15.107:0/745690901 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:34:47.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:34:47.104+0000 7fc6b3400700 1 --2- 172.21.15.107:0/745690901 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc6b40ff150 0x7fc6b410fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:34:47.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:34:47.104+0000 7fc6b2a00700 1 -- 172.21.15.107:0/745690901 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc6b40ff150 msgr2=0x7fc6b410fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:34:47.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:34:47.104+0000 7fc6b2a00700 1 --2- 172.21.15.107:0/745690901 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc6b40ff150 0x7fc6b410fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:34:47.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:34:47.104+0000 7fc6b2a00700 1 -- 172.21.15.107:0/745690901 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6a4005040 con 0x7fc6b4100360 2024-03-04T13:34:47.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:34:47.105+0000 7fc6b2a00700 1 --2- 172.21.15.107:0/745690901 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6b4100360 0x7fc6b41100d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc6a0002a50 tx=0x7fc6a00046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:34:47.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:34:47.105+0000 7fc6ab400700 1 -- 172.21.15.107:0/745690901 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc6a0007cc0 con 0x7fc6b4100360 2024-03-04T13:34:47.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:34:47.105+0000 7fc6ab400700 1 -- 172.21.15.107:0/745690901 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc6a000f040 con 0x7fc6b4100360 2024-03-04T13:34:47.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:34:47.105+0000 7fc6ab400700 1 -- 172.21.15.107:0/745690901 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc6a0009640 con 0x7fc6b4100360 2024-03-04T13:34:47.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:34:47.105+0000 7fc6ba800700 1 -- 172.21.15.107:0/745690901 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6b4111250 con 0x7fc6b4100360 2024-03-04T13:34:47.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:34:47.105+0000 7fc6ba800700 1 -- 172.21.15.107:0/745690901 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6b4111790 con 0x7fc6b4100360 2024-03-04T13:34:47.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:34:47.106+0000 7fc6ba800700 1 -- 172.21.15.107:0/745690901 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6b404bed0 con 0x7fc6b4100360 2024-03-04T13:34:47.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:34:47.107+0000 7fc6ab400700 1 -- 172.21.15.107:0/745690901 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 86567+0+0 (secure 0 0 0) 0x7fc6a00097a0 con 0x7fc6b4100360 2024-03-04T13:34:47.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:34:47.108+0000 7fc6ab400700 1 --2- 172.21.15.107:0/745690901 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fc698069210 0x7fc69806b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:34:47.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:34:47.108+0000 7fc6ab400700 1 -- 172.21.15.107:0/745690901 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 2410+0+0 (secure 0 0 0) 0x7fc6a0086250 con 0x7fc6b4100360 2024-03-04T13:34:47.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:34:47.108+0000 7fc6b3400700 1 --2- 172.21.15.107:0/745690901 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fc698069210 0x7fc69806b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:34:47.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:34:47.109+0000 7fc6b3400700 1 --2- 172.21.15.107:0/745690901 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fc698069210 0x7fc69806b6c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc6a400b3b0 tx=0x7fc6a4009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:34:47.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:34:47.114+0000 7fc6ab400700 1 -- 172.21.15.107:0/745690901 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fc6a0056c40 con 0x7fc6b4100360 2024-03-04T13:34:47.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:34:47.337+0000 7fc6ba800700 1 -- 172.21.15.107:0/745690901 --> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fc6b4111c30 con 0x7fc698069210 2024-03-04T13:34:47.582 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:47 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T13:34:47.582 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:47 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T13:34:47.582 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:47 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:34:47.853 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:47 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T13:34:47.854 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:47 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T13:34:47.854 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:47 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:34:48.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:48 smithi107 ceph-mon[28739]: pgmap v73: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-04T13:34:48.838 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:48 smithi107 ceph-mon[28739]: from='client.14284 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:34:48.838 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:48 smithi107 ceph-mon[28739]: Detected new or changed devices on smithi107 2024-03-04T13:34:48.838 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:48 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:34:48.838 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:48 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:34:48.838 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:48 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-04T13:34:48.838 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:48 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-04T13:34:48.838 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:48 smithi107 ceph-mon[28739]: Adjusting osd_memory_target on smithi107 to 6433M 2024-03-04T13:34:48.838 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:48 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:34:48.838 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:48 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:34:48.838 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:48 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:34:48.838 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:48 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:34:48.838 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:48 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:34:49.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:48 smithi169 ceph-mon[36313]: pgmap v73: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-04T13:34:49.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:48 smithi169 ceph-mon[36313]: from='client.14284 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:34:49.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:48 smithi169 ceph-mon[36313]: Detected new or changed devices on smithi107 2024-03-04T13:34:49.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:48 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:34:49.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:48 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:34:49.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:48 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-04T13:34:49.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:48 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-04T13:34:49.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:48 smithi169 ceph-mon[36313]: Adjusting osd_memory_target on smithi107 to 6433M 2024-03-04T13:34:49.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:48 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:34:49.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:48 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:34:49.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:48 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:34:49.105 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:48 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:34:49.105 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:48 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:34:51.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:50 smithi107 ceph-mon[28739]: pgmap v74: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-04T13:34:51.103 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:50 smithi169 ceph-mon[36313]: pgmap v74: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-04T13:34:53.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:52 smithi107 ceph-mon[28739]: pgmap v75: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-04T13:34:53.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:52 smithi107 ceph-mon[28739]: from='client.? 172.21.15.107:0/3713376691' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "580a07f0-7bd4-4341-9c1f-b3383d64c720"}]: dispatch 2024-03-04T13:34:53.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:52 smithi107 ceph-mon[28739]: from='client.? 172.21.15.107:0/3713376691' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "580a07f0-7bd4-4341-9c1f-b3383d64c720"}]': finished 2024-03-04T13:34:53.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:52 smithi107 ceph-mon[28739]: osdmap e16: 3 total, 2 up, 3 in 2024-03-04T13:34:53.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:52 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-04T13:34:53.103 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:52 smithi169 ceph-mon[36313]: pgmap v75: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-04T13:34:53.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:52 smithi169 ceph-mon[36313]: from='client.? 172.21.15.107:0/3713376691' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "580a07f0-7bd4-4341-9c1f-b3383d64c720"}]: dispatch 2024-03-04T13:34:53.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:52 smithi169 ceph-mon[36313]: from='client.? 172.21.15.107:0/3713376691' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "580a07f0-7bd4-4341-9c1f-b3383d64c720"}]': finished 2024-03-04T13:34:53.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:52 smithi169 ceph-mon[36313]: osdmap e16: 3 total, 2 up, 3 in 2024-03-04T13:34:53.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:52 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-04T13:34:54.084 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:53 smithi107 ceph-mon[28739]: from='client.? 172.21.15.107:0/506777024' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T13:34:54.103 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:53 smithi169 ceph-mon[36313]: from='client.? 172.21.15.107:0/506777024' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T13:34:55.086 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:54 smithi107 ceph-mon[28739]: pgmap v77: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-04T13:34:55.103 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:54 smithi169 ceph-mon[36313]: pgmap v77: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-04T13:34:56.086 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:55 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:34:56.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:55 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:34:56.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:55 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:34:56.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:55 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:34:56.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:55 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:34:56.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:55 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:34:56.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:55 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:34:56.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:55 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:34:56.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:55 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:34:56.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:55 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:34:56.836 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:56 smithi107 ceph-mon[28739]: pgmap v78: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-04T13:34:57.103 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:56 smithi169 ceph-mon[36313]: pgmap v78: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-04T13:34:58.086 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:57 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:34:58.103 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:57 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:34:59.086 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:34:58 smithi107 ceph-mon[28739]: pgmap v79: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-04T13:34:59.103 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:34:58 smithi169 ceph-mon[36313]: pgmap v79: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-04T13:35:00.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:00 smithi107 ceph-mon[28739]: pgmap v80: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-04T13:35:01.103 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:00 smithi169 ceph-mon[36313]: pgmap v80: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-04T13:35:02.086 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:01 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-04T13:35:02.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:01 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:35:02.103 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:01 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-04T13:35:02.103 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:01 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:35:03.065 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:02 smithi107 ceph-mon[28739]: Deploying daemon osd.2 on smithi107 2024-03-04T13:35:03.065 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:02 smithi107 ceph-mon[28739]: pgmap v81: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-04T13:35:03.103 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:02 smithi169 ceph-mon[36313]: Deploying daemon osd.2 on smithi107 2024-03-04T13:35:03.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:02 smithi169 ceph-mon[36313]: pgmap v81: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-04T13:35:05.049 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:04 smithi107 ceph-mon[28739]: pgmap v82: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-04T13:35:05.103 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:04 smithi169 ceph-mon[36313]: pgmap v82: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-04T13:35:07.086 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:06 smithi107 ceph-mon[28739]: pgmap v83: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-04T13:35:07.103 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:06 smithi169 ceph-mon[36313]: pgmap v83: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-04T13:35:08.940 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:08 smithi107 ceph-mon[28739]: pgmap v84: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-04T13:35:09.353 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:08 smithi169 ceph-mon[36313]: pgmap v84: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-04T13:35:10.587 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:10 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:35:10.587 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:10 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:35:10.587 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:10 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:35:10.587 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:10 smithi107 ceph-mon[28739]: pgmap v85: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-04T13:35:10.603 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:10 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:35:10.603 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:10 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:35:10.603 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:10 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:35:10.604 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:10 smithi169 ceph-mon[36313]: pgmap v85: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-04T13:35:12.836 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:12 smithi107 ceph-mon[28739]: pgmap v86: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-04T13:35:12.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:12 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:35:12.853 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:12 smithi169 ceph-mon[36313]: pgmap v86: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-04T13:35:12.853 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:12 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:35:13.834 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:13 smithi107 ceph-mon[28739]: from='osd.2 [v2:172.21.15.107:6818/2331133561,v1:172.21.15.107:6819/2331133561]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-04T13:35:13.855 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:13 smithi169 ceph-mon[36313]: from='osd.2 [v2:172.21.15.107:6818/2331133561,v1:172.21.15.107:6819/2331133561]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-04T13:35:14.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:35:14.280+0000 7fc6ab400700 1 -- 172.21.15.107:0/745690901 <== mgr.14212 v2:172.21.15.107:6800/3848495603 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fc6b4111c30 con 0x7fc698069210 2024-03-04T13:35:14.275 INFO:teuthology.orchestra.run.smithi107.stdout:Created osd(s) 2 on host 'smithi107' 2024-03-04T13:35:14.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:35:14.284+0000 7fc6ba800700 1 -- 172.21.15.107:0/745690901 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fc698069210 msgr2=0x7fc69806b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:35:14.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:35:14.284+0000 7fc6ba800700 1 --2- 172.21.15.107:0/745690901 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fc698069210 0x7fc69806b6c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc6a400b3b0 tx=0x7fc6a4009e30 comp rx=0 tx=0).stop 2024-03-04T13:35:14.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:35:14.284+0000 7fc6ba800700 1 -- 172.21.15.107:0/745690901 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6b4100360 msgr2=0x7fc6b41100d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:35:14.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:35:14.284+0000 7fc6ba800700 1 --2- 172.21.15.107:0/745690901 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6b4100360 0x7fc6b41100d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc6a0002a50 tx=0x7fc6a00046b0 comp rx=0 tx=0).stop 2024-03-04T13:35:14.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:35:14.284+0000 7fc6ba800700 1 -- 172.21.15.107:0/745690901 shutdown_connections 2024-03-04T13:35:14.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:35:14.284+0000 7fc6ba800700 1 --2- 172.21.15.107:0/745690901 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fc698069210 0x7fc69806b6c0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:35:14.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:35:14.284+0000 7fc6ba800700 1 --2- 172.21.15.107:0/745690901 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc6b40ff150 0x7fc6b410fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:35:14.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:35:14.284+0000 7fc6ba800700 1 --2- 172.21.15.107:0/745690901 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6b4100360 0x7fc6b41100d0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:35:14.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:35:14.284+0000 7fc6ba800700 1 -- 172.21.15.107:0/745690901 >> 172.21.15.107:0/745690901 conn(0x7fc6b40fa920 msgr2=0x7fc6b4103590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:35:14.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:35:14.284+0000 7fc6ba800700 1 -- 172.21.15.107:0/745690901 shutdown_connections 2024-03-04T13:35:14.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:35:14.284+0000 7fc6ba800700 1 -- 172.21.15.107:0/745690901 wait complete. 2024-03-04T13:35:14.836 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:14 smithi107 ceph-mon[28739]: pgmap v87: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-04T13:35:14.836 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:14 smithi107 ceph-mon[28739]: from='osd.2 [v2:172.21.15.107:6818/2331133561,v1:172.21.15.107:6819/2331133561]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-04T13:35:14.836 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:14 smithi107 ceph-mon[28739]: osdmap e17: 3 total, 2 up, 3 in 2024-03-04T13:35:14.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:14 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-04T13:35:14.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:14 smithi107 ceph-mon[28739]: from='osd.2 [v2:172.21.15.107:6818/2331133561,v1:172.21.15.107:6819/2331133561]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-03-04T13:35:14.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:14 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:35:14.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:14 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:35:14.854 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:14 smithi169 ceph-mon[36313]: pgmap v87: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-04T13:35:14.854 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:14 smithi169 ceph-mon[36313]: from='osd.2 [v2:172.21.15.107:6818/2331133561,v1:172.21.15.107:6819/2331133561]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-04T13:35:14.854 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:14 smithi169 ceph-mon[36313]: osdmap e17: 3 total, 2 up, 3 in 2024-03-04T13:35:14.854 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:14 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-04T13:35:14.854 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:14 smithi169 ceph-mon[36313]: from='osd.2 [v2:172.21.15.107:6818/2331133561,v1:172.21.15.107:6819/2331133561]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-03-04T13:35:14.854 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:14 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:35:14.854 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:14 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:35:15.635 DEBUG:teuthology.orchestra.run.smithi107:osd.2> sudo journalctl -f -n 0 -u ceph-1fc09a58-da2b-11ee-95c6-87774f69a715@osd.2.service 2024-03-04T13:35:15.639 INFO:tasks.cephadm:Deploying osd.3 on smithi169 with /dev/vg_nvme/lv_4... 2024-03-04T13:35:15.640 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-04T13:35:15.838 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:15 smithi107 ceph-mon[28739]: from='osd.2 [v2:172.21.15.107:6818/2331133561,v1:172.21.15.107:6819/2331133561]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-03-04T13:35:15.838 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:15 smithi107 ceph-mon[28739]: osdmap e18: 3 total, 2 up, 3 in 2024-03-04T13:35:15.838 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:15 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-04T13:35:15.838 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:15 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-04T13:35:15.853 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:15 smithi169 ceph-mon[36313]: from='osd.2 [v2:172.21.15.107:6818/2331133561,v1:172.21.15.107:6819/2331133561]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-03-04T13:35:15.854 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:15 smithi169 ceph-mon[36313]: osdmap e18: 3 total, 2 up, 3 in 2024-03-04T13:35:15.854 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:15 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-04T13:35:15.854 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:15 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-04T13:35:16.107 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/mon.smithi169/config 2024-03-04T13:35:16.836 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:16 smithi107 ceph-mon[28739]: purged_snaps scrub starts 2024-03-04T13:35:16.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:16 smithi107 ceph-mon[28739]: purged_snaps scrub ok 2024-03-04T13:35:16.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:16 smithi107 ceph-mon[28739]: pgmap v90: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-04T13:35:16.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:16 smithi107 ceph-mon[28739]: osd.2 [v2:172.21.15.107:6818/2331133561,v1:172.21.15.107:6819/2331133561] boot 2024-03-04T13:35:16.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:16 smithi107 ceph-mon[28739]: osdmap e19: 3 total, 3 up, 3 in 2024-03-04T13:35:16.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:16 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-04T13:35:16.853 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:16 smithi169 ceph-mon[36313]: purged_snaps scrub starts 2024-03-04T13:35:16.853 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:16 smithi169 ceph-mon[36313]: purged_snaps scrub ok 2024-03-04T13:35:16.854 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:16 smithi169 ceph-mon[36313]: pgmap v90: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-04T13:35:16.854 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:16 smithi169 ceph-mon[36313]: osd.2 [v2:172.21.15.107:6818/2331133561,v1:172.21.15.107:6819/2331133561] boot 2024-03-04T13:35:16.854 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:16 smithi169 ceph-mon[36313]: osdmap e19: 3 total, 3 up, 3 in 2024-03-04T13:35:16.854 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:16 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-04T13:35:18.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:17 smithi169 ceph-mon[36313]: osdmap e20: 3 total, 3 up, 3 in 2024-03-04T13:35:18.336 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:17 smithi107 ceph-mon[28739]: osdmap e20: 3 total, 3 up, 3 in 2024-03-04T13:35:18.657 INFO:teuthology.orchestra.run.smithi169.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-04T13:35:18.657 INFO:teuthology.orchestra.run.smithi169.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-04T13:35:18.657 INFO:teuthology.orchestra.run.smithi169.stderr: stderr: 10+0 records in 2024-03-04T13:35:18.657 INFO:teuthology.orchestra.run.smithi169.stderr:10+0 records out 2024-03-04T13:35:18.658 INFO:teuthology.orchestra.run.smithi169.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0334543 s, 313 MB/s 2024-03-04T13:35:18.658 INFO:teuthology.orchestra.run.smithi169.stderr:--> Zapping successful for: 2024-03-04T13:35:19.261 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:18 smithi107 ceph-mon[28739]: pgmap v93: 0 pgs: ; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-03-04T13:35:19.261 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:18 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-03-04T13:35:19.261 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:18 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:35:19.261 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:18 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:35:19.261 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:18 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-03-04T13:35:19.262 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:18 smithi107 ceph-mon[28739]: osdmap e21: 3 total, 3 up, 3 in 2024-03-04T13:35:19.262 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:18 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-04T13:35:19.354 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:18 smithi169 ceph-mon[36313]: pgmap v93: 0 pgs: ; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-03-04T13:35:19.354 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:18 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-03-04T13:35:19.354 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:18 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:35:19.354 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:18 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:35:19.354 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:18 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-03-04T13:35:19.354 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:18 smithi169 ceph-mon[36313]: osdmap e21: 3 total, 3 up, 3 in 2024-03-04T13:35:19.355 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:18 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-04T13:35:19.439 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 -- ceph orch daemon add osd smithi169:vg_nvme/lv_4 2024-03-04T13:35:19.894 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/mon.smithi169/config 2024-03-04T13:35:20.224 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:19 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-04T13:35:20.224 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:19 smithi107 ceph-mon[28739]: osdmap e22: 3 total, 3 up, 3 in 2024-03-04T13:35:20.225 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 04 13:35:19 smithi107 sudo[64089]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-04T13:35:20.225 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 04 13:35:19 smithi107 sudo[64089]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-04T13:35:20.225 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 04 13:35:19 smithi107 sudo[64089]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-04T13:35:20.225 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 04 13:35:19 smithi107 sudo[64089]: pam_unix(sudo:session): session closed for user root 2024-03-04T13:35:20.226 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:19 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-04T13:35:20.226 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:19 smithi169 ceph-mon[36313]: osdmap e22: 3 total, 3 up, 3 in 2024-03-04T13:35:20.817 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 04 13:35:20 smithi107 sudo[64109]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-04T13:35:20.817 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 04 13:35:20 smithi107 sudo[64109]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-04T13:35:20.817 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 04 13:35:20 smithi107 sudo[64109]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-04T13:35:20.817 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 04 13:35:20 smithi107 sudo[64109]: pam_unix(sudo:session): session closed for user root 2024-03-04T13:35:21.058 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:20 smithi169 ceph-mon[36313]: pgmap v96: 1 pgs: 1 unknown; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-03-04T13:35:21.058 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:20 smithi169 ceph-mon[36313]: osdmap e23: 3 total, 3 up, 3 in 2024-03-04T13:35:21.058 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:20 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:35:21.072 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:20 smithi107 ceph-mon[28739]: pgmap v96: 1 pgs: 1 unknown; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-03-04T13:35:21.072 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:20 smithi107 ceph-mon[28739]: osdmap e23: 3 total, 3 up, 3 in 2024-03-04T13:35:21.072 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:20 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:35:21.325 INFO:journalctl@ceph.osd.1.smithi107.stdout:Mar 04 13:35:21 smithi107 sudo[64115]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-04T13:35:21.325 INFO:journalctl@ceph.osd.1.smithi107.stdout:Mar 04 13:35:21 smithi107 sudo[64115]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-04T13:35:21.325 INFO:journalctl@ceph.osd.1.smithi107.stdout:Mar 04 13:35:21 smithi107 sudo[64115]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-04T13:35:21.325 INFO:journalctl@ceph.osd.1.smithi107.stdout:Mar 04 13:35:21 smithi107 sudo[64115]: pam_unix(sudo:session): session closed for user root 2024-03-04T13:35:21.793 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:35:21.799+0000 7f131e600700 1 -- 172.21.15.169:0/56438193 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1318103540 msgr2=0x7f1318103950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:35:21.793 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:35:21.799+0000 7f131e600700 1 --2- 172.21.15.169:0/56438193 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1318103540 0x7f1318103950 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f1308009ce0 tx=0x7f1308004d10 comp rx=0 tx=0).stop 2024-03-04T13:35:21.793 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:35:21.799+0000 7f131e600700 1 -- 172.21.15.169:0/56438193 shutdown_connections 2024-03-04T13:35:21.793 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:35:21.799+0000 7f131e600700 1 --2- 172.21.15.169:0/56438193 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1318104750 0x7f1318104ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:35:21.793 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:35:21.799+0000 7f131e600700 1 --2- 172.21.15.169:0/56438193 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1318103540 0x7f1318103950 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:35:21.793 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:35:21.799+0000 7f131e600700 1 -- 172.21.15.169:0/56438193 >> 172.21.15.169:0/56438193 conn(0x7f13180727d0 msgr2=0x7f1318074c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:35:21.793 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:35:21.800+0000 7f131e600700 1 -- 172.21.15.169:0/56438193 shutdown_connections 2024-03-04T13:35:21.793 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:35:21.800+0000 7f131e600700 1 -- 172.21.15.169:0/56438193 wait complete. 2024-03-04T13:35:21.795 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:35:21.800+0000 7f131e600700 1 Processor -- start 2024-03-04T13:35:21.796 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:35:21.800+0000 7f131e600700 1 -- start start 2024-03-04T13:35:21.796 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:35:21.801+0000 7f131e600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1318103540 0x7f1318114160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:35:21.796 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:35:21.801+0000 7f131e600700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1318104750 0x7f13181146a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:35:21.796 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:35:21.801+0000 7f131e600700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1318114ca0 con 0x7f1318103540 2024-03-04T13:35:21.796 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:35:21.801+0000 7f131e600700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1318115540 con 0x7f1318104750 2024-03-04T13:35:21.796 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:35:21.801+0000 7f130ea00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1318104750 0x7f13181146a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:35:21.797 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:35:21.801+0000 7f130ea00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1318104750 0x7f13181146a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:43018/0 (socket says 172.21.15.169:43018) 2024-03-04T13:35:21.797 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:35:21.801+0000 7f130ea00700 1 -- 172.21.15.169:0/441141614 learned_addr learned my addr 172.21.15.169:0/441141614 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-03-04T13:35:21.797 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:35:21.801+0000 7f1317400700 1 --2- 172.21.15.169:0/441141614 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1318103540 0x7f1318114160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:35:21.797 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:35:21.802+0000 7f130ea00700 1 -- 172.21.15.169:0/441141614 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1318103540 msgr2=0x7f1318114160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:35:21.797 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:35:21.802+0000 7f130ea00700 1 --2- 172.21.15.169:0/441141614 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1318103540 0x7f1318114160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:35:21.797 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:35:21.802+0000 7f130ea00700 1 -- 172.21.15.169:0/441141614 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1308009940 con 0x7f1318104750 2024-03-04T13:35:21.798 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:35:21.802+0000 7f130ea00700 1 --2- 172.21.15.169:0/441141614 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1318104750 0x7f13181146a0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f1300002a50 tx=0x7f13000046b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:35:21.798 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:35:21.802+0000 7f130fe00700 1 -- 172.21.15.169:0/441141614 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1300007cc0 con 0x7f1318104750 2024-03-04T13:35:21.798 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:35:21.802+0000 7f130fe00700 1 -- 172.21.15.169:0/441141614 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f130000f040 con 0x7f1318104750 2024-03-04T13:35:21.798 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:35:21.802+0000 7f131e600700 1 -- 172.21.15.169:0/441141614 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1318115820 con 0x7f1318104750 2024-03-04T13:35:21.798 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:35:21.802+0000 7f130fe00700 1 -- 172.21.15.169:0/441141614 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1300009640 con 0x7f1318104750 2024-03-04T13:35:21.798 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:35:21.802+0000 7f131e600700 1 -- 172.21.15.169:0/441141614 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1318115d60 con 0x7f1318104750 2024-03-04T13:35:21.799 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:35:21.804+0000 7f131e600700 1 -- 172.21.15.169:0/441141614 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f131804bed0 con 0x7f1318104750 2024-03-04T13:35:21.799 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:35:21.804+0000 7f130fe00700 1 -- 172.21.15.169:0/441141614 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 18) v1 ==== 86567+0+0 (secure 0 0 0) 0x7f13000097a0 con 0x7f1318104750 2024-03-04T13:35:21.799 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:35:21.805+0000 7f130fe00700 1 --2- 172.21.15.169:0/441141614 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f12fc069210 0x7f12fc06b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:35:21.799 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:35:21.805+0000 7f130fe00700 1 -- 172.21.15.169:0/441141614 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3228+0+0 (secure 0 0 0) 0x7f1300086690 con 0x7f1318104750 2024-03-04T13:35:21.800 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:35:21.806+0000 7f1317400700 1 --2- 172.21.15.169:0/441141614 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f12fc069210 0x7f12fc06b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:35:21.800 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:35:21.807+0000 7f1317400700 1 --2- 172.21.15.169:0/441141614 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f12fc069210 0x7f12fc06b6c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1308009470 tx=0x7f1308004d10 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:35:21.804 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:35:21.811+0000 7f130fe00700 1 -- 172.21.15.169:0/441141614 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f1300056cd0 con 0x7f1318104750 2024-03-04T13:35:21.986 INFO:journalctl@ceph.osd.1.smithi107.stdout:Mar 04 13:35:21 smithi107 sudo[64145]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-04T13:35:21.987 INFO:journalctl@ceph.osd.1.smithi107.stdout:Mar 04 13:35:21 smithi107 sudo[64145]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-04T13:35:21.987 INFO:journalctl@ceph.osd.1.smithi107.stdout:Mar 04 13:35:21 smithi107 sudo[64145]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-04T13:35:21.987 INFO:journalctl@ceph.osd.1.smithi107.stdout:Mar 04 13:35:21 smithi107 sudo[64145]: pam_unix(sudo:session): session closed for user root 2024-03-04T13:35:21.995 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:35:22.002+0000 7f130fe00700 1 -- 172.21.15.169:0/441141614 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mgrmap(e 19) v1 ==== 86623+0+0 (secure 0 0 0) 0x7f1300053610 con 0x7f1318104750 2024-03-04T13:35:22.020 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:35:22.026+0000 7f131e600700 1 -- 172.21.15.169:0/441141614 --> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f1318116170 con 0x7f12fc069210 2024-03-04T13:35:22.241 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:21 smithi107 ceph-mon[28739]: pgmap v98: 1 pgs: 1 active+clean; 449 KiB data, 871 MiB used, 267 GiB / 268 GiB avail 2024-03-04T13:35:22.241 INFO:journalctl@ceph.osd.2.smithi107.stdout:Mar 04 13:35:22 smithi107 sudo[64163]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-04T13:35:22.241 INFO:journalctl@ceph.osd.2.smithi107.stdout:Mar 04 13:35:22 smithi107 sudo[64163]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-04T13:35:22.241 INFO:journalctl@ceph.osd.2.smithi107.stdout:Mar 04 13:35:22 smithi107 sudo[64163]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-04T13:35:22.241 INFO:journalctl@ceph.osd.2.smithi107.stdout:Mar 04 13:35:22 smithi107 sudo[64163]: pam_unix(sudo:session): session closed for user root 2024-03-04T13:35:22.353 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:21 smithi169 ceph-mon[36313]: pgmap v98: 1 pgs: 1 active+clean; 449 KiB data, 871 MiB used, 267 GiB / 268 GiB avail 2024-03-04T13:35:23.086 INFO:journalctl@ceph.osd.2.smithi107.stdout:Mar 04 13:35:22 smithi107 sudo[64920]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-04T13:35:23.086 INFO:journalctl@ceph.osd.2.smithi107.stdout:Mar 04 13:35:22 smithi107 sudo[64920]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-04T13:35:23.087 INFO:journalctl@ceph.osd.2.smithi107.stdout:Mar 04 13:35:22 smithi107 sudo[64920]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-04T13:35:23.087 INFO:journalctl@ceph.osd.2.smithi107.stdout:Mar 04 13:35:22 smithi107 sudo[64920]: pam_unix(sudo:session): session closed for user root 2024-03-04T13:35:23.352 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:23 smithi107 sudo[66845]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-04T13:35:23.836 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:23 smithi107 sudo[66845]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-04T13:35:23.836 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:23 smithi107 sudo[66845]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-04T13:35:23.836 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:23 smithi107 sudo[66845]: pam_unix(sudo:session): session closed for user root 2024-03-04T13:35:23.974 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:23 smithi169 ceph-mon[36313]: mgrmap e19: smithi107.wpzgwl(active, since 2m), standbys: smithi169.eihczr 2024-03-04T13:35:23.974 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:23 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T13:35:23.974 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:23 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T13:35:23.974 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:23 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:35:23.974 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:23 smithi169 ceph-mon[36313]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-04T13:35:24.337 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:23 smithi107 sudo[66852]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-04T13:35:24.337 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:23 smithi107 sudo[66852]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-04T13:35:24.337 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:23 smithi107 sudo[66852]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-04T13:35:24.337 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:23 smithi107 sudo[66852]: pam_unix(sudo:session): session closed for user root 2024-03-04T13:35:24.337 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:23 smithi107 ceph-mon[28739]: mgrmap e19: smithi107.wpzgwl(active, since 2m), standbys: smithi169.eihczr 2024-03-04T13:35:24.337 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:23 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T13:35:24.337 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:23 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T13:35:24.337 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:23 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:35:24.337 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:23 smithi107 ceph-mon[28739]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-04T13:35:24.603 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:24 smithi169 sudo[39685]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-04T13:35:24.603 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:24 smithi169 sudo[39685]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-04T13:35:24.604 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:24 smithi169 sudo[39685]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-04T13:35:24.604 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:24 smithi169 sudo[39685]: pam_unix(sudo:session): session closed for user root 2024-03-04T13:35:25.337 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:25 smithi107 ceph-mon[28739]: from='client.24135 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:35:25.337 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:25 smithi107 ceph-mon[28739]: pgmap v99: 1 pgs: 1 active+clean; 449 KiB data, 873 MiB used, 267 GiB / 268 GiB avail 2024-03-04T13:35:25.337 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:25 smithi107 ceph-mon[28739]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-04T13:35:25.337 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:25 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mon metadata", "id": "smithi107"}]: dispatch 2024-03-04T13:35:25.337 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:25 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mon metadata", "id": "smithi169"}]: dispatch 2024-03-04T13:35:25.337 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:25 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mon metadata", "id": "smithi107"}]: dispatch 2024-03-04T13:35:25.337 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:25 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mon metadata", "id": "smithi169"}]: dispatch 2024-03-04T13:35:25.354 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:25 smithi169 sudo[39707]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-04T13:35:25.354 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:25 smithi169 sudo[39707]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-04T13:35:25.354 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:25 smithi169 sudo[39707]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-04T13:35:25.354 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:25 smithi169 sudo[39707]: pam_unix(sudo:session): session closed for user root 2024-03-04T13:35:25.354 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:25 smithi169 ceph-mon[36313]: from='client.24135 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:35:25.354 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:25 smithi169 ceph-mon[36313]: pgmap v99: 1 pgs: 1 active+clean; 449 KiB data, 873 MiB used, 267 GiB / 268 GiB avail 2024-03-04T13:35:25.354 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:25 smithi169 ceph-mon[36313]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-04T13:35:25.354 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:25 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mon metadata", "id": "smithi107"}]: dispatch 2024-03-04T13:35:25.354 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:25 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mon metadata", "id": "smithi169"}]: dispatch 2024-03-04T13:35:25.354 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:25 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mon metadata", "id": "smithi107"}]: dispatch 2024-03-04T13:35:25.355 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:25 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mon metadata", "id": "smithi169"}]: dispatch 2024-03-04T13:35:26.337 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:26 smithi107 ceph-mon[28739]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-04T13:35:26.337 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:26 smithi107 ceph-mon[28739]: Detected new or changed devices on smithi107 2024-03-04T13:35:26.337 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:26 smithi107 ceph-mon[28739]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-04T13:35:26.337 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:26 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:35:26.337 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:26 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:35:26.338 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:26 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-04T13:35:26.338 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:26 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-04T13:35:26.338 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:26 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-04T13:35:26.338 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:26 smithi107 ceph-mon[28739]: Adjusting osd_memory_target on smithi107 to 4288M 2024-03-04T13:35:26.338 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:26 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:35:26.338 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:26 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:35:26.338 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:26 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:35:26.338 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:26 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:35:26.338 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:26 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:35:26.338 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:26 smithi107 ceph-mon[28739]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.7 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-03-04T13:35:26.338 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:26 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:35:26.338 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:26 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:35:26.338 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:26 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:35:26.338 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:26 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:35:26.338 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:26 smithi107 ceph-mon[28739]: from='client.? 172.21.15.169:0/909751572' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7acac14d-998a-4651-b78c-e42584e13b43"}]: dispatch 2024-03-04T13:35:26.339 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:26 smithi107 ceph-mon[28739]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7acac14d-998a-4651-b78c-e42584e13b43"}]: dispatch 2024-03-04T13:35:26.339 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:26 smithi107 ceph-mon[28739]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7acac14d-998a-4651-b78c-e42584e13b43"}]': finished 2024-03-04T13:35:26.339 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:26 smithi107 ceph-mon[28739]: osdmap e24: 4 total, 3 up, 4 in 2024-03-04T13:35:26.339 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:26 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-04T13:35:26.353 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:26 smithi169 ceph-mon[36313]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-04T13:35:26.354 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:26 smithi169 ceph-mon[36313]: Detected new or changed devices on smithi107 2024-03-04T13:35:26.354 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:26 smithi169 ceph-mon[36313]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-04T13:35:26.354 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:26 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:35:26.354 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:26 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:35:26.354 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:26 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-04T13:35:26.354 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:26 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-04T13:35:26.354 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:26 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-04T13:35:26.354 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:26 smithi169 ceph-mon[36313]: Adjusting osd_memory_target on smithi107 to 4288M 2024-03-04T13:35:26.354 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:26 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:35:26.354 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:26 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:35:26.354 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:26 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:35:26.354 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:26 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:35:26.354 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:26 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:35:26.354 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:26 smithi169 ceph-mon[36313]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.7 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-03-04T13:35:26.355 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:26 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:35:26.355 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:26 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:35:26.355 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:26 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:35:26.355 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:26 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:35:26.355 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:26 smithi169 ceph-mon[36313]: from='client.? 172.21.15.169:0/909751572' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7acac14d-998a-4651-b78c-e42584e13b43"}]: dispatch 2024-03-04T13:35:26.355 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:26 smithi169 ceph-mon[36313]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7acac14d-998a-4651-b78c-e42584e13b43"}]: dispatch 2024-03-04T13:35:26.355 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:26 smithi169 ceph-mon[36313]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7acac14d-998a-4651-b78c-e42584e13b43"}]': finished 2024-03-04T13:35:26.355 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:26 smithi169 ceph-mon[36313]: osdmap e24: 4 total, 3 up, 4 in 2024-03-04T13:35:26.355 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:26 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-04T13:35:27.336 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:27 smithi107 ceph-mon[28739]: from='client.? 172.21.15.169:0/2978408695' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T13:35:27.353 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:27 smithi169 ceph-mon[36313]: from='client.? 172.21.15.169:0/2978408695' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T13:35:28.353 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:28 smithi169 ceph-mon[36313]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.2 KiB/s rd, 71 KiB/s wr, 4 op/s 2024-03-04T13:35:28.354 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:28 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:35:28.586 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:28 smithi107 ceph-mon[28739]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.2 KiB/s rd, 71 KiB/s wr, 4 op/s 2024-03-04T13:35:28.586 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:28 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:35:30.586 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:30 smithi107 ceph-mon[28739]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 990 B/s rd, 56 KiB/s wr, 3 op/s 2024-03-04T13:35:30.604 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:30 smithi169 ceph-mon[36313]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 990 B/s rd, 56 KiB/s wr, 3 op/s 2024-03-04T13:35:32.586 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:32 smithi107 ceph-mon[28739]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.7 KiB/s rd, 131 KiB/s wr, 8 op/s 2024-03-04T13:35:32.603 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:32 smithi169 ceph-mon[36313]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.7 KiB/s rd, 131 KiB/s wr, 8 op/s 2024-03-04T13:35:34.836 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:34 smithi107 ceph-mon[28739]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.7 KiB/s rd, 131 KiB/s wr, 8 op/s 2024-03-04T13:35:34.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:34 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-04T13:35:34.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:34 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:35:34.853 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:34 smithi169 ceph-mon[36313]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.7 KiB/s rd, 131 KiB/s wr, 8 op/s 2024-03-04T13:35:34.853 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:34 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-04T13:35:34.853 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:34 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:35:35.603 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:35 smithi169 ceph-mon[36313]: Deploying daemon osd.3 on smithi169 2024-03-04T13:35:35.839 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:35 smithi107 ceph-mon[28739]: Deploying daemon osd.3 on smithi169 2024-03-04T13:35:36.794 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:36 smithi169 ceph-mon[36313]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-03-04T13:35:36.836 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:36 smithi107 ceph-mon[28739]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-03-04T13:35:38.711 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:38 smithi169 ceph-mon[36313]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.5 KiB/s rd, 66 KiB/s wr, 4 op/s 2024-03-04T13:35:38.836 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:38 smithi107 ceph-mon[28739]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.5 KiB/s rd, 66 KiB/s wr, 4 op/s 2024-03-04T13:35:40.813 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:40 smithi169 ceph-mon[36313]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-03-04T13:35:40.836 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:40 smithi107 ceph-mon[28739]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-03-04T13:35:42.753 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:42 smithi169 ceph-mon[36313]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-03-04T13:35:42.753 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:42 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:35:42.753 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:42 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:35:42.753 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:42 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:35:42.753 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:42 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:35:43.086 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:42 smithi107 ceph-mon[28739]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-03-04T13:35:43.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:42 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:35:43.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:42 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:35:43.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:42 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:35:43.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:42 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:35:45.064 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:44 smithi169 ceph-mon[36313]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-03-04T13:35:45.086 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:44 smithi107 ceph-mon[28739]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-03-04T13:35:46.491 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:46 smithi169 ceph-mon[36313]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-03-04T13:35:46.491 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:46 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:35:46.491 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:46 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:35:46.491 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:46 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:35:46.491 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:46 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:35:46.491 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:46 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:35:46.491 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:46 smithi169 ceph-mon[36313]: from='osd.3 [v2:172.21.15.169:6800/2475630313,v1:172.21.15.169:6801/2475630313]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-04T13:35:46.491 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:46 smithi169 ceph-mon[36313]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-04T13:35:46.668 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:35:46.673+0000 7f130fe00700 1 -- 172.21.15.169:0/441141614 <== mgr.14212 v2:172.21.15.107:6800/3848495603 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f1318116170 con 0x7f12fc069210 2024-03-04T13:35:46.669 INFO:teuthology.orchestra.run.smithi169.stdout:Created osd(s) 3 on host 'smithi169' 2024-03-04T13:35:46.671 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:35:46.677+0000 7f131e600700 1 -- 172.21.15.169:0/441141614 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f12fc069210 msgr2=0x7f12fc06b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:35:46.671 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:35:46.677+0000 7f131e600700 1 --2- 172.21.15.169:0/441141614 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f12fc069210 0x7f12fc06b6c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1308009470 tx=0x7f1308004d10 comp rx=0 tx=0).stop 2024-03-04T13:35:46.671 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:35:46.677+0000 7f131e600700 1 -- 172.21.15.169:0/441141614 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1318104750 msgr2=0x7f13181146a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:35:46.672 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:35:46.677+0000 7f131e600700 1 --2- 172.21.15.169:0/441141614 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1318104750 0x7f13181146a0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f1300002a50 tx=0x7f13000046b0 comp rx=0 tx=0).stop 2024-03-04T13:35:46.672 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:35:46.677+0000 7f131e600700 1 -- 172.21.15.169:0/441141614 shutdown_connections 2024-03-04T13:35:46.672 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:35:46.677+0000 7f131e600700 1 --2- 172.21.15.169:0/441141614 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f12fc069210 0x7f12fc06b6c0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:35:46.672 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:35:46.677+0000 7f131e600700 1 --2- 172.21.15.169:0/441141614 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1318103540 0x7f1318114160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:35:46.672 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:35:46.677+0000 7f131e600700 1 --2- 172.21.15.169:0/441141614 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1318104750 0x7f13181146a0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:35:46.672 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:35:46.677+0000 7f131e600700 1 -- 172.21.15.169:0/441141614 >> 172.21.15.169:0/441141614 conn(0x7f13180727d0 msgr2=0x7f1318107980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:35:46.672 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:35:46.677+0000 7f131e600700 1 -- 172.21.15.169:0/441141614 shutdown_connections 2024-03-04T13:35:46.672 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:35:46.678+0000 7f131e600700 1 -- 172.21.15.169:0/441141614 wait complete. 2024-03-04T13:35:46.836 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:46 smithi107 ceph-mon[28739]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-03-04T13:35:46.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:46 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:35:46.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:46 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:35:46.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:46 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:35:46.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:46 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:35:46.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:46 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:35:46.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:46 smithi107 ceph-mon[28739]: from='osd.3 [v2:172.21.15.169:6800/2475630313,v1:172.21.15.169:6801/2475630313]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-04T13:35:46.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:46 smithi107 ceph-mon[28739]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-04T13:35:47.414 DEBUG:teuthology.orchestra.run.smithi169:osd.3> sudo journalctl -f -n 0 -u ceph-1fc09a58-da2b-11ee-95c6-87774f69a715@osd.3.service 2024-03-04T13:35:47.420 INFO:tasks.cephadm:Deploying osd.4 on smithi169 with /dev/vg_nvme/lv_3... 2024-03-04T13:35:47.420 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-04T13:35:47.626 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:47 smithi169 ceph-mon[36313]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-04T13:35:47.627 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:47 smithi169 ceph-mon[36313]: osdmap e25: 4 total, 3 up, 4 in 2024-03-04T13:35:47.627 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:47 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-04T13:35:47.627 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:47 smithi169 ceph-mon[36313]: from='osd.3 [v2:172.21.15.169:6800/2475630313,v1:172.21.15.169:6801/2475630313]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-03-04T13:35:47.627 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:47 smithi169 ceph-mon[36313]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-03-04T13:35:47.627 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:47 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:35:47.627 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:47 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:35:47.627 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:47 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:35:47.627 INFO:journalctl@ceph.osd.3.smithi169.stdout:Mar 04 13:35:47 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-3[41844]: 2024-03-04T13:35:47.545+0000 7f125de00700 -1 osd.3 0 waiting for initial osdmap 2024-03-04T13:35:47.627 INFO:journalctl@ceph.osd.3.smithi169.stdout:Mar 04 13:35:47 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-3[41844]: 2024-03-04T13:35:47.554+0000 7f1258e00700 -1 osd.3 26 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-04T13:35:47.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:47 smithi107 ceph-mon[28739]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-04T13:35:47.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:47 smithi107 ceph-mon[28739]: osdmap e25: 4 total, 3 up, 4 in 2024-03-04T13:35:47.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:47 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-04T13:35:47.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:47 smithi107 ceph-mon[28739]: from='osd.3 [v2:172.21.15.169:6800/2475630313,v1:172.21.15.169:6801/2475630313]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-03-04T13:35:47.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:47 smithi107 ceph-mon[28739]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-03-04T13:35:47.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:47 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:35:47.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:47 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:35:47.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:47 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:35:48.604 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:48 smithi169 ceph-mon[36313]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-03-04T13:35:48.604 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:48 smithi169 ceph-mon[36313]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-03-04T13:35:48.604 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:48 smithi169 ceph-mon[36313]: osdmap e26: 4 total, 3 up, 4 in 2024-03-04T13:35:48.604 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:48 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-04T13:35:48.604 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:48 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-04T13:35:48.666 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/mon.smithi169/config 2024-03-04T13:35:48.836 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:48 smithi107 ceph-mon[28739]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-03-04T13:35:48.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:48 smithi107 ceph-mon[28739]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-03-04T13:35:48.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:48 smithi107 ceph-mon[28739]: osdmap e26: 4 total, 3 up, 4 in 2024-03-04T13:35:48.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:48 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-04T13:35:48.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:48 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-04T13:35:49.735 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:49 smithi169 ceph-mon[36313]: purged_snaps scrub starts 2024-03-04T13:35:49.735 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:49 smithi169 ceph-mon[36313]: purged_snaps scrub ok 2024-03-04T13:35:49.735 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:49 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-04T13:35:49.735 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:49 smithi169 ceph-mon[36313]: osd.3 [v2:172.21.15.169:6800/2475630313,v1:172.21.15.169:6801/2475630313] boot 2024-03-04T13:35:49.735 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:49 smithi169 ceph-mon[36313]: osdmap e27: 4 total, 4 up, 4 in 2024-03-04T13:35:49.735 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:49 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-04T13:35:49.836 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:49 smithi107 ceph-mon[28739]: purged_snaps scrub starts 2024-03-04T13:35:49.836 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:49 smithi107 ceph-mon[28739]: purged_snaps scrub ok 2024-03-04T13:35:49.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:49 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-04T13:35:49.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:49 smithi107 ceph-mon[28739]: osd.3 [v2:172.21.15.169:6800/2475630313,v1:172.21.15.169:6801/2475630313] boot 2024-03-04T13:35:49.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:49 smithi107 ceph-mon[28739]: osdmap e27: 4 total, 4 up, 4 in 2024-03-04T13:35:49.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:49 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-04T13:35:50.775 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:50 smithi169 ceph-mon[36313]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-03-04T13:35:50.775 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:50 smithi169 ceph-mon[36313]: osdmap e28: 4 total, 4 up, 4 in 2024-03-04T13:35:50.836 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:50 smithi107 ceph-mon[28739]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-03-04T13:35:50.836 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:50 smithi107 ceph-mon[28739]: osdmap e28: 4 total, 4 up, 4 in 2024-03-04T13:35:51.647 INFO:teuthology.orchestra.run.smithi169.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-04T13:35:51.647 INFO:teuthology.orchestra.run.smithi169.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-04T13:35:51.647 INFO:teuthology.orchestra.run.smithi169.stderr: stderr: 10+0 records in 2024-03-04T13:35:51.647 INFO:teuthology.orchestra.run.smithi169.stderr:10+0 records out 2024-03-04T13:35:51.647 INFO:teuthology.orchestra.run.smithi169.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0615395 s, 170 MB/s 2024-03-04T13:35:51.647 INFO:teuthology.orchestra.run.smithi169.stderr:--> Zapping successful for: 2024-03-04T13:35:51.836 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:51 smithi107 ceph-mon[28739]: osdmap e29: 4 total, 4 up, 4 in 2024-03-04T13:35:51.853 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:51 smithi169 ceph-mon[36313]: osdmap e29: 4 total, 4 up, 4 in 2024-03-04T13:35:52.359 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 -- ceph orch daemon add osd smithi169:vg_nvme/lv_3 2024-03-04T13:35:52.828 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/mon.smithi169/config 2024-03-04T13:35:52.840 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:52 smithi107 ceph-mon[28739]: pgmap v119: 1 pgs: 1 remapped+peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-04T13:35:52.840 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:52 smithi107 ceph-mon[28739]: Detected new or changed devices on smithi169 2024-03-04T13:35:52.840 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:52 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:35:52.840 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:52 smithi107 ceph-mon[28739]: osdmap e30: 4 total, 4 up, 4 in 2024-03-04T13:35:52.841 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:52 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:35:52.841 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:52 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-04T13:35:52.841 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:52 smithi107 ceph-mon[28739]: Adjusting osd_memory_target on smithi169 to 15939M 2024-03-04T13:35:52.841 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:52 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:35:52.841 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:52 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:35:52.841 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:52 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:35:52.841 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:52 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:35:52.853 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:52 smithi169 ceph-mon[36313]: pgmap v119: 1 pgs: 1 remapped+peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-04T13:35:52.854 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:52 smithi169 ceph-mon[36313]: Detected new or changed devices on smithi169 2024-03-04T13:35:52.854 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:52 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:35:52.854 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:52 smithi169 ceph-mon[36313]: osdmap e30: 4 total, 4 up, 4 in 2024-03-04T13:35:52.854 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:52 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:35:52.854 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:52 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-04T13:35:52.854 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:52 smithi169 ceph-mon[36313]: Adjusting osd_memory_target on smithi169 to 15939M 2024-03-04T13:35:52.854 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:52 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:35:52.854 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:52 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:35:52.855 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:52 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:35:52.855 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:52 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:35:53.836 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:53 smithi107 ceph-mon[28739]: osdmap e31: 4 total, 4 up, 4 in 2024-03-04T13:35:53.853 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:53 smithi169 ceph-mon[36313]: osdmap e31: 4 total, 4 up, 4 in 2024-03-04T13:35:54.749 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:35:54.755+0000 7fe8b0000700 1 -- 172.21.15.169:0/2025707216 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8a80708c0 msgr2=0x7fe8a8070d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:35:54.749 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:35:54.755+0000 7fe8b0000700 1 --2- 172.21.15.169:0/2025707216 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8a80708c0 0x7fe8a8070d10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe894004940 tx=0x7fe894009e30 comp rx=0 tx=0).stop 2024-03-04T13:35:54.749 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:35:54.755+0000 7fe8b0000700 1 -- 172.21.15.169:0/2025707216 shutdown_connections 2024-03-04T13:35:54.749 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:35:54.755+0000 7fe8b0000700 1 --2- 172.21.15.169:0/2025707216 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8a80708c0 0x7fe8a8070d10 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:35:54.749 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:35:54.755+0000 7fe8b0000700 1 --2- 172.21.15.169:0/2025707216 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8a8071f10 0x7fe8a8070380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:35:54.749 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:35:54.755+0000 7fe8b0000700 1 -- 172.21.15.169:0/2025707216 >> 172.21.15.169:0/2025707216 conn(0x7fe8a80f8620 msgr2=0x7fe8a80faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:35:54.749 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:35:54.755+0000 7fe8b0000700 1 -- 172.21.15.169:0/2025707216 shutdown_connections 2024-03-04T13:35:54.749 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:35:54.755+0000 7fe8b0000700 1 -- 172.21.15.169:0/2025707216 wait complete. 2024-03-04T13:35:54.750 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:35:54.757+0000 7fe8b0000700 1 Processor -- start 2024-03-04T13:35:54.750 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:35:54.757+0000 7fe8b0000700 1 -- start start 2024-03-04T13:35:54.752 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:35:54.758+0000 7fe8b0000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8a80708c0 0x7fe8a810fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:35:54.752 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:35:54.758+0000 7fe8b0000700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8a8071f10 0x7fe8a8110020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:35:54.752 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:35:54.758+0000 7fe8b0000700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8a8110620 con 0x7fe8a80708c0 2024-03-04T13:35:54.752 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:35:54.758+0000 7fe8b0000700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8a8110ec0 con 0x7fe8a8071f10 2024-03-04T13:35:54.752 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:35:54.758+0000 7fe8acc00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8a8071f10 0x7fe8a8110020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:35:54.752 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:35:54.758+0000 7fe8acc00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8a8071f10 0x7fe8a8110020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:36838/0 (socket says 172.21.15.169:36838) 2024-03-04T13:35:54.752 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:35:54.758+0000 7fe8acc00700 1 -- 172.21.15.169:0/610752805 learned_addr learned my addr 172.21.15.169:0/610752805 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-03-04T13:35:54.752 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:35:54.758+0000 7fe8ad600700 1 --2- 172.21.15.169:0/610752805 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8a80708c0 0x7fe8a810fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:35:54.752 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:35:54.758+0000 7fe8acc00700 1 -- 172.21.15.169:0/610752805 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8a80708c0 msgr2=0x7fe8a810fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:35:54.752 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:35:54.758+0000 7fe8acc00700 1 --2- 172.21.15.169:0/610752805 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8a80708c0 0x7fe8a810fae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:35:54.752 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:35:54.758+0000 7fe8acc00700 1 -- 172.21.15.169:0/610752805 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe894005040 con 0x7fe8a8071f10 2024-03-04T13:35:54.753 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:35:54.759+0000 7fe8acc00700 1 --2- 172.21.15.169:0/610752805 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8a8071f10 0x7fe8a8110020 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe894009360 tx=0x7fe894009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:35:54.753 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:35:54.759+0000 7fe8a5600700 1 -- 172.21.15.169:0/610752805 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe89400e040 con 0x7fe8a8071f10 2024-03-04T13:35:54.754 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:35:54.759+0000 7fe8b0000700 1 -- 172.21.15.169:0/610752805 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8a8111140 con 0x7fe8a8071f10 2024-03-04T13:35:54.754 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:35:54.759+0000 7fe8a5600700 1 -- 172.21.15.169:0/610752805 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe89400bcc0 con 0x7fe8a8071f10 2024-03-04T13:35:54.754 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:35:54.759+0000 7fe8a5600700 1 -- 172.21.15.169:0/610752805 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe89401c6a0 con 0x7fe8a8071f10 2024-03-04T13:35:54.754 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:35:54.759+0000 7fe8b0000700 1 -- 172.21.15.169:0/610752805 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8a8111680 con 0x7fe8a8071f10 2024-03-04T13:35:54.754 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:35:54.760+0000 7fe8b0000700 1 -- 172.21.15.169:0/610752805 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8a804bed0 con 0x7fe8a8071f10 2024-03-04T13:35:54.754 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:35:54.761+0000 7fe8a5600700 1 -- 172.21.15.169:0/610752805 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 19) v1 ==== 86623+0+0 (secure 0 0 0) 0x7fe894013b10 con 0x7fe8a8071f10 2024-03-04T13:35:54.755 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:35:54.762+0000 7fe8a5600700 1 --2- 172.21.15.169:0/610752805 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fe8900692a0 0x7fe89006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:35:54.755 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:35:54.762+0000 7fe8a5600700 1 -- 172.21.15.169:0/610752805 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 3770+0+0 (secure 0 0 0) 0x7fe894087f00 con 0x7fe8a8071f10 2024-03-04T13:35:54.756 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:35:54.763+0000 7fe8ad600700 1 --2- 172.21.15.169:0/610752805 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fe8900692a0 0x7fe89006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:35:54.757 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:35:54.764+0000 7fe8ad600700 1 --2- 172.21.15.169:0/610752805 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fe8900692a0 0x7fe89006b750 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe89c004d60 tx=0x7fe89c00b040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:35:54.760 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:35:54.768+0000 7fe8a5600700 1 -- 172.21.15.169:0/610752805 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fe894058340 con 0x7fe8a8071f10 2024-03-04T13:35:54.853 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:54 smithi169 ceph-mon[36313]: pgmap v122: 1 pgs: 1 remapped+peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-04T13:35:54.977 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:35:54.983+0000 7fe8b0000700 1 -- 172.21.15.169:0/610752805 --> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fe8a8111b20 con 0x7fe8900692a0 2024-03-04T13:35:55.086 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:54 smithi107 ceph-mon[28739]: pgmap v122: 1 pgs: 1 remapped+peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-04T13:35:55.887 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:55 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T13:35:55.887 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:55 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T13:35:55.887 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:55 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:35:56.086 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:55 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T13:35:56.086 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:55 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T13:35:56.086 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:55 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:35:56.743 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:56 smithi169 ceph-mon[36313]: from='client.24155 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:35:56.744 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:56 smithi169 ceph-mon[36313]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-03-04T13:35:57.086 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:56 smithi107 ceph-mon[28739]: from='client.24155 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:35:57.086 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:56 smithi107 ceph-mon[28739]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-03-04T13:35:58.086 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:57 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:35:58.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:57 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:35:58.984 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:58 smithi169 ceph-mon[36313]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-03-04T13:35:58.984 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:58 smithi169 ceph-mon[36313]: from='client.? 172.21.15.169:0/456254284' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "01d2b77c-66ec-4b50-8170-43d95eeae471"}]: dispatch 2024-03-04T13:35:58.984 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:58 smithi169 ceph-mon[36313]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "01d2b77c-66ec-4b50-8170-43d95eeae471"}]: dispatch 2024-03-04T13:35:58.984 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:58 smithi169 ceph-mon[36313]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "01d2b77c-66ec-4b50-8170-43d95eeae471"}]': finished 2024-03-04T13:35:58.985 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:58 smithi169 ceph-mon[36313]: osdmap e32: 5 total, 4 up, 5 in 2024-03-04T13:35:58.985 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:58 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-04T13:35:59.086 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:58 smithi107 ceph-mon[28739]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-03-04T13:35:59.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:58 smithi107 ceph-mon[28739]: from='client.? 172.21.15.169:0/456254284' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "01d2b77c-66ec-4b50-8170-43d95eeae471"}]: dispatch 2024-03-04T13:35:59.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:58 smithi107 ceph-mon[28739]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "01d2b77c-66ec-4b50-8170-43d95eeae471"}]: dispatch 2024-03-04T13:35:59.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:58 smithi107 ceph-mon[28739]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "01d2b77c-66ec-4b50-8170-43d95eeae471"}]': finished 2024-03-04T13:35:59.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:58 smithi107 ceph-mon[28739]: osdmap e32: 5 total, 4 up, 5 in 2024-03-04T13:35:59.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:58 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-04T13:36:00.086 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:35:59 smithi107 ceph-mon[28739]: from='client.? 172.21.15.169:0/1689427731' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T13:36:00.103 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:35:59 smithi169 ceph-mon[36313]: from='client.? 172.21.15.169:0/1689427731' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T13:36:01.086 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:00 smithi107 ceph-mon[28739]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-03-04T13:36:01.103 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:00 smithi169 ceph-mon[36313]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-03-04T13:36:03.086 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:02 smithi107 ceph-mon[28739]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-03-04T13:36:03.103 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:02 smithi169 ceph-mon[36313]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-03-04T13:36:05.057 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:04 smithi169 ceph-mon[36313]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-04T13:36:05.086 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:04 smithi107 ceph-mon[28739]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-04T13:36:07.084 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:06 smithi169 ceph-mon[36313]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-04T13:36:07.086 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:06 smithi107 ceph-mon[28739]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-04T13:36:08.072 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:07 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-04T13:36:08.072 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:07 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:36:08.086 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:07 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-04T13:36:08.086 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:07 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:36:09.086 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:08 smithi107 ceph-mon[28739]: Deploying daemon osd.4 on smithi169 2024-03-04T13:36:09.086 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:08 smithi107 ceph-mon[28739]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-04T13:36:09.103 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:08 smithi169 ceph-mon[36313]: Deploying daemon osd.4 on smithi169 2024-03-04T13:36:09.103 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:08 smithi169 ceph-mon[36313]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-04T13:36:11.086 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:10 smithi107 ceph-mon[28739]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-04T13:36:11.103 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:10 smithi169 ceph-mon[36313]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-04T13:36:13.086 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:12 smithi107 ceph-mon[28739]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-04T13:36:13.086 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:12 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:36:13.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:12 smithi169 ceph-mon[36313]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-04T13:36:13.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:12 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:36:15.024 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:14 smithi169 ceph-mon[36313]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-04T13:36:15.024 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:14 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:36:15.024 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:14 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:36:15.024 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:14 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:36:15.086 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:14 smithi107 ceph-mon[28739]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-04T13:36:15.086 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:14 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:36:15.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:14 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:36:15.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:14 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:36:17.017 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:16 smithi169 ceph-mon[36313]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-04T13:36:17.086 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:16 smithi107 ceph-mon[28739]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-04T13:36:18.859 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:18 smithi169 ceph-mon[36313]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-04T13:36:18.859 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:18 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:36:18.859 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:18 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:36:18.859 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:18 smithi169 ceph-mon[36313]: from='osd.4 [v2:172.21.15.169:6808/592652532,v1:172.21.15.169:6809/592652532]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-04T13:36:18.859 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:18 smithi169 ceph-mon[36313]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-04T13:36:19.086 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:18 smithi107 ceph-mon[28739]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-04T13:36:19.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:18 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:36:19.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:18 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:36:19.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:18 smithi107 ceph-mon[28739]: from='osd.4 [v2:172.21.15.169:6808/592652532,v1:172.21.15.169:6809/592652532]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-04T13:36:19.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:18 smithi107 ceph-mon[28739]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-04T13:36:19.590 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:36:19.597+0000 7fe8a5600700 1 -- 172.21.15.169:0/610752805 <== mgr.14212 v2:172.21.15.107:6800/3848495603 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fe8a8111b20 con 0x7fe8900692a0 2024-03-04T13:36:19.591 INFO:teuthology.orchestra.run.smithi169.stdout:Created osd(s) 4 on host 'smithi169' 2024-03-04T13:36:19.594 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:36:19.600+0000 7fe8b0000700 1 -- 172.21.15.169:0/610752805 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fe8900692a0 msgr2=0x7fe89006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:36:19.594 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:36:19.600+0000 7fe8b0000700 1 --2- 172.21.15.169:0/610752805 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fe8900692a0 0x7fe89006b750 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe89c004d60 tx=0x7fe89c00b040 comp rx=0 tx=0).stop 2024-03-04T13:36:19.595 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:36:19.600+0000 7fe8b0000700 1 -- 172.21.15.169:0/610752805 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8a8071f10 msgr2=0x7fe8a8110020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:36:19.595 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:36:19.600+0000 7fe8b0000700 1 --2- 172.21.15.169:0/610752805 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8a8071f10 0x7fe8a8110020 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe894009360 tx=0x7fe894009e30 comp rx=0 tx=0).stop 2024-03-04T13:36:19.595 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:36:19.601+0000 7fe8b0000700 1 -- 172.21.15.169:0/610752805 shutdown_connections 2024-03-04T13:36:19.595 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:36:19.601+0000 7fe8b0000700 1 --2- 172.21.15.169:0/610752805 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fe8900692a0 0x7fe89006b750 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:36:19.595 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:36:19.601+0000 7fe8b0000700 1 --2- 172.21.15.169:0/610752805 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8a80708c0 0x7fe8a810fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:36:19.596 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:36:19.601+0000 7fe8b0000700 1 --2- 172.21.15.169:0/610752805 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8a8071f10 0x7fe8a8110020 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:36:19.596 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:36:19.601+0000 7fe8b0000700 1 -- 172.21.15.169:0/610752805 >> 172.21.15.169:0/610752805 conn(0x7fe8a80f8620 msgr2=0x7fe8a8102dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:36:19.596 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:36:19.601+0000 7fe8b0000700 1 -- 172.21.15.169:0/610752805 shutdown_connections 2024-03-04T13:36:19.596 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:36:19.601+0000 7fe8b0000700 1 -- 172.21.15.169:0/610752805 wait complete. 2024-03-04T13:36:20.289 DEBUG:teuthology.orchestra.run.smithi169:osd.4> sudo journalctl -f -n 0 -u ceph-1fc09a58-da2b-11ee-95c6-87774f69a715@osd.4.service 2024-03-04T13:36:20.297 INFO:tasks.cephadm:Deploying osd.5 on smithi169 with /dev/vg_nvme/lv_2... 2024-03-04T13:36:20.297 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-04T13:36:20.797 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:20 smithi169 ceph-mon[36313]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-04T13:36:20.797 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:20 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:36:20.797 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:20 smithi169 ceph-mon[36313]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-04T13:36:20.797 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:20 smithi169 ceph-mon[36313]: osdmap e33: 5 total, 4 up, 5 in 2024-03-04T13:36:20.797 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:20 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-04T13:36:20.797 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:20 smithi169 ceph-mon[36313]: from='osd.4 [v2:172.21.15.169:6808/592652532,v1:172.21.15.169:6809/592652532]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-03-04T13:36:20.798 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:20 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:36:20.798 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:20 smithi169 ceph-mon[36313]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-03-04T13:36:20.798 INFO:journalctl@ceph.osd.4.smithi169.stdout:Mar 04 13:36:20 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-4[48277]: 2024-03-04T13:36:20.574+0000 7fe808a00700 -1 osd.4 0 waiting for initial osdmap 2024-03-04T13:36:20.798 INFO:journalctl@ceph.osd.4.smithi169.stdout:Mar 04 13:36:20 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-4[48277]: 2024-03-04T13:36:20.591+0000 7fe801c00700 -1 osd.4 34 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-04T13:36:20.836 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:20 smithi107 ceph-mon[28739]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-04T13:36:20.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:20 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:36:20.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:20 smithi107 ceph-mon[28739]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-04T13:36:20.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:20 smithi107 ceph-mon[28739]: osdmap e33: 5 total, 4 up, 5 in 2024-03-04T13:36:20.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:20 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-04T13:36:20.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:20 smithi107 ceph-mon[28739]: from='osd.4 [v2:172.21.15.169:6808/592652532,v1:172.21.15.169:6809/592652532]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-03-04T13:36:20.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:20 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:36:20.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:20 smithi107 ceph-mon[28739]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-03-04T13:36:21.603 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:21 smithi169 ceph-mon[36313]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-03-04T13:36:21.604 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:21 smithi169 ceph-mon[36313]: osdmap e34: 5 total, 4 up, 5 in 2024-03-04T13:36:21.604 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:21 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-04T13:36:21.604 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:21 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-04T13:36:21.630 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/mon.smithi169/config 2024-03-04T13:36:21.836 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:21 smithi107 ceph-mon[28739]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-03-04T13:36:21.836 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:21 smithi107 ceph-mon[28739]: osdmap e34: 5 total, 4 up, 5 in 2024-03-04T13:36:21.836 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:21 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-04T13:36:21.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:21 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-04T13:36:22.835 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:22 smithi169 ceph-mon[36313]: purged_snaps scrub starts 2024-03-04T13:36:22.835 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:22 smithi169 ceph-mon[36313]: purged_snaps scrub ok 2024-03-04T13:36:22.835 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:22 smithi169 ceph-mon[36313]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-04T13:36:22.835 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:22 smithi169 ceph-mon[36313]: osd.4 [v2:172.21.15.169:6808/592652532,v1:172.21.15.169:6809/592652532] boot 2024-03-04T13:36:22.835 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:22 smithi169 ceph-mon[36313]: osdmap e35: 5 total, 5 up, 5 in 2024-03-04T13:36:22.835 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:22 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-04T13:36:22.836 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:22 smithi107 ceph-mon[28739]: purged_snaps scrub starts 2024-03-04T13:36:22.836 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:22 smithi107 ceph-mon[28739]: purged_snaps scrub ok 2024-03-04T13:36:22.836 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:22 smithi107 ceph-mon[28739]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-04T13:36:22.836 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:22 smithi107 ceph-mon[28739]: osd.4 [v2:172.21.15.169:6808/592652532,v1:172.21.15.169:6809/592652532] boot 2024-03-04T13:36:22.836 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:22 smithi107 ceph-mon[28739]: osdmap e35: 5 total, 5 up, 5 in 2024-03-04T13:36:22.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:22 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-04T13:36:23.732 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:23 smithi169 ceph-mon[36313]: osdmap e36: 5 total, 5 up, 5 in 2024-03-04T13:36:23.836 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:23 smithi107 ceph-mon[28739]: osdmap e36: 5 total, 5 up, 5 in 2024-03-04T13:36:24.596 INFO:teuthology.orchestra.run.smithi169.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-04T13:36:24.596 INFO:teuthology.orchestra.run.smithi169.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-04T13:36:24.596 INFO:teuthology.orchestra.run.smithi169.stderr: stderr: 10+0 records in 2024-03-04T13:36:24.597 INFO:teuthology.orchestra.run.smithi169.stderr:10+0 records out 2024-03-04T13:36:24.597 INFO:teuthology.orchestra.run.smithi169.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0591196 s, 177 MB/s 2024-03-04T13:36:24.597 INFO:teuthology.orchestra.run.smithi169.stderr:--> Zapping successful for: 2024-03-04T13:36:24.853 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:24 smithi169 ceph-mon[36313]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-04T13:36:24.853 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:24 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:36:24.854 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:24 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:36:24.854 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:24 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-04T13:36:24.854 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:24 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-04T13:36:24.854 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:24 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:36:25.086 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:24 smithi107 ceph-mon[28739]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-04T13:36:25.086 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:24 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:36:25.086 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:24 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:36:25.086 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:24 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-04T13:36:25.086 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:24 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-04T13:36:25.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:24 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:36:25.335 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 -- ceph orch daemon add osd smithi169:vg_nvme/lv_2 2024-03-04T13:36:25.809 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/mon.smithi169/config 2024-03-04T13:36:25.853 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:25 smithi169 ceph-mon[36313]: Detected new or changed devices on smithi169 2024-03-04T13:36:25.853 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:25 smithi169 ceph-mon[36313]: Adjusting osd_memory_target on smithi169 to 7969M 2024-03-04T13:36:25.853 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:25 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:36:25.853 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:25 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:36:25.853 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:25 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:36:25.853 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:25 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:36:26.086 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:25 smithi107 ceph-mon[28739]: Detected new or changed devices on smithi169 2024-03-04T13:36:26.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:25 smithi107 ceph-mon[28739]: Adjusting osd_memory_target on smithi169 to 7969M 2024-03-04T13:36:26.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:25 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:36:26.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:25 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:36:26.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:25 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:36:26.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:25 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:36:26.735 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:26 smithi169 ceph-mon[36313]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-04T13:36:26.735 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:26 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:36:26.735 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:26 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:36:26.735 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:26 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:36:27.086 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:26 smithi107 ceph-mon[28739]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-04T13:36:27.086 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:26 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:36:27.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:26 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:36:27.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:26 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:36:27.806 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:36:27.813+0000 7fc33c800700 1 -- 172.21.15.169:0/4127544841 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc334100360 msgr2=0x7fc3341007b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:36:27.807 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:36:27.813+0000 7fc33c800700 1 --2- 172.21.15.169:0/4127544841 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc334100360 0x7fc3341007b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc324004940 tx=0x7fc324009e30 comp rx=0 tx=0).stop 2024-03-04T13:36:27.807 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:36:27.813+0000 7fc33c800700 1 -- 172.21.15.169:0/4127544841 shutdown_connections 2024-03-04T13:36:27.807 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:36:27.813+0000 7fc33c800700 1 --2- 172.21.15.169:0/4127544841 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc334100360 0x7fc3341007b0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:36:27.807 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:36:27.813+0000 7fc33c800700 1 --2- 172.21.15.169:0/4127544841 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3340ff150 0x7fc3340ff560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:36:27.807 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:36:27.813+0000 7fc33c800700 1 -- 172.21.15.169:0/4127544841 >> 172.21.15.169:0/4127544841 conn(0x7fc3340fa920 msgr2=0x7fc3340fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:36:27.807 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:36:27.813+0000 7fc33c800700 1 -- 172.21.15.169:0/4127544841 shutdown_connections 2024-03-04T13:36:27.807 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:36:27.813+0000 7fc33c800700 1 -- 172.21.15.169:0/4127544841 wait complete. 2024-03-04T13:36:27.808 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:36:27.814+0000 7fc33c800700 1 Processor -- start 2024-03-04T13:36:27.808 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:36:27.814+0000 7fc33c800700 1 -- start start 2024-03-04T13:36:27.808 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:36:27.815+0000 7fc33c800700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3340ff150 0x7fc33410fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:36:27.809 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:36:27.815+0000 7fc33c800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc334100360 0x7fc3341100b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:36:27.809 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:36:27.815+0000 7fc33c800700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3341106b0 con 0x7fc334100360 2024-03-04T13:36:27.809 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:36:27.815+0000 7fc33c800700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc334110f50 con 0x7fc3340ff150 2024-03-04T13:36:27.809 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:36:27.815+0000 7fc339e00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3340ff150 0x7fc33410fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:36:27.809 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:36:27.815+0000 7fc339e00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3340ff150 0x7fc33410fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:43416/0 (socket says 172.21.15.169:43416) 2024-03-04T13:36:27.809 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:36:27.815+0000 7fc339e00700 1 -- 172.21.15.169:0/3232071555 learned_addr learned my addr 172.21.15.169:0/3232071555 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-03-04T13:36:27.809 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:36:27.815+0000 7fc339400700 1 --2- 172.21.15.169:0/3232071555 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc334100360 0x7fc3341100b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:36:27.809 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:36:27.815+0000 7fc339e00700 1 -- 172.21.15.169:0/3232071555 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc334100360 msgr2=0x7fc3341100b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:36:27.809 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:36:27.815+0000 7fc339e00700 1 --2- 172.21.15.169:0/3232071555 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc334100360 0x7fc3341100b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:36:27.809 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:36:27.815+0000 7fc339e00700 1 -- 172.21.15.169:0/3232071555 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc324005040 con 0x7fc3340ff150 2024-03-04T13:36:27.810 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:36:27.816+0000 7fc339e00700 1 --2- 172.21.15.169:0/3232071555 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3340ff150 0x7fc33410fb70 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc32800c8e0 tx=0x7fc3280044c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:36:27.810 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:36:27.816+0000 7fc32e000700 1 -- 172.21.15.169:0/3232071555 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc32800a940 con 0x7fc3340ff150 2024-03-04T13:36:27.810 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:36:27.816+0000 7fc32e000700 1 -- 172.21.15.169:0/3232071555 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc32800ce80 con 0x7fc3340ff150 2024-03-04T13:36:27.810 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:36:27.816+0000 7fc32e000700 1 -- 172.21.15.169:0/3232071555 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc3280113a0 con 0x7fc3340ff150 2024-03-04T13:36:27.810 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:36:27.816+0000 7fc33c800700 1 -- 172.21.15.169:0/3232071555 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc334111230 con 0x7fc3340ff150 2024-03-04T13:36:27.810 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:36:27.816+0000 7fc33c800700 1 -- 172.21.15.169:0/3232071555 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc334111770 con 0x7fc3340ff150 2024-03-04T13:36:27.810 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:36:27.817+0000 7fc33c800700 1 -- 172.21.15.169:0/3232071555 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc33404bed0 con 0x7fc3340ff150 2024-03-04T13:36:27.811 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:36:27.818+0000 7fc32e000700 1 -- 172.21.15.169:0/3232071555 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 19) v1 ==== 86623+0+0 (secure 0 0 0) 0x7fc328012400 con 0x7fc3340ff150 2024-03-04T13:36:27.812 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:36:27.819+0000 7fc32e000700 1 --2- 172.21.15.169:0/3232071555 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fc31c0692b0 0x7fc31c06b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:36:27.813 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:36:27.819+0000 7fc32e000700 1 -- 172.21.15.169:0/3232071555 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4202+0+0 (secure 0 0 0) 0x7fc328086cc0 con 0x7fc3340ff150 2024-03-04T13:36:27.813 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:36:27.820+0000 7fc339400700 1 --2- 172.21.15.169:0/3232071555 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fc31c0692b0 0x7fc31c06b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:36:27.814 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:36:27.821+0000 7fc339400700 1 --2- 172.21.15.169:0/3232071555 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fc31c0692b0 0x7fc31c06b760 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc32400b020 tx=0x7fc324009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:36:27.817 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:36:27.824+0000 7fc32e000700 1 -- 172.21.15.169:0/3232071555 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fc328057010 con 0x7fc3340ff150 2024-03-04T13:36:28.033 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:36:28.039+0000 7fc33c800700 1 -- 172.21.15.169:0/3232071555 --> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fc334104cb0 con 0x7fc31c0692b0 2024-03-04T13:36:28.033 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:27 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:36:28.086 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:27 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:36:28.900 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:28 smithi169 ceph-mon[36313]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-04T13:36:28.900 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:28 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T13:36:28.900 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:28 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T13:36:28.900 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:28 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:36:29.086 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:28 smithi107 ceph-mon[28739]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-04T13:36:29.086 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:28 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T13:36:29.086 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:28 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T13:36:29.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:28 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:36:29.713 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:29 smithi169 ceph-mon[36313]: from='client.24173 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:36:30.086 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:29 smithi107 ceph-mon[28739]: from='client.24173 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi169:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:36:31.086 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:30 smithi107 ceph-mon[28739]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-04T13:36:31.103 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:30 smithi169 ceph-mon[36313]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-04T13:36:31.988 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:31 smithi169 ceph-mon[36313]: from='client.? 172.21.15.169:0/3215231408' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0164ead7-c5d0-4541-bf22-f671558f8607"}]: dispatch 2024-03-04T13:36:31.989 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:31 smithi169 ceph-mon[36313]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0164ead7-c5d0-4541-bf22-f671558f8607"}]: dispatch 2024-03-04T13:36:31.989 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:31 smithi169 ceph-mon[36313]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0164ead7-c5d0-4541-bf22-f671558f8607"}]': finished 2024-03-04T13:36:31.989 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:31 smithi169 ceph-mon[36313]: osdmap e37: 6 total, 5 up, 6 in 2024-03-04T13:36:31.989 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:31 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-04T13:36:32.086 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:31 smithi107 ceph-mon[28739]: from='client.? 172.21.15.169:0/3215231408' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0164ead7-c5d0-4541-bf22-f671558f8607"}]: dispatch 2024-03-04T13:36:32.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:31 smithi107 ceph-mon[28739]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0164ead7-c5d0-4541-bf22-f671558f8607"}]: dispatch 2024-03-04T13:36:32.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:31 smithi107 ceph-mon[28739]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0164ead7-c5d0-4541-bf22-f671558f8607"}]': finished 2024-03-04T13:36:32.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:31 smithi107 ceph-mon[28739]: osdmap e37: 6 total, 5 up, 6 in 2024-03-04T13:36:32.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:31 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-04T13:36:33.086 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:32 smithi107 ceph-mon[28739]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-04T13:36:33.086 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:32 smithi107 ceph-mon[28739]: from='client.? 172.21.15.169:0/93146020' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T13:36:33.103 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:32 smithi169 ceph-mon[36313]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-04T13:36:33.103 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:32 smithi169 ceph-mon[36313]: from='client.? 172.21.15.169:0/93146020' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T13:36:35.086 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:34 smithi107 ceph-mon[28739]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-04T13:36:35.103 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:34 smithi169 ceph-mon[36313]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-04T13:36:36.853 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:36 smithi169 ceph-mon[36313]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-04T13:36:37.086 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:36 smithi107 ceph-mon[28739]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-04T13:36:38.898 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:38 smithi169 ceph-mon[36313]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-04T13:36:39.086 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:38 smithi107 ceph-mon[28739]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-04T13:36:40.975 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:40 smithi169 ceph-mon[36313]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-04T13:36:40.975 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:40 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-04T13:36:40.975 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:40 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:36:41.086 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:40 smithi107 ceph-mon[28739]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-04T13:36:41.086 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:40 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-04T13:36:41.086 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:40 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:36:41.823 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:41 smithi169 ceph-mon[36313]: Deploying daemon osd.5 on smithi169 2024-03-04T13:36:42.086 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:41 smithi107 ceph-mon[28739]: Deploying daemon osd.5 on smithi169 2024-03-04T13:36:42.964 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:42 smithi169 ceph-mon[36313]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-04T13:36:42.965 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:42 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:36:43.086 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:42 smithi107 ceph-mon[28739]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-04T13:36:43.086 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:42 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:36:45.024 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:44 smithi169 ceph-mon[36313]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-04T13:36:45.086 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:44 smithi107 ceph-mon[28739]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-04T13:36:47.086 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:46 smithi107 ceph-mon[28739]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-04T13:36:47.103 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:46 smithi169 ceph-mon[36313]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-04T13:36:49.074 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:48 smithi169 ceph-mon[36313]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-04T13:36:49.074 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:48 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:36:49.074 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:48 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:36:49.074 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:48 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:36:49.086 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:48 smithi107 ceph-mon[28739]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-04T13:36:49.086 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:48 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:36:49.086 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:48 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:36:49.086 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:48 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:36:50.805 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:50 smithi169 ceph-mon[36313]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-04T13:36:51.086 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:50 smithi107 ceph-mon[28739]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-04T13:36:51.846 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:51 smithi169 ceph-mon[36313]: from='osd.5 [v2:172.21.15.169:6816/182182184,v1:172.21.15.169:6817/182182184]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-04T13:36:51.846 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:51 smithi169 ceph-mon[36313]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-04T13:36:51.931 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:36:51.937+0000 7fc32e000700 1 -- 172.21.15.169:0/3232071555 <== mgr.14212 v2:172.21.15.107:6800/3848495603 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fc334104cb0 con 0x7fc31c0692b0 2024-03-04T13:36:51.931 INFO:teuthology.orchestra.run.smithi169.stdout:Created osd(s) 5 on host 'smithi169' 2024-03-04T13:36:51.936 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:36:51.942+0000 7fc33c800700 1 -- 172.21.15.169:0/3232071555 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fc31c0692b0 msgr2=0x7fc31c06b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:36:51.936 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:36:51.942+0000 7fc33c800700 1 --2- 172.21.15.169:0/3232071555 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fc31c0692b0 0x7fc31c06b760 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc32400b020 tx=0x7fc324009e30 comp rx=0 tx=0).stop 2024-03-04T13:36:51.936 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:36:51.942+0000 7fc33c800700 1 -- 172.21.15.169:0/3232071555 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3340ff150 msgr2=0x7fc33410fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:36:51.936 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:36:51.942+0000 7fc33c800700 1 --2- 172.21.15.169:0/3232071555 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3340ff150 0x7fc33410fb70 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc32800c8e0 tx=0x7fc3280044c0 comp rx=0 tx=0).stop 2024-03-04T13:36:51.936 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:36:51.942+0000 7fc33c800700 1 -- 172.21.15.169:0/3232071555 shutdown_connections 2024-03-04T13:36:51.936 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:36:51.943+0000 7fc33c800700 1 --2- 172.21.15.169:0/3232071555 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fc31c0692b0 0x7fc31c06b760 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:36:51.936 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:36:51.943+0000 7fc33c800700 1 --2- 172.21.15.169:0/3232071555 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3340ff150 0x7fc33410fb70 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:36:51.936 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:36:51.943+0000 7fc33c800700 1 --2- 172.21.15.169:0/3232071555 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc334100360 0x7fc3341100b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:36:51.936 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:36:51.943+0000 7fc33c800700 1 -- 172.21.15.169:0/3232071555 >> 172.21.15.169:0/3232071555 conn(0x7fc3340fa920 msgr2=0x7fc334103590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:36:51.937 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:36:51.943+0000 7fc33c800700 1 -- 172.21.15.169:0/3232071555 shutdown_connections 2024-03-04T13:36:51.937 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:36:51.943+0000 7fc33c800700 1 -- 172.21.15.169:0/3232071555 wait complete. 2024-03-04T13:36:52.086 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:51 smithi107 ceph-mon[28739]: from='osd.5 [v2:172.21.15.169:6816/182182184,v1:172.21.15.169:6817/182182184]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-04T13:36:52.086 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:51 smithi107 ceph-mon[28739]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-04T13:36:52.564 DEBUG:teuthology.orchestra.run.smithi169:osd.5> sudo journalctl -f -n 0 -u ceph-1fc09a58-da2b-11ee-95c6-87774f69a715@osd.5.service 2024-03-04T13:36:52.571 INFO:tasks.cephadm:Waiting for 6 OSDs to come up... 2024-03-04T13:36:52.572 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 -- ceph osd stat -f json 2024-03-04T13:36:52.769 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:52 smithi107 ceph-mon[28739]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-04T13:36:52.769 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:52 smithi107 ceph-mon[28739]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-04T13:36:52.769 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:52 smithi107 ceph-mon[28739]: osdmap e38: 6 total, 5 up, 6 in 2024-03-04T13:36:52.769 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:52 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-04T13:36:52.770 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:52 smithi107 ceph-mon[28739]: from='osd.5 [v2:172.21.15.169:6816/182182184,v1:172.21.15.169:6817/182182184]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-03-04T13:36:52.770 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:52 smithi107 ceph-mon[28739]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-03-04T13:36:52.770 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:52 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:36:52.770 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:52 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:36:52.770 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:52 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:36:52.770 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:52 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:36:52.853 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:52 smithi169 ceph-mon[36313]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-04T13:36:52.853 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:52 smithi169 ceph-mon[36313]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-04T13:36:52.853 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:52 smithi169 ceph-mon[36313]: osdmap e38: 6 total, 5 up, 6 in 2024-03-04T13:36:52.853 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:52 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-04T13:36:52.853 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:52 smithi169 ceph-mon[36313]: from='osd.5 [v2:172.21.15.169:6816/182182184,v1:172.21.15.169:6817/182182184]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-03-04T13:36:52.853 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:52 smithi169 ceph-mon[36313]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-03-04T13:36:52.854 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:52 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:36:52.854 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:52 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:36:52.854 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:52 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:36:52.854 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:52 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:36:52.854 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 13:36:52 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-5[54856]: 2024-03-04T13:36:52.669+0000 7f909ce00700 -1 osd.5 0 waiting for initial osdmap 2024-03-04T13:36:52.854 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 13:36:52 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-5[54856]: 2024-03-04T13:36:52.700+0000 7f9096000700 -1 osd.5 39 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-04T13:36:53.025 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/mon.smithi107/config 2024-03-04T13:36:53.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:53 smithi107 ceph-mon[28739]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-03-04T13:36:53.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:53 smithi107 ceph-mon[28739]: osdmap e39: 6 total, 5 up, 6 in 2024-03-04T13:36:53.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:53 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-04T13:36:53.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:53 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-04T13:36:54.103 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:53 smithi169 ceph-mon[36313]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]': finished 2024-03-04T13:36:54.103 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:53 smithi169 ceph-mon[36313]: osdmap e39: 6 total, 5 up, 6 in 2024-03-04T13:36:54.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:53 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-04T13:36:54.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:53 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-04T13:36:54.944 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:54 smithi107 ceph-mon[28739]: purged_snaps scrub starts 2024-03-04T13:36:54.945 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:54 smithi107 ceph-mon[28739]: purged_snaps scrub ok 2024-03-04T13:36:54.945 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:54 smithi107 ceph-mon[28739]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-04T13:36:54.945 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:54 smithi107 ceph-mon[28739]: osd.5 [v2:172.21.15.169:6816/182182184,v1:172.21.15.169:6817/182182184] boot 2024-03-04T13:36:54.945 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:54 smithi107 ceph-mon[28739]: osdmap e40: 6 total, 6 up, 6 in 2024-03-04T13:36:54.945 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:54 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-04T13:36:55.103 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:54 smithi169 ceph-mon[36313]: purged_snaps scrub starts 2024-03-04T13:36:55.103 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:54 smithi169 ceph-mon[36313]: purged_snaps scrub ok 2024-03-04T13:36:55.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:54 smithi169 ceph-mon[36313]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-04T13:36:55.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:54 smithi169 ceph-mon[36313]: osd.5 [v2:172.21.15.169:6816/182182184,v1:172.21.15.169:6817/182182184] boot 2024-03-04T13:36:55.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:54 smithi169 ceph-mon[36313]: osdmap e40: 6 total, 6 up, 6 in 2024-03-04T13:36:55.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:54 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-04T13:36:55.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:36:55.246+0000 7fc37c200700 1 -- 172.21.15.107:0/3441882359 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc374100360 msgr2=0x7fc3741007b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:36:55.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:36:55.246+0000 7fc37c200700 1 --2- 172.21.15.107:0/3441882359 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc374100360 0x7fc3741007b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc360004940 tx=0x7fc360009e30 comp rx=0 tx=0).stop 2024-03-04T13:36:55.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:36:55.246+0000 7fc37c200700 1 -- 172.21.15.107:0/3441882359 shutdown_connections 2024-03-04T13:36:55.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:36:55.246+0000 7fc37c200700 1 --2- 172.21.15.107:0/3441882359 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc374100360 0x7fc3741007b0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:36:55.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:36:55.246+0000 7fc37c200700 1 --2- 172.21.15.107:0/3441882359 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3740ff150 0x7fc3740ff560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:36:55.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:36:55.246+0000 7fc37c200700 1 -- 172.21.15.107:0/3441882359 >> 172.21.15.107:0/3441882359 conn(0x7fc3740fa920 msgr2=0x7fc3740fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:36:55.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:36:55.246+0000 7fc37c200700 1 -- 172.21.15.107:0/3441882359 shutdown_connections 2024-03-04T13:36:55.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:36:55.246+0000 7fc37c200700 1 -- 172.21.15.107:0/3441882359 wait complete. 2024-03-04T13:36:55.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:36:55.247+0000 7fc37c200700 1 Processor -- start 2024-03-04T13:36:55.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:36:55.247+0000 7fc37c200700 1 -- start start 2024-03-04T13:36:55.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:36:55.247+0000 7fc37c200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3740ff150 0x7fc37410fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:36:55.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:36:55.248+0000 7fc37c200700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc374100360 0x7fc374110010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:36:55.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:36:55.248+0000 7fc37c200700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc374110610 con 0x7fc3740ff150 2024-03-04T13:36:55.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:36:55.248+0000 7fc37c200700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc374110eb0 con 0x7fc374100360 2024-03-04T13:36:55.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:36:55.248+0000 7fc379800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3740ff150 0x7fc37410fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:36:55.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:36:55.248+0000 7fc379800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3740ff150 0x7fc37410fad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38074/0 (socket says 172.21.15.107:38074) 2024-03-04T13:36:55.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:36:55.248+0000 7fc379800700 1 -- 172.21.15.107:0/3259745914 learned_addr learned my addr 172.21.15.107:0/3259745914 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:36:55.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:36:55.248+0000 7fc378e00700 1 --2- 172.21.15.107:0/3259745914 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc374100360 0x7fc374110010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:36:55.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:36:55.248+0000 7fc379800700 1 -- 172.21.15.107:0/3259745914 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc374100360 msgr2=0x7fc374110010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:36:55.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:36:55.248+0000 7fc379800700 1 --2- 172.21.15.107:0/3259745914 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc374100360 0x7fc374110010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:36:55.244 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:36:55.248+0000 7fc379800700 1 -- 172.21.15.107:0/3259745914 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc360005040 con 0x7fc3740ff150 2024-03-04T13:36:55.244 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:36:55.248+0000 7fc379800700 1 --2- 172.21.15.107:0/3259745914 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3740ff150 0x7fc37410fad0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc36800eb70 tx=0x7fc36800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:36:55.244 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:36:55.249+0000 7fc371600700 1 -- 172.21.15.107:0/3259745914 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc368007cc0 con 0x7fc3740ff150 2024-03-04T13:36:55.244 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:36:55.249+0000 7fc371600700 1 -- 172.21.15.107:0/3259745914 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc36800f040 con 0x7fc3740ff150 2024-03-04T13:36:55.244 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:36:55.249+0000 7fc371600700 1 -- 172.21.15.107:0/3259745914 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc36800cbd0 con 0x7fc3740ff150 2024-03-04T13:36:55.244 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:36:55.249+0000 7fc37c200700 1 -- 172.21.15.107:0/3259745914 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc374111130 con 0x7fc3740ff150 2024-03-04T13:36:55.244 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:36:55.249+0000 7fc37c200700 1 -- 172.21.15.107:0/3259745914 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc374111670 con 0x7fc3740ff150 2024-03-04T13:36:55.245 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:36:55.250+0000 7fc37c200700 1 -- 172.21.15.107:0/3259745914 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc37404bed0 con 0x7fc3740ff150 2024-03-04T13:36:55.245 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:36:55.251+0000 7fc371600700 1 -- 172.21.15.107:0/3259745914 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86623+0+0 (secure 0 0 0) 0x7fc36800cd30 con 0x7fc3740ff150 2024-03-04T13:36:55.245 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:36:55.252+0000 7fc371600700 1 --2- 172.21.15.107:0/3259745914 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fc35c0692b0 0x7fc35c06b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:36:55.245 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:36:55.252+0000 7fc371600700 1 -- 172.21.15.107:0/3259745914 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fc368086940 con 0x7fc3740ff150 2024-03-04T13:36:55.246 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:36:55.252+0000 7fc378e00700 1 --2- 172.21.15.107:0/3259745914 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fc35c0692b0 0x7fc35c06b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:36:55.246 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:36:55.253+0000 7fc378e00700 1 --2- 172.21.15.107:0/3259745914 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fc35c0692b0 0x7fc35c06b760 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc360009fd0 tx=0x7fc360009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:36:55.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:36:55.257+0000 7fc371600700 1 -- 172.21.15.107:0/3259745914 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fc36804dbd0 con 0x7fc3740ff150 2024-03-04T13:36:55.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:36:55.464+0000 7fc37c200700 1 -- 172.21.15.107:0/3259745914 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7fc374111d30 con 0x7fc3740ff150 2024-03-04T13:36:55.458 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:36:55.465+0000 7fc371600700 1 -- 172.21.15.107:0/3259745914 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v41) v1 ==== 74+0+130 (secure 0 0 0) 0x7fc368056a50 con 0x7fc3740ff150 2024-03-04T13:36:55.458 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:36:55.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:36:55.467+0000 7fc37c200700 1 -- 172.21.15.107:0/3259745914 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fc35c0692b0 msgr2=0x7fc35c06b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:36:55.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:36:55.467+0000 7fc37c200700 1 --2- 172.21.15.107:0/3259745914 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fc35c0692b0 0x7fc35c06b760 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc360009fd0 tx=0x7fc360009e30 comp rx=0 tx=0).stop 2024-03-04T13:36:55.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:36:55.467+0000 7fc37c200700 1 -- 172.21.15.107:0/3259745914 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3740ff150 msgr2=0x7fc37410fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:36:55.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:36:55.467+0000 7fc37c200700 1 --2- 172.21.15.107:0/3259745914 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3740ff150 0x7fc37410fad0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc36800eb70 tx=0x7fc36800b040 comp rx=0 tx=0).stop 2024-03-04T13:36:55.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:36:55.468+0000 7fc37c200700 1 -- 172.21.15.107:0/3259745914 shutdown_connections 2024-03-04T13:36:55.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:36:55.468+0000 7fc37c200700 1 --2- 172.21.15.107:0/3259745914 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fc35c0692b0 0x7fc35c06b760 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:36:55.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:36:55.468+0000 7fc37c200700 1 --2- 172.21.15.107:0/3259745914 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3740ff150 0x7fc37410fad0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:36:55.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:36:55.468+0000 7fc37c200700 1 --2- 172.21.15.107:0/3259745914 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc374100360 0x7fc374110010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:36:55.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:36:55.468+0000 7fc37c200700 1 -- 172.21.15.107:0/3259745914 >> 172.21.15.107:0/3259745914 conn(0x7fc3740fa920 msgr2=0x7fc374103590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:36:55.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:36:55.468+0000 7fc37c200700 1 -- 172.21.15.107:0/3259745914 shutdown_connections 2024-03-04T13:36:55.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:36:55.468+0000 7fc37c200700 1 -- 172.21.15.107:0/3259745914 wait complete. 2024-03-04T13:36:55.822 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:55 smithi107 ceph-mon[28739]: osdmap e41: 6 total, 6 up, 6 in 2024-03-04T13:36:55.822 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:55 smithi107 ceph-mon[28739]: from='client.? 172.21.15.107:0/3259745914' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-04T13:36:56.103 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:55 smithi169 ceph-mon[36313]: osdmap e41: 6 total, 6 up, 6 in 2024-03-04T13:36:56.103 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:55 smithi169 ceph-mon[36313]: from='client.? 172.21.15.107:0/3259745914' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-04T13:36:56.233 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":41,"num_osds":6,"num_up_osds":6,"osd_up_since":1709559413,"num_in_osds":6,"osd_in_since":1709559391,"num_remapped_pgs":0} 2024-03-04T13:36:56.234 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 -- ceph osd dump --format=json 2024-03-04T13:36:56.721 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/mon.smithi107/config 2024-03-04T13:36:57.056 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:56 smithi107 ceph-mon[28739]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-04T13:36:57.103 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:56 smithi169 ceph-mon[36313]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-04T13:36:58.086 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:57 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:36:58.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:57 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:36:58.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:57 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:36:58.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:57 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-04T13:36:58.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:57 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-04T13:36:58.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:57 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-04T13:36:58.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:57 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:36:58.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:57 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:36:58.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:57 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:36:58.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:57 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:36:58.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:57 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:36:58.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:57 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:36:58.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:57 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:36:58.088 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:57 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:36:58.103 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:57 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:36:58.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:57 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:36:58.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:57 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:36:58.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:57 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-04T13:36:58.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:57 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-04T13:36:58.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:57 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-04T13:36:58.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:57 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:36:58.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:57 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:36:58.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:57 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:36:58.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:57 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:36:58.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:57 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:36:58.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:57 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:36:58.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:57 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:36:58.105 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:57 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:36:58.831 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:58 smithi107 ceph-mon[28739]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-04T13:36:58.831 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:58 smithi107 ceph-mon[28739]: Detected new or changed devices on smithi169 2024-03-04T13:36:58.831 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:58 smithi107 ceph-mon[28739]: Adjusting osd_memory_target on smithi169 to 5313M 2024-03-04T13:36:58.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:36:58.996+0000 7f08da800700 1 -- 172.21.15.107:0/931745593 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f08d40fe900 msgr2=0x7f08d4106f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:36:58.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:36:58.996+0000 7f08da800700 1 --2- 172.21.15.107:0/931745593 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f08d40fe900 0x7f08d4106f40 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f08c0004940 tx=0x7f08c0009e30 comp rx=0 tx=0).stop 2024-03-04T13:36:58.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:36:58.997+0000 7f08da800700 1 -- 172.21.15.107:0/931745593 shutdown_connections 2024-03-04T13:36:58.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:36:58.997+0000 7f08da800700 1 --2- 172.21.15.107:0/931745593 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f08d40fe900 0x7f08d4106f40 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:36:58.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:36:58.997+0000 7f08da800700 1 --2- 172.21.15.107:0/931745593 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08d40fdf60 0x7f08d40fe330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:36:58.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:36:58.997+0000 7f08da800700 1 -- 172.21.15.107:0/931745593 >> 172.21.15.107:0/931745593 conn(0x7f08d40f7a10 msgr2=0x7f08d40f9e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:36:58.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:36:58.997+0000 7f08da800700 1 -- 172.21.15.107:0/931745593 shutdown_connections 2024-03-04T13:36:58.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:36:58.997+0000 7f08da800700 1 -- 172.21.15.107:0/931745593 wait complete. 2024-03-04T13:36:58.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:36:58.998+0000 7f08da800700 1 Processor -- start 2024-03-04T13:36:58.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:36:58.998+0000 7f08da800700 1 -- start start 2024-03-04T13:36:58.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:36:58.998+0000 7f08da800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f08d40fdf60 0x7f08d4113b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:36:58.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:36:58.998+0000 7f08da800700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08d40fe900 0x7f08d4114050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:36:58.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:36:58.999+0000 7f08da800700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08d4114730 con 0x7f08d40fdf60 2024-03-04T13:36:58.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:36:58.999+0000 7f08da800700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08d410d9d0 con 0x7f08d40fe900 2024-03-04T13:36:58.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:36:58.999+0000 7f08d3400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f08d40fdf60 0x7f08d4113b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:36:58.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:36:58.999+0000 7f08d3400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f08d40fdf60 0x7f08d4113b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42044/0 (socket says 172.21.15.107:42044) 2024-03-04T13:36:58.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:36:58.999+0000 7f08d3400700 1 -- 172.21.15.107:0/2677830462 learned_addr learned my addr 172.21.15.107:0/2677830462 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:36:58.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:36:58.999+0000 7f08d2a00700 1 --2- 172.21.15.107:0/2677830462 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08d40fe900 0x7f08d4114050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:36:58.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:36:58.999+0000 7f08d3400700 1 -- 172.21.15.107:0/2677830462 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08d40fe900 msgr2=0x7f08d4114050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:36:58.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:36:58.999+0000 7f08d3400700 1 --2- 172.21.15.107:0/2677830462 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08d40fe900 0x7f08d4114050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:36:58.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:36:58.999+0000 7f08d3400700 1 -- 172.21.15.107:0/2677830462 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08c0005040 con 0x7f08d40fdf60 2024-03-04T13:36:58.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:36:58.999+0000 7f08d3400700 1 --2- 172.21.15.107:0/2677830462 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f08d40fdf60 0x7f08d4113b10 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f08c400b790 tx=0x7f08c4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:36:58.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:36:59.000+0000 7f08cb400700 1 -- 172.21.15.107:0/2677830462 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f08c400f770 con 0x7f08d40fdf60 2024-03-04T13:36:58.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:36:59.000+0000 7f08cb400700 1 -- 172.21.15.107:0/2677830462 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f08c400fda0 con 0x7f08d40fdf60 2024-03-04T13:36:58.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:36:59.000+0000 7f08da800700 1 -- 172.21.15.107:0/2677830462 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08d410dcb0 con 0x7f08d40fdf60 2024-03-04T13:36:58.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:36:59.000+0000 7f08cb400700 1 -- 172.21.15.107:0/2677830462 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f08c400d310 con 0x7f08d40fdf60 2024-03-04T13:36:58.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:36:59.000+0000 7f08da800700 1 -- 172.21.15.107:0/2677830462 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08d410e1f0 con 0x7f08d40fdf60 2024-03-04T13:36:58.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:36:59.001+0000 7f08da800700 1 -- 172.21.15.107:0/2677830462 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08d4104550 con 0x7f08d40fdf60 2024-03-04T13:36:58.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:36:59.002+0000 7f08cb400700 1 -- 172.21.15.107:0/2677830462 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86623+0+0 (secure 0 0 0) 0x7f08c4010420 con 0x7f08d40fdf60 2024-03-04T13:36:58.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:36:59.003+0000 7f08cb400700 1 --2- 172.21.15.107:0/2677830462 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f08b8069380 0x7f08b806b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:36:58.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:36:59.003+0000 7f08cb400700 1 -- 172.21.15.107:0/2677830462 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f08c4086e20 con 0x7f08d40fdf60 2024-03-04T13:36:58.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:36:59.003+0000 7f08d2a00700 1 --2- 172.21.15.107:0/2677830462 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f08b8069380 0x7f08b806b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:36:58.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:36:59.004+0000 7f08d2a00700 1 --2- 172.21.15.107:0/2677830462 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f08b8069380 0x7f08b806b830 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f08c0000c00 tx=0x7f08c0009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:36:59.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:36:59.007+0000 7f08cb400700 1 -- 172.21.15.107:0/2677830462 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f08c4056f00 con 0x7f08d40fdf60 2024-03-04T13:36:59.103 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:58 smithi169 ceph-mon[36313]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-04T13:36:59.103 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:58 smithi169 ceph-mon[36313]: Detected new or changed devices on smithi169 2024-03-04T13:36:59.103 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:58 smithi169 ceph-mon[36313]: Adjusting osd_memory_target on smithi169 to 5313M 2024-03-04T13:36:59.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:36:59.215+0000 7f08da800700 1 -- 172.21.15.107:0/2677830462 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f08d404bed0 con 0x7f08d40fdf60 2024-03-04T13:36:59.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:36:59.217+0000 7f08cb400700 1 -- 172.21.15.107:0/2677830462 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+11122 (secure 0 0 0) 0x7f08c4056a90 con 0x7f08d40fdf60 2024-03-04T13:36:59.211 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:36:59.211 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":41,"fsid":"1fc09a58-da2b-11ee-95c6-87774f69a715","created":"2024-03-04T13:29:42.985888+0000","modified":"2024-03-04T13:36:54.674997+0000","last_up_change":"2024-03-04T13:36:53.661634+0000","last_in_change":"2024-03-04T13:36:31.424613+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T13:35:17.915858+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"c857e387-3d2b-478d-9807-24ea8b9fe02d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":3721776817},{"type":"v1","addr":"172.21.15.107:6803","nonce":3721776817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":3721776817},{"type":"v1","addr":"172.21.15.107:6805","nonce":3721776817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":3721776817},{"type":"v1","addr":"172.21.15.107:6809","nonce":3721776817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":3721776817},{"type":"v1","addr":"172.21.15.107:6807","nonce":3721776817}]},"public_addr":"172.21.15.107:6803/3721776817","cluster_addr":"172.21.15.107:6805/3721776817","heartbeat_back_addr":"172.21.15.107:6809/3721776817","heartbeat_front_addr":"172.21.15.107:6807/3721776817","state":["exists","up"]},{"osd":1,"uuid":"fa29dc8f-419b-4318-b43e-e13fffd87204","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":4224626475},{"type":"v1","addr":"172.21.15.107:6811","nonce":4224626475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":4224626475},{"type":"v1","addr":"172.21.15.107:6813","nonce":4224626475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4224626475},{"type":"v1","addr":"172.21.15.107:6817","nonce":4224626475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":4224626475},{"type":"v1","addr":"172.21.15.107:6815","nonce":4224626475}]},"public_addr":"172.21.15.107:6811/4224626475","cluster_addr":"172.21.15.107:6813/4224626475","heartbeat_back_addr":"172.21.15.107:6817/4224626475","heartbeat_front_addr":"172.21.15.107:6815/4224626475","state":["exists","up"]},{"osd":2,"uuid":"580a07f0-7bd4-4341-9c1f-b3383d64c720","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2331133561},{"type":"v1","addr":"172.21.15.107:6819","nonce":2331133561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2331133561},{"type":"v1","addr":"172.21.15.107:6821","nonce":2331133561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":2331133561},{"type":"v1","addr":"172.21.15.107:6825","nonce":2331133561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2331133561},{"type":"v1","addr":"172.21.15.107:6823","nonce":2331133561}]},"public_addr":"172.21.15.107:6819/2331133561","cluster_addr":"172.21.15.107:6821/2331133561","heartbeat_back_addr":"172.21.15.107:6825/2331133561","heartbeat_front_addr":"172.21.15.107:6823/2331133561","state":["exists","up"]},{"osd":3,"uuid":"7acac14d-998a-4651-b78c-e42584e13b43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2475630313},{"type":"v1","addr":"172.21.15.169:6801","nonce":2475630313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2475630313},{"type":"v1","addr":"172.21.15.169:6803","nonce":2475630313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2475630313},{"type":"v1","addr":"172.21.15.169:6807","nonce":2475630313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2475630313},{"type":"v1","addr":"172.21.15.169:6805","nonce":2475630313}]},"public_addr":"172.21.15.169:6801/2475630313","cluster_addr":"172.21.15.169:6803/2475630313","heartbeat_back_addr":"172.21.15.169:6807/2475630313","heartbeat_front_addr":"172.21.15.169:6805/2475630313","state":["exists","up"]},{"osd":4,"uuid":"01d2b77c-66ec-4b50-8170-43d95eeae471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":592652532},{"type":"v1","addr":"172.21.15.169:6809","nonce":592652532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":592652532},{"type":"v1","addr":"172.21.15.169:6811","nonce":592652532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":592652532},{"type":"v1","addr":"172.21.15.169:6815","nonce":592652532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":592652532},{"type":"v1","addr":"172.21.15.169:6813","nonce":592652532}]},"public_addr":"172.21.15.169:6809/592652532","cluster_addr":"172.21.15.169:6811/592652532","heartbeat_back_addr":"172.21.15.169:6815/592652532","heartbeat_front_addr":"172.21.15.169:6813/592652532","state":["exists","up"]},{"osd":5,"uuid":"0164ead7-c5d0-4541-bf22-f671558f8607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":182182184},{"type":"v1","addr":"172.21.15.169:6817","nonce":182182184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":182182184},{"type":"v1","addr":"172.21.15.169:6819","nonce":182182184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":182182184},{"type":"v1","addr":"172.21.15.169:6823","nonce":182182184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":182182184},{"type":"v1","addr":"172.21.15.169:6821","nonce":182182184}]},"public_addr":"172.21.15.169:6817/182182184","cluster_addr":"172.21.15.169:6819/182182184","heartbeat_back_addr":"172.21.15.169:6823/182182184","heartbeat_front_addr":"172.21.15.169:6821/182182184","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T13:33:59.729659+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T13:34:36.600326+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T13:35:14.068513+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T13:35:46.608007+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T13:36:19.595680+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T13:36:52.586933+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/3503768751":"2024-03-05T13:32:27.229455+0000","172.21.15.107:0/601795289":"2024-03-05T13:32:27.229455+0000","172.21.15.107:6801/2040330534":"2024-03-05T13:32:27.229455+0000","172.21.15.107:6800/2040330534":"2024-03-05T13:32:27.229455+0000","172.21.15.107:0/136342223":"2024-03-05T13:30:56.353578+0000","172.21.15.107:0/2314831039":"2024-03-05T13:30:56.353578+0000","172.21.15.107:6801/2371648646":"2024-03-05T13:30:56.353578+0000","172.21.15.107:6800/2371648646":"2024-03-05T13:30:56.353578+0000","172.21.15.107:0/2723084639":"2024-03-05T13:32:27.229455+0000","172.21.15.107:6801/1723390229":"2024-03-05T13:30:16.230256+0000","172.21.15.107:6800/1723390229":"2024-03-05T13:30:16.230256+0000","172.21.15.107:0/2928681569":"2024-03-05T13:30:56.353578+0000","172.21.15.107:0/1780539770":"2024-03-05T13:30:16.230256+0000","172.21.15.107:0/3451280196":"2024-03-05T13:30:16.230256+0000","172.21.15.107:0/2357638826":"2024-03-05T13:30:16.230256+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T13:36:59.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:36:59.220+0000 7f08da800700 1 -- 172.21.15.107:0/2677830462 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f08b8069380 msgr2=0x7f08b806b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:36:59.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:36:59.220+0000 7f08da800700 1 --2- 172.21.15.107:0/2677830462 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f08b8069380 0x7f08b806b830 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f08c0000c00 tx=0x7f08c0009e30 comp rx=0 tx=0).stop 2024-03-04T13:36:59.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:36:59.220+0000 7f08da800700 1 -- 172.21.15.107:0/2677830462 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f08d40fdf60 msgr2=0x7f08d4113b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:36:59.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:36:59.220+0000 7f08da800700 1 --2- 172.21.15.107:0/2677830462 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f08d40fdf60 0x7f08d4113b10 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f08c400b790 tx=0x7f08c4009250 comp rx=0 tx=0).stop 2024-03-04T13:36:59.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:36:59.220+0000 7f08da800700 1 -- 172.21.15.107:0/2677830462 shutdown_connections 2024-03-04T13:36:59.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:36:59.220+0000 7f08da800700 1 --2- 172.21.15.107:0/2677830462 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f08b8069380 0x7f08b806b830 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:36:59.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:36:59.220+0000 7f08da800700 1 --2- 172.21.15.107:0/2677830462 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f08d40fdf60 0x7f08d4113b10 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:36:59.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:36:59.220+0000 7f08da800700 1 --2- 172.21.15.107:0/2677830462 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f08d40fe900 0x7f08d4114050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:36:59.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:36:59.220+0000 7f08da800700 1 -- 172.21.15.107:0/2677830462 >> 172.21.15.107:0/2677830462 conn(0x7f08d40f7a10 msgr2=0x7f08d41016d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:36:59.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:36:59.220+0000 7f08da800700 1 -- 172.21.15.107:0/2677830462 shutdown_connections 2024-03-04T13:36:59.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:36:59.220+0000 7f08da800700 1 -- 172.21.15.107:0/2677830462 wait complete. 2024-03-04T13:37:00.016 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:36:59 smithi107 ceph-mon[28739]: from='client.? 172.21.15.107:0/2677830462' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T13:37:00.101 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-04T13:35:17.915858+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2024-03-04T13:37:00.102 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 -- ceph osd pool get .mgr pg_num 2024-03-04T13:37:00.106 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:36:59 smithi169 ceph-mon[36313]: from='client.? 172.21.15.107:0/2677830462' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T13:37:00.582 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/mon.smithi107/config 2024-03-04T13:37:01.046 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:37:00 smithi107 ceph-mon[28739]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-04T13:37:01.103 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:37:00 smithi169 ceph-mon[36313]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-04T13:37:02.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:02.541+0000 7f8549800700 1 -- 172.21.15.107:0/3986372448 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85440fc650 msgr2=0x7f85441093b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:37:02.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:02.541+0000 7f8549800700 1 --2- 172.21.15.107:0/3986372448 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85440fc650 0x7f85441093b0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8530004940 tx=0x7f8530009e30 comp rx=0 tx=0).stop 2024-03-04T13:37:02.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:02.542+0000 7f8549800700 1 -- 172.21.15.107:0/3986372448 shutdown_connections 2024-03-04T13:37:02.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:02.542+0000 7f8549800700 1 --2- 172.21.15.107:0/3986372448 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85440fc650 0x7f85441093b0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:02.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:02.542+0000 7f8549800700 1 --2- 172.21.15.107:0/3986372448 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f85440fbd40 0x7f85440fc110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:02.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:02.542+0000 7f8549800700 1 -- 172.21.15.107:0/3986372448 >> 172.21.15.107:0/3986372448 conn(0x7f85440f7a10 msgr2=0x7f85440f9e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:37:02.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:02.542+0000 7f8549800700 1 -- 172.21.15.107:0/3986372448 shutdown_connections 2024-03-04T13:37:02.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:02.542+0000 7f8549800700 1 -- 172.21.15.107:0/3986372448 wait complete. 2024-03-04T13:37:02.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:02.542+0000 7f8549800700 1 Processor -- start 2024-03-04T13:37:02.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:02.543+0000 7f8549800700 1 -- start start 2024-03-04T13:37:02.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:02.543+0000 7f8549800700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f85440fbd40 0x7f8544115cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:37:02.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:02.543+0000 7f8549800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85440fc650 0x7f8544116210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:37:02.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:02.543+0000 7f8549800700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8544116860 con 0x7f85440fc650 2024-03-04T13:37:02.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:02.543+0000 7f8549800700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f854410fb70 con 0x7f85440fbd40 2024-03-04T13:37:02.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:02.543+0000 7f8542000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85440fc650 0x7f8544116210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:37:02.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:02.543+0000 7f8542000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85440fc650 0x7f8544116210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42054/0 (socket says 172.21.15.107:42054) 2024-03-04T13:37:02.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:02.543+0000 7f8542000700 1 -- 172.21.15.107:0/3637037097 learned_addr learned my addr 172.21.15.107:0/3637037097 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:37:02.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:02.544+0000 7f8542a00700 1 --2- 172.21.15.107:0/3637037097 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f85440fbd40 0x7f8544115cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:37:02.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:02.544+0000 7f8542000700 1 -- 172.21.15.107:0/3637037097 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f85440fbd40 msgr2=0x7f8544115cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:37:02.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:02.544+0000 7f8542000700 1 --2- 172.21.15.107:0/3637037097 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f85440fbd40 0x7f8544115cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:02.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:02.544+0000 7f8542000700 1 -- 172.21.15.107:0/3637037097 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8530005040 con 0x7f85440fc650 2024-03-04T13:37:02.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:02.544+0000 7f8542000700 1 --2- 172.21.15.107:0/3637037097 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85440fc650 0x7f8544116210 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8530000c00 tx=0x7f8530009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:37:02.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:02.544+0000 7f853aa00700 1 -- 172.21.15.107:0/3637037097 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f853000e040 con 0x7f85440fc650 2024-03-04T13:37:02.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:02.544+0000 7f853aa00700 1 -- 172.21.15.107:0/3637037097 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f853000bb90 con 0x7f85440fc650 2024-03-04T13:37:02.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:02.544+0000 7f8549800700 1 -- 172.21.15.107:0/3637037097 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f854410fdf0 con 0x7f85440fc650 2024-03-04T13:37:02.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:02.544+0000 7f853aa00700 1 -- 172.21.15.107:0/3637037097 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f853001c630 con 0x7f85440fc650 2024-03-04T13:37:02.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:02.545+0000 7f8549800700 1 -- 172.21.15.107:0/3637037097 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8544110330 con 0x7f85440fc650 2024-03-04T13:37:02.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:02.546+0000 7f8549800700 1 -- 172.21.15.107:0/3637037097 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8544106ae0 con 0x7f85440fc650 2024-03-04T13:37:02.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:02.547+0000 7f853aa00700 1 -- 172.21.15.107:0/3637037097 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86623+0+0 (secure 0 0 0) 0x7f853000bd00 con 0x7f85440fc650 2024-03-04T13:37:02.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:02.548+0000 7f853aa00700 1 --2- 172.21.15.107:0/3637037097 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f85280692b0 0x7f852806b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:37:02.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:02.548+0000 7f853aa00700 1 -- 172.21.15.107:0/3637037097 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f8530087f20 con 0x7f85440fc650 2024-03-04T13:37:02.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:02.548+0000 7f8542a00700 1 --2- 172.21.15.107:0/3637037097 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f85280692b0 0x7f852806b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:37:02.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:02.549+0000 7f8542a00700 1 --2- 172.21.15.107:0/3637037097 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f85280692b0 0x7f852806b760 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8534007fd0 tx=0x7f8534009040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:37:02.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:02.553+0000 7f853aa00700 1 -- 172.21.15.107:0/3637037097 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f85300b9bd0 con 0x7f85440fc650 2024-03-04T13:37:02.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:02.757+0000 7f8549800700 1 -- 172.21.15.107:0/3637037097 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f85441112b0 con 0x7f85440fc650 2024-03-04T13:37:02.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:02.758+0000 7f853aa00700 1 -- 172.21.15.107:0/3637037097 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v41) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8530057fa0 con 0x7f85440fc650 2024-03-04T13:37:02.752 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 1 2024-03-04T13:37:02.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:02.760+0000 7f8549800700 1 -- 172.21.15.107:0/3637037097 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f85280692b0 msgr2=0x7f852806b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:37:02.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:02.760+0000 7f8549800700 1 --2- 172.21.15.107:0/3637037097 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f85280692b0 0x7f852806b760 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8534007fd0 tx=0x7f8534009040 comp rx=0 tx=0).stop 2024-03-04T13:37:02.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:02.760+0000 7f8549800700 1 -- 172.21.15.107:0/3637037097 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85440fc650 msgr2=0x7f8544116210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:37:02.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:02.760+0000 7f8549800700 1 --2- 172.21.15.107:0/3637037097 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85440fc650 0x7f8544116210 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8530000c00 tx=0x7f8530009e30 comp rx=0 tx=0).stop 2024-03-04T13:37:02.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:02.761+0000 7f8549800700 1 -- 172.21.15.107:0/3637037097 shutdown_connections 2024-03-04T13:37:02.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:02.761+0000 7f8549800700 1 --2- 172.21.15.107:0/3637037097 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f85280692b0 0x7f852806b760 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:02.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:02.761+0000 7f8549800700 1 --2- 172.21.15.107:0/3637037097 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f85440fbd40 0x7f8544115cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:02.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:02.761+0000 7f8549800700 1 --2- 172.21.15.107:0/3637037097 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85440fc650 0x7f8544116210 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:02.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:02.761+0000 7f8549800700 1 -- 172.21.15.107:0/3637037097 >> 172.21.15.107:0/3637037097 conn(0x7f85440f7a10 msgr2=0x7f85440f91c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:37:02.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:02.761+0000 7f8549800700 1 -- 172.21.15.107:0/3637037097 shutdown_connections 2024-03-04T13:37:02.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:02.761+0000 7f8549800700 1 -- 172.21.15.107:0/3637037097 wait complete. 2024-03-04T13:37:03.086 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:37:02 smithi107 ceph-mon[28739]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-04T13:37:03.103 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:37:02 smithi169 ceph-mon[36313]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-04T13:37:03.460 INFO:tasks.cephadm:Setting up client nodes... 2024-03-04T13:37:03.460 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-03-04T13:37:03.942 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/mon.smithi107/config 2024-03-04T13:37:04.086 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:37:03 smithi107 ceph-mon[28739]: from='client.? 172.21.15.107:0/3637037097' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-04T13:37:04.103 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:37:03 smithi169 ceph-mon[36313]: from='client.? 172.21.15.107:0/3637037097' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-04T13:37:05.086 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:37:04 smithi107 ceph-mon[28739]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-04T13:37:05.103 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:37:04 smithi169 ceph-mon[36313]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-04T13:37:06.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:06.018+0000 7f63dc400700 1 -- 172.21.15.107:0/1514500929 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63d4071eb0 msgr2=0x7f63d4070320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:37:06.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:06.018+0000 7f63dc400700 1 --2- 172.21.15.107:0/1514500929 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63d4071eb0 0x7f63d4070320 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f63c8004940 tx=0x7f63c8009e30 comp rx=0 tx=0).stop 2024-03-04T13:37:06.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:06.018+0000 7f63dc400700 1 -- 172.21.15.107:0/1514500929 shutdown_connections 2024-03-04T13:37:06.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:06.018+0000 7f63dc400700 1 --2- 172.21.15.107:0/1514500929 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63d4070860 0x7f63d4070cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:06.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:06.018+0000 7f63dc400700 1 --2- 172.21.15.107:0/1514500929 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63d4071eb0 0x7f63d4070320 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:06.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:06.018+0000 7f63dc400700 1 -- 172.21.15.107:0/1514500929 >> 172.21.15.107:0/1514500929 conn(0x7f63d40f8620 msgr2=0x7f63d40faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:37:06.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:06.018+0000 7f63dc400700 1 -- 172.21.15.107:0/1514500929 shutdown_connections 2024-03-04T13:37:06.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:06.018+0000 7f63dc400700 1 -- 172.21.15.107:0/1514500929 wait complete. 2024-03-04T13:37:06.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:06.019+0000 7f63dc400700 1 Processor -- start 2024-03-04T13:37:06.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:06.019+0000 7f63dc400700 1 -- start start 2024-03-04T13:37:06.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:06.020+0000 7f63dc400700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63d4070860 0x7f63d410fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:37:06.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:06.020+0000 7f63dc400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63d4071eb0 0x7f63d4110080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:37:06.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:06.020+0000 7f63dc400700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d4110680 con 0x7f63d4071eb0 2024-03-04T13:37:06.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:06.020+0000 7f63dc400700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d4110f20 con 0x7f63d4070860 2024-03-04T13:37:06.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:06.020+0000 7f63d9000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63d4071eb0 0x7f63d4110080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:37:06.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:06.020+0000 7f63d9000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63d4071eb0 0x7f63d4110080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42072/0 (socket says 172.21.15.107:42072) 2024-03-04T13:37:06.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:06.020+0000 7f63d9000700 1 -- 172.21.15.107:0/1955308558 learned_addr learned my addr 172.21.15.107:0/1955308558 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:37:06.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:06.020+0000 7f63d9a00700 1 --2- 172.21.15.107:0/1955308558 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63d4070860 0x7f63d410fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:37:06.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:06.020+0000 7f63d9000700 1 -- 172.21.15.107:0/1955308558 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63d4070860 msgr2=0x7f63d410fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:37:06.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:06.020+0000 7f63d9000700 1 --2- 172.21.15.107:0/1955308558 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63d4070860 0x7f63d410fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:06.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:06.020+0000 7f63d9000700 1 -- 172.21.15.107:0/1955308558 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63c8005040 con 0x7f63d4071eb0 2024-03-04T13:37:06.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:06.020+0000 7f63d9000700 1 --2- 172.21.15.107:0/1955308558 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63d4071eb0 0x7f63d4110080 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f63c0002a50 tx=0x7f63c00046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:37:06.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:06.021+0000 7f63d1600700 1 -- 172.21.15.107:0/1955308558 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f63c0007cc0 con 0x7f63d4071eb0 2024-03-04T13:37:06.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:06.021+0000 7f63d1600700 1 -- 172.21.15.107:0/1955308558 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f63c000f040 con 0x7f63d4071eb0 2024-03-04T13:37:06.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:06.021+0000 7f63dc400700 1 -- 172.21.15.107:0/1955308558 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63d4111200 con 0x7f63d4071eb0 2024-03-04T13:37:06.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:06.021+0000 7f63d1600700 1 -- 172.21.15.107:0/1955308558 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f63c0009640 con 0x7f63d4071eb0 2024-03-04T13:37:06.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:06.021+0000 7f63dc400700 1 -- 172.21.15.107:0/1955308558 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63d4111740 con 0x7f63d4071eb0 2024-03-04T13:37:06.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:06.022+0000 7f63dc400700 1 -- 172.21.15.107:0/1955308558 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63d404bed0 con 0x7f63d4071eb0 2024-03-04T13:37:06.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:06.023+0000 7f63d1600700 1 -- 172.21.15.107:0/1955308558 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86623+0+0 (secure 0 0 0) 0x7f63c00097a0 con 0x7f63d4071eb0 2024-03-04T13:37:06.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:06.025+0000 7f63d1600700 1 --2- 172.21.15.107:0/1955308558 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f63bc0692a0 0x7f63bc06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:37:06.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:06.025+0000 7f63d1600700 1 -- 172.21.15.107:0/1955308558 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f63c0086d20 con 0x7f63d4071eb0 2024-03-04T13:37:06.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:06.025+0000 7f63d9a00700 1 --2- 172.21.15.107:0/1955308558 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f63bc0692a0 0x7f63bc06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:37:06.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:06.026+0000 7f63d9a00700 1 --2- 172.21.15.107:0/1955308558 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f63bc0692a0 0x7f63bc06b750 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f63c8004f90 tx=0x7f63c8009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:37:06.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:06.029+0000 7f63d1600700 1 -- 172.21.15.107:0/1955308558 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f63c0056d00 con 0x7f63d4071eb0 2024-03-04T13:37:06.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:06.316+0000 7f63dc400700 1 -- 172.21.15.107:0/1955308558 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f63d4107c00 con 0x7f63d4071eb0 2024-03-04T13:37:06.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:06.325+0000 7f63d1600700 1 -- 172.21.15.107:0/1955308558 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v14) v1 ==== 170+0+59 (secure 0 0 0) 0x7f63c0056890 con 0x7f63d4071eb0 2024-03-04T13:37:06.319 INFO:teuthology.orchestra.run.smithi107.stdout:[client.0] 2024-03-04T13:37:06.319 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQCCzuVlXncCExAAPYjDQISXLzUyZP7JjWPRsA== 2024-03-04T13:37:06.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:06.328+0000 7f63dc400700 1 -- 172.21.15.107:0/1955308558 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f63bc0692a0 msgr2=0x7f63bc06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:37:06.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:06.328+0000 7f63dc400700 1 --2- 172.21.15.107:0/1955308558 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f63bc0692a0 0x7f63bc06b750 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f63c8004f90 tx=0x7f63c8009e30 comp rx=0 tx=0).stop 2024-03-04T13:37:06.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:06.328+0000 7f63dc400700 1 -- 172.21.15.107:0/1955308558 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63d4071eb0 msgr2=0x7f63d4110080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:37:06.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:06.328+0000 7f63dc400700 1 --2- 172.21.15.107:0/1955308558 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63d4071eb0 0x7f63d4110080 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f63c0002a50 tx=0x7f63c00046b0 comp rx=0 tx=0).stop 2024-03-04T13:37:06.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:06.328+0000 7f63dc400700 1 -- 172.21.15.107:0/1955308558 shutdown_connections 2024-03-04T13:37:06.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:06.328+0000 7f63dc400700 1 --2- 172.21.15.107:0/1955308558 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f63bc0692a0 0x7f63bc06b750 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:06.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:06.328+0000 7f63dc400700 1 --2- 172.21.15.107:0/1955308558 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63d4070860 0x7f63d410fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:06.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:06.328+0000 7f63dc400700 1 --2- 172.21.15.107:0/1955308558 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63d4071eb0 0x7f63d4110080 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:06.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:06.328+0000 7f63dc400700 1 -- 172.21.15.107:0/1955308558 >> 172.21.15.107:0/1955308558 conn(0x7f63d40f8620 msgr2=0x7f63d4102e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:37:06.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:06.329+0000 7f63dc400700 1 -- 172.21.15.107:0/1955308558 shutdown_connections 2024-03-04T13:37:06.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:06.329+0000 7f63dc400700 1 -- 172.21.15.107:0/1955308558 wait complete. 2024-03-04T13:37:07.020 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:37:06 smithi107 ceph-mon[28739]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-04T13:37:07.020 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:37:06 smithi107 ceph-mon[28739]: from='client.? 172.21.15.107:0/1955308558' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-04T13:37:07.021 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:37:06 smithi107 ceph-mon[28739]: from='client.? 172.21.15.107:0/1955308558' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-04T13:37:07.101 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-03-04T13:37:07.101 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-03-04T13:37:07.101 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-04T13:37:07.103 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:37:06 smithi169 ceph-mon[36313]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-04T13:37:07.103 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:37:06 smithi169 ceph-mon[36313]: from='client.? 172.21.15.107:0/1955308558' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-04T13:37:07.103 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:37:06 smithi169 ceph-mon[36313]: from='client.? 172.21.15.107:0/1955308558' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-04T13:37:07.247 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-03-04T13:37:07.726 INFO:teuthology.orchestra.run.smithi169.stderr:Inferring config /var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/mon.smithi169/config 2024-03-04T13:37:09.075 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:37:08 smithi169 ceph-mon[36313]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-04T13:37:09.086 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:37:08 smithi107 ceph-mon[28739]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-04T13:37:09.819 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:37:09.825+0000 7fbe2f600700 1 -- 172.21.15.169:0/2298219249 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe28071f10 msgr2=0x7fbe28070380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:37:09.819 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:37:09.825+0000 7fbe2f600700 1 --2- 172.21.15.169:0/2298219249 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe28071f10 0x7fbe28070380 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fbe18004940 tx=0x7fbe18009e30 comp rx=0 tx=0).stop 2024-03-04T13:37:09.819 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:37:09.825+0000 7fbe2f600700 1 -- 172.21.15.169:0/2298219249 shutdown_connections 2024-03-04T13:37:09.819 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:37:09.825+0000 7fbe2f600700 1 --2- 172.21.15.169:0/2298219249 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe280708c0 0x7fbe28070d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:09.819 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:37:09.825+0000 7fbe2f600700 1 --2- 172.21.15.169:0/2298219249 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe28071f10 0x7fbe28070380 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:09.819 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:37:09.825+0000 7fbe2f600700 1 -- 172.21.15.169:0/2298219249 >> 172.21.15.169:0/2298219249 conn(0x7fbe280f8620 msgr2=0x7fbe280faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:37:09.819 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:37:09.825+0000 7fbe2f600700 1 -- 172.21.15.169:0/2298219249 shutdown_connections 2024-03-04T13:37:09.819 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:37:09.825+0000 7fbe2f600700 1 -- 172.21.15.169:0/2298219249 wait complete. 2024-03-04T13:37:09.820 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:37:09.826+0000 7fbe2f600700 1 Processor -- start 2024-03-04T13:37:09.820 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:37:09.826+0000 7fbe2f600700 1 -- start start 2024-03-04T13:37:09.821 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:37:09.826+0000 7fbe2f600700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe280708c0 0x7fbe2810fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:37:09.821 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:37:09.827+0000 7fbe2f600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe28071f10 0x7fbe28110040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:37:09.821 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:37:09.827+0000 7fbe2f600700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe28110640 con 0x7fbe28071f10 2024-03-04T13:37:09.821 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:37:09.827+0000 7fbe2f600700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe28110ee0 con 0x7fbe280708c0 2024-03-04T13:37:09.821 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:37:09.827+0000 7fbe2cc00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe280708c0 0x7fbe2810fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:37:09.821 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:37:09.827+0000 7fbe2cc00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe280708c0 0x7fbe2810fb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:54852/0 (socket says 172.21.15.169:54852) 2024-03-04T13:37:09.821 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:37:09.827+0000 7fbe2cc00700 1 -- 172.21.15.169:0/3825511192 learned_addr learned my addr 172.21.15.169:0/3825511192 (peer_addr_for_me v2:172.21.15.169:0/0) 2024-03-04T13:37:09.822 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:37:09.827+0000 7fbe2cc00700 1 -- 172.21.15.169:0/3825511192 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe28071f10 msgr2=0x7fbe28110040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:37:09.822 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:37:09.827+0000 7fbe1fe00700 1 --2- 172.21.15.169:0/3825511192 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe28071f10 0x7fbe28110040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:37:09.822 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:37:09.827+0000 7fbe2cc00700 1 --2- 172.21.15.169:0/3825511192 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe28071f10 0x7fbe28110040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:09.822 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:37:09.827+0000 7fbe2cc00700 1 -- 172.21.15.169:0/3825511192 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe18005040 con 0x7fbe280708c0 2024-03-04T13:37:09.822 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:37:09.827+0000 7fbe2cc00700 1 --2- 172.21.15.169:0/3825511192 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe280708c0 0x7fbe2810fb00 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fbe18009040 tx=0x7fbe18009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:37:09.822 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:37:09.828+0000 7fbe25600700 1 -- 172.21.15.169:0/3825511192 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe1800e040 con 0x7fbe280708c0 2024-03-04T13:37:09.822 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:37:09.828+0000 7fbe25600700 1 -- 172.21.15.169:0/3825511192 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbe1800bd40 con 0x7fbe280708c0 2024-03-04T13:37:09.822 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:37:09.828+0000 7fbe2f600700 1 -- 172.21.15.169:0/3825511192 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe28111160 con 0x7fbe280708c0 2024-03-04T13:37:09.822 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:37:09.828+0000 7fbe25600700 1 -- 172.21.15.169:0/3825511192 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe1801c6c0 con 0x7fbe280708c0 2024-03-04T13:37:09.822 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:37:09.828+0000 7fbe2f600700 1 -- 172.21.15.169:0/3825511192 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe281116a0 con 0x7fbe280708c0 2024-03-04T13:37:09.823 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:37:09.829+0000 7fbe2f600700 1 -- 172.21.15.169:0/3825511192 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe2804bed0 con 0x7fbe280708c0 2024-03-04T13:37:09.823 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:37:09.830+0000 7fbe25600700 1 -- 172.21.15.169:0/3825511192 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 19) v1 ==== 86623+0+0 (secure 0 0 0) 0x7fbe18013420 con 0x7fbe280708c0 2024-03-04T13:37:09.824 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:37:09.831+0000 7fbe25600700 1 --2- 172.21.15.169:0/3825511192 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fbe0c0692b0 0x7fbe0c06b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:37:09.824 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:37:09.831+0000 7fbe25600700 1 -- 172.21.15.169:0/3825511192 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fbe180882a0 con 0x7fbe280708c0 2024-03-04T13:37:09.824 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:37:09.831+0000 7fbe1fe00700 1 --2- 172.21.15.169:0/3825511192 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fbe0c0692b0 0x7fbe0c06b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:37:09.825 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:37:09.832+0000 7fbe1fe00700 1 --2- 172.21.15.169:0/3825511192 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fbe0c0692b0 0x7fbe0c06b760 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fbe10004d60 tx=0x7fbe1000b040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:37:09.828 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:37:09.836+0000 7fbe25600700 1 -- 172.21.15.169:0/3825511192 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fbe18058440 con 0x7fbe280708c0 2024-03-04T13:37:10.116 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:37:10.123+0000 7fbe2f600700 1 -- 172.21.15.169:0/3825511192 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7fbe28111b50 con 0x7fbe280708c0 2024-03-04T13:37:10.128 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:37:10.135+0000 7fbe25600700 1 -- 172.21.15.169:0/3825511192 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v15) v1 ==== 170+0+59 (secure 0 0 0) 0x7fbe18057fd0 con 0x7fbe280708c0 2024-03-04T13:37:10.128 INFO:teuthology.orchestra.run.smithi169.stdout:[client.1] 2024-03-04T13:37:10.128 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQCGzuVlueiIBxAAW/dad6RTkLJmuHbzwJMuYw== 2024-03-04T13:37:10.130 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:37:10.137+0000 7fbe2f600700 1 -- 172.21.15.169:0/3825511192 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fbe0c0692b0 msgr2=0x7fbe0c06b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:37:10.130 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:37:10.137+0000 7fbe2f600700 1 --2- 172.21.15.169:0/3825511192 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fbe0c0692b0 0x7fbe0c06b760 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fbe10004d60 tx=0x7fbe1000b040 comp rx=0 tx=0).stop 2024-03-04T13:37:10.130 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:37:10.137+0000 7fbe2f600700 1 -- 172.21.15.169:0/3825511192 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe280708c0 msgr2=0x7fbe2810fb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:37:10.131 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:37:10.137+0000 7fbe2f600700 1 --2- 172.21.15.169:0/3825511192 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe280708c0 0x7fbe2810fb00 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fbe18009040 tx=0x7fbe18009e30 comp rx=0 tx=0).stop 2024-03-04T13:37:10.131 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:37:10.138+0000 7fbe2f600700 1 -- 172.21.15.169:0/3825511192 shutdown_connections 2024-03-04T13:37:10.131 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:37:10.138+0000 7fbe2f600700 1 --2- 172.21.15.169:0/3825511192 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fbe0c0692b0 0x7fbe0c06b760 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:10.131 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:37:10.138+0000 7fbe2f600700 1 --2- 172.21.15.169:0/3825511192 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe280708c0 0x7fbe2810fb00 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:10.131 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:37:10.138+0000 7fbe2f600700 1 --2- 172.21.15.169:0/3825511192 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe28071f10 0x7fbe28110040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:10.131 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:37:10.138+0000 7fbe2f600700 1 -- 172.21.15.169:0/3825511192 >> 172.21.15.169:0/3825511192 conn(0x7fbe280f8620 msgr2=0x7fbe28102dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:37:10.131 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:37:10.138+0000 7fbe2f600700 1 -- 172.21.15.169:0/3825511192 shutdown_connections 2024-03-04T13:37:10.131 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:37:10.138+0000 7fbe2f600700 1 -- 172.21.15.169:0/3825511192 wait complete. 2024-03-04T13:37:10.913 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-03-04T13:37:10.914 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-03-04T13:37:10.914 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-03-04T13:37:11.038 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:37:10 smithi169 ceph-mon[36313]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-04T13:37:11.039 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:37:10 smithi169 ceph-mon[36313]: from='client.? 172.21.15.169:0/3825511192' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-04T13:37:11.039 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:37:10 smithi169 ceph-mon[36313]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-04T13:37:11.039 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:37:10 smithi169 ceph-mon[36313]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-04T13:37:11.054 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-04T13:37:11.055 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-03-04T13:37:11.055 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 -- ceph mgr dump --format=json 2024-03-04T13:37:11.086 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:37:10 smithi107 ceph-mon[28739]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-04T13:37:11.086 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:37:10 smithi107 ceph-mon[28739]: from='client.? 172.21.15.169:0/3825511192' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-04T13:37:11.086 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:37:10 smithi107 ceph-mon[28739]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-04T13:37:11.086 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:37:10 smithi107 ceph-mon[28739]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-04T13:37:11.537 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/mon.smithi107/config 2024-03-04T13:37:12.836 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:37:12 smithi107 ceph-mon[28739]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-04T13:37:12.836 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:37:12 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:37:13.103 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:37:12 smithi169 ceph-mon[36313]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-04T13:37:13.103 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:37:12 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:37:13.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:13.641+0000 7fb49f600700 1 -- 172.21.15.107:0/234514376 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4980fe8f0 msgr2=0x7fb498106f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:37:13.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:13.641+0000 7fb49f600700 1 --2- 172.21.15.107:0/234514376 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4980fe8f0 0x7fb498106f30 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb484004940 tx=0x7fb484009e30 comp rx=0 tx=0).stop 2024-03-04T13:37:13.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:13.641+0000 7fb49f600700 1 -- 172.21.15.107:0/234514376 shutdown_connections 2024-03-04T13:37:13.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:13.641+0000 7fb49f600700 1 --2- 172.21.15.107:0/234514376 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4980fe8f0 0x7fb498106f30 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:13.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:13.641+0000 7fb49f600700 1 --2- 172.21.15.107:0/234514376 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4980fdf50 0x7fb4980fe320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:13.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:13.641+0000 7fb49f600700 1 -- 172.21.15.107:0/234514376 >> 172.21.15.107:0/234514376 conn(0x7fb4980f79e0 msgr2=0x7fb4980f9df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:37:13.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:13.641+0000 7fb49f600700 1 -- 172.21.15.107:0/234514376 shutdown_connections 2024-03-04T13:37:13.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:13.641+0000 7fb49f600700 1 -- 172.21.15.107:0/234514376 wait complete. 2024-03-04T13:37:13.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:13.642+0000 7fb49f600700 1 Processor -- start 2024-03-04T13:37:13.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:13.643+0000 7fb49f600700 1 -- start start 2024-03-04T13:37:13.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:13.643+0000 7fb49f600700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4980fdf50 0x7fb498115bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:37:13.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:13.643+0000 7fb49f600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4980fe8f0 0x7fb498116130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:37:13.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:13.643+0000 7fb49f600700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb498116780 con 0x7fb4980fe8f0 2024-03-04T13:37:13.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:13.643+0000 7fb49f600700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb49810fad0 con 0x7fb4980fdf50 2024-03-04T13:37:13.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:13.643+0000 7fb497e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4980fe8f0 0x7fb498116130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:37:13.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:13.643+0000 7fb497e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4980fe8f0 0x7fb498116130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47784/0 (socket says 172.21.15.107:47784) 2024-03-04T13:37:13.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:13.643+0000 7fb497e00700 1 -- 172.21.15.107:0/2320987338 learned_addr learned my addr 172.21.15.107:0/2320987338 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:37:13.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:13.644+0000 7fb49cc00700 1 --2- 172.21.15.107:0/2320987338 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4980fdf50 0x7fb498115bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:37:13.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:13.644+0000 7fb497e00700 1 -- 172.21.15.107:0/2320987338 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4980fdf50 msgr2=0x7fb498115bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:37:13.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:13.644+0000 7fb497e00700 1 --2- 172.21.15.107:0/2320987338 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4980fdf50 0x7fb498115bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:13.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:13.644+0000 7fb497e00700 1 -- 172.21.15.107:0/2320987338 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb484005040 con 0x7fb4980fe8f0 2024-03-04T13:37:13.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:13.644+0000 7fb497e00700 1 --2- 172.21.15.107:0/2320987338 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4980fe8f0 0x7fb498116130 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb484009180 tx=0x7fb484009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:37:13.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:13.644+0000 7fb494c00700 1 -- 172.21.15.107:0/2320987338 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb48400e040 con 0x7fb4980fe8f0 2024-03-04T13:37:13.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:13.644+0000 7fb494c00700 1 -- 172.21.15.107:0/2320987338 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb484004290 con 0x7fb4980fe8f0 2024-03-04T13:37:13.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:13.644+0000 7fb494c00700 1 -- 172.21.15.107:0/2320987338 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb48401c630 con 0x7fb4980fe8f0 2024-03-04T13:37:13.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:13.644+0000 7fb49f600700 1 -- 172.21.15.107:0/2320987338 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb49810fd50 con 0x7fb4980fe8f0 2024-03-04T13:37:13.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:13.645+0000 7fb49f600700 1 -- 172.21.15.107:0/2320987338 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb498110240 con 0x7fb4980fe8f0 2024-03-04T13:37:13.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:13.646+0000 7fb49f600700 1 -- 172.21.15.107:0/2320987338 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4981069a0 con 0x7fb4980fe8f0 2024-03-04T13:37:13.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:13.647+0000 7fb494c00700 1 -- 172.21.15.107:0/2320987338 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86623+0+0 (secure 0 0 0) 0x7fb4840139f0 con 0x7fb4980fe8f0 2024-03-04T13:37:13.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:13.647+0000 7fb494c00700 1 --2- 172.21.15.107:0/2320987338 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fb4800692b0 0x7fb48006b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:37:13.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:13.648+0000 7fb494c00700 1 -- 172.21.15.107:0/2320987338 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fb4840883b0 con 0x7fb4980fe8f0 2024-03-04T13:37:13.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:13.648+0000 7fb49cc00700 1 --2- 172.21.15.107:0/2320987338 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fb4800692b0 0x7fb48006b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:37:13.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:13.648+0000 7fb49cc00700 1 --2- 172.21.15.107:0/2320987338 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fb4800692b0 0x7fb48006b760 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb48c004990 tx=0x7fb48c009ee0 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:37:13.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:13.652+0000 7fb494c00700 1 -- 172.21.15.107:0/2320987338 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fb484054980 con 0x7fb4980fe8f0 2024-03-04T13:37:13.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:13.913+0000 7fb49f600700 1 -- 172.21.15.107:0/2320987338 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fb49804bed0 con 0x7fb4980fe8f0 2024-03-04T13:37:13.913 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:13.919+0000 7fb494c00700 1 -- 172.21.15.107:0/2320987338 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v19) v1 ==== 74+0+167179 (secure 0 0 0) 0x7fb484004db0 con 0x7fb4980fe8f0 2024-03-04T13:37:13.917 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:37:13.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:13.923+0000 7fb49f600700 1 -- 172.21.15.107:0/2320987338 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fb4800692b0 msgr2=0x7fb48006b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:37:13.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:13.923+0000 7fb49f600700 1 --2- 172.21.15.107:0/2320987338 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fb4800692b0 0x7fb48006b760 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb48c004990 tx=0x7fb48c009ee0 comp rx=0 tx=0).stop 2024-03-04T13:37:13.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:13.923+0000 7fb49f600700 1 -- 172.21.15.107:0/2320987338 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4980fe8f0 msgr2=0x7fb498116130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:37:13.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:13.923+0000 7fb49f600700 1 --2- 172.21.15.107:0/2320987338 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4980fe8f0 0x7fb498116130 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb484009180 tx=0x7fb484009e30 comp rx=0 tx=0).stop 2024-03-04T13:37:13.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:13.924+0000 7fb49f600700 1 -- 172.21.15.107:0/2320987338 shutdown_connections 2024-03-04T13:37:13.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:13.924+0000 7fb49f600700 1 --2- 172.21.15.107:0/2320987338 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fb4800692b0 0x7fb48006b760 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:13.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:13.924+0000 7fb49f600700 1 --2- 172.21.15.107:0/2320987338 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4980fdf50 0x7fb498115bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:13.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:13.924+0000 7fb49f600700 1 --2- 172.21.15.107:0/2320987338 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4980fe8f0 0x7fb498116130 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:13.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:13.924+0000 7fb49f600700 1 -- 172.21.15.107:0/2320987338 >> 172.21.15.107:0/2320987338 conn(0x7fb4980f79e0 msgr2=0x7fb4980fc540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:37:13.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:13.924+0000 7fb49f600700 1 -- 172.21.15.107:0/2320987338 shutdown_connections 2024-03-04T13:37:13.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:13.924+0000 7fb49f600700 1 -- 172.21.15.107:0/2320987338 wait complete. 2024-03-04T13:37:14.783 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":19,"active_gid":14212,"active_name":"smithi107.wpzgwl","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":3848495603},{"type":"v1","addr":"172.21.15.107:6801","nonce":3848495603}]},"active_addr":"172.21.15.107:6801/3848495603","active_change":"2024-03-04T13:32:27.229683+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":14234,"name":"smithi169.eihczr","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.107:8443/","prometheus":"http://172.21.15.107:9283/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":5,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.107:0","nonce":3355652811}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.107:0","nonce":395365799}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.107:0","nonce":2605062903}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.107:0","nonce":2217624526}]}]} 2024-03-04T13:37:14.787 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-03-04T13:37:14.787 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-03-04T13:37:14.787 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 -- ceph osd dump --format=json 2024-03-04T13:37:15.086 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:37:14 smithi107 ceph-mon[28739]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-04T13:37:15.086 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:37:14 smithi107 ceph-mon[28739]: from='client.? 172.21.15.107:0/2320987338' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-04T13:37:15.103 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:37:14 smithi169 ceph-mon[36313]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-04T13:37:15.103 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:37:14 smithi169 ceph-mon[36313]: from='client.? 172.21.15.107:0/2320987338' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-04T13:37:15.301 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/mon.smithi107/config 2024-03-04T13:37:17.047 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:37:16 smithi107 ceph-mon[28739]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-04T13:37:17.103 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:37:16 smithi169 ceph-mon[36313]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-04T13:37:17.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:17.513+0000 7f9b78000700 1 -- 172.21.15.107:0/2352690695 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b701095f0 msgr2=0x7f9b701099c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:37:17.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:17.513+0000 7f9b78000700 1 --2- 172.21.15.107:0/2352690695 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b701095f0 0x7f9b701099c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9b64004940 tx=0x7f9b64009e30 comp rx=0 tx=0).stop 2024-03-04T13:37:17.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:17.513+0000 7f9b78000700 1 -- 172.21.15.107:0/2352690695 shutdown_connections 2024-03-04T13:37:17.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:17.513+0000 7f9b78000700 1 --2- 172.21.15.107:0/2352690695 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b70103dc0 0x7f9b70104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:17.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:17.513+0000 7f9b78000700 1 --2- 172.21.15.107:0/2352690695 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b701095f0 0x7f9b701099c0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:17.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:17.513+0000 7f9b78000700 1 -- 172.21.15.107:0/2352690695 >> 172.21.15.107:0/2352690695 conn(0x7f9b70072d50 msgr2=0x7f9b70075160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:37:17.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:17.513+0000 7f9b78000700 1 -- 172.21.15.107:0/2352690695 shutdown_connections 2024-03-04T13:37:17.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:17.513+0000 7f9b78000700 1 -- 172.21.15.107:0/2352690695 wait complete. 2024-03-04T13:37:17.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:17.514+0000 7f9b78000700 1 Processor -- start 2024-03-04T13:37:17.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:17.514+0000 7f9b78000700 1 -- start start 2024-03-04T13:37:17.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:17.515+0000 7f9b78000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b70103dc0 0x7f9b7011a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:37:17.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:17.515+0000 7f9b78000700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b701095f0 0x7f9b7011a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:37:17.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:17.515+0000 7f9b78000700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b7011ac70 con 0x7f9b70103dc0 2024-03-04T13:37:17.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:17.515+0000 7f9b78000700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b70113f80 con 0x7f9b701095f0 2024-03-04T13:37:17.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:17.515+0000 7f9b75600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b70103dc0 0x7f9b7011a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:37:17.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:17.515+0000 7f9b75600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b70103dc0 0x7f9b7011a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47808/0 (socket says 172.21.15.107:47808) 2024-03-04T13:37:17.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:17.515+0000 7f9b75600700 1 -- 172.21.15.107:0/615580096 learned_addr learned my addr 172.21.15.107:0/615580096 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:37:17.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:17.515+0000 7f9b74c00700 1 --2- 172.21.15.107:0/615580096 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b701095f0 0x7f9b7011a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:37:17.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:17.515+0000 7f9b75600700 1 -- 172.21.15.107:0/615580096 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b701095f0 msgr2=0x7f9b7011a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:37:17.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:17.515+0000 7f9b75600700 1 --2- 172.21.15.107:0/615580096 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b701095f0 0x7f9b7011a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:17.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:17.515+0000 7f9b75600700 1 -- 172.21.15.107:0/615580096 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b64005040 con 0x7f9b70103dc0 2024-03-04T13:37:17.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:17.515+0000 7f9b74c00700 1 --2- 172.21.15.107:0/615580096 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b701095f0 0x7f9b7011a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T13:37:17.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:17.516+0000 7f9b75600700 1 --2- 172.21.15.107:0/615580096 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b70103dc0 0x7f9b7011a0e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9b6400b3b0 tx=0x7f9b64009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:37:17.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:17.516+0000 7f9b6d600700 1 -- 172.21.15.107:0/615580096 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b6400e040 con 0x7f9b70103dc0 2024-03-04T13:37:17.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:17.516+0000 7f9b6d600700 1 -- 172.21.15.107:0/615580096 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9b6400bbf0 con 0x7f9b70103dc0 2024-03-04T13:37:17.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:17.516+0000 7f9b78000700 1 -- 172.21.15.107:0/615580096 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b70114200 con 0x7f9b70103dc0 2024-03-04T13:37:17.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:17.516+0000 7f9b6d600700 1 -- 172.21.15.107:0/615580096 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b6401c630 con 0x7f9b70103dc0 2024-03-04T13:37:17.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:17.516+0000 7f9b78000700 1 -- 172.21.15.107:0/615580096 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b70114740 con 0x7f9b70103dc0 2024-03-04T13:37:17.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:17.517+0000 7f9b78000700 1 -- 172.21.15.107:0/615580096 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b70106850 con 0x7f9b70103dc0 2024-03-04T13:37:17.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:17.518+0000 7f9b6d600700 1 -- 172.21.15.107:0/615580096 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86623+0+0 (secure 0 0 0) 0x7f9b6400bd60 con 0x7f9b70103dc0 2024-03-04T13:37:17.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:17.519+0000 7f9b6d600700 1 --2- 172.21.15.107:0/615580096 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f9b580692a0 0x7f9b5806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:37:17.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:17.519+0000 7f9b6d600700 1 -- 172.21.15.107:0/615580096 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f9b64088170 con 0x7f9b70103dc0 2024-03-04T13:37:17.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:17.520+0000 7f9b74c00700 1 --2- 172.21.15.107:0/615580096 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f9b580692a0 0x7f9b5806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:37:17.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:17.520+0000 7f9b74c00700 1 --2- 172.21.15.107:0/615580096 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f9b580692a0 0x7f9b5806b750 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9b5c009fd0 tx=0x7f9b5c00b040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:37:17.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:17.524+0000 7f9b6d600700 1 -- 172.21.15.107:0/615580096 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f9b64058310 con 0x7f9b70103dc0 2024-03-04T13:37:17.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:17.735+0000 7f9b78000700 1 -- 172.21.15.107:0/615580096 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9b7004bed0 con 0x7f9b70103dc0 2024-03-04T13:37:17.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:17.736+0000 7f9b6d600700 1 -- 172.21.15.107:0/615580096 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+11122 (secure 0 0 0) 0x7f9b64057ea0 con 0x7f9b70103dc0 2024-03-04T13:37:17.729 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:37:17.730 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":41,"fsid":"1fc09a58-da2b-11ee-95c6-87774f69a715","created":"2024-03-04T13:29:42.985888+0000","modified":"2024-03-04T13:36:54.674997+0000","last_up_change":"2024-03-04T13:36:53.661634+0000","last_in_change":"2024-03-04T13:36:31.424613+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T13:35:17.915858+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"c857e387-3d2b-478d-9807-24ea8b9fe02d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":3721776817},{"type":"v1","addr":"172.21.15.107:6803","nonce":3721776817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":3721776817},{"type":"v1","addr":"172.21.15.107:6805","nonce":3721776817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":3721776817},{"type":"v1","addr":"172.21.15.107:6809","nonce":3721776817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":3721776817},{"type":"v1","addr":"172.21.15.107:6807","nonce":3721776817}]},"public_addr":"172.21.15.107:6803/3721776817","cluster_addr":"172.21.15.107:6805/3721776817","heartbeat_back_addr":"172.21.15.107:6809/3721776817","heartbeat_front_addr":"172.21.15.107:6807/3721776817","state":["exists","up"]},{"osd":1,"uuid":"fa29dc8f-419b-4318-b43e-e13fffd87204","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":4224626475},{"type":"v1","addr":"172.21.15.107:6811","nonce":4224626475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":4224626475},{"type":"v1","addr":"172.21.15.107:6813","nonce":4224626475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4224626475},{"type":"v1","addr":"172.21.15.107:6817","nonce":4224626475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":4224626475},{"type":"v1","addr":"172.21.15.107:6815","nonce":4224626475}]},"public_addr":"172.21.15.107:6811/4224626475","cluster_addr":"172.21.15.107:6813/4224626475","heartbeat_back_addr":"172.21.15.107:6817/4224626475","heartbeat_front_addr":"172.21.15.107:6815/4224626475","state":["exists","up"]},{"osd":2,"uuid":"580a07f0-7bd4-4341-9c1f-b3383d64c720","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2331133561},{"type":"v1","addr":"172.21.15.107:6819","nonce":2331133561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2331133561},{"type":"v1","addr":"172.21.15.107:6821","nonce":2331133561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":2331133561},{"type":"v1","addr":"172.21.15.107:6825","nonce":2331133561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2331133561},{"type":"v1","addr":"172.21.15.107:6823","nonce":2331133561}]},"public_addr":"172.21.15.107:6819/2331133561","cluster_addr":"172.21.15.107:6821/2331133561","heartbeat_back_addr":"172.21.15.107:6825/2331133561","heartbeat_front_addr":"172.21.15.107:6823/2331133561","state":["exists","up"]},{"osd":3,"uuid":"7acac14d-998a-4651-b78c-e42584e13b43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2475630313},{"type":"v1","addr":"172.21.15.169:6801","nonce":2475630313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2475630313},{"type":"v1","addr":"172.21.15.169:6803","nonce":2475630313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2475630313},{"type":"v1","addr":"172.21.15.169:6807","nonce":2475630313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2475630313},{"type":"v1","addr":"172.21.15.169:6805","nonce":2475630313}]},"public_addr":"172.21.15.169:6801/2475630313","cluster_addr":"172.21.15.169:6803/2475630313","heartbeat_back_addr":"172.21.15.169:6807/2475630313","heartbeat_front_addr":"172.21.15.169:6805/2475630313","state":["exists","up"]},{"osd":4,"uuid":"01d2b77c-66ec-4b50-8170-43d95eeae471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":592652532},{"type":"v1","addr":"172.21.15.169:6809","nonce":592652532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":592652532},{"type":"v1","addr":"172.21.15.169:6811","nonce":592652532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":592652532},{"type":"v1","addr":"172.21.15.169:6815","nonce":592652532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":592652532},{"type":"v1","addr":"172.21.15.169:6813","nonce":592652532}]},"public_addr":"172.21.15.169:6809/592652532","cluster_addr":"172.21.15.169:6811/592652532","heartbeat_back_addr":"172.21.15.169:6815/592652532","heartbeat_front_addr":"172.21.15.169:6813/592652532","state":["exists","up"]},{"osd":5,"uuid":"0164ead7-c5d0-4541-bf22-f671558f8607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":182182184},{"type":"v1","addr":"172.21.15.169:6817","nonce":182182184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":182182184},{"type":"v1","addr":"172.21.15.169:6819","nonce":182182184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":182182184},{"type":"v1","addr":"172.21.15.169:6823","nonce":182182184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":182182184},{"type":"v1","addr":"172.21.15.169:6821","nonce":182182184}]},"public_addr":"172.21.15.169:6817/182182184","cluster_addr":"172.21.15.169:6819/182182184","heartbeat_back_addr":"172.21.15.169:6823/182182184","heartbeat_front_addr":"172.21.15.169:6821/182182184","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T13:33:59.729659+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T13:34:36.600326+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T13:35:14.068513+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T13:35:46.608007+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T13:36:19.595680+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T13:36:52.586933+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/3503768751":"2024-03-05T13:32:27.229455+0000","172.21.15.107:0/601795289":"2024-03-05T13:32:27.229455+0000","172.21.15.107:6801/2040330534":"2024-03-05T13:32:27.229455+0000","172.21.15.107:6800/2040330534":"2024-03-05T13:32:27.229455+0000","172.21.15.107:0/136342223":"2024-03-05T13:30:56.353578+0000","172.21.15.107:0/2314831039":"2024-03-05T13:30:56.353578+0000","172.21.15.107:6801/2371648646":"2024-03-05T13:30:56.353578+0000","172.21.15.107:6800/2371648646":"2024-03-05T13:30:56.353578+0000","172.21.15.107:0/2723084639":"2024-03-05T13:32:27.229455+0000","172.21.15.107:6801/1723390229":"2024-03-05T13:30:16.230256+0000","172.21.15.107:6800/1723390229":"2024-03-05T13:30:16.230256+0000","172.21.15.107:0/2928681569":"2024-03-05T13:30:56.353578+0000","172.21.15.107:0/1780539770":"2024-03-05T13:30:16.230256+0000","172.21.15.107:0/3451280196":"2024-03-05T13:30:16.230256+0000","172.21.15.107:0/2357638826":"2024-03-05T13:30:16.230256+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T13:37:17.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:17.738+0000 7f9b78000700 1 -- 172.21.15.107:0/615580096 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f9b580692a0 msgr2=0x7f9b5806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:37:17.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:17.738+0000 7f9b78000700 1 --2- 172.21.15.107:0/615580096 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f9b580692a0 0x7f9b5806b750 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9b5c009fd0 tx=0x7f9b5c00b040 comp rx=0 tx=0).stop 2024-03-04T13:37:17.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:17.738+0000 7f9b78000700 1 -- 172.21.15.107:0/615580096 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b70103dc0 msgr2=0x7f9b7011a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:37:17.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:17.738+0000 7f9b78000700 1 --2- 172.21.15.107:0/615580096 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b70103dc0 0x7f9b7011a0e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9b6400b3b0 tx=0x7f9b64009e30 comp rx=0 tx=0).stop 2024-03-04T13:37:17.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:17.739+0000 7f9b78000700 1 -- 172.21.15.107:0/615580096 shutdown_connections 2024-03-04T13:37:17.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:17.739+0000 7f9b78000700 1 --2- 172.21.15.107:0/615580096 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f9b580692a0 0x7f9b5806b750 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:17.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:17.739+0000 7f9b78000700 1 --2- 172.21.15.107:0/615580096 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b70103dc0 0x7f9b7011a0e0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:17.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:17.739+0000 7f9b78000700 1 --2- 172.21.15.107:0/615580096 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b701095f0 0x7f9b7011a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:17.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:17.739+0000 7f9b78000700 1 -- 172.21.15.107:0/615580096 >> 172.21.15.107:0/615580096 conn(0x7f9b70072d50 msgr2=0x7f9b70074790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:37:17.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:17.739+0000 7f9b78000700 1 -- 172.21.15.107:0/615580096 shutdown_connections 2024-03-04T13:37:17.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:17.739+0000 7f9b78000700 1 -- 172.21.15.107:0/615580096 wait complete. 2024-03-04T13:37:18.078 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:37:17 smithi107 ceph-mon[28739]: from='client.? 172.21.15.107:0/615580096' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T13:37:18.103 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:37:17 smithi169 ceph-mon[36313]: from='client.? 172.21.15.107:0/615580096' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T13:37:18.482 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-03-04T13:37:18.482 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 -- ceph osd dump --format=json 2024-03-04T13:37:18.941 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/mon.smithi107/config 2024-03-04T13:37:19.086 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:37:18 smithi107 ceph-mon[28739]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-04T13:37:19.103 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:37:18 smithi169 ceph-mon[36313]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-04T13:37:21.086 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:37:20 smithi107 ceph-mon[28739]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-04T13:37:21.126 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:37:20 smithi169 ceph-mon[36313]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-04T13:37:21.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:21.291+0000 7f27e5200700 1 -- 172.21.15.107:0/3341762860 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f27e00fe8f0 msgr2=0x7f27e0107050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:37:21.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:21.291+0000 7f27e5200700 1 --2- 172.21.15.107:0/3341762860 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f27e00fe8f0 0x7f27e0107050 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f27cc004970 tx=0x7f27cc009e30 comp rx=0 tx=0).stop 2024-03-04T13:37:21.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:21.291+0000 7f27e5200700 1 -- 172.21.15.107:0/3341762860 shutdown_connections 2024-03-04T13:37:21.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:21.291+0000 7f27e5200700 1 --2- 172.21.15.107:0/3341762860 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f27e00fe8f0 0x7f27e0107050 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:21.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:21.291+0000 7f27e5200700 1 --2- 172.21.15.107:0/3341762860 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27e00fdfe0 0x7f27e00fe3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:21.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:21.291+0000 7f27e5200700 1 -- 172.21.15.107:0/3341762860 >> 172.21.15.107:0/3341762860 conn(0x7f27e0071d90 msgr2=0x7f27e0072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:37:21.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:21.291+0000 7f27e5200700 1 -- 172.21.15.107:0/3341762860 shutdown_connections 2024-03-04T13:37:21.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:21.291+0000 7f27e5200700 1 -- 172.21.15.107:0/3341762860 wait complete. 2024-03-04T13:37:21.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:21.292+0000 7f27e5200700 1 Processor -- start 2024-03-04T13:37:21.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:21.292+0000 7f27e5200700 1 -- start start 2024-03-04T13:37:21.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:21.293+0000 7f27e5200700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27e00fdfe0 0x7f27e00fbea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:37:21.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:21.293+0000 7f27e5200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f27e00fe8f0 0x7f27e00fc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:37:21.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:21.293+0000 7f27e5200700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27e00fc9e0 con 0x7f27e00fe8f0 2024-03-04T13:37:21.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:21.293+0000 7f27e5200700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27e00fcb50 con 0x7f27e00fdfe0 2024-03-04T13:37:21.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:21.293+0000 7f27dd400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f27e00fe8f0 0x7f27e00fc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:37:21.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:21.293+0000 7f27dd400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f27e00fe8f0 0x7f27e00fc3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34354/0 (socket says 172.21.15.107:34354) 2024-03-04T13:37:21.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:21.293+0000 7f27dd400700 1 -- 172.21.15.107:0/564529967 learned_addr learned my addr 172.21.15.107:0/564529967 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:37:21.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:21.293+0000 7f27dde00700 1 --2- 172.21.15.107:0/564529967 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27e00fdfe0 0x7f27e00fbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:37:21.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:21.293+0000 7f27dd400700 1 -- 172.21.15.107:0/564529967 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27e00fdfe0 msgr2=0x7f27e00fbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:37:21.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:21.293+0000 7f27dd400700 1 --2- 172.21.15.107:0/564529967 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27e00fdfe0 0x7f27e00fbea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:21.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:21.293+0000 7f27dd400700 1 -- 172.21.15.107:0/564529967 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27cc005040 con 0x7f27e00fe8f0 2024-03-04T13:37:21.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:21.294+0000 7f27dd400700 1 --2- 172.21.15.107:0/564529967 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f27e00fe8f0 0x7f27e00fc3e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f27cc000c00 tx=0x7f27cc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:37:21.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:21.294+0000 7f27d6000700 1 -- 172.21.15.107:0/564529967 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f27cc00e040 con 0x7f27e00fe8f0 2024-03-04T13:37:21.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:21.294+0000 7f27d6000700 1 -- 172.21.15.107:0/564529967 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f27cc004290 con 0x7f27e00fe8f0 2024-03-04T13:37:21.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:21.294+0000 7f27e5200700 1 -- 172.21.15.107:0/564529967 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27e0119c30 con 0x7f27e00fe8f0 2024-03-04T13:37:21.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:21.294+0000 7f27e5200700 1 -- 172.21.15.107:0/564529967 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27e011a090 con 0x7f27e00fe8f0 2024-03-04T13:37:21.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:21.295+0000 7f27d6000700 1 -- 172.21.15.107:0/564529967 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f27cc01c630 con 0x7f27e00fe8f0 2024-03-04T13:37:21.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:21.296+0000 7f27e5200700 1 -- 172.21.15.107:0/564529967 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27c00052e0 con 0x7f27e00fe8f0 2024-03-04T13:37:21.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:21.296+0000 7f27d6000700 1 -- 172.21.15.107:0/564529967 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86623+0+0 (secure 0 0 0) 0x7f27cc01c8a0 con 0x7f27e00fe8f0 2024-03-04T13:37:21.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:21.297+0000 7f27d6000700 1 --2- 172.21.15.107:0/564529967 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f27c40692a0 0x7f27c406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:37:21.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:21.297+0000 7f27d6000700 1 -- 172.21.15.107:0/564529967 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f27cc088150 con 0x7f27e00fe8f0 2024-03-04T13:37:21.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:21.298+0000 7f27dde00700 1 --2- 172.21.15.107:0/564529967 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f27c40692a0 0x7f27c406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:37:21.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:21.298+0000 7f27dde00700 1 --2- 172.21.15.107:0/564529967 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f27c40692a0 0x7f27c406b750 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f27d0007fd0 tx=0x7f27d0009040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:37:21.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:21.306+0000 7f27d6000700 1 -- 172.21.15.107:0/564529967 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f27cc054720 con 0x7f27e00fe8f0 2024-03-04T13:37:21.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:21.519+0000 7f27e5200700 1 -- 172.21.15.107:0/564529967 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f27c0005590 con 0x7f27e00fe8f0 2024-03-04T13:37:21.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:21.520+0000 7f27d6000700 1 -- 172.21.15.107:0/564529967 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+11122 (secure 0 0 0) 0x7f27cc057d40 con 0x7f27e00fe8f0 2024-03-04T13:37:21.514 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:37:21.514 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":41,"fsid":"1fc09a58-da2b-11ee-95c6-87774f69a715","created":"2024-03-04T13:29:42.985888+0000","modified":"2024-03-04T13:36:54.674997+0000","last_up_change":"2024-03-04T13:36:53.661634+0000","last_in_change":"2024-03-04T13:36:31.424613+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T13:35:17.915858+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"c857e387-3d2b-478d-9807-24ea8b9fe02d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":3721776817},{"type":"v1","addr":"172.21.15.107:6803","nonce":3721776817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":3721776817},{"type":"v1","addr":"172.21.15.107:6805","nonce":3721776817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":3721776817},{"type":"v1","addr":"172.21.15.107:6809","nonce":3721776817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":3721776817},{"type":"v1","addr":"172.21.15.107:6807","nonce":3721776817}]},"public_addr":"172.21.15.107:6803/3721776817","cluster_addr":"172.21.15.107:6805/3721776817","heartbeat_back_addr":"172.21.15.107:6809/3721776817","heartbeat_front_addr":"172.21.15.107:6807/3721776817","state":["exists","up"]},{"osd":1,"uuid":"fa29dc8f-419b-4318-b43e-e13fffd87204","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":4224626475},{"type":"v1","addr":"172.21.15.107:6811","nonce":4224626475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":4224626475},{"type":"v1","addr":"172.21.15.107:6813","nonce":4224626475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4224626475},{"type":"v1","addr":"172.21.15.107:6817","nonce":4224626475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":4224626475},{"type":"v1","addr":"172.21.15.107:6815","nonce":4224626475}]},"public_addr":"172.21.15.107:6811/4224626475","cluster_addr":"172.21.15.107:6813/4224626475","heartbeat_back_addr":"172.21.15.107:6817/4224626475","heartbeat_front_addr":"172.21.15.107:6815/4224626475","state":["exists","up"]},{"osd":2,"uuid":"580a07f0-7bd4-4341-9c1f-b3383d64c720","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2331133561},{"type":"v1","addr":"172.21.15.107:6819","nonce":2331133561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2331133561},{"type":"v1","addr":"172.21.15.107:6821","nonce":2331133561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":2331133561},{"type":"v1","addr":"172.21.15.107:6825","nonce":2331133561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2331133561},{"type":"v1","addr":"172.21.15.107:6823","nonce":2331133561}]},"public_addr":"172.21.15.107:6819/2331133561","cluster_addr":"172.21.15.107:6821/2331133561","heartbeat_back_addr":"172.21.15.107:6825/2331133561","heartbeat_front_addr":"172.21.15.107:6823/2331133561","state":["exists","up"]},{"osd":3,"uuid":"7acac14d-998a-4651-b78c-e42584e13b43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2475630313},{"type":"v1","addr":"172.21.15.169:6801","nonce":2475630313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2475630313},{"type":"v1","addr":"172.21.15.169:6803","nonce":2475630313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2475630313},{"type":"v1","addr":"172.21.15.169:6807","nonce":2475630313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2475630313},{"type":"v1","addr":"172.21.15.169:6805","nonce":2475630313}]},"public_addr":"172.21.15.169:6801/2475630313","cluster_addr":"172.21.15.169:6803/2475630313","heartbeat_back_addr":"172.21.15.169:6807/2475630313","heartbeat_front_addr":"172.21.15.169:6805/2475630313","state":["exists","up"]},{"osd":4,"uuid":"01d2b77c-66ec-4b50-8170-43d95eeae471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":592652532},{"type":"v1","addr":"172.21.15.169:6809","nonce":592652532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":592652532},{"type":"v1","addr":"172.21.15.169:6811","nonce":592652532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":592652532},{"type":"v1","addr":"172.21.15.169:6815","nonce":592652532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":592652532},{"type":"v1","addr":"172.21.15.169:6813","nonce":592652532}]},"public_addr":"172.21.15.169:6809/592652532","cluster_addr":"172.21.15.169:6811/592652532","heartbeat_back_addr":"172.21.15.169:6815/592652532","heartbeat_front_addr":"172.21.15.169:6813/592652532","state":["exists","up"]},{"osd":5,"uuid":"0164ead7-c5d0-4541-bf22-f671558f8607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":182182184},{"type":"v1","addr":"172.21.15.169:6817","nonce":182182184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":182182184},{"type":"v1","addr":"172.21.15.169:6819","nonce":182182184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":182182184},{"type":"v1","addr":"172.21.15.169:6823","nonce":182182184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":182182184},{"type":"v1","addr":"172.21.15.169:6821","nonce":182182184}]},"public_addr":"172.21.15.169:6817/182182184","cluster_addr":"172.21.15.169:6819/182182184","heartbeat_back_addr":"172.21.15.169:6823/182182184","heartbeat_front_addr":"172.21.15.169:6821/182182184","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T13:33:59.729659+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T13:34:36.600326+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T13:35:14.068513+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T13:35:46.608007+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T13:36:19.595680+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T13:36:52.586933+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/3503768751":"2024-03-05T13:32:27.229455+0000","172.21.15.107:0/601795289":"2024-03-05T13:32:27.229455+0000","172.21.15.107:6801/2040330534":"2024-03-05T13:32:27.229455+0000","172.21.15.107:6800/2040330534":"2024-03-05T13:32:27.229455+0000","172.21.15.107:0/136342223":"2024-03-05T13:30:56.353578+0000","172.21.15.107:0/2314831039":"2024-03-05T13:30:56.353578+0000","172.21.15.107:6801/2371648646":"2024-03-05T13:30:56.353578+0000","172.21.15.107:6800/2371648646":"2024-03-05T13:30:56.353578+0000","172.21.15.107:0/2723084639":"2024-03-05T13:32:27.229455+0000","172.21.15.107:6801/1723390229":"2024-03-05T13:30:16.230256+0000","172.21.15.107:6800/1723390229":"2024-03-05T13:30:16.230256+0000","172.21.15.107:0/2928681569":"2024-03-05T13:30:56.353578+0000","172.21.15.107:0/1780539770":"2024-03-05T13:30:16.230256+0000","172.21.15.107:0/3451280196":"2024-03-05T13:30:16.230256+0000","172.21.15.107:0/2357638826":"2024-03-05T13:30:16.230256+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T13:37:21.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:21.523+0000 7f27e5200700 1 -- 172.21.15.107:0/564529967 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f27c40692a0 msgr2=0x7f27c406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:37:21.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:21.523+0000 7f27e5200700 1 --2- 172.21.15.107:0/564529967 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f27c40692a0 0x7f27c406b750 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f27d0007fd0 tx=0x7f27d0009040 comp rx=0 tx=0).stop 2024-03-04T13:37:21.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:21.523+0000 7f27e5200700 1 -- 172.21.15.107:0/564529967 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f27e00fe8f0 msgr2=0x7f27e00fc3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:37:21.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:21.523+0000 7f27e5200700 1 --2- 172.21.15.107:0/564529967 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f27e00fe8f0 0x7f27e00fc3e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f27cc000c00 tx=0x7f27cc009e30 comp rx=0 tx=0).stop 2024-03-04T13:37:21.518 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:21.523+0000 7f27e5200700 1 -- 172.21.15.107:0/564529967 shutdown_connections 2024-03-04T13:37:21.518 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:21.523+0000 7f27e5200700 1 --2- 172.21.15.107:0/564529967 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f27c40692a0 0x7f27c406b750 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:21.518 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:21.523+0000 7f27e5200700 1 --2- 172.21.15.107:0/564529967 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27e00fdfe0 0x7f27e00fbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:21.518 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:21.523+0000 7f27e5200700 1 --2- 172.21.15.107:0/564529967 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f27e00fe8f0 0x7f27e00fc3e0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:21.518 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:21.523+0000 7f27e5200700 1 -- 172.21.15.107:0/564529967 >> 172.21.15.107:0/564529967 conn(0x7f27e0071d90 msgr2=0x7f27e0101840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:37:21.518 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:21.524+0000 7f27e5200700 1 -- 172.21.15.107:0/564529967 shutdown_connections 2024-03-04T13:37:21.518 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:21.524+0000 7f27e5200700 1 -- 172.21.15.107:0/564529967 wait complete. 2024-03-04T13:37:22.086 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:37:21 smithi107 ceph-mon[28739]: from='client.? 172.21.15.107:0/564529967' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T13:37:22.103 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:37:21 smithi169 ceph-mon[36313]: from='client.? 172.21.15.107:0/564529967' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T13:37:22.355 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-03-04T13:37:22.355 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-03-04T13:37:22.356 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-03-04T13:37:22.356 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2024-03-04T13:37:22.357 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2024-03-04T13:37:22.357 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2024-03-04T13:37:22.891 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/mon.smithi107/config 2024-03-04T13:37:23.086 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:37:22 smithi107 ceph-mon[28739]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-04T13:37:23.103 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:37:22 smithi169 ceph-mon[36313]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-04T13:37:23.670 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/mon.smithi107/config 2024-03-04T13:37:24.057 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/mon.smithi107/config 2024-03-04T13:37:24.058 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/mon.smithi107/config 2024-03-04T13:37:24.902 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:37:24 smithi107 ceph-mon[28739]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-04T13:37:25.103 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:37:24 smithi169 ceph-mon[36313]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-04T13:37:26.356 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:26.237+0000 7f5984800700 1 -- 172.21.15.107:0/3300463718 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f597c102f50 msgr2=0x7f597c103320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:37:26.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:26.237+0000 7f5984800700 1 --2- 172.21.15.107:0/3300463718 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f597c102f50 0x7f597c103320 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5970004940 tx=0x7f5970009e30 comp rx=0 tx=0).stop 2024-03-04T13:37:26.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:26.237+0000 7f5984800700 1 -- 172.21.15.107:0/3300463718 shutdown_connections 2024-03-04T13:37:26.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:26.237+0000 7f5984800700 1 --2- 172.21.15.107:0/3300463718 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f597c1038f0 0x7f597c0fcf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:26.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:26.237+0000 7f5984800700 1 --2- 172.21.15.107:0/3300463718 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f597c102f50 0x7f597c103320 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:26.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:26.237+0000 7f5984800700 1 -- 172.21.15.107:0/3300463718 >> 172.21.15.107:0/3300463718 conn(0x7f597c0f8ae0 msgr2=0x7f597c0faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:37:26.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:26.237+0000 7f5984800700 1 -- 172.21.15.107:0/3300463718 shutdown_connections 2024-03-04T13:37:26.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:26.237+0000 7f5984800700 1 -- 172.21.15.107:0/3300463718 wait complete. 2024-03-04T13:37:26.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:26.238+0000 7f5984800700 1 Processor -- start 2024-03-04T13:37:26.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:26.238+0000 7f5984800700 1 -- start start 2024-03-04T13:37:26.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:26.239+0000 7f5984800700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f597c102f50 0x7f597c115c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:37:26.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:26.239+0000 7f5984800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f597c1038f0 0x7f597c1161c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:37:26.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:26.239+0000 7f5984800700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f597c116810 con 0x7f597c1038f0 2024-03-04T13:37:26.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:26.239+0000 7f5984800700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f597c10fb40 con 0x7f597c102f50 2024-03-04T13:37:26.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:26.239+0000 7f5981400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f597c1038f0 0x7f597c1161c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:37:26.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:26.239+0000 7f5981400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f597c1038f0 0x7f597c1161c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34370/0 (socket says 172.21.15.107:34370) 2024-03-04T13:37:26.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:26.239+0000 7f5981400700 1 -- 172.21.15.107:0/2168928424 learned_addr learned my addr 172.21.15.107:0/2168928424 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:37:26.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:26.240+0000 7f5981e00700 1 --2- 172.21.15.107:0/2168928424 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f597c102f50 0x7f597c115c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:37:26.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:26.240+0000 7f5981400700 1 -- 172.21.15.107:0/2168928424 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f597c102f50 msgr2=0x7f597c115c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:37:26.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:26.240+0000 7f5981400700 1 --2- 172.21.15.107:0/2168928424 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f597c102f50 0x7f597c115c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:26.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:26.240+0000 7f5981400700 1 -- 172.21.15.107:0/2168928424 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5970005040 con 0x7f597c1038f0 2024-03-04T13:37:26.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:26.240+0000 7f5981400700 1 --2- 172.21.15.107:0/2168928424 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f597c1038f0 0x7f597c1161c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f596c00c800 tx=0x7f596c004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:37:26.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:26.240+0000 7f5976000700 1 -- 172.21.15.107:0/2168928424 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f596c00a760 con 0x7f597c1038f0 2024-03-04T13:37:26.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:26.240+0000 7f5976000700 1 -- 172.21.15.107:0/2168928424 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f596c00ad90 con 0x7f597c1038f0 2024-03-04T13:37:26.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:26.240+0000 7f5984800700 1 -- 172.21.15.107:0/2168928424 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f597c10fe20 con 0x7f597c1038f0 2024-03-04T13:37:26.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:26.240+0000 7f5976000700 1 -- 172.21.15.107:0/2168928424 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f596c0113a0 con 0x7f597c1038f0 2024-03-04T13:37:26.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:26.240+0000 7f5984800700 1 -- 172.21.15.107:0/2168928424 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f597c110360 con 0x7f597c1038f0 2024-03-04T13:37:26.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:26.241+0000 7f5984800700 1 -- 172.21.15.107:0/2168928424 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f597c1108f0 con 0x7f597c1038f0 2024-03-04T13:37:26.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:26.243+0000 7f5976000700 1 -- 172.21.15.107:0/2168928424 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86623+0+0 (secure 0 0 0) 0x7f596c012400 con 0x7f597c1038f0 2024-03-04T13:37:26.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:26.244+0000 7f5976000700 1 --2- 172.21.15.107:0/2168928424 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f59640692b0 0x7f596406b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:37:26.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:26.244+0000 7f5976000700 1 -- 172.21.15.107:0/2168928424 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f596c0877e0 con 0x7f597c1038f0 2024-03-04T13:37:26.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:26.244+0000 7f5981e00700 1 --2- 172.21.15.107:0/2168928424 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f59640692b0 0x7f596406b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:37:26.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:26.244+0000 7f5976000700 1 --2- 172.21.15.107:0/2168928424 >> [v2:172.21.15.107:6802/3721776817,v1:172.21.15.107:6803/3721776817] conn(0x7f596406ef80 0x7f5964071390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:37:26.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:26.244+0000 7f5976000700 1 -- 172.21.15.107:0/2168928424 --> [v2:172.21.15.107:6802/3721776817,v1:172.21.15.107:6803/3721776817] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5964071a40 con 0x7f596406ef80 2024-03-04T13:37:26.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:26.244+0000 7f5976000700 1 -- 172.21.15.107:0/2168928424 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f596c087be0 con 0x7f597c1038f0 2024-03-04T13:37:26.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:26.244+0000 7f5982800700 1 --2- 172.21.15.107:0/2168928424 >> [v2:172.21.15.107:6802/3721776817,v1:172.21.15.107:6803/3721776817] conn(0x7f596406ef80 0x7f5964071390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:37:26.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:26.245+0000 7f5981e00700 1 --2- 172.21.15.107:0/2168928424 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f59640692b0 0x7f596406b760 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f597000b020 tx=0x7f5970009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:37:26.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:26.245+0000 7f5982800700 1 --2- 172.21.15.107:0/2168928424 >> [v2:172.21.15.107:6802/3721776817,v1:172.21.15.107:6803/3721776817] conn(0x7f596406ef80 0x7f5964071390 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:37:26.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:26.247+0000 7f5976000700 1 -- 172.21.15.107:0/2168928424 <== osd.0 v2:172.21.15.107:6802/3721776817 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f5964071a40 con 0x7f596406ef80 2024-03-04T13:37:26.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:26.263+0000 7f5984800700 1 -- 172.21.15.107:0/2168928424 --> [v2:172.21.15.107:6802/3721776817,v1:172.21.15.107:6803/3721776817] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f597c0fad80 con 0x7f596406ef80 2024-03-04T13:37:26.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:26.264+0000 7f5976000700 1 -- 172.21.15.107:0/2168928424 <== osd.0 v2:172.21.15.107:6802/3721776817 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f597c0fad80 con 0x7f596406ef80 2024-03-04T13:37:26.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:26.264+0000 7f5984800700 1 -- 172.21.15.107:0/2168928424 >> [v2:172.21.15.107:6802/3721776817,v1:172.21.15.107:6803/3721776817] conn(0x7f596406ef80 msgr2=0x7f5964071390 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:37:26.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:26.264+0000 7f5984800700 1 --2- 172.21.15.107:0/2168928424 >> [v2:172.21.15.107:6802/3721776817,v1:172.21.15.107:6803/3721776817] conn(0x7f596406ef80 0x7f5964071390 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:26.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:26.264+0000 7f5984800700 1 -- 172.21.15.107:0/2168928424 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f59640692b0 msgr2=0x7f596406b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:37:26.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:26.264+0000 7f5984800700 1 --2- 172.21.15.107:0/2168928424 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f59640692b0 0x7f596406b760 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f597000b020 tx=0x7f5970009e30 comp rx=0 tx=0).stop 2024-03-04T13:37:26.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:26.264+0000 7f5984800700 1 -- 172.21.15.107:0/2168928424 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f597c1038f0 msgr2=0x7f597c1161c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:37:26.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:26.264+0000 7f5984800700 1 --2- 172.21.15.107:0/2168928424 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f597c1038f0 0x7f597c1161c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f596c00c800 tx=0x7f596c004470 comp rx=0 tx=0).stop 2024-03-04T13:37:26.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:26.264+0000 7f5984800700 1 -- 172.21.15.107:0/2168928424 shutdown_connections 2024-03-04T13:37:26.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:26.264+0000 7f5984800700 1 --2- 172.21.15.107:0/2168928424 >> [v2:172.21.15.107:6802/3721776817,v1:172.21.15.107:6803/3721776817] conn(0x7f596406ef80 0x7f5964071390 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:26.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:26.264+0000 7f5984800700 1 --2- 172.21.15.107:0/2168928424 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f59640692b0 0x7f596406b760 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:26.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:26.264+0000 7f5984800700 1 --2- 172.21.15.107:0/2168928424 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f597c102f50 0x7f597c115c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:26.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:26.264+0000 7f5984800700 1 --2- 172.21.15.107:0/2168928424 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f597c1038f0 0x7f597c1161c0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:26.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:26.264+0000 7f5984800700 1 -- 172.21.15.107:0/2168928424 >> 172.21.15.107:0/2168928424 conn(0x7f597c0f8ae0 msgr2=0x7f597c0fa400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:37:26.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:26.264+0000 7f5984800700 1 -- 172.21.15.107:0/2168928424 shutdown_connections 2024-03-04T13:37:26.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:26.265+0000 7f5984800700 1 -- 172.21.15.107:0/2168928424 wait complete. 2024-03-04T13:37:26.619 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/mon.smithi107/config 2024-03-04T13:37:26.622 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/mon.smithi107/config 2024-03-04T13:37:26.979 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:37:26 smithi107 ceph-mon[28739]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-04T13:37:27.103 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:37:26 smithi169 ceph-mon[36313]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-04T13:37:27.830 INFO:teuthology.orchestra.run.smithi107.stdout:38654705708 2024-03-04T13:37:27.830 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-03-04T13:37:28.084 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:37:27 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:37:28.103 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:37:27 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:37:28.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.467+0000 7fb15bc00700 1 -- 172.21.15.107:0/4220188662 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb154070c40 msgr2=0x7fb154109260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:37:28.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.467+0000 7fb15bc00700 1 --2- 172.21.15.107:0/4220188662 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb154070c40 0x7fb154109260 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb140004970 tx=0x7fb140009e30 comp rx=0 tx=0).stop 2024-03-04T13:37:28.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.467+0000 7fb15bc00700 1 -- 172.21.15.107:0/4220188662 shutdown_connections 2024-03-04T13:37:28.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.467+0000 7fb15bc00700 1 --2- 172.21.15.107:0/4220188662 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb154070c40 0x7fb154109260 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:28.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.467+0000 7fb15bc00700 1 --2- 172.21.15.107:0/4220188662 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb154070330 0x7fb154070700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:28.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.467+0000 7fb15bc00700 1 -- 172.21.15.107:0/4220188662 >> 172.21.15.107:0/4220188662 conn(0x7fb1540f8ae0 msgr2=0x7fb1540faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:37:28.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.467+0000 7fb15bc00700 1 -- 172.21.15.107:0/4220188662 shutdown_connections 2024-03-04T13:37:28.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.467+0000 7fb15bc00700 1 -- 172.21.15.107:0/4220188662 wait complete. 2024-03-04T13:37:28.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.468+0000 7fb15bc00700 1 Processor -- start 2024-03-04T13:37:28.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.468+0000 7fb15bc00700 1 -- start start 2024-03-04T13:37:28.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.468+0000 7fb15bc00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb154070330 0x7fb154115c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:37:28.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.468+0000 7fb15bc00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb154070c40 0x7fb154116140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:37:28.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.468+0000 7fb15bc00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb154116790 con 0x7fb154070330 2024-03-04T13:37:28.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.469+0000 7fb15bc00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb15410fae0 con 0x7fb154070c40 2024-03-04T13:37:28.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.469+0000 7fb159200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb154070330 0x7fb154115c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:37:28.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.469+0000 7fb159200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb154070330 0x7fb154115c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39902/0 (socket says 172.21.15.107:39902) 2024-03-04T13:37:28.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.469+0000 7fb159200700 1 -- 172.21.15.107:0/2878178591 learned_addr learned my addr 172.21.15.107:0/2878178591 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:37:28.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.469+0000 7fb153e00700 1 --2- 172.21.15.107:0/2878178591 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb154070c40 0x7fb154116140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:37:28.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.469+0000 7fb159200700 1 -- 172.21.15.107:0/2878178591 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb154070c40 msgr2=0x7fb154116140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:37:28.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.469+0000 7fb159200700 1 --2- 172.21.15.107:0/2878178591 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb154070c40 0x7fb154116140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:28.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.469+0000 7fb159200700 1 -- 172.21.15.107:0/2878178591 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb140005040 con 0x7fb154070330 2024-03-04T13:37:28.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.469+0000 7fb159200700 1 --2- 172.21.15.107:0/2878178591 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb154070330 0x7fb154115c00 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb148001d80 tx=0x7fb148009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:37:28.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.470+0000 7fb150c00700 1 -- 172.21.15.107:0/2878178591 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb148007d10 con 0x7fb154070330 2024-03-04T13:37:28.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.470+0000 7fb150c00700 1 -- 172.21.15.107:0/2878178591 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb148007e70 con 0x7fb154070330 2024-03-04T13:37:28.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.470+0000 7fb150c00700 1 -- 172.21.15.107:0/2878178591 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb148014640 con 0x7fb154070330 2024-03-04T13:37:28.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.470+0000 7fb15bc00700 1 -- 172.21.15.107:0/2878178591 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb15410fdc0 con 0x7fb154070330 2024-03-04T13:37:28.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.470+0000 7fb15bc00700 1 -- 172.21.15.107:0/2878178591 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb154110300 con 0x7fb154070330 2024-03-04T13:37:28.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.471+0000 7fb15bc00700 1 -- 172.21.15.107:0/2878178591 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb154110850 con 0x7fb154070330 2024-03-04T13:37:28.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.472+0000 7fb150c00700 1 -- 172.21.15.107:0/2878178591 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86623+0+0 (secure 0 0 0) 0x7fb14800e960 con 0x7fb154070330 2024-03-04T13:37:28.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.473+0000 7fb150c00700 1 --2- 172.21.15.107:0/2878178591 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fb13c0692b0 0x7fb13c06b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:37:28.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.473+0000 7fb150c00700 1 -- 172.21.15.107:0/2878178591 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fb148086c10 con 0x7fb154070330 2024-03-04T13:37:28.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.473+0000 7fb153e00700 1 --2- 172.21.15.107:0/2878178591 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fb13c0692b0 0x7fb13c06b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:37:28.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.473+0000 7fb150c00700 1 --2- 172.21.15.107:0/2878178591 >> [v2:172.21.15.107:6810/4224626475,v1:172.21.15.107:6811/4224626475] conn(0x7fb13c06ef80 0x7fb13c071390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:37:28.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.473+0000 7fb150c00700 1 -- 172.21.15.107:0/2878178591 --> [v2:172.21.15.107:6810/4224626475,v1:172.21.15.107:6811/4224626475] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb13c071a40 con 0x7fb13c06ef80 2024-03-04T13:37:28.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.473+0000 7fb150c00700 1 -- 172.21.15.107:0/2878178591 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb14808b220 con 0x7fb154070330 2024-03-04T13:37:28.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.473+0000 7fb159c00700 1 --2- 172.21.15.107:0/2878178591 >> [v2:172.21.15.107:6810/4224626475,v1:172.21.15.107:6811/4224626475] conn(0x7fb13c06ef80 0x7fb13c071390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:37:28.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.474+0000 7fb153e00700 1 --2- 172.21.15.107:0/2878178591 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fb13c0692b0 0x7fb13c06b760 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb140000c00 tx=0x7fb140009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:37:28.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.474+0000 7fb159c00700 1 --2- 172.21.15.107:0/2878178591 >> [v2:172.21.15.107:6810/4224626475,v1:172.21.15.107:6811/4224626475] conn(0x7fb13c06ef80 0x7fb13c071390 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:37:28.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.475+0000 7fb150c00700 1 -- 172.21.15.107:0/2878178591 <== osd.1 v2:172.21.15.107:6810/4224626475 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7fb13c071a40 con 0x7fb13c06ef80 2024-03-04T13:37:28.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.491+0000 7fb15bc00700 1 -- 172.21.15.107:0/2878178591 --> [v2:172.21.15.107:6810/4224626475,v1:172.21.15.107:6811/4224626475] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb15405e640 con 0x7fb13c06ef80 2024-03-04T13:37:28.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.491+0000 7fb150c00700 1 -- 172.21.15.107:0/2878178591 <== osd.1 v2:172.21.15.107:6810/4224626475 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb15405e640 con 0x7fb13c06ef80 2024-03-04T13:37:28.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.492+0000 7fb15bc00700 1 -- 172.21.15.107:0/2878178591 >> [v2:172.21.15.107:6810/4224626475,v1:172.21.15.107:6811/4224626475] conn(0x7fb13c06ef80 msgr2=0x7fb13c071390 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:37:28.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.492+0000 7fb15bc00700 1 --2- 172.21.15.107:0/2878178591 >> [v2:172.21.15.107:6810/4224626475,v1:172.21.15.107:6811/4224626475] conn(0x7fb13c06ef80 0x7fb13c071390 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:28.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.492+0000 7fb15bc00700 1 -- 172.21.15.107:0/2878178591 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fb13c0692b0 msgr2=0x7fb13c06b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:37:28.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.492+0000 7fb15bc00700 1 --2- 172.21.15.107:0/2878178591 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fb13c0692b0 0x7fb13c06b760 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb140000c00 tx=0x7fb140009e30 comp rx=0 tx=0).stop 2024-03-04T13:37:28.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.492+0000 7fb15bc00700 1 -- 172.21.15.107:0/2878178591 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb154070330 msgr2=0x7fb154115c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:37:28.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.492+0000 7fb15bc00700 1 --2- 172.21.15.107:0/2878178591 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb154070330 0x7fb154115c00 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb148001d80 tx=0x7fb148009ec0 comp rx=0 tx=0).stop 2024-03-04T13:37:28.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.492+0000 7fb15bc00700 1 -- 172.21.15.107:0/2878178591 shutdown_connections 2024-03-04T13:37:28.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.492+0000 7fb15bc00700 1 --2- 172.21.15.107:0/2878178591 >> [v2:172.21.15.107:6810/4224626475,v1:172.21.15.107:6811/4224626475] conn(0x7fb13c06ef80 0x7fb13c071390 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:28.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.492+0000 7fb15bc00700 1 --2- 172.21.15.107:0/2878178591 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fb13c0692b0 0x7fb13c06b760 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:28.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.492+0000 7fb15bc00700 1 --2- 172.21.15.107:0/2878178591 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb154070330 0x7fb154115c00 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:28.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.492+0000 7fb15bc00700 1 --2- 172.21.15.107:0/2878178591 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb154070c40 0x7fb154116140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:28.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.492+0000 7fb15bc00700 1 -- 172.21.15.107:0/2878178591 >> 172.21.15.107:0/2878178591 conn(0x7fb1540f8ae0 msgr2=0x7fb154103aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:37:28.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.492+0000 7fb15bc00700 1 -- 172.21.15.107:0/2878178591 shutdown_connections 2024-03-04T13:37:28.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.492+0000 7fb15bc00700 1 -- 172.21.15.107:0/2878178591 wait complete. 2024-03-04T13:37:28.944 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:37:28 smithi107 ceph-mon[28739]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-04T13:37:29.103 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:37:28 smithi169 ceph-mon[36313]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-04T13:37:29.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.689+0000 7fb5ca400700 1 -- 172.21.15.107:0/3268323916 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb5c4103dc0 msgr2=0x7fb5c4104230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:37:29.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.689+0000 7fb5ca400700 1 --2- 172.21.15.107:0/3268323916 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb5c4103dc0 0x7fb5c4104230 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb5b0004940 tx=0x7fb5b0009e30 comp rx=0 tx=0).stop 2024-03-04T13:37:29.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.689+0000 7fb5ca400700 1 -- 172.21.15.107:0/3268323916 shutdown_connections 2024-03-04T13:37:29.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.689+0000 7fb5ca400700 1 --2- 172.21.15.107:0/3268323916 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb5c4103dc0 0x7fb5c4104230 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:29.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.689+0000 7fb5ca400700 1 --2- 172.21.15.107:0/3268323916 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb5c41095b0 0x7fb5c4109980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:29.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.689+0000 7fb5ca400700 1 -- 172.21.15.107:0/3268323916 >> 172.21.15.107:0/3268323916 conn(0x7fb5c4072d70 msgr2=0x7fb5c4075180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:37:29.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.689+0000 7fb5ca400700 1 -- 172.21.15.107:0/3268323916 shutdown_connections 2024-03-04T13:37:29.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.689+0000 7fb5ca400700 1 -- 172.21.15.107:0/3268323916 wait complete. 2024-03-04T13:37:29.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.690+0000 7fb5ca400700 1 Processor -- start 2024-03-04T13:37:29.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.690+0000 7fb5ca400700 1 -- start start 2024-03-04T13:37:29.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.691+0000 7fb5ca400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb5c4103dc0 0x7fb5c411a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:37:29.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.691+0000 7fb5ca400700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb5c41095b0 0x7fb5c411a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:37:29.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.691+0000 7fb5ca400700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c411ac00 con 0x7fb5c4103dc0 2024-03-04T13:37:29.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.691+0000 7fb5ca400700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c4113f10 con 0x7fb5c41095b0 2024-03-04T13:37:29.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.691+0000 7fb5c3400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb5c4103dc0 0x7fb5c411a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:37:29.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.691+0000 7fb5c3400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb5c4103dc0 0x7fb5c411a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39934/0 (socket says 172.21.15.107:39934) 2024-03-04T13:37:29.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.691+0000 7fb5c3400700 1 -- 172.21.15.107:0/1352717428 learned_addr learned my addr 172.21.15.107:0/1352717428 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:37:29.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.691+0000 7fb5c2a00700 1 --2- 172.21.15.107:0/1352717428 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb5c41095b0 0x7fb5c411a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:37:29.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.692+0000 7fb5c3400700 1 -- 172.21.15.107:0/1352717428 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb5c41095b0 msgr2=0x7fb5c411a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:37:29.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.692+0000 7fb5c3400700 1 --2- 172.21.15.107:0/1352717428 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb5c41095b0 0x7fb5c411a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:29.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.692+0000 7fb5c3400700 1 -- 172.21.15.107:0/1352717428 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5b0005040 con 0x7fb5c4103dc0 2024-03-04T13:37:29.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.692+0000 7fb5c3400700 1 --2- 172.21.15.107:0/1352717428 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb5c4103dc0 0x7fb5c411a070 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb5b4002a80 tx=0x7fb5b4004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:37:29.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.692+0000 7fb5bb400700 1 -- 172.21.15.107:0/1352717428 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb5b4007cc0 con 0x7fb5c4103dc0 2024-03-04T13:37:29.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.692+0000 7fb5bb400700 1 -- 172.21.15.107:0/1352717428 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb5b400f040 con 0x7fb5c4103dc0 2024-03-04T13:37:29.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.692+0000 7fb5ca400700 1 -- 172.21.15.107:0/1352717428 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5c41141f0 con 0x7fb5c4103dc0 2024-03-04T13:37:29.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.692+0000 7fb5bb400700 1 -- 172.21.15.107:0/1352717428 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb5b400dad0 con 0x7fb5c4103dc0 2024-03-04T13:37:29.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.692+0000 7fb5ca400700 1 -- 172.21.15.107:0/1352717428 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5c4114730 con 0x7fb5c4103dc0 2024-03-04T13:37:29.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.693+0000 7fb5ca400700 1 -- 172.21.15.107:0/1352717428 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb5c4114c60 con 0x7fb5c4103dc0 2024-03-04T13:37:29.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.694+0000 7fb5bb400700 1 -- 172.21.15.107:0/1352717428 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86623+0+0 (secure 0 0 0) 0x7fb5b400dc30 con 0x7fb5c4103dc0 2024-03-04T13:37:29.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.695+0000 7fb5bb400700 1 --2- 172.21.15.107:0/1352717428 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fb5a80692b0 0x7fb5a806b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:37:29.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.695+0000 7fb5bb400700 1 -- 172.21.15.107:0/1352717428 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fb5b4086bd0 con 0x7fb5c4103dc0 2024-03-04T13:37:29.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.695+0000 7fb5c2a00700 1 --2- 172.21.15.107:0/1352717428 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fb5a80692b0 0x7fb5a806b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:37:29.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.695+0000 7fb5bb400700 1 --2- 172.21.15.107:0/1352717428 >> [v2:172.21.15.169:6800/2475630313,v1:172.21.15.169:6801/2475630313] conn(0x7fb5a806ef80 0x7fb5a8071390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:37:29.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.695+0000 7fb5bb400700 1 -- 172.21.15.107:0/1352717428 --> [v2:172.21.15.169:6800/2475630313,v1:172.21.15.169:6801/2475630313] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb5a8071a40 con 0x7fb5a806ef80 2024-03-04T13:37:29.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.695+0000 7fb5bb400700 1 -- 172.21.15.107:0/1352717428 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb5b408b220 con 0x7fb5c4103dc0 2024-03-04T13:37:29.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.696+0000 7fb5c2a00700 1 --2- 172.21.15.107:0/1352717428 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fb5a80692b0 0x7fb5a806b760 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb5c4115910 tx=0x7fb5b0009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:37:29.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.696+0000 7fb5c3e00700 1 --2- 172.21.15.107:0/1352717428 >> [v2:172.21.15.169:6800/2475630313,v1:172.21.15.169:6801/2475630313] conn(0x7fb5a806ef80 0x7fb5a8071390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:37:29.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.697+0000 7fb5c3e00700 1 --2- 172.21.15.107:0/1352717428 >> [v2:172.21.15.169:6800/2475630313,v1:172.21.15.169:6801/2475630313] conn(0x7fb5a806ef80 0x7fb5a8071390 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:37:29.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.699+0000 7fb5bb400700 1 -- 172.21.15.107:0/1352717428 <== osd.3 v2:172.21.15.169:6800/2475630313 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7fb5a8071a40 con 0x7fb5a806ef80 2024-03-04T13:37:29.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.716+0000 7fb5ca400700 1 -- 172.21.15.107:0/1352717428 --> [v2:172.21.15.169:6800/2475630313,v1:172.21.15.169:6801/2475630313] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb5c405e640 con 0x7fb5a806ef80 2024-03-04T13:37:29.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.716+0000 7fb5bb400700 1 -- 172.21.15.107:0/1352717428 <== osd.3 v2:172.21.15.169:6800/2475630313 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fb5c405e640 con 0x7fb5a806ef80 2024-03-04T13:37:29.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.717+0000 7fb5ca400700 1 -- 172.21.15.107:0/1352717428 >> [v2:172.21.15.169:6800/2475630313,v1:172.21.15.169:6801/2475630313] conn(0x7fb5a806ef80 msgr2=0x7fb5a8071390 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:37:29.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.717+0000 7fb5ca400700 1 --2- 172.21.15.107:0/1352717428 >> [v2:172.21.15.169:6800/2475630313,v1:172.21.15.169:6801/2475630313] conn(0x7fb5a806ef80 0x7fb5a8071390 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:29.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.717+0000 7fb5ca400700 1 -- 172.21.15.107:0/1352717428 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fb5a80692b0 msgr2=0x7fb5a806b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:37:29.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.717+0000 7fb5ca400700 1 --2- 172.21.15.107:0/1352717428 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fb5a80692b0 0x7fb5a806b760 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb5c4115910 tx=0x7fb5b0009e30 comp rx=0 tx=0).stop 2024-03-04T13:37:29.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.717+0000 7fb5ca400700 1 -- 172.21.15.107:0/1352717428 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb5c4103dc0 msgr2=0x7fb5c411a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:37:29.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.717+0000 7fb5ca400700 1 --2- 172.21.15.107:0/1352717428 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb5c4103dc0 0x7fb5c411a070 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb5b4002a80 tx=0x7fb5b4004600 comp rx=0 tx=0).stop 2024-03-04T13:37:29.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.717+0000 7fb5ca400700 1 -- 172.21.15.107:0/1352717428 shutdown_connections 2024-03-04T13:37:29.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.717+0000 7fb5ca400700 1 --2- 172.21.15.107:0/1352717428 >> [v2:172.21.15.169:6800/2475630313,v1:172.21.15.169:6801/2475630313] conn(0x7fb5a806ef80 0x7fb5a8071390 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:29.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.717+0000 7fb5ca400700 1 --2- 172.21.15.107:0/1352717428 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fb5a80692b0 0x7fb5a806b760 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:29.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.717+0000 7fb5ca400700 1 --2- 172.21.15.107:0/1352717428 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb5c4103dc0 0x7fb5c411a070 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:29.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.717+0000 7fb5ca400700 1 --2- 172.21.15.107:0/1352717428 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb5c41095b0 0x7fb5c411a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:29.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.717+0000 7fb5ca400700 1 -- 172.21.15.107:0/1352717428 >> 172.21.15.107:0/1352717428 conn(0x7fb5c4072d70 msgr2=0x7fb5c4074960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:37:29.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.718+0000 7fb5ca400700 1 -- 172.21.15.107:0/1352717428 shutdown_connections 2024-03-04T13:37:29.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:28.718+0000 7fb5ca400700 1 -- 172.21.15.107:0/1352717428 wait complete. 2024-03-04T13:37:30.608 INFO:teuthology.orchestra.run.smithi107.stdout:115964117014 2024-03-04T13:37:30.609 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 -- ceph osd last-stat-seq osd.3 2024-03-04T13:37:31.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:37:30 smithi169 ceph-mon[36313]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-04T13:37:31.117 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:37:30 smithi107 ceph-mon[28739]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-04T13:37:32.138 INFO:teuthology.orchestra.run.smithi107.stdout:60129542179 2024-03-04T13:37:32.138 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-03-04T13:37:32.768 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/mon.smithi107/config 2024-03-04T13:37:33.103 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:37:32 smithi169 ceph-mon[36313]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-04T13:37:33.112 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:37:32 smithi107 ceph-mon[28739]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-04T13:37:33.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:33.282+0000 7f666c600700 1 -- 172.21.15.107:0/3116098229 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6664105b10 msgr2=0x7f66640ff180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:37:33.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:33.282+0000 7f666c600700 1 --2- 172.21.15.107:0/3116098229 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6664105b10 0x7f66640ff180 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6650004940 tx=0x7f6650009e30 comp rx=0 tx=0).stop 2024-03-04T13:37:33.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:33.282+0000 7f666c600700 1 -- 172.21.15.107:0/3116098229 shutdown_connections 2024-03-04T13:37:33.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:33.282+0000 7f666c600700 1 --2- 172.21.15.107:0/3116098229 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6664105b10 0x7f66640ff180 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:33.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:33.282+0000 7f666c600700 1 --2- 172.21.15.107:0/3116098229 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6664105170 0x7f6664105540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:33.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:33.282+0000 7f666c600700 1 -- 172.21.15.107:0/3116098229 >> 172.21.15.107:0/3116098229 conn(0x7f66640face0 msgr2=0x7f66640fd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:37:33.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:33.282+0000 7f666c600700 1 -- 172.21.15.107:0/3116098229 shutdown_connections 2024-03-04T13:37:33.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:33.282+0000 7f666c600700 1 -- 172.21.15.107:0/3116098229 wait complete. 2024-03-04T13:37:33.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:33.283+0000 7f666c600700 1 Processor -- start 2024-03-04T13:37:33.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:33.283+0000 7f666c600700 1 -- start start 2024-03-04T13:37:33.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:33.284+0000 7f666c600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6664105170 0x7f6664115ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:37:33.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:33.284+0000 7f666c600700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6664105b10 0x7f6664116220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:37:33.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:33.284+0000 7f666c600700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6664116870 con 0x7f6664105170 2024-03-04T13:37:33.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:33.284+0000 7f666c600700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f666410fb80 con 0x7f6664105b10 2024-03-04T13:37:33.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:33.284+0000 7f6669c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6664105170 0x7f6664115ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:37:33.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:33.284+0000 7f6669c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6664105170 0x7f6664115ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39954/0 (socket says 172.21.15.107:39954) 2024-03-04T13:37:33.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:33.284+0000 7f6669c00700 1 -- 172.21.15.107:0/276619638 learned_addr learned my addr 172.21.15.107:0/276619638 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:37:33.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:33.284+0000 7f6669c00700 1 -- 172.21.15.107:0/276619638 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6664105b10 msgr2=0x7f6664116220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:37:33.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:33.284+0000 7f6669c00700 1 --2- 172.21.15.107:0/276619638 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6664105b10 0x7f6664116220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:33.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:33.284+0000 7f6669c00700 1 -- 172.21.15.107:0/276619638 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6650005040 con 0x7f6664105170 2024-03-04T13:37:33.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:33.285+0000 7f6669c00700 1 --2- 172.21.15.107:0/276619638 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6664105170 0x7f6664115ce0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f665800c830 tx=0x7f6658004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:37:33.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:33.285+0000 7f6661600700 1 -- 172.21.15.107:0/276619638 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f665800a760 con 0x7f6664105170 2024-03-04T13:37:33.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:33.285+0000 7f6661600700 1 -- 172.21.15.107:0/276619638 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f665800ad90 con 0x7f6664105170 2024-03-04T13:37:33.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:33.285+0000 7f666c600700 1 -- 172.21.15.107:0/276619638 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f666410fe60 con 0x7f6664105170 2024-03-04T13:37:33.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:33.285+0000 7f6661600700 1 -- 172.21.15.107:0/276619638 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f66580113a0 con 0x7f6664105170 2024-03-04T13:37:33.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:33.285+0000 7f666c600700 1 -- 172.21.15.107:0/276619638 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66641103a0 con 0x7f6664105170 2024-03-04T13:37:33.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:33.286+0000 7f666c600700 1 -- 172.21.15.107:0/276619638 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6664110930 con 0x7f6664105170 2024-03-04T13:37:33.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:33.287+0000 7f6661600700 1 -- 172.21.15.107:0/276619638 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86623+0+0 (secure 0 0 0) 0x7f6658012400 con 0x7f6664105170 2024-03-04T13:37:33.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:33.288+0000 7f6661600700 1 --2- 172.21.15.107:0/276619638 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f664c0692b0 0x7f664c06b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:37:33.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:33.288+0000 7f6661600700 1 -- 172.21.15.107:0/276619638 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f6658086ec0 con 0x7f6664105170 2024-03-04T13:37:33.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:33.288+0000 7f6669200700 1 --2- 172.21.15.107:0/276619638 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f664c0692b0 0x7f664c06b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:37:33.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:33.288+0000 7f6661600700 1 --2- 172.21.15.107:0/276619638 >> [v2:172.21.15.169:6808/592652532,v1:172.21.15.169:6809/592652532] conn(0x7f664c06ef80 0x7f664c071390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:37:33.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:33.288+0000 7f6661600700 1 -- 172.21.15.107:0/276619638 --> [v2:172.21.15.169:6808/592652532,v1:172.21.15.169:6809/592652532] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f664c071a40 con 0x7f664c06ef80 2024-03-04T13:37:33.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:33.288+0000 7f6661600700 1 -- 172.21.15.107:0/276619638 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f665808c430 con 0x7f6664105170 2024-03-04T13:37:33.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:33.289+0000 7f6669200700 1 --2- 172.21.15.107:0/276619638 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f664c0692b0 0x7f664c06b760 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6650004f90 tx=0x7f6650009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:37:33.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:33.289+0000 7f666a600700 1 --2- 172.21.15.107:0/276619638 >> [v2:172.21.15.169:6808/592652532,v1:172.21.15.169:6809/592652532] conn(0x7f664c06ef80 0x7f664c071390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:37:33.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:33.290+0000 7f666a600700 1 --2- 172.21.15.107:0/276619638 >> [v2:172.21.15.169:6808/592652532,v1:172.21.15.169:6809/592652532] conn(0x7f664c06ef80 0x7f664c071390 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:37:33.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:33.292+0000 7f6661600700 1 -- 172.21.15.107:0/276619638 <== osd.4 v2:172.21.15.169:6808/592652532 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f664c071a40 con 0x7f664c06ef80 2024-03-04T13:37:33.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:33.308+0000 7f666c600700 1 -- 172.21.15.107:0/276619638 --> [v2:172.21.15.169:6808/592652532,v1:172.21.15.169:6809/592652532] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f666405e640 con 0x7f664c06ef80 2024-03-04T13:37:33.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:33.309+0000 7f6661600700 1 -- 172.21.15.107:0/276619638 <== osd.4 v2:172.21.15.169:6808/592652532 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f666405e640 con 0x7f664c06ef80 2024-03-04T13:37:33.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:33.310+0000 7f666c600700 1 -- 172.21.15.107:0/276619638 >> [v2:172.21.15.169:6808/592652532,v1:172.21.15.169:6809/592652532] conn(0x7f664c06ef80 msgr2=0x7f664c071390 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:37:33.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:33.310+0000 7f666c600700 1 --2- 172.21.15.107:0/276619638 >> [v2:172.21.15.169:6808/592652532,v1:172.21.15.169:6809/592652532] conn(0x7f664c06ef80 0x7f664c071390 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:33.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:33.310+0000 7f666c600700 1 -- 172.21.15.107:0/276619638 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f664c0692b0 msgr2=0x7f664c06b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:37:33.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:33.310+0000 7f666c600700 1 --2- 172.21.15.107:0/276619638 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f664c0692b0 0x7f664c06b760 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6650004f90 tx=0x7f6650009e30 comp rx=0 tx=0).stop 2024-03-04T13:37:33.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:33.310+0000 7f666c600700 1 -- 172.21.15.107:0/276619638 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6664105170 msgr2=0x7f6664115ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:37:33.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:33.310+0000 7f666c600700 1 --2- 172.21.15.107:0/276619638 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6664105170 0x7f6664115ce0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f665800c830 tx=0x7f6658004470 comp rx=0 tx=0).stop 2024-03-04T13:37:33.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:33.310+0000 7f666c600700 1 -- 172.21.15.107:0/276619638 shutdown_connections 2024-03-04T13:37:33.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:33.310+0000 7f666c600700 1 --2- 172.21.15.107:0/276619638 >> [v2:172.21.15.169:6808/592652532,v1:172.21.15.169:6809/592652532] conn(0x7f664c06ef80 0x7f664c071390 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:33.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:33.310+0000 7f666c600700 1 --2- 172.21.15.107:0/276619638 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f664c0692b0 0x7f664c06b760 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:33.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:33.310+0000 7f666c600700 1 --2- 172.21.15.107:0/276619638 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6664105170 0x7f6664115ce0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:33.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:33.310+0000 7f666c600700 1 --2- 172.21.15.107:0/276619638 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6664105b10 0x7f6664116220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:33.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:33.310+0000 7f666c600700 1 -- 172.21.15.107:0/276619638 >> 172.21.15.107:0/276619638 conn(0x7f66640face0 msgr2=0x7f66640fc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:37:33.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:33.310+0000 7f666c600700 1 -- 172.21.15.107:0/276619638 shutdown_connections 2024-03-04T13:37:33.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:33.310+0000 7f666c600700 1 -- 172.21.15.107:0/276619638 wait complete. 2024-03-04T13:37:34.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:34.699+0000 7f0560000700 1 -- 172.21.15.107:0/2801205719 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0558105150 msgr2=0x7f0558105520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:37:34.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:34.699+0000 7f0560000700 1 --2- 172.21.15.107:0/2801205719 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0558105150 0x7f0558105520 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f054c004940 tx=0x7f054c009e30 comp rx=0 tx=0).stop 2024-03-04T13:37:34.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:34.699+0000 7f0560000700 1 -- 172.21.15.107:0/2801205719 shutdown_connections 2024-03-04T13:37:34.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:34.699+0000 7f0560000700 1 --2- 172.21.15.107:0/2801205719 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0558105af0 0x7f05580ff160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:34.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:34.699+0000 7f0560000700 1 --2- 172.21.15.107:0/2801205719 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0558105150 0x7f0558105520 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:34.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:34.699+0000 7f0560000700 1 -- 172.21.15.107:0/2801205719 >> 172.21.15.107:0/2801205719 conn(0x7f05580face0 msgr2=0x7f05580fd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:37:34.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:34.699+0000 7f0560000700 1 -- 172.21.15.107:0/2801205719 shutdown_connections 2024-03-04T13:37:34.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:34.699+0000 7f0560000700 1 -- 172.21.15.107:0/2801205719 wait complete. 2024-03-04T13:37:34.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:34.700+0000 7f0560000700 1 Processor -- start 2024-03-04T13:37:34.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:34.700+0000 7f0560000700 1 -- start start 2024-03-04T13:37:34.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:34.701+0000 7f0560000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0558105150 0x7f0558115c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:37:34.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:34.701+0000 7f0560000700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0558105af0 0x7f05581161b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:37:34.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:34.701+0000 7f0560000700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0558116800 con 0x7f0558105150 2024-03-04T13:37:34.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:34.701+0000 7f0560000700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f055810fb30 con 0x7f0558105af0 2024-03-04T13:37:34.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:34.701+0000 7f055d600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0558105150 0x7f0558115c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:37:34.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:34.701+0000 7f055d600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0558105150 0x7f0558115c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39964/0 (socket says 172.21.15.107:39964) 2024-03-04T13:37:34.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:34.701+0000 7f055d600700 1 -- 172.21.15.107:0/2162788350 learned_addr learned my addr 172.21.15.107:0/2162788350 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:37:34.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:34.701+0000 7f055cc00700 1 --2- 172.21.15.107:0/2162788350 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0558105af0 0x7f05581161b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:37:34.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:34.702+0000 7f055d600700 1 -- 172.21.15.107:0/2162788350 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0558105af0 msgr2=0x7f05581161b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:37:34.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:34.702+0000 7f055d600700 1 --2- 172.21.15.107:0/2162788350 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0558105af0 0x7f05581161b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:34.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:34.702+0000 7f055d600700 1 -- 172.21.15.107:0/2162788350 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f054c005040 con 0x7f0558105150 2024-03-04T13:37:34.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:34.702+0000 7f055d600700 1 --2- 172.21.15.107:0/2162788350 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0558105150 0x7f0558115c70 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f054c009fd0 tx=0x7f054c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:37:34.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:34.702+0000 7f0555600700 1 -- 172.21.15.107:0/2162788350 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f054c00e040 con 0x7f0558105150 2024-03-04T13:37:34.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:34.702+0000 7f0555600700 1 -- 172.21.15.107:0/2162788350 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f054c00bbf0 con 0x7f0558105150 2024-03-04T13:37:34.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:34.702+0000 7f0560000700 1 -- 172.21.15.107:0/2162788350 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f055810fdb0 con 0x7f0558105150 2024-03-04T13:37:34.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:34.702+0000 7f0555600700 1 -- 172.21.15.107:0/2162788350 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f054c01c3f0 con 0x7f0558105150 2024-03-04T13:37:34.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:34.702+0000 7f0560000700 1 -- 172.21.15.107:0/2162788350 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05581102f0 con 0x7f0558105150 2024-03-04T13:37:34.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:34.703+0000 7f0560000700 1 -- 172.21.15.107:0/2162788350 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f05581108e0 con 0x7f0558105150 2024-03-04T13:37:34.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:34.704+0000 7f0555600700 1 -- 172.21.15.107:0/2162788350 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86623+0+0 (secure 0 0 0) 0x7f054c00bd60 con 0x7f0558105150 2024-03-04T13:37:34.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:34.705+0000 7f0555600700 1 --2- 172.21.15.107:0/2162788350 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f05400692b0 0x7f054006b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:37:34.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:34.705+0000 7f0555600700 1 -- 172.21.15.107:0/2162788350 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f054c088bc0 con 0x7f0558105150 2024-03-04T13:37:34.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:34.705+0000 7f055cc00700 1 --2- 172.21.15.107:0/2162788350 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f05400692b0 0x7f054006b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:37:34.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:34.706+0000 7f0555600700 1 --2- 172.21.15.107:0/2162788350 >> [v2:172.21.15.169:6816/182182184,v1:172.21.15.169:6817/182182184] conn(0x7f054006ef80 0x7f0540071390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:37:34.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:34.706+0000 7f0555600700 1 -- 172.21.15.107:0/2162788350 --> [v2:172.21.15.169:6816/182182184,v1:172.21.15.169:6817/182182184] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0540071a40 con 0x7f054006ef80 2024-03-04T13:37:34.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:34.706+0000 7f0555600700 1 -- 172.21.15.107:0/2162788350 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f054c088f00 con 0x7f0558105150 2024-03-04T13:37:34.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:34.706+0000 7f055cc00700 1 --2- 172.21.15.107:0/2162788350 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f05400692b0 0x7f054006b760 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f05581114a0 tx=0x7f054400b040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:37:34.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:34.706+0000 7f055e000700 1 --2- 172.21.15.107:0/2162788350 >> [v2:172.21.15.169:6816/182182184,v1:172.21.15.169:6817/182182184] conn(0x7f054006ef80 0x7f0540071390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:37:34.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:34.707+0000 7f055e000700 1 --2- 172.21.15.107:0/2162788350 >> [v2:172.21.15.169:6816/182182184,v1:172.21.15.169:6817/182182184] conn(0x7f054006ef80 0x7f0540071390 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:37:34.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:34.709+0000 7f0555600700 1 -- 172.21.15.107:0/2162788350 <== osd.5 v2:172.21.15.169:6816/182182184 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f0540071a40 con 0x7f054006ef80 2024-03-04T13:37:34.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:34.725+0000 7f0560000700 1 -- 172.21.15.107:0/2162788350 --> [v2:172.21.15.169:6816/182182184,v1:172.21.15.169:6817/182182184] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f05580fd080 con 0x7f054006ef80 2024-03-04T13:37:34.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:34.726+0000 7f0555600700 1 -- 172.21.15.107:0/2162788350 <== osd.5 v2:172.21.15.169:6816/182182184 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f05580fd080 con 0x7f054006ef80 2024-03-04T13:37:34.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:34.726+0000 7f0560000700 1 -- 172.21.15.107:0/2162788350 >> [v2:172.21.15.169:6816/182182184,v1:172.21.15.169:6817/182182184] conn(0x7f054006ef80 msgr2=0x7f0540071390 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:37:34.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:34.726+0000 7f0560000700 1 --2- 172.21.15.107:0/2162788350 >> [v2:172.21.15.169:6816/182182184,v1:172.21.15.169:6817/182182184] conn(0x7f054006ef80 0x7f0540071390 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:34.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:34.727+0000 7f0560000700 1 -- 172.21.15.107:0/2162788350 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f05400692b0 msgr2=0x7f054006b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:37:34.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:34.727+0000 7f0560000700 1 --2- 172.21.15.107:0/2162788350 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f05400692b0 0x7f054006b760 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f05581114a0 tx=0x7f054400b040 comp rx=0 tx=0).stop 2024-03-04T13:37:34.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:34.727+0000 7f0560000700 1 -- 172.21.15.107:0/2162788350 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0558105150 msgr2=0x7f0558115c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:37:34.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:34.727+0000 7f0560000700 1 --2- 172.21.15.107:0/2162788350 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0558105150 0x7f0558115c70 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f054c009fd0 tx=0x7f054c009e30 comp rx=0 tx=0).stop 2024-03-04T13:37:34.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:34.727+0000 7f0560000700 1 -- 172.21.15.107:0/2162788350 shutdown_connections 2024-03-04T13:37:34.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:34.727+0000 7f0560000700 1 --2- 172.21.15.107:0/2162788350 >> [v2:172.21.15.169:6816/182182184,v1:172.21.15.169:6817/182182184] conn(0x7f054006ef80 0x7f0540071390 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:34.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:34.727+0000 7f0560000700 1 --2- 172.21.15.107:0/2162788350 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f05400692b0 0x7f054006b760 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:34.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:34.727+0000 7f0560000700 1 --2- 172.21.15.107:0/2162788350 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0558105150 0x7f0558115c70 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:34.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:34.727+0000 7f0560000700 1 --2- 172.21.15.107:0/2162788350 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0558105af0 0x7f05581161b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:34.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:34.727+0000 7f0560000700 1 -- 172.21.15.107:0/2162788350 >> 172.21.15.107:0/2162788350 conn(0x7f05580face0 msgr2=0x7f05580fc700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:37:34.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:34.727+0000 7f0560000700 1 -- 172.21.15.107:0/2162788350 shutdown_connections 2024-03-04T13:37:34.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:34.727+0000 7f0560000700 1 -- 172.21.15.107:0/2162788350 wait complete. 2024-03-04T13:37:35.179 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:37:34 smithi107 ceph-mon[28739]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-04T13:37:35.353 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:37:34 smithi169 ceph-mon[36313]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-04T13:37:36.035 INFO:teuthology.orchestra.run.smithi107.stdout:150323855376 2024-03-04T13:37:36.036 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 -- ceph osd last-stat-seq osd.4 2024-03-04T13:37:36.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:36.790+0000 7ffba5e00700 1 -- 172.21.15.107:0/4247755928 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffba0102f50 msgr2=0x7ffba0103320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:37:36.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:36.790+0000 7ffba5e00700 1 --2- 172.21.15.107:0/4247755928 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffba0102f50 0x7ffba0103320 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ffb90004940 tx=0x7ffb90009e30 comp rx=0 tx=0).stop 2024-03-04T13:37:36.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:36.790+0000 7ffba5e00700 1 -- 172.21.15.107:0/4247755928 shutdown_connections 2024-03-04T13:37:36.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:36.790+0000 7ffba5e00700 1 --2- 172.21.15.107:0/4247755928 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffba01038f0 0x7ffba00fcf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:36.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:36.790+0000 7ffba5e00700 1 --2- 172.21.15.107:0/4247755928 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffba0102f50 0x7ffba0103320 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:36.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:36.790+0000 7ffba5e00700 1 -- 172.21.15.107:0/4247755928 >> 172.21.15.107:0/4247755928 conn(0x7ffba00f8ae0 msgr2=0x7ffba00faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:37:36.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:36.790+0000 7ffba5e00700 1 -- 172.21.15.107:0/4247755928 shutdown_connections 2024-03-04T13:37:36.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:36.790+0000 7ffba5e00700 1 -- 172.21.15.107:0/4247755928 wait complete. 2024-03-04T13:37:36.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:36.791+0000 7ffba5e00700 1 Processor -- start 2024-03-04T13:37:36.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:36.791+0000 7ffba5e00700 1 -- start start 2024-03-04T13:37:36.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:36.791+0000 7ffba5e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffba0102f50 0x7ffba0115bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:37:36.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:36.792+0000 7ffba5e00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffba01038f0 0x7ffba01160f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:37:36.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:36.792+0000 7ffba5e00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffba0116740 con 0x7ffba0102f50 2024-03-04T13:37:36.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:36.792+0000 7ffba5e00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffba010fb00 con 0x7ffba01038f0 2024-03-04T13:37:36.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:36.792+0000 7ffb9ea00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffba0102f50 0x7ffba0115bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:37:36.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:36.792+0000 7ffb9ea00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffba0102f50 0x7ffba0115bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39978/0 (socket says 172.21.15.107:39978) 2024-03-04T13:37:36.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:36.792+0000 7ffb9ea00700 1 -- 172.21.15.107:0/581203176 learned_addr learned my addr 172.21.15.107:0/581203176 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:37:36.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:36.792+0000 7ffb9e000700 1 --2- 172.21.15.107:0/581203176 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffba01038f0 0x7ffba01160f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:37:36.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:36.792+0000 7ffb9ea00700 1 -- 172.21.15.107:0/581203176 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffba01038f0 msgr2=0x7ffba01160f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:37:36.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:36.792+0000 7ffb9ea00700 1 --2- 172.21.15.107:0/581203176 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffba01038f0 0x7ffba01160f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:36.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:36.792+0000 7ffb9ea00700 1 -- 172.21.15.107:0/581203176 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb90005040 con 0x7ffba0102f50 2024-03-04T13:37:36.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:36.793+0000 7ffb9ea00700 1 --2- 172.21.15.107:0/581203176 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffba0102f50 0x7ffba0115bb0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ffb90009fd0 tx=0x7ffb90009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:37:36.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:36.793+0000 7ffb96a00700 1 -- 172.21.15.107:0/581203176 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffb9000e040 con 0x7ffba0102f50 2024-03-04T13:37:36.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:36.793+0000 7ffb96a00700 1 -- 172.21.15.107:0/581203176 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffb9000bbf0 con 0x7ffba0102f50 2024-03-04T13:37:36.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:36.793+0000 7ffba5e00700 1 -- 172.21.15.107:0/581203176 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffba010fd80 con 0x7ffba0102f50 2024-03-04T13:37:36.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:36.793+0000 7ffb96a00700 1 -- 172.21.15.107:0/581203176 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffb9001c3f0 con 0x7ffba0102f50 2024-03-04T13:37:36.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:36.793+0000 7ffba5e00700 1 -- 172.21.15.107:0/581203176 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffba01102c0 con 0x7ffba0102f50 2024-03-04T13:37:36.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:36.794+0000 7ffba5e00700 1 -- 172.21.15.107:0/581203176 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ffba01107a0 con 0x7ffba0102f50 2024-03-04T13:37:36.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:36.795+0000 7ffb96a00700 1 -- 172.21.15.107:0/581203176 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86623+0+0 (secure 0 0 0) 0x7ffb90013420 con 0x7ffba0102f50 2024-03-04T13:37:36.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:36.796+0000 7ffb96a00700 1 --2- 172.21.15.107:0/581203176 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7ffb840692b0 0x7ffb8406b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:37:36.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:36.796+0000 7ffb96a00700 1 -- 172.21.15.107:0/581203176 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7ffb90088240 con 0x7ffba0102f50 2024-03-04T13:37:36.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:36.797+0000 7ffb96a00700 1 --2- 172.21.15.107:0/581203176 >> [v2:172.21.15.107:6818/2331133561,v1:172.21.15.107:6819/2331133561] conn(0x7ffb8406ef80 0x7ffb84071390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:37:36.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:36.797+0000 7ffb96a00700 1 -- 172.21.15.107:0/581203176 --> [v2:172.21.15.107:6818/2331133561,v1:172.21.15.107:6819/2331133561] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ffb84071a40 con 0x7ffb8406ef80 2024-03-04T13:37:36.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:36.797+0000 7ffb96a00700 1 -- 172.21.15.107:0/581203176 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffb9008a7f0 con 0x7ffba0102f50 2024-03-04T13:37:36.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:36.797+0000 7ffb9e000700 1 --2- 172.21.15.107:0/581203176 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7ffb840692b0 0x7ffb8406b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:37:36.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:36.797+0000 7ffb9f400700 1 --2- 172.21.15.107:0/581203176 >> [v2:172.21.15.107:6818/2331133561,v1:172.21.15.107:6819/2331133561] conn(0x7ffb8406ef80 0x7ffb84071390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:37:36.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:36.797+0000 7ffb9e000700 1 --2- 172.21.15.107:0/581203176 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7ffb840692b0 0x7ffb8406b760 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c007fd0 tx=0x7ffb8c009040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:37:36.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:36.798+0000 7ffb9f400700 1 --2- 172.21.15.107:0/581203176 >> [v2:172.21.15.107:6818/2331133561,v1:172.21.15.107:6819/2331133561] conn(0x7ffb8406ef80 0x7ffb84071390 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:37:36.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:36.798+0000 7ffb96a00700 1 -- 172.21.15.107:0/581203176 <== osd.2 v2:172.21.15.107:6818/2331133561 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7ffb84071a40 con 0x7ffb8406ef80 2024-03-04T13:37:36.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:36.816+0000 7ffba5e00700 1 -- 172.21.15.107:0/581203176 --> [v2:172.21.15.107:6818/2331133561,v1:172.21.15.107:6819/2331133561] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ffba0110910 con 0x7ffb8406ef80 2024-03-04T13:37:36.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:36.816+0000 7ffb96a00700 1 -- 172.21.15.107:0/581203176 <== osd.2 v2:172.21.15.107:6818/2331133561 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ffba0110910 con 0x7ffb8406ef80 2024-03-04T13:37:36.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:36.817+0000 7ffba5e00700 1 -- 172.21.15.107:0/581203176 >> [v2:172.21.15.107:6818/2331133561,v1:172.21.15.107:6819/2331133561] conn(0x7ffb8406ef80 msgr2=0x7ffb84071390 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:37:36.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:36.817+0000 7ffba5e00700 1 --2- 172.21.15.107:0/581203176 >> [v2:172.21.15.107:6818/2331133561,v1:172.21.15.107:6819/2331133561] conn(0x7ffb8406ef80 0x7ffb84071390 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:36.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:36.817+0000 7ffba5e00700 1 -- 172.21.15.107:0/581203176 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7ffb840692b0 msgr2=0x7ffb8406b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:37:36.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:36.817+0000 7ffba5e00700 1 --2- 172.21.15.107:0/581203176 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7ffb840692b0 0x7ffb8406b760 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c007fd0 tx=0x7ffb8c009040 comp rx=0 tx=0).stop 2024-03-04T13:37:36.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:36.817+0000 7ffba5e00700 1 -- 172.21.15.107:0/581203176 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffba0102f50 msgr2=0x7ffba0115bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:37:36.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:36.817+0000 7ffba5e00700 1 --2- 172.21.15.107:0/581203176 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffba0102f50 0x7ffba0115bb0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ffb90009fd0 tx=0x7ffb90009e30 comp rx=0 tx=0).stop 2024-03-04T13:37:36.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:36.817+0000 7ffba5e00700 1 -- 172.21.15.107:0/581203176 shutdown_connections 2024-03-04T13:37:36.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:36.817+0000 7ffba5e00700 1 --2- 172.21.15.107:0/581203176 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7ffb840692b0 0x7ffb8406b760 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:36.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:36.817+0000 7ffba5e00700 1 --2- 172.21.15.107:0/581203176 >> [v2:172.21.15.107:6818/2331133561,v1:172.21.15.107:6819/2331133561] conn(0x7ffb8406ef80 0x7ffb84071390 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:36.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:36.817+0000 7ffba5e00700 1 --2- 172.21.15.107:0/581203176 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffba0102f50 0x7ffba0115bb0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:36.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:36.817+0000 7ffba5e00700 1 --2- 172.21.15.107:0/581203176 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffba01038f0 0x7ffba01160f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:36.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:36.817+0000 7ffba5e00700 1 -- 172.21.15.107:0/581203176 >> 172.21.15.107:0/581203176 conn(0x7ffba00f8ae0 msgr2=0x7ffba00fa3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:37:36.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:36.817+0000 7ffba5e00700 1 -- 172.21.15.107:0/581203176 shutdown_connections 2024-03-04T13:37:36.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:36.817+0000 7ffba5e00700 1 -- 172.21.15.107:0/581203176 wait complete. 2024-03-04T13:37:37.217 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:37:36 smithi107 ceph-mon[28739]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-04T13:37:37.353 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:37:36 smithi169 ceph-mon[36313]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-04T13:37:38.015 INFO:teuthology.orchestra.run.smithi107.stdout:171798691852 2024-03-04T13:37:38.015 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 -- ceph osd last-stat-seq osd.5 2024-03-04T13:37:38.176 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/mon.smithi107/config 2024-03-04T13:37:38.603 INFO:teuthology.orchestra.run.smithi107.stdout:81604378654 2024-03-04T13:37:38.603 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-03-04T13:37:38.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:38.839+0000 7f6c02800700 1 -- 172.21.15.107:0/3711355380 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6bfc1038f0 msgr2=0x7f6bfc0fcf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:37:38.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:38.839+0000 7f6c02800700 1 --2- 172.21.15.107:0/3711355380 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6bfc1038f0 0x7f6bfc0fcf60 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6be8004940 tx=0x7f6be8009e30 comp rx=0 tx=0).stop 2024-03-04T13:37:38.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:38.839+0000 7f6c02800700 1 -- 172.21.15.107:0/3711355380 shutdown_connections 2024-03-04T13:37:38.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:38.839+0000 7f6c02800700 1 --2- 172.21.15.107:0/3711355380 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6bfc1038f0 0x7f6bfc0fcf60 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:38.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:38.839+0000 7f6c02800700 1 --2- 172.21.15.107:0/3711355380 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6bfc102f50 0x7f6bfc103320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:38.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:38.839+0000 7f6c02800700 1 -- 172.21.15.107:0/3711355380 >> 172.21.15.107:0/3711355380 conn(0x7f6bfc0f8ae0 msgr2=0x7f6bfc0faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:37:38.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:38.839+0000 7f6c02800700 1 -- 172.21.15.107:0/3711355380 shutdown_connections 2024-03-04T13:37:38.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:38.840+0000 7f6c02800700 1 -- 172.21.15.107:0/3711355380 wait complete. 2024-03-04T13:37:38.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:38.840+0000 7f6c02800700 1 Processor -- start 2024-03-04T13:37:38.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:38.841+0000 7f6c02800700 1 -- start start 2024-03-04T13:37:38.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:38.841+0000 7f6c02800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6bfc102f50 0x7f6bfc10a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:37:38.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:38.841+0000 7f6c02800700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6bfc1038f0 0x7f6bfc10ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:37:38.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:38.841+0000 7f6c02800700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bfc10b350 con 0x7f6bfc102f50 2024-03-04T13:37:38.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:38.841+0000 7f6c02800700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bfc10b490 con 0x7f6bfc1038f0 2024-03-04T13:37:38.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:38.842+0000 7f6bfaa00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6bfc1038f0 0x7f6bfc10ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:37:38.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:38.842+0000 7f6bfb400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6bfc102f50 0x7f6bfc10a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:37:38.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:38.842+0000 7f6bfaa00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6bfc1038f0 0x7f6bfc10ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.107:50644/0 (socket says 172.21.15.107:50644) 2024-03-04T13:37:38.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:38.842+0000 7f6bfaa00700 1 -- 172.21.15.107:0/898408398 learned_addr learned my addr 172.21.15.107:0/898408398 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:37:38.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:38.843+0000 7f6bfaa00700 1 -- 172.21.15.107:0/898408398 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6bfc102f50 msgr2=0x7f6bfc10a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:37:38.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:38.843+0000 7f6bfaa00700 1 --2- 172.21.15.107:0/898408398 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6bfc102f50 0x7f6bfc10a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:38.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:38.843+0000 7f6bfaa00700 1 -- 172.21.15.107:0/898408398 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6be8005040 con 0x7f6bfc1038f0 2024-03-04T13:37:38.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:38.843+0000 7f6bfb400700 1 --2- 172.21.15.107:0/898408398 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6bfc102f50 0x7f6bfc10a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T13:37:38.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:38.843+0000 7f6bfaa00700 1 --2- 172.21.15.107:0/898408398 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6bfc1038f0 0x7f6bfc10ae10 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f6be8009030 tx=0x7f6be8009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:37:38.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:38.844+0000 7f6bf3400700 1 -- 172.21.15.107:0/898408398 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6be800e040 con 0x7f6bfc1038f0 2024-03-04T13:37:38.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:38.844+0000 7f6c02800700 1 -- 172.21.15.107:0/898408398 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bfc10b710 con 0x7f6bfc1038f0 2024-03-04T13:37:38.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:38.844+0000 7f6bf3400700 1 -- 172.21.15.107:0/898408398 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6be800bbf0 con 0x7f6bfc1038f0 2024-03-04T13:37:38.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:38.844+0000 7f6bf3400700 1 -- 172.21.15.107:0/898408398 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6be801c3f0 con 0x7f6bfc1038f0 2024-03-04T13:37:38.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:38.844+0000 7f6c02800700 1 -- 172.21.15.107:0/898408398 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bfc11e570 con 0x7f6bfc1038f0 2024-03-04T13:37:38.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:38.846+0000 7f6c02800700 1 -- 172.21.15.107:0/898408398 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bfc0fc6e0 con 0x7f6bfc1038f0 2024-03-04T13:37:38.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:38.847+0000 7f6bf3400700 1 -- 172.21.15.107:0/898408398 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 19) v1 ==== 86623+0+0 (secure 0 0 0) 0x7f6be800bd60 con 0x7f6bfc1038f0 2024-03-04T13:37:38.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:38.848+0000 7f6bf3400700 1 --2- 172.21.15.107:0/898408398 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f6be00692b0 0x7f6be006b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:37:38.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:38.848+0000 7f6bf3400700 1 -- 172.21.15.107:0/898408398 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f6be80880f0 con 0x7f6bfc1038f0 2024-03-04T13:37:38.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:38.848+0000 7f6bfb400700 1 --2- 172.21.15.107:0/898408398 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f6be00692b0 0x7f6be006b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:37:38.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:38.849+0000 7f6bfb400700 1 --2- 172.21.15.107:0/898408398 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f6be00692b0 0x7f6be006b760 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6bec007fd0 tx=0x7f6bec009040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:37:38.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:38.862+0000 7f6bf3400700 1 -- 172.21.15.107:0/898408398 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f6be8054740 con 0x7f6bfc1038f0 2024-03-04T13:37:39.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:39.086+0000 7f6c02800700 1 -- 172.21.15.107:0/898408398 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f6bfc10a030 con 0x7f6bfc1038f0 2024-03-04T13:37:39.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:39.087+0000 7f6bf3400700 1 -- 172.21.15.107:0/898408398 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6be8057d60 con 0x7f6bfc1038f0 2024-03-04T13:37:39.080 INFO:teuthology.orchestra.run.smithi107.stdout:38654705710 2024-03-04T13:37:39.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:39.090+0000 7f6c02800700 1 -- 172.21.15.107:0/898408398 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f6be00692b0 msgr2=0x7f6be006b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:37:39.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:39.090+0000 7f6c02800700 1 --2- 172.21.15.107:0/898408398 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f6be00692b0 0x7f6be006b760 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6bec007fd0 tx=0x7f6bec009040 comp rx=0 tx=0).stop 2024-03-04T13:37:39.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:39.090+0000 7f6c02800700 1 -- 172.21.15.107:0/898408398 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6bfc1038f0 msgr2=0x7f6bfc10ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:37:39.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:39.090+0000 7f6c02800700 1 --2- 172.21.15.107:0/898408398 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6bfc1038f0 0x7f6bfc10ae10 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f6be8009030 tx=0x7f6be8009e30 comp rx=0 tx=0).stop 2024-03-04T13:37:39.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:39.090+0000 7f6c02800700 1 -- 172.21.15.107:0/898408398 shutdown_connections 2024-03-04T13:37:39.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:39.090+0000 7f6c02800700 1 --2- 172.21.15.107:0/898408398 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f6be00692b0 0x7f6be006b760 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:39.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:39.090+0000 7f6c02800700 1 --2- 172.21.15.107:0/898408398 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6bfc102f50 0x7f6bfc10a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:39.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:39.090+0000 7f6c02800700 1 --2- 172.21.15.107:0/898408398 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6bfc1038f0 0x7f6bfc10ae10 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:39.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:39.090+0000 7f6c02800700 1 -- 172.21.15.107:0/898408398 >> 172.21.15.107:0/898408398 conn(0x7f6bfc0f8ae0 msgr2=0x7f6bfc0fa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:37:39.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:39.090+0000 7f6c02800700 1 -- 172.21.15.107:0/898408398 shutdown_connections 2024-03-04T13:37:39.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:39.090+0000 7f6c02800700 1 -- 172.21.15.107:0/898408398 wait complete. 2024-03-04T13:37:39.084 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:37:38 smithi107 ceph-mon[28739]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-04T13:37:39.353 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:37:38 smithi169 ceph-mon[36313]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-04T13:37:39.829 INFO:tasks.cephadm.ceph_manager.ceph:need seq 38654705708 got 38654705710 for osd.0 2024-03-04T13:37:39.829 DEBUG:teuthology.parallel:result is None 2024-03-04T13:37:39.829 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/mon.smithi107/config 2024-03-04T13:37:39.834 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/mon.smithi107/config 2024-03-04T13:37:40.070 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:37:39 smithi107 ceph-mon[28739]: from='client.? 172.21.15.107:0/898408398' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-04T13:37:40.255 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:37:39 smithi169 ceph-mon[36313]: from='client.? 172.21.15.107:0/898408398' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-04T13:37:40.278 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/mon.smithi107/config 2024-03-04T13:37:41.086 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:37:40 smithi107 ceph-mon[28739]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-04T13:37:41.353 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:37:40 smithi169 ceph-mon[36313]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-04T13:37:43.334 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:37:42 smithi107 ceph-mon[28739]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-04T13:37:43.334 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:37:42 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:37:43.353 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:37:42 smithi169 ceph-mon[36313]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-04T13:37:43.353 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:37:42 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:37:44.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:44.329+0000 7f2334000700 1 -- 172.21.15.107:0/1863747827 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f232c0fbdd0 msgr2=0x7f232c0fc1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:37:44.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:44.329+0000 7f2334000700 1 --2- 172.21.15.107:0/1863747827 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f232c0fbdd0 0x7f232c0fc1a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2320004940 tx=0x7f2320009e30 comp rx=0 tx=0).stop 2024-03-04T13:37:44.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:44.329+0000 7f2334000700 1 -- 172.21.15.107:0/1863747827 shutdown_connections 2024-03-04T13:37:44.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:44.329+0000 7f2334000700 1 --2- 172.21.15.107:0/1863747827 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f232c0fc6e0 0x7f232c100e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:44.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:44.329+0000 7f2334000700 1 --2- 172.21.15.107:0/1863747827 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f232c0fbdd0 0x7f232c0fc1a0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:44.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:44.329+0000 7f2334000700 1 -- 172.21.15.107:0/1863747827 >> 172.21.15.107:0/1863747827 conn(0x7f232c071d90 msgr2=0x7f232c072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:37:44.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:44.329+0000 7f2334000700 1 -- 172.21.15.107:0/1863747827 shutdown_connections 2024-03-04T13:37:44.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:44.329+0000 7f2334000700 1 -- 172.21.15.107:0/1863747827 wait complete. 2024-03-04T13:37:44.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:44.330+0000 7f2334000700 1 Processor -- start 2024-03-04T13:37:44.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:44.330+0000 7f2334000700 1 -- start start 2024-03-04T13:37:44.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:44.331+0000 7f2334000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f232c0fbdd0 0x7f232c11a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:37:44.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:44.331+0000 7f2334000700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f232c0fc6e0 0x7f232c11a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:37:44.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:44.331+0000 7f2334000700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f232c11ac30 con 0x7f232c0fbdd0 2024-03-04T13:37:44.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:44.331+0000 7f2334000700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f232c113f40 con 0x7f232c0fc6e0 2024-03-04T13:37:44.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:44.331+0000 7f2331600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f232c0fbdd0 0x7f232c11a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:37:44.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:44.331+0000 7f2331600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f232c0fbdd0 0x7f232c11a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49750/0 (socket says 172.21.15.107:49750) 2024-03-04T13:37:44.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:44.331+0000 7f2331600700 1 -- 172.21.15.107:0/1143752877 learned_addr learned my addr 172.21.15.107:0/1143752877 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:37:44.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:44.331+0000 7f2330c00700 1 --2- 172.21.15.107:0/1143752877 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f232c0fc6e0 0x7f232c11a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:37:44.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:44.331+0000 7f2331600700 1 -- 172.21.15.107:0/1143752877 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f232c0fc6e0 msgr2=0x7f232c11a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:37:44.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:44.331+0000 7f2331600700 1 --2- 172.21.15.107:0/1143752877 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f232c0fc6e0 0x7f232c11a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:44.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:44.331+0000 7f2331600700 1 -- 172.21.15.107:0/1143752877 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2320005040 con 0x7f232c0fbdd0 2024-03-04T13:37:44.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:44.332+0000 7f2331600700 1 --2- 172.21.15.107:0/1143752877 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f232c0fbdd0 0x7f232c11a0a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2320009150 tx=0x7f2320009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:37:44.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:44.332+0000 7f2329600700 1 -- 172.21.15.107:0/1143752877 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f232000e040 con 0x7f232c0fbdd0 2024-03-04T13:37:44.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:44.332+0000 7f2329600700 1 -- 172.21.15.107:0/1143752877 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f232000bbf0 con 0x7f232c0fbdd0 2024-03-04T13:37:44.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:44.332+0000 7f2334000700 1 -- 172.21.15.107:0/1143752877 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f232c1141c0 con 0x7f232c0fbdd0 2024-03-04T13:37:44.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:44.332+0000 7f2329600700 1 -- 172.21.15.107:0/1143752877 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f232001c630 con 0x7f232c0fbdd0 2024-03-04T13:37:44.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:44.332+0000 7f2334000700 1 -- 172.21.15.107:0/1143752877 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f232c114700 con 0x7f232c0fbdd0 2024-03-04T13:37:44.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:44.334+0000 7f2334000700 1 -- 172.21.15.107:0/1143752877 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f232c0fe540 con 0x7f232c0fbdd0 2024-03-04T13:37:44.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:44.334+0000 7f2329600700 1 -- 172.21.15.107:0/1143752877 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86623+0+0 (secure 0 0 0) 0x7f232000bd60 con 0x7f232c0fbdd0 2024-03-04T13:37:44.355 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:44.335+0000 7f2329600700 1 --2- 172.21.15.107:0/1143752877 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f23140692b0 0x7f231406b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:37:44.355 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:44.335+0000 7f2329600700 1 -- 172.21.15.107:0/1143752877 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f2320088310 con 0x7f232c0fbdd0 2024-03-04T13:37:44.355 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:44.335+0000 7f2330c00700 1 --2- 172.21.15.107:0/1143752877 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f23140692b0 0x7f231406b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:37:44.355 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:44.336+0000 7f2330c00700 1 --2- 172.21.15.107:0/1143752877 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f23140692b0 0x7f231406b760 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f232c115940 tx=0x7f231800b040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:37:44.355 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:44.340+0000 7f2329600700 1 -- 172.21.15.107:0/1143752877 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f2320058370 con 0x7f232c0fbdd0 2024-03-04T13:37:44.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:44.551+0000 7f2334000700 1 -- 172.21.15.107:0/1143752877 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f232c04bed0 con 0x7f232c0fbdd0 2024-03-04T13:37:44.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:44.551+0000 7f2329600700 1 -- 172.21.15.107:0/1143752877 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2320057f00 con 0x7f232c0fbdd0 2024-03-04T13:37:44.545 INFO:teuthology.orchestra.run.smithi107.stdout:115964117017 2024-03-04T13:37:44.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:44.554+0000 7f2334000700 1 -- 172.21.15.107:0/1143752877 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f23140692b0 msgr2=0x7f231406b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:37:44.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:44.554+0000 7f2334000700 1 --2- 172.21.15.107:0/1143752877 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f23140692b0 0x7f231406b760 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f232c115940 tx=0x7f231800b040 comp rx=0 tx=0).stop 2024-03-04T13:37:44.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:44.554+0000 7f2334000700 1 -- 172.21.15.107:0/1143752877 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f232c0fbdd0 msgr2=0x7f232c11a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:37:44.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:44.554+0000 7f2334000700 1 --2- 172.21.15.107:0/1143752877 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f232c0fbdd0 0x7f232c11a0a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2320009150 tx=0x7f2320009e30 comp rx=0 tx=0).stop 2024-03-04T13:37:44.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:44.555+0000 7f2334000700 1 -- 172.21.15.107:0/1143752877 shutdown_connections 2024-03-04T13:37:44.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:44.555+0000 7f2334000700 1 --2- 172.21.15.107:0/1143752877 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f23140692b0 0x7f231406b760 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:44.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:44.555+0000 7f2334000700 1 --2- 172.21.15.107:0/1143752877 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f232c0fbdd0 0x7f232c11a0a0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:44.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:44.555+0000 7f2334000700 1 --2- 172.21.15.107:0/1143752877 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f232c0fc6e0 0x7f232c11a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:44.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:44.555+0000 7f2334000700 1 -- 172.21.15.107:0/1143752877 >> 172.21.15.107:0/1143752877 conn(0x7f232c071d90 msgr2=0x7f232c10d4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:37:44.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:44.555+0000 7f2334000700 1 -- 172.21.15.107:0/1143752877 shutdown_connections 2024-03-04T13:37:44.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:44.555+0000 7f2334000700 1 -- 172.21.15.107:0/1143752877 wait complete. 2024-03-04T13:37:45.077 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:37:44 smithi107 ceph-mon[28739]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-04T13:37:45.077 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:37:44 smithi107 ceph-mon[28739]: from='client.? 172.21.15.107:0/1143752877' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-04T13:37:45.353 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:37:44 smithi169 ceph-mon[36313]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-04T13:37:45.353 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:37:44 smithi169 ceph-mon[36313]: from='client.? 172.21.15.107:0/1143752877' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-04T13:37:45.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:45.803+0000 7f8e6d400700 1 -- 172.21.15.107:0/3202278487 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e680702b0 msgr2=0x7f8e68070680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:37:45.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:45.803+0000 7f8e6d400700 1 --2- 172.21.15.107:0/3202278487 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e680702b0 0x7f8e68070680 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f8e58004940 tx=0x7f8e58009e30 comp rx=0 tx=0).stop 2024-03-04T13:37:45.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:45.804+0000 7f8e6d400700 1 -- 172.21.15.107:0/3202278487 shutdown_connections 2024-03-04T13:37:45.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:45.804+0000 7f8e6d400700 1 --2- 172.21.15.107:0/3202278487 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8e68070bc0 0x7f8e681092e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:45.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:45.804+0000 7f8e6d400700 1 --2- 172.21.15.107:0/3202278487 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e680702b0 0x7f8e68070680 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:45.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:45.804+0000 7f8e6d400700 1 -- 172.21.15.107:0/3202278487 >> 172.21.15.107:0/3202278487 conn(0x7f8e680f8ae0 msgr2=0x7f8e680faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:37:45.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:45.804+0000 7f8e6d400700 1 -- 172.21.15.107:0/3202278487 shutdown_connections 2024-03-04T13:37:45.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:45.804+0000 7f8e6d400700 1 -- 172.21.15.107:0/3202278487 wait complete. 2024-03-04T13:37:45.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:45.804+0000 7f8e6d400700 1 Processor -- start 2024-03-04T13:37:45.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:45.805+0000 7f8e6d400700 1 -- start start 2024-03-04T13:37:45.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:45.805+0000 7f8e6d400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e680702b0 0x7f8e68115c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:37:45.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:45.805+0000 7f8e6d400700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8e68070bc0 0x7f8e681161c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:37:45.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:45.805+0000 7f8e6d400700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e68116810 con 0x7f8e680702b0 2024-03-04T13:37:45.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:45.805+0000 7f8e6d400700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e6810fb40 con 0x7f8e68070bc0 2024-03-04T13:37:45.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:45.805+0000 7f8e66800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e680702b0 0x7f8e68115c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:37:45.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:45.805+0000 7f8e66800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e680702b0 0x7f8e68115c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49768/0 (socket says 172.21.15.107:49768) 2024-03-04T13:37:45.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:45.805+0000 7f8e66800700 1 -- 172.21.15.107:0/2607131877 learned_addr learned my addr 172.21.15.107:0/2607131877 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:37:45.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:45.806+0000 7f8e65e00700 1 --2- 172.21.15.107:0/2607131877 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8e68070bc0 0x7f8e681161c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:37:45.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:45.806+0000 7f8e66800700 1 -- 172.21.15.107:0/2607131877 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8e68070bc0 msgr2=0x7f8e681161c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:37:45.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:45.806+0000 7f8e66800700 1 --2- 172.21.15.107:0/2607131877 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8e68070bc0 0x7f8e681161c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:45.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:45.806+0000 7f8e66800700 1 -- 172.21.15.107:0/2607131877 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e58005040 con 0x7f8e680702b0 2024-03-04T13:37:45.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:45.806+0000 7f8e66800700 1 --2- 172.21.15.107:0/2607131877 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e680702b0 0x7f8e68115c80 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8e58009150 tx=0x7f8e58009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:37:45.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:45.806+0000 7f8e5ea00700 1 -- 172.21.15.107:0/2607131877 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8e5800e040 con 0x7f8e680702b0 2024-03-04T13:37:45.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:45.806+0000 7f8e6d400700 1 -- 172.21.15.107:0/2607131877 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e6810fdc0 con 0x7f8e680702b0 2024-03-04T13:37:45.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:45.806+0000 7f8e5ea00700 1 -- 172.21.15.107:0/2607131877 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8e5800bbf0 con 0x7f8e680702b0 2024-03-04T13:37:45.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:45.806+0000 7f8e5ea00700 1 -- 172.21.15.107:0/2607131877 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8e5801c630 con 0x7f8e680702b0 2024-03-04T13:37:45.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:45.806+0000 7f8e6d400700 1 -- 172.21.15.107:0/2607131877 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e68110300 con 0x7f8e680702b0 2024-03-04T13:37:45.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:45.808+0000 7f8e53e00700 1 -- 172.21.15.107:0/2607131877 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e680fc6e0 con 0x7f8e680702b0 2024-03-04T13:37:45.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:45.809+0000 7f8e5ea00700 1 -- 172.21.15.107:0/2607131877 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86623+0+0 (secure 0 0 0) 0x7f8e5800bd60 con 0x7f8e680702b0 2024-03-04T13:37:45.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:45.810+0000 7f8e5ea00700 1 --2- 172.21.15.107:0/2607131877 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f8e4c0692b0 0x7f8e4c06b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:37:45.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:45.810+0000 7f8e5ea00700 1 -- 172.21.15.107:0/2607131877 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f8e58088260 con 0x7f8e680702b0 2024-03-04T13:37:45.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:45.810+0000 7f8e65e00700 1 --2- 172.21.15.107:0/2607131877 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f8e4c0692b0 0x7f8e4c06b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:37:45.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:45.811+0000 7f8e65e00700 1 --2- 172.21.15.107:0/2607131877 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f8e4c0692b0 0x7f8e4c06b760 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8e681114b0 tx=0x7f8e54009040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:37:45.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:45.815+0000 7f8e5ea00700 1 -- 172.21.15.107:0/2607131877 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f8e58058340 con 0x7f8e680702b0 2024-03-04T13:37:45.886 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/mon.smithi107/config 2024-03-04T13:37:46.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:45.934+0000 7f6f12e00700 1 -- 172.21.15.107:0/1247222847 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f0c102f50 msgr2=0x7f6f0c103320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:37:46.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:45.934+0000 7f6f12e00700 1 --2- 172.21.15.107:0/1247222847 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f0c102f50 0x7f6f0c103320 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6efc004940 tx=0x7f6efc009e30 comp rx=0 tx=0).stop 2024-03-04T13:37:46.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:45.935+0000 7f6f12e00700 1 -- 172.21.15.107:0/1247222847 shutdown_connections 2024-03-04T13:37:46.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:45.935+0000 7f6f12e00700 1 --2- 172.21.15.107:0/1247222847 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f0c1038f0 0x7f6f0c0fcf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:46.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:45.935+0000 7f6f12e00700 1 --2- 172.21.15.107:0/1247222847 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f0c102f50 0x7f6f0c103320 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:46.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:45.935+0000 7f6f12e00700 1 -- 172.21.15.107:0/1247222847 >> 172.21.15.107:0/1247222847 conn(0x7f6f0c0f8ae0 msgr2=0x7f6f0c0faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:37:46.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:45.935+0000 7f6f12e00700 1 -- 172.21.15.107:0/1247222847 shutdown_connections 2024-03-04T13:37:46.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:45.935+0000 7f6f12e00700 1 -- 172.21.15.107:0/1247222847 wait complete. 2024-03-04T13:37:46.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:45.936+0000 7f6f12e00700 1 Processor -- start 2024-03-04T13:37:46.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:45.936+0000 7f6f12e00700 1 -- start start 2024-03-04T13:37:46.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:45.937+0000 7f6f12e00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f0c102f50 0x7f6f0c070290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:37:46.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:45.937+0000 7f6f12e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f0c1038f0 0x7f6f0c0707d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:37:46.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:45.937+0000 7f6f12e00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f0c070dd0 con 0x7f6f0c1038f0 2024-03-04T13:37:46.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:45.937+0000 7f6f12e00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f0c070f40 con 0x7f6f0c102f50 2024-03-04T13:37:46.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:45.937+0000 7f6f0b400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f0c1038f0 0x7f6f0c0707d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:37:46.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:45.937+0000 7f6f0b400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f0c1038f0 0x7f6f0c0707d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49780/0 (socket says 172.21.15.107:49780) 2024-03-04T13:37:46.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:45.937+0000 7f6f0b400700 1 -- 172.21.15.107:0/3008273079 learned_addr learned my addr 172.21.15.107:0/3008273079 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:37:46.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:45.937+0000 7f6f0be00700 1 --2- 172.21.15.107:0/3008273079 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f0c102f50 0x7f6f0c070290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:37:46.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:45.938+0000 7f6f0b400700 1 -- 172.21.15.107:0/3008273079 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f0c102f50 msgr2=0x7f6f0c070290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:37:46.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:45.938+0000 7f6f0b400700 1 --2- 172.21.15.107:0/3008273079 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f0c102f50 0x7f6f0c070290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:46.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:45.938+0000 7f6f0b400700 1 -- 172.21.15.107:0/3008273079 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6efc005040 con 0x7f6f0c1038f0 2024-03-04T13:37:46.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:45.939+0000 7f6f0b400700 1 --2- 172.21.15.107:0/3008273079 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f0c1038f0 0x7f6f0c0707d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f6ef8002a50 tx=0x7f6ef8004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:37:46.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:45.939+0000 7f6f03e00700 1 -- 172.21.15.107:0/3008273079 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ef8007cc0 con 0x7f6f0c1038f0 2024-03-04T13:37:46.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:45.939+0000 7f6f03e00700 1 -- 172.21.15.107:0/3008273079 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6ef800f040 con 0x7f6f0c1038f0 2024-03-04T13:37:46.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:45.939+0000 7f6f03e00700 1 -- 172.21.15.107:0/3008273079 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ef800d9c0 con 0x7f6f0c1038f0 2024-03-04T13:37:46.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:45.939+0000 7f6f12e00700 1 -- 172.21.15.107:0/3008273079 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f0c119c90 con 0x7f6f0c1038f0 2024-03-04T13:37:46.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:45.939+0000 7f6f12e00700 1 -- 172.21.15.107:0/3008273079 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f0c11a170 con 0x7f6f0c1038f0 2024-03-04T13:37:46.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:45.941+0000 7f6f12e00700 1 -- 172.21.15.107:0/3008273079 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f0c0fc6e0 con 0x7f6f0c1038f0 2024-03-04T13:37:46.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:45.944+0000 7f6f03e00700 1 -- 172.21.15.107:0/3008273079 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86623+0+0 (secure 0 0 0) 0x7f6ef8009730 con 0x7f6f0c1038f0 2024-03-04T13:37:46.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:45.945+0000 7f6f03e00700 1 --2- 172.21.15.107:0/3008273079 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f6ef00692a0 0x7f6ef006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:37:46.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:45.945+0000 7f6f03e00700 1 -- 172.21.15.107:0/3008273079 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f6ef8086b20 con 0x7f6f0c1038f0 2024-03-04T13:37:46.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:45.945+0000 7f6f0be00700 1 --2- 172.21.15.107:0/3008273079 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f6ef00692a0 0x7f6ef006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:37:46.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:45.946+0000 7f6f0be00700 1 --2- 172.21.15.107:0/3008273079 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f6ef00692a0 0x7f6ef006b750 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6efc004f90 tx=0x7f6efc009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:37:46.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:45.950+0000 7f6f03e00700 1 -- 172.21.15.107:0/3008273079 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f6ef80530f0 con 0x7f6f0c1038f0 2024-03-04T13:37:46.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:46.041+0000 7f8e53e00700 1 -- 172.21.15.107:0/2607131877 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f8e6804bed0 con 0x7f8e680702b0 2024-03-04T13:37:46.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:46.041+0000 7f8e5ea00700 1 -- 172.21.15.107:0/2607131877 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8e58057ed0 con 0x7f8e680702b0 2024-03-04T13:37:46.037 INFO:teuthology.orchestra.run.smithi107.stdout:171798691854 2024-03-04T13:37:46.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:46.044+0000 7f8e53e00700 1 -- 172.21.15.107:0/2607131877 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f8e4c0692b0 msgr2=0x7f8e4c06b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:37:46.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:46.044+0000 7f8e53e00700 1 --2- 172.21.15.107:0/2607131877 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f8e4c0692b0 0x7f8e4c06b760 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8e681114b0 tx=0x7f8e54009040 comp rx=0 tx=0).stop 2024-03-04T13:37:46.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:46.044+0000 7f8e53e00700 1 -- 172.21.15.107:0/2607131877 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e680702b0 msgr2=0x7f8e68115c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:37:46.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:46.044+0000 7f8e53e00700 1 --2- 172.21.15.107:0/2607131877 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e680702b0 0x7f8e68115c80 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8e58009150 tx=0x7f8e58009e30 comp rx=0 tx=0).stop 2024-03-04T13:37:46.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:46.045+0000 7f8e53e00700 1 -- 172.21.15.107:0/2607131877 shutdown_connections 2024-03-04T13:37:46.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:46.045+0000 7f8e53e00700 1 --2- 172.21.15.107:0/2607131877 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f8e4c0692b0 0x7f8e4c06b760 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:46.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:46.045+0000 7f8e53e00700 1 --2- 172.21.15.107:0/2607131877 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e680702b0 0x7f8e68115c80 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:46.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:46.045+0000 7f8e53e00700 1 --2- 172.21.15.107:0/2607131877 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8e68070bc0 0x7f8e681161c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:46.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:46.045+0000 7f8e53e00700 1 -- 172.21.15.107:0/2607131877 >> 172.21.15.107:0/2607131877 conn(0x7f8e680f8ae0 msgr2=0x7f8e68103b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:37:46.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:46.045+0000 7f8e53e00700 1 -- 172.21.15.107:0/2607131877 shutdown_connections 2024-03-04T13:37:46.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:46.045+0000 7f8e53e00700 1 -- 172.21.15.107:0/2607131877 wait complete. 2024-03-04T13:37:46.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:46.169+0000 7f6f12e00700 1 -- 172.21.15.107:0/3008273079 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f6f0c071a70 con 0x7f6f0c1038f0 2024-03-04T13:37:46.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:46.170+0000 7f6f03e00700 1 -- 172.21.15.107:0/3008273079 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6ef8056710 con 0x7f6f0c1038f0 2024-03-04T13:37:46.164 INFO:teuthology.orchestra.run.smithi107.stdout:150323855378 2024-03-04T13:37:46.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:46.173+0000 7f6f12e00700 1 -- 172.21.15.107:0/3008273079 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f6ef00692a0 msgr2=0x7f6ef006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:37:46.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:46.173+0000 7f6f12e00700 1 --2- 172.21.15.107:0/3008273079 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f6ef00692a0 0x7f6ef006b750 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6efc004f90 tx=0x7f6efc009e30 comp rx=0 tx=0).stop 2024-03-04T13:37:46.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:46.173+0000 7f6f12e00700 1 -- 172.21.15.107:0/3008273079 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f0c1038f0 msgr2=0x7f6f0c0707d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:37:46.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:46.173+0000 7f6f12e00700 1 --2- 172.21.15.107:0/3008273079 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f0c1038f0 0x7f6f0c0707d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f6ef8002a50 tx=0x7f6ef8004600 comp rx=0 tx=0).stop 2024-03-04T13:37:46.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:46.173+0000 7f6f12e00700 1 -- 172.21.15.107:0/3008273079 shutdown_connections 2024-03-04T13:37:46.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:46.173+0000 7f6f12e00700 1 --2- 172.21.15.107:0/3008273079 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f6ef00692a0 0x7f6ef006b750 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:46.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:46.173+0000 7f6f12e00700 1 --2- 172.21.15.107:0/3008273079 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6f0c102f50 0x7f6f0c070290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:46.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:46.173+0000 7f6f12e00700 1 --2- 172.21.15.107:0/3008273079 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f0c1038f0 0x7f6f0c0707d0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:46.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:46.173+0000 7f6f12e00700 1 -- 172.21.15.107:0/3008273079 >> 172.21.15.107:0/3008273079 conn(0x7f6f0c0f8ae0 msgr2=0x7f6f0c0fa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:37:46.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:46.173+0000 7f6f12e00700 1 -- 172.21.15.107:0/3008273079 shutdown_connections 2024-03-04T13:37:46.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:46.173+0000 7f6f12e00700 1 -- 172.21.15.107:0/3008273079 wait complete. 2024-03-04T13:37:46.958 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:37:46 smithi107 ceph-mon[28739]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-04T13:37:46.958 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:37:46 smithi107 ceph-mon[28739]: from='client.? 172.21.15.107:0/2607131877' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-04T13:37:46.958 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:37:46 smithi107 ceph-mon[28739]: from='client.? 172.21.15.107:0/3008273079' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-04T13:37:47.008 INFO:tasks.cephadm.ceph_manager.ceph:need seq 150323855376 got 150323855378 for osd.4 2024-03-04T13:37:47.009 DEBUG:teuthology.parallel:result is None 2024-03-04T13:37:47.353 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:37:46 smithi169 ceph-mon[36313]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-04T13:37:47.354 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:37:46 smithi169 ceph-mon[36313]: from='client.? 172.21.15.107:0/2607131877' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-04T13:37:47.354 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:37:46 smithi169 ceph-mon[36313]: from='client.? 172.21.15.107:0/3008273079' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-04T13:37:47.671 INFO:tasks.cephadm.ceph_manager.ceph:need seq 115964117014 got 115964117017 for osd.3 2024-03-04T13:37:47.671 DEBUG:teuthology.parallel:result is None 2024-03-04T13:37:48.461 INFO:tasks.cephadm.ceph_manager.ceph:need seq 171798691852 got 171798691854 for osd.5 2024-03-04T13:37:48.461 DEBUG:teuthology.parallel:result is None 2024-03-04T13:37:48.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:48.259+0000 7efe91000700 1 -- 172.21.15.107:0/906740359 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe8c0fdf50 msgr2=0x7efe8c0fe320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:37:48.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:48.259+0000 7efe91000700 1 --2- 172.21.15.107:0/906740359 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe8c0fdf50 0x7efe8c0fe320 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7efe7c004940 tx=0x7efe7c009e30 comp rx=0 tx=0).stop 2024-03-04T13:37:48.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:48.259+0000 7efe91000700 1 -- 172.21.15.107:0/906740359 shutdown_connections 2024-03-04T13:37:48.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:48.259+0000 7efe91000700 1 --2- 172.21.15.107:0/906740359 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe8c0fe8f0 0x7efe8c106f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:48.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:48.259+0000 7efe91000700 1 --2- 172.21.15.107:0/906740359 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe8c0fdf50 0x7efe8c0fe320 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:48.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:48.259+0000 7efe91000700 1 -- 172.21.15.107:0/906740359 >> 172.21.15.107:0/906740359 conn(0x7efe8c0f79e0 msgr2=0x7efe8c0f9df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:37:48.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:48.259+0000 7efe91000700 1 -- 172.21.15.107:0/906740359 shutdown_connections 2024-03-04T13:37:48.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:48.259+0000 7efe91000700 1 -- 172.21.15.107:0/906740359 wait complete. 2024-03-04T13:37:48.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:48.260+0000 7efe91000700 1 Processor -- start 2024-03-04T13:37:48.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:48.260+0000 7efe91000700 1 -- start start 2024-03-04T13:37:48.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:48.260+0000 7efe91000700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe8c0fdf50 0x7efe8c0fbd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:37:48.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:48.260+0000 7efe91000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe8c0fe8f0 0x7efe8c0fc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:37:48.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:48.260+0000 7efe91000700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe8c111950 con 0x7efe8c0fe8f0 2024-03-04T13:37:48.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:48.260+0000 7efe91000700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe8c111ac0 con 0x7efe8c0fdf50 2024-03-04T13:37:48.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:48.261+0000 7efe89400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe8c0fe8f0 0x7efe8c0fc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:37:48.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:48.261+0000 7efe89400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe8c0fe8f0 0x7efe8c0fc2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39612/0 (socket says 172.21.15.107:39612) 2024-03-04T13:37:48.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:48.261+0000 7efe89400700 1 -- 172.21.15.107:0/759843668 learned_addr learned my addr 172.21.15.107:0/759843668 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:37:48.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:48.261+0000 7efe89e00700 1 --2- 172.21.15.107:0/759843668 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe8c0fdf50 0x7efe8c0fbd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:37:48.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:48.261+0000 7efe89400700 1 -- 172.21.15.107:0/759843668 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe8c0fdf50 msgr2=0x7efe8c0fbd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:37:48.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:48.261+0000 7efe89400700 1 --2- 172.21.15.107:0/759843668 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe8c0fdf50 0x7efe8c0fbd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:48.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:48.261+0000 7efe89400700 1 -- 172.21.15.107:0/759843668 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe7c005040 con 0x7efe8c0fe8f0 2024-03-04T13:37:48.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:48.261+0000 7efe89400700 1 --2- 172.21.15.107:0/759843668 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe8c0fe8f0 0x7efe8c0fc2c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7efe78002a50 tx=0x7efe780046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:37:48.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:48.262+0000 7efe82000700 1 -- 172.21.15.107:0/759843668 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe78007cc0 con 0x7efe8c0fe8f0 2024-03-04T13:37:48.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:48.262+0000 7efe91000700 1 -- 172.21.15.107:0/759843668 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe8c0fc950 con 0x7efe8c0fe8f0 2024-03-04T13:37:48.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:48.262+0000 7efe82000700 1 -- 172.21.15.107:0/759843668 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efe7800f040 con 0x7efe8c0fe8f0 2024-03-04T13:37:48.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:48.262+0000 7efe82000700 1 -- 172.21.15.107:0/759843668 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe780095e0 con 0x7efe8c0fe8f0 2024-03-04T13:37:48.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:48.262+0000 7efe91000700 1 -- 172.21.15.107:0/759843668 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe8c11e650 con 0x7efe8c0fe8f0 2024-03-04T13:37:48.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:48.263+0000 7efe91000700 1 -- 172.21.15.107:0/759843668 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe8c104540 con 0x7efe8c0fe8f0 2024-03-04T13:37:48.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:48.264+0000 7efe82000700 1 -- 172.21.15.107:0/759843668 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86623+0+0 (secure 0 0 0) 0x7efe78009740 con 0x7efe8c0fe8f0 2024-03-04T13:37:48.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:48.265+0000 7efe82000700 1 --2- 172.21.15.107:0/759843668 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7efe700692a0 0x7efe7006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:37:48.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:48.265+0000 7efe82000700 1 -- 172.21.15.107:0/759843668 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7efe78086c20 con 0x7efe8c0fe8f0 2024-03-04T13:37:48.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:48.265+0000 7efe89e00700 1 --2- 172.21.15.107:0/759843668 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7efe700692a0 0x7efe7006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:37:48.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:48.265+0000 7efe89e00700 1 --2- 172.21.15.107:0/759843668 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7efe700692a0 0x7efe7006b750 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7efe7c004f90 tx=0x7efe7c009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:37:48.529 INFO:teuthology.orchestra.run.smithi107.stdout:60129542183 2024-03-04T13:37:48.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:48.269+0000 7efe82000700 1 -- 172.21.15.107:0/759843668 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7efe780532a0 con 0x7efe8c0fe8f0 2024-03-04T13:37:48.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:48.481+0000 7efe91000700 1 -- 172.21.15.107:0/759843668 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7efe8c0fd4c0 con 0x7efe8c0fe8f0 2024-03-04T13:37:48.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:48.481+0000 7efe82000700 1 -- 172.21.15.107:0/759843668 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7efe780568c0 con 0x7efe8c0fe8f0 2024-03-04T13:37:48.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:48.484+0000 7efe91000700 1 -- 172.21.15.107:0/759843668 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7efe700692a0 msgr2=0x7efe7006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:37:48.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:48.484+0000 7efe91000700 1 --2- 172.21.15.107:0/759843668 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7efe700692a0 0x7efe7006b750 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7efe7c004f90 tx=0x7efe7c009e30 comp rx=0 tx=0).stop 2024-03-04T13:37:48.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:48.484+0000 7efe91000700 1 -- 172.21.15.107:0/759843668 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe8c0fe8f0 msgr2=0x7efe8c0fc2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:37:48.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:48.484+0000 7efe91000700 1 --2- 172.21.15.107:0/759843668 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe8c0fe8f0 0x7efe8c0fc2c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7efe78002a50 tx=0x7efe780046b0 comp rx=0 tx=0).stop 2024-03-04T13:37:48.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:48.484+0000 7efe91000700 1 -- 172.21.15.107:0/759843668 shutdown_connections 2024-03-04T13:37:48.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:48.484+0000 7efe91000700 1 --2- 172.21.15.107:0/759843668 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7efe700692a0 0x7efe7006b750 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:48.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:48.484+0000 7efe91000700 1 --2- 172.21.15.107:0/759843668 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe8c0fdf50 0x7efe8c0fbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:48.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:48.484+0000 7efe91000700 1 --2- 172.21.15.107:0/759843668 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe8c0fe8f0 0x7efe8c0fc2c0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:48.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:48.484+0000 7efe91000700 1 -- 172.21.15.107:0/759843668 >> 172.21.15.107:0/759843668 conn(0x7efe8c0f79e0 msgr2=0x7efe8c1016c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:37:48.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:48.484+0000 7efe91000700 1 -- 172.21.15.107:0/759843668 shutdown_connections 2024-03-04T13:37:48.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:48.484+0000 7efe91000700 1 -- 172.21.15.107:0/759843668 wait complete. 2024-03-04T13:37:49.260 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:37:48 smithi107 ceph-mon[28739]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-04T13:37:49.260 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:37:48 smithi107 ceph-mon[28739]: from='client.? 172.21.15.107:0/759843668' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-04T13:37:49.353 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:37:48 smithi169 ceph-mon[36313]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-04T13:37:49.353 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:37:48 smithi169 ceph-mon[36313]: from='client.? 172.21.15.107:0/759843668' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-04T13:37:49.409 INFO:tasks.cephadm.ceph_manager.ceph:need seq 60129542179 got 60129542183 for osd.1 2024-03-04T13:37:49.410 DEBUG:teuthology.parallel:result is None 2024-03-04T13:37:51.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:51.038+0000 7fb636600700 1 -- 172.21.15.107:0/3392129972 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb630104f70 msgr2=0x7fb630105340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:37:51.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:51.038+0000 7fb636600700 1 --2- 172.21.15.107:0/3392129972 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb630104f70 0x7fb630105340 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb624004940 tx=0x7fb624009e30 comp rx=0 tx=0).stop 2024-03-04T13:37:51.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:51.038+0000 7fb636600700 1 -- 172.21.15.107:0/3392129972 shutdown_connections 2024-03-04T13:37:51.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:51.038+0000 7fb636600700 1 --2- 172.21.15.107:0/3392129972 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb630105910 0x7fb6300fef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:51.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:51.038+0000 7fb636600700 1 --2- 172.21.15.107:0/3392129972 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb630104f70 0x7fb630105340 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:51.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:51.038+0000 7fb636600700 1 -- 172.21.15.107:0/3392129972 >> 172.21.15.107:0/3392129972 conn(0x7fb6300faae0 msgr2=0x7fb6300fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:37:51.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:51.038+0000 7fb636600700 1 -- 172.21.15.107:0/3392129972 shutdown_connections 2024-03-04T13:37:51.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:51.038+0000 7fb636600700 1 -- 172.21.15.107:0/3392129972 wait complete. 2024-03-04T13:37:51.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:51.039+0000 7fb636600700 1 Processor -- start 2024-03-04T13:37:51.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:51.039+0000 7fb636600700 1 -- start start 2024-03-04T13:37:51.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:51.039+0000 7fb636600700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb630104f70 0x7fb630115a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:37:51.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:51.039+0000 7fb636600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb630105910 0x7fb630115f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:37:51.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:51.040+0000 7fb636600700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6301165b0 con 0x7fb630105910 2024-03-04T13:37:51.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:51.040+0000 7fb636600700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb63010f8e0 con 0x7fb630104f70 2024-03-04T13:37:51.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:51.040+0000 7fb62fe00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb630105910 0x7fb630115f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:37:51.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:51.040+0000 7fb62fe00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb630105910 0x7fb630115f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39636/0 (socket says 172.21.15.107:39636) 2024-03-04T13:37:51.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:51.040+0000 7fb62fe00700 1 -- 172.21.15.107:0/2501614002 learned_addr learned my addr 172.21.15.107:0/2501614002 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:37:51.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:51.040+0000 7fb635200700 1 --2- 172.21.15.107:0/2501614002 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb630104f70 0x7fb630115a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:37:51.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:51.040+0000 7fb62fe00700 1 -- 172.21.15.107:0/2501614002 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb630104f70 msgr2=0x7fb630115a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:37:51.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:51.040+0000 7fb62fe00700 1 --2- 172.21.15.107:0/2501614002 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb630104f70 0x7fb630115a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:51.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:51.040+0000 7fb62fe00700 1 -- 172.21.15.107:0/2501614002 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb624005040 con 0x7fb630105910 2024-03-04T13:37:51.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:51.041+0000 7fb62fe00700 1 --2- 172.21.15.107:0/2501614002 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb630105910 0x7fb630115f60 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb61c00c800 tx=0x7fb61c004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:37:51.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:51.041+0000 7fb62cc00700 1 -- 172.21.15.107:0/2501614002 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb61c00a760 con 0x7fb630105910 2024-03-04T13:37:51.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:51.041+0000 7fb62cc00700 1 -- 172.21.15.107:0/2501614002 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb61c00ad90 con 0x7fb630105910 2024-03-04T13:37:51.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:51.041+0000 7fb62cc00700 1 -- 172.21.15.107:0/2501614002 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb61c0113a0 con 0x7fb630105910 2024-03-04T13:37:51.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:51.041+0000 7fb636600700 1 -- 172.21.15.107:0/2501614002 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb63010fbc0 con 0x7fb630105910 2024-03-04T13:37:51.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:51.041+0000 7fb636600700 1 -- 172.21.15.107:0/2501614002 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb630110180 con 0x7fb630105910 2024-03-04T13:37:51.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:51.042+0000 7fb636600700 1 -- 172.21.15.107:0/2501614002 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb63005f5c0 con 0x7fb630105910 2024-03-04T13:37:51.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:51.043+0000 7fb62cc00700 1 -- 172.21.15.107:0/2501614002 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86623+0+0 (secure 0 0 0) 0x7fb61c012400 con 0x7fb630105910 2024-03-04T13:37:51.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:51.044+0000 7fb62cc00700 1 --2- 172.21.15.107:0/2501614002 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fb6180692a0 0x7fb61806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:37:51.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:51.044+0000 7fb62cc00700 1 -- 172.21.15.107:0/2501614002 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fb61c086dd0 con 0x7fb630105910 2024-03-04T13:37:51.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:51.044+0000 7fb635200700 1 --2- 172.21.15.107:0/2501614002 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fb6180692a0 0x7fb61806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:37:51.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:51.045+0000 7fb635200700 1 --2- 172.21.15.107:0/2501614002 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fb6180692a0 0x7fb61806b750 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb624004f90 tx=0x7fb624009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:37:51.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:51.049+0000 7fb62cc00700 1 -- 172.21.15.107:0/2501614002 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fb61c056f70 con 0x7fb630105910 2024-03-04T13:37:51.247 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:51.253+0000 7fb636600700 1 -- 172.21.15.107:0/2501614002 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fb630110450 con 0x7fb630105910 2024-03-04T13:37:51.247 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:37:50 smithi107 ceph-mon[28739]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-04T13:37:51.248 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:51.254+0000 7fb62cc00700 1 -- 172.21.15.107:0/2501614002 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb61c056b00 con 0x7fb630105910 2024-03-04T13:37:51.248 INFO:teuthology.orchestra.run.smithi107.stdout:81604378656 2024-03-04T13:37:51.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:51.257+0000 7fb636600700 1 -- 172.21.15.107:0/2501614002 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fb6180692a0 msgr2=0x7fb61806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:37:51.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:51.257+0000 7fb636600700 1 --2- 172.21.15.107:0/2501614002 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fb6180692a0 0x7fb61806b750 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb624004f90 tx=0x7fb624009e30 comp rx=0 tx=0).stop 2024-03-04T13:37:51.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:51.257+0000 7fb636600700 1 -- 172.21.15.107:0/2501614002 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb630105910 msgr2=0x7fb630115f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:37:51.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:51.257+0000 7fb636600700 1 --2- 172.21.15.107:0/2501614002 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb630105910 0x7fb630115f60 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb61c00c800 tx=0x7fb61c004470 comp rx=0 tx=0).stop 2024-03-04T13:37:51.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:51.258+0000 7fb636600700 1 -- 172.21.15.107:0/2501614002 shutdown_connections 2024-03-04T13:37:51.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:51.258+0000 7fb636600700 1 --2- 172.21.15.107:0/2501614002 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fb6180692a0 0x7fb61806b750 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:51.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:51.258+0000 7fb636600700 1 --2- 172.21.15.107:0/2501614002 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb630104f70 0x7fb630115a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:51.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:51.258+0000 7fb636600700 1 --2- 172.21.15.107:0/2501614002 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb630105910 0x7fb630115f60 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:51.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:51.258+0000 7fb636600700 1 -- 172.21.15.107:0/2501614002 >> 172.21.15.107:0/2501614002 conn(0x7fb6300faae0 msgr2=0x7fb6300fc560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:37:51.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:51.258+0000 7fb636600700 1 -- 172.21.15.107:0/2501614002 shutdown_connections 2024-03-04T13:37:51.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:51.258+0000 7fb636600700 1 -- 172.21.15.107:0/2501614002 wait complete. 2024-03-04T13:37:51.353 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:37:50 smithi169 ceph-mon[36313]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-04T13:37:52.189 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:37:51 smithi107 ceph-mon[28739]: from='client.? 172.21.15.107:0/2501614002' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-04T13:37:52.197 INFO:tasks.cephadm.ceph_manager.ceph:need seq 81604378654 got 81604378656 for osd.2 2024-03-04T13:37:52.197 DEBUG:teuthology.parallel:result is None 2024-03-04T13:37:52.197 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-03-04T13:37:52.197 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 -- ceph pg dump --format=json 2024-03-04T13:37:52.353 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:37:51 smithi169 ceph-mon[36313]: from='client.? 172.21.15.107:0/2501614002' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-04T13:37:52.676 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/mon.smithi107/config 2024-03-04T13:37:53.086 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:37:52 smithi107 ceph-mon[28739]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-04T13:37:53.353 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:37:52 smithi169 ceph-mon[36313]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-04T13:37:54.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:54.880+0000 7f903d800700 1 -- 172.21.15.107:0/1951498991 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9038102f50 msgr2=0x7f9038103320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:37:54.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:54.880+0000 7f903d800700 1 --2- 172.21.15.107:0/1951498991 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9038102f50 0x7f9038103320 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9028004940 tx=0x7f9028009e30 comp rx=0 tx=0).stop 2024-03-04T13:37:54.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:54.880+0000 7f903d800700 1 -- 172.21.15.107:0/1951498991 shutdown_connections 2024-03-04T13:37:54.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:54.880+0000 7f903d800700 1 --2- 172.21.15.107:0/1951498991 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f90381038f0 0x7f90380fcf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:54.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:54.880+0000 7f903d800700 1 --2- 172.21.15.107:0/1951498991 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9038102f50 0x7f9038103320 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:54.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:54.880+0000 7f903d800700 1 -- 172.21.15.107:0/1951498991 >> 172.21.15.107:0/1951498991 conn(0x7f90380f8ae0 msgr2=0x7f90380faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:37:54.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:54.880+0000 7f903d800700 1 -- 172.21.15.107:0/1951498991 shutdown_connections 2024-03-04T13:37:54.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:54.880+0000 7f903d800700 1 -- 172.21.15.107:0/1951498991 wait complete. 2024-03-04T13:37:54.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:54.881+0000 7f903d800700 1 Processor -- start 2024-03-04T13:37:54.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:54.881+0000 7f903d800700 1 -- start start 2024-03-04T13:37:54.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:54.882+0000 7f903d800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9038102f50 0x7f9038115c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:37:54.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:54.882+0000 7f903d800700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f90381038f0 0x7f9038116140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:37:54.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:54.882+0000 7f903d800700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9038116790 con 0x7f9038102f50 2024-03-04T13:37:54.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:54.882+0000 7f903d800700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f903810fae0 con 0x7f90381038f0 2024-03-04T13:37:54.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:54.882+0000 7f9036a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9038102f50 0x7f9038115c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:37:54.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:54.882+0000 7f9036a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9038102f50 0x7f9038115c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39654/0 (socket says 172.21.15.107:39654) 2024-03-04T13:37:54.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:54.882+0000 7f9036a00700 1 -- 172.21.15.107:0/1394584414 learned_addr learned my addr 172.21.15.107:0/1394584414 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:37:54.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:54.882+0000 7f9036000700 1 --2- 172.21.15.107:0/1394584414 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f90381038f0 0x7f9038116140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:37:54.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:54.882+0000 7f9036a00700 1 -- 172.21.15.107:0/1394584414 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f90381038f0 msgr2=0x7f9038116140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:37:54.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:54.882+0000 7f9036a00700 1 --2- 172.21.15.107:0/1394584414 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f90381038f0 0x7f9038116140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:54.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:54.882+0000 7f9036a00700 1 -- 172.21.15.107:0/1394584414 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9028005040 con 0x7f9038102f50 2024-03-04T13:37:54.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:54.883+0000 7f9036a00700 1 --2- 172.21.15.107:0/1394584414 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9038102f50 0x7f9038115c00 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9028009fd0 tx=0x7f9028009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:37:54.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:54.883+0000 7f902ea00700 1 -- 172.21.15.107:0/1394584414 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f902800e040 con 0x7f9038102f50 2024-03-04T13:37:54.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:54.883+0000 7f902ea00700 1 -- 172.21.15.107:0/1394584414 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f902800bbf0 con 0x7f9038102f50 2024-03-04T13:37:54.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:54.883+0000 7f902ea00700 1 -- 172.21.15.107:0/1394584414 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f902801c3f0 con 0x7f9038102f50 2024-03-04T13:37:54.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:54.883+0000 7f903d800700 1 -- 172.21.15.107:0/1394584414 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f903810fd60 con 0x7f9038102f50 2024-03-04T13:37:54.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:54.883+0000 7f903d800700 1 -- 172.21.15.107:0/1394584414 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90381102a0 con 0x7f9038102f50 2024-03-04T13:37:54.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:54.884+0000 7f903d800700 1 -- 172.21.15.107:0/1394584414 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90380fc6e0 con 0x7f9038102f50 2024-03-04T13:37:54.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:54.885+0000 7f902ea00700 1 -- 172.21.15.107:0/1394584414 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86623+0+0 (secure 0 0 0) 0x7f902800bd60 con 0x7f9038102f50 2024-03-04T13:37:54.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:54.886+0000 7f902ea00700 1 --2- 172.21.15.107:0/1394584414 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f901c0692b0 0x7f901c06b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:37:54.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:54.886+0000 7f902ea00700 1 -- 172.21.15.107:0/1394584414 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f9028088170 con 0x7f9038102f50 2024-03-04T13:37:54.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:54.886+0000 7f9036000700 1 --2- 172.21.15.107:0/1394584414 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f901c0692b0 0x7f901c06b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:37:54.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:54.887+0000 7f9036000700 1 --2- 172.21.15.107:0/1394584414 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f901c0692b0 0x7f901c06b760 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f90381114a0 tx=0x7f902400b040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:37:54.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:54.890+0000 7f902ea00700 1 -- 172.21.15.107:0/1394584414 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f90280547c0 con 0x7f9038102f50 2024-03-04T13:37:55.086 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:37:54 smithi107 ceph-mon[28739]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-04T13:37:55.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:55.097+0000 7f903d800700 1 -- 172.21.15.107:0/1394584414 --> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9038111280 con 0x7f901c0692b0 2024-03-04T13:37:55.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:55.099+0000 7f902ea00700 1 -- 172.21.15.107:0/1394584414 <== mgr.14212 v2:172.21.15.107:6800/3848495603 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+27317 (secure 0 0 0) 0x7f9038111280 con 0x7f901c0692b0 2024-03-04T13:37:55.093 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:37:55.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:55.102+0000 7f903d800700 1 -- 172.21.15.107:0/1394584414 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f901c0692b0 msgr2=0x7f901c06b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:37:55.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:55.102+0000 7f903d800700 1 --2- 172.21.15.107:0/1394584414 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f901c0692b0 0x7f901c06b760 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f90381114a0 tx=0x7f902400b040 comp rx=0 tx=0).stop 2024-03-04T13:37:55.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:55.102+0000 7f903d800700 1 -- 172.21.15.107:0/1394584414 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9038102f50 msgr2=0x7f9038115c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:37:55.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:55.102+0000 7f903d800700 1 --2- 172.21.15.107:0/1394584414 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9038102f50 0x7f9038115c00 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9028009fd0 tx=0x7f9028009e30 comp rx=0 tx=0).stop 2024-03-04T13:37:55.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:55.102+0000 7f903d800700 1 -- 172.21.15.107:0/1394584414 shutdown_connections 2024-03-04T13:37:55.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:55.102+0000 7f903d800700 1 --2- 172.21.15.107:0/1394584414 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f901c0692b0 0x7f901c06b760 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:55.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:55.102+0000 7f903d800700 1 --2- 172.21.15.107:0/1394584414 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9038102f50 0x7f9038115c00 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:55.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:55.102+0000 7f903d800700 1 --2- 172.21.15.107:0/1394584414 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f90381038f0 0x7f9038116140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:55.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:55.102+0000 7f903d800700 1 -- 172.21.15.107:0/1394584414 >> 172.21.15.107:0/1394584414 conn(0x7f90380f8ae0 msgr2=0x7f90380fa370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:37:55.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:55.102+0000 7f903d800700 1 -- 172.21.15.107:0/1394584414 shutdown_connections 2024-03-04T13:37:55.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:55.103+0000 7f903d800700 1 -- 172.21.15.107:0/1394584414 wait complete. 2024-03-04T13:37:55.097 INFO:teuthology.orchestra.run.smithi107.stderr:dumped all 2024-03-04T13:37:55.353 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:37:54 smithi169 ceph-mon[36313]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-04T13:37:55.909 INFO:teuthology.orchestra.run.smithi107.stdout:{"pg_ready":true,"pg_map":{"version":192,"stamp":"2024-03-04T13:37:53.320768+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":1787652,"kb_used_data":3924,"kb_used_omap":0,"kb_used_meta":1783680,"kb_avail":560683260,"statfs":{"total":575970213888,"available":574139658240,"internally_reserved":0,"allocated":4018176,"data_stored":2624466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1826488320},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003142"},"pg_stats":[{"pgid":"1.0","version":"23'117","reported_seq":189,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-04T13:36:53.673117+0000","last_change":"2024-03-04T13:36:21.589247+0000","last_active":"2024-03-04T13:36:53.673117+0000","last_peered":"2024-03-04T13:36:53.673117+0000","last_clean":"2024-03-04T13:36:53.673117+0000","last_became_active":"2024-03-04T13:36:21.588134+0000","last_became_peered":"2024-03-04T13:36:21.588134+0000","last_unstale":"2024-03-04T13:36:53.673117+0000","last_undegraded":"2024-03-04T13:36:53.673117+0000","last_fullsized":"2024-03-04T13:36:53.673117+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T13:35:17.966141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T13:35:17.966141+0000","last_clean_scrub_stamp":"2024-03-04T13:35:17.966141+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:11:18.675453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":40,"seq":171798691855,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297652,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93447500,"statfs":{"total":95995035648,"available":95690240000,"internally_reserved":0,"allocated":372736,"data_stored":142227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.173}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90700000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95299999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.069}]}]},{"osd":4,"up_from":35,"seq":150323855380,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297652,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93447500,"statfs":{"total":95995035648,"available":95690240000,"internally_reserved":0,"allocated":372736,"data_stored":142227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Mar 4 13:37:21 2024","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.4199999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":1.177},{"interface":"front","average":{"1min":1.056,"5min":1.056,"15min":1.056},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":1.341}]},{"osd":1,"last update":"Mon Mar 4 13:37:21 2024","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":1.01,"15min":1.01},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.6870000000000001,"5min":1.6870000000000001,"15min":1.6870000000000001},"last":0.90600000000000003},{"interface":"front","average":{"1min":1.036,"5min":1.036,"15min":1.036},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.603,"5min":1.603,"15min":1.603},"last":1.1100000000000001}]},{"osd":2,"last update":"Mon Mar 4 13:37:21 2024","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":1.048,"15min":1.048},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.571,"5min":1.571,"15min":1.571},"last":1.008},{"interface":"front","average":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.534,"5min":1.534,"15min":1.534},"last":1.2549999999999999}]},{"osd":3,"last update":"Mon Mar 4 13:37:21 2024","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.7969999999999999,"5min":1.7969999999999999,"15min":1.7969999999999999},"last":1.458},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":2.2759999999999998,"5min":2.2759999999999998,"15min":2.2759999999999998},"last":1.143}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.228}]}]},{"osd":3,"up_from":27,"seq":115964117019,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298232,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446920,"statfs":{"total":95995035648,"available":95689646080,"internally_reserved":0,"allocated":966656,"data_stored":732595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Mar 4 13:36:50 2024","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.014,"15min":1.014},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.393,"5min":1.393,"15min":1.393},"last":0.997},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":1.083}]},{"osd":1,"last update":"Mon Mar 4 13:36:50 2024","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"min":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":0.84399999999999997},{"interface":"front","average":{"1min":1.04,"5min":1.04,"15min":1.04},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.8029999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":0.753}]},{"osd":2,"last update":"Mon Mar 4 13:36:50 2024","interfaces":[{"interface":"back","average":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.7,"5min":1.7,"15min":1.7},"last":1.1539999999999999},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":1.109}]},{"osd":4,"last update":"Mon Mar 4 13:37:23 2024","interfaces":[{"interface":"back","average":{"1min":0.999,"5min":0.999,"15min":0.999},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.8700000000000001,"5min":1.8700000000000001,"15min":1.8700000000000001},"last":0.80600000000000005},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.7529999999999999,"5min":1.7529999999999999,"15min":1.7529999999999999},"last":1.24}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1879999999999999}]}]},{"osd":2,"up_from":19,"seq":81604378657,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297652,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93447500,"statfs":{"total":95995035648,"available":95690240000,"internally_reserved":0,"allocated":372736,"data_stored":142227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Mar 4 13:37:17 2024","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.80800000000000005,"15min":0.79700000000000004},"min":{"1min":0.626,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.88,"15min":0.879},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.66500000000000004}]},{"osd":1,"last update":"Mon Mar 4 13:37:17 2024","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.91500000000000004,"15min":0.92300000000000004},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.0329999999999999,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.63900000000000001},{"interface":"front","average":{"1min":1,"5min":0.98799999999999999,"15min":0.98599999999999999},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.4059999999999999,"5min":1.55,"15min":1.55},"last":0.71599999999999997}]},{"osd":3,"last update":"Mon Mar 4 13:37:51 2024","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.155,"15min":1.137},"min":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":2.1179999999999999,"5min":2.1179999999999999,"15min":2.1179999999999999},"last":1.0820000000000001},{"interface":"front","average":{"1min":1.181,"5min":1.1180000000000001,"15min":1.1080000000000001},"min":{"1min":0.82499999999999996,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.8839999999999999,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":0.85799999999999998}]},{"osd":4,"last update":"Mon Mar 4 13:37:25 2024","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"min":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"max":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.93200000000000005},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"min":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"max":{"1min":1.6040000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":0.96099999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91700000000000004}]}]},{"osd":0,"up_from":9,"seq":38654705714,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298232,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446920,"statfs":{"total":95995035648,"available":95689646080,"internally_reserved":0,"allocated":966656,"data_stored":732595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Mar 4 13:37:49 2024","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.76600000000000001,"15min":0.71599999999999997},"min":{"1min":0.55900000000000005,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.6679999999999999,"5min":1.6679999999999999,"15min":1.6679999999999999},"last":0.501},{"interface":"front","average":{"1min":1.016,"5min":0.79700000000000004,"15min":0.74299999999999999},"min":{"1min":0.63100000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.8380000000000001,"5min":1.8380000000000001,"15min":1.8380000000000001},"last":0.82499999999999996}]},{"osd":2,"last update":"Mon Mar 4 13:37:22 2024","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":0.94899999999999995,"15min":0.93799999999999994},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":2.1259999999999999,"5min":2.3149999999999999,"15min":2.3149999999999999},"last":0.48199999999999998},{"interface":"front","average":{"1min":1.056,"5min":1.018,"15min":1.012},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.446,"5min":1.958,"15min":1.958},"last":0.74299999999999999}]},{"osd":3,"last update":"Mon Mar 4 13:36:53 2024","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"min":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":1.619,"5min":1.619,"15min":1.619},"last":1.0369999999999999},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"min":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"max":{"1min":1.7370000000000001,"5min":1.7370000000000001,"15min":1.7370000000000001},"last":1.159}]},{"osd":4,"last update":"Mon Mar 4 13:37:22 2024","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"min":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":2.2160000000000002,"5min":2.2160000000000002,"15min":2.2160000000000002},"last":1.006},{"interface":"front","average":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.7689999999999999,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":0.97999999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91400000000000003}]}]},{"osd":1,"up_from":14,"seq":60129542184,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298232,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446920,"statfs":{"total":95995035648,"available":95689646080,"internally_reserved":0,"allocated":966656,"data_stored":732595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Mar 4 13:37:41 2024","interfaces":[{"interface":"back","average":{"1min":0.94999999999999996,"5min":0.745,"15min":0.68999999999999995},"min":{"1min":0.44900000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.4870000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.82199999999999995,"15min":0.78300000000000003},"min":{"1min":0.45800000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.7190000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":0.72299999999999998}]},{"osd":2,"last update":"Mon Mar 4 13:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.89100000000000001,"15min":0.89500000000000002},"min":{"1min":0.51500000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.2,"5min":1.7170000000000001,"15min":1.7170000000000001},"last":1.0649999999999999},{"interface":"front","average":{"1min":1.0049999999999999,"5min":0.94699999999999995,"15min":0.93700000000000006},"min":{"1min":0.65600000000000003,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.2470000000000001,"5min":1.8,"15min":1.8},"last":1.03}]},{"osd":3,"last update":"Mon Mar 4 13:36:51 2024","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":1.093,"15min":1.093},"min":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"max":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":1.2949999999999999},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.5269999999999999,"5min":1.5269999999999999,"15min":1.5269999999999999},"last":1.266}]},{"osd":4,"last update":"Mon Mar 4 13:37:23 2024","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":1.167,"15min":1.167},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.645,"5min":1.645,"15min":1.645},"last":1.0029999999999999},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"min":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"max":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":1.5980000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7250000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-04T13:37:55.910 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 -- ceph pg dump --format=json 2024-03-04T13:37:56.382 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/mon.smithi107/config 2024-03-04T13:37:57.336 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:37:56 smithi107 ceph-mon[28739]: from='client.14418 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-04T13:37:57.336 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:37:56 smithi107 ceph-mon[28739]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-04T13:37:57.353 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:37:56 smithi169 ceph-mon[36313]: from='client.14418 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-04T13:37:57.353 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:37:56 smithi169 ceph-mon[36313]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-04T13:37:57.985 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:37:57 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:37:58.353 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:37:57 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:37:58.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:58.424+0000 7efd6e600700 1 -- 172.21.15.107:0/282476096 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd681038f0 msgr2=0x7efd680fcf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:37:58.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:58.424+0000 7efd6e600700 1 --2- 172.21.15.107:0/282476096 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd681038f0 0x7efd680fcf60 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7efd54004940 tx=0x7efd54009e30 comp rx=0 tx=0).stop 2024-03-04T13:37:58.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:58.424+0000 7efd6e600700 1 -- 172.21.15.107:0/282476096 shutdown_connections 2024-03-04T13:37:58.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:58.424+0000 7efd6e600700 1 --2- 172.21.15.107:0/282476096 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd681038f0 0x7efd680fcf60 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:58.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:58.424+0000 7efd6e600700 1 --2- 172.21.15.107:0/282476096 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd68102f50 0x7efd68103320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:58.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:58.424+0000 7efd6e600700 1 -- 172.21.15.107:0/282476096 >> 172.21.15.107:0/282476096 conn(0x7efd680f8ae0 msgr2=0x7efd680faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:37:58.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:58.424+0000 7efd6e600700 1 -- 172.21.15.107:0/282476096 shutdown_connections 2024-03-04T13:37:58.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:58.424+0000 7efd6e600700 1 -- 172.21.15.107:0/282476096 wait complete. 2024-03-04T13:37:58.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:58.425+0000 7efd6e600700 1 Processor -- start 2024-03-04T13:37:58.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:58.425+0000 7efd6e600700 1 -- start start 2024-03-04T13:37:58.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:58.426+0000 7efd6e600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd68102f50 0x7efd68115c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:37:58.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:58.426+0000 7efd6e600700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd681038f0 0x7efd681161c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:37:58.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:58.426+0000 7efd6e600700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd68116810 con 0x7efd68102f50 2024-03-04T13:37:58.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:58.426+0000 7efd6e600700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd6810fb40 con 0x7efd681038f0 2024-03-04T13:37:58.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:58.426+0000 7efd67400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd68102f50 0x7efd68115c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:37:58.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:58.426+0000 7efd67400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd68102f50 0x7efd68115c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56258/0 (socket says 172.21.15.107:56258) 2024-03-04T13:37:58.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:58.426+0000 7efd67400700 1 -- 172.21.15.107:0/2811885342 learned_addr learned my addr 172.21.15.107:0/2811885342 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:37:58.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:58.426+0000 7efd66a00700 1 --2- 172.21.15.107:0/2811885342 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd681038f0 0x7efd681161c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:37:58.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:58.427+0000 7efd67400700 1 -- 172.21.15.107:0/2811885342 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd681038f0 msgr2=0x7efd681161c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:37:58.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:58.427+0000 7efd67400700 1 --2- 172.21.15.107:0/2811885342 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd681038f0 0x7efd681161c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:58.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:58.427+0000 7efd67400700 1 -- 172.21.15.107:0/2811885342 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd54005040 con 0x7efd68102f50 2024-03-04T13:37:58.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:58.427+0000 7efd66a00700 1 --2- 172.21.15.107:0/2811885342 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd681038f0 0x7efd681161c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T13:37:58.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:58.427+0000 7efd67400700 1 --2- 172.21.15.107:0/2811885342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd68102f50 0x7efd68115c80 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7efd5800c8c0 tx=0x7efd580044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:37:58.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:58.427+0000 7efd5f400700 1 -- 172.21.15.107:0/2811885342 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efd5800a980 con 0x7efd68102f50 2024-03-04T13:37:58.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:58.427+0000 7efd6e600700 1 -- 172.21.15.107:0/2811885342 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd6810fe20 con 0x7efd68102f50 2024-03-04T13:37:58.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:58.427+0000 7efd5f400700 1 -- 172.21.15.107:0/2811885342 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efd5800ce60 con 0x7efd68102f50 2024-03-04T13:37:58.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:58.427+0000 7efd5f400700 1 -- 172.21.15.107:0/2811885342 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efd580115e0 con 0x7efd68102f50 2024-03-04T13:37:58.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:58.427+0000 7efd6e600700 1 -- 172.21.15.107:0/2811885342 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd68110360 con 0x7efd68102f50 2024-03-04T13:37:58.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:58.429+0000 7efd6e600700 1 -- 172.21.15.107:0/2811885342 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd680fc6e0 con 0x7efd68102f50 2024-03-04T13:37:58.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:58.430+0000 7efd5f400700 1 -- 172.21.15.107:0/2811885342 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86623+0+0 (secure 0 0 0) 0x7efd5800aae0 con 0x7efd68102f50 2024-03-04T13:37:58.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:58.431+0000 7efd5f400700 1 --2- 172.21.15.107:0/2811885342 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7efd4c0692a0 0x7efd4c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:37:58.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:58.431+0000 7efd5f400700 1 -- 172.21.15.107:0/2811885342 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7efd58086ff0 con 0x7efd68102f50 2024-03-04T13:37:58.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:58.432+0000 7efd66a00700 1 --2- 172.21.15.107:0/2811885342 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7efd4c0692a0 0x7efd4c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:37:58.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:58.432+0000 7efd66a00700 1 --2- 172.21.15.107:0/2811885342 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7efd4c0692a0 0x7efd4c06b750 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7efd681114b0 tx=0x7efd54009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:37:58.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:58.436+0000 7efd5f400700 1 -- 172.21.15.107:0/2811885342 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7efd58057090 con 0x7efd68102f50 2024-03-04T13:37:58.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:58.643+0000 7efd6e600700 1 -- 172.21.15.107:0/2811885342 --> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7efd681111f0 con 0x7efd4c0692a0 2024-03-04T13:37:58.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:58.645+0000 7efd5f400700 1 -- 172.21.15.107:0/2811885342 <== mgr.14212 v2:172.21.15.107:6800/3848495603 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+29762 (secure 0 0 0) 0x7efd681111f0 con 0x7efd4c0692a0 2024-03-04T13:37:58.639 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:37:58.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:58.648+0000 7efd6e600700 1 -- 172.21.15.107:0/2811885342 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7efd4c0692a0 msgr2=0x7efd4c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:37:58.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:58.648+0000 7efd6e600700 1 --2- 172.21.15.107:0/2811885342 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7efd4c0692a0 0x7efd4c06b750 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7efd681114b0 tx=0x7efd54009e30 comp rx=0 tx=0).stop 2024-03-04T13:37:58.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:58.648+0000 7efd6e600700 1 -- 172.21.15.107:0/2811885342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd68102f50 msgr2=0x7efd68115c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:37:58.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:58.648+0000 7efd6e600700 1 --2- 172.21.15.107:0/2811885342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd68102f50 0x7efd68115c80 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7efd5800c8c0 tx=0x7efd580044c0 comp rx=0 tx=0).stop 2024-03-04T13:37:58.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:58.648+0000 7efd6e600700 1 -- 172.21.15.107:0/2811885342 shutdown_connections 2024-03-04T13:37:58.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:58.648+0000 7efd6e600700 1 --2- 172.21.15.107:0/2811885342 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7efd4c0692a0 0x7efd4c06b750 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:58.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:58.648+0000 7efd6e600700 1 --2- 172.21.15.107:0/2811885342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd68102f50 0x7efd68115c80 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:58.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:58.648+0000 7efd6e600700 1 --2- 172.21.15.107:0/2811885342 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd681038f0 0x7efd681161c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:37:58.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:58.648+0000 7efd6e600700 1 -- 172.21.15.107:0/2811885342 >> 172.21.15.107:0/2811885342 conn(0x7efd680f8ae0 msgr2=0x7efd680fa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:37:58.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:58.648+0000 7efd6e600700 1 -- 172.21.15.107:0/2811885342 shutdown_connections 2024-03-04T13:37:58.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:37:58.649+0000 7efd6e600700 1 -- 172.21.15.107:0/2811885342 wait complete. 2024-03-04T13:37:58.643 INFO:teuthology.orchestra.run.smithi107.stderr:dumped all 2024-03-04T13:37:59.259 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:37:58 smithi107 ceph-mon[28739]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-04T13:37:59.259 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:37:58 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:37:59.346 INFO:teuthology.orchestra.run.smithi107.stdout:{"pg_ready":true,"pg_map":{"version":194,"stamp":"2024-03-04T13:37:57.322994+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":1787652,"kb_used_data":3924,"kb_used_omap":0,"kb_used_meta":1783680,"kb_avail":560683260,"statfs":{"total":575970213888,"available":574139658240,"internally_reserved":0,"allocated":4018176,"data_stored":2624466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1826488320},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004292"},"pg_stats":[{"pgid":"1.0","version":"23'117","reported_seq":189,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-04T13:36:53.673117+0000","last_change":"2024-03-04T13:36:21.589247+0000","last_active":"2024-03-04T13:36:53.673117+0000","last_peered":"2024-03-04T13:36:53.673117+0000","last_clean":"2024-03-04T13:36:53.673117+0000","last_became_active":"2024-03-04T13:36:21.588134+0000","last_became_peered":"2024-03-04T13:36:21.588134+0000","last_unstale":"2024-03-04T13:36:53.673117+0000","last_undegraded":"2024-03-04T13:36:53.673117+0000","last_fullsized":"2024-03-04T13:36:53.673117+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T13:35:17.966141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T13:35:17.966141+0000","last_clean_scrub_stamp":"2024-03-04T13:35:17.966141+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:11:18.675453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":40,"seq":171798691856,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297652,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93447500,"statfs":{"total":95995035648,"available":95690240000,"internally_reserved":0,"allocated":372736,"data_stored":142227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Mar 4 13:37:53 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.393,"5min":1.393,"15min":1.393},"last":0.80300000000000005},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":1.1919999999999999}]},{"osd":1,"last update":"Mon Mar 4 13:37:53 2024","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.3680000000000001,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":1.3680000000000001},{"interface":"front","average":{"1min":1.052,"5min":1.052,"15min":1.052},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.554,"5min":1.554,"15min":1.554},"last":1}]},{"osd":2,"last update":"Mon Mar 4 13:37:53 2024","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.379,"5min":1.379,"15min":1.379},"last":1.145},{"interface":"front","average":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"min":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":1.613,"5min":1.613,"15min":1.613},"last":1.52}]},{"osd":3,"last update":"Mon Mar 4 13:37:53 2024","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":1.093,"15min":1.093},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.726,"5min":1.726,"15min":1.726},"last":0.96599999999999997},{"interface":"front","average":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"min":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":2.036,"5min":2.036,"15min":2.036},"last":1.4550000000000001}]},{"osd":4,"last update":"Mon Mar 4 13:37:53 2024","interfaces":[{"interface":"back","average":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"min":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"max":{"1min":2.9039999999999999,"5min":2.9039999999999999,"15min":2.9039999999999999},"last":1.2689999999999999},{"interface":"front","average":{"1min":1.363,"5min":1.363,"15min":1.363},"min":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"max":{"1min":2.2149999999999999,"5min":2.2149999999999999,"15min":2.2149999999999999},"last":1.3220000000000001}]}]},{"osd":4,"up_from":35,"seq":150323855380,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297652,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93447500,"statfs":{"total":95995035648,"available":95690240000,"internally_reserved":0,"allocated":372736,"data_stored":142227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Mar 4 13:37:21 2024","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.4199999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":1.177},{"interface":"front","average":{"1min":1.056,"5min":1.056,"15min":1.056},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":1.341}]},{"osd":1,"last update":"Mon Mar 4 13:37:21 2024","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":1.01,"15min":1.01},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.6870000000000001,"5min":1.6870000000000001,"15min":1.6870000000000001},"last":0.90600000000000003},{"interface":"front","average":{"1min":1.036,"5min":1.036,"15min":1.036},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.603,"5min":1.603,"15min":1.603},"last":1.1100000000000001}]},{"osd":2,"last update":"Mon Mar 4 13:37:21 2024","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":1.048,"15min":1.048},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.571,"5min":1.571,"15min":1.571},"last":1.008},{"interface":"front","average":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.534,"5min":1.534,"15min":1.534},"last":1.2549999999999999}]},{"osd":3,"last update":"Mon Mar 4 13:37:21 2024","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.7969999999999999,"5min":1.7969999999999999,"15min":1.7969999999999999},"last":1.458},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":2.2759999999999998,"5min":2.2759999999999998,"15min":2.2759999999999998},"last":1.143}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.228}]}]},{"osd":3,"up_from":27,"seq":115964117020,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298232,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446920,"statfs":{"total":95995035648,"available":95689646080,"internally_reserved":0,"allocated":966656,"data_stored":732595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Mar 4 13:37:53 2024","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":1.0229999999999999,"15min":1.0169999999999999},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.716,"5min":1.716,"15min":1.716},"last":1.1859999999999999},{"interface":"front","average":{"1min":1.24,"5min":0.99099999999999999,"15min":0.94899999999999995},"min":{"1min":0.747,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":4.1520000000000001,"5min":4.1520000000000001,"15min":4.1520000000000001},"last":0.91500000000000004}]},{"osd":1,"last update":"Mon Mar 4 13:37:53 2024","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":1.018,"15min":1.018},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.5069999999999999,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":1.5069999999999999},{"interface":"front","average":{"1min":1.022,"5min":1.036,"15min":1.038},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.4039999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":1.0880000000000001}]},{"osd":2,"last update":"Mon Mar 4 13:37:53 2024","interfaces":[{"interface":"back","average":{"1min":1.0860000000000001,"5min":1.0680000000000001,"15min":1.0649999999999999},"min":{"1min":0.81200000000000006,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.3720000000000001,"5min":1.7,"15min":1.7},"last":1.0389999999999999},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.117,"15min":1.103},"min":{"1min":0.80900000000000005,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":1.401}]},{"osd":4,"last update":"Mon Mar 4 13:37:23 2024","interfaces":[{"interface":"back","average":{"1min":0.999,"5min":0.999,"15min":0.999},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.8700000000000001,"5min":1.8700000000000001,"15min":1.8700000000000001},"last":1.1359999999999999},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.7529999999999999,"5min":1.7529999999999999,"15min":1.7529999999999999},"last":1.5680000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.254}]}]},{"osd":2,"up_from":19,"seq":81604378658,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297652,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93447500,"statfs":{"total":95995035648,"available":95690240000,"internally_reserved":0,"allocated":372736,"data_stored":142227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Mar 4 13:37:17 2024","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.80800000000000005,"15min":0.79700000000000004},"min":{"1min":0.626,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.88,"15min":0.879},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.89200000000000002}]},{"osd":1,"last update":"Mon Mar 4 13:37:17 2024","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.91500000000000004,"15min":0.92300000000000004},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.0329999999999999,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.82899999999999996},{"interface":"front","average":{"1min":1,"5min":0.98799999999999999,"15min":0.98599999999999999},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.4059999999999999,"5min":1.55,"15min":1.55},"last":1.012}]},{"osd":3,"last update":"Mon Mar 4 13:37:51 2024","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.155,"15min":1.137},"min":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":2.1179999999999999,"5min":2.1179999999999999,"15min":2.1179999999999999},"last":1.4330000000000001},{"interface":"front","average":{"1min":1.181,"5min":1.1180000000000001,"15min":1.1080000000000001},"min":{"1min":0.82499999999999996,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.8839999999999999,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":1.0629999999999999}]},{"osd":4,"last update":"Mon Mar 4 13:37:25 2024","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"min":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"max":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":1.1000000000000001},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"min":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"max":{"1min":1.6040000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":1.1870000000000001}]},{"osd":5,"last update":"Mon Mar 4 13:37:53 2024","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.218,"15min":1.218},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":2.2069999999999999,"5min":2.2069999999999999,"15min":2.2069999999999999},"last":1.3089999999999999},{"interface":"front","average":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"min":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":2.2810000000000001,"5min":2.2810000000000001,"15min":2.2810000000000001},"last":1.2529999999999999}]}]},{"osd":0,"up_from":9,"seq":38654705714,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298232,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446920,"statfs":{"total":95995035648,"available":95689646080,"internally_reserved":0,"allocated":966656,"data_stored":732595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Mar 4 13:37:49 2024","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.76600000000000001,"15min":0.71599999999999997},"min":{"1min":0.55900000000000005,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.6679999999999999,"5min":1.6679999999999999,"15min":1.6679999999999999},"last":0.501},{"interface":"front","average":{"1min":1.016,"5min":0.79700000000000004,"15min":0.74299999999999999},"min":{"1min":0.63100000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.8380000000000001,"5min":1.8380000000000001,"15min":1.8380000000000001},"last":0.82499999999999996}]},{"osd":2,"last update":"Mon Mar 4 13:37:22 2024","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":0.94899999999999995,"15min":0.93799999999999994},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":2.1259999999999999,"5min":2.3149999999999999,"15min":2.3149999999999999},"last":0.48199999999999998},{"interface":"front","average":{"1min":1.056,"5min":1.018,"15min":1.012},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.446,"5min":1.958,"15min":1.958},"last":0.74299999999999999}]},{"osd":3,"last update":"Mon Mar 4 13:36:53 2024","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"min":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":1.619,"5min":1.619,"15min":1.619},"last":1.0369999999999999},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"min":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"max":{"1min":1.7370000000000001,"5min":1.7370000000000001,"15min":1.7370000000000001},"last":1.159}]},{"osd":4,"last update":"Mon Mar 4 13:37:22 2024","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"min":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":2.2160000000000002,"5min":2.2160000000000002,"15min":2.2160000000000002},"last":1.006},{"interface":"front","average":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.7689999999999999,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":0.97999999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91400000000000003}]}]},{"osd":1,"up_from":14,"seq":60129542185,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298232,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446920,"statfs":{"total":95995035648,"available":95689646080,"internally_reserved":0,"allocated":966656,"data_stored":732595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Mar 4 13:37:41 2024","interfaces":[{"interface":"back","average":{"1min":0.94999999999999996,"5min":0.745,"15min":0.68999999999999995},"min":{"1min":0.44900000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.4870000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":1.002},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.82199999999999995,"15min":0.78300000000000003},"min":{"1min":0.45800000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.7190000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":0.745}]},{"osd":2,"last update":"Mon Mar 4 13:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.89100000000000001,"15min":0.89500000000000002},"min":{"1min":0.51500000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.2,"5min":1.7170000000000001,"15min":1.7170000000000001},"last":1.129},{"interface":"front","average":{"1min":1.0049999999999999,"5min":0.94699999999999995,"15min":0.93700000000000006},"min":{"1min":0.65600000000000003,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.2470000000000001,"5min":1.8,"15min":1.8},"last":1.1679999999999999}]},{"osd":3,"last update":"Mon Mar 4 13:36:51 2024","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":1.093,"15min":1.093},"min":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"max":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":1.341},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.5269999999999999,"5min":1.5269999999999999,"15min":1.5269999999999999},"last":1.423}]},{"osd":4,"last update":"Mon Mar 4 13:37:23 2024","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":1.167,"15min":1.167},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.645,"5min":1.645,"15min":1.645},"last":1.3049999999999999},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"min":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"max":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":1.5249999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.355}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-04T13:37:59.347 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-03-04T13:37:59.347 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-04T13:37:59.347 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-03-04T13:37:59.348 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 -- ceph health --format=json 2024-03-04T13:37:59.353 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:37:58 smithi169 ceph-mon[36313]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-04T13:37:59.353 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:37:58 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:37:59.825 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/mon.smithi107/config 2024-03-04T13:38:00.196 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:37:59 smithi107 ceph-mon[28739]: from='client.14422 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-04T13:38:00.196 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:37:59 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:38:00.196 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:37:59 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:38:00.197 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:37:59 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:38:00.353 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:37:59 smithi169 ceph-mon[36313]: from='client.14422 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-04T13:38:00.353 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:37:59 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:38:00.353 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:37:59 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:38:00.353 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:37:59 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:38:01.086 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:00 smithi107 ceph-mon[28739]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-04T13:38:01.353 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:00 smithi169 ceph-mon[36313]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-04T13:38:01.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:01.903+0000 7f4e56e00700 1 -- 172.21.15.107:0/308929529 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4e50105b10 msgr2=0x7f4e500ff180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:38:01.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:01.903+0000 7f4e56e00700 1 --2- 172.21.15.107:0/308929529 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4e50105b10 0x7f4e500ff180 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4e38004940 tx=0x7f4e38009e30 comp rx=0 tx=0).stop 2024-03-04T13:38:01.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:01.903+0000 7f4e56e00700 1 -- 172.21.15.107:0/308929529 shutdown_connections 2024-03-04T13:38:01.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:01.903+0000 7f4e56e00700 1 --2- 172.21.15.107:0/308929529 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4e50105b10 0x7f4e500ff180 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:38:01.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:01.903+0000 7f4e56e00700 1 --2- 172.21.15.107:0/308929529 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e50105170 0x7f4e50105540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:38:01.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:01.903+0000 7f4e56e00700 1 -- 172.21.15.107:0/308929529 >> 172.21.15.107:0/308929529 conn(0x7f4e500face0 msgr2=0x7f4e500fd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:38:01.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:01.904+0000 7f4e56e00700 1 -- 172.21.15.107:0/308929529 shutdown_connections 2024-03-04T13:38:01.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:01.904+0000 7f4e56e00700 1 -- 172.21.15.107:0/308929529 wait complete. 2024-03-04T13:38:01.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:01.904+0000 7f4e56e00700 1 Processor -- start 2024-03-04T13:38:01.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:01.904+0000 7f4e56e00700 1 -- start start 2024-03-04T13:38:01.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:01.905+0000 7f4e56e00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e50105170 0x7f4e50115de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:38:01.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:01.905+0000 7f4e56e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4e50105b10 0x7f4e50116320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:38:01.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:01.905+0000 7f4e56e00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e50116a00 con 0x7f4e50105b10 2024-03-04T13:38:01.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:01.905+0000 7f4e56e00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e5010fc80 con 0x7f4e50105170 2024-03-04T13:38:01.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:01.905+0000 7f4e47400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4e50105b10 0x7f4e50116320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:38:01.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:01.905+0000 7f4e47400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4e50105b10 0x7f4e50116320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56282/0 (socket says 172.21.15.107:56282) 2024-03-04T13:38:01.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:01.905+0000 7f4e47400700 1 -- 172.21.15.107:0/1159640190 learned_addr learned my addr 172.21.15.107:0/1159640190 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:38:01.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:01.905+0000 7f4e47400700 1 -- 172.21.15.107:0/1159640190 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e50105170 msgr2=0x7f4e50115de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:38:01.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:01.905+0000 7f4e4fe00700 1 --2- 172.21.15.107:0/1159640190 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e50105170 0x7f4e50115de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:38:01.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:01.905+0000 7f4e47400700 1 --2- 172.21.15.107:0/1159640190 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e50105170 0x7f4e50115de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:38:01.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:01.905+0000 7f4e47400700 1 -- 172.21.15.107:0/1159640190 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e38005040 con 0x7f4e50105b10 2024-03-04T13:38:01.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:01.905+0000 7f4e4fe00700 1 --2- 172.21.15.107:0/1159640190 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e50105170 0x7f4e50115de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T13:38:01.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:01.906+0000 7f4e47400700 1 --2- 172.21.15.107:0/1159640190 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4e50105b10 0x7f4e50116320 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4e38009100 tx=0x7f4e38009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:38:01.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:01.906+0000 7f4e4cc00700 1 -- 172.21.15.107:0/1159640190 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4e3800e040 con 0x7f4e50105b10 2024-03-04T13:38:01.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:01.906+0000 7f4e56e00700 1 -- 172.21.15.107:0/1159640190 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e5010fea0 con 0x7f4e50105b10 2024-03-04T13:38:01.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:01.906+0000 7f4e4cc00700 1 -- 172.21.15.107:0/1159640190 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4e38004290 con 0x7f4e50105b10 2024-03-04T13:38:01.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:01.906+0000 7f4e4cc00700 1 -- 172.21.15.107:0/1159640190 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4e3801c3f0 con 0x7f4e50105b10 2024-03-04T13:38:01.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:01.906+0000 7f4e56e00700 1 -- 172.21.15.107:0/1159640190 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e501103e0 con 0x7f4e50105b10 2024-03-04T13:38:01.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:01.907+0000 7f4e56e00700 1 -- 172.21.15.107:0/1159640190 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e500fe900 con 0x7f4e50105b10 2024-03-04T13:38:01.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:01.908+0000 7f4e4cc00700 1 -- 172.21.15.107:0/1159640190 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86623+0+0 (secure 0 0 0) 0x7f4e3800bbf0 con 0x7f4e50105b10 2024-03-04T13:38:01.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:01.909+0000 7f4e4cc00700 1 --2- 172.21.15.107:0/1159640190 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f4e340692b0 0x7f4e3406b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:38:01.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:01.909+0000 7f4e4cc00700 1 -- 172.21.15.107:0/1159640190 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f4e380881e0 con 0x7f4e50105b10 2024-03-04T13:38:01.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:01.909+0000 7f4e4fe00700 1 --2- 172.21.15.107:0/1159640190 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f4e340692b0 0x7f4e3406b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:38:01.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:01.910+0000 7f4e4fe00700 1 --2- 172.21.15.107:0/1159640190 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f4e340692b0 0x7f4e3406b760 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4e40007fd0 tx=0x7f4e400074f0 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:38:01.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:01.914+0000 7f4e4cc00700 1 -- 172.21.15.107:0/1159640190 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f4e38058240 con 0x7f4e50105b10 2024-03-04T13:38:02.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:02.165+0000 7f4e56e00700 1 -- 172.21.15.107:0/1159640190 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f4e5010c2c0 con 0x7f4e50105b10 2024-03-04T13:38:02.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:02.166+0000 7f4e4cc00700 1 -- 172.21.15.107:0/1159640190 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f4e38057dd0 con 0x7f4e50105b10 2024-03-04T13:38:02.160 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:38:02.160 INFO:teuthology.orchestra.run.smithi107.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-04T13:38:02.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:02.169+0000 7f4e56e00700 1 -- 172.21.15.107:0/1159640190 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f4e340692b0 msgr2=0x7f4e3406b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:38:02.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:02.169+0000 7f4e56e00700 1 --2- 172.21.15.107:0/1159640190 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f4e340692b0 0x7f4e3406b760 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4e40007fd0 tx=0x7f4e400074f0 comp rx=0 tx=0).stop 2024-03-04T13:38:02.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:02.169+0000 7f4e56e00700 1 -- 172.21.15.107:0/1159640190 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4e50105b10 msgr2=0x7f4e50116320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:38:02.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:02.169+0000 7f4e56e00700 1 --2- 172.21.15.107:0/1159640190 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4e50105b10 0x7f4e50116320 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4e38009100 tx=0x7f4e38009e30 comp rx=0 tx=0).stop 2024-03-04T13:38:02.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:02.170+0000 7f4e56e00700 1 -- 172.21.15.107:0/1159640190 shutdown_connections 2024-03-04T13:38:02.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:02.170+0000 7f4e56e00700 1 --2- 172.21.15.107:0/1159640190 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f4e340692b0 0x7f4e3406b760 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:38:02.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:02.170+0000 7f4e56e00700 1 --2- 172.21.15.107:0/1159640190 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e50105170 0x7f4e50115de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:38:02.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:02.170+0000 7f4e56e00700 1 --2- 172.21.15.107:0/1159640190 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4e50105b10 0x7f4e50116320 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:38:02.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:02.170+0000 7f4e56e00700 1 -- 172.21.15.107:0/1159640190 >> 172.21.15.107:0/1159640190 conn(0x7f4e500face0 msgr2=0x7f4e500fc710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:38:02.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:02.170+0000 7f4e56e00700 1 -- 172.21.15.107:0/1159640190 shutdown_connections 2024-03-04T13:38:02.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:02.170+0000 7f4e56e00700 1 -- 172.21.15.107:0/1159640190 wait complete. 2024-03-04T13:38:03.015 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-03-04T13:38:03.015 INFO:tasks.cephadm:Setup complete, yielding 2024-03-04T13:38:03.016 INFO:teuthology.run_tasks:Running task print... 2024-03-04T13:38:03.025 INFO:teuthology.task.print:**** done end installing quincy cephadm ... 2024-03-04T13:38:03.025 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-04T13:38:03.045 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi107.front.sepia.ceph.com 2024-03-04T13:38:03.045 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 -- bash -c 'ceph config set mgr mgr/cephadm/use_repo_digest true --force' 2024-03-04T13:38:03.336 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:02 smithi107 ceph-mon[28739]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-04T13:38:03.336 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:02 smithi107 ceph-mon[28739]: from='client.? 172.21.15.107:0/1159640190' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-04T13:38:03.353 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:02 smithi169 ceph-mon[36313]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-04T13:38:03.353 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:02 smithi169 ceph-mon[36313]: from='client.? 172.21.15.107:0/1159640190' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-04T13:38:03.518 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/mon.smithi107/config 2024-03-04T13:38:05.047 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:04 smithi107 ceph-mon[28739]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-04T13:38:05.353 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:04 smithi169 ceph-mon[36313]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-04T13:38:05.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:05.526+0000 7feda2a00700 1 -- 172.21.15.107:0/358577264 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed9c104760 msgr2=0x7fed9c104bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:38:05.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:05.526+0000 7feda2a00700 1 --2- 172.21.15.107:0/358577264 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed9c104760 0x7fed9c104bb0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fed88004940 tx=0x7fed88009e30 comp rx=0 tx=0).stop 2024-03-04T13:38:05.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:05.526+0000 7feda2a00700 1 -- 172.21.15.107:0/358577264 shutdown_connections 2024-03-04T13:38:05.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:05.526+0000 7feda2a00700 1 --2- 172.21.15.107:0/358577264 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed9c104760 0x7fed9c104bb0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:38:05.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:05.526+0000 7feda2a00700 1 --2- 172.21.15.107:0/358577264 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed9c1035a0 0x7fed9c1039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:38:05.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:05.526+0000 7feda2a00700 1 -- 172.21.15.107:0/358577264 >> 172.21.15.107:0/358577264 conn(0x7fed9c072690 msgr2=0x7fed9c074ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:38:05.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:05.526+0000 7feda2a00700 1 -- 172.21.15.107:0/358577264 shutdown_connections 2024-03-04T13:38:05.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:05.527+0000 7feda2a00700 1 -- 172.21.15.107:0/358577264 wait complete. 2024-03-04T13:38:05.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:05.528+0000 7feda2a00700 1 Processor -- start 2024-03-04T13:38:05.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:05.528+0000 7feda2a00700 1 -- start start 2024-03-04T13:38:05.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:05.529+0000 7feda2a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed9c1035a0 0x7fed9c113f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:38:05.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:05.529+0000 7feda2a00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed9c104760 0x7fed9c1144b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:38:05.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:05.529+0000 7feda2a00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed9c114ab0 con 0x7fed9c1035a0 2024-03-04T13:38:05.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:05.529+0000 7feda2a00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed9c115350 con 0x7fed9c104760 2024-03-04T13:38:05.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:05.529+0000 7fed9be00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed9c1035a0 0x7fed9c113f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:38:05.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:05.529+0000 7fed9be00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed9c1035a0 0x7fed9c113f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56304/0 (socket says 172.21.15.107:56304) 2024-03-04T13:38:05.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:05.529+0000 7fed9be00700 1 -- 172.21.15.107:0/2455627497 learned_addr learned my addr 172.21.15.107:0/2455627497 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:38:05.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:05.529+0000 7fed9b400700 1 --2- 172.21.15.107:0/2455627497 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed9c104760 0x7fed9c1144b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:38:05.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:05.529+0000 7fed9be00700 1 -- 172.21.15.107:0/2455627497 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed9c104760 msgr2=0x7fed9c1144b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:38:05.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:05.529+0000 7fed9be00700 1 --2- 172.21.15.107:0/2455627497 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed9c104760 0x7fed9c1144b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:38:05.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:05.529+0000 7fed9be00700 1 -- 172.21.15.107:0/2455627497 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed88005040 con 0x7fed9c1035a0 2024-03-04T13:38:05.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:05.529+0000 7fed9b400700 1 --2- 172.21.15.107:0/2455627497 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed9c104760 0x7fed9c1144b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T13:38:05.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:05.530+0000 7fed9be00700 1 --2- 172.21.15.107:0/2455627497 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed9c1035a0 0x7fed9c113f70 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fed8c00c8b0 tx=0x7fed8c004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:38:05.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:05.530+0000 7fed93e00700 1 -- 172.21.15.107:0/2455627497 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fed8c00a760 con 0x7fed9c1035a0 2024-03-04T13:38:05.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:05.530+0000 7fed93e00700 1 -- 172.21.15.107:0/2455627497 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fed8c00ad90 con 0x7fed9c1035a0 2024-03-04T13:38:05.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:05.530+0000 7feda2a00700 1 -- 172.21.15.107:0/2455627497 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed9c115630 con 0x7fed9c1035a0 2024-03-04T13:38:05.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:05.530+0000 7fed93e00700 1 -- 172.21.15.107:0/2455627497 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fed8c0113a0 con 0x7fed9c1035a0 2024-03-04T13:38:05.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:05.530+0000 7feda2a00700 1 -- 172.21.15.107:0/2455627497 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed9c115b70 con 0x7fed9c1035a0 2024-03-04T13:38:05.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:05.531+0000 7feda2a00700 1 -- 172.21.15.107:0/2455627497 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed9c04bed0 con 0x7fed9c1035a0 2024-03-04T13:38:05.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:05.532+0000 7fed93e00700 1 -- 172.21.15.107:0/2455627497 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86623+0+0 (secure 0 0 0) 0x7fed8c012400 con 0x7fed9c1035a0 2024-03-04T13:38:05.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:05.533+0000 7fed93e00700 1 --2- 172.21.15.107:0/2455627497 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fed800692b0 0x7fed8006b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:38:05.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:05.533+0000 7fed93e00700 1 -- 172.21.15.107:0/2455627497 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fed8c086e50 con 0x7fed9c1035a0 2024-03-04T13:38:05.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:05.533+0000 7fed9b400700 1 --2- 172.21.15.107:0/2455627497 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fed800692b0 0x7fed8006b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:38:05.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:05.534+0000 7fed9b400700 1 --2- 172.21.15.107:0/2455627497 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fed800692b0 0x7fed8006b760 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fed8800b7a0 tx=0x7fed88009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:38:05.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:05.537+0000 7fed93e00700 1 -- 172.21.15.107:0/2455627497 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fed8c056f30 con 0x7fed9c1035a0 2024-03-04T13:38:05.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:05.745+0000 7feda2a00700 1 -- 172.21.15.107:0/2455627497 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/use_repo_digest}] v 0) v1 -- 0x7fed9c10c030 con 0x7fed9c1035a0 2024-03-04T13:38:05.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:05.752+0000 7fed93e00700 1 -- 172.21.15.107:0/2455627497 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/use_repo_digest}]=0 v20)=0 v20) v1 ==== 143+0+0 (secure 0 0 0) 0x7fed8c056ac0 con 0x7fed9c1035a0 2024-03-04T13:38:05.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:05.755+0000 7feda2a00700 1 -- 172.21.15.107:0/2455627497 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fed800692b0 msgr2=0x7fed8006b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:38:05.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:05.755+0000 7feda2a00700 1 --2- 172.21.15.107:0/2455627497 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fed800692b0 0x7fed8006b760 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fed8800b7a0 tx=0x7fed88009e30 comp rx=0 tx=0).stop 2024-03-04T13:38:05.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:05.755+0000 7feda2a00700 1 -- 172.21.15.107:0/2455627497 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed9c1035a0 msgr2=0x7fed9c113f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:38:05.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:05.755+0000 7feda2a00700 1 --2- 172.21.15.107:0/2455627497 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed9c1035a0 0x7fed9c113f70 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fed8c00c8b0 tx=0x7fed8c004470 comp rx=0 tx=0).stop 2024-03-04T13:38:05.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:05.756+0000 7feda2a00700 1 -- 172.21.15.107:0/2455627497 shutdown_connections 2024-03-04T13:38:05.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:05.756+0000 7feda2a00700 1 --2- 172.21.15.107:0/2455627497 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fed800692b0 0x7fed8006b760 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:38:05.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:05.756+0000 7feda2a00700 1 --2- 172.21.15.107:0/2455627497 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed9c1035a0 0x7fed9c113f70 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:38:05.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:05.756+0000 7feda2a00700 1 --2- 172.21.15.107:0/2455627497 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed9c104760 0x7fed9c1144b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:38:05.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:05.756+0000 7feda2a00700 1 -- 172.21.15.107:0/2455627497 >> 172.21.15.107:0/2455627497 conn(0x7fed9c072690 msgr2=0x7fed9c107990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:38:05.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:05.756+0000 7feda2a00700 1 -- 172.21.15.107:0/2455627497 shutdown_connections 2024-03-04T13:38:05.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:05.756+0000 7feda2a00700 1 -- 172.21.15.107:0/2455627497 wait complete. 2024-03-04T13:38:06.650 INFO:teuthology.run_tasks:Running task print... 2024-03-04T13:38:06.658 INFO:teuthology.task.print:**** done cephadm.shell ceph config set mgr... 2024-03-04T13:38:06.658 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-04T13:38:06.695 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi107.front.sepia.ceph.com 2024-03-04T13:38:06.695 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 -- bash -c 'ceph orch status' 2024-03-04T13:38:06.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:06 smithi107 ceph-mon[28739]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-04T13:38:06.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:06 smithi107 ceph-mon[28739]: from='client.? 172.21.15.107:0/2455627497' entity='client.admin' 2024-03-04T13:38:06.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:06 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:38:06.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:06 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:38:06.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:06 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:38:06.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:06 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:38:07.103 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:06 smithi169 ceph-mon[36313]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-04T13:38:07.103 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:06 smithi169 ceph-mon[36313]: from='client.? 172.21.15.107:0/2455627497' entity='client.admin' 2024-03-04T13:38:07.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:06 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:38:07.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:06 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:38:07.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:06 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:38:07.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:06 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:38:07.182 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/mon.smithi107/config 2024-03-04T13:38:08.952 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:08 smithi107 ceph-mon[28739]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-04T13:38:09.103 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:08 smithi169 ceph-mon[36313]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-04T13:38:09.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:09.499+0000 7f59e3600700 1 -- 172.21.15.107:0/266442682 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59dc0fcef0 msgr2=0x7f59dc0fd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:38:09.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:09.499+0000 7f59e3600700 1 --2- 172.21.15.107:0/266442682 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59dc0fcef0 0x7f59dc0fd320 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f59c4004940 tx=0x7f59c4009e30 comp rx=0 tx=0).stop 2024-03-04T13:38:09.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:09.499+0000 7f59e3600700 1 -- 172.21.15.107:0/266442682 shutdown_connections 2024-03-04T13:38:09.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:09.499+0000 7f59e3600700 1 --2- 172.21.15.107:0/266442682 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f59dc0fe0e0 0x7f59dc0fe550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:38:09.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:09.499+0000 7f59e3600700 1 --2- 172.21.15.107:0/266442682 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59dc0fcef0 0x7f59dc0fd320 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:38:09.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:09.499+0000 7f59e3600700 1 -- 172.21.15.107:0/266442682 >> 172.21.15.107:0/266442682 conn(0x7f59dc0f8620 msgr2=0x7f59dc0faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:38:09.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:09.499+0000 7f59e3600700 1 -- 172.21.15.107:0/266442682 shutdown_connections 2024-03-04T13:38:09.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:09.499+0000 7f59e3600700 1 -- 172.21.15.107:0/266442682 wait complete. 2024-03-04T13:38:09.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:09.500+0000 7f59e3600700 1 Processor -- start 2024-03-04T13:38:09.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:09.500+0000 7f59e3600700 1 -- start start 2024-03-04T13:38:09.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:09.501+0000 7f59e3600700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f59dc0fcef0 0x7f59dc113f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:38:09.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:09.501+0000 7f59e3600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59dc0fe0e0 0x7f59dc1144c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:38:09.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:09.501+0000 7f59e3600700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59dc114ac0 con 0x7f59dc0fe0e0 2024-03-04T13:38:09.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:09.501+0000 7f59e3600700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59dc115360 con 0x7f59dc0fcef0 2024-03-04T13:38:09.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:09.501+0000 7f59dbe00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59dc0fe0e0 0x7f59dc1144c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:38:09.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:09.501+0000 7f59dbe00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59dc0fe0e0 0x7f59dc1144c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44794/0 (socket says 172.21.15.107:44794) 2024-03-04T13:38:09.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:09.501+0000 7f59dbe00700 1 -- 172.21.15.107:0/1624033646 learned_addr learned my addr 172.21.15.107:0/1624033646 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:38:09.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:09.501+0000 7f59e0c00700 1 --2- 172.21.15.107:0/1624033646 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f59dc0fcef0 0x7f59dc113f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:38:09.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:09.501+0000 7f59dbe00700 1 -- 172.21.15.107:0/1624033646 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f59dc0fcef0 msgr2=0x7f59dc113f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:38:09.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:09.501+0000 7f59dbe00700 1 --2- 172.21.15.107:0/1624033646 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f59dc0fcef0 0x7f59dc113f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:38:09.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:09.502+0000 7f59dbe00700 1 -- 172.21.15.107:0/1624033646 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59c4005040 con 0x7f59dc0fe0e0 2024-03-04T13:38:09.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:09.502+0000 7f59dbe00700 1 --2- 172.21.15.107:0/1624033646 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59dc0fe0e0 0x7f59dc1144c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f59cc002a50 tx=0x7f59cc0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:38:09.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:09.502+0000 7f59d8c00700 1 -- 172.21.15.107:0/1624033646 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f59cc007cc0 con 0x7f59dc0fe0e0 2024-03-04T13:38:09.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:09.502+0000 7f59d8c00700 1 -- 172.21.15.107:0/1624033646 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f59cc00f040 con 0x7f59dc0fe0e0 2024-03-04T13:38:09.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:09.502+0000 7f59e3600700 1 -- 172.21.15.107:0/1624033646 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59dc115640 con 0x7f59dc0fe0e0 2024-03-04T13:38:09.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:09.502+0000 7f59d8c00700 1 -- 172.21.15.107:0/1624033646 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f59cc009640 con 0x7f59dc0fe0e0 2024-03-04T13:38:09.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:09.502+0000 7f59e3600700 1 -- 172.21.15.107:0/1624033646 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59dc115b80 con 0x7f59dc0fe0e0 2024-03-04T13:38:09.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:09.504+0000 7f59e3600700 1 -- 172.21.15.107:0/1624033646 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59dc102210 con 0x7f59dc0fe0e0 2024-03-04T13:38:09.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:09.504+0000 7f59d8c00700 1 -- 172.21.15.107:0/1624033646 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86623+0+0 (secure 0 0 0) 0x7f59cc0097a0 con 0x7f59dc0fe0e0 2024-03-04T13:38:09.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:09.505+0000 7f59d8c00700 1 --2- 172.21.15.107:0/1624033646 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f59c00692b0 0x7f59c006b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:38:09.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:09.505+0000 7f59d8c00700 1 -- 172.21.15.107:0/1624033646 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f59cc086c40 con 0x7f59dc0fe0e0 2024-03-04T13:38:09.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:09.506+0000 7f59e0c00700 1 --2- 172.21.15.107:0/1624033646 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f59c00692b0 0x7f59c006b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:38:09.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:09.506+0000 7f59e0c00700 1 --2- 172.21.15.107:0/1624033646 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f59c00692b0 0x7f59c006b760 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f59c400b3b0 tx=0x7f59c4009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:38:09.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:09.510+0000 7f59d8c00700 1 -- 172.21.15.107:0/1624033646 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f59cc056ca0 con 0x7f59dc0fe0e0 2024-03-04T13:38:09.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:09.730+0000 7f59e3600700 1 -- 172.21.15.107:0/1624033646 --> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7f59dc115e60 con 0x7f59c00692b0 2024-03-04T13:38:09.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:09.732+0000 7f59d8c00700 1 -- 172.21.15.107:0/1624033646 <== mgr.14212 v2:172.21.15.107:6800/3848495603 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f59dc115e60 con 0x7f59c00692b0 2024-03-04T13:38:09.726 INFO:teuthology.orchestra.run.smithi107.stdout:Backend: cephadm 2024-03-04T13:38:09.726 INFO:teuthology.orchestra.run.smithi107.stdout:Available: Yes 2024-03-04T13:38:09.726 INFO:teuthology.orchestra.run.smithi107.stdout:Paused: No 2024-03-04T13:38:09.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:09.735+0000 7f59e3600700 1 -- 172.21.15.107:0/1624033646 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f59c00692b0 msgr2=0x7f59c006b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:38:09.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:09.735+0000 7f59e3600700 1 --2- 172.21.15.107:0/1624033646 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f59c00692b0 0x7f59c006b760 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f59c400b3b0 tx=0x7f59c4009e30 comp rx=0 tx=0).stop 2024-03-04T13:38:09.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:09.735+0000 7f59e3600700 1 -- 172.21.15.107:0/1624033646 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59dc0fe0e0 msgr2=0x7f59dc1144c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:38:09.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:09.735+0000 7f59e3600700 1 --2- 172.21.15.107:0/1624033646 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59dc0fe0e0 0x7f59dc1144c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f59cc002a50 tx=0x7f59cc0046b0 comp rx=0 tx=0).stop 2024-03-04T13:38:09.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:09.736+0000 7f59e3600700 1 -- 172.21.15.107:0/1624033646 shutdown_connections 2024-03-04T13:38:09.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:09.736+0000 7f59e3600700 1 --2- 172.21.15.107:0/1624033646 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f59c00692b0 0x7f59c006b760 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:38:09.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:09.736+0000 7f59e3600700 1 --2- 172.21.15.107:0/1624033646 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f59dc0fcef0 0x7f59dc113f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:38:09.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:09.736+0000 7f59e3600700 1 --2- 172.21.15.107:0/1624033646 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59dc0fe0e0 0x7f59dc1144c0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:38:09.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:09.736+0000 7f59e3600700 1 -- 172.21.15.107:0/1624033646 >> 172.21.15.107:0/1624033646 conn(0x7f59dc0f8620 msgr2=0x7f59dc0ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:38:09.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:09.736+0000 7f59e3600700 1 -- 172.21.15.107:0/1624033646 shutdown_connections 2024-03-04T13:38:09.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:09.736+0000 7f59e3600700 1 -- 172.21.15.107:0/1624033646 wait complete. 2024-03-04T13:38:10.594 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 -- bash -c 'ceph orch ps' 2024-03-04T13:38:10.836 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:10 smithi107 ceph-mon[28739]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-04T13:38:10.836 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:10 smithi107 ceph-mon[28739]: from='client.14434 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:38:11.076 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/mon.smithi107/config 2024-03-04T13:38:11.103 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:10 smithi169 ceph-mon[36313]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-04T13:38:11.103 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:10 smithi169 ceph-mon[36313]: from='client.14434 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:38:12.928 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:12 smithi107 ceph-mon[28739]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-04T13:38:12.928 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:12 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:38:13.103 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:12 smithi169 ceph-mon[36313]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-04T13:38:13.103 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:12 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:38:13.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:13.260+0000 7fb8d7400700 1 -- 172.21.15.107:0/1644870890 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8d00fcef0 msgr2=0x7fb8d00fd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:38:13.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:13.260+0000 7fb8d7400700 1 --2- 172.21.15.107:0/1644870890 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8d00fcef0 0x7fb8d00fd320 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb8c4004940 tx=0x7fb8c4009e30 comp rx=0 tx=0).stop 2024-03-04T13:38:13.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:13.260+0000 7fb8d7400700 1 -- 172.21.15.107:0/1644870890 shutdown_connections 2024-03-04T13:38:13.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:13.260+0000 7fb8d7400700 1 --2- 172.21.15.107:0/1644870890 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8d00fe0e0 0x7fb8d00fe550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:38:13.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:13.260+0000 7fb8d7400700 1 --2- 172.21.15.107:0/1644870890 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8d00fcef0 0x7fb8d00fd320 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:38:13.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:13.260+0000 7fb8d7400700 1 -- 172.21.15.107:0/1644870890 >> 172.21.15.107:0/1644870890 conn(0x7fb8d00f8620 msgr2=0x7fb8d00faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:38:13.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:13.260+0000 7fb8d7400700 1 -- 172.21.15.107:0/1644870890 shutdown_connections 2024-03-04T13:38:13.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:13.260+0000 7fb8d7400700 1 -- 172.21.15.107:0/1644870890 wait complete. 2024-03-04T13:38:13.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:13.261+0000 7fb8d7400700 1 Processor -- start 2024-03-04T13:38:13.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:13.261+0000 7fb8d7400700 1 -- start start 2024-03-04T13:38:13.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:13.261+0000 7fb8d7400700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8d00fcef0 0x7fb8d010d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:38:13.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:13.261+0000 7fb8d7400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8d00fe0e0 0x7fb8d010deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:38:13.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:13.261+0000 7fb8d7400700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d010e4b0 con 0x7fb8d00fe0e0 2024-03-04T13:38:13.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:13.261+0000 7fb8d7400700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d010ed50 con 0x7fb8d00fcef0 2024-03-04T13:38:13.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:13.262+0000 7fb8cfe00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8d00fe0e0 0x7fb8d010deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:38:13.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:13.262+0000 7fb8cfe00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8d00fe0e0 0x7fb8d010deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44814/0 (socket says 172.21.15.107:44814) 2024-03-04T13:38:13.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:13.262+0000 7fb8cfe00700 1 -- 172.21.15.107:0/2687850316 learned_addr learned my addr 172.21.15.107:0/2687850316 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:38:13.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:13.262+0000 7fb8d4a00700 1 --2- 172.21.15.107:0/2687850316 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8d00fcef0 0x7fb8d010d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:38:13.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:13.262+0000 7fb8cfe00700 1 -- 172.21.15.107:0/2687850316 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8d00fcef0 msgr2=0x7fb8d010d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:38:13.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:13.262+0000 7fb8cfe00700 1 --2- 172.21.15.107:0/2687850316 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8d00fcef0 0x7fb8d010d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:38:13.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:13.262+0000 7fb8cfe00700 1 -- 172.21.15.107:0/2687850316 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8c4005040 con 0x7fb8d00fe0e0 2024-03-04T13:38:13.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:13.262+0000 7fb8d4a00700 1 --2- 172.21.15.107:0/2687850316 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8d00fcef0 0x7fb8d010d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T13:38:13.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:13.263+0000 7fb8cfe00700 1 --2- 172.21.15.107:0/2687850316 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8d00fe0e0 0x7fb8d010deb0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc00c8c0 tx=0x7fb8bc0044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:38:13.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:13.263+0000 7fb8ccc00700 1 -- 172.21.15.107:0/2687850316 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb8bc00a980 con 0x7fb8d00fe0e0 2024-03-04T13:38:13.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:13.263+0000 7fb8ccc00700 1 -- 172.21.15.107:0/2687850316 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb8bc00ce60 con 0x7fb8d00fe0e0 2024-03-04T13:38:13.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:13.263+0000 7fb8ccc00700 1 -- 172.21.15.107:0/2687850316 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb8bc0115e0 con 0x7fb8d00fe0e0 2024-03-04T13:38:13.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:13.263+0000 7fb8d7400700 1 -- 172.21.15.107:0/2687850316 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8d010f030 con 0x7fb8d00fe0e0 2024-03-04T13:38:13.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:13.263+0000 7fb8d7400700 1 -- 172.21.15.107:0/2687850316 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8d010f570 con 0x7fb8d00fe0e0 2024-03-04T13:38:13.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:13.265+0000 7fb8d7400700 1 -- 172.21.15.107:0/2687850316 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8d004bed0 con 0x7fb8d00fe0e0 2024-03-04T13:38:13.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:13.265+0000 7fb8ccc00700 1 -- 172.21.15.107:0/2687850316 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86623+0+0 (secure 0 0 0) 0x7fb8bc012400 con 0x7fb8d00fe0e0 2024-03-04T13:38:13.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:13.267+0000 7fb8ccc00700 1 --2- 172.21.15.107:0/2687850316 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fb8b80692b0 0x7fb8b806b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:38:13.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:13.267+0000 7fb8ccc00700 1 -- 172.21.15.107:0/2687850316 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fb8bc086e60 con 0x7fb8d00fe0e0 2024-03-04T13:38:13.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:13.267+0000 7fb8d4a00700 1 --2- 172.21.15.107:0/2687850316 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fb8b80692b0 0x7fb8b806b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:38:13.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:13.268+0000 7fb8d4a00700 1 --2- 172.21.15.107:0/2687850316 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fb8b80692b0 0x7fb8b806b760 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb8c400b3b0 tx=0x7fb8c4009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:38:13.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:13.271+0000 7fb8ccc00700 1 -- 172.21.15.107:0/2687850316 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fb8bc057000 con 0x7fb8d00fe0e0 2024-03-04T13:38:13.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:13.487+0000 7fb8d7400700 1 -- 172.21.15.107:0/2687850316 --> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fb8d010fa10 con 0x7fb8b80692b0 2024-03-04T13:38:13.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:13.512+0000 7fb8ccc00700 1 -- 172.21.15.107:0/2687850316 <== mgr.14212 v2:172.21.15.107:6800/3848495603 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+2718 (secure 0 0 0) 0x7fb8d010fa10 con 0x7fb8b80692b0 2024-03-04T13:38:13.506 INFO:teuthology.orchestra.run.smithi107.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-04T13:38:13.506 INFO:teuthology.orchestra.run.smithi107.stdout:alertmanager.smithi107 smithi107 *:9093,9094 running (4m) 2m ago 6m 14.5M - 0.25.0 c8568f914cd2 3eef48872dfe 2024-03-04T13:38:13.506 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi107 smithi107 running (6m) 2m ago 6m 6665k - 17.2.7-545-g42608bb1 9eec3f22859d 03e1570c2eff 2024-03-04T13:38:13.506 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi169 smithi169 running (5m) 81s ago 5m 6656k - 17.2.7-545-g42608bb1 9eec3f22859d 9c8881a5338e 2024-03-04T13:38:13.506 INFO:teuthology.orchestra.run.smithi107.stdout:grafana.smithi107 smithi107 *:3000 running (4m) 2m ago 6m 74.6M - 9.4.7 954c08fa6188 2d8536fd3802 2024-03-04T13:38:13.506 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi107.wpzgwl smithi107 *:9283 running (8m) 2m ago 8m 470M - 17.2.7-545-g42608bb1 9eec3f22859d 4831dce06263 2024-03-04T13:38:13.506 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi169.eihczr smithi169 *:8443,9283 running (5m) 81s ago 5m 431M - 17.2.7-545-g42608bb1 9eec3f22859d 001762248bc2 2024-03-04T13:38:13.506 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi107 smithi107 running (8m) 2m ago 8m 40.2M 2048M 17.2.7-545-g42608bb1 9eec3f22859d 49f5da990d35 2024-03-04T13:38:13.506 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi169 smithi169 running (5m) 81s ago 5m 33.3M 2048M 17.2.7-545-g42608bb1 9eec3f22859d ba4ee916297f 2024-03-04T13:38:13.506 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi107 smithi107 *:9100 running (6m) 2m ago 6m 7784k - 1.5.0 0da6a335fe13 00c53f7970cf 2024-03-04T13:38:13.506 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi169 smithi169 *:9100 running (5m) 81s ago 5m 7784k - 1.5.0 0da6a335fe13 b2dae07f2da2 2024-03-04T13:38:13.506 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0 smithi107 running (4m) 2m ago 4m 34.8M 4288M 17.2.7-545-g42608bb1 9eec3f22859d c0605b68f696 2024-03-04T13:38:13.507 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1 smithi107 running (3m) 2m ago 3m 34.8M 4288M 17.2.7-545-g42608bb1 9eec3f22859d 5fb2c0dd5769 2024-03-04T13:38:13.507 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2 smithi107 running (3m) 2m ago 3m 10.5M 4288M 17.2.7-545-g42608bb1 9eec3f22859d 61f30f643549 2024-03-04T13:38:13.507 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3 smithi169 running (2m) 81s ago 2m 38.9M 5313M 17.2.7-545-g42608bb1 9eec3f22859d cd91d8694764 2024-03-04T13:38:13.507 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4 smithi169 running (119s) 81s ago 118s 36.4M 5313M 17.2.7-545-g42608bb1 9eec3f22859d 08545e3d0659 2024-03-04T13:38:13.507 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5 smithi169 running (86s) 81s ago 85s 10.5M 5313M 17.2.7-545-g42608bb1 9eec3f22859d dcffe6cb3eac 2024-03-04T13:38:13.507 INFO:teuthology.orchestra.run.smithi107.stdout:prometheus.smithi107 smithi107 *:9095 running (4m) 2m ago 5m 35.4M - 2.43.0 a07b618ecd1d 226d4ba6d37a 2024-03-04T13:38:13.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:13.514+0000 7fb8d7400700 1 -- 172.21.15.107:0/2687850316 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fb8b80692b0 msgr2=0x7fb8b806b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:38:13.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:13.514+0000 7fb8d7400700 1 --2- 172.21.15.107:0/2687850316 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fb8b80692b0 0x7fb8b806b760 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb8c400b3b0 tx=0x7fb8c4009e30 comp rx=0 tx=0).stop 2024-03-04T13:38:13.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:13.514+0000 7fb8d7400700 1 -- 172.21.15.107:0/2687850316 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8d00fe0e0 msgr2=0x7fb8d010deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:38:13.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:13.514+0000 7fb8d7400700 1 --2- 172.21.15.107:0/2687850316 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8d00fe0e0 0x7fb8d010deb0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc00c8c0 tx=0x7fb8bc0044c0 comp rx=0 tx=0).stop 2024-03-04T13:38:13.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:13.515+0000 7fb8d7400700 1 -- 172.21.15.107:0/2687850316 shutdown_connections 2024-03-04T13:38:13.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:13.515+0000 7fb8d7400700 1 --2- 172.21.15.107:0/2687850316 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fb8b80692b0 0x7fb8b806b760 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:38:13.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:13.515+0000 7fb8d7400700 1 --2- 172.21.15.107:0/2687850316 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8d00fcef0 0x7fb8d010d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:38:13.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:13.515+0000 7fb8d7400700 1 --2- 172.21.15.107:0/2687850316 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8d00fe0e0 0x7fb8d010deb0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:38:13.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:13.515+0000 7fb8d7400700 1 -- 172.21.15.107:0/2687850316 >> 172.21.15.107:0/2687850316 conn(0x7fb8d00f8620 msgr2=0x7fb8d01013a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:38:13.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:13.515+0000 7fb8d7400700 1 -- 172.21.15.107:0/2687850316 shutdown_connections 2024-03-04T13:38:13.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:13.515+0000 7fb8d7400700 1 -- 172.21.15.107:0/2687850316 wait complete. 2024-03-04T13:38:14.495 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 -- bash -c 'ceph orch ls' 2024-03-04T13:38:14.974 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/mon.smithi107/config 2024-03-04T13:38:15.086 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:14 smithi107 ceph-mon[28739]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-04T13:38:15.086 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:14 smithi107 ceph-mon[28739]: from='client.14438 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:38:15.103 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:14 smithi169 ceph-mon[36313]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-04T13:38:15.103 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:14 smithi169 ceph-mon[36313]: from='client.14438 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:38:16.881 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:16 smithi107 ceph-mon[28739]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-04T13:38:17.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:17.049+0000 7fe9e1600700 1 -- 172.21.15.107:0/4024980487 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9dc066c30 msgr2=0x7fe9dc0ffb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:38:17.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:17.049+0000 7fe9e1600700 1 --2- 172.21.15.107:0/4024980487 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9dc066c30 0x7fe9dc0ffb30 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc004940 tx=0x7fe9cc009e30 comp rx=0 tx=0).stop 2024-03-04T13:38:17.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:17.049+0000 7fe9e1600700 1 -- 172.21.15.107:0/4024980487 shutdown_connections 2024-03-04T13:38:17.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:17.049+0000 7fe9e1600700 1 --2- 172.21.15.107:0/4024980487 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9dc100100 0x7fe9dc102520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:38:17.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:17.049+0000 7fe9e1600700 1 --2- 172.21.15.107:0/4024980487 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9dc066c30 0x7fe9dc0ffb30 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:38:17.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:17.049+0000 7fe9e1600700 1 -- 172.21.15.107:0/4024980487 >> 172.21.15.107:0/4024980487 conn(0x7fe9dc0f7520 msgr2=0x7fe9dc0f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:38:17.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:17.049+0000 7fe9e1600700 1 -- 172.21.15.107:0/4024980487 shutdown_connections 2024-03-04T13:38:17.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:17.049+0000 7fe9e1600700 1 -- 172.21.15.107:0/4024980487 wait complete. 2024-03-04T13:38:17.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:17.050+0000 7fe9e1600700 1 Processor -- start 2024-03-04T13:38:17.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:17.050+0000 7fe9e1600700 1 -- start start 2024-03-04T13:38:17.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:17.050+0000 7fe9e1600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9dc066c30 0x7fe9dc0fd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:38:17.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:17.050+0000 7fe9e1600700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9dc100100 0x7fe9dc0fba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:38:17.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:17.050+0000 7fe9e1600700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9dc0fc030 con 0x7fe9dc066c30 2024-03-04T13:38:17.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:17.050+0000 7fe9e1600700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9dc0fc1a0 con 0x7fe9dc100100 2024-03-04T13:38:17.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:17.050+0000 7fe9daa00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9dc066c30 0x7fe9dc0fd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:38:17.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:17.051+0000 7fe9daa00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9dc066c30 0x7fe9dc0fd5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44828/0 (socket says 172.21.15.107:44828) 2024-03-04T13:38:17.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:17.051+0000 7fe9daa00700 1 -- 172.21.15.107:0/900052266 learned_addr learned my addr 172.21.15.107:0/900052266 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:38:17.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:17.051+0000 7fe9da000700 1 --2- 172.21.15.107:0/900052266 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9dc100100 0x7fe9dc0fba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:38:17.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:17.051+0000 7fe9daa00700 1 -- 172.21.15.107:0/900052266 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9dc100100 msgr2=0x7fe9dc0fba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:38:17.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:17.051+0000 7fe9daa00700 1 --2- 172.21.15.107:0/900052266 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9dc100100 0x7fe9dc0fba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:38:17.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:17.051+0000 7fe9daa00700 1 -- 172.21.15.107:0/900052266 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9cc005040 con 0x7fe9dc066c30 2024-03-04T13:38:17.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:17.051+0000 7fe9daa00700 1 --2- 172.21.15.107:0/900052266 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9dc066c30 0x7fe9dc0fd5b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc00b3b0 tx=0x7fe9cc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:38:17.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:17.052+0000 7fe9d2a00700 1 -- 172.21.15.107:0/900052266 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe9cc00e040 con 0x7fe9dc066c30 2024-03-04T13:38:17.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:17.052+0000 7fe9d2a00700 1 -- 172.21.15.107:0/900052266 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe9cc00bd90 con 0x7fe9dc066c30 2024-03-04T13:38:17.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:17.052+0000 7fe9d2a00700 1 -- 172.21.15.107:0/900052266 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe9cc01c630 con 0x7fe9dc066c30 2024-03-04T13:38:17.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:17.052+0000 7fe9e1600700 1 -- 172.21.15.107:0/900052266 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9dc0fc420 con 0x7fe9dc066c30 2024-03-04T13:38:17.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:17.052+0000 7fe9e1600700 1 -- 172.21.15.107:0/900052266 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9dc1159c0 con 0x7fe9dc066c30 2024-03-04T13:38:17.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:17.053+0000 7fe9e1600700 1 -- 172.21.15.107:0/900052266 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9dc107a90 con 0x7fe9dc066c30 2024-03-04T13:38:17.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:17.054+0000 7fe9d2a00700 1 -- 172.21.15.107:0/900052266 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86623+0+0 (secure 0 0 0) 0x7fe9cc0139f0 con 0x7fe9dc066c30 2024-03-04T13:38:17.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:17.056+0000 7fe9d2a00700 1 --2- 172.21.15.107:0/900052266 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fe9c00692b0 0x7fe9c006b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:38:17.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:17.056+0000 7fe9d2a00700 1 -- 172.21.15.107:0/900052266 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fe9cc088390 con 0x7fe9dc066c30 2024-03-04T13:38:17.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:17.056+0000 7fe9da000700 1 --2- 172.21.15.107:0/900052266 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fe9c00692b0 0x7fe9c006b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:38:17.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:17.057+0000 7fe9da000700 1 --2- 172.21.15.107:0/900052266 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fe9c00692b0 0x7fe9c006b760 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe9c8007fd0 tx=0x7fe9c8009040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:38:17.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:17.061+0000 7fe9d2a00700 1 -- 172.21.15.107:0/900052266 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fe9cc058520 con 0x7fe9dc066c30 2024-03-04T13:38:17.103 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:16 smithi169 ceph-mon[36313]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-04T13:38:17.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:17.279+0000 7fe9e1600700 1 -- 172.21.15.107:0/900052266 --> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fe9dc02b310 con 0x7fe9c00692b0 2024-03-04T13:38:17.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:17.286+0000 7fe9d2a00700 1 -- 172.21.15.107:0/900052266 <== mgr.14212 v2:172.21.15.107:6800/3848495603 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+1179 (secure 0 0 0) 0x7fe9dc02b310 con 0x7fe9c00692b0 2024-03-04T13:38:17.280 INFO:teuthology.orchestra.run.smithi107.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-03-04T13:38:17.280 INFO:teuthology.orchestra.run.smithi107.stdout:alertmanager ?:9093,9094 1/1 2m ago 7m count:1 2024-03-04T13:38:17.280 INFO:teuthology.orchestra.run.smithi107.stdout:crash 2/2 2m ago 7m * 2024-03-04T13:38:17.280 INFO:teuthology.orchestra.run.smithi107.stdout:grafana ?:3000 1/1 2m ago 7m count:1 2024-03-04T13:38:17.280 INFO:teuthology.orchestra.run.smithi107.stdout:mgr 2/2 2m ago 7m count:2 2024-03-04T13:38:17.280 INFO:teuthology.orchestra.run.smithi107.stdout:mon 2/2 2m ago 6m smithi107:172.21.15.107=smithi107;smithi169:172.21.15.169=smithi169;count:2 2024-03-04T13:38:17.280 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter ?:9100 2/2 2m ago 7m * 2024-03-04T13:38:17.280 INFO:teuthology.orchestra.run.smithi107.stdout:osd 6 2m ago - 2024-03-04T13:38:17.280 INFO:teuthology.orchestra.run.smithi107.stdout:prometheus ?:9095 1/1 2m ago 7m count:1 2024-03-04T13:38:17.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:17.289+0000 7fe9e1600700 1 -- 172.21.15.107:0/900052266 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fe9c00692b0 msgr2=0x7fe9c006b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:38:17.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:17.289+0000 7fe9e1600700 1 --2- 172.21.15.107:0/900052266 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fe9c00692b0 0x7fe9c006b760 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe9c8007fd0 tx=0x7fe9c8009040 comp rx=0 tx=0).stop 2024-03-04T13:38:17.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:17.289+0000 7fe9e1600700 1 -- 172.21.15.107:0/900052266 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9dc066c30 msgr2=0x7fe9dc0fd5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:38:17.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:17.289+0000 7fe9e1600700 1 --2- 172.21.15.107:0/900052266 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9dc066c30 0x7fe9dc0fd5b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc00b3b0 tx=0x7fe9cc009e30 comp rx=0 tx=0).stop 2024-03-04T13:38:17.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:17.289+0000 7fe9e1600700 1 -- 172.21.15.107:0/900052266 shutdown_connections 2024-03-04T13:38:17.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:17.289+0000 7fe9e1600700 1 --2- 172.21.15.107:0/900052266 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fe9c00692b0 0x7fe9c006b760 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:38:17.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:17.289+0000 7fe9e1600700 1 --2- 172.21.15.107:0/900052266 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9dc066c30 0x7fe9dc0fd5b0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:38:17.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:17.289+0000 7fe9e1600700 1 --2- 172.21.15.107:0/900052266 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9dc100100 0x7fe9dc0fba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:38:17.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:17.289+0000 7fe9e1600700 1 -- 172.21.15.107:0/900052266 >> 172.21.15.107:0/900052266 conn(0x7fe9dc0f7520 msgr2=0x7fe9dc0f9900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:38:17.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:17.290+0000 7fe9e1600700 1 -- 172.21.15.107:0/900052266 shutdown_connections 2024-03-04T13:38:17.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:17.290+0000 7fe9e1600700 1 -- 172.21.15.107:0/900052266 wait complete. 2024-03-04T13:38:18.248 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 -- bash -c 'ceph orch host ls' 2024-03-04T13:38:18.712 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/mon.smithi107/config 2024-03-04T13:38:19.049 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:18 smithi107 ceph-mon[28739]: from='client.14442 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:38:19.049 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:18 smithi107 ceph-mon[28739]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-04T13:38:19.103 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:18 smithi169 ceph-mon[36313]: from='client.14442 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:38:19.103 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:18 smithi169 ceph-mon[36313]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-04T13:38:21.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:21.049+0000 7fa41b600700 1 -- 172.21.15.107:0/598760910 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4140fe0e0 msgr2=0x7fa4140fe550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:38:21.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:21.049+0000 7fa41b600700 1 --2- 172.21.15.107:0/598760910 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4140fe0e0 0x7fa4140fe550 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa400004940 tx=0x7fa400009e30 comp rx=0 tx=0).stop 2024-03-04T13:38:21.043 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:20 smithi107 ceph-mon[28739]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-04T13:38:21.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:21.049+0000 7fa41b600700 1 -- 172.21.15.107:0/598760910 shutdown_connections 2024-03-04T13:38:21.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:21.049+0000 7fa41b600700 1 --2- 172.21.15.107:0/598760910 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4140fe0e0 0x7fa4140fe550 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:38:21.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:21.049+0000 7fa41b600700 1 --2- 172.21.15.107:0/598760910 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa4140fcef0 0x7fa4140fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:38:21.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:21.049+0000 7fa41b600700 1 -- 172.21.15.107:0/598760910 >> 172.21.15.107:0/598760910 conn(0x7fa4140f8620 msgr2=0x7fa4140faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:38:21.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:21.049+0000 7fa41b600700 1 -- 172.21.15.107:0/598760910 shutdown_connections 2024-03-04T13:38:21.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:21.049+0000 7fa41b600700 1 -- 172.21.15.107:0/598760910 wait complete. 2024-03-04T13:38:21.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:21.050+0000 7fa41b600700 1 Processor -- start 2024-03-04T13:38:21.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:21.050+0000 7fa41b600700 1 -- start start 2024-03-04T13:38:21.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:21.051+0000 7fa41b600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4140fcef0 0x7fa41410b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:38:21.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:21.051+0000 7fa41b600700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa4140fe0e0 0x7fa41410bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:38:21.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:21.051+0000 7fa41b600700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa41410c340 con 0x7fa4140fcef0 2024-03-04T13:38:21.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:21.051+0000 7fa41b600700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa41410cb50 con 0x7fa4140fe0e0 2024-03-04T13:38:21.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:21.051+0000 7fa418c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4140fcef0 0x7fa41410b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:38:21.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:21.051+0000 7fa418c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4140fcef0 0x7fa41410b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41734/0 (socket says 172.21.15.107:41734) 2024-03-04T13:38:21.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:21.051+0000 7fa418c00700 1 -- 172.21.15.107:0/1209944073 learned_addr learned my addr 172.21.15.107:0/1209944073 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:38:21.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:21.051+0000 7fa413e00700 1 --2- 172.21.15.107:0/1209944073 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa4140fe0e0 0x7fa41410bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:38:21.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:21.051+0000 7fa418c00700 1 -- 172.21.15.107:0/1209944073 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa4140fe0e0 msgr2=0x7fa41410bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:38:21.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:21.051+0000 7fa418c00700 1 --2- 172.21.15.107:0/1209944073 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa4140fe0e0 0x7fa41410bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:38:21.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:21.051+0000 7fa418c00700 1 -- 172.21.15.107:0/1209944073 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa400005040 con 0x7fa4140fcef0 2024-03-04T13:38:21.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:21.052+0000 7fa418c00700 1 --2- 172.21.15.107:0/1209944073 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4140fcef0 0x7fa41410b720 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa40800b790 tx=0x7fa408009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:38:21.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:21.052+0000 7fa410c00700 1 -- 172.21.15.107:0/1209944073 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa40800f6f0 con 0x7fa4140fcef0 2024-03-04T13:38:21.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:21.052+0000 7fa410c00700 1 -- 172.21.15.107:0/1209944073 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa40800fd20 con 0x7fa4140fcef0 2024-03-04T13:38:21.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:21.052+0000 7fa41b600700 1 -- 172.21.15.107:0/1209944073 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa41410ce30 con 0x7fa4140fcef0 2024-03-04T13:38:21.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:21.052+0000 7fa410c00700 1 -- 172.21.15.107:0/1209944073 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa4080103a0 con 0x7fa4140fcef0 2024-03-04T13:38:21.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:21.052+0000 7fa41b600700 1 -- 172.21.15.107:0/1209944073 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa41410d370 con 0x7fa4140fcef0 2024-03-04T13:38:21.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:21.054+0000 7fa41b600700 1 -- 172.21.15.107:0/1209944073 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa41404bed0 con 0x7fa4140fcef0 2024-03-04T13:38:21.048 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:21.055+0000 7fa410c00700 1 -- 172.21.15.107:0/1209944073 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86623+0+0 (secure 0 0 0) 0x7fa408011400 con 0x7fa4140fcef0 2024-03-04T13:38:21.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:21.056+0000 7fa410c00700 1 --2- 172.21.15.107:0/1209944073 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fa3fc0692a0 0x7fa3fc06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:38:21.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:21.056+0000 7fa410c00700 1 -- 172.21.15.107:0/1209944073 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fa408086d90 con 0x7fa4140fcef0 2024-03-04T13:38:21.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:21.056+0000 7fa413e00700 1 --2- 172.21.15.107:0/1209944073 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fa3fc0692a0 0x7fa3fc06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:38:21.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:21.057+0000 7fa413e00700 1 --2- 172.21.15.107:0/1209944073 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fa3fc0692a0 0x7fa3fc06b750 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa400004f90 tx=0x7fa400009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:38:21.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:21.060+0000 7fa410c00700 1 -- 172.21.15.107:0/1209944073 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fa4080532e0 con 0x7fa4140fcef0 2024-03-04T13:38:21.103 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:20 smithi169 ceph-mon[36313]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-04T13:38:21.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:21.277+0000 7fa41b600700 1 -- 172.21.15.107:0/1209944073 --> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fa41410d770 con 0x7fa3fc0692a0 2024-03-04T13:38:21.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:21.280+0000 7fa410c00700 1 -- 172.21.15.107:0/1209944073 <== mgr.14212 v2:172.21.15.107:6800/3848495603 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+148 (secure 0 0 0) 0x7fa41410d770 con 0x7fa3fc0692a0 2024-03-04T13:38:21.273 INFO:teuthology.orchestra.run.smithi107.stdout:HOST ADDR LABELS STATUS 2024-03-04T13:38:21.273 INFO:teuthology.orchestra.run.smithi107.stdout:smithi107 172.21.15.107 2024-03-04T13:38:21.273 INFO:teuthology.orchestra.run.smithi107.stdout:smithi169 172.21.15.169 2024-03-04T13:38:21.273 INFO:teuthology.orchestra.run.smithi107.stdout:2 hosts in cluster 2024-03-04T13:38:21.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:21.282+0000 7fa41b600700 1 -- 172.21.15.107:0/1209944073 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fa3fc0692a0 msgr2=0x7fa3fc06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:38:21.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:21.282+0000 7fa41b600700 1 --2- 172.21.15.107:0/1209944073 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fa3fc0692a0 0x7fa3fc06b750 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa400004f90 tx=0x7fa400009e30 comp rx=0 tx=0).stop 2024-03-04T13:38:21.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:21.282+0000 7fa41b600700 1 -- 172.21.15.107:0/1209944073 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4140fcef0 msgr2=0x7fa41410b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:38:21.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:21.282+0000 7fa41b600700 1 --2- 172.21.15.107:0/1209944073 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4140fcef0 0x7fa41410b720 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa40800b790 tx=0x7fa408009220 comp rx=0 tx=0).stop 2024-03-04T13:38:21.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:21.283+0000 7fa41b600700 1 -- 172.21.15.107:0/1209944073 shutdown_connections 2024-03-04T13:38:21.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:21.283+0000 7fa41b600700 1 --2- 172.21.15.107:0/1209944073 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fa3fc0692a0 0x7fa3fc06b750 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:38:21.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:21.283+0000 7fa41b600700 1 --2- 172.21.15.107:0/1209944073 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4140fcef0 0x7fa41410b720 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:38:21.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:21.283+0000 7fa41b600700 1 --2- 172.21.15.107:0/1209944073 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa4140fe0e0 0x7fa41410bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:38:21.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:21.283+0000 7fa41b600700 1 -- 172.21.15.107:0/1209944073 >> 172.21.15.107:0/1209944073 conn(0x7fa4140f8620 msgr2=0x7fa4140ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:38:21.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:21.283+0000 7fa41b600700 1 -- 172.21.15.107:0/1209944073 shutdown_connections 2024-03-04T13:38:21.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:21.283+0000 7fa41b600700 1 -- 172.21.15.107:0/1209944073 wait complete. 2024-03-04T13:38:22.237 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 -- bash -c 'ceph orch device ls' 2024-03-04T13:38:22.686 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/mon.smithi107/config 2024-03-04T13:38:23.086 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:22 smithi107 ceph-mon[28739]: from='client.14446 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:38:23.086 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:22 smithi107 ceph-mon[28739]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-04T13:38:23.103 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:22 smithi169 ceph-mon[36313]: from='client.14446 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:38:23.103 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:22 smithi169 ceph-mon[36313]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-04T13:38:24.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:24.866+0000 7f99ce400700 1 -- 172.21.15.107:0/1935584597 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99c80ff150 msgr2=0x7f99c80ff560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:38:24.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:24.866+0000 7f99ce400700 1 --2- 172.21.15.107:0/1935584597 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99c80ff150 0x7f99c80ff560 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f99b8004940 tx=0x7f99b8009e30 comp rx=0 tx=0).stop 2024-03-04T13:38:24.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:24.866+0000 7f99ce400700 1 -- 172.21.15.107:0/1935584597 shutdown_connections 2024-03-04T13:38:24.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:24.866+0000 7f99ce400700 1 --2- 172.21.15.107:0/1935584597 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99c8100360 0x7f99c81007b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:38:24.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:24.866+0000 7f99ce400700 1 --2- 172.21.15.107:0/1935584597 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99c80ff150 0x7f99c80ff560 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:38:24.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:24.866+0000 7f99ce400700 1 -- 172.21.15.107:0/1935584597 >> 172.21.15.107:0/1935584597 conn(0x7f99c80fa920 msgr2=0x7f99c80fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:38:24.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:24.866+0000 7f99ce400700 1 -- 172.21.15.107:0/1935584597 shutdown_connections 2024-03-04T13:38:24.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:24.866+0000 7f99ce400700 1 -- 172.21.15.107:0/1935584597 wait complete. 2024-03-04T13:38:24.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:24.867+0000 7f99ce400700 1 Processor -- start 2024-03-04T13:38:24.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:24.867+0000 7f99ce400700 1 -- start start 2024-03-04T13:38:24.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:24.868+0000 7f99ce400700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99c80ff150 0x7f99c810fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:38:24.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:24.868+0000 7f99ce400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99c8100360 0x7f99c81100f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:38:24.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:24.868+0000 7f99ce400700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99c81106f0 con 0x7f99c8100360 2024-03-04T13:38:24.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:24.868+0000 7f99ce400700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99c8110f90 con 0x7f99c80ff150 2024-03-04T13:38:24.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:24.868+0000 7f99c6a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99c8100360 0x7f99c81100f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:38:24.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:24.868+0000 7f99c6a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99c8100360 0x7f99c81100f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41752/0 (socket says 172.21.15.107:41752) 2024-03-04T13:38:24.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:24.868+0000 7f99c6a00700 1 -- 172.21.15.107:0/1700519765 learned_addr learned my addr 172.21.15.107:0/1700519765 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:38:24.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:24.868+0000 7f99c7400700 1 --2- 172.21.15.107:0/1700519765 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99c80ff150 0x7f99c810fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:38:24.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:24.868+0000 7f99c6a00700 1 -- 172.21.15.107:0/1700519765 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99c80ff150 msgr2=0x7f99c810fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:38:24.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:24.868+0000 7f99c6a00700 1 --2- 172.21.15.107:0/1700519765 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99c80ff150 0x7f99c810fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:38:24.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:24.868+0000 7f99c6a00700 1 -- 172.21.15.107:0/1700519765 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99b8005040 con 0x7f99c8100360 2024-03-04T13:38:24.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:24.869+0000 7f99c6a00700 1 --2- 172.21.15.107:0/1700519765 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99c8100360 0x7f99c81100f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f99b4002a50 tx=0x7f99b40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:38:24.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:24.869+0000 7f99bf400700 1 -- 172.21.15.107:0/1700519765 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f99b4007cc0 con 0x7f99c8100360 2024-03-04T13:38:24.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:24.869+0000 7f99bf400700 1 -- 172.21.15.107:0/1700519765 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f99b400f040 con 0x7f99c8100360 2024-03-04T13:38:24.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:24.869+0000 7f99ce400700 1 -- 172.21.15.107:0/1700519765 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99c8111270 con 0x7f99c8100360 2024-03-04T13:38:24.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:24.869+0000 7f99bf400700 1 -- 172.21.15.107:0/1700519765 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f99b400db70 con 0x7f99c8100360 2024-03-04T13:38:24.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:24.869+0000 7f99ce400700 1 -- 172.21.15.107:0/1700519765 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99c81117b0 con 0x7f99c8100360 2024-03-04T13:38:24.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:24.870+0000 7f99ce400700 1 -- 172.21.15.107:0/1700519765 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99c804bed0 con 0x7f99c8100360 2024-03-04T13:38:24.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:24.871+0000 7f99bf400700 1 -- 172.21.15.107:0/1700519765 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86623+0+0 (secure 0 0 0) 0x7f99b400dd10 con 0x7f99c8100360 2024-03-04T13:38:24.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:24.872+0000 7f99bf400700 1 --2- 172.21.15.107:0/1700519765 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f99ac0692b0 0x7f99ac06b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:38:24.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:24.872+0000 7f99bf400700 1 -- 172.21.15.107:0/1700519765 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f99b4086550 con 0x7f99c8100360 2024-03-04T13:38:24.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:24.872+0000 7f99c7400700 1 --2- 172.21.15.107:0/1700519765 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f99ac0692b0 0x7f99ac06b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:38:24.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:24.873+0000 7f99c7400700 1 --2- 172.21.15.107:0/1700519765 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f99ac0692b0 0x7f99ac06b760 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f99b800b3b0 tx=0x7f99b8009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:38:24.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:24.876+0000 7f99bf400700 1 -- 172.21.15.107:0/1700519765 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f99b40565b0 con 0x7f99c8100360 2024-03-04T13:38:25.086 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:24 smithi107 ceph-mon[28739]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-04T13:38:25.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:25.095+0000 7f99ce400700 1 -- 172.21.15.107:0/1700519765 --> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f99c8111c50 con 0x7f99ac0692b0 2024-03-04T13:38:25.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:25.098+0000 7f99bf400700 1 -- 172.21.15.107:0/1700519765 <== mgr.14212 v2:172.21.15.107:6800/3848495603 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+399 (secure 0 0 0) 0x7f99c8111c50 con 0x7f99ac0692b0 2024-03-04T13:38:25.092 INFO:teuthology.orchestra.run.smithi107.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-03-04T13:38:25.092 INFO:teuthology.orchestra.run.smithi107.stdout:smithi107 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400NK400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-03-04T13:38:25.092 INFO:teuthology.orchestra.run.smithi107.stdout:smithi169 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300LN400BGN 372G No 87s ago Has a FileSystem, LVM detected 2024-03-04T13:38:25.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:25.101+0000 7f99ce400700 1 -- 172.21.15.107:0/1700519765 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f99ac0692b0 msgr2=0x7f99ac06b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:38:25.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:25.101+0000 7f99ce400700 1 --2- 172.21.15.107:0/1700519765 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f99ac0692b0 0x7f99ac06b760 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f99b800b3b0 tx=0x7f99b8009e30 comp rx=0 tx=0).stop 2024-03-04T13:38:25.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:25.101+0000 7f99ce400700 1 -- 172.21.15.107:0/1700519765 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99c8100360 msgr2=0x7f99c81100f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:38:25.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:25.101+0000 7f99ce400700 1 --2- 172.21.15.107:0/1700519765 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99c8100360 0x7f99c81100f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f99b4002a50 tx=0x7f99b40046b0 comp rx=0 tx=0).stop 2024-03-04T13:38:25.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:25.101+0000 7f99ce400700 1 -- 172.21.15.107:0/1700519765 shutdown_connections 2024-03-04T13:38:25.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:25.101+0000 7f99ce400700 1 --2- 172.21.15.107:0/1700519765 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f99ac0692b0 0x7f99ac06b760 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:38:25.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:25.101+0000 7f99ce400700 1 --2- 172.21.15.107:0/1700519765 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99c80ff150 0x7f99c810fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:38:25.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:25.101+0000 7f99ce400700 1 --2- 172.21.15.107:0/1700519765 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99c8100360 0x7f99c81100f0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:38:25.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:25.101+0000 7f99ce400700 1 -- 172.21.15.107:0/1700519765 >> 172.21.15.107:0/1700519765 conn(0x7f99c80fa920 msgr2=0x7f99c8103590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:38:25.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:25.102+0000 7f99ce400700 1 -- 172.21.15.107:0/1700519765 shutdown_connections 2024-03-04T13:38:25.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:25.102+0000 7f99ce400700 1 -- 172.21.15.107:0/1700519765 wait complete. 2024-03-04T13:38:25.103 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:24 smithi169 ceph-mon[36313]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-04T13:38:25.893 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-04T13:38:25.903 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi107.front.sepia.ceph.com 2024-03-04T13:38:25.903 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 -- bash -c 'ceph fs volume create cephfs --placement=4' 2024-03-04T13:38:26.386 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/mon.smithi107/config 2024-03-04T13:38:27.086 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:26 smithi107 ceph-mon[28739]: from='client.14450 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:38:27.086 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:26 smithi107 ceph-mon[28739]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-04T13:38:27.103 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:26 smithi169 ceph-mon[36313]: from='client.14450 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:38:27.103 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:26 smithi169 ceph-mon[36313]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-04T13:38:28.054 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:27 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:38:28.103 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:27 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:38:28.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:28.585+0000 7fdd80400700 1 -- 172.21.15.107:0/3452189238 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd78100360 msgr2=0x7fdd781007b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:38:28.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:28.585+0000 7fdd80400700 1 --2- 172.21.15.107:0/3452189238 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd78100360 0x7fdd781007b0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fdd64004940 tx=0x7fdd64009e30 comp rx=0 tx=0).stop 2024-03-04T13:38:28.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:28.585+0000 7fdd80400700 1 -- 172.21.15.107:0/3452189238 shutdown_connections 2024-03-04T13:38:28.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:28.585+0000 7fdd80400700 1 --2- 172.21.15.107:0/3452189238 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd78100360 0x7fdd781007b0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:38:28.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:28.585+0000 7fdd80400700 1 --2- 172.21.15.107:0/3452189238 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd780ff150 0x7fdd780ff560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:38:28.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:28.585+0000 7fdd80400700 1 -- 172.21.15.107:0/3452189238 >> 172.21.15.107:0/3452189238 conn(0x7fdd780fa920 msgr2=0x7fdd780fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:38:28.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:28.585+0000 7fdd80400700 1 -- 172.21.15.107:0/3452189238 shutdown_connections 2024-03-04T13:38:28.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:28.585+0000 7fdd80400700 1 -- 172.21.15.107:0/3452189238 wait complete. 2024-03-04T13:38:28.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:28.586+0000 7fdd80400700 1 Processor -- start 2024-03-04T13:38:28.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:28.586+0000 7fdd80400700 1 -- start start 2024-03-04T13:38:28.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:28.586+0000 7fdd80400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd780ff150 0x7fdd7810fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:38:28.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:28.586+0000 7fdd80400700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd78100360 0x7fdd781100b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:38:28.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:28.586+0000 7fdd80400700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd781106b0 con 0x7fdd780ff150 2024-03-04T13:38:28.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:28.586+0000 7fdd80400700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd78110f50 con 0x7fdd78100360 2024-03-04T13:38:28.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:28.587+0000 7fdd7da00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd780ff150 0x7fdd7810fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:38:28.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:28.587+0000 7fdd7da00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd780ff150 0x7fdd7810fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37362/0 (socket says 172.21.15.107:37362) 2024-03-04T13:38:28.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:28.587+0000 7fdd7da00700 1 -- 172.21.15.107:0/2460066429 learned_addr learned my addr 172.21.15.107:0/2460066429 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:38:28.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:28.587+0000 7fdd7d000700 1 --2- 172.21.15.107:0/2460066429 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd78100360 0x7fdd781100b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:38:28.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:28.587+0000 7fdd7da00700 1 -- 172.21.15.107:0/2460066429 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd78100360 msgr2=0x7fdd781100b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:38:28.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:28.587+0000 7fdd7da00700 1 --2- 172.21.15.107:0/2460066429 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd78100360 0x7fdd781100b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:38:28.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:28.587+0000 7fdd7da00700 1 -- 172.21.15.107:0/2460066429 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd64005040 con 0x7fdd780ff150 2024-03-04T13:38:28.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:28.587+0000 7fdd7da00700 1 --2- 172.21.15.107:0/2460066429 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd780ff150 0x7fdd7810fb70 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c00c8b0 tx=0x7fdd6c004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:38:28.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:28.588+0000 7fdd75600700 1 -- 172.21.15.107:0/2460066429 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd6c00a760 con 0x7fdd780ff150 2024-03-04T13:38:28.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:28.588+0000 7fdd75600700 1 -- 172.21.15.107:0/2460066429 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdd6c00ad90 con 0x7fdd780ff150 2024-03-04T13:38:28.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:28.588+0000 7fdd75600700 1 -- 172.21.15.107:0/2460066429 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd6c0113a0 con 0x7fdd780ff150 2024-03-04T13:38:28.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:28.588+0000 7fdd80400700 1 -- 172.21.15.107:0/2460066429 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd78111230 con 0x7fdd780ff150 2024-03-04T13:38:28.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:28.588+0000 7fdd80400700 1 -- 172.21.15.107:0/2460066429 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd78111770 con 0x7fdd780ff150 2024-03-04T13:38:28.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:28.589+0000 7fdd80400700 1 -- 172.21.15.107:0/2460066429 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd7804bed0 con 0x7fdd780ff150 2024-03-04T13:38:28.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:28.590+0000 7fdd75600700 1 -- 172.21.15.107:0/2460066429 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86623+0+0 (secure 0 0 0) 0x7fdd6c012400 con 0x7fdd780ff150 2024-03-04T13:38:28.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:28.591+0000 7fdd75600700 1 --2- 172.21.15.107:0/2460066429 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fdd600692b0 0x7fdd6006b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:38:28.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:28.591+0000 7fdd75600700 1 -- 172.21.15.107:0/2460066429 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fdd6c086e50 con 0x7fdd780ff150 2024-03-04T13:38:28.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:28.591+0000 7fdd7d000700 1 --2- 172.21.15.107:0/2460066429 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fdd600692b0 0x7fdd6006b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:38:28.584 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:28.592+0000 7fdd7d000700 1 --2- 172.21.15.107:0/2460066429 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fdd600692b0 0x7fdd6006b760 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fdd6400b840 tx=0x7fdd64009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:38:28.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:28.596+0000 7fdd75600700 1 -- 172.21.15.107:0/2460066429 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fdd6c056f30 con 0x7fdd780ff150 2024-03-04T13:38:28.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:28.831+0000 7fdd80400700 1 -- 172.21.15.107:0/2460066429 --> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] -- mgr_command(tid 0: {"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7fdd78111c90 con 0x7fdd600692b0 2024-03-04T13:38:29.086 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:28 smithi107 ceph-mon[28739]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-04T13:38:29.103 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:28 smithi169 ceph-mon[36313]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-04T13:38:30.086 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:29 smithi107 ceph-mon[28739]: from='client.14454 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:38:30.086 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:29 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-03-04T13:38:30.103 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:29 smithi169 ceph-mon[36313]: from='client.14454 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:38:30.103 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:29 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-03-04T13:38:30.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:30.894+0000 7fdd75600700 1 -- 172.21.15.107:0/2460066429 <== mgr.14212 v2:172.21.15.107:6800/3848495603 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fdd78111c90 con 0x7fdd600692b0 2024-03-04T13:38:30.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:30.897+0000 7fdd80400700 1 -- 172.21.15.107:0/2460066429 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fdd600692b0 msgr2=0x7fdd6006b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:38:30.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:30.897+0000 7fdd80400700 1 --2- 172.21.15.107:0/2460066429 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fdd600692b0 0x7fdd6006b760 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fdd6400b840 tx=0x7fdd64009e30 comp rx=0 tx=0).stop 2024-03-04T13:38:30.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:30.897+0000 7fdd80400700 1 -- 172.21.15.107:0/2460066429 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd780ff150 msgr2=0x7fdd7810fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:38:30.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:30.897+0000 7fdd80400700 1 --2- 172.21.15.107:0/2460066429 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd780ff150 0x7fdd7810fb70 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c00c8b0 tx=0x7fdd6c004470 comp rx=0 tx=0).stop 2024-03-04T13:38:30.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:30.897+0000 7fdd80400700 1 -- 172.21.15.107:0/2460066429 shutdown_connections 2024-03-04T13:38:30.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:30.897+0000 7fdd80400700 1 --2- 172.21.15.107:0/2460066429 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fdd600692b0 0x7fdd6006b760 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:38:30.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:30.897+0000 7fdd80400700 1 --2- 172.21.15.107:0/2460066429 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd780ff150 0x7fdd7810fb70 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:38:30.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:30.897+0000 7fdd80400700 1 --2- 172.21.15.107:0/2460066429 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd78100360 0x7fdd781100b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:38:30.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:30.897+0000 7fdd80400700 1 -- 172.21.15.107:0/2460066429 >> 172.21.15.107:0/2460066429 conn(0x7fdd780fa920 msgr2=0x7fdd78103590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:38:30.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:30.897+0000 7fdd80400700 1 -- 172.21.15.107:0/2460066429 shutdown_connections 2024-03-04T13:38:30.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:30.897+0000 7fdd80400700 1 -- 172.21.15.107:0/2460066429 wait complete. 2024-03-04T13:38:31.283 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:30 smithi107 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-mon-smithi107[28734]: 2024-03-04T13:38:30.854+0000 7f1a36000700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-04T13:38:31.283 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:30 smithi107 ceph-mon[28739]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-04T13:38:31.283 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:30 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-03-04T13:38:31.283 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:30 smithi107 ceph-mon[28739]: osdmap e42: 6 total, 6 up, 6 in 2024-03-04T13:38:31.283 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:30 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-03-04T13:38:31.353 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:30 smithi169 ceph-mon[36313]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-04T13:38:31.353 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:30 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-03-04T13:38:31.353 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:30 smithi169 ceph-mon[36313]: osdmap e42: 6 total, 6 up, 6 in 2024-03-04T13:38:31.353 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:30 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-03-04T13:38:31.803 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 -- bash -c 'ceph fs dump' 2024-03-04T13:38:31.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:31 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-03-04T13:38:31.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:31 smithi107 ceph-mon[28739]: osdmap e43: 6 total, 6 up, 6 in 2024-03-04T13:38:31.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:31 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-03-04T13:38:31.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:31 smithi107 ceph-mon[28739]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-04T13:38:31.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:31 smithi107 ceph-mon[28739]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-04T13:38:31.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:31 smithi107 ceph-mon[28739]: osdmap e44: 6 total, 6 up, 6 in 2024-03-04T13:38:31.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:31 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-03-04T13:38:31.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:31 smithi107 ceph-mon[28739]: fsmap cephfs:0 2024-03-04T13:38:31.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:31 smithi107 ceph-mon[28739]: Saving service mds.cephfs spec with placement count:4 2024-03-04T13:38:31.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:31 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:38:31.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:31 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:38:31.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:31 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:38:31.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:31 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:38:31.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:31 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:38:31.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:31 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi107.riexmm", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-04T13:38:31.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:31 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi107.riexmm", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-04T13:38:31.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:31 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:38:31.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:31 smithi107 ceph-mon[28739]: Deploying daemon mds.cephfs.smithi107.riexmm on smithi107 2024-03-04T13:38:32.354 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:31 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-03-04T13:38:32.354 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:31 smithi169 ceph-mon[36313]: osdmap e43: 6 total, 6 up, 6 in 2024-03-04T13:38:32.354 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:31 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-03-04T13:38:32.354 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:31 smithi169 ceph-mon[36313]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-04T13:38:32.354 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:31 smithi169 ceph-mon[36313]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-04T13:38:32.354 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:31 smithi169 ceph-mon[36313]: osdmap e44: 6 total, 6 up, 6 in 2024-03-04T13:38:32.354 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:31 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-03-04T13:38:32.354 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:31 smithi169 ceph-mon[36313]: fsmap cephfs:0 2024-03-04T13:38:32.354 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:31 smithi169 ceph-mon[36313]: Saving service mds.cephfs spec with placement count:4 2024-03-04T13:38:32.354 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:31 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:38:32.355 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:31 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:38:32.355 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:31 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:38:32.355 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:31 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:38:32.355 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:31 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:38:32.355 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:31 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi107.riexmm", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-04T13:38:32.355 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:31 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi107.riexmm", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-04T13:38:32.355 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:31 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:38:32.355 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:31 smithi169 ceph-mon[36313]: Deploying daemon mds.cephfs.smithi107.riexmm on smithi107 2024-03-04T13:38:32.663 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/mon.smithi107/config 2024-03-04T13:38:33.086 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:32 smithi107 ceph-mon[28739]: pgmap v214: 65 pgs: 64 unknown, 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-04T13:38:33.086 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:32 smithi107 ceph-mon[28739]: osdmap e45: 6 total, 6 up, 6 in 2024-03-04T13:38:33.353 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:32 smithi169 ceph-mon[36313]: pgmap v214: 65 pgs: 64 unknown, 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-04T13:38:33.353 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:32 smithi169 ceph-mon[36313]: osdmap e45: 6 total, 6 up, 6 in 2024-03-04T13:38:34.222 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:33 smithi107 ceph-mon[28739]: osdmap e46: 6 total, 6 up, 6 in 2024-03-04T13:38:34.353 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:33 smithi169 ceph-mon[36313]: osdmap e46: 6 total, 6 up, 6 in 2024-03-04T13:38:35.198 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:34 smithi107 ceph-mon[28739]: pgmap v217: 65 pgs: 4 creating+peering, 21 active+clean, 40 unknown; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-04T13:38:35.353 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:34 smithi169 ceph-mon[36313]: pgmap v217: 65 pgs: 4 creating+peering, 21 active+clean, 40 unknown; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-04T13:38:35.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:35.463+0000 7fc99b200700 1 -- 172.21.15.107:0/1417127829 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc994104750 msgr2=0x7fc994104ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:38:35.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:35.463+0000 7fc99b200700 1 --2- 172.21.15.107:0/1417127829 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc994104750 0x7fc994104ba0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc980004940 tx=0x7fc980009e30 comp rx=0 tx=0).stop 2024-03-04T13:38:35.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:35.463+0000 7fc99b200700 1 -- 172.21.15.107:0/1417127829 shutdown_connections 2024-03-04T13:38:35.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:35.463+0000 7fc99b200700 1 --2- 172.21.15.107:0/1417127829 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc994104750 0x7fc994104ba0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:38:35.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:35.463+0000 7fc99b200700 1 --2- 172.21.15.107:0/1417127829 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc994103540 0x7fc994103950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:38:35.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:35.463+0000 7fc99b200700 1 -- 172.21.15.107:0/1417127829 >> 172.21.15.107:0/1417127829 conn(0x7fc9940727d0 msgr2=0x7fc994074c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:38:35.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:35.463+0000 7fc99b200700 1 -- 172.21.15.107:0/1417127829 shutdown_connections 2024-03-04T13:38:35.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:35.463+0000 7fc99b200700 1 -- 172.21.15.107:0/1417127829 wait complete. 2024-03-04T13:38:35.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:35.464+0000 7fc99b200700 1 Processor -- start 2024-03-04T13:38:35.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:35.464+0000 7fc99b200700 1 -- start start 2024-03-04T13:38:35.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:35.464+0000 7fc99b200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc994103540 0x7fc994113f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:38:35.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:35.464+0000 7fc99b200700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc994104750 0x7fc9941144a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:38:35.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:35.465+0000 7fc99b200700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc994114aa0 con 0x7fc994103540 2024-03-04T13:38:35.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:35.465+0000 7fc99b200700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc994115340 con 0x7fc994104750 2024-03-04T13:38:35.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:35.465+0000 7fc993e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc994103540 0x7fc994113f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:38:35.458 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:35.465+0000 7fc993e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc994103540 0x7fc994113f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37400/0 (socket says 172.21.15.107:37400) 2024-03-04T13:38:35.458 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:35.465+0000 7fc993e00700 1 -- 172.21.15.107:0/686403479 learned_addr learned my addr 172.21.15.107:0/686403479 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:38:35.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:35.465+0000 7fc993400700 1 --2- 172.21.15.107:0/686403479 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc994104750 0x7fc9941144a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:38:35.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:35.465+0000 7fc993e00700 1 -- 172.21.15.107:0/686403479 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc994104750 msgr2=0x7fc9941144a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:38:35.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:35.465+0000 7fc993e00700 1 --2- 172.21.15.107:0/686403479 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc994104750 0x7fc9941144a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:38:35.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:35.465+0000 7fc993e00700 1 -- 172.21.15.107:0/686403479 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc980005040 con 0x7fc994103540 2024-03-04T13:38:35.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:35.465+0000 7fc993400700 1 --2- 172.21.15.107:0/686403479 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc994104750 0x7fc9941144a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T13:38:35.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:35.466+0000 7fc993e00700 1 --2- 172.21.15.107:0/686403479 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc994103540 0x7fc994113f60 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc98400ca90 tx=0x7fc984007f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:38:35.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:35.466+0000 7fc98be00700 1 -- 172.21.15.107:0/686403479 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc98400aa50 con 0x7fc994103540 2024-03-04T13:38:35.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:35.466+0000 7fc98be00700 1 -- 172.21.15.107:0/686403479 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc98400abb0 con 0x7fc994103540 2024-03-04T13:38:35.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:35.466+0000 7fc99b200700 1 -- 172.21.15.107:0/686403479 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc994115620 con 0x7fc994103540 2024-03-04T13:38:35.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:35.466+0000 7fc98be00700 1 -- 172.21.15.107:0/686403479 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc9840115e0 con 0x7fc994103540 2024-03-04T13:38:35.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:35.466+0000 7fc99b200700 1 -- 172.21.15.107:0/686403479 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc994115b60 con 0x7fc994103540 2024-03-04T13:38:35.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:35.467+0000 7fc99b200700 1 -- 172.21.15.107:0/686403479 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc99404bed0 con 0x7fc994103540 2024-03-04T13:38:35.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:35.468+0000 7fc98be00700 1 -- 172.21.15.107:0/686403479 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86623+0+0 (secure 0 0 0) 0x7fc98400ad20 con 0x7fc994103540 2024-03-04T13:38:35.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:35.469+0000 7fc98be00700 1 --2- 172.21.15.107:0/686403479 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fc9780692a0 0x7fc97806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:38:35.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:35.470+0000 7fc98be00700 1 -- 172.21.15.107:0/686403479 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fc984087980 con 0x7fc994103540 2024-03-04T13:38:35.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:35.470+0000 7fc993400700 1 --2- 172.21.15.107:0/686403479 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fc9780692a0 0x7fc97806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:38:35.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:35.470+0000 7fc993400700 1 --2- 172.21.15.107:0/686403479 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fc9780692a0 0x7fc97806b750 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc980009fd0 tx=0x7fc980009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:38:35.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:35.475+0000 7fc98be00700 1 -- 172.21.15.107:0/686403479 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fc984057620 con 0x7fc994103540 2024-03-04T13:38:35.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:35.756+0000 7fc99b200700 1 -- 172.21.15.107:0/686403479 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fc99410c020 con 0x7fc994103540 2024-03-04T13:38:35.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:35.757+0000 7fc98be00700 1 -- 172.21.15.107:0/686403479 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 3 v3) v1 ==== 75+0+1249 (secure 0 0 0) 0x7fc9840571b0 con 0x7fc994103540 2024-03-04T13:38:35.750 INFO:teuthology.orchestra.run.smithi107.stdout:e3 2024-03-04T13:38:35.750 INFO:teuthology.orchestra.run.smithi107.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-04T13:38:35.750 INFO:teuthology.orchestra.run.smithi107.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-04T13:38:35.750 INFO:teuthology.orchestra.run.smithi107.stdout:legacy client fscid: 1 2024-03-04T13:38:35.750 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:38:35.750 INFO:teuthology.orchestra.run.smithi107.stdout:Filesystem 'cephfs' (1) 2024-03-04T13:38:35.750 INFO:teuthology.orchestra.run.smithi107.stdout:fs_name cephfs 2024-03-04T13:38:35.750 INFO:teuthology.orchestra.run.smithi107.stdout:epoch 3 2024-03-04T13:38:35.750 INFO:teuthology.orchestra.run.smithi107.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-04T13:38:35.750 INFO:teuthology.orchestra.run.smithi107.stdout:created 2024-03-04T13:38:30.854004+0000 2024-03-04T13:38:35.751 INFO:teuthology.orchestra.run.smithi107.stdout:modified 2024-03-04T13:38:35.579643+0000 2024-03-04T13:38:35.751 INFO:teuthology.orchestra.run.smithi107.stdout:tableserver 0 2024-03-04T13:38:35.751 INFO:teuthology.orchestra.run.smithi107.stdout:root 0 2024-03-04T13:38:35.751 INFO:teuthology.orchestra.run.smithi107.stdout:session_timeout 60 2024-03-04T13:38:35.751 INFO:teuthology.orchestra.run.smithi107.stdout:session_autoclose 300 2024-03-04T13:38:35.751 INFO:teuthology.orchestra.run.smithi107.stdout:max_file_size 1099511627776 2024-03-04T13:38:35.751 INFO:teuthology.orchestra.run.smithi107.stdout:required_client_features {} 2024-03-04T13:38:35.751 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure 0 2024-03-04T13:38:35.751 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure_osd_epoch 0 2024-03-04T13:38:35.751 INFO:teuthology.orchestra.run.smithi107.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-04T13:38:35.751 INFO:teuthology.orchestra.run.smithi107.stdout:max_mds 1 2024-03-04T13:38:35.751 INFO:teuthology.orchestra.run.smithi107.stdout:in 0 2024-03-04T13:38:35.751 INFO:teuthology.orchestra.run.smithi107.stdout:up {0=14458} 2024-03-04T13:38:35.751 INFO:teuthology.orchestra.run.smithi107.stdout:failed 2024-03-04T13:38:35.751 INFO:teuthology.orchestra.run.smithi107.stdout:damaged 2024-03-04T13:38:35.752 INFO:teuthology.orchestra.run.smithi107.stdout:stopped 2024-03-04T13:38:35.752 INFO:teuthology.orchestra.run.smithi107.stdout:data_pools [3] 2024-03-04T13:38:35.752 INFO:teuthology.orchestra.run.smithi107.stdout:metadata_pool 2 2024-03-04T13:38:35.752 INFO:teuthology.orchestra.run.smithi107.stdout:inline_data disabled 2024-03-04T13:38:35.752 INFO:teuthology.orchestra.run.smithi107.stdout:balancer 2024-03-04T13:38:35.752 INFO:teuthology.orchestra.run.smithi107.stdout:standby_count_wanted 0 2024-03-04T13:38:35.752 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.riexmm{0:14458} state up:creating seq 1 addr [v2:172.21.15.107:6826/3814968386,v1:172.21.15.107:6827/3814968386] compat {c=[1],r=[1],i=[7ff]}] 2024-03-04T13:38:35.752 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:38:35.752 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:38:35.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:35.760+0000 7fc99b200700 1 -- 172.21.15.107:0/686403479 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fc9780692a0 msgr2=0x7fc97806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:38:35.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:35.760+0000 7fc99b200700 1 --2- 172.21.15.107:0/686403479 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fc9780692a0 0x7fc97806b750 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc980009fd0 tx=0x7fc980009e30 comp rx=0 tx=0).stop 2024-03-04T13:38:35.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:35.760+0000 7fc99b200700 1 -- 172.21.15.107:0/686403479 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc994103540 msgr2=0x7fc994113f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:38:35.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:35.760+0000 7fc99b200700 1 --2- 172.21.15.107:0/686403479 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc994103540 0x7fc994113f60 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc98400ca90 tx=0x7fc984007f00 comp rx=0 tx=0).stop 2024-03-04T13:38:35.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:35.760+0000 7fc99b200700 1 -- 172.21.15.107:0/686403479 shutdown_connections 2024-03-04T13:38:35.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:35.760+0000 7fc99b200700 1 --2- 172.21.15.107:0/686403479 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fc9780692a0 0x7fc97806b750 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:38:35.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:35.760+0000 7fc99b200700 1 --2- 172.21.15.107:0/686403479 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc994103540 0x7fc994113f60 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:38:35.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:35.760+0000 7fc99b200700 1 --2- 172.21.15.107:0/686403479 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc994104750 0x7fc9941144a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:38:35.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:35.760+0000 7fc99b200700 1 -- 172.21.15.107:0/686403479 >> 172.21.15.107:0/686403479 conn(0x7fc9940727d0 msgr2=0x7fc994107980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:38:35.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:35.760+0000 7fc99b200700 1 -- 172.21.15.107:0/686403479 shutdown_connections 2024-03-04T13:38:35.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:35.760+0000 7fc99b200700 1 -- 172.21.15.107:0/686403479 wait complete. 2024-03-04T13:38:35.754 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 3 2024-03-04T13:38:36.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:35 smithi107 ceph-mon[28739]: daemon mds.cephfs.smithi107.riexmm assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-03-04T13:38:36.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:35 smithi107 ceph-mon[28739]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-04T13:38:36.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:35 smithi107 ceph-mon[28739]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-04T13:38:36.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:35 smithi107 ceph-mon[28739]: Cluster is now healthy 2024-03-04T13:38:36.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:35 smithi107 ceph-mon[28739]: mds.? [v2:172.21.15.107:6826/3814968386,v1:172.21.15.107:6827/3814968386] up:boot 2024-03-04T13:38:36.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:35 smithi107 ceph-mon[28739]: fsmap cephfs:1 {0=cephfs.smithi107.riexmm=up:creating} 2024-03-04T13:38:36.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:35 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi107.riexmm"}]: dispatch 2024-03-04T13:38:36.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:35 smithi107 ceph-mon[28739]: daemon mds.cephfs.smithi107.riexmm is now active in filesystem cephfs as rank 0 2024-03-04T13:38:36.088 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:35 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:38:36.088 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:35 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:38:36.088 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:35 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:38:36.088 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:35 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi169.jusudl", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-04T13:38:36.088 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:35 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi169.jusudl", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-04T13:38:36.088 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:35 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:38:36.088 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:35 smithi107 ceph-mon[28739]: from='client.? 172.21.15.107:0/686403479' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-04T13:38:36.353 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:35 smithi169 ceph-mon[36313]: daemon mds.cephfs.smithi107.riexmm assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-03-04T13:38:36.353 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:35 smithi169 ceph-mon[36313]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-04T13:38:36.353 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:35 smithi169 ceph-mon[36313]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-04T13:38:36.354 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:35 smithi169 ceph-mon[36313]: Cluster is now healthy 2024-03-04T13:38:36.354 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:35 smithi169 ceph-mon[36313]: mds.? [v2:172.21.15.107:6826/3814968386,v1:172.21.15.107:6827/3814968386] up:boot 2024-03-04T13:38:36.354 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:35 smithi169 ceph-mon[36313]: fsmap cephfs:1 {0=cephfs.smithi107.riexmm=up:creating} 2024-03-04T13:38:36.354 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:35 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi107.riexmm"}]: dispatch 2024-03-04T13:38:36.354 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:35 smithi169 ceph-mon[36313]: daemon mds.cephfs.smithi107.riexmm is now active in filesystem cephfs as rank 0 2024-03-04T13:38:36.354 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:35 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:38:36.354 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:35 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:38:36.354 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:35 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:38:36.354 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:35 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi169.jusudl", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-04T13:38:36.354 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:35 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi169.jusudl", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-04T13:38:36.354 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:35 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:38:36.354 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:35 smithi169 ceph-mon[36313]: from='client.? 172.21.15.107:0/686403479' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-04T13:38:36.587 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-04T13:38:36.597 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi107.front.sepia.ceph.com 2024-03-04T13:38:36.597 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 -- bash -c 'ceph fs set cephfs max_mds 1' 2024-03-04T13:38:37.061 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/mon.smithi107/config 2024-03-04T13:38:37.096 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:36 smithi169 ceph-mon[36313]: pgmap v218: 65 pgs: 4 creating+peering, 61 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-04T13:38:37.097 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:36 smithi169 ceph-mon[36313]: Deploying daemon mds.cephfs.smithi169.jusudl on smithi169 2024-03-04T13:38:37.097 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:36 smithi169 ceph-mon[36313]: mds.? [v2:172.21.15.107:6826/3814968386,v1:172.21.15.107:6827/3814968386] up:active 2024-03-04T13:38:37.097 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:36 smithi169 ceph-mon[36313]: fsmap cephfs:1 {0=cephfs.smithi107.riexmm=up:active} 2024-03-04T13:38:37.212 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:36 smithi107 ceph-mon[28739]: pgmap v218: 65 pgs: 4 creating+peering, 61 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-04T13:38:37.212 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:36 smithi107 ceph-mon[28739]: Deploying daemon mds.cephfs.smithi169.jusudl on smithi169 2024-03-04T13:38:37.212 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:36 smithi107 ceph-mon[28739]: mds.? [v2:172.21.15.107:6826/3814968386,v1:172.21.15.107:6827/3814968386] up:active 2024-03-04T13:38:37.212 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:36 smithi107 ceph-mon[28739]: fsmap cephfs:1 {0=cephfs.smithi107.riexmm=up:active} 2024-03-04T13:38:39.008 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:38 smithi107 ceph-mon[28739]: pgmap v219: 65 pgs: 65 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s wr, 3 op/s 2024-03-04T13:38:39.103 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:38 smithi169 ceph-mon[36313]: pgmap v219: 65 pgs: 65 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s wr, 3 op/s 2024-03-04T13:38:39.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:39.335+0000 7f6d08e00700 1 -- 172.21.15.107:0/1874974750 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d040fe0e0 msgr2=0x7f6d040fe550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:38:39.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:39.335+0000 7f6d08e00700 1 --2- 172.21.15.107:0/1874974750 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d040fe0e0 0x7f6d040fe550 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6cec004940 tx=0x7f6cec009e30 comp rx=0 tx=0).stop 2024-03-04T13:38:39.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:39.335+0000 7f6d08e00700 1 -- 172.21.15.107:0/1874974750 shutdown_connections 2024-03-04T13:38:39.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:39.335+0000 7f6d08e00700 1 --2- 172.21.15.107:0/1874974750 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d040fe0e0 0x7f6d040fe550 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:38:39.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:39.335+0000 7f6d08e00700 1 --2- 172.21.15.107:0/1874974750 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d040fcef0 0x7f6d040fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:38:39.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:39.335+0000 7f6d08e00700 1 -- 172.21.15.107:0/1874974750 >> 172.21.15.107:0/1874974750 conn(0x7f6d040f8620 msgr2=0x7f6d040faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:38:39.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:39.336+0000 7f6d08e00700 1 -- 172.21.15.107:0/1874974750 shutdown_connections 2024-03-04T13:38:39.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:39.336+0000 7f6d08e00700 1 -- 172.21.15.107:0/1874974750 wait complete. 2024-03-04T13:38:39.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:39.336+0000 7f6d08e00700 1 Processor -- start 2024-03-04T13:38:39.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:39.337+0000 7f6d08e00700 1 -- start start 2024-03-04T13:38:39.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:39.337+0000 7f6d08e00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d040fcef0 0x7f6d041066c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:38:39.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:39.337+0000 7f6d08e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d040fe0e0 0x7f6d04106c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:38:39.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:39.337+0000 7f6d08e00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d04107250 con 0x7f6d040fe0e0 2024-03-04T13:38:39.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:39.337+0000 7f6d08e00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d04107af0 con 0x7f6d040fcef0 2024-03-04T13:38:39.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:39.337+0000 7f6d01400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d040fe0e0 0x7f6d04106c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:38:39.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:39.337+0000 7f6d01400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d040fe0e0 0x7f6d04106c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48780/0 (socket says 172.21.15.107:48780) 2024-03-04T13:38:39.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:39.337+0000 7f6d01400700 1 -- 172.21.15.107:0/3371745252 learned_addr learned my addr 172.21.15.107:0/3371745252 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:38:39.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:39.337+0000 7f6d01e00700 1 --2- 172.21.15.107:0/3371745252 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d040fcef0 0x7f6d041066c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:38:39.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:39.338+0000 7f6d01400700 1 -- 172.21.15.107:0/3371745252 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d040fcef0 msgr2=0x7f6d041066c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:38:39.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:39.338+0000 7f6d01400700 1 --2- 172.21.15.107:0/3371745252 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d040fcef0 0x7f6d041066c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:38:39.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:39.338+0000 7f6d01400700 1 -- 172.21.15.107:0/3371745252 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cec005040 con 0x7f6d040fe0e0 2024-03-04T13:38:39.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:39.338+0000 7f6d01400700 1 --2- 172.21.15.107:0/3371745252 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d040fe0e0 0x7f6d04106c00 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6cec00b3b0 tx=0x7f6cec009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:38:39.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:39.338+0000 7f6cfa000700 1 -- 172.21.15.107:0/3371745252 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6cec00e040 con 0x7f6d040fe0e0 2024-03-04T13:38:39.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:39.338+0000 7f6d08e00700 1 -- 172.21.15.107:0/3371745252 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d04107d70 con 0x7f6d040fe0e0 2024-03-04T13:38:39.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:39.338+0000 7f6cfa000700 1 -- 172.21.15.107:0/3371745252 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6cec00bd90 con 0x7f6d040fe0e0 2024-03-04T13:38:39.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:39.338+0000 7f6cfa000700 1 -- 172.21.15.107:0/3371745252 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6cec01c630 con 0x7f6d040fe0e0 2024-03-04T13:38:39.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:39.338+0000 7f6d08e00700 1 -- 172.21.15.107:0/3371745252 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d041082b0 con 0x7f6d040fe0e0 2024-03-04T13:38:39.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:39.340+0000 7f6d08e00700 1 -- 172.21.15.107:0/3371745252 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d04102210 con 0x7f6d040fe0e0 2024-03-04T13:38:39.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:39.341+0000 7f6cfa000700 1 -- 172.21.15.107:0/3371745252 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86623+0+0 (secure 0 0 0) 0x7f6cec0139f0 con 0x7f6d040fe0e0 2024-03-04T13:38:39.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:39.341+0000 7f6cfa000700 1 --2- 172.21.15.107:0/3371745252 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f6ce80692b0 0x7f6ce806b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:38:39.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:39.342+0000 7f6cfa000700 1 -- 172.21.15.107:0/3371745252 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f6cec0886d0 con 0x7f6d040fe0e0 2024-03-04T13:38:39.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:39.342+0000 7f6d01e00700 1 --2- 172.21.15.107:0/3371745252 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f6ce80692b0 0x7f6ce806b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:38:39.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:39.343+0000 7f6d01e00700 1 --2- 172.21.15.107:0/3371745252 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f6ce80692b0 0x7f6ce806b760 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6cf4009fd0 tx=0x7f6cf4009f00 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:38:39.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:39.347+0000 7f6cfa000700 1 -- 172.21.15.107:0/3371745252 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f6cec054a90 con 0x7f6d040fe0e0 2024-03-04T13:38:39.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:39.604+0000 7f6d08e00700 1 -- 172.21.15.107:0/3371745252 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"} v 0) v1 -- 0x7f6d04105a30 con 0x7f6d040fe0e0 2024-03-04T13:38:40.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:40.075+0000 7f6cfa000700 1 -- 172.21.15.107:0/3371745252 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]=0 v7) v1 ==== 105+0+0 (secure 0 0 0) 0x7f6cec0580b0 con 0x7f6d040fe0e0 2024-03-04T13:38:40.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:40.080+0000 7f6d08e00700 1 -- 172.21.15.107:0/3371745252 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f6ce80692b0 msgr2=0x7f6ce806b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:38:40.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:40.080+0000 7f6d08e00700 1 --2- 172.21.15.107:0/3371745252 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f6ce80692b0 0x7f6ce806b760 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6cf4009fd0 tx=0x7f6cf4009f00 comp rx=0 tx=0).stop 2024-03-04T13:38:40.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:40.080+0000 7f6d08e00700 1 -- 172.21.15.107:0/3371745252 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d040fe0e0 msgr2=0x7f6d04106c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:38:40.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:40.080+0000 7f6d08e00700 1 --2- 172.21.15.107:0/3371745252 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d040fe0e0 0x7f6d04106c00 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6cec00b3b0 tx=0x7f6cec009e30 comp rx=0 tx=0).stop 2024-03-04T13:38:40.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:40.081+0000 7f6d08e00700 1 -- 172.21.15.107:0/3371745252 shutdown_connections 2024-03-04T13:38:40.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:40.081+0000 7f6d08e00700 1 --2- 172.21.15.107:0/3371745252 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f6ce80692b0 0x7f6ce806b760 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:38:40.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:40.081+0000 7f6d08e00700 1 --2- 172.21.15.107:0/3371745252 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d040fcef0 0x7f6d041066c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:38:40.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:40.081+0000 7f6d08e00700 1 --2- 172.21.15.107:0/3371745252 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d040fe0e0 0x7f6d04106c00 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:38:40.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:40.081+0000 7f6d08e00700 1 -- 172.21.15.107:0/3371745252 >> 172.21.15.107:0/3371745252 conn(0x7f6d040f8620 msgr2=0x7f6d040ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:38:40.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:40.081+0000 7f6d08e00700 1 -- 172.21.15.107:0/3371745252 shutdown_connections 2024-03-04T13:38:40.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:40.081+0000 7f6d08e00700 1 -- 172.21.15.107:0/3371745252 wait complete. 2024-03-04T13:38:40.337 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:39 smithi107 ceph-mon[28739]: mds.? [v2:172.21.15.169:6824/2994866076,v1:172.21.15.169:6825/2994866076] up:boot 2024-03-04T13:38:40.337 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:39 smithi107 ceph-mon[28739]: fsmap cephfs:1 {0=cephfs.smithi107.riexmm=up:active} 1 up:standby 2024-03-04T13:38:40.337 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:39 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi169.jusudl"}]: dispatch 2024-03-04T13:38:40.337 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:39 smithi107 ceph-mon[28739]: fsmap cephfs:1 {0=cephfs.smithi107.riexmm=up:active} 1 up:standby 2024-03-04T13:38:40.337 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:39 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:38:40.337 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:39 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:38:40.337 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:39 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:38:40.337 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:39 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi107.thycho", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-04T13:38:40.337 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:39 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi107.thycho", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-04T13:38:40.337 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:39 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:38:40.337 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:39 smithi107 ceph-mon[28739]: from='client.? 172.21.15.107:0/3371745252' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]: dispatch 2024-03-04T13:38:40.353 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:39 smithi169 ceph-mon[36313]: mds.? [v2:172.21.15.169:6824/2994866076,v1:172.21.15.169:6825/2994866076] up:boot 2024-03-04T13:38:40.354 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:39 smithi169 ceph-mon[36313]: fsmap cephfs:1 {0=cephfs.smithi107.riexmm=up:active} 1 up:standby 2024-03-04T13:38:40.354 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:39 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi169.jusudl"}]: dispatch 2024-03-04T13:38:40.354 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:39 smithi169 ceph-mon[36313]: fsmap cephfs:1 {0=cephfs.smithi107.riexmm=up:active} 1 up:standby 2024-03-04T13:38:40.354 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:39 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:38:40.354 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:39 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:38:40.354 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:39 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:38:40.354 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:39 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi107.thycho", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-04T13:38:40.354 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:39 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi107.thycho", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-04T13:38:40.354 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:39 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:38:40.354 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:39 smithi169 ceph-mon[36313]: from='client.? 172.21.15.107:0/3371745252' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]: dispatch 2024-03-04T13:38:41.086 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:40 smithi107 ceph-mon[28739]: Deploying daemon mds.cephfs.smithi107.thycho on smithi107 2024-03-04T13:38:41.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:40 smithi107 ceph-mon[28739]: pgmap v220: 65 pgs: 65 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1023 B/s wr, 3 op/s 2024-03-04T13:38:41.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:40 smithi107 ceph-mon[28739]: from='client.? 172.21.15.107:0/3371745252' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]': finished 2024-03-04T13:38:41.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:40 smithi107 ceph-mon[28739]: fsmap cephfs:1 {0=cephfs.smithi107.riexmm=up:active} 1 up:standby 2024-03-04T13:38:41.353 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:40 smithi169 ceph-mon[36313]: Deploying daemon mds.cephfs.smithi107.thycho on smithi107 2024-03-04T13:38:41.353 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:40 smithi169 ceph-mon[36313]: pgmap v220: 65 pgs: 65 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1023 B/s wr, 3 op/s 2024-03-04T13:38:41.353 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:40 smithi169 ceph-mon[36313]: from='client.? 172.21.15.107:0/3371745252' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]': finished 2024-03-04T13:38:41.353 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:40 smithi169 ceph-mon[36313]: fsmap cephfs:1 {0=cephfs.smithi107.riexmm=up:active} 1 up:standby 2024-03-04T13:38:41.839 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-04T13:38:41.852 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi107.front.sepia.ceph.com 2024-03-04T13:38:41.852 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 -- bash -c 'ceph fs set cephfs allow_standby_replay true' 2024-03-04T13:38:42.904 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/mon.smithi107/config 2024-03-04T13:38:43.218 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:42 smithi107 ceph-mon[28739]: pgmap v221: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.4 KiB/s wr, 4 op/s 2024-03-04T13:38:43.219 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:42 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:38:43.219 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:42 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:38:43.219 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:42 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:38:43.219 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:42 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:38:43.219 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:42 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:38:43.219 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:42 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi169.bxitzk", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-04T13:38:43.219 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:42 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi169.bxitzk", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-04T13:38:43.219 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:42 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:38:43.353 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:42 smithi169 ceph-mon[36313]: pgmap v221: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.4 KiB/s wr, 4 op/s 2024-03-04T13:38:43.353 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:42 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:38:43.353 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:42 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:38:43.353 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:42 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:38:43.353 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:42 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:38:43.353 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:42 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:38:43.353 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:42 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi169.bxitzk", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-04T13:38:43.353 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:42 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi169.bxitzk", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-04T13:38:43.354 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:42 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:38:44.309 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:43 smithi107 ceph-mon[28739]: Deploying daemon mds.cephfs.smithi169.bxitzk on smithi169 2024-03-04T13:38:44.309 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:43 smithi107 ceph-mon[28739]: mds.? [v2:172.21.15.107:6828/1985577060,v1:172.21.15.107:6829/1985577060] up:boot 2024-03-04T13:38:44.309 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:43 smithi107 ceph-mon[28739]: fsmap cephfs:1 {0=cephfs.smithi107.riexmm=up:active} 2 up:standby 2024-03-04T13:38:44.310 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:43 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi107.thycho"}]: dispatch 2024-03-04T13:38:44.353 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:43 smithi169 ceph-mon[36313]: Deploying daemon mds.cephfs.smithi169.bxitzk on smithi169 2024-03-04T13:38:44.353 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:43 smithi169 ceph-mon[36313]: mds.? [v2:172.21.15.107:6828/1985577060,v1:172.21.15.107:6829/1985577060] up:boot 2024-03-04T13:38:44.353 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:43 smithi169 ceph-mon[36313]: fsmap cephfs:1 {0=cephfs.smithi107.riexmm=up:active} 2 up:standby 2024-03-04T13:38:44.354 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:43 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi107.thycho"}]: dispatch 2024-03-04T13:38:45.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:45.070+0000 7f4b97a00700 1 -- 172.21.15.107:0/3041726509 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b900708c0 msgr2=0x7f4b90070d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:38:45.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:45.070+0000 7f4b97a00700 1 --2- 172.21.15.107:0/3041726509 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b900708c0 0x7f4b90070d10 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c004940 tx=0x7f4b7c009e30 comp rx=0 tx=0).stop 2024-03-04T13:38:45.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:45.070+0000 7f4b97a00700 1 -- 172.21.15.107:0/3041726509 shutdown_connections 2024-03-04T13:38:45.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:45.070+0000 7f4b97a00700 1 --2- 172.21.15.107:0/3041726509 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b900708c0 0x7f4b90070d10 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:38:45.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:45.070+0000 7f4b97a00700 1 --2- 172.21.15.107:0/3041726509 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b90071f10 0x7f4b90070380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:38:45.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:45.070+0000 7f4b97a00700 1 -- 172.21.15.107:0/3041726509 >> 172.21.15.107:0/3041726509 conn(0x7f4b900f8620 msgr2=0x7f4b900faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:38:45.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:45.070+0000 7f4b97a00700 1 -- 172.21.15.107:0/3041726509 shutdown_connections 2024-03-04T13:38:45.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:45.070+0000 7f4b97a00700 1 -- 172.21.15.107:0/3041726509 wait complete. 2024-03-04T13:38:45.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:45.071+0000 7f4b97a00700 1 Processor -- start 2024-03-04T13:38:45.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:45.071+0000 7f4b97a00700 1 -- start start 2024-03-04T13:38:45.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:45.072+0000 7f4b97a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b900708c0 0x7f4b9010fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:38:45.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:45.072+0000 7f4b97a00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b90071f10 0x7f4b901100b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:38:45.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:45.072+0000 7f4b97a00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b901106b0 con 0x7f4b900708c0 2024-03-04T13:38:45.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:45.072+0000 7f4b97a00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b90110f50 con 0x7f4b90071f10 2024-03-04T13:38:45.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:45.072+0000 7f4b95000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b900708c0 0x7f4b9010fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:38:45.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:45.072+0000 7f4b95000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b900708c0 0x7f4b9010fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48814/0 (socket says 172.21.15.107:48814) 2024-03-04T13:38:45.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:45.072+0000 7f4b95000700 1 -- 172.21.15.107:0/3806271929 learned_addr learned my addr 172.21.15.107:0/3806271929 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:38:45.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:45.072+0000 7f4b87e00700 1 --2- 172.21.15.107:0/3806271929 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b90071f10 0x7f4b901100b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:38:45.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:45.072+0000 7f4b95000700 1 -- 172.21.15.107:0/3806271929 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b90071f10 msgr2=0x7f4b901100b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:38:45.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:45.072+0000 7f4b95000700 1 --2- 172.21.15.107:0/3806271929 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b90071f10 0x7f4b901100b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:38:45.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:45.072+0000 7f4b95000700 1 -- 172.21.15.107:0/3806271929 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b7c005040 con 0x7f4b900708c0 2024-03-04T13:38:45.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:45.073+0000 7f4b95000700 1 --2- 172.21.15.107:0/3806271929 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b900708c0 0x7f4b9010fb70 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f4b8000cb10 tx=0x7f4b80007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:38:45.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:45.073+0000 7f4b8d600700 1 -- 172.21.15.107:0/3806271929 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b80004200 con 0x7f4b900708c0 2024-03-04T13:38:45.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:45.073+0000 7f4b8d600700 1 -- 172.21.15.107:0/3806271929 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4b8000acc0 con 0x7f4b900708c0 2024-03-04T13:38:45.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:45.073+0000 7f4b8d600700 1 -- 172.21.15.107:0/3806271929 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b80003d90 con 0x7f4b900708c0 2024-03-04T13:38:45.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:45.073+0000 7f4b97a00700 1 -- 172.21.15.107:0/3806271929 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b90111230 con 0x7f4b900708c0 2024-03-04T13:38:45.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:45.073+0000 7f4b97a00700 1 -- 172.21.15.107:0/3806271929 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b90111770 con 0x7f4b900708c0 2024-03-04T13:38:45.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:45.075+0000 7f4b97a00700 1 -- 172.21.15.107:0/3806271929 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b9004bed0 con 0x7f4b900708c0 2024-03-04T13:38:45.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:45.076+0000 7f4b8d600700 1 -- 172.21.15.107:0/3806271929 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86623+0+0 (secure 0 0 0) 0x7f4b80003430 con 0x7f4b900708c0 2024-03-04T13:38:45.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:45.077+0000 7f4b8d600700 1 --2- 172.21.15.107:0/3806271929 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f4b740692b0 0x7f4b7406b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:38:45.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:45.077+0000 7f4b8d600700 1 -- 172.21.15.107:0/3806271929 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f4b80087720 con 0x7f4b900708c0 2024-03-04T13:38:45.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:45.077+0000 7f4b87e00700 1 --2- 172.21.15.107:0/3806271929 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f4b740692b0 0x7f4b7406b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:38:45.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:45.078+0000 7f4b87e00700 1 --2- 172.21.15.107:0/3806271929 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f4b740692b0 0x7f4b7406b760 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c004f90 tx=0x7f4b7c009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:38:45.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:45.082+0000 7f4b8d600700 1 -- 172.21.15.107:0/3806271929 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f4b800574e0 con 0x7f4b900708c0 2024-03-04T13:38:45.103 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:44 smithi169 ceph-mon[36313]: pgmap v222: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s wr, 4 op/s 2024-03-04T13:38:45.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:45.335+0000 7f4b97a00700 1 -- 172.21.15.107:0/3806271929 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7f4b90111c00 con 0x7f4b900708c0 2024-03-04T13:38:45.329 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:44 smithi107 ceph-mon[28739]: pgmap v222: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s wr, 4 op/s 2024-03-04T13:38:45.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:45.591+0000 7f4b8d600700 1 -- 172.21.15.107:0/3806271929 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v9) v1 ==== 121+0+0 (secure 0 0 0) 0x7f4b80057070 con 0x7f4b900708c0 2024-03-04T13:38:45.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:45.596+0000 7f4b97a00700 1 -- 172.21.15.107:0/3806271929 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f4b740692b0 msgr2=0x7f4b7406b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:38:45.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:45.596+0000 7f4b97a00700 1 --2- 172.21.15.107:0/3806271929 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f4b740692b0 0x7f4b7406b760 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c004f90 tx=0x7f4b7c009e30 comp rx=0 tx=0).stop 2024-03-04T13:38:45.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:45.596+0000 7f4b97a00700 1 -- 172.21.15.107:0/3806271929 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b900708c0 msgr2=0x7f4b9010fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:38:45.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:45.596+0000 7f4b97a00700 1 --2- 172.21.15.107:0/3806271929 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b900708c0 0x7f4b9010fb70 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f4b8000cb10 tx=0x7f4b80007f90 comp rx=0 tx=0).stop 2024-03-04T13:38:45.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:45.596+0000 7f4b97a00700 1 -- 172.21.15.107:0/3806271929 shutdown_connections 2024-03-04T13:38:45.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:45.596+0000 7f4b97a00700 1 --2- 172.21.15.107:0/3806271929 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f4b740692b0 0x7f4b7406b760 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:38:45.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:45.596+0000 7f4b97a00700 1 --2- 172.21.15.107:0/3806271929 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b900708c0 0x7f4b9010fb70 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:38:45.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:45.596+0000 7f4b97a00700 1 --2- 172.21.15.107:0/3806271929 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b90071f10 0x7f4b901100b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:38:45.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:45.596+0000 7f4b97a00700 1 -- 172.21.15.107:0/3806271929 >> 172.21.15.107:0/3806271929 conn(0x7f4b900f8620 msgr2=0x7f4b90102dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:38:45.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:45.596+0000 7f4b97a00700 1 -- 172.21.15.107:0/3806271929 shutdown_connections 2024-03-04T13:38:45.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:45.597+0000 7f4b97a00700 1 -- 172.21.15.107:0/3806271929 wait complete. 2024-03-04T13:38:46.131 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:45 smithi169 ceph-mon[36313]: from='client.? 172.21.15.107:0/3806271929' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-04T13:38:46.131 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:45 smithi169 ceph-mon[36313]: from='client.? 172.21.15.107:0/3806271929' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-04T13:38:46.131 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:45 smithi169 ceph-mon[36313]: fsmap cephfs:1 {0=cephfs.smithi107.riexmm=up:active} 1 up:standby-replay 1 up:standby 2024-03-04T13:38:46.336 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:45 smithi107 ceph-mon[28739]: from='client.? 172.21.15.107:0/3806271929' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-04T13:38:46.337 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:45 smithi107 ceph-mon[28739]: from='client.? 172.21.15.107:0/3806271929' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-04T13:38:46.337 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:45 smithi107 ceph-mon[28739]: fsmap cephfs:1 {0=cephfs.smithi107.riexmm=up:active} 1 up:standby-replay 1 up:standby 2024-03-04T13:38:46.589 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-04T13:38:46.598 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi107.front.sepia.ceph.com 2024-03-04T13:38:46.598 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 -- bash -c 'ceph fs set cephfs inline_data true --yes-i-really-really-mean-it' 2024-03-04T13:38:47.085 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/mon.smithi107/config 2024-03-04T13:38:47.229 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:46 smithi107 ceph-mon[28739]: pgmap v223: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s wr, 3 op/s 2024-03-04T13:38:47.230 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:46 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:38:47.230 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:46 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:38:47.230 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:46 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:38:47.230 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:46 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:38:47.230 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:46 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:38:47.230 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:46 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:38:47.353 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:46 smithi169 ceph-mon[36313]: pgmap v223: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s wr, 3 op/s 2024-03-04T13:38:47.353 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:46 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:38:47.353 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:46 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:38:47.353 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:46 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:38:47.353 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:46 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:38:47.353 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:46 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:38:47.353 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:46 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:38:48.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:47 smithi169 ceph-mon[36313]: mds.? [v2:172.21.15.169:6826/3696090493,v1:172.21.15.169:6827/3696090493] up:boot 2024-03-04T13:38:48.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:47 smithi169 ceph-mon[36313]: mds.? [v2:172.21.15.107:6828/1985577060,v1:172.21.15.107:6829/1985577060] up:standby 2024-03-04T13:38:48.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:47 smithi169 ceph-mon[36313]: mds.? [v2:172.21.15.169:6824/2994866076,v1:172.21.15.169:6825/2994866076] up:standby-replay 2024-03-04T13:38:48.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:47 smithi169 ceph-mon[36313]: Dropping low affinity active daemon mds.cephfs.smithi107.riexmm in favor of higher affinity standby. 2024-03-04T13:38:48.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:47 smithi169 ceph-mon[36313]: Replacing daemon mds.cephfs.smithi107.riexmm as rank 0 with standby daemon mds.cephfs.smithi169.jusudl 2024-03-04T13:38:48.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:47 smithi169 ceph-mon[36313]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-04T13:38:48.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:47 smithi169 ceph-mon[36313]: fsmap cephfs:1 {0=cephfs.smithi107.riexmm=up:active} 1 up:standby-replay 2 up:standby 2024-03-04T13:38:48.105 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:47 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi169.bxitzk"}]: dispatch 2024-03-04T13:38:48.105 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:47 smithi169 ceph-mon[36313]: osdmap e47: 6 total, 6 up, 6 in 2024-03-04T13:38:48.105 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:47 smithi169 ceph-mon[36313]: fsmap cephfs:1/1 {0=cephfs.smithi169.jusudl=up:replay} 2 up:standby 2024-03-04T13:38:48.105 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:47 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:38:48.248 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:47 smithi107 ceph-mon[28739]: mds.? [v2:172.21.15.169:6826/3696090493,v1:172.21.15.169:6827/3696090493] up:boot 2024-03-04T13:38:48.248 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:47 smithi107 ceph-mon[28739]: mds.? [v2:172.21.15.107:6828/1985577060,v1:172.21.15.107:6829/1985577060] up:standby 2024-03-04T13:38:48.248 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:47 smithi107 ceph-mon[28739]: mds.? [v2:172.21.15.169:6824/2994866076,v1:172.21.15.169:6825/2994866076] up:standby-replay 2024-03-04T13:38:48.248 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:47 smithi107 ceph-mon[28739]: Dropping low affinity active daemon mds.cephfs.smithi107.riexmm in favor of higher affinity standby. 2024-03-04T13:38:48.248 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:47 smithi107 ceph-mon[28739]: Replacing daemon mds.cephfs.smithi107.riexmm as rank 0 with standby daemon mds.cephfs.smithi169.jusudl 2024-03-04T13:38:48.248 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:47 smithi107 ceph-mon[28739]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-04T13:38:48.248 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:47 smithi107 ceph-mon[28739]: fsmap cephfs:1 {0=cephfs.smithi107.riexmm=up:active} 1 up:standby-replay 2 up:standby 2024-03-04T13:38:48.248 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:47 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi169.bxitzk"}]: dispatch 2024-03-04T13:38:48.248 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:47 smithi107 ceph-mon[28739]: osdmap e47: 6 total, 6 up, 6 in 2024-03-04T13:38:48.249 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:47 smithi107 ceph-mon[28739]: fsmap cephfs:1/1 {0=cephfs.smithi169.jusudl=up:replay} 2 up:standby 2024-03-04T13:38:48.249 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:47 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:38:49.336 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:49 smithi107 ceph-mon[28739]: pgmap v225: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 204 B/s rd, 614 B/s wr, 2 op/s 2024-03-04T13:38:49.336 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:49 smithi107 ceph-mon[28739]: mds.? [v2:172.21.15.107:6826/3261622517,v1:172.21.15.107:6827/3261622517] up:boot 2024-03-04T13:38:49.336 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:49 smithi107 ceph-mon[28739]: mds.? [v2:172.21.15.169:6824/2994866076,v1:172.21.15.169:6825/2994866076] up:reconnect 2024-03-04T13:38:49.336 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:49 smithi107 ceph-mon[28739]: fsmap cephfs:1/1 {0=cephfs.smithi169.jusudl=up:reconnect} 3 up:standby 2024-03-04T13:38:49.337 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:49 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi107.riexmm"}]: dispatch 2024-03-04T13:38:49.353 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:49 smithi169 ceph-mon[36313]: pgmap v225: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 204 B/s rd, 614 B/s wr, 2 op/s 2024-03-04T13:38:49.353 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:49 smithi169 ceph-mon[36313]: mds.? [v2:172.21.15.107:6826/3261622517,v1:172.21.15.107:6827/3261622517] up:boot 2024-03-04T13:38:49.353 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:49 smithi169 ceph-mon[36313]: mds.? [v2:172.21.15.169:6824/2994866076,v1:172.21.15.169:6825/2994866076] up:reconnect 2024-03-04T13:38:49.353 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:49 smithi169 ceph-mon[36313]: fsmap cephfs:1/1 {0=cephfs.smithi169.jusudl=up:reconnect} 3 up:standby 2024-03-04T13:38:49.353 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:49 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi107.riexmm"}]: dispatch 2024-03-04T13:38:49.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:49.357+0000 7f0b73000700 1 -- 172.21.15.107:0/288796544 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b6c103540 msgr2=0x7f0b6c103950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:38:49.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:49.357+0000 7f0b73000700 1 --2- 172.21.15.107:0/288796544 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b6c103540 0x7f0b6c103950 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c004940 tx=0x7f0b5c009e30 comp rx=0 tx=0).stop 2024-03-04T13:38:49.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:49.358+0000 7f0b73000700 1 -- 172.21.15.107:0/288796544 shutdown_connections 2024-03-04T13:38:49.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:49.358+0000 7f0b73000700 1 --2- 172.21.15.107:0/288796544 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0b6c104750 0x7f0b6c104ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:38:49.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:49.358+0000 7f0b73000700 1 --2- 172.21.15.107:0/288796544 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b6c103540 0x7f0b6c103950 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:38:49.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:49.358+0000 7f0b73000700 1 -- 172.21.15.107:0/288796544 >> 172.21.15.107:0/288796544 conn(0x7f0b6c0727d0 msgr2=0x7f0b6c074c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:38:49.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:49.358+0000 7f0b73000700 1 -- 172.21.15.107:0/288796544 shutdown_connections 2024-03-04T13:38:49.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:49.358+0000 7f0b73000700 1 -- 172.21.15.107:0/288796544 wait complete. 2024-03-04T13:38:49.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:49.358+0000 7f0b73000700 1 Processor -- start 2024-03-04T13:38:49.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:49.359+0000 7f0b73000700 1 -- start start 2024-03-04T13:38:49.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:49.359+0000 7f0b73000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b6c103540 0x7f0b6c113ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:38:49.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:49.359+0000 7f0b73000700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0b6c104750 0x7f0b6c114420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:38:49.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:49.359+0000 7f0b73000700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b6c114a20 con 0x7f0b6c103540 2024-03-04T13:38:49.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:49.359+0000 7f0b73000700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b6c1152c0 con 0x7f0b6c104750 2024-03-04T13:38:49.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:49.359+0000 7f0b6be00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b6c103540 0x7f0b6c113ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:38:49.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:49.359+0000 7f0b6be00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b6c103540 0x7f0b6c113ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48560/0 (socket says 172.21.15.107:48560) 2024-03-04T13:38:49.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:49.359+0000 7f0b6be00700 1 -- 172.21.15.107:0/786566459 learned_addr learned my addr 172.21.15.107:0/786566459 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:38:49.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:49.359+0000 7f0b6b400700 1 --2- 172.21.15.107:0/786566459 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0b6c104750 0x7f0b6c114420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:38:49.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:49.360+0000 7f0b6be00700 1 -- 172.21.15.107:0/786566459 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0b6c104750 msgr2=0x7f0b6c114420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:38:49.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:49.360+0000 7f0b6be00700 1 --2- 172.21.15.107:0/786566459 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0b6c104750 0x7f0b6c114420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:38:49.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:49.360+0000 7f0b6be00700 1 -- 172.21.15.107:0/786566459 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b5c005040 con 0x7f0b6c103540 2024-03-04T13:38:49.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:49.360+0000 7f0b6be00700 1 --2- 172.21.15.107:0/786566459 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b6c103540 0x7f0b6c113ee0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c009040 tx=0x7f0b5c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:38:49.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:49.360+0000 7f0b63e00700 1 -- 172.21.15.107:0/786566459 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0b5c021070 con 0x7f0b6c103540 2024-03-04T13:38:49.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:49.360+0000 7f0b63e00700 1 -- 172.21.15.107:0/786566459 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0b5c009430 con 0x7f0b6c103540 2024-03-04T13:38:49.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:49.360+0000 7f0b73000700 1 -- 172.21.15.107:0/786566459 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b6c115540 con 0x7f0b6c103540 2024-03-04T13:38:49.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:49.360+0000 7f0b63e00700 1 -- 172.21.15.107:0/786566459 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0b5c00e070 con 0x7f0b6c103540 2024-03-04T13:38:49.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:49.360+0000 7f0b73000700 1 -- 172.21.15.107:0/786566459 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b6c115a80 con 0x7f0b6c103540 2024-03-04T13:38:49.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:49.361+0000 7f0b73000700 1 -- 172.21.15.107:0/786566459 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b6c04bed0 con 0x7f0b6c103540 2024-03-04T13:38:49.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:49.362+0000 7f0b63e00700 1 -- 172.21.15.107:0/786566459 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86623+0+0 (secure 0 0 0) 0x7f0b5c01c400 con 0x7f0b6c103540 2024-03-04T13:38:49.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:49.364+0000 7f0b63e00700 1 --2- 172.21.15.107:0/786566459 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f0b500692b0 0x7f0b5006b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:38:49.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:49.364+0000 7f0b63e00700 1 -- 172.21.15.107:0/786566459 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f0b5c088400 con 0x7f0b6c103540 2024-03-04T13:38:49.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:49.364+0000 7f0b6b400700 1 --2- 172.21.15.107:0/786566459 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f0b500692b0 0x7f0b5006b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:38:49.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:49.365+0000 7f0b6b400700 1 --2- 172.21.15.107:0/786566459 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f0b500692b0 0x7f0b5006b760 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0b54009b80 tx=0x7f0b5400b040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:38:49.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:49.368+0000 7f0b63e00700 1 -- 172.21.15.107:0/786566459 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f0b5c058170 con 0x7f0b6c103540 2024-03-04T13:38:49.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:49.627+0000 7f0b73000700 1 -- 172.21.15.107:0/786566459 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0b6c115ee0 con 0x7f0b6c103540 2024-03-04T13:38:50.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:50.014+0000 7f0b63e00700 1 -- 172.21.15.107:0/786566459 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]=0 inline data enabled v14) v1 ==== 168+0+0 (secure 0 0 0) 0x7f0b5c004490 con 0x7f0b6c103540 2024-03-04T13:38:50.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:50.019+0000 7f0b73000700 1 -- 172.21.15.107:0/786566459 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f0b500692b0 msgr2=0x7f0b5006b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:38:50.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:50.019+0000 7f0b73000700 1 --2- 172.21.15.107:0/786566459 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f0b500692b0 0x7f0b5006b760 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0b54009b80 tx=0x7f0b5400b040 comp rx=0 tx=0).stop 2024-03-04T13:38:50.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:50.019+0000 7f0b73000700 1 -- 172.21.15.107:0/786566459 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b6c103540 msgr2=0x7f0b6c113ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:38:50.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:50.019+0000 7f0b73000700 1 --2- 172.21.15.107:0/786566459 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b6c103540 0x7f0b6c113ee0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c009040 tx=0x7f0b5c009e30 comp rx=0 tx=0).stop 2024-03-04T13:38:50.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:50.019+0000 7f0b73000700 1 -- 172.21.15.107:0/786566459 shutdown_connections 2024-03-04T13:38:50.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:50.019+0000 7f0b73000700 1 --2- 172.21.15.107:0/786566459 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f0b500692b0 0x7f0b5006b760 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:38:50.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:50.019+0000 7f0b73000700 1 --2- 172.21.15.107:0/786566459 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b6c103540 0x7f0b6c113ee0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:38:50.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:50.019+0000 7f0b73000700 1 --2- 172.21.15.107:0/786566459 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0b6c104750 0x7f0b6c114420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:38:50.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:50.019+0000 7f0b73000700 1 -- 172.21.15.107:0/786566459 >> 172.21.15.107:0/786566459 conn(0x7f0b6c0727d0 msgr2=0x7f0b6c107980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:38:50.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:50.020+0000 7f0b73000700 1 -- 172.21.15.107:0/786566459 shutdown_connections 2024-03-04T13:38:50.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:50.020+0000 7f0b73000700 1 -- 172.21.15.107:0/786566459 wait complete. 2024-03-04T13:38:50.015 INFO:teuthology.orchestra.run.smithi107.stderr:inline data enabled 2024-03-04T13:38:50.336 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:50 smithi107 ceph-mon[28739]: mds.? [v2:172.21.15.169:6824/2994866076,v1:172.21.15.169:6825/2994866076] up:rejoin 2024-03-04T13:38:50.336 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:50 smithi107 ceph-mon[28739]: fsmap cephfs:1/1 {0=cephfs.smithi169.jusudl=up:rejoin} 3 up:standby 2024-03-04T13:38:50.336 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:50 smithi107 ceph-mon[28739]: daemon mds.cephfs.smithi169.jusudl is now active in filesystem cephfs as rank 0 2024-03-04T13:38:50.336 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:50 smithi107 ceph-mon[28739]: from='client.? 172.21.15.107:0/786566459' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-04T13:38:50.336 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:50 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:38:50.337 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:50 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:38:50.337 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:50 smithi107 ceph-mon[28739]: Health check failed: 1 filesystem with deprecated feature inline_data (FS_INLINE_DATA_DEPRECATED) 2024-03-04T13:38:50.337 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:50 smithi107 ceph-mon[28739]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-04T13:38:50.337 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:50 smithi107 ceph-mon[28739]: mds.? [v2:172.21.15.169:6824/2994866076,v1:172.21.15.169:6825/2994866076] up:active 2024-03-04T13:38:50.337 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:50 smithi107 ceph-mon[28739]: from='client.? 172.21.15.107:0/786566459' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]': finished 2024-03-04T13:38:50.337 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:50 smithi107 ceph-mon[28739]: fsmap cephfs:1 {0=cephfs.smithi169.jusudl=up:active} 3 up:standby 2024-03-04T13:38:50.353 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:50 smithi169 ceph-mon[36313]: mds.? [v2:172.21.15.169:6824/2994866076,v1:172.21.15.169:6825/2994866076] up:rejoin 2024-03-04T13:38:50.353 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:50 smithi169 ceph-mon[36313]: fsmap cephfs:1/1 {0=cephfs.smithi169.jusudl=up:rejoin} 3 up:standby 2024-03-04T13:38:50.354 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:50 smithi169 ceph-mon[36313]: daemon mds.cephfs.smithi169.jusudl is now active in filesystem cephfs as rank 0 2024-03-04T13:38:50.354 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:50 smithi169 ceph-mon[36313]: from='client.? 172.21.15.107:0/786566459' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-04T13:38:50.354 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:50 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:38:50.354 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:50 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:38:50.354 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:50 smithi169 ceph-mon[36313]: Health check failed: 1 filesystem with deprecated feature inline_data (FS_INLINE_DATA_DEPRECATED) 2024-03-04T13:38:50.354 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:50 smithi169 ceph-mon[36313]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-04T13:38:50.354 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:50 smithi169 ceph-mon[36313]: mds.? [v2:172.21.15.169:6824/2994866076,v1:172.21.15.169:6825/2994866076] up:active 2024-03-04T13:38:50.354 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:50 smithi169 ceph-mon[36313]: from='client.? 172.21.15.107:0/786566459' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]': finished 2024-03-04T13:38:50.354 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:50 smithi169 ceph-mon[36313]: fsmap cephfs:1 {0=cephfs.smithi169.jusudl=up:active} 3 up:standby 2024-03-04T13:38:50.787 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-04T13:38:50.798 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi107.front.sepia.ceph.com 2024-03-04T13:38:50.798 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 -- bash -c 'ceph fs dump' 2024-03-04T13:38:51.337 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:51 smithi107 ceph-mon[28739]: pgmap v226: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 307 B/s rd, 511 B/s wr, 2 op/s 2024-03-04T13:38:51.337 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:51 smithi107 ceph-mon[28739]: fsmap cephfs:1 {0=cephfs.smithi169.jusudl=up:active} 1 up:standby-replay 2 up:standby 2024-03-04T13:38:51.353 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:51 smithi169 ceph-mon[36313]: pgmap v226: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 307 B/s rd, 511 B/s wr, 2 op/s 2024-03-04T13:38:51.353 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:51 smithi169 ceph-mon[36313]: fsmap cephfs:1 {0=cephfs.smithi169.jusudl=up:active} 1 up:standby-replay 2 up:standby 2024-03-04T13:38:51.618 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/mon.smithi107/config 2024-03-04T13:38:52.336 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:52 smithi107 ceph-mon[28739]: mds.? [v2:172.21.15.169:6826/3696090493,v1:172.21.15.169:6827/3696090493] up:standby 2024-03-04T13:38:52.336 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:52 smithi107 ceph-mon[28739]: fsmap cephfs:1 {0=cephfs.smithi169.jusudl=up:active} 1 up:standby-replay 2 up:standby 2024-03-04T13:38:52.336 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:52 smithi107 ceph-mon[28739]: pgmap v227: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-04T13:38:52.354 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:52 smithi169 ceph-mon[36313]: mds.? [v2:172.21.15.169:6826/3696090493,v1:172.21.15.169:6827/3696090493] up:standby 2024-03-04T13:38:52.354 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:52 smithi169 ceph-mon[36313]: fsmap cephfs:1 {0=cephfs.smithi169.jusudl=up:active} 1 up:standby-replay 2 up:standby 2024-03-04T13:38:52.354 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:52 smithi169 ceph-mon[36313]: pgmap v227: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-04T13:38:54.770 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:54 smithi107 ceph-mon[28739]: pgmap v228: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 4.5 KiB/s rd, 0 B/s wr, 4 op/s 2024-03-04T13:38:54.853 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:54 smithi169 ceph-mon[36313]: pgmap v228: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 4.5 KiB/s rd, 0 B/s wr, 4 op/s 2024-03-04T13:38:54.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:54.775+0000 7ff8b9600700 1 -- 172.21.15.107:0/809015606 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8b40fdf50 msgr2=0x7ff8b40fe3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:38:54.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:54.775+0000 7ff8b9600700 1 --2- 172.21.15.107:0/809015606 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8b40fdf50 0x7ff8b40fe3a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff89c004940 tx=0x7ff89c009e30 comp rx=0 tx=0).stop 2024-03-04T13:38:54.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:54.775+0000 7ff8b9600700 1 -- 172.21.15.107:0/809015606 shutdown_connections 2024-03-04T13:38:54.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:54.775+0000 7ff8b9600700 1 --2- 172.21.15.107:0/809015606 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8b40fdf50 0x7ff8b40fe3a0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:38:54.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:54.775+0000 7ff8b9600700 1 --2- 172.21.15.107:0/809015606 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff8b40fcd40 0x7ff8b40fd150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:38:54.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:54.775+0000 7ff8b9600700 1 -- 172.21.15.107:0/809015606 >> 172.21.15.107:0/809015606 conn(0x7ff8b40f83e0 msgr2=0x7ff8b40fa830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:38:54.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:54.775+0000 7ff8b9600700 1 -- 172.21.15.107:0/809015606 shutdown_connections 2024-03-04T13:38:54.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:54.775+0000 7ff8b9600700 1 -- 172.21.15.107:0/809015606 wait complete. 2024-03-04T13:38:54.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:54.776+0000 7ff8b9600700 1 Processor -- start 2024-03-04T13:38:54.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:54.777+0000 7ff8b9600700 1 -- start start 2024-03-04T13:38:54.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:54.777+0000 7ff8b9600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8b40fcd40 0x7ff8b410f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:38:54.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:54.777+0000 7ff8b9600700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff8b40fdf50 0x7ff8b410fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:38:54.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:54.777+0000 7ff8b9600700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8b4110470 con 0x7ff8b40fcd40 2024-03-04T13:38:54.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:54.777+0000 7ff8b9600700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8b4110d10 con 0x7ff8b40fdf50 2024-03-04T13:38:54.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:54.777+0000 7ff8b3e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8b40fcd40 0x7ff8b410f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:38:54.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:54.777+0000 7ff8b3e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8b40fcd40 0x7ff8b410f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48586/0 (socket says 172.21.15.107:48586) 2024-03-04T13:38:54.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:54.777+0000 7ff8b3e00700 1 -- 172.21.15.107:0/260338194 learned_addr learned my addr 172.21.15.107:0/260338194 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:38:54.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:54.778+0000 7ff8ab400700 1 --2- 172.21.15.107:0/260338194 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff8b40fdf50 0x7ff8b410fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:38:54.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:54.778+0000 7ff8b3e00700 1 -- 172.21.15.107:0/260338194 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff8b40fdf50 msgr2=0x7ff8b410fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:38:54.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:54.778+0000 7ff8b3e00700 1 --2- 172.21.15.107:0/260338194 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff8b40fdf50 0x7ff8b410fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:38:54.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:54.778+0000 7ff8b3e00700 1 -- 172.21.15.107:0/260338194 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff89c005040 con 0x7ff8b40fcd40 2024-03-04T13:38:54.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:54.778+0000 7ff8b3e00700 1 --2- 172.21.15.107:0/260338194 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8b40fcd40 0x7ff8b410f930 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff8a4001ce0 tx=0x7ff8a4009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:38:54.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:54.778+0000 7ff8b0c00700 1 -- 172.21.15.107:0/260338194 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff8a400db60 con 0x7ff8b40fcd40 2024-03-04T13:38:54.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:54.778+0000 7ff8b0c00700 1 -- 172.21.15.107:0/260338194 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff8a400f040 con 0x7ff8b40fcd40 2024-03-04T13:38:54.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:54.778+0000 7ff8b9600700 1 -- 172.21.15.107:0/260338194 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8b4110ff0 con 0x7ff8b40fcd40 2024-03-04T13:38:54.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:54.778+0000 7ff8b0c00700 1 -- 172.21.15.107:0/260338194 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff8a4009790 con 0x7ff8b40fcd40 2024-03-04T13:38:54.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:54.778+0000 7ff8b9600700 1 -- 172.21.15.107:0/260338194 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8b4111530 con 0x7ff8b40fcd40 2024-03-04T13:38:54.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:54.780+0000 7ff8b9600700 1 -- 172.21.15.107:0/260338194 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8b4101ff0 con 0x7ff8b40fcd40 2024-03-04T13:38:54.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:54.780+0000 7ff8b0c00700 1 -- 172.21.15.107:0/260338194 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86623+0+0 (secure 0 0 0) 0x7ff8a4007920 con 0x7ff8b40fcd40 2024-03-04T13:38:54.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:54.781+0000 7ff8b0c00700 1 --2- 172.21.15.107:0/260338194 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7ff8980692b0 0x7ff89806b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:38:54.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:54.781+0000 7ff8b0c00700 1 -- 172.21.15.107:0/260338194 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5550+0+0 (secure 0 0 0) 0x7ff8a4086cf0 con 0x7ff8b40fcd40 2024-03-04T13:38:54.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:54.782+0000 7ff8ab400700 1 --2- 172.21.15.107:0/260338194 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7ff8980692b0 0x7ff89806b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:38:54.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:54.782+0000 7ff8ab400700 1 --2- 172.21.15.107:0/260338194 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7ff8980692b0 0x7ff89806b760 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff89c00b840 tx=0x7ff89c009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:38:54.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:54.786+0000 7ff8b0c00700 1 -- 172.21.15.107:0/260338194 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7ff8a404cbd0 con 0x7ff8b40fcd40 2024-03-04T13:38:55.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:55.048+0000 7ff8b9600700 1 -- 172.21.15.107:0/260338194 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ff8b4111850 con 0x7ff8b40fcd40 2024-03-04T13:38:55.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:55.049+0000 7ff8b0c00700 1 -- 172.21.15.107:0/260338194 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 16 v16) v1 ==== 76+0+1842 (secure 0 0 0) 0x7ff8a4056a90 con 0x7ff8b40fcd40 2024-03-04T13:38:55.043 INFO:teuthology.orchestra.run.smithi107.stdout:e16 2024-03-04T13:38:55.043 INFO:teuthology.orchestra.run.smithi107.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-04T13:38:55.043 INFO:teuthology.orchestra.run.smithi107.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-04T13:38:55.043 INFO:teuthology.orchestra.run.smithi107.stdout:legacy client fscid: 1 2024-03-04T13:38:55.043 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:38:55.043 INFO:teuthology.orchestra.run.smithi107.stdout:Filesystem 'cephfs' (1) 2024-03-04T13:38:55.043 INFO:teuthology.orchestra.run.smithi107.stdout:fs_name cephfs 2024-03-04T13:38:55.044 INFO:teuthology.orchestra.run.smithi107.stdout:epoch 15 2024-03-04T13:38:55.044 INFO:teuthology.orchestra.run.smithi107.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-04T13:38:55.044 INFO:teuthology.orchestra.run.smithi107.stdout:created 2024-03-04T13:38:30.854004+0000 2024-03-04T13:38:55.044 INFO:teuthology.orchestra.run.smithi107.stdout:modified 2024-03-04T13:38:50.015989+0000 2024-03-04T13:38:55.044 INFO:teuthology.orchestra.run.smithi107.stdout:tableserver 0 2024-03-04T13:38:55.044 INFO:teuthology.orchestra.run.smithi107.stdout:root 0 2024-03-04T13:38:55.044 INFO:teuthology.orchestra.run.smithi107.stdout:session_timeout 60 2024-03-04T13:38:55.044 INFO:teuthology.orchestra.run.smithi107.stdout:session_autoclose 300 2024-03-04T13:38:55.044 INFO:teuthology.orchestra.run.smithi107.stdout:max_file_size 1099511627776 2024-03-04T13:38:55.044 INFO:teuthology.orchestra.run.smithi107.stdout:required_client_features {} 2024-03-04T13:38:55.044 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure 0 2024-03-04T13:38:55.044 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure_osd_epoch 47 2024-03-04T13:38:55.044 INFO:teuthology.orchestra.run.smithi107.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-04T13:38:55.044 INFO:teuthology.orchestra.run.smithi107.stdout:max_mds 1 2024-03-04T13:38:55.045 INFO:teuthology.orchestra.run.smithi107.stdout:in 0 2024-03-04T13:38:55.045 INFO:teuthology.orchestra.run.smithi107.stdout:up {0=24247} 2024-03-04T13:38:55.045 INFO:teuthology.orchestra.run.smithi107.stdout:failed 2024-03-04T13:38:55.045 INFO:teuthology.orchestra.run.smithi107.stdout:damaged 2024-03-04T13:38:55.045 INFO:teuthology.orchestra.run.smithi107.stdout:stopped 2024-03-04T13:38:55.045 INFO:teuthology.orchestra.run.smithi107.stdout:data_pools [3] 2024-03-04T13:38:55.045 INFO:teuthology.orchestra.run.smithi107.stdout:metadata_pool 2 2024-03-04T13:38:55.045 INFO:teuthology.orchestra.run.smithi107.stdout:inline_data enabled 2024-03-04T13:38:55.045 INFO:teuthology.orchestra.run.smithi107.stdout:balancer 2024-03-04T13:38:55.045 INFO:teuthology.orchestra.run.smithi107.stdout:standby_count_wanted 1 2024-03-04T13:38:55.045 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi169.jusudl{0:24247} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.169:6824/2994866076,v1:172.21.15.169:6825/2994866076] compat {c=[1],r=[1],i=[7ff]}] 2024-03-04T13:38:55.045 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.thycho{0:14470} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.107:6828/1985577060,v1:172.21.15.107:6829/1985577060] compat {c=[1],r=[1],i=[7ff]}] 2024-03-04T13:38:55.045 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:38:55.045 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:38:55.046 INFO:teuthology.orchestra.run.smithi107.stdout:Standby daemons: 2024-03-04T13:38:55.046 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:38:55.046 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.riexmm{-1:14480} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.107:6826/3261622517,v1:172.21.15.107:6827/3261622517] compat {c=[1],r=[1],i=[7ff]}] 2024-03-04T13:38:55.046 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi169.bxitzk{-1:24259} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.169:6826/3696090493,v1:172.21.15.169:6827/3696090493] compat {c=[1],r=[1],i=[7ff]}] 2024-03-04T13:38:55.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:55.051+0000 7ff8b9600700 1 -- 172.21.15.107:0/260338194 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7ff8980692b0 msgr2=0x7ff89806b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:38:55.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:55.051+0000 7ff8b9600700 1 --2- 172.21.15.107:0/260338194 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7ff8980692b0 0x7ff89806b760 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff89c00b840 tx=0x7ff89c009e30 comp rx=0 tx=0).stop 2024-03-04T13:38:55.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:55.051+0000 7ff8b9600700 1 -- 172.21.15.107:0/260338194 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8b40fcd40 msgr2=0x7ff8b410f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:38:55.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:55.051+0000 7ff8b9600700 1 --2- 172.21.15.107:0/260338194 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8b40fcd40 0x7ff8b410f930 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff8a4001ce0 tx=0x7ff8a4009e60 comp rx=0 tx=0).stop 2024-03-04T13:38:55.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:55.052+0000 7ff8b9600700 1 -- 172.21.15.107:0/260338194 shutdown_connections 2024-03-04T13:38:55.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:55.052+0000 7ff8b9600700 1 --2- 172.21.15.107:0/260338194 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7ff8980692b0 0x7ff89806b760 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:38:55.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:55.052+0000 7ff8b9600700 1 --2- 172.21.15.107:0/260338194 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8b40fcd40 0x7ff8b410f930 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:38:55.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:55.052+0000 7ff8b9600700 1 --2- 172.21.15.107:0/260338194 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff8b40fdf50 0x7ff8b410fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:38:55.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:55.052+0000 7ff8b9600700 1 -- 172.21.15.107:0/260338194 >> 172.21.15.107:0/260338194 conn(0x7ff8b40f83e0 msgr2=0x7ff8b40ff170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:38:55.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:55.052+0000 7ff8b9600700 1 -- 172.21.15.107:0/260338194 shutdown_connections 2024-03-04T13:38:55.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:55.052+0000 7ff8b9600700 1 -- 172.21.15.107:0/260338194 wait complete. 2024-03-04T13:38:55.047 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 16 2024-03-04T13:38:55.679 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:55 smithi107 ceph-mon[28739]: from='client.? 172.21.15.107:0/260338194' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-04T13:38:55.853 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:55 smithi169 ceph-mon[36313]: from='client.? 172.21.15.107:0/260338194' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-04T13:38:56.516 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 -- bash -c 'ceph --format=json fs dump | jq -e ".filesystems | length == 1"' 2024-03-04T13:38:56.836 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:56 smithi107 ceph-mon[28739]: pgmap v229: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 6.7 KiB/s rd, 0 B/s wr, 7 op/s 2024-03-04T13:38:56.853 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:56 smithi169 ceph-mon[36313]: pgmap v229: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 6.7 KiB/s rd, 0 B/s wr, 7 op/s 2024-03-04T13:38:57.013 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/mon.smithi107/config 2024-03-04T13:38:58.222 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:58 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:38:58.222 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:58 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:38:58.222 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:58 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:38:58.222 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:58 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:38:58.222 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:58 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:38:58.222 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:58 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:38:58.223 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:58 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:38:58.223 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:58 smithi107 ceph-mon[28739]: pgmap v230: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 6.5 KiB/s rd, 0 B/s wr, 7 op/s 2024-03-04T13:38:58.223 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:58 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:38:58.354 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:58 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:38:58.354 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:58 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:38:58.354 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:58 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:38:58.354 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:58 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:38:58.354 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:58 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:38:58.354 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:58 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:38:58.354 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:58 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:38:58.354 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:58 smithi169 ceph-mon[36313]: pgmap v230: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 6.5 KiB/s rd, 0 B/s wr, 7 op/s 2024-03-04T13:38:58.354 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:58 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:38:59.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:58.955+0000 7f1b2e800700 1 -- 172.21.15.107:0/2875882148 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b28072890 msgr2=0x7f1b28072ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:38:59.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:58.955+0000 7f1b2e800700 1 --2- 172.21.15.107:0/2875882148 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b28072890 0x7f1b28072ca0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1b1c004940 tx=0x7f1b1c009e30 comp rx=0 tx=0).stop 2024-03-04T13:38:59.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:58.955+0000 7f1b2e800700 1 -- 172.21.15.107:0/2875882148 shutdown_connections 2024-03-04T13:38:59.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:58.955+0000 7f1b2e800700 1 --2- 172.21.15.107:0/2875882148 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b28073af0 0x7f1b28073f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:38:59.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:58.955+0000 7f1b2e800700 1 --2- 172.21.15.107:0/2875882148 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b28072890 0x7f1b28072ca0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:38:59.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:58.955+0000 7f1b2e800700 1 -- 172.21.15.107:0/2875882148 >> 172.21.15.107:0/2875882148 conn(0x7f1b280fa760 msgr2=0x7f1b280fcb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:38:59.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:58.956+0000 7f1b2e800700 1 -- 172.21.15.107:0/2875882148 shutdown_connections 2024-03-04T13:38:59.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:58.956+0000 7f1b2e800700 1 -- 172.21.15.107:0/2875882148 wait complete. 2024-03-04T13:38:59.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:58.956+0000 7f1b2e800700 1 Processor -- start 2024-03-04T13:38:59.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:58.956+0000 7f1b2e800700 1 -- start start 2024-03-04T13:38:59.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:58.957+0000 7f1b2e800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b28072890 0x7f1b28113d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:38:59.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:58.957+0000 7f1b2e800700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b28073af0 0x7f1b281142d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:38:59.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:58.957+0000 7f1b2e800700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b281150c0 con 0x7f1b28072890 2024-03-04T13:38:59.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:58.957+0000 7f1b2e800700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b28115230 con 0x7f1b28073af0 2024-03-04T13:38:59.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:58.957+0000 7f1b2d400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b28072890 0x7f1b28113d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:38:59.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:58.957+0000 7f1b2d400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b28072890 0x7f1b28113d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57886/0 (socket says 172.21.15.107:57886) 2024-03-04T13:38:59.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:58.957+0000 7f1b2d400700 1 -- 172.21.15.107:0/1264136408 learned_addr learned my addr 172.21.15.107:0/1264136408 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:38:59.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:58.957+0000 7f1b2d400700 1 -- 172.21.15.107:0/1264136408 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b28073af0 msgr2=0x7f1b281142d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:38:59.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:58.957+0000 7f1b2d400700 1 --2- 172.21.15.107:0/1264136408 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b28073af0 0x7f1b281142d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:38:59.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:58.957+0000 7f1b2d400700 1 -- 172.21.15.107:0/1264136408 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b1c005040 con 0x7f1b28072890 2024-03-04T13:38:59.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:58.958+0000 7f1b2d400700 1 --2- 172.21.15.107:0/1264136408 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b28072890 0x7f1b28113d90 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f1b1c009fd0 tx=0x7f1b1c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:38:59.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:58.958+0000 7f1b25600700 1 -- 172.21.15.107:0/1264136408 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1b1c00e040 con 0x7f1b28072890 2024-03-04T13:38:59.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:58.958+0000 7f1b25600700 1 -- 172.21.15.107:0/1264136408 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1b1c00bcc0 con 0x7f1b28072890 2024-03-04T13:38:59.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:58.958+0000 7f1b2e800700 1 -- 172.21.15.107:0/1264136408 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b281154b0 con 0x7f1b28072890 2024-03-04T13:38:59.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:58.958+0000 7f1b25600700 1 -- 172.21.15.107:0/1264136408 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1b1c01c630 con 0x7f1b28072890 2024-03-04T13:38:59.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:58.958+0000 7f1b2e800700 1 -- 172.21.15.107:0/1264136408 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b28115a70 con 0x7f1b28072890 2024-03-04T13:38:59.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:58.959+0000 7f1b2e800700 1 -- 172.21.15.107:0/1264136408 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b2805f5c0 con 0x7f1b28072890 2024-03-04T13:38:59.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:58.960+0000 7f1b25600700 1 -- 172.21.15.107:0/1264136408 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86623+0+0 (secure 0 0 0) 0x7f1b1c013980 con 0x7f1b28072890 2024-03-04T13:38:59.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:58.961+0000 7f1b25600700 1 --2- 172.21.15.107:0/1264136408 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f1b100692b0 0x7f1b1006b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:38:59.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:58.961+0000 7f1b25600700 1 -- 172.21.15.107:0/1264136408 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f1b1c088740 con 0x7f1b28072890 2024-03-04T13:38:59.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:58.961+0000 7f1b2ca00700 1 --2- 172.21.15.107:0/1264136408 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f1b100692b0 0x7f1b1006b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:38:59.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:58.962+0000 7f1b2ca00700 1 --2- 172.21.15.107:0/1264136408 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f1b100692b0 0x7f1b1006b760 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1b28113b20 tx=0x7f1b14009ce0 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:38:59.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:58.966+0000 7f1b25600700 1 -- 172.21.15.107:0/1264136408 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f1b1c054980 con 0x7f1b28072890 2024-03-04T13:38:59.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:59.237+0000 7f1b2e800700 1 -- 172.21.15.107:0/1264136408 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1b2804bed0 con 0x7f1b28072890 2024-03-04T13:38:59.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:59.238+0000 7f1b25600700 1 -- 172.21.15.107:0/1264136408 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4708 (secure 0 0 0) 0x7f1b1c057fa0 con 0x7f1b28072890 2024-03-04T13:38:59.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:59.241+0000 7f1b2e800700 1 -- 172.21.15.107:0/1264136408 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f1b100692b0 msgr2=0x7f1b1006b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:38:59.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:59.241+0000 7f1b2e800700 1 --2- 172.21.15.107:0/1264136408 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f1b100692b0 0x7f1b1006b760 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1b28113b20 tx=0x7f1b14009ce0 comp rx=0 tx=0).stop 2024-03-04T13:38:59.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:59.241+0000 7f1b2e800700 1 -- 172.21.15.107:0/1264136408 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b28072890 msgr2=0x7f1b28113d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:38:59.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:59.241+0000 7f1b2e800700 1 --2- 172.21.15.107:0/1264136408 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b28072890 0x7f1b28113d90 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f1b1c009fd0 tx=0x7f1b1c009e30 comp rx=0 tx=0).stop 2024-03-04T13:38:59.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:59.241+0000 7f1b2e800700 1 -- 172.21.15.107:0/1264136408 shutdown_connections 2024-03-04T13:38:59.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:59.241+0000 7f1b2e800700 1 --2- 172.21.15.107:0/1264136408 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f1b100692b0 0x7f1b1006b760 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:38:59.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:59.241+0000 7f1b2e800700 1 --2- 172.21.15.107:0/1264136408 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b28072890 0x7f1b28113d90 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:38:59.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:59.241+0000 7f1b2e800700 1 --2- 172.21.15.107:0/1264136408 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b28073af0 0x7f1b281142d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:38:59.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:59.241+0000 7f1b2e800700 1 -- 172.21.15.107:0/1264136408 >> 172.21.15.107:0/1264136408 conn(0x7f1b280fa760 msgr2=0x7f1b281092a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:38:59.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:59.241+0000 7f1b2e800700 1 -- 172.21.15.107:0/1264136408 shutdown_connections 2024-03-04T13:38:59.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:38:59.241+0000 7f1b2e800700 1 -- 172.21.15.107:0/1264136408 wait complete. 2024-03-04T13:38:59.237 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 16 2024-03-04T13:38:59.250 INFO:teuthology.orchestra.run.smithi107.stdout:true 2024-03-04T13:38:59.586 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:38:59 smithi107 ceph-mon[28739]: from='client.? 172.21.15.107:0/1264136408' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T13:38:59.603 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:38:59 smithi169 ceph-mon[36313]: from='client.? 172.21.15.107:0/1264136408' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T13:39:00.502 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 -- bash -c 'while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done' 2024-03-04T13:39:00.586 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:39:00 smithi107 ceph-mon[28739]: pgmap v231: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 5.4 KiB/s rd, 0 B/s wr, 5 op/s 2024-03-04T13:39:00.603 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:39:00 smithi169 ceph-mon[36313]: pgmap v231: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 5.4 KiB/s rd, 0 B/s wr, 5 op/s 2024-03-04T13:39:01.559 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/mon.smithi107/config 2024-03-04T13:39:01.836 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:39:01 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:39:01.836 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:39:01 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:39:01.853 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:39:01 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:39:01.853 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:39:01 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:39:02.854 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:39:02 smithi169 ceph-mon[36313]: pgmap v232: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 5.7 KiB/s rd, 0 B/s wr, 6 op/s 2024-03-04T13:39:03.086 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:39:02 smithi107 ceph-mon[28739]: pgmap v232: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 5.7 KiB/s rd, 0 B/s wr, 6 op/s 2024-03-04T13:39:04.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:04.817+0000 7f67a3800700 1 -- 172.21.15.107:0/2255479480 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f679c100360 msgr2=0x7f679c1007b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:39:04.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:04.817+0000 7f67a3800700 1 --2- 172.21.15.107:0/2255479480 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f679c100360 0x7f679c1007b0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6788004940 tx=0x7f6788009e30 comp rx=0 tx=0).stop 2024-03-04T13:39:04.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:04.817+0000 7f67a3800700 1 -- 172.21.15.107:0/2255479480 shutdown_connections 2024-03-04T13:39:04.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:04.817+0000 7f67a3800700 1 --2- 172.21.15.107:0/2255479480 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f679c100360 0x7f679c1007b0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:39:04.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:04.817+0000 7f67a3800700 1 --2- 172.21.15.107:0/2255479480 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f679c0ff150 0x7f679c0ff560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:39:04.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:04.817+0000 7f67a3800700 1 -- 172.21.15.107:0/2255479480 >> 172.21.15.107:0/2255479480 conn(0x7f679c0fa920 msgr2=0x7f679c0fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:39:04.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:04.817+0000 7f67a3800700 1 -- 172.21.15.107:0/2255479480 shutdown_connections 2024-03-04T13:39:04.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:04.817+0000 7f67a3800700 1 -- 172.21.15.107:0/2255479480 wait complete. 2024-03-04T13:39:04.812 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:39:04 smithi107 ceph-mon[28739]: pgmap v233: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 4.7 KiB/s rd, 0 B/s wr, 5 op/s 2024-03-04T13:39:04.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:04.818+0000 7f67a3800700 1 Processor -- start 2024-03-04T13:39:04.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:04.818+0000 7f67a3800700 1 -- start start 2024-03-04T13:39:04.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:04.818+0000 7f67a3800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f679c0ff150 0x7f679c10fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:39:04.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:04.818+0000 7f67a3800700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f679c100360 0x7f679c1100b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:39:04.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:04.819+0000 7f67a3800700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f679c1106b0 con 0x7f679c0ff150 2024-03-04T13:39:04.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:04.819+0000 7f67a3800700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f679c110f50 con 0x7f679c100360 2024-03-04T13:39:04.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:04.819+0000 7f67a0e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f679c0ff150 0x7f679c10fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:39:04.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:04.819+0000 7f67a0e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f679c0ff150 0x7f679c10fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57894/0 (socket says 172.21.15.107:57894) 2024-03-04T13:39:04.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:04.819+0000 7f67a0e00700 1 -- 172.21.15.107:0/3888189863 learned_addr learned my addr 172.21.15.107:0/3888189863 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:39:04.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:04.819+0000 7f679be00700 1 --2- 172.21.15.107:0/3888189863 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f679c100360 0x7f679c1100b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:39:04.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:04.819+0000 7f67a0e00700 1 -- 172.21.15.107:0/3888189863 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f679c100360 msgr2=0x7f679c1100b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:39:04.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:04.819+0000 7f67a0e00700 1 --2- 172.21.15.107:0/3888189863 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f679c100360 0x7f679c1100b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:39:04.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:04.819+0000 7f67a0e00700 1 -- 172.21.15.107:0/3888189863 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6788005040 con 0x7f679c0ff150 2024-03-04T13:39:04.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:04.820+0000 7f67a0e00700 1 --2- 172.21.15.107:0/3888189863 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f679c0ff150 0x7f679c10fb70 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f679000c8b0 tx=0x7f6790004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:39:04.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:04.820+0000 7f6798c00700 1 -- 172.21.15.107:0/3888189863 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f679000a760 con 0x7f679c0ff150 2024-03-04T13:39:04.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:04.820+0000 7f6798c00700 1 -- 172.21.15.107:0/3888189863 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f679000ad90 con 0x7f679c0ff150 2024-03-04T13:39:04.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:04.820+0000 7f6798c00700 1 -- 172.21.15.107:0/3888189863 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f67900113a0 con 0x7f679c0ff150 2024-03-04T13:39:04.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:04.820+0000 7f67a3800700 1 -- 172.21.15.107:0/3888189863 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f679c111230 con 0x7f679c0ff150 2024-03-04T13:39:04.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:04.820+0000 7f67a3800700 1 -- 172.21.15.107:0/3888189863 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f679c111770 con 0x7f679c0ff150 2024-03-04T13:39:04.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:04.821+0000 7f67a3800700 1 -- 172.21.15.107:0/3888189863 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f679c04bed0 con 0x7f679c0ff150 2024-03-04T13:39:04.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:04.822+0000 7f6798c00700 1 -- 172.21.15.107:0/3888189863 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86623+0+0 (secure 0 0 0) 0x7f6790012400 con 0x7f679c0ff150 2024-03-04T13:39:04.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:04.824+0000 7f6798c00700 1 --2- 172.21.15.107:0/3888189863 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f67840692b0 0x7f678406b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:39:04.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:04.824+0000 7f6798c00700 1 -- 172.21.15.107:0/3888189863 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f6790087220 con 0x7f679c0ff150 2024-03-04T13:39:04.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:04.824+0000 7f679be00700 1 --2- 172.21.15.107:0/3888189863 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f67840692b0 0x7f678406b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:39:04.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:04.825+0000 7f679be00700 1 --2- 172.21.15.107:0/3888189863 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f67840692b0 0x7f678406b760 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f678800b020 tx=0x7f6788009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:39:04.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:04.828+0000 7f6798c00700 1 -- 172.21.15.107:0/3888189863 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f6790056f70 con 0x7f679c0ff150 2024-03-04T13:39:04.853 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:39:04 smithi169 ceph-mon[36313]: pgmap v233: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 4.7 KiB/s rd, 0 B/s wr, 5 op/s 2024-03-04T13:39:05.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:05.137+0000 7f67a3800700 1 -- 172.21.15.107:0/3888189863 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mds versions", "format": "json"} v 0) v1 -- 0x7f679c107c30 con 0x7f679c0ff150 2024-03-04T13:39:05.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:05.137+0000 7f6798c00700 1 -- 172.21.15.107:0/3888189863 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mds versions", "format": "json"}]=0 v16) v1 ==== 78+0+99 (secure 0 0 0) 0x7f6790056b00 con 0x7f679c0ff150 2024-03-04T13:39:05.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:05.140+0000 7f67a3800700 1 -- 172.21.15.107:0/3888189863 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f67840692b0 msgr2=0x7f678406b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:39:05.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:05.140+0000 7f67a3800700 1 --2- 172.21.15.107:0/3888189863 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f67840692b0 0x7f678406b760 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f678800b020 tx=0x7f6788009e30 comp rx=0 tx=0).stop 2024-03-04T13:39:05.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:05.140+0000 7f67a3800700 1 -- 172.21.15.107:0/3888189863 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f679c0ff150 msgr2=0x7f679c10fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:39:05.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:05.140+0000 7f67a3800700 1 --2- 172.21.15.107:0/3888189863 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f679c0ff150 0x7f679c10fb70 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f679000c8b0 tx=0x7f6790004470 comp rx=0 tx=0).stop 2024-03-04T13:39:05.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:05.140+0000 7f67a3800700 1 -- 172.21.15.107:0/3888189863 shutdown_connections 2024-03-04T13:39:05.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:05.140+0000 7f67a3800700 1 --2- 172.21.15.107:0/3888189863 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f67840692b0 0x7f678406b760 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:39:05.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:05.140+0000 7f67a3800700 1 --2- 172.21.15.107:0/3888189863 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f679c0ff150 0x7f679c10fb70 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:39:05.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:05.140+0000 7f67a3800700 1 --2- 172.21.15.107:0/3888189863 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f679c100360 0x7f679c1100b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:39:05.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:05.140+0000 7f67a3800700 1 -- 172.21.15.107:0/3888189863 >> 172.21.15.107:0/3888189863 conn(0x7f679c0fa920 msgr2=0x7f679c103590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:39:05.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:05.140+0000 7f67a3800700 1 -- 172.21.15.107:0/3888189863 shutdown_connections 2024-03-04T13:39:05.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:05.140+0000 7f67a3800700 1 -- 172.21.15.107:0/3888189863 wait complete. 2024-03-04T13:39:05.152 INFO:teuthology.orchestra.run.smithi107.stdout:true 2024-03-04T13:39:05.836 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:39:05 smithi107 ceph-mon[28739]: from='client.? 172.21.15.107:0/3888189863' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-03-04T13:39:05.887 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:39:05 smithi169 ceph-mon[36313]: from='client.? 172.21.15.107:0/3888189863' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-03-04T13:39:06.264 INFO:teuthology.run_tasks:Running task fs.pre_upgrade_save... 2024-03-04T13:39:06.279 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-04T13:39:06.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:06.503+0000 7ff7a1400640 1 -- 172.21.15.107:0/2570460377 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff79c0fd3e0 msgr2=0x7ff79c0fd850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:39:06.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:06.503+0000 7ff7a1400640 1 --2- 172.21.15.107:0/2570460377 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff79c0fd3e0 0x7ff79c0fd850 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ff7880048d0 tx=0x7ff788009da0 comp rx=0 tx=0).stop 2024-03-04T13:39:06.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:06.503+0000 7ff7a1400640 1 -- 172.21.15.107:0/2570460377 shutdown_connections 2024-03-04T13:39:06.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:06.503+0000 7ff7a1400640 1 --2- 172.21.15.107:0/2570460377 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff79c0fd3e0 0x7ff79c0fd850 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:39:06.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:06.503+0000 7ff7a1400640 1 --2- 172.21.15.107:0/2570460377 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff79c0fc110 0x7ff79c0fc520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:39:06.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:06.503+0000 7ff7a1400640 1 -- 172.21.15.107:0/2570460377 >> 172.21.15.107:0/2570460377 conn(0x7ff79c0f7890 msgr2=0x7ff79c0f9d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:39:06.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:06.503+0000 7ff7a1400640 1 -- 172.21.15.107:0/2570460377 shutdown_connections 2024-03-04T13:39:06.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:06.503+0000 7ff7a1400640 1 -- 172.21.15.107:0/2570460377 wait complete. 2024-03-04T13:39:06.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:06.504+0000 7ff7a1400640 1 Processor -- start 2024-03-04T13:39:06.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:06.504+0000 7ff7a1400640 1 -- start start 2024-03-04T13:39:06.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:06.504+0000 7ff7a1400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff79c0fc110 0x7ff79c10ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:39:06.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:06.505+0000 7ff7a1400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff79c0fd3e0 0x7ff79c10f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:39:06.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:06.505+0000 7ff7a1400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff79c10f7b0 con 0x7ff79c0fd3e0 2024-03-04T13:39:06.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:06.505+0000 7ff7a1400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff79c1141c0 con 0x7ff79c0fc110 2024-03-04T13:39:06.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:06.505+0000 7ff79b400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff79c0fd3e0 0x7ff79c10f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:39:06.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:06.505+0000 7ff79b400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff79c0fd3e0 0x7ff79c10f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57914/0 (socket says 172.21.15.107:57914) 2024-03-04T13:39:06.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:06.505+0000 7ff79b400640 1 -- 172.21.15.107:0/1710681955 learned_addr learned my addr 172.21.15.107:0/1710681955 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:39:06.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:06.505+0000 7ff79b400640 1 -- 172.21.15.107:0/1710681955 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff79c0fc110 msgr2=0x7ff79c10ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:39:06.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:06.505+0000 7ff79be00640 1 --2- 172.21.15.107:0/1710681955 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff79c0fc110 0x7ff79c10ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:39:06.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:06.505+0000 7ff79b400640 1 --2- 172.21.15.107:0/1710681955 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff79c0fc110 0x7ff79c10ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:39:06.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:06.505+0000 7ff79b400640 1 -- 172.21.15.107:0/1710681955 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff788005040 con 0x7ff79c0fd3e0 2024-03-04T13:39:06.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:06.506+0000 7ff79b400640 1 --2- 172.21.15.107:0/1710681955 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff79c0fd3e0 0x7ff79c10f160 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff788007da0 tx=0x7ff788009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:39:06.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:06.506+0000 7ff793e00640 1 -- 172.21.15.107:0/1710681955 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff7880094c0 con 0x7ff79c0fd3e0 2024-03-04T13:39:06.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:06.506+0000 7ff793e00640 1 -- 172.21.15.107:0/1710681955 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff788009620 con 0x7ff79c0fd3e0 2024-03-04T13:39:06.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:06.506+0000 7ff7a1400640 1 -- 172.21.15.107:0/1710681955 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff79c114300 con 0x7ff79c0fd3e0 2024-03-04T13:39:06.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:06.506+0000 7ff793e00640 1 -- 172.21.15.107:0/1710681955 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff78803b620 con 0x7ff79c0fd3e0 2024-03-04T13:39:06.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:06.506+0000 7ff7a1400640 1 -- 172.21.15.107:0/1710681955 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff79c1147d0 con 0x7ff79c0fd3e0 2024-03-04T13:39:06.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:06.508+0000 7ff7a1400640 1 -- 172.21.15.107:0/1710681955 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff79c1013f0 con 0x7ff79c0fd3e0 2024-03-04T13:39:06.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:06.508+0000 7ff793e00640 1 -- 172.21.15.107:0/1710681955 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86623+0+0 (secure 0 0 0) 0x7ff78802f070 con 0x7ff79c0fd3e0 2024-03-04T13:39:06.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:06.509+0000 7ff793e00640 1 --2- 172.21.15.107:0/1710681955 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7ff780069210 0x7ff78006b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:39:06.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:06.509+0000 7ff793e00640 1 -- 172.21.15.107:0/1710681955 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5550+0+0 (secure 0 0 0) 0x7ff7880a9840 con 0x7ff79c0fd3e0 2024-03-04T13:39:06.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:06.510+0000 7ff79be00640 1 --2- 172.21.15.107:0/1710681955 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7ff780069210 0x7ff78006b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:39:06.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:06.510+0000 7ff79be00640 1 --2- 172.21.15.107:0/1710681955 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7ff780069210 0x7ff78006b6c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff79c110130 tx=0x7ff78c00b040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:39:06.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:06.514+0000 7ff793e00640 1 -- 172.21.15.107:0/1710681955 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7ff788075aa0 con 0x7ff79c0fd3e0 2024-03-04T13:39:06.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:06.720+0000 7ff7a1400640 1 -- 172.21.15.107:0/1710681955 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff79c104c80 con 0x7ff79c0fd3e0 2024-03-04T13:39:06.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:06.722+0000 7ff793e00640 1 -- 172.21.15.107:0/1710681955 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+14439 (secure 0 0 0) 0x7ff7880790c0 con 0x7ff79c0fd3e0 2024-03-04T13:39:06.716 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:39:06.716 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":47,"fsid":"1fc09a58-da2b-11ee-95c6-87774f69a715","created":"2024-03-04T13:29:42.985888+0000","modified":"2024-03-04T13:38:46.993432+0000","last_up_change":"2024-03-04T13:36:53.661634+0000","last_in_change":"2024-03-04T13:36:31.424613+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T13:35:17.915858+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-03-04T13:38:29.362118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-03-04T13:38:30.373574+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c857e387-3d2b-478d-9807-24ea8b9fe02d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":3721776817},{"type":"v1","addr":"172.21.15.107:6803","nonce":3721776817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":3721776817},{"type":"v1","addr":"172.21.15.107:6805","nonce":3721776817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":3721776817},{"type":"v1","addr":"172.21.15.107:6809","nonce":3721776817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":3721776817},{"type":"v1","addr":"172.21.15.107:6807","nonce":3721776817}]},"public_addr":"172.21.15.107:6803/3721776817","cluster_addr":"172.21.15.107:6805/3721776817","heartbeat_back_addr":"172.21.15.107:6809/3721776817","heartbeat_front_addr":"172.21.15.107:6807/3721776817","state":["exists","up"]},{"osd":1,"uuid":"fa29dc8f-419b-4318-b43e-e13fffd87204","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":4224626475},{"type":"v1","addr":"172.21.15.107:6811","nonce":4224626475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":4224626475},{"type":"v1","addr":"172.21.15.107:6813","nonce":4224626475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4224626475},{"type":"v1","addr":"172.21.15.107:6817","nonce":4224626475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":4224626475},{"type":"v1","addr":"172.21.15.107:6815","nonce":4224626475}]},"public_addr":"172.21.15.107:6811/4224626475","cluster_addr":"172.21.15.107:6813/4224626475","heartbeat_back_addr":"172.21.15.107:6817/4224626475","heartbeat_front_addr":"172.21.15.107:6815/4224626475","state":["exists","up"]},{"osd":2,"uuid":"580a07f0-7bd4-4341-9c1f-b3383d64c720","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2331133561},{"type":"v1","addr":"172.21.15.107:6819","nonce":2331133561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2331133561},{"type":"v1","addr":"172.21.15.107:6821","nonce":2331133561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":2331133561},{"type":"v1","addr":"172.21.15.107:6825","nonce":2331133561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2331133561},{"type":"v1","addr":"172.21.15.107:6823","nonce":2331133561}]},"public_addr":"172.21.15.107:6819/2331133561","cluster_addr":"172.21.15.107:6821/2331133561","heartbeat_back_addr":"172.21.15.107:6825/2331133561","heartbeat_front_addr":"172.21.15.107:6823/2331133561","state":["exists","up"]},{"osd":3,"uuid":"7acac14d-998a-4651-b78c-e42584e13b43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":2475630313},{"type":"v1","addr":"172.21.15.169:6801","nonce":2475630313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":2475630313},{"type":"v1","addr":"172.21.15.169:6803","nonce":2475630313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":2475630313},{"type":"v1","addr":"172.21.15.169:6807","nonce":2475630313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":2475630313},{"type":"v1","addr":"172.21.15.169:6805","nonce":2475630313}]},"public_addr":"172.21.15.169:6801/2475630313","cluster_addr":"172.21.15.169:6803/2475630313","heartbeat_back_addr":"172.21.15.169:6807/2475630313","heartbeat_front_addr":"172.21.15.169:6805/2475630313","state":["exists","up"]},{"osd":4,"uuid":"01d2b77c-66ec-4b50-8170-43d95eeae471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":592652532},{"type":"v1","addr":"172.21.15.169:6809","nonce":592652532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":592652532},{"type":"v1","addr":"172.21.15.169:6811","nonce":592652532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":592652532},{"type":"v1","addr":"172.21.15.169:6815","nonce":592652532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":592652532},{"type":"v1","addr":"172.21.15.169:6813","nonce":592652532}]},"public_addr":"172.21.15.169:6809/592652532","cluster_addr":"172.21.15.169:6811/592652532","heartbeat_back_addr":"172.21.15.169:6815/592652532","heartbeat_front_addr":"172.21.15.169:6813/592652532","state":["exists","up"]},{"osd":5,"uuid":"0164ead7-c5d0-4541-bf22-f671558f8607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":182182184},{"type":"v1","addr":"172.21.15.169:6817","nonce":182182184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":182182184},{"type":"v1","addr":"172.21.15.169:6819","nonce":182182184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":182182184},{"type":"v1","addr":"172.21.15.169:6823","nonce":182182184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":182182184},{"type":"v1","addr":"172.21.15.169:6821","nonce":182182184}]},"public_addr":"172.21.15.169:6817/182182184","cluster_addr":"172.21.15.169:6819/182182184","heartbeat_back_addr":"172.21.15.169:6823/182182184","heartbeat_front_addr":"172.21.15.169:6821/182182184","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T13:33:59.729659+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T13:34:36.600326+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T13:35:14.068513+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T13:35:46.608007+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T13:36:19.595680+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T13:36:52.586933+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/2357638826":"2024-03-05T13:30:16.230256+0000","172.21.15.107:0/1780539770":"2024-03-05T13:30:16.230256+0000","172.21.15.107:0/2928681569":"2024-03-05T13:30:56.353578+0000","172.21.15.107:6826/3814968386":"2024-03-05T13:38:46.993392+0000","172.21.15.107:6800/1723390229":"2024-03-05T13:30:16.230256+0000","172.21.15.107:6800/2371648646":"2024-03-05T13:30:56.353578+0000","172.21.15.107:6827/3814968386":"2024-03-05T13:38:46.993392+0000","172.21.15.107:6801/2371648646":"2024-03-05T13:30:56.353578+0000","172.21.15.107:0/2314831039":"2024-03-05T13:30:56.353578+0000","172.21.15.107:6801/1723390229":"2024-03-05T13:30:16.230256+0000","172.21.15.107:0/2723084639":"2024-03-05T13:32:27.229455+0000","172.21.15.107:0/136342223":"2024-03-05T13:30:56.353578+0000","172.21.15.107:0/3451280196":"2024-03-05T13:30:16.230256+0000","172.21.15.107:6801/2040330534":"2024-03-05T13:32:27.229455+0000","172.21.15.107:6800/2040330534":"2024-03-05T13:32:27.229455+0000","172.21.15.107:0/601795289":"2024-03-05T13:32:27.229455+0000","172.21.15.107:0/3503768751":"2024-03-05T13:32:27.229455+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T13:39:06.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:06.724+0000 7ff7a1400640 1 -- 172.21.15.107:0/1710681955 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7ff780069210 msgr2=0x7ff78006b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:39:06.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:06.724+0000 7ff7a1400640 1 --2- 172.21.15.107:0/1710681955 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7ff780069210 0x7ff78006b6c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff79c110130 tx=0x7ff78c00b040 comp rx=0 tx=0).stop 2024-03-04T13:39:06.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:06.724+0000 7ff7a1400640 1 -- 172.21.15.107:0/1710681955 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff79c0fd3e0 msgr2=0x7ff79c10f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:39:06.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:06.724+0000 7ff7a1400640 1 --2- 172.21.15.107:0/1710681955 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff79c0fd3e0 0x7ff79c10f160 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff788007da0 tx=0x7ff788009da0 comp rx=0 tx=0).stop 2024-03-04T13:39:06.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:06.725+0000 7ff7a1400640 1 -- 172.21.15.107:0/1710681955 shutdown_connections 2024-03-04T13:39:06.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:06.725+0000 7ff7a1400640 1 --2- 172.21.15.107:0/1710681955 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff79c0fd3e0 0x7ff79c10f160 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:39:06.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:06.725+0000 7ff7a1400640 1 --2- 172.21.15.107:0/1710681955 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7ff780069210 0x7ff78006b6c0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:39:06.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:06.725+0000 7ff7a1400640 1 --2- 172.21.15.107:0/1710681955 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff79c0fc110 0x7ff79c10ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:39:06.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:06.725+0000 7ff7a1400640 1 -- 172.21.15.107:0/1710681955 >> 172.21.15.107:0/1710681955 conn(0x7ff79c0f7890 msgr2=0x7ff79c0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:39:06.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:06.725+0000 7ff7a1400640 1 -- 172.21.15.107:0/1710681955 shutdown_connections 2024-03-04T13:39:06.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:06.725+0000 7ff7a1400640 1 -- 172.21.15.107:0/1710681955 wait complete. 2024-03-04T13:39:06.740 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-04T13:35:17.915858+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-03-04T13:38:29.362118+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-03-04T13:38:30.373574+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-04T13:39:06.741 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-04T13:39:06.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:06.940+0000 7f34d1c00640 1 -- 172.21.15.107:0/2134944453 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34cc071300 msgr2=0x7f34cc06f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:39:06.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:06.940+0000 7f34d1c00640 1 --2- 172.21.15.107:0/2134944453 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34cc071300 0x7f34cc06f830 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f34bc0048d0 tx=0x7f34bc009da0 comp rx=0 tx=0).stop 2024-03-04T13:39:06.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:06.940+0000 7f34d1c00640 1 -- 172.21.15.107:0/2134944453 shutdown_connections 2024-03-04T13:39:06.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:06.940+0000 7f34d1c00640 1 --2- 172.21.15.107:0/2134944453 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34cc06fd70 0x7f34cc0701e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:39:06.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:06.940+0000 7f34d1c00640 1 --2- 172.21.15.107:0/2134944453 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34cc071300 0x7f34cc06f830 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:39:06.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:06.940+0000 7f34d1c00640 1 -- 172.21.15.107:0/2134944453 >> 172.21.15.107:0/2134944453 conn(0x7f34cc0f7a60 msgr2=0x7f34cc0f9e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:39:06.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:06.940+0000 7f34d1c00640 1 -- 172.21.15.107:0/2134944453 shutdown_connections 2024-03-04T13:39:06.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:06.940+0000 7f34d1c00640 1 -- 172.21.15.107:0/2134944453 wait complete. 2024-03-04T13:39:06.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:06.941+0000 7f34d1c00640 1 Processor -- start 2024-03-04T13:39:06.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:06.941+0000 7f34d1c00640 1 -- start start 2024-03-04T13:39:06.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:06.941+0000 7f34d1c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34cc06fd70 0x7f34cc10ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:39:06.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:06.941+0000 7f34d1c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34cc071300 0x7f34cc10f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:39:06.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:06.941+0000 7f34d1c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34cc10f8b0 con 0x7f34cc06fd70 2024-03-04T13:39:06.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:06.941+0000 7f34d1c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34cc114300 con 0x7f34cc071300 2024-03-04T13:39:06.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:06.941+0000 7f34caa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34cc06fd70 0x7f34cc10ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:39:06.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:06.942+0000 7f34caa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34cc06fd70 0x7f34cc10ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57932/0 (socket says 172.21.15.107:57932) 2024-03-04T13:39:06.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:06.942+0000 7f34caa00640 1 -- 172.21.15.107:0/3745065552 learned_addr learned my addr 172.21.15.107:0/3745065552 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:39:06.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:06.942+0000 7f34caa00640 1 -- 172.21.15.107:0/3745065552 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34cc071300 msgr2=0x7f34cc10f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:39:06.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:06.942+0000 7f34caa00640 1 --2- 172.21.15.107:0/3745065552 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34cc071300 0x7f34cc10f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:39:06.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:06.942+0000 7f34caa00640 1 -- 172.21.15.107:0/3745065552 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34bc005040 con 0x7f34cc06fd70 2024-03-04T13:39:06.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:06.942+0000 7f34caa00640 1 --2- 172.21.15.107:0/3745065552 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34cc06fd70 0x7f34cc10ed20 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f34bc034040 tx=0x7f34bc009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:39:06.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:06.942+0000 7f34c2a00640 1 -- 172.21.15.107:0/3745065552 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f34bc009590 con 0x7f34cc06fd70 2024-03-04T13:39:06.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:06.942+0000 7f34c2a00640 1 -- 172.21.15.107:0/3745065552 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f34bc0096f0 con 0x7f34cc06fd70 2024-03-04T13:39:06.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:06.942+0000 7f34c2a00640 1 -- 172.21.15.107:0/3745065552 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f34bc03b6b0 con 0x7f34cc06fd70 2024-03-04T13:39:06.938 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:06.942+0000 7f34d1c00640 1 -- 172.21.15.107:0/3745065552 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34cc114440 con 0x7f34cc06fd70 2024-03-04T13:39:06.938 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:06.943+0000 7f34d1c00640 1 -- 172.21.15.107:0/3745065552 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34cc114930 con 0x7f34cc06fd70 2024-03-04T13:39:06.938 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:06.944+0000 7f34d1c00640 1 -- 172.21.15.107:0/3745065552 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34cc106fe0 con 0x7f34cc06fd70 2024-03-04T13:39:06.938 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:06.945+0000 7f34c2a00640 1 -- 172.21.15.107:0/3745065552 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86623+0+0 (secure 0 0 0) 0x7f34bc02f070 con 0x7f34cc06fd70 2024-03-04T13:39:06.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:06.945+0000 7f34c2a00640 1 --2- 172.21.15.107:0/3745065552 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f34b0069210 0x7f34b006b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:39:06.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:06.946+0000 7f34c2a00640 1 -- 172.21.15.107:0/3745065552 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f34bc0a9990 con 0x7f34cc06fd70 2024-03-04T13:39:06.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:06.946+0000 7f34c3e00640 1 --2- 172.21.15.107:0/3745065552 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f34b0069210 0x7f34b006b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:39:06.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:06.946+0000 7f34c3e00640 1 --2- 172.21.15.107:0/3745065552 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f34b0069210 0x7f34b006b6c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f34cc110270 tx=0x7f34b4007c30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:39:06.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:06.950+0000 7f34c2a00640 1 -- 172.21.15.107:0/3745065552 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f34bc075bf0 con 0x7f34cc06fd70 2024-03-04T13:39:07.086 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:39:06 smithi107 ceph-mon[28739]: pgmap v234: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 0 B/s wr, 3 op/s 2024-03-04T13:39:07.103 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:39:06 smithi169 ceph-mon[36313]: pgmap v234: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 0 B/s wr, 3 op/s 2024-03-04T13:39:07.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.142+0000 7f34d1c00640 1 -- 172.21.15.107:0/3745065552 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f34cc114ac0 con 0x7f34cc06fd70 2024-03-04T13:39:07.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.143+0000 7f34c2a00640 1 -- 172.21.15.107:0/3745065552 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v47) v1 ==== 93+0+10 (secure 0 0 0) 0x7f34bc079210 con 0x7f34cc06fd70 2024-03-04T13:39:07.137 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 1 2024-03-04T13:39:07.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.145+0000 7f34d1c00640 1 -- 172.21.15.107:0/3745065552 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f34b0069210 msgr2=0x7f34b006b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:39:07.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.145+0000 7f34d1c00640 1 --2- 172.21.15.107:0/3745065552 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f34b0069210 0x7f34b006b6c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f34cc110270 tx=0x7f34b4007c30 comp rx=0 tx=0).stop 2024-03-04T13:39:07.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.145+0000 7f34d1c00640 1 -- 172.21.15.107:0/3745065552 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34cc06fd70 msgr2=0x7f34cc10ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:39:07.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.145+0000 7f34d1c00640 1 --2- 172.21.15.107:0/3745065552 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34cc06fd70 0x7f34cc10ed20 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f34bc034040 tx=0x7f34bc009da0 comp rx=0 tx=0).stop 2024-03-04T13:39:07.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.145+0000 7f34d1c00640 1 -- 172.21.15.107:0/3745065552 shutdown_connections 2024-03-04T13:39:07.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.145+0000 7f34d1c00640 1 --2- 172.21.15.107:0/3745065552 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f34b0069210 0x7f34b006b6c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:39:07.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.145+0000 7f34d1c00640 1 --2- 172.21.15.107:0/3745065552 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34cc071300 0x7f34cc10f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:39:07.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.145+0000 7f34d1c00640 1 --2- 172.21.15.107:0/3745065552 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34cc06fd70 0x7f34cc10ed20 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:39:07.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.145+0000 7f34d1c00640 1 -- 172.21.15.107:0/3745065552 >> 172.21.15.107:0/3745065552 conn(0x7f34cc0f7a60 msgr2=0x7f34cc102230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:39:07.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.146+0000 7f34d1c00640 1 -- 172.21.15.107:0/3745065552 shutdown_connections 2024-03-04T13:39:07.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.146+0000 7f34d1c00640 1 -- 172.21.15.107:0/3745065552 wait complete. 2024-03-04T13:39:07.159 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-03-04T13:39:07.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.346+0000 7fc368600640 1 -- 172.21.15.107:0/3886543521 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3600ff3d0 msgr2=0x7fc3601017b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:39:07.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.346+0000 7fc368600640 1 --2- 172.21.15.107:0/3886543521 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3600ff3d0 0x7fc3601017b0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc34c0048d0 tx=0x7fc34c009da0 comp rx=0 tx=0).stop 2024-03-04T13:39:07.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.346+0000 7fc368600640 1 -- 172.21.15.107:0/3886543521 shutdown_connections 2024-03-04T13:39:07.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.346+0000 7fc368600640 1 --2- 172.21.15.107:0/3886543521 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3600ff3d0 0x7fc3601017b0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:39:07.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.346+0000 7fc368600640 1 --2- 172.21.15.107:0/3886543521 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc360066350 0x7fc3600fee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:39:07.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.346+0000 7fc368600640 1 -- 172.21.15.107:0/3886543521 >> 172.21.15.107:0/3886543521 conn(0x7fc3600f6960 msgr2=0x7fc3600f8d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:39:07.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.347+0000 7fc368600640 1 -- 172.21.15.107:0/3886543521 shutdown_connections 2024-03-04T13:39:07.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.347+0000 7fc368600640 1 -- 172.21.15.107:0/3886543521 wait complete. 2024-03-04T13:39:07.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.347+0000 7fc368600640 1 Processor -- start 2024-03-04T13:39:07.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.348+0000 7fc368600640 1 -- start start 2024-03-04T13:39:07.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.348+0000 7fc368600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc360066350 0x7fc3600fc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:39:07.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.348+0000 7fc368600640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3600ff3d0 0x7fc3600fad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:39:07.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.348+0000 7fc368600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3600fcd10 con 0x7fc360066350 2024-03-04T13:39:07.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.348+0000 7fc368600640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3600fb2d0 con 0x7fc3600ff3d0 2024-03-04T13:39:07.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.348+0000 7fc365c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc360066350 0x7fc3600fc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:39:07.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.348+0000 7fc365c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc360066350 0x7fc3600fc6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57952/0 (socket says 172.21.15.107:57952) 2024-03-04T13:39:07.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.348+0000 7fc365c00640 1 -- 172.21.15.107:0/2932134046 learned_addr learned my addr 172.21.15.107:0/2932134046 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:39:07.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.349+0000 7fc365200640 1 --2- 172.21.15.107:0/2932134046 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3600ff3d0 0x7fc3600fad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:39:07.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.349+0000 7fc365c00640 1 -- 172.21.15.107:0/2932134046 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3600ff3d0 msgr2=0x7fc3600fad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:39:07.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.349+0000 7fc365c00640 1 --2- 172.21.15.107:0/2932134046 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3600ff3d0 0x7fc3600fad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:39:07.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.349+0000 7fc365c00640 1 -- 172.21.15.107:0/2932134046 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc34c005040 con 0x7fc360066350 2024-03-04T13:39:07.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.349+0000 7fc365200640 1 --2- 172.21.15.107:0/2932134046 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3600ff3d0 0x7fc3600fad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T13:39:07.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.349+0000 7fc365c00640 1 --2- 172.21.15.107:0/2932134046 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc360066350 0x7fc3600fc6c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc35400c810 tx=0x7fc3540043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:39:07.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.349+0000 7fc35d600640 1 -- 172.21.15.107:0/2932134046 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc35400a820 con 0x7fc360066350 2024-03-04T13:39:07.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.349+0000 7fc35d600640 1 -- 172.21.15.107:0/2932134046 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc35400ae50 con 0x7fc360066350 2024-03-04T13:39:07.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.349+0000 7fc35d600640 1 -- 172.21.15.107:0/2932134046 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc3540115c0 con 0x7fc360066350 2024-03-04T13:39:07.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.349+0000 7fc368600640 1 -- 172.21.15.107:0/2932134046 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3600fb5b0 con 0x7fc360066350 2024-03-04T13:39:07.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.350+0000 7fc368600640 1 -- 172.21.15.107:0/2932134046 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3600fbaa0 con 0x7fc360066350 2024-03-04T13:39:07.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.351+0000 7fc368600640 1 -- 172.21.15.107:0/2932134046 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc360064480 con 0x7fc360066350 2024-03-04T13:39:07.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.352+0000 7fc35d600640 1 -- 172.21.15.107:0/2932134046 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86623+0+0 (secure 0 0 0) 0x7fc354012420 con 0x7fc360066350 2024-03-04T13:39:07.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.352+0000 7fc35d600640 1 --2- 172.21.15.107:0/2932134046 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fc348069220 0x7fc34806b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:39:07.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.353+0000 7fc35d600640 1 -- 172.21.15.107:0/2932134046 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fc3540876a0 con 0x7fc360066350 2024-03-04T13:39:07.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.353+0000 7fc365200640 1 --2- 172.21.15.107:0/2932134046 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fc348069220 0x7fc34806b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:39:07.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.353+0000 7fc365200640 1 --2- 172.21.15.107:0/2932134046 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fc348069220 0x7fc34806b6d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc34c034040 tx=0x7fc34c009da0 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:39:07.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.357+0000 7fc35d600640 1 -- 172.21.15.107:0/2932134046 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fc354053a40 con 0x7fc360066350 2024-03-04T13:39:07.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.544+0000 7fc368600640 1 -- 172.21.15.107:0/2932134046 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7fc360106d80 con 0x7fc360066350 2024-03-04T13:39:07.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.544+0000 7fc35d600640 1 -- 172.21.15.107:0/2932134046 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v47) v1 ==== 107+0+11 (secure 0 0 0) 0x7fc354057060 con 0x7fc360066350 2024-03-04T13:39:07.539 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 32 2024-03-04T13:39:07.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.547+0000 7fc368600640 1 -- 172.21.15.107:0/2932134046 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fc348069220 msgr2=0x7fc34806b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:39:07.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.547+0000 7fc368600640 1 --2- 172.21.15.107:0/2932134046 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fc348069220 0x7fc34806b6d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc34c034040 tx=0x7fc34c009da0 comp rx=0 tx=0).stop 2024-03-04T13:39:07.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.547+0000 7fc368600640 1 -- 172.21.15.107:0/2932134046 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc360066350 msgr2=0x7fc3600fc6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:39:07.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.547+0000 7fc368600640 1 --2- 172.21.15.107:0/2932134046 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc360066350 0x7fc3600fc6c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc35400c810 tx=0x7fc3540043c0 comp rx=0 tx=0).stop 2024-03-04T13:39:07.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.547+0000 7fc368600640 1 -- 172.21.15.107:0/2932134046 shutdown_connections 2024-03-04T13:39:07.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.547+0000 7fc368600640 1 --2- 172.21.15.107:0/2932134046 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fc348069220 0x7fc34806b6d0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:39:07.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.547+0000 7fc368600640 1 --2- 172.21.15.107:0/2932134046 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc3600ff3d0 0x7fc3600fad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:39:07.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.547+0000 7fc368600640 1 --2- 172.21.15.107:0/2932134046 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc360066350 0x7fc3600fc6c0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:39:07.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.547+0000 7fc368600640 1 -- 172.21.15.107:0/2932134046 >> 172.21.15.107:0/2932134046 conn(0x7fc3600f6960 msgr2=0x7fc3600f8d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:39:07.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.547+0000 7fc368600640 1 -- 172.21.15.107:0/2932134046 shutdown_connections 2024-03-04T13:39:07.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.547+0000 7fc368600640 1 -- 172.21.15.107:0/2932134046 wait complete. 2024-03-04T13:39:07.559 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-03-04T13:39:07.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.765+0000 7fae2ea00640 1 -- 172.21.15.107:0/3434862693 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae280ff440 msgr2=0x7fae28101820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:39:07.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.765+0000 7fae2ea00640 1 --2- 172.21.15.107:0/3434862693 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae280ff440 0x7fae28101820 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fae140048d0 tx=0x7fae14009da0 comp rx=0 tx=0).stop 2024-03-04T13:39:07.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.765+0000 7fae2ea00640 1 -- 172.21.15.107:0/3434862693 shutdown_connections 2024-03-04T13:39:07.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.765+0000 7fae2ea00640 1 --2- 172.21.15.107:0/3434862693 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae280ff440 0x7fae28101820 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:39:07.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.765+0000 7fae2ea00640 1 --2- 172.21.15.107:0/3434862693 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fae28066310 0x7fae280fef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:39:07.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.765+0000 7fae2ea00640 1 -- 172.21.15.107:0/3434862693 >> 172.21.15.107:0/3434862693 conn(0x7fae280f69b0 msgr2=0x7fae280f8dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:39:07.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.765+0000 7fae2ea00640 1 -- 172.21.15.107:0/3434862693 shutdown_connections 2024-03-04T13:39:07.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.765+0000 7fae2ea00640 1 -- 172.21.15.107:0/3434862693 wait complete. 2024-03-04T13:39:07.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.766+0000 7fae2ea00640 1 Processor -- start 2024-03-04T13:39:07.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.766+0000 7fae2ea00640 1 -- start start 2024-03-04T13:39:07.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.766+0000 7fae2ea00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fae28066310 0x7fae2810edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:39:07.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.766+0000 7fae2ea00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae280ff440 0x7fae2810f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:39:07.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.766+0000 7fae2ea00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae2810f960 con 0x7fae280ff440 2024-03-04T13:39:07.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.766+0000 7fae2ea00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae281143b0 con 0x7fae28066310 2024-03-04T13:39:07.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.766+0000 7fae27400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae280ff440 0x7fae2810f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:39:07.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.767+0000 7fae27400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae280ff440 0x7fae2810f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57968/0 (socket says 172.21.15.107:57968) 2024-03-04T13:39:07.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.767+0000 7fae27400640 1 -- 172.21.15.107:0/3282012064 learned_addr learned my addr 172.21.15.107:0/3282012064 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:39:07.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.767+0000 7fae27e00640 1 --2- 172.21.15.107:0/3282012064 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fae28066310 0x7fae2810edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:39:07.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.767+0000 7fae27400640 1 -- 172.21.15.107:0/3282012064 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fae28066310 msgr2=0x7fae2810edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:39:07.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.767+0000 7fae27400640 1 --2- 172.21.15.107:0/3282012064 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fae28066310 0x7fae2810edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:39:07.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.767+0000 7fae27400640 1 -- 172.21.15.107:0/3282012064 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae14005040 con 0x7fae280ff440 2024-03-04T13:39:07.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.767+0000 7fae27400640 1 --2- 172.21.15.107:0/3282012064 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae280ff440 0x7fae2810f310 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fae1402bda0 tx=0x7fae14009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:39:07.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.767+0000 7fae1fe00640 1 -- 172.21.15.107:0/3282012064 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fae140094c0 con 0x7fae280ff440 2024-03-04T13:39:07.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.768+0000 7fae1fe00640 1 -- 172.21.15.107:0/3282012064 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fae14009620 con 0x7fae280ff440 2024-03-04T13:39:07.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.768+0000 7fae2ea00640 1 -- 172.21.15.107:0/3282012064 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae281144f0 con 0x7fae280ff440 2024-03-04T13:39:07.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.768+0000 7fae1fe00640 1 -- 172.21.15.107:0/3282012064 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fae140435f0 con 0x7fae280ff440 2024-03-04T13:39:07.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.768+0000 7fae2ea00640 1 -- 172.21.15.107:0/3282012064 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae28114980 con 0x7fae280ff440 2024-03-04T13:39:07.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.769+0000 7fae2ea00640 1 -- 172.21.15.107:0/3282012064 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae280f8450 con 0x7fae280ff440 2024-03-04T13:39:07.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.770+0000 7fae1fe00640 1 -- 172.21.15.107:0/3282012064 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86623+0+0 (secure 0 0 0) 0x7fae1402f070 con 0x7fae280ff440 2024-03-04T13:39:07.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.771+0000 7fae1fe00640 1 --2- 172.21.15.107:0/3282012064 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fae0c069220 0x7fae0c06b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:39:07.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.771+0000 7fae1fe00640 1 -- 172.21.15.107:0/3282012064 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fae140a9110 con 0x7fae280ff440 2024-03-04T13:39:07.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.771+0000 7fae27e00640 1 --2- 172.21.15.107:0/3282012064 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fae0c069220 0x7fae0c06b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:39:07.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.772+0000 7fae27e00640 1 --2- 172.21.15.107:0/3282012064 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fae0c069220 0x7fae0c06b6d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fae18009fd0 tx=0x7fae18009e50 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:39:07.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.776+0000 7fae1fe00640 1 -- 172.21.15.107:0/3282012064 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fae14075370 con 0x7fae280ff440 2024-03-04T13:39:07.836 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:39:07 smithi107 ceph-mon[28739]: from='client.? 172.21.15.107:0/1710681955' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T13:39:07.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:39:07 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:39:07.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:39:07 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:39:07.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:39:07 smithi107 ceph-mon[28739]: from='client.? 172.21.15.107:0/3745065552' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-04T13:39:07.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:39:07 smithi107 ceph-mon[28739]: from='client.? 172.21.15.107:0/2932134046' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-04T13:39:07.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.960+0000 7fae2ea00640 1 -- 172.21.15.107:0/3282012064 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7fae28104e40 con 0x7fae280ff440 2024-03-04T13:39:07.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.962+0000 7fae1fe00640 1 -- 172.21.15.107:0/3282012064 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v47) v1 ==== 107+0+11 (secure 0 0 0) 0x7fae14078990 con 0x7fae280ff440 2024-03-04T13:39:07.956 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 32 2024-03-04T13:39:07.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.964+0000 7fae2ea00640 1 -- 172.21.15.107:0/3282012064 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fae0c069220 msgr2=0x7fae0c06b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:39:07.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.964+0000 7fae2ea00640 1 --2- 172.21.15.107:0/3282012064 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fae0c069220 0x7fae0c06b6d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fae18009fd0 tx=0x7fae18009e50 comp rx=0 tx=0).stop 2024-03-04T13:39:07.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.964+0000 7fae2ea00640 1 -- 172.21.15.107:0/3282012064 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae280ff440 msgr2=0x7fae2810f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:39:07.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.964+0000 7fae2ea00640 1 --2- 172.21.15.107:0/3282012064 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae280ff440 0x7fae2810f310 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fae1402bda0 tx=0x7fae14009da0 comp rx=0 tx=0).stop 2024-03-04T13:39:07.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.964+0000 7fae2ea00640 1 -- 172.21.15.107:0/3282012064 shutdown_connections 2024-03-04T13:39:07.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.964+0000 7fae2ea00640 1 --2- 172.21.15.107:0/3282012064 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae280ff440 0x7fae2810f310 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:39:07.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.964+0000 7fae2ea00640 1 --2- 172.21.15.107:0/3282012064 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fae0c069220 0x7fae0c06b6d0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:39:07.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.964+0000 7fae2ea00640 1 --2- 172.21.15.107:0/3282012064 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fae28066310 0x7fae2810edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:39:07.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.964+0000 7fae2ea00640 1 -- 172.21.15.107:0/3282012064 >> 172.21.15.107:0/3282012064 conn(0x7fae280f69b0 msgr2=0x7fae281000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:39:07.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.964+0000 7fae2ea00640 1 -- 172.21.15.107:0/3282012064 shutdown_connections 2024-03-04T13:39:07.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:07.964+0000 7fae2ea00640 1 -- 172.21.15.107:0/3282012064 wait complete. 2024-03-04T13:39:07.975 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T13:39:08.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:39:07 smithi169 ceph-mon[36313]: from='client.? 172.21.15.107:0/1710681955' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T13:39:08.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:39:07 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:39:08.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:39:07 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:39:08.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:39:07 smithi169 ceph-mon[36313]: from='client.? 172.21.15.107:0/3745065552' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-04T13:39:08.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:39:07 smithi169 ceph-mon[36313]: from='client.? 172.21.15.107:0/2932134046' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-04T13:39:08.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:08.185+0000 7f87be800640 1 -- 172.21.15.107:0/2209498746 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f87b80ff6c0 msgr2=0x7f87b80ffb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:39:08.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:08.185+0000 7f87be800640 1 --2- 172.21.15.107:0/2209498746 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f87b80ff6c0 0x7f87b80ffb30 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f87a40048d0 tx=0x7f87a4009da0 comp rx=0 tx=0).stop 2024-03-04T13:39:08.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:08.185+0000 7f87be800640 1 -- 172.21.15.107:0/2209498746 shutdown_connections 2024-03-04T13:39:08.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:08.185+0000 7f87be800640 1 --2- 172.21.15.107:0/2209498746 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f87b80ff6c0 0x7f87b80ffb30 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:39:08.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:08.185+0000 7f87be800640 1 --2- 172.21.15.107:0/2209498746 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f87b80fe490 0x7f87b80fe8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:39:08.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:08.185+0000 7f87be800640 1 -- 172.21.15.107:0/2209498746 >> 172.21.15.107:0/2209498746 conn(0x7f87b80f9c50 msgr2=0x7f87b80fc030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:39:08.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:08.185+0000 7f87be800640 1 -- 172.21.15.107:0/2209498746 shutdown_connections 2024-03-04T13:39:08.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:08.186+0000 7f87be800640 1 -- 172.21.15.107:0/2209498746 wait complete. 2024-03-04T13:39:08.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:08.186+0000 7f87be800640 1 Processor -- start 2024-03-04T13:39:08.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:08.186+0000 7f87be800640 1 -- start start 2024-03-04T13:39:08.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:08.187+0000 7f87be800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f87b80fe490 0x7f87b810ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:39:08.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:08.187+0000 7f87be800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f87b80ff6c0 0x7f87b810f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:39:08.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:08.187+0000 7f87be800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b810f970 con 0x7f87b80ff6c0 2024-03-04T13:39:08.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:08.187+0000 7f87be800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b81143c0 con 0x7f87b80fe490 2024-03-04T13:39:08.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:08.187+0000 7f87b6a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f87b80ff6c0 0x7f87b810f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:39:08.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:08.187+0000 7f87b6a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f87b80ff6c0 0x7f87b810f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42266/0 (socket says 172.21.15.107:42266) 2024-03-04T13:39:08.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:08.187+0000 7f87b6a00640 1 -- 172.21.15.107:0/654550155 learned_addr learned my addr 172.21.15.107:0/654550155 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:39:08.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:08.187+0000 7f87b7400640 1 --2- 172.21.15.107:0/654550155 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f87b80fe490 0x7f87b810ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:39:08.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:08.187+0000 7f87b6a00640 1 -- 172.21.15.107:0/654550155 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f87b80fe490 msgr2=0x7f87b810ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:39:08.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:08.187+0000 7f87b6a00640 1 --2- 172.21.15.107:0/654550155 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f87b80fe490 0x7f87b810ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:39:08.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:08.187+0000 7f87b6a00640 1 -- 172.21.15.107:0/654550155 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87a4005040 con 0x7f87b80ff6c0 2024-03-04T13:39:08.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:08.188+0000 7f87b6a00640 1 --2- 172.21.15.107:0/654550155 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f87b80ff6c0 0x7f87b810f320 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f87a4007da0 tx=0x7f87a4009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:39:08.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:08.188+0000 7f87af400640 1 -- 172.21.15.107:0/654550155 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f87a40094c0 con 0x7f87b80ff6c0 2024-03-04T13:39:08.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:08.188+0000 7f87af400640 1 -- 172.21.15.107:0/654550155 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f87a4009620 con 0x7f87b80ff6c0 2024-03-04T13:39:08.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:08.188+0000 7f87af400640 1 -- 172.21.15.107:0/654550155 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f87a403b620 con 0x7f87b80ff6c0 2024-03-04T13:39:08.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:08.188+0000 7f87be800640 1 -- 172.21.15.107:0/654550155 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87b8114500 con 0x7f87b80ff6c0 2024-03-04T13:39:08.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:08.188+0000 7f87be800640 1 -- 172.21.15.107:0/654550155 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87b81149f0 con 0x7f87b80ff6c0 2024-03-04T13:39:08.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:08.189+0000 7f87be800640 1 -- 172.21.15.107:0/654550155 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87b8107010 con 0x7f87b80ff6c0 2024-03-04T13:39:08.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:08.190+0000 7f87af400640 1 -- 172.21.15.107:0/654550155 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86623+0+0 (secure 0 0 0) 0x7f87a402f070 con 0x7f87b80ff6c0 2024-03-04T13:39:08.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:08.191+0000 7f87af400640 1 --2- 172.21.15.107:0/654550155 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f879c069210 0x7f879c06b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:39:08.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:08.191+0000 7f87af400640 1 -- 172.21.15.107:0/654550155 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f87a40a9810 con 0x7f87b80ff6c0 2024-03-04T13:39:08.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:08.191+0000 7f87b7400640 1 --2- 172.21.15.107:0/654550155 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f879c069210 0x7f879c06b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:39:08.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:08.192+0000 7f87b7400640 1 --2- 172.21.15.107:0/654550155 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f879c069210 0x7f879c06b6c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f87b80ff520 tx=0x7f87a800b040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:39:08.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:08.195+0000 7f87af400640 1 -- 172.21.15.107:0/654550155 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f87a4075a50 con 0x7f87b80ff6c0 2024-03-04T13:39:08.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:08.416+0000 7f87be800640 1 -- 172.21.15.107:0/654550155 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f87b8114bd0 con 0x7f87b80ff6c0 2024-03-04T13:39:08.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:08.417+0000 7f87af400640 1 -- 172.21.15.107:0/654550155 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4708 (secure 0 0 0) 0x7f87a4079070 con 0x7f87b80ff6c0 2024-03-04T13:39:08.412 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:39:08.412 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14480,"name":"cephfs.smithi107.riexmm","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6827/3261622517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3261622517},{"type":"v1","addr":"172.21.15.107:6827","nonce":3261622517}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24259,"name":"cephfs.smithi169.bxitzk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.169:6827/3696090493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3696090493},{"type":"v1","addr":"172.21.15.169:6827","nonce":3696090493}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16}],"filesystems":[{"mdsmap":{"epoch":15,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-04T13:38:30.854004+0000","modified":"2024-03-04T13:38:50.015989+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24247},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14470":{"gid":14470,"name":"cephfs.smithi107.thycho","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":2,"addr":"172.21.15.107:6829/1985577060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":1985577060},{"type":"v1","addr":"172.21.15.107:6829","nonce":1985577060}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24247":{"gid":24247,"name":"cephfs.smithi169.jusudl","rank":0,"incarnation":11,"state":"up:active","state_seq":6,"addr":"172.21.15.169:6825/2994866076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2994866076},{"type":"v1","addr":"172.21.15.169:6825","nonce":2994866076}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":1},"id":1}]} 2024-03-04T13:39:08.412 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 16 2024-03-04T13:39:08.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:08.420+0000 7f87be800640 1 -- 172.21.15.107:0/654550155 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f879c069210 msgr2=0x7f879c06b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:39:08.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:08.420+0000 7f87be800640 1 --2- 172.21.15.107:0/654550155 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f879c069210 0x7f879c06b6c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f87b80ff520 tx=0x7f87a800b040 comp rx=0 tx=0).stop 2024-03-04T13:39:08.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:08.420+0000 7f87be800640 1 -- 172.21.15.107:0/654550155 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f87b80ff6c0 msgr2=0x7f87b810f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:39:08.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:08.420+0000 7f87be800640 1 --2- 172.21.15.107:0/654550155 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f87b80ff6c0 0x7f87b810f320 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f87a4007da0 tx=0x7f87a4009da0 comp rx=0 tx=0).stop 2024-03-04T13:39:08.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:08.420+0000 7f87be800640 1 -- 172.21.15.107:0/654550155 shutdown_connections 2024-03-04T13:39:08.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:08.420+0000 7f87be800640 1 --2- 172.21.15.107:0/654550155 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f87b80ff6c0 0x7f87b810f320 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:39:08.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:08.420+0000 7f87be800640 1 --2- 172.21.15.107:0/654550155 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f879c069210 0x7f879c06b6c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:39:08.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:08.420+0000 7f87be800640 1 --2- 172.21.15.107:0/654550155 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f87b80fe490 0x7f87b810ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:39:08.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:08.420+0000 7f87be800640 1 -- 172.21.15.107:0/654550155 >> 172.21.15.107:0/654550155 conn(0x7f87b80f9c50 msgr2=0x7f87b81028f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:39:08.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:08.420+0000 7f87be800640 1 -- 172.21.15.107:0/654550155 shutdown_connections 2024-03-04T13:39:08.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:08.421+0000 7f87be800640 1 -- 172.21.15.107:0/654550155 wait complete. 2024-03-04T13:39:08.432 DEBUG:tasks.fs:fs fscid=1,name=cephfs state = {'epoch': 15, 'max_mds': 1, 'flags': 50} 2024-03-04T13:39:08.432 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-03-04T13:39:08.462 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-03-04T13:39:08.463 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.1': {}} 2024-03-04T13:39:08.463 INFO:tasks.ceph_fuse:client.0 config is {} 2024-03-04T13:39:08.463 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-04T13:39:08.463 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-04T13:39:08.463 INFO:tasks.ceph_fuse:client.1 config is {} 2024-03-04T13:39:08.463 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-04T13:39:08.463 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-04T13:39:08.464 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T13:39:08.464 DEBUG:teuthology.orchestra.run.smithi169:> ip netns list 2024-03-04T13:39:08.524 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T13:39:08.524 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link delete ceph-brx 2024-03-04T13:39:08.627 INFO:teuthology.orchestra.run.smithi169.stderr:Cannot find device "ceph-brx" 2024-03-04T13:39:08.631 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-04T13:39:08.632 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T13:39:08.632 DEBUG:teuthology.orchestra.run.smithi107:> ip netns list 2024-03-04T13:39:08.690 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T13:39:08.690 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link delete ceph-brx 2024-03-04T13:39:08.762 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:39:08 smithi107 ceph-mon[28739]: pgmap v235: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:39:08.762 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:39:08 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:39:08.762 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:39:08 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:39:08.762 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:39:08 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:39:08.762 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:39:08 smithi107 ceph-mon[28739]: from='client.? 172.21.15.107:0/3282012064' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-04T13:39:08.763 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:39:08 smithi107 ceph-mon[28739]: from='client.? 172.21.15.107:0/654550155' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T13:39:08.778 INFO:teuthology.orchestra.run.smithi107.stderr:Cannot find device "ceph-brx" 2024-03-04T13:39:08.782 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-04T13:39:08.782 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-03-04T13:39:08.783 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 -- ceph fs ls 2024-03-04T13:39:09.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:39:08 smithi169 ceph-mon[36313]: pgmap v235: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:39:09.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:39:08 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:39:09.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:39:08 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:39:09.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:39:08 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:39:09.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:39:08 smithi169 ceph-mon[36313]: from='client.? 172.21.15.107:0/3282012064' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-04T13:39:09.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:39:08 smithi169 ceph-mon[36313]: from='client.? 172.21.15.107:0/654550155' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T13:39:09.240 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/mon.smithi107/config 2024-03-04T13:39:10.906 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:39:10 smithi169 ceph-mon[36313]: pgmap v236: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:39:10.920 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:39:10 smithi107 ceph-mon[28739]: pgmap v236: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:39:11.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:11.470+0000 7ff8e8400700 1 -- 172.21.15.107:0/3086766400 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8e0105170 msgr2=0x7ff8e0105540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:39:11.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:11.470+0000 7ff8e8400700 1 --2- 172.21.15.107:0/3086766400 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8e0105170 0x7ff8e0105540 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff8d4004940 tx=0x7ff8d4009e30 comp rx=0 tx=0).stop 2024-03-04T13:39:11.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:11.471+0000 7ff8e8400700 1 -- 172.21.15.107:0/3086766400 shutdown_connections 2024-03-04T13:39:11.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:11.471+0000 7ff8e8400700 1 --2- 172.21.15.107:0/3086766400 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff8e0105b10 0x7ff8e00ff180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:39:11.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:11.471+0000 7ff8e8400700 1 --2- 172.21.15.107:0/3086766400 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8e0105170 0x7ff8e0105540 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:39:11.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:11.471+0000 7ff8e8400700 1 -- 172.21.15.107:0/3086766400 >> 172.21.15.107:0/3086766400 conn(0x7ff8e00face0 msgr2=0x7ff8e00fd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:39:11.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:11.471+0000 7ff8e8400700 1 -- 172.21.15.107:0/3086766400 shutdown_connections 2024-03-04T13:39:11.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:11.471+0000 7ff8e8400700 1 -- 172.21.15.107:0/3086766400 wait complete. 2024-03-04T13:39:11.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:11.472+0000 7ff8e8400700 1 Processor -- start 2024-03-04T13:39:11.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:11.472+0000 7ff8e8400700 1 -- start start 2024-03-04T13:39:11.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:11.473+0000 7ff8e8400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8e0105170 0x7ff8e0115c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:39:11.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:11.473+0000 7ff8e8400700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff8e0105b10 0x7ff8e01161c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:39:11.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:11.473+0000 7ff8e8400700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e01168a0 con 0x7ff8e0105170 2024-03-04T13:39:11.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:11.473+0000 7ff8e8400700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e010fb40 con 0x7ff8e0105b10 2024-03-04T13:39:11.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:11.473+0000 7ff8e5a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8e0105170 0x7ff8e0115c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:39:11.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:11.473+0000 7ff8e5a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8e0105170 0x7ff8e0115c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42286/0 (socket says 172.21.15.107:42286) 2024-03-04T13:39:11.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:11.473+0000 7ff8e5a00700 1 -- 172.21.15.107:0/2556324871 learned_addr learned my addr 172.21.15.107:0/2556324871 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:39:11.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:11.473+0000 7ff8e5000700 1 --2- 172.21.15.107:0/2556324871 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff8e0105b10 0x7ff8e01161c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:39:11.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:11.474+0000 7ff8e5a00700 1 -- 172.21.15.107:0/2556324871 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff8e0105b10 msgr2=0x7ff8e01161c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:39:11.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:11.474+0000 7ff8e5a00700 1 --2- 172.21.15.107:0/2556324871 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff8e0105b10 0x7ff8e01161c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:39:11.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:11.474+0000 7ff8e5a00700 1 -- 172.21.15.107:0/2556324871 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8d4005040 con 0x7ff8e0105170 2024-03-04T13:39:11.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:11.474+0000 7ff8e5000700 1 --2- 172.21.15.107:0/2556324871 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff8e0105b10 0x7ff8e01161c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T13:39:11.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:11.474+0000 7ff8e5a00700 1 --2- 172.21.15.107:0/2556324871 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8e0105170 0x7ff8e0115c80 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff8d4009150 tx=0x7ff8d4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:39:11.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:11.474+0000 7ff8dd600700 1 -- 172.21.15.107:0/2556324871 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff8d400e040 con 0x7ff8e0105170 2024-03-04T13:39:11.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:11.474+0000 7ff8dd600700 1 -- 172.21.15.107:0/2556324871 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff8d400bbf0 con 0x7ff8e0105170 2024-03-04T13:39:11.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:11.474+0000 7ff8dd600700 1 -- 172.21.15.107:0/2556324871 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff8d401c630 con 0x7ff8e0105170 2024-03-04T13:39:11.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:11.474+0000 7ff8e8400700 1 -- 172.21.15.107:0/2556324871 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8e010fdc0 con 0x7ff8e0105170 2024-03-04T13:39:11.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:11.474+0000 7ff8e8400700 1 -- 172.21.15.107:0/2556324871 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8e0110300 con 0x7ff8e0105170 2024-03-04T13:39:11.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:11.476+0000 7ff8e8400700 1 -- 172.21.15.107:0/2556324871 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8e00fe900 con 0x7ff8e0105170 2024-03-04T13:39:11.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:11.476+0000 7ff8dd600700 1 -- 172.21.15.107:0/2556324871 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86623+0+0 (secure 0 0 0) 0x7ff8d400bd60 con 0x7ff8e0105170 2024-03-04T13:39:11.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:11.477+0000 7ff8dd600700 1 --2- 172.21.15.107:0/2556324871 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7ff8c80692b0 0x7ff8c806b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:39:11.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:11.477+0000 7ff8dd600700 1 -- 172.21.15.107:0/2556324871 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5550+0+0 (secure 0 0 0) 0x7ff8d40885f0 con 0x7ff8e0105170 2024-03-04T13:39:11.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:11.477+0000 7ff8e5000700 1 --2- 172.21.15.107:0/2556324871 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7ff8c80692b0 0x7ff8c806b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:39:11.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:11.478+0000 7ff8e5000700 1 --2- 172.21.15.107:0/2556324871 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7ff8c80692b0 0x7ff8c806b760 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc006fd0 tx=0x7ff8cc008040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:39:11.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:11.482+0000 7ff8dd600700 1 -- 172.21.15.107:0/2556324871 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7ff8d4058340 con 0x7ff8e0105170 2024-03-04T13:39:11.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:11.737+0000 7ff8e8400700 1 -- 172.21.15.107:0/2556324871 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff8e0111290 con 0x7ff8e0105170 2024-03-04T13:39:11.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:11.738+0000 7ff8dd600700 1 -- 172.21.15.107:0/2556324871 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) v1 ==== 53+0+83 (secure 0 0 0) 0x7ff8d4057ed0 con 0x7ff8e0105170 2024-03-04T13:39:11.732 INFO:teuthology.orchestra.run.smithi107.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-03-04T13:39:11.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:11.741+0000 7ff8e8400700 1 -- 172.21.15.107:0/2556324871 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7ff8c80692b0 msgr2=0x7ff8c806b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:39:11.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:11.741+0000 7ff8e8400700 1 --2- 172.21.15.107:0/2556324871 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7ff8c80692b0 0x7ff8c806b760 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc006fd0 tx=0x7ff8cc008040 comp rx=0 tx=0).stop 2024-03-04T13:39:11.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:11.741+0000 7ff8e8400700 1 -- 172.21.15.107:0/2556324871 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8e0105170 msgr2=0x7ff8e0115c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:39:11.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:11.741+0000 7ff8e8400700 1 --2- 172.21.15.107:0/2556324871 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8e0105170 0x7ff8e0115c80 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff8d4009150 tx=0x7ff8d4009e30 comp rx=0 tx=0).stop 2024-03-04T13:39:11.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:11.741+0000 7ff8e8400700 1 -- 172.21.15.107:0/2556324871 shutdown_connections 2024-03-04T13:39:11.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:11.741+0000 7ff8e8400700 1 --2- 172.21.15.107:0/2556324871 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7ff8c80692b0 0x7ff8c806b760 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:39:11.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:11.741+0000 7ff8e8400700 1 --2- 172.21.15.107:0/2556324871 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8e0105170 0x7ff8e0115c80 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:39:11.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:11.741+0000 7ff8e8400700 1 --2- 172.21.15.107:0/2556324871 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff8e0105b10 0x7ff8e01161c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:39:11.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:11.741+0000 7ff8e8400700 1 -- 172.21.15.107:0/2556324871 >> 172.21.15.107:0/2556324871 conn(0x7ff8e00face0 msgr2=0x7ff8e00fc630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:39:11.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:11.742+0000 7ff8e8400700 1 -- 172.21.15.107:0/2556324871 shutdown_connections 2024-03-04T13:39:11.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:11.742+0000 7ff8e8400700 1 -- 172.21.15.107:0/2556324871 wait complete. 2024-03-04T13:39:12.637 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-04T13:39:12.637 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-04T13:39:12.638 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi107.front.sepia.ceph.com 2024-03-04T13:39:12.638 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-04T13:39:12.638 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-04T13:39:12.638 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-04T13:39:12.638 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-04T13:39:12.638 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-04T13:39:12.638 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-04T13:39:12.638 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T13:39:12.638 DEBUG:teuthology.orchestra.run.smithi107:> ip addr 2024-03-04T13:39:12.693 INFO:teuthology.orchestra.run.smithi107.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-04T13:39:12.694 INFO:teuthology.orchestra.run.smithi107.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-04T13:39:12.694 INFO:teuthology.orchestra.run.smithi107.stdout: inet 127.0.0.1/8 scope host lo 2024-03-04T13:39:12.694 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2024-03-04T13:39:12.694 INFO:teuthology.orchestra.run.smithi107.stdout: inet6 ::1/128 scope host 2024-03-04T13:39:12.694 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2024-03-04T13:39:12.694 INFO:teuthology.orchestra.run.smithi107.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-04T13:39:12.694 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 0c:c4:7a:d8:a2:0a brd ff:ff:ff:ff:ff:ff 2024-03-04T13:39:12.694 INFO:teuthology.orchestra.run.smithi107.stdout: altname enp7s0f0 2024-03-04T13:39:12.694 INFO:teuthology.orchestra.run.smithi107.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-04T13:39:12.694 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 0c:c4:7a:d8:a2:0b brd ff:ff:ff:ff:ff:ff 2024-03-04T13:39:12.694 INFO:teuthology.orchestra.run.smithi107.stdout: altname enp7s0f1 2024-03-04T13:39:12.695 INFO:teuthology.orchestra.run.smithi107.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-04T13:39:12.695 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 0c:c4:7a:8f:47:ea brd ff:ff:ff:ff:ff:ff 2024-03-04T13:39:12.695 INFO:teuthology.orchestra.run.smithi107.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-04T13:39:12.695 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 0c:c4:7a:8f:47:eb brd ff:ff:ff:ff:ff:ff 2024-03-04T13:39:12.695 INFO:teuthology.orchestra.run.smithi107.stdout: inet 172.21.15.107/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-04T13:39:12.695 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft 41847sec preferred_lft 41847sec 2024-03-04T13:39:12.695 INFO:teuthology.orchestra.run.smithi107.stdout: inet6 fe80::ec4:7aff:fe8f:47eb/64 scope link 2024-03-04T13:39:12.695 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2024-03-04T13:39:12.696 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-04T13:39:12.696 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T13:39:12.696 DEBUG:teuthology.orchestra.run.smithi107:> (cd / && exec bash -c ' 2024-03-04T13:39:12.696 DEBUG:teuthology.orchestra.run.smithi107:> set -e 2024-03-04T13:39:12.696 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link add name ceph-brx type bridge 2024-03-04T13:39:12.696 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip addr flush dev ceph-brx 2024-03-04T13:39:12.697 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link set ceph-brx up 2024-03-04T13:39:12.697 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-04T13:39:12.697 DEBUG:teuthology.orchestra.run.smithi107:> ') 2024-03-04T13:39:12.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:39:12 smithi107 ceph-mon[28739]: pgmap v237: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:39:12.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:39:12 smithi107 ceph-mon[28739]: from='client.? 172.21.15.107:0/2556324871' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-04T13:39:12.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:39:12 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:39:12.854 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:39:12 smithi169 ceph-mon[36313]: pgmap v237: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:39:12.854 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:39:12 smithi169 ceph-mon[36313]: from='client.? 172.21.15.107:0/2556324871' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-04T13:39:12.854 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:39:12 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:39:13.132 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T13:39:13.132 DEBUG:teuthology.orchestra.run.smithi107:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-04T13:39:13.211 INFO:teuthology.orchestra.run.smithi107.stdout:1 2024-03-04T13:39:13.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T13:39:13.215 DEBUG:teuthology.orchestra.run.smithi107:> route 2024-03-04T13:39:13.271 INFO:teuthology.orchestra.run.smithi107.stdout:Kernel IP routing table 2024-03-04T13:39:13.271 INFO:teuthology.orchestra.run.smithi107.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-04T13:39:13.272 INFO:teuthology.orchestra.run.smithi107.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-04T13:39:13.272 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-04T13:39:13.272 INFO:teuthology.orchestra.run.smithi107.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-04T13:39:13.272 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T13:39:13.272 DEBUG:teuthology.orchestra.run.smithi107:> (cd / && exec bash -c ' 2024-03-04T13:39:13.272 DEBUG:teuthology.orchestra.run.smithi107:> set -e 2024-03-04T13:39:13.273 DEBUG:teuthology.orchestra.run.smithi107:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-04T13:39:13.273 DEBUG:teuthology.orchestra.run.smithi107:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-04T13:39:13.273 DEBUG:teuthology.orchestra.run.smithi107:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-04T13:39:13.273 DEBUG:teuthology.orchestra.run.smithi107:> ') 2024-03-04T13:39:14.034 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T13:39:14.034 DEBUG:teuthology.orchestra.run.smithi107:> ip netns list 2024-03-04T13:39:14.090 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T13:39:14.091 DEBUG:teuthology.orchestra.run.smithi107:> ip netns list-id 2024-03-04T13:39:14.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T13:39:14.140 DEBUG:teuthology.orchestra.run.smithi107:> (cd / && exec bash -c ' 2024-03-04T13:39:14.140 DEBUG:teuthology.orchestra.run.smithi107:> set -e 2024-03-04T13:39:14.140 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-04T13:39:14.140 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-03-04T13:39:14.140 DEBUG:teuthology.orchestra.run.smithi107:> ') 2024-03-04T13:39:14.282 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-03-04T13:39:14.282 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T13:39:14.282 DEBUG:teuthology.orchestra.run.smithi107:> (cd / && exec bash -c ' 2024-03-04T13:39:14.282 DEBUG:teuthology.orchestra.run.smithi107:> set -e 2024-03-04T13:39:14.282 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-03-04T13:39:14.283 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-04T13:39:14.283 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-03-04T13:39:14.283 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-03-04T13:39:14.283 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-03-04T13:39:14.283 DEBUG:teuthology.orchestra.run.smithi107:> ') 2024-03-04T13:39:14.573 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T13:39:14.574 DEBUG:teuthology.orchestra.run.smithi107:> (cd / && exec bash -c ' 2024-03-04T13:39:14.574 DEBUG:teuthology.orchestra.run.smithi107:> set -e 2024-03-04T13:39:14.574 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link set brx.0 up 2024-03-04T13:39:14.574 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link set dev brx.0 master ceph-brx 2024-03-04T13:39:14.574 DEBUG:teuthology.orchestra.run.smithi107:> ') 2024-03-04T13:39:14.740 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-03-04T13:39:14.740 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-04T13:39:14.741 DEBUG:teuthology.orchestra.run.smithi107:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-04T13:39:14.786 INFO:teuthology.orchestra.run.smithi107.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-04T13:39:14.786 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-04T13:39:14.787 DEBUG:teuthology.orchestra.run.smithi107:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-04T13:39:14.880 DEBUG:teuthology.orchestra.run.smithi107:> sudo modprobe fuse 2024-03-04T13:39:14.949 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:39:14 smithi107 ceph-mon[28739]: pgmap v238: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:39:14.962 DEBUG:teuthology.orchestra.run.smithi107:> cat /proc/self/mounts | awk '{print $2}' 2024-03-04T13:39:15.008 INFO:teuthology.orchestra.run.smithi107.stdout:/proc 2024-03-04T13:39:15.008 INFO:teuthology.orchestra.run.smithi107.stdout:/sys 2024-03-04T13:39:15.008 INFO:teuthology.orchestra.run.smithi107.stdout:/dev 2024-03-04T13:39:15.008 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/kernel/security 2024-03-04T13:39:15.008 INFO:teuthology.orchestra.run.smithi107.stdout:/dev/shm 2024-03-04T13:39:15.008 INFO:teuthology.orchestra.run.smithi107.stdout:/dev/pts 2024-03-04T13:39:15.008 INFO:teuthology.orchestra.run.smithi107.stdout:/run 2024-03-04T13:39:15.008 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/fs/cgroup 2024-03-04T13:39:15.009 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/fs/pstore 2024-03-04T13:39:15.009 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/fs/bpf 2024-03-04T13:39:15.009 INFO:teuthology.orchestra.run.smithi107.stdout:/ 2024-03-04T13:39:15.009 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/fs/selinux 2024-03-04T13:39:15.009 INFO:teuthology.orchestra.run.smithi107.stdout:/proc/sys/fs/binfmt_misc 2024-03-04T13:39:15.009 INFO:teuthology.orchestra.run.smithi107.stdout:/dev/hugepages 2024-03-04T13:39:15.009 INFO:teuthology.orchestra.run.smithi107.stdout:/dev/mqueue 2024-03-04T13:39:15.009 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/kernel/debug 2024-03-04T13:39:15.009 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/kernel/tracing 2024-03-04T13:39:15.009 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/fs/fuse/connections 2024-03-04T13:39:15.009 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/kernel/config 2024-03-04T13:39:15.010 INFO:teuthology.orchestra.run.smithi107.stdout:/run/credentials/systemd-sysctl.service 2024-03-04T13:39:15.010 INFO:teuthology.orchestra.run.smithi107.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-04T13:39:15.010 INFO:teuthology.orchestra.run.smithi107.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-04T13:39:15.010 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/nfs/rpc_pipefs 2024-03-04T13:39:15.010 INFO:teuthology.orchestra.run.smithi107.stdout:/run/user/1000 2024-03-04T13:39:15.010 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/ceph 2024-03-04T13:39:15.010 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay 2024-03-04T13:39:15.010 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/28ce5b8ca75064ab44b164310a74fba35dcc0ced8dce1016843ef0bad81445f0/merged 2024-03-04T13:39:15.010 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/d955db762285aaeb18748b15ec066233216a7f7aa63988802eb294972e7876a4/merged 2024-03-04T13:39:15.010 INFO:teuthology.orchestra.run.smithi107.stdout:/run/user/0 2024-03-04T13:39:15.010 INFO:teuthology.orchestra.run.smithi107.stdout:/proc/sys/fs/binfmt_misc 2024-03-04T13:39:15.010 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/e42b2877e4f8a64c4ea0a334ff01569b03d9861562997bb2b4f3d2a879b0b2b9/merged 2024-03-04T13:39:15.010 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/2aae4ee1aea22f5ed0eb0a7f0267ebbce5d653380707f0a6da540d32d26d11cc/merged 2024-03-04T13:39:15.010 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/dc6944013a88995fff9883691b329a21d3cfaf9501ecc579bd12dec00765f1c0/merged 2024-03-04T13:39:15.011 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/a1e2d2c824eb88d36d79b1c3d93eb4e79d38c2c7f9869826e6b139456d947d60/merged 2024-03-04T13:39:15.011 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/aee8422d922f5a6fc23d360282a7288cb620ac0c064e6f107a81ab138e9952e2/merged 2024-03-04T13:39:15.011 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/f38e26dba0edb0df8cf42e6ccc54041200effdbf096ea5c732ea0f9d63d4277f/merged 2024-03-04T13:39:15.011 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/172a96860704d88f2d56123e353cc97b69542c7f41426ab9988d020e1419ad9c/merged 2024-03-04T13:39:15.011 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/30776d7db4aae543a5572f17f58acdb4964545b6f1814172ffd393997158df3c/merged 2024-03-04T13:39:15.011 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/248dddd27909f8c57548cfe1f30c0b15d29c7a02294c70ebbde91e6bf90ebc8f/merged 2024-03-04T13:39:15.011 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/e0510f246cdf32d97e39284e77a6c994fa8c275a70f8d9671472296499b34f70/merged 2024-03-04T13:39:15.011 INFO:teuthology.orchestra.run.smithi107.stdout:/run/netns 2024-03-04T13:39:15.011 INFO:teuthology.orchestra.run.smithi107.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-04T13:39:15.011 INFO:teuthology.orchestra.run.smithi107.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-04T13:39:15.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T13:39:15.012 DEBUG:teuthology.orchestra.run.smithi107:> ls /sys/fs/fuse/connections 2024-03-04T13:39:15.103 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:39:14 smithi169 ceph-mon[36313]: pgmap v238: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:39:15.106 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-04T13:39:15.106 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-03-04T13:39:15.110 DEBUG:teuthology.orchestra.run.smithi107:> sudo modprobe fuse 2024-03-04T13:39:15.186 DEBUG:teuthology.orchestra.run.smithi107:> cat /proc/self/mounts | awk '{print $2}' 2024-03-04T13:39:15.233 INFO:teuthology.orchestra.run.smithi107.stdout:/proc 2024-03-04T13:39:15.233 INFO:teuthology.orchestra.run.smithi107.stdout:/sys 2024-03-04T13:39:15.233 INFO:teuthology.orchestra.run.smithi107.stdout:/dev 2024-03-04T13:39:15.233 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/kernel/security 2024-03-04T13:39:15.233 INFO:teuthology.orchestra.run.smithi107.stdout:/dev/shm 2024-03-04T13:39:15.233 INFO:teuthology.orchestra.run.smithi107.stdout:/dev/pts 2024-03-04T13:39:15.233 INFO:teuthology.orchestra.run.smithi107.stdout:/run 2024-03-04T13:39:15.233 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/fs/cgroup 2024-03-04T13:39:15.233 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/fs/pstore 2024-03-04T13:39:15.234 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/fs/bpf 2024-03-04T13:39:15.234 INFO:teuthology.orchestra.run.smithi107.stdout:/ 2024-03-04T13:39:15.234 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/fs/selinux 2024-03-04T13:39:15.234 INFO:teuthology.orchestra.run.smithi107.stdout:/proc/sys/fs/binfmt_misc 2024-03-04T13:39:15.234 INFO:teuthology.orchestra.run.smithi107.stdout:/dev/hugepages 2024-03-04T13:39:15.234 INFO:teuthology.orchestra.run.smithi107.stdout:/dev/mqueue 2024-03-04T13:39:15.234 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/kernel/debug 2024-03-04T13:39:15.234 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/kernel/tracing 2024-03-04T13:39:15.234 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/fs/fuse/connections 2024-03-04T13:39:15.234 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/kernel/config 2024-03-04T13:39:15.234 INFO:teuthology.orchestra.run.smithi107.stdout:/run/credentials/systemd-sysctl.service 2024-03-04T13:39:15.234 INFO:teuthology.orchestra.run.smithi107.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-04T13:39:15.235 INFO:teuthology.orchestra.run.smithi107.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-04T13:39:15.235 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/nfs/rpc_pipefs 2024-03-04T13:39:15.235 INFO:teuthology.orchestra.run.smithi107.stdout:/run/user/1000 2024-03-04T13:39:15.235 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/ceph 2024-03-04T13:39:15.235 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay 2024-03-04T13:39:15.235 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/28ce5b8ca75064ab44b164310a74fba35dcc0ced8dce1016843ef0bad81445f0/merged 2024-03-04T13:39:15.235 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/d955db762285aaeb18748b15ec066233216a7f7aa63988802eb294972e7876a4/merged 2024-03-04T13:39:15.235 INFO:teuthology.orchestra.run.smithi107.stdout:/run/user/0 2024-03-04T13:39:15.235 INFO:teuthology.orchestra.run.smithi107.stdout:/proc/sys/fs/binfmt_misc 2024-03-04T13:39:15.235 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/e42b2877e4f8a64c4ea0a334ff01569b03d9861562997bb2b4f3d2a879b0b2b9/merged 2024-03-04T13:39:15.235 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/2aae4ee1aea22f5ed0eb0a7f0267ebbce5d653380707f0a6da540d32d26d11cc/merged 2024-03-04T13:39:15.235 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/dc6944013a88995fff9883691b329a21d3cfaf9501ecc579bd12dec00765f1c0/merged 2024-03-04T13:39:15.236 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/a1e2d2c824eb88d36d79b1c3d93eb4e79d38c2c7f9869826e6b139456d947d60/merged 2024-03-04T13:39:15.236 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/aee8422d922f5a6fc23d360282a7288cb620ac0c064e6f107a81ab138e9952e2/merged 2024-03-04T13:39:15.236 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/f38e26dba0edb0df8cf42e6ccc54041200effdbf096ea5c732ea0f9d63d4277f/merged 2024-03-04T13:39:15.236 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/172a96860704d88f2d56123e353cc97b69542c7f41426ab9988d020e1419ad9c/merged 2024-03-04T13:39:15.236 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/30776d7db4aae543a5572f17f58acdb4964545b6f1814172ffd393997158df3c/merged 2024-03-04T13:39:15.236 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/248dddd27909f8c57548cfe1f30c0b15d29c7a02294c70ebbde91e6bf90ebc8f/merged 2024-03-04T13:39:15.236 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/e0510f246cdf32d97e39284e77a6c994fa8c275a70f8d9671472296499b34f70/merged 2024-03-04T13:39:15.236 INFO:teuthology.orchestra.run.smithi107.stdout:/run/netns 2024-03-04T13:39:15.236 INFO:teuthology.orchestra.run.smithi107.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-04T13:39:15.236 INFO:teuthology.orchestra.run.smithi107.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-04T13:39:15.237 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T13:39:15.237 DEBUG:teuthology.orchestra.run.smithi107:> ls /sys/fs/fuse/connections 2024-03-04T13:39:15.304 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi107.stderr:2024-03-04T13:39:15.310+0000 7fef4b85f300 -1 init, newargv = 0x55bfa30efd60 newargc=15 2024-03-04T13:39:15.304 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi107.stderr:ceph-fuse[75882]: starting ceph client 2024-03-04T13:39:15.318 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi107.stderr:ceph-fuse[75882]: starting fuse 2024-03-04T13:39:16.381 DEBUG:teuthology.orchestra.run.smithi107:> sudo modprobe fuse 2024-03-04T13:39:16.485 DEBUG:teuthology.orchestra.run.smithi107:> cat /proc/self/mounts | awk '{print $2}' 2024-03-04T13:39:16.539 INFO:teuthology.orchestra.run.smithi107.stdout:/proc 2024-03-04T13:39:16.539 INFO:teuthology.orchestra.run.smithi107.stdout:/sys 2024-03-04T13:39:16.539 INFO:teuthology.orchestra.run.smithi107.stdout:/dev 2024-03-04T13:39:16.539 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/kernel/security 2024-03-04T13:39:16.540 INFO:teuthology.orchestra.run.smithi107.stdout:/dev/shm 2024-03-04T13:39:16.540 INFO:teuthology.orchestra.run.smithi107.stdout:/dev/pts 2024-03-04T13:39:16.540 INFO:teuthology.orchestra.run.smithi107.stdout:/run 2024-03-04T13:39:16.540 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/fs/cgroup 2024-03-04T13:39:16.540 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/fs/pstore 2024-03-04T13:39:16.540 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/fs/bpf 2024-03-04T13:39:16.540 INFO:teuthology.orchestra.run.smithi107.stdout:/ 2024-03-04T13:39:16.540 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/fs/selinux 2024-03-04T13:39:16.540 INFO:teuthology.orchestra.run.smithi107.stdout:/proc/sys/fs/binfmt_misc 2024-03-04T13:39:16.540 INFO:teuthology.orchestra.run.smithi107.stdout:/dev/hugepages 2024-03-04T13:39:16.540 INFO:teuthology.orchestra.run.smithi107.stdout:/dev/mqueue 2024-03-04T13:39:16.540 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/kernel/debug 2024-03-04T13:39:16.540 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/kernel/tracing 2024-03-04T13:39:16.541 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/fs/fuse/connections 2024-03-04T13:39:16.541 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/kernel/config 2024-03-04T13:39:16.541 INFO:teuthology.orchestra.run.smithi107.stdout:/run/credentials/systemd-sysctl.service 2024-03-04T13:39:16.541 INFO:teuthology.orchestra.run.smithi107.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-04T13:39:16.541 INFO:teuthology.orchestra.run.smithi107.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-04T13:39:16.541 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/nfs/rpc_pipefs 2024-03-04T13:39:16.541 INFO:teuthology.orchestra.run.smithi107.stdout:/run/user/1000 2024-03-04T13:39:16.541 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/ceph 2024-03-04T13:39:16.541 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay 2024-03-04T13:39:16.541 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/28ce5b8ca75064ab44b164310a74fba35dcc0ced8dce1016843ef0bad81445f0/merged 2024-03-04T13:39:16.541 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/d955db762285aaeb18748b15ec066233216a7f7aa63988802eb294972e7876a4/merged 2024-03-04T13:39:16.541 INFO:teuthology.orchestra.run.smithi107.stdout:/run/user/0 2024-03-04T13:39:16.541 INFO:teuthology.orchestra.run.smithi107.stdout:/proc/sys/fs/binfmt_misc 2024-03-04T13:39:16.541 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/e42b2877e4f8a64c4ea0a334ff01569b03d9861562997bb2b4f3d2a879b0b2b9/merged 2024-03-04T13:39:16.542 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/2aae4ee1aea22f5ed0eb0a7f0267ebbce5d653380707f0a6da540d32d26d11cc/merged 2024-03-04T13:39:16.542 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/dc6944013a88995fff9883691b329a21d3cfaf9501ecc579bd12dec00765f1c0/merged 2024-03-04T13:39:16.542 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/a1e2d2c824eb88d36d79b1c3d93eb4e79d38c2c7f9869826e6b139456d947d60/merged 2024-03-04T13:39:16.542 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/aee8422d922f5a6fc23d360282a7288cb620ac0c064e6f107a81ab138e9952e2/merged 2024-03-04T13:39:16.542 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/f38e26dba0edb0df8cf42e6ccc54041200effdbf096ea5c732ea0f9d63d4277f/merged 2024-03-04T13:39:16.542 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/172a96860704d88f2d56123e353cc97b69542c7f41426ab9988d020e1419ad9c/merged 2024-03-04T13:39:16.542 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/30776d7db4aae543a5572f17f58acdb4964545b6f1814172ffd393997158df3c/merged 2024-03-04T13:39:16.542 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/248dddd27909f8c57548cfe1f30c0b15d29c7a02294c70ebbde91e6bf90ebc8f/merged 2024-03-04T13:39:16.542 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/e0510f246cdf32d97e39284e77a6c994fa8c275a70f8d9671472296499b34f70/merged 2024-03-04T13:39:16.542 INFO:teuthology.orchestra.run.smithi107.stdout:/run/netns 2024-03-04T13:39:16.542 INFO:teuthology.orchestra.run.smithi107.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-04T13:39:16.542 INFO:teuthology.orchestra.run.smithi107.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-04T13:39:16.542 INFO:teuthology.orchestra.run.smithi107.stdout:/home/ubuntu/cephtest/mnt.0 2024-03-04T13:39:16.543 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T13:39:16.543 DEBUG:teuthology.orchestra.run.smithi107:> ls /sys/fs/fuse/connections 2024-03-04T13:39:16.593 INFO:teuthology.orchestra.run.smithi107.stdout:147 2024-03-04T13:39:16.593 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [147] 2024-03-04T13:39:16.594 DEBUG:teuthology.orchestra.run.smithi107:> sudo stdin-killer -- python3 -c ' 2024-03-04T13:39:16.594 DEBUG:teuthology.orchestra.run.smithi107:> import glob 2024-03-04T13:39:16.594 DEBUG:teuthology.orchestra.run.smithi107:> import re 2024-03-04T13:39:16.594 DEBUG:teuthology.orchestra.run.smithi107:> import os 2024-03-04T13:39:16.594 DEBUG:teuthology.orchestra.run.smithi107:> import subprocess 2024-03-04T13:39:16.594 DEBUG:teuthology.orchestra.run.smithi107:> 2024-03-04T13:39:16.594 DEBUG:teuthology.orchestra.run.smithi107:> def _find_admin_socket(client_name): 2024-03-04T13:39:16.594 DEBUG:teuthology.orchestra.run.smithi107:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-03-04T13:39:16.594 DEBUG:teuthology.orchestra.run.smithi107:> files = glob.glob(asok_path) 2024-03-04T13:39:16.594 DEBUG:teuthology.orchestra.run.smithi107:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-03-04T13:39:16.594 DEBUG:teuthology.orchestra.run.smithi107:> 2024-03-04T13:39:16.594 DEBUG:teuthology.orchestra.run.smithi107:> # Given a non-glob path, it better be there 2024-03-04T13:39:16.595 DEBUG:teuthology.orchestra.run.smithi107:> if "*" not in asok_path: 2024-03-04T13:39:16.595 DEBUG:teuthology.orchestra.run.smithi107:> assert(len(files) == 1) 2024-03-04T13:39:16.595 DEBUG:teuthology.orchestra.run.smithi107:> return files[0] 2024-03-04T13:39:16.595 DEBUG:teuthology.orchestra.run.smithi107:> 2024-03-04T13:39:16.595 DEBUG:teuthology.orchestra.run.smithi107:> for f in files: 2024-03-04T13:39:16.595 DEBUG:teuthology.orchestra.run.smithi107:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-04T13:39:16.595 DEBUG:teuthology.orchestra.run.smithi107:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-04T13:39:16.595 DEBUG:teuthology.orchestra.run.smithi107:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-04T13:39:16.595 DEBUG:teuthology.orchestra.run.smithi107:> contents = proc_f.read() 2024-03-04T13:39:16.595 DEBUG:teuthology.orchestra.run.smithi107:> if mountpoint in contents: 2024-03-04T13:39:16.595 DEBUG:teuthology.orchestra.run.smithi107:> return f 2024-03-04T13:39:16.595 DEBUG:teuthology.orchestra.run.smithi107:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-04T13:39:16.595 DEBUG:teuthology.orchestra.run.smithi107:> 2024-03-04T13:39:16.595 DEBUG:teuthology.orchestra.run.smithi107:> print(_find_admin_socket("client.0")) 2024-03-04T13:39:16.595 DEBUG:teuthology.orchestra.run.smithi107:> ' 2024-03-04T13:39:16.762 INFO:teuthology.orchestra.run.smithi107.stdout:/var/run/ceph/ceph-client.0.75882.asok 2024-03-04T13:39:16.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-04T13:39:16.775 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:39:16 smithi107 ceph-mon[28739]: pgmap v239: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:39:16.779 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.75882.asok 2024-03-04T13:39:16.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T13:39:16.779 DEBUG:teuthology.orchestra.run.smithi107:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.75882.asok status 2024-03-04T13:39:17.001 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-03-04T13:39:17.001 INFO:teuthology.orchestra.run.smithi107.stdout: "metadata": { 2024-03-04T13:39:17.002 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_sha1": "42608bb191117be10dba720e6bf8a02f3171cd34", 2024-03-04T13:39:17.002 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_version": "ceph version 17.2.7-545-g42608bb1 (42608bb191117be10dba720e6bf8a02f3171cd34) quincy (stable)", 2024-03-04T13:39:17.002 INFO:teuthology.orchestra.run.smithi107.stdout: "entity_id": "0", 2024-03-04T13:39:17.002 INFO:teuthology.orchestra.run.smithi107.stdout: "hostname": "smithi107", 2024-03-04T13:39:17.002 INFO:teuthology.orchestra.run.smithi107.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-03-04T13:39:17.002 INFO:teuthology.orchestra.run.smithi107.stdout: "pid": "75882", 2024-03-04T13:39:17.002 INFO:teuthology.orchestra.run.smithi107.stdout: "root": "/" 2024-03-04T13:39:17.002 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:39:17.002 INFO:teuthology.orchestra.run.smithi107.stdout: "dentry_count": 0, 2024-03-04T13:39:17.002 INFO:teuthology.orchestra.run.smithi107.stdout: "dentry_pinned_count": 0, 2024-03-04T13:39:17.002 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 14524, 2024-03-04T13:39:17.002 INFO:teuthology.orchestra.run.smithi107.stdout: "inst": { 2024-03-04T13:39:17.002 INFO:teuthology.orchestra.run.smithi107.stdout: "name": { 2024-03-04T13:39:17.002 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "client", 2024-03-04T13:39:17.003 INFO:teuthology.orchestra.run.smithi107.stdout: "num": 14524 2024-03-04T13:39:17.003 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:39:17.003 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": { 2024-03-04T13:39:17.003 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-03-04T13:39:17.003 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "192.168.0.1:0", 2024-03-04T13:39:17.003 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 317786685 2024-03-04T13:39:17.003 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-04T13:39:17.003 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:39:17.003 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": { 2024-03-04T13:39:17.003 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-03-04T13:39:17.003 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "192.168.0.1:0", 2024-03-04T13:39:17.003 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 317786685 2024-03-04T13:39:17.004 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:39:17.004 INFO:teuthology.orchestra.run.smithi107.stdout: "inst_str": "client.14524 192.168.0.1:0/317786685", 2024-03-04T13:39:17.004 INFO:teuthology.orchestra.run.smithi107.stdout: "addr_str": "192.168.0.1:0/317786685", 2024-03-04T13:39:17.004 INFO:teuthology.orchestra.run.smithi107.stdout: "inode_count": 1, 2024-03-04T13:39:17.004 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_epoch": 15, 2024-03-04T13:39:17.004 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_epoch": 47, 2024-03-04T13:39:17.004 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_epoch_barrier": 0, 2024-03-04T13:39:17.004 INFO:teuthology.orchestra.run.smithi107.stdout: "blocklisted": false, 2024-03-04T13:39:17.004 INFO:teuthology.orchestra.run.smithi107.stdout: "fs_name": "cephfs" 2024-03-04T13:39:17.004 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-03-04T13:39:17.016 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 -- ceph fs ls 2024-03-04T13:39:17.103 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:39:16 smithi169 ceph-mon[36313]: pgmap v239: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:39:17.494 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/mon.smithi107/config 2024-03-04T13:39:18.710 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:39:18 smithi107 ceph-mon[28739]: pgmap v240: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:39:19.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:39:18 smithi169 ceph-mon[36313]: pgmap v240: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:39:19.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:19.599+0000 7f05a2c00700 1 -- 172.21.15.107:0/359468049 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f059c105170 msgr2=0x7f059c105540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:39:19.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:19.599+0000 7f05a2c00700 1 --2- 172.21.15.107:0/359468049 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f059c105170 0x7f059c105540 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f058c004940 tx=0x7f058c009e30 comp rx=0 tx=0).stop 2024-03-04T13:39:19.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:19.599+0000 7f05a2c00700 1 -- 172.21.15.107:0/359468049 shutdown_connections 2024-03-04T13:39:19.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:19.599+0000 7f05a2c00700 1 --2- 172.21.15.107:0/359468049 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f059c105b10 0x7f059c0ff180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:39:19.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:19.599+0000 7f05a2c00700 1 --2- 172.21.15.107:0/359468049 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f059c105170 0x7f059c105540 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:39:19.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:19.599+0000 7f05a2c00700 1 -- 172.21.15.107:0/359468049 >> 172.21.15.107:0/359468049 conn(0x7f059c0face0 msgr2=0x7f059c0fd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:39:19.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:19.599+0000 7f05a2c00700 1 -- 172.21.15.107:0/359468049 shutdown_connections 2024-03-04T13:39:19.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:19.599+0000 7f05a2c00700 1 -- 172.21.15.107:0/359468049 wait complete. 2024-03-04T13:39:19.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:19.600+0000 7f05a2c00700 1 Processor -- start 2024-03-04T13:39:19.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:19.600+0000 7f05a2c00700 1 -- start start 2024-03-04T13:39:19.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:19.601+0000 7f05a2c00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f059c105170 0x7f059c115d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:39:19.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:19.601+0000 7f05a2c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f059c105b10 0x7f059c116290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:39:19.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:19.601+0000 7f05a2c00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f059c1168e0 con 0x7f059c105b10 2024-03-04T13:39:19.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:19.601+0000 7f05a2c00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f059c10fbf0 con 0x7f059c105170 2024-03-04T13:39:19.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:19.601+0000 7f059b400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f059c105b10 0x7f059c116290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:39:19.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:19.601+0000 7f059b400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f059c105b10 0x7f059c116290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49748/0 (socket says 172.21.15.107:49748) 2024-03-04T13:39:19.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:19.601+0000 7f059b400700 1 -- 172.21.15.107:0/441175570 learned_addr learned my addr 172.21.15.107:0/441175570 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:39:19.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:19.601+0000 7f059be00700 1 --2- 172.21.15.107:0/441175570 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f059c105170 0x7f059c115d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:39:19.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:19.602+0000 7f059b400700 1 -- 172.21.15.107:0/441175570 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f059c105170 msgr2=0x7f059c115d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:39:19.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:19.602+0000 7f059b400700 1 --2- 172.21.15.107:0/441175570 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f059c105170 0x7f059c115d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:39:19.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:19.602+0000 7f059b400700 1 -- 172.21.15.107:0/441175570 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f058c005040 con 0x7f059c105b10 2024-03-04T13:39:19.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:19.602+0000 7f059b400700 1 --2- 172.21.15.107:0/441175570 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f059c105b10 0x7f059c116290 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f0588002a80 tx=0x7f0588004630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:39:19.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:19.602+0000 7f0593e00700 1 -- 172.21.15.107:0/441175570 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0588007cc0 con 0x7f059c105b10 2024-03-04T13:39:19.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:19.602+0000 7f0593e00700 1 -- 172.21.15.107:0/441175570 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f058800f040 con 0x7f059c105b10 2024-03-04T13:39:19.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:19.602+0000 7f0593e00700 1 -- 172.21.15.107:0/441175570 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f058800daf0 con 0x7f059c105b10 2024-03-04T13:39:19.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:19.602+0000 7f05a2c00700 1 -- 172.21.15.107:0/441175570 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f059c10fed0 con 0x7f059c105b10 2024-03-04T13:39:19.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:19.603+0000 7f05a2c00700 1 -- 172.21.15.107:0/441175570 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f059c110410 con 0x7f059c105b10 2024-03-04T13:39:19.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:19.604+0000 7f05a2c00700 1 -- 172.21.15.107:0/441175570 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f059c102480 con 0x7f059c105b10 2024-03-04T13:39:19.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:19.605+0000 7f0593e00700 1 -- 172.21.15.107:0/441175570 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86623+0+0 (secure 0 0 0) 0x7f058800dc90 con 0x7f059c105b10 2024-03-04T13:39:19.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:19.606+0000 7f0593e00700 1 --2- 172.21.15.107:0/441175570 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f05800692b0 0x7f058006b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:39:19.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:19.606+0000 7f0593e00700 1 -- 172.21.15.107:0/441175570 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f05880868e0 con 0x7f059c105b10 2024-03-04T13:39:19.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:19.606+0000 7f059be00700 1 --2- 172.21.15.107:0/441175570 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f05800692b0 0x7f058006b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:39:19.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:19.607+0000 7f059be00700 1 --2- 172.21.15.107:0/441175570 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f05800692b0 0x7f058006b760 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f058c00b020 tx=0x7f058c009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:39:19.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:19.610+0000 7f0593e00700 1 -- 172.21.15.107:0/441175570 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f058804cbd0 con 0x7f059c105b10 2024-03-04T13:39:19.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:19.863+0000 7f05a2c00700 1 -- 172.21.15.107:0/441175570 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f059c0fe900 con 0x7f059c105b10 2024-03-04T13:39:19.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:19.864+0000 7f0593e00700 1 -- 172.21.15.107:0/441175570 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) v1 ==== 53+0+83 (secure 0 0 0) 0x7f05880565b0 con 0x7f059c105b10 2024-03-04T13:39:19.859 INFO:teuthology.orchestra.run.smithi107.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-03-04T13:39:19.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:19.867+0000 7f05a2c00700 1 -- 172.21.15.107:0/441175570 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f05800692b0 msgr2=0x7f058006b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:39:19.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:19.867+0000 7f05a2c00700 1 --2- 172.21.15.107:0/441175570 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f05800692b0 0x7f058006b760 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f058c00b020 tx=0x7f058c009e30 comp rx=0 tx=0).stop 2024-03-04T13:39:19.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:19.867+0000 7f05a2c00700 1 -- 172.21.15.107:0/441175570 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f059c105b10 msgr2=0x7f059c116290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:39:19.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:19.867+0000 7f05a2c00700 1 --2- 172.21.15.107:0/441175570 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f059c105b10 0x7f059c116290 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f0588002a80 tx=0x7f0588004630 comp rx=0 tx=0).stop 2024-03-04T13:39:19.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:19.867+0000 7f05a2c00700 1 -- 172.21.15.107:0/441175570 shutdown_connections 2024-03-04T13:39:19.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:19.867+0000 7f05a2c00700 1 --2- 172.21.15.107:0/441175570 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f05800692b0 0x7f058006b760 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:39:19.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:19.867+0000 7f05a2c00700 1 --2- 172.21.15.107:0/441175570 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f059c105170 0x7f059c115d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:39:19.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:19.867+0000 7f05a2c00700 1 --2- 172.21.15.107:0/441175570 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f059c105b10 0x7f059c116290 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:39:19.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:19.867+0000 7f05a2c00700 1 -- 172.21.15.107:0/441175570 >> 172.21.15.107:0/441175570 conn(0x7f059c0face0 msgr2=0x7f059c0fc830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:39:19.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:19.867+0000 7f05a2c00700 1 -- 172.21.15.107:0/441175570 shutdown_connections 2024-03-04T13:39:19.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:19.867+0000 7f05a2c00700 1 -- 172.21.15.107:0/441175570 wait complete. 2024-03-04T13:39:20.786 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-04T13:39:20.787 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-04T13:39:20.787 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi169.front.sepia.ceph.com 2024-03-04T13:39:20.787 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-03-04T13:39:20.787 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-04T13:39:20.787 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-04T13:39:20.787 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-04T13:39:20.787 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-04T13:39:20.787 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-03-04T13:39:20.787 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T13:39:20.787 DEBUG:teuthology.orchestra.run.smithi169:> ip addr 2024-03-04T13:39:20.849 INFO:teuthology.orchestra.run.smithi169.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-04T13:39:20.849 INFO:teuthology.orchestra.run.smithi169.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-04T13:39:20.849 INFO:teuthology.orchestra.run.smithi169.stdout: inet 127.0.0.1/8 scope host lo 2024-03-04T13:39:20.849 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2024-03-04T13:39:20.849 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 ::1/128 scope host 2024-03-04T13:39:20.849 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2024-03-04T13:39:20.849 INFO:teuthology.orchestra.run.smithi169.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-04T13:39:20.849 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:d9:3b:b4 brd ff:ff:ff:ff:ff:ff 2024-03-04T13:39:20.849 INFO:teuthology.orchestra.run.smithi169.stdout: altname enp7s0f0 2024-03-04T13:39:20.849 INFO:teuthology.orchestra.run.smithi169.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-04T13:39:20.849 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:d9:3b:b5 brd ff:ff:ff:ff:ff:ff 2024-03-04T13:39:20.849 INFO:teuthology.orchestra.run.smithi169.stdout: altname enp7s0f1 2024-03-04T13:39:20.849 INFO:teuthology.orchestra.run.smithi169.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-04T13:39:20.849 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:88:7f:4c brd ff:ff:ff:ff:ff:ff 2024-03-04T13:39:20.850 INFO:teuthology.orchestra.run.smithi169.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-04T13:39:20.850 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:88:7f:4d brd ff:ff:ff:ff:ff:ff 2024-03-04T13:39:20.850 INFO:teuthology.orchestra.run.smithi169.stdout: inet 172.21.15.169/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-04T13:39:20.850 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft 41703sec preferred_lft 41703sec 2024-03-04T13:39:20.850 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 fe80::ec4:7aff:fe88:7f4d/64 scope link 2024-03-04T13:39:20.850 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2024-03-04T13:39:20.851 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-04T13:39:20.851 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T13:39:20.851 DEBUG:teuthology.orchestra.run.smithi169:> (cd / && exec bash -c ' 2024-03-04T13:39:20.851 DEBUG:teuthology.orchestra.run.smithi169:> set -e 2024-03-04T13:39:20.851 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link add name ceph-brx type bridge 2024-03-04T13:39:20.851 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip addr flush dev ceph-brx 2024-03-04T13:39:20.851 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link set ceph-brx up 2024-03-04T13:39:20.851 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-04T13:39:20.851 DEBUG:teuthology.orchestra.run.smithi169:> ') 2024-03-04T13:39:20.938 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:39:20 smithi169 ceph-mon[36313]: pgmap v241: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:39:20.938 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:39:20 smithi169 ceph-mon[36313]: from='client.? 172.21.15.107:0/441175570' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-04T13:39:21.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:39:20 smithi107 ceph-mon[28739]: pgmap v241: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:39:21.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:39:20 smithi107 ceph-mon[28739]: from='client.? 172.21.15.107:0/441175570' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-04T13:39:21.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T13:39:21.292 DEBUG:teuthology.orchestra.run.smithi169:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-04T13:39:21.373 INFO:teuthology.orchestra.run.smithi169.stdout:1 2024-03-04T13:39:21.377 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T13:39:21.377 DEBUG:teuthology.orchestra.run.smithi169:> route 2024-03-04T13:39:21.433 INFO:teuthology.orchestra.run.smithi169.stdout:Kernel IP routing table 2024-03-04T13:39:21.433 INFO:teuthology.orchestra.run.smithi169.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-04T13:39:21.433 INFO:teuthology.orchestra.run.smithi169.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-04T13:39:21.433 INFO:teuthology.orchestra.run.smithi169.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-04T13:39:21.434 INFO:teuthology.orchestra.run.smithi169.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-04T13:39:21.434 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T13:39:21.434 DEBUG:teuthology.orchestra.run.smithi169:> (cd / && exec bash -c ' 2024-03-04T13:39:21.434 DEBUG:teuthology.orchestra.run.smithi169:> set -e 2024-03-04T13:39:21.434 DEBUG:teuthology.orchestra.run.smithi169:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-04T13:39:21.434 DEBUG:teuthology.orchestra.run.smithi169:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-04T13:39:21.434 DEBUG:teuthology.orchestra.run.smithi169:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-04T13:39:21.435 DEBUG:teuthology.orchestra.run.smithi169:> ') 2024-03-04T13:39:22.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T13:39:22.141 DEBUG:teuthology.orchestra.run.smithi169:> ip netns list 2024-03-04T13:39:22.196 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T13:39:22.197 DEBUG:teuthology.orchestra.run.smithi169:> ip netns list-id 2024-03-04T13:39:22.247 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T13:39:22.247 DEBUG:teuthology.orchestra.run.smithi169:> (cd / && exec bash -c ' 2024-03-04T13:39:22.247 DEBUG:teuthology.orchestra.run.smithi169:> set -e 2024-03-04T13:39:22.247 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-04T13:39:22.247 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-03-04T13:39:22.247 DEBUG:teuthology.orchestra.run.smithi169:> ') 2024-03-04T13:39:22.384 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-03-04T13:39:22.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T13:39:22.384 DEBUG:teuthology.orchestra.run.smithi169:> (cd / && exec bash -c ' 2024-03-04T13:39:22.384 DEBUG:teuthology.orchestra.run.smithi169:> set -e 2024-03-04T13:39:22.384 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-03-04T13:39:22.385 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-04T13:39:22.385 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-03-04T13:39:22.385 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-03-04T13:39:22.385 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-03-04T13:39:22.385 DEBUG:teuthology.orchestra.run.smithi169:> ') 2024-03-04T13:39:22.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T13:39:22.663 DEBUG:teuthology.orchestra.run.smithi169:> (cd / && exec bash -c ' 2024-03-04T13:39:22.664 DEBUG:teuthology.orchestra.run.smithi169:> set -e 2024-03-04T13:39:22.664 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link set brx.0 up 2024-03-04T13:39:22.664 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link set dev brx.0 master ceph-brx 2024-03-04T13:39:22.664 DEBUG:teuthology.orchestra.run.smithi169:> ') 2024-03-04T13:39:22.741 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:39:22 smithi169 ceph-mon[36313]: pgmap v242: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-03-04T13:39:22.766 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:39:22 smithi107 ceph-mon[28739]: pgmap v242: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-03-04T13:39:22.827 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-03-04T13:39:22.827 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-04T13:39:22.827 DEBUG:teuthology.orchestra.run.smithi169:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-03-04T13:39:22.869 INFO:teuthology.orchestra.run.smithi169.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-03-04T13:39:22.869 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-04T13:39:22.869 DEBUG:teuthology.orchestra.run.smithi169:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-03-04T13:39:22.962 DEBUG:teuthology.orchestra.run.smithi169:> sudo modprobe fuse 2024-03-04T13:39:23.054 DEBUG:teuthology.orchestra.run.smithi169:> cat /proc/self/mounts | awk '{print $2}' 2024-03-04T13:39:23.116 INFO:teuthology.orchestra.run.smithi169.stdout:/proc 2024-03-04T13:39:23.116 INFO:teuthology.orchestra.run.smithi169.stdout:/sys 2024-03-04T13:39:23.116 INFO:teuthology.orchestra.run.smithi169.stdout:/dev 2024-03-04T13:39:23.116 INFO:teuthology.orchestra.run.smithi169.stdout:/sys/kernel/security 2024-03-04T13:39:23.116 INFO:teuthology.orchestra.run.smithi169.stdout:/dev/shm 2024-03-04T13:39:23.116 INFO:teuthology.orchestra.run.smithi169.stdout:/dev/pts 2024-03-04T13:39:23.116 INFO:teuthology.orchestra.run.smithi169.stdout:/run 2024-03-04T13:39:23.116 INFO:teuthology.orchestra.run.smithi169.stdout:/sys/fs/cgroup 2024-03-04T13:39:23.116 INFO:teuthology.orchestra.run.smithi169.stdout:/sys/fs/pstore 2024-03-04T13:39:23.116 INFO:teuthology.orchestra.run.smithi169.stdout:/sys/fs/bpf 2024-03-04T13:39:23.116 INFO:teuthology.orchestra.run.smithi169.stdout:/ 2024-03-04T13:39:23.116 INFO:teuthology.orchestra.run.smithi169.stdout:/sys/fs/selinux 2024-03-04T13:39:23.117 INFO:teuthology.orchestra.run.smithi169.stdout:/proc/sys/fs/binfmt_misc 2024-03-04T13:39:23.117 INFO:teuthology.orchestra.run.smithi169.stdout:/dev/hugepages 2024-03-04T13:39:23.117 INFO:teuthology.orchestra.run.smithi169.stdout:/dev/mqueue 2024-03-04T13:39:23.117 INFO:teuthology.orchestra.run.smithi169.stdout:/sys/kernel/debug 2024-03-04T13:39:23.117 INFO:teuthology.orchestra.run.smithi169.stdout:/sys/kernel/tracing 2024-03-04T13:39:23.117 INFO:teuthology.orchestra.run.smithi169.stdout:/sys/fs/fuse/connections 2024-03-04T13:39:23.117 INFO:teuthology.orchestra.run.smithi169.stdout:/sys/kernel/config 2024-03-04T13:39:23.117 INFO:teuthology.orchestra.run.smithi169.stdout:/run/credentials/systemd-sysctl.service 2024-03-04T13:39:23.117 INFO:teuthology.orchestra.run.smithi169.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-04T13:39:23.117 INFO:teuthology.orchestra.run.smithi169.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-04T13:39:23.117 INFO:teuthology.orchestra.run.smithi169.stdout:/var/lib/nfs/rpc_pipefs 2024-03-04T13:39:23.117 INFO:teuthology.orchestra.run.smithi169.stdout:/run/user/1000 2024-03-04T13:39:23.117 INFO:teuthology.orchestra.run.smithi169.stdout:/var/lib/ceph 2024-03-04T13:39:23.117 INFO:teuthology.orchestra.run.smithi169.stdout:/run/user/0 2024-03-04T13:39:23.118 INFO:teuthology.orchestra.run.smithi169.stdout:/proc/sys/fs/binfmt_misc 2024-03-04T13:39:23.118 INFO:teuthology.orchestra.run.smithi169.stdout:/var/lib/containers/storage/overlay 2024-03-04T13:39:23.118 INFO:teuthology.orchestra.run.smithi169.stdout:/var/lib/containers/storage/overlay/a8041d79629251f7db32e6707ed17c178635072c7060dac1fa3764951a323c4f/merged 2024-03-04T13:39:23.118 INFO:teuthology.orchestra.run.smithi169.stdout:/var/lib/containers/storage/overlay/0b792b6caed30fafcbab3cdc7febe6e6a8085acaed58917e12a891445fd62d6d/merged 2024-03-04T13:39:23.118 INFO:teuthology.orchestra.run.smithi169.stdout:/var/lib/containers/storage/overlay/3263944e4c0b2403accc77a3117602861586c541e6af4ccb6040f89bc2c8a1dc/merged 2024-03-04T13:39:23.118 INFO:teuthology.orchestra.run.smithi169.stdout:/var/lib/containers/storage/overlay/9c8cb79262e6191c25f07033d44bf323850335f775401da8bb956a9c33bedd67/merged 2024-03-04T13:39:23.118 INFO:teuthology.orchestra.run.smithi169.stdout:/var/lib/containers/storage/overlay/f2f84d13729381f54592be3031a905be172d7fd318735d4e0a4bfca4c130dafd/merged 2024-03-04T13:39:23.118 INFO:teuthology.orchestra.run.smithi169.stdout:/var/lib/containers/storage/overlay/5cc96f67b66adba34cbd7325a1ef5e681cbca4bd5dba6106e68a355bded1c37b/merged 2024-03-04T13:39:23.118 INFO:teuthology.orchestra.run.smithi169.stdout:/var/lib/containers/storage/overlay/e0e577ff83f3176be80a2ffbf59417df8d318d1b45c6b7424196e817ae5c8e5a/merged 2024-03-04T13:39:23.118 INFO:teuthology.orchestra.run.smithi169.stdout:/var/lib/containers/storage/overlay/d22fda54cee6b712d2dbe4eb4991a30ee0c07a0e45e917259106421fcd31476b/merged 2024-03-04T13:39:23.118 INFO:teuthology.orchestra.run.smithi169.stdout:/var/lib/containers/storage/overlay/b9330b5fbbd2fccf2fec5589aecee0994b5fe0f2c7482a5721e944313fdfcb28/merged 2024-03-04T13:39:23.118 INFO:teuthology.orchestra.run.smithi169.stdout:/run/netns 2024-03-04T13:39:23.118 INFO:teuthology.orchestra.run.smithi169.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-04T13:39:23.118 INFO:teuthology.orchestra.run.smithi169.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-04T13:39:23.119 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T13:39:23.119 DEBUG:teuthology.orchestra.run.smithi169:> ls /sys/fs/fuse/connections 2024-03-04T13:39:23.166 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-04T13:39:23.167 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-03-04T13:39:23.171 DEBUG:teuthology.orchestra.run.smithi169:> sudo modprobe fuse 2024-03-04T13:39:23.241 DEBUG:teuthology.orchestra.run.smithi169:> cat /proc/self/mounts | awk '{print $2}' 2024-03-04T13:39:23.284 INFO:teuthology.orchestra.run.smithi169.stdout:/proc 2024-03-04T13:39:23.285 INFO:teuthology.orchestra.run.smithi169.stdout:/sys 2024-03-04T13:39:23.285 INFO:teuthology.orchestra.run.smithi169.stdout:/dev 2024-03-04T13:39:23.285 INFO:teuthology.orchestra.run.smithi169.stdout:/sys/kernel/security 2024-03-04T13:39:23.285 INFO:teuthology.orchestra.run.smithi169.stdout:/dev/shm 2024-03-04T13:39:23.285 INFO:teuthology.orchestra.run.smithi169.stdout:/dev/pts 2024-03-04T13:39:23.285 INFO:teuthology.orchestra.run.smithi169.stdout:/run 2024-03-04T13:39:23.285 INFO:teuthology.orchestra.run.smithi169.stdout:/sys/fs/cgroup 2024-03-04T13:39:23.285 INFO:teuthology.orchestra.run.smithi169.stdout:/sys/fs/pstore 2024-03-04T13:39:23.285 INFO:teuthology.orchestra.run.smithi169.stdout:/sys/fs/bpf 2024-03-04T13:39:23.285 INFO:teuthology.orchestra.run.smithi169.stdout:/ 2024-03-04T13:39:23.285 INFO:teuthology.orchestra.run.smithi169.stdout:/sys/fs/selinux 2024-03-04T13:39:23.285 INFO:teuthology.orchestra.run.smithi169.stdout:/proc/sys/fs/binfmt_misc 2024-03-04T13:39:23.285 INFO:teuthology.orchestra.run.smithi169.stdout:/dev/hugepages 2024-03-04T13:39:23.285 INFO:teuthology.orchestra.run.smithi169.stdout:/dev/mqueue 2024-03-04T13:39:23.285 INFO:teuthology.orchestra.run.smithi169.stdout:/sys/kernel/debug 2024-03-04T13:39:23.286 INFO:teuthology.orchestra.run.smithi169.stdout:/sys/kernel/tracing 2024-03-04T13:39:23.286 INFO:teuthology.orchestra.run.smithi169.stdout:/sys/fs/fuse/connections 2024-03-04T13:39:23.286 INFO:teuthology.orchestra.run.smithi169.stdout:/sys/kernel/config 2024-03-04T13:39:23.286 INFO:teuthology.orchestra.run.smithi169.stdout:/run/credentials/systemd-sysctl.service 2024-03-04T13:39:23.286 INFO:teuthology.orchestra.run.smithi169.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-04T13:39:23.286 INFO:teuthology.orchestra.run.smithi169.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-04T13:39:23.286 INFO:teuthology.orchestra.run.smithi169.stdout:/var/lib/nfs/rpc_pipefs 2024-03-04T13:39:23.286 INFO:teuthology.orchestra.run.smithi169.stdout:/run/user/1000 2024-03-04T13:39:23.286 INFO:teuthology.orchestra.run.smithi169.stdout:/var/lib/ceph 2024-03-04T13:39:23.286 INFO:teuthology.orchestra.run.smithi169.stdout:/run/user/0 2024-03-04T13:39:23.286 INFO:teuthology.orchestra.run.smithi169.stdout:/proc/sys/fs/binfmt_misc 2024-03-04T13:39:23.286 INFO:teuthology.orchestra.run.smithi169.stdout:/var/lib/containers/storage/overlay 2024-03-04T13:39:23.286 INFO:teuthology.orchestra.run.smithi169.stdout:/var/lib/containers/storage/overlay/a8041d79629251f7db32e6707ed17c178635072c7060dac1fa3764951a323c4f/merged 2024-03-04T13:39:23.286 INFO:teuthology.orchestra.run.smithi169.stdout:/var/lib/containers/storage/overlay/0b792b6caed30fafcbab3cdc7febe6e6a8085acaed58917e12a891445fd62d6d/merged 2024-03-04T13:39:23.286 INFO:teuthology.orchestra.run.smithi169.stdout:/var/lib/containers/storage/overlay/3263944e4c0b2403accc77a3117602861586c541e6af4ccb6040f89bc2c8a1dc/merged 2024-03-04T13:39:23.286 INFO:teuthology.orchestra.run.smithi169.stdout:/var/lib/containers/storage/overlay/9c8cb79262e6191c25f07033d44bf323850335f775401da8bb956a9c33bedd67/merged 2024-03-04T13:39:23.287 INFO:teuthology.orchestra.run.smithi169.stdout:/var/lib/containers/storage/overlay/f2f84d13729381f54592be3031a905be172d7fd318735d4e0a4bfca4c130dafd/merged 2024-03-04T13:39:23.287 INFO:teuthology.orchestra.run.smithi169.stdout:/var/lib/containers/storage/overlay/5cc96f67b66adba34cbd7325a1ef5e681cbca4bd5dba6106e68a355bded1c37b/merged 2024-03-04T13:39:23.287 INFO:teuthology.orchestra.run.smithi169.stdout:/var/lib/containers/storage/overlay/e0e577ff83f3176be80a2ffbf59417df8d318d1b45c6b7424196e817ae5c8e5a/merged 2024-03-04T13:39:23.287 INFO:teuthology.orchestra.run.smithi169.stdout:/var/lib/containers/storage/overlay/d22fda54cee6b712d2dbe4eb4991a30ee0c07a0e45e917259106421fcd31476b/merged 2024-03-04T13:39:23.287 INFO:teuthology.orchestra.run.smithi169.stdout:/var/lib/containers/storage/overlay/b9330b5fbbd2fccf2fec5589aecee0994b5fe0f2c7482a5721e944313fdfcb28/merged 2024-03-04T13:39:23.287 INFO:teuthology.orchestra.run.smithi169.stdout:/run/netns 2024-03-04T13:39:23.287 INFO:teuthology.orchestra.run.smithi169.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-04T13:39:23.287 INFO:teuthology.orchestra.run.smithi169.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-04T13:39:23.288 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T13:39:23.288 DEBUG:teuthology.orchestra.run.smithi169:> ls /sys/fs/fuse/connections 2024-03-04T13:39:23.390 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi169.stderr:2024-03-04T13:39:23.396+0000 7f65a56c3300 -1 init, newargv = 0x5629a6187090 newargc=15 2024-03-04T13:39:23.390 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi169.stderr:ceph-fuse[61093]: starting ceph client 2024-03-04T13:39:23.399 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi169.stderr:ceph-fuse[61093]: starting fuse 2024-03-04T13:39:24.476 DEBUG:teuthology.orchestra.run.smithi169:> sudo modprobe fuse 2024-03-04T13:39:24.579 DEBUG:teuthology.orchestra.run.smithi169:> cat /proc/self/mounts | awk '{print $2}' 2024-03-04T13:39:24.634 INFO:teuthology.orchestra.run.smithi169.stdout:/proc 2024-03-04T13:39:24.634 INFO:teuthology.orchestra.run.smithi169.stdout:/sys 2024-03-04T13:39:24.634 INFO:teuthology.orchestra.run.smithi169.stdout:/dev 2024-03-04T13:39:24.634 INFO:teuthology.orchestra.run.smithi169.stdout:/sys/kernel/security 2024-03-04T13:39:24.634 INFO:teuthology.orchestra.run.smithi169.stdout:/dev/shm 2024-03-04T13:39:24.634 INFO:teuthology.orchestra.run.smithi169.stdout:/dev/pts 2024-03-04T13:39:24.634 INFO:teuthology.orchestra.run.smithi169.stdout:/run 2024-03-04T13:39:24.634 INFO:teuthology.orchestra.run.smithi169.stdout:/sys/fs/cgroup 2024-03-04T13:39:24.634 INFO:teuthology.orchestra.run.smithi169.stdout:/sys/fs/pstore 2024-03-04T13:39:24.634 INFO:teuthology.orchestra.run.smithi169.stdout:/sys/fs/bpf 2024-03-04T13:39:24.634 INFO:teuthology.orchestra.run.smithi169.stdout:/ 2024-03-04T13:39:24.634 INFO:teuthology.orchestra.run.smithi169.stdout:/sys/fs/selinux 2024-03-04T13:39:24.635 INFO:teuthology.orchestra.run.smithi169.stdout:/proc/sys/fs/binfmt_misc 2024-03-04T13:39:24.635 INFO:teuthology.orchestra.run.smithi169.stdout:/dev/hugepages 2024-03-04T13:39:24.635 INFO:teuthology.orchestra.run.smithi169.stdout:/dev/mqueue 2024-03-04T13:39:24.635 INFO:teuthology.orchestra.run.smithi169.stdout:/sys/kernel/debug 2024-03-04T13:39:24.635 INFO:teuthology.orchestra.run.smithi169.stdout:/sys/kernel/tracing 2024-03-04T13:39:24.635 INFO:teuthology.orchestra.run.smithi169.stdout:/sys/fs/fuse/connections 2024-03-04T13:39:24.635 INFO:teuthology.orchestra.run.smithi169.stdout:/sys/kernel/config 2024-03-04T13:39:24.635 INFO:teuthology.orchestra.run.smithi169.stdout:/run/credentials/systemd-sysctl.service 2024-03-04T13:39:24.635 INFO:teuthology.orchestra.run.smithi169.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-04T13:39:24.635 INFO:teuthology.orchestra.run.smithi169.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-04T13:39:24.635 INFO:teuthology.orchestra.run.smithi169.stdout:/var/lib/nfs/rpc_pipefs 2024-03-04T13:39:24.635 INFO:teuthology.orchestra.run.smithi169.stdout:/run/user/1000 2024-03-04T13:39:24.635 INFO:teuthology.orchestra.run.smithi169.stdout:/var/lib/ceph 2024-03-04T13:39:24.635 INFO:teuthology.orchestra.run.smithi169.stdout:/run/user/0 2024-03-04T13:39:24.635 INFO:teuthology.orchestra.run.smithi169.stdout:/proc/sys/fs/binfmt_misc 2024-03-04T13:39:24.636 INFO:teuthology.orchestra.run.smithi169.stdout:/var/lib/containers/storage/overlay 2024-03-04T13:39:24.636 INFO:teuthology.orchestra.run.smithi169.stdout:/var/lib/containers/storage/overlay/a8041d79629251f7db32e6707ed17c178635072c7060dac1fa3764951a323c4f/merged 2024-03-04T13:39:24.636 INFO:teuthology.orchestra.run.smithi169.stdout:/var/lib/containers/storage/overlay/0b792b6caed30fafcbab3cdc7febe6e6a8085acaed58917e12a891445fd62d6d/merged 2024-03-04T13:39:24.636 INFO:teuthology.orchestra.run.smithi169.stdout:/var/lib/containers/storage/overlay/3263944e4c0b2403accc77a3117602861586c541e6af4ccb6040f89bc2c8a1dc/merged 2024-03-04T13:39:24.636 INFO:teuthology.orchestra.run.smithi169.stdout:/var/lib/containers/storage/overlay/9c8cb79262e6191c25f07033d44bf323850335f775401da8bb956a9c33bedd67/merged 2024-03-04T13:39:24.636 INFO:teuthology.orchestra.run.smithi169.stdout:/var/lib/containers/storage/overlay/f2f84d13729381f54592be3031a905be172d7fd318735d4e0a4bfca4c130dafd/merged 2024-03-04T13:39:24.636 INFO:teuthology.orchestra.run.smithi169.stdout:/var/lib/containers/storage/overlay/5cc96f67b66adba34cbd7325a1ef5e681cbca4bd5dba6106e68a355bded1c37b/merged 2024-03-04T13:39:24.636 INFO:teuthology.orchestra.run.smithi169.stdout:/var/lib/containers/storage/overlay/e0e577ff83f3176be80a2ffbf59417df8d318d1b45c6b7424196e817ae5c8e5a/merged 2024-03-04T13:39:24.636 INFO:teuthology.orchestra.run.smithi169.stdout:/var/lib/containers/storage/overlay/d22fda54cee6b712d2dbe4eb4991a30ee0c07a0e45e917259106421fcd31476b/merged 2024-03-04T13:39:24.636 INFO:teuthology.orchestra.run.smithi169.stdout:/var/lib/containers/storage/overlay/b9330b5fbbd2fccf2fec5589aecee0994b5fe0f2c7482a5721e944313fdfcb28/merged 2024-03-04T13:39:24.636 INFO:teuthology.orchestra.run.smithi169.stdout:/run/netns 2024-03-04T13:39:24.636 INFO:teuthology.orchestra.run.smithi169.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-04T13:39:24.636 INFO:teuthology.orchestra.run.smithi169.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-04T13:39:24.636 INFO:teuthology.orchestra.run.smithi169.stdout:/home/ubuntu/cephtest/mnt.1 2024-03-04T13:39:24.637 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T13:39:24.637 DEBUG:teuthology.orchestra.run.smithi169:> ls /sys/fs/fuse/connections 2024-03-04T13:39:24.681 INFO:teuthology.orchestra.run.smithi169.stdout:114 2024-03-04T13:39:24.681 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [114] 2024-03-04T13:39:24.683 DEBUG:teuthology.orchestra.run.smithi169:> sudo stdin-killer -- python3 -c ' 2024-03-04T13:39:24.683 DEBUG:teuthology.orchestra.run.smithi169:> import glob 2024-03-04T13:39:24.683 DEBUG:teuthology.orchestra.run.smithi169:> import re 2024-03-04T13:39:24.683 DEBUG:teuthology.orchestra.run.smithi169:> import os 2024-03-04T13:39:24.683 DEBUG:teuthology.orchestra.run.smithi169:> import subprocess 2024-03-04T13:39:24.683 DEBUG:teuthology.orchestra.run.smithi169:> 2024-03-04T13:39:24.683 DEBUG:teuthology.orchestra.run.smithi169:> def _find_admin_socket(client_name): 2024-03-04T13:39:24.683 DEBUG:teuthology.orchestra.run.smithi169:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-03-04T13:39:24.683 DEBUG:teuthology.orchestra.run.smithi169:> files = glob.glob(asok_path) 2024-03-04T13:39:24.683 DEBUG:teuthology.orchestra.run.smithi169:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-03-04T13:39:24.683 DEBUG:teuthology.orchestra.run.smithi169:> 2024-03-04T13:39:24.683 DEBUG:teuthology.orchestra.run.smithi169:> # Given a non-glob path, it better be there 2024-03-04T13:39:24.683 DEBUG:teuthology.orchestra.run.smithi169:> if "*" not in asok_path: 2024-03-04T13:39:24.683 DEBUG:teuthology.orchestra.run.smithi169:> assert(len(files) == 1) 2024-03-04T13:39:24.684 DEBUG:teuthology.orchestra.run.smithi169:> return files[0] 2024-03-04T13:39:24.684 DEBUG:teuthology.orchestra.run.smithi169:> 2024-03-04T13:39:24.684 DEBUG:teuthology.orchestra.run.smithi169:> for f in files: 2024-03-04T13:39:24.684 DEBUG:teuthology.orchestra.run.smithi169:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-04T13:39:24.684 DEBUG:teuthology.orchestra.run.smithi169:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-04T13:39:24.684 DEBUG:teuthology.orchestra.run.smithi169:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-04T13:39:24.684 DEBUG:teuthology.orchestra.run.smithi169:> contents = proc_f.read() 2024-03-04T13:39:24.684 DEBUG:teuthology.orchestra.run.smithi169:> if mountpoint in contents: 2024-03-04T13:39:24.684 DEBUG:teuthology.orchestra.run.smithi169:> return f 2024-03-04T13:39:24.684 DEBUG:teuthology.orchestra.run.smithi169:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-04T13:39:24.684 DEBUG:teuthology.orchestra.run.smithi169:> 2024-03-04T13:39:24.684 DEBUG:teuthology.orchestra.run.smithi169:> print(_find_admin_socket("client.1")) 2024-03-04T13:39:24.684 DEBUG:teuthology.orchestra.run.smithi169:> ' 2024-03-04T13:39:24.854 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:39:24 smithi169 ceph-mon[36313]: pgmap v243: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-03-04T13:39:24.904 INFO:teuthology.orchestra.run.smithi169.stdout:/var/run/ceph/ceph-client.1.61093.asok 2024-03-04T13:39:24.908 INFO:teuthology.orchestra.run.smithi169.stderr:2024-03-04T13:39:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-04T13:39:24.922 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.61093.asok 2024-03-04T13:39:24.923 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T13:39:24.923 DEBUG:teuthology.orchestra.run.smithi169:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.61093.asok status 2024-03-04T13:39:25.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:39:24 smithi107 ceph-mon[28739]: pgmap v243: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-03-04T13:39:25.156 INFO:teuthology.orchestra.run.smithi169.stdout:{ 2024-03-04T13:39:25.157 INFO:teuthology.orchestra.run.smithi169.stdout: "metadata": { 2024-03-04T13:39:25.157 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_sha1": "42608bb191117be10dba720e6bf8a02f3171cd34", 2024-03-04T13:39:25.157 INFO:teuthology.orchestra.run.smithi169.stdout: "ceph_version": "ceph version 17.2.7-545-g42608bb1 (42608bb191117be10dba720e6bf8a02f3171cd34) quincy (stable)", 2024-03-04T13:39:25.157 INFO:teuthology.orchestra.run.smithi169.stdout: "entity_id": "1", 2024-03-04T13:39:25.157 INFO:teuthology.orchestra.run.smithi169.stdout: "hostname": "smithi169", 2024-03-04T13:39:25.157 INFO:teuthology.orchestra.run.smithi169.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-03-04T13:39:25.157 INFO:teuthology.orchestra.run.smithi169.stdout: "pid": "61093", 2024-03-04T13:39:25.157 INFO:teuthology.orchestra.run.smithi169.stdout: "root": "/" 2024-03-04T13:39:25.157 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-03-04T13:39:25.158 INFO:teuthology.orchestra.run.smithi169.stdout: "dentry_count": 0, 2024-03-04T13:39:25.158 INFO:teuthology.orchestra.run.smithi169.stdout: "dentry_pinned_count": 0, 2024-03-04T13:39:25.158 INFO:teuthology.orchestra.run.smithi169.stdout: "id": 24293, 2024-03-04T13:39:25.158 INFO:teuthology.orchestra.run.smithi169.stdout: "inst": { 2024-03-04T13:39:25.158 INFO:teuthology.orchestra.run.smithi169.stdout: "name": { 2024-03-04T13:39:25.158 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "client", 2024-03-04T13:39:25.158 INFO:teuthology.orchestra.run.smithi169.stdout: "num": 24293 2024-03-04T13:39:25.158 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-03-04T13:39:25.158 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": { 2024-03-04T13:39:25.158 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2024-03-04T13:39:25.158 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "192.168.0.1:0", 2024-03-04T13:39:25.159 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2380524016 2024-03-04T13:39:25.159 INFO:teuthology.orchestra.run.smithi169.stdout: } 2024-03-04T13:39:25.159 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-03-04T13:39:25.159 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": { 2024-03-04T13:39:25.159 INFO:teuthology.orchestra.run.smithi169.stdout: "type": "v1", 2024-03-04T13:39:25.159 INFO:teuthology.orchestra.run.smithi169.stdout: "addr": "192.168.0.1:0", 2024-03-04T13:39:25.159 INFO:teuthology.orchestra.run.smithi169.stdout: "nonce": 2380524016 2024-03-04T13:39:25.159 INFO:teuthology.orchestra.run.smithi169.stdout: }, 2024-03-04T13:39:25.159 INFO:teuthology.orchestra.run.smithi169.stdout: "inst_str": "client.24293 192.168.0.1:0/2380524016", 2024-03-04T13:39:25.159 INFO:teuthology.orchestra.run.smithi169.stdout: "addr_str": "192.168.0.1:0/2380524016", 2024-03-04T13:39:25.159 INFO:teuthology.orchestra.run.smithi169.stdout: "inode_count": 1, 2024-03-04T13:39:25.160 INFO:teuthology.orchestra.run.smithi169.stdout: "mds_epoch": 15, 2024-03-04T13:39:25.160 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_epoch": 47, 2024-03-04T13:39:25.160 INFO:teuthology.orchestra.run.smithi169.stdout: "osd_epoch_barrier": 0, 2024-03-04T13:39:25.160 INFO:teuthology.orchestra.run.smithi169.stdout: "blocklisted": false, 2024-03-04T13:39:25.160 INFO:teuthology.orchestra.run.smithi169.stdout: "fs_name": "cephfs" 2024-03-04T13:39:25.160 INFO:teuthology.orchestra.run.smithi169.stdout:} 2024-03-04T13:39:25.171 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T13:39:25.171 DEBUG:teuthology.orchestra.run.smithi107:> stat --file-system '--printf=%T 2024-03-04T13:39:25.171 DEBUG:teuthology.orchestra.run.smithi107:> ' -- /home/ubuntu/cephtest/mnt.0 2024-03-04T13:39:25.234 INFO:teuthology.orchestra.run.smithi107.stdout:fuseblk 2024-03-04T13:39:25.234 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-03-04T13:39:25.234 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T13:39:25.235 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-04T13:39:25.342 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T13:39:25.343 DEBUG:teuthology.orchestra.run.smithi169:> stat --file-system '--printf=%T 2024-03-04T13:39:25.343 DEBUG:teuthology.orchestra.run.smithi169:> ' -- /home/ubuntu/cephtest/mnt.1 2024-03-04T13:39:25.400 INFO:teuthology.orchestra.run.smithi169.stdout:fuseblk 2024-03-04T13:39:25.400 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-03-04T13:39:25.400 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T13:39:25.400 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-03-04T13:39:25.503 INFO:teuthology.run_tasks:Running task print... 2024-03-04T13:39:25.516 INFO:teuthology.task.print:**** done client 2024-03-04T13:39:25.517 INFO:teuthology.run_tasks:Running task parallel... 2024-03-04T13:39:25.542 INFO:teuthology.task.parallel:starting parallel... 2024-03-04T13:39:25.542 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-03-04T13:39:25.544 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-03-04T13:39:25.545 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi107.front.sepia.ceph.com 2024-03-04T13:39:25.545 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 -e sha1=717ce3cce8d0166e4e577faa8004e6f9cb4128c0 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force' 2024-03-04T13:39:25.546 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-03-04T13:39:25.546 INFO:teuthology.task.sequential:In sequential, running task workunit... 2024-03-04T13:39:25.549 INFO:tasks.workunit:Pulling workunits from ref 717ce3cce8d0166e4e577faa8004e6f9cb4128c0 2024-03-04T13:39:25.550 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-03-04T13:39:25.550 INFO:tasks.workunit:timeout=3h 2024-03-04T13:39:25.550 INFO:tasks.workunit:cleanup=True 2024-03-04T13:39:25.550 DEBUG:teuthology.orchestra.run.smithi107:> stat -- /home/ubuntu/cephtest/mnt.0 2024-03-04T13:39:25.606 INFO:teuthology.orchestra.run.smithi107.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-03-04T13:39:25.606 INFO:teuthology.orchestra.run.smithi107.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-03-04T13:39:25.606 INFO:teuthology.orchestra.run.smithi107.stdout:Device: 93h/147d Inode: 1 Links: 2 2024-03-04T13:39:25.606 INFO:teuthology.orchestra.run.smithi107.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-04T13:39:25.606 INFO:teuthology.orchestra.run.smithi107.stdout:Context: system_u:object_r:fusefs_t:s0 2024-03-04T13:39:25.606 INFO:teuthology.orchestra.run.smithi107.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-03-04T13:39:25.607 INFO:teuthology.orchestra.run.smithi107.stdout:Modify: 2024-03-04 13:38:35.585408251 +0000 2024-03-04T13:39:25.607 INFO:teuthology.orchestra.run.smithi107.stdout:Change: 2024-03-04 13:39:25.341126972 +0000 2024-03-04T13:39:25.607 INFO:teuthology.orchestra.run.smithi107.stdout: Birth: - 2024-03-04T13:39:25.607 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-03-04T13:39:25.607 DEBUG:teuthology.orchestra.run.smithi107:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-03-04T13:39:25.746 DEBUG:teuthology.orchestra.run.smithi169:> stat -- /home/ubuntu/cephtest/mnt.1 2024-03-04T13:39:25.804 INFO:teuthology.orchestra.run.smithi169.stdout: File: /home/ubuntu/cephtest/mnt.1 2024-03-04T13:39:25.804 INFO:teuthology.orchestra.run.smithi169.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-03-04T13:39:25.805 INFO:teuthology.orchestra.run.smithi169.stdout:Device: 72h/114d Inode: 1 Links: 3 2024-03-04T13:39:25.805 INFO:teuthology.orchestra.run.smithi169.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-04T13:39:25.805 INFO:teuthology.orchestra.run.smithi169.stdout:Context: system_u:object_r:fusefs_t:s0 2024-03-04T13:39:25.805 INFO:teuthology.orchestra.run.smithi169.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-03-04T13:39:25.805 INFO:teuthology.orchestra.run.smithi169.stdout:Modify: 2024-03-04 13:39:25.736851689 +0000 2024-03-04T13:39:25.805 INFO:teuthology.orchestra.run.smithi169.stdout:Change: 2024-03-04 13:39:25.736851689 +0000 2024-03-04T13:39:25.805 INFO:teuthology.orchestra.run.smithi169.stdout: Birth: - 2024-03-04T13:39:25.805 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.1 2024-03-04T13:39:25.805 DEBUG:teuthology.orchestra.run.smithi169:> cd -- /home/ubuntu/cephtest/mnt.1 && sudo install -d -m 0755 --owner=ubuntu -- client.1 2024-03-04T13:39:25.914 DEBUG:teuthology.orchestra.run.smithi107:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 717ce3cce8d0166e4e577faa8004e6f9cb4128c0 2024-03-04T13:39:25.915 DEBUG:teuthology.orchestra.run.smithi169:> rm -rf /home/ubuntu/cephtest/clone.client.1 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.1 && cd /home/ubuntu/cephtest/clone.client.1 && git checkout 717ce3cce8d0166e4e577faa8004e6f9cb4128c0 2024-03-04T13:39:25.956 INFO:tasks.workunit.client.0.smithi107.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-03-04T13:39:25.971 INFO:tasks.workunit.client.1.smithi169.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.1'... 2024-03-04T13:39:26.018 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/mon.smithi107/config 2024-03-04T13:39:26.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:39:26 smithi107 ceph-mon[28739]: pgmap v244: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.4 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-04T13:39:27.105 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:39:26 smithi169 ceph-mon[36313]: pgmap v244: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.4 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-04T13:39:28.035 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:39:27 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:39:28.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:39:27 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:39:28.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:28.208+0000 7fa91d600700 1 -- 172.21.15.107:0/723201744 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa918066c70 msgr2=0x7fa9180ffaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:39:28.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:28.208+0000 7fa91d600700 1 --2- 172.21.15.107:0/723201744 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa918066c70 0x7fa9180ffaf0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa908004940 tx=0x7fa908009e30 comp rx=0 tx=0).stop 2024-03-04T13:39:28.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:28.208+0000 7fa91d600700 1 -- 172.21.15.107:0/723201744 shutdown_connections 2024-03-04T13:39:28.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:28.208+0000 7fa91d600700 1 --2- 172.21.15.107:0/723201744 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa9181000c0 0x7fa918102520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:39:28.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:28.208+0000 7fa91d600700 1 --2- 172.21.15.107:0/723201744 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa918066c70 0x7fa9180ffaf0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:39:28.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:28.208+0000 7fa91d600700 1 -- 172.21.15.107:0/723201744 >> 172.21.15.107:0/723201744 conn(0x7fa9180f7520 msgr2=0x7fa9180f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:39:28.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:28.208+0000 7fa91d600700 1 -- 172.21.15.107:0/723201744 shutdown_connections 2024-03-04T13:39:28.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:28.208+0000 7fa91d600700 1 -- 172.21.15.107:0/723201744 wait complete. 2024-03-04T13:39:28.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:28.209+0000 7fa91d600700 1 Processor -- start 2024-03-04T13:39:28.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:28.210+0000 7fa91d600700 1 -- start start 2024-03-04T13:39:28.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:28.210+0000 7fa91d600700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa918066c70 0x7fa91810fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:39:28.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:28.210+0000 7fa91d600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9181000c0 0x7fa9181100b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:39:28.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:28.210+0000 7fa91d600700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9181106b0 con 0x7fa9181000c0 2024-03-04T13:39:28.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:28.210+0000 7fa91d600700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa918110f50 con 0x7fa918066c70 2024-03-04T13:39:28.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:28.210+0000 7fa916000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9181000c0 0x7fa9181100b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:39:28.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:28.210+0000 7fa916000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9181000c0 0x7fa9181100b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46614/0 (socket says 172.21.15.107:46614) 2024-03-04T13:39:28.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:28.210+0000 7fa916000700 1 -- 172.21.15.107:0/496233474 learned_addr learned my addr 172.21.15.107:0/496233474 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:39:28.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:28.211+0000 7fa916a00700 1 --2- 172.21.15.107:0/496233474 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa918066c70 0x7fa91810fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:39:28.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:28.211+0000 7fa916000700 1 -- 172.21.15.107:0/496233474 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa918066c70 msgr2=0x7fa91810fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:39:28.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:28.211+0000 7fa916000700 1 --2- 172.21.15.107:0/496233474 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa918066c70 0x7fa91810fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:39:28.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:28.211+0000 7fa916000700 1 -- 172.21.15.107:0/496233474 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa908005040 con 0x7fa9181000c0 2024-03-04T13:39:28.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:28.211+0000 7fa916000700 1 --2- 172.21.15.107:0/496233474 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9181000c0 0x7fa9181100b0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa904002a50 tx=0x7fa904004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:39:28.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:28.211+0000 7fa90ea00700 1 -- 172.21.15.107:0/496233474 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa904007cc0 con 0x7fa9181000c0 2024-03-04T13:39:28.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:28.211+0000 7fa90ea00700 1 -- 172.21.15.107:0/496233474 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa90400f040 con 0x7fa9181000c0 2024-03-04T13:39:28.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:28.211+0000 7fa90ea00700 1 -- 172.21.15.107:0/496233474 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa90400daf0 con 0x7fa9181000c0 2024-03-04T13:39:28.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:28.211+0000 7fa91d600700 1 -- 172.21.15.107:0/496233474 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa918111230 con 0x7fa9181000c0 2024-03-04T13:39:28.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:28.212+0000 7fa91d600700 1 -- 172.21.15.107:0/496233474 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa918111770 con 0x7fa9181000c0 2024-03-04T13:39:28.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:28.213+0000 7fa91d600700 1 -- 172.21.15.107:0/496233474 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa91804bed0 con 0x7fa9181000c0 2024-03-04T13:39:28.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:28.214+0000 7fa90ea00700 1 -- 172.21.15.107:0/496233474 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86623+0+0 (secure 0 0 0) 0x7fa904009810 con 0x7fa9181000c0 2024-03-04T13:39:28.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:28.214+0000 7fa90ea00700 1 --2- 172.21.15.107:0/496233474 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fa8fc0692b0 0x7fa8fc06b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:39:28.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:28.215+0000 7fa90ea00700 1 -- 172.21.15.107:0/496233474 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fa904086fe0 con 0x7fa9181000c0 2024-03-04T13:39:28.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:28.215+0000 7fa916a00700 1 --2- 172.21.15.107:0/496233474 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fa8fc0692b0 0x7fa8fc06b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:39:28.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:28.215+0000 7fa916a00700 1 --2- 172.21.15.107:0/496233474 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fa8fc0692b0 0x7fa8fc06b760 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa90800b3b0 tx=0x7fa908009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:39:28.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:28.219+0000 7fa90ea00700 1 -- 172.21.15.107:0/496233474 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fa904056c70 con 0x7fa9181000c0 2024-03-04T13:39:28.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:28.427+0000 7fa91d600700 1 -- 172.21.15.107:0/496233474 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}] v 0) v1 -- 0x7fa918105a30 con 0x7fa9181000c0 2024-03-04T13:39:28.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:28.428+0000 7fa90ea00700 1 -- 172.21.15.107:0/496233474 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}]=0 v21)=0 v21) v1 ==== 155+0+0 (secure 0 0 0) 0x7fa904056800 con 0x7fa9181000c0 2024-03-04T13:39:28.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:28.431+0000 7fa91d600700 1 -- 172.21.15.107:0/496233474 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fa8fc0692b0 msgr2=0x7fa8fc06b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:39:28.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:28.431+0000 7fa91d600700 1 --2- 172.21.15.107:0/496233474 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fa8fc0692b0 0x7fa8fc06b760 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa90800b3b0 tx=0x7fa908009e30 comp rx=0 tx=0).stop 2024-03-04T13:39:28.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:28.431+0000 7fa91d600700 1 -- 172.21.15.107:0/496233474 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9181000c0 msgr2=0x7fa9181100b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:39:28.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:28.431+0000 7fa91d600700 1 --2- 172.21.15.107:0/496233474 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9181000c0 0x7fa9181100b0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa904002a50 tx=0x7fa904004600 comp rx=0 tx=0).stop 2024-03-04T13:39:28.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:28.431+0000 7fa91d600700 1 -- 172.21.15.107:0/496233474 shutdown_connections 2024-03-04T13:39:28.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:28.431+0000 7fa91d600700 1 --2- 172.21.15.107:0/496233474 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fa8fc0692b0 0x7fa8fc06b760 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:39:28.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:28.431+0000 7fa91d600700 1 --2- 172.21.15.107:0/496233474 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa918066c70 0x7fa91810fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:39:28.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:28.431+0000 7fa91d600700 1 --2- 172.21.15.107:0/496233474 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9181000c0 0x7fa9181100b0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:39:28.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:28.431+0000 7fa91d600700 1 -- 172.21.15.107:0/496233474 >> 172.21.15.107:0/496233474 conn(0x7fa9180f7520 msgr2=0x7fa9180fb8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:39:28.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:28.431+0000 7fa91d600700 1 -- 172.21.15.107:0/496233474 shutdown_connections 2024-03-04T13:39:28.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:28.432+0000 7fa91d600700 1 -- 172.21.15.107:0/496233474 wait complete. 2024-03-04T13:39:29.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:39:28 smithi107 ceph-mon[28739]: pgmap v245: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 255 B/s wr, 2 op/s 2024-03-04T13:39:29.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:39:28 smithi169 ceph-mon[36313]: pgmap v245: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 255 B/s wr, 2 op/s 2024-03-04T13:39:29.237 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 -e sha1=717ce3cce8d0166e4e577faa8004e6f9cb4128c0 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force' 2024-03-04T13:39:29.772 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/mon.smithi107/config 2024-03-04T13:39:31.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:39:30 smithi107 ceph-mon[28739]: pgmap v246: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 255 B/s wr, 2 op/s 2024-03-04T13:39:31.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:39:30 smithi169 ceph-mon[36313]: pgmap v246: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 255 B/s wr, 2 op/s 2024-03-04T13:39:32.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:32.086+0000 7f2aa3800700 1 -- 172.21.15.107:0/4215156414 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2a9c073af0 msgr2=0x7f2a9c073f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:39:32.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:32.086+0000 7f2aa3800700 1 --2- 172.21.15.107:0/4215156414 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2a9c073af0 0x7f2a9c073f60 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f2a84004940 tx=0x7f2a84009e30 comp rx=0 tx=0).stop 2024-03-04T13:39:32.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:32.086+0000 7f2aa3800700 1 -- 172.21.15.107:0/4215156414 shutdown_connections 2024-03-04T13:39:32.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:32.086+0000 7f2aa3800700 1 --2- 172.21.15.107:0/4215156414 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2a9c073af0 0x7f2a9c073f60 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:39:32.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:32.086+0000 7f2aa3800700 1 --2- 172.21.15.107:0/4215156414 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a9c072890 0x7f2a9c072ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:39:32.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:32.086+0000 7f2aa3800700 1 -- 172.21.15.107:0/4215156414 >> 172.21.15.107:0/4215156414 conn(0x7f2a9c0fa920 msgr2=0x7f2a9c0fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:39:32.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:32.086+0000 7f2aa3800700 1 -- 172.21.15.107:0/4215156414 shutdown_connections 2024-03-04T13:39:32.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:32.086+0000 7f2aa3800700 1 -- 172.21.15.107:0/4215156414 wait complete. 2024-03-04T13:39:32.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:32.088+0000 7f2aa3800700 1 Processor -- start 2024-03-04T13:39:32.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:32.088+0000 7f2aa3800700 1 -- start start 2024-03-04T13:39:32.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:32.088+0000 7f2aa3800700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a9c072890 0x7f2a9c113f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:39:32.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:32.088+0000 7f2aa3800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2a9c073af0 0x7f2a9c1144c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:39:32.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:32.088+0000 7f2aa3800700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a9c114ac0 con 0x7f2a9c073af0 2024-03-04T13:39:32.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:32.088+0000 7f2aa3800700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a9c115360 con 0x7f2a9c072890 2024-03-04T13:39:32.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:32.088+0000 7f2a93e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2a9c073af0 0x7f2a9c1144c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:39:32.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:32.089+0000 7f2a93e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2a9c073af0 0x7f2a9c1144c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46636/0 (socket says 172.21.15.107:46636) 2024-03-04T13:39:32.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:32.089+0000 7f2a93e00700 1 -- 172.21.15.107:0/3055203626 learned_addr learned my addr 172.21.15.107:0/3055203626 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:39:32.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:32.089+0000 7f2aa0e00700 1 --2- 172.21.15.107:0/3055203626 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a9c072890 0x7f2a9c113f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:39:32.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:32.089+0000 7f2a93e00700 1 -- 172.21.15.107:0/3055203626 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a9c072890 msgr2=0x7f2a9c113f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:39:32.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:32.089+0000 7f2a93e00700 1 --2- 172.21.15.107:0/3055203626 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a9c072890 0x7f2a9c113f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:39:32.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:32.089+0000 7f2a93e00700 1 -- 172.21.15.107:0/3055203626 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a84005040 con 0x7f2a9c073af0 2024-03-04T13:39:32.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:32.089+0000 7f2aa0e00700 1 --2- 172.21.15.107:0/3055203626 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a9c072890 0x7f2a9c113f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T13:39:32.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:32.089+0000 7f2a93e00700 1 --2- 172.21.15.107:0/3055203626 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2a9c073af0 0x7f2a9c1144c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f2a8400b3b0 tx=0x7f2a84009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:39:32.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:32.089+0000 7f2a99600700 1 -- 172.21.15.107:0/3055203626 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a8400e040 con 0x7f2a9c073af0 2024-03-04T13:39:32.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:32.090+0000 7f2a99600700 1 -- 172.21.15.107:0/3055203626 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2a8400bd90 con 0x7f2a9c073af0 2024-03-04T13:39:32.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:32.090+0000 7f2a99600700 1 -- 172.21.15.107:0/3055203626 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a8401c630 con 0x7f2a9c073af0 2024-03-04T13:39:32.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:32.090+0000 7f2aa3800700 1 -- 172.21.15.107:0/3055203626 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a9c1155e0 con 0x7f2a9c073af0 2024-03-04T13:39:32.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:32.090+0000 7f2aa3800700 1 -- 172.21.15.107:0/3055203626 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a9c115b20 con 0x7f2a9c073af0 2024-03-04T13:39:32.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:32.091+0000 7f2aa3800700 1 -- 172.21.15.107:0/3055203626 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a9c04bed0 con 0x7f2a9c073af0 2024-03-04T13:39:32.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:32.092+0000 7f2a99600700 1 -- 172.21.15.107:0/3055203626 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86623+0+0 (secure 0 0 0) 0x7f2a840139f0 con 0x7f2a9c073af0 2024-03-04T13:39:32.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:32.093+0000 7f2a99600700 1 --2- 172.21.15.107:0/3055203626 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f2a800692b0 0x7f2a8006b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:39:32.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:32.093+0000 7f2a99600700 1 -- 172.21.15.107:0/3055203626 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f2a84088720 con 0x7f2a9c073af0 2024-03-04T13:39:32.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:32.093+0000 7f2aa0e00700 1 --2- 172.21.15.107:0/3055203626 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f2a800692b0 0x7f2a8006b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:39:32.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:32.094+0000 7f2aa0e00700 1 --2- 172.21.15.107:0/3055203626 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f2a800692b0 0x7f2a8006b760 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c001250 tx=0x7f2a8c00b040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:39:32.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:32.098+0000 7f2a99600700 1 -- 172.21.15.107:0/3055203626 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f2a84058520 con 0x7f2a9c073af0 2024-03-04T13:39:32.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:32.305+0000 7f2aa3800700 1 -- 172.21.15.107:0/3055203626 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}] v 0) v1 -- 0x7f2a9c1161d0 con 0x7f2a9c073af0 2024-03-04T13:39:32.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:32.306+0000 7f2a99600700 1 -- 172.21.15.107:0/3055203626 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}]=0 v21)=0 v21) v1 ==== 163+0+0 (secure 0 0 0) 0x7f2a9c1161d0 con 0x7f2a9c073af0 2024-03-04T13:39:32.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:32.309+0000 7f2aa3800700 1 -- 172.21.15.107:0/3055203626 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f2a800692b0 msgr2=0x7f2a8006b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:39:32.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:32.309+0000 7f2aa3800700 1 --2- 172.21.15.107:0/3055203626 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f2a800692b0 0x7f2a8006b760 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c001250 tx=0x7f2a8c00b040 comp rx=0 tx=0).stop 2024-03-04T13:39:32.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:32.309+0000 7f2aa3800700 1 -- 172.21.15.107:0/3055203626 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2a9c073af0 msgr2=0x7f2a9c1144c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:39:32.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:32.309+0000 7f2aa3800700 1 --2- 172.21.15.107:0/3055203626 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2a9c073af0 0x7f2a9c1144c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f2a8400b3b0 tx=0x7f2a84009e30 comp rx=0 tx=0).stop 2024-03-04T13:39:32.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:32.309+0000 7f2aa3800700 1 -- 172.21.15.107:0/3055203626 shutdown_connections 2024-03-04T13:39:32.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:32.309+0000 7f2aa3800700 1 --2- 172.21.15.107:0/3055203626 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f2a800692b0 0x7f2a8006b760 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:39:32.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:32.309+0000 7f2aa3800700 1 --2- 172.21.15.107:0/3055203626 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a9c072890 0x7f2a9c113f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:39:32.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:32.309+0000 7f2aa3800700 1 --2- 172.21.15.107:0/3055203626 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2a9c073af0 0x7f2a9c1144c0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:39:32.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:32.309+0000 7f2aa3800700 1 -- 172.21.15.107:0/3055203626 >> 172.21.15.107:0/3055203626 conn(0x7f2a9c0fa920 msgr2=0x7f2a9c109480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:39:32.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:32.309+0000 7f2aa3800700 1 -- 172.21.15.107:0/3055203626 shutdown_connections 2024-03-04T13:39:32.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:32.309+0000 7f2aa3800700 1 -- 172.21.15.107:0/3055203626 wait complete. 2024-03-04T13:39:33.088 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:39:32 smithi107 ceph-mon[28739]: pgmap v247: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 767 B/s wr, 3 op/s 2024-03-04T13:39:33.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:39:32 smithi169 ceph-mon[36313]: pgmap v247: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 767 B/s wr, 3 op/s 2024-03-04T13:39:33.221 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 -e sha1=717ce3cce8d0166e4e577faa8004e6f9cb4128c0 -- bash -c 'ceph config set global log_to_journald false --force' 2024-03-04T13:39:33.718 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/mon.smithi107/config 2024-03-04T13:39:35.070 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:39:34 smithi107 ceph-mon[28739]: pgmap v248: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 597 B/s wr, 2 op/s 2024-03-04T13:39:35.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:39:34 smithi169 ceph-mon[36313]: pgmap v248: 65 pgs: 65 active+clean; 584 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 597 B/s wr, 2 op/s 2024-03-04T13:39:35.913 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:35.918+0000 7f7e3fe00700 1 -- 172.21.15.107:0/1634508396 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e38103540 msgr2=0x7f7e38103950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:39:35.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:35.918+0000 7f7e3fe00700 1 --2- 172.21.15.107:0/1634508396 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e38103540 0x7f7e38103950 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c004940 tx=0x7f7e2c009e30 comp rx=0 tx=0).stop 2024-03-04T13:39:35.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:35.919+0000 7f7e3fe00700 1 -- 172.21.15.107:0/1634508396 shutdown_connections 2024-03-04T13:39:35.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:35.919+0000 7f7e3fe00700 1 --2- 172.21.15.107:0/1634508396 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e38104750 0x7f7e38104ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:39:35.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:35.919+0000 7f7e3fe00700 1 --2- 172.21.15.107:0/1634508396 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e38103540 0x7f7e38103950 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:39:35.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:35.919+0000 7f7e3fe00700 1 -- 172.21.15.107:0/1634508396 >> 172.21.15.107:0/1634508396 conn(0x7f7e380727d0 msgr2=0x7f7e38074c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:39:35.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:35.919+0000 7f7e3fe00700 1 -- 172.21.15.107:0/1634508396 shutdown_connections 2024-03-04T13:39:35.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:35.919+0000 7f7e3fe00700 1 -- 172.21.15.107:0/1634508396 wait complete. 2024-03-04T13:39:35.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:35.920+0000 7f7e3fe00700 1 Processor -- start 2024-03-04T13:39:35.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:35.920+0000 7f7e3fe00700 1 -- start start 2024-03-04T13:39:35.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:35.920+0000 7f7e3fe00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e38104750 0x7f7e38114120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:39:35.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:35.920+0000 7f7e3fe00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e38114660 0x7f7e38115500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:39:35.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:35.920+0000 7f7e3fe00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e38114b20 con 0x7f7e38114660 2024-03-04T13:39:35.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:35.920+0000 7f7e3fe00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e38114c60 con 0x7f7e38104750 2024-03-04T13:39:35.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:35.921+0000 7f7e3ca00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e38114660 0x7f7e38115500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:39:35.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:35.921+0000 7f7e3ca00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e38114660 0x7f7e38115500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46650/0 (socket says 172.21.15.107:46650) 2024-03-04T13:39:35.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:35.921+0000 7f7e3ca00700 1 -- 172.21.15.107:0/3151029558 learned_addr learned my addr 172.21.15.107:0/3151029558 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:39:35.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:35.921+0000 7f7e3d400700 1 --2- 172.21.15.107:0/3151029558 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e38104750 0x7f7e38114120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:39:35.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:35.921+0000 7f7e3ca00700 1 -- 172.21.15.107:0/3151029558 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e38104750 msgr2=0x7f7e38114120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:39:35.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:35.921+0000 7f7e3ca00700 1 --2- 172.21.15.107:0/3151029558 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e38104750 0x7f7e38114120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:39:35.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:35.921+0000 7f7e3ca00700 1 -- 172.21.15.107:0/3151029558 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e2c005040 con 0x7f7e38114660 2024-03-04T13:39:35.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:35.922+0000 7f7e3ca00700 1 --2- 172.21.15.107:0/3151029558 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e38114660 0x7f7e38115500 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f7e24002a50 tx=0x7f7e240046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:39:35.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:35.922+0000 7f7e35600700 1 -- 172.21.15.107:0/3151029558 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7e24007cc0 con 0x7f7e38114660 2024-03-04T13:39:35.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:35.922+0000 7f7e35600700 1 -- 172.21.15.107:0/3151029558 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7e2400f040 con 0x7f7e38114660 2024-03-04T13:39:35.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:35.922+0000 7f7e3fe00700 1 -- 172.21.15.107:0/3151029558 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e38115b60 con 0x7f7e38114660 2024-03-04T13:39:35.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:35.922+0000 7f7e35600700 1 -- 172.21.15.107:0/3151029558 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7e24009640 con 0x7f7e38114660 2024-03-04T13:39:35.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:35.922+0000 7f7e3fe00700 1 -- 172.21.15.107:0/3151029558 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e38116070 con 0x7f7e38114660 2024-03-04T13:39:35.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:35.923+0000 7f7e3fe00700 1 -- 172.21.15.107:0/3151029558 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e3810e060 con 0x7f7e38114660 2024-03-04T13:39:35.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:35.924+0000 7f7e35600700 1 -- 172.21.15.107:0/3151029558 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86623+0+0 (secure 0 0 0) 0x7f7e240097a0 con 0x7f7e38114660 2024-03-04T13:39:35.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:35.926+0000 7f7e35600700 1 --2- 172.21.15.107:0/3151029558 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f7e200692b0 0x7f7e2006b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:39:35.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:35.926+0000 7f7e35600700 1 -- 172.21.15.107:0/3151029558 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f7e24086f60 con 0x7f7e38114660 2024-03-04T13:39:35.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:35.926+0000 7f7e3d400700 1 --2- 172.21.15.107:0/3151029558 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f7e200692b0 0x7f7e2006b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:39:35.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:35.927+0000 7f7e3d400700 1 --2- 172.21.15.107:0/3151029558 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f7e200692b0 0x7f7e2006b760 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c00b840 tx=0x7f7e2c009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:39:35.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:35.931+0000 7f7e35600700 1 -- 172.21.15.107:0/3151029558 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f7e24056c70 con 0x7f7e38114660 2024-03-04T13:39:36.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:36.138+0000 7f7e3fe00700 1 -- 172.21.15.107:0/3151029558 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command([{prefix=config set, name=log_to_journald}] v 0) v1 -- 0x7f7e3804bed0 con 0x7f7e38114660 2024-03-04T13:39:36.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:36.140+0000 7f7e35600700 1 -- 172.21.15.107:0/3151029558 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{prefix=config set, name=log_to_journald}]=0 v21)=0 v21) v1 ==== 135+0+0 (secure 0 0 0) 0x7f7e24056800 con 0x7f7e38114660 2024-03-04T13:39:36.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:36.142+0000 7f7e3fe00700 1 -- 172.21.15.107:0/3151029558 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f7e200692b0 msgr2=0x7f7e2006b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:39:36.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:36.142+0000 7f7e3fe00700 1 --2- 172.21.15.107:0/3151029558 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f7e200692b0 0x7f7e2006b760 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c00b840 tx=0x7f7e2c009e30 comp rx=0 tx=0).stop 2024-03-04T13:39:36.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:36.142+0000 7f7e3fe00700 1 -- 172.21.15.107:0/3151029558 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e38114660 msgr2=0x7f7e38115500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:39:36.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:36.142+0000 7f7e3fe00700 1 --2- 172.21.15.107:0/3151029558 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e38114660 0x7f7e38115500 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f7e24002a50 tx=0x7f7e240046b0 comp rx=0 tx=0).stop 2024-03-04T13:39:36.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:36.142+0000 7f7e3fe00700 1 -- 172.21.15.107:0/3151029558 shutdown_connections 2024-03-04T13:39:36.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:36.142+0000 7f7e3fe00700 1 --2- 172.21.15.107:0/3151029558 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f7e200692b0 0x7f7e2006b760 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:39:36.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:36.142+0000 7f7e3fe00700 1 --2- 172.21.15.107:0/3151029558 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e38104750 0x7f7e38114120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:39:36.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:36.142+0000 7f7e3fe00700 1 --2- 172.21.15.107:0/3151029558 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e38114660 0x7f7e38115500 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:39:36.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:36.142+0000 7f7e3fe00700 1 -- 172.21.15.107:0/3151029558 >> 172.21.15.107:0/3151029558 conn(0x7f7e380727d0 msgr2=0x7f7e38075b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:39:36.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:36.143+0000 7f7e3fe00700 1 -- 172.21.15.107:0/3151029558 shutdown_connections 2024-03-04T13:39:36.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:36.143+0000 7f7e3fe00700 1 -- 172.21.15.107:0/3151029558 wait complete. 2024-03-04T13:39:36.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:39:36 smithi107 ceph-mon[28739]: pgmap v249: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 1.1 KiB/s wr, 3 op/s 2024-03-04T13:39:37.019 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 -e sha1=717ce3cce8d0166e4e577faa8004e6f9cb4128c0 -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 --daemon-types mgr' 2024-03-04T13:39:37.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:39:36 smithi169 ceph-mon[36313]: pgmap v249: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 1.1 KiB/s wr, 3 op/s 2024-03-04T13:39:37.512 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/mon.smithi107/config 2024-03-04T13:39:39.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:39:38 smithi107 ceph-mon[28739]: pgmap v250: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 1023 B/s wr, 2 op/s 2024-03-04T13:39:39.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:39:38 smithi169 ceph-mon[36313]: pgmap v250: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 1023 B/s wr, 2 op/s 2024-03-04T13:39:39.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:39.619+0000 7fafa9000700 1 -- 172.21.15.107:0/2881086478 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fafa40fcef0 msgr2=0x7fafa40fd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:39:39.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:39.619+0000 7fafa9000700 1 --2- 172.21.15.107:0/2881086478 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fafa40fcef0 0x7fafa40fd320 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7faf94004940 tx=0x7faf94009e30 comp rx=0 tx=0).stop 2024-03-04T13:39:39.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:39.619+0000 7fafa9000700 1 -- 172.21.15.107:0/2881086478 shutdown_connections 2024-03-04T13:39:39.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:39.619+0000 7fafa9000700 1 --2- 172.21.15.107:0/2881086478 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fafa40fe0e0 0x7fafa40fe550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:39:39.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:39.619+0000 7fafa9000700 1 --2- 172.21.15.107:0/2881086478 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fafa40fcef0 0x7fafa40fd320 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:39:39.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:39.619+0000 7fafa9000700 1 -- 172.21.15.107:0/2881086478 >> 172.21.15.107:0/2881086478 conn(0x7fafa40f8620 msgr2=0x7fafa40faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:39:39.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:39.619+0000 7fafa9000700 1 -- 172.21.15.107:0/2881086478 shutdown_connections 2024-03-04T13:39:39.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:39.619+0000 7fafa9000700 1 -- 172.21.15.107:0/2881086478 wait complete. 2024-03-04T13:39:39.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:39.620+0000 7fafa9000700 1 Processor -- start 2024-03-04T13:39:39.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:39.620+0000 7fafa9000700 1 -- start start 2024-03-04T13:39:39.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:39.621+0000 7fafa9000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fafa40fcef0 0x7fafa410d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:39:39.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:39.621+0000 7fafa9000700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fafa40fe0e0 0x7fafa410de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:39:39.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:39.621+0000 7fafa9000700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafa410e420 con 0x7fafa40fcef0 2024-03-04T13:39:39.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:39.621+0000 7fafa9000700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafa410ecc0 con 0x7fafa40fe0e0 2024-03-04T13:39:39.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:39.621+0000 7fafa1e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fafa40fcef0 0x7fafa410d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:39:39.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:39.621+0000 7fafa1e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fafa40fcef0 0x7fafa410d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47962/0 (socket says 172.21.15.107:47962) 2024-03-04T13:39:39.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:39.621+0000 7fafa1e00700 1 -- 172.21.15.107:0/4080746776 learned_addr learned my addr 172.21.15.107:0/4080746776 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:39:39.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:39.621+0000 7fafa1400700 1 --2- 172.21.15.107:0/4080746776 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fafa40fe0e0 0x7fafa410de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:39:39.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:39.621+0000 7fafa1e00700 1 -- 172.21.15.107:0/4080746776 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fafa40fe0e0 msgr2=0x7fafa410de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:39:39.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:39.621+0000 7fafa1e00700 1 --2- 172.21.15.107:0/4080746776 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fafa40fe0e0 0x7fafa410de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:39:39.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:39.621+0000 7fafa1e00700 1 -- 172.21.15.107:0/4080746776 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf94005040 con 0x7fafa40fcef0 2024-03-04T13:39:39.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:39.622+0000 7fafa1e00700 1 --2- 172.21.15.107:0/4080746776 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fafa40fcef0 0x7fafa410d8e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7faf94000c00 tx=0x7faf94009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:39:39.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:39.622+0000 7faf9a000700 1 -- 172.21.15.107:0/4080746776 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faf9400e070 con 0x7fafa40fcef0 2024-03-04T13:39:39.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:39.622+0000 7faf9a000700 1 -- 172.21.15.107:0/4080746776 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faf9401c400 con 0x7fafa40fcef0 2024-03-04T13:39:39.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:39.622+0000 7fafa9000700 1 -- 172.21.15.107:0/4080746776 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafa410ef40 con 0x7fafa40fcef0 2024-03-04T13:39:39.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:39.622+0000 7faf9a000700 1 -- 172.21.15.107:0/4080746776 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faf9400b670 con 0x7fafa40fcef0 2024-03-04T13:39:39.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:39.622+0000 7fafa9000700 1 -- 172.21.15.107:0/4080746776 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafa410f420 con 0x7fafa40fcef0 2024-03-04T13:39:39.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:39.623+0000 7fafa9000700 1 -- 172.21.15.107:0/4080746776 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafa404bed0 con 0x7fafa40fcef0 2024-03-04T13:39:39.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:39.624+0000 7faf9a000700 1 -- 172.21.15.107:0/4080746776 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86623+0+0 (secure 0 0 0) 0x7faf9401c9e0 con 0x7fafa40fcef0 2024-03-04T13:39:39.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:39.625+0000 7faf9a000700 1 --2- 172.21.15.107:0/4080746776 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7faf880692a0 0x7faf8806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:39:39.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:39.625+0000 7faf9a000700 1 -- 172.21.15.107:0/4080746776 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5550+0+0 (secure 0 0 0) 0x7faf94088970 con 0x7fafa40fcef0 2024-03-04T13:39:39.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:39.625+0000 7fafa1400700 1 --2- 172.21.15.107:0/4080746776 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7faf880692a0 0x7faf8806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:39:39.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:39.626+0000 7fafa1400700 1 --2- 172.21.15.107:0/4080746776 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7faf880692a0 0x7faf8806b750 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7faf90009fd0 tx=0x7faf9000b040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:39:39.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:39.629+0000 7faf9a000700 1 -- 172.21.15.107:0/4080746776 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7faf94058640 con 0x7fafa40fcef0 2024-03-04T13:39:39.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:39:39.857+0000 7fafa9000700 1 -- 172.21.15.107:0/4080746776 --> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] -- mgr_command(tid 0: {"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:717ce3cce8d0166e4e577faa8004e6f9cb4128c0", "daemon_types": "mgr", "target": ["mon-mgr", ""]}) v1 -- 0x7fafa4102ac0 con 0x7faf880692a0 2024-03-04T13:39:41.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:39:40 smithi107 ceph-mon[28739]: pgmap v251: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 1023 B/s wr, 2 op/s 2024-03-04T13:39:41.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:39:40 smithi107 ceph-mon[28739]: from='client.14548 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:717ce3cce8d0166e4e577faa8004e6f9cb4128c0", "daemon_types": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:39:41.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:39:40 smithi169 ceph-mon[36313]: pgmap v251: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 1023 B/s wr, 2 op/s 2024-03-04T13:39:41.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:39:40 smithi169 ceph-mon[36313]: from='client.14548 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:717ce3cce8d0166e4e577faa8004e6f9cb4128c0", "daemon_types": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:39:43.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:39:42 smithi107 ceph-mon[28739]: pgmap v252: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 1023 B/s wr, 3 op/s 2024-03-04T13:39:43.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:39:42 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:39:43.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:39:42 smithi169 ceph-mon[36313]: pgmap v252: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 1023 B/s wr, 3 op/s 2024-03-04T13:39:43.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:39:42 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:39:45.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:39:44 smithi107 ceph-mon[28739]: pgmap v253: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-04T13:39:45.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:39:44 smithi169 ceph-mon[36313]: pgmap v253: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-04T13:39:47.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:39:46 smithi107 ceph-mon[28739]: pgmap v254: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-04T13:39:47.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:39:46 smithi169 ceph-mon[36313]: pgmap v254: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-04T13:39:49.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:39:48 smithi107 ceph-mon[28739]: pgmap v255: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:39:49.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:39:48 smithi169 ceph-mon[36313]: pgmap v255: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:39:51.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:39:50 smithi107 ceph-mon[28739]: pgmap v256: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:39:51.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:39:50 smithi169 ceph-mon[36313]: pgmap v256: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:39:52.854 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:39:52 smithi169 ceph-mon[36313]: pgmap v257: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:39:53.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:39:52 smithi107 ceph-mon[28739]: pgmap v257: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:39:55.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:39:54 smithi107 ceph-mon[28739]: pgmap v258: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:39:55.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:39:54 smithi169 ceph-mon[36313]: pgmap v258: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:39:57.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:39:56 smithi107 ceph-mon[28739]: pgmap v259: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:39:57.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:39:56 smithi169 ceph-mon[36313]: pgmap v259: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:39:58.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:39:57 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:39:58.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:39:57 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:39:59.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:39:58 smithi107 ceph-mon[28739]: pgmap v260: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:39:59.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:39:58 smithi169 ceph-mon[36313]: pgmap v260: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:40:01.088 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:40:00 smithi107 ceph-mon[28739]: pgmap v261: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:40:01.088 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:40:00 smithi107 ceph-mon[28739]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-03-04T13:40:01.088 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:40:00 smithi107 ceph-mon[28739]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-04T13:40:01.088 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:40:00 smithi107 ceph-mon[28739]: fs cephfs has deprecated feature inline_data enabled. 2024-03-04T13:40:01.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:40:00 smithi169 ceph-mon[36313]: pgmap v261: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:40:01.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:40:00 smithi169 ceph-mon[36313]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-03-04T13:40:01.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:40:00 smithi169 ceph-mon[36313]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-04T13:40:01.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:40:00 smithi169 ceph-mon[36313]: fs cephfs has deprecated feature inline_data enabled. 2024-03-04T13:40:03.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:40:02 smithi107 ceph-mon[28739]: pgmap v262: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:40:03.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:40:02 smithi169 ceph-mon[36313]: pgmap v262: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:40:05.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:40:04 smithi107 ceph-mon[28739]: pgmap v263: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:40:05.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:40:04 smithi169 ceph-mon[36313]: pgmap v263: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:40:07.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:40:06 smithi107 ceph-mon[28739]: pgmap v264: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:40:07.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:40:06 smithi169 ceph-mon[36313]: pgmap v264: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:40:09.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:40:08 smithi107 ceph-mon[28739]: pgmap v265: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:40:09.088 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:40:08 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:40:09.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:40:08 smithi169 ceph-mon[36313]: pgmap v265: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:40:09.105 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:40:08 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:40:11.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:40:10 smithi107 ceph-mon[28739]: pgmap v266: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:40:11.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:40:10 smithi169 ceph-mon[36313]: pgmap v266: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:40:13.074 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:40:12 smithi107 ceph-mon[28739]: pgmap v267: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:40:13.074 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:40:13 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:40:13.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:40:12 smithi169 ceph-mon[36313]: pgmap v267: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:40:13.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:40:12 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:40:13.105 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:40:12 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:40:13.105 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:40:12 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:40:13.105 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:40:12 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:40:13.337 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:40:13 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:40:13.338 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:40:13 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:40:13.338 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:40:13 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:40:15.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:40:14 smithi107 ceph-mon[28739]: pgmap v268: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:40:15.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:40:14 smithi169 ceph-mon[36313]: pgmap v268: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:40:17.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:40:16 smithi107 ceph-mon[28739]: pgmap v269: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:40:17.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:40:16 smithi169 ceph-mon[36313]: pgmap v269: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:40:18.930 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:40:18 smithi107 ceph-mon[28739]: pgmap v270: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:40:19.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:40:18 smithi169 ceph-mon[36313]: pgmap v270: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:40:19.427 INFO:teuthology.orchestra.run.smithi107.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:717ce3cce8d0166e4e577faa8004e6f9cb4128c0 2024-03-04T13:40:19.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:19.432+0000 7faf9a000700 1 -- 172.21.15.107:0/4080746776 <== mgr.14212 v2:172.21.15.107:6800/3848495603 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+89 (secure 0 0 0) 0x7fafa4102ac0 con 0x7faf880692a0 2024-03-04T13:40:19.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:19.435+0000 7fafa9000700 1 -- 172.21.15.107:0/4080746776 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7faf880692a0 msgr2=0x7faf8806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:40:19.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:19.435+0000 7fafa9000700 1 --2- 172.21.15.107:0/4080746776 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7faf880692a0 0x7faf8806b750 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7faf90009fd0 tx=0x7faf9000b040 comp rx=0 tx=0).stop 2024-03-04T13:40:19.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:19.435+0000 7fafa9000700 1 -- 172.21.15.107:0/4080746776 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fafa40fcef0 msgr2=0x7fafa410d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:40:19.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:19.435+0000 7fafa9000700 1 --2- 172.21.15.107:0/4080746776 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fafa40fcef0 0x7fafa410d8e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7faf94000c00 tx=0x7faf94009e30 comp rx=0 tx=0).stop 2024-03-04T13:40:19.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:19.435+0000 7fafa9000700 1 -- 172.21.15.107:0/4080746776 shutdown_connections 2024-03-04T13:40:19.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:19.435+0000 7fafa9000700 1 --2- 172.21.15.107:0/4080746776 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7faf880692a0 0x7faf8806b750 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:40:19.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:19.435+0000 7fafa9000700 1 --2- 172.21.15.107:0/4080746776 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fafa40fcef0 0x7fafa410d8e0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:40:19.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:19.435+0000 7fafa9000700 1 --2- 172.21.15.107:0/4080746776 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fafa40fe0e0 0x7fafa410de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:40:19.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:19.435+0000 7fafa9000700 1 -- 172.21.15.107:0/4080746776 >> 172.21.15.107:0/4080746776 conn(0x7fafa40f8620 msgr2=0x7fafa41013a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:40:19.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:19.436+0000 7fafa9000700 1 -- 172.21.15.107:0/4080746776 shutdown_connections 2024-03-04T13:40:19.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:19.436+0000 7fafa9000700 1 -- 172.21.15.107:0/4080746776 wait complete. 2024-03-04T13:40:20.216 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 -e sha1=717ce3cce8d0166e4e577faa8004e6f9cb4128c0 -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true && ! ceph orch upgrade status | jq '"'"'.message'"'"' | grep Error ; do ceph orch ps ; ceph versions ; ceph orch upgrade status ; sleep 30 ; done' 2024-03-04T13:40:20.425 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:40:20 smithi107 ceph-mon[28739]: pgmap v271: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:40:20.425 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:40:20 smithi107 ceph-mon[28739]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:717ce3cce8d0166e4e577faa8004e6f9cb4128c0 2024-03-04T13:40:20.425 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:40:20 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:40:20.426 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:40:20 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:40:20.426 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:40:20 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:40:20.426 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:40:20 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:40:20.426 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:40:20 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:40:20.426 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:40:20 smithi107 ceph-mon[28739]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:717ce3cce8d0166e4e577faa8004e6f9cb4128c0 2024-03-04T13:40:20.842 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/mon.smithi107/config 2024-03-04T13:40:20.854 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:40:20 smithi169 ceph-mon[36313]: pgmap v271: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:40:20.855 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:40:20 smithi169 ceph-mon[36313]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:717ce3cce8d0166e4e577faa8004e6f9cb4128c0 2024-03-04T13:40:20.855 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:40:20 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:40:20.855 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:40:20 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:40:20.855 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:40:20 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:40:20.855 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:40:20 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:40:20.855 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:40:20 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:40:20.855 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:40:20 smithi169 ceph-mon[36313]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:717ce3cce8d0166e4e577faa8004e6f9cb4128c0 2024-03-04T13:40:23.066 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:40:22 smithi107 ceph-mon[28739]: pgmap v272: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:40:23.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:40:22 smithi169 ceph-mon[36313]: pgmap v272: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:40:24.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:24.105+0000 7f95d5800700 1 -- 172.21.15.107:0/2922051018 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95d0071f10 msgr2=0x7f95d0070380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:40:24.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:24.105+0000 7f95d5800700 1 --2- 172.21.15.107:0/2922051018 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95d0071f10 0x7f95d0070380 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f95c0004940 tx=0x7f95c0009e30 comp rx=0 tx=0).stop 2024-03-04T13:40:24.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:24.105+0000 7f95d5800700 1 -- 172.21.15.107:0/2922051018 shutdown_connections 2024-03-04T13:40:24.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:24.105+0000 7f95d5800700 1 --2- 172.21.15.107:0/2922051018 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95d00708c0 0x7f95d0070d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:40:24.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:24.105+0000 7f95d5800700 1 --2- 172.21.15.107:0/2922051018 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95d0071f10 0x7f95d0070380 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:40:24.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:24.105+0000 7f95d5800700 1 -- 172.21.15.107:0/2922051018 >> 172.21.15.107:0/2922051018 conn(0x7f95d00f8620 msgr2=0x7f95d00faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:40:24.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:24.105+0000 7f95d5800700 1 -- 172.21.15.107:0/2922051018 shutdown_connections 2024-03-04T13:40:24.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:24.105+0000 7f95d5800700 1 -- 172.21.15.107:0/2922051018 wait complete. 2024-03-04T13:40:24.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:24.106+0000 7f95d5800700 1 Processor -- start 2024-03-04T13:40:24.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:24.107+0000 7f95d5800700 1 -- start start 2024-03-04T13:40:24.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:24.107+0000 7f95d5800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95d00708c0 0x7f95d006f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:40:24.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:24.107+0000 7f95d5800700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95d0071f10 0x7f95d006f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:40:24.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:24.107+0000 7f95d5800700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95d006fc30 con 0x7f95d00708c0 2024-03-04T13:40:24.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:24.107+0000 7f95d5800700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95d0119830 con 0x7f95d0071f10 2024-03-04T13:40:24.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:24.107+0000 7f95cea00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95d00708c0 0x7f95d006f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:40:24.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:24.107+0000 7f95cea00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95d00708c0 0x7f95d006f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56980/0 (socket says 172.21.15.107:56980) 2024-03-04T13:40:24.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:24.107+0000 7f95cea00700 1 -- 172.21.15.107:0/1319749594 learned_addr learned my addr 172.21.15.107:0/1319749594 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:40:24.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:24.108+0000 7f95ce000700 1 --2- 172.21.15.107:0/1319749594 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95d0071f10 0x7f95d006f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:40:24.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:24.108+0000 7f95cea00700 1 -- 172.21.15.107:0/1319749594 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95d0071f10 msgr2=0x7f95d006f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:40:24.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:24.108+0000 7f95cea00700 1 --2- 172.21.15.107:0/1319749594 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95d0071f10 0x7f95d006f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:40:24.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:24.108+0000 7f95cea00700 1 -- 172.21.15.107:0/1319749594 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95c0005040 con 0x7f95d00708c0 2024-03-04T13:40:24.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:24.108+0000 7f95ce000700 1 --2- 172.21.15.107:0/1319749594 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95d0071f10 0x7f95d006f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T13:40:24.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:24.108+0000 7f95cea00700 1 --2- 172.21.15.107:0/1319749594 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95d00708c0 0x7f95d006f0f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f95c0009fd0 tx=0x7f95c0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:40:24.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:24.109+0000 7f95c6a00700 1 -- 172.21.15.107:0/1319749594 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f95c000e040 con 0x7f95d00708c0 2024-03-04T13:40:24.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:24.109+0000 7f95c6a00700 1 -- 172.21.15.107:0/1319749594 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f95c0004290 con 0x7f95d00708c0 2024-03-04T13:40:24.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:24.109+0000 7f95c6a00700 1 -- 172.21.15.107:0/1319749594 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f95c001c630 con 0x7f95d00708c0 2024-03-04T13:40:24.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:24.109+0000 7f95d5800700 1 -- 172.21.15.107:0/1319749594 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95d0119ab0 con 0x7f95d00708c0 2024-03-04T13:40:24.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:24.109+0000 7f95d5800700 1 -- 172.21.15.107:0/1319749594 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95d0119ff0 con 0x7f95d00708c0 2024-03-04T13:40:24.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:24.110+0000 7f95d5800700 1 -- 172.21.15.107:0/1319749594 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95d004bed0 con 0x7f95d00708c0 2024-03-04T13:40:24.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:24.112+0000 7f95c6a00700 1 -- 172.21.15.107:0/1319749594 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86623+0+0 (secure 0 0 0) 0x7f95c000bc60 con 0x7f95d00708c0 2024-03-04T13:40:24.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:24.112+0000 7f95c6a00700 1 --2- 172.21.15.107:0/1319749594 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f95b40692b0 0x7f95b406b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:40:24.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:24.112+0000 7f95c6a00700 1 -- 172.21.15.107:0/1319749594 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f95c00890e0 con 0x7f95d00708c0 2024-03-04T13:40:24.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:24.114+0000 7f95ce000700 1 --2- 172.21.15.107:0/1319749594 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f95b40692b0 0x7f95b406b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:40:24.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:24.115+0000 7f95ce000700 1 --2- 172.21.15.107:0/1319749594 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f95b40692b0 0x7f95b406b760 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f95bc009fd0 tx=0x7f95bc009f20 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:40:24.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:24.121+0000 7f95c6a00700 1 -- 172.21.15.107:0/1319749594 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f95c0058e30 con 0x7f95d00708c0 2024-03-04T13:40:24.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:24.472+0000 7f95d5800700 1 -- 172.21.15.107:0/1319749594 --> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f95d01044f0 con 0x7f95b40692b0 2024-03-04T13:40:24.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:24.474+0000 7f95c6a00700 1 -- 172.21.15.107:0/1319749594 <== mgr.14212 v2:172.21.15.107:6800/3848495603 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7f95d01044f0 con 0x7f95b40692b0 2024-03-04T13:40:24.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:24.476+0000 7f95d5800700 1 -- 172.21.15.107:0/1319749594 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f95b40692b0 msgr2=0x7f95b406b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:40:24.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:24.476+0000 7f95d5800700 1 --2- 172.21.15.107:0/1319749594 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f95b40692b0 0x7f95b406b760 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f95bc009fd0 tx=0x7f95bc009f20 comp rx=0 tx=0).stop 2024-03-04T13:40:24.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:24.476+0000 7f95d5800700 1 -- 172.21.15.107:0/1319749594 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95d00708c0 msgr2=0x7f95d006f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:40:24.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:24.476+0000 7f95d5800700 1 --2- 172.21.15.107:0/1319749594 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95d00708c0 0x7f95d006f0f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f95c0009fd0 tx=0x7f95c0009e30 comp rx=0 tx=0).stop 2024-03-04T13:40:24.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:24.477+0000 7f95d5800700 1 -- 172.21.15.107:0/1319749594 shutdown_connections 2024-03-04T13:40:24.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:24.477+0000 7f95d5800700 1 --2- 172.21.15.107:0/1319749594 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f95b40692b0 0x7f95b406b760 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:40:24.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:24.477+0000 7f95d5800700 1 --2- 172.21.15.107:0/1319749594 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95d00708c0 0x7f95d006f0f0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:40:24.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:24.477+0000 7f95d5800700 1 --2- 172.21.15.107:0/1319749594 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95d0071f10 0x7f95d006f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:40:24.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:24.477+0000 7f95d5800700 1 -- 172.21.15.107:0/1319749594 >> 172.21.15.107:0/1319749594 conn(0x7f95d00f8620 msgr2=0x7f95d0102dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:40:24.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:24.477+0000 7f95d5800700 1 -- 172.21.15.107:0/1319749594 shutdown_connections 2024-03-04T13:40:24.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:24.477+0000 7f95d5800700 1 -- 172.21.15.107:0/1319749594 wait complete. 2024-03-04T13:40:24.492 INFO:teuthology.orchestra.run.smithi107.stdout:true 2024-03-04T13:40:24.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:24.662+0000 7f373fa00700 1 -- 172.21.15.107:0/498730390 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f37380fe0e0 msgr2=0x7f37380fe550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:40:24.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:24.662+0000 7f373fa00700 1 --2- 172.21.15.107:0/498730390 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f37380fe0e0 0x7f37380fe550 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f3724004940 tx=0x7f3724009e30 comp rx=0 tx=0).stop 2024-03-04T13:40:24.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:24.662+0000 7f373fa00700 1 -- 172.21.15.107:0/498730390 shutdown_connections 2024-03-04T13:40:24.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:24.662+0000 7f373fa00700 1 --2- 172.21.15.107:0/498730390 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f37380fe0e0 0x7f37380fe550 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:40:24.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:24.662+0000 7f373fa00700 1 --2- 172.21.15.107:0/498730390 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37380fcef0 0x7f37380fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:40:24.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:24.662+0000 7f373fa00700 1 -- 172.21.15.107:0/498730390 >> 172.21.15.107:0/498730390 conn(0x7f37380f8620 msgr2=0x7f37380faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:40:24.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:24.663+0000 7f373fa00700 1 -- 172.21.15.107:0/498730390 shutdown_connections 2024-03-04T13:40:24.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:24.663+0000 7f373fa00700 1 -- 172.21.15.107:0/498730390 wait complete. 2024-03-04T13:40:24.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:24.664+0000 7f373fa00700 1 Processor -- start 2024-03-04T13:40:24.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:24.664+0000 7f373fa00700 1 -- start start 2024-03-04T13:40:24.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:24.664+0000 7f373fa00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f37380fcef0 0x7f37381066c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:40:24.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:24.664+0000 7f373fa00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37380fe0e0 0x7f3738106c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:40:24.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:24.664+0000 7f373fa00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3738107250 con 0x7f37380fcef0 2024-03-04T13:40:24.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:24.664+0000 7f373fa00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3738107af0 con 0x7f37380fe0e0 2024-03-04T13:40:24.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:24.665+0000 7f373d000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f37380fcef0 0x7f37381066c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:40:24.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:24.665+0000 7f3737e00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37380fe0e0 0x7f3738106c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:40:24.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:24.665+0000 7f3737e00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37380fe0e0 0x7f3738106c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.107:45846/0 (socket says 172.21.15.107:45846) 2024-03-04T13:40:24.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:24.665+0000 7f3737e00700 1 -- 172.21.15.107:0/1578759817 learned_addr learned my addr 172.21.15.107:0/1578759817 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:40:24.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:24.666+0000 7f373d000700 1 -- 172.21.15.107:0/1578759817 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37380fe0e0 msgr2=0x7f3738106c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:40:24.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:24.666+0000 7f373d000700 1 --2- 172.21.15.107:0/1578759817 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37380fe0e0 0x7f3738106c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:40:24.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:24.666+0000 7f373d000700 1 -- 172.21.15.107:0/1578759817 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3724005040 con 0x7f37380fcef0 2024-03-04T13:40:24.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:24.666+0000 7f373d000700 1 --2- 172.21.15.107:0/1578759817 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f37380fcef0 0x7f37381066c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f37380fcc30 tx=0x7f372c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:40:24.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:24.666+0000 7f3734c00700 1 -- 172.21.15.107:0/1578759817 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f372c004e90 con 0x7f37380fcef0 2024-03-04T13:40:24.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:24.666+0000 7f3734c00700 1 -- 172.21.15.107:0/1578759817 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f372c00f040 con 0x7f37380fcef0 2024-03-04T13:40:24.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:24.666+0000 7f3734c00700 1 -- 172.21.15.107:0/1578759817 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f372c00c980 con 0x7f37380fcef0 2024-03-04T13:40:24.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:24.666+0000 7f373fa00700 1 -- 172.21.15.107:0/1578759817 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3738107cf0 con 0x7f37380fcef0 2024-03-04T13:40:24.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:24.667+0000 7f373fa00700 1 -- 172.21.15.107:0/1578759817 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3738108200 con 0x7f37380fcef0 2024-03-04T13:40:24.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:24.668+0000 7f373fa00700 1 -- 172.21.15.107:0/1578759817 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3738102210 con 0x7f37380fcef0 2024-03-04T13:40:24.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:24.670+0000 7f3734c00700 1 -- 172.21.15.107:0/1578759817 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86623+0+0 (secure 0 0 0) 0x7f372c0090d0 con 0x7f37380fcef0 2024-03-04T13:40:24.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:24.671+0000 7f3734c00700 1 --2- 172.21.15.107:0/1578759817 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f37200692b0 0x7f372006b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:40:24.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:24.671+0000 7f3734c00700 1 -- 172.21.15.107:0/1578759817 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f372c086b90 con 0x7f37380fcef0 2024-03-04T13:40:24.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:24.672+0000 7f3737e00700 1 --2- 172.21.15.107:0/1578759817 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f37200692b0 0x7f372006b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:40:24.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:24.673+0000 7f3737e00700 1 --2- 172.21.15.107:0/1578759817 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f37200692b0 0x7f372006b760 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3724013740 tx=0x7f3724009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:40:24.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:24.675+0000 7f3734c00700 1 -- 172.21.15.107:0/1578759817 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f372c04dbd0 con 0x7f37380fcef0 2024-03-04T13:40:24.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:40:24 smithi107 ceph-mon[28739]: pgmap v273: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:40:24.838 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:40:24 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:40:24.838 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:40:24 smithi107 ceph-mon[28739]: Upgrade: Target is version 19.0.0-1845-g717ce3cc (unknown) 2024-03-04T13:40:24.838 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:40:24 smithi107 ceph-mon[28739]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, digests ['quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377'] 2024-03-04T13:40:24.838 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:40:24 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:40:24.838 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:40:24 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:40:24.838 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:40:24 smithi107 ceph-mon[28739]: Upgrade: Need to upgrade myself (mgr.smithi107.wpzgwl) 2024-03-04T13:40:24.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:24.894+0000 7f373fa00700 1 -- 172.21.15.107:0/1578759817 --> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f373802b410 con 0x7f37200692b0 2024-03-04T13:40:24.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:24.896+0000 7f3734c00700 1 -- 172.21.15.107:0/1578759817 <== mgr.14212 v2:172.21.15.107:6800/3848495603 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7f373802b410 con 0x7f37200692b0 2024-03-04T13:40:24.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:24.898+0000 7f373fa00700 1 -- 172.21.15.107:0/1578759817 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f37200692b0 msgr2=0x7f372006b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:40:24.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:24.898+0000 7f373fa00700 1 --2- 172.21.15.107:0/1578759817 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f37200692b0 0x7f372006b760 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3724013740 tx=0x7f3724009e30 comp rx=0 tx=0).stop 2024-03-04T13:40:24.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:24.898+0000 7f373fa00700 1 -- 172.21.15.107:0/1578759817 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f37380fcef0 msgr2=0x7f37381066c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:40:24.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:24.898+0000 7f373fa00700 1 --2- 172.21.15.107:0/1578759817 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f37380fcef0 0x7f37381066c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f37380fcc30 tx=0x7f372c00b040 comp rx=0 tx=0).stop 2024-03-04T13:40:24.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:24.898+0000 7f373fa00700 1 -- 172.21.15.107:0/1578759817 shutdown_connections 2024-03-04T13:40:24.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:24.898+0000 7f373fa00700 1 --2- 172.21.15.107:0/1578759817 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f37200692b0 0x7f372006b760 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:40:24.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:24.898+0000 7f373fa00700 1 --2- 172.21.15.107:0/1578759817 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f37380fcef0 0x7f37381066c0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:40:24.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:24.898+0000 7f373fa00700 1 --2- 172.21.15.107:0/1578759817 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37380fe0e0 0x7f3738106c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:40:24.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:24.898+0000 7f373fa00700 1 -- 172.21.15.107:0/1578759817 >> 172.21.15.107:0/1578759817 conn(0x7f37380f8620 msgr2=0x7f37380ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:40:24.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:24.899+0000 7f373fa00700 1 -- 172.21.15.107:0/1578759817 shutdown_connections 2024-03-04T13:40:24.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:24.899+0000 7f373fa00700 1 -- 172.21.15.107:0/1578759817 wait complete. 2024-03-04T13:40:25.048 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:25.054+0000 7f842e600700 1 -- 172.21.15.107:0/22322976 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f84280663c0 msgr2=0x7f8428101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:40:25.048 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:25.054+0000 7f842e600700 1 --2- 172.21.15.107:0/22322976 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f84280663c0 0x7f8428101d10 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f841c004940 tx=0x7f841c009e30 comp rx=0 tx=0).stop 2024-03-04T13:40:25.048 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:25.054+0000 7f842e600700 1 -- 172.21.15.107:0/22322976 shutdown_connections 2024-03-04T13:40:25.048 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:25.054+0000 7f842e600700 1 --2- 172.21.15.107:0/22322976 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8428102250 0x7f8428104630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:40:25.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:25.054+0000 7f842e600700 1 --2- 172.21.15.107:0/22322976 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f84280663c0 0x7f8428101d10 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:40:25.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:25.054+0000 7f842e600700 1 -- 172.21.15.107:0/22322976 >> 172.21.15.107:0/22322976 conn(0x7f84280f72c0 msgr2=0x7f84280f9730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:40:25.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:25.054+0000 7f842e600700 1 -- 172.21.15.107:0/22322976 shutdown_connections 2024-03-04T13:40:25.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:25.054+0000 7f842e600700 1 -- 172.21.15.107:0/22322976 wait complete. 2024-03-04T13:40:25.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:25.055+0000 7f842e600700 1 Processor -- start 2024-03-04T13:40:25.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:25.056+0000 7f842e600700 1 -- start start 2024-03-04T13:40:25.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:25.056+0000 7f842e600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8428102250 0x7f842810fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:40:25.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:25.056+0000 7f842e600700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8428110020 0x7f8428110ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:40:25.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:25.056+0000 7f842e600700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84281104e0 con 0x7f8428102250 2024-03-04T13:40:25.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:25.056+0000 7f842e600700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8428110620 con 0x7f8428110020 2024-03-04T13:40:25.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:25.056+0000 7f842d200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8428102250 0x7f842810fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:40:25.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:25.057+0000 7f842d200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8428102250 0x7f842810fae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57020/0 (socket says 172.21.15.107:57020) 2024-03-04T13:40:25.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:25.057+0000 7f842d200700 1 -- 172.21.15.107:0/2016916787 learned_addr learned my addr 172.21.15.107:0/2016916787 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:40:25.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:25.057+0000 7f8427e00700 1 --2- 172.21.15.107:0/2016916787 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8428110020 0x7f8428110ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:40:25.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:25.057+0000 7f842d200700 1 -- 172.21.15.107:0/2016916787 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8428110020 msgr2=0x7f8428110ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:40:25.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:25.057+0000 7f842d200700 1 --2- 172.21.15.107:0/2016916787 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8428110020 0x7f8428110ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:40:25.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:25.057+0000 7f842d200700 1 -- 172.21.15.107:0/2016916787 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f841c005040 con 0x7f8428102250 2024-03-04T13:40:25.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:25.058+0000 7f842d200700 1 --2- 172.21.15.107:0/2016916787 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8428102250 0x7f842810fae0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f841c00b3b0 tx=0x7f841c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:40:25.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:25.058+0000 7f8424c00700 1 -- 172.21.15.107:0/2016916787 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f841c00e040 con 0x7f8428102250 2024-03-04T13:40:25.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:25.058+0000 7f8424c00700 1 -- 172.21.15.107:0/2016916787 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f841c00bbf0 con 0x7f8428102250 2024-03-04T13:40:25.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:25.058+0000 7f842e600700 1 -- 172.21.15.107:0/2016916787 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84281114c0 con 0x7f8428102250 2024-03-04T13:40:25.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:25.058+0000 7f842e600700 1 -- 172.21.15.107:0/2016916787 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8428111a50 con 0x7f8428102250 2024-03-04T13:40:25.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:25.058+0000 7f8424c00700 1 -- 172.21.15.107:0/2016916787 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f841c01c3f0 con 0x7f8428102250 2024-03-04T13:40:25.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:25.060+0000 7f842e600700 1 -- 172.21.15.107:0/2016916787 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84080052e0 con 0x7f8428102250 2024-03-04T13:40:25.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:25.060+0000 7f8424c00700 1 -- 172.21.15.107:0/2016916787 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86623+0+0 (secure 0 0 0) 0x7f841c013420 con 0x7f8428102250 2024-03-04T13:40:25.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:25.062+0000 7f8424c00700 1 --2- 172.21.15.107:0/2016916787 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f84100692a0 0x7f841006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:40:25.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:25.062+0000 7f8424c00700 1 -- 172.21.15.107:0/2016916787 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f841c0884d0 con 0x7f8428102250 2024-03-04T13:40:25.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:25.062+0000 7f8427e00700 1 --2- 172.21.15.107:0/2016916787 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f84100692a0 0x7f841006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:40:25.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:25.063+0000 7f8427e00700 1 --2- 172.21.15.107:0/2016916787 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f84100692a0 0x7f841006b750 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8428066100 tx=0x7f841400b040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:40:25.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:25.069+0000 7f8424c00700 1 -- 172.21.15.107:0/2016916787 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f841c058220 con 0x7f8428102250 2024-03-04T13:40:25.105 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:40:24 smithi169 ceph-mon[36313]: pgmap v273: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:40:25.105 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:40:24 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:40:25.105 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:40:24 smithi169 ceph-mon[36313]: Upgrade: Target is version 19.0.0-1845-g717ce3cc (unknown) 2024-03-04T13:40:25.105 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:40:24 smithi169 ceph-mon[36313]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, digests ['quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377'] 2024-03-04T13:40:25.105 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:40:24 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:40:25.105 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:40:24 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:40:25.105 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:40:24 smithi169 ceph-mon[36313]: Upgrade: Need to upgrade myself (mgr.smithi107.wpzgwl) 2024-03-04T13:40:25.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:25.280+0000 7f842e600700 1 -- 172.21.15.107:0/2016916787 --> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f8408000bf0 con 0x7f84100692a0 2024-03-04T13:40:25.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:25.293+0000 7f8424c00700 1 -- 172.21.15.107:0/2016916787 <== mgr.14212 v2:172.21.15.107:6800/3848495603 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3410 (secure 0 0 0) 0x7f8408000bf0 con 0x7f84100692a0 2024-03-04T13:40:25.287 INFO:teuthology.orchestra.run.smithi107.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-04T13:40:25.287 INFO:teuthology.orchestra.run.smithi107.stdout:alertmanager.smithi107 smithi107 *:9093,9094 running (7m) 78s ago 8m 13.7M - 0.25.0 c8568f914cd2 3eef48872dfe 2024-03-04T13:40:25.287 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi107 smithi107 running (9m) 78s ago 9m 6665k - 17.2.7-545-g42608bb1 9eec3f22859d 03e1570c2eff 2024-03-04T13:40:25.287 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi169 smithi169 running (7m) 84s ago 7m 6656k - 17.2.7-545-g42608bb1 9eec3f22859d 9c8881a5338e 2024-03-04T13:40:25.287 INFO:teuthology.orchestra.run.smithi107.stdout:grafana.smithi107 smithi107 *:3000 running (6m) 78s ago 8m 75.2M - 9.4.7 954c08fa6188 2d8536fd3802 2024-03-04T13:40:25.288 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.riexmm smithi107 running (110s) 78s ago 109s 10.4M - 17.2.7-545-g42608bb1 9eec3f22859d fc921f7d4328 2024-03-04T13:40:25.288 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.thycho smithi107 running (102s) 78s ago 102s 11.6M - 17.2.7-545-g42608bb1 9eec3f22859d 4069c6e0e063 2024-03-04T13:40:25.288 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi169.bxitzk smithi169 running (99s) 84s ago 99s 10.5M - 17.2.7-545-g42608bb1 9eec3f22859d 7d4115228c7c 2024-03-04T13:40:25.288 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi169.jusudl smithi169 running (106s) 84s ago 106s 13.0M - 17.2.7-545-g42608bb1 9eec3f22859d 2d2f2cc452e8 2024-03-04T13:40:25.288 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi107.wpzgwl smithi107 *:9283 running (10m) 78s ago 10m 480M - 17.2.7-545-g42608bb1 9eec3f22859d 4831dce06263 2024-03-04T13:40:25.288 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi169.eihczr smithi169 *:8443,9283 running (7m) 84s ago 7m 432M - 17.2.7-545-g42608bb1 9eec3f22859d 001762248bc2 2024-03-04T13:40:25.288 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi107 smithi107 running (10m) 78s ago 10m 45.4M 2048M 17.2.7-545-g42608bb1 9eec3f22859d 49f5da990d35 2024-03-04T13:40:25.288 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi169 smithi169 running (7m) 84s ago 7m 34.7M 2048M 17.2.7-545-g42608bb1 9eec3f22859d ba4ee916297f 2024-03-04T13:40:25.288 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi107 smithi107 *:9100 running (9m) 78s ago 8m 7750k - 1.5.0 0da6a335fe13 00c53f7970cf 2024-03-04T13:40:25.288 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi169 smithi169 *:9100 running (7m) 84s ago 7m 8044k - 1.5.0 0da6a335fe13 b2dae07f2da2 2024-03-04T13:40:25.288 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0 smithi107 running (6m) 78s ago 6m 45.2M 4288M 17.2.7-545-g42608bb1 9eec3f22859d c0605b68f696 2024-03-04T13:40:25.288 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1 smithi107 running (5m) 78s ago 5m 45.0M 4288M 17.2.7-545-g42608bb1 9eec3f22859d 5fb2c0dd5769 2024-03-04T13:40:25.288 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2 smithi107 running (5m) 78s ago 5m 43.5M 4288M 17.2.7-545-g42608bb1 9eec3f22859d 61f30f643549 2024-03-04T13:40:25.288 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3 smithi169 running (4m) 84s ago 4m 43.1M 5313M 17.2.7-545-g42608bb1 9eec3f22859d cd91d8694764 2024-03-04T13:40:25.289 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4 smithi169 running (4m) 84s ago 4m 41.0M 5313M 17.2.7-545-g42608bb1 9eec3f22859d 08545e3d0659 2024-03-04T13:40:25.289 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5 smithi169 running (3m) 84s ago 3m 40.1M 5313M 17.2.7-545-g42608bb1 9eec3f22859d dcffe6cb3eac 2024-03-04T13:40:25.289 INFO:teuthology.orchestra.run.smithi107.stdout:prometheus.smithi107 smithi107 *:9095 running (6m) 78s ago 8m 46.1M - 2.43.0 a07b618ecd1d 226d4ba6d37a 2024-03-04T13:40:25.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:25.296+0000 7f842e600700 1 -- 172.21.15.107:0/2016916787 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f84100692a0 msgr2=0x7f841006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:40:25.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:25.296+0000 7f842e600700 1 --2- 172.21.15.107:0/2016916787 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f84100692a0 0x7f841006b750 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8428066100 tx=0x7f841400b040 comp rx=0 tx=0).stop 2024-03-04T13:40:25.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:25.296+0000 7f842e600700 1 -- 172.21.15.107:0/2016916787 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8428102250 msgr2=0x7f842810fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:40:25.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:25.296+0000 7f842e600700 1 --2- 172.21.15.107:0/2016916787 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8428102250 0x7f842810fae0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f841c00b3b0 tx=0x7f841c009e30 comp rx=0 tx=0).stop 2024-03-04T13:40:25.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:25.296+0000 7f842e600700 1 -- 172.21.15.107:0/2016916787 shutdown_connections 2024-03-04T13:40:25.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:25.296+0000 7f842e600700 1 --2- 172.21.15.107:0/2016916787 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f84100692a0 0x7f841006b750 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:40:25.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:25.296+0000 7f842e600700 1 --2- 172.21.15.107:0/2016916787 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8428102250 0x7f842810fae0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:40:25.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:25.296+0000 7f842e600700 1 --2- 172.21.15.107:0/2016916787 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8428110020 0x7f8428110ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:40:25.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:25.296+0000 7f842e600700 1 -- 172.21.15.107:0/2016916787 >> 172.21.15.107:0/2016916787 conn(0x7f84280f72c0 msgr2=0x7f84280f9730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:40:25.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:25.296+0000 7f842e600700 1 -- 172.21.15.107:0/2016916787 shutdown_connections 2024-03-04T13:40:25.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:25.296+0000 7f842e600700 1 -- 172.21.15.107:0/2016916787 wait complete. 2024-03-04T13:40:25.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:25.462+0000 7f7ea5000700 1 -- 172.21.15.107:0/2354051034 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ea0100180 msgr2=0x7f7ea01005d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:40:25.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:25.462+0000 7f7ea5000700 1 --2- 172.21.15.107:0/2354051034 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ea0100180 0x7f7ea01005d0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c004940 tx=0x7f7e8c009e30 comp rx=0 tx=0).stop 2024-03-04T13:40:25.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:25.463+0000 7f7ea5000700 1 -- 172.21.15.107:0/2354051034 shutdown_connections 2024-03-04T13:40:25.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:25.463+0000 7f7ea5000700 1 --2- 172.21.15.107:0/2354051034 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ea0100180 0x7f7ea01005d0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:40:25.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:25.463+0000 7f7ea5000700 1 --2- 172.21.15.107:0/2354051034 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ea00fef70 0x7f7ea00ff380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:40:25.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:25.463+0000 7f7ea5000700 1 -- 172.21.15.107:0/2354051034 >> 172.21.15.107:0/2354051034 conn(0x7f7ea00fa760 msgr2=0x7f7ea00fcb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:40:25.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:25.463+0000 7f7ea5000700 1 -- 172.21.15.107:0/2354051034 shutdown_connections 2024-03-04T13:40:25.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:25.463+0000 7f7ea5000700 1 -- 172.21.15.107:0/2354051034 wait complete. 2024-03-04T13:40:25.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:25.464+0000 7f7ea5000700 1 Processor -- start 2024-03-04T13:40:25.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:25.464+0000 7f7ea5000700 1 -- start start 2024-03-04T13:40:25.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:25.465+0000 7f7ea5000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ea00fef70 0x7f7ea010fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:40:25.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:25.465+0000 7f7ea5000700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ea0110000 0x7f7ea0110ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:40:25.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:25.465+0000 7f7ea5000700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea01104c0 con 0x7f7ea00fef70 2024-03-04T13:40:25.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:25.465+0000 7f7ea5000700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea0110600 con 0x7f7ea0110000 2024-03-04T13:40:25.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:25.465+0000 7f7e9f400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ea00fef70 0x7f7ea010fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:40:25.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:25.465+0000 7f7e9ea00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ea0110000 0x7f7ea0110ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:40:25.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:25.465+0000 7f7e9ea00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ea0110000 0x7f7ea0110ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.107:45884/0 (socket says 172.21.15.107:45884) 2024-03-04T13:40:25.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:25.465+0000 7f7e9ea00700 1 -- 172.21.15.107:0/3168588899 learned_addr learned my addr 172.21.15.107:0/3168588899 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:40:25.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:25.466+0000 7f7e9ea00700 1 -- 172.21.15.107:0/3168588899 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ea00fef70 msgr2=0x7f7ea010fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:40:25.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:25.466+0000 7f7e9ea00700 1 --2- 172.21.15.107:0/3168588899 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ea00fef70 0x7f7ea010fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:40:25.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:25.466+0000 7f7e9ea00700 1 -- 172.21.15.107:0/3168588899 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e8c005040 con 0x7f7ea0110000 2024-03-04T13:40:25.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:25.466+0000 7f7e9f400700 1 --2- 172.21.15.107:0/3168588899 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ea00fef70 0x7f7ea010fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T13:40:25.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:25.467+0000 7f7e9ea00700 1 --2- 172.21.15.107:0/3168588899 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ea0110000 0x7f7ea0110ea0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c00b3b0 tx=0x7f7e8c009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:40:25.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:25.467+0000 7f7e97400700 1 -- 172.21.15.107:0/3168588899 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7e8c00e040 con 0x7f7ea0110000 2024-03-04T13:40:25.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:25.467+0000 7f7e97400700 1 -- 172.21.15.107:0/3168588899 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7e8c009af0 con 0x7f7ea0110000 2024-03-04T13:40:25.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:25.467+0000 7f7e97400700 1 -- 172.21.15.107:0/3168588899 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7e8c01c3f0 con 0x7f7ea0110000 2024-03-04T13:40:25.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:25.467+0000 7f7ea5000700 1 -- 172.21.15.107:0/3168588899 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ea01114a0 con 0x7f7ea0110000 2024-03-04T13:40:25.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:25.468+0000 7f7ea5000700 1 -- 172.21.15.107:0/3168588899 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ea0111a30 con 0x7f7ea0110000 2024-03-04T13:40:25.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:25.469+0000 7f7ea5000700 1 -- 172.21.15.107:0/3168588899 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ea004bed0 con 0x7f7ea0110000 2024-03-04T13:40:25.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:25.471+0000 7f7e97400700 1 -- 172.21.15.107:0/3168588899 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 19) v1 ==== 86623+0+0 (secure 0 0 0) 0x7f7e8c00bbf0 con 0x7f7ea0110000 2024-03-04T13:40:25.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:25.472+0000 7f7e97400700 1 --2- 172.21.15.107:0/3168588899 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f7e840692a0 0x7f7e8406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:40:25.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:25.473+0000 7f7e97400700 1 -- 172.21.15.107:0/3168588899 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f7e8c0886a0 con 0x7f7ea0110000 2024-03-04T13:40:25.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:25.473+0000 7f7e9f400700 1 --2- 172.21.15.107:0/3168588899 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f7e840692a0 0x7f7e8406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:40:25.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:25.474+0000 7f7e9f400700 1 --2- 172.21.15.107:0/3168588899 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f7e840692a0 0x7f7e8406b750 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7ea00fffe0 tx=0x7f7e9000b040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:40:25.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:25.480+0000 7f7e97400700 1 -- 172.21.15.107:0/3168588899 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f7e8c0583f0 con 0x7f7ea0110000 2024-03-04T13:40:25.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:25.887+0000 7f7ea5000700 1 -- 172.21.15.107:0/3168588899 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f7ea0107a50 con 0x7f7ea0110000 2024-03-04T13:40:25.882 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:40:25 smithi107 ceph-mon[28739]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377 on smithi169 2024-03-04T13:40:25.882 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:40:25 smithi107 ceph-mon[28739]: from='client.14552 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:40:25.882 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:40:25 smithi107 ceph-mon[28739]: from='client.14556 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:40:25.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:25.888+0000 7f7e97400700 1 -- 172.21.15.107:0/3168588899 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+638 (secure 0 0 0) 0x7f7e8c057f80 con 0x7f7ea0110000 2024-03-04T13:40:25.882 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-03-04T13:40:25.882 INFO:teuthology.orchestra.run.smithi107.stdout: "mon": { 2024-03-04T13:40:25.883 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-545-g42608bb1 (42608bb191117be10dba720e6bf8a02f3171cd34) quincy (stable)": 2 2024-03-04T13:40:25.883 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:40:25.883 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr": { 2024-03-04T13:40:25.883 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-545-g42608bb1 (42608bb191117be10dba720e6bf8a02f3171cd34) quincy (stable)": 2 2024-03-04T13:40:25.883 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:40:25.883 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": { 2024-03-04T13:40:25.883 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-545-g42608bb1 (42608bb191117be10dba720e6bf8a02f3171cd34) quincy (stable)": 6 2024-03-04T13:40:25.883 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:40:25.883 INFO:teuthology.orchestra.run.smithi107.stdout: "mds": { 2024-03-04T13:40:25.883 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-545-g42608bb1 (42608bb191117be10dba720e6bf8a02f3171cd34) quincy (stable)": 4 2024-03-04T13:40:25.883 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:40:25.883 INFO:teuthology.orchestra.run.smithi107.stdout: "overall": { 2024-03-04T13:40:25.883 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-545-g42608bb1 (42608bb191117be10dba720e6bf8a02f3171cd34) quincy (stable)": 14 2024-03-04T13:40:25.883 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-04T13:40:25.884 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-03-04T13:40:25.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:25.890+0000 7f7ea5000700 1 -- 172.21.15.107:0/3168588899 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f7e840692a0 msgr2=0x7f7e8406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:40:25.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:25.890+0000 7f7ea5000700 1 --2- 172.21.15.107:0/3168588899 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f7e840692a0 0x7f7e8406b750 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7ea00fffe0 tx=0x7f7e9000b040 comp rx=0 tx=0).stop 2024-03-04T13:40:25.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:25.890+0000 7f7ea5000700 1 -- 172.21.15.107:0/3168588899 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ea0110000 msgr2=0x7f7ea0110ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:40:25.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:25.890+0000 7f7ea5000700 1 --2- 172.21.15.107:0/3168588899 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ea0110000 0x7f7ea0110ea0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c00b3b0 tx=0x7f7e8c009e30 comp rx=0 tx=0).stop 2024-03-04T13:40:25.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:25.891+0000 7f7ea5000700 1 -- 172.21.15.107:0/3168588899 shutdown_connections 2024-03-04T13:40:25.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:25.891+0000 7f7ea5000700 1 --2- 172.21.15.107:0/3168588899 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f7e840692a0 0x7f7e8406b750 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:40:25.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:25.891+0000 7f7ea5000700 1 --2- 172.21.15.107:0/3168588899 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ea00fef70 0x7f7ea010fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:40:25.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:25.891+0000 7f7ea5000700 1 --2- 172.21.15.107:0/3168588899 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ea0110000 0x7f7ea0110ea0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:40:25.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:25.891+0000 7f7ea5000700 1 -- 172.21.15.107:0/3168588899 >> 172.21.15.107:0/3168588899 conn(0x7f7ea00fa760 msgr2=0x7f7ea01033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:40:25.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:25.891+0000 7f7ea5000700 1 -- 172.21.15.107:0/3168588899 shutdown_connections 2024-03-04T13:40:25.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:25.891+0000 7f7ea5000700 1 -- 172.21.15.107:0/3168588899 wait complete. 2024-03-04T13:40:25.889 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:40:25 smithi169 ceph-mon[36313]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377 on smithi169 2024-03-04T13:40:25.889 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:40:25 smithi169 ceph-mon[36313]: from='client.14552 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:40:25.889 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:40:25 smithi169 ceph-mon[36313]: from='client.14556 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:40:26.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:26.043+0000 7fbe89c00700 1 -- 172.21.15.107:0/2692731999 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe84071f10 msgr2=0x7fbe84070380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:40:26.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:26.043+0000 7fbe89c00700 1 --2- 172.21.15.107:0/2692731999 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe84071f10 0x7fbe84070380 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fbe74004940 tx=0x7fbe74009e30 comp rx=0 tx=0).stop 2024-03-04T13:40:26.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:26.044+0000 7fbe89c00700 1 -- 172.21.15.107:0/2692731999 shutdown_connections 2024-03-04T13:40:26.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:26.044+0000 7fbe89c00700 1 --2- 172.21.15.107:0/2692731999 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe840708c0 0x7fbe84070d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:40:26.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:26.044+0000 7fbe89c00700 1 --2- 172.21.15.107:0/2692731999 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe84071f10 0x7fbe84070380 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:40:26.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:26.044+0000 7fbe89c00700 1 -- 172.21.15.107:0/2692731999 >> 172.21.15.107:0/2692731999 conn(0x7fbe840f8620 msgr2=0x7fbe840faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:40:26.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:26.044+0000 7fbe89c00700 1 -- 172.21.15.107:0/2692731999 shutdown_connections 2024-03-04T13:40:26.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:26.044+0000 7fbe89c00700 1 -- 172.21.15.107:0/2692731999 wait complete. 2024-03-04T13:40:26.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:26.045+0000 7fbe89c00700 1 Processor -- start 2024-03-04T13:40:26.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:26.045+0000 7fbe89c00700 1 -- start start 2024-03-04T13:40:26.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:26.046+0000 7fbe89c00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe840708c0 0x7fbe8410fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:40:26.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:26.046+0000 7fbe89c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe84071f10 0x7fbe841100b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:40:26.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:26.046+0000 7fbe89c00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe841106b0 con 0x7fbe84071f10 2024-03-04T13:40:26.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:26.046+0000 7fbe89c00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe84110f50 con 0x7fbe840708c0 2024-03-04T13:40:26.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:26.046+0000 7fbe82000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe84071f10 0x7fbe841100b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:40:26.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:26.046+0000 7fbe82a00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe840708c0 0x7fbe8410fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:40:26.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:26.046+0000 7fbe82a00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe840708c0 0x7fbe8410fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.107:45890/0 (socket says 172.21.15.107:45890) 2024-03-04T13:40:26.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:26.046+0000 7fbe82a00700 1 -- 172.21.15.107:0/3736270852 learned_addr learned my addr 172.21.15.107:0/3736270852 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:40:26.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:26.047+0000 7fbe82000700 1 -- 172.21.15.107:0/3736270852 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe840708c0 msgr2=0x7fbe8410fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:40:26.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:26.047+0000 7fbe82000700 1 --2- 172.21.15.107:0/3736270852 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe840708c0 0x7fbe8410fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:40:26.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:26.047+0000 7fbe82000700 1 -- 172.21.15.107:0/3736270852 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe74005040 con 0x7fbe84071f10 2024-03-04T13:40:26.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:26.047+0000 7fbe82000700 1 --2- 172.21.15.107:0/3736270852 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe84071f10 0x7fbe841100b0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fbe70009fd0 tx=0x7fbe7000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:40:26.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:26.048+0000 7fbe7aa00700 1 -- 172.21.15.107:0/3736270852 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe70004e70 con 0x7fbe84071f10 2024-03-04T13:40:26.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:26.048+0000 7fbe7aa00700 1 -- 172.21.15.107:0/3736270852 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbe7000f040 con 0x7fbe84071f10 2024-03-04T13:40:26.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:26.048+0000 7fbe7aa00700 1 -- 172.21.15.107:0/3736270852 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe7000c980 con 0x7fbe84071f10 2024-03-04T13:40:26.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:26.048+0000 7fbe89c00700 1 -- 172.21.15.107:0/3736270852 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe841111d0 con 0x7fbe84071f10 2024-03-04T13:40:26.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:26.048+0000 7fbe89c00700 1 -- 172.21.15.107:0/3736270852 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe84111710 con 0x7fbe84071f10 2024-03-04T13:40:26.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:26.049+0000 7fbe89c00700 1 -- 172.21.15.107:0/3736270852 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe8404bed0 con 0x7fbe84071f10 2024-03-04T13:40:26.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:26.051+0000 7fbe7aa00700 1 -- 172.21.15.107:0/3736270852 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86623+0+0 (secure 0 0 0) 0x7fbe70007cc0 con 0x7fbe84071f10 2024-03-04T13:40:26.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:26.053+0000 7fbe7aa00700 1 --2- 172.21.15.107:0/3736270852 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fbe680692b0 0x7fbe6806b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:40:26.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:26.053+0000 7fbe7aa00700 1 -- 172.21.15.107:0/3736270852 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fbe70086d70 con 0x7fbe84071f10 2024-03-04T13:40:26.048 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:26.053+0000 7fbe82a00700 1 --2- 172.21.15.107:0/3736270852 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fbe680692b0 0x7fbe6806b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:40:26.048 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:26.054+0000 7fbe82a00700 1 --2- 172.21.15.107:0/3736270852 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fbe680692b0 0x7fbe6806b760 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fbe7400b0d0 tx=0x7fbe74009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:40:26.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:26.060+0000 7fbe7aa00700 1 -- 172.21.15.107:0/3736270852 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fbe70056a40 con 0x7fbe84071f10 2024-03-04T13:40:26.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:26.313+0000 7fbe89c00700 1 -- 172.21.15.107:0/3736270852 --> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbe84111be0 con 0x7fbe680692b0 2024-03-04T13:40:26.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:26.316+0000 7fbe7aa00700 1 -- 172.21.15.107:0/3736270852 <== mgr.14212 v2:172.21.15.107:6800/3848495603 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7fbe84111be0 con 0x7fbe680692b0 2024-03-04T13:40:26.310 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-03-04T13:40:26.310 INFO:teuthology.orchestra.run.smithi107.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377", 2024-03-04T13:40:26.310 INFO:teuthology.orchestra.run.smithi107.stdout: "in_progress": true, 2024-03-04T13:40:26.310 INFO:teuthology.orchestra.run.smithi107.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-03-04T13:40:26.311 INFO:teuthology.orchestra.run.smithi107.stdout: "services_complete": [], 2024-03-04T13:40:26.311 INFO:teuthology.orchestra.run.smithi107.stdout: "progress": "0/2 daemons upgraded", 2024-03-04T13:40:26.311 INFO:teuthology.orchestra.run.smithi107.stdout: "message": "Pulling quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377 image on host smithi169", 2024-03-04T13:40:26.311 INFO:teuthology.orchestra.run.smithi107.stdout: "is_paused": false 2024-03-04T13:40:26.311 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-03-04T13:40:26.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:26.319+0000 7fbe89c00700 1 -- 172.21.15.107:0/3736270852 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fbe680692b0 msgr2=0x7fbe6806b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:40:26.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:26.319+0000 7fbe89c00700 1 --2- 172.21.15.107:0/3736270852 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fbe680692b0 0x7fbe6806b760 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fbe7400b0d0 tx=0x7fbe74009e30 comp rx=0 tx=0).stop 2024-03-04T13:40:26.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:26.319+0000 7fbe89c00700 1 -- 172.21.15.107:0/3736270852 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe84071f10 msgr2=0x7fbe841100b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:40:26.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:26.319+0000 7fbe89c00700 1 --2- 172.21.15.107:0/3736270852 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe84071f10 0x7fbe841100b0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fbe70009fd0 tx=0x7fbe7000b040 comp rx=0 tx=0).stop 2024-03-04T13:40:26.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:26.319+0000 7fbe89c00700 1 -- 172.21.15.107:0/3736270852 shutdown_connections 2024-03-04T13:40:26.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:26.319+0000 7fbe89c00700 1 --2- 172.21.15.107:0/3736270852 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fbe680692b0 0x7fbe6806b760 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:40:26.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:26.319+0000 7fbe89c00700 1 --2- 172.21.15.107:0/3736270852 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe840708c0 0x7fbe8410fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:40:26.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:26.319+0000 7fbe89c00700 1 --2- 172.21.15.107:0/3736270852 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe84071f10 0x7fbe841100b0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:40:26.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:26.319+0000 7fbe89c00700 1 -- 172.21.15.107:0/3736270852 >> 172.21.15.107:0/3736270852 conn(0x7fbe840f8620 msgr2=0x7fbe84102dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:40:26.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:26.320+0000 7fbe89c00700 1 -- 172.21.15.107:0/3736270852 shutdown_connections 2024-03-04T13:40:26.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:26.320+0000 7fbe89c00700 1 -- 172.21.15.107:0/3736270852 wait complete. 2024-03-04T13:40:27.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:40:26 smithi107 ceph-mon[28739]: from='client.14560 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:40:27.088 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:40:26 smithi107 ceph-mon[28739]: pgmap v274: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:40:27.088 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:40:26 smithi107 ceph-mon[28739]: from='client.? 172.21.15.107:0/3168588899' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:40:27.105 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:40:26 smithi169 ceph-mon[36313]: from='client.14560 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:40:27.105 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:40:26 smithi169 ceph-mon[36313]: pgmap v274: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:40:27.105 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:40:26 smithi169 ceph-mon[36313]: from='client.? 172.21.15.107:0/3168588899' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:40:28.087 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:40:27 smithi107 ceph-mon[28739]: from='client.14568 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:40:28.088 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:40:27 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:40:28.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:40:27 smithi169 ceph-mon[36313]: from='client.14568 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:40:28.104 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:40:27 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:40:29.088 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:40:28 smithi107 ceph-mon[28739]: pgmap v275: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:40:29.105 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:40:28 smithi169 ceph-mon[36313]: pgmap v275: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:40:31.088 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:40:30 smithi107 ceph-mon[28739]: pgmap v276: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:40:31.105 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:40:30 smithi169 ceph-mon[36313]: pgmap v276: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:40:32.838 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:40:32 smithi107 ceph-mon[28739]: pgmap v277: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:40:33.105 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:40:32 smithi169 ceph-mon[36313]: pgmap v277: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:40:35.088 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:40:34 smithi107 ceph-mon[28739]: pgmap v278: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:40:35.105 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:40:34 smithi169 ceph-mon[36313]: pgmap v278: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:40:37.088 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:40:36 smithi107 ceph-mon[28739]: pgmap v279: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:40:37.105 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:40:36 smithi169 ceph-mon[36313]: pgmap v279: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:40:39.088 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:40:38 smithi107 ceph-mon[28739]: pgmap v280: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:40:39.105 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:40:38 smithi169 ceph-mon[36313]: pgmap v280: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:40:40.907 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:40:40 smithi169 ceph-mon[36313]: pgmap v281: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:40:41.088 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:40:40 smithi107 ceph-mon[28739]: pgmap v281: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:40:43.088 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:40:42 smithi107 ceph-mon[28739]: pgmap v282: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:40:43.088 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:40:42 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:40:43.105 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:40:42 smithi169 ceph-mon[36313]: pgmap v282: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:40:43.105 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:40:42 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:40:45.088 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:40:44 smithi107 ceph-mon[28739]: pgmap v283: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:40:45.105 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:40:44 smithi169 ceph-mon[36313]: pgmap v283: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:40:47.088 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:40:46 smithi107 ceph-mon[28739]: pgmap v284: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:40:47.105 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:40:46 smithi169 ceph-mon[36313]: pgmap v284: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:40:47.985 INFO:tasks.workunit.client.0.smithi107.stderr:Updating files: 63% (7822/12271) Updating files: 64% (7854/12271) Updating files: 65% (7977/12271) Updating files: 66% (8099/12271) Updating files: 67% (8222/12271) Updating files: 68% (8345/12271) Updating files: 69% (8467/12271) Updating files: 70% (8590/12271) Updating files: 71% (8713/12271) Updating files: 72% (8836/12271) Updating files: 73% (8958/12271) Updating files: 74% (9081/12271) Updating files: 75% (9204/12271) Updating files: 76% (9326/12271) Updating files: 77% (9449/12271) Updating files: 78% (9572/12271) Updating files: 79% (9695/12271) Updating files: 80% (9817/12271) Updating files: 81% (9940/12271) Updating files: 82% (10063/12271) Updating files: 83% (10185/12271) Updating files: 84% (10308/12271) Updating files: 85% (10431/12271) Updating files: 86% (10554/12271) Updating files: 87% (10676/12271) Updating files: 88% (10799/12271) Updating files: 89% (10922/12271) Updating files: 90% (11044/12271) Updating files: 91% (11167/12271) Updating files: 92% (11290/12271) Updating files: 93% (11413/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-03-04T13:40:48.206 INFO:tasks.workunit.client.0.smithi107.stderr:Note: switching to '717ce3cce8d0166e4e577faa8004e6f9cb4128c0'. 2024-03-04T13:40:48.206 INFO:tasks.workunit.client.0.smithi107.stderr: 2024-03-04T13:40:48.206 INFO:tasks.workunit.client.0.smithi107.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-03-04T13:40:48.206 INFO:tasks.workunit.client.0.smithi107.stderr:changes and commit them, and you can discard any commits you make in this 2024-03-04T13:40:48.206 INFO:tasks.workunit.client.0.smithi107.stderr:state without impacting any branches by switching back to a branch. 2024-03-04T13:40:48.206 INFO:tasks.workunit.client.0.smithi107.stderr: 2024-03-04T13:40:48.206 INFO:tasks.workunit.client.0.smithi107.stderr:If you want to create a new branch to retain commits you create, you may 2024-03-04T13:40:48.206 INFO:tasks.workunit.client.0.smithi107.stderr:do so (now or later) by using -c with the switch command. Example: 2024-03-04T13:40:48.206 INFO:tasks.workunit.client.0.smithi107.stderr: 2024-03-04T13:40:48.206 INFO:tasks.workunit.client.0.smithi107.stderr: git switch -c 2024-03-04T13:40:48.206 INFO:tasks.workunit.client.0.smithi107.stderr: 2024-03-04T13:40:48.206 INFO:tasks.workunit.client.0.smithi107.stderr:Or undo this operation with: 2024-03-04T13:40:48.207 INFO:tasks.workunit.client.0.smithi107.stderr: 2024-03-04T13:40:48.207 INFO:tasks.workunit.client.0.smithi107.stderr: git switch - 2024-03-04T13:40:48.207 INFO:tasks.workunit.client.0.smithi107.stderr: 2024-03-04T13:40:48.207 INFO:tasks.workunit.client.0.smithi107.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-03-04T13:40:48.207 INFO:tasks.workunit.client.0.smithi107.stderr: 2024-03-04T13:40:48.207 INFO:tasks.workunit.client.0.smithi107.stderr:HEAD is now at 717ce3cce8d Merge branch 'wip-64616' into wip-vshankar-testing-20240304.042522 2024-03-04T13:40:48.214 DEBUG:teuthology.orchestra.run.smithi107:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-03-04T13:40:48.275 INFO:tasks.workunit.client.0.smithi107.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-03-04T13:40:48.284 INFO:tasks.workunit.client.0.smithi107.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-04T13:40:48.284 INFO:tasks.workunit.client.0.smithi107.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-03-04T13:40:49.088 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:40:48 smithi107 ceph-mon[28739]: pgmap v285: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:40:49.105 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:40:48 smithi169 ceph-mon[36313]: pgmap v285: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:40:49.289 INFO:tasks.workunit.client.0.smithi107.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-03-04T13:40:49.464 INFO:tasks.workunit.client.0.smithi107.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-03-04T13:40:49.548 INFO:tasks.workunit.client.0.smithi107.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-04T13:40:49.553 INFO:tasks.workunit.client.0.smithi107.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-04T13:40:49.553 INFO:tasks.workunit.client.0.smithi107.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-03-04T13:40:49.638 INFO:tasks.workunit.client.0.smithi107.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-04T13:40:49.649 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-03-04T13:40:49.649 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-03-04T13:40:49.700 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.0... 2024-03-04T13:40:49.705 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-03-04T13:40:49.705 DEBUG:teuthology.orchestra.run.smithi107:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=717ce3cce8d0166e4e577faa8004e6f9cb4128c0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh 2024-03-04T13:40:49.786 INFO:tasks.workunit.client.0.smithi107.stderr:+ mkdir -p fsstress 2024-03-04T13:40:49.795 INFO:tasks.workunit.client.0.smithi107.stderr:+ pushd fsstress 2024-03-04T13:40:49.798 INFO:tasks.workunit.client.0.smithi107.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-03-04T13:40:49.798 INFO:tasks.workunit.client.0.smithi107.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-03-04T13:40:50.284 INFO:tasks.workunit.client.1.smithi169.stderr:Updating files: 56% (6924/12271) Updating files: 57% (6995/12271) Updating files: 58% (7118/12271) Updating files: 59% (7240/12271) Updating files: 60% (7363/12271) Updating files: 61% (7486/12271) Updating files: 62% (7609/12271) Updating files: 63% (7731/12271) Updating files: 64% (7854/12271) Updating files: 65% (7977/12271) Updating files: 66% (8099/12271) Updating files: 67% (8222/12271) Updating files: 68% (8345/12271) Updating files: 69% (8467/12271) Updating files: 70% (8590/12271) Updating files: 71% (8713/12271) Updating files: 72% (8836/12271) Updating files: 73% (8958/12271) Updating files: 74% (9081/12271) Updating files: 75% (9204/12271) Updating files: 76% (9326/12271) Updating files: 77% (9449/12271) Updating files: 78% (9572/12271) Updating files: 79% (9695/12271) Updating files: 80% (9817/12271) Updating files: 81% (9940/12271) Updating files: 82% (10063/12271) Updating files: 83% (10185/12271) Updating files: 84% (10308/12271) Updating files: 85% (10431/12271) Updating files: 86% (10554/12271) Updating files: 86% (10601/12271) Updating files: 87% (10676/12271) Updating files: 88% (10799/12271) Updating files: 89% (10922/12271) Updating files: 90% (11044/12271) Updating files: 91% (11167/12271) Updating files: 92% (11290/12271) Updating files: 93% (11413/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-03-04T13:40:50.625 INFO:tasks.workunit.client.1.smithi169.stderr:Note: switching to '717ce3cce8d0166e4e577faa8004e6f9cb4128c0'. 2024-03-04T13:40:50.626 INFO:tasks.workunit.client.1.smithi169.stderr: 2024-03-04T13:40:50.626 INFO:tasks.workunit.client.1.smithi169.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-03-04T13:40:50.626 INFO:tasks.workunit.client.1.smithi169.stderr:changes and commit them, and you can discard any commits you make in this 2024-03-04T13:40:50.626 INFO:tasks.workunit.client.1.smithi169.stderr:state without impacting any branches by switching back to a branch. 2024-03-04T13:40:50.626 INFO:tasks.workunit.client.1.smithi169.stderr: 2024-03-04T13:40:50.626 INFO:tasks.workunit.client.1.smithi169.stderr:If you want to create a new branch to retain commits you create, you may 2024-03-04T13:40:50.626 INFO:tasks.workunit.client.1.smithi169.stderr:do so (now or later) by using -c with the switch command. Example: 2024-03-04T13:40:50.626 INFO:tasks.workunit.client.1.smithi169.stderr: 2024-03-04T13:40:50.626 INFO:tasks.workunit.client.1.smithi169.stderr: git switch -c 2024-03-04T13:40:50.626 INFO:tasks.workunit.client.1.smithi169.stderr: 2024-03-04T13:40:50.626 INFO:tasks.workunit.client.1.smithi169.stderr:Or undo this operation with: 2024-03-04T13:40:50.626 INFO:tasks.workunit.client.1.smithi169.stderr: 2024-03-04T13:40:50.626 INFO:tasks.workunit.client.1.smithi169.stderr: git switch - 2024-03-04T13:40:50.627 INFO:tasks.workunit.client.1.smithi169.stderr: 2024-03-04T13:40:50.627 INFO:tasks.workunit.client.1.smithi169.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-03-04T13:40:50.627 INFO:tasks.workunit.client.1.smithi169.stderr: 2024-03-04T13:40:50.627 INFO:tasks.workunit.client.1.smithi169.stderr:HEAD is now at 717ce3cce8d Merge branch 'wip-64616' into wip-vshankar-testing-20240304.042522 2024-03-04T13:40:50.632 DEBUG:teuthology.orchestra.run.smithi169:> cd -- /home/ubuntu/cephtest/clone.client.1/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.1 2024-03-04T13:40:50.768 INFO:tasks.workunit.client.1.smithi169.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-03-04T13:40:50.776 INFO:tasks.workunit.client.1.smithi169.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-03-04T13:40:50.776 INFO:tasks.workunit.client.1.smithi169.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-03-04T13:40:51.088 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:40:50 smithi107 ceph-mon[28739]: pgmap v286: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:40:51.105 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:40:50 smithi169 ceph-mon[36313]: pgmap v286: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:40:51.621 INFO:tasks.workunit.client.0.smithi107.stderr:+ tar xzf ltp-full.tgz 2024-03-04T13:40:51.677 INFO:tasks.workunit.client.1.smithi169.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-03-04T13:40:51.808 INFO:tasks.workunit.client.1.smithi169.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-03-04T13:40:51.904 INFO:tasks.workunit.client.1.smithi169.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-03-04T13:40:51.909 INFO:tasks.workunit.client.1.smithi169.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-03-04T13:40:51.909 INFO:tasks.workunit.client.1.smithi169.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-03-04T13:40:51.990 INFO:tasks.workunit.client.1.smithi169.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-03-04T13:40:52.000 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2024-03-04T13:40:52.000 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/home/ubuntu/cephtest/workunits.list.client.1 of=/dev/stdout 2024-03-04T13:40:52.045 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.1... 2024-03-04T13:40:52.046 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-03-04T13:40:52.046 DEBUG:teuthology.orchestra.run.smithi169:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=717ce3cce8d0166e4e577faa8004e6f9cb4128c0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/fsstress.sh 2024-03-04T13:40:52.161 INFO:tasks.workunit.client.1.smithi169.stderr:+ mkdir -p fsstress 2024-03-04T13:40:52.167 INFO:tasks.workunit.client.1.smithi169.stderr:+ pushd fsstress 2024-03-04T13:40:52.172 INFO:tasks.workunit.client.1.smithi169.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-03-04T13:40:52.173 INFO:tasks.workunit.client.1.smithi169.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-03-04T13:40:52.826 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:40:52 smithi107 ceph-mon[28739]: pgmap v287: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:40:53.105 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:40:52 smithi169 ceph-mon[36313]: pgmap v287: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:40:55.088 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:40:54 smithi107 ceph-mon[28739]: pgmap v288: 65 pgs: 65 active+clean; 4.6 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 343 KiB/s wr, 2 op/s 2024-03-04T13:40:55.106 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:40:54 smithi169 ceph-mon[36313]: pgmap v288: 65 pgs: 65 active+clean; 4.6 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 343 KiB/s wr, 2 op/s 2024-03-04T13:40:55.681 INFO:tasks.workunit.client.1.smithi169.stderr:+ tar xzf ltp-full.tgz 2024-03-04T13:40:56.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:56.496+0000 7f477e000700 1 -- 172.21.15.107:0/3918075513 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47780fda80 msgr2=0x7f47780ffe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:40:56.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:56.496+0000 7f477e000700 1 --2- 172.21.15.107:0/3918075513 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47780fda80 0x7f47780ffe60 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f4768004940 tx=0x7f4768009e30 comp rx=0 tx=0).stop 2024-03-04T13:40:56.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:56.496+0000 7f477e000700 1 -- 172.21.15.107:0/3918075513 shutdown_connections 2024-03-04T13:40:56.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:56.496+0000 7f477e000700 1 --2- 172.21.15.107:0/3918075513 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f47781003a0 0x7f4778102780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:40:56.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:56.496+0000 7f477e000700 1 --2- 172.21.15.107:0/3918075513 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47780fda80 0x7f47780ffe60 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:40:56.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:56.496+0000 7f477e000700 1 -- 172.21.15.107:0/3918075513 >> 172.21.15.107:0/3918075513 conn(0x7f47780f7520 msgr2=0x7f47780f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:40:56.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:56.496+0000 7f477e000700 1 -- 172.21.15.107:0/3918075513 shutdown_connections 2024-03-04T13:40:56.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:56.496+0000 7f477e000700 1 -- 172.21.15.107:0/3918075513 wait complete. 2024-03-04T13:40:56.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:56.497+0000 7f477e000700 1 Processor -- start 2024-03-04T13:40:56.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:56.498+0000 7f477e000700 1 -- start start 2024-03-04T13:40:56.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:56.498+0000 7f477e000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47780fda80 0x7f477810fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:40:56.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:56.498+0000 7f477e000700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f47781003a0 0x7f4778110050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:40:56.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:56.498+0000 7f477e000700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4778110650 con 0x7f47780fda80 2024-03-04T13:40:56.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:56.498+0000 7f477e000700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4778110ef0 con 0x7f47781003a0 2024-03-04T13:40:56.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:56.498+0000 7f4777400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47780fda80 0x7f477810fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:40:56.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:56.498+0000 7f4777400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47780fda80 0x7f477810fb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51896/0 (socket says 172.21.15.107:51896) 2024-03-04T13:40:56.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:56.498+0000 7f4777400700 1 -- 172.21.15.107:0/2829462688 learned_addr learned my addr 172.21.15.107:0/2829462688 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:40:56.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:56.499+0000 7f4777400700 1 -- 172.21.15.107:0/2829462688 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f47781003a0 msgr2=0x7f4778110050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:40:56.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:56.499+0000 7f4776a00700 1 --2- 172.21.15.107:0/2829462688 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f47781003a0 0x7f4778110050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:40:56.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:56.499+0000 7f4777400700 1 --2- 172.21.15.107:0/2829462688 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f47781003a0 0x7f4778110050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:40:56.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:56.499+0000 7f4777400700 1 -- 172.21.15.107:0/2829462688 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4768005040 con 0x7f47780fda80 2024-03-04T13:40:56.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:56.499+0000 7f4777400700 1 --2- 172.21.15.107:0/2829462688 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47780fda80 0x7f477810fb10 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f47680093b0 tx=0x7f4768009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:40:56.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:56.500+0000 7f476f400700 1 -- 172.21.15.107:0/2829462688 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f476800e040 con 0x7f47780fda80 2024-03-04T13:40:56.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:56.500+0000 7f476f400700 1 -- 172.21.15.107:0/2829462688 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f476800bcc0 con 0x7f47780fda80 2024-03-04T13:40:56.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:56.500+0000 7f476f400700 1 -- 172.21.15.107:0/2829462688 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f476801c630 con 0x7f47780fda80 2024-03-04T13:40:56.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:56.500+0000 7f477e000700 1 -- 172.21.15.107:0/2829462688 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4778111170 con 0x7f47780fda80 2024-03-04T13:40:56.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:56.500+0000 7f477e000700 1 -- 172.21.15.107:0/2829462688 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47781116b0 con 0x7f47780fda80 2024-03-04T13:40:56.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:56.501+0000 7f477e000700 1 -- 172.21.15.107:0/2829462688 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f477804bed0 con 0x7f47780fda80 2024-03-04T13:40:56.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:56.502+0000 7f476f400700 1 -- 172.21.15.107:0/2829462688 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86623+0+0 (secure 0 0 0) 0x7f47680139f0 con 0x7f47780fda80 2024-03-04T13:40:56.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:56.503+0000 7f476f400700 1 --2- 172.21.15.107:0/2829462688 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f475c0692a0 0x7f475c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:40:56.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:56.503+0000 7f476f400700 1 -- 172.21.15.107:0/2829462688 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f4768088660 con 0x7f47780fda80 2024-03-04T13:40:56.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:56.503+0000 7f4776a00700 1 --2- 172.21.15.107:0/2829462688 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f475c0692a0 0x7f475c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:40:56.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:56.504+0000 7f4776a00700 1 --2- 172.21.15.107:0/2829462688 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f475c0692a0 0x7f475c06b750 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f47640041d0 tx=0x7f4764009040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:40:56.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:56.508+0000 7f476f400700 1 -- 172.21.15.107:0/2829462688 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f47680583b0 con 0x7f47780fda80 2024-03-04T13:40:56.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:56.765+0000 7f477e000700 1 -- 172.21.15.107:0/2829462688 --> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4778111300 con 0x7f475c0692a0 2024-03-04T13:40:56.762 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:40:56 smithi107 ceph-mon[28739]: Upgrade: Updating mgr.smithi169.eihczr 2024-03-04T13:40:56.762 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:40:56 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:40:56.762 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:40:56 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi169.eihczr", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-04T13:40:56.762 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:40:56 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-04T13:40:56.762 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:40:56 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:40:56.762 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:40:56 smithi107 ceph-mon[28739]: Deploying daemon mgr.smithi169.eihczr on smithi169 2024-03-04T13:40:56.763 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:40:56 smithi107 ceph-mon[28739]: pgmap v289: 65 pgs: 65 active+clean; 12 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 53 KiB/s rd, 1001 KiB/s wr, 10 op/s 2024-03-04T13:40:56.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:56.768+0000 7f476f400700 1 -- 172.21.15.107:0/2829462688 <== mgr.14212 v2:172.21.15.107:6800/3848495603 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7f4778111300 con 0x7f475c0692a0 2024-03-04T13:40:56.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:56.771+0000 7f477e000700 1 -- 172.21.15.107:0/2829462688 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f475c0692a0 msgr2=0x7f475c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:40:56.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:56.771+0000 7f477e000700 1 --2- 172.21.15.107:0/2829462688 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f475c0692a0 0x7f475c06b750 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f47640041d0 tx=0x7f4764009040 comp rx=0 tx=0).stop 2024-03-04T13:40:56.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:56.771+0000 7f477e000700 1 -- 172.21.15.107:0/2829462688 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47780fda80 msgr2=0x7f477810fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:40:56.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:56.771+0000 7f477e000700 1 --2- 172.21.15.107:0/2829462688 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47780fda80 0x7f477810fb10 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f47680093b0 tx=0x7f4768009e30 comp rx=0 tx=0).stop 2024-03-04T13:40:56.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:56.771+0000 7f477e000700 1 -- 172.21.15.107:0/2829462688 shutdown_connections 2024-03-04T13:40:56.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:56.771+0000 7f477e000700 1 --2- 172.21.15.107:0/2829462688 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f475c0692a0 0x7f475c06b750 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:40:56.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:56.771+0000 7f477e000700 1 --2- 172.21.15.107:0/2829462688 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47780fda80 0x7f477810fb10 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:40:56.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:56.771+0000 7f477e000700 1 --2- 172.21.15.107:0/2829462688 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f47781003a0 0x7f4778110050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:40:56.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:56.771+0000 7f477e000700 1 -- 172.21.15.107:0/2829462688 >> 172.21.15.107:0/2829462688 conn(0x7f47780f7520 msgr2=0x7f47780fb870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:40:56.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:56.771+0000 7f477e000700 1 -- 172.21.15.107:0/2829462688 shutdown_connections 2024-03-04T13:40:56.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:56.771+0000 7f477e000700 1 -- 172.21.15.107:0/2829462688 wait complete. 2024-03-04T13:40:56.784 INFO:teuthology.orchestra.run.smithi107.stdout:true 2024-03-04T13:40:56.856 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:40:56 smithi169 ceph-mon[36313]: Upgrade: Updating mgr.smithi169.eihczr 2024-03-04T13:40:56.856 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:40:56 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:40:56.856 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:40:56 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi169.eihczr", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-04T13:40:56.856 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:40:56 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-04T13:40:56.856 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:40:56 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:40:56.856 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:40:56 smithi169 ceph-mon[36313]: Deploying daemon mgr.smithi169.eihczr on smithi169 2024-03-04T13:40:56.856 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:40:56 smithi169 ceph-mon[36313]: pgmap v289: 65 pgs: 65 active+clean; 12 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 53 KiB/s rd, 1001 KiB/s wr, 10 op/s 2024-03-04T13:40:56.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:56.936+0000 7f7412e00700 1 -- 172.21.15.107:0/3415482195 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f740c0fb650 msgr2=0x7f740c0fbf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:40:56.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:56.936+0000 7f7412e00700 1 --2- 172.21.15.107:0/3415482195 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f740c0fb650 0x7f740c0fbf40 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f73f4004940 tx=0x7f73f4009e30 comp rx=0 tx=0).stop 2024-03-04T13:40:56.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:56.936+0000 7f7412e00700 1 -- 172.21.15.107:0/3415482195 shutdown_connections 2024-03-04T13:40:56.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:56.936+0000 7f7412e00700 1 --2- 172.21.15.107:0/3415482195 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f740c0fb650 0x7f740c0fbf40 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:40:56.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:56.936+0000 7f7412e00700 1 --2- 172.21.15.107:0/3415482195 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f740c0fad00 0x7f740c0fb110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:40:56.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:56.936+0000 7f7412e00700 1 -- 172.21.15.107:0/3415482195 >> 172.21.15.107:0/3415482195 conn(0x7f740c0f6840 msgr2=0x7f740c0f8c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:40:56.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:56.936+0000 7f7412e00700 1 -- 172.21.15.107:0/3415482195 shutdown_connections 2024-03-04T13:40:56.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:56.936+0000 7f7412e00700 1 -- 172.21.15.107:0/3415482195 wait complete. 2024-03-04T13:40:56.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:56.937+0000 7f7412e00700 1 Processor -- start 2024-03-04T13:40:56.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:56.938+0000 7f7412e00700 1 -- start start 2024-03-04T13:40:56.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:56.938+0000 7f7412e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f740c0fad00 0x7f740c10fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:40:56.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:56.938+0000 7f7412e00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f740c0fb650 0x7f740c110040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:40:56.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:56.938+0000 7f7412e00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f740c110640 con 0x7f740c0fad00 2024-03-04T13:40:56.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:56.938+0000 7f7412e00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f740c110ee0 con 0x7f740c0fb650 2024-03-04T13:40:56.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:56.938+0000 7f740be00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f740c0fad00 0x7f740c10fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:40:56.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:56.938+0000 7f740be00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f740c0fad00 0x7f740c10fb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51914/0 (socket says 172.21.15.107:51914) 2024-03-04T13:40:56.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:56.938+0000 7f740be00700 1 -- 172.21.15.107:0/3921089960 learned_addr learned my addr 172.21.15.107:0/3921089960 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:40:56.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:56.939+0000 7f7403400700 1 --2- 172.21.15.107:0/3921089960 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f740c0fb650 0x7f740c110040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:40:56.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:56.939+0000 7f740be00700 1 -- 172.21.15.107:0/3921089960 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f740c0fb650 msgr2=0x7f740c110040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:40:56.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:56.939+0000 7f740be00700 1 --2- 172.21.15.107:0/3921089960 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f740c0fb650 0x7f740c110040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:40:56.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:56.939+0000 7f740be00700 1 -- 172.21.15.107:0/3921089960 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73f4005040 con 0x7f740c0fad00 2024-03-04T13:40:56.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:56.939+0000 7f740be00700 1 --2- 172.21.15.107:0/3921089960 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f740c0fad00 0x7f740c10fb00 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f73fc00c830 tx=0x7f73fc004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:40:56.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:56.939+0000 7f7408c00700 1 -- 172.21.15.107:0/3921089960 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f73fc00a760 con 0x7f740c0fad00 2024-03-04T13:40:56.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:56.939+0000 7f7408c00700 1 -- 172.21.15.107:0/3921089960 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f73fc00ad90 con 0x7f740c0fad00 2024-03-04T13:40:56.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:56.939+0000 7f7412e00700 1 -- 172.21.15.107:0/3921089960 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f740c1111c0 con 0x7f740c0fad00 2024-03-04T13:40:56.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:56.939+0000 7f7408c00700 1 -- 172.21.15.107:0/3921089960 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f73fc0113a0 con 0x7f740c0fad00 2024-03-04T13:40:56.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:56.939+0000 7f7412e00700 1 -- 172.21.15.107:0/3921089960 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f740c111700 con 0x7f740c0fad00 2024-03-04T13:40:56.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:56.941+0000 7f7412e00700 1 -- 172.21.15.107:0/3921089960 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f740c04bed0 con 0x7f740c0fad00 2024-03-04T13:40:56.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:56.942+0000 7f7408c00700 1 -- 172.21.15.107:0/3921089960 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86623+0+0 (secure 0 0 0) 0x7f73fc012400 con 0x7f740c0fad00 2024-03-04T13:40:56.938 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:56.943+0000 7f7408c00700 1 --2- 172.21.15.107:0/3921089960 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f73f00692b0 0x7f73f006b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:40:56.938 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:56.943+0000 7f7408c00700 1 -- 172.21.15.107:0/3921089960 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f73fc087320 con 0x7f740c0fad00 2024-03-04T13:40:56.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:56.943+0000 7f7403400700 1 --2- 172.21.15.107:0/3921089960 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f73f00692b0 0x7f73f006b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:40:56.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:56.943+0000 7f7403400700 1 --2- 172.21.15.107:0/3921089960 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f73f00692b0 0x7f73f006b760 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f73f400b840 tx=0x7f73f4009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:40:56.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:56.947+0000 7f7408c00700 1 -- 172.21.15.107:0/3921089960 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f73fc056ff0 con 0x7f740c0fad00 2024-03-04T13:40:57.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:57.197+0000 7f7412e00700 1 -- 172.21.15.107:0/3921089960 --> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f740c1062c0 con 0x7f73f00692b0 2024-03-04T13:40:57.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:57.199+0000 7f7408c00700 1 -- 172.21.15.107:0/3921089960 <== mgr.14212 v2:172.21.15.107:6800/3848495603 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7f740c1062c0 con 0x7f73f00692b0 2024-03-04T13:40:57.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:57.202+0000 7f7412e00700 1 -- 172.21.15.107:0/3921089960 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f73f00692b0 msgr2=0x7f73f006b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:40:57.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:57.202+0000 7f7412e00700 1 --2- 172.21.15.107:0/3921089960 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f73f00692b0 0x7f73f006b760 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f73f400b840 tx=0x7f73f4009e30 comp rx=0 tx=0).stop 2024-03-04T13:40:57.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:57.203+0000 7f7412e00700 1 -- 172.21.15.107:0/3921089960 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f740c0fad00 msgr2=0x7f740c10fb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:40:57.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:57.203+0000 7f7412e00700 1 --2- 172.21.15.107:0/3921089960 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f740c0fad00 0x7f740c10fb00 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f73fc00c830 tx=0x7f73fc004470 comp rx=0 tx=0).stop 2024-03-04T13:40:57.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:57.203+0000 7f7412e00700 1 -- 172.21.15.107:0/3921089960 shutdown_connections 2024-03-04T13:40:57.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:57.203+0000 7f7412e00700 1 --2- 172.21.15.107:0/3921089960 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f73f00692b0 0x7f73f006b760 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:40:57.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:57.203+0000 7f7412e00700 1 --2- 172.21.15.107:0/3921089960 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f740c0fad00 0x7f740c10fb00 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:40:57.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:57.203+0000 7f7412e00700 1 --2- 172.21.15.107:0/3921089960 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f740c0fb650 0x7f740c110040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:40:57.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:57.203+0000 7f7412e00700 1 -- 172.21.15.107:0/3921089960 >> 172.21.15.107:0/3921089960 conn(0x7f740c0f6840 msgr2=0x7f740c104ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:40:57.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:57.203+0000 7f7412e00700 1 -- 172.21.15.107:0/3921089960 shutdown_connections 2024-03-04T13:40:57.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:57.205+0000 7f7412e00700 1 -- 172.21.15.107:0/3921089960 wait complete. 2024-03-04T13:40:57.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:57.353+0000 7fe405c00700 1 -- 172.21.15.107:0/3709284988 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4001003a0 msgr2=0x7fe400102780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:40:57.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:57.353+0000 7fe405c00700 1 --2- 172.21.15.107:0/3709284988 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4001003a0 0x7fe400102780 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec004940 tx=0x7fe3ec009e30 comp rx=0 tx=0).stop 2024-03-04T13:40:57.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:57.354+0000 7fe405c00700 1 -- 172.21.15.107:0/3709284988 shutdown_connections 2024-03-04T13:40:57.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:57.354+0000 7fe405c00700 1 --2- 172.21.15.107:0/3709284988 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4001003a0 0x7fe400102780 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:40:57.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:57.354+0000 7fe405c00700 1 --2- 172.21.15.107:0/3709284988 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4000fda80 0x7fe4000ffe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:40:57.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:57.354+0000 7fe405c00700 1 -- 172.21.15.107:0/3709284988 >> 172.21.15.107:0/3709284988 conn(0x7fe4000f74e0 msgr2=0x7fe4000f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:40:57.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:57.354+0000 7fe405c00700 1 -- 172.21.15.107:0/3709284988 shutdown_connections 2024-03-04T13:40:57.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:57.354+0000 7fe405c00700 1 -- 172.21.15.107:0/3709284988 wait complete. 2024-03-04T13:40:57.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:57.355+0000 7fe405c00700 1 Processor -- start 2024-03-04T13:40:57.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:57.355+0000 7fe405c00700 1 -- start start 2024-03-04T13:40:57.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:57.355+0000 7fe405c00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4000fda80 0x7fe40010fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:40:57.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:57.355+0000 7fe405c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe400110290 0x7fe400111050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:40:57.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:57.356+0000 7fe405c00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe400110750 con 0x7fe400110290 2024-03-04T13:40:57.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:57.356+0000 7fe405c00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4001108c0 con 0x7fe4000fda80 2024-03-04T13:40:57.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:57.356+0000 7fe3fe000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe400110290 0x7fe400111050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:40:57.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:57.356+0000 7fe3fe000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe400110290 0x7fe400111050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51936/0 (socket says 172.21.15.107:51936) 2024-03-04T13:40:57.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:57.356+0000 7fe3fe000700 1 -- 172.21.15.107:0/342054236 learned_addr learned my addr 172.21.15.107:0/342054236 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:40:57.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:57.356+0000 7fe3fea00700 1 --2- 172.21.15.107:0/342054236 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4000fda80 0x7fe40010fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:40:57.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:57.356+0000 7fe3fe000700 1 -- 172.21.15.107:0/342054236 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4000fda80 msgr2=0x7fe40010fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:40:57.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:57.356+0000 7fe3fe000700 1 --2- 172.21.15.107:0/342054236 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4000fda80 0x7fe40010fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:40:57.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:57.356+0000 7fe3fe000700 1 -- 172.21.15.107:0/342054236 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3ec005040 con 0x7fe400110290 2024-03-04T13:40:57.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:57.357+0000 7fe3fe000700 1 --2- 172.21.15.107:0/342054236 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe400110290 0x7fe400111050 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec00b3b0 tx=0x7fe3ec009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:40:57.355 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:57.357+0000 7fe3f6a00700 1 -- 172.21.15.107:0/342054236 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe3ec00e040 con 0x7fe400110290 2024-03-04T13:40:57.356 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:57.357+0000 7fe3f6a00700 1 -- 172.21.15.107:0/342054236 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe3ec004290 con 0x7fe400110290 2024-03-04T13:40:57.356 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:57.357+0000 7fe3f6a00700 1 -- 172.21.15.107:0/342054236 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe3ec01c630 con 0x7fe400110290 2024-03-04T13:40:57.356 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:57.358+0000 7fe405c00700 1 -- 172.21.15.107:0/342054236 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4001116d0 con 0x7fe400110290 2024-03-04T13:40:57.356 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:57.358+0000 7fe405c00700 1 -- 172.21.15.107:0/342054236 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe400111c10 con 0x7fe400110290 2024-03-04T13:40:57.356 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:57.359+0000 7fe405c00700 1 -- 172.21.15.107:0/342054236 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe40004bed0 con 0x7fe400110290 2024-03-04T13:40:57.356 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:57.360+0000 7fe3f6a00700 1 -- 172.21.15.107:0/342054236 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86623+0+0 (secure 0 0 0) 0x7fe3ec00bbf0 con 0x7fe400110290 2024-03-04T13:40:57.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:57.361+0000 7fe3f6a00700 1 --2- 172.21.15.107:0/342054236 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fe3e40692b0 0x7fe3e406b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:40:57.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:57.361+0000 7fe3f6a00700 1 -- 172.21.15.107:0/342054236 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fe3ec088300 con 0x7fe400110290 2024-03-04T13:40:57.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:57.362+0000 7fe3fea00700 1 --2- 172.21.15.107:0/342054236 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fe3e40692b0 0x7fe3e406b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:40:57.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:57.362+0000 7fe3fea00700 1 --2- 172.21.15.107:0/342054236 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fe3e40692b0 0x7fe3e406b760 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe3f0009fd0 tx=0x7fe3f0009f20 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:40:57.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:57.366+0000 7fe3f6a00700 1 -- 172.21.15.107:0/342054236 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7fe3ec0545c0 con 0x7fe400110290 2024-03-04T13:40:57.588 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:40:57 smithi107 ceph-mon[28739]: from='client.14572 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:40:57.589 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:40:57 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:40:57.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:57.599+0000 7fe405c00700 1 -- 172.21.15.107:0/342054236 --> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fe4000fbf80 con 0x7fe3e40692b0 2024-03-04T13:40:57.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:57.637+0000 7fe3f6a00700 1 -- 172.21.15.107:0/342054236 <== mgr.14212 v2:172.21.15.107:6800/3848495603 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3366 (secure 0 0 0) 0x7fe4000fbf80 con 0x7fe3e40692b0 2024-03-04T13:40:57.634 INFO:teuthology.orchestra.run.smithi107.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-04T13:40:57.634 INFO:teuthology.orchestra.run.smithi107.stdout:alertmanager.smithi107 smithi107 *:9093,9094 running (7m) 110s ago 9m 13.7M - 0.25.0 c8568f914cd2 3eef48872dfe 2024-03-04T13:40:57.634 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi107 smithi107 running (9m) 110s ago 9m 6665k - 17.2.7-545-g42608bb1 9eec3f22859d 03e1570c2eff 2024-03-04T13:40:57.634 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi169 smithi169 running (8m) 117s ago 8m 6656k - 17.2.7-545-g42608bb1 9eec3f22859d 9c8881a5338e 2024-03-04T13:40:57.634 INFO:teuthology.orchestra.run.smithi107.stdout:grafana.smithi107 smithi107 *:3000 running (7m) 110s ago 8m 75.2M - 9.4.7 954c08fa6188 2d8536fd3802 2024-03-04T13:40:57.634 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.riexmm smithi107 running (2m) 110s ago 2m 10.4M - 17.2.7-545-g42608bb1 9eec3f22859d fc921f7d4328 2024-03-04T13:40:57.634 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.thycho smithi107 running (2m) 110s ago 2m 11.6M - 17.2.7-545-g42608bb1 9eec3f22859d 4069c6e0e063 2024-03-04T13:40:57.634 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi169.bxitzk smithi169 running (2m) 117s ago 2m 10.5M - 17.2.7-545-g42608bb1 9eec3f22859d 7d4115228c7c 2024-03-04T13:40:57.634 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi169.jusudl smithi169 running (2m) 117s ago 2m 13.0M - 17.2.7-545-g42608bb1 9eec3f22859d 2d2f2cc452e8 2024-03-04T13:40:57.634 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi107.wpzgwl smithi107 *:9283 running (11m) 110s ago 11m 480M - 17.2.7-545-g42608bb1 9eec3f22859d 4831dce06263 2024-03-04T13:40:57.634 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi169.eihczr smithi169 *:8443,9283 running (8m) 117s ago 8m 432M - 17.2.7-545-g42608bb1 9eec3f22859d 001762248bc2 2024-03-04T13:40:57.634 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi107 smithi107 running (11m) 110s ago 11m 45.4M 2048M 17.2.7-545-g42608bb1 9eec3f22859d 49f5da990d35 2024-03-04T13:40:57.635 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi169 smithi169 running (7m) 117s ago 7m 34.7M 2048M 17.2.7-545-g42608bb1 9eec3f22859d ba4ee916297f 2024-03-04T13:40:57.635 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi107 smithi107 *:9100 running (9m) 110s ago 9m 7750k - 1.5.0 0da6a335fe13 00c53f7970cf 2024-03-04T13:40:57.635 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi169 smithi169 *:9100 running (8m) 117s ago 8m 8044k - 1.5.0 0da6a335fe13 b2dae07f2da2 2024-03-04T13:40:57.635 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0 smithi107 running (7m) 110s ago 7m 45.2M 4288M 17.2.7-545-g42608bb1 9eec3f22859d c0605b68f696 2024-03-04T13:40:57.635 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1 smithi107 running (6m) 110s ago 6m 45.0M 4288M 17.2.7-545-g42608bb1 9eec3f22859d 5fb2c0dd5769 2024-03-04T13:40:57.635 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2 smithi107 running (5m) 110s ago 5m 43.5M 4288M 17.2.7-545-g42608bb1 9eec3f22859d 61f30f643549 2024-03-04T13:40:57.635 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3 smithi169 running (5m) 117s ago 5m 43.1M 5313M 17.2.7-545-g42608bb1 9eec3f22859d cd91d8694764 2024-03-04T13:40:57.635 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4 smithi169 running (4m) 117s ago 4m 41.0M 5313M 17.2.7-545-g42608bb1 9eec3f22859d 08545e3d0659 2024-03-04T13:40:57.635 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5 smithi169 running (4m) 117s ago 4m 40.1M 5313M 17.2.7-545-g42608bb1 9eec3f22859d dcffe6cb3eac 2024-03-04T13:40:57.635 INFO:teuthology.orchestra.run.smithi107.stdout:prometheus.smithi107 smithi107 *:9095 running (7m) 110s ago 8m 46.1M - 2.43.0 a07b618ecd1d 226d4ba6d37a 2024-03-04T13:40:57.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:57.640+0000 7fe405c00700 1 -- 172.21.15.107:0/342054236 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fe3e40692b0 msgr2=0x7fe3e406b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:40:57.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:57.640+0000 7fe405c00700 1 --2- 172.21.15.107:0/342054236 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fe3e40692b0 0x7fe3e406b760 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe3f0009fd0 tx=0x7fe3f0009f20 comp rx=0 tx=0).stop 2024-03-04T13:40:57.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:57.640+0000 7fe405c00700 1 -- 172.21.15.107:0/342054236 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe400110290 msgr2=0x7fe400111050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:40:57.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:57.640+0000 7fe405c00700 1 --2- 172.21.15.107:0/342054236 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe400110290 0x7fe400111050 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec00b3b0 tx=0x7fe3ec009e30 comp rx=0 tx=0).stop 2024-03-04T13:40:57.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:57.641+0000 7fe405c00700 1 -- 172.21.15.107:0/342054236 shutdown_connections 2024-03-04T13:40:57.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:57.641+0000 7fe405c00700 1 --2- 172.21.15.107:0/342054236 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7fe3e40692b0 0x7fe3e406b760 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:40:57.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:57.641+0000 7fe405c00700 1 --2- 172.21.15.107:0/342054236 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4000fda80 0x7fe40010fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:40:57.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:57.641+0000 7fe405c00700 1 --2- 172.21.15.107:0/342054236 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe400110290 0x7fe400111050 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:40:57.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:57.641+0000 7fe405c00700 1 -- 172.21.15.107:0/342054236 >> 172.21.15.107:0/342054236 conn(0x7fe4000f74e0 msgr2=0x7fe4000fb870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:40:57.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:57.641+0000 7fe405c00700 1 -- 172.21.15.107:0/342054236 shutdown_connections 2024-03-04T13:40:57.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:57.641+0000 7fe405c00700 1 -- 172.21.15.107:0/342054236 wait complete. 2024-03-04T13:40:57.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:57.805+0000 7f79c1200700 1 -- 172.21.15.107:0/1069267930 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79bc0fbe40 msgr2=0x7f79bc0fc250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:40:57.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:57.805+0000 7f79c1200700 1 --2- 172.21.15.107:0/1069267930 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79bc0fbe40 0x7f79bc0fc250 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f79ac004940 tx=0x7f79ac009e30 comp rx=0 tx=0).stop 2024-03-04T13:40:57.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:57.805+0000 7f79c1200700 1 -- 172.21.15.107:0/1069267930 shutdown_connections 2024-03-04T13:40:57.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:57.805+0000 7f79c1200700 1 --2- 172.21.15.107:0/1069267930 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79bc0fcfb0 0x7f79bc0fd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:40:57.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:57.805+0000 7f79c1200700 1 --2- 172.21.15.107:0/1069267930 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79bc0fbe40 0x7f79bc0fc250 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:40:57.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:57.805+0000 7f79c1200700 1 -- 172.21.15.107:0/1069267930 >> 172.21.15.107:0/1069267930 conn(0x7f79bc0f7520 msgr2=0x7f79bc0f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:40:57.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:57.805+0000 7f79c1200700 1 -- 172.21.15.107:0/1069267930 shutdown_connections 2024-03-04T13:40:57.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:57.805+0000 7f79c1200700 1 -- 172.21.15.107:0/1069267930 wait complete. 2024-03-04T13:40:57.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:57.806+0000 7f79c1200700 1 Processor -- start 2024-03-04T13:40:57.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:57.806+0000 7f79c1200700 1 -- start start 2024-03-04T13:40:57.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:57.806+0000 7f79c1200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79bc0fbe40 0x7f79bc0fff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:40:57.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:57.806+0000 7f79c1200700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79bc0fcfb0 0x7f79bc0fe380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:40:57.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:57.806+0000 7f79c1200700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79bc0fea10 con 0x7f79bc0fbe40 2024-03-04T13:40:57.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:57.806+0000 7f79c1200700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79bc0feb80 con 0x7f79bc0fcfb0 2024-03-04T13:40:57.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:57.807+0000 7f79b9e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79bc0fbe40 0x7f79bc0fff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:40:57.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:57.807+0000 7f79b9e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79bc0fbe40 0x7f79bc0fff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51946/0 (socket says 172.21.15.107:51946) 2024-03-04T13:40:57.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:57.807+0000 7f79b9e00700 1 -- 172.21.15.107:0/2109773038 learned_addr learned my addr 172.21.15.107:0/2109773038 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:40:57.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:57.807+0000 7f79b3e00700 1 --2- 172.21.15.107:0/2109773038 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79bc0fcfb0 0x7f79bc0fe380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:40:57.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:57.807+0000 7f79b9e00700 1 -- 172.21.15.107:0/2109773038 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79bc0fcfb0 msgr2=0x7f79bc0fe380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:40:57.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:57.807+0000 7f79b9e00700 1 --2- 172.21.15.107:0/2109773038 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79bc0fcfb0 0x7f79bc0fe380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:40:57.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:57.807+0000 7f79b9e00700 1 -- 172.21.15.107:0/2109773038 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79ac005040 con 0x7f79bc0fbe40 2024-03-04T13:40:57.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:57.807+0000 7f79b3e00700 1 --2- 172.21.15.107:0/2109773038 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79bc0fcfb0 0x7f79bc0fe380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T13:40:57.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:57.808+0000 7f79b9e00700 1 --2- 172.21.15.107:0/2109773038 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79bc0fbe40 0x7f79bc0fff00 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f79ac0093b0 tx=0x7f79ac009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:40:57.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:57.808+0000 7f79b2000700 1 -- 172.21.15.107:0/2109773038 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f79ac00e040 con 0x7f79bc0fbe40 2024-03-04T13:40:57.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:57.808+0000 7f79b2000700 1 -- 172.21.15.107:0/2109773038 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f79ac00bcc0 con 0x7f79bc0fbe40 2024-03-04T13:40:57.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:57.808+0000 7f79b2000700 1 -- 172.21.15.107:0/2109773038 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f79ac01c630 con 0x7f79bc0fbe40 2024-03-04T13:40:57.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:57.808+0000 7f79c1200700 1 -- 172.21.15.107:0/2109773038 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79bc0fee00 con 0x7f79bc0fbe40 2024-03-04T13:40:57.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:57.808+0000 7f79c1200700 1 -- 172.21.15.107:0/2109773038 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79bc115960 con 0x7f79bc0fbe40 2024-03-04T13:40:57.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:57.809+0000 7f79c1200700 1 -- 172.21.15.107:0/2109773038 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79bc04bed0 con 0x7f79bc0fbe40 2024-03-04T13:40:57.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:57.810+0000 7f79b2000700 1 -- 172.21.15.107:0/2109773038 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86623+0+0 (secure 0 0 0) 0x7f79ac0139f0 con 0x7f79bc0fbe40 2024-03-04T13:40:57.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:57.811+0000 7f79b2000700 1 --2- 172.21.15.107:0/2109773038 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f79a00692a0 0x7f79a006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:40:57.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:57.811+0000 7f79b2000700 1 -- 172.21.15.107:0/2109773038 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f79ac0885a0 con 0x7f79bc0fbe40 2024-03-04T13:40:57.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:57.811+0000 7f79b3e00700 1 --2- 172.21.15.107:0/2109773038 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f79a00692a0 0x7f79a006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:40:57.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:57.812+0000 7f79b3e00700 1 --2- 172.21.15.107:0/2109773038 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f79a00692a0 0x7f79a006b750 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f79a4006fd0 tx=0x7f79a4008040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:40:57.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:57.815+0000 7f79b2000700 1 -- 172.21.15.107:0/2109773038 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7f79ac0583b0 con 0x7f79bc0fbe40 2024-03-04T13:40:57.855 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:40:57 smithi169 ceph-mon[36313]: from='client.14572 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:40:57.855 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:40:57 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:40:58.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:58.153+0000 7f79c1200700 1 -- 172.21.15.107:0/2109773038 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f79bc115df0 con 0x7f79bc0fbe40 2024-03-04T13:40:58.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:58.154+0000 7f79b2000700 1 -- 172.21.15.107:0/2109773038 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+638 (secure 0 0 0) 0x7f79ac057f40 con 0x7f79bc0fbe40 2024-03-04T13:40:58.152 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-03-04T13:40:58.152 INFO:teuthology.orchestra.run.smithi107.stdout: "mon": { 2024-03-04T13:40:58.152 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-545-g42608bb1 (42608bb191117be10dba720e6bf8a02f3171cd34) quincy (stable)": 2 2024-03-04T13:40:58.152 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:40:58.152 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr": { 2024-03-04T13:40:58.152 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-545-g42608bb1 (42608bb191117be10dba720e6bf8a02f3171cd34) quincy (stable)": 2 2024-03-04T13:40:58.152 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:40:58.153 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": { 2024-03-04T13:40:58.153 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-545-g42608bb1 (42608bb191117be10dba720e6bf8a02f3171cd34) quincy (stable)": 6 2024-03-04T13:40:58.153 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:40:58.153 INFO:teuthology.orchestra.run.smithi107.stdout: "mds": { 2024-03-04T13:40:58.153 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-545-g42608bb1 (42608bb191117be10dba720e6bf8a02f3171cd34) quincy (stable)": 4 2024-03-04T13:40:58.153 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:40:58.153 INFO:teuthology.orchestra.run.smithi107.stdout: "overall": { 2024-03-04T13:40:58.153 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-545-g42608bb1 (42608bb191117be10dba720e6bf8a02f3171cd34) quincy (stable)": 14 2024-03-04T13:40:58.153 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-04T13:40:58.153 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-03-04T13:40:58.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:58.157+0000 7f79c1200700 1 -- 172.21.15.107:0/2109773038 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f79a00692a0 msgr2=0x7f79a006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:40:58.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:58.157+0000 7f79c1200700 1 --2- 172.21.15.107:0/2109773038 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f79a00692a0 0x7f79a006b750 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f79a4006fd0 tx=0x7f79a4008040 comp rx=0 tx=0).stop 2024-03-04T13:40:58.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:58.157+0000 7f79c1200700 1 -- 172.21.15.107:0/2109773038 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79bc0fbe40 msgr2=0x7f79bc0fff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:40:58.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:58.157+0000 7f79c1200700 1 --2- 172.21.15.107:0/2109773038 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79bc0fbe40 0x7f79bc0fff00 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f79ac0093b0 tx=0x7f79ac009e30 comp rx=0 tx=0).stop 2024-03-04T13:40:58.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:58.158+0000 7f79c1200700 1 -- 172.21.15.107:0/2109773038 shutdown_connections 2024-03-04T13:40:58.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:58.158+0000 7f79c1200700 1 --2- 172.21.15.107:0/2109773038 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7f79a00692a0 0x7f79a006b750 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:40:58.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:58.158+0000 7f79c1200700 1 --2- 172.21.15.107:0/2109773038 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79bc0fbe40 0x7f79bc0fff00 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:40:58.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:58.158+0000 7f79c1200700 1 --2- 172.21.15.107:0/2109773038 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79bc0fcfb0 0x7f79bc0fe380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:40:58.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:58.158+0000 7f79c1200700 1 -- 172.21.15.107:0/2109773038 >> 172.21.15.107:0/2109773038 conn(0x7f79bc0f7520 msgr2=0x7f79bc065f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:40:58.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:58.158+0000 7f79c1200700 1 -- 172.21.15.107:0/2109773038 shutdown_connections 2024-03-04T13:40:58.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:58.158+0000 7f79c1200700 1 -- 172.21.15.107:0/2109773038 wait complete. 2024-03-04T13:40:58.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:58.319+0000 7ff548000700 1 -- 172.21.15.107:0/2512551431 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5401002f0 msgr2=0x7ff540100740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:40:58.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:58.319+0000 7ff548000700 1 --2- 172.21.15.107:0/2512551431 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5401002f0 0x7ff540100740 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff52c004940 tx=0x7ff52c009e30 comp rx=0 tx=0).stop 2024-03-04T13:40:58.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:58.319+0000 7ff548000700 1 -- 172.21.15.107:0/2512551431 shutdown_connections 2024-03-04T13:40:58.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:58.319+0000 7ff548000700 1 --2- 172.21.15.107:0/2512551431 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5401002f0 0x7ff540100740 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:40:58.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:58.319+0000 7ff548000700 1 --2- 172.21.15.107:0/2512551431 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff5400ff0e0 0x7ff5400ff4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:40:58.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:58.319+0000 7ff548000700 1 -- 172.21.15.107:0/2512551431 >> 172.21.15.107:0/2512551431 conn(0x7ff5400fa7c0 msgr2=0x7ff5400fcbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:40:58.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:58.319+0000 7ff548000700 1 -- 172.21.15.107:0/2512551431 shutdown_connections 2024-03-04T13:40:58.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:58.319+0000 7ff548000700 1 -- 172.21.15.107:0/2512551431 wait complete. 2024-03-04T13:40:58.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:58.320+0000 7ff548000700 1 Processor -- start 2024-03-04T13:40:58.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:58.320+0000 7ff548000700 1 -- start start 2024-03-04T13:40:58.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:58.321+0000 7ff548000700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff5400ff0e0 0x7ff54010fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:40:58.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:58.321+0000 7ff548000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5401002f0 0x7ff540110000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:40:58.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:58.321+0000 7ff548000700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff540110690 con 0x7ff5401002f0 2024-03-04T13:40:58.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:58.321+0000 7ff548000700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff540110ed0 con 0x7ff5400ff0e0 2024-03-04T13:40:58.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:58.321+0000 7ff544c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5401002f0 0x7ff540110000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:40:58.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:58.321+0000 7ff544c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5401002f0 0x7ff540110000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51796/0 (socket says 172.21.15.107:51796) 2024-03-04T13:40:58.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:58.321+0000 7ff544c00700 1 -- 172.21.15.107:0/3141041962 learned_addr learned my addr 172.21.15.107:0/3141041962 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:40:58.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:58.321+0000 7ff545600700 1 --2- 172.21.15.107:0/3141041962 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff5400ff0e0 0x7ff54010fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:40:58.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:58.321+0000 7ff544c00700 1 -- 172.21.15.107:0/3141041962 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff5400ff0e0 msgr2=0x7ff54010fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:40:58.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:58.321+0000 7ff544c00700 1 --2- 172.21.15.107:0/3141041962 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff5400ff0e0 0x7ff54010fac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:40:58.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:58.322+0000 7ff544c00700 1 -- 172.21.15.107:0/3141041962 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff52c005040 con 0x7ff5401002f0 2024-03-04T13:40:58.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:58.322+0000 7ff544c00700 1 --2- 172.21.15.107:0/3141041962 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5401002f0 0x7ff540110000 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ff52c0093b0 tx=0x7ff52c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:40:58.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:58.322+0000 7ff53d600700 1 -- 172.21.15.107:0/3141041962 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff52c00e040 con 0x7ff5401002f0 2024-03-04T13:40:58.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:58.322+0000 7ff53d600700 1 -- 172.21.15.107:0/3141041962 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff52c00bcc0 con 0x7ff5401002f0 2024-03-04T13:40:58.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:58.322+0000 7ff53d600700 1 -- 172.21.15.107:0/3141041962 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff52c01c630 con 0x7ff5401002f0 2024-03-04T13:40:58.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:58.322+0000 7ff548000700 1 -- 172.21.15.107:0/3141041962 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff540111150 con 0x7ff5401002f0 2024-03-04T13:40:58.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:58.322+0000 7ff548000700 1 -- 172.21.15.107:0/3141041962 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff540111710 con 0x7ff5401002f0 2024-03-04T13:40:58.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:58.324+0000 7ff548000700 1 -- 172.21.15.107:0/3141041962 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff54004bed0 con 0x7ff5401002f0 2024-03-04T13:40:58.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:58.325+0000 7ff53d600700 1 -- 172.21.15.107:0/3141041962 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86623+0+0 (secure 0 0 0) 0x7ff52c0139f0 con 0x7ff5401002f0 2024-03-04T13:40:58.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:58.326+0000 7ff53d600700 1 --2- 172.21.15.107:0/3141041962 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7ff5280692a0 0x7ff52806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:40:58.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:58.326+0000 7ff53d600700 1 -- 172.21.15.107:0/3141041962 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5550+0+0 (secure 0 0 0) 0x7ff52c088610 con 0x7ff5401002f0 2024-03-04T13:40:58.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:58.326+0000 7ff545600700 1 --2- 172.21.15.107:0/3141041962 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7ff5280692a0 0x7ff52806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:40:58.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:58.327+0000 7ff545600700 1 --2- 172.21.15.107:0/3141041962 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7ff5280692a0 0x7ff52806b750 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff534006fd0 tx=0x7ff534008040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:40:58.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:58.330+0000 7ff53d600700 1 -- 172.21.15.107:0/3141041962 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175031 (secure 0 0 0) 0x7ff52c058360 con 0x7ff5401002f0 2024-03-04T13:40:58.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:58.581+0000 7ff548000700 1 -- 172.21.15.107:0/3141041962 --> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff5400008d0 con 0x7ff5280692a0 2024-03-04T13:40:58.577 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:40:58 smithi107 ceph-mon[28739]: from='client.14576 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:40:58.577 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:40:58 smithi107 ceph-mon[28739]: pgmap v290: 65 pgs: 65 active+clean; 13 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 52 KiB/s rd, 1.0 MiB/s wr, 15 op/s 2024-03-04T13:40:58.577 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:40:58 smithi107 ceph-mon[28739]: from='client.14580 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:40:58.577 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:40:58 smithi107 ceph-mon[28739]: from='client.? 172.21.15.107:0/2109773038' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:40:58.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:58.583+0000 7ff53d600700 1 -- 172.21.15.107:0/3141041962 <== mgr.14212 v2:172.21.15.107:6800/3848495603 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7ff5400008d0 con 0x7ff5280692a0 2024-03-04T13:40:58.579 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-03-04T13:40:58.580 INFO:teuthology.orchestra.run.smithi107.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377", 2024-03-04T13:40:58.580 INFO:teuthology.orchestra.run.smithi107.stdout: "in_progress": true, 2024-03-04T13:40:58.580 INFO:teuthology.orchestra.run.smithi107.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-03-04T13:40:58.580 INFO:teuthology.orchestra.run.smithi107.stdout: "services_complete": [], 2024-03-04T13:40:58.580 INFO:teuthology.orchestra.run.smithi107.stdout: "progress": "0/2 daemons upgraded", 2024-03-04T13:40:58.580 INFO:teuthology.orchestra.run.smithi107.stdout: "message": "Currently upgrading mgr daemons", 2024-03-04T13:40:58.580 INFO:teuthology.orchestra.run.smithi107.stdout: "is_paused": false 2024-03-04T13:40:58.580 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-03-04T13:40:58.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:58.586+0000 7ff548000700 1 -- 172.21.15.107:0/3141041962 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7ff5280692a0 msgr2=0x7ff52806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:40:58.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:58.586+0000 7ff548000700 1 --2- 172.21.15.107:0/3141041962 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7ff5280692a0 0x7ff52806b750 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff534006fd0 tx=0x7ff534008040 comp rx=0 tx=0).stop 2024-03-04T13:40:58.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:58.586+0000 7ff548000700 1 -- 172.21.15.107:0/3141041962 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5401002f0 msgr2=0x7ff540110000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:40:58.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:58.586+0000 7ff548000700 1 --2- 172.21.15.107:0/3141041962 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5401002f0 0x7ff540110000 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ff52c0093b0 tx=0x7ff52c009e30 comp rx=0 tx=0).stop 2024-03-04T13:40:58.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:58.587+0000 7ff548000700 1 -- 172.21.15.107:0/3141041962 shutdown_connections 2024-03-04T13:40:58.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:58.587+0000 7ff548000700 1 --2- 172.21.15.107:0/3141041962 >> [v2:172.21.15.107:6800/3848495603,v1:172.21.15.107:6801/3848495603] conn(0x7ff5280692a0 0x7ff52806b750 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:40:58.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:58.587+0000 7ff548000700 1 --2- 172.21.15.107:0/3141041962 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff5400ff0e0 0x7ff54010fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:40:58.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:58.587+0000 7ff548000700 1 --2- 172.21.15.107:0/3141041962 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5401002f0 0x7ff540110000 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:40:58.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:58.587+0000 7ff548000700 1 -- 172.21.15.107:0/3141041962 >> 172.21.15.107:0/3141041962 conn(0x7ff5400fa7c0 msgr2=0x7ff540103520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:40:58.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:58.587+0000 7ff548000700 1 -- 172.21.15.107:0/3141041962 shutdown_connections 2024-03-04T13:40:58.584 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:40:58.587+0000 7ff548000700 1 -- 172.21.15.107:0/3141041962 wait complete. 2024-03-04T13:40:58.855 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:40:58 smithi169 ceph-mon[36313]: from='client.14576 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:40:58.855 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:40:58 smithi169 ceph-mon[36313]: pgmap v290: 65 pgs: 65 active+clean; 13 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 52 KiB/s rd, 1.0 MiB/s wr, 15 op/s 2024-03-04T13:40:58.855 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:40:58 smithi169 ceph-mon[36313]: from='client.14580 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:40:58.856 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:40:58 smithi169 ceph-mon[36313]: from='client.? 172.21.15.107:0/2109773038' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:40:59.588 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:40:59 smithi107 ceph-mon[28739]: from='client.14588 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:40:59.771 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:40:59 smithi169 ceph-mon[36313]: from='client.14588 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:41:00.838 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:00 smithi107 ceph-mon[28739]: pgmap v291: 65 pgs: 65 active+clean; 13 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 52 KiB/s rd, 1.0 MiB/s wr, 16 op/s 2024-03-04T13:41:00.855 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:00 smithi169 ceph-mon[36313]: pgmap v291: 65 pgs: 65 active+clean; 13 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 52 KiB/s rd, 1.0 MiB/s wr, 16 op/s 2024-03-04T13:41:02.856 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:02 smithi169 ceph-mon[36313]: pgmap v292: 65 pgs: 65 active+clean; 26 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 210 KiB/s rd, 2.1 MiB/s wr, 33 op/s 2024-03-04T13:41:03.088 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:02 smithi107 ceph-mon[28739]: pgmap v292: 65 pgs: 65 active+clean; 26 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 210 KiB/s rd, 2.1 MiB/s wr, 33 op/s 2024-03-04T13:41:05.091 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:04 smithi107 ceph-mon[28739]: pgmap v293: 65 pgs: 65 active+clean; 27 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 210 KiB/s rd, 2.2 MiB/s wr, 45 op/s 2024-03-04T13:41:05.106 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:04 smithi169 ceph-mon[36313]: pgmap v293: 65 pgs: 65 active+clean; 27 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 210 KiB/s rd, 2.2 MiB/s wr, 45 op/s 2024-03-04T13:41:07.088 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:06 smithi107 ceph-mon[28739]: pgmap v294: 65 pgs: 65 active+clean; 31 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 342 KiB/s rd, 2.2 MiB/s wr, 66 op/s 2024-03-04T13:41:07.105 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:06 smithi169 ceph-mon[36313]: pgmap v294: 65 pgs: 65 active+clean; 31 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 342 KiB/s rd, 2.2 MiB/s wr, 66 op/s 2024-03-04T13:41:08.923 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:08 smithi169 ceph-mon[36313]: pgmap v295: 65 pgs: 65 active+clean; 32 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 290 KiB/s rd, 1.6 MiB/s wr, 66 op/s 2024-03-04T13:41:09.088 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:08 smithi107 ceph-mon[28739]: pgmap v295: 65 pgs: 65 active+clean; 32 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 290 KiB/s rd, 1.6 MiB/s wr, 66 op/s 2024-03-04T13:41:11.089 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:10 smithi107 ceph-mon[28739]: pgmap v296: 65 pgs: 65 active+clean; 32 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 290 KiB/s rd, 1.6 MiB/s wr, 63 op/s 2024-03-04T13:41:11.089 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:10 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:41:11.089 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:10 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:41:11.089 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:10 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:41:11.106 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:10 smithi169 ceph-mon[36313]: pgmap v296: 65 pgs: 65 active+clean; 32 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 290 KiB/s rd, 1.6 MiB/s wr, 63 op/s 2024-03-04T13:41:11.106 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:10 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:41:11.106 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:10 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:41:11.106 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:10 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:41:12.339 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:12 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:41:12.339 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:12 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:41:12.339 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:12 smithi107 ceph-mon[28739]: pgmap v297: 65 pgs: 65 active+clean; 36 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 818 KiB/s rd, 2.0 MiB/s wr, 77 op/s 2024-03-04T13:41:12.355 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:12 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:41:12.355 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:12 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:41:12.356 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:12 smithi169 ceph-mon[36313]: pgmap v297: 65 pgs: 65 active+clean; 36 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 818 KiB/s rd, 2.0 MiB/s wr, 77 op/s 2024-03-04T13:41:13.338 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:13 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:41:13.356 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:13 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:41:14.338 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:14 smithi107 ceph-mon[28739]: pgmap v298: 65 pgs: 65 active+clean; 37 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 660 KiB/s rd, 990 KiB/s wr, 71 op/s 2024-03-04T13:41:14.355 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:14 smithi169 ceph-mon[36313]: pgmap v298: 65 pgs: 65 active+clean; 37 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 660 KiB/s rd, 990 KiB/s wr, 71 op/s 2024-03-04T13:41:16.056 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:15 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:41:16.056 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:15 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:41:16.088 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:15 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:41:16.088 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:15 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:41:17.037 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:16 smithi169 ceph-mon[36313]: pgmap v299: 65 pgs: 65 active+clean; 39 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 816 KiB/s rd, 1.1 MiB/s wr, 77 op/s 2024-03-04T13:41:17.037 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:16 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:41:17.037 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:16 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:41:17.088 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:16 smithi107 ceph-mon[28739]: pgmap v299: 65 pgs: 65 active+clean; 39 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 816 KiB/s rd, 1.1 MiB/s wr, 77 op/s 2024-03-04T13:41:17.089 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:16 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:41:17.089 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:16 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:41:18.856 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:18 smithi169 ceph-mon[36313]: pgmap v300: 65 pgs: 65 active+clean; 44 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 70 op/s 2024-03-04T13:41:19.088 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:18 smithi107 ceph-mon[28739]: pgmap v300: 65 pgs: 65 active+clean; 44 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 70 op/s 2024-03-04T13:41:20.683 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:20 smithi169 ceph-mon[36313]: pgmap v301: 65 pgs: 65 active+clean; 44 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 65 op/s 2024-03-04T13:41:21.089 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:20 smithi107 ceph-mon[28739]: pgmap v301: 65 pgs: 65 active+clean; 44 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 65 op/s 2024-03-04T13:41:22.781 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:22 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:41:22.781 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:22 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:41:22.781 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:22 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:41:22.781 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:22 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:41:22.781 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:22 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:41:22.781 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:22 smithi169 ceph-mon[36313]: Standby manager daemon smithi169.eihczr restarted 2024-03-04T13:41:22.781 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:22 smithi169 ceph-mon[36313]: Standby manager daemon smithi169.eihczr started 2024-03-04T13:41:22.781 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:22 smithi169 ceph-mon[36313]: from='mgr.? 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi169.eihczr/crt"}]: dispatch 2024-03-04T13:41:22.781 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:22 smithi169 ceph-mon[36313]: from='mgr.? 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-04T13:41:22.781 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:22 smithi169 ceph-mon[36313]: from='mgr.? 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi169.eihczr/key"}]: dispatch 2024-03-04T13:41:22.781 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:22 smithi169 ceph-mon[36313]: from='mgr.? 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-04T13:41:22.782 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:22 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:41:22.782 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:22 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mgr fail", "who": "smithi107.wpzgwl"}]: dispatch 2024-03-04T13:41:22.782 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:22 smithi169 ceph-mon[36313]: osdmap e48: 6 total, 6 up, 6 in 2024-03-04T13:41:22.782 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:22 smithi169 ceph-mon[36313]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd='[{"prefix": "mgr fail", "who": "smithi107.wpzgwl"}]': finished 2024-03-04T13:41:22.782 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:22 smithi169 ceph-mon[36313]: mgrmap e20: smithi169.eihczr(active, starting, since 0.0184387s) 2024-03-04T13:41:22.782 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:22 smithi169 ceph-mon[36313]: from='mgr.24331 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "mon metadata", "id": "smithi107"}]: dispatch 2024-03-04T13:41:22.782 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:22 smithi169 ceph-mon[36313]: from='mgr.24331 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "mon metadata", "id": "smithi169"}]: dispatch 2024-03-04T13:41:22.782 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:22 smithi169 ceph-mon[36313]: from='mgr.24331 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi107.thycho"}]: dispatch 2024-03-04T13:41:22.782 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:22 smithi169 ceph-mon[36313]: from='mgr.24331 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi107.riexmm"}]: dispatch 2024-03-04T13:41:22.782 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:22 smithi169 ceph-mon[36313]: from='mgr.24331 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi169.jusudl"}]: dispatch 2024-03-04T13:41:22.782 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:22 smithi169 ceph-mon[36313]: from='mgr.24331 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi169.bxitzk"}]: dispatch 2024-03-04T13:41:22.782 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:22 smithi169 ceph-mon[36313]: from='mgr.24331 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "mgr metadata", "who": "smithi169.eihczr", "id": "smithi169.eihczr"}]: dispatch 2024-03-04T13:41:22.782 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:22 smithi169 ceph-mon[36313]: from='mgr.24331 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-04T13:41:22.782 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:22 smithi169 ceph-mon[36313]: from='mgr.24331 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-04T13:41:22.782 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:22 smithi169 ceph-mon[36313]: from='mgr.24331 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-04T13:41:22.782 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:22 smithi169 ceph-mon[36313]: from='mgr.24331 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-04T13:41:22.783 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:22 smithi169 ceph-mon[36313]: from='mgr.24331 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-04T13:41:22.783 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:22 smithi169 ceph-mon[36313]: from='mgr.24331 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-04T13:41:22.783 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:22 smithi169 ceph-mon[36313]: from='mgr.24331 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-04T13:41:22.783 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:22 smithi169 ceph-mon[36313]: from='mgr.24331 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-04T13:41:22.783 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:22 smithi169 ceph-mon[36313]: from='mgr.24331 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-04T13:41:22.783 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:22 smithi169 ceph-mon[36313]: Manager daemon smithi169.eihczr is now available 2024-03-04T13:41:22.783 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:22 smithi169 ceph-mon[36313]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' 2024-03-04T13:41:22.783 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:22 smithi169 ceph-mon[36313]: from='mgr.24331 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:41:22.783 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:22 smithi169 ceph-mon[36313]: from='mgr.24331 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:41:22.783 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:22 smithi169 ceph-mon[36313]: from='mgr.24331 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:41:22.783 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:22 smithi169 ceph-mon[36313]: from='mgr.24331 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi169.eihczr/mirror_snapshot_schedule"}]: dispatch 2024-03-04T13:41:22.783 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:22 smithi169 ceph-mon[36313]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi169.eihczr/mirror_snapshot_schedule"}]: dispatch 2024-03-04T13:41:22.783 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:22 smithi169 ceph-mon[36313]: from='mgr.24331 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi169.eihczr/trash_purge_schedule"}]: dispatch 2024-03-04T13:41:22.783 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:22 smithi169 ceph-mon[36313]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi169.eihczr/trash_purge_schedule"}]: dispatch 2024-03-04T13:41:22.783 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:22 smithi169 ceph-mon[36313]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' 2024-03-04T13:41:22.783 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:22 smithi169 ceph-mon[36313]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' 2024-03-04T13:41:22.962 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:22 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:41:22.962 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:22 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:41:22.962 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:22 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:41:22.962 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:22 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:41:22.962 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:22 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:41:22.963 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:22 smithi107 ceph-mon[28739]: Standby manager daemon smithi169.eihczr restarted 2024-03-04T13:41:22.963 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:22 smithi107 ceph-mon[28739]: Standby manager daemon smithi169.eihczr started 2024-03-04T13:41:22.963 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:22 smithi107 ceph-mon[28739]: from='mgr.? 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi169.eihczr/crt"}]: dispatch 2024-03-04T13:41:22.963 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:22 smithi107 ceph-mon[28739]: from='mgr.? 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-04T13:41:22.963 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:22 smithi107 ceph-mon[28739]: from='mgr.? 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi169.eihczr/key"}]: dispatch 2024-03-04T13:41:22.963 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:22 smithi107 ceph-mon[28739]: from='mgr.? 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-04T13:41:22.963 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:22 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:41:22.963 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:22 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mgr fail", "who": "smithi107.wpzgwl"}]: dispatch 2024-03-04T13:41:22.963 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:22 smithi107 ceph-mon[28739]: osdmap e48: 6 total, 6 up, 6 in 2024-03-04T13:41:22.963 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:22 smithi107 ceph-mon[28739]: from='mgr.14212 172.21.15.107:0/2022936759' entity='mgr.smithi107.wpzgwl' cmd='[{"prefix": "mgr fail", "who": "smithi107.wpzgwl"}]': finished 2024-03-04T13:41:22.963 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:22 smithi107 ceph-mon[28739]: mgrmap e20: smithi169.eihczr(active, starting, since 0.0184387s) 2024-03-04T13:41:22.963 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:22 smithi107 ceph-mon[28739]: from='mgr.24331 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "mon metadata", "id": "smithi107"}]: dispatch 2024-03-04T13:41:22.963 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:22 smithi107 ceph-mon[28739]: from='mgr.24331 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "mon metadata", "id": "smithi169"}]: dispatch 2024-03-04T13:41:22.963 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:22 smithi107 ceph-mon[28739]: from='mgr.24331 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi107.thycho"}]: dispatch 2024-03-04T13:41:22.963 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:22 smithi107 ceph-mon[28739]: from='mgr.24331 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi107.riexmm"}]: dispatch 2024-03-04T13:41:22.964 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:22 smithi107 ceph-mon[28739]: from='mgr.24331 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi169.jusudl"}]: dispatch 2024-03-04T13:41:22.964 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:22 smithi107 ceph-mon[28739]: from='mgr.24331 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi169.bxitzk"}]: dispatch 2024-03-04T13:41:22.964 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:22 smithi107 ceph-mon[28739]: from='mgr.24331 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "mgr metadata", "who": "smithi169.eihczr", "id": "smithi169.eihczr"}]: dispatch 2024-03-04T13:41:22.964 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:22 smithi107 ceph-mon[28739]: from='mgr.24331 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-04T13:41:22.964 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:22 smithi107 ceph-mon[28739]: from='mgr.24331 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-04T13:41:22.964 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:22 smithi107 ceph-mon[28739]: from='mgr.24331 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-04T13:41:22.964 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:22 smithi107 ceph-mon[28739]: from='mgr.24331 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-04T13:41:22.964 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:22 smithi107 ceph-mon[28739]: from='mgr.24331 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-04T13:41:22.964 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:22 smithi107 ceph-mon[28739]: from='mgr.24331 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-04T13:41:22.965 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:22 smithi107 ceph-mon[28739]: from='mgr.24331 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-04T13:41:22.965 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:22 smithi107 ceph-mon[28739]: from='mgr.24331 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-04T13:41:22.965 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:22 smithi107 ceph-mon[28739]: from='mgr.24331 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-04T13:41:22.965 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:22 smithi107 ceph-mon[28739]: Manager daemon smithi169.eihczr is now available 2024-03-04T13:41:22.965 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:22 smithi107 ceph-mon[28739]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' 2024-03-04T13:41:22.965 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:22 smithi107 ceph-mon[28739]: from='mgr.24331 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:41:22.965 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:22 smithi107 ceph-mon[28739]: from='mgr.24331 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:41:22.965 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:22 smithi107 ceph-mon[28739]: from='mgr.24331 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:41:22.965 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:22 smithi107 ceph-mon[28739]: from='mgr.24331 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi169.eihczr/mirror_snapshot_schedule"}]: dispatch 2024-03-04T13:41:22.965 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:22 smithi107 ceph-mon[28739]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi169.eihczr/mirror_snapshot_schedule"}]: dispatch 2024-03-04T13:41:22.965 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:22 smithi107 ceph-mon[28739]: from='mgr.24331 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi169.eihczr/trash_purge_schedule"}]: dispatch 2024-03-04T13:41:22.965 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:22 smithi107 ceph-mon[28739]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi169.eihczr/trash_purge_schedule"}]: dispatch 2024-03-04T13:41:22.965 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:22 smithi107 ceph-mon[28739]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' 2024-03-04T13:41:22.965 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:22 smithi107 ceph-mon[28739]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' 2024-03-04T13:41:24.133 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:23 smithi107 ceph-mon[28739]: Deploying cephadm binary to smithi107 2024-03-04T13:41:24.133 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:23 smithi107 ceph-mon[28739]: Deploying cephadm binary to smithi169 2024-03-04T13:41:24.133 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:23 smithi107 ceph-mon[28739]: mgrmap e21: smithi169.eihczr(active, since 1.01405s) 2024-03-04T13:41:24.133 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:23 smithi107 ceph-mon[28739]: pgmap v3: 65 pgs: 65 active+clean; 63 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail 2024-03-04T13:41:24.355 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:23 smithi169 ceph-mon[36313]: Deploying cephadm binary to smithi107 2024-03-04T13:41:24.356 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:23 smithi169 ceph-mon[36313]: Deploying cephadm binary to smithi169 2024-03-04T13:41:24.356 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:23 smithi169 ceph-mon[36313]: mgrmap e21: smithi169.eihczr(active, since 1.01405s) 2024-03-04T13:41:24.356 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:23 smithi169 ceph-mon[36313]: pgmap v3: 65 pgs: 65 active+clean; 63 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail 2024-03-04T13:41:25.001 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:24 smithi107 ceph-mon[28739]: [04/Mar/2024:13:41:23] ENGINE Bus STARTING 2024-03-04T13:41:25.001 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:24 smithi107 ceph-mon[28739]: pgmap v4: 65 pgs: 65 active+clean; 63 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail 2024-03-04T13:41:25.001 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:24 smithi107 ceph-mon[28739]: mgrmap e22: smithi169.eihczr(active, since 2s) 2024-03-04T13:41:25.002 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:24 smithi107 ceph-mon[28739]: [04/Mar/2024:13:41:24] ENGINE Serving on https://172.21.15.169:7150 2024-03-04T13:41:25.356 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:24 smithi169 ceph-mon[36313]: [04/Mar/2024:13:41:23] ENGINE Bus STARTING 2024-03-04T13:41:25.356 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:24 smithi169 ceph-mon[36313]: pgmap v4: 65 pgs: 65 active+clean; 63 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail 2024-03-04T13:41:25.356 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:24 smithi169 ceph-mon[36313]: mgrmap e22: smithi169.eihczr(active, since 2s) 2024-03-04T13:41:25.356 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:24 smithi169 ceph-mon[36313]: [04/Mar/2024:13:41:24] ENGINE Serving on https://172.21.15.169:7150 2024-03-04T13:41:26.338 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:25 smithi107 ceph-mon[28739]: [04/Mar/2024:13:41:24] ENGINE Serving on http://172.21.15.169:8765 2024-03-04T13:41:26.339 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:25 smithi107 ceph-mon[28739]: [04/Mar/2024:13:41:24] ENGINE Bus STARTED 2024-03-04T13:41:26.355 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:25 smithi169 ceph-mon[36313]: [04/Mar/2024:13:41:24] ENGINE Serving on http://172.21.15.169:8765 2024-03-04T13:41:26.355 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:25 smithi169 ceph-mon[36313]: [04/Mar/2024:13:41:24] ENGINE Bus STARTED 2024-03-04T13:41:27.223 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:26 smithi107 ceph-mon[28739]: pgmap v5: 65 pgs: 65 active+clean; 63 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail 2024-03-04T13:41:27.223 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:26 smithi107 ceph-mon[28739]: mgrmap e23: smithi169.eihczr(active, since 4s) 2024-03-04T13:41:27.356 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:26 smithi169 ceph-mon[36313]: pgmap v5: 65 pgs: 65 active+clean; 63 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail 2024-03-04T13:41:27.356 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:26 smithi169 ceph-mon[36313]: mgrmap e23: smithi169.eihczr(active, since 4s) 2024-03-04T13:41:28.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:28.777+0000 7fdd04600700 1 -- 172.21.15.107:0/3770981440 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdcfc066c70 msgr2=0x7fdcfc0ffaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:41:28.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:28.777+0000 7fdd04600700 1 --2- 172.21.15.107:0/3770981440 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdcfc066c70 0x7fdcfc0ffaf0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fdcf0004940 tx=0x7fdcf0009e30 comp rx=0 tx=0).stop 2024-03-04T13:41:28.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:28.777+0000 7fdd04600700 1 -- 172.21.15.107:0/3770981440 shutdown_connections 2024-03-04T13:41:28.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:28.777+0000 7fdd04600700 1 --2- 172.21.15.107:0/3770981440 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdcfc1000c0 0x7fdcfc102520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:41:28.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:28.777+0000 7fdd04600700 1 --2- 172.21.15.107:0/3770981440 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdcfc066c70 0x7fdcfc0ffaf0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:41:28.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:28.777+0000 7fdd04600700 1 -- 172.21.15.107:0/3770981440 >> 172.21.15.107:0/3770981440 conn(0x7fdcfc0f7520 msgr2=0x7fdcfc0f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:41:28.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:28.777+0000 7fdd04600700 1 -- 172.21.15.107:0/3770981440 shutdown_connections 2024-03-04T13:41:28.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:28.777+0000 7fdd04600700 1 -- 172.21.15.107:0/3770981440 wait complete. 2024-03-04T13:41:28.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:28.778+0000 7fdd04600700 1 Processor -- start 2024-03-04T13:41:28.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:28.778+0000 7fdd04600700 1 -- start start 2024-03-04T13:41:28.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:28.778+0000 7fdd04600700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdcfc066c70 0x7fdcfc10fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:41:28.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:28.779+0000 7fdd04600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdcfc1000c0 0x7fdcfc1100a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:41:28.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:28.779+0000 7fdd04600700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcfc1106a0 con 0x7fdcfc1000c0 2024-03-04T13:41:28.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:28.779+0000 7fdd04600700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcfc110f40 con 0x7fdcfc066c70 2024-03-04T13:41:28.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:28.779+0000 7fdd01200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdcfc1000c0 0x7fdcfc1100a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:41:28.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:28.779+0000 7fdd01200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdcfc1000c0 0x7fdcfc1100a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57810/0 (socket says 172.21.15.107:57810) 2024-03-04T13:41:28.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:28.779+0000 7fdd01200700 1 -- 172.21.15.107:0/3497414478 learned_addr learned my addr 172.21.15.107:0/3497414478 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:41:28.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:28.779+0000 7fdd01c00700 1 --2- 172.21.15.107:0/3497414478 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdcfc066c70 0x7fdcfc10fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:41:28.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:28.780+0000 7fdd01c00700 1 -- 172.21.15.107:0/3497414478 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdcfc1000c0 msgr2=0x7fdcfc1100a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:41:28.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:28.780+0000 7fdd01c00700 1 --2- 172.21.15.107:0/3497414478 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdcfc1000c0 0x7fdcfc1100a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:41:28.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:28.780+0000 7fdd01c00700 1 -- 172.21.15.107:0/3497414478 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcf0005040 con 0x7fdcfc066c70 2024-03-04T13:41:28.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:28.780+0000 7fdd01c00700 1 --2- 172.21.15.107:0/3497414478 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdcfc066c70 0x7fdcfc10fb60 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fdcf0000c00 tx=0x7fdcf0009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:41:28.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:28.781+0000 7fdcf9600700 1 -- 172.21.15.107:0/3497414478 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdcf000e040 con 0x7fdcfc066c70 2024-03-04T13:41:28.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:28.781+0000 7fdd04600700 1 -- 172.21.15.107:0/3497414478 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcfc1111c0 con 0x7fdcfc066c70 2024-03-04T13:41:28.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:28.781+0000 7fdd04600700 1 -- 172.21.15.107:0/3497414478 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcfc111700 con 0x7fdcfc066c70 2024-03-04T13:41:28.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:28.781+0000 7fdcf9600700 1 -- 172.21.15.107:0/3497414478 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdcf0004290 con 0x7fdcfc066c70 2024-03-04T13:41:28.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:28.781+0000 7fdcf9600700 1 -- 172.21.15.107:0/3497414478 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdcf0013dd0 con 0x7fdcfc066c70 2024-03-04T13:41:28.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:28.782+0000 7fdd04600700 1 -- 172.21.15.107:0/3497414478 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcdc0052e0 con 0x7fdcfc066c70 2024-03-04T13:41:28.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:28.783+0000 7fdcf9600700 1 -- 172.21.15.107:0/3497414478 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 23) v1 ==== 45929+0+0 (secure 0 0 0) 0x7fdcf001c440 con 0x7fdcfc066c70 2024-03-04T13:41:28.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:28.783+0000 7fdcf9600700 1 --2- 172.21.15.107:0/3497414478 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7fdce4038a70 0x7fdce403af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:41:28.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:28.783+0000 7fdcf9600700 1 -- 172.21.15.107:0/3497414478 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5808+0+0 (secure 0 0 0) 0x7fdcf004daf0 con 0x7fdcfc066c70 2024-03-04T13:41:28.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:28.784+0000 7fdd01200700 1 --2- 172.21.15.107:0/3497414478 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7fdce4038a70 0x7fdce403af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:41:28.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:28.785+0000 7fdd01200700 1 --2- 172.21.15.107:0/3497414478 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7fdce4038a70 0x7fdce403af20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fdce8009fd0 tx=0x7fdce800b040 comp rx=0 tx=0).ready entity=mgr.24331 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:41:28.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:28.800+0000 7fdcf9600700 1 -- 172.21.15.107:0/3497414478 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7fdcf0011990 con 0x7fdcfc066c70 2024-03-04T13:41:29.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:29.074+0000 7fdd04600700 1 -- 172.21.15.107:0/3497414478 --> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdcdc000bf0 con 0x7fdce4038a70 2024-03-04T13:41:29.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:29.080+0000 7fdcf9600700 1 -- 172.21.15.107:0/3497414478 <== mgr.24331 v2:172.21.15.169:6828/3524931243 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7fdcdc000bf0 con 0x7fdce4038a70 2024-03-04T13:41:29.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:29.083+0000 7fdd04600700 1 -- 172.21.15.107:0/3497414478 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7fdce4038a70 msgr2=0x7fdce403af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:41:29.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:29.083+0000 7fdd04600700 1 --2- 172.21.15.107:0/3497414478 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7fdce4038a70 0x7fdce403af20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fdce8009fd0 tx=0x7fdce800b040 comp rx=0 tx=0).stop 2024-03-04T13:41:29.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:29.083+0000 7fdd04600700 1 -- 172.21.15.107:0/3497414478 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdcfc066c70 msgr2=0x7fdcfc10fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:41:29.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:29.083+0000 7fdd04600700 1 --2- 172.21.15.107:0/3497414478 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdcfc066c70 0x7fdcfc10fb60 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fdcf0000c00 tx=0x7fdcf0009e30 comp rx=0 tx=0).stop 2024-03-04T13:41:29.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:29.083+0000 7fdd04600700 1 -- 172.21.15.107:0/3497414478 shutdown_connections 2024-03-04T13:41:29.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:29.083+0000 7fdd04600700 1 --2- 172.21.15.107:0/3497414478 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7fdce4038a70 0x7fdce403af20 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:41:29.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:29.083+0000 7fdd04600700 1 --2- 172.21.15.107:0/3497414478 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdcfc066c70 0x7fdcfc10fb60 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:41:29.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:29.083+0000 7fdd04600700 1 --2- 172.21.15.107:0/3497414478 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdcfc1000c0 0x7fdcfc1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:41:29.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:29.083+0000 7fdd04600700 1 -- 172.21.15.107:0/3497414478 >> 172.21.15.107:0/3497414478 conn(0x7fdcfc0f7520 msgr2=0x7fdcfc0fb8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:41:29.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:29.084+0000 7fdd04600700 1 -- 172.21.15.107:0/3497414478 shutdown_connections 2024-03-04T13:41:29.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:29.084+0000 7fdd04600700 1 -- 172.21.15.107:0/3497414478 wait complete. 2024-03-04T13:41:29.094 INFO:teuthology.orchestra.run.smithi107.stdout:true 2024-03-04T13:41:29.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:29.276+0000 7f03ae400700 1 -- 172.21.15.107:0/4039876439 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03a80fcfb0 msgr2=0x7f03a80fd420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:41:29.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:29.276+0000 7f03ae400700 1 --2- 172.21.15.107:0/4039876439 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03a80fcfb0 0x7f03a80fd420 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f03900049c0 tx=0x7f0390009e30 comp rx=0 tx=0).stop 2024-03-04T13:41:29.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:29.276+0000 7f03ae400700 1 -- 172.21.15.107:0/4039876439 shutdown_connections 2024-03-04T13:41:29.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:29.277+0000 7f03ae400700 1 --2- 172.21.15.107:0/4039876439 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03a80fcfb0 0x7f03a80fd420 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:41:29.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:29.277+0000 7f03ae400700 1 --2- 172.21.15.107:0/4039876439 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03a80fbe40 0x7f03a80fc250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:41:29.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:29.277+0000 7f03ae400700 1 -- 172.21.15.107:0/4039876439 >> 172.21.15.107:0/4039876439 conn(0x7f03a80f74e0 msgr2=0x7f03a80f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:41:29.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:29.277+0000 7f03ae400700 1 -- 172.21.15.107:0/4039876439 shutdown_connections 2024-03-04T13:41:29.273 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:28 smithi107 ceph-mon[28739]: pgmap v6: 65 pgs: 65 active+clean; 69 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 1.3 MiB/s wr, 59 op/s 2024-03-04T13:41:29.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:29.278+0000 7f03ae400700 1 -- 172.21.15.107:0/4039876439 wait complete. 2024-03-04T13:41:29.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:29.279+0000 7f03ae400700 1 Processor -- start 2024-03-04T13:41:29.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:29.279+0000 7f03ae400700 1 -- start start 2024-03-04T13:41:29.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:29.280+0000 7f03ae400700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03a80fbe40 0x7f03a810fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:41:29.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:29.280+0000 7f03ae400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03a8110280 0x7f03a8111040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:41:29.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:29.280+0000 7f03ae400700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03a8110740 con 0x7f03a8110280 2024-03-04T13:41:29.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:29.280+0000 7f03ae400700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03a81108b0 con 0x7f03a80fbe40 2024-03-04T13:41:29.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:29.281+0000 7f03a7400700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03a80fbe40 0x7f03a810fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:41:29.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:29.281+0000 7f03a7400700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03a80fbe40 0x7f03a810fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.107:43132/0 (socket says 172.21.15.107:43132) 2024-03-04T13:41:29.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:29.281+0000 7f03a7400700 1 -- 172.21.15.107:0/2639329006 learned_addr learned my addr 172.21.15.107:0/2639329006 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:41:29.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:29.281+0000 7f039fe00700 1 --2- 172.21.15.107:0/2639329006 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03a8110280 0x7f03a8111040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:41:29.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:29.282+0000 7f03a7400700 1 -- 172.21.15.107:0/2639329006 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03a8110280 msgr2=0x7f03a8111040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:41:29.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:29.282+0000 7f03a7400700 1 --2- 172.21.15.107:0/2639329006 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03a8110280 0x7f03a8111040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:41:29.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:29.282+0000 7f03a7400700 1 -- 172.21.15.107:0/2639329006 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0390005040 con 0x7f03a80fbe40 2024-03-04T13:41:29.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:29.282+0000 7f03a7400700 1 --2- 172.21.15.107:0/2639329006 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03a80fbe40 0x7f03a810fd40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0398001d80 tx=0x7f0398009ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:41:29.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:29.283+0000 7f039f400700 1 -- 172.21.15.107:0/2639329006 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0398007d10 con 0x7f03a80fbe40 2024-03-04T13:41:29.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:29.283+0000 7f039f400700 1 -- 172.21.15.107:0/2639329006 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0398007e70 con 0x7f03a80fbe40 2024-03-04T13:41:29.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:29.283+0000 7f039f400700 1 -- 172.21.15.107:0/2639329006 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0398014640 con 0x7f03a80fbe40 2024-03-04T13:41:29.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:29.283+0000 7f03ae400700 1 -- 172.21.15.107:0/2639329006 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03a8111720 con 0x7f03a80fbe40 2024-03-04T13:41:29.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:29.283+0000 7f03ae400700 1 -- 172.21.15.107:0/2639329006 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03a8111c60 con 0x7f03a80fbe40 2024-03-04T13:41:29.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:29.284+0000 7f03ae400700 1 -- 172.21.15.107:0/2639329006 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03a804bed0 con 0x7f03a80fbe40 2024-03-04T13:41:29.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:29.285+0000 7f039f400700 1 -- 172.21.15.107:0/2639329006 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 23) v1 ==== 45929+0+0 (secure 0 0 0) 0x7f039800e960 con 0x7f03a80fbe40 2024-03-04T13:41:29.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:29.286+0000 7f039f400700 1 --2- 172.21.15.107:0/2639329006 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7f038c038ab0 0x7f038c03af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:41:29.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:29.286+0000 7f039f400700 1 -- 172.21.15.107:0/2639329006 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f039804c320 con 0x7f03a80fbe40 2024-03-04T13:41:29.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:29.287+0000 7f039fe00700 1 --2- 172.21.15.107:0/2639329006 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7f038c038ab0 0x7f038c03af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:41:29.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:29.289+0000 7f039fe00700 1 --2- 172.21.15.107:0/2639329006 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7f038c038ab0 0x7f038c03af60 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0390009040 tx=0x7f039000b040 comp rx=0 tx=0).ready entity=mgr.24331 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:41:29.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:29.294+0000 7f039f400700 1 -- 172.21.15.107:0/2639329006 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f03980124e0 con 0x7f03a80fbe40 2024-03-04T13:41:29.356 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:28 smithi169 ceph-mon[36313]: pgmap v6: 65 pgs: 65 active+clean; 69 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 1.3 MiB/s wr, 59 op/s 2024-03-04T13:41:29.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:29.616+0000 7f03ae400700 1 -- 172.21.15.107:0/2639329006 --> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f03a8101a10 con 0x7f038c038ab0 2024-03-04T13:41:29.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:29.618+0000 7f039f400700 1 -- 172.21.15.107:0/2639329006 <== mgr.24331 v2:172.21.15.169:6828/3524931243 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f03a8101a10 con 0x7f038c038ab0 2024-03-04T13:41:29.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:29.623+0000 7f03ae400700 1 -- 172.21.15.107:0/2639329006 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7f038c038ab0 msgr2=0x7f038c03af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:41:29.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:29.623+0000 7f03ae400700 1 --2- 172.21.15.107:0/2639329006 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7f038c038ab0 0x7f038c03af60 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0390009040 tx=0x7f039000b040 comp rx=0 tx=0).stop 2024-03-04T13:41:29.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:29.623+0000 7f03ae400700 1 -- 172.21.15.107:0/2639329006 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03a80fbe40 msgr2=0x7f03a810fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:41:29.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:29.623+0000 7f03ae400700 1 --2- 172.21.15.107:0/2639329006 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03a80fbe40 0x7f03a810fd40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0398001d80 tx=0x7f0398009ec0 comp rx=0 tx=0).stop 2024-03-04T13:41:29.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:29.623+0000 7f03ae400700 1 -- 172.21.15.107:0/2639329006 shutdown_connections 2024-03-04T13:41:29.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:29.623+0000 7f03ae400700 1 --2- 172.21.15.107:0/2639329006 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7f038c038ab0 0x7f038c03af60 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:41:29.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:29.623+0000 7f03ae400700 1 --2- 172.21.15.107:0/2639329006 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f03a80fbe40 0x7f03a810fd40 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:41:29.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:29.623+0000 7f03ae400700 1 --2- 172.21.15.107:0/2639329006 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03a8110280 0x7f03a8111040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:41:29.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:29.623+0000 7f03ae400700 1 -- 172.21.15.107:0/2639329006 >> 172.21.15.107:0/2639329006 conn(0x7f03a80f74e0 msgr2=0x7f03a8100290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:41:29.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:29.623+0000 7f03ae400700 1 -- 172.21.15.107:0/2639329006 shutdown_connections 2024-03-04T13:41:29.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:29.623+0000 7f03ae400700 1 -- 172.21.15.107:0/2639329006 wait complete. 2024-03-04T13:41:29.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:29.798+0000 7f7b06a00700 1 -- 172.21.15.107:0/1602613052 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b001000c0 msgr2=0x7f7b00102520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:41:29.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:29.798+0000 7f7b06a00700 1 --2- 172.21.15.107:0/1602613052 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b001000c0 0x7f7b00102520 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f7aec004940 tx=0x7f7aec009e30 comp rx=0 tx=0).stop 2024-03-04T13:41:29.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:29.798+0000 7f7b06a00700 1 -- 172.21.15.107:0/1602613052 shutdown_connections 2024-03-04T13:41:29.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:29.798+0000 7f7b06a00700 1 --2- 172.21.15.107:0/1602613052 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b001000c0 0x7f7b00102520 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:41:29.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:29.798+0000 7f7b06a00700 1 --2- 172.21.15.107:0/1602613052 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b000663e0 0x7f7b000ffaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:41:29.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:29.798+0000 7f7b06a00700 1 -- 172.21.15.107:0/1602613052 >> 172.21.15.107:0/1602613052 conn(0x7f7b000f74e0 msgr2=0x7f7b000f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:41:29.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:29.799+0000 7f7b06a00700 1 -- 172.21.15.107:0/1602613052 shutdown_connections 2024-03-04T13:41:29.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:29.799+0000 7f7b06a00700 1 -- 172.21.15.107:0/1602613052 wait complete. 2024-03-04T13:41:29.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:29.800+0000 7f7b06a00700 1 Processor -- start 2024-03-04T13:41:29.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:29.800+0000 7f7b06a00700 1 -- start start 2024-03-04T13:41:29.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:29.800+0000 7f7b06a00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b000663e0 0x7f7b0006dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:41:29.797 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:29.800+0000 7f7b06a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b001000c0 0x7f7b0006e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:41:29.797 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:29.800+0000 7f7b06a00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0006e8f0 con 0x7f7b001000c0 2024-03-04T13:41:29.797 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:29.801+0000 7f7b06a00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0006f190 con 0x7f7b000663e0 2024-03-04T13:41:29.797 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:29.801+0000 7f7aff400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b001000c0 0x7f7b0006e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:41:29.797 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:29.801+0000 7f7aff400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b001000c0 0x7f7b0006e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57848/0 (socket says 172.21.15.107:57848) 2024-03-04T13:41:29.797 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:29.801+0000 7f7aff400700 1 -- 172.21.15.107:0/4148833252 learned_addr learned my addr 172.21.15.107:0/4148833252 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:41:29.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:29.801+0000 7f7affe00700 1 --2- 172.21.15.107:0/4148833252 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b000663e0 0x7f7b0006dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:41:29.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:29.802+0000 7f7aff400700 1 -- 172.21.15.107:0/4148833252 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b000663e0 msgr2=0x7f7b0006dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:41:29.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:29.802+0000 7f7aff400700 1 --2- 172.21.15.107:0/4148833252 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b000663e0 0x7f7b0006dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:41:29.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:29.802+0000 7f7aff400700 1 -- 172.21.15.107:0/4148833252 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7aec005040 con 0x7f7b001000c0 2024-03-04T13:41:29.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:29.802+0000 7f7aff400700 1 --2- 172.21.15.107:0/4148833252 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b001000c0 0x7f7b0006e2a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f7aec009040 tx=0x7f7aec009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:41:29.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:29.803+0000 7f7af7e00700 1 -- 172.21.15.107:0/4148833252 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7aec021070 con 0x7f7b001000c0 2024-03-04T13:41:29.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:29.803+0000 7f7b06a00700 1 -- 172.21.15.107:0/4148833252 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b0006f410 con 0x7f7b001000c0 2024-03-04T13:41:29.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:29.803+0000 7f7b06a00700 1 -- 172.21.15.107:0/4148833252 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b0006f950 con 0x7f7b001000c0 2024-03-04T13:41:29.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:29.803+0000 7f7af7e00700 1 -- 172.21.15.107:0/4148833252 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7aec009430 con 0x7f7b001000c0 2024-03-04T13:41:29.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:29.803+0000 7f7af7e00700 1 -- 172.21.15.107:0/4148833252 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7aec00e070 con 0x7f7b001000c0 2024-03-04T13:41:29.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:29.805+0000 7f7b06a00700 1 -- 172.21.15.107:0/4148833252 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ae00052e0 con 0x7f7b001000c0 2024-03-04T13:41:29.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:29.805+0000 7f7af7e00700 1 -- 172.21.15.107:0/4148833252 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 23) v1 ==== 45929+0+0 (secure 0 0 0) 0x7f7aec013af0 con 0x7f7b001000c0 2024-03-04T13:41:29.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:29.806+0000 7f7af7e00700 1 --2- 172.21.15.107:0/4148833252 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7f7ae4038a70 0x7f7ae403af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:41:29.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:29.806+0000 7f7af7e00700 1 -- 172.21.15.107:0/4148833252 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f7aec04d7a0 con 0x7f7b001000c0 2024-03-04T13:41:29.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:29.806+0000 7f7affe00700 1 --2- 172.21.15.107:0/4148833252 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7f7ae4038a70 0x7f7ae403af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:41:29.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:29.807+0000 7f7affe00700 1 --2- 172.21.15.107:0/4148833252 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7f7ae4038a70 0x7f7ae403af20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7b0006daf0 tx=0x7f7af000b040 comp rx=0 tx=0).ready entity=mgr.24331 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:41:29.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:29.814+0000 7f7af7e00700 1 -- 172.21.15.107:0/4148833252 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f7aec024be0 con 0x7f7b001000c0 2024-03-04T13:41:30.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:30.077+0000 7f7b06a00700 1 -- 172.21.15.107:0/4148833252 --> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f7ae0000bf0 con 0x7f7ae4038a70 2024-03-04T13:41:30.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:30.104+0000 7f7af7e00700 1 -- 172.21.15.107:0/4148833252 <== mgr.24331 v2:172.21.15.169:6828/3524931243 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3388 (secure 0 0 0) 0x7f7ae0000bf0 con 0x7f7ae4038a70 2024-03-04T13:41:30.100 INFO:teuthology.orchestra.run.smithi107.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-04T13:41:30.100 INFO:teuthology.orchestra.run.smithi107.stdout:alertmanager.smithi107 smithi107 *:9093,9094 running (8m) 2m ago 9m 13.7M - 0.25.0 c8568f914cd2 3eef48872dfe 2024-03-04T13:41:30.100 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi107 smithi107 running (10m) 2m ago 10m 6665k - 17.2.7-545-g42608bb1 9eec3f22859d 03e1570c2eff 2024-03-04T13:41:30.100 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi169 smithi169 running (8m) 15s ago 8m 6656k - 17.2.7-545-g42608bb1 9eec3f22859d 9c8881a5338e 2024-03-04T13:41:30.100 INFO:teuthology.orchestra.run.smithi107.stdout:grafana.smithi107 smithi107 *:3000 running (8m) 2m ago 9m 75.2M - 9.4.7 954c08fa6188 2d8536fd3802 2024-03-04T13:41:30.100 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.riexmm smithi107 running (2m) 2m ago 2m 10.4M - 17.2.7-545-g42608bb1 9eec3f22859d fc921f7d4328 2024-03-04T13:41:30.101 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.thycho smithi107 running (2m) 2m ago 2m 11.6M - 17.2.7-545-g42608bb1 9eec3f22859d 4069c6e0e063 2024-03-04T13:41:30.101 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi169.bxitzk smithi169 running (2m) 15s ago 2m 11.2M - 17.2.7-545-g42608bb1 9eec3f22859d 7d4115228c7c 2024-03-04T13:41:30.101 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi169.jusudl smithi169 running (2m) 15s ago 2m 57.2M - 17.2.7-545-g42608bb1 9eec3f22859d 2d2f2cc452e8 2024-03-04T13:41:30.101 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi107.wpzgwl smithi107 *:9283 running (11m) 2m ago 11m 480M - 17.2.7-545-g42608bb1 9eec3f22859d 4831dce06263 2024-03-04T13:41:30.101 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi169.eihczr smithi169 *:8443,9283 running (20s) 15s ago 8m 55.4M - 19.0.0-1845-g717ce3cc d71b839abdf8 18b4dfa896b5 2024-03-04T13:41:30.101 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi107 smithi107 running (11m) 2m ago 11m 45.4M 2048M 17.2.7-545-g42608bb1 9eec3f22859d 49f5da990d35 2024-03-04T13:41:30.101 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi169 smithi169 running (8m) 15s ago 8m 34.9M 2048M 17.2.7-545-g42608bb1 9eec3f22859d ba4ee916297f 2024-03-04T13:41:30.101 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi107 smithi107 *:9100 running (10m) 2m ago 10m 7750k - 1.5.0 0da6a335fe13 00c53f7970cf 2024-03-04T13:41:30.101 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi169 smithi169 *:9100 running (8m) 15s ago 8m 8024k - 1.5.0 0da6a335fe13 b2dae07f2da2 2024-03-04T13:41:30.101 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0 smithi107 running (7m) 2m ago 7m 45.2M 4288M 17.2.7-545-g42608bb1 9eec3f22859d c0605b68f696 2024-03-04T13:41:30.101 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1 smithi107 running (6m) 2m ago 6m 45.0M 4288M 17.2.7-545-g42608bb1 9eec3f22859d 5fb2c0dd5769 2024-03-04T13:41:30.101 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2 smithi107 running (6m) 2m ago 6m 43.5M 4288M 17.2.7-545-g42608bb1 9eec3f22859d 61f30f643549 2024-03-04T13:41:30.101 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3 smithi169 running (5m) 15s ago 5m 56.5M 5313M 17.2.7-545-g42608bb1 9eec3f22859d cd91d8694764 2024-03-04T13:41:30.101 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4 smithi169 running (5m) 15s ago 5m 53.7M 5313M 17.2.7-545-g42608bb1 9eec3f22859d 08545e3d0659 2024-03-04T13:41:30.102 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5 smithi169 running (4m) 15s ago 4m 52.2M 5313M 17.2.7-545-g42608bb1 9eec3f22859d dcffe6cb3eac 2024-03-04T13:41:30.102 INFO:teuthology.orchestra.run.smithi107.stdout:prometheus.smithi107 smithi107 *:9095 running (7m) 2m ago 9m 46.1M - 2.43.0 a07b618ecd1d 226d4ba6d37a 2024-03-04T13:41:30.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:30.108+0000 7f7b06a00700 1 -- 172.21.15.107:0/4148833252 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7f7ae4038a70 msgr2=0x7f7ae403af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:41:30.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:30.108+0000 7f7b06a00700 1 --2- 172.21.15.107:0/4148833252 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7f7ae4038a70 0x7f7ae403af20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7b0006daf0 tx=0x7f7af000b040 comp rx=0 tx=0).stop 2024-03-04T13:41:30.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:30.108+0000 7f7b06a00700 1 -- 172.21.15.107:0/4148833252 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b001000c0 msgr2=0x7f7b0006e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:41:30.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:30.108+0000 7f7b06a00700 1 --2- 172.21.15.107:0/4148833252 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b001000c0 0x7f7b0006e2a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f7aec009040 tx=0x7f7aec009e30 comp rx=0 tx=0).stop 2024-03-04T13:41:30.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:30.109+0000 7f7b06a00700 1 -- 172.21.15.107:0/4148833252 shutdown_connections 2024-03-04T13:41:30.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:30.109+0000 7f7b06a00700 1 --2- 172.21.15.107:0/4148833252 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7f7ae4038a70 0x7f7ae403af20 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:41:30.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:30.109+0000 7f7b06a00700 1 --2- 172.21.15.107:0/4148833252 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b000663e0 0x7f7b0006dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:41:30.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:30.109+0000 7f7b06a00700 1 --2- 172.21.15.107:0/4148833252 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b001000c0 0x7f7b0006e2a0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:41:30.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:30.109+0000 7f7b06a00700 1 -- 172.21.15.107:0/4148833252 >> 172.21.15.107:0/4148833252 conn(0x7f7b000f74e0 msgr2=0x7f7b00100d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:41:30.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:30.109+0000 7f7b06a00700 1 -- 172.21.15.107:0/4148833252 shutdown_connections 2024-03-04T13:41:30.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:30.109+0000 7f7b06a00700 1 -- 172.21.15.107:0/4148833252 wait complete. 2024-03-04T13:41:30.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:30.281+0000 7fb159800700 1 -- 172.21.15.107:0/3782938406 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb1540708c0 msgr2=0x7fb154070d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:41:30.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:30.281+0000 7fb159800700 1 --2- 172.21.15.107:0/3782938406 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb1540708c0 0x7fb154070d10 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fb140004940 tx=0x7fb140009e30 comp rx=0 tx=0).stop 2024-03-04T13:41:30.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:30.281+0000 7fb159800700 1 -- 172.21.15.107:0/3782938406 shutdown_connections 2024-03-04T13:41:30.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:30.281+0000 7fb159800700 1 --2- 172.21.15.107:0/3782938406 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb1540708c0 0x7fb154070d10 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:41:30.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:30.281+0000 7fb159800700 1 --2- 172.21.15.107:0/3782938406 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb154071f10 0x7fb154070380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:41:30.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:30.281+0000 7fb159800700 1 -- 172.21.15.107:0/3782938406 >> 172.21.15.107:0/3782938406 conn(0x7fb1540f8620 msgr2=0x7fb1540faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:41:30.277 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:29 smithi107 ceph-mon[28739]: from='client.24363 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:41:30.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:30.282+0000 7fb159800700 1 -- 172.21.15.107:0/3782938406 shutdown_connections 2024-03-04T13:41:30.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:30.282+0000 7fb159800700 1 -- 172.21.15.107:0/3782938406 wait complete. 2024-03-04T13:41:30.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:30.283+0000 7fb159800700 1 Processor -- start 2024-03-04T13:41:30.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:30.284+0000 7fb159800700 1 -- start start 2024-03-04T13:41:30.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:30.284+0000 7fb159800700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1540708c0 0x7fb15410fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:41:30.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:30.284+0000 7fb159800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb154071f10 0x7fb154110050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:41:30.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:30.284+0000 7fb159800700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb154110650 con 0x7fb154071f10 2024-03-04T13:41:30.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:30.284+0000 7fb159800700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb154110ef0 con 0x7fb1540708c0 2024-03-04T13:41:30.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:30.284+0000 7fb152a00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1540708c0 0x7fb15410fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:41:30.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:30.284+0000 7fb152000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb154071f10 0x7fb154110050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:41:30.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:30.285+0000 7fb152a00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1540708c0 0x7fb15410fb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.107:43188/0 (socket says 172.21.15.107:43188) 2024-03-04T13:41:30.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:30.285+0000 7fb152a00700 1 -- 172.21.15.107:0/2286539315 learned_addr learned my addr 172.21.15.107:0/2286539315 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:41:30.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:30.285+0000 7fb152a00700 1 -- 172.21.15.107:0/2286539315 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb154071f10 msgr2=0x7fb154110050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:41:30.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:30.285+0000 7fb152a00700 1 --2- 172.21.15.107:0/2286539315 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb154071f10 0x7fb154110050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:41:30.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:30.285+0000 7fb152a00700 1 -- 172.21.15.107:0/2286539315 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb140005040 con 0x7fb1540708c0 2024-03-04T13:41:30.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:30.285+0000 7fb152a00700 1 --2- 172.21.15.107:0/2286539315 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1540708c0 0x7fb15410fb10 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb14400eb40 tx=0x7fb14400b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:41:30.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:30.286+0000 7fb14aa00700 1 -- 172.21.15.107:0/2286539315 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb144007cc0 con 0x7fb1540708c0 2024-03-04T13:41:30.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:30.286+0000 7fb159800700 1 -- 172.21.15.107:0/2286539315 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb154111170 con 0x7fb1540708c0 2024-03-04T13:41:30.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:30.286+0000 7fb14aa00700 1 -- 172.21.15.107:0/2286539315 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb14400f040 con 0x7fb1540708c0 2024-03-04T13:41:30.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:30.286+0000 7fb14aa00700 1 -- 172.21.15.107:0/2286539315 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb144009620 con 0x7fb1540708c0 2024-03-04T13:41:30.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:30.287+0000 7fb159800700 1 -- 172.21.15.107:0/2286539315 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1541116b0 con 0x7fb1540708c0 2024-03-04T13:41:30.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:30.288+0000 7fb14aa00700 1 -- 172.21.15.107:0/2286539315 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 23) v1 ==== 45929+0+0 (secure 0 0 0) 0x7fb144009780 con 0x7fb1540708c0 2024-03-04T13:41:30.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:30.288+0000 7fb159800700 1 -- 172.21.15.107:0/2286539315 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb15404bed0 con 0x7fb1540708c0 2024-03-04T13:41:30.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:30.289+0000 7fb14aa00700 1 --2- 172.21.15.107:0/2286539315 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7fb138038a70 0x7fb13803af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:41:30.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:30.289+0000 7fb14aa00700 1 -- 172.21.15.107:0/2286539315 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5808+0+0 (secure 0 0 0) 0x7fb14404c260 con 0x7fb1540708c0 2024-03-04T13:41:30.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:30.290+0000 7fb152000700 1 --2- 172.21.15.107:0/2286539315 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7fb138038a70 0x7fb13803af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:41:30.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:30.291+0000 7fb152000700 1 --2- 172.21.15.107:0/2286539315 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7fb138038a70 0x7fb13803af20 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb14000b0d0 tx=0x7fb140009e30 comp rx=0 tx=0).ready entity=mgr.24331 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:41:30.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:30.304+0000 7fb14aa00700 1 -- 172.21.15.107:0/2286539315 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7fb144012520 con 0x7fb1540708c0 2024-03-04T13:41:30.355 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:29 smithi169 ceph-mon[36313]: from='client.24363 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:41:30.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:30.764+0000 7fb159800700 1 -- 172.21.15.107:0/2286539315 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb154111d70 con 0x7fb1540708c0 2024-03-04T13:41:30.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:30.765+0000 7fb14aa00700 1 -- 172.21.15.107:0/2286539315 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7fb144012340 con 0x7fb1540708c0 2024-03-04T13:41:30.761 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-03-04T13:41:30.761 INFO:teuthology.orchestra.run.smithi107.stdout: "mon": { 2024-03-04T13:41:30.761 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-545-g42608bb1 (42608bb191117be10dba720e6bf8a02f3171cd34) quincy (stable)": 2 2024-03-04T13:41:30.761 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:41:30.761 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr": { 2024-03-04T13:41:30.761 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-1845-g717ce3cc (717ce3cce8d0166e4e577faa8004e6f9cb4128c0) squid (dev)": 1 2024-03-04T13:41:30.761 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:41:30.761 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": { 2024-03-04T13:41:30.762 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-545-g42608bb1 (42608bb191117be10dba720e6bf8a02f3171cd34) quincy (stable)": 6 2024-03-04T13:41:30.762 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:41:30.762 INFO:teuthology.orchestra.run.smithi107.stdout: "mds": { 2024-03-04T13:41:30.762 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-545-g42608bb1 (42608bb191117be10dba720e6bf8a02f3171cd34) quincy (stable)": 4 2024-03-04T13:41:30.762 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:41:30.762 INFO:teuthology.orchestra.run.smithi107.stdout: "overall": { 2024-03-04T13:41:30.762 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-545-g42608bb1 (42608bb191117be10dba720e6bf8a02f3171cd34) quincy (stable)": 12, 2024-03-04T13:41:30.762 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-1845-g717ce3cc (717ce3cce8d0166e4e577faa8004e6f9cb4128c0) squid (dev)": 1 2024-03-04T13:41:30.762 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-04T13:41:30.762 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-03-04T13:41:30.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:30.769+0000 7fb159800700 1 -- 172.21.15.107:0/2286539315 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7fb138038a70 msgr2=0x7fb13803af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:41:30.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:30.769+0000 7fb159800700 1 --2- 172.21.15.107:0/2286539315 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7fb138038a70 0x7fb13803af20 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb14000b0d0 tx=0x7fb140009e30 comp rx=0 tx=0).stop 2024-03-04T13:41:30.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:30.769+0000 7fb159800700 1 -- 172.21.15.107:0/2286539315 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1540708c0 msgr2=0x7fb15410fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:41:30.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:30.769+0000 7fb159800700 1 --2- 172.21.15.107:0/2286539315 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1540708c0 0x7fb15410fb10 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb14400eb40 tx=0x7fb14400b040 comp rx=0 tx=0).stop 2024-03-04T13:41:30.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:30.769+0000 7fb159800700 1 -- 172.21.15.107:0/2286539315 shutdown_connections 2024-03-04T13:41:30.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:30.769+0000 7fb159800700 1 --2- 172.21.15.107:0/2286539315 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7fb138038a70 0x7fb13803af20 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:41:30.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:30.769+0000 7fb159800700 1 --2- 172.21.15.107:0/2286539315 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1540708c0 0x7fb15410fb10 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:41:30.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:30.769+0000 7fb159800700 1 --2- 172.21.15.107:0/2286539315 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb154071f10 0x7fb154110050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:41:30.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:30.769+0000 7fb159800700 1 -- 172.21.15.107:0/2286539315 >> 172.21.15.107:0/2286539315 conn(0x7fb1540f8620 msgr2=0x7fb154102dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:41:30.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:30.769+0000 7fb159800700 1 -- 172.21.15.107:0/2286539315 shutdown_connections 2024-03-04T13:41:30.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:30.769+0000 7fb159800700 1 -- 172.21.15.107:0/2286539315 wait complete. 2024-03-04T13:41:30.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:30.925+0000 7fdd5e000700 1 -- 172.21.15.107:0/2371999940 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd581003a0 msgr2=0x7fdd58102780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:41:30.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:30.925+0000 7fdd5e000700 1 --2- 172.21.15.107:0/2371999940 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd581003a0 0x7fdd58102780 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fdd44004940 tx=0x7fdd44009e30 comp rx=0 tx=0).stop 2024-03-04T13:41:30.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:30.927+0000 7fdd5e000700 1 -- 172.21.15.107:0/2371999940 shutdown_connections 2024-03-04T13:41:30.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:30.927+0000 7fdd5e000700 1 --2- 172.21.15.107:0/2371999940 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd581003a0 0x7fdd58102780 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:41:30.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:30.927+0000 7fdd5e000700 1 --2- 172.21.15.107:0/2371999940 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd580fda80 0x7fdd580ffe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:41:30.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:30.927+0000 7fdd5e000700 1 -- 172.21.15.107:0/2371999940 >> 172.21.15.107:0/2371999940 conn(0x7fdd580f7520 msgr2=0x7fdd580f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:41:30.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:30.927+0000 7fdd5e000700 1 -- 172.21.15.107:0/2371999940 shutdown_connections 2024-03-04T13:41:30.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:30.927+0000 7fdd5e000700 1 -- 172.21.15.107:0/2371999940 wait complete. 2024-03-04T13:41:30.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:30.928+0000 7fdd5e000700 1 Processor -- start 2024-03-04T13:41:30.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:30.929+0000 7fdd5e000700 1 -- start start 2024-03-04T13:41:30.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:30.929+0000 7fdd5e000700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd580fda80 0x7fdd580fd510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:41:30.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:30.929+0000 7fdd5e000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd581003a0 0x7fdd580fb990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:41:30.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:30.929+0000 7fdd5e000700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd580fbf90 con 0x7fdd581003a0 2024-03-04T13:41:30.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:30.929+0000 7fdd5e000700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd580fc100 con 0x7fdd580fda80 2024-03-04T13:41:30.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:30.929+0000 7fdd56a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd581003a0 0x7fdd580fb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:41:30.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:30.929+0000 7fdd56a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd581003a0 0x7fdd580fb990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57886/0 (socket says 172.21.15.107:57886) 2024-03-04T13:41:30.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:30.929+0000 7fdd56a00700 1 -- 172.21.15.107:0/3341090337 learned_addr learned my addr 172.21.15.107:0/3341090337 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:41:30.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:30.929+0000 7fdd57400700 1 --2- 172.21.15.107:0/3341090337 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd580fda80 0x7fdd580fd510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:41:30.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:30.930+0000 7fdd56a00700 1 -- 172.21.15.107:0/3341090337 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd580fda80 msgr2=0x7fdd580fd510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:41:30.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:30.930+0000 7fdd56a00700 1 --2- 172.21.15.107:0/3341090337 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd580fda80 0x7fdd580fd510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:41:30.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:30.930+0000 7fdd56a00700 1 -- 172.21.15.107:0/3341090337 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd44005040 con 0x7fdd581003a0 2024-03-04T13:41:30.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:30.930+0000 7fdd56a00700 1 --2- 172.21.15.107:0/3341090337 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd581003a0 0x7fdd580fb990 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fdd44009fd0 tx=0x7fdd44009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:41:30.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:30.931+0000 7fdd4f400700 1 -- 172.21.15.107:0/3341090337 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd44021070 con 0x7fdd581003a0 2024-03-04T13:41:30.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:30.931+0000 7fdd4f400700 1 -- 172.21.15.107:0/3341090337 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdd44013de0 con 0x7fdd581003a0 2024-03-04T13:41:30.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:30.931+0000 7fdd5e000700 1 -- 172.21.15.107:0/3341090337 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd580fc380 con 0x7fdd581003a0 2024-03-04T13:41:30.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:30.931+0000 7fdd5e000700 1 -- 172.21.15.107:0/3341090337 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd58115910 con 0x7fdd581003a0 2024-03-04T13:41:30.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:30.931+0000 7fdd4f400700 1 -- 172.21.15.107:0/3341090337 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd4400e070 con 0x7fdd581003a0 2024-03-04T13:41:30.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:30.932+0000 7fdd5e000700 1 -- 172.21.15.107:0/3341090337 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd58107a50 con 0x7fdd581003a0 2024-03-04T13:41:30.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:30.933+0000 7fdd4f400700 1 -- 172.21.15.107:0/3341090337 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 23) v1 ==== 45929+0+0 (secure 0 0 0) 0x7fdd4401c690 con 0x7fdd581003a0 2024-03-04T13:41:30.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:30.933+0000 7fdd4f400700 1 --2- 172.21.15.107:0/3341090337 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7fdd3c038a70 0x7fdd3c03af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:41:30.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:30.933+0000 7fdd4f400700 1 -- 172.21.15.107:0/3341090337 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5808+0+0 (secure 0 0 0) 0x7fdd4404d850 con 0x7fdd581003a0 2024-03-04T13:41:30.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:30.934+0000 7fdd57400700 1 --2- 172.21.15.107:0/3341090337 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7fdd3c038a70 0x7fdd3c03af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:41:30.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:30.935+0000 7fdd57400700 1 --2- 172.21.15.107:0/3341090337 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7fdd3c038a70 0x7fdd3c03af20 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fdd580fd2a0 tx=0x7fdd4800b040 comp rx=0 tx=0).ready entity=mgr.24331 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:41:30.938 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:30.943+0000 7fdd4f400700 1 -- 172.21.15.107:0/3341090337 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7fdd44024be0 con 0x7fdd581003a0 2024-03-04T13:41:31.058 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:30 smithi107 ceph-mon[28739]: from='client.24367 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:41:31.058 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:30 smithi107 ceph-mon[28739]: pgmap v7: 65 pgs: 65 active+clean; 75 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 617 KiB/s rd, 1.8 MiB/s wr, 94 op/s 2024-03-04T13:41:31.058 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:30 smithi107 ceph-mon[28739]: from='client.14624 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:41:31.058 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:30 smithi107 ceph-mon[28739]: from='client.? 172.21.15.107:0/2286539315' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:41:31.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:31.274+0000 7fdd5e000700 1 -- 172.21.15.107:0/3341090337 --> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdd5802b3b0 con 0x7fdd3c038a70 2024-03-04T13:41:31.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:31.276+0000 7fdd4f400700 1 -- 172.21.15.107:0/3341090337 <== mgr.24331 v2:172.21.15.169:6828/3524931243 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7fdd5802b3b0 con 0x7fdd3c038a70 2024-03-04T13:41:31.271 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-03-04T13:41:31.271 INFO:teuthology.orchestra.run.smithi107.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377", 2024-03-04T13:41:31.271 INFO:teuthology.orchestra.run.smithi107.stdout: "in_progress": true, 2024-03-04T13:41:31.271 INFO:teuthology.orchestra.run.smithi107.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-03-04T13:41:31.271 INFO:teuthology.orchestra.run.smithi107.stdout: "services_complete": [], 2024-03-04T13:41:31.272 INFO:teuthology.orchestra.run.smithi107.stdout: "progress": "1/2 daemons upgraded", 2024-03-04T13:41:31.272 INFO:teuthology.orchestra.run.smithi107.stdout: "message": "", 2024-03-04T13:41:31.272 INFO:teuthology.orchestra.run.smithi107.stdout: "is_paused": false 2024-03-04T13:41:31.272 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-03-04T13:41:31.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:31.278+0000 7fdd5e000700 1 -- 172.21.15.107:0/3341090337 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7fdd3c038a70 msgr2=0x7fdd3c03af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:41:31.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:31.278+0000 7fdd5e000700 1 --2- 172.21.15.107:0/3341090337 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7fdd3c038a70 0x7fdd3c03af20 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fdd580fd2a0 tx=0x7fdd4800b040 comp rx=0 tx=0).stop 2024-03-04T13:41:31.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:31.278+0000 7fdd5e000700 1 -- 172.21.15.107:0/3341090337 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd581003a0 msgr2=0x7fdd580fb990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:41:31.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:31.278+0000 7fdd5e000700 1 --2- 172.21.15.107:0/3341090337 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd581003a0 0x7fdd580fb990 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fdd44009fd0 tx=0x7fdd44009e30 comp rx=0 tx=0).stop 2024-03-04T13:41:31.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:31.279+0000 7fdd5e000700 1 -- 172.21.15.107:0/3341090337 shutdown_connections 2024-03-04T13:41:31.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:31.279+0000 7fdd5e000700 1 --2- 172.21.15.107:0/3341090337 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7fdd3c038a70 0x7fdd3c03af20 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:41:31.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:31.279+0000 7fdd5e000700 1 --2- 172.21.15.107:0/3341090337 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdd580fda80 0x7fdd580fd510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:41:31.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:31.279+0000 7fdd5e000700 1 --2- 172.21.15.107:0/3341090337 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd581003a0 0x7fdd580fb990 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:41:31.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:31.279+0000 7fdd5e000700 1 -- 172.21.15.107:0/3341090337 >> 172.21.15.107:0/3341090337 conn(0x7fdd580f7520 msgr2=0x7fdd580f9900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:41:31.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:31.279+0000 7fdd5e000700 1 -- 172.21.15.107:0/3341090337 shutdown_connections 2024-03-04T13:41:31.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:41:31.279+0000 7fdd5e000700 1 -- 172.21.15.107:0/3341090337 wait complete. 2024-03-04T13:41:31.356 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:30 smithi169 ceph-mon[36313]: from='client.24367 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:41:31.356 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:30 smithi169 ceph-mon[36313]: pgmap v7: 65 pgs: 65 active+clean; 75 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 617 KiB/s rd, 1.8 MiB/s wr, 94 op/s 2024-03-04T13:41:31.356 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:30 smithi169 ceph-mon[36313]: from='client.14624 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:41:31.356 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:30 smithi169 ceph-mon[36313]: from='client.? 172.21.15.107:0/2286539315' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:41:33.252 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:32 smithi107 ceph-mon[28739]: from='client.14632 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:41:33.252 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:32 smithi107 ceph-mon[28739]: pgmap v8: 65 pgs: 65 active+clean; 75 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 480 KiB/s rd, 1.4 MiB/s wr, 73 op/s 2024-03-04T13:41:33.252 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:32 smithi107 ceph-mon[28739]: Standby manager daemon smithi107.wpzgwl started 2024-03-04T13:41:33.252 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:32 smithi107 ceph-mon[28739]: from='mgr.? 172.21.15.107:0/4215519068' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi107.wpzgwl/crt"}]: dispatch 2024-03-04T13:41:33.252 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:32 smithi107 ceph-mon[28739]: from='mgr.? 172.21.15.107:0/4215519068' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-04T13:41:33.253 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:32 smithi107 ceph-mon[28739]: from='mgr.? 172.21.15.107:0/4215519068' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi107.wpzgwl/key"}]: dispatch 2024-03-04T13:41:33.253 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:32 smithi107 ceph-mon[28739]: from='mgr.? 172.21.15.107:0/4215519068' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-04T13:41:33.356 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:32 smithi169 ceph-mon[36313]: from='client.14632 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:41:33.356 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:32 smithi169 ceph-mon[36313]: pgmap v8: 65 pgs: 65 active+clean; 75 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 480 KiB/s rd, 1.4 MiB/s wr, 73 op/s 2024-03-04T13:41:33.356 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:32 smithi169 ceph-mon[36313]: Standby manager daemon smithi107.wpzgwl started 2024-03-04T13:41:33.356 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:32 smithi169 ceph-mon[36313]: from='mgr.? 172.21.15.107:0/4215519068' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi107.wpzgwl/crt"}]: dispatch 2024-03-04T13:41:33.356 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:32 smithi169 ceph-mon[36313]: from='mgr.? 172.21.15.107:0/4215519068' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-04T13:41:33.356 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:32 smithi169 ceph-mon[36313]: from='mgr.? 172.21.15.107:0/4215519068' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi107.wpzgwl/key"}]: dispatch 2024-03-04T13:41:33.356 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:32 smithi169 ceph-mon[36313]: from='mgr.? 172.21.15.107:0/4215519068' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-04T13:41:34.339 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:33 smithi107 ceph-mon[28739]: mgrmap e24: smithi169.eihczr(active, since 11s), standbys: smithi107.wpzgwl 2024-03-04T13:41:34.339 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:33 smithi107 ceph-mon[28739]: from='mgr.24331 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "mgr metadata", "who": "smithi107.wpzgwl", "id": "smithi107.wpzgwl"}]: dispatch 2024-03-04T13:41:34.355 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:33 smithi169 ceph-mon[36313]: mgrmap e24: smithi169.eihczr(active, since 11s), standbys: smithi107.wpzgwl 2024-03-04T13:41:34.356 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:33 smithi169 ceph-mon[36313]: from='mgr.24331 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "mgr metadata", "who": "smithi107.wpzgwl", "id": "smithi107.wpzgwl"}]: dispatch 2024-03-04T13:41:35.246 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:34 smithi169 ceph-mon[36313]: pgmap v9: 65 pgs: 65 active+clean; 91 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 128 op/s 2024-03-04T13:41:35.339 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:34 smithi107 ceph-mon[28739]: pgmap v9: 65 pgs: 65 active+clean; 91 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 128 op/s 2024-03-04T13:41:37.356 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:37 smithi169 ceph-mon[36313]: pgmap v10: 65 pgs: 65 active+clean; 91 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.0 MiB/s rd, 2.4 MiB/s wr, 117 op/s 2024-03-04T13:41:37.356 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:37 smithi169 ceph-mon[36313]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' 2024-03-04T13:41:37.356 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:37 smithi169 ceph-mon[36313]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' 2024-03-04T13:41:37.356 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:37 smithi169 ceph-mon[36313]: from='mgr.24331 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:41:37.589 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:37 smithi107 ceph-mon[28739]: pgmap v10: 65 pgs: 65 active+clean; 91 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.0 MiB/s rd, 2.4 MiB/s wr, 117 op/s 2024-03-04T13:41:37.589 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:37 smithi107 ceph-mon[28739]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' 2024-03-04T13:41:37.589 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:37 smithi107 ceph-mon[28739]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' 2024-03-04T13:41:37.589 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:37 smithi107 ceph-mon[28739]: from='mgr.24331 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:41:38.589 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:38 smithi107 ceph-mon[28739]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' 2024-03-04T13:41:38.589 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:38 smithi107 ceph-mon[28739]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' 2024-03-04T13:41:38.606 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:38 smithi169 ceph-mon[36313]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' 2024-03-04T13:41:38.606 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:38 smithi169 ceph-mon[36313]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' 2024-03-04T13:41:39.589 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:39 smithi107 ceph-mon[28739]: pgmap v11: 65 pgs: 65 active+clean; 97 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.4 MiB/s rd, 2.9 MiB/s wr, 145 op/s 2024-03-04T13:41:39.589 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:39 smithi107 ceph-mon[28739]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' 2024-03-04T13:41:39.589 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:39 smithi107 ceph-mon[28739]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' 2024-03-04T13:41:39.590 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:39 smithi107 ceph-mon[28739]: from='mgr.24331 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-04T13:41:39.590 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:39 smithi107 ceph-mon[28739]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-04T13:41:39.590 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:39 smithi107 ceph-mon[28739]: from='mgr.24331 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-04T13:41:39.590 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:39 smithi107 ceph-mon[28739]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-04T13:41:39.590 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:39 smithi107 ceph-mon[28739]: from='mgr.24331 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-04T13:41:39.590 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:39 smithi107 ceph-mon[28739]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-04T13:41:39.590 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:39 smithi107 ceph-mon[28739]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' 2024-03-04T13:41:39.606 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:39 smithi169 ceph-mon[36313]: pgmap v11: 65 pgs: 65 active+clean; 97 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.4 MiB/s rd, 2.9 MiB/s wr, 145 op/s 2024-03-04T13:41:39.606 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:39 smithi169 ceph-mon[36313]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' 2024-03-04T13:41:39.606 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:39 smithi169 ceph-mon[36313]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' 2024-03-04T13:41:39.606 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:39 smithi169 ceph-mon[36313]: from='mgr.24331 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-04T13:41:39.606 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:39 smithi169 ceph-mon[36313]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-04T13:41:39.606 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:39 smithi169 ceph-mon[36313]: from='mgr.24331 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-04T13:41:39.606 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:39 smithi169 ceph-mon[36313]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-04T13:41:39.606 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:39 smithi169 ceph-mon[36313]: from='mgr.24331 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-04T13:41:39.606 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:39 smithi169 ceph-mon[36313]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-04T13:41:39.606 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:39 smithi169 ceph-mon[36313]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' 2024-03-04T13:41:40.507 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:40 smithi169 ceph-mon[36313]: Adjusting osd_memory_target on smithi107 to 1558M 2024-03-04T13:41:40.507 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:40 smithi169 ceph-mon[36313]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' 2024-03-04T13:41:40.507 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:40 smithi169 ceph-mon[36313]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' 2024-03-04T13:41:40.508 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:40 smithi169 ceph-mon[36313]: pgmap v12: 65 pgs: 65 active+clean; 98 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.4 MiB/s rd, 2.5 MiB/s wr, 130 op/s 2024-03-04T13:41:40.589 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:40 smithi107 ceph-mon[28739]: Adjusting osd_memory_target on smithi107 to 1558M 2024-03-04T13:41:40.589 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:40 smithi107 ceph-mon[28739]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' 2024-03-04T13:41:40.589 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:40 smithi107 ceph-mon[28739]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' 2024-03-04T13:41:40.589 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:40 smithi107 ceph-mon[28739]: pgmap v12: 65 pgs: 65 active+clean; 98 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.4 MiB/s rd, 2.5 MiB/s wr, 130 op/s 2024-03-04T13:41:43.339 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:42 smithi107 ceph-mon[28739]: pgmap v13: 65 pgs: 65 active+clean; 98 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.0 MiB/s rd, 2.0 MiB/s wr, 100 op/s 2024-03-04T13:41:43.356 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:42 smithi169 ceph-mon[36313]: pgmap v13: 65 pgs: 65 active+clean; 98 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.0 MiB/s rd, 2.0 MiB/s wr, 100 op/s 2024-03-04T13:41:45.106 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:45 smithi169 ceph-mon[36313]: pgmap v14: 65 pgs: 65 active+clean; 107 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 2.8 MiB/s wr, 138 op/s 2024-03-04T13:41:45.339 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:45 smithi107 ceph-mon[28739]: pgmap v14: 65 pgs: 65 active+clean; 107 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 2.8 MiB/s wr, 138 op/s 2024-03-04T13:41:46.589 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:46 smithi107 ceph-mon[28739]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' 2024-03-04T13:41:46.589 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:46 smithi107 ceph-mon[28739]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' 2024-03-04T13:41:46.589 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:46 smithi107 ceph-mon[28739]: from='mgr.24331 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-04T13:41:46.589 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:46 smithi107 ceph-mon[28739]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-04T13:41:46.590 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:46 smithi107 ceph-mon[28739]: from='mgr.24331 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-04T13:41:46.590 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:46 smithi107 ceph-mon[28739]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-04T13:41:46.590 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:46 smithi107 ceph-mon[28739]: from='mgr.24331 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-04T13:41:46.590 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:46 smithi107 ceph-mon[28739]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-04T13:41:46.590 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:46 smithi107 ceph-mon[28739]: Adjusting osd_memory_target on smithi169 to 2582M 2024-03-04T13:41:46.590 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:46 smithi107 ceph-mon[28739]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' 2024-03-04T13:41:46.590 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:46 smithi107 ceph-mon[28739]: from='mgr.24331 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:41:46.590 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:46 smithi107 ceph-mon[28739]: from='mgr.24331 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:41:46.590 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:46 smithi107 ceph-mon[28739]: Updating smithi107:/etc/ceph/ceph.conf 2024-03-04T13:41:46.590 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:46 smithi107 ceph-mon[28739]: Updating smithi169:/etc/ceph/ceph.conf 2024-03-04T13:41:46.590 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:46 smithi107 ceph-mon[28739]: pgmap v15: 65 pgs: 65 active+clean; 107 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 75 op/s 2024-03-04T13:41:46.590 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:46 smithi107 ceph-mon[28739]: Updating smithi169:/var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-04T13:41:46.590 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:46 smithi107 ceph-mon[28739]: Updating smithi107:/var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-04T13:41:46.606 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:46 smithi169 ceph-mon[36313]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' 2024-03-04T13:41:46.607 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:46 smithi169 ceph-mon[36313]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' 2024-03-04T13:41:46.607 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:46 smithi169 ceph-mon[36313]: from='mgr.24331 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-04T13:41:46.607 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:46 smithi169 ceph-mon[36313]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-04T13:41:46.607 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:46 smithi169 ceph-mon[36313]: from='mgr.24331 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-04T13:41:46.607 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:46 smithi169 ceph-mon[36313]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-04T13:41:46.607 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:46 smithi169 ceph-mon[36313]: from='mgr.24331 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-04T13:41:46.607 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:46 smithi169 ceph-mon[36313]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-04T13:41:46.607 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:46 smithi169 ceph-mon[36313]: Adjusting osd_memory_target on smithi169 to 2582M 2024-03-04T13:41:46.607 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:46 smithi169 ceph-mon[36313]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' 2024-03-04T13:41:46.607 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:46 smithi169 ceph-mon[36313]: from='mgr.24331 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:41:46.607 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:46 smithi169 ceph-mon[36313]: from='mgr.24331 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:41:46.607 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:46 smithi169 ceph-mon[36313]: Updating smithi107:/etc/ceph/ceph.conf 2024-03-04T13:41:46.607 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:46 smithi169 ceph-mon[36313]: Updating smithi169:/etc/ceph/ceph.conf 2024-03-04T13:41:46.608 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:46 smithi169 ceph-mon[36313]: pgmap v15: 65 pgs: 65 active+clean; 107 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 75 op/s 2024-03-04T13:41:46.608 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:46 smithi169 ceph-mon[36313]: Updating smithi169:/var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-04T13:41:46.608 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:46 smithi169 ceph-mon[36313]: Updating smithi107:/var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-04T13:41:48.089 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:47 smithi107 ceph-mon[28739]: Updating smithi169:/etc/ceph/ceph.client.admin.keyring 2024-03-04T13:41:48.089 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:47 smithi107 ceph-mon[28739]: Updating smithi107:/etc/ceph/ceph.client.admin.keyring 2024-03-04T13:41:48.106 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:47 smithi169 ceph-mon[36313]: Updating smithi169:/etc/ceph/ceph.client.admin.keyring 2024-03-04T13:41:48.107 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:47 smithi169 ceph-mon[36313]: Updating smithi107:/etc/ceph/ceph.client.admin.keyring 2024-03-04T13:41:49.339 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:49 smithi107 ceph-mon[28739]: Updating smithi169:/var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/config/ceph.client.admin.keyring 2024-03-04T13:41:49.339 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:49 smithi107 ceph-mon[28739]: Updating smithi107:/var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/config/ceph.client.admin.keyring 2024-03-04T13:41:49.339 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:49 smithi107 ceph-mon[28739]: pgmap v16: 65 pgs: 65 active+clean; 112 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 102 op/s 2024-03-04T13:41:49.339 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:49 smithi107 ceph-mon[28739]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' 2024-03-04T13:41:49.339 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:49 smithi107 ceph-mon[28739]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' 2024-03-04T13:41:49.339 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:49 smithi107 ceph-mon[28739]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' 2024-03-04T13:41:49.340 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:49 smithi107 ceph-mon[28739]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' 2024-03-04T13:41:49.340 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:49 smithi107 ceph-mon[28739]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' 2024-03-04T13:41:49.340 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:49 smithi107 ceph-mon[28739]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' 2024-03-04T13:41:49.340 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:49 smithi107 ceph-mon[28739]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' 2024-03-04T13:41:49.340 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:49 smithi107 ceph-mon[28739]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' 2024-03-04T13:41:49.340 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:49 smithi107 ceph-mon[28739]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' 2024-03-04T13:41:49.356 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:49 smithi169 ceph-mon[36313]: Updating smithi169:/var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/config/ceph.client.admin.keyring 2024-03-04T13:41:49.356 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:49 smithi169 ceph-mon[36313]: Updating smithi107:/var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/config/ceph.client.admin.keyring 2024-03-04T13:41:49.356 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:49 smithi169 ceph-mon[36313]: pgmap v16: 65 pgs: 65 active+clean; 112 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 102 op/s 2024-03-04T13:41:49.356 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:49 smithi169 ceph-mon[36313]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' 2024-03-04T13:41:49.356 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:49 smithi169 ceph-mon[36313]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' 2024-03-04T13:41:49.356 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:49 smithi169 ceph-mon[36313]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' 2024-03-04T13:41:49.357 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:49 smithi169 ceph-mon[36313]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' 2024-03-04T13:41:49.357 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:49 smithi169 ceph-mon[36313]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' 2024-03-04T13:41:49.357 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:49 smithi169 ceph-mon[36313]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' 2024-03-04T13:41:49.357 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:49 smithi169 ceph-mon[36313]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' 2024-03-04T13:41:49.357 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:49 smithi169 ceph-mon[36313]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' 2024-03-04T13:41:49.357 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:49 smithi169 ceph-mon[36313]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' 2024-03-04T13:41:50.237 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:50 smithi107 ceph-mon[28739]: Reconfiguring node-exporter.smithi107 (dependencies changed)... 2024-03-04T13:41:50.237 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:50 smithi107 ceph-mon[28739]: Reconfiguring daemon node-exporter.smithi107 on smithi107 2024-03-04T13:41:50.356 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:50 smithi169 ceph-mon[36313]: Reconfiguring node-exporter.smithi107 (dependencies changed)... 2024-03-04T13:41:50.356 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:50 smithi169 ceph-mon[36313]: Reconfiguring daemon node-exporter.smithi107 on smithi107 2024-03-04T13:41:51.339 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:51 smithi107 ceph-mon[28739]: pgmap v17: 65 pgs: 65 active+clean; 116 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 85 op/s 2024-03-04T13:41:51.356 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:51 smithi169 ceph-mon[36313]: pgmap v17: 65 pgs: 65 active+clean; 116 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 85 op/s 2024-03-04T13:41:53.178 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:52 smithi107 ceph-mon[28739]: pgmap v18: 65 pgs: 65 active+clean; 116 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 76 op/s 2024-03-04T13:41:53.178 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:52 smithi107 ceph-mon[28739]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' 2024-03-04T13:41:53.178 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:52 smithi107 ceph-mon[28739]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' 2024-03-04T13:41:53.178 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:52 smithi107 ceph-mon[28739]: Reconfiguring alertmanager.smithi107 (dependencies changed)... 2024-03-04T13:41:53.178 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:52 smithi107 ceph-mon[28739]: from='mgr.24331 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:41:53.179 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:52 smithi107 ceph-mon[28739]: Reconfiguring daemon alertmanager.smithi107 on smithi107 2024-03-04T13:41:53.356 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:52 smithi169 ceph-mon[36313]: pgmap v18: 65 pgs: 65 active+clean; 116 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 76 op/s 2024-03-04T13:41:53.356 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:52 smithi169 ceph-mon[36313]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' 2024-03-04T13:41:53.356 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:52 smithi169 ceph-mon[36313]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' 2024-03-04T13:41:53.356 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:52 smithi169 ceph-mon[36313]: Reconfiguring alertmanager.smithi107 (dependencies changed)... 2024-03-04T13:41:53.356 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:52 smithi169 ceph-mon[36313]: from='mgr.24331 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:41:53.357 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:52 smithi169 ceph-mon[36313]: Reconfiguring daemon alertmanager.smithi107 on smithi107 2024-03-04T13:41:55.309 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:54 smithi107 ceph-mon[28739]: pgmap v19: 65 pgs: 65 active+clean; 122 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 117 op/s 2024-03-04T13:41:55.359 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:54 smithi169 ceph-mon[36313]: pgmap v19: 65 pgs: 65 active+clean; 122 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 117 op/s 2024-03-04T13:41:57.089 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:56 smithi107 ceph-mon[28739]: pgmap v20: 65 pgs: 65 active+clean; 122 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 79 op/s 2024-03-04T13:41:57.106 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:56 smithi169 ceph-mon[36313]: pgmap v20: 65 pgs: 65 active+clean; 122 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 79 op/s 2024-03-04T13:41:59.260 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:41:58 smithi107 ceph-mon[28739]: pgmap v21: 65 pgs: 65 active+clean; 133 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 108 op/s 2024-03-04T13:41:59.356 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:41:58 smithi169 ceph-mon[36313]: pgmap v21: 65 pgs: 65 active+clean; 133 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 108 op/s 2024-03-04T13:42:01.090 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:42:00 smithi107 ceph-mon[28739]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' 2024-03-04T13:42:01.090 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:42:00 smithi107 ceph-mon[28739]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' 2024-03-04T13:42:01.090 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:42:00 smithi107 ceph-mon[28739]: Reconfiguring grafana.smithi107 (dependencies changed)... 2024-03-04T13:42:01.090 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:42:00 smithi107 ceph-mon[28739]: Reconfiguring daemon grafana.smithi107 on smithi107 2024-03-04T13:42:01.090 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:42:00 smithi107 ceph-mon[28739]: pgmap v22: 65 pgs: 65 active+clean; 134 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 90 op/s 2024-03-04T13:42:01.106 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:42:00 smithi169 ceph-mon[36313]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' 2024-03-04T13:42:01.106 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:42:00 smithi169 ceph-mon[36313]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' 2024-03-04T13:42:01.107 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:42:00 smithi169 ceph-mon[36313]: Reconfiguring grafana.smithi107 (dependencies changed)... 2024-03-04T13:42:01.107 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:42:00 smithi169 ceph-mon[36313]: Reconfiguring daemon grafana.smithi107 on smithi107 2024-03-04T13:42:01.107 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:42:00 smithi169 ceph-mon[36313]: pgmap v22: 65 pgs: 65 active+clean; 134 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 90 op/s 2024-03-04T13:42:01.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:01.452+0000 7f75a3e00700 1 -- 172.21.15.107:0/3912639714 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75a40fd880 msgr2=0x7f75a40ffc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:42:01.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:01.452+0000 7f75a3e00700 1 --2- 172.21.15.107:0/3912639714 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75a40fd880 0x7f75a40ffc60 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f7594004940 tx=0x7f7594009e30 comp rx=0 tx=0).stop 2024-03-04T13:42:01.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:01.452+0000 7f75a3e00700 1 -- 172.21.15.107:0/3912639714 shutdown_connections 2024-03-04T13:42:01.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:01.452+0000 7f75a3e00700 1 --2- 172.21.15.107:0/3912639714 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75a41001a0 0x7f75a4102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:42:01.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:01.452+0000 7f75a3e00700 1 --2- 172.21.15.107:0/3912639714 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75a40fd880 0x7f75a40ffc60 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:42:01.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:01.452+0000 7f75a3e00700 1 -- 172.21.15.107:0/3912639714 >> 172.21.15.107:0/3912639714 conn(0x7f75a40f72c0 msgr2=0x7f75a40f9730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:42:01.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:01.452+0000 7f75a3e00700 1 -- 172.21.15.107:0/3912639714 shutdown_connections 2024-03-04T13:42:01.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:01.452+0000 7f75a3e00700 1 -- 172.21.15.107:0/3912639714 wait complete. 2024-03-04T13:42:01.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:01.453+0000 7f75a3e00700 1 Processor -- start 2024-03-04T13:42:01.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:01.454+0000 7f75a3e00700 1 -- start start 2024-03-04T13:42:01.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:01.454+0000 7f75a3e00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75a40fd880 0x7f75a410f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:42:01.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:01.454+0000 7f75a3e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75a41001a0 0x7f75a410fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:42:01.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:01.454+0000 7f75a3e00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75a41104c0 con 0x7f75a41001a0 2024-03-04T13:42:01.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:01.454+0000 7f75a3e00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75a4110d00 con 0x7f75a40fd880 2024-03-04T13:42:01.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:01.454+0000 7f75a2000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75a41001a0 0x7f75a410fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:42:01.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:01.454+0000 7f75a2000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75a41001a0 0x7f75a410fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38768/0 (socket says 172.21.15.107:38768) 2024-03-04T13:42:01.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:01.455+0000 7f75a2000700 1 -- 172.21.15.107:0/850345172 learned_addr learned my addr 172.21.15.107:0/850345172 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:42:01.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:01.455+0000 7f75a2a00700 1 --2- 172.21.15.107:0/850345172 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75a40fd880 0x7f75a410f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:42:01.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:01.455+0000 7f75a2000700 1 -- 172.21.15.107:0/850345172 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75a40fd880 msgr2=0x7f75a410f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:42:01.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:01.455+0000 7f75a2000700 1 --2- 172.21.15.107:0/850345172 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75a40fd880 0x7f75a410f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:42:01.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:01.455+0000 7f75a2000700 1 -- 172.21.15.107:0/850345172 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7594005040 con 0x7f75a41001a0 2024-03-04T13:42:01.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:01.455+0000 7f75a2000700 1 --2- 172.21.15.107:0/850345172 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75a41001a0 0x7f75a410fe30 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f759000b790 tx=0x7f7590009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:42:01.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:01.456+0000 7f759aa00700 1 -- 172.21.15.107:0/850345172 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f759000f770 con 0x7f75a41001a0 2024-03-04T13:42:01.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:01.456+0000 7f759aa00700 1 -- 172.21.15.107:0/850345172 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f759000fda0 con 0x7f75a41001a0 2024-03-04T13:42:01.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:01.456+0000 7f75a3e00700 1 -- 172.21.15.107:0/850345172 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75a4110fe0 con 0x7f75a41001a0 2024-03-04T13:42:01.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:01.456+0000 7f759aa00700 1 -- 172.21.15.107:0/850345172 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f759000d310 con 0x7f75a41001a0 2024-03-04T13:42:01.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:01.456+0000 7f75a3e00700 1 -- 172.21.15.107:0/850345172 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75a41115a0 con 0x7f75a41001a0 2024-03-04T13:42:01.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:01.457+0000 7f75a3e00700 1 -- 172.21.15.107:0/850345172 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75a40f8e80 con 0x7f75a41001a0 2024-03-04T13:42:01.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:01.458+0000 7f759aa00700 1 -- 172.21.15.107:0/850345172 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) v1 ==== 89013+0+0 (secure 0 0 0) 0x7f7590010420 con 0x7f75a41001a0 2024-03-04T13:42:01.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:01.459+0000 7f759aa00700 1 --2- 172.21.15.107:0/850345172 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7f758806b2e0 0x7f758806d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:42:01.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:01.459+0000 7f759aa00700 1 -- 172.21.15.107:0/850345172 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f7590089ca0 con 0x7f75a41001a0 2024-03-04T13:42:01.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:01.460+0000 7f75a2a00700 1 --2- 172.21.15.107:0/850345172 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7f758806b2e0 0x7f758806d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:42:01.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:01.461+0000 7f75a2a00700 1 --2- 172.21.15.107:0/850345172 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7f758806b2e0 0x7f758806d790 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f759400b3b0 tx=0x7f7594009e30 comp rx=0 tx=0).ready entity=mgr.24331 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:42:01.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:01.465+0000 7f759aa00700 1 -- 172.21.15.107:0/850345172 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f75900578f0 con 0x7f75a41001a0 2024-03-04T13:42:01.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:01.770+0000 7f75a3e00700 1 -- 172.21.15.107:0/850345172 --> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f75a4111990 con 0x7f758806b2e0 2024-03-04T13:42:01.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:01.772+0000 7f759aa00700 1 -- 172.21.15.107:0/850345172 <== mgr.24331 v2:172.21.15.169:6828/3524931243 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f75a4111990 con 0x7f758806b2e0 2024-03-04T13:42:01.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:01.775+0000 7f75a3e00700 1 -- 172.21.15.107:0/850345172 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7f758806b2e0 msgr2=0x7f758806d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:42:01.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:01.775+0000 7f75a3e00700 1 --2- 172.21.15.107:0/850345172 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7f758806b2e0 0x7f758806d790 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f759400b3b0 tx=0x7f7594009e30 comp rx=0 tx=0).stop 2024-03-04T13:42:01.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:01.775+0000 7f75a3e00700 1 -- 172.21.15.107:0/850345172 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75a41001a0 msgr2=0x7f75a410fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:42:01.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:01.775+0000 7f75a3e00700 1 --2- 172.21.15.107:0/850345172 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75a41001a0 0x7f75a410fe30 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f759000b790 tx=0x7f7590009250 comp rx=0 tx=0).stop 2024-03-04T13:42:01.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:01.776+0000 7f75a3e00700 1 -- 172.21.15.107:0/850345172 shutdown_connections 2024-03-04T13:42:01.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:01.776+0000 7f75a3e00700 1 --2- 172.21.15.107:0/850345172 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7f758806b2e0 0x7f758806d790 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:42:01.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:01.776+0000 7f75a3e00700 1 --2- 172.21.15.107:0/850345172 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75a40fd880 0x7f75a410f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:42:01.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:01.776+0000 7f75a3e00700 1 --2- 172.21.15.107:0/850345172 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75a41001a0 0x7f75a410fe30 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:42:01.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:01.776+0000 7f75a3e00700 1 -- 172.21.15.107:0/850345172 >> 172.21.15.107:0/850345172 conn(0x7f75a40f72c0 msgr2=0x7f75a4100e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:42:01.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:01.776+0000 7f75a3e00700 1 -- 172.21.15.107:0/850345172 shutdown_connections 2024-03-04T13:42:01.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:01.777+0000 7f75a3e00700 1 -- 172.21.15.107:0/850345172 wait complete. 2024-03-04T13:42:01.791 INFO:teuthology.orchestra.run.smithi107.stdout:true 2024-03-04T13:42:01.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:01.950+0000 7fbe67c00700 1 -- 172.21.15.107:0/4047675381 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe60066c70 msgr2=0x7fbe60101ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:42:01.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:01.950+0000 7fbe67c00700 1 --2- 172.21.15.107:0/4047675381 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe60066c70 0x7fbe60101ef0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fbe54004940 tx=0x7fbe54009e30 comp rx=0 tx=0).stop 2024-03-04T13:42:01.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:01.951+0000 7fbe67c00700 1 -- 172.21.15.107:0/4047675381 shutdown_connections 2024-03-04T13:42:01.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:01.951+0000 7fbe67c00700 1 --2- 172.21.15.107:0/4047675381 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe60102430 0x7fbe60104810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:42:01.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:01.951+0000 7fbe67c00700 1 --2- 172.21.15.107:0/4047675381 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe60066c70 0x7fbe60101ef0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:42:01.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:01.951+0000 7fbe67c00700 1 -- 172.21.15.107:0/4047675381 >> 172.21.15.107:0/4047675381 conn(0x7fbe600f7500 msgr2=0x7fbe600f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:42:01.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:01.951+0000 7fbe67c00700 1 -- 172.21.15.107:0/4047675381 shutdown_connections 2024-03-04T13:42:01.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:01.951+0000 7fbe67c00700 1 -- 172.21.15.107:0/4047675381 wait complete. 2024-03-04T13:42:01.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:01.952+0000 7fbe67c00700 1 Processor -- start 2024-03-04T13:42:01.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:01.952+0000 7fbe67c00700 1 -- start start 2024-03-04T13:42:01.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:01.953+0000 7fbe67c00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe60066c70 0x7fbe6010dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:42:01.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:01.953+0000 7fbe67c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe60102430 0x7fbe6010e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:42:01.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:01.953+0000 7fbe67c00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe6010cbf0 con 0x7fbe60102430 2024-03-04T13:42:01.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:01.953+0000 7fbe67c00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe6010cd30 con 0x7fbe60066c70 2024-03-04T13:42:01.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:01.953+0000 7fbe5fe00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe60102430 0x7fbe6010e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:42:01.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:01.953+0000 7fbe5fe00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe60102430 0x7fbe6010e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38786/0 (socket says 172.21.15.107:38786) 2024-03-04T13:42:01.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:01.953+0000 7fbe5fe00700 1 -- 172.21.15.107:0/608624044 learned_addr learned my addr 172.21.15.107:0/608624044 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:42:01.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:01.953+0000 7fbe65200700 1 --2- 172.21.15.107:0/608624044 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe60066c70 0x7fbe6010dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:42:01.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:01.953+0000 7fbe5fe00700 1 -- 172.21.15.107:0/608624044 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe60066c70 msgr2=0x7fbe6010dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:42:01.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:01.953+0000 7fbe5fe00700 1 --2- 172.21.15.107:0/608624044 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe60066c70 0x7fbe6010dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:42:01.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:01.954+0000 7fbe5fe00700 1 -- 172.21.15.107:0/608624044 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe54005040 con 0x7fbe60102430 2024-03-04T13:42:01.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:01.954+0000 7fbe5fe00700 1 --2- 172.21.15.107:0/608624044 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe60102430 0x7fbe6010e4f0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c00eb40 tx=0x7fbe4c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:42:01.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:01.954+0000 7fbe5cc00700 1 -- 172.21.15.107:0/608624044 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe4c007cc0 con 0x7fbe60102430 2024-03-04T13:42:01.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:01.954+0000 7fbe5cc00700 1 -- 172.21.15.107:0/608624044 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbe4c00f040 con 0x7fbe60102430 2024-03-04T13:42:01.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:01.954+0000 7fbe5cc00700 1 -- 172.21.15.107:0/608624044 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe4c00c9d0 con 0x7fbe60102430 2024-03-04T13:42:01.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:01.954+0000 7fbe67c00700 1 -- 172.21.15.107:0/608624044 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe6010cfb0 con 0x7fbe60102430 2024-03-04T13:42:01.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:01.954+0000 7fbe67c00700 1 -- 172.21.15.107:0/608624044 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe6010d4f0 con 0x7fbe60102430 2024-03-04T13:42:01.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:01.955+0000 7fbe67c00700 1 -- 172.21.15.107:0/608624044 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe6004bed0 con 0x7fbe60102430 2024-03-04T13:42:01.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:01.957+0000 7fbe5cc00700 1 -- 172.21.15.107:0/608624044 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) v1 ==== 89013+0+0 (secure 0 0 0) 0x7fbe4c00cb30 con 0x7fbe60102430 2024-03-04T13:42:01.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:01.958+0000 7fbe5cc00700 1 --2- 172.21.15.107:0/608624044 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7fbe4806b2e0 0x7fbe4806d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:42:01.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:01.958+0000 7fbe5cc00700 1 -- 172.21.15.107:0/608624044 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5808+0+0 (secure 0 0 0) 0x7fbe4c0897b0 con 0x7fbe60102430 2024-03-04T13:42:01.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:01.959+0000 7fbe65200700 1 --2- 172.21.15.107:0/608624044 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7fbe4806b2e0 0x7fbe4806d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:42:01.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:01.959+0000 7fbe65200700 1 --2- 172.21.15.107:0/608624044 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7fbe4806b2e0 0x7fbe4806d790 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fbe5400b0d0 tx=0x7fbe54009e30 comp rx=0 tx=0).ready entity=mgr.24331 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:42:01.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:01.963+0000 7fbe5cc00700 1 -- 172.21.15.107:0/608624044 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7fbe4c04f620 con 0x7fbe60102430 2024-03-04T13:42:02.247 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:02.250+0000 7fbe67c00700 1 -- 172.21.15.107:0/608624044 --> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbe60100840 con 0x7fbe4806b2e0 2024-03-04T13:42:02.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:02.253+0000 7fbe5cc00700 1 -- 172.21.15.107:0/608624044 <== mgr.24331 v2:172.21.15.169:6828/3524931243 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7fbe60100840 con 0x7fbe4806b2e0 2024-03-04T13:42:02.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:02.256+0000 7fbe67c00700 1 -- 172.21.15.107:0/608624044 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7fbe4806b2e0 msgr2=0x7fbe4806d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:42:02.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:02.256+0000 7fbe67c00700 1 --2- 172.21.15.107:0/608624044 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7fbe4806b2e0 0x7fbe4806d790 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fbe5400b0d0 tx=0x7fbe54009e30 comp rx=0 tx=0).stop 2024-03-04T13:42:02.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:02.256+0000 7fbe67c00700 1 -- 172.21.15.107:0/608624044 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe60102430 msgr2=0x7fbe6010e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:42:02.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:02.256+0000 7fbe67c00700 1 --2- 172.21.15.107:0/608624044 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe60102430 0x7fbe6010e4f0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c00eb40 tx=0x7fbe4c00b040 comp rx=0 tx=0).stop 2024-03-04T13:42:02.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:02.256+0000 7fbe67c00700 1 -- 172.21.15.107:0/608624044 shutdown_connections 2024-03-04T13:42:02.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:02.256+0000 7fbe67c00700 1 --2- 172.21.15.107:0/608624044 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7fbe4806b2e0 0x7fbe4806d790 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:42:02.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:02.256+0000 7fbe67c00700 1 --2- 172.21.15.107:0/608624044 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe60066c70 0x7fbe6010dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:42:02.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:02.256+0000 7fbe67c00700 1 --2- 172.21.15.107:0/608624044 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe60102430 0x7fbe6010e4f0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:42:02.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:02.256+0000 7fbe67c00700 1 -- 172.21.15.107:0/608624044 >> 172.21.15.107:0/608624044 conn(0x7fbe600f7500 msgr2=0x7fbe60100130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:42:02.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:02.257+0000 7fbe67c00700 1 -- 172.21.15.107:0/608624044 shutdown_connections 2024-03-04T13:42:02.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:02.257+0000 7fbe67c00700 1 -- 172.21.15.107:0/608624044 wait complete. 2024-03-04T13:42:02.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:02.413+0000 7fa702600700 1 -- 172.21.15.107:0/84973550 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa6fc070860 msgr2=0x7fa6fc070cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:42:02.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:02.413+0000 7fa702600700 1 --2- 172.21.15.107:0/84973550 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa6fc070860 0x7fa6fc070cd0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fa6e8004940 tx=0x7fa6e8009e30 comp rx=0 tx=0).stop 2024-03-04T13:42:02.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:02.414+0000 7fa702600700 1 -- 172.21.15.107:0/84973550 shutdown_connections 2024-03-04T13:42:02.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:02.414+0000 7fa702600700 1 --2- 172.21.15.107:0/84973550 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa6fc070860 0x7fa6fc070cd0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:42:02.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:02.414+0000 7fa702600700 1 --2- 172.21.15.107:0/84973550 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa6fc071eb0 0x7fa6fc070320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:42:02.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:02.414+0000 7fa702600700 1 -- 172.21.15.107:0/84973550 >> 172.21.15.107:0/84973550 conn(0x7fa6fc0f8620 msgr2=0x7fa6fc0faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:42:02.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:02.414+0000 7fa702600700 1 -- 172.21.15.107:0/84973550 shutdown_connections 2024-03-04T13:42:02.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:02.414+0000 7fa702600700 1 -- 172.21.15.107:0/84973550 wait complete. 2024-03-04T13:42:02.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:02.415+0000 7fa702600700 1 Processor -- start 2024-03-04T13:42:02.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:02.415+0000 7fa702600700 1 -- start start 2024-03-04T13:42:02.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:02.415+0000 7fa702600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa6fc070860 0x7fa6fc10fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:42:02.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:02.415+0000 7fa702600700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa6fc071eb0 0x7fa6fc1100c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:42:02.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:02.415+0000 7fa702600700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6fc1106c0 con 0x7fa6fc070860 2024-03-04T13:42:02.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:02.415+0000 7fa702600700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6fc110f60 con 0x7fa6fc071eb0 2024-03-04T13:42:02.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:02.415+0000 7fa6fb400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa6fc070860 0x7fa6fc10fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:42:02.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:02.416+0000 7fa6fb400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa6fc070860 0x7fa6fc10fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38802/0 (socket says 172.21.15.107:38802) 2024-03-04T13:42:02.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:02.416+0000 7fa6fb400700 1 -- 172.21.15.107:0/2249126750 learned_addr learned my addr 172.21.15.107:0/2249126750 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:42:02.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:02.416+0000 7fa6faa00700 1 --2- 172.21.15.107:0/2249126750 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa6fc071eb0 0x7fa6fc1100c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:42:02.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:02.416+0000 7fa6fb400700 1 -- 172.21.15.107:0/2249126750 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa6fc071eb0 msgr2=0x7fa6fc1100c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:42:02.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:02.416+0000 7fa6fb400700 1 --2- 172.21.15.107:0/2249126750 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa6fc071eb0 0x7fa6fc1100c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:42:02.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:02.416+0000 7fa6fb400700 1 -- 172.21.15.107:0/2249126750 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6e8005040 con 0x7fa6fc070860 2024-03-04T13:42:02.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:02.416+0000 7fa6fb400700 1 --2- 172.21.15.107:0/2249126750 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa6fc070860 0x7fa6fc10fb80 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec002a50 tx=0x7fa6ec0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:42:02.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:02.417+0000 7fa6f3400700 1 -- 172.21.15.107:0/2249126750 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa6ec007cc0 con 0x7fa6fc070860 2024-03-04T13:42:02.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:02.417+0000 7fa702600700 1 -- 172.21.15.107:0/2249126750 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6fc111240 con 0x7fa6fc070860 2024-03-04T13:42:02.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:02.417+0000 7fa6f3400700 1 -- 172.21.15.107:0/2249126750 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa6ec00f040 con 0x7fa6fc070860 2024-03-04T13:42:02.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:02.417+0000 7fa6f3400700 1 -- 172.21.15.107:0/2249126750 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa6ec00db70 con 0x7fa6fc070860 2024-03-04T13:42:02.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:02.417+0000 7fa702600700 1 -- 172.21.15.107:0/2249126750 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6fc111780 con 0x7fa6fc070860 2024-03-04T13:42:02.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:02.418+0000 7fa702600700 1 -- 172.21.15.107:0/2249126750 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6fc04bed0 con 0x7fa6fc070860 2024-03-04T13:42:02.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:02.419+0000 7fa6f3400700 1 -- 172.21.15.107:0/2249126750 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) v1 ==== 89013+0+0 (secure 0 0 0) 0x7fa6ec00dd10 con 0x7fa6fc070860 2024-03-04T13:42:02.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:02.420+0000 7fa6f3400700 1 --2- 172.21.15.107:0/2249126750 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7fa6e006b2e0 0x7fa6e006d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:42:02.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:02.420+0000 7fa6f3400700 1 -- 172.21.15.107:0/2249126750 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5808+0+0 (secure 0 0 0) 0x7fa6ec0894f0 con 0x7fa6fc070860 2024-03-04T13:42:02.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:02.421+0000 7fa6faa00700 1 --2- 172.21.15.107:0/2249126750 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7fa6e006b2e0 0x7fa6e006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:42:02.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:02.422+0000 7fa6faa00700 1 --2- 172.21.15.107:0/2249126750 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7fa6e006b2e0 0x7fa6e006d790 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa6e800b020 tx=0x7fa6e8009e30 comp rx=0 tx=0).ready entity=mgr.24331 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:42:02.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:02.425+0000 7fa6f3400700 1 -- 172.21.15.107:0/2249126750 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7fa6ec056fb0 con 0x7fa6fc070860 2024-03-04T13:42:02.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:02.687+0000 7fa702600700 1 -- 172.21.15.107:0/2249126750 --> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fa6fc111be0 con 0x7fa6e006b2e0 2024-03-04T13:42:02.725 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:02.728+0000 7fa6f3400700 1 -- 172.21.15.107:0/2249126750 <== mgr.24331 v2:172.21.15.169:6828/3524931243 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3388 (secure 0 0 0) 0x7fa6fc111be0 con 0x7fa6e006b2e0 2024-03-04T13:42:02.726 INFO:teuthology.orchestra.run.smithi107.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-04T13:42:02.726 INFO:teuthology.orchestra.run.smithi107.stdout:alertmanager.smithi107 smithi107 *:9093,9094 starting - - - - 2024-03-04T13:42:02.726 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi107 smithi107 running (10m) 26s ago 10m 6665k - 17.2.7-545-g42608bb1 9eec3f22859d 03e1570c2eff 2024-03-04T13:42:02.726 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi169 smithi169 running (9m) 25s ago 9m 6656k - 17.2.7-545-g42608bb1 9eec3f22859d 9c8881a5338e 2024-03-04T13:42:02.726 INFO:teuthology.orchestra.run.smithi107.stdout:grafana.smithi107 smithi107 *:3000 running (8m) 26s ago 10m 75.2M - 9.4.7 954c08fa6188 2d8536fd3802 2024-03-04T13:42:02.726 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.riexmm smithi107 running (3m) 26s ago 3m 11.4M - 17.2.7-545-g42608bb1 9eec3f22859d fc921f7d4328 2024-03-04T13:42:02.726 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.thycho smithi107 running (3m) 26s ago 3m 85.9M - 17.2.7-545-g42608bb1 9eec3f22859d 4069c6e0e063 2024-03-04T13:42:02.726 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi169.bxitzk smithi169 running (3m) 25s ago 3m 11.5M - 17.2.7-545-g42608bb1 9eec3f22859d 7d4115228c7c 2024-03-04T13:42:02.726 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi169.jusudl smithi169 running (3m) 25s ago 3m 101M - 17.2.7-545-g42608bb1 9eec3f22859d 2d2f2cc452e8 2024-03-04T13:42:02.726 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi107.wpzgwl smithi107 *:9283 running (12m) 26s ago 12m 367M - 17.2.7-545-g42608bb1 9eec3f22859d 4831dce06263 2024-03-04T13:42:02.726 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi169.eihczr smithi169 *:8443,9283 running (53s) 25s ago 9m 489M - 19.0.0-1845-g717ce3cc d71b839abdf8 18b4dfa896b5 2024-03-04T13:42:02.726 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi107 smithi107 running (12m) 26s ago 12m 50.0M 2048M 17.2.7-545-g42608bb1 9eec3f22859d 49f5da990d35 2024-03-04T13:42:02.726 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi169 smithi169 running (9m) 25s ago 9m 38.2M 2048M 17.2.7-545-g42608bb1 9eec3f22859d ba4ee916297f 2024-03-04T13:42:02.727 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi107 smithi107 *:9100 starting - - - - 2024-03-04T13:42:02.727 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi169 smithi169 *:9100 running (9m) 25s ago 9m 7801k - 1.5.0 0da6a335fe13 b2dae07f2da2 2024-03-04T13:42:02.727 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0 smithi107 running (8m) 26s ago 8m 64.8M 1558M 17.2.7-545-g42608bb1 9eec3f22859d c0605b68f696 2024-03-04T13:42:02.727 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1 smithi107 running (7m) 26s ago 7m 64.4M 1558M 17.2.7-545-g42608bb1 9eec3f22859d 5fb2c0dd5769 2024-03-04T13:42:02.727 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2 smithi107 running (6m) 26s ago 6m 59.0M 1558M 17.2.7-545-g42608bb1 9eec3f22859d 61f30f643549 2024-03-04T13:42:02.727 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3 smithi169 running (6m) 25s ago 6m 69.7M 2582M 17.2.7-545-g42608bb1 9eec3f22859d cd91d8694764 2024-03-04T13:42:02.727 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4 smithi169 running (5m) 25s ago 5m 64.6M 2582M 17.2.7-545-g42608bb1 9eec3f22859d 08545e3d0659 2024-03-04T13:42:02.727 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5 smithi169 running (5m) 25s ago 5m 62.3M 2582M 17.2.7-545-g42608bb1 9eec3f22859d dcffe6cb3eac 2024-03-04T13:42:02.727 INFO:teuthology.orchestra.run.smithi107.stdout:prometheus.smithi107 smithi107 *:9095 running (8m) 26s ago 9m 51.3M - 2.43.0 a07b618ecd1d 226d4ba6d37a 2024-03-04T13:42:02.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:02.731+0000 7fa702600700 1 -- 172.21.15.107:0/2249126750 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7fa6e006b2e0 msgr2=0x7fa6e006d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:42:02.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:02.731+0000 7fa702600700 1 --2- 172.21.15.107:0/2249126750 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7fa6e006b2e0 0x7fa6e006d790 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa6e800b020 tx=0x7fa6e8009e30 comp rx=0 tx=0).stop 2024-03-04T13:42:02.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:02.731+0000 7fa702600700 1 -- 172.21.15.107:0/2249126750 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa6fc070860 msgr2=0x7fa6fc10fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:42:02.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:02.731+0000 7fa702600700 1 --2- 172.21.15.107:0/2249126750 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa6fc070860 0x7fa6fc10fb80 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec002a50 tx=0x7fa6ec0046b0 comp rx=0 tx=0).stop 2024-03-04T13:42:02.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:02.732+0000 7fa702600700 1 -- 172.21.15.107:0/2249126750 shutdown_connections 2024-03-04T13:42:02.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:02.732+0000 7fa702600700 1 --2- 172.21.15.107:0/2249126750 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7fa6e006b2e0 0x7fa6e006d790 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:42:02.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:02.732+0000 7fa702600700 1 --2- 172.21.15.107:0/2249126750 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa6fc070860 0x7fa6fc10fb80 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:42:02.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:02.732+0000 7fa702600700 1 --2- 172.21.15.107:0/2249126750 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa6fc071eb0 0x7fa6fc1100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:42:02.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:02.732+0000 7fa702600700 1 -- 172.21.15.107:0/2249126750 >> 172.21.15.107:0/2249126750 conn(0x7fa6fc0f8620 msgr2=0x7fa6fc102e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:42:02.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:02.732+0000 7fa702600700 1 -- 172.21.15.107:0/2249126750 shutdown_connections 2024-03-04T13:42:02.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:02.732+0000 7fa702600700 1 -- 172.21.15.107:0/2249126750 wait complete. 2024-03-04T13:42:02.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:02.901+0000 7f32c1200700 1 -- 172.21.15.107:0/1679790030 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32bc0fcef0 msgr2=0x7f32bc0fd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:42:02.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:02.901+0000 7f32c1200700 1 --2- 172.21.15.107:0/1679790030 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32bc0fcef0 0x7f32bc0fd320 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f32ac004940 tx=0x7f32ac009e30 comp rx=0 tx=0).stop 2024-03-04T13:42:02.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:02.901+0000 7f32c1200700 1 -- 172.21.15.107:0/1679790030 shutdown_connections 2024-03-04T13:42:02.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:02.901+0000 7f32c1200700 1 --2- 172.21.15.107:0/1679790030 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32bc0fe0e0 0x7f32bc0fe550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:42:02.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:02.901+0000 7f32c1200700 1 --2- 172.21.15.107:0/1679790030 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32bc0fcef0 0x7f32bc0fd320 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:42:02.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:02.901+0000 7f32c1200700 1 -- 172.21.15.107:0/1679790030 >> 172.21.15.107:0/1679790030 conn(0x7f32bc0f8620 msgr2=0x7f32bc0faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:42:02.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:02.901+0000 7f32c1200700 1 -- 172.21.15.107:0/1679790030 shutdown_connections 2024-03-04T13:42:02.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:02.901+0000 7f32c1200700 1 -- 172.21.15.107:0/1679790030 wait complete. 2024-03-04T13:42:02.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:02.902+0000 7f32c1200700 1 Processor -- start 2024-03-04T13:42:02.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:02.902+0000 7f32c1200700 1 -- start start 2024-03-04T13:42:02.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:02.902+0000 7f32c1200700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32bc0fcef0 0x7f32bc10fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:42:02.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:02.902+0000 7f32c1200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32bc0fe0e0 0x7f32bc110180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:42:02.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:02.902+0000 7f32c1200700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32bc110780 con 0x7f32bc0fe0e0 2024-03-04T13:42:02.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:02.902+0000 7f32c1200700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32bc110fd0 con 0x7f32bc0fcef0 2024-03-04T13:42:02.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:02.903+0000 7f32b9400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32bc0fe0e0 0x7f32bc110180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:42:02.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:02.903+0000 7f32b9e00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32bc0fcef0 0x7f32bc10fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:42:02.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:02.903+0000 7f32b9400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32bc0fe0e0 0x7f32bc110180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38820/0 (socket says 172.21.15.107:38820) 2024-03-04T13:42:02.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:02.903+0000 7f32b9400700 1 -- 172.21.15.107:0/2180308684 learned_addr learned my addr 172.21.15.107:0/2180308684 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:42:02.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:02.903+0000 7f32b9400700 1 -- 172.21.15.107:0/2180308684 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32bc0fcef0 msgr2=0x7f32bc10fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:42:02.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:02.903+0000 7f32b9400700 1 --2- 172.21.15.107:0/2180308684 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32bc0fcef0 0x7f32bc10fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:42:02.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:02.903+0000 7f32b9400700 1 -- 172.21.15.107:0/2180308684 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32ac005040 con 0x7f32bc0fe0e0 2024-03-04T13:42:02.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:02.904+0000 7f32b9400700 1 --2- 172.21.15.107:0/2180308684 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32bc0fe0e0 0x7f32bc110180 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f32a8002a50 tx=0x7f32a80046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:42:02.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:02.904+0000 7f32b2000700 1 -- 172.21.15.107:0/2180308684 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f32a8007cc0 con 0x7f32bc0fe0e0 2024-03-04T13:42:02.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:02.904+0000 7f32c1200700 1 -- 172.21.15.107:0/2180308684 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32bc1112b0 con 0x7f32bc0fe0e0 2024-03-04T13:42:02.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:02.904+0000 7f32b2000700 1 -- 172.21.15.107:0/2180308684 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f32a800f040 con 0x7f32bc0fe0e0 2024-03-04T13:42:02.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:02.904+0000 7f32b2000700 1 -- 172.21.15.107:0/2180308684 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f32a8009640 con 0x7f32bc0fe0e0 2024-03-04T13:42:02.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:02.904+0000 7f32c1200700 1 -- 172.21.15.107:0/2180308684 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32bc1117f0 con 0x7f32bc0fe0e0 2024-03-04T13:42:02.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:02.905+0000 7f32c1200700 1 -- 172.21.15.107:0/2180308684 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32bc102210 con 0x7f32bc0fe0e0 2024-03-04T13:42:02.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:02.907+0000 7f32b2000700 1 -- 172.21.15.107:0/2180308684 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) v1 ==== 89013+0+0 (secure 0 0 0) 0x7f32a80097a0 con 0x7f32bc0fe0e0 2024-03-04T13:42:02.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:02.908+0000 7f32b2000700 1 --2- 172.21.15.107:0/2180308684 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7f32a006b2e0 0x7f32a006d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:42:02.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:02.908+0000 7f32b2000700 1 -- 172.21.15.107:0/2180308684 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f32a808a150 con 0x7f32bc0fe0e0 2024-03-04T13:42:02.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:02.909+0000 7f32b9e00700 1 --2- 172.21.15.107:0/2180308684 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7f32a006b2e0 0x7f32a006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:42:02.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:02.910+0000 7f32b9e00700 1 --2- 172.21.15.107:0/2180308684 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7f32a006b2e0 0x7f32a006d790 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f32ac00b3b0 tx=0x7f32ac009e30 comp rx=0 tx=0).ready entity=mgr.24331 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:42:02.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:02.912+0000 7f32b2000700 1 -- 172.21.15.107:0/2180308684 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f32a8057c90 con 0x7f32bc0fe0e0 2024-03-04T13:42:03.262 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:42:03 smithi107 ceph-mon[28739]: from='client.14636 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:42:03.262 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:42:03 smithi107 ceph-mon[28739]: pgmap v23: 65 pgs: 65 active+clean; 134 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 819 KiB/s rd, 1.5 MiB/s wr, 80 op/s 2024-03-04T13:42:03.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:03.299+0000 7f32c1200700 1 -- 172.21.15.107:0/2180308684 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f32bc04bed0 con 0x7f32bc0fe0e0 2024-03-04T13:42:03.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:03.300+0000 7f32b2000700 1 -- 172.21.15.107:0/2180308684 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+846 (secure 0 0 0) 0x7f32a8057820 con 0x7f32bc0fe0e0 2024-03-04T13:42:03.297 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-03-04T13:42:03.297 INFO:teuthology.orchestra.run.smithi107.stdout: "mon": { 2024-03-04T13:42:03.297 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-545-g42608bb1 (42608bb191117be10dba720e6bf8a02f3171cd34) quincy (stable)": 2 2024-03-04T13:42:03.297 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:42:03.298 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr": { 2024-03-04T13:42:03.298 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-545-g42608bb1 (42608bb191117be10dba720e6bf8a02f3171cd34) quincy (stable)": 1, 2024-03-04T13:42:03.298 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-1845-g717ce3cc (717ce3cce8d0166e4e577faa8004e6f9cb4128c0) squid (dev)": 1 2024-03-04T13:42:03.298 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:42:03.298 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": { 2024-03-04T13:42:03.298 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-545-g42608bb1 (42608bb191117be10dba720e6bf8a02f3171cd34) quincy (stable)": 6 2024-03-04T13:42:03.298 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:42:03.298 INFO:teuthology.orchestra.run.smithi107.stdout: "mds": { 2024-03-04T13:42:03.298 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-545-g42608bb1 (42608bb191117be10dba720e6bf8a02f3171cd34) quincy (stable)": 4 2024-03-04T13:42:03.298 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:42:03.298 INFO:teuthology.orchestra.run.smithi107.stdout: "overall": { 2024-03-04T13:42:03.298 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-545-g42608bb1 (42608bb191117be10dba720e6bf8a02f3171cd34) quincy (stable)": 13, 2024-03-04T13:42:03.298 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-1845-g717ce3cc (717ce3cce8d0166e4e577faa8004e6f9cb4128c0) squid (dev)": 1 2024-03-04T13:42:03.299 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-04T13:42:03.299 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-03-04T13:42:03.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:03.303+0000 7f32c1200700 1 -- 172.21.15.107:0/2180308684 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7f32a006b2e0 msgr2=0x7f32a006d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:42:03.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:03.303+0000 7f32c1200700 1 --2- 172.21.15.107:0/2180308684 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7f32a006b2e0 0x7f32a006d790 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f32ac00b3b0 tx=0x7f32ac009e30 comp rx=0 tx=0).stop 2024-03-04T13:42:03.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:03.303+0000 7f32c1200700 1 -- 172.21.15.107:0/2180308684 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32bc0fe0e0 msgr2=0x7f32bc110180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:42:03.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:03.303+0000 7f32c1200700 1 --2- 172.21.15.107:0/2180308684 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32bc0fe0e0 0x7f32bc110180 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f32a8002a50 tx=0x7f32a80046b0 comp rx=0 tx=0).stop 2024-03-04T13:42:03.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:03.304+0000 7f32c1200700 1 -- 172.21.15.107:0/2180308684 shutdown_connections 2024-03-04T13:42:03.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:03.304+0000 7f32c1200700 1 --2- 172.21.15.107:0/2180308684 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7f32a006b2e0 0x7f32a006d790 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:42:03.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:03.304+0000 7f32c1200700 1 --2- 172.21.15.107:0/2180308684 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32bc0fcef0 0x7f32bc10fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:42:03.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:03.304+0000 7f32c1200700 1 --2- 172.21.15.107:0/2180308684 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32bc0fe0e0 0x7f32bc110180 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:42:03.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:03.304+0000 7f32c1200700 1 -- 172.21.15.107:0/2180308684 >> 172.21.15.107:0/2180308684 conn(0x7f32bc0f8620 msgr2=0x7f32bc0ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:42:03.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:03.304+0000 7f32c1200700 1 -- 172.21.15.107:0/2180308684 shutdown_connections 2024-03-04T13:42:03.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:03.304+0000 7f32c1200700 1 -- 172.21.15.107:0/2180308684 wait complete. 2024-03-04T13:42:03.356 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:42:03 smithi169 ceph-mon[36313]: from='client.14636 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:42:03.356 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:42:03 smithi169 ceph-mon[36313]: pgmap v23: 65 pgs: 65 active+clean; 134 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 819 KiB/s rd, 1.5 MiB/s wr, 80 op/s 2024-03-04T13:42:03.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:03.462+0000 7ffbf2200700 1 -- 172.21.15.107:0/1633736903 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffbec1003a0 msgr2=0x7ffbec102780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:42:03.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:03.462+0000 7ffbf2200700 1 --2- 172.21.15.107:0/1633736903 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffbec1003a0 0x7ffbec102780 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7ffbd8004940 tx=0x7ffbd8009e30 comp rx=0 tx=0).stop 2024-03-04T13:42:03.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:03.462+0000 7ffbf2200700 1 -- 172.21.15.107:0/1633736903 shutdown_connections 2024-03-04T13:42:03.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:03.462+0000 7ffbf2200700 1 --2- 172.21.15.107:0/1633736903 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffbec1003a0 0x7ffbec102780 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:42:03.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:03.462+0000 7ffbf2200700 1 --2- 172.21.15.107:0/1633736903 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffbec0fda80 0x7ffbec0ffe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:42:03.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:03.462+0000 7ffbf2200700 1 -- 172.21.15.107:0/1633736903 >> 172.21.15.107:0/1633736903 conn(0x7ffbec0f74e0 msgr2=0x7ffbec0f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:42:03.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:03.463+0000 7ffbf2200700 1 -- 172.21.15.107:0/1633736903 shutdown_connections 2024-03-04T13:42:03.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:03.463+0000 7ffbf2200700 1 -- 172.21.15.107:0/1633736903 wait complete. 2024-03-04T13:42:03.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:03.464+0000 7ffbf2200700 1 Processor -- start 2024-03-04T13:42:03.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:03.464+0000 7ffbf2200700 1 -- start start 2024-03-04T13:42:03.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:03.464+0000 7ffbf2200700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffbec0fda80 0x7ffbec0fd510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:42:03.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:03.464+0000 7ffbf2200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffbec1003a0 0x7ffbec0fb990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:42:03.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:03.464+0000 7ffbf2200700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbec0fbf90 con 0x7ffbec1003a0 2024-03-04T13:42:03.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:03.464+0000 7ffbf2200700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbec0fc100 con 0x7ffbec0fda80 2024-03-04T13:42:03.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:03.465+0000 7ffbeb400700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffbec0fda80 0x7ffbec0fd510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:42:03.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:03.465+0000 7ffbeb400700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffbec0fda80 0x7ffbec0fd510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.107:47348/0 (socket says 172.21.15.107:47348) 2024-03-04T13:42:03.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:03.465+0000 7ffbeb400700 1 -- 172.21.15.107:0/3596429392 learned_addr learned my addr 172.21.15.107:0/3596429392 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:42:03.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:03.465+0000 7ffbeaa00700 1 --2- 172.21.15.107:0/3596429392 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffbec1003a0 0x7ffbec0fb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:42:03.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:03.465+0000 7ffbeb400700 1 -- 172.21.15.107:0/3596429392 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffbec1003a0 msgr2=0x7ffbec0fb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:42:03.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:03.465+0000 7ffbeb400700 1 --2- 172.21.15.107:0/3596429392 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffbec1003a0 0x7ffbec0fb990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:42:03.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:03.465+0000 7ffbeb400700 1 -- 172.21.15.107:0/3596429392 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbd8005040 con 0x7ffbec0fda80 2024-03-04T13:42:03.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:03.466+0000 7ffbeb400700 1 --2- 172.21.15.107:0/3596429392 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffbec0fda80 0x7ffbec0fd510 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc00eb40 tx=0x7ffbdc00b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:42:03.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:03.466+0000 7ffbe3400700 1 -- 172.21.15.107:0/3596429392 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffbdc007cc0 con 0x7ffbec0fda80 2024-03-04T13:42:03.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:03.466+0000 7ffbf2200700 1 -- 172.21.15.107:0/3596429392 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbec0fc380 con 0x7ffbec0fda80 2024-03-04T13:42:03.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:03.466+0000 7ffbe3400700 1 -- 172.21.15.107:0/3596429392 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffbdc00f040 con 0x7ffbec0fda80 2024-03-04T13:42:03.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:03.466+0000 7ffbe3400700 1 -- 172.21.15.107:0/3596429392 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffbdc00ca10 con 0x7ffbec0fda80 2024-03-04T13:42:03.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:03.466+0000 7ffbf2200700 1 -- 172.21.15.107:0/3596429392 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbec115910 con 0x7ffbec0fda80 2024-03-04T13:42:03.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:03.467+0000 7ffbf2200700 1 -- 172.21.15.107:0/3596429392 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbec107a50 con 0x7ffbec0fda80 2024-03-04T13:42:03.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:03.469+0000 7ffbe3400700 1 -- 172.21.15.107:0/3596429392 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 24) v1 ==== 89013+0+0 (secure 0 0 0) 0x7ffbdc00cc80 con 0x7ffbec0fda80 2024-03-04T13:42:03.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:03.470+0000 7ffbe3400700 1 --2- 172.21.15.107:0/3596429392 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7ffbd006b2e0 0x7ffbd006d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:42:03.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:03.470+0000 7ffbe3400700 1 -- 172.21.15.107:0/3596429392 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5808+0+0 (secure 0 0 0) 0x7ffbdc089cc0 con 0x7ffbec0fda80 2024-03-04T13:42:03.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:03.471+0000 7ffbeaa00700 1 --2- 172.21.15.107:0/3596429392 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7ffbd006b2e0 0x7ffbd006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:42:03.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:03.472+0000 7ffbeaa00700 1 --2- 172.21.15.107:0/3596429392 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7ffbd006b2e0 0x7ffbd006d790 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ffbd8009b30 tx=0x7ffbd8009e30 comp rx=0 tx=0).ready entity=mgr.24331 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:42:03.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:03.490+0000 7ffbe3400700 1 -- 172.21.15.107:0/3596429392 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7ffbdc057830 con 0x7ffbec0fda80 2024-03-04T13:42:03.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:03.790+0000 7ffbf2200700 1 -- 172.21.15.107:0/3596429392 --> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ffbec02b3b0 con 0x7ffbd006b2e0 2024-03-04T13:42:03.791 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:03.794+0000 7ffbe3400700 1 -- 172.21.15.107:0/3596429392 <== mgr.24331 v2:172.21.15.169:6828/3524931243 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7ffbec02b3b0 con 0x7ffbd006b2e0 2024-03-04T13:42:03.791 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-03-04T13:42:03.791 INFO:teuthology.orchestra.run.smithi107.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377", 2024-03-04T13:42:03.791 INFO:teuthology.orchestra.run.smithi107.stdout: "in_progress": true, 2024-03-04T13:42:03.791 INFO:teuthology.orchestra.run.smithi107.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-03-04T13:42:03.791 INFO:teuthology.orchestra.run.smithi107.stdout: "services_complete": [], 2024-03-04T13:42:03.791 INFO:teuthology.orchestra.run.smithi107.stdout: "progress": "1/2 daemons upgraded", 2024-03-04T13:42:03.792 INFO:teuthology.orchestra.run.smithi107.stdout: "message": "", 2024-03-04T13:42:03.792 INFO:teuthology.orchestra.run.smithi107.stdout: "is_paused": false 2024-03-04T13:42:03.792 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-03-04T13:42:03.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:03.797+0000 7ffbf2200700 1 -- 172.21.15.107:0/3596429392 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7ffbd006b2e0 msgr2=0x7ffbd006d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:42:03.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:03.797+0000 7ffbf2200700 1 --2- 172.21.15.107:0/3596429392 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7ffbd006b2e0 0x7ffbd006d790 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ffbd8009b30 tx=0x7ffbd8009e30 comp rx=0 tx=0).stop 2024-03-04T13:42:03.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:03.797+0000 7ffbf2200700 1 -- 172.21.15.107:0/3596429392 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffbec0fda80 msgr2=0x7ffbec0fd510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:42:03.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:03.797+0000 7ffbf2200700 1 --2- 172.21.15.107:0/3596429392 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffbec0fda80 0x7ffbec0fd510 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc00eb40 tx=0x7ffbdc00b040 comp rx=0 tx=0).stop 2024-03-04T13:42:03.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:03.797+0000 7ffbf2200700 1 -- 172.21.15.107:0/3596429392 shutdown_connections 2024-03-04T13:42:03.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:03.797+0000 7ffbf2200700 1 --2- 172.21.15.107:0/3596429392 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7ffbd006b2e0 0x7ffbd006d790 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:42:03.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:03.797+0000 7ffbf2200700 1 --2- 172.21.15.107:0/3596429392 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffbec0fda80 0x7ffbec0fd510 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:42:03.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:03.797+0000 7ffbf2200700 1 --2- 172.21.15.107:0/3596429392 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffbec1003a0 0x7ffbec0fb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:42:03.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:03.797+0000 7ffbf2200700 1 -- 172.21.15.107:0/3596429392 >> 172.21.15.107:0/3596429392 conn(0x7ffbec0f74e0 msgr2=0x7ffbec0f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:42:03.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:03.797+0000 7ffbf2200700 1 -- 172.21.15.107:0/3596429392 shutdown_connections 2024-03-04T13:42:03.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:03.798+0000 7ffbf2200700 1 -- 172.21.15.107:0/3596429392 wait complete. 2024-03-04T13:42:04.089 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:42:04 smithi107 ceph-mon[28739]: from='client.14640 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:42:04.089 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:42:04 smithi107 ceph-mon[28739]: from='client.14644 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:42:04.089 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:42:04 smithi107 ceph-mon[28739]: from='client.? 172.21.15.107:0/2180308684' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:42:04.357 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:42:04 smithi169 ceph-mon[36313]: from='client.14640 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:42:04.357 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:42:04 smithi169 ceph-mon[36313]: from='client.14644 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:42:04.358 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:42:04 smithi169 ceph-mon[36313]: from='client.? 172.21.15.107:0/2180308684' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:42:05.339 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:42:05 smithi107 ceph-mon[28739]: from='client.24397 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:42:05.339 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:42:05 smithi107 ceph-mon[28739]: pgmap v24: 65 pgs: 65 active+clean; 142 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 135 op/s 2024-03-04T13:42:05.356 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:42:05 smithi169 ceph-mon[36313]: from='client.24397 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:42:05.356 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:42:05 smithi169 ceph-mon[36313]: pgmap v24: 65 pgs: 65 active+clean; 142 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 135 op/s 2024-03-04T13:42:07.090 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:42:06 smithi107 ceph-mon[28739]: pgmap v25: 65 pgs: 65 active+clean; 142 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 93 op/s 2024-03-04T13:42:07.106 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:42:06 smithi169 ceph-mon[36313]: pgmap v25: 65 pgs: 65 active+clean; 142 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 93 op/s 2024-03-04T13:42:08.089 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:42:07 smithi107 ceph-mon[28739]: from='mgr.24331 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:42:08.106 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:42:07 smithi169 ceph-mon[36313]: from='mgr.24331 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:42:09.089 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:42:08 smithi107 ceph-mon[28739]: pgmap v26: 65 pgs: 65 active+clean; 144 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 133 op/s 2024-03-04T13:42:09.106 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:42:08 smithi169 ceph-mon[36313]: pgmap v26: 65 pgs: 65 active+clean; 144 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 133 op/s 2024-03-04T13:42:11.339 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:42:11 smithi107 ceph-mon[28739]: pgmap v27: 65 pgs: 65 active+clean; 149 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 891 KiB/s rd, 1.4 MiB/s wr, 112 op/s 2024-03-04T13:42:11.356 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:42:11 smithi169 ceph-mon[36313]: pgmap v27: 65 pgs: 65 active+clean; 149 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 891 KiB/s rd, 1.4 MiB/s wr, 112 op/s 2024-03-04T13:42:13.262 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:42:13 smithi169 ceph-mon[36313]: pgmap v28: 65 pgs: 65 active+clean; 149 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 891 KiB/s rd, 1.3 MiB/s wr, 103 op/s 2024-03-04T13:42:13.339 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:42:13 smithi107 ceph-mon[28739]: pgmap v28: 65 pgs: 65 active+clean; 149 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 891 KiB/s rd, 1.3 MiB/s wr, 103 op/s 2024-03-04T13:42:15.339 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:42:15 smithi107 ceph-mon[28739]: pgmap v29: 65 pgs: 65 active+clean; 158 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 165 op/s 2024-03-04T13:42:15.356 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:42:15 smithi169 ceph-mon[36313]: pgmap v29: 65 pgs: 65 active+clean; 158 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 165 op/s 2024-03-04T13:42:17.089 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:42:16 smithi107 ceph-mon[28739]: pgmap v30: 65 pgs: 65 active+clean; 158 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 724 KiB/s rd, 1.4 MiB/s wr, 110 op/s 2024-03-04T13:42:17.106 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:42:16 smithi169 ceph-mon[36313]: pgmap v30: 65 pgs: 65 active+clean; 158 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 724 KiB/s rd, 1.4 MiB/s wr, 110 op/s 2024-03-04T13:42:19.260 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:42:19 smithi107 ceph-mon[28739]: pgmap v31: 65 pgs: 65 active+clean; 166 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 815 KiB/s rd, 2.1 MiB/s wr, 197 op/s 2024-03-04T13:42:19.356 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:42:19 smithi169 ceph-mon[36313]: pgmap v31: 65 pgs: 65 active+clean; 166 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 815 KiB/s rd, 2.1 MiB/s wr, 197 op/s 2024-03-04T13:42:21.340 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:42:21 smithi107 ceph-mon[28739]: pgmap v32: 65 pgs: 65 active+clean; 170 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 169 op/s 2024-03-04T13:42:21.356 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:42:21 smithi169 ceph-mon[36313]: pgmap v32: 65 pgs: 65 active+clean; 170 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 169 op/s 2024-03-04T13:42:23.234 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:42:23 smithi107 ceph-mon[28739]: pgmap v33: 65 pgs: 65 active+clean; 170 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 774 KiB/s rd, 1.9 MiB/s wr, 160 op/s 2024-03-04T13:42:23.234 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:42:23 smithi107 ceph-mon[28739]: from='mgr.24331 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:42:23.356 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:42:23 smithi169 ceph-mon[36313]: pgmap v33: 65 pgs: 65 active+clean; 170 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 774 KiB/s rd, 1.9 MiB/s wr, 160 op/s 2024-03-04T13:42:23.356 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:42:23 smithi169 ceph-mon[36313]: from='mgr.24331 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:42:25.340 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:42:25 smithi107 ceph-mon[28739]: pgmap v34: 65 pgs: 65 active+clean; 179 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.3 MiB/s rd, 2.6 MiB/s wr, 247 op/s 2024-03-04T13:42:25.356 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:42:25 smithi169 ceph-mon[36313]: pgmap v34: 65 pgs: 65 active+clean; 179 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.3 MiB/s rd, 2.6 MiB/s wr, 247 op/s 2024-03-04T13:42:26.793 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:42:26 smithi107 ceph-mon[28739]: pgmap v35: 65 pgs: 65 active+clean; 179 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 985 KiB/s rd, 1.7 MiB/s wr, 185 op/s 2024-03-04T13:42:27.107 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:42:26 smithi169 ceph-mon[36313]: pgmap v35: 65 pgs: 65 active+clean; 179 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 985 KiB/s rd, 1.7 MiB/s wr, 185 op/s 2024-03-04T13:42:28.590 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:42:28 smithi107 ceph-mon[28739]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' 2024-03-04T13:42:28.590 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:42:28 smithi107 ceph-mon[28739]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' 2024-03-04T13:42:28.590 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:42:28 smithi107 ceph-mon[28739]: Reconfiguring prometheus.smithi107 (dependencies changed)... 2024-03-04T13:42:28.590 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:42:28 smithi107 ceph-mon[28739]: Reconfiguring daemon prometheus.smithi107 on smithi107 2024-03-04T13:42:28.590 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:42:28 smithi107 ceph-mon[28739]: pgmap v36: 65 pgs: 65 active+clean; 190 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 266 op/s 2024-03-04T13:42:28.607 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:42:28 smithi169 ceph-mon[36313]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' 2024-03-04T13:42:28.607 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:42:28 smithi169 ceph-mon[36313]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' 2024-03-04T13:42:28.607 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:42:28 smithi169 ceph-mon[36313]: Reconfiguring prometheus.smithi107 (dependencies changed)... 2024-03-04T13:42:28.607 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:42:28 smithi169 ceph-mon[36313]: Reconfiguring daemon prometheus.smithi107 on smithi107 2024-03-04T13:42:28.607 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:42:28 smithi169 ceph-mon[36313]: pgmap v36: 65 pgs: 65 active+clean; 190 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 266 op/s 2024-03-04T13:42:31.281 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:42:31 smithi107 ceph-mon[28739]: pgmap v37: 65 pgs: 65 active+clean; 190 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 186 op/s 2024-03-04T13:42:31.357 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:42:31 smithi169 ceph-mon[36313]: pgmap v37: 65 pgs: 65 active+clean; 190 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 186 op/s 2024-03-04T13:42:32.980 INFO:tasks.workunit.client.1.smithi169.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-03-04T13:42:32.999 INFO:tasks.workunit.client.1.smithi169.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-03-04T13:42:32.999 INFO:tasks.workunit.client.1.smithi169.stderr:+ make 2024-03-04T13:42:33.340 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:42:33 smithi107 ceph-mon[28739]: pgmap v38: 65 pgs: 65 active+clean; 190 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 935 KiB/s rd, 1.7 MiB/s wr, 174 op/s 2024-03-04T13:42:33.357 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:42:33 smithi169 ceph-mon[36313]: pgmap v38: 65 pgs: 65 active+clean; 190 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 935 KiB/s rd, 1.7 MiB/s wr, 174 op/s 2024-03-04T13:42:33.927 INFO:tasks.workunit.client.1.smithi169.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-03-04T13:42:33.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:33.974+0000 7f4cfba00700 1 -- 172.21.15.107:0/2521372257 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4cf4100360 msgr2=0x7f4cf41007b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:42:33.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:33.974+0000 7f4cfba00700 1 --2- 172.21.15.107:0/2521372257 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4cf4100360 0x7f4cf41007b0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f4ce0004940 tx=0x7f4ce0009e30 comp rx=0 tx=0).stop 2024-03-04T13:42:33.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:33.975+0000 7f4cfba00700 1 -- 172.21.15.107:0/2521372257 shutdown_connections 2024-03-04T13:42:33.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:33.975+0000 7f4cfba00700 1 --2- 172.21.15.107:0/2521372257 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4cf4100360 0x7f4cf41007b0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:42:33.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:33.975+0000 7f4cfba00700 1 --2- 172.21.15.107:0/2521372257 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4cf40ff150 0x7f4cf40ff560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:42:33.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:33.975+0000 7f4cfba00700 1 -- 172.21.15.107:0/2521372257 >> 172.21.15.107:0/2521372257 conn(0x7f4cf40fa920 msgr2=0x7f4cf40fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:42:33.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:33.975+0000 7f4cfba00700 1 -- 172.21.15.107:0/2521372257 shutdown_connections 2024-03-04T13:42:33.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:33.975+0000 7f4cfba00700 1 -- 172.21.15.107:0/2521372257 wait complete. 2024-03-04T13:42:33.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:33.976+0000 7f4cfba00700 1 Processor -- start 2024-03-04T13:42:33.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:33.976+0000 7f4cfba00700 1 -- start start 2024-03-04T13:42:33.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:33.976+0000 7f4cfba00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4cf40ff150 0x7f4cf4072780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:42:33.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:33.976+0000 7f4cfba00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4cf4100360 0x7f4cf4072cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:42:33.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:33.976+0000 7f4cfba00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf4073310 con 0x7f4cf4100360 2024-03-04T13:42:33.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:33.976+0000 7f4cfba00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf4073bb0 con 0x7f4cf40ff150 2024-03-04T13:42:33.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:33.977+0000 7f4cf3e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4cf4100360 0x7f4cf4072cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:42:33.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:33.977+0000 7f4cf3e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4cf4100360 0x7f4cf4072cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43264/0 (socket says 172.21.15.107:43264) 2024-03-04T13:42:33.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:33.977+0000 7f4cf3e00700 1 -- 172.21.15.107:0/2293619702 learned_addr learned my addr 172.21.15.107:0/2293619702 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:42:33.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:33.978+0000 7f4cf9000700 1 --2- 172.21.15.107:0/2293619702 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4cf40ff150 0x7f4cf4072780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:42:33.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:33.978+0000 7f4cf3e00700 1 -- 172.21.15.107:0/2293619702 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4cf40ff150 msgr2=0x7f4cf4072780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:42:33.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:33.978+0000 7f4cf3e00700 1 --2- 172.21.15.107:0/2293619702 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4cf40ff150 0x7f4cf4072780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:42:33.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:33.978+0000 7f4cf3e00700 1 -- 172.21.15.107:0/2293619702 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ce0005040 con 0x7f4cf4100360 2024-03-04T13:42:33.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:33.978+0000 7f4cf3e00700 1 --2- 172.21.15.107:0/2293619702 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4cf4100360 0x7f4cf4072cc0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f4ce0009fd0 tx=0x7f4ce0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:42:33.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:33.979+0000 7f4cf0c00700 1 -- 172.21.15.107:0/2293619702 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ce0021070 con 0x7f4cf4100360 2024-03-04T13:42:33.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:33.979+0000 7f4cf0c00700 1 -- 172.21.15.107:0/2293619702 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4ce001c400 con 0x7f4cf4100360 2024-03-04T13:42:33.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:33.979+0000 7f4cfba00700 1 -- 172.21.15.107:0/2293619702 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cf4073db0 con 0x7f4cf4100360 2024-03-04T13:42:33.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:33.979+0000 7f4cf0c00700 1 -- 172.21.15.107:0/2293619702 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ce000e070 con 0x7f4cf4100360 2024-03-04T13:42:33.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:33.979+0000 7f4cfba00700 1 -- 172.21.15.107:0/2293619702 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cf4074260 con 0x7f4cf4100360 2024-03-04T13:42:33.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:33.980+0000 7f4cfba00700 1 -- 172.21.15.107:0/2293619702 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cf404bed0 con 0x7f4cf4100360 2024-03-04T13:42:33.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:33.981+0000 7f4cf0c00700 1 -- 172.21.15.107:0/2293619702 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) v1 ==== 89013+0+0 (secure 0 0 0) 0x7f4ce001c570 con 0x7f4cf4100360 2024-03-04T13:42:33.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:33.982+0000 7f4cf0c00700 1 --2- 172.21.15.107:0/2293619702 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7f4cdc06b2e0 0x7f4cdc06d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:42:33.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:33.982+0000 7f4cf0c00700 1 -- 172.21.15.107:0/2293619702 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f4ce008b190 con 0x7f4cf4100360 2024-03-04T13:42:33.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:33.983+0000 7f4cf9000700 1 --2- 172.21.15.107:0/2293619702 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7f4cdc06b2e0 0x7f4cdc06d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:42:33.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:33.984+0000 7f4cf9000700 1 --2- 172.21.15.107:0/2293619702 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7f4cdc06b2e0 0x7f4cdc06d790 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f4cf4072510 tx=0x7f4ce800b040 comp rx=0 tx=0).ready entity=mgr.24331 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:42:33.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:33.989+0000 7f4cf0c00700 1 -- 172.21.15.107:0/2293619702 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f4ce0055120 con 0x7f4cf4100360 2024-03-04T13:42:34.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:34.300+0000 7f4cfba00700 1 -- 172.21.15.107:0/2293619702 --> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4cf4074a10 con 0x7f4cdc06b2e0 2024-03-04T13:42:34.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:34.303+0000 7f4cf0c00700 1 -- 172.21.15.107:0/2293619702 <== mgr.24331 v2:172.21.15.169:6828/3524931243 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f4cf4074a10 con 0x7f4cdc06b2e0 2024-03-04T13:42:34.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:34.307+0000 7f4cfba00700 1 -- 172.21.15.107:0/2293619702 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7f4cdc06b2e0 msgr2=0x7f4cdc06d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:42:34.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:34.307+0000 7f4cfba00700 1 --2- 172.21.15.107:0/2293619702 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7f4cdc06b2e0 0x7f4cdc06d790 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f4cf4072510 tx=0x7f4ce800b040 comp rx=0 tx=0).stop 2024-03-04T13:42:34.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:34.307+0000 7f4cfba00700 1 -- 172.21.15.107:0/2293619702 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4cf4100360 msgr2=0x7f4cf4072cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:42:34.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:34.307+0000 7f4cfba00700 1 --2- 172.21.15.107:0/2293619702 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4cf4100360 0x7f4cf4072cc0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f4ce0009fd0 tx=0x7f4ce0009e30 comp rx=0 tx=0).stop 2024-03-04T13:42:34.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:34.307+0000 7f4cfba00700 1 -- 172.21.15.107:0/2293619702 shutdown_connections 2024-03-04T13:42:34.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:34.307+0000 7f4cfba00700 1 --2- 172.21.15.107:0/2293619702 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7f4cdc06b2e0 0x7f4cdc06d790 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:42:34.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:34.307+0000 7f4cfba00700 1 --2- 172.21.15.107:0/2293619702 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4cf40ff150 0x7f4cf4072780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:42:34.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:34.307+0000 7f4cfba00700 1 --2- 172.21.15.107:0/2293619702 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4cf4100360 0x7f4cf4072cc0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:42:34.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:34.307+0000 7f4cfba00700 1 -- 172.21.15.107:0/2293619702 >> 172.21.15.107:0/2293619702 conn(0x7f4cf40fa920 msgr2=0x7f4cf4103590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:42:34.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:34.307+0000 7f4cfba00700 1 -- 172.21.15.107:0/2293619702 shutdown_connections 2024-03-04T13:42:34.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:34.307+0000 7f4cfba00700 1 -- 172.21.15.107:0/2293619702 wait complete. 2024-03-04T13:42:34.322 INFO:teuthology.orchestra.run.smithi107.stdout:true 2024-03-04T13:42:34.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:34.479+0000 7f445fe00700 1 -- 172.21.15.107:0/923338548 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44600fe000 msgr2=0x7f4460100480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:42:34.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:34.479+0000 7f445fe00700 1 --2- 172.21.15.107:0/923338548 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44600fe000 0x7f4460100480 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f444c004940 tx=0x7f444c009e30 comp rx=0 tx=0).stop 2024-03-04T13:42:34.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:34.479+0000 7f445fe00700 1 -- 172.21.15.107:0/923338548 shutdown_connections 2024-03-04T13:42:34.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:34.479+0000 7f445fe00700 1 --2- 172.21.15.107:0/923338548 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44600fe000 0x7f4460100480 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:42:34.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:34.479+0000 7f445fe00700 1 --2- 172.21.15.107:0/923338548 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44600fb6a0 0x7f44600fdac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:42:34.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:34.479+0000 7f445fe00700 1 -- 172.21.15.107:0/923338548 >> 172.21.15.107:0/923338548 conn(0x7f44600f7330 msgr2=0x7f44600f9760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:42:34.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:34.479+0000 7f445fe00700 1 -- 172.21.15.107:0/923338548 shutdown_connections 2024-03-04T13:42:34.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:34.479+0000 7f445fe00700 1 -- 172.21.15.107:0/923338548 wait complete. 2024-03-04T13:42:34.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:34.480+0000 7f445fe00700 1 Processor -- start 2024-03-04T13:42:34.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:34.480+0000 7f445fe00700 1 -- start start 2024-03-04T13:42:34.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:34.480+0000 7f445fe00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44600fb6a0 0x7f4460113d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:42:34.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:34.480+0000 7f445fe00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44600fe000 0x7f4460114280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:42:34.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:34.480+0000 7f445fe00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4460115040 con 0x7f44600fe000 2024-03-04T13:42:34.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:34.480+0000 7f445fe00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44601151b0 con 0x7f44600fb6a0 2024-03-04T13:42:34.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:34.481+0000 7f445e000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44600fe000 0x7f4460114280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:42:34.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:34.481+0000 7f445e000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44600fe000 0x7f4460114280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43284/0 (socket says 172.21.15.107:43284) 2024-03-04T13:42:34.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:34.481+0000 7f445e000700 1 -- 172.21.15.107:0/3270622367 learned_addr learned my addr 172.21.15.107:0/3270622367 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:42:34.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:34.481+0000 7f445ea00700 1 --2- 172.21.15.107:0/3270622367 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44600fb6a0 0x7f4460113d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:42:34.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:34.481+0000 7f445e000700 1 -- 172.21.15.107:0/3270622367 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44600fb6a0 msgr2=0x7f4460113d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:42:34.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:34.481+0000 7f445e000700 1 --2- 172.21.15.107:0/3270622367 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44600fb6a0 0x7f4460113d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:42:34.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:34.481+0000 7f445e000700 1 -- 172.21.15.107:0/3270622367 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f444c005040 con 0x7f44600fe000 2024-03-04T13:42:34.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:34.482+0000 7f445e000700 1 --2- 172.21.15.107:0/3270622367 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44600fe000 0x7f4460114280 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f444c00b3b0 tx=0x7f444c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:42:34.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:34.482+0000 7f4456a00700 1 -- 172.21.15.107:0/3270622367 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f444c00e040 con 0x7f44600fe000 2024-03-04T13:42:34.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:34.482+0000 7f4456a00700 1 -- 172.21.15.107:0/3270622367 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f444c00bd90 con 0x7f44600fe000 2024-03-04T13:42:34.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:34.482+0000 7f4456a00700 1 -- 172.21.15.107:0/3270622367 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f444c01c630 con 0x7f44600fe000 2024-03-04T13:42:34.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:34.482+0000 7f445fe00700 1 -- 172.21.15.107:0/3270622367 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44601153d0 con 0x7f44600fe000 2024-03-04T13:42:34.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:34.482+0000 7f445fe00700 1 -- 172.21.15.107:0/3270622367 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4460115990 con 0x7f44600fe000 2024-03-04T13:42:34.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:34.484+0000 7f445fe00700 1 -- 172.21.15.107:0/3270622367 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f446010df00 con 0x7f44600fe000 2024-03-04T13:42:34.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:34.485+0000 7f4456a00700 1 -- 172.21.15.107:0/3270622367 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) v1 ==== 89013+0+0 (secure 0 0 0) 0x7f444c0139f0 con 0x7f44600fe000 2024-03-04T13:42:34.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:34.486+0000 7f4456a00700 1 --2- 172.21.15.107:0/3270622367 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7f444406b2e0 0x7f444406d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:42:34.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:34.486+0000 7f4456a00700 1 -- 172.21.15.107:0/3270622367 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f444c08b270 con 0x7f44600fe000 2024-03-04T13:42:34.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:34.486+0000 7f445ea00700 1 --2- 172.21.15.107:0/3270622367 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7f444406b2e0 0x7f444406d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:42:34.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:34.488+0000 7f445ea00700 1 --2- 172.21.15.107:0/3270622367 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7f444406b2e0 0x7f444406d790 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f4450006fd0 tx=0x7f4450008040 comp rx=0 tx=0).ready entity=mgr.24331 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:42:34.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:34.491+0000 7f4456a00700 1 -- 172.21.15.107:0/3270622367 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f444c0553f0 con 0x7f44600fe000 2024-03-04T13:42:34.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:34.756+0000 7f445fe00700 1 -- 172.21.15.107:0/3270622367 --> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4460115c70 con 0x7f444406b2e0 2024-03-04T13:42:34.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:34.760+0000 7f4456a00700 1 -- 172.21.15.107:0/3270622367 <== mgr.24331 v2:172.21.15.169:6828/3524931243 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f4460115c70 con 0x7f444406b2e0 2024-03-04T13:42:34.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:34.763+0000 7f445fe00700 1 -- 172.21.15.107:0/3270622367 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7f444406b2e0 msgr2=0x7f444406d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:42:34.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:34.763+0000 7f445fe00700 1 --2- 172.21.15.107:0/3270622367 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7f444406b2e0 0x7f444406d790 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f4450006fd0 tx=0x7f4450008040 comp rx=0 tx=0).stop 2024-03-04T13:42:34.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:34.763+0000 7f445fe00700 1 -- 172.21.15.107:0/3270622367 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44600fe000 msgr2=0x7f4460114280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:42:34.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:34.763+0000 7f445fe00700 1 --2- 172.21.15.107:0/3270622367 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44600fe000 0x7f4460114280 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f444c00b3b0 tx=0x7f444c009e30 comp rx=0 tx=0).stop 2024-03-04T13:42:34.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:34.763+0000 7f445fe00700 1 -- 172.21.15.107:0/3270622367 shutdown_connections 2024-03-04T13:42:34.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:34.763+0000 7f445fe00700 1 --2- 172.21.15.107:0/3270622367 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7f444406b2e0 0x7f444406d790 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:42:34.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:34.763+0000 7f445fe00700 1 --2- 172.21.15.107:0/3270622367 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44600fb6a0 0x7f4460113d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:42:34.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:34.763+0000 7f445fe00700 1 --2- 172.21.15.107:0/3270622367 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44600fe000 0x7f4460114280 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:42:34.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:34.763+0000 7f445fe00700 1 -- 172.21.15.107:0/3270622367 >> 172.21.15.107:0/3270622367 conn(0x7f44600f7330 msgr2=0x7f44600f9760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:42:34.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:34.763+0000 7f445fe00700 1 -- 172.21.15.107:0/3270622367 shutdown_connections 2024-03-04T13:42:34.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:34.763+0000 7f445fe00700 1 -- 172.21.15.107:0/3270622367 wait complete. 2024-03-04T13:42:34.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:34.924+0000 7fb18fe00700 1 -- 172.21.15.107:0/1959526244 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb1900fd880 msgr2=0x7fb1900ffc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:42:34.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:34.924+0000 7fb18fe00700 1 --2- 172.21.15.107:0/1959526244 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb1900fd880 0x7fb1900ffc60 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fb180004940 tx=0x7fb180009e30 comp rx=0 tx=0).stop 2024-03-04T13:42:34.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:34.924+0000 7fb18fe00700 1 -- 172.21.15.107:0/1959526244 shutdown_connections 2024-03-04T13:42:34.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:34.924+0000 7fb18fe00700 1 --2- 172.21.15.107:0/1959526244 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1901001a0 0x7fb190102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:42:34.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:34.924+0000 7fb18fe00700 1 --2- 172.21.15.107:0/1959526244 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb1900fd880 0x7fb1900ffc60 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:42:34.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:34.924+0000 7fb18fe00700 1 -- 172.21.15.107:0/1959526244 >> 172.21.15.107:0/1959526244 conn(0x7fb1900f72c0 msgr2=0x7fb1900f9730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:42:34.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:34.924+0000 7fb18fe00700 1 -- 172.21.15.107:0/1959526244 shutdown_connections 2024-03-04T13:42:34.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:34.924+0000 7fb18fe00700 1 -- 172.21.15.107:0/1959526244 wait complete. 2024-03-04T13:42:34.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:34.925+0000 7fb18fe00700 1 Processor -- start 2024-03-04T13:42:34.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:34.925+0000 7fb18fe00700 1 -- start start 2024-03-04T13:42:34.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:34.925+0000 7fb18fe00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1900fd880 0x7fb19010f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:42:34.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:34.925+0000 7fb18fe00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb1901001a0 0x7fb19010fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:42:34.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:34.925+0000 7fb18fe00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb190110440 con 0x7fb1901001a0 2024-03-04T13:42:34.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:34.925+0000 7fb18fe00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb190110ce0 con 0x7fb1900fd880 2024-03-04T13:42:34.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:34.926+0000 7fb18e000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb1901001a0 0x7fb19010fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:42:34.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:34.926+0000 7fb18e000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb1901001a0 0x7fb19010fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43300/0 (socket says 172.21.15.107:43300) 2024-03-04T13:42:34.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:34.926+0000 7fb18e000700 1 -- 172.21.15.107:0/3608690219 learned_addr learned my addr 172.21.15.107:0/3608690219 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:42:34.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:34.926+0000 7fb18ea00700 1 --2- 172.21.15.107:0/3608690219 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1900fd880 0x7fb19010f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:42:34.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:34.926+0000 7fb18e000700 1 -- 172.21.15.107:0/3608690219 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1900fd880 msgr2=0x7fb19010f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:42:34.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:34.926+0000 7fb18e000700 1 --2- 172.21.15.107:0/3608690219 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1900fd880 0x7fb19010f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:42:34.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:34.926+0000 7fb18e000700 1 -- 172.21.15.107:0/3608690219 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb180005040 con 0x7fb1901001a0 2024-03-04T13:42:34.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:34.927+0000 7fb18e000700 1 --2- 172.21.15.107:0/3608690219 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb1901001a0 0x7fb19010fe40 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fb17c002a50 tx=0x7fb17c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:42:34.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:34.927+0000 7fb186a00700 1 -- 172.21.15.107:0/3608690219 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb17c007cc0 con 0x7fb1901001a0 2024-03-04T13:42:34.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:34.927+0000 7fb186a00700 1 -- 172.21.15.107:0/3608690219 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb17c00f040 con 0x7fb1901001a0 2024-03-04T13:42:34.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:34.927+0000 7fb186a00700 1 -- 172.21.15.107:0/3608690219 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb17c00da30 con 0x7fb1901001a0 2024-03-04T13:42:34.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:34.927+0000 7fb18fe00700 1 -- 172.21.15.107:0/3608690219 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb190110fc0 con 0x7fb1901001a0 2024-03-04T13:42:34.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:34.927+0000 7fb18fe00700 1 -- 172.21.15.107:0/3608690219 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb190111500 con 0x7fb1901001a0 2024-03-04T13:42:34.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:34.928+0000 7fb18fe00700 1 -- 172.21.15.107:0/3608690219 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1900f8e80 con 0x7fb1901001a0 2024-03-04T13:42:34.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:34.930+0000 7fb186a00700 1 -- 172.21.15.107:0/3608690219 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) v1 ==== 89013+0+0 (secure 0 0 0) 0x7fb17c00dbb0 con 0x7fb1901001a0 2024-03-04T13:42:34.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:34.930+0000 7fb186a00700 1 --2- 172.21.15.107:0/3608690219 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7fb17406b2e0 0x7fb17406d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:42:34.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:34.931+0000 7fb186a00700 1 -- 172.21.15.107:0/3608690219 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5808+0+0 (secure 0 0 0) 0x7fb17c089b10 con 0x7fb1901001a0 2024-03-04T13:42:34.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:34.931+0000 7fb18ea00700 1 --2- 172.21.15.107:0/3608690219 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7fb17406b2e0 0x7fb17406d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:42:34.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:34.932+0000 7fb18ea00700 1 --2- 172.21.15.107:0/3608690219 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7fb17406b2e0 0x7fb17406d790 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fb1800043d0 tx=0x7fb180009e30 comp rx=0 tx=0).ready entity=mgr.24331 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:42:34.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:34.935+0000 7fb186a00700 1 -- 172.21.15.107:0/3608690219 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7fb17c057700 con 0x7fb1901001a0 2024-03-04T13:42:35.080 INFO:tasks.workunit.client.1.smithi169.stderr:++ readlink -f fsstress 2024-03-04T13:42:35.086 INFO:tasks.workunit.client.1.smithi169.stderr:+ BIN=/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-03-04T13:42:35.087 INFO:tasks.workunit.client.1.smithi169.stderr:+ popd 2024-03-04T13:42:35.091 INFO:tasks.workunit.client.1.smithi169.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-03-04T13:42:35.091 INFO:tasks.workunit.client.1.smithi169.stderr:+ popd 2024-03-04T13:42:35.094 INFO:tasks.workunit.client.1.smithi169.stdout:~/cephtest/mnt.1/client.1/tmp 2024-03-04T13:42:35.095 INFO:tasks.workunit.client.1.smithi169.stderr:++ mktemp -d -p . 2024-03-04T13:42:35.101 INFO:tasks.workunit.client.1.smithi169.stderr:+ T=./tmp.a3Z4RkHcIM 2024-03-04T13:42:35.101 INFO:tasks.workunit.client.1.smithi169.stderr:+ /home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.a3Z4RkHcIM -l 1 -n 1000 -p 10 -v 2024-03-04T13:42:35.115 INFO:tasks.workunit.client.1.smithi169.stdout:seed = 1709448204 2024-03-04T13:42:35.125 INFO:tasks.workunit.client.1.smithi169.stdout:0/0: write - no filename 2024-03-04T13:42:35.126 INFO:tasks.workunit.client.1.smithi169.stdout:0/1: chown . 65 1 2024-03-04T13:42:35.126 INFO:tasks.workunit.client.1.smithi169.stdout:0/2: unlink - no file 2024-03-04T13:42:35.126 INFO:tasks.workunit.client.1.smithi169.stdout:0/3: dread - no filename 2024-03-04T13:42:35.126 INFO:tasks.workunit.client.1.smithi169.stdout:0/4: write - no filename 2024-03-04T13:42:35.129 INFO:tasks.workunit.client.1.smithi169.stdout:0/5: creat f0 x:0 0 0 2024-03-04T13:42:35.130 INFO:tasks.workunit.client.1.smithi169.stdout:0/6: chown f0 58 1 2024-03-04T13:42:35.131 INFO:tasks.workunit.client.1.smithi169.stdout:9/0: getdents . 0 2024-03-04T13:42:35.131 INFO:tasks.workunit.client.1.smithi169.stdout:9/1: rename - no filename 2024-03-04T13:42:35.131 INFO:tasks.workunit.client.1.smithi169.stdout:9/2: read - no filename 2024-03-04T13:42:35.131 INFO:tasks.workunit.client.1.smithi169.stdout:9/3: dwrite - no filename 2024-03-04T13:42:35.133 INFO:tasks.workunit.client.1.smithi169.stdout:1/0: dwrite - no filename 2024-03-04T13:42:35.133 INFO:tasks.workunit.client.1.smithi169.stdout:1/1: chown . 102362 1 2024-03-04T13:42:35.135 INFO:tasks.workunit.client.1.smithi169.stdout:9/4: getdents . 0 2024-03-04T13:42:35.135 INFO:tasks.workunit.client.1.smithi169.stdout:9/5: dread - no filename 2024-03-04T13:42:35.136 INFO:tasks.workunit.client.1.smithi169.stdout:2/0: dread - no filename 2024-03-04T13:42:35.136 INFO:tasks.workunit.client.1.smithi169.stdout:2/1: dwrite - no filename 2024-03-04T13:42:35.137 INFO:tasks.workunit.client.1.smithi169.stdout:1/2: creat f0 x:0 0 0 2024-03-04T13:42:35.137 INFO:tasks.workunit.client.1.smithi169.stdout:1/3: dread - f0 zero size 2024-03-04T13:42:35.140 INFO:tasks.workunit.client.1.smithi169.stdout:3/0: truncate - no filename 2024-03-04T13:42:35.141 INFO:tasks.workunit.client.1.smithi169.stdout:3/1: chown . 2513038 1 2024-03-04T13:42:35.141 INFO:tasks.workunit.client.1.smithi169.stdout:3/2: rmdir - no directory 2024-03-04T13:42:35.141 INFO:tasks.workunit.client.1.smithi169.stdout:3/3: readlink - no filename 2024-03-04T13:42:35.141 INFO:tasks.workunit.client.1.smithi169.stdout:3/4: fdatasync - no filename 2024-03-04T13:42:35.141 INFO:tasks.workunit.client.1.smithi169.stdout:9/6: symlink l0 0 2024-03-04T13:42:35.142 INFO:tasks.workunit.client.1.smithi169.stdout:1/4: rename f0 to f1 0 2024-03-04T13:42:35.142 INFO:tasks.workunit.client.1.smithi169.stdout:2/2: creat f0 x:0 0 0 2024-03-04T13:42:35.143 INFO:tasks.workunit.client.1.smithi169.stdout:1/5: chown f1 2 1 2024-03-04T13:42:35.144 INFO:tasks.workunit.client.1.smithi169.stdout:2/3: stat f0 0 2024-03-04T13:42:35.146 INFO:tasks.workunit.client.1.smithi169.stdout:9/7: creat f1 x:0 0 0 2024-03-04T13:42:35.146 INFO:tasks.workunit.client.1.smithi169.stdout:2/4: write f0 [28339,59048] 0 2024-03-04T13:42:35.147 INFO:tasks.workunit.client.1.smithi169.stdout:3/5: creat f0 x:0 0 0 2024-03-04T13:42:35.150 INFO:tasks.workunit.client.1.smithi169.stdout:3/6: write f0 [836632,89104] 0 2024-03-04T13:42:35.151 INFO:tasks.workunit.client.1.smithi169.stdout:5/0: dread - no filename 2024-03-04T13:42:35.151 INFO:tasks.workunit.client.1.smithi169.stdout:5/1: dwrite - no filename 2024-03-04T13:42:35.151 INFO:tasks.workunit.client.1.smithi169.stdout:1/6: mknod c2 0 2024-03-04T13:42:35.151 INFO:tasks.workunit.client.1.smithi169.stdout:5/2: chown . 13646182 1 2024-03-04T13:42:35.151 INFO:tasks.workunit.client.1.smithi169.stdout:5/3: truncate - no filename 2024-03-04T13:42:35.151 INFO:tasks.workunit.client.1.smithi169.stdout:5/4: dwrite - no filename 2024-03-04T13:42:35.152 INFO:tasks.workunit.client.1.smithi169.stdout:5/5: chown . 29273 1 2024-03-04T13:42:35.154 INFO:tasks.workunit.client.1.smithi169.stdout:5/6: chown . 14 1 2024-03-04T13:42:35.156 INFO:tasks.workunit.client.1.smithi169.stdout:2/5: mkdir d1 0 2024-03-04T13:42:35.158 INFO:tasks.workunit.client.1.smithi169.stdout:6/0: stat - no entries 2024-03-04T13:42:35.159 INFO:tasks.workunit.client.1.smithi169.stdout:4/0: creat f0 x:0 0 0 2024-03-04T13:42:35.161 INFO:tasks.workunit.client.1.smithi169.stdout:4/1: chown f0 731986921 1 2024-03-04T13:42:35.164 INFO:tasks.workunit.client.1.smithi169.stdout:4/2: chown f0 1 1 2024-03-04T13:42:35.171 INFO:tasks.workunit.client.1.smithi169.stdout:1/7: write f1 [695675,46543] 0 2024-03-04T13:42:35.175 INFO:tasks.workunit.client.1.smithi169.stdout:5/7: creat f0 x:0 0 0 2024-03-04T13:42:35.175 INFO:tasks.workunit.client.1.smithi169.stdout:4/3: rename f0 to f1 0 2024-03-04T13:42:35.185 INFO:tasks.workunit.client.1.smithi169.stdout:1/8: chown c2 215402398 1 2024-03-04T13:42:35.193 INFO:tasks.workunit.client.1.smithi169.stdout:7/0: mknod c0 0 2024-03-04T13:42:35.194 INFO:tasks.workunit.client.1.smithi169.stdout:7/1: write - no filename 2024-03-04T13:42:35.203 INFO:tasks.workunit.client.1.smithi169.stdout:9/8: dwrite f1 [0,4194304] 0 2024-03-04T13:42:35.203 INFO:tasks.workunit.client.1.smithi169.stdout:7/2: rename c0 to c1 0 2024-03-04T13:42:35.203 INFO:tasks.workunit.client.1.smithi169.stdout:7/3: dwrite - no filename 2024-03-04T13:42:35.204 INFO:tasks.workunit.client.1.smithi169.stdout:7/4: truncate - no filename 2024-03-04T13:42:35.204 INFO:tasks.workunit.client.1.smithi169.stdout:2/6: dwrite f0 [0,4194304] 0 2024-03-04T13:42:35.209 INFO:tasks.workunit.client.1.smithi169.stdout:3/7: dwrite f0 [0,4194304] 0 2024-03-04T13:42:35.211 INFO:tasks.workunit.client.1.smithi169.stdout:9/9: write f1 [3415969,103333] 0 2024-03-04T13:42:35.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:35.217+0000 7fb18fe00700 1 -- 172.21.15.107:0/3608690219 --> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fb1901118f0 con 0x7fb17406b2e0 2024-03-04T13:42:35.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:35.242+0000 7fb186a00700 1 -- 172.21.15.107:0/3608690219 <== mgr.24331 v2:172.21.15.169:6828/3524931243 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3388 (secure 0 0 0) 0x7fb1901118f0 con 0x7fb17406b2e0 2024-03-04T13:42:35.241 INFO:teuthology.orchestra.run.smithi107.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-04T13:42:35.242 INFO:teuthology.orchestra.run.smithi107.stdout:alertmanager.smithi107 smithi107 *:9093,9094 starting - - - - 2024-03-04T13:42:35.242 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi107 smithi107 running (11m) 59s ago 11m 6665k - 17.2.7-545-g42608bb1 9eec3f22859d 03e1570c2eff 2024-03-04T13:42:35.242 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi169 smithi169 running (9m) 58s ago 9m 6656k - 17.2.7-545-g42608bb1 9eec3f22859d 9c8881a5338e 2024-03-04T13:42:35.242 INFO:teuthology.orchestra.run.smithi107.stdout:grafana.smithi107 smithi107 *:3000 starting - - - - 2024-03-04T13:42:35.242 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.riexmm smithi107 running (4m) 59s ago 3m 11.4M - 17.2.7-545-g42608bb1 9eec3f22859d fc921f7d4328 2024-03-04T13:42:35.242 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.thycho smithi107 running (3m) 59s ago 3m 85.9M - 17.2.7-545-g42608bb1 9eec3f22859d 4069c6e0e063 2024-03-04T13:42:35.242 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi169.bxitzk smithi169 running (3m) 58s ago 3m 11.5M - 17.2.7-545-g42608bb1 9eec3f22859d 7d4115228c7c 2024-03-04T13:42:35.242 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi169.jusudl smithi169 running (3m) 58s ago 3m 101M - 17.2.7-545-g42608bb1 9eec3f22859d 2d2f2cc452e8 2024-03-04T13:42:35.242 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi107.wpzgwl smithi107 *:9283 running (12m) 59s ago 12m 367M - 17.2.7-545-g42608bb1 9eec3f22859d 4831dce06263 2024-03-04T13:42:35.242 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi169.eihczr smithi169 *:8443,9283 running (85s) 58s ago 9m 489M - 19.0.0-1845-g717ce3cc d71b839abdf8 18b4dfa896b5 2024-03-04T13:42:35.242 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi107 smithi107 running (12m) 59s ago 12m 50.0M 2048M 17.2.7-545-g42608bb1 9eec3f22859d 49f5da990d35 2024-03-04T13:42:35.242 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi169 smithi169 running (9m) 58s ago 9m 38.2M 2048M 17.2.7-545-g42608bb1 9eec3f22859d ba4ee916297f 2024-03-04T13:42:35.242 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi107 smithi107 *:9100 starting - - - - 2024-03-04T13:42:35.243 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi169 smithi169 *:9100 running (9m) 58s ago 9m 7801k - 1.5.0 0da6a335fe13 b2dae07f2da2 2024-03-04T13:42:35.243 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0 smithi107 running (8m) 59s ago 8m 64.8M 1558M 17.2.7-545-g42608bb1 9eec3f22859d c0605b68f696 2024-03-04T13:42:35.243 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1 smithi107 running (8m) 59s ago 8m 64.4M 1558M 17.2.7-545-g42608bb1 9eec3f22859d 5fb2c0dd5769 2024-03-04T13:42:35.243 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2 smithi107 running (7m) 59s ago 7m 59.0M 1558M 17.2.7-545-g42608bb1 9eec3f22859d 61f30f643549 2024-03-04T13:42:35.243 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3 smithi169 running (6m) 58s ago 6m 69.7M 2582M 17.2.7-545-g42608bb1 9eec3f22859d cd91d8694764 2024-03-04T13:42:35.243 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4 smithi169 running (6m) 58s ago 6m 64.6M 2582M 17.2.7-545-g42608bb1 9eec3f22859d 08545e3d0659 2024-03-04T13:42:35.243 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5 smithi169 running (5m) 58s ago 5m 62.3M 2582M 17.2.7-545-g42608bb1 9eec3f22859d dcffe6cb3eac 2024-03-04T13:42:35.243 INFO:teuthology.orchestra.run.smithi107.stdout:prometheus.smithi107 smithi107 *:9095 running (8m) 59s ago 10m 51.3M - 2.43.0 a07b618ecd1d 226d4ba6d37a 2024-03-04T13:42:35.244 INFO:tasks.workunit.client.1.smithi169.stdout:2/7: dwrite f0 [0,4194304] 0 2024-03-04T13:42:35.244 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:35.246+0000 7fb18fe00700 1 -- 172.21.15.107:0/3608690219 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7fb17406b2e0 msgr2=0x7fb17406d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:42:35.244 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:35.246+0000 7fb18fe00700 1 --2- 172.21.15.107:0/3608690219 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7fb17406b2e0 0x7fb17406d790 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fb1800043d0 tx=0x7fb180009e30 comp rx=0 tx=0).stop 2024-03-04T13:42:35.244 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:35.246+0000 7fb18fe00700 1 -- 172.21.15.107:0/3608690219 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb1901001a0 msgr2=0x7fb19010fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:42:35.244 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:35.246+0000 7fb18fe00700 1 --2- 172.21.15.107:0/3608690219 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb1901001a0 0x7fb19010fe40 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fb17c002a50 tx=0x7fb17c0046b0 comp rx=0 tx=0).stop 2024-03-04T13:42:35.244 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:35.246+0000 7fb18fe00700 1 -- 172.21.15.107:0/3608690219 shutdown_connections 2024-03-04T13:42:35.244 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:35.246+0000 7fb18fe00700 1 --2- 172.21.15.107:0/3608690219 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7fb17406b2e0 0x7fb17406d790 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:42:35.244 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:35.246+0000 7fb18fe00700 1 --2- 172.21.15.107:0/3608690219 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1900fd880 0x7fb19010f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:42:35.244 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:35.246+0000 7fb18fe00700 1 --2- 172.21.15.107:0/3608690219 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb1901001a0 0x7fb19010fe40 secure :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fb17c002a50 tx=0x7fb17c0046b0 comp rx=0 tx=0).stop 2024-03-04T13:42:35.244 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:35.246+0000 7fb18fe00700 1 -- 172.21.15.107:0/3608690219 >> 172.21.15.107:0/3608690219 conn(0x7fb1900f72c0 msgr2=0x7fb190100e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:42:35.245 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:35.247+0000 7fb18fe00700 1 -- 172.21.15.107:0/3608690219 shutdown_connections 2024-03-04T13:42:35.245 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:35.248+0000 7fb18fe00700 1 -- 172.21.15.107:0/3608690219 wait complete. 2024-03-04T13:42:35.250 INFO:tasks.workunit.client.1.smithi169.stdout:2/8: creat d1/f2 x:0 0 0 2024-03-04T13:42:35.253 INFO:tasks.workunit.client.1.smithi169.stdout:2/9: creat d1/f3 x:0 0 0 2024-03-04T13:42:35.256 INFO:tasks.workunit.client.1.smithi169.stdout:2/10: mknod d1/c4 0 2024-03-04T13:42:35.261 INFO:tasks.workunit.client.1.smithi169.stdout:2/11: mkdir d1/d5 0 2024-03-04T13:42:35.264 INFO:tasks.workunit.client.1.smithi169.stdout:2/12: symlink d1/l6 0 2024-03-04T13:42:35.267 INFO:tasks.workunit.client.1.smithi169.stdout:2/13: symlink d1/d5/l7 0 2024-03-04T13:42:35.303 INFO:tasks.workunit.client.1.smithi169.stdout:0/7: fsync f0 0 2024-03-04T13:42:35.306 INFO:tasks.workunit.client.1.smithi169.stdout:0/8: rename f0 to f1 0 2024-03-04T13:42:35.309 INFO:tasks.workunit.client.1.smithi169.stdout:0/9: write f1 [638273,58708] 0 2024-03-04T13:42:35.312 INFO:tasks.workunit.client.1.smithi169.stdout:9/10: fsync f1 0 2024-03-04T13:42:35.318 INFO:tasks.workunit.client.1.smithi169.stdout:5/8: fsync f0 0 2024-03-04T13:42:35.318 INFO:tasks.workunit.client.1.smithi169.stdout:0/10: dread f1 [0,4194304] 0 2024-03-04T13:42:35.320 INFO:tasks.workunit.client.1.smithi169.stdout:4/4: unlink f1 0 2024-03-04T13:42:35.321 INFO:tasks.workunit.client.1.smithi169.stdout:5/9: truncate f0 577107 0 2024-03-04T13:42:35.324 INFO:tasks.workunit.client.1.smithi169.stdout:4/5: creat f2 x:0 0 0 2024-03-04T13:42:35.325 INFO:tasks.workunit.client.1.smithi169.stdout:5/10: dread f0 [0,4194304] 0 2024-03-04T13:42:35.330 INFO:tasks.workunit.client.1.smithi169.stdout:7/5: getdents . 0 2024-03-04T13:42:35.333 INFO:tasks.workunit.client.1.smithi169.stdout:9/11: dwrite f1 [0,4194304] 0 2024-03-04T13:42:35.334 INFO:tasks.workunit.client.1.smithi169.stdout:7/6: chown c1 156324360 1 2024-03-04T13:42:35.334 INFO:tasks.workunit.client.1.smithi169.stdout:3/8: truncate f0 3301754 0 2024-03-04T13:42:35.335 INFO:tasks.workunit.client.1.smithi169.stdout:3/9: rmdir - no directory 2024-03-04T13:42:35.339 INFO:tasks.workunit.client.1.smithi169.stdout:7/7: mkdir d2 0 2024-03-04T13:42:35.339 INFO:tasks.workunit.client.1.smithi169.stdout:7/8: truncate - no filename 2024-03-04T13:42:35.339 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:42:35 smithi107 ceph-mon[28739]: pgmap v39: 65 pgs: 65 active+clean; 200 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.3 MiB/s rd, 2.6 MiB/s wr, 254 op/s 2024-03-04T13:42:35.345 INFO:tasks.workunit.client.1.smithi169.stdout:3/10: creat f1 x:0 0 0 2024-03-04T13:42:35.345 INFO:tasks.workunit.client.1.smithi169.stdout:3/11: readlink - no filename 2024-03-04T13:42:35.346 INFO:tasks.workunit.client.1.smithi169.stdout:2/14: rmdir d1 39 2024-03-04T13:42:35.349 INFO:tasks.workunit.client.1.smithi169.stdout:7/9: symlink d2/l3 0 2024-03-04T13:42:35.351 INFO:tasks.workunit.client.1.smithi169.stdout:3/12: dread - f1 zero size 2024-03-04T13:42:35.354 INFO:tasks.workunit.client.1.smithi169.stdout:2/15: unlink f0 0 2024-03-04T13:42:35.355 INFO:tasks.workunit.client.1.smithi169.stdout:0/11: dwrite f1 [0,4194304] 0 2024-03-04T13:42:35.357 INFO:tasks.workunit.client.1.smithi169.stdout:7/10: unlink c1 0 2024-03-04T13:42:35.357 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:42:35 smithi169 ceph-mon[36313]: pgmap v39: 65 pgs: 65 active+clean; 200 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.3 MiB/s rd, 2.6 MiB/s wr, 254 op/s 2024-03-04T13:42:35.362 INFO:tasks.workunit.client.1.smithi169.stdout:9/12: dread f1 [0,4194304] 0 2024-03-04T13:42:35.364 INFO:tasks.workunit.client.1.smithi169.stdout:4/6: getdents . 0 2024-03-04T13:42:35.364 INFO:tasks.workunit.client.1.smithi169.stdout:5/11: dwrite f0 [0,4194304] 0 2024-03-04T13:42:35.366 INFO:tasks.workunit.client.1.smithi169.stdout:9/13: chown l0 1186338 1 2024-03-04T13:42:35.369 INFO:tasks.workunit.client.1.smithi169.stdout:3/13: creat f2 x:0 0 0 2024-03-04T13:42:35.371 INFO:tasks.workunit.client.1.smithi169.stdout:0/12: mknod c2 0 2024-03-04T13:42:35.371 INFO:tasks.workunit.client.1.smithi169.stdout:7/11: rmdir d2 39 2024-03-04T13:42:35.373 INFO:tasks.workunit.client.1.smithi169.stdout:5/12: mkdir d1 0 2024-03-04T13:42:35.375 INFO:tasks.workunit.client.1.smithi169.stdout:4/7: unlink f2 0 2024-03-04T13:42:35.375 INFO:tasks.workunit.client.1.smithi169.stdout:4/8: truncate - no filename 2024-03-04T13:42:35.377 INFO:tasks.workunit.client.1.smithi169.stdout:9/14: symlink l2 0 2024-03-04T13:42:35.381 INFO:tasks.workunit.client.1.smithi169.stdout:0/13: mknod c3 0 2024-03-04T13:42:35.388 INFO:tasks.workunit.client.1.smithi169.stdout:4/9: creat f3 x:0 0 0 2024-03-04T13:42:35.391 INFO:tasks.workunit.client.1.smithi169.stdout:3/14: link f0 f3 0 2024-03-04T13:42:35.391 INFO:tasks.workunit.client.1.smithi169.stdout:2/16: dwrite d1/f3 [0,4194304] 0 2024-03-04T13:42:35.392 INFO:tasks.workunit.client.1.smithi169.stdout:3/15: read - f1 zero size 2024-03-04T13:42:35.393 INFO:tasks.workunit.client.1.smithi169.stdout:9/15: rename f1 to f3 0 2024-03-04T13:42:35.398 INFO:tasks.workunit.client.1.smithi169.stdout:0/14: symlink l4 0 2024-03-04T13:42:35.399 INFO:tasks.workunit.client.1.smithi169.stdout:7/12: creat d2/f4 x:0 0 0 2024-03-04T13:42:35.399 INFO:tasks.workunit.client.1.smithi169.stdout:5/13: getdents d1 0 2024-03-04T13:42:35.400 INFO:tasks.workunit.client.1.smithi169.stdout:4/10: mkdir d4 0 2024-03-04T13:42:35.400 INFO:tasks.workunit.client.1.smithi169.stdout:5/14: fdatasync f0 0 2024-03-04T13:42:35.401 INFO:tasks.workunit.client.1.smithi169.stdout:7/13: read - d2/f4 zero size 2024-03-04T13:42:35.403 INFO:tasks.workunit.client.1.smithi169.stdout:7/14: truncate d2/f4 44740 0 2024-03-04T13:42:35.406 INFO:tasks.workunit.client.1.smithi169.stdout:3/16: rename f3 to f4 0 2024-03-04T13:42:35.407 INFO:tasks.workunit.client.1.smithi169.stdout:0/15: mkdir d5 0 2024-03-04T13:42:35.416 INFO:tasks.workunit.client.1.smithi169.stdout:5/15: creat d1/f2 x:0 0 0 2024-03-04T13:42:35.418 INFO:tasks.workunit.client.1.smithi169.stdout:0/16: creat d5/f6 x:0 0 0 2024-03-04T13:42:35.421 INFO:tasks.workunit.client.1.smithi169.stdout:4/11: fsync f3 0 2024-03-04T13:42:35.422 INFO:tasks.workunit.client.1.smithi169.stdout:0/17: fdatasync f1 0 2024-03-04T13:42:35.424 INFO:tasks.workunit.client.1.smithi169.stdout:7/15: link d2/l3 d2/l5 0 2024-03-04T13:42:35.430 INFO:tasks.workunit.client.1.smithi169.stdout:7/16: write d2/f4 [982188,52318] 0 2024-03-04T13:42:35.431 INFO:tasks.workunit.client.1.smithi169.stdout:0/18: rename f1 to d5/f7 0 2024-03-04T13:42:35.433 INFO:tasks.workunit.client.1.smithi169.stdout:9/16: dwrite f3 [4194304,4194304] 0 2024-03-04T13:42:35.433 INFO:tasks.workunit.client.1.smithi169.stdout:4/12: creat d4/f5 x:0 0 0 2024-03-04T13:42:35.438 INFO:tasks.workunit.client.1.smithi169.stdout:4/13: chown f3 534730208 1 2024-03-04T13:42:35.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:35.442+0000 7fb87e800700 1 -- 172.21.15.107:0/3546277686 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb878070860 msgr2=0x7fb878070cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:42:35.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:35.442+0000 7fb87e800700 1 --2- 172.21.15.107:0/3546277686 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb878070860 0x7fb878070cd0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fb864004940 tx=0x7fb864009e30 comp rx=0 tx=0).stop 2024-03-04T13:42:35.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:35.443+0000 7fb87e800700 1 -- 172.21.15.107:0/3546277686 shutdown_connections 2024-03-04T13:42:35.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:35.443+0000 7fb87e800700 1 --2- 172.21.15.107:0/3546277686 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb878070860 0x7fb878070cd0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:42:35.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:35.443+0000 7fb87e800700 1 --2- 172.21.15.107:0/3546277686 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb878071eb0 0x7fb878070320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:42:35.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:35.443+0000 7fb87e800700 1 -- 172.21.15.107:0/3546277686 >> 172.21.15.107:0/3546277686 conn(0x7fb8780f8440 msgr2=0x7fb8780fa890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:42:35.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:35.444+0000 7fb87e800700 1 -- 172.21.15.107:0/3546277686 shutdown_connections 2024-03-04T13:42:35.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:35.444+0000 7fb87e800700 1 -- 172.21.15.107:0/3546277686 wait complete. 2024-03-04T13:42:35.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:35.445+0000 7fb87e800700 1 Processor -- start 2024-03-04T13:42:35.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:35.446+0000 7fb87e800700 1 -- start start 2024-03-04T13:42:35.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:35.446+0000 7fb87e800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb878070860 0x7fb8780fcc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:42:35.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:35.446+0000 7fb87e800700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb878071eb0 0x7fb8780fd160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:42:35.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:35.446+0000 7fb87e800700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8780fd7b0 con 0x7fb878070860 2024-03-04T13:42:35.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:35.446+0000 7fb87e800700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8780fe050 con 0x7fb878071eb0 2024-03-04T13:42:35.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:35.447+0000 7fb87d400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb878070860 0x7fb8780fcc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:42:35.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:35.447+0000 7fb87d400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb878070860 0x7fb8780fcc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43310/0 (socket says 172.21.15.107:43310) 2024-03-04T13:42:35.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:35.447+0000 7fb87d400700 1 -- 172.21.15.107:0/405889542 learned_addr learned my addr 172.21.15.107:0/405889542 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:42:35.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:35.447+0000 7fb87d400700 1 -- 172.21.15.107:0/405889542 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb878071eb0 msgr2=0x7fb8780fd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:42:35.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:35.447+0000 7fb87d400700 1 --2- 172.21.15.107:0/405889542 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb878071eb0 0x7fb8780fd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:42:35.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:35.447+0000 7fb87d400700 1 -- 172.21.15.107:0/405889542 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb864005040 con 0x7fb878070860 2024-03-04T13:42:35.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:35.448+0000 7fb87d400700 1 --2- 172.21.15.107:0/405889542 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb878070860 0x7fb8780fcc20 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fb86c00eb40 tx=0x7fb86c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:42:35.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:35.448+0000 7fb875600700 1 -- 172.21.15.107:0/405889542 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb86c007cc0 con 0x7fb878070860 2024-03-04T13:42:35.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:35.448+0000 7fb875600700 1 -- 172.21.15.107:0/405889542 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb86c00f040 con 0x7fb878070860 2024-03-04T13:42:35.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:35.448+0000 7fb875600700 1 -- 172.21.15.107:0/405889542 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb86c009620 con 0x7fb878070860 2024-03-04T13:42:35.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:35.449+0000 7fb87e800700 1 -- 172.21.15.107:0/405889542 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8780fe2d0 con 0x7fb878070860 2024-03-04T13:42:35.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:35.449+0000 7fb87e800700 1 -- 172.21.15.107:0/405889542 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8780fe840 con 0x7fb878070860 2024-03-04T13:42:35.447 INFO:tasks.workunit.client.1.smithi169.stdout:0/19: rename d5/f7 to d5/f8 0 2024-03-04T13:42:35.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:35.451+0000 7fb87e800700 1 -- 172.21.15.107:0/405889542 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb87804bed0 con 0x7fb878070860 2024-03-04T13:42:35.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:35.452+0000 7fb875600700 1 -- 172.21.15.107:0/405889542 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) v1 ==== 89013+0+0 (secure 0 0 0) 0x7fb86c009780 con 0x7fb878070860 2024-03-04T13:42:35.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:35.453+0000 7fb875600700 1 --2- 172.21.15.107:0/405889542 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7fb86006b2e0 0x7fb86006d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:42:35.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:35.453+0000 7fb875600700 1 -- 172.21.15.107:0/405889542 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5808+0+0 (secure 0 0 0) 0x7fb86c089fb0 con 0x7fb878070860 2024-03-04T13:42:35.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:35.454+0000 7fb87ca00700 1 --2- 172.21.15.107:0/405889542 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7fb86006b2e0 0x7fb86006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:42:35.451 INFO:tasks.workunit.client.1.smithi169.stdout:5/16: dread f0 [0,4194304] 0 2024-03-04T13:42:35.451 INFO:tasks.workunit.client.1.smithi169.stdout:0/20: read - d5/f6 zero size 2024-03-04T13:42:35.451 INFO:tasks.workunit.client.1.smithi169.stdout:7/17: dread d2/f4 [0,4194304] 0 2024-03-04T13:42:35.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:35.455+0000 7fb87ca00700 1 --2- 172.21.15.107:0/405889542 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7fb86006b2e0 0x7fb86006d790 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fb864009fd0 tx=0x7fb864009e30 comp rx=0 tx=0).ready entity=mgr.24331 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:42:35.454 INFO:tasks.workunit.client.1.smithi169.stdout:0/21: chown d5 51922 1 2024-03-04T13:42:35.454 INFO:tasks.workunit.client.1.smithi169.stdout:5/17: fsync d1/f2 0 2024-03-04T13:42:35.454 INFO:tasks.workunit.client.1.smithi169.stdout:2/17: dwrite d1/f3 [4194304,4194304] 0 2024-03-04T13:42:35.458 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:35.461+0000 7fb875600700 1 -- 172.21.15.107:0/405889542 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7fb86c053f80 con 0x7fb878070860 2024-03-04T13:42:35.461 INFO:tasks.workunit.client.1.smithi169.stdout:9/17: truncate f3 7288327 0 2024-03-04T13:42:35.462 INFO:tasks.workunit.client.1.smithi169.stdout:7/18: creat d2/f6 x:0 0 0 2024-03-04T13:42:35.462 INFO:tasks.workunit.client.1.smithi169.stdout:9/18: readlink l0 0 2024-03-04T13:42:35.465 INFO:tasks.workunit.client.1.smithi169.stdout:5/18: symlink d1/l3 0 2024-03-04T13:42:35.466 INFO:tasks.workunit.client.1.smithi169.stdout:2/18: creat d1/d5/f8 x:0 0 0 2024-03-04T13:42:35.469 INFO:tasks.workunit.client.1.smithi169.stdout:7/19: creat d2/f7 x:0 0 0 2024-03-04T13:42:35.474 INFO:tasks.workunit.client.1.smithi169.stdout:4/14: dwrite f3 [0,4194304] 0 2024-03-04T13:42:35.481 INFO:tasks.workunit.client.1.smithi169.stdout:4/15: write f3 [986360,7139] 0 2024-03-04T13:42:35.487 INFO:tasks.workunit.client.1.smithi169.stdout:7/20: rmdir d2 39 2024-03-04T13:42:35.490 INFO:tasks.workunit.client.1.smithi169.stdout:0/22: dwrite d5/f6 [0,4194304] 0 2024-03-04T13:42:35.495 INFO:tasks.workunit.client.1.smithi169.stdout:7/21: stat d2/f7 0 2024-03-04T13:42:35.502 INFO:tasks.workunit.client.1.smithi169.stdout:9/19: dread f3 [4194304,4194304] 0 2024-03-04T13:42:35.502 INFO:tasks.workunit.client.1.smithi169.stdout:7/22: rename d2/f6 to d2/f8 0 2024-03-04T13:42:35.503 INFO:tasks.workunit.client.1.smithi169.stdout:5/19: dwrite d1/f2 [0,4194304] 0 2024-03-04T13:42:35.504 INFO:tasks.workunit.client.1.smithi169.stdout:9/20: chown l0 27 1 2024-03-04T13:42:35.512 INFO:tasks.workunit.client.1.smithi169.stdout:5/20: rename f0 to d1/f4 0 2024-03-04T13:42:35.512 INFO:tasks.workunit.client.1.smithi169.stdout:7/23: creat d2/f9 x:0 0 0 2024-03-04T13:42:35.522 INFO:tasks.workunit.client.1.smithi169.stdout:0/23: fsync d5/f6 0 2024-03-04T13:42:35.527 INFO:tasks.workunit.client.1.smithi169.stdout:0/24: mkdir d5/d9 0 2024-03-04T13:42:35.528 INFO:tasks.workunit.client.1.smithi169.stdout:5/21: rmdir d1 39 2024-03-04T13:42:35.536 INFO:tasks.workunit.client.1.smithi169.stdout:9/21: dwrite f3 [4194304,4194304] 0 2024-03-04T13:42:35.538 INFO:tasks.workunit.client.1.smithi169.stdout:7/24: dwrite d2/f7 [0,4194304] 0 2024-03-04T13:42:35.538 INFO:tasks.workunit.client.1.smithi169.stdout:0/25: link d5/f8 d5/fa 0 2024-03-04T13:42:35.547 INFO:tasks.workunit.client.1.smithi169.stdout:0/26: creat d5/fb x:0 0 0 2024-03-04T13:42:35.550 INFO:tasks.workunit.client.1.smithi169.stdout:0/27: read d5/f8 [422247,90405] 0 2024-03-04T13:42:35.553 INFO:tasks.workunit.client.1.smithi169.stdout:5/22: dread d1/f4 [0,4194304] 0 2024-03-04T13:42:35.558 INFO:tasks.workunit.client.1.smithi169.stdout:5/23: unlink d1/f2 0 2024-03-04T13:42:35.563 INFO:tasks.workunit.client.1.smithi169.stdout:9/22: truncate f3 5698118 0 2024-03-04T13:42:35.565 INFO:tasks.workunit.client.1.smithi169.stdout:5/24: unlink d1/l3 0 2024-03-04T13:42:35.570 INFO:tasks.workunit.client.1.smithi169.stdout:7/25: dwrite d2/f8 [0,4194304] 0 2024-03-04T13:42:35.573 INFO:tasks.workunit.client.1.smithi169.stdout:0/28: dread d5/fa [0,4194304] 0 2024-03-04T13:42:35.574 INFO:tasks.workunit.client.1.smithi169.stdout:9/23: fsync f3 0 2024-03-04T13:42:35.576 INFO:tasks.workunit.client.1.smithi169.stdout:7/26: dread d2/f4 [0,4194304] 0 2024-03-04T13:42:35.577 INFO:tasks.workunit.client.1.smithi169.stdout:7/27: dread - d2/f9 zero size 2024-03-04T13:42:35.579 INFO:tasks.workunit.client.1.smithi169.stdout:7/28: write d2/f7 [434586,6378] 0 2024-03-04T13:42:35.581 INFO:tasks.workunit.client.1.smithi169.stdout:9/24: truncate f3 543880 0 2024-03-04T13:42:35.583 INFO:tasks.workunit.client.1.smithi169.stdout:9/25: chown f3 47 1 2024-03-04T13:42:35.583 INFO:tasks.workunit.client.1.smithi169.stdout:5/25: dread d1/f4 [0,4194304] 0 2024-03-04T13:42:35.583 INFO:tasks.workunit.client.1.smithi169.stdout:7/29: write d2/f8 [1624474,123480] 0 2024-03-04T13:42:35.588 INFO:tasks.workunit.client.1.smithi169.stdout:7/30: rename d2/f4 to d2/fa 0 2024-03-04T13:42:35.591 INFO:tasks.workunit.client.1.smithi169.stdout:0/29: write d5/fa [575117,58361] 0 2024-03-04T13:42:35.593 INFO:tasks.workunit.client.1.smithi169.stdout:9/26: getdents . 0 2024-03-04T13:42:35.595 INFO:tasks.workunit.client.1.smithi169.stdout:7/31: mknod d2/cb 0 2024-03-04T13:42:35.597 INFO:tasks.workunit.client.1.smithi169.stdout:5/26: dread d1/f4 [0,4194304] 0 2024-03-04T13:42:35.601 INFO:tasks.workunit.client.1.smithi169.stdout:0/30: mkdir d5/d9/dc 0 2024-03-04T13:42:35.602 INFO:tasks.workunit.client.1.smithi169.stdout:7/32: fsync d2/fa 0 2024-03-04T13:42:35.602 INFO:tasks.workunit.client.1.smithi169.stdout:9/27: getdents . 0 2024-03-04T13:42:35.604 INFO:tasks.workunit.client.1.smithi169.stdout:5/27: write d1/f4 [69890,52966] 0 2024-03-04T13:42:35.604 INFO:tasks.workunit.client.1.smithi169.stdout:0/31: mknod d5/cd 0 2024-03-04T13:42:35.605 INFO:tasks.workunit.client.1.smithi169.stdout:0/32: stat d5/d9 0 2024-03-04T13:42:35.606 INFO:tasks.workunit.client.1.smithi169.stdout:7/33: symlink d2/lc 0 2024-03-04T13:42:35.608 INFO:tasks.workunit.client.1.smithi169.stdout:0/33: write d5/f8 [5054064,64466] 0 2024-03-04T13:42:35.608 INFO:tasks.workunit.client.1.smithi169.stdout:7/34: write d2/f8 [4813695,38272] 0 2024-03-04T13:42:35.611 INFO:tasks.workunit.client.1.smithi169.stdout:5/28: unlink d1/f4 0 2024-03-04T13:42:35.611 INFO:tasks.workunit.client.1.smithi169.stdout:5/29: write - no filename 2024-03-04T13:42:35.611 INFO:tasks.workunit.client.1.smithi169.stdout:5/30: write - no filename 2024-03-04T13:42:35.611 INFO:tasks.workunit.client.1.smithi169.stdout:5/31: fdatasync - no filename 2024-03-04T13:42:35.611 INFO:tasks.workunit.client.1.smithi169.stdout:5/32: truncate - no filename 2024-03-04T13:42:35.611 INFO:tasks.workunit.client.1.smithi169.stdout:5/33: dread - no filename 2024-03-04T13:42:35.611 INFO:tasks.workunit.client.1.smithi169.stdout:5/34: fdatasync - no filename 2024-03-04T13:42:35.611 INFO:tasks.workunit.client.1.smithi169.stdout:5/35: write - no filename 2024-03-04T13:42:35.612 INFO:tasks.workunit.client.1.smithi169.stdout:7/35: write d2/f7 [2544311,95810] 0 2024-03-04T13:42:35.612 INFO:tasks.workunit.client.1.smithi169.stdout:5/36: unlink - no file 2024-03-04T13:42:35.612 INFO:tasks.workunit.client.1.smithi169.stdout:0/34: chown d5/f8 1736207 1 2024-03-04T13:42:35.612 INFO:tasks.workunit.client.1.smithi169.stdout:5/37: chown d1 102583 1 2024-03-04T13:42:35.616 INFO:tasks.workunit.client.1.smithi169.stdout:7/36: fsync d2/f7 0 2024-03-04T13:42:35.618 INFO:tasks.workunit.client.1.smithi169.stdout:7/37: write d2/f9 [757308,11198] 0 2024-03-04T13:42:35.619 INFO:tasks.workunit.client.1.smithi169.stdout:5/38: creat d1/f5 x:0 0 0 2024-03-04T13:42:35.619 INFO:tasks.workunit.client.1.smithi169.stdout:7/38: stat d2/fa 0 2024-03-04T13:42:35.621 INFO:tasks.workunit.client.1.smithi169.stdout:0/35: rename c3 to d5/d9/ce 0 2024-03-04T13:42:35.623 INFO:tasks.workunit.client.1.smithi169.stdout:9/28: write f3 [1070792,63163] 0 2024-03-04T13:42:35.624 INFO:tasks.workunit.client.1.smithi169.stdout:7/39: creat d2/fd x:0 0 0 2024-03-04T13:42:35.626 INFO:tasks.workunit.client.1.smithi169.stdout:5/39: rename d1/f5 to d1/f6 0 2024-03-04T13:42:35.626 INFO:tasks.workunit.client.1.smithi169.stdout:7/40: write d2/fd [500960,67203] 0 2024-03-04T13:42:35.627 INFO:tasks.workunit.client.1.smithi169.stdout:9/29: creat f4 x:0 0 0 2024-03-04T13:42:35.627 INFO:tasks.workunit.client.1.smithi169.stdout:5/40: read - d1/f6 zero size 2024-03-04T13:42:35.629 INFO:tasks.workunit.client.1.smithi169.stdout:9/30: rename l2 to l5 0 2024-03-04T13:42:35.630 INFO:tasks.workunit.client.1.smithi169.stdout:7/41: mkdir d2/de 0 2024-03-04T13:42:35.632 INFO:tasks.workunit.client.1.smithi169.stdout:9/31: write f4 [872781,37848] 0 2024-03-04T13:42:35.632 INFO:tasks.workunit.client.1.smithi169.stdout:5/41: mknod d1/c7 0 2024-03-04T13:42:35.633 INFO:tasks.workunit.client.1.smithi169.stdout:5/42: chown d1 31 1 2024-03-04T13:42:35.634 INFO:tasks.workunit.client.1.smithi169.stdout:0/36: dread d5/f6 [0,4194304] 0 2024-03-04T13:42:35.635 INFO:tasks.workunit.client.1.smithi169.stdout:9/32: creat f6 x:0 0 0 2024-03-04T13:42:35.636 INFO:tasks.workunit.client.1.smithi169.stdout:7/42: rename d2/f9 to d2/de/ff 0 2024-03-04T13:42:35.638 INFO:tasks.workunit.client.1.smithi169.stdout:5/43: write d1/f6 [241494,57376] 0 2024-03-04T13:42:35.641 INFO:tasks.workunit.client.1.smithi169.stdout:5/44: truncate d1/f6 1261273 0 2024-03-04T13:42:35.655 INFO:tasks.workunit.client.1.smithi169.stdout:7/43: dwrite d2/fd [0,4194304] 0 2024-03-04T13:42:35.659 INFO:tasks.workunit.client.1.smithi169.stdout:7/44: mknod d2/c10 0 2024-03-04T13:42:35.663 INFO:tasks.workunit.client.1.smithi169.stdout:7/45: symlink d2/de/l11 0 2024-03-04T13:42:35.664 INFO:tasks.workunit.client.1.smithi169.stdout:5/45: getdents d1 0 2024-03-04T13:42:35.664 INFO:tasks.workunit.client.1.smithi169.stdout:5/46: stat d1 0 2024-03-04T13:42:35.668 INFO:tasks.workunit.client.1.smithi169.stdout:5/47: creat d1/f8 x:0 0 0 2024-03-04T13:42:35.670 INFO:tasks.workunit.client.1.smithi169.stdout:5/48: chown d1 31710 1 2024-03-04T13:42:35.673 INFO:tasks.workunit.client.1.smithi169.stdout:9/33: truncate f3 722674 0 2024-03-04T13:42:35.676 INFO:tasks.workunit.client.1.smithi169.stdout:7/46: dread d2/f7 [0,4194304] 0 2024-03-04T13:42:35.677 INFO:tasks.workunit.client.1.smithi169.stdout:9/34: mkdir d7 0 2024-03-04T13:42:35.683 INFO:tasks.workunit.client.1.smithi169.stdout:7/47: creat d2/de/f12 x:0 0 0 2024-03-04T13:42:35.690 INFO:tasks.workunit.client.1.smithi169.stdout:7/48: symlink d2/de/l13 0 2024-03-04T13:42:35.690 INFO:tasks.workunit.client.1.smithi169.stdout:5/49: dwrite d1/f8 [0,4194304] 0 2024-03-04T13:42:35.690 INFO:tasks.workunit.client.1.smithi169.stdout:7/49: readlink d2/lc 0 2024-03-04T13:42:35.697 INFO:tasks.workunit.client.1.smithi169.stdout:7/50: unlink d2/c10 0 2024-03-04T13:42:35.701 INFO:tasks.workunit.client.1.smithi169.stdout:5/50: creat d1/f9 x:0 0 0 2024-03-04T13:42:35.702 INFO:tasks.workunit.client.1.smithi169.stdout:7/51: creat d2/f14 x:0 0 0 2024-03-04T13:42:35.707 INFO:tasks.workunit.client.1.smithi169.stdout:5/51: creat d1/fa x:0 0 0 2024-03-04T13:42:35.708 INFO:tasks.workunit.client.1.smithi169.stdout:5/52: rename d1 to d1/db 22 2024-03-04T13:42:35.717 INFO:tasks.workunit.client.1.smithi169.stdout:7/52: dread d2/f8 [0,4194304] 0 2024-03-04T13:42:35.722 INFO:tasks.workunit.client.1.smithi169.stdout:7/53: mkdir d2/d15 0 2024-03-04T13:42:35.725 INFO:tasks.workunit.client.1.smithi169.stdout:7/54: dread - d2/de/f12 zero size 2024-03-04T13:42:35.731 INFO:tasks.workunit.client.1.smithi169.stdout:7/55: unlink d2/f8 0 2024-03-04T13:42:35.732 INFO:tasks.workunit.client.1.smithi169.stdout:5/53: dwrite d1/fa [0,4194304] 0 2024-03-04T13:42:35.747 INFO:tasks.workunit.client.1.smithi169.stdout:5/54: dread d1/f6 [0,4194304] 0 2024-03-04T13:42:35.748 INFO:tasks.workunit.client.1.smithi169.stdout:5/55: chown d1/f6 297 1 2024-03-04T13:42:35.751 INFO:tasks.workunit.client.1.smithi169.stdout:5/56: symlink d1/lc 0 2024-03-04T13:42:35.752 INFO:tasks.workunit.client.1.smithi169.stdout:5/57: chown d1/fa 30731040 1 2024-03-04T13:42:35.753 INFO:tasks.workunit.client.1.smithi169.stdout:5/58: chown d1/f6 26 1 2024-03-04T13:42:35.754 INFO:tasks.workunit.client.1.smithi169.stdout:5/59: stat d1/f8 0 2024-03-04T13:42:35.755 INFO:tasks.workunit.client.1.smithi169.stdout:5/60: chown d1/f8 15 1 2024-03-04T13:42:35.757 INFO:tasks.workunit.client.1.smithi169.stdout:9/35: write f3 [1063259,112157] 0 2024-03-04T13:42:35.764 INFO:tasks.workunit.client.1.smithi169.stdout:9/36: rename l0 to d7/l8 0 2024-03-04T13:42:35.767 INFO:tasks.workunit.client.1.smithi169.stdout:5/61: dread d1/fa [0,4194304] 0 2024-03-04T13:42:35.772 INFO:tasks.workunit.client.1.smithi169.stdout:7/56: rmdir d2 39 2024-03-04T13:42:35.773 INFO:tasks.workunit.client.1.smithi169.stdout:5/62: symlink d1/ld 0 2024-03-04T13:42:35.784 INFO:tasks.workunit.client.1.smithi169.stdout:7/57: mknod d2/d15/c16 0 2024-03-04T13:42:35.786 INFO:tasks.workunit.client.1.smithi169.stdout:9/37: dwrite f4 [0,4194304] 0 2024-03-04T13:42:35.795 INFO:tasks.workunit.client.1.smithi169.stdout:5/63: dwrite d1/f8 [0,4194304] 0 2024-03-04T13:42:35.799 INFO:tasks.workunit.client.1.smithi169.stdout:9/38: link f6 d7/f9 0 2024-03-04T13:42:35.800 INFO:tasks.workunit.client.1.smithi169.stdout:5/64: write d1/f8 [2015939,119942] 0 2024-03-04T13:42:35.805 INFO:tasks.workunit.client.1.smithi169.stdout:9/39: rename f3 to d7/fa 0 2024-03-04T13:42:35.807 INFO:tasks.workunit.client.1.smithi169.stdout:5/65: mknod d1/ce 0 2024-03-04T13:42:35.811 INFO:tasks.workunit.client.1.smithi169.stdout:5/66: symlink d1/lf 0 2024-03-04T13:42:35.812 INFO:tasks.workunit.client.1.smithi169.stdout:5/67: dread - d1/f9 zero size 2024-03-04T13:42:35.816 INFO:tasks.workunit.client.1.smithi169.stdout:5/68: truncate d1/f6 1251147 0 2024-03-04T13:42:35.833 INFO:tasks.workunit.client.1.smithi169.stdout:9/40: fsync d7/fa 0 2024-03-04T13:42:35.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:35.856+0000 7fb87e800700 1 -- 172.21.15.107:0/405889542 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb878107a60 con 0x7fb878070860 2024-03-04T13:42:35.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:35.857+0000 7fb875600700 1 -- 172.21.15.107:0/405889542 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+846 (secure 0 0 0) 0x7fb86c057640 con 0x7fb878070860 2024-03-04T13:42:35.854 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-03-04T13:42:35.854 INFO:teuthology.orchestra.run.smithi107.stdout: "mon": { 2024-03-04T13:42:35.855 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-545-g42608bb1 (42608bb191117be10dba720e6bf8a02f3171cd34) quincy (stable)": 2 2024-03-04T13:42:35.855 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:42:35.855 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr": { 2024-03-04T13:42:35.855 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-545-g42608bb1 (42608bb191117be10dba720e6bf8a02f3171cd34) quincy (stable)": 1, 2024-03-04T13:42:35.855 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-1845-g717ce3cc (717ce3cce8d0166e4e577faa8004e6f9cb4128c0) squid (dev)": 1 2024-03-04T13:42:35.855 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:42:35.855 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": { 2024-03-04T13:42:35.855 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-545-g42608bb1 (42608bb191117be10dba720e6bf8a02f3171cd34) quincy (stable)": 6 2024-03-04T13:42:35.855 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:42:35.855 INFO:teuthology.orchestra.run.smithi107.stdout: "mds": { 2024-03-04T13:42:35.855 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-545-g42608bb1 (42608bb191117be10dba720e6bf8a02f3171cd34) quincy (stable)": 4 2024-03-04T13:42:35.855 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:42:35.855 INFO:teuthology.orchestra.run.smithi107.stdout: "overall": { 2024-03-04T13:42:35.855 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-545-g42608bb1 (42608bb191117be10dba720e6bf8a02f3171cd34) quincy (stable)": 13, 2024-03-04T13:42:35.855 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-1845-g717ce3cc (717ce3cce8d0166e4e577faa8004e6f9cb4128c0) squid (dev)": 1 2024-03-04T13:42:35.855 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-04T13:42:35.856 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-03-04T13:42:35.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:35.860+0000 7fb87e800700 1 -- 172.21.15.107:0/405889542 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7fb86006b2e0 msgr2=0x7fb86006d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:42:35.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:35.860+0000 7fb87e800700 1 --2- 172.21.15.107:0/405889542 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7fb86006b2e0 0x7fb86006d790 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fb864009fd0 tx=0x7fb864009e30 comp rx=0 tx=0).stop 2024-03-04T13:42:35.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:35.860+0000 7fb87e800700 1 -- 172.21.15.107:0/405889542 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb878070860 msgr2=0x7fb8780fcc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:42:35.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:35.860+0000 7fb87e800700 1 --2- 172.21.15.107:0/405889542 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb878070860 0x7fb8780fcc20 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fb86c00eb40 tx=0x7fb86c00b040 comp rx=0 tx=0).stop 2024-03-04T13:42:35.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:35.860+0000 7fb87e800700 1 -- 172.21.15.107:0/405889542 shutdown_connections 2024-03-04T13:42:35.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:35.860+0000 7fb87e800700 1 --2- 172.21.15.107:0/405889542 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7fb86006b2e0 0x7fb86006d790 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:42:35.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:35.860+0000 7fb87e800700 1 --2- 172.21.15.107:0/405889542 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb878070860 0x7fb8780fcc20 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:42:35.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:35.860+0000 7fb87e800700 1 --2- 172.21.15.107:0/405889542 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb878071eb0 0x7fb8780fd160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:42:35.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:35.860+0000 7fb87e800700 1 -- 172.21.15.107:0/405889542 >> 172.21.15.107:0/405889542 conn(0x7fb8780f8440 msgr2=0x7fb878102c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:42:35.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:35.861+0000 7fb87e800700 1 -- 172.21.15.107:0/405889542 shutdown_connections 2024-03-04T13:42:35.859 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:35.861+0000 7fb87e800700 1 -- 172.21.15.107:0/405889542 wait complete. 2024-03-04T13:42:35.886 INFO:tasks.workunit.client.1.smithi169.stdout:9/41: creat d7/fb x:0 0 0 2024-03-04T13:42:35.889 INFO:tasks.workunit.client.1.smithi169.stdout:9/42: mkdir d7/dc 0 2024-03-04T13:42:35.890 INFO:tasks.workunit.client.1.smithi169.stdout:9/43: fdatasync f4 0 2024-03-04T13:42:35.892 INFO:tasks.workunit.client.1.smithi169.stdout:5/69: rmdir d1 39 2024-03-04T13:42:35.894 INFO:tasks.workunit.client.1.smithi169.stdout:5/70: dread - d1/f9 zero size 2024-03-04T13:42:35.896 INFO:tasks.workunit.client.1.smithi169.stdout:9/44: rename d7/f9 to d7/dc/fd 0 2024-03-04T13:42:35.897 INFO:tasks.workunit.client.1.smithi169.stdout:5/71: mknod d1/c10 0 2024-03-04T13:42:35.901 INFO:tasks.workunit.client.1.smithi169.stdout:9/45: truncate d7/fa 380292 0 2024-03-04T13:42:35.904 INFO:tasks.workunit.client.1.smithi169.stdout:9/46: creat d7/fe x:0 0 0 2024-03-04T13:42:35.905 INFO:tasks.workunit.client.1.smithi169.stdout:9/47: write d7/fe [401852,23576] 0 2024-03-04T13:42:35.907 INFO:tasks.workunit.client.1.smithi169.stdout:9/48: read - d7/fb zero size 2024-03-04T13:42:35.911 INFO:tasks.workunit.client.1.smithi169.stdout:9/49: symlink d7/dc/lf 0 2024-03-04T13:42:35.913 INFO:tasks.workunit.client.1.smithi169.stdout:5/72: dwrite d1/f8 [0,4194304] 0 2024-03-04T13:42:35.915 INFO:tasks.workunit.client.1.smithi169.stdout:5/73: write d1/f9 [530489,118532] 0 2024-03-04T13:42:35.955 INFO:tasks.workunit.client.1.smithi169.stdout:9/50: getdents d7/dc 0 2024-03-04T13:42:35.957 INFO:tasks.workunit.client.1.smithi169.stdout:9/51: creat d7/f10 x:0 0 0 2024-03-04T13:42:35.958 INFO:tasks.workunit.client.1.smithi169.stdout:9/52: read - d7/fb zero size 2024-03-04T13:42:35.962 INFO:tasks.workunit.client.1.smithi169.stdout:9/53: rename d7/fa to d7/f11 0 2024-03-04T13:42:36.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:36.041+0000 7f407e200700 1 -- 172.21.15.107:0/3558183668 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40780fdf70 msgr2=0x7f40780fe3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:42:36.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:36.041+0000 7f407e200700 1 --2- 172.21.15.107:0/3558183668 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40780fdf70 0x7f40780fe3c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f4068004940 tx=0x7f4068009e30 comp rx=0 tx=0).stop 2024-03-04T13:42:36.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:36.041+0000 7f407e200700 1 -- 172.21.15.107:0/3558183668 shutdown_connections 2024-03-04T13:42:36.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:36.041+0000 7f407e200700 1 --2- 172.21.15.107:0/3558183668 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40780fdf70 0x7f40780fe3c0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:42:36.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:36.041+0000 7f407e200700 1 --2- 172.21.15.107:0/3558183668 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40780fcd60 0x7f40780fd170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:42:36.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:36.041+0000 7f407e200700 1 -- 172.21.15.107:0/3558183668 >> 172.21.15.107:0/3558183668 conn(0x7f40780f83e0 msgr2=0x7f40780fa850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:42:36.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:36.041+0000 7f407e200700 1 -- 172.21.15.107:0/3558183668 shutdown_connections 2024-03-04T13:42:36.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:36.042+0000 7f407e200700 1 -- 172.21.15.107:0/3558183668 wait complete. 2024-03-04T13:42:36.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:36.042+0000 7f407e200700 1 Processor -- start 2024-03-04T13:42:36.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:36.043+0000 7f407e200700 1 -- start start 2024-03-04T13:42:36.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:36.043+0000 7f407e200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40780fcd60 0x7f407810f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:42:36.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:36.043+0000 7f407e200700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40780fdf70 0x7f407810fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:42:36.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:36.043+0000 7f407e200700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40781104f0 con 0x7f40780fcd60 2024-03-04T13:42:36.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:36.043+0000 7f407e200700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4078110d90 con 0x7f40780fdf70 2024-03-04T13:42:36.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:36.043+0000 7f407ce00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40780fcd60 0x7f407810f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:42:36.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:36.043+0000 7f407ce00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40780fcd60 0x7f407810f9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43334/0 (socket says 172.21.15.107:43334) 2024-03-04T13:42:36.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:36.043+0000 7f407ce00700 1 -- 172.21.15.107:0/3021847497 learned_addr learned my addr 172.21.15.107:0/3021847497 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:42:36.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:36.043+0000 7f4077e00700 1 --2- 172.21.15.107:0/3021847497 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40780fdf70 0x7f407810fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:42:36.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:36.044+0000 7f407ce00700 1 -- 172.21.15.107:0/3021847497 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40780fdf70 msgr2=0x7f407810fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:42:36.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:36.044+0000 7f407ce00700 1 --2- 172.21.15.107:0/3021847497 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40780fdf70 0x7f407810fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:42:36.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:36.044+0000 7f407ce00700 1 -- 172.21.15.107:0/3021847497 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4068005040 con 0x7f40780fcd60 2024-03-04T13:42:36.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:36.044+0000 7f407ce00700 1 --2- 172.21.15.107:0/3021847497 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40780fcd60 0x7f407810f9b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f406c002a50 tx=0x7f406c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:42:36.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:36.044+0000 7f4074c00700 1 -- 172.21.15.107:0/3021847497 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f406c007cc0 con 0x7f40780fcd60 2024-03-04T13:42:36.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:36.044+0000 7f4074c00700 1 -- 172.21.15.107:0/3021847497 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f406c00f040 con 0x7f40780fcd60 2024-03-04T13:42:36.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:36.044+0000 7f407e200700 1 -- 172.21.15.107:0/3021847497 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4078111070 con 0x7f40780fcd60 2024-03-04T13:42:36.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:36.044+0000 7f4074c00700 1 -- 172.21.15.107:0/3021847497 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f406c00da40 con 0x7f40780fcd60 2024-03-04T13:42:36.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:36.045+0000 7f407e200700 1 -- 172.21.15.107:0/3021847497 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40781115b0 con 0x7f40780fcd60 2024-03-04T13:42:36.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:36.046+0000 7f407e200700 1 -- 172.21.15.107:0/3021847497 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4078102010 con 0x7f40780fcd60 2024-03-04T13:42:36.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:36.047+0000 7f4074c00700 1 -- 172.21.15.107:0/3021847497 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) v1 ==== 89013+0+0 (secure 0 0 0) 0x7f406c00dc00 con 0x7f40780fcd60 2024-03-04T13:42:36.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:36.048+0000 7f4074c00700 1 --2- 172.21.15.107:0/3021847497 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7f406006b2e0 0x7f406006d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:42:36.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:36.048+0000 7f4074c00700 1 -- 172.21.15.107:0/3021847497 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f406c089ad0 con 0x7f40780fcd60 2024-03-04T13:42:36.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:36.049+0000 7f4077e00700 1 --2- 172.21.15.107:0/3021847497 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7f406006b2e0 0x7f406006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:42:36.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:36.050+0000 7f4077e00700 1 --2- 172.21.15.107:0/3021847497 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7f406006b2e0 0x7f406006d790 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f406800b020 tx=0x7f4068009e30 comp rx=0 tx=0).ready entity=mgr.24331 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:42:36.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:36.054+0000 7f4074c00700 1 -- 172.21.15.107:0/3021847497 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f406c057610 con 0x7f40780fcd60 2024-03-04T13:42:36.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:36.332+0000 7f407e200700 1 -- 172.21.15.107:0/3021847497 --> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4078111990 con 0x7f406006b2e0 2024-03-04T13:42:36.330 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:42:36 smithi107 ceph-mon[28739]: from='client.14656 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:42:36.330 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:42:36 smithi107 ceph-mon[28739]: from='client.14660 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:42:36.330 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:42:36 smithi107 ceph-mon[28739]: from='client.? 172.21.15.107:0/405889542' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:42:36.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:36.336+0000 7f4074c00700 1 -- 172.21.15.107:0/3021847497 <== mgr.24331 v2:172.21.15.169:6828/3524931243 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f4078111990 con 0x7f406006b2e0 2024-03-04T13:42:36.333 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-03-04T13:42:36.333 INFO:teuthology.orchestra.run.smithi107.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377", 2024-03-04T13:42:36.333 INFO:teuthology.orchestra.run.smithi107.stdout: "in_progress": true, 2024-03-04T13:42:36.333 INFO:teuthology.orchestra.run.smithi107.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-03-04T13:42:36.333 INFO:teuthology.orchestra.run.smithi107.stdout: "services_complete": [], 2024-03-04T13:42:36.333 INFO:teuthology.orchestra.run.smithi107.stdout: "progress": "1/2 daemons upgraded", 2024-03-04T13:42:36.333 INFO:teuthology.orchestra.run.smithi107.stdout: "message": "", 2024-03-04T13:42:36.333 INFO:teuthology.orchestra.run.smithi107.stdout: "is_paused": false 2024-03-04T13:42:36.334 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-03-04T13:42:36.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:36.340+0000 7f407e200700 1 -- 172.21.15.107:0/3021847497 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7f406006b2e0 msgr2=0x7f406006d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:42:36.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:36.340+0000 7f407e200700 1 --2- 172.21.15.107:0/3021847497 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7f406006b2e0 0x7f406006d790 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f406800b020 tx=0x7f4068009e30 comp rx=0 tx=0).stop 2024-03-04T13:42:36.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:36.340+0000 7f407e200700 1 -- 172.21.15.107:0/3021847497 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40780fcd60 msgr2=0x7f407810f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:42:36.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:36.340+0000 7f407e200700 1 --2- 172.21.15.107:0/3021847497 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40780fcd60 0x7f407810f9b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f406c002a50 tx=0x7f406c004600 comp rx=0 tx=0).stop 2024-03-04T13:42:36.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:36.340+0000 7f407e200700 1 -- 172.21.15.107:0/3021847497 shutdown_connections 2024-03-04T13:42:36.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:36.340+0000 7f407e200700 1 --2- 172.21.15.107:0/3021847497 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7f406006b2e0 0x7f406006d790 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:42:36.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:36.340+0000 7f407e200700 1 --2- 172.21.15.107:0/3021847497 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40780fcd60 0x7f407810f9b0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:42:36.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:36.340+0000 7f407e200700 1 --2- 172.21.15.107:0/3021847497 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40780fdf70 0x7f407810fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:42:36.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:36.340+0000 7f407e200700 1 -- 172.21.15.107:0/3021847497 >> 172.21.15.107:0/3021847497 conn(0x7f40780f83e0 msgr2=0x7f40780ff190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:42:36.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:36.340+0000 7f407e200700 1 -- 172.21.15.107:0/3021847497 shutdown_connections 2024-03-04T13:42:36.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:42:36.340+0000 7f407e200700 1 -- 172.21.15.107:0/3021847497 wait complete. 2024-03-04T13:42:36.357 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:42:36 smithi169 ceph-mon[36313]: from='client.14656 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:42:36.357 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:42:36 smithi169 ceph-mon[36313]: from='client.14660 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:42:36.357 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:42:36 smithi169 ceph-mon[36313]: from='client.? 172.21.15.107:0/405889542' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:42:37.340 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:42:37 smithi107 ceph-mon[28739]: from='client.14664 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:42:37.340 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:42:37 smithi107 ceph-mon[28739]: pgmap v40: 65 pgs: 65 active+clean; 200 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 696 KiB/s rd, 1.9 MiB/s wr, 167 op/s 2024-03-04T13:42:37.357 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:42:37 smithi169 ceph-mon[36313]: from='client.14664 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:42:37.357 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:42:37 smithi169 ceph-mon[36313]: pgmap v40: 65 pgs: 65 active+clean; 200 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 696 KiB/s rd, 1.9 MiB/s wr, 167 op/s 2024-03-04T13:42:38.340 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:42:38 smithi107 ceph-mon[28739]: from='client.14672 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:42:38.340 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:42:38 smithi107 ceph-mon[28739]: from='mgr.24331 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:42:38.357 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:42:38 smithi169 ceph-mon[36313]: from='client.14672 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:42:38.357 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:42:38 smithi169 ceph-mon[36313]: from='mgr.24331 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:42:38.736 INFO:tasks.workunit.client.1.smithi169.stdout:6/1: sync 2024-03-04T13:42:38.736 INFO:tasks.workunit.client.1.smithi169.stdout:6/2: dwrite - no filename 2024-03-04T13:42:38.736 INFO:tasks.workunit.client.1.smithi169.stdout:6/3: stat - no entries 2024-03-04T13:42:38.747 INFO:tasks.workunit.client.1.smithi169.stdout:6/4: mkdir d0 0 2024-03-04T13:42:38.756 INFO:tasks.workunit.client.1.smithi169.stdout:6/5: creat d0/f1 x:0 0 0 2024-03-04T13:42:38.764 INFO:tasks.workunit.client.1.smithi169.stdout:6/6: symlink d0/l2 0 2024-03-04T13:42:38.956 INFO:tasks.workunit.client.1.smithi169.stdout:1/9: sync 2024-03-04T13:42:38.956 INFO:tasks.workunit.client.1.smithi169.stdout:0/37: sync 2024-03-04T13:42:38.958 INFO:tasks.workunit.client.1.smithi169.stdout:0/38: chown d5/d9/dc 648484 1 2024-03-04T13:42:38.960 INFO:tasks.workunit.client.1.smithi169.stdout:0/39: rename d5 to d5/df 22 2024-03-04T13:42:38.978 INFO:tasks.workunit.client.1.smithi169.stdout:0/40: dread d5/f6 [0,4194304] 0 2024-03-04T13:42:39.000 INFO:tasks.workunit.client.1.smithi169.stdout:0/41: dwrite d5/f8 [0,4194304] 0 2024-03-04T13:42:39.006 INFO:tasks.workunit.client.1.smithi169.stdout:0/42: mknod d5/d9/dc/c10 0 2024-03-04T13:42:39.007 INFO:tasks.workunit.client.1.smithi169.stdout:0/43: read - d5/fb zero size 2024-03-04T13:42:39.022 INFO:tasks.workunit.client.1.smithi169.stdout:0/44: read d5/f8 [4307400,113834] 0 2024-03-04T13:42:39.024 INFO:tasks.workunit.client.1.smithi169.stdout:0/45: dread - d5/fb zero size 2024-03-04T13:42:39.028 INFO:tasks.workunit.client.1.smithi169.stdout:0/46: write d5/f8 [2175987,120369] 0 2024-03-04T13:42:39.032 INFO:tasks.workunit.client.1.smithi169.stdout:0/47: mknod d5/d9/dc/c11 0 2024-03-04T13:42:39.035 INFO:tasks.workunit.client.1.smithi169.stdout:0/48: truncate d5/fb 648397 0 2024-03-04T13:42:39.057 INFO:tasks.workunit.client.1.smithi169.stdout:0/49: dwrite d5/f6 [0,4194304] 0 2024-03-04T13:42:39.061 INFO:tasks.workunit.client.1.smithi169.stdout:0/50: mkdir d5/d9/dc/d12 0 2024-03-04T13:42:39.077 INFO:tasks.workunit.client.1.smithi169.stdout:0/51: dread d5/f6 [0,4194304] 0 2024-03-04T13:42:39.078 INFO:tasks.workunit.client.1.smithi169.stdout:0/52: readlink l4 0 2024-03-04T13:42:39.082 INFO:tasks.workunit.client.1.smithi169.stdout:0/53: symlink d5/l13 0 2024-03-04T13:42:39.085 INFO:tasks.workunit.client.1.smithi169.stdout:0/54: mknod d5/c14 0 2024-03-04T13:42:39.088 INFO:tasks.workunit.client.1.smithi169.stdout:0/55: write d5/f6 [1231066,77813] 0 2024-03-04T13:42:39.091 INFO:tasks.workunit.client.1.smithi169.stdout:6/7: getdents d0 0 2024-03-04T13:42:39.109 INFO:tasks.workunit.client.1.smithi169.stdout:0/56: dwrite d5/f6 [0,4194304] 0 2024-03-04T13:42:39.110 INFO:tasks.workunit.client.1.smithi169.stdout:6/8: dwrite d0/f1 [0,4194304] 0 2024-03-04T13:42:39.116 INFO:tasks.workunit.client.1.smithi169.stdout:0/57: mkdir d5/d9/dc/d12/d15 0 2024-03-04T13:42:39.120 INFO:tasks.workunit.client.1.smithi169.stdout:0/58: write d5/fb [1186951,81311] 0 2024-03-04T13:42:39.123 INFO:tasks.workunit.client.1.smithi169.stdout:8/0: sync 2024-03-04T13:42:39.123 INFO:tasks.workunit.client.1.smithi169.stdout:8/1: unlink - no file 2024-03-04T13:42:39.123 INFO:tasks.workunit.client.1.smithi169.stdout:8/2: link - no file 2024-03-04T13:42:39.123 INFO:tasks.workunit.client.1.smithi169.stdout:3/17: sync 2024-03-04T13:42:39.124 INFO:tasks.workunit.client.1.smithi169.stdout:8/3: chown . 244 1 2024-03-04T13:42:39.124 INFO:tasks.workunit.client.1.smithi169.stdout:8/4: unlink - no file 2024-03-04T13:42:39.124 INFO:tasks.workunit.client.1.smithi169.stdout:8/5: dread - no filename 2024-03-04T13:42:39.124 INFO:tasks.workunit.client.1.smithi169.stdout:8/6: readlink - no filename 2024-03-04T13:42:39.126 INFO:tasks.workunit.client.1.smithi169.stdout:0/59: creat d5/f16 x:0 0 0 2024-03-04T13:42:39.130 INFO:tasks.workunit.client.1.smithi169.stdout:3/18: unlink f0 0 2024-03-04T13:42:39.135 INFO:tasks.workunit.client.1.smithi169.stdout:8/7: mkdir d0 0 2024-03-04T13:42:39.136 INFO:tasks.workunit.client.1.smithi169.stdout:6/9: dwrite d0/f1 [0,4194304] 0 2024-03-04T13:42:39.137 INFO:tasks.workunit.client.1.smithi169.stdout:3/19: creat f5 x:0 0 0 2024-03-04T13:42:39.144 INFO:tasks.workunit.client.1.smithi169.stdout:6/10: symlink d0/l3 0 2024-03-04T13:42:39.144 INFO:tasks.workunit.client.1.smithi169.stdout:0/60: dread d5/f8 [0,4194304] 0 2024-03-04T13:42:39.146 INFO:tasks.workunit.client.1.smithi169.stdout:3/20: fsync f4 0 2024-03-04T13:42:39.147 INFO:tasks.workunit.client.1.smithi169.stdout:5/74: sync 2024-03-04T13:42:39.147 INFO:tasks.workunit.client.1.smithi169.stdout:7/58: sync 2024-03-04T13:42:39.148 INFO:tasks.workunit.client.1.smithi169.stdout:4/16: sync 2024-03-04T13:42:39.148 INFO:tasks.workunit.client.1.smithi169.stdout:9/54: sync 2024-03-04T13:42:39.148 INFO:tasks.workunit.client.1.smithi169.stdout:1/10: sync 2024-03-04T13:42:39.148 INFO:tasks.workunit.client.1.smithi169.stdout:2/19: sync 2024-03-04T13:42:39.148 INFO:tasks.workunit.client.1.smithi169.stdout:3/21: dread - f1 zero size 2024-03-04T13:42:39.149 INFO:tasks.workunit.client.1.smithi169.stdout:8/8: rmdir d0 0 2024-03-04T13:42:39.150 INFO:tasks.workunit.client.1.smithi169.stdout:7/59: fsync d2/f14 0 2024-03-04T13:42:39.154 INFO:tasks.workunit.client.1.smithi169.stdout:3/22: chown f4 7920315 1 2024-03-04T13:42:39.154 INFO:tasks.workunit.client.1.smithi169.stdout:4/17: write f3 [2053192,105216] 0 2024-03-04T13:42:39.154 INFO:tasks.workunit.client.1.smithi169.stdout:3/23: chown f4 1070412 1 2024-03-04T13:42:39.154 INFO:tasks.workunit.client.1.smithi169.stdout:7/60: write d2/fa [472174,6609] 0 2024-03-04T13:42:39.158 INFO:tasks.workunit.client.1.smithi169.stdout:0/61: symlink d5/l17 0 2024-03-04T13:42:39.161 INFO:tasks.workunit.client.1.smithi169.stdout:9/55: mkdir d7/dc/d12 0 2024-03-04T13:42:39.162 INFO:tasks.workunit.client.1.smithi169.stdout:0/62: rename d5 to d5/d9/dc/d18 22 2024-03-04T13:42:39.164 INFO:tasks.workunit.client.1.smithi169.stdout:5/75: symlink d1/l11 0 2024-03-04T13:42:39.171 INFO:tasks.workunit.client.1.smithi169.stdout:8/9: symlink l1 0 2024-03-04T13:42:39.171 INFO:tasks.workunit.client.1.smithi169.stdout:8/10: truncate - no filename 2024-03-04T13:42:39.174 INFO:tasks.workunit.client.1.smithi169.stdout:2/20: dread d1/f3 [0,4194304] 0 2024-03-04T13:42:39.176 INFO:tasks.workunit.client.1.smithi169.stdout:3/24: rename f2 to f6 0 2024-03-04T13:42:39.182 INFO:tasks.workunit.client.1.smithi169.stdout:5/76: mknod d1/c12 0 2024-03-04T13:42:39.189 INFO:tasks.workunit.client.1.smithi169.stdout:3/25: mkdir d7 0 2024-03-04T13:42:39.192 INFO:tasks.workunit.client.1.smithi169.stdout:6/11: dwrite d0/f1 [0,4194304] 0 2024-03-04T13:42:39.196 INFO:tasks.workunit.client.1.smithi169.stdout:2/21: creat d1/d5/f9 x:0 0 0 2024-03-04T13:42:39.196 INFO:tasks.workunit.client.1.smithi169.stdout:1/11: dwrite f1 [0,4194304] 0 2024-03-04T13:42:39.199 INFO:tasks.workunit.client.1.smithi169.stdout:0/63: mkdir d5/d9/dc/d12/d15/d19 0 2024-03-04T13:42:39.201 INFO:tasks.workunit.client.1.smithi169.stdout:7/61: dwrite d2/f14 [0,4194304] 0 2024-03-04T13:42:39.201 INFO:tasks.workunit.client.1.smithi169.stdout:4/18: dwrite f3 [0,4194304] 0 2024-03-04T13:42:39.208 INFO:tasks.workunit.client.1.smithi169.stdout:3/26: fsync f5 0 2024-03-04T13:42:39.208 INFO:tasks.workunit.client.1.smithi169.stdout:6/12: mknod d0/c4 0 2024-03-04T13:42:39.211 INFO:tasks.workunit.client.1.smithi169.stdout:8/11: link l1 l2 0 2024-03-04T13:42:39.211 INFO:tasks.workunit.client.1.smithi169.stdout:8/12: write - no filename 2024-03-04T13:42:39.212 INFO:tasks.workunit.client.1.smithi169.stdout:2/22: mkdir d1/da 0 2024-03-04T13:42:39.213 INFO:tasks.workunit.client.1.smithi169.stdout:9/56: dwrite d7/f10 [0,4194304] 0 2024-03-04T13:42:39.213 INFO:tasks.workunit.client.1.smithi169.stdout:7/62: mkdir d2/d15/d17 0 2024-03-04T13:42:39.215 INFO:tasks.workunit.client.1.smithi169.stdout:9/57: chown d7/fe 0 1 2024-03-04T13:42:39.217 INFO:tasks.workunit.client.1.smithi169.stdout:2/23: rename d1/da to d1/da/db 22 2024-03-04T13:42:39.219 INFO:tasks.workunit.client.1.smithi169.stdout:2/24: write d1/f2 [871925,60710] 0 2024-03-04T13:42:39.220 INFO:tasks.workunit.client.1.smithi169.stdout:7/63: write d2/de/f12 [846747,103252] 0 2024-03-04T13:42:39.223 INFO:tasks.workunit.client.1.smithi169.stdout:7/64: chown d2 4 1 2024-03-04T13:42:39.224 INFO:tasks.workunit.client.1.smithi169.stdout:2/25: write d1/d5/f9 [887985,61957] 0 2024-03-04T13:42:39.225 INFO:tasks.workunit.client.1.smithi169.stdout:8/13: symlink l3 0 2024-03-04T13:42:39.231 INFO:tasks.workunit.client.1.smithi169.stdout:8/14: readlink l3 0 2024-03-04T13:42:39.231 INFO:tasks.workunit.client.1.smithi169.stdout:8/15: dwrite - no filename 2024-03-04T13:42:39.236 INFO:tasks.workunit.client.1.smithi169.stdout:9/58: dread d7/fe [0,4194304] 0 2024-03-04T13:42:39.239 INFO:tasks.workunit.client.1.smithi169.stdout:8/16: chown l3 115 1 2024-03-04T13:42:39.241 INFO:tasks.workunit.client.1.smithi169.stdout:6/13: dread d0/f1 [0,4194304] 0 2024-03-04T13:42:39.244 INFO:tasks.workunit.client.1.smithi169.stdout:9/59: write d7/fb [1003493,114467] 0 2024-03-04T13:42:39.245 INFO:tasks.workunit.client.1.smithi169.stdout:3/27: symlink d7/l8 0 2024-03-04T13:42:39.247 INFO:tasks.workunit.client.1.smithi169.stdout:1/12: dwrite f1 [0,4194304] 0 2024-03-04T13:42:39.248 INFO:tasks.workunit.client.1.smithi169.stdout:0/64: dwrite d5/f6 [0,4194304] 0 2024-03-04T13:42:39.251 INFO:tasks.workunit.client.1.smithi169.stdout:1/13: truncate f1 4437232 0 2024-03-04T13:42:39.251 INFO:tasks.workunit.client.1.smithi169.stdout:4/19: getdents d4 0 2024-03-04T13:42:39.254 INFO:tasks.workunit.client.1.smithi169.stdout:8/17: readlink l1 0 2024-03-04T13:42:39.254 INFO:tasks.workunit.client.1.smithi169.stdout:5/77: truncate d1/fa 4105869 0 2024-03-04T13:42:39.254 INFO:tasks.workunit.client.1.smithi169.stdout:8/18: truncate - no filename 2024-03-04T13:42:39.255 INFO:tasks.workunit.client.1.smithi169.stdout:4/20: write d4/f5 [26218,21778] 0 2024-03-04T13:42:39.256 INFO:tasks.workunit.client.1.smithi169.stdout:7/65: symlink d2/d15/d17/l18 0 2024-03-04T13:42:39.259 INFO:tasks.workunit.client.1.smithi169.stdout:7/66: fdatasync d2/de/f12 0 2024-03-04T13:42:39.259 INFO:tasks.workunit.client.1.smithi169.stdout:6/14: write d0/f1 [147396,23554] 0 2024-03-04T13:42:39.267 INFO:tasks.workunit.client.1.smithi169.stdout:3/28: creat d7/f9 x:0 0 0 2024-03-04T13:42:39.269 INFO:tasks.workunit.client.1.smithi169.stdout:0/65: creat d5/d9/f1a x:0 0 0 2024-03-04T13:42:39.270 INFO:tasks.workunit.client.1.smithi169.stdout:1/14: creat f3 x:0 0 0 2024-03-04T13:42:39.270 INFO:tasks.workunit.client.1.smithi169.stdout:5/78: dread d1/f9 [0,4194304] 0 2024-03-04T13:42:39.270 INFO:tasks.workunit.client.1.smithi169.stdout:2/26: dread d1/f2 [0,4194304] 0 2024-03-04T13:42:39.275 INFO:tasks.workunit.client.1.smithi169.stdout:8/19: rename l1 to l4 0 2024-03-04T13:42:39.277 INFO:tasks.workunit.client.1.smithi169.stdout:8/20: chown l3 137473395 1 2024-03-04T13:42:39.280 INFO:tasks.workunit.client.1.smithi169.stdout:4/21: unlink f3 0 2024-03-04T13:42:39.283 INFO:tasks.workunit.client.1.smithi169.stdout:6/15: dread d0/f1 [0,4194304] 0 2024-03-04T13:42:39.288 INFO:tasks.workunit.client.1.smithi169.stdout:4/22: dread d4/f5 [0,4194304] 0 2024-03-04T13:42:39.288 INFO:tasks.workunit.client.1.smithi169.stdout:9/60: write d7/dc/fd [449902,41523] 0 2024-03-04T13:42:39.288 INFO:tasks.workunit.client.1.smithi169.stdout:7/67: symlink d2/d15/d17/l19 0 2024-03-04T13:42:39.291 INFO:tasks.workunit.client.1.smithi169.stdout:4/23: write d4/f5 [800234,40302] 0 2024-03-04T13:42:39.296 INFO:tasks.workunit.client.1.smithi169.stdout:5/79: creat d1/f13 x:0 0 0 2024-03-04T13:42:39.296 INFO:tasks.workunit.client.1.smithi169.stdout:2/27: mknod d1/cc 0 2024-03-04T13:42:39.301 INFO:tasks.workunit.client.1.smithi169.stdout:8/21: creat f5 x:0 0 0 2024-03-04T13:42:39.303 INFO:tasks.workunit.client.1.smithi169.stdout:1/15: dwrite f3 [0,4194304] 0 2024-03-04T13:42:39.307 INFO:tasks.workunit.client.1.smithi169.stdout:9/61: symlink d7/l13 0 2024-03-04T13:42:39.308 INFO:tasks.workunit.client.1.smithi169.stdout:0/66: dwrite d5/f6 [0,4194304] 0 2024-03-04T13:42:39.313 INFO:tasks.workunit.client.1.smithi169.stdout:1/16: mkdir d4 0 2024-03-04T13:42:39.321 INFO:tasks.workunit.client.1.smithi169.stdout:8/22: chown l2 7 1 2024-03-04T13:42:39.321 INFO:tasks.workunit.client.1.smithi169.stdout:4/24: dread d4/f5 [0,4194304] 0 2024-03-04T13:42:39.323 INFO:tasks.workunit.client.1.smithi169.stdout:2/28: rmdir d1/da 0 2024-03-04T13:42:39.323 INFO:tasks.workunit.client.1.smithi169.stdout:3/29: getdents d7 0 2024-03-04T13:42:39.323 INFO:tasks.workunit.client.1.smithi169.stdout:8/23: fdatasync f5 0 2024-03-04T13:42:39.323 INFO:tasks.workunit.client.1.smithi169.stdout:4/25: chown d4 42 1 2024-03-04T13:42:39.325 INFO:tasks.workunit.client.1.smithi169.stdout:6/16: dwrite d0/f1 [0,4194304] 0 2024-03-04T13:42:39.325 INFO:tasks.workunit.client.1.smithi169.stdout:1/17: mkdir d4/d5 0 2024-03-04T13:42:39.331 INFO:tasks.workunit.client.1.smithi169.stdout:8/24: fdatasync f5 0 2024-03-04T13:42:39.331 INFO:tasks.workunit.client.1.smithi169.stdout:7/68: dread d2/fd [0,4194304] 0 2024-03-04T13:42:39.333 INFO:tasks.workunit.client.1.smithi169.stdout:1/18: write f1 [418378,14056] 0 2024-03-04T13:42:39.339 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:42:39 smithi107 ceph-mon[28739]: pgmap v41: 65 pgs: 65 active+clean; 250 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.3 MiB/s rd, 8.7 MiB/s wr, 225 op/s 2024-03-04T13:42:39.340 INFO:tasks.workunit.client.1.smithi169.stdout:2/29: dread d1/f2 [0,4194304] 0 2024-03-04T13:42:39.343 INFO:tasks.workunit.client.1.smithi169.stdout:3/30: chown f5 206 1 2024-03-04T13:42:39.343 INFO:tasks.workunit.client.1.smithi169.stdout:5/80: dwrite d1/f13 [0,4194304] 0 2024-03-04T13:42:39.353 INFO:tasks.workunit.client.1.smithi169.stdout:4/26: rename d4/f5 to d4/f6 0 2024-03-04T13:42:39.353 INFO:tasks.workunit.client.1.smithi169.stdout:8/25: creat f6 x:0 0 0 2024-03-04T13:42:39.353 INFO:tasks.workunit.client.1.smithi169.stdout:1/19: rename c2 to d4/c6 0 2024-03-04T13:42:39.353 INFO:tasks.workunit.client.1.smithi169.stdout:8/26: mkdir d7 0 2024-03-04T13:42:39.353 INFO:tasks.workunit.client.1.smithi169.stdout:3/31: getdents d7 0 2024-03-04T13:42:39.355 INFO:tasks.workunit.client.1.smithi169.stdout:9/62: dread d7/f10 [0,4194304] 0 2024-03-04T13:42:39.356 INFO:tasks.workunit.client.1.smithi169.stdout:2/30: rename d1/d5/l7 to d1/d5/ld 0 2024-03-04T13:42:39.356 INFO:tasks.workunit.client.1.smithi169.stdout:1/20: read f1 [3432407,86300] 0 2024-03-04T13:42:39.357 INFO:tasks.workunit.client.1.smithi169.stdout:3/32: fsync d7/f9 0 2024-03-04T13:42:39.360 INFO:tasks.workunit.client.1.smithi169.stdout:3/33: read - d7/f9 zero size 2024-03-04T13:42:39.364 INFO:tasks.workunit.client.1.smithi169.stdout:7/69: getdents d2/d15 0 2024-03-04T13:42:39.365 INFO:tasks.workunit.client.1.smithi169.stdout:6/17: dread d0/f1 [0,4194304] 0 2024-03-04T13:42:39.369 INFO:tasks.workunit.client.1.smithi169.stdout:2/31: creat d1/d5/fe x:0 0 0 2024-03-04T13:42:39.371 INFO:tasks.workunit.client.1.smithi169.stdout:3/34: dread - f6 zero size 2024-03-04T13:42:39.373 INFO:tasks.workunit.client.1.smithi169.stdout:6/18: fdatasync d0/f1 0 2024-03-04T13:42:39.373 INFO:tasks.workunit.client.1.smithi169.stdout:0/67: truncate d5/f8 5015982 0 2024-03-04T13:42:39.376 INFO:tasks.workunit.client.1.smithi169.stdout:7/70: mknod d2/de/c1a 0 2024-03-04T13:42:39.379 INFO:tasks.workunit.client.1.smithi169.stdout:9/63: dread d7/f10 [0,4194304] 0 2024-03-04T13:42:39.380 INFO:tasks.workunit.client.1.smithi169.stdout:0/68: chown d5/d9/dc/d12/d15/d19 121 1 2024-03-04T13:42:39.381 INFO:tasks.workunit.client.1.smithi169.stdout:1/21: rename f1 to d4/d5/f7 0 2024-03-04T13:42:39.382 INFO:tasks.workunit.client.1.smithi169.stdout:9/64: readlink d7/dc/lf 0 2024-03-04T13:42:39.383 INFO:tasks.workunit.client.1.smithi169.stdout:5/81: write d1/fa [1777678,37519] 0 2024-03-04T13:42:39.383 INFO:tasks.workunit.client.1.smithi169.stdout:3/35: read f4 [102757,6402] 0 2024-03-04T13:42:39.385 INFO:tasks.workunit.client.1.smithi169.stdout:8/27: dwrite f6 [0,4194304] 0 2024-03-04T13:42:39.387 INFO:tasks.workunit.client.1.smithi169.stdout:7/71: fsync d2/fd 0 2024-03-04T13:42:39.387 INFO:tasks.workunit.client.1.smithi169.stdout:0/69: mkdir d5/d1b 0 2024-03-04T13:42:39.394 INFO:tasks.workunit.client.1.smithi169.stdout:3/36: mkdir d7/da 0 2024-03-04T13:42:39.396 INFO:tasks.workunit.client.1.smithi169.stdout:7/72: write d2/fd [2715693,730] 0 2024-03-04T13:42:39.397 INFO:tasks.workunit.client.1.smithi169.stdout:9/65: unlink d7/l8 0 2024-03-04T13:42:39.398 INFO:tasks.workunit.client.1.smithi169.stdout:2/32: dread d1/f3 [0,4194304] 0 2024-03-04T13:42:39.401 INFO:tasks.workunit.client.1.smithi169.stdout:8/28: symlink d7/l8 0 2024-03-04T13:42:39.405 INFO:tasks.workunit.client.1.smithi169.stdout:6/19: getdents d0 0 2024-03-04T13:42:39.405 INFO:tasks.workunit.client.1.smithi169.stdout:8/29: read - f5 zero size 2024-03-04T13:42:39.405 INFO:tasks.workunit.client.1.smithi169.stdout:7/73: rename d2/lc to d2/l1b 0 2024-03-04T13:42:39.407 INFO:tasks.workunit.client.1.smithi169.stdout:1/22: dread d4/d5/f7 [0,4194304] 0 2024-03-04T13:42:39.407 INFO:tasks.workunit.client.1.smithi169.stdout:6/20: write d0/f1 [2764272,53685] 0 2024-03-04T13:42:39.407 INFO:tasks.workunit.client.1.smithi169.stdout:2/33: rename d1/f2 to d1/ff 0 2024-03-04T13:42:39.409 INFO:tasks.workunit.client.1.smithi169.stdout:9/66: write d7/f10 [1285701,23211] 0 2024-03-04T13:42:39.414 INFO:tasks.workunit.client.1.smithi169.stdout:8/30: creat d7/f9 x:0 0 0 2024-03-04T13:42:39.416 INFO:tasks.workunit.client.1.smithi169.stdout:7/74: rmdir d2/d15/d17 39 2024-03-04T13:42:39.417 INFO:tasks.workunit.client.1.smithi169.stdout:5/82: link d1/f6 d1/f14 0 2024-03-04T13:42:39.419 INFO:tasks.workunit.client.1.smithi169.stdout:2/34: mknod d1/d5/c10 0 2024-03-04T13:42:39.425 INFO:tasks.workunit.client.1.smithi169.stdout:9/67: mknod d7/dc/c14 0 2024-03-04T13:42:39.428 INFO:tasks.workunit.client.1.smithi169.stdout:7/75: creat d2/f1c x:0 0 0 2024-03-04T13:42:39.428 INFO:tasks.workunit.client.1.smithi169.stdout:8/31: mknod d7/ca 0 2024-03-04T13:42:39.436 INFO:tasks.workunit.client.1.smithi169.stdout:8/32: unlink f6 0 2024-03-04T13:42:39.438 INFO:tasks.workunit.client.1.smithi169.stdout:3/37: dwrite f1 [0,4194304] 0 2024-03-04T13:42:39.440 INFO:tasks.workunit.client.1.smithi169.stdout:7/76: unlink d2/f14 0 2024-03-04T13:42:39.443 INFO:tasks.workunit.client.1.smithi169.stdout:1/23: dread d4/d5/f7 [0,4194304] 0 2024-03-04T13:42:39.443 INFO:tasks.workunit.client.1.smithi169.stdout:9/68: symlink d7/dc/d12/l15 0 2024-03-04T13:42:39.446 INFO:tasks.workunit.client.1.smithi169.stdout:7/77: fdatasync d2/de/f12 0 2024-03-04T13:42:39.449 INFO:tasks.workunit.client.1.smithi169.stdout:9/69: fsync d7/fb 0 2024-03-04T13:42:39.449 INFO:tasks.workunit.client.1.smithi169.stdout:8/33: creat d7/fb x:0 0 0 2024-03-04T13:42:39.451 INFO:tasks.workunit.client.1.smithi169.stdout:6/21: truncate d0/f1 1304363 0 2024-03-04T13:42:39.454 INFO:tasks.workunit.client.1.smithi169.stdout:0/70: truncate d5/fa 2749534 0 2024-03-04T13:42:39.457 INFO:tasks.workunit.client.1.smithi169.stdout:2/35: dwrite d1/d5/f9 [0,4194304] 0 2024-03-04T13:42:39.457 INFO:tasks.workunit.client.1.smithi169.stdout:5/83: dread d1/fa [0,4194304] 0 2024-03-04T13:42:39.459 INFO:tasks.workunit.client.1.smithi169.stdout:0/71: write d5/fb [1333280,16488] 0 2024-03-04T13:42:39.460 INFO:tasks.workunit.client.1.smithi169.stdout:3/38: rename f1 to d7/fb 0 2024-03-04T13:42:39.463 INFO:tasks.workunit.client.1.smithi169.stdout:7/78: creat d2/d15/d17/f1d x:0 0 0 2024-03-04T13:42:39.463 INFO:tasks.workunit.client.1.smithi169.stdout:8/34: mknod d7/cc 0 2024-03-04T13:42:39.467 INFO:tasks.workunit.client.1.smithi169.stdout:7/79: read - d2/d15/d17/f1d zero size 2024-03-04T13:42:39.468 INFO:tasks.workunit.client.1.smithi169.stdout:8/35: write d7/fb [43061,49135] 0 2024-03-04T13:42:39.470 INFO:tasks.workunit.client.1.smithi169.stdout:9/70: rename l5 to d7/dc/d12/l16 0 2024-03-04T13:42:39.472 INFO:tasks.workunit.client.1.smithi169.stdout:5/84: dread d1/f9 [0,4194304] 0 2024-03-04T13:42:39.477 INFO:tasks.workunit.client.1.smithi169.stdout:5/85: chown d1/c10 13336087 1 2024-03-04T13:42:39.479 INFO:tasks.workunit.client.1.smithi169.stdout:9/71: symlink d7/l17 0 2024-03-04T13:42:39.480 INFO:tasks.workunit.client.1.smithi169.stdout:5/86: readlink d1/ld 0 2024-03-04T13:42:39.482 INFO:tasks.workunit.client.1.smithi169.stdout:2/36: rename d1/d5/ld to d1/l11 0 2024-03-04T13:42:39.486 INFO:tasks.workunit.client.1.smithi169.stdout:1/24: write d4/d5/f7 [1561687,43025] 0 2024-03-04T13:42:39.486 INFO:tasks.workunit.client.1.smithi169.stdout:5/87: read d1/f13 [4060390,61360] 0 2024-03-04T13:42:39.491 INFO:tasks.workunit.client.1.smithi169.stdout:9/72: creat d7/dc/d12/f18 x:0 0 0 2024-03-04T13:42:39.497 INFO:tasks.workunit.client.1.smithi169.stdout:8/36: rename l3 to d7/ld 0 2024-03-04T13:42:39.498 INFO:tasks.workunit.client.1.smithi169.stdout:8/37: rename d7 to d7/de 22 2024-03-04T13:42:39.500 INFO:tasks.workunit.client.1.smithi169.stdout:0/72: dwrite d5/f16 [0,4194304] 0 2024-03-04T13:42:39.501 INFO:tasks.workunit.client.1.smithi169.stdout:5/88: creat d1/f15 x:0 0 0 2024-03-04T13:42:39.503 INFO:tasks.workunit.client.1.smithi169.stdout:7/80: dwrite d2/fd [0,4194304] 0 2024-03-04T13:42:39.515 INFO:tasks.workunit.client.1.smithi169.stdout:7/81: write d2/fd [1616316,66321] 0 2024-03-04T13:42:39.523 INFO:tasks.workunit.client.1.smithi169.stdout:0/73: fsync d5/fa 0 2024-03-04T13:42:39.527 INFO:tasks.workunit.client.1.smithi169.stdout:8/38: rmdir d7 39 2024-03-04T13:42:39.531 INFO:tasks.workunit.client.1.smithi169.stdout:5/89: rmdir d1 39 2024-03-04T13:42:39.534 INFO:tasks.workunit.client.1.smithi169.stdout:7/82: write d2/f1c [81202,13383] 0 2024-03-04T13:42:39.535 INFO:tasks.workunit.client.1.smithi169.stdout:6/22: dwrite d0/f1 [0,4194304] 0 2024-03-04T13:42:39.548 INFO:tasks.workunit.client.1.smithi169.stdout:2/37: dwrite d1/ff [0,4194304] 0 2024-03-04T13:42:39.550 INFO:tasks.workunit.client.1.smithi169.stdout:5/90: mkdir d1/d16 0 2024-03-04T13:42:39.550 INFO:tasks.workunit.client.1.smithi169.stdout:1/25: dwrite d4/d5/f7 [4194304,4194304] 0 2024-03-04T13:42:39.552 INFO:tasks.workunit.client.1.smithi169.stdout:1/26: readlink - no filename 2024-03-04T13:42:39.556 INFO:tasks.workunit.client.1.smithi169.stdout:8/39: dread d7/fb [0,4194304] 0 2024-03-04T13:42:39.556 INFO:tasks.workunit.client.1.smithi169.stdout:1/27: fsync f3 0 2024-03-04T13:42:39.557 INFO:tasks.workunit.client.1.smithi169.stdout:0/74: truncate d5/fa 520462 0 2024-03-04T13:42:39.558 INFO:tasks.workunit.client.1.smithi169.stdout:7/83: creat d2/de/f1e x:0 0 0 2024-03-04T13:42:39.563 INFO:tasks.workunit.client.1.smithi169.stdout:2/38: creat d1/d5/f12 x:0 0 0 2024-03-04T13:42:39.566 INFO:tasks.workunit.client.1.smithi169.stdout:1/28: creat d4/d5/f8 x:0 0 0 2024-03-04T13:42:39.566 INFO:tasks.workunit.client.1.smithi169.stdout:5/91: dread d1/f9 [0,4194304] 0 2024-03-04T13:42:39.568 INFO:tasks.workunit.client.1.smithi169.stdout:8/40: creat d7/ff x:0 0 0 2024-03-04T13:42:39.569 INFO:tasks.workunit.client.1.smithi169.stdout:7/84: creat d2/f1f x:0 0 0 2024-03-04T13:42:39.571 INFO:tasks.workunit.client.1.smithi169.stdout:9/73: dwrite d7/fe [0,4194304] 0 2024-03-04T13:42:39.572 INFO:tasks.workunit.client.1.smithi169.stdout:5/92: readlink d1/ld 0 2024-03-04T13:42:39.578 INFO:tasks.workunit.client.1.smithi169.stdout:1/29: unlink d4/d5/f7 0 2024-03-04T13:42:39.580 INFO:tasks.workunit.client.1.smithi169.stdout:8/41: fdatasync d7/fb 0 2024-03-04T13:42:39.583 INFO:tasks.workunit.client.1.smithi169.stdout:6/23: dread d0/f1 [0,4194304] 0 2024-03-04T13:42:39.584 INFO:tasks.workunit.client.1.smithi169.stdout:5/93: mkdir d1/d16/d17 0 2024-03-04T13:42:39.587 INFO:tasks.workunit.client.1.smithi169.stdout:5/94: stat d1/f9 0 2024-03-04T13:42:39.587 INFO:tasks.workunit.client.1.smithi169.stdout:0/75: dread d5/f16 [0,4194304] 0 2024-03-04T13:42:39.591 INFO:tasks.workunit.client.1.smithi169.stdout:0/76: chown d5 18573408 1 2024-03-04T13:42:39.599 INFO:tasks.workunit.client.1.smithi169.stdout:1/30: stat d4/c6 0 2024-03-04T13:42:39.600 INFO:tasks.workunit.client.1.smithi169.stdout:7/85: truncate d2/de/f12 250741 0 2024-03-04T13:42:39.603 INFO:tasks.workunit.client.1.smithi169.stdout:6/24: unlink d0/c4 0 2024-03-04T13:42:39.606 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:42:39 smithi169 ceph-mon[36313]: pgmap v41: 65 pgs: 65 active+clean; 250 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.3 MiB/s rd, 8.7 MiB/s wr, 225 op/s 2024-03-04T13:42:39.610 INFO:tasks.workunit.client.1.smithi169.stdout:0/77: mkdir d5/d9/dc/d1c 0 2024-03-04T13:42:39.612 INFO:tasks.workunit.client.1.smithi169.stdout:2/39: dwrite d1/d5/fe [0,4194304] 0 2024-03-04T13:42:39.615 INFO:tasks.workunit.client.1.smithi169.stdout:1/31: creat d4/f9 x:0 0 0 2024-03-04T13:42:39.615 INFO:tasks.workunit.client.1.smithi169.stdout:7/86: truncate d2/d15/d17/f1d 974768 0 2024-03-04T13:42:39.617 INFO:tasks.workunit.client.1.smithi169.stdout:6/25: unlink d0/l2 0 2024-03-04T13:42:39.624 INFO:tasks.workunit.client.1.smithi169.stdout:0/78: fsync d5/d9/f1a 0 2024-03-04T13:42:39.624 INFO:tasks.workunit.client.1.smithi169.stdout:5/95: creat d1/d16/d17/f18 x:0 0 0 2024-03-04T13:42:39.632 INFO:tasks.workunit.client.1.smithi169.stdout:8/42: dwrite d7/fb [0,4194304] 0 2024-03-04T13:42:39.633 INFO:tasks.workunit.client.1.smithi169.stdout:1/32: mknod d4/d5/ca 0 2024-03-04T13:42:39.633 INFO:tasks.workunit.client.1.smithi169.stdout:5/96: fsync d1/f15 0 2024-03-04T13:42:39.633 INFO:tasks.workunit.client.1.smithi169.stdout:9/74: dwrite d7/f10 [0,4194304] 0 2024-03-04T13:42:39.638 INFO:tasks.workunit.client.1.smithi169.stdout:8/43: stat d7/fb 0 2024-03-04T13:42:39.638 INFO:tasks.workunit.client.1.smithi169.stdout:6/26: write d0/f1 [1957908,77661] 0 2024-03-04T13:42:39.639 INFO:tasks.workunit.client.1.smithi169.stdout:9/75: truncate f4 4868851 0 2024-03-04T13:42:39.639 INFO:tasks.workunit.client.1.smithi169.stdout:5/97: chown d1/lf 90 1 2024-03-04T13:42:39.639 INFO:tasks.workunit.client.1.smithi169.stdout:9/76: fdatasync f6 0 2024-03-04T13:42:39.639 INFO:tasks.workunit.client.1.smithi169.stdout:0/79: creat d5/d9/dc/f1d x:0 0 0 2024-03-04T13:42:39.640 INFO:tasks.workunit.client.1.smithi169.stdout:5/98: chown d1/lf 4 1 2024-03-04T13:42:39.646 INFO:tasks.workunit.client.1.smithi169.stdout:8/44: mkdir d7/d10 0 2024-03-04T13:42:39.649 INFO:tasks.workunit.client.1.smithi169.stdout:4/27: sync 2024-03-04T13:42:39.649 INFO:tasks.workunit.client.1.smithi169.stdout:2/40: dread d1/d5/fe [0,4194304] 0 2024-03-04T13:42:39.650 INFO:tasks.workunit.client.1.smithi169.stdout:8/45: dread - d7/f9 zero size 2024-03-04T13:42:39.654 INFO:tasks.workunit.client.1.smithi169.stdout:6/27: fdatasync d0/f1 0 2024-03-04T13:42:39.657 INFO:tasks.workunit.client.1.smithi169.stdout:1/33: rename d4/f9 to d4/d5/fb 0 2024-03-04T13:42:39.659 INFO:tasks.workunit.client.1.smithi169.stdout:5/99: write d1/f13 [2558409,34261] 0 2024-03-04T13:42:39.664 INFO:tasks.workunit.client.1.smithi169.stdout:4/28: creat d4/f7 x:0 0 0 2024-03-04T13:42:39.664 INFO:tasks.workunit.client.1.smithi169.stdout:2/41: creat d1/d5/f13 x:0 0 0 2024-03-04T13:42:39.666 INFO:tasks.workunit.client.1.smithi169.stdout:0/80: link d5/c14 d5/d9/dc/d12/c1e 0 2024-03-04T13:42:39.667 INFO:tasks.workunit.client.1.smithi169.stdout:8/46: write d7/fb [3500993,123956] 0 2024-03-04T13:42:39.668 INFO:tasks.workunit.client.1.smithi169.stdout:6/28: rename d0/l3 to d0/l5 0 2024-03-04T13:42:39.673 INFO:tasks.workunit.client.1.smithi169.stdout:1/34: symlink d4/d5/lc 0 2024-03-04T13:42:39.676 INFO:tasks.workunit.client.1.smithi169.stdout:2/42: symlink d1/l14 0 2024-03-04T13:42:39.677 INFO:tasks.workunit.client.1.smithi169.stdout:8/47: write d7/fb [3590950,103807] 0 2024-03-04T13:42:39.678 INFO:tasks.workunit.client.1.smithi169.stdout:7/87: dwrite d2/de/ff [0,4194304] 0 2024-03-04T13:42:39.682 INFO:tasks.workunit.client.1.smithi169.stdout:1/35: chown d4/d5/ca 315 1 2024-03-04T13:42:39.682 INFO:tasks.workunit.client.1.smithi169.stdout:6/29: mkdir d0/d6 0 2024-03-04T13:42:39.685 INFO:tasks.workunit.client.1.smithi169.stdout:6/30: chown d0/d6 13417 1 2024-03-04T13:42:39.685 INFO:tasks.workunit.client.1.smithi169.stdout:9/77: dwrite d7/dc/d12/f18 [0,4194304] 0 2024-03-04T13:42:39.685 INFO:tasks.workunit.client.1.smithi169.stdout:1/36: fsync f3 0 2024-03-04T13:42:39.692 INFO:tasks.workunit.client.1.smithi169.stdout:2/43: mknod d1/c15 0 2024-03-04T13:42:39.695 INFO:tasks.workunit.client.1.smithi169.stdout:4/29: link d4/f7 d4/f8 0 2024-03-04T13:42:39.700 INFO:tasks.workunit.client.1.smithi169.stdout:3/39: sync 2024-03-04T13:42:39.707 INFO:tasks.workunit.client.1.smithi169.stdout:7/88: symlink d2/d15/l20 0 2024-03-04T13:42:39.715 INFO:tasks.workunit.client.1.smithi169.stdout:0/81: rename d5/d9/dc/c11 to d5/d9/dc/d12/d15/d19/c1f 0 2024-03-04T13:42:39.718 INFO:tasks.workunit.client.1.smithi169.stdout:2/44: rename d1/d5/f12 to d1/d5/f16 0 2024-03-04T13:42:39.725 INFO:tasks.workunit.client.1.smithi169.stdout:6/31: rmdir d0/d6 0 2024-03-04T13:42:39.726 INFO:tasks.workunit.client.1.smithi169.stdout:7/89: rename d2/cb to d2/c21 0 2024-03-04T13:42:39.726 INFO:tasks.workunit.client.1.smithi169.stdout:0/82: write d5/f6 [3939129,47589] 0 2024-03-04T13:42:39.728 INFO:tasks.workunit.client.1.smithi169.stdout:5/100: dwrite d1/d16/d17/f18 [0,4194304] 0 2024-03-04T13:42:39.729 INFO:tasks.workunit.client.1.smithi169.stdout:1/37: dread f3 [0,4194304] 0 2024-03-04T13:42:39.739 INFO:tasks.workunit.client.1.smithi169.stdout:2/45: rename d1/l6 to d1/l17 0 2024-03-04T13:42:39.739 INFO:tasks.workunit.client.1.smithi169.stdout:8/48: dwrite d7/ff [0,4194304] 0 2024-03-04T13:42:39.748 INFO:tasks.workunit.client.1.smithi169.stdout:1/38: rmdir d4/d5 39 2024-03-04T13:42:39.755 INFO:tasks.workunit.client.1.smithi169.stdout:5/101: fsync d1/f6 0 2024-03-04T13:42:39.755 INFO:tasks.workunit.client.1.smithi169.stdout:8/49: chown d7/ld 3025505 1 2024-03-04T13:42:39.759 INFO:tasks.workunit.client.1.smithi169.stdout:6/32: dwrite d0/f1 [0,4194304] 0 2024-03-04T13:42:39.762 INFO:tasks.workunit.client.1.smithi169.stdout:2/46: rename d1/l14 to d1/d5/l18 0 2024-03-04T13:42:39.764 INFO:tasks.workunit.client.1.smithi169.stdout:4/30: dwrite d4/f8 [0,4194304] 0 2024-03-04T13:42:39.766 INFO:tasks.workunit.client.1.smithi169.stdout:1/39: chown d4/d5 5 1 2024-03-04T13:42:39.769 INFO:tasks.workunit.client.1.smithi169.stdout:0/83: dread d5/f6 [0,4194304] 0 2024-03-04T13:42:39.775 INFO:tasks.workunit.client.1.smithi169.stdout:8/50: unlink d7/f9 0 2024-03-04T13:42:39.779 INFO:tasks.workunit.client.1.smithi169.stdout:2/47: dread d1/d5/f9 [0,4194304] 0 2024-03-04T13:42:39.779 INFO:tasks.workunit.client.1.smithi169.stdout:2/48: chown d1 40497 1 2024-03-04T13:42:39.780 INFO:tasks.workunit.client.1.smithi169.stdout:6/33: mknod d0/c7 0 2024-03-04T13:42:39.782 INFO:tasks.workunit.client.1.smithi169.stdout:4/31: dread d4/f7 [0,4194304] 0 2024-03-04T13:42:39.783 INFO:tasks.workunit.client.1.smithi169.stdout:5/102: truncate d1/f9 513948 0 2024-03-04T13:42:39.785 INFO:tasks.workunit.client.1.smithi169.stdout:9/78: truncate d7/fe 3630551 0 2024-03-04T13:42:39.794 INFO:tasks.workunit.client.1.smithi169.stdout:0/84: dread d5/f8 [0,4194304] 0 2024-03-04T13:42:39.796 INFO:tasks.workunit.client.1.smithi169.stdout:2/49: mkdir d1/d5/d19 0 2024-03-04T13:42:39.798 INFO:tasks.workunit.client.1.smithi169.stdout:6/34: creat d0/f8 x:0 0 0 2024-03-04T13:42:39.801 INFO:tasks.workunit.client.1.smithi169.stdout:9/79: symlink d7/l19 0 2024-03-04T13:42:39.802 INFO:tasks.workunit.client.1.smithi169.stdout:9/80: stat f6 0 2024-03-04T13:42:39.803 INFO:tasks.workunit.client.1.smithi169.stdout:5/103: mknod d1/d16/d17/c19 0 2024-03-04T13:42:39.808 INFO:tasks.workunit.client.1.smithi169.stdout:4/32: rename d4/f8 to d4/f9 0 2024-03-04T13:42:39.808 INFO:tasks.workunit.client.1.smithi169.stdout:6/35: dread d0/f1 [0,4194304] 0 2024-03-04T13:42:39.810 INFO:tasks.workunit.client.1.smithi169.stdout:6/36: chown d0/c7 637381702 1 2024-03-04T13:42:39.810 INFO:tasks.workunit.client.1.smithi169.stdout:1/40: mkdir d4/dd 0 2024-03-04T13:42:39.819 INFO:tasks.workunit.client.1.smithi169.stdout:0/85: creat d5/d9/dc/d12/d15/d19/f20 x:0 0 0 2024-03-04T13:42:39.827 INFO:tasks.workunit.client.1.smithi169.stdout:5/104: unlink d1/c10 0 2024-03-04T13:42:39.827 INFO:tasks.workunit.client.1.smithi169.stdout:1/41: dwrite f3 [0,4194304] 0 2024-03-04T13:42:39.831 INFO:tasks.workunit.client.1.smithi169.stdout:4/33: write d4/f9 [3388903,103968] 0 2024-03-04T13:42:39.832 INFO:tasks.workunit.client.1.smithi169.stdout:6/37: creat d0/f9 x:0 0 0 2024-03-04T13:42:39.835 INFO:tasks.workunit.client.1.smithi169.stdout:2/50: mkdir d1/d5/d19/d1a 0 2024-03-04T13:42:39.839 INFO:tasks.workunit.client.1.smithi169.stdout:2/51: chown d1/d5/f8 1755848986 1 2024-03-04T13:42:39.841 INFO:tasks.workunit.client.1.smithi169.stdout:2/52: dread - d1/d5/f16 zero size 2024-03-04T13:42:39.842 INFO:tasks.workunit.client.1.smithi169.stdout:0/86: dwrite d5/d9/dc/f1d [0,4194304] 0 2024-03-04T13:42:39.843 INFO:tasks.workunit.client.1.smithi169.stdout:2/53: read d1/d5/fe [1077576,50010] 0 2024-03-04T13:42:39.845 INFO:tasks.workunit.client.1.smithi169.stdout:8/51: truncate d7/fb 1392773 0 2024-03-04T13:42:39.847 INFO:tasks.workunit.client.1.smithi169.stdout:5/105: truncate d1/f6 488834 0 2024-03-04T13:42:39.848 INFO:tasks.workunit.client.1.smithi169.stdout:8/52: write d7/ff [4437777,65420] 0 2024-03-04T13:42:39.849 INFO:tasks.workunit.client.1.smithi169.stdout:4/34: rmdir d4 39 2024-03-04T13:42:39.851 INFO:tasks.workunit.client.1.smithi169.stdout:5/106: readlink d1/ld 0 2024-03-04T13:42:39.863 INFO:tasks.workunit.client.1.smithi169.stdout:1/42: symlink d4/le 0 2024-03-04T13:42:39.863 INFO:tasks.workunit.client.1.smithi169.stdout:2/54: dwrite d1/ff [0,4194304] 0 2024-03-04T13:42:39.873 INFO:tasks.workunit.client.1.smithi169.stdout:7/90: sync 2024-03-04T13:42:39.873 INFO:tasks.workunit.client.1.smithi169.stdout:3/40: sync 2024-03-04T13:42:39.874 INFO:tasks.workunit.client.1.smithi169.stdout:4/35: chown d4 40 1 2024-03-04T13:42:39.878 INFO:tasks.workunit.client.1.smithi169.stdout:8/53: unlink f5 0 2024-03-04T13:42:39.889 INFO:tasks.workunit.client.1.smithi169.stdout:2/55: write d1/d5/f16 [338163,87200] 0 2024-03-04T13:42:39.889 INFO:tasks.workunit.client.1.smithi169.stdout:8/54: dread d7/ff [0,4194304] 0 2024-03-04T13:42:39.893 INFO:tasks.workunit.client.1.smithi169.stdout:0/87: creat d5/d9/dc/d1c/f21 x:0 0 0 2024-03-04T13:42:39.893 INFO:tasks.workunit.client.1.smithi169.stdout:7/91: creat d2/f22 x:0 0 0 2024-03-04T13:42:39.896 INFO:tasks.workunit.client.1.smithi169.stdout:4/36: creat d4/fa x:0 0 0 2024-03-04T13:42:39.902 INFO:tasks.workunit.client.1.smithi169.stdout:1/43: mkdir d4/dd/df 0 2024-03-04T13:42:39.905 INFO:tasks.workunit.client.1.smithi169.stdout:7/92: dread d2/de/ff [0,4194304] 0 2024-03-04T13:42:39.907 INFO:tasks.workunit.client.1.smithi169.stdout:8/55: rename d7/l8 to d7/l11 0 2024-03-04T13:42:39.908 INFO:tasks.workunit.client.1.smithi169.stdout:8/56: chown l2 5261 1 2024-03-04T13:42:39.908 INFO:tasks.workunit.client.1.smithi169.stdout:3/41: getdents d7/da 0 2024-03-04T13:42:39.909 INFO:tasks.workunit.client.1.smithi169.stdout:0/88: mknod d5/d9/dc/d1c/c22 0 2024-03-04T13:42:39.920 INFO:tasks.workunit.client.1.smithi169.stdout:6/38: getdents d0 0 2024-03-04T13:42:39.923 INFO:tasks.workunit.client.1.smithi169.stdout:2/56: dwrite d1/d5/fe [0,4194304] 0 2024-03-04T13:42:39.925 INFO:tasks.workunit.client.1.smithi169.stdout:6/39: chown d0/f9 140 1 2024-03-04T13:42:39.928 INFO:tasks.workunit.client.1.smithi169.stdout:7/93: dwrite d2/fd [0,4194304] 0 2024-03-04T13:42:39.929 INFO:tasks.workunit.client.1.smithi169.stdout:3/42: write f5 [743688,78378] 0 2024-03-04T13:42:39.933 INFO:tasks.workunit.client.1.smithi169.stdout:6/40: chown d0/c7 742870 1 2024-03-04T13:42:39.933 INFO:tasks.workunit.client.1.smithi169.stdout:8/57: mkdir d7/d12 0 2024-03-04T13:42:39.942 INFO:tasks.workunit.client.1.smithi169.stdout:4/37: dwrite d4/f6 [0,4194304] 0 2024-03-04T13:42:39.944 INFO:tasks.workunit.client.1.smithi169.stdout:2/57: creat d1/d5/d19/f1b x:0 0 0 2024-03-04T13:42:39.944 INFO:tasks.workunit.client.1.smithi169.stdout:7/94: symlink d2/de/l23 0 2024-03-04T13:42:39.947 INFO:tasks.workunit.client.1.smithi169.stdout:6/41: rename d0/c7 to d0/ca 0 2024-03-04T13:42:39.948 INFO:tasks.workunit.client.1.smithi169.stdout:4/38: truncate d4/fa 352502 0 2024-03-04T13:42:39.948 INFO:tasks.workunit.client.1.smithi169.stdout:7/95: chown d2/de/ff 356965 1 2024-03-04T13:42:39.951 INFO:tasks.workunit.client.1.smithi169.stdout:9/81: dwrite d7/fe [4194304,4194304] 0 2024-03-04T13:42:39.952 INFO:tasks.workunit.client.1.smithi169.stdout:5/107: dread d1/f14 [0,4194304] 0 2024-03-04T13:42:39.952 INFO:tasks.workunit.client.1.smithi169.stdout:3/43: getdents d7/da 0 2024-03-04T13:42:39.964 INFO:tasks.workunit.client.1.smithi169.stdout:6/42: truncate d0/f8 572374 0 2024-03-04T13:42:39.964 INFO:tasks.workunit.client.1.smithi169.stdout:9/82: chown d7/f10 967070 1 2024-03-04T13:42:39.964 INFO:tasks.workunit.client.1.smithi169.stdout:4/39: creat d4/fb x:0 0 0 2024-03-04T13:42:39.966 INFO:tasks.workunit.client.1.smithi169.stdout:3/44: mknod d7/cc 0 2024-03-04T13:42:39.967 INFO:tasks.workunit.client.1.smithi169.stdout:8/58: rename d7/fb to d7/f13 0 2024-03-04T13:42:39.969 INFO:tasks.workunit.client.1.smithi169.stdout:9/83: chown d7/dc/d12 194076 1 2024-03-04T13:42:39.973 INFO:tasks.workunit.client.1.smithi169.stdout:0/89: dwrite d5/f16 [4194304,4194304] 0 2024-03-04T13:42:39.976 INFO:tasks.workunit.client.1.smithi169.stdout:4/40: symlink d4/lc 0 2024-03-04T13:42:39.980 INFO:tasks.workunit.client.1.smithi169.stdout:0/90: readlink d5/l17 0 2024-03-04T13:42:39.981 INFO:tasks.workunit.client.1.smithi169.stdout:9/84: mknod d7/dc/c1a 0 2024-03-04T13:42:39.988 INFO:tasks.workunit.client.1.smithi169.stdout:9/85: readlink d7/dc/lf 0 2024-03-04T13:42:39.990 INFO:tasks.workunit.client.1.smithi169.stdout:8/59: mkdir d7/d10/d14 0 2024-03-04T13:42:39.991 INFO:tasks.workunit.client.1.smithi169.stdout:4/41: mknod d4/cd 0 2024-03-04T13:42:39.993 INFO:tasks.workunit.client.1.smithi169.stdout:1/44: dwrite d4/d5/fb [0,4194304] 0 2024-03-04T13:42:40.001 INFO:tasks.workunit.client.1.smithi169.stdout:4/42: symlink d4/le 0 2024-03-04T13:42:40.009 INFO:tasks.workunit.client.1.smithi169.stdout:6/43: dwrite d0/f8 [0,4194304] 0 2024-03-04T13:42:40.012 INFO:tasks.workunit.client.1.smithi169.stdout:8/60: creat d7/d12/f15 x:0 0 0 2024-03-04T13:42:40.012 INFO:tasks.workunit.client.1.smithi169.stdout:4/43: dread d4/fa [0,4194304] 0 2024-03-04T13:42:40.013 INFO:tasks.workunit.client.1.smithi169.stdout:7/96: dread d2/de/ff [0,4194304] 0 2024-03-04T13:42:40.014 INFO:tasks.workunit.client.1.smithi169.stdout:2/58: dwrite d1/d5/f9 [0,4194304] 0 2024-03-04T13:42:40.015 INFO:tasks.workunit.client.1.smithi169.stdout:6/44: readlink d0/l5 0 2024-03-04T13:42:40.016 INFO:tasks.workunit.client.1.smithi169.stdout:9/86: dread d7/fb [0,4194304] 0 2024-03-04T13:42:40.025 INFO:tasks.workunit.client.1.smithi169.stdout:4/44: mknod d4/cf 0 2024-03-04T13:42:40.029 INFO:tasks.workunit.client.1.smithi169.stdout:8/61: creat d7/d10/f16 x:0 0 0 2024-03-04T13:42:40.029 INFO:tasks.workunit.client.1.smithi169.stdout:7/97: chown d2/l5 99 1 2024-03-04T13:42:40.031 INFO:tasks.workunit.client.1.smithi169.stdout:9/87: mknod d7/c1b 0 2024-03-04T13:42:40.032 INFO:tasks.workunit.client.1.smithi169.stdout:6/45: creat d0/fb x:0 0 0 2024-03-04T13:42:40.033 INFO:tasks.workunit.client.1.smithi169.stdout:4/45: write d4/f7 [167653,85496] 0 2024-03-04T13:42:40.033 INFO:tasks.workunit.client.1.smithi169.stdout:3/45: dwrite f6 [0,4194304] 0 2024-03-04T13:42:40.038 INFO:tasks.workunit.client.1.smithi169.stdout:2/59: symlink d1/d5/l1c 0 2024-03-04T13:42:40.039 INFO:tasks.workunit.client.1.smithi169.stdout:1/45: dread f3 [0,4194304] 0 2024-03-04T13:42:40.040 INFO:tasks.workunit.client.1.smithi169.stdout:3/46: chown d7/f9 31558441 1 2024-03-04T13:42:40.042 INFO:tasks.workunit.client.1.smithi169.stdout:6/46: truncate d0/f9 978588 0 2024-03-04T13:42:40.044 INFO:tasks.workunit.client.1.smithi169.stdout:6/47: stat d0/f8 0 2024-03-04T13:42:40.045 INFO:tasks.workunit.client.1.smithi169.stdout:9/88: symlink d7/dc/l1c 0 2024-03-04T13:42:40.046 INFO:tasks.workunit.client.1.smithi169.stdout:4/46: rename d4/lc to d4/l10 0 2024-03-04T13:42:40.047 INFO:tasks.workunit.client.1.smithi169.stdout:1/46: read d4/d5/fb [2629877,249] 0 2024-03-04T13:42:40.047 INFO:tasks.workunit.client.1.smithi169.stdout:0/91: dwrite d5/fa [0,4194304] 0 2024-03-04T13:42:40.047 INFO:tasks.workunit.client.1.smithi169.stdout:6/48: dread - d0/fb zero size 2024-03-04T13:42:40.047 INFO:tasks.workunit.client.1.smithi169.stdout:8/62: truncate d7/ff 4409830 0 2024-03-04T13:42:40.052 INFO:tasks.workunit.client.1.smithi169.stdout:8/63: truncate d7/d12/f15 244136 0 2024-03-04T13:42:40.054 INFO:tasks.workunit.client.1.smithi169.stdout:3/47: symlink d7/ld 0 2024-03-04T13:42:40.056 INFO:tasks.workunit.client.1.smithi169.stdout:9/89: creat d7/dc/f1d x:0 0 0 2024-03-04T13:42:40.060 INFO:tasks.workunit.client.1.smithi169.stdout:4/47: mkdir d4/d11 0 2024-03-04T13:42:40.060 INFO:tasks.workunit.client.1.smithi169.stdout:1/47: dread d4/d5/fb [0,4194304] 0 2024-03-04T13:42:40.062 INFO:tasks.workunit.client.1.smithi169.stdout:1/48: dread - d4/d5/f8 zero size 2024-03-04T13:42:40.062 INFO:tasks.workunit.client.1.smithi169.stdout:4/48: truncate d4/fb 805380 0 2024-03-04T13:42:40.068 INFO:tasks.workunit.client.1.smithi169.stdout:8/64: mkdir d7/d10/d17 0 2024-03-04T13:42:40.069 INFO:tasks.workunit.client.1.smithi169.stdout:7/98: creat d2/d15/f24 x:0 0 0 2024-03-04T13:42:40.071 INFO:tasks.workunit.client.1.smithi169.stdout:3/48: mkdir d7/de 0 2024-03-04T13:42:40.073 INFO:tasks.workunit.client.1.smithi169.stdout:9/90: rmdir d7 39 2024-03-04T13:42:40.074 INFO:tasks.workunit.client.1.smithi169.stdout:2/60: getdents d1/d5/d19/d1a 0 2024-03-04T13:42:40.080 INFO:tasks.workunit.client.1.smithi169.stdout:0/92: rename d5/fa to d5/d9/dc/d1c/f23 0 2024-03-04T13:42:40.082 INFO:tasks.workunit.client.1.smithi169.stdout:8/65: rmdir d7/d10 39 2024-03-04T13:42:40.088 INFO:tasks.workunit.client.1.smithi169.stdout:6/49: dwrite d0/f1 [0,4194304] 0 2024-03-04T13:42:40.090 INFO:tasks.workunit.client.1.smithi169.stdout:1/49: symlink d4/l10 0 2024-03-04T13:42:40.091 INFO:tasks.workunit.client.1.smithi169.stdout:4/49: getdents d4/d11 0 2024-03-04T13:42:40.092 INFO:tasks.workunit.client.1.smithi169.stdout:0/93: rename d5/fb to d5/d9/dc/d12/f24 0 2024-03-04T13:42:40.093 INFO:tasks.workunit.client.1.smithi169.stdout:3/49: creat d7/de/ff x:0 0 0 2024-03-04T13:42:40.094 INFO:tasks.workunit.client.1.smithi169.stdout:9/91: creat d7/dc/f1e x:0 0 0 2024-03-04T13:42:40.096 INFO:tasks.workunit.client.1.smithi169.stdout:6/50: symlink d0/lc 0 2024-03-04T13:42:40.097 INFO:tasks.workunit.client.1.smithi169.stdout:1/50: fdatasync f3 0 2024-03-04T13:42:40.099 INFO:tasks.workunit.client.1.smithi169.stdout:5/108: sync 2024-03-04T13:42:40.099 INFO:tasks.workunit.client.1.smithi169.stdout:8/66: unlink d7/f13 0 2024-03-04T13:42:40.101 INFO:tasks.workunit.client.1.smithi169.stdout:8/67: chown d7/ld 5599063 1 2024-03-04T13:42:40.105 INFO:tasks.workunit.client.1.smithi169.stdout:9/92: creat d7/dc/d12/f1f x:0 0 0 2024-03-04T13:42:40.107 INFO:tasks.workunit.client.1.smithi169.stdout:2/61: rmdir d1/d5/d19/d1a 0 2024-03-04T13:42:40.107 INFO:tasks.workunit.client.1.smithi169.stdout:6/51: symlink d0/ld 0 2024-03-04T13:42:40.108 INFO:tasks.workunit.client.1.smithi169.stdout:6/52: fsync d0/f8 0 2024-03-04T13:42:40.108 INFO:tasks.workunit.client.1.smithi169.stdout:0/94: dread d5/f8 [0,4194304] 0 2024-03-04T13:42:40.112 INFO:tasks.workunit.client.1.smithi169.stdout:7/99: getdents d2/d15/d17 0 2024-03-04T13:42:40.112 INFO:tasks.workunit.client.1.smithi169.stdout:1/51: fdatasync d4/d5/fb 0 2024-03-04T13:42:40.113 INFO:tasks.workunit.client.1.smithi169.stdout:1/52: fdatasync f3 0 2024-03-04T13:42:40.114 INFO:tasks.workunit.client.1.smithi169.stdout:6/53: truncate d0/f1 4403635 0 2024-03-04T13:42:40.119 INFO:tasks.workunit.client.1.smithi169.stdout:9/93: dread d7/dc/fd [0,4194304] 0 2024-03-04T13:42:40.120 INFO:tasks.workunit.client.1.smithi169.stdout:4/50: link d4/cd d4/c12 0 2024-03-04T13:42:40.122 INFO:tasks.workunit.client.1.smithi169.stdout:4/51: chown d4/le 292 1 2024-03-04T13:42:40.122 INFO:tasks.workunit.client.1.smithi169.stdout:5/109: dread d1/fa [0,4194304] 0 2024-03-04T13:42:40.123 INFO:tasks.workunit.client.1.smithi169.stdout:0/95: mknod d5/c25 0 2024-03-04T13:42:40.123 INFO:tasks.workunit.client.1.smithi169.stdout:4/52: readlink d4/le 0 2024-03-04T13:42:40.125 INFO:tasks.workunit.client.1.smithi169.stdout:5/110: chown d1/d16/d17 1456 1 2024-03-04T13:42:40.127 INFO:tasks.workunit.client.1.smithi169.stdout:6/54: stat d0/ca 0 2024-03-04T13:42:40.128 INFO:tasks.workunit.client.1.smithi169.stdout:6/55: fdatasync d0/f9 0 2024-03-04T13:42:40.131 INFO:tasks.workunit.client.1.smithi169.stdout:8/68: mkdir d7/d10/d14/d18 0 2024-03-04T13:42:40.133 INFO:tasks.workunit.client.1.smithi169.stdout:6/56: write d0/fb [1000345,11904] 0 2024-03-04T13:42:40.134 INFO:tasks.workunit.client.1.smithi169.stdout:0/96: unlink d5/d9/f1a 0 2024-03-04T13:42:40.137 INFO:tasks.workunit.client.1.smithi169.stdout:2/62: dwrite d1/d5/f8 [0,4194304] 0 2024-03-04T13:42:40.139 INFO:tasks.workunit.client.1.smithi169.stdout:3/50: write f4 [591670,124578] 0 2024-03-04T13:42:40.139 INFO:tasks.workunit.client.1.smithi169.stdout:5/111: rmdir d1/d16 39 2024-03-04T13:42:40.141 INFO:tasks.workunit.client.1.smithi169.stdout:6/57: chown d0/ca 0 1 2024-03-04T13:42:40.141 INFO:tasks.workunit.client.1.smithi169.stdout:1/53: dread d4/d5/fb [0,4194304] 0 2024-03-04T13:42:40.142 INFO:tasks.workunit.client.1.smithi169.stdout:3/51: dread - d7/de/ff zero size 2024-03-04T13:42:40.145 INFO:tasks.workunit.client.1.smithi169.stdout:5/112: dread - d1/f15 zero size 2024-03-04T13:42:40.148 INFO:tasks.workunit.client.1.smithi169.stdout:6/58: mknod d0/ce 0 2024-03-04T13:42:40.153 INFO:tasks.workunit.client.1.smithi169.stdout:4/53: dwrite d4/f6 [0,4194304] 0 2024-03-04T13:42:40.163 INFO:tasks.workunit.client.1.smithi169.stdout:1/54: truncate f3 4003793 0 2024-03-04T13:42:40.168 INFO:tasks.workunit.client.1.smithi169.stdout:4/54: mknod d4/d11/c13 0 2024-03-04T13:42:40.170 INFO:tasks.workunit.client.1.smithi169.stdout:9/94: write d7/f11 [1303621,74806] 0 2024-03-04T13:42:40.176 INFO:tasks.workunit.client.1.smithi169.stdout:1/55: chown d4/dd 110 1 2024-03-04T13:42:40.181 INFO:tasks.workunit.client.1.smithi169.stdout:5/113: creat d1/d16/d17/f1a x:0 0 0 2024-03-04T13:42:40.181 INFO:tasks.workunit.client.1.smithi169.stdout:4/55: read d4/f9 [2690836,37943] 0 2024-03-04T13:42:40.182 INFO:tasks.workunit.client.1.smithi169.stdout:3/52: dread f6 [0,4194304] 0 2024-03-04T13:42:40.189 INFO:tasks.workunit.client.1.smithi169.stdout:2/63: dwrite d1/d5/f13 [0,4194304] 0 2024-03-04T13:42:40.189 INFO:tasks.workunit.client.1.smithi169.stdout:9/95: mkdir d7/dc/d12/d20 0 2024-03-04T13:42:40.191 INFO:tasks.workunit.client.1.smithi169.stdout:5/114: write d1/f13 [1257623,54331] 0 2024-03-04T13:42:40.198 INFO:tasks.workunit.client.1.smithi169.stdout:8/69: dwrite d7/d10/f16 [0,4194304] 0 2024-03-04T13:42:40.199 INFO:tasks.workunit.client.1.smithi169.stdout:4/56: creat d4/f14 x:0 0 0 2024-03-04T13:42:40.199 INFO:tasks.workunit.client.1.smithi169.stdout:9/96: fdatasync d7/f10 0 2024-03-04T13:42:40.203 INFO:tasks.workunit.client.1.smithi169.stdout:3/53: symlink d7/da/l10 0 2024-03-04T13:42:40.206 INFO:tasks.workunit.client.1.smithi169.stdout:2/64: write d1/d5/f16 [653632,44185] 0 2024-03-04T13:42:40.207 INFO:tasks.workunit.client.1.smithi169.stdout:8/70: read d7/d12/f15 [140960,90687] 0 2024-03-04T13:42:40.216 INFO:tasks.workunit.client.1.smithi169.stdout:6/59: dwrite d0/f9 [0,4194304] 0 2024-03-04T13:42:40.218 INFO:tasks.workunit.client.1.smithi169.stdout:9/97: symlink d7/l21 0 2024-03-04T13:42:40.220 INFO:tasks.workunit.client.1.smithi169.stdout:2/65: stat d1/c15 0 2024-03-04T13:42:40.220 INFO:tasks.workunit.client.1.smithi169.stdout:0/97: dwrite d5/d9/dc/d1c/f21 [0,4194304] 0 2024-03-04T13:42:40.220 INFO:tasks.workunit.client.1.smithi169.stdout:8/71: chown d7/d10/d14 114 1 2024-03-04T13:42:40.222 INFO:tasks.workunit.client.1.smithi169.stdout:4/57: creat d4/f15 x:0 0 0 2024-03-04T13:42:40.224 INFO:tasks.workunit.client.1.smithi169.stdout:3/54: symlink d7/da/l11 0 2024-03-04T13:42:40.235 INFO:tasks.workunit.client.1.smithi169.stdout:2/66: rename d1/d5/f13 to d1/f1d 0 2024-03-04T13:42:40.236 INFO:tasks.workunit.client.1.smithi169.stdout:9/98: write d7/dc/d12/f18 [4403345,102844] 0 2024-03-04T13:42:40.236 INFO:tasks.workunit.client.1.smithi169.stdout:4/58: chown d4/f7 16310272 1 2024-03-04T13:42:40.238 INFO:tasks.workunit.client.1.smithi169.stdout:3/55: write d7/f9 [337151,64545] 0 2024-03-04T13:42:40.242 INFO:tasks.workunit.client.1.smithi169.stdout:1/56: dwrite d4/d5/fb [0,4194304] 0 2024-03-04T13:42:40.242 INFO:tasks.workunit.client.1.smithi169.stdout:6/60: dread d0/fb [0,4194304] 0 2024-03-04T13:42:40.244 INFO:tasks.workunit.client.1.smithi169.stdout:0/98: getdents d5/d9 0 2024-03-04T13:42:40.249 INFO:tasks.workunit.client.1.smithi169.stdout:4/59: truncate d4/fb 1676991 0 2024-03-04T13:42:40.255 INFO:tasks.workunit.client.1.smithi169.stdout:2/67: chown d1/d5/c10 193129 1 2024-03-04T13:42:40.255 INFO:tasks.workunit.client.1.smithi169.stdout:5/115: dwrite d1/d16/d17/f1a [0,4194304] 0 2024-03-04T13:42:40.255 INFO:tasks.workunit.client.1.smithi169.stdout:3/56: write f6 [5203674,68073] 0 2024-03-04T13:42:40.269 INFO:tasks.workunit.client.1.smithi169.stdout:4/60: symlink d4/l16 0 2024-03-04T13:42:40.271 INFO:tasks.workunit.client.1.smithi169.stdout:3/57: creat d7/da/f12 x:0 0 0 2024-03-04T13:42:40.272 INFO:tasks.workunit.client.1.smithi169.stdout:2/68: mkdir d1/d5/d19/d1e 0 2024-03-04T13:42:40.272 INFO:tasks.workunit.client.1.smithi169.stdout:4/61: fsync d4/f15 0 2024-03-04T13:42:40.274 INFO:tasks.workunit.client.1.smithi169.stdout:7/100: sync 2024-03-04T13:42:40.275 INFO:tasks.workunit.client.1.smithi169.stdout:5/116: mkdir d1/d16/d17/d1b 0 2024-03-04T13:42:40.278 INFO:tasks.workunit.client.1.smithi169.stdout:0/99: rename d5/d9/dc/d12/d15/d19/c1f to d5/d9/dc/d1c/c26 0 2024-03-04T13:42:40.281 INFO:tasks.workunit.client.1.smithi169.stdout:6/61: getdents d0 0 2024-03-04T13:42:40.282 INFO:tasks.workunit.client.1.smithi169.stdout:9/99: dread d7/f11 [0,4194304] 0 2024-03-04T13:42:40.285 INFO:tasks.workunit.client.1.smithi169.stdout:8/72: dwrite d7/d12/f15 [0,4194304] 0 2024-03-04T13:42:40.286 INFO:tasks.workunit.client.1.smithi169.stdout:2/69: symlink d1/d5/d19/l1f 0 2024-03-04T13:42:40.290 INFO:tasks.workunit.client.1.smithi169.stdout:1/57: dwrite d4/d5/f8 [0,4194304] 0 2024-03-04T13:42:40.292 INFO:tasks.workunit.client.1.smithi169.stdout:7/101: chown d2/fd 52 1 2024-03-04T13:42:40.292 INFO:tasks.workunit.client.1.smithi169.stdout:3/58: write d7/de/ff [468624,121524] 0 2024-03-04T13:42:40.341 INFO:tasks.workunit.client.1.smithi169.stdout:4/62: rename d4/cd to d4/d11/c17 0 2024-03-04T13:42:40.344 INFO:tasks.workunit.client.1.smithi169.stdout:5/117: rename d1/d16/d17/d1b to d1/d16/d17/d1b/d1c 22 2024-03-04T13:42:40.345 INFO:tasks.workunit.client.1.smithi169.stdout:8/73: fsync d7/ff 0 2024-03-04T13:42:40.347 INFO:tasks.workunit.client.1.smithi169.stdout:3/59: mkdir d7/da/d13 0 2024-03-04T13:42:40.348 INFO:tasks.workunit.client.1.smithi169.stdout:4/63: truncate d4/f14 55243 0 2024-03-04T13:42:40.354 INFO:tasks.workunit.client.1.smithi169.stdout:9/100: getdents d7/dc/d12 0 2024-03-04T13:42:40.356 INFO:tasks.workunit.client.1.smithi169.stdout:6/62: dwrite d0/f8 [0,4194304] 0 2024-03-04T13:42:40.356 INFO:tasks.workunit.client.1.smithi169.stdout:3/60: dread d7/f9 [0,4194304] 0 2024-03-04T13:42:40.358 INFO:tasks.workunit.client.1.smithi169.stdout:3/61: readlink d7/l8 0 2024-03-04T13:42:40.361 INFO:tasks.workunit.client.1.smithi169.stdout:5/118: rmdir d1 39 2024-03-04T13:42:40.364 INFO:tasks.workunit.client.1.smithi169.stdout:2/70: dread d1/d5/fe [0,4194304] 0 2024-03-04T13:42:40.366 INFO:tasks.workunit.client.1.smithi169.stdout:7/102: creat d2/d15/f25 x:0 0 0 2024-03-04T13:42:40.367 INFO:tasks.workunit.client.1.smithi169.stdout:4/64: truncate d4/fa 146375 0 2024-03-04T13:42:40.368 INFO:tasks.workunit.client.1.smithi169.stdout:7/103: read d2/f7 [2194158,129251] 0 2024-03-04T13:42:40.369 INFO:tasks.workunit.client.1.smithi169.stdout:4/65: fdatasync d4/f6 0 2024-03-04T13:42:40.373 INFO:tasks.workunit.client.1.smithi169.stdout:9/101: symlink d7/dc/l22 0 2024-03-04T13:42:40.376 INFO:tasks.workunit.client.1.smithi169.stdout:9/102: readlink d7/dc/l22 0 2024-03-04T13:42:40.376 INFO:tasks.workunit.client.1.smithi169.stdout:3/62: rename d7/da to d7/de/d14 0 2024-03-04T13:42:40.377 INFO:tasks.workunit.client.1.smithi169.stdout:6/63: dwrite d0/f8 [0,4194304] 0 2024-03-04T13:42:40.380 INFO:tasks.workunit.client.1.smithi169.stdout:8/74: symlink d7/d10/d17/l19 0 2024-03-04T13:42:40.388 INFO:tasks.workunit.client.1.smithi169.stdout:9/103: truncate d7/dc/f1d 930224 0 2024-03-04T13:42:40.392 INFO:tasks.workunit.client.1.smithi169.stdout:5/119: symlink d1/d16/l1d 0 2024-03-04T13:42:40.395 INFO:tasks.workunit.client.1.smithi169.stdout:6/64: creat d0/ff x:0 0 0 2024-03-04T13:42:40.396 INFO:tasks.workunit.client.1.smithi169.stdout:1/58: dwrite f3 [0,4194304] 0 2024-03-04T13:42:40.397 INFO:tasks.workunit.client.1.smithi169.stdout:4/66: rename d4/d11/c17 to d4/d11/c18 0 2024-03-04T13:42:40.397 INFO:tasks.workunit.client.1.smithi169.stdout:2/71: dwrite d1/d5/f8 [4194304,4194304] 0 2024-03-04T13:42:40.399 INFO:tasks.workunit.client.1.smithi169.stdout:4/67: stat d4/cf 0 2024-03-04T13:42:40.399 INFO:tasks.workunit.client.1.smithi169.stdout:2/72: readlink d1/d5/d19/l1f 0 2024-03-04T13:42:40.399 INFO:tasks.workunit.client.1.smithi169.stdout:1/59: chown d4/le 106 1 2024-03-04T13:42:40.401 INFO:tasks.workunit.client.1.smithi169.stdout:1/60: chown d4/d5 7552 1 2024-03-04T13:42:40.401 INFO:tasks.workunit.client.1.smithi169.stdout:9/104: creat d7/dc/f23 x:0 0 0 2024-03-04T13:42:40.408 INFO:tasks.workunit.client.1.smithi169.stdout:6/65: symlink d0/l10 0 2024-03-04T13:42:40.408 INFO:tasks.workunit.client.1.smithi169.stdout:7/104: rename d2/de/l23 to d2/d15/l26 0 2024-03-04T13:42:40.416 INFO:tasks.workunit.client.1.smithi169.stdout:5/120: creat d1/d16/d17/d1b/f1e x:0 0 0 2024-03-04T13:42:40.419 INFO:tasks.workunit.client.1.smithi169.stdout:3/63: rename d7/de/ff to d7/de/d14/d13/f15 0 2024-03-04T13:42:40.421 INFO:tasks.workunit.client.1.smithi169.stdout:5/121: dread - d1/d16/d17/d1b/f1e zero size 2024-03-04T13:42:40.424 INFO:tasks.workunit.client.1.smithi169.stdout:7/105: dread d2/de/f12 [0,4194304] 0 2024-03-04T13:42:40.424 INFO:tasks.workunit.client.1.smithi169.stdout:1/61: dread f3 [0,4194304] 0 2024-03-04T13:42:40.427 INFO:tasks.workunit.client.1.smithi169.stdout:9/105: dwrite f4 [0,4194304] 0 2024-03-04T13:42:40.430 INFO:tasks.workunit.client.1.smithi169.stdout:5/122: unlink d1/f13 0 2024-03-04T13:42:40.431 INFO:tasks.workunit.client.1.smithi169.stdout:3/64: mknod d7/de/d14/c16 0 2024-03-04T13:42:40.432 INFO:tasks.workunit.client.1.smithi169.stdout:9/106: readlink d7/dc/d12/l15 0 2024-03-04T13:42:40.437 INFO:tasks.workunit.client.1.smithi169.stdout:7/106: symlink d2/de/l27 0 2024-03-04T13:42:40.442 INFO:tasks.workunit.client.1.smithi169.stdout:2/73: link d1/l17 d1/d5/l20 0 2024-03-04T13:42:40.443 INFO:tasks.workunit.client.1.smithi169.stdout:8/75: truncate d7/d12/f15 1633302 0 2024-03-04T13:42:40.447 INFO:tasks.workunit.client.1.smithi169.stdout:6/66: getdents d0 0 2024-03-04T13:42:40.456 INFO:tasks.workunit.client.1.smithi169.stdout:7/107: mkdir d2/d28 0 2024-03-04T13:42:40.459 INFO:tasks.workunit.client.1.smithi169.stdout:1/62: rename f3 to d4/f11 0 2024-03-04T13:42:40.460 INFO:tasks.workunit.client.1.smithi169.stdout:7/108: truncate d2/f1c 1115476 0 2024-03-04T13:42:40.460 INFO:tasks.workunit.client.1.smithi169.stdout:9/107: dwrite d7/dc/f1d [0,4194304] 0 2024-03-04T13:42:40.463 INFO:tasks.workunit.client.1.smithi169.stdout:6/67: rename d0/l5 to d0/l11 0 2024-03-04T13:42:40.464 INFO:tasks.workunit.client.1.smithi169.stdout:6/68: stat d0/l10 0 2024-03-04T13:42:40.468 INFO:tasks.workunit.client.1.smithi169.stdout:2/74: symlink d1/d5/d19/d1e/l21 0 2024-03-04T13:42:40.468 INFO:tasks.workunit.client.1.smithi169.stdout:5/123: write d1/f14 [412547,129788] 0 2024-03-04T13:42:40.472 INFO:tasks.workunit.client.1.smithi169.stdout:9/108: unlink f4 0 2024-03-04T13:42:40.472 INFO:tasks.workunit.client.1.smithi169.stdout:1/63: write d4/d5/f8 [1404827,45490] 0 2024-03-04T13:42:40.482 INFO:tasks.workunit.client.1.smithi169.stdout:1/64: write d4/d5/fb [1545190,104675] 0 2024-03-04T13:42:40.482 INFO:tasks.workunit.client.1.smithi169.stdout:6/69: readlink d0/l11 0 2024-03-04T13:42:40.487 INFO:tasks.workunit.client.1.smithi169.stdout:2/75: mkdir d1/d5/d22 0 2024-03-04T13:42:40.491 INFO:tasks.workunit.client.1.smithi169.stdout:7/109: dwrite d2/f1f [0,4194304] 0 2024-03-04T13:42:40.492 INFO:tasks.workunit.client.1.smithi169.stdout:9/109: creat d7/dc/d12/d20/f24 x:0 0 0 2024-03-04T13:42:40.493 INFO:tasks.workunit.client.1.smithi169.stdout:1/65: write d4/d5/f8 [4443465,46808] 0 2024-03-04T13:42:40.499 INFO:tasks.workunit.client.1.smithi169.stdout:6/70: creat d0/f12 x:0 0 0 2024-03-04T13:42:40.504 INFO:tasks.workunit.client.1.smithi169.stdout:8/76: dread d7/d12/f15 [0,4194304] 0 2024-03-04T13:42:40.505 INFO:tasks.workunit.client.1.smithi169.stdout:9/110: mkdir d7/d25 0 2024-03-04T13:42:40.506 INFO:tasks.workunit.client.1.smithi169.stdout:5/124: write d1/f9 [1414933,102945] 0 2024-03-04T13:42:40.507 INFO:tasks.workunit.client.1.smithi169.stdout:4/68: dwrite d4/fa [0,4194304] 0 2024-03-04T13:42:40.508 INFO:tasks.workunit.client.1.smithi169.stdout:6/71: read d0/f9 [3206697,5999] 0 2024-03-04T13:42:40.510 INFO:tasks.workunit.client.1.smithi169.stdout:4/69: chown d4/f6 7 1 2024-03-04T13:42:40.511 INFO:tasks.workunit.client.1.smithi169.stdout:4/70: readlink d4/l16 0 2024-03-04T13:42:40.515 INFO:tasks.workunit.client.1.smithi169.stdout:2/76: getdents d1 0 2024-03-04T13:42:40.518 INFO:tasks.workunit.client.1.smithi169.stdout:9/111: rmdir d7/dc/d12/d20 39 2024-03-04T13:42:40.522 INFO:tasks.workunit.client.1.smithi169.stdout:7/110: dread d2/f1c [0,4194304] 0 2024-03-04T13:42:40.523 INFO:tasks.workunit.client.1.smithi169.stdout:1/66: link d4/c6 d4/dd/c12 0 2024-03-04T13:42:40.526 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:42:40 smithi107 ceph-mon[28739]: pgmap v42: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.1 MiB/s rd, 9.1 MiB/s wr, 150 op/s 2024-03-04T13:42:40.527 INFO:tasks.workunit.client.1.smithi169.stdout:8/77: mkdir d7/d1a 0 2024-03-04T13:42:40.527 INFO:tasks.workunit.client.1.smithi169.stdout:1/67: stat d4/le 0 2024-03-04T13:42:40.528 INFO:tasks.workunit.client.1.smithi169.stdout:2/77: symlink d1/d5/d19/l23 0 2024-03-04T13:42:40.532 INFO:tasks.workunit.client.1.smithi169.stdout:1/68: readlink d4/d5/lc 0 2024-03-04T13:42:40.534 INFO:tasks.workunit.client.1.smithi169.stdout:2/78: fdatasync d1/d5/f9 0 2024-03-04T13:42:40.541 INFO:tasks.workunit.client.1.smithi169.stdout:1/69: mkdir d4/d13 0 2024-03-04T13:42:40.542 INFO:tasks.workunit.client.1.smithi169.stdout:5/125: dwrite d1/d16/d17/f1a [0,4194304] 0 2024-03-04T13:42:40.546 INFO:tasks.workunit.client.1.smithi169.stdout:8/78: getdents d7 0 2024-03-04T13:42:40.550 INFO:tasks.workunit.client.1.smithi169.stdout:1/70: symlink d4/d5/l14 0 2024-03-04T13:42:40.551 INFO:tasks.workunit.client.1.smithi169.stdout:2/79: dread d1/d5/f16 [0,4194304] 0 2024-03-04T13:42:40.554 INFO:tasks.workunit.client.1.smithi169.stdout:8/79: chown d7/ca 359582 1 2024-03-04T13:42:40.556 INFO:tasks.workunit.client.1.smithi169.stdout:5/126: mknod d1/d16/d17/d1b/c1f 0 2024-03-04T13:42:40.557 INFO:tasks.workunit.client.1.smithi169.stdout:8/80: stat d7/ld 0 2024-03-04T13:42:40.559 INFO:tasks.workunit.client.1.smithi169.stdout:6/72: dwrite d0/ff [0,4194304] 0 2024-03-04T13:42:40.560 INFO:tasks.workunit.client.1.smithi169.stdout:1/71: unlink d4/le 0 2024-03-04T13:42:40.561 INFO:tasks.workunit.client.1.smithi169.stdout:4/71: dread d4/f7 [0,4194304] 0 2024-03-04T13:42:40.563 INFO:tasks.workunit.client.1.smithi169.stdout:8/81: chown d7/ld 421256263 1 2024-03-04T13:42:40.563 INFO:tasks.workunit.client.1.smithi169.stdout:9/112: dwrite d7/dc/f1d [0,4194304] 0 2024-03-04T13:42:40.565 INFO:tasks.workunit.client.1.smithi169.stdout:5/127: chown d1/d16/l1d 0 1 2024-03-04T13:42:40.565 INFO:tasks.workunit.client.1.smithi169.stdout:7/111: dwrite d2/f22 [0,4194304] 0 2024-03-04T13:42:40.567 INFO:tasks.workunit.client.1.smithi169.stdout:9/113: chown d7/dc/f1e 194 1 2024-03-04T13:42:40.576 INFO:tasks.workunit.client.1.smithi169.stdout:6/73: mknod d0/c13 0 2024-03-04T13:42:40.577 INFO:tasks.workunit.client.1.smithi169.stdout:4/72: mkdir d4/d19 0 2024-03-04T13:42:40.582 INFO:tasks.workunit.client.1.smithi169.stdout:8/82: dread d7/d10/f16 [0,4194304] 0 2024-03-04T13:42:40.582 INFO:tasks.workunit.client.1.smithi169.stdout:7/112: unlink d2/d15/f25 0 2024-03-04T13:42:40.587 INFO:tasks.workunit.client.1.smithi169.stdout:5/128: rename d1/f6 to d1/d16/f20 0 2024-03-04T13:42:40.588 INFO:tasks.workunit.client.1.smithi169.stdout:6/74: mkdir d0/d14 0 2024-03-04T13:42:40.589 INFO:tasks.workunit.client.1.smithi169.stdout:4/73: rename d4 to d4/d19/d1a 22 2024-03-04T13:42:40.589 INFO:tasks.workunit.client.1.smithi169.stdout:9/114: dread - d7/dc/d12/d20/f24 zero size 2024-03-04T13:42:40.590 INFO:tasks.workunit.client.1.smithi169.stdout:8/83: truncate d7/ff 3116014 0 2024-03-04T13:42:40.591 INFO:tasks.workunit.client.1.smithi169.stdout:0/100: sync 2024-03-04T13:42:40.591 INFO:tasks.workunit.client.1.smithi169.stdout:7/113: mknod d2/d15/c29 0 2024-03-04T13:42:40.592 INFO:tasks.workunit.client.1.smithi169.stdout:2/80: dwrite d1/f1d [4194304,4194304] 0 2024-03-04T13:42:40.592 INFO:tasks.workunit.client.1.smithi169.stdout:6/75: creat d0/f15 x:0 0 0 2024-03-04T13:42:40.595 INFO:tasks.workunit.client.1.smithi169.stdout:6/76: readlink d0/lc 0 2024-03-04T13:42:40.597 INFO:tasks.workunit.client.1.smithi169.stdout:0/101: write d5/f16 [4218505,36716] 0 2024-03-04T13:42:40.599 INFO:tasks.workunit.client.1.smithi169.stdout:8/84: symlink d7/d10/d17/l1b 0 2024-03-04T13:42:40.602 INFO:tasks.workunit.client.1.smithi169.stdout:2/81: truncate d1/d5/d19/f1b 456245 0 2024-03-04T13:42:40.606 INFO:tasks.workunit.client.1.smithi169.stdout:1/72: truncate d4/f11 3941664 0 2024-03-04T13:42:40.607 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:42:40 smithi169 ceph-mon[36313]: pgmap v42: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.1 MiB/s rd, 9.1 MiB/s wr, 150 op/s 2024-03-04T13:42:40.613 INFO:tasks.workunit.client.1.smithi169.stdout:7/114: dread d2/f22 [0,4194304] 0 2024-03-04T13:42:40.615 INFO:tasks.workunit.client.1.smithi169.stdout:4/74: write d4/f7 [4342434,15142] 0 2024-03-04T13:42:40.618 INFO:tasks.workunit.client.1.smithi169.stdout:5/129: link d1/c7 d1/c21 0 2024-03-04T13:42:40.624 INFO:tasks.workunit.client.1.smithi169.stdout:9/115: truncate d7/dc/f1d 3403041 0 2024-03-04T13:42:40.624 INFO:tasks.workunit.client.1.smithi169.stdout:8/85: write d7/ff [1321237,54759] 0 2024-03-04T13:42:40.626 INFO:tasks.workunit.client.1.smithi169.stdout:7/115: creat d2/de/f2a x:0 0 0 2024-03-04T13:42:40.627 INFO:tasks.workunit.client.1.smithi169.stdout:5/130: fsync d1/fa 0 2024-03-04T13:42:40.631 INFO:tasks.workunit.client.1.smithi169.stdout:9/116: creat d7/f26 x:0 0 0 2024-03-04T13:42:40.631 INFO:tasks.workunit.client.1.smithi169.stdout:8/86: chown d7/ff 271789 1 2024-03-04T13:42:40.635 INFO:tasks.workunit.client.1.smithi169.stdout:7/116: write d2/fa [500236,70384] 0 2024-03-04T13:42:40.635 INFO:tasks.workunit.client.1.smithi169.stdout:5/131: stat d1/d16/d17/d1b/f1e 0 2024-03-04T13:42:40.637 INFO:tasks.workunit.client.1.smithi169.stdout:3/65: sync 2024-03-04T13:42:40.637 INFO:tasks.workunit.client.1.smithi169.stdout:6/77: dwrite d0/f9 [0,4194304] 0 2024-03-04T13:42:40.643 INFO:tasks.workunit.client.1.smithi169.stdout:0/102: dwrite d5/f6 [0,4194304] 0 2024-03-04T13:42:40.646 INFO:tasks.workunit.client.1.smithi169.stdout:7/117: readlink d2/de/l13 0 2024-03-04T13:42:40.647 INFO:tasks.workunit.client.1.smithi169.stdout:9/117: dread f6 [0,4194304] 0 2024-03-04T13:42:40.652 INFO:tasks.workunit.client.1.smithi169.stdout:8/87: truncate d7/d12/f15 608094 0 2024-03-04T13:42:40.652 INFO:tasks.workunit.client.1.smithi169.stdout:7/118: fsync d2/f1c 0 2024-03-04T13:42:40.653 INFO:tasks.workunit.client.1.smithi169.stdout:2/82: dwrite d1/d5/f16 [0,4194304] 0 2024-03-04T13:42:40.655 INFO:tasks.workunit.client.1.smithi169.stdout:9/118: write d7/f26 [266838,119986] 0 2024-03-04T13:42:40.669 INFO:tasks.workunit.client.1.smithi169.stdout:8/88: creat d7/d12/f1c x:0 0 0 2024-03-04T13:42:40.669 INFO:tasks.workunit.client.1.smithi169.stdout:9/119: mkdir d7/d27 0 2024-03-04T13:42:40.671 INFO:tasks.workunit.client.1.smithi169.stdout:5/132: link d1/d16/d17/c19 d1/d16/d17/d1b/c22 0 2024-03-04T13:42:40.672 INFO:tasks.workunit.client.1.smithi169.stdout:4/75: dwrite d4/f15 [0,4194304] 0 2024-03-04T13:42:40.673 INFO:tasks.workunit.client.1.smithi169.stdout:6/78: truncate d0/f1 3755361 0 2024-03-04T13:42:40.677 INFO:tasks.workunit.client.1.smithi169.stdout:8/89: unlink d7/d10/f16 0 2024-03-04T13:42:40.680 INFO:tasks.workunit.client.1.smithi169.stdout:3/66: dwrite f4 [0,4194304] 0 2024-03-04T13:42:40.682 INFO:tasks.workunit.client.1.smithi169.stdout:0/103: link d5/d9/dc/f1d d5/d9/dc/d12/f27 0 2024-03-04T13:42:40.689 INFO:tasks.workunit.client.1.smithi169.stdout:6/79: creat d0/f16 x:0 0 0 2024-03-04T13:42:40.691 INFO:tasks.workunit.client.1.smithi169.stdout:3/67: symlink d7/de/l17 0 2024-03-04T13:42:40.693 INFO:tasks.workunit.client.1.smithi169.stdout:8/90: symlink d7/l1d 0 2024-03-04T13:42:40.698 INFO:tasks.workunit.client.1.smithi169.stdout:0/104: creat d5/d9/dc/d12/d15/d19/f28 x:0 0 0 2024-03-04T13:42:40.703 INFO:tasks.workunit.client.1.smithi169.stdout:3/68: mkdir d7/de/d14/d18 0 2024-03-04T13:42:40.705 INFO:tasks.workunit.client.1.smithi169.stdout:6/80: rename d0/c13 to d0/c17 0 2024-03-04T13:42:40.706 INFO:tasks.workunit.client.1.smithi169.stdout:8/91: creat d7/d10/f1e x:0 0 0 2024-03-04T13:42:40.711 INFO:tasks.workunit.client.1.smithi169.stdout:6/81: stat d0 0 2024-03-04T13:42:40.711 INFO:tasks.workunit.client.1.smithi169.stdout:0/105: write d5/f6 [1038491,127154] 0 2024-03-04T13:42:40.720 INFO:tasks.workunit.client.1.smithi169.stdout:2/83: dwrite d1/d5/f8 [8388608,4194304] 0 2024-03-04T13:42:40.726 INFO:tasks.workunit.client.1.smithi169.stdout:9/120: dwrite d7/dc/d12/f1f [0,4194304] 0 2024-03-04T13:42:40.728 INFO:tasks.workunit.client.1.smithi169.stdout:4/76: dread d4/f7 [0,4194304] 0 2024-03-04T13:42:40.734 INFO:tasks.workunit.client.1.smithi169.stdout:9/121: dread - d7/dc/d12/d20/f24 zero size 2024-03-04T13:42:40.736 INFO:tasks.workunit.client.1.smithi169.stdout:5/133: dwrite d1/f9 [0,4194304] 0 2024-03-04T13:42:40.736 INFO:tasks.workunit.client.1.smithi169.stdout:4/77: fdatasync d4/f6 0 2024-03-04T13:42:40.737 INFO:tasks.workunit.client.1.smithi169.stdout:8/92: mknod d7/d1a/c1f 0 2024-03-04T13:42:40.739 INFO:tasks.workunit.client.1.smithi169.stdout:8/93: fsync d7/ff 0 2024-03-04T13:42:40.740 INFO:tasks.workunit.client.1.smithi169.stdout:9/122: truncate d7/fb 1224305 0 2024-03-04T13:42:40.740 INFO:tasks.workunit.client.1.smithi169.stdout:3/69: creat d7/de/d14/d18/f19 x:0 0 0 2024-03-04T13:42:40.740 INFO:tasks.workunit.client.1.smithi169.stdout:8/94: readlink d7/l11 0 2024-03-04T13:42:40.741 INFO:tasks.workunit.client.1.smithi169.stdout:4/78: write d4/f14 [605993,77307] 0 2024-03-04T13:42:40.741 INFO:tasks.workunit.client.1.smithi169.stdout:5/134: write d1/f9 [3194437,34642] 0 2024-03-04T13:42:40.745 INFO:tasks.workunit.client.1.smithi169.stdout:4/79: chown d4/f9 550708 1 2024-03-04T13:42:40.746 INFO:tasks.workunit.client.1.smithi169.stdout:2/84: rename d1/d5/d19/f1b to d1/d5/d19/d1e/f24 0 2024-03-04T13:42:40.750 INFO:tasks.workunit.client.1.smithi169.stdout:6/82: dread d0/ff [0,4194304] 0 2024-03-04T13:42:40.760 INFO:tasks.workunit.client.1.smithi169.stdout:8/95: creat d7/d10/d17/f20 x:0 0 0 2024-03-04T13:42:40.769 INFO:tasks.workunit.client.1.smithi169.stdout:4/80: creat d4/d11/f1b x:0 0 0 2024-03-04T13:42:40.769 INFO:tasks.workunit.client.1.smithi169.stdout:3/70: dwrite d7/de/d14/d18/f19 [0,4194304] 0 2024-03-04T13:42:40.771 INFO:tasks.workunit.client.1.smithi169.stdout:2/85: mknod d1/d5/c25 0 2024-03-04T13:42:40.775 INFO:tasks.workunit.client.1.smithi169.stdout:9/123: rename d7/l21 to d7/dc/l28 0 2024-03-04T13:42:40.776 INFO:tasks.workunit.client.1.smithi169.stdout:6/83: dwrite d0/f8 [0,4194304] 0 2024-03-04T13:42:40.783 INFO:tasks.workunit.client.1.smithi169.stdout:9/124: dread f6 [0,4194304] 0 2024-03-04T13:42:40.783 INFO:tasks.workunit.client.1.smithi169.stdout:8/96: dread d7/ff [0,4194304] 0 2024-03-04T13:42:40.783 INFO:tasks.workunit.client.1.smithi169.stdout:0/106: link d5/cd d5/d9/dc/c29 0 2024-03-04T13:42:40.788 INFO:tasks.workunit.client.1.smithi169.stdout:0/107: chown d5/d9/dc/d1c/f23 63603 1 2024-03-04T13:42:40.792 INFO:tasks.workunit.client.1.smithi169.stdout:5/135: dwrite d1/d16/f20 [0,4194304] 0 2024-03-04T13:42:40.795 INFO:tasks.workunit.client.1.smithi169.stdout:5/136: readlink d1/d16/l1d 0 2024-03-04T13:42:40.796 INFO:tasks.workunit.client.1.smithi169.stdout:1/73: sync 2024-03-04T13:42:40.798 INFO:tasks.workunit.client.1.smithi169.stdout:2/86: unlink d1/f1d 0 2024-03-04T13:42:40.801 INFO:tasks.workunit.client.1.smithi169.stdout:3/71: mkdir d7/de/d14/d13/d1a 0 2024-03-04T13:42:40.810 INFO:tasks.workunit.client.1.smithi169.stdout:4/81: dwrite d4/f9 [0,4194304] 0 2024-03-04T13:42:40.812 INFO:tasks.workunit.client.1.smithi169.stdout:0/108: rename c2 to d5/d9/dc/d12/d15/c2a 0 2024-03-04T13:42:40.813 INFO:tasks.workunit.client.1.smithi169.stdout:4/82: write d4/d11/f1b [442211,120833] 0 2024-03-04T13:42:40.819 INFO:tasks.workunit.client.1.smithi169.stdout:1/74: mknod d4/d5/c15 0 2024-03-04T13:42:40.819 INFO:tasks.workunit.client.1.smithi169.stdout:4/83: dread d4/f14 [0,4194304] 0 2024-03-04T13:42:40.822 INFO:tasks.workunit.client.1.smithi169.stdout:3/72: rmdir d7/de 39 2024-03-04T13:42:40.824 INFO:tasks.workunit.client.1.smithi169.stdout:9/125: symlink d7/dc/d12/l29 0 2024-03-04T13:42:40.826 INFO:tasks.workunit.client.1.smithi169.stdout:8/97: getdents d7/d10/d14/d18 0 2024-03-04T13:42:40.829 INFO:tasks.workunit.client.1.smithi169.stdout:0/109: rename d5/l13 to d5/l2b 0 2024-03-04T13:42:40.834 INFO:tasks.workunit.client.1.smithi169.stdout:5/137: mkdir d1/d23 0 2024-03-04T13:42:40.837 INFO:tasks.workunit.client.1.smithi169.stdout:2/87: dwrite d1/d5/d19/d1e/f24 [0,4194304] 0 2024-03-04T13:42:40.838 INFO:tasks.workunit.client.1.smithi169.stdout:0/110: write d5/d9/dc/d12/d15/d19/f20 [860131,12061] 0 2024-03-04T13:42:40.838 INFO:tasks.workunit.client.1.smithi169.stdout:4/84: mkdir d4/d1c 0 2024-03-04T13:42:40.842 INFO:tasks.workunit.client.1.smithi169.stdout:9/126: mknod d7/dc/d12/d20/c2a 0 2024-03-04T13:42:40.845 INFO:tasks.workunit.client.1.smithi169.stdout:8/98: rmdir d7/d12 39 2024-03-04T13:42:40.847 INFO:tasks.workunit.client.1.smithi169.stdout:9/127: fsync d7/dc/d12/f1f 0 2024-03-04T13:42:40.849 INFO:tasks.workunit.client.1.smithi169.stdout:1/75: fsync d4/f11 0 2024-03-04T13:42:40.852 INFO:tasks.workunit.client.1.smithi169.stdout:4/85: chown d4/c12 338320311 1 2024-03-04T13:42:40.859 INFO:tasks.workunit.client.1.smithi169.stdout:8/99: mknod d7/d1a/c21 0 2024-03-04T13:42:40.860 INFO:tasks.workunit.client.1.smithi169.stdout:2/88: dwrite d1/d5/f16 [0,4194304] 0 2024-03-04T13:42:40.863 INFO:tasks.workunit.client.1.smithi169.stdout:3/73: rename d7/de/d14/c16 to d7/de/d14/d18/c1b 0 2024-03-04T13:42:40.864 INFO:tasks.workunit.client.1.smithi169.stdout:2/89: chown d1/d5/d19/l23 0 1 2024-03-04T13:42:40.871 INFO:tasks.workunit.client.1.smithi169.stdout:8/100: dread d7/ff [0,4194304] 0 2024-03-04T13:42:40.872 INFO:tasks.workunit.client.1.smithi169.stdout:1/76: mkdir d4/dd/d16 0 2024-03-04T13:42:40.874 INFO:tasks.workunit.client.1.smithi169.stdout:4/86: mknod d4/d11/c1d 0 2024-03-04T13:42:40.881 INFO:tasks.workunit.client.1.smithi169.stdout:5/138: creat d1/f24 x:0 0 0 2024-03-04T13:42:40.886 INFO:tasks.workunit.client.1.smithi169.stdout:0/111: write d5/d9/dc/d12/f27 [4213577,75029] 0 2024-03-04T13:42:40.890 INFO:tasks.workunit.client.1.smithi169.stdout:0/112: write d5/d9/dc/d1c/f21 [3104804,19743] 0 2024-03-04T13:42:40.890 INFO:tasks.workunit.client.1.smithi169.stdout:1/77: dwrite d4/d5/fb [0,4194304] 0 2024-03-04T13:42:40.890 INFO:tasks.workunit.client.1.smithi169.stdout:9/128: creat d7/d27/f2b x:0 0 0 2024-03-04T13:42:40.899 INFO:tasks.workunit.client.1.smithi169.stdout:4/87: rename d4/d11/c13 to d4/d11/c1e 0 2024-03-04T13:42:40.906 INFO:tasks.workunit.client.1.smithi169.stdout:2/90: unlink d1/d5/f16 0 2024-03-04T13:42:40.908 INFO:tasks.workunit.client.1.smithi169.stdout:7/119: sync 2024-03-04T13:42:40.909 INFO:tasks.workunit.client.1.smithi169.stdout:8/101: dwrite d7/d12/f1c [0,4194304] 0 2024-03-04T13:42:40.912 INFO:tasks.workunit.client.1.smithi169.stdout:8/102: write d7/d10/f1e [136139,103451] 0 2024-03-04T13:42:40.919 INFO:tasks.workunit.client.1.smithi169.stdout:0/113: rename d5/d9/dc/d12/f27 to d5/d9/dc/d1c/f2c 0 2024-03-04T13:42:40.924 INFO:tasks.workunit.client.1.smithi169.stdout:3/74: dwrite d7/f9 [0,4194304] 0 2024-03-04T13:42:40.929 INFO:tasks.workunit.client.1.smithi169.stdout:2/91: write d1/d5/fe [3639025,90934] 0 2024-03-04T13:42:40.933 INFO:tasks.workunit.client.1.smithi169.stdout:9/129: mknod d7/d25/c2c 0 2024-03-04T13:42:40.941 INFO:tasks.workunit.client.1.smithi169.stdout:8/103: rename d7/d10/d14/d18 to d7/d10/d22 0 2024-03-04T13:42:40.943 INFO:tasks.workunit.client.1.smithi169.stdout:0/114: creat d5/d9/f2d x:0 0 0 2024-03-04T13:42:40.944 INFO:tasks.workunit.client.1.smithi169.stdout:0/115: write d5/f6 [1940709,38857] 0 2024-03-04T13:42:40.945 INFO:tasks.workunit.client.1.smithi169.stdout:4/88: mknod d4/d1c/c1f 0 2024-03-04T13:42:40.946 INFO:tasks.workunit.client.1.smithi169.stdout:4/89: chown d4/f14 13 1 2024-03-04T13:42:40.949 INFO:tasks.workunit.client.1.smithi169.stdout:7/120: symlink d2/d15/l2b 0 2024-03-04T13:42:40.950 INFO:tasks.workunit.client.1.smithi169.stdout:7/121: fsync d2/fa 0 2024-03-04T13:42:40.951 INFO:tasks.workunit.client.1.smithi169.stdout:1/78: link d4/d5/c15 d4/d5/c17 0 2024-03-04T13:42:40.952 INFO:tasks.workunit.client.1.smithi169.stdout:7/122: readlink d2/d15/d17/l19 0 2024-03-04T13:42:40.954 INFO:tasks.workunit.client.1.smithi169.stdout:8/104: symlink d7/d10/d14/l23 0 2024-03-04T13:42:40.955 INFO:tasks.workunit.client.1.smithi169.stdout:5/139: getdents d1/d16/d17/d1b 0 2024-03-04T13:42:40.957 INFO:tasks.workunit.client.1.smithi169.stdout:4/90: mkdir d4/d20 0 2024-03-04T13:42:40.958 INFO:tasks.workunit.client.1.smithi169.stdout:6/84: sync 2024-03-04T13:42:40.958 INFO:tasks.workunit.client.1.smithi169.stdout:4/91: chown d4/l16 1891563309 1 2024-03-04T13:42:40.960 INFO:tasks.workunit.client.1.smithi169.stdout:0/116: rename d5/f6 to d5/d9/dc/d12/d15/d19/f2e 0 2024-03-04T13:42:40.961 INFO:tasks.workunit.client.1.smithi169.stdout:0/117: chown l4 402669 1 2024-03-04T13:42:40.962 INFO:tasks.workunit.client.1.smithi169.stdout:2/92: symlink d1/d5/d22/l26 0 2024-03-04T13:42:40.963 INFO:tasks.workunit.client.1.smithi169.stdout:9/130: truncate d7/dc/f1d 3507262 0 2024-03-04T13:42:40.964 INFO:tasks.workunit.client.1.smithi169.stdout:1/79: dread d4/d5/f8 [0,4194304] 0 2024-03-04T13:42:40.965 INFO:tasks.workunit.client.1.smithi169.stdout:7/123: creat d2/f2c x:0 0 0 2024-03-04T13:42:40.965 INFO:tasks.workunit.client.1.smithi169.stdout:8/105: mkdir d7/d10/d24 0 2024-03-04T13:42:40.969 INFO:tasks.workunit.client.1.smithi169.stdout:6/85: unlink d0/ff 0 2024-03-04T13:42:40.971 INFO:tasks.workunit.client.1.smithi169.stdout:4/92: rename d4/d20 to d4/d19/d21 0 2024-03-04T13:42:40.972 INFO:tasks.workunit.client.1.smithi169.stdout:2/93: creat d1/d5/d19/f27 x:0 0 0 2024-03-04T13:42:40.974 INFO:tasks.workunit.client.1.smithi169.stdout:2/94: stat d1/d5/l1c 0 2024-03-04T13:42:40.975 INFO:tasks.workunit.client.1.smithi169.stdout:9/131: mkdir d7/d2d 0 2024-03-04T13:42:40.977 INFO:tasks.workunit.client.1.smithi169.stdout:1/80: write d4/d5/f8 [3094363,4933] 0 2024-03-04T13:42:40.977 INFO:tasks.workunit.client.1.smithi169.stdout:7/124: creat d2/f2d x:0 0 0 2024-03-04T13:42:40.978 INFO:tasks.workunit.client.1.smithi169.stdout:5/140: link d1/f8 d1/d16/d17/d1b/f25 0 2024-03-04T13:42:40.979 INFO:tasks.workunit.client.1.smithi169.stdout:2/95: write d1/d5/f8 [9676655,9148] 0 2024-03-04T13:42:40.982 INFO:tasks.workunit.client.1.smithi169.stdout:3/75: getdents d7 0 2024-03-04T13:42:40.982 INFO:tasks.workunit.client.1.smithi169.stdout:4/93: truncate d4/f14 640140 0 2024-03-04T13:42:40.987 INFO:tasks.workunit.client.1.smithi169.stdout:5/141: write d1/d16/d17/d1b/f1e [547495,74677] 0 2024-03-04T13:42:40.988 INFO:tasks.workunit.client.1.smithi169.stdout:1/81: rename d4/l10 to d4/d13/l18 0 2024-03-04T13:42:40.991 INFO:tasks.workunit.client.1.smithi169.stdout:4/94: symlink d4/l22 0 2024-03-04T13:42:40.992 INFO:tasks.workunit.client.1.smithi169.stdout:9/132: symlink d7/d2d/l2e 0 2024-03-04T13:42:40.993 INFO:tasks.workunit.client.1.smithi169.stdout:0/118: dwrite d5/d9/dc/d12/d15/d19/f20 [0,4194304] 0 2024-03-04T13:42:40.995 INFO:tasks.workunit.client.1.smithi169.stdout:6/86: dwrite d0/f8 [0,4194304] 0 2024-03-04T13:42:40.998 INFO:tasks.workunit.client.1.smithi169.stdout:7/125: symlink d2/d28/l2e 0 2024-03-04T13:42:41.007 INFO:tasks.workunit.client.1.smithi169.stdout:4/95: mkdir d4/d11/d23 0 2024-03-04T13:42:41.009 INFO:tasks.workunit.client.1.smithi169.stdout:9/133: creat d7/d27/f2f x:0 0 0 2024-03-04T13:42:41.012 INFO:tasks.workunit.client.1.smithi169.stdout:3/76: symlink d7/de/d14/d13/d1a/l1c 0 2024-03-04T13:42:41.014 INFO:tasks.workunit.client.1.smithi169.stdout:1/82: mknod d4/c19 0 2024-03-04T13:42:41.018 INFO:tasks.workunit.client.1.smithi169.stdout:6/87: dread d0/f8 [0,4194304] 0 2024-03-04T13:42:41.021 INFO:tasks.workunit.client.1.smithi169.stdout:7/126: dwrite d2/de/f1e [0,4194304] 0 2024-03-04T13:42:41.028 INFO:tasks.workunit.client.1.smithi169.stdout:0/119: link d5/d9/dc/d1c/f21 d5/d9/dc/d12/d15/d19/f2f 0 2024-03-04T13:42:41.032 INFO:tasks.workunit.client.1.smithi169.stdout:3/77: rename d7/de/d14/d18/f19 to d7/f1d 0 2024-03-04T13:42:41.035 INFO:tasks.workunit.client.1.smithi169.stdout:2/96: getdents d1/d5/d19 0 2024-03-04T13:42:41.037 INFO:tasks.workunit.client.1.smithi169.stdout:5/142: getdents d1/d16 0 2024-03-04T13:42:41.038 INFO:tasks.workunit.client.1.smithi169.stdout:2/97: chown d1/d5/d22 418682218 1 2024-03-04T13:42:41.041 INFO:tasks.workunit.client.1.smithi169.stdout:9/134: dwrite d7/dc/fd [0,4194304] 0 2024-03-04T13:42:41.045 INFO:tasks.workunit.client.1.smithi169.stdout:8/106: dwrite d7/d12/f15 [0,4194304] 0 2024-03-04T13:42:41.046 INFO:tasks.workunit.client.1.smithi169.stdout:0/120: write d5/d9/dc/d12/f24 [1602863,96338] 0 2024-03-04T13:42:41.046 INFO:tasks.workunit.client.1.smithi169.stdout:6/88: mknod d0/d14/c18 0 2024-03-04T13:42:41.047 INFO:tasks.workunit.client.1.smithi169.stdout:9/135: chown d7/dc/d12/d20/c2a 6 1 2024-03-04T13:42:41.050 INFO:tasks.workunit.client.1.smithi169.stdout:7/127: symlink d2/d15/l2f 0 2024-03-04T13:42:41.052 INFO:tasks.workunit.client.1.smithi169.stdout:7/128: chown d2/d15/l20 34 1 2024-03-04T13:42:41.057 INFO:tasks.workunit.client.1.smithi169.stdout:5/143: stat d1/d16/d17/c19 0 2024-03-04T13:42:41.061 INFO:tasks.workunit.client.1.smithi169.stdout:2/98: rename d1/d5/c10 to d1/d5/d19/c28 0 2024-03-04T13:42:41.067 INFO:tasks.workunit.client.1.smithi169.stdout:0/121: mknod d5/c30 0 2024-03-04T13:42:41.067 INFO:tasks.workunit.client.1.smithi169.stdout:5/144: write d1/f15 [958614,66598] 0 2024-03-04T13:42:41.073 INFO:tasks.workunit.client.1.smithi169.stdout:1/83: getdents d4/dd 0 2024-03-04T13:42:41.075 INFO:tasks.workunit.client.1.smithi169.stdout:8/107: dread d7/d12/f1c [0,4194304] 0 2024-03-04T13:42:41.075 INFO:tasks.workunit.client.1.smithi169.stdout:6/89: dwrite d0/f12 [0,4194304] 0 2024-03-04T13:42:41.079 INFO:tasks.workunit.client.1.smithi169.stdout:0/122: chown d5/d9/dc/d12/d15/d19/f20 62716 1 2024-03-04T13:42:41.079 INFO:tasks.workunit.client.1.smithi169.stdout:2/99: mkdir d1/d5/d19/d1e/d29 0 2024-03-04T13:42:41.085 INFO:tasks.workunit.client.1.smithi169.stdout:8/108: dread - d7/d10/d17/f20 zero size 2024-03-04T13:42:41.086 INFO:tasks.workunit.client.1.smithi169.stdout:9/136: dwrite d7/d27/f2b [0,4194304] 0 2024-03-04T13:42:41.087 INFO:tasks.workunit.client.1.smithi169.stdout:4/96: dread d4/f14 [0,4194304] 0 2024-03-04T13:42:41.096 INFO:tasks.workunit.client.1.smithi169.stdout:1/84: creat d4/d5/f1a x:0 0 0 2024-03-04T13:42:41.097 INFO:tasks.workunit.client.1.smithi169.stdout:6/90: mknod d0/c19 0 2024-03-04T13:42:41.097 INFO:tasks.workunit.client.1.smithi169.stdout:3/78: dread d7/f9 [0,4194304] 0 2024-03-04T13:42:41.101 INFO:tasks.workunit.client.1.smithi169.stdout:0/123: mknod d5/d9/dc/d12/c31 0 2024-03-04T13:42:41.101 INFO:tasks.workunit.client.1.smithi169.stdout:2/100: rename d1/c4 to d1/d5/d19/d1e/d29/c2a 0 2024-03-04T13:42:41.106 INFO:tasks.workunit.client.1.smithi169.stdout:4/97: rename d4/d11/c1d to d4/d11/c24 0 2024-03-04T13:42:41.107 INFO:tasks.workunit.client.1.smithi169.stdout:5/145: write d1/f8 [4467422,66381] 0 2024-03-04T13:42:41.107 INFO:tasks.workunit.client.1.smithi169.stdout:1/85: unlink d4/d5/c15 0 2024-03-04T13:42:41.108 INFO:tasks.workunit.client.1.smithi169.stdout:7/129: link d2/d15/d17/l19 d2/d15/l30 0 2024-03-04T13:42:41.113 INFO:tasks.workunit.client.1.smithi169.stdout:8/109: link d7/d12/f15 d7/d10/d22/f25 0 2024-03-04T13:42:41.118 INFO:tasks.workunit.client.1.smithi169.stdout:0/124: write d5/d9/dc/f1d [4035929,129900] 0 2024-03-04T13:42:41.123 INFO:tasks.workunit.client.1.smithi169.stdout:6/91: rmdir d0 39 2024-03-04T13:42:41.125 INFO:tasks.workunit.client.1.smithi169.stdout:7/130: dread d2/fa [0,4194304] 0 2024-03-04T13:42:41.127 INFO:tasks.workunit.client.1.smithi169.stdout:0/125: creat d5/d9/dc/d12/d15/d19/f32 x:0 0 0 2024-03-04T13:42:41.132 INFO:tasks.workunit.client.1.smithi169.stdout:1/86: creat d4/f1b x:0 0 0 2024-03-04T13:42:41.135 INFO:tasks.workunit.client.1.smithi169.stdout:2/101: dwrite d1/d5/f9 [0,4194304] 0 2024-03-04T13:42:41.141 INFO:tasks.workunit.client.1.smithi169.stdout:2/102: readlink d1/d5/d19/l23 0 2024-03-04T13:42:41.142 INFO:tasks.workunit.client.1.smithi169.stdout:3/79: getdents d7/de/d14/d18 0 2024-03-04T13:42:41.143 INFO:tasks.workunit.client.1.smithi169.stdout:5/146: dwrite d1/d16/d17/f1a [0,4194304] 0 2024-03-04T13:42:41.144 INFO:tasks.workunit.client.1.smithi169.stdout:6/92: truncate d0/f1 2855789 0 2024-03-04T13:42:41.151 INFO:tasks.workunit.client.1.smithi169.stdout:8/110: dwrite d7/d12/f15 [0,4194304] 0 2024-03-04T13:42:41.152 INFO:tasks.workunit.client.1.smithi169.stdout:7/131: dread d2/f7 [0,4194304] 0 2024-03-04T13:42:41.152 INFO:tasks.workunit.client.1.smithi169.stdout:5/147: mknod d1/d16/d17/d1b/c26 0 2024-03-04T13:42:41.155 INFO:tasks.workunit.client.1.smithi169.stdout:5/148: fsync d1/f15 0 2024-03-04T13:42:41.158 INFO:tasks.workunit.client.1.smithi169.stdout:0/126: dread d5/d9/dc/d12/d15/d19/f2e [0,4194304] 0 2024-03-04T13:42:41.163 INFO:tasks.workunit.client.1.smithi169.stdout:5/149: fdatasync d1/d16/d17/f18 0 2024-03-04T13:42:41.166 INFO:tasks.workunit.client.1.smithi169.stdout:2/103: dwrite d1/ff [0,4194304] 0 2024-03-04T13:42:41.169 INFO:tasks.workunit.client.1.smithi169.stdout:0/127: creat d5/d9/dc/d1c/f33 x:0 0 0 2024-03-04T13:42:41.174 INFO:tasks.workunit.client.1.smithi169.stdout:6/93: link d0/l10 d0/l1a 0 2024-03-04T13:42:41.175 INFO:tasks.workunit.client.1.smithi169.stdout:3/80: getdents d7/de/d14/d13/d1a 0 2024-03-04T13:42:41.178 INFO:tasks.workunit.client.1.smithi169.stdout:0/128: chown d5/d9/dc/d12 13065 1 2024-03-04T13:42:41.179 INFO:tasks.workunit.client.1.smithi169.stdout:1/87: rename d4/d5/f8 to d4/dd/df/f1c 0 2024-03-04T13:42:41.180 INFO:tasks.workunit.client.1.smithi169.stdout:6/94: write d0/f15 [1004302,60835] 0 2024-03-04T13:42:41.187 INFO:tasks.workunit.client.1.smithi169.stdout:7/132: getdents d2/d15 0 2024-03-04T13:42:41.191 INFO:tasks.workunit.client.1.smithi169.stdout:6/95: creat d0/f1b x:0 0 0 2024-03-04T13:42:41.192 INFO:tasks.workunit.client.1.smithi169.stdout:0/129: dread - d5/d9/dc/d12/d15/d19/f28 zero size 2024-03-04T13:42:41.193 INFO:tasks.workunit.client.1.smithi169.stdout:7/133: mknod d2/c31 0 2024-03-04T13:42:41.195 INFO:tasks.workunit.client.1.smithi169.stdout:5/150: rename d1/d16/d17/d1b/c26 to d1/c27 0 2024-03-04T13:42:41.196 INFO:tasks.workunit.client.1.smithi169.stdout:3/81: link d7/de/d14/l11 d7/de/d14/d13/l1e 0 2024-03-04T13:42:41.199 INFO:tasks.workunit.client.1.smithi169.stdout:8/111: dwrite d7/ff [0,4194304] 0 2024-03-04T13:42:41.203 INFO:tasks.workunit.client.1.smithi169.stdout:7/134: rename d2/de/f12 to d2/d28/f32 0 2024-03-04T13:42:41.204 INFO:tasks.workunit.client.1.smithi169.stdout:2/104: dwrite d1/d5/fe [4194304,4194304] 0 2024-03-04T13:42:41.207 INFO:tasks.workunit.client.1.smithi169.stdout:2/105: chown d1/ff 36203310 1 2024-03-04T13:42:41.210 INFO:tasks.workunit.client.1.smithi169.stdout:0/130: rmdir d5/d9/dc/d12/d15 39 2024-03-04T13:42:41.212 INFO:tasks.workunit.client.1.smithi169.stdout:1/88: dread d4/f11 [0,4194304] 0 2024-03-04T13:42:41.216 INFO:tasks.workunit.client.1.smithi169.stdout:1/89: truncate d4/dd/df/f1c 5479217 0 2024-03-04T13:42:41.218 INFO:tasks.workunit.client.1.smithi169.stdout:8/112: rename d7/l1d to d7/d12/l26 0 2024-03-04T13:42:41.218 INFO:tasks.workunit.client.1.smithi169.stdout:8/113: chown l2 95016501 1 2024-03-04T13:42:41.220 INFO:tasks.workunit.client.1.smithi169.stdout:7/135: creat d2/d28/f33 x:0 0 0 2024-03-04T13:42:41.221 INFO:tasks.workunit.client.1.smithi169.stdout:2/106: symlink d1/d5/l2b 0 2024-03-04T13:42:41.226 INFO:tasks.workunit.client.1.smithi169.stdout:6/96: dwrite d0/f15 [0,4194304] 0 2024-03-04T13:42:41.230 INFO:tasks.workunit.client.1.smithi169.stdout:7/136: creat d2/f34 x:0 0 0 2024-03-04T13:42:41.232 INFO:tasks.workunit.client.1.smithi169.stdout:8/114: creat d7/d10/d17/f27 x:0 0 0 2024-03-04T13:42:41.233 INFO:tasks.workunit.client.1.smithi169.stdout:1/90: mkdir d4/d1d 0 2024-03-04T13:42:41.233 INFO:tasks.workunit.client.1.smithi169.stdout:0/131: unlink d5/d9/dc/c29 0 2024-03-04T13:42:41.234 INFO:tasks.workunit.client.1.smithi169.stdout:2/107: fdatasync d1/d5/fe 0 2024-03-04T13:42:41.236 INFO:tasks.workunit.client.1.smithi169.stdout:3/82: link d7/de/d14/d13/d1a/l1c d7/de/d14/d13/l1f 0 2024-03-04T13:42:41.245 INFO:tasks.workunit.client.1.smithi169.stdout:3/83: creat d7/de/d14/f20 x:0 0 0 2024-03-04T13:42:41.245 INFO:tasks.workunit.client.1.smithi169.stdout:8/115: symlink d7/d10/d24/l28 0 2024-03-04T13:42:41.246 INFO:tasks.workunit.client.1.smithi169.stdout:0/132: dread - d5/d9/dc/d12/d15/d19/f28 zero size 2024-03-04T13:42:41.251 INFO:tasks.workunit.client.1.smithi169.stdout:6/97: dwrite d0/f9 [0,4194304] 0 2024-03-04T13:42:41.251 INFO:tasks.workunit.client.1.smithi169.stdout:8/116: creat d7/d10/d17/f29 x:0 0 0 2024-03-04T13:42:41.257 INFO:tasks.workunit.client.1.smithi169.stdout:2/108: link d1/d5/d19/c28 d1/c2c 0 2024-03-04T13:42:41.258 INFO:tasks.workunit.client.1.smithi169.stdout:1/91: dread d4/d5/fb [0,4194304] 0 2024-03-04T13:42:41.259 INFO:tasks.workunit.client.1.smithi169.stdout:8/117: creat d7/d10/d17/f2a x:0 0 0 2024-03-04T13:42:41.263 INFO:tasks.workunit.client.1.smithi169.stdout:2/109: mknod d1/d5/c2d 0 2024-03-04T13:42:41.265 INFO:tasks.workunit.client.1.smithi169.stdout:7/137: rmdir d2 39 2024-03-04T13:42:41.268 INFO:tasks.workunit.client.1.smithi169.stdout:3/84: dwrite f5 [0,4194304] 0 2024-03-04T13:42:41.269 INFO:tasks.workunit.client.1.smithi169.stdout:1/92: write d4/d5/fb [2563295,5131] 0 2024-03-04T13:42:41.273 INFO:tasks.workunit.client.1.smithi169.stdout:3/85: write d7/de/d14/f20 [231644,12897] 0 2024-03-04T13:42:41.274 INFO:tasks.workunit.client.1.smithi169.stdout:2/110: readlink d1/d5/l18 0 2024-03-04T13:42:41.274 INFO:tasks.workunit.client.1.smithi169.stdout:6/98: link d0/d14/c18 d0/c1c 0 2024-03-04T13:42:41.276 INFO:tasks.workunit.client.1.smithi169.stdout:6/99: stat d0/f16 0 2024-03-04T13:42:41.281 INFO:tasks.workunit.client.1.smithi169.stdout:3/86: mkdir d7/de/d14/d18/d21 0 2024-03-04T13:42:41.286 INFO:tasks.workunit.client.1.smithi169.stdout:2/111: creat d1/d5/d19/f2e x:0 0 0 2024-03-04T13:42:41.286 INFO:tasks.workunit.client.1.smithi169.stdout:6/100: creat d0/f1d x:0 0 0 2024-03-04T13:42:41.289 INFO:tasks.workunit.client.1.smithi169.stdout:8/118: dwrite d7/d10/d17/f2a [0,4194304] 0 2024-03-04T13:42:41.289 INFO:tasks.workunit.client.1.smithi169.stdout:7/138: symlink d2/d15/l35 0 2024-03-04T13:42:41.292 INFO:tasks.workunit.client.1.smithi169.stdout:7/139: dread - d2/d28/f33 zero size 2024-03-04T13:42:41.292 INFO:tasks.workunit.client.1.smithi169.stdout:8/119: write d7/ff [2278954,121092] 0 2024-03-04T13:42:41.293 INFO:tasks.workunit.client.1.smithi169.stdout:6/101: mknod d0/d14/c1e 0 2024-03-04T13:42:41.296 INFO:tasks.workunit.client.1.smithi169.stdout:2/112: write d1/f3 [4234421,4749] 0 2024-03-04T13:42:41.300 INFO:tasks.workunit.client.1.smithi169.stdout:7/140: mknod d2/de/c36 0 2024-03-04T13:42:41.311 INFO:tasks.workunit.client.1.smithi169.stdout:3/87: dread d7/f1d [0,4194304] 0 2024-03-04T13:42:41.316 INFO:tasks.workunit.client.1.smithi169.stdout:8/120: dwrite d7/d10/d17/f29 [0,4194304] 0 2024-03-04T13:42:41.316 INFO:tasks.workunit.client.1.smithi169.stdout:3/88: chown d7/de/d14/d18/c1b 30302132 1 2024-03-04T13:42:41.317 INFO:tasks.workunit.client.1.smithi169.stdout:1/93: write d4/f11 [525999,87847] 0 2024-03-04T13:42:41.321 INFO:tasks.workunit.client.1.smithi169.stdout:6/102: fsync d0/f1d 0 2024-03-04T13:42:41.323 INFO:tasks.workunit.client.1.smithi169.stdout:8/121: write d7/d10/d17/f2a [1344820,57046] 0 2024-03-04T13:42:41.324 INFO:tasks.workunit.client.1.smithi169.stdout:6/103: fsync d0/f9 0 2024-03-04T13:42:41.327 INFO:tasks.workunit.client.1.smithi169.stdout:1/94: creat d4/dd/f1e x:0 0 0 2024-03-04T13:42:41.331 INFO:tasks.workunit.client.1.smithi169.stdout:6/104: mknod d0/c1f 0 2024-03-04T13:42:41.333 INFO:tasks.workunit.client.1.smithi169.stdout:1/95: readlink d4/d5/l14 0 2024-03-04T13:42:41.333 INFO:tasks.workunit.client.1.smithi169.stdout:6/105: write d0/f9 [4882413,113485] 0 2024-03-04T13:42:41.347 INFO:tasks.workunit.client.1.smithi169.stdout:7/141: dread d2/fd [0,4194304] 0 2024-03-04T13:42:41.347 INFO:tasks.workunit.client.1.smithi169.stdout:1/96: link d4/d5/c17 d4/dd/c1f 0 2024-03-04T13:42:41.349 INFO:tasks.workunit.client.1.smithi169.stdout:6/106: truncate d0/f15 765471 0 2024-03-04T13:42:41.349 INFO:tasks.workunit.client.1.smithi169.stdout:8/122: dwrite d7/d10/d17/f2a [0,4194304] 0 2024-03-04T13:42:41.353 INFO:tasks.workunit.client.1.smithi169.stdout:6/107: symlink d0/d14/l20 0 2024-03-04T13:42:41.354 INFO:tasks.workunit.client.1.smithi169.stdout:1/97: creat d4/f20 x:0 0 0 2024-03-04T13:42:41.358 INFO:tasks.workunit.client.1.smithi169.stdout:1/98: chown d4 1387 1 2024-03-04T13:42:41.358 INFO:tasks.workunit.client.1.smithi169.stdout:7/142: truncate d2/d15/f24 784433 0 2024-03-04T13:42:41.359 INFO:tasks.workunit.client.1.smithi169.stdout:8/123: dread d7/d10/f1e [0,4194304] 0 2024-03-04T13:42:41.360 INFO:tasks.workunit.client.1.smithi169.stdout:6/108: write d0/fb [460276,112393] 0 2024-03-04T13:42:41.362 INFO:tasks.workunit.client.1.smithi169.stdout:8/124: fdatasync d7/d10/d17/f27 0 2024-03-04T13:42:41.363 INFO:tasks.workunit.client.1.smithi169.stdout:7/143: fsync d2/d15/d17/f1d 0 2024-03-04T13:42:41.369 INFO:tasks.workunit.client.1.smithi169.stdout:7/144: mkdir d2/de/d37 0 2024-03-04T13:42:41.371 INFO:tasks.workunit.client.1.smithi169.stdout:7/145: rmdir d2/d15 39 2024-03-04T13:42:41.383 INFO:tasks.workunit.client.1.smithi169.stdout:1/99: dwrite d4/dd/f1e [0,4194304] 0 2024-03-04T13:42:41.384 INFO:tasks.workunit.client.1.smithi169.stdout:8/125: dwrite d7/ff [0,4194304] 0 2024-03-04T13:42:41.384 INFO:tasks.workunit.client.1.smithi169.stdout:6/109: dread d0/f12 [0,4194304] 0 2024-03-04T13:42:41.390 INFO:tasks.workunit.client.1.smithi169.stdout:6/110: creat d0/d14/f21 x:0 0 0 2024-03-04T13:42:41.393 INFO:tasks.workunit.client.1.smithi169.stdout:1/100: rename d4/c6 to d4/dd/c21 0 2024-03-04T13:42:41.395 INFO:tasks.workunit.client.1.smithi169.stdout:6/111: symlink d0/d14/l22 0 2024-03-04T13:42:41.396 INFO:tasks.workunit.client.1.smithi169.stdout:6/112: write d0/f9 [1260601,124024] 0 2024-03-04T13:42:41.400 INFO:tasks.workunit.client.1.smithi169.stdout:7/146: dwrite d2/de/ff [0,4194304] 0 2024-03-04T13:42:41.401 INFO:tasks.workunit.client.1.smithi169.stdout:1/101: mknod d4/d5/c22 0 2024-03-04T13:42:41.406 INFO:tasks.workunit.client.1.smithi169.stdout:7/147: truncate d2/f2c 452531 0 2024-03-04T13:42:41.407 INFO:tasks.workunit.client.1.smithi169.stdout:6/113: creat d0/d14/f23 x:0 0 0 2024-03-04T13:42:41.409 INFO:tasks.workunit.client.1.smithi169.stdout:1/102: creat d4/d1d/f23 x:0 0 0 2024-03-04T13:42:41.413 INFO:tasks.workunit.client.1.smithi169.stdout:8/126: dwrite d7/d10/d17/f2a [0,4194304] 0 2024-03-04T13:42:41.413 INFO:tasks.workunit.client.1.smithi169.stdout:1/103: dread - d4/d1d/f23 zero size 2024-03-04T13:42:41.413 INFO:tasks.workunit.client.1.smithi169.stdout:1/104: stat d4/d1d 0 2024-03-04T13:42:41.414 INFO:tasks.workunit.client.1.smithi169.stdout:6/114: symlink d0/l24 0 2024-03-04T13:42:41.415 INFO:tasks.workunit.client.1.smithi169.stdout:7/148: chown d2/d15/l2b 3307 1 2024-03-04T13:42:41.419 INFO:tasks.workunit.client.1.smithi169.stdout:6/115: truncate d0/d14/f21 786128 0 2024-03-04T13:42:41.422 INFO:tasks.workunit.client.1.smithi169.stdout:7/149: symlink d2/d28/l38 0 2024-03-04T13:42:41.423 INFO:tasks.workunit.client.1.smithi169.stdout:7/150: stat d2/d15/f24 0 2024-03-04T13:42:41.423 INFO:tasks.workunit.client.1.smithi169.stdout:6/116: symlink d0/d14/l25 0 2024-03-04T13:42:41.431 INFO:tasks.workunit.client.1.smithi169.stdout:7/151: write d2/fd [75723,56460] 0 2024-03-04T13:42:41.433 INFO:tasks.workunit.client.1.smithi169.stdout:1/105: dwrite d4/f20 [0,4194304] 0 2024-03-04T13:42:41.435 INFO:tasks.workunit.client.1.smithi169.stdout:7/152: creat d2/de/f39 x:0 0 0 2024-03-04T13:42:41.436 INFO:tasks.workunit.client.1.smithi169.stdout:1/106: write d4/dd/f1e [4864276,27441] 0 2024-03-04T13:42:41.438 INFO:tasks.workunit.client.1.smithi169.stdout:9/137: sync 2024-03-04T13:42:41.438 INFO:tasks.workunit.client.1.smithi169.stdout:1/107: stat d4/dd/d16 0 2024-03-04T13:42:41.440 INFO:tasks.workunit.client.1.smithi169.stdout:1/108: write d4/f11 [525471,43811] 0 2024-03-04T13:42:41.446 INFO:tasks.workunit.client.1.smithi169.stdout:7/153: creat d2/d15/d17/f3a x:0 0 0 2024-03-04T13:42:41.447 INFO:tasks.workunit.client.1.smithi169.stdout:9/138: rename d7/dc/d12/l29 to d7/l30 0 2024-03-04T13:42:41.449 INFO:tasks.workunit.client.1.smithi169.stdout:9/139: stat d7/dc/d12/f18 0 2024-03-04T13:42:41.454 INFO:tasks.workunit.client.1.smithi169.stdout:1/109: rename d4/d5/fb to d4/dd/df/f24 0 2024-03-04T13:42:41.459 INFO:tasks.workunit.client.1.smithi169.stdout:7/154: symlink d2/d15/l3b 0 2024-03-04T13:42:41.462 INFO:tasks.workunit.client.1.smithi169.stdout:7/155: write d2/d15/d17/f1d [1381632,85541] 0 2024-03-04T13:42:41.465 INFO:tasks.workunit.client.1.smithi169.stdout:6/117: rmdir d0/d14 39 2024-03-04T13:42:41.577 INFO:tasks.workunit.client.1.smithi169.stdout:6/118: dread d0/f15 [0,4194304] 0 2024-03-04T13:42:41.579 INFO:tasks.workunit.client.1.smithi169.stdout:6/119: chown d0/ld 417 1 2024-03-04T13:42:41.579 INFO:tasks.workunit.client.1.smithi169.stdout:9/140: dwrite d7/f11 [0,4194304] 0 2024-03-04T13:42:41.579 INFO:tasks.workunit.client.1.smithi169.stdout:1/110: dwrite d4/f11 [0,4194304] 0 2024-03-04T13:42:41.582 INFO:tasks.workunit.client.1.smithi169.stdout:1/111: write d4/f1b [974362,7283] 0 2024-03-04T13:42:41.585 INFO:tasks.workunit.client.1.smithi169.stdout:6/120: write d0/f12 [2984687,74845] 0 2024-03-04T13:42:41.585 INFO:tasks.workunit.client.1.smithi169.stdout:9/141: unlink d7/dc/d12/l15 0 2024-03-04T13:42:41.585 INFO:tasks.workunit.client.1.smithi169.stdout:1/112: write d4/d1d/f23 [173613,47335] 0 2024-03-04T13:42:41.588 INFO:tasks.workunit.client.1.smithi169.stdout:6/121: chown d0/ca 461012 1 2024-03-04T13:42:41.593 INFO:tasks.workunit.client.1.smithi169.stdout:9/142: unlink d7/dc/d12/f1f 0 2024-03-04T13:42:41.602 INFO:tasks.workunit.client.1.smithi169.stdout:9/143: rmdir d7/d27 39 2024-03-04T13:42:41.605 INFO:tasks.workunit.client.1.smithi169.stdout:6/122: creat d0/d14/f26 x:0 0 0 2024-03-04T13:42:41.608 INFO:tasks.workunit.client.1.smithi169.stdout:1/113: creat d4/f25 x:0 0 0 2024-03-04T13:42:41.609 INFO:tasks.workunit.client.1.smithi169.stdout:9/144: mknod d7/dc/c31 0 2024-03-04T13:42:41.613 INFO:tasks.workunit.client.1.smithi169.stdout:6/123: unlink d0/c17 0 2024-03-04T13:42:41.614 INFO:tasks.workunit.client.1.smithi169.stdout:6/124: chown d0/fb 1691 1 2024-03-04T13:42:41.616 INFO:tasks.workunit.client.1.smithi169.stdout:6/125: read - d0/f1d zero size 2024-03-04T13:42:41.622 INFO:tasks.workunit.client.1.smithi169.stdout:9/145: dread d7/dc/f1d [0,4194304] 0 2024-03-04T13:42:41.625 INFO:tasks.workunit.client.1.smithi169.stdout:1/114: dwrite d4/f25 [0,4194304] 0 2024-03-04T13:42:41.630 INFO:tasks.workunit.client.1.smithi169.stdout:6/126: getdents d0/d14 0 2024-03-04T13:42:41.636 INFO:tasks.workunit.client.1.smithi169.stdout:1/115: mkdir d4/d13/d26 0 2024-03-04T13:42:41.638 INFO:tasks.workunit.client.1.smithi169.stdout:6/127: rename d0/d14/f26 to d0/d14/f27 0 2024-03-04T13:42:41.639 INFO:tasks.workunit.client.1.smithi169.stdout:1/116: truncate d4/f1b 1984382 0 2024-03-04T13:42:41.640 INFO:tasks.workunit.client.1.smithi169.stdout:6/128: chown d0/d14/c1e 3830 1 2024-03-04T13:42:41.642 INFO:tasks.workunit.client.1.smithi169.stdout:9/146: dwrite f6 [4194304,4194304] 0 2024-03-04T13:42:41.643 INFO:tasks.workunit.client.1.smithi169.stdout:1/117: truncate d4/d1d/f23 854840 0 2024-03-04T13:42:41.644 INFO:tasks.workunit.client.1.smithi169.stdout:9/147: chown d7/dc/c1a 1822 1 2024-03-04T13:42:41.645 INFO:tasks.workunit.client.1.smithi169.stdout:1/118: chown d4/d5/l14 0 1 2024-03-04T13:42:41.647 INFO:tasks.workunit.client.1.smithi169.stdout:1/119: fsync d4/f1b 0 2024-03-04T13:42:41.649 INFO:tasks.workunit.client.1.smithi169.stdout:6/129: dread d0/fb [0,4194304] 0 2024-03-04T13:42:41.649 INFO:tasks.workunit.client.1.smithi169.stdout:9/148: mkdir d7/dc/d12/d32 0 2024-03-04T13:42:41.657 INFO:tasks.workunit.client.1.smithi169.stdout:9/149: creat d7/f33 x:0 0 0 2024-03-04T13:42:41.659 INFO:tasks.workunit.client.1.smithi169.stdout:6/130: creat d0/f28 x:0 0 0 2024-03-04T13:42:41.661 INFO:tasks.workunit.client.1.smithi169.stdout:1/120: getdents d4/dd 0 2024-03-04T13:42:41.661 INFO:tasks.workunit.client.1.smithi169.stdout:6/131: dread - d0/d14/f23 zero size 2024-03-04T13:42:41.662 INFO:tasks.workunit.client.1.smithi169.stdout:9/150: mkdir d7/d34 0 2024-03-04T13:42:41.666 INFO:tasks.workunit.client.1.smithi169.stdout:6/132: creat d0/f29 x:0 0 0 2024-03-04T13:42:41.667 INFO:tasks.workunit.client.1.smithi169.stdout:4/98: sync 2024-03-04T13:42:41.669 INFO:tasks.workunit.client.1.smithi169.stdout:6/133: creat d0/d14/f2a x:0 0 0 2024-03-04T13:42:41.669 INFO:tasks.workunit.client.1.smithi169.stdout:9/151: link d7/dc/c31 d7/d2d/c35 0 2024-03-04T13:42:41.674 INFO:tasks.workunit.client.1.smithi169.stdout:6/134: truncate d0/d14/f27 841420 0 2024-03-04T13:42:41.676 INFO:tasks.workunit.client.1.smithi169.stdout:6/135: read d0/f12 [350426,87022] 0 2024-03-04T13:42:41.681 INFO:tasks.workunit.client.1.smithi169.stdout:6/136: creat d0/f2b x:0 0 0 2024-03-04T13:42:41.784 INFO:tasks.workunit.client.1.smithi169.stdout:4/99: dwrite d4/f15 [0,4194304] 0 2024-03-04T13:42:41.788 INFO:tasks.workunit.client.1.smithi169.stdout:9/152: dwrite d7/f11 [0,4194304] 0 2024-03-04T13:42:41.788 INFO:tasks.workunit.client.1.smithi169.stdout:6/137: rename d0/l24 to d0/d14/l2c 0 2024-03-04T13:42:41.790 INFO:tasks.workunit.client.1.smithi169.stdout:9/153: readlink d7/dc/d12/l16 0 2024-03-04T13:42:41.800 INFO:tasks.workunit.client.1.smithi169.stdout:6/138: rename d0/c1c to d0/d14/c2d 0 2024-03-04T13:42:41.800 INFO:tasks.workunit.client.1.smithi169.stdout:4/100: dread d4/f7 [0,4194304] 0 2024-03-04T13:42:41.803 INFO:tasks.workunit.client.1.smithi169.stdout:4/101: mkdir d4/d1c/d25 0 2024-03-04T13:42:41.811 INFO:tasks.workunit.client.1.smithi169.stdout:9/154: dwrite d7/f11 [0,4194304] 0 2024-03-04T13:42:41.814 INFO:tasks.workunit.client.1.smithi169.stdout:9/155: write d7/dc/fd [2920712,99486] 0 2024-03-04T13:42:41.818 INFO:tasks.workunit.client.1.smithi169.stdout:9/156: chown d7/l30 0 1 2024-03-04T13:42:41.825 INFO:tasks.workunit.client.1.smithi169.stdout:5/151: sync 2024-03-04T13:42:41.838 INFO:tasks.workunit.client.1.smithi169.stdout:5/152: dread d1/f15 [0,4194304] 0 2024-03-04T13:42:41.840 INFO:tasks.workunit.client.1.smithi169.stdout:5/153: write d1/f15 [1008466,126933] 0 2024-03-04T13:42:41.841 INFO:tasks.workunit.client.1.smithi169.stdout:9/157: dwrite d7/dc/f23 [0,4194304] 0 2024-03-04T13:42:41.843 INFO:tasks.workunit.client.1.smithi169.stdout:5/154: write d1/f24 [461666,79077] 0 2024-03-04T13:42:41.843 INFO:tasks.workunit.client.1.smithi169.stdout:9/158: chown d7/fb 1 1 2024-03-04T13:42:41.851 INFO:tasks.workunit.client.1.smithi169.stdout:9/159: unlink d7/dc/c31 0 2024-03-04T13:42:41.855 INFO:tasks.workunit.client.1.smithi169.stdout:6/139: getdents d0 0 2024-03-04T13:42:41.859 INFO:tasks.workunit.client.1.smithi169.stdout:6/140: mknod d0/c2e 0 2024-03-04T13:42:41.870 INFO:tasks.workunit.client.1.smithi169.stdout:8/127: sync 2024-03-04T13:42:41.871 INFO:tasks.workunit.client.1.smithi169.stdout:0/133: sync 2024-03-04T13:42:41.871 INFO:tasks.workunit.client.1.smithi169.stdout:2/113: sync 2024-03-04T13:42:41.871 INFO:tasks.workunit.client.1.smithi169.stdout:3/89: sync 2024-03-04T13:42:41.871 INFO:tasks.workunit.client.1.smithi169.stdout:4/102: dwrite d4/f14 [0,4194304] 0 2024-03-04T13:42:41.873 INFO:tasks.workunit.client.1.smithi169.stdout:3/90: read d7/f9 [5718,55635] 0 2024-03-04T13:42:41.874 INFO:tasks.workunit.client.1.smithi169.stdout:8/128: write d7/d10/d17/f29 [1507474,116296] 0 2024-03-04T13:42:41.876 INFO:tasks.workunit.client.1.smithi169.stdout:0/134: mknod d5/c34 0 2024-03-04T13:42:41.879 INFO:tasks.workunit.client.1.smithi169.stdout:9/160: getdents d7/dc 0 2024-03-04T13:42:41.885 INFO:tasks.workunit.client.1.smithi169.stdout:2/114: symlink d1/d5/d19/d1e/d29/l2f 0 2024-03-04T13:42:41.885 INFO:tasks.workunit.client.1.smithi169.stdout:6/141: link d0/f1 d0/f2f 0 2024-03-04T13:42:41.889 INFO:tasks.workunit.client.1.smithi169.stdout:5/155: dread d1/fa [0,4194304] 0 2024-03-04T13:42:41.893 INFO:tasks.workunit.client.1.smithi169.stdout:8/129: dread d7/ff [0,4194304] 0 2024-03-04T13:42:41.894 INFO:tasks.workunit.client.1.smithi169.stdout:5/156: read d1/d16/d17/f18 [2543116,62027] 0 2024-03-04T13:42:41.894 INFO:tasks.workunit.client.1.smithi169.stdout:4/103: dread d4/f15 [0,4194304] 0 2024-03-04T13:42:41.900 INFO:tasks.workunit.client.1.smithi169.stdout:8/130: chown d7/d10/d17/f2a 896 1 2024-03-04T13:42:41.915 INFO:tasks.workunit.client.1.smithi169.stdout:0/135: rename d5/c30 to d5/d9/dc/d12/c35 0 2024-03-04T13:42:41.916 INFO:tasks.workunit.client.1.smithi169.stdout:9/161: dwrite d7/d27/f2b [4194304,4194304] 0 2024-03-04T13:42:41.916 INFO:tasks.workunit.client.1.smithi169.stdout:2/115: dwrite d1/d5/d19/d1e/f24 [0,4194304] 0 2024-03-04T13:42:41.917 INFO:tasks.workunit.client.1.smithi169.stdout:3/91: truncate d7/f9 2599312 0 2024-03-04T13:42:41.920 INFO:tasks.workunit.client.1.smithi169.stdout:4/104: symlink d4/d1c/d25/l26 0 2024-03-04T13:42:41.920 INFO:tasks.workunit.client.1.smithi169.stdout:3/92: stat f6 0 2024-03-04T13:42:41.927 INFO:tasks.workunit.client.1.smithi169.stdout:5/157: dwrite d1/f14 [0,4194304] 0 2024-03-04T13:42:41.927 INFO:tasks.workunit.client.1.smithi169.stdout:0/136: creat d5/d9/dc/d1c/f36 x:0 0 0 2024-03-04T13:42:41.928 INFO:tasks.workunit.client.1.smithi169.stdout:2/116: creat d1/f30 x:0 0 0 2024-03-04T13:42:41.928 INFO:tasks.workunit.client.1.smithi169.stdout:3/93: rmdir d7/de 39 2024-03-04T13:42:41.933 INFO:tasks.workunit.client.1.smithi169.stdout:3/94: write f4 [2264110,54342] 0 2024-03-04T13:42:41.935 INFO:tasks.workunit.client.1.smithi169.stdout:2/117: read d1/d5/f8 [7218036,53451] 0 2024-03-04T13:42:41.939 INFO:tasks.workunit.client.1.smithi169.stdout:8/131: dwrite d7/d10/d22/f25 [0,4194304] 0 2024-03-04T13:42:41.943 INFO:tasks.workunit.client.1.smithi169.stdout:1/121: sync 2024-03-04T13:42:41.943 INFO:tasks.workunit.client.1.smithi169.stdout:7/156: sync 2024-03-04T13:42:41.946 INFO:tasks.workunit.client.1.smithi169.stdout:4/105: creat d4/d11/d23/f27 x:0 0 0 2024-03-04T13:42:41.948 INFO:tasks.workunit.client.1.smithi169.stdout:2/118: creat d1/d5/f31 x:0 0 0 2024-03-04T13:42:41.949 INFO:tasks.workunit.client.1.smithi169.stdout:9/162: creat d7/d2d/f36 x:0 0 0 2024-03-04T13:42:41.949 INFO:tasks.workunit.client.1.smithi169.stdout:4/106: chown d4/fa 947073 1 2024-03-04T13:42:41.957 INFO:tasks.workunit.client.1.smithi169.stdout:1/122: stat d4/dd/c12 0 2024-03-04T13:42:41.961 INFO:tasks.workunit.client.1.smithi169.stdout:6/142: dread d0/f2f [0,4194304] 0 2024-03-04T13:42:41.972 INFO:tasks.workunit.client.1.smithi169.stdout:7/157: creat d2/d15/f3c x:0 0 0 2024-03-04T13:42:41.972 INFO:tasks.workunit.client.1.smithi169.stdout:5/158: dwrite d1/d16/f20 [0,4194304] 0 2024-03-04T13:42:41.974 INFO:tasks.workunit.client.1.smithi169.stdout:1/123: mkdir d4/d5/d27 0 2024-03-04T13:42:41.979 INFO:tasks.workunit.client.1.smithi169.stdout:8/132: dwrite d7/d10/d17/f27 [0,4194304] 0 2024-03-04T13:42:41.992 INFO:tasks.workunit.client.1.smithi169.stdout:7/158: creat d2/de/f3d x:0 0 0 2024-03-04T13:42:41.994 INFO:tasks.workunit.client.1.smithi169.stdout:4/107: dread d4/f7 [0,4194304] 0 2024-03-04T13:42:41.994 INFO:tasks.workunit.client.1.smithi169.stdout:5/159: mkdir d1/d16/d17/d28 0 2024-03-04T13:42:41.999 INFO:tasks.workunit.client.1.smithi169.stdout:6/143: creat d0/f30 x:0 0 0 2024-03-04T13:42:41.999 INFO:tasks.workunit.client.1.smithi169.stdout:1/124: dread d4/d1d/f23 [0,4194304] 0 2024-03-04T13:42:42.000 INFO:tasks.workunit.client.1.smithi169.stdout:8/133: mknod d7/d10/d17/c2b 0 2024-03-04T13:42:42.008 INFO:tasks.workunit.client.1.smithi169.stdout:7/159: mkdir d2/de/d3e 0 2024-03-04T13:42:42.008 INFO:tasks.workunit.client.1.smithi169.stdout:3/95: dwrite d7/fb [0,4194304] 0 2024-03-04T13:42:42.008 INFO:tasks.workunit.client.1.smithi169.stdout:3/96: read f4 [3559993,62655] 0 2024-03-04T13:42:42.008 INFO:tasks.workunit.client.1.smithi169.stdout:4/108: mknod d4/d11/c28 0 2024-03-04T13:42:42.010 INFO:tasks.workunit.client.1.smithi169.stdout:9/163: truncate d7/fe 2542268 0 2024-03-04T13:42:42.016 INFO:tasks.workunit.client.1.smithi169.stdout:5/160: unlink d1/f15 0 2024-03-04T13:42:42.026 INFO:tasks.workunit.client.1.smithi169.stdout:5/161: chown d1/d16/d17/c19 2 1 2024-03-04T13:42:42.028 INFO:tasks.workunit.client.1.smithi169.stdout:5/162: chown d1/l11 481062 1 2024-03-04T13:42:42.029 INFO:tasks.workunit.client.1.smithi169.stdout:6/144: dread d0/fb [0,4194304] 0 2024-03-04T13:42:42.030 INFO:tasks.workunit.client.1.smithi169.stdout:1/125: dread d4/f1b [0,4194304] 0 2024-03-04T13:42:42.030 INFO:tasks.workunit.client.1.smithi169.stdout:8/134: dwrite d7/d10/d22/f25 [0,4194304] 0 2024-03-04T13:42:42.032 INFO:tasks.workunit.client.1.smithi169.stdout:9/164: symlink d7/d25/l37 0 2024-03-04T13:42:42.034 INFO:tasks.workunit.client.1.smithi169.stdout:4/109: rename d4/f6 to d4/d19/d21/f29 0 2024-03-04T13:42:42.035 INFO:tasks.workunit.client.1.smithi169.stdout:3/97: truncate d7/f1d 5046385 0 2024-03-04T13:42:42.037 INFO:tasks.workunit.client.1.smithi169.stdout:7/160: creat d2/de/d37/f3f x:0 0 0 2024-03-04T13:42:42.039 INFO:tasks.workunit.client.1.smithi169.stdout:0/137: dread d5/f16 [0,4194304] 0 2024-03-04T13:42:42.041 INFO:tasks.workunit.client.1.smithi169.stdout:8/135: creat d7/d10/d24/f2c x:0 0 0 2024-03-04T13:42:42.043 INFO:tasks.workunit.client.1.smithi169.stdout:1/126: rename d4/f11 to d4/dd/df/f28 0 2024-03-04T13:42:42.046 INFO:tasks.workunit.client.1.smithi169.stdout:0/138: creat d5/f37 x:0 0 0 2024-03-04T13:42:42.052 INFO:tasks.workunit.client.1.smithi169.stdout:8/136: mkdir d7/d10/d2d 0 2024-03-04T13:42:42.058 INFO:tasks.workunit.client.1.smithi169.stdout:3/98: mkdir d7/de/d22 0 2024-03-04T13:42:42.060 INFO:tasks.workunit.client.1.smithi169.stdout:7/161: mkdir d2/d40 0 2024-03-04T13:42:42.060 INFO:tasks.workunit.client.1.smithi169.stdout:9/165: link d7/l13 d7/d34/l38 0 2024-03-04T13:42:42.061 INFO:tasks.workunit.client.1.smithi169.stdout:7/162: stat d2/de/d37 0 2024-03-04T13:42:42.062 INFO:tasks.workunit.client.1.smithi169.stdout:7/163: chown d2/de/d37 23731 1 2024-03-04T13:42:42.063 INFO:tasks.workunit.client.1.smithi169.stdout:7/164: write d2/f34 [743972,28828] 0 2024-03-04T13:42:42.065 INFO:tasks.workunit.client.1.smithi169.stdout:8/137: creat d7/d10/d17/f2e x:0 0 0 2024-03-04T13:42:42.066 INFO:tasks.workunit.client.1.smithi169.stdout:7/165: dread - d2/de/f3d zero size 2024-03-04T13:42:42.068 INFO:tasks.workunit.client.1.smithi169.stdout:8/138: truncate d7/d10/f1e 589342 0 2024-03-04T13:42:42.069 INFO:tasks.workunit.client.1.smithi169.stdout:7/166: chown d2/fd 3633 1 2024-03-04T13:42:42.070 INFO:tasks.workunit.client.1.smithi169.stdout:7/167: dread - d2/de/f39 zero size 2024-03-04T13:42:42.070 INFO:tasks.workunit.client.1.smithi169.stdout:3/99: write d7/f1d [922437,12014] 0 2024-03-04T13:42:42.071 INFO:tasks.workunit.client.1.smithi169.stdout:8/139: chown d7/d10/d24/l28 63108 1 2024-03-04T13:42:42.073 INFO:tasks.workunit.client.1.smithi169.stdout:7/168: write d2/fa [924618,112745] 0 2024-03-04T13:42:42.076 INFO:tasks.workunit.client.1.smithi169.stdout:0/139: symlink d5/d9/dc/d12/l38 0 2024-03-04T13:42:42.081 INFO:tasks.workunit.client.1.smithi169.stdout:8/140: symlink d7/d10/d17/l2f 0 2024-03-04T13:42:42.083 INFO:tasks.workunit.client.1.smithi169.stdout:7/169: dread d2/d15/d17/f1d [0,4194304] 0 2024-03-04T13:42:42.087 INFO:tasks.workunit.client.1.smithi169.stdout:6/145: write d0/f2f [1791034,51285] 0 2024-03-04T13:42:42.088 INFO:tasks.workunit.client.1.smithi169.stdout:0/140: mkdir d5/d9/d39 0 2024-03-04T13:42:42.090 INFO:tasks.workunit.client.1.smithi169.stdout:1/127: link d4/d5/ca d4/dd/df/c29 0 2024-03-04T13:42:42.094 INFO:tasks.workunit.client.1.smithi169.stdout:8/141: creat d7/d10/d22/f30 x:0 0 0 2024-03-04T13:42:42.096 INFO:tasks.workunit.client.1.smithi169.stdout:6/146: write d0/f2b [750770,122180] 0 2024-03-04T13:42:42.099 INFO:tasks.workunit.client.1.smithi169.stdout:4/110: truncate d4/f9 1476016 0 2024-03-04T13:42:42.101 INFO:tasks.workunit.client.1.smithi169.stdout:3/100: link d7/de/d14/l11 d7/de/d14/l23 0 2024-03-04T13:42:42.103 INFO:tasks.workunit.client.1.smithi169.stdout:7/170: rmdir d2/d40 0 2024-03-04T13:42:42.104 INFO:tasks.workunit.client.1.smithi169.stdout:6/147: write d0/d14/f23 [614214,115482] 0 2024-03-04T13:42:42.108 INFO:tasks.workunit.client.1.smithi169.stdout:3/101: unlink d7/de/d14/d18/c1b 0 2024-03-04T13:42:42.109 INFO:tasks.workunit.client.1.smithi169.stdout:8/142: dread d7/ff [0,4194304] 0 2024-03-04T13:42:42.110 INFO:tasks.workunit.client.1.smithi169.stdout:7/171: truncate d2/d15/d17/f3a 90320 0 2024-03-04T13:42:42.112 INFO:tasks.workunit.client.1.smithi169.stdout:1/128: dwrite d4/dd/f1e [0,4194304] 0 2024-03-04T13:42:42.116 INFO:tasks.workunit.client.1.smithi169.stdout:8/143: write d7/d10/d17/f20 [443220,61628] 0 2024-03-04T13:42:42.117 INFO:tasks.workunit.client.1.smithi169.stdout:3/102: unlink d7/de/d14/f12 0 2024-03-04T13:42:42.120 INFO:tasks.workunit.client.1.smithi169.stdout:3/103: write d7/fb [454132,32444] 0 2024-03-04T13:42:42.121 INFO:tasks.workunit.client.1.smithi169.stdout:7/172: write d2/d15/d17/f1d [331040,21496] 0 2024-03-04T13:42:42.124 INFO:tasks.workunit.client.1.smithi169.stdout:7/173: chown d2/de/f39 215339 1 2024-03-04T13:42:42.126 INFO:tasks.workunit.client.1.smithi169.stdout:1/129: write d4/dd/df/f24 [2897459,43198] 0 2024-03-04T13:42:42.126 INFO:tasks.workunit.client.1.smithi169.stdout:9/166: truncate f6 5064596 0 2024-03-04T13:42:42.129 INFO:tasks.workunit.client.1.smithi169.stdout:6/148: dread d0/f8 [0,4194304] 0 2024-03-04T13:42:42.140 INFO:tasks.workunit.client.1.smithi169.stdout:3/104: getdents d7/de/d22 0 2024-03-04T13:42:42.145 INFO:tasks.workunit.client.1.smithi169.stdout:6/149: write d0/f28 [739375,40011] 0 2024-03-04T13:42:42.147 INFO:tasks.workunit.client.1.smithi169.stdout:7/174: dwrite d2/d28/f33 [0,4194304] 0 2024-03-04T13:42:42.149 INFO:tasks.workunit.client.1.smithi169.stdout:8/144: dwrite d7/d10/d17/f27 [0,4194304] 0 2024-03-04T13:42:42.166 INFO:tasks.workunit.client.1.smithi169.stdout:1/130: link d4/d5/l14 d4/l2a 0 2024-03-04T13:42:42.167 INFO:tasks.workunit.client.1.smithi169.stdout:3/105: getdents d7/de/d14/d13/d1a 0 2024-03-04T13:42:42.169 INFO:tasks.workunit.client.1.smithi169.stdout:3/106: chown d7/de/d14/d13/d1a 56662 1 2024-03-04T13:42:42.169 INFO:tasks.workunit.client.1.smithi169.stdout:7/175: mkdir d2/de/d3e/d41 0 2024-03-04T13:42:42.170 INFO:tasks.workunit.client.1.smithi169.stdout:9/167: link d7/dc/l1c d7/d2d/l39 0 2024-03-04T13:42:42.171 INFO:tasks.workunit.client.1.smithi169.stdout:8/145: link d7/d10/d17/f29 d7/d10/d17/f31 0 2024-03-04T13:42:42.171 INFO:tasks.workunit.client.1.smithi169.stdout:2/119: sync 2024-03-04T13:42:42.177 INFO:tasks.workunit.client.1.smithi169.stdout:6/150: rename d0/d14 to d0/d31 0 2024-03-04T13:42:42.179 INFO:tasks.workunit.client.1.smithi169.stdout:9/168: mknod d7/d25/c3a 0 2024-03-04T13:42:42.180 INFO:tasks.workunit.client.1.smithi169.stdout:1/131: write d4/dd/df/f28 [4025996,12854] 0 2024-03-04T13:42:42.182 INFO:tasks.workunit.client.1.smithi169.stdout:7/176: write d2/de/d37/f3f [588568,93398] 0 2024-03-04T13:42:42.182 INFO:tasks.workunit.client.1.smithi169.stdout:2/120: creat d1/d5/d22/f32 x:0 0 0 2024-03-04T13:42:42.184 INFO:tasks.workunit.client.1.smithi169.stdout:3/107: rename d7/de/d22 to d7/de/d24 0 2024-03-04T13:42:42.185 INFO:tasks.workunit.client.1.smithi169.stdout:4/111: dwrite d4/f7 [0,4194304] 0 2024-03-04T13:42:42.199 INFO:tasks.workunit.client.1.smithi169.stdout:6/151: mkdir d0/d32 0 2024-03-04T13:42:42.204 INFO:tasks.workunit.client.1.smithi169.stdout:4/112: symlink d4/d11/l2a 0 2024-03-04T13:42:42.206 INFO:tasks.workunit.client.1.smithi169.stdout:6/152: write d0/d31/f21 [1461199,30700] 0 2024-03-04T13:42:42.206 INFO:tasks.workunit.client.1.smithi169.stdout:7/177: fsync d2/d28/f32 0 2024-03-04T13:42:42.215 INFO:tasks.workunit.client.1.smithi169.stdout:8/146: dwrite d7/d10/d17/f2e [0,4194304] 0 2024-03-04T13:42:42.219 INFO:tasks.workunit.client.1.smithi169.stdout:6/153: chown d0/f8 329188459 1 2024-03-04T13:42:42.224 INFO:tasks.workunit.client.1.smithi169.stdout:1/132: dwrite d4/dd/df/f24 [4194304,4194304] 0 2024-03-04T13:42:42.224 INFO:tasks.workunit.client.1.smithi169.stdout:9/169: dwrite d7/dc/f23 [0,4194304] 0 2024-03-04T13:42:42.231 INFO:tasks.workunit.client.1.smithi169.stdout:9/170: write d7/f10 [4783377,107342] 0 2024-03-04T13:42:42.232 INFO:tasks.workunit.client.1.smithi169.stdout:2/121: dwrite d1/ff [0,4194304] 0 2024-03-04T13:42:42.235 INFO:tasks.workunit.client.1.smithi169.stdout:7/178: getdents d2 0 2024-03-04T13:42:42.236 INFO:tasks.workunit.client.1.smithi169.stdout:2/122: fdatasync d1/f3 0 2024-03-04T13:42:42.239 INFO:tasks.workunit.client.1.smithi169.stdout:6/154: creat d0/d32/f33 x:0 0 0 2024-03-04T13:42:42.243 INFO:tasks.workunit.client.1.smithi169.stdout:9/171: creat d7/dc/d12/d32/f3b x:0 0 0 2024-03-04T13:42:42.243 INFO:tasks.workunit.client.1.smithi169.stdout:1/133: write d4/d5/f1a [783310,2317] 0 2024-03-04T13:42:42.247 INFO:tasks.workunit.client.1.smithi169.stdout:4/113: dread d4/f7 [0,4194304] 0 2024-03-04T13:42:42.251 INFO:tasks.workunit.client.1.smithi169.stdout:6/155: mknod d0/d31/c34 0 2024-03-04T13:42:42.258 INFO:tasks.workunit.client.1.smithi169.stdout:4/114: truncate d4/d19/d21/f29 4270844 0 2024-03-04T13:42:42.261 INFO:tasks.workunit.client.1.smithi169.stdout:1/134: creat d4/dd/df/f2b x:0 0 0 2024-03-04T13:42:42.261 INFO:tasks.workunit.client.1.smithi169.stdout:4/115: dread - d4/d11/d23/f27 zero size 2024-03-04T13:42:42.266 INFO:tasks.workunit.client.1.smithi169.stdout:4/116: write d4/f14 [2797208,12322] 0 2024-03-04T13:42:42.276 INFO:tasks.workunit.client.1.smithi169.stdout:8/147: dwrite d7/ff [0,4194304] 0 2024-03-04T13:42:42.277 INFO:tasks.workunit.client.1.smithi169.stdout:4/117: symlink d4/d1c/d25/l2b 0 2024-03-04T13:42:42.281 INFO:tasks.workunit.client.1.smithi169.stdout:6/156: creat d0/f35 x:0 0 0 2024-03-04T13:42:42.281 INFO:tasks.workunit.client.1.smithi169.stdout:7/179: truncate d2/f22 681879 0 2024-03-04T13:42:42.281 INFO:tasks.workunit.client.1.smithi169.stdout:4/118: readlink d4/l22 0 2024-03-04T13:42:42.283 INFO:tasks.workunit.client.1.smithi169.stdout:9/172: dread d7/dc/fd [0,4194304] 0 2024-03-04T13:42:42.286 INFO:tasks.workunit.client.1.smithi169.stdout:2/123: dwrite d1/d5/d22/f32 [0,4194304] 0 2024-03-04T13:42:42.287 INFO:tasks.workunit.client.1.smithi169.stdout:7/180: creat d2/d15/d17/f42 x:0 0 0 2024-03-04T13:42:42.291 INFO:tasks.workunit.client.1.smithi169.stdout:9/173: write d7/dc/d12/d32/f3b [498369,39782] 0 2024-03-04T13:42:42.293 INFO:tasks.workunit.client.1.smithi169.stdout:4/119: rename d4/d11/d23/f27 to d4/d1c/f2c 0 2024-03-04T13:42:42.296 INFO:tasks.workunit.client.1.smithi169.stdout:1/135: dread d4/f1b [0,4194304] 0 2024-03-04T13:42:42.297 INFO:tasks.workunit.client.1.smithi169.stdout:6/157: symlink d0/d32/l36 0 2024-03-04T13:42:42.297 INFO:tasks.workunit.client.1.smithi169.stdout:9/174: read - d7/d2d/f36 zero size 2024-03-04T13:42:42.301 INFO:tasks.workunit.client.1.smithi169.stdout:2/124: unlink d1/d5/f31 0 2024-03-04T13:42:42.302 INFO:tasks.workunit.client.1.smithi169.stdout:1/136: write d4/f25 [2439003,79068] 0 2024-03-04T13:42:42.302 INFO:tasks.workunit.client.1.smithi169.stdout:4/120: unlink d4/d11/c18 0 2024-03-04T13:42:42.302 INFO:tasks.workunit.client.1.smithi169.stdout:8/148: link l2 d7/d12/l32 0 2024-03-04T13:42:42.304 INFO:tasks.workunit.client.1.smithi169.stdout:2/125: readlink d1/d5/d22/l26 0 2024-03-04T13:42:42.307 INFO:tasks.workunit.client.1.smithi169.stdout:1/137: dread - d4/dd/df/f2b zero size 2024-03-04T13:42:42.310 INFO:tasks.workunit.client.1.smithi169.stdout:2/126: write d1/ff [2580707,101088] 0 2024-03-04T13:42:42.311 INFO:tasks.workunit.client.1.smithi169.stdout:9/175: truncate d7/f26 572671 0 2024-03-04T13:42:42.312 INFO:tasks.workunit.client.1.smithi169.stdout:8/149: rmdir d7/d10 39 2024-03-04T13:42:42.314 INFO:tasks.workunit.client.1.smithi169.stdout:1/138: write d4/dd/df/f28 [4463458,26352] 0 2024-03-04T13:42:42.316 INFO:tasks.workunit.client.1.smithi169.stdout:2/127: mknod d1/d5/c33 0 2024-03-04T13:42:42.316 INFO:tasks.workunit.client.1.smithi169.stdout:9/176: write d7/f10 [2497417,58990] 0 2024-03-04T13:42:42.323 INFO:tasks.workunit.client.1.smithi169.stdout:8/150: write d7/d10/d22/f30 [512551,78710] 0 2024-03-04T13:42:42.324 INFO:tasks.workunit.client.1.smithi169.stdout:5/163: sync 2024-03-04T13:42:42.326 INFO:tasks.workunit.client.1.smithi169.stdout:7/181: dwrite d2/de/f2a [0,4194304] 0 2024-03-04T13:42:42.326 INFO:tasks.workunit.client.1.smithi169.stdout:8/151: read - d7/d10/d24/f2c zero size 2024-03-04T13:42:42.331 INFO:tasks.workunit.client.1.smithi169.stdout:7/182: chown d2/d15/d17/f42 3773 1 2024-03-04T13:42:42.331 INFO:tasks.workunit.client.1.smithi169.stdout:5/164: truncate d1/d16/d17/d1b/f1e 1101380 0 2024-03-04T13:42:42.332 INFO:tasks.workunit.client.1.smithi169.stdout:8/152: write d7/d10/d24/f2c [571307,16012] 0 2024-03-04T13:42:42.333 INFO:tasks.workunit.client.1.smithi169.stdout:7/183: chown d2/de 1712 1 2024-03-04T13:42:42.334 INFO:tasks.workunit.client.1.smithi169.stdout:9/177: dread d7/dc/f1d [0,4194304] 0 2024-03-04T13:42:42.334 INFO:tasks.workunit.client.1.smithi169.stdout:4/121: dread d4/f15 [0,4194304] 0 2024-03-04T13:42:42.337 INFO:tasks.workunit.client.1.smithi169.stdout:9/178: chown d7/f11 250468 1 2024-03-04T13:42:42.337 INFO:tasks.workunit.client.1.smithi169.stdout:5/165: chown d1/d16/d17/d1b/f25 24306 1 2024-03-04T13:42:42.337 INFO:tasks.workunit.client.1.smithi169.stdout:7/184: write d2/d15/f3c [118453,70647] 0 2024-03-04T13:42:42.341 INFO:tasks.workunit.client.1.smithi169.stdout:9/179: readlink d7/dc/d12/l16 0 2024-03-04T13:42:42.345 INFO:tasks.workunit.client.1.smithi169.stdout:2/128: symlink d1/l34 0 2024-03-04T13:42:42.347 INFO:tasks.workunit.client.1.smithi169.stdout:9/180: write d7/f10 [5641771,101645] 0 2024-03-04T13:42:42.347 INFO:tasks.workunit.client.1.smithi169.stdout:1/139: symlink d4/l2c 0 2024-03-04T13:42:42.352 INFO:tasks.workunit.client.1.smithi169.stdout:8/153: rmdir d7/d10/d22 39 2024-03-04T13:42:42.353 INFO:tasks.workunit.client.1.smithi169.stdout:4/122: fdatasync d4/d19/d21/f29 0 2024-03-04T13:42:42.355 INFO:tasks.workunit.client.1.smithi169.stdout:7/185: rename d2/l5 to d2/d15/d17/l43 0 2024-03-04T13:42:42.355 INFO:tasks.workunit.client.1.smithi169.stdout:6/158: rmdir d0 39 2024-03-04T13:42:42.355 INFO:tasks.workunit.client.1.smithi169.stdout:1/140: truncate d4/dd/df/f2b 468138 0 2024-03-04T13:42:42.362 INFO:tasks.workunit.client.1.smithi169.stdout:4/123: mknod d4/c2d 0 2024-03-04T13:42:42.366 INFO:tasks.workunit.client.1.smithi169.stdout:8/154: creat d7/d1a/f33 x:0 0 0 2024-03-04T13:42:42.367 INFO:tasks.workunit.client.1.smithi169.stdout:5/166: dwrite d1/f24 [0,4194304] 0 2024-03-04T13:42:42.369 INFO:tasks.workunit.client.1.smithi169.stdout:2/129: rename d1/d5/d19/d1e/l21 to d1/d5/l35 0 2024-03-04T13:42:42.374 INFO:tasks.workunit.client.1.smithi169.stdout:3/108: sync 2024-03-04T13:42:42.374 INFO:tasks.workunit.client.1.smithi169.stdout:0/141: sync 2024-03-04T13:42:42.382 INFO:tasks.workunit.client.1.smithi169.stdout:9/181: rename d7/d34 to d7/d25/d3c 0 2024-03-04T13:42:42.384 INFO:tasks.workunit.client.1.smithi169.stdout:2/130: mkdir d1/d5/d19/d36 0 2024-03-04T13:42:42.398 INFO:tasks.workunit.client.1.smithi169.stdout:5/167: link d1/d16/f20 d1/d16/d17/d1b/f29 0 2024-03-04T13:42:42.405 INFO:tasks.workunit.client.1.smithi169.stdout:9/182: unlink d7/f26 0 2024-03-04T13:42:42.408 INFO:tasks.workunit.client.1.smithi169.stdout:1/141: fsync d4/dd/df/f1c 0 2024-03-04T13:42:42.411 INFO:tasks.workunit.client.1.smithi169.stdout:0/142: link d5/d9/dc/f1d d5/d9/d39/f3a 0 2024-03-04T13:42:42.412 INFO:tasks.workunit.client.1.smithi169.stdout:4/124: dread d4/fa [0,4194304] 0 2024-03-04T13:42:42.416 INFO:tasks.workunit.client.1.smithi169.stdout:8/155: creat d7/f34 x:0 0 0 2024-03-04T13:42:42.418 INFO:tasks.workunit.client.1.smithi169.stdout:5/168: mkdir d1/d16/d2a 0 2024-03-04T13:42:42.420 INFO:tasks.workunit.client.1.smithi169.stdout:5/169: fdatasync d1/f9 0 2024-03-04T13:42:42.421 INFO:tasks.workunit.client.1.smithi169.stdout:5/170: read d1/f24 [822290,2772] 0 2024-03-04T13:42:42.424 INFO:tasks.workunit.client.1.smithi169.stdout:9/183: creat d7/d25/f3d x:0 0 0 2024-03-04T13:42:42.425 INFO:tasks.workunit.client.1.smithi169.stdout:3/109: creat d7/f25 x:0 0 0 2024-03-04T13:42:42.427 INFO:tasks.workunit.client.1.smithi169.stdout:6/159: link d0/f29 d0/f37 0 2024-03-04T13:42:42.427 INFO:tasks.workunit.client.1.smithi169.stdout:3/110: write d7/fb [3554531,90725] 0 2024-03-04T13:42:42.429 INFO:tasks.workunit.client.1.smithi169.stdout:3/111: truncate d7/f25 1035343 0 2024-03-04T13:42:42.432 INFO:tasks.workunit.client.1.smithi169.stdout:7/186: symlink d2/de/d3e/l44 0 2024-03-04T13:42:42.432 INFO:tasks.workunit.client.1.smithi169.stdout:2/131: truncate d1/ff 3118703 0 2024-03-04T13:42:42.438 INFO:tasks.workunit.client.1.smithi169.stdout:5/171: rename d1/ce to d1/d16/d17/d28/c2b 0 2024-03-04T13:42:42.441 INFO:tasks.workunit.client.1.smithi169.stdout:5/172: write d1/f8 [1111148,95039] 0 2024-03-04T13:42:42.443 INFO:tasks.workunit.client.1.smithi169.stdout:1/142: getdents d4/d5/d27 0 2024-03-04T13:42:42.453 INFO:tasks.workunit.client.1.smithi169.stdout:1/143: dread d4/dd/df/f28 [4194304,4194304] 0 2024-03-04T13:42:42.461 INFO:tasks.workunit.client.1.smithi169.stdout:9/184: write d7/dc/f1d [4086922,91027] 0 2024-03-04T13:42:42.465 INFO:tasks.workunit.client.1.smithi169.stdout:0/143: dread d5/d9/dc/d12/d15/d19/f20 [0,4194304] 0 2024-03-04T13:42:42.465 INFO:tasks.workunit.client.1.smithi169.stdout:2/132: dwrite d1/d5/d19/f2e [0,4194304] 0 2024-03-04T13:42:42.466 INFO:tasks.workunit.client.1.smithi169.stdout:7/187: chown d2/d15/d17/l19 87933 1 2024-03-04T13:42:42.466 INFO:tasks.workunit.client.1.smithi169.stdout:0/144: chown d5/d9/dc/d12/l38 497 1 2024-03-04T13:42:42.467 INFO:tasks.workunit.client.1.smithi169.stdout:1/144: dread d4/f1b [0,4194304] 0 2024-03-04T13:42:42.471 INFO:tasks.workunit.client.1.smithi169.stdout:8/156: dwrite d7/d10/f1e [0,4194304] 0 2024-03-04T13:42:42.472 INFO:tasks.workunit.client.1.smithi169.stdout:6/160: creat d0/f38 x:0 0 0 2024-03-04T13:42:42.474 INFO:tasks.workunit.client.1.smithi169.stdout:8/157: chown d7/ca 26 1 2024-03-04T13:42:42.477 INFO:tasks.workunit.client.1.smithi169.stdout:4/125: rename d4/d19 to d4/d11/d2e 0 2024-03-04T13:42:42.488 INFO:tasks.workunit.client.1.smithi169.stdout:7/188: mknod d2/de/d37/c45 0 2024-03-04T13:42:42.493 INFO:tasks.workunit.client.1.smithi169.stdout:5/173: link d1/d16/f20 d1/d16/d17/d28/f2c 0 2024-03-04T13:42:42.496 INFO:tasks.workunit.client.1.smithi169.stdout:1/145: dwrite d4/dd/f1e [0,4194304] 0 2024-03-04T13:42:42.496 INFO:tasks.workunit.client.1.smithi169.stdout:2/133: mknod d1/d5/d19/d36/c37 0 2024-03-04T13:42:42.496 INFO:tasks.workunit.client.1.smithi169.stdout:3/112: link d7/l8 d7/de/d14/d18/l26 0 2024-03-04T13:42:42.496 INFO:tasks.workunit.client.1.smithi169.stdout:0/145: dwrite d5/d9/dc/d12/f24 [0,4194304] 0 2024-03-04T13:42:42.499 INFO:tasks.workunit.client.1.smithi169.stdout:2/134: fdatasync d1/d5/fe 0 2024-03-04T13:42:42.504 INFO:tasks.workunit.client.1.smithi169.stdout:5/174: mkdir d1/d16/d17/d2d 0 2024-03-04T13:42:42.506 INFO:tasks.workunit.client.1.smithi169.stdout:2/135: write d1/f30 [852298,50227] 0 2024-03-04T13:42:42.506 INFO:tasks.workunit.client.1.smithi169.stdout:3/113: creat d7/de/d14/d18/f27 x:0 0 0 2024-03-04T13:42:42.507 INFO:tasks.workunit.client.1.smithi169.stdout:7/189: creat d2/f46 x:0 0 0 2024-03-04T13:42:42.510 INFO:tasks.workunit.client.1.smithi169.stdout:1/146: mkdir d4/d5/d2d 0 2024-03-04T13:42:42.513 INFO:tasks.workunit.client.1.smithi169.stdout:5/175: rmdir d1/d16/d17/d28 39 2024-03-04T13:42:42.513 INFO:tasks.workunit.client.1.smithi169.stdout:3/114: chown d7/f25 5128851 1 2024-03-04T13:42:42.518 INFO:tasks.workunit.client.1.smithi169.stdout:8/158: dwrite d7/f34 [0,4194304] 0 2024-03-04T13:42:42.518 INFO:tasks.workunit.client.1.smithi169.stdout:3/115: fsync f5 0 2024-03-04T13:42:42.520 INFO:tasks.workunit.client.1.smithi169.stdout:7/190: symlink d2/de/l47 0 2024-03-04T13:42:42.524 INFO:tasks.workunit.client.1.smithi169.stdout:3/116: write d7/de/d14/f20 [7486,126483] 0 2024-03-04T13:42:42.527 INFO:tasks.workunit.client.1.smithi169.stdout:4/126: dwrite d4/fb [0,4194304] 0 2024-03-04T13:42:42.536 INFO:tasks.workunit.client.1.smithi169.stdout:1/147: rmdir d4/dd/df 39 2024-03-04T13:42:42.536 INFO:tasks.workunit.client.1.smithi169.stdout:7/191: stat d2/d28 0 2024-03-04T13:42:42.539 INFO:tasks.workunit.client.1.smithi169.stdout:2/136: dwrite d1/d5/d19/f2e [0,4194304] 0 2024-03-04T13:42:42.550 INFO:tasks.workunit.client.1.smithi169.stdout:3/117: creat d7/de/d14/d13/d1a/f28 x:0 0 0 2024-03-04T13:42:42.554 INFO:tasks.workunit.client.1.smithi169.stdout:5/176: symlink d1/d16/d2a/l2e 0 2024-03-04T13:42:42.555 INFO:tasks.workunit.client.1.smithi169.stdout:1/148: fdatasync d4/dd/df/f2b 0 2024-03-04T13:42:42.559 INFO:tasks.workunit.client.1.smithi169.stdout:5/177: chown d1/c12 663 1 2024-03-04T13:42:42.566 INFO:tasks.workunit.client.1.smithi169.stdout:2/137: symlink d1/l38 0 2024-03-04T13:42:42.569 INFO:tasks.workunit.client.1.smithi169.stdout:8/159: mknod d7/d10/d22/c35 0 2024-03-04T13:42:42.569 INFO:tasks.workunit.client.1.smithi169.stdout:6/161: dwrite d0/f37 [0,4194304] 0 2024-03-04T13:42:42.572 INFO:tasks.workunit.client.1.smithi169.stdout:4/127: dwrite d4/fb [0,4194304] 0 2024-03-04T13:42:42.573 INFO:tasks.workunit.client.1.smithi169.stdout:8/160: fdatasync d7/d10/d17/f2e 0 2024-03-04T13:42:42.579 INFO:tasks.workunit.client.1.smithi169.stdout:5/178: mkdir d1/d16/d2f 0 2024-03-04T13:42:42.581 INFO:tasks.workunit.client.1.smithi169.stdout:5/179: chown d1/ld 1 1 2024-03-04T13:42:42.581 INFO:tasks.workunit.client.1.smithi169.stdout:2/138: creat d1/d5/d19/d1e/d29/f39 x:0 0 0 2024-03-04T13:42:42.583 INFO:tasks.workunit.client.1.smithi169.stdout:6/162: mknod d0/d31/c39 0 2024-03-04T13:42:42.588 INFO:tasks.workunit.client.1.smithi169.stdout:4/128: creat d4/d11/d2e/f2f x:0 0 0 2024-03-04T13:42:42.589 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:42:42 smithi107 ceph-mon[28739]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' 2024-03-04T13:42:42.590 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:42:42 smithi107 ceph-mon[28739]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' 2024-03-04T13:42:42.590 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:42:42 smithi107 ceph-mon[28739]: Reconfiguring node-exporter.smithi169 (dependencies changed)... 2024-03-04T13:42:42.590 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:42:42 smithi107 ceph-mon[28739]: Reconfiguring daemon node-exporter.smithi169 on smithi169 2024-03-04T13:42:42.590 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:42:42 smithi107 ceph-mon[28739]: pgmap v43: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.1 MiB/s rd, 9.1 MiB/s wr, 142 op/s 2024-03-04T13:42:42.593 INFO:tasks.workunit.client.1.smithi169.stdout:1/149: creat d4/d13/d26/f2e x:0 0 0 2024-03-04T13:42:42.598 INFO:tasks.workunit.client.1.smithi169.stdout:2/139: unlink d1/d5/l18 0 2024-03-04T13:42:42.598 INFO:tasks.workunit.client.1.smithi169.stdout:0/146: dwrite d5/d9/dc/f1d [0,4194304] 0 2024-03-04T13:42:42.604 INFO:tasks.workunit.client.1.smithi169.stdout:5/180: mknod d1/d16/d2a/c30 0 2024-03-04T13:42:42.604 INFO:tasks.workunit.client.1.smithi169.stdout:8/161: dwrite d7/d10/f1e [4194304,4194304] 0 2024-03-04T13:42:42.606 INFO:tasks.workunit.client.1.smithi169.stdout:7/192: getdents d2 0 2024-03-04T13:42:42.607 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:42:42 smithi169 ceph-mon[36313]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' 2024-03-04T13:42:42.607 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:42:42 smithi169 ceph-mon[36313]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' 2024-03-04T13:42:42.607 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:42:42 smithi169 ceph-mon[36313]: Reconfiguring node-exporter.smithi169 (dependencies changed)... 2024-03-04T13:42:42.607 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:42:42 smithi169 ceph-mon[36313]: Reconfiguring daemon node-exporter.smithi169 on smithi169 2024-03-04T13:42:42.607 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:42:42 smithi169 ceph-mon[36313]: pgmap v43: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.1 MiB/s rd, 9.1 MiB/s wr, 142 op/s 2024-03-04T13:42:42.616 INFO:tasks.workunit.client.1.smithi169.stdout:5/181: chown d1/d16/d2a/l2e 3825 1 2024-03-04T13:42:42.616 INFO:tasks.workunit.client.1.smithi169.stdout:6/163: dread d0/f8 [0,4194304] 0 2024-03-04T13:42:42.617 INFO:tasks.workunit.client.1.smithi169.stdout:2/140: mknod d1/c3a 0 2024-03-04T13:42:42.623 INFO:tasks.workunit.client.1.smithi169.stdout:4/129: dwrite d4/d11/d2e/d21/f29 [4194304,4194304] 0 2024-03-04T13:42:42.623 INFO:tasks.workunit.client.1.smithi169.stdout:3/118: link d7/de/d14/l23 d7/de/l29 0 2024-03-04T13:42:42.624 INFO:tasks.workunit.client.1.smithi169.stdout:2/141: dread - d1/d5/d19/f27 zero size 2024-03-04T13:42:42.628 INFO:tasks.workunit.client.1.smithi169.stdout:2/142: chown d1/d5/f8 5 1 2024-03-04T13:42:42.634 INFO:tasks.workunit.client.1.smithi169.stdout:1/150: symlink d4/d5/d2d/l2f 0 2024-03-04T13:42:42.634 INFO:tasks.workunit.client.1.smithi169.stdout:6/164: unlink d0/d31/c39 0 2024-03-04T13:42:42.639 INFO:tasks.workunit.client.1.smithi169.stdout:3/119: readlink d7/de/d14/d18/l26 0 2024-03-04T13:42:42.639 INFO:tasks.workunit.client.1.smithi169.stdout:8/162: dread d7/d10/d17/f27 [0,4194304] 0 2024-03-04T13:42:42.641 INFO:tasks.workunit.client.1.smithi169.stdout:6/165: chown d0/f28 7 1 2024-03-04T13:42:42.643 INFO:tasks.workunit.client.1.smithi169.stdout:2/143: mkdir d1/d5/d22/d3b 0 2024-03-04T13:42:42.643 INFO:tasks.workunit.client.1.smithi169.stdout:7/193: dwrite d2/d15/f3c [0,4194304] 0 2024-03-04T13:42:42.644 INFO:tasks.workunit.client.1.smithi169.stdout:0/147: rmdir d5/d1b 0 2024-03-04T13:42:42.647 INFO:tasks.workunit.client.1.smithi169.stdout:8/163: write d7/d1a/f33 [1043650,72522] 0 2024-03-04T13:42:42.650 INFO:tasks.workunit.client.1.smithi169.stdout:5/182: truncate d1/fa 358284 0 2024-03-04T13:42:42.653 INFO:tasks.workunit.client.1.smithi169.stdout:5/183: fsync d1/d16/d17/f1a 0 2024-03-04T13:42:42.654 INFO:tasks.workunit.client.1.smithi169.stdout:7/194: mknod d2/c48 0 2024-03-04T13:42:42.655 INFO:tasks.workunit.client.1.smithi169.stdout:1/151: mknod d4/c30 0 2024-03-04T13:42:42.656 INFO:tasks.workunit.client.1.smithi169.stdout:1/152: fdatasync d4/f1b 0 2024-03-04T13:42:42.658 INFO:tasks.workunit.client.1.smithi169.stdout:1/153: fsync d4/dd/df/f2b 0 2024-03-04T13:42:42.661 INFO:tasks.workunit.client.1.smithi169.stdout:3/120: link d7/f25 d7/de/d14/f2a 0 2024-03-04T13:42:42.661 INFO:tasks.workunit.client.1.smithi169.stdout:3/121: chown d7/de/d14 9041826 1 2024-03-04T13:42:42.664 INFO:tasks.workunit.client.1.smithi169.stdout:8/164: mkdir d7/d10/d14/d36 0 2024-03-04T13:42:42.665 INFO:tasks.workunit.client.1.smithi169.stdout:4/130: getdents d4/d11/d2e/d21 0 2024-03-04T13:42:42.666 INFO:tasks.workunit.client.1.smithi169.stdout:2/144: dwrite d1/f3 [0,4194304] 0 2024-03-04T13:42:42.671 INFO:tasks.workunit.client.1.smithi169.stdout:5/184: creat d1/d16/f31 x:0 0 0 2024-03-04T13:42:42.673 INFO:tasks.workunit.client.1.smithi169.stdout:1/154: chown d4/d5/l14 571524649 1 2024-03-04T13:42:42.675 INFO:tasks.workunit.client.1.smithi169.stdout:8/165: rmdir d7/d10/d22 39 2024-03-04T13:42:42.679 INFO:tasks.workunit.client.1.smithi169.stdout:4/131: rename d4/fb to d4/d11/f30 0 2024-03-04T13:42:42.680 INFO:tasks.workunit.client.1.smithi169.stdout:2/145: mkdir d1/d5/d19/d3c 0 2024-03-04T13:42:42.683 INFO:tasks.workunit.client.1.smithi169.stdout:2/146: write d1/d5/f9 [2290177,7329] 0 2024-03-04T13:42:42.686 INFO:tasks.workunit.client.1.smithi169.stdout:2/147: write d1/f30 [407189,113330] 0 2024-03-04T13:42:42.691 INFO:tasks.workunit.client.1.smithi169.stdout:1/155: unlink d4/dd/f1e 0 2024-03-04T13:42:42.696 INFO:tasks.workunit.client.1.smithi169.stdout:4/132: creat d4/d11/d2e/f31 x:0 0 0 2024-03-04T13:42:42.698 INFO:tasks.workunit.client.1.smithi169.stdout:4/133: write d4/d11/d2e/f2f [1002556,88226] 0 2024-03-04T13:42:42.700 INFO:tasks.workunit.client.1.smithi169.stdout:4/134: chown d4/d11/d2e 15920 1 2024-03-04T13:42:42.700 INFO:tasks.workunit.client.1.smithi169.stdout:0/148: link d5/l2b d5/d9/d39/l3b 0 2024-03-04T13:42:42.707 INFO:tasks.workunit.client.1.smithi169.stdout:7/195: write d2/de/f39 [1046918,108745] 0 2024-03-04T13:42:42.710 INFO:tasks.workunit.client.1.smithi169.stdout:1/156: dwrite d4/f20 [0,4194304] 0 2024-03-04T13:42:42.713 INFO:tasks.workunit.client.1.smithi169.stdout:0/149: write d5/d9/dc/d12/d15/d19/f2e [4650451,38242] 0 2024-03-04T13:42:42.718 INFO:tasks.workunit.client.1.smithi169.stdout:0/150: chown d5/d9/dc/d12/d15/d19/f2e 3 1 2024-03-04T13:42:42.722 INFO:tasks.workunit.client.1.smithi169.stdout:8/166: rename d7/d1a/c21 to d7/c37 0 2024-03-04T13:42:42.728 INFO:tasks.workunit.client.1.smithi169.stdout:4/135: symlink d4/d1c/d25/l32 0 2024-03-04T13:42:42.730 INFO:tasks.workunit.client.1.smithi169.stdout:4/136: truncate d4/d1c/f2c 854194 0 2024-03-04T13:42:42.733 INFO:tasks.workunit.client.1.smithi169.stdout:1/157: symlink d4/d5/l31 0 2024-03-04T13:42:42.738 INFO:tasks.workunit.client.1.smithi169.stdout:7/196: rename d2/d15/f3c to d2/de/d3e/d41/f49 0 2024-03-04T13:42:42.741 INFO:tasks.workunit.client.1.smithi169.stdout:4/137: creat d4/d1c/d25/f33 x:0 0 0 2024-03-04T13:42:42.746 INFO:tasks.workunit.client.1.smithi169.stdout:2/148: write d1/ff [3998028,45675] 0 2024-03-04T13:42:42.751 INFO:tasks.workunit.client.1.smithi169.stdout:4/138: chown d4/f14 1544030 1 2024-03-04T13:42:42.753 INFO:tasks.workunit.client.1.smithi169.stdout:1/158: dwrite d4/dd/df/f1c [4194304,4194304] 0 2024-03-04T13:42:42.757 INFO:tasks.workunit.client.1.smithi169.stdout:2/149: creat d1/f3d x:0 0 0 2024-03-04T13:42:42.759 INFO:tasks.workunit.client.1.smithi169.stdout:5/185: dwrite d1/d16/d17/d28/f2c [4194304,4194304] 0 2024-03-04T13:42:42.759 INFO:tasks.workunit.client.1.smithi169.stdout:1/159: write d4/dd/df/f2b [635229,88845] 0 2024-03-04T13:42:42.761 INFO:tasks.workunit.client.1.smithi169.stdout:8/167: rename d7/c37 to d7/d10/d14/c38 0 2024-03-04T13:42:42.765 INFO:tasks.workunit.client.1.smithi169.stdout:4/139: mknod d4/d11/d2e/d21/c34 0 2024-03-04T13:42:42.770 INFO:tasks.workunit.client.1.smithi169.stdout:5/186: symlink d1/d16/d2a/l32 0 2024-03-04T13:42:42.773 INFO:tasks.workunit.client.1.smithi169.stdout:8/168: unlink d7/l11 0 2024-03-04T13:42:42.777 INFO:tasks.workunit.client.1.smithi169.stdout:7/197: rename d2/d15/d17/f3a to d2/de/f4a 0 2024-03-04T13:42:42.786 INFO:tasks.workunit.client.1.smithi169.stdout:5/187: chown d1/d16/d17/d28/c2b 18 1 2024-03-04T13:42:42.788 INFO:tasks.workunit.client.1.smithi169.stdout:2/150: dwrite d1/d5/d19/f27 [0,4194304] 0 2024-03-04T13:42:42.791 INFO:tasks.workunit.client.1.smithi169.stdout:4/140: rename d4/d11/d2e/f2f to d4/d11/d23/f35 0 2024-03-04T13:42:42.794 INFO:tasks.workunit.client.1.smithi169.stdout:2/151: write d1/f3 [155314,42822] 0 2024-03-04T13:42:42.794 INFO:tasks.workunit.client.1.smithi169.stdout:7/198: dread d2/de/f39 [0,4194304] 0 2024-03-04T13:42:42.801 INFO:tasks.workunit.client.1.smithi169.stdout:9/185: sync 2024-03-04T13:42:42.803 INFO:tasks.workunit.client.1.smithi169.stdout:4/141: symlink d4/d11/d23/l36 0 2024-03-04T13:42:42.816 INFO:tasks.workunit.client.1.smithi169.stdout:2/152: creat d1/d5/d19/d1e/d29/f3e x:0 0 0 2024-03-04T13:42:42.819 INFO:tasks.workunit.client.1.smithi169.stdout:7/199: creat d2/d28/f4b x:0 0 0 2024-03-04T13:42:42.821 INFO:tasks.workunit.client.1.smithi169.stdout:1/160: dwrite d4/dd/df/f28 [4194304,4194304] 0 2024-03-04T13:42:42.824 INFO:tasks.workunit.client.1.smithi169.stdout:2/153: rmdir d1/d5/d19/d36 39 2024-03-04T13:42:42.828 INFO:tasks.workunit.client.1.smithi169.stdout:8/169: rename l2 to d7/d10/l39 0 2024-03-04T13:42:42.828 INFO:tasks.workunit.client.1.smithi169.stdout:7/200: dread d2/f2c [0,4194304] 0 2024-03-04T13:42:42.831 INFO:tasks.workunit.client.1.smithi169.stdout:1/161: unlink d4/d5/f1a 0 2024-03-04T13:42:42.838 INFO:tasks.workunit.client.1.smithi169.stdout:8/170: read d7/d10/d17/f2a [2690681,111362] 0 2024-03-04T13:42:42.840 INFO:tasks.workunit.client.1.smithi169.stdout:7/201: write d2/de/f4a [454096,60791] 0 2024-03-04T13:42:42.844 INFO:tasks.workunit.client.1.smithi169.stdout:1/162: symlink d4/d13/d26/l32 0 2024-03-04T13:42:42.845 INFO:tasks.workunit.client.1.smithi169.stdout:9/186: write d7/fe [1159534,5038] 0 2024-03-04T13:42:42.847 INFO:tasks.workunit.client.1.smithi169.stdout:5/188: rename d1/fa to d1/d16/d2f/f33 0 2024-03-04T13:42:42.848 INFO:tasks.workunit.client.1.smithi169.stdout:9/187: write d7/dc/f1e [1040981,18826] 0 2024-03-04T13:42:42.849 INFO:tasks.workunit.client.1.smithi169.stdout:2/154: creat d1/d5/d19/f3f x:0 0 0 2024-03-04T13:42:42.850 INFO:tasks.workunit.client.1.smithi169.stdout:5/189: write d1/f8 [407868,90240] 0 2024-03-04T13:42:42.854 INFO:tasks.workunit.client.1.smithi169.stdout:2/155: write d1/d5/d19/f2e [1800555,103728] 0 2024-03-04T13:42:42.856 INFO:tasks.workunit.client.1.smithi169.stdout:7/202: dread d2/d28/f33 [0,4194304] 0 2024-03-04T13:42:42.858 INFO:tasks.workunit.client.1.smithi169.stdout:7/203: dread - d2/d28/f4b zero size 2024-03-04T13:42:42.861 INFO:tasks.workunit.client.1.smithi169.stdout:5/190: creat d1/d16/d17/d1b/f34 x:0 0 0 2024-03-04T13:42:42.864 INFO:tasks.workunit.client.1.smithi169.stdout:2/156: creat d1/f40 x:0 0 0 2024-03-04T13:42:42.865 INFO:tasks.workunit.client.1.smithi169.stdout:7/204: creat d2/de/d3e/d41/f4c x:0 0 0 2024-03-04T13:42:42.866 INFO:tasks.workunit.client.1.smithi169.stdout:5/191: truncate d1/d16/d17/d1b/f34 437811 0 2024-03-04T13:42:42.868 INFO:tasks.workunit.client.1.smithi169.stdout:2/157: fdatasync d1/d5/d19/d1e/d29/f3e 0 2024-03-04T13:42:42.869 INFO:tasks.workunit.client.1.smithi169.stdout:5/192: chown d1/d16/l1d 1944616727 1 2024-03-04T13:42:42.872 INFO:tasks.workunit.client.1.smithi169.stdout:7/205: symlink d2/de/l4d 0 2024-03-04T13:42:42.873 INFO:tasks.workunit.client.1.smithi169.stdout:7/206: readlink d2/d28/l2e 0 2024-03-04T13:42:42.877 INFO:tasks.workunit.client.1.smithi169.stdout:7/207: mkdir d2/d15/d17/d4e 0 2024-03-04T13:42:42.882 INFO:tasks.workunit.client.1.smithi169.stdout:7/208: rmdir d2/d28 39 2024-03-04T13:42:42.884 INFO:tasks.workunit.client.1.smithi169.stdout:5/193: dread d1/f24 [0,4194304] 0 2024-03-04T13:42:42.889 INFO:tasks.workunit.client.1.smithi169.stdout:1/163: dread d4/dd/df/f1c [0,4194304] 0 2024-03-04T13:42:42.894 INFO:tasks.workunit.client.1.smithi169.stdout:7/209: readlink d2/d15/l35 0 2024-03-04T13:42:42.899 INFO:tasks.workunit.client.1.smithi169.stdout:7/210: write d2/f46 [308489,57279] 0 2024-03-04T13:42:42.901 INFO:tasks.workunit.client.1.smithi169.stdout:5/194: dwrite d1/f8 [0,4194304] 0 2024-03-04T13:42:42.903 INFO:tasks.workunit.client.1.smithi169.stdout:1/164: dread d4/dd/df/f28 [4194304,4194304] 0 2024-03-04T13:42:42.904 INFO:tasks.workunit.client.1.smithi169.stdout:1/165: write d4/f25 [448219,109896] 0 2024-03-04T13:42:42.916 INFO:tasks.workunit.client.1.smithi169.stdout:2/158: dread d1/d5/f8 [8388608,4194304] 0 2024-03-04T13:42:42.920 INFO:tasks.workunit.client.1.smithi169.stdout:5/195: dread d1/f24 [0,4194304] 0 2024-03-04T13:42:42.920 INFO:tasks.workunit.client.1.smithi169.stdout:9/188: dwrite d7/dc/fd [4194304,4194304] 0 2024-03-04T13:42:42.921 INFO:tasks.workunit.client.1.smithi169.stdout:5/196: readlink d1/lf 0 2024-03-04T13:42:42.924 INFO:tasks.workunit.client.1.smithi169.stdout:9/189: mkdir d7/dc/d3e 0 2024-03-04T13:42:42.926 INFO:tasks.workunit.client.1.smithi169.stdout:5/197: write d1/d16/d17/d1b/f25 [3517250,53021] 0 2024-03-04T13:42:42.932 INFO:tasks.workunit.client.1.smithi169.stdout:7/211: dwrite d2/de/d3e/d41/f4c [0,4194304] 0 2024-03-04T13:42:42.942 INFO:tasks.workunit.client.1.smithi169.stdout:5/198: fsync d1/f8 0 2024-03-04T13:42:42.942 INFO:tasks.workunit.client.1.smithi169.stdout:2/159: dwrite d1/d5/d19/f27 [0,4194304] 0 2024-03-04T13:42:42.952 INFO:tasks.workunit.client.1.smithi169.stdout:9/190: dwrite d7/f10 [4194304,4194304] 0 2024-03-04T13:42:42.952 INFO:tasks.workunit.client.1.smithi169.stdout:5/199: mknod d1/d16/d2f/c35 0 2024-03-04T13:42:42.958 INFO:tasks.workunit.client.1.smithi169.stdout:2/160: chown d1/d5/l35 67622976 1 2024-03-04T13:42:42.958 INFO:tasks.workunit.client.1.smithi169.stdout:2/161: stat d1/d5/d19/f27 0 2024-03-04T13:42:42.963 INFO:tasks.workunit.client.1.smithi169.stdout:7/212: dread d2/de/d3e/d41/f4c [0,4194304] 0 2024-03-04T13:42:42.967 INFO:tasks.workunit.client.1.smithi169.stdout:1/166: dread d4/f25 [0,4194304] 0 2024-03-04T13:42:42.972 INFO:tasks.workunit.client.1.smithi169.stdout:9/191: dwrite f6 [8388608,4194304] 0 2024-03-04T13:42:43.011 INFO:tasks.workunit.client.1.smithi169.stdout:6/166: sync 2024-03-04T13:42:43.042 INFO:tasks.workunit.client.1.smithi169.stdout:5/200: rename d1/d16/d17/c19 to d1/d16/d2a/c36 0 2024-03-04T13:42:43.045 INFO:tasks.workunit.client.1.smithi169.stdout:2/162: truncate d1/d5/f8 7175431 0 2024-03-04T13:42:43.046 INFO:tasks.workunit.client.1.smithi169.stdout:9/192: symlink d7/d25/l3f 0 2024-03-04T13:42:43.047 INFO:tasks.workunit.client.1.smithi169.stdout:1/167: rename d4 to d4/dd/d33 22 2024-03-04T13:42:43.050 INFO:tasks.workunit.client.1.smithi169.stdout:5/201: symlink d1/d23/l37 0 2024-03-04T13:42:43.050 INFO:tasks.workunit.client.1.smithi169.stdout:1/168: creat d4/d1d/f34 x:0 0 0 2024-03-04T13:42:43.053 INFO:tasks.workunit.client.1.smithi169.stdout:1/169: chown d4/f20 188849 1 2024-03-04T13:42:43.055 INFO:tasks.workunit.client.1.smithi169.stdout:9/193: creat d7/d2d/f40 x:0 0 0 2024-03-04T13:42:43.057 INFO:tasks.workunit.client.1.smithi169.stdout:1/170: write d4/f25 [3165501,4180] 0 2024-03-04T13:42:43.057 INFO:tasks.workunit.client.1.smithi169.stdout:9/194: chown f6 20724 1 2024-03-04T13:42:43.063 INFO:tasks.workunit.client.1.smithi169.stdout:7/213: dwrite d2/f1c [0,4194304] 0 2024-03-04T13:42:43.063 INFO:tasks.workunit.client.1.smithi169.stdout:1/171: chown d4/d5/d2d 1151 1 2024-03-04T13:42:43.063 INFO:tasks.workunit.client.1.smithi169.stdout:2/163: truncate d1/d5/d19/f27 2177306 0 2024-03-04T13:42:43.070 INFO:tasks.workunit.client.1.smithi169.stdout:5/202: dread d1/f24 [0,4194304] 0 2024-03-04T13:42:43.070 INFO:tasks.workunit.client.1.smithi169.stdout:7/214: mknod d2/de/d3e/d41/c4f 0 2024-03-04T13:42:43.072 INFO:tasks.workunit.client.1.smithi169.stdout:1/172: rename d4/dd to d4/d5/d27/d35 0 2024-03-04T13:42:43.075 INFO:tasks.workunit.client.1.smithi169.stdout:2/164: mkdir d1/d5/d22/d3b/d41 0 2024-03-04T13:42:43.077 INFO:tasks.workunit.client.1.smithi169.stdout:7/215: unlink d2/de/f3d 0 2024-03-04T13:42:43.080 INFO:tasks.workunit.client.1.smithi169.stdout:2/165: write d1/d5/d19/f3f [925225,81971] 0 2024-03-04T13:42:43.082 INFO:tasks.workunit.client.1.smithi169.stdout:2/166: read d1/d5/d19/f2e [2170107,5139] 0 2024-03-04T13:42:43.084 INFO:tasks.workunit.client.1.smithi169.stdout:7/216: fdatasync d2/de/d3e/d41/f4c 0 2024-03-04T13:42:43.086 INFO:tasks.workunit.client.1.smithi169.stdout:9/195: truncate d7/f10 5199597 0 2024-03-04T13:42:43.087 INFO:tasks.workunit.client.1.smithi169.stdout:5/203: getdents d1/d16 0 2024-03-04T13:42:43.088 INFO:tasks.workunit.client.1.smithi169.stdout:5/204: read d1/f24 [238045,81186] 0 2024-03-04T13:42:43.089 INFO:tasks.workunit.client.1.smithi169.stdout:1/173: rename d4/d5/d27 to d4/d5/d36 0 2024-03-04T13:42:43.091 INFO:tasks.workunit.client.1.smithi169.stdout:2/167: mkdir d1/d5/d42 0 2024-03-04T13:42:43.095 INFO:tasks.workunit.client.1.smithi169.stdout:5/205: mkdir d1/d16/d2f/d38 0 2024-03-04T13:42:43.096 INFO:tasks.workunit.client.1.smithi169.stdout:1/174: creat d4/d13/f37 x:0 0 0 2024-03-04T13:42:43.097 INFO:tasks.workunit.client.1.smithi169.stdout:9/196: creat d7/dc/d3e/f41 x:0 0 0 2024-03-04T13:42:43.099 INFO:tasks.workunit.client.1.smithi169.stdout:2/168: creat d1/d5/d19/d1e/f43 x:0 0 0 2024-03-04T13:42:43.101 INFO:tasks.workunit.client.1.smithi169.stdout:9/197: write d7/f11 [1367192,114023] 0 2024-03-04T13:42:43.103 INFO:tasks.workunit.client.1.smithi169.stdout:7/217: getdents d2/de 0 2024-03-04T13:42:43.105 INFO:tasks.workunit.client.1.smithi169.stdout:1/175: truncate d4/d5/d36/d35/df/f2b 1318212 0 2024-03-04T13:42:43.107 INFO:tasks.workunit.client.1.smithi169.stdout:1/176: truncate d4/f20 5084464 0 2024-03-04T13:42:43.108 INFO:tasks.workunit.client.1.smithi169.stdout:7/218: write d2/de/d37/f3f [1727977,89493] 0 2024-03-04T13:42:43.108 INFO:tasks.workunit.client.1.smithi169.stdout:2/169: symlink d1/d5/d22/l44 0 2024-03-04T13:42:43.116 INFO:tasks.workunit.client.1.smithi169.stdout:1/177: fsync d4/d5/d36/d35/df/f1c 0 2024-03-04T13:42:43.116 INFO:tasks.workunit.client.1.smithi169.stdout:5/206: dread d1/d16/d17/d28/f2c [0,4194304] 0 2024-03-04T13:42:43.117 INFO:tasks.workunit.client.1.smithi169.stdout:5/207: fsync d1/f8 0 2024-03-04T13:42:43.118 INFO:tasks.workunit.client.1.smithi169.stdout:7/219: rename d2/d15/l35 to d2/d28/l50 0 2024-03-04T13:42:43.118 INFO:tasks.workunit.client.1.smithi169.stdout:5/208: fsync d1/d16/d17/f18 0 2024-03-04T13:42:43.122 INFO:tasks.workunit.client.1.smithi169.stdout:5/209: read d1/d16/d17/f1a [35417,77087] 0 2024-03-04T13:42:43.124 INFO:tasks.workunit.client.1.smithi169.stdout:1/178: creat d4/d5/f38 x:0 0 0 2024-03-04T13:42:43.125 INFO:tasks.workunit.client.1.smithi169.stdout:2/170: dwrite d1/f30 [0,4194304] 0 2024-03-04T13:42:43.127 INFO:tasks.workunit.client.1.smithi169.stdout:7/220: symlink d2/d15/l51 0 2024-03-04T13:42:43.132 INFO:tasks.workunit.client.1.smithi169.stdout:7/221: symlink d2/d15/l52 0 2024-03-04T13:42:43.135 INFO:tasks.workunit.client.1.smithi169.stdout:7/222: chown d2/f2c 170745101 1 2024-03-04T13:42:43.138 INFO:tasks.workunit.client.1.smithi169.stdout:7/223: unlink d2/f46 0 2024-03-04T13:42:43.139 INFO:tasks.workunit.client.1.smithi169.stdout:5/210: write d1/d16/d2f/f33 [705673,18009] 0 2024-03-04T13:42:43.140 INFO:tasks.workunit.client.1.smithi169.stdout:9/198: truncate d7/f10 150879 0 2024-03-04T13:42:43.146 INFO:tasks.workunit.client.1.smithi169.stdout:9/199: chown d7/d2d/l2e 99 1 2024-03-04T13:42:43.148 INFO:tasks.workunit.client.1.smithi169.stdout:2/171: dwrite d1/f40 [0,4194304] 0 2024-03-04T13:42:43.148 INFO:tasks.workunit.client.1.smithi169.stdout:9/200: read - d7/dc/d12/d20/f24 zero size 2024-03-04T13:42:43.151 INFO:tasks.workunit.client.1.smithi169.stdout:1/179: dwrite d4/d5/d36/d35/df/f28 [0,4194304] 0 2024-03-04T13:42:43.155 INFO:tasks.workunit.client.1.smithi169.stdout:2/172: chown d1/d5/d22/l44 12 1 2024-03-04T13:42:43.161 INFO:tasks.workunit.client.1.smithi169.stdout:1/180: mknod d4/d5/d2d/c39 0 2024-03-04T13:42:43.161 INFO:tasks.workunit.client.1.smithi169.stdout:2/173: mkdir d1/d5/d22/d45 0 2024-03-04T13:42:43.162 INFO:tasks.workunit.client.1.smithi169.stdout:9/201: dread d7/dc/d12/d32/f3b [0,4194304] 0 2024-03-04T13:42:43.171 INFO:tasks.workunit.client.1.smithi169.stdout:9/202: write d7/dc/f1d [1387920,36632] 0 2024-03-04T13:42:43.175 INFO:tasks.workunit.client.1.smithi169.stdout:1/181: getdents d4/d13 0 2024-03-04T13:42:43.176 INFO:tasks.workunit.client.1.smithi169.stdout:2/174: getdents d1/d5/d19/d1e 0 2024-03-04T13:42:43.179 INFO:tasks.workunit.client.1.smithi169.stdout:9/203: readlink d7/d25/d3c/l38 0 2024-03-04T13:42:43.180 INFO:tasks.workunit.client.1.smithi169.stdout:7/224: dwrite d2/fd [0,4194304] 0 2024-03-04T13:42:43.181 INFO:tasks.workunit.client.1.smithi169.stdout:5/211: dwrite d1/f14 [4194304,4194304] 0 2024-03-04T13:42:43.181 INFO:tasks.workunit.client.1.smithi169.stdout:1/182: mkdir d4/d5/d36/d35/d16/d3a 0 2024-03-04T13:42:43.183 INFO:tasks.workunit.client.1.smithi169.stdout:2/175: symlink d1/d5/d19/d3c/l46 0 2024-03-04T13:42:43.187 INFO:tasks.workunit.client.1.smithi169.stdout:1/183: mknod d4/d5/d36/c3b 0 2024-03-04T13:42:43.187 INFO:tasks.workunit.client.1.smithi169.stdout:2/176: write d1/d5/f9 [3773692,64509] 0 2024-03-04T13:42:43.187 INFO:tasks.workunit.client.1.smithi169.stdout:9/204: mkdir d7/dc/d12/d32/d42 0 2024-03-04T13:42:43.192 INFO:tasks.workunit.client.1.smithi169.stdout:7/225: rmdir d2/d15/d17/d4e 0 2024-03-04T13:42:43.192 INFO:tasks.workunit.client.1.smithi169.stdout:1/184: dread d4/d1d/f23 [0,4194304] 0 2024-03-04T13:42:43.200 INFO:tasks.workunit.client.1.smithi169.stdout:7/226: creat d2/de/f53 x:0 0 0 2024-03-04T13:42:43.206 INFO:tasks.workunit.client.1.smithi169.stdout:3/122: sync 2024-03-04T13:42:43.207 INFO:tasks.workunit.client.1.smithi169.stdout:9/205: dread f6 [0,4194304] 0 2024-03-04T13:42:43.209 INFO:tasks.workunit.client.1.smithi169.stdout:7/227: unlink d2/c21 0 2024-03-04T13:42:43.218 INFO:tasks.workunit.client.1.smithi169.stdout:9/206: dread d7/dc/f1e [0,4194304] 0 2024-03-04T13:42:43.218 INFO:tasks.workunit.client.1.smithi169.stdout:2/177: write d1/d5/d19/f27 [3188828,62203] 0 2024-03-04T13:42:43.220 INFO:tasks.workunit.client.1.smithi169.stdout:9/207: write d7/d27/f2f [44496,119526] 0 2024-03-04T13:42:43.222 INFO:tasks.workunit.client.1.smithi169.stdout:1/185: truncate d4/f25 821527 0 2024-03-04T13:42:43.223 INFO:tasks.workunit.client.1.smithi169.stdout:7/228: mkdir d2/d15/d54 0 2024-03-04T13:42:43.226 INFO:tasks.workunit.client.1.smithi169.stdout:1/186: mknod d4/d5/d36/c3c 0 2024-03-04T13:42:43.228 INFO:tasks.workunit.client.1.smithi169.stdout:5/212: dread d1/d16/d17/f1a [0,4194304] 0 2024-03-04T13:42:43.230 INFO:tasks.workunit.client.1.smithi169.stdout:2/178: creat d1/d5/d19/f47 x:0 0 0 2024-03-04T13:42:43.231 INFO:tasks.workunit.client.1.smithi169.stdout:7/229: dread d2/de/f4a [0,4194304] 0 2024-03-04T13:42:43.231 INFO:tasks.workunit.client.1.smithi169.stdout:3/123: truncate d7/f1d 1490784 0 2024-03-04T13:42:43.233 INFO:tasks.workunit.client.1.smithi169.stdout:5/213: mkdir d1/d16/d2f/d39 0 2024-03-04T13:42:43.236 INFO:tasks.workunit.client.1.smithi169.stdout:3/124: mkdir d7/de/d2b 0 2024-03-04T13:42:43.237 INFO:tasks.workunit.client.1.smithi169.stdout:2/179: mknod d1/d5/d19/c48 0 2024-03-04T13:42:43.239 INFO:tasks.workunit.client.1.smithi169.stdout:2/180: fsync d1/d5/fe 0 2024-03-04T13:42:43.269 INFO:tasks.workunit.client.1.smithi169.stdout:3/125: fdatasync d7/de/d14/f2a 0 2024-03-04T13:42:43.269 INFO:tasks.workunit.client.1.smithi169.stdout:2/181: write d1/f40 [1418665,33170] 0 2024-03-04T13:42:43.271 INFO:tasks.workunit.client.1.smithi169.stdout:5/214: truncate d1/f8 740144 0 2024-03-04T13:42:43.277 INFO:tasks.workunit.client.1.smithi169.stdout:3/126: mkdir d7/de/d14/d18/d21/d2c 0 2024-03-04T13:42:43.279 INFO:tasks.workunit.client.1.smithi169.stdout:3/127: chown d7/l8 195378064 1 2024-03-04T13:42:43.283 INFO:tasks.workunit.client.1.smithi169.stdout:9/208: dwrite d7/f10 [0,4194304] 0 2024-03-04T13:42:43.284 INFO:tasks.workunit.client.1.smithi169.stdout:7/230: dwrite d2/de/d3e/d41/f4c [0,4194304] 0 2024-03-04T13:42:43.284 INFO:tasks.workunit.client.1.smithi169.stdout:2/182: rename d1/d5/d19/c28 to d1/d5/d22/d3b/d41/c49 0 2024-03-04T13:42:43.291 INFO:tasks.workunit.client.1.smithi169.stdout:7/231: symlink d2/d28/l55 0 2024-03-04T13:42:43.295 INFO:tasks.workunit.client.1.smithi169.stdout:5/215: dwrite d1/d16/d17/f18 [0,4194304] 0 2024-03-04T13:42:43.300 INFO:tasks.workunit.client.1.smithi169.stdout:9/209: link d7/f11 d7/dc/d12/d32/d42/f43 0 2024-03-04T13:42:43.300 INFO:tasks.workunit.client.1.smithi169.stdout:2/183: truncate d1/d5/f8 5760445 0 2024-03-04T13:42:43.305 INFO:tasks.workunit.client.1.smithi169.stdout:2/184: fsync d1/d5/d19/d1e/f24 0 2024-03-04T13:42:43.305 INFO:tasks.workunit.client.1.smithi169.stdout:5/216: creat d1/d16/f3a x:0 0 0 2024-03-04T13:42:43.305 INFO:tasks.workunit.client.1.smithi169.stdout:9/210: symlink d7/dc/l44 0 2024-03-04T13:42:43.309 INFO:tasks.workunit.client.1.smithi169.stdout:2/185: mkdir d1/d5/d22/d45/d4a 0 2024-03-04T13:42:43.310 INFO:tasks.workunit.client.1.smithi169.stdout:5/217: dread d1/f8 [0,4194304] 0 2024-03-04T13:42:43.312 INFO:tasks.workunit.client.1.smithi169.stdout:9/211: unlink d7/dc/d12/f18 0 2024-03-04T13:42:43.315 INFO:tasks.workunit.client.1.smithi169.stdout:2/186: rmdir d1/d5/d22/d45 39 2024-03-04T13:42:43.315 INFO:tasks.workunit.client.1.smithi169.stdout:9/212: chown d7/dc/fd 4270 1 2024-03-04T13:42:43.318 INFO:tasks.workunit.client.1.smithi169.stdout:5/218: link d1/f14 d1/d16/f3b 0 2024-03-04T13:42:43.319 INFO:tasks.workunit.client.1.smithi169.stdout:7/232: dwrite d2/f7 [0,4194304] 0 2024-03-04T13:42:43.320 INFO:tasks.workunit.client.1.smithi169.stdout:9/213: symlink d7/dc/l45 0 2024-03-04T13:42:43.325 INFO:tasks.workunit.client.1.smithi169.stdout:7/233: creat d2/d28/f56 x:0 0 0 2024-03-04T13:42:43.327 INFO:tasks.workunit.client.1.smithi169.stdout:5/219: mknod d1/d16/d2f/d38/c3c 0 2024-03-04T13:42:43.327 INFO:tasks.workunit.client.1.smithi169.stdout:7/234: read d2/de/f4a [387827,56874] 0 2024-03-04T13:42:43.336 INFO:tasks.workunit.client.1.smithi169.stdout:3/128: dread f4 [0,4194304] 0 2024-03-04T13:42:43.338 INFO:tasks.workunit.client.1.smithi169.stdout:3/129: stat d7/de/d14/f20 0 2024-03-04T13:42:43.341 INFO:tasks.workunit.client.1.smithi169.stdout:3/130: mkdir d7/de/d14/d13/d1a/d2d 0 2024-03-04T13:42:43.342 INFO:tasks.workunit.client.1.smithi169.stdout:2/187: write d1/d5/d19/f2e [2219226,114467] 0 2024-03-04T13:42:43.352 INFO:tasks.workunit.client.1.smithi169.stdout:2/188: creat d1/d5/d19/d36/f4b x:0 0 0 2024-03-04T13:42:43.356 INFO:tasks.workunit.client.1.smithi169.stdout:3/131: write d7/f25 [297166,101779] 0 2024-03-04T13:42:43.361 INFO:tasks.workunit.client.1.smithi169.stdout:3/132: creat d7/de/d14/d13/d1a/d2d/f2e x:0 0 0 2024-03-04T13:42:43.365 INFO:tasks.workunit.client.1.smithi169.stdout:7/235: dread d2/de/f1e [0,4194304] 0 2024-03-04T13:42:43.366 INFO:tasks.workunit.client.1.smithi169.stdout:5/220: dread d1/f9 [0,4194304] 0 2024-03-04T13:42:43.371 INFO:tasks.workunit.client.1.smithi169.stdout:2/189: dwrite d1/ff [0,4194304] 0 2024-03-04T13:42:43.371 INFO:tasks.workunit.client.1.smithi169.stdout:7/236: chown d2/f2d 204 1 2024-03-04T13:42:43.372 INFO:tasks.workunit.client.1.smithi169.stdout:3/133: write f4 [3315951,109987] 0 2024-03-04T13:42:43.375 INFO:tasks.workunit.client.1.smithi169.stdout:5/221: rename d1/d16/d2a/c36 to d1/d16/d17/d2d/c3d 0 2024-03-04T13:42:43.378 INFO:tasks.workunit.client.1.smithi169.stdout:0/151: sync 2024-03-04T13:42:43.382 INFO:tasks.workunit.client.1.smithi169.stdout:3/134: dread d7/de/d14/f20 [0,4194304] 0 2024-03-04T13:42:43.384 INFO:tasks.workunit.client.1.smithi169.stdout:3/135: stat d7/f25 0 2024-03-04T13:42:43.387 INFO:tasks.workunit.client.1.smithi169.stdout:0/152: getdents d5/d9/dc/d1c 0 2024-03-04T13:42:43.389 INFO:tasks.workunit.client.1.smithi169.stdout:5/222: write d1/d16/d17/d1b/f25 [198180,37448] 0 2024-03-04T13:42:43.393 INFO:tasks.workunit.client.1.smithi169.stdout:7/237: dwrite d2/f2c [0,4194304] 0 2024-03-04T13:42:43.395 INFO:tasks.workunit.client.1.smithi169.stdout:2/190: dwrite d1/d5/d22/f32 [0,4194304] 0 2024-03-04T13:42:43.395 INFO:tasks.workunit.client.1.smithi169.stdout:5/223: read d1/d16/d17/d1b/f1e [949500,99281] 0 2024-03-04T13:42:43.399 INFO:tasks.workunit.client.1.smithi169.stdout:5/224: read d1/d16/d17/f18 [3799937,60144] 0 2024-03-04T13:42:43.404 INFO:tasks.workunit.client.1.smithi169.stdout:3/136: truncate f5 2244954 0 2024-03-04T13:42:43.409 INFO:tasks.workunit.client.1.smithi169.stdout:2/191: creat d1/d5/d22/d3b/f4c x:0 0 0 2024-03-04T13:42:43.411 INFO:tasks.workunit.client.1.smithi169.stdout:2/192: fdatasync d1/d5/d19/f2e 0 2024-03-04T13:42:43.412 INFO:tasks.workunit.client.1.smithi169.stdout:5/225: creat d1/d16/d2a/f3e x:0 0 0 2024-03-04T13:42:43.412 INFO:tasks.workunit.client.1.smithi169.stdout:7/238: mknod d2/d15/d54/c57 0 2024-03-04T13:42:43.415 INFO:tasks.workunit.client.1.smithi169.stdout:3/137: mknod d7/de/d14/d18/c2f 0 2024-03-04T13:42:43.415 INFO:tasks.workunit.client.1.smithi169.stdout:2/193: fdatasync d1/d5/fe 0 2024-03-04T13:42:43.415 INFO:tasks.workunit.client.1.smithi169.stdout:7/239: stat d2/d28/f32 0 2024-03-04T13:42:43.417 INFO:tasks.workunit.client.1.smithi169.stdout:0/153: creat d5/d9/dc/d12/f3c x:0 0 0 2024-03-04T13:42:43.418 INFO:tasks.workunit.client.1.smithi169.stdout:5/226: unlink d1/f9 0 2024-03-04T13:42:43.423 INFO:tasks.workunit.client.1.smithi169.stdout:3/138: creat d7/de/d14/d18/f30 x:0 0 0 2024-03-04T13:42:43.427 INFO:tasks.workunit.client.1.smithi169.stdout:3/139: fsync d7/de/d14/d18/f27 0 2024-03-04T13:42:43.428 INFO:tasks.workunit.client.1.smithi169.stdout:2/194: creat d1/d5/d22/d3b/f4d x:0 0 0 2024-03-04T13:42:43.432 INFO:tasks.workunit.client.1.smithi169.stdout:3/140: chown d7/cc 1416758 1 2024-03-04T13:42:43.433 INFO:tasks.workunit.client.1.smithi169.stdout:3/141: stat d7/de/d14/d13 0 2024-03-04T13:42:43.435 INFO:tasks.workunit.client.1.smithi169.stdout:3/142: chown f5 15942714 1 2024-03-04T13:42:43.435 INFO:tasks.workunit.client.1.smithi169.stdout:5/227: dread d1/d16/f20 [4194304,4194304] 0 2024-03-04T13:42:43.437 INFO:tasks.workunit.client.1.smithi169.stdout:7/240: dwrite d2/de/d37/f3f [0,4194304] 0 2024-03-04T13:42:43.439 INFO:tasks.workunit.client.1.smithi169.stdout:2/195: creat d1/d5/f4e x:0 0 0 2024-03-04T13:42:43.440 INFO:tasks.workunit.client.1.smithi169.stdout:3/143: dread - d7/de/d14/d18/f30 zero size 2024-03-04T13:42:43.441 INFO:tasks.workunit.client.1.smithi169.stdout:2/196: stat d1/d5/d22/l26 0 2024-03-04T13:42:43.443 INFO:tasks.workunit.client.1.smithi169.stdout:7/241: fdatasync d2/f34 0 2024-03-04T13:42:43.445 INFO:tasks.workunit.client.1.smithi169.stdout:7/242: chown d2/de/l11 2433 1 2024-03-04T13:42:43.449 INFO:tasks.workunit.client.1.smithi169.stdout:3/144: creat d7/de/d14/d18/d21/f31 x:0 0 0 2024-03-04T13:42:43.449 INFO:tasks.workunit.client.1.smithi169.stdout:2/197: creat d1/d5/f4f x:0 0 0 2024-03-04T13:42:43.451 INFO:tasks.workunit.client.1.smithi169.stdout:0/154: write d5/f8 [2920021,18246] 0 2024-03-04T13:42:43.452 INFO:tasks.workunit.client.1.smithi169.stdout:2/198: write d1/d5/fe [7138722,95248] 0 2024-03-04T13:42:43.453 INFO:tasks.workunit.client.1.smithi169.stdout:7/243: symlink d2/de/l58 0 2024-03-04T13:42:43.454 INFO:tasks.workunit.client.1.smithi169.stdout:0/155: chown d5/d9/dc/d1c/c26 317319790 1 2024-03-04T13:42:43.455 INFO:tasks.workunit.client.1.smithi169.stdout:3/145: mkdir d7/de/d14/d18/d21/d32 0 2024-03-04T13:42:43.458 INFO:tasks.workunit.client.1.smithi169.stdout:2/199: creat d1/f50 x:0 0 0 2024-03-04T13:42:43.458 INFO:tasks.workunit.client.1.smithi169.stdout:0/156: stat l4 0 2024-03-04T13:42:43.460 INFO:tasks.workunit.client.1.smithi169.stdout:2/200: dread - d1/d5/d22/d3b/f4d zero size 2024-03-04T13:42:43.461 INFO:tasks.workunit.client.1.smithi169.stdout:5/228: write d1/d16/d17/d1b/f1e [689781,73593] 0 2024-03-04T13:42:43.467 INFO:tasks.workunit.client.1.smithi169.stdout:3/146: creat d7/f33 x:0 0 0 2024-03-04T13:42:43.467 INFO:tasks.workunit.client.1.smithi169.stdout:2/201: link d1/f30 d1/f51 0 2024-03-04T13:42:43.469 INFO:tasks.workunit.client.1.smithi169.stdout:5/229: chown d1/d16/d17/f1a 113640 1 2024-03-04T13:42:43.473 INFO:tasks.workunit.client.1.smithi169.stdout:5/230: write d1/d16/d2f/f33 [149992,124514] 0 2024-03-04T13:42:43.475 INFO:tasks.workunit.client.1.smithi169.stdout:7/244: dwrite d2/fd [0,4194304] 0 2024-03-04T13:42:43.476 INFO:tasks.workunit.client.1.smithi169.stdout:2/202: truncate d1/d5/d19/d1e/d29/f3e 947501 0 2024-03-04T13:42:43.482 INFO:tasks.workunit.client.1.smithi169.stdout:6/167: sync 2024-03-04T13:42:43.482 INFO:tasks.workunit.client.1.smithi169.stdout:8/171: sync 2024-03-04T13:42:43.482 INFO:tasks.workunit.client.1.smithi169.stdout:4/142: sync 2024-03-04T13:42:43.483 INFO:tasks.workunit.client.1.smithi169.stdout:8/172: chown d7 10 1 2024-03-04T13:42:43.484 INFO:tasks.workunit.client.1.smithi169.stdout:0/157: dwrite d5/d9/dc/d12/d15/d19/f20 [0,4194304] 0 2024-03-04T13:42:43.488 INFO:tasks.workunit.client.1.smithi169.stdout:7/245: rename d2/d28/f32 to d2/de/d3e/d41/f59 0 2024-03-04T13:42:43.491 INFO:tasks.workunit.client.1.smithi169.stdout:0/158: mkdir d5/d9/d3d 0 2024-03-04T13:42:43.491 INFO:tasks.workunit.client.1.smithi169.stdout:7/246: chown d2/f1c 929384 1 2024-03-04T13:42:43.494 INFO:tasks.workunit.client.1.smithi169.stdout:2/203: dread d1/f40 [0,4194304] 0 2024-03-04T13:42:43.494 INFO:tasks.workunit.client.1.smithi169.stdout:7/247: read d2/fd [1455205,80361] 0 2024-03-04T13:42:43.496 INFO:tasks.workunit.client.1.smithi169.stdout:6/168: rename d0/d31 to d0/d32/d3a 0 2024-03-04T13:42:43.497 INFO:tasks.workunit.client.1.smithi169.stdout:7/248: read d2/de/f4a [32993,69298] 0 2024-03-04T13:42:43.499 INFO:tasks.workunit.client.1.smithi169.stdout:4/143: unlink d4/c2d 0 2024-03-04T13:42:43.503 INFO:tasks.workunit.client.1.smithi169.stdout:2/204: readlink d1/l17 0 2024-03-04T13:42:43.504 INFO:tasks.workunit.client.1.smithi169.stdout:6/169: read d0/d32/d3a/f23 [578133,5867] 0 2024-03-04T13:42:43.510 INFO:tasks.workunit.client.1.smithi169.stdout:8/173: getdents d7/d10/d17 0 2024-03-04T13:42:43.510 INFO:tasks.workunit.client.1.smithi169.stdout:3/147: dread d7/fb [0,4194304] 0 2024-03-04T13:42:43.512 INFO:tasks.workunit.client.1.smithi169.stdout:0/159: mknod d5/d9/dc/d12/c3e 0 2024-03-04T13:42:43.512 INFO:tasks.workunit.client.1.smithi169.stdout:3/148: read - d7/de/d14/d18/f30 zero size 2024-03-04T13:42:43.515 INFO:tasks.workunit.client.1.smithi169.stdout:2/205: creat d1/d5/d22/d3b/f52 x:0 0 0 2024-03-04T13:42:43.515 INFO:tasks.workunit.client.1.smithi169.stdout:3/149: read - d7/de/d14/d13/d1a/f28 zero size 2024-03-04T13:42:43.518 INFO:tasks.workunit.client.1.smithi169.stdout:7/249: link d2/fd d2/de/d3e/d41/f5a 0 2024-03-04T13:42:43.524 INFO:tasks.workunit.client.1.smithi169.stdout:8/174: creat d7/d12/f3a x:0 0 0 2024-03-04T13:42:43.525 INFO:tasks.workunit.client.1.smithi169.stdout:5/231: write d1/d16/f20 [1776795,28369] 0 2024-03-04T13:42:43.529 INFO:tasks.workunit.client.1.smithi169.stdout:6/170: creat d0/f3b x:0 0 0 2024-03-04T13:42:43.530 INFO:tasks.workunit.client.1.smithi169.stdout:0/160: unlink d5/cd 0 2024-03-04T13:42:43.532 INFO:tasks.workunit.client.1.smithi169.stdout:5/232: truncate d1/d16/d2f/f33 848553 0 2024-03-04T13:42:43.534 INFO:tasks.workunit.client.1.smithi169.stdout:8/175: creat d7/d10/f3b x:0 0 0 2024-03-04T13:42:43.535 INFO:tasks.workunit.client.1.smithi169.stdout:7/250: dread d2/de/d3e/d41/f5a [0,4194304] 0 2024-03-04T13:42:43.536 INFO:tasks.workunit.client.1.smithi169.stdout:0/161: chown d5/d9/dc/d12/l38 339648611 1 2024-03-04T13:42:43.538 INFO:tasks.workunit.client.1.smithi169.stdout:4/144: dwrite d4/f15 [0,4194304] 0 2024-03-04T13:42:43.541 INFO:tasks.workunit.client.1.smithi169.stdout:7/251: chown d2/l3 2 1 2024-03-04T13:42:43.545 INFO:tasks.workunit.client.1.smithi169.stdout:7/252: chown d2/de/d3e/d41 10 1 2024-03-04T13:42:43.546 INFO:tasks.workunit.client.1.smithi169.stdout:3/150: dwrite d7/de/d14/d18/d21/f31 [0,4194304] 0 2024-03-04T13:42:43.552 INFO:tasks.workunit.client.1.smithi169.stdout:8/176: rename d7/f34 to d7/d10/d24/f3c 0 2024-03-04T13:42:43.552 INFO:tasks.workunit.client.1.smithi169.stdout:0/162: mknod d5/d9/d39/c3f 0 2024-03-04T13:42:43.553 INFO:tasks.workunit.client.1.smithi169.stdout:8/177: chown d7/d10/d2d 77319861 1 2024-03-04T13:42:43.554 INFO:tasks.workunit.client.1.smithi169.stdout:3/151: dread d7/de/d14/f20 [0,4194304] 0 2024-03-04T13:42:43.559 INFO:tasks.workunit.client.1.smithi169.stdout:0/163: fdatasync d5/d9/dc/d12/f3c 0 2024-03-04T13:42:43.560 INFO:tasks.workunit.client.1.smithi169.stdout:3/152: chown d7/de/d14/d18/d21 254 1 2024-03-04T13:42:43.561 INFO:tasks.workunit.client.1.smithi169.stdout:0/164: readlink l4 0 2024-03-04T13:42:43.564 INFO:tasks.workunit.client.1.smithi169.stdout:8/178: fsync d7/d10/d22/f30 0 2024-03-04T13:42:43.570 INFO:tasks.workunit.client.1.smithi169.stdout:5/233: dread d1/d16/f3b [4194304,4194304] 0 2024-03-04T13:42:43.571 INFO:tasks.workunit.client.1.smithi169.stdout:3/153: symlink d7/de/d24/l34 0 2024-03-04T13:42:43.572 INFO:tasks.workunit.client.1.smithi169.stdout:7/253: dwrite d2/de/f2a [0,4194304] 0 2024-03-04T13:42:43.575 INFO:tasks.workunit.client.1.smithi169.stdout:4/145: dwrite d4/d11/d2e/f31 [0,4194304] 0 2024-03-04T13:42:43.577 INFO:tasks.workunit.client.1.smithi169.stdout:0/165: truncate d5/d9/dc/d12/d15/d19/f32 169690 0 2024-03-04T13:42:43.582 INFO:tasks.workunit.client.1.smithi169.stdout:3/154: symlink d7/de/d14/d18/d21/l35 0 2024-03-04T13:42:43.583 INFO:tasks.workunit.client.1.smithi169.stdout:7/254: truncate d2/de/f39 208170 0 2024-03-04T13:42:43.583 INFO:tasks.workunit.client.1.smithi169.stdout:4/146: write d4/f15 [4559490,71747] 0 2024-03-04T13:42:43.585 INFO:tasks.workunit.client.1.smithi169.stdout:5/234: creat d1/d16/d17/d28/f3f x:0 0 0 2024-03-04T13:42:43.586 INFO:tasks.workunit.client.1.smithi169.stdout:3/155: write f6 [2455054,92663] 0 2024-03-04T13:42:43.592 INFO:tasks.workunit.client.1.smithi169.stdout:7/255: mkdir d2/de/d3e/d41/d5b 0 2024-03-04T13:42:43.604 INFO:tasks.workunit.client.1.smithi169.stdout:8/179: dwrite d7/d10/d24/f2c [0,4194304] 0 2024-03-04T13:42:43.610 INFO:tasks.workunit.client.1.smithi169.stdout:4/147: getdents d4 0 2024-03-04T13:42:43.612 INFO:tasks.workunit.client.1.smithi169.stdout:3/156: creat d7/f36 x:0 0 0 2024-03-04T13:42:43.612 INFO:tasks.workunit.client.1.smithi169.stdout:4/148: chown d4/l16 37204120 1 2024-03-04T13:42:43.616 INFO:tasks.workunit.client.1.smithi169.stdout:7/256: read d2/f1f [261472,118326] 0 2024-03-04T13:42:43.618 INFO:tasks.workunit.client.1.smithi169.stdout:7/257: write d2/f2d [1005564,87711] 0 2024-03-04T13:42:43.627 INFO:tasks.workunit.client.1.smithi169.stdout:5/235: dread d1/d16/d17/f18 [0,4194304] 0 2024-03-04T13:42:43.628 INFO:tasks.workunit.client.1.smithi169.stdout:8/180: symlink d7/d10/d14/d36/l3d 0 2024-03-04T13:42:43.630 INFO:tasks.workunit.client.1.smithi169.stdout:7/258: unlink d2/f2c 0 2024-03-04T13:42:43.639 INFO:tasks.workunit.client.1.smithi169.stdout:0/166: dread d5/f8 [0,4194304] 0 2024-03-04T13:42:43.640 INFO:tasks.workunit.client.1.smithi169.stdout:8/181: rename d7/d1a to d7/d10/d24/d3e 0 2024-03-04T13:42:43.642 INFO:tasks.workunit.client.1.smithi169.stdout:3/157: rmdir d7/de/d14/d18/d21/d32 0 2024-03-04T13:42:43.644 INFO:tasks.workunit.client.1.smithi169.stdout:4/149: dwrite d4/d1c/f2c [0,4194304] 0 2024-03-04T13:42:43.645 INFO:tasks.workunit.client.1.smithi169.stdout:3/158: readlink d7/ld 0 2024-03-04T13:42:43.647 INFO:tasks.workunit.client.1.smithi169.stdout:4/150: fsync d4/d11/f1b 0 2024-03-04T13:42:43.647 INFO:tasks.workunit.client.1.smithi169.stdout:3/159: read f4 [404919,90834] 0 2024-03-04T13:42:43.649 INFO:tasks.workunit.client.1.smithi169.stdout:7/259: dread d2/de/f1e [0,4194304] 0 2024-03-04T13:42:43.653 INFO:tasks.workunit.client.1.smithi169.stdout:4/151: mkdir d4/d11/d2e/d37 0 2024-03-04T13:42:43.655 INFO:tasks.workunit.client.1.smithi169.stdout:3/160: creat d7/de/d14/d13/d1a/d2d/f37 x:0 0 0 2024-03-04T13:42:43.657 INFO:tasks.workunit.client.1.smithi169.stdout:3/161: chown d7/de/d2b 1616 1 2024-03-04T13:42:43.662 INFO:tasks.workunit.client.1.smithi169.stdout:0/167: dread d5/d9/dc/d1c/f23 [0,4194304] 0 2024-03-04T13:42:43.664 INFO:tasks.workunit.client.1.smithi169.stdout:4/152: unlink d4/f14 0 2024-03-04T13:42:43.665 INFO:tasks.workunit.client.1.smithi169.stdout:5/236: dwrite d1/d16/d17/d28/f2c [4194304,4194304] 0 2024-03-04T13:42:43.671 INFO:tasks.workunit.client.1.smithi169.stdout:7/260: dread d2/de/d3e/d41/f4c [0,4194304] 0 2024-03-04T13:42:43.672 INFO:tasks.workunit.client.1.smithi169.stdout:9/214: sync 2024-03-04T13:42:43.672 INFO:tasks.workunit.client.1.smithi169.stdout:7/261: chown d2/f34 89546216 1 2024-03-04T13:42:43.674 INFO:tasks.workunit.client.1.smithi169.stdout:1/187: sync 2024-03-04T13:42:43.678 INFO:tasks.workunit.client.1.smithi169.stdout:3/162: mknod d7/c38 0 2024-03-04T13:42:43.681 INFO:tasks.workunit.client.1.smithi169.stdout:5/237: rename d1/d16/d17/d28/f3f to d1/d23/f40 0 2024-03-04T13:42:43.683 INFO:tasks.workunit.client.1.smithi169.stdout:5/238: read d1/d16/d17/d1b/f34 [368416,11371] 0 2024-03-04T13:42:43.693 INFO:tasks.workunit.client.1.smithi169.stdout:1/188: symlink d4/d5/d36/d35/l3d 0 2024-03-04T13:42:43.695 INFO:tasks.workunit.client.1.smithi169.stdout:5/239: creat d1/d16/d17/d28/f41 x:0 0 0 2024-03-04T13:42:43.698 INFO:tasks.workunit.client.1.smithi169.stdout:1/189: write d4/d5/d36/d35/df/f2b [2213383,57654] 0 2024-03-04T13:42:43.698 INFO:tasks.workunit.client.1.smithi169.stdout:9/215: dwrite d7/dc/d12/d20/f24 [0,4194304] 0 2024-03-04T13:42:43.698 INFO:tasks.workunit.client.1.smithi169.stdout:3/163: mkdir d7/de/d2b/d39 0 2024-03-04T13:42:43.699 INFO:tasks.workunit.client.1.smithi169.stdout:5/240: write d1/d16/d2f/f33 [858466,101750] 0 2024-03-04T13:42:43.700 INFO:tasks.workunit.client.1.smithi169.stdout:8/182: truncate d7/d10/d17/f2a 1698062 0 2024-03-04T13:42:43.715 INFO:tasks.workunit.client.1.smithi169.stdout:0/168: write d5/d9/dc/d1c/f21 [3621522,6368] 0 2024-03-04T13:42:43.717 INFO:tasks.workunit.client.1.smithi169.stdout:8/183: dread d7/d10/f1e [4194304,4194304] 0 2024-03-04T13:42:43.718 INFO:tasks.workunit.client.1.smithi169.stdout:7/262: link d2/de/l27 d2/de/d3e/d41/d5b/l5c 0 2024-03-04T13:42:43.724 INFO:tasks.workunit.client.1.smithi169.stdout:1/190: dwrite d4/d5/d36/d35/df/f28 [4194304,4194304] 0 2024-03-04T13:42:43.729 INFO:tasks.workunit.client.1.smithi169.stdout:0/169: stat d5/l2b 0 2024-03-04T13:42:43.730 INFO:tasks.workunit.client.1.smithi169.stdout:3/164: creat d7/de/d14/d18/d21/d2c/f3a x:0 0 0 2024-03-04T13:42:43.731 INFO:tasks.workunit.client.1.smithi169.stdout:0/170: dread - d5/d9/dc/d1c/f36 zero size 2024-03-04T13:42:43.731 INFO:tasks.workunit.client.1.smithi169.stdout:7/263: chown d2/de/f39 1 1 2024-03-04T13:42:43.743 INFO:tasks.workunit.client.1.smithi169.stdout:1/191: symlink d4/d13/l3e 0 2024-03-04T13:42:43.743 INFO:tasks.workunit.client.1.smithi169.stdout:4/153: mkdir d4/d11/d38 0 2024-03-04T13:42:43.745 INFO:tasks.workunit.client.1.smithi169.stdout:5/241: creat d1/f42 x:0 0 0 2024-03-04T13:42:43.747 INFO:tasks.workunit.client.1.smithi169.stdout:9/216: truncate d7/dc/f1e 288514 0 2024-03-04T13:42:43.749 INFO:tasks.workunit.client.1.smithi169.stdout:9/217: write d7/f33 [795293,66067] 0 2024-03-04T13:42:43.752 INFO:tasks.workunit.client.1.smithi169.stdout:7/264: write d2/de/f4a [1391125,79018] 0 2024-03-04T13:42:43.753 INFO:tasks.workunit.client.1.smithi169.stdout:9/218: truncate d7/d25/f3d 449189 0 2024-03-04T13:42:43.755 INFO:tasks.workunit.client.1.smithi169.stdout:9/219: stat d7/d25/d3c 0 2024-03-04T13:42:43.756 INFO:tasks.workunit.client.1.smithi169.stdout:0/171: creat d5/d9/dc/d12/d15/f40 x:0 0 0 2024-03-04T13:42:43.757 INFO:tasks.workunit.client.1.smithi169.stdout:2/206: sync 2024-03-04T13:42:43.765 INFO:tasks.workunit.client.1.smithi169.stdout:1/192: dwrite d4/f1b [0,4194304] 0 2024-03-04T13:42:43.769 INFO:tasks.workunit.client.1.smithi169.stdout:7/265: creat d2/d15/d17/f5d x:0 0 0 2024-03-04T13:42:43.777 INFO:tasks.workunit.client.1.smithi169.stdout:5/242: dwrite d1/d16/d17/d1b/f29 [0,4194304] 0 2024-03-04T13:42:43.779 INFO:tasks.workunit.client.1.smithi169.stdout:3/165: creat d7/f3b x:0 0 0 2024-03-04T13:42:43.780 INFO:tasks.workunit.client.1.smithi169.stdout:3/166: stat d7/f33 0 2024-03-04T13:42:43.790 INFO:tasks.workunit.client.1.smithi169.stdout:9/220: dread d7/dc/fd [0,4194304] 0 2024-03-04T13:42:43.792 INFO:tasks.workunit.client.1.smithi169.stdout:2/207: dwrite d1/d5/d19/d1e/f24 [0,4194304] 0 2024-03-04T13:42:43.795 INFO:tasks.workunit.client.1.smithi169.stdout:7/266: dread d2/d28/f33 [0,4194304] 0 2024-03-04T13:42:43.800 INFO:tasks.workunit.client.1.smithi169.stdout:5/243: dread d1/d16/d17/d1b/f1e [0,4194304] 0 2024-03-04T13:42:43.809 INFO:tasks.workunit.client.1.smithi169.stdout:3/167: dwrite d7/f3b [0,4194304] 0 2024-03-04T13:42:43.813 INFO:tasks.workunit.client.1.smithi169.stdout:2/208: dread d1/d5/d22/f32 [0,4194304] 0 2024-03-04T13:42:43.816 INFO:tasks.workunit.client.1.smithi169.stdout:2/209: readlink d1/d5/d19/d1e/d29/l2f 0 2024-03-04T13:42:43.828 INFO:tasks.workunit.client.1.smithi169.stdout:0/172: creat d5/d9/d3d/f41 x:0 0 0 2024-03-04T13:42:43.842 INFO:tasks.workunit.client.1.smithi169.stdout:2/210: dwrite d1/d5/d19/d1e/d29/f3e [0,4194304] 0 2024-03-04T13:42:43.842 INFO:tasks.workunit.client.1.smithi169.stdout:6/171: sync 2024-03-04T13:42:43.842 INFO:tasks.workunit.client.1.smithi169.stdout:4/154: rmdir d4/d11/d38 0 2024-03-04T13:42:43.847 INFO:tasks.workunit.client.1.smithi169.stdout:1/193: creat d4/d5/d36/d35/d16/d3a/f3f x:0 0 0 2024-03-04T13:42:43.864 INFO:tasks.workunit.client.1.smithi169.stdout:9/221: write d7/dc/fd [12977654,125835] 0 2024-03-04T13:42:43.864 INFO:tasks.workunit.client.1.smithi169.stdout:8/184: truncate d7/ff 2328473 0 2024-03-04T13:42:43.867 INFO:tasks.workunit.client.1.smithi169.stdout:7/267: write d2/fd [3991005,101911] 0 2024-03-04T13:42:43.876 INFO:tasks.workunit.client.1.smithi169.stdout:5/244: symlink d1/d16/d17/d28/l43 0 2024-03-04T13:42:43.878 INFO:tasks.workunit.client.1.smithi169.stdout:5/245: chown d1/d16/d2f/c35 295373 1 2024-03-04T13:42:43.879 INFO:tasks.workunit.client.1.smithi169.stdout:3/168: creat d7/de/d24/f3c x:0 0 0 2024-03-04T13:42:43.884 INFO:tasks.workunit.client.1.smithi169.stdout:0/173: creat d5/d9/f42 x:0 0 0 2024-03-04T13:42:43.887 INFO:tasks.workunit.client.1.smithi169.stdout:7/268: dread d2/de/f1e [0,4194304] 0 2024-03-04T13:42:43.897 INFO:tasks.workunit.client.1.smithi169.stdout:4/155: dread d4/f15 [4194304,4194304] 0 2024-03-04T13:42:43.911 INFO:tasks.workunit.client.1.smithi169.stdout:8/185: write d7/d10/d17/f27 [4949531,2002] 0 2024-03-04T13:42:43.925 INFO:tasks.workunit.client.1.smithi169.stdout:7/269: truncate d2/f1f 1943248 0 2024-03-04T13:42:43.926 INFO:tasks.workunit.client.1.smithi169.stdout:7/270: fsync d2/de/ff 0 2024-03-04T13:42:43.932 INFO:tasks.workunit.client.1.smithi169.stdout:4/156: mkdir d4/d11/d2e/d39 0 2024-03-04T13:42:43.936 INFO:tasks.workunit.client.1.smithi169.stdout:1/194: link d4/d5/d36/d35/df/f1c d4/d1d/f40 0 2024-03-04T13:42:43.939 INFO:tasks.workunit.client.1.smithi169.stdout:9/222: creat d7/dc/d12/d32/d42/f46 x:0 0 0 2024-03-04T13:42:43.942 INFO:tasks.workunit.client.1.smithi169.stdout:8/186: fsync d7/d10/f1e 0 2024-03-04T13:42:43.952 INFO:tasks.workunit.client.1.smithi169.stdout:9/223: dread d7/dc/d12/d20/f24 [0,4194304] 0 2024-03-04T13:42:43.957 INFO:tasks.workunit.client.1.smithi169.stdout:7/271: creat d2/d15/d54/f5e x:0 0 0 2024-03-04T13:42:43.960 INFO:tasks.workunit.client.1.smithi169.stdout:4/157: chown d4/l22 0 1 2024-03-04T13:42:43.962 INFO:tasks.workunit.client.1.smithi169.stdout:2/211: creat d1/d5/d22/d45/d4a/f53 x:0 0 0 2024-03-04T13:42:43.968 INFO:tasks.workunit.client.1.smithi169.stdout:1/195: rmdir d4/d5/d36/d35/d16 39 2024-03-04T13:42:43.973 INFO:tasks.workunit.client.1.smithi169.stdout:8/187: mknod d7/d10/d24/c3f 0 2024-03-04T13:42:43.978 INFO:tasks.workunit.client.1.smithi169.stdout:6/172: getdents d0 0 2024-03-04T13:42:43.979 INFO:tasks.workunit.client.1.smithi169.stdout:9/224: mkdir d7/d25/d47 0 2024-03-04T13:42:43.987 INFO:tasks.workunit.client.1.smithi169.stdout:2/212: rename d1/ff to d1/d5/d19/d1e/f54 0 2024-03-04T13:42:43.987 INFO:tasks.workunit.client.1.smithi169.stdout:1/196: read - d4/d5/d36/d35/d16/d3a/f3f zero size 2024-03-04T13:42:43.988 INFO:tasks.workunit.client.1.smithi169.stdout:2/213: write d1/d5/d19/f3f [1691331,54572] 0 2024-03-04T13:42:43.990 INFO:tasks.workunit.client.1.smithi169.stdout:5/246: getdents d1 0 2024-03-04T13:42:43.991 INFO:tasks.workunit.client.1.smithi169.stdout:8/188: unlink d7/d10/d17/f2e 0 2024-03-04T13:42:43.994 INFO:tasks.workunit.client.1.smithi169.stdout:8/189: readlink d7/d10/d24/l28 0 2024-03-04T13:42:43.995 INFO:tasks.workunit.client.1.smithi169.stdout:3/169: getdents d7/de/d14 0 2024-03-04T13:42:43.997 INFO:tasks.workunit.client.1.smithi169.stdout:0/174: getdents d5/d9/dc/d12/d15/d19 0 2024-03-04T13:42:43.999 INFO:tasks.workunit.client.1.smithi169.stdout:0/175: fdatasync d5/d9/dc/d1c/f2c 0 2024-03-04T13:42:44.005 INFO:tasks.workunit.client.1.smithi169.stdout:4/158: dwrite d4/d1c/f2c [0,4194304] 0 2024-03-04T13:42:44.005 INFO:tasks.workunit.client.1.smithi169.stdout:6/173: mkdir d0/d3c 0 2024-03-04T13:42:44.012 INFO:tasks.workunit.client.1.smithi169.stdout:1/197: symlink d4/d5/d2d/l41 0 2024-03-04T13:42:44.016 INFO:tasks.workunit.client.1.smithi169.stdout:5/247: creat d1/d16/d17/d1b/f44 x:0 0 0 2024-03-04T13:42:44.018 INFO:tasks.workunit.client.1.smithi169.stdout:8/190: creat d7/d10/d24/d3e/f40 x:0 0 0 2024-03-04T13:42:44.019 INFO:tasks.workunit.client.1.smithi169.stdout:3/170: mknod d7/de/d14/d13/d1a/c3d 0 2024-03-04T13:42:44.023 INFO:tasks.workunit.client.1.smithi169.stdout:3/171: dread - d7/de/d14/d18/f27 zero size 2024-03-04T13:42:44.028 INFO:tasks.workunit.client.1.smithi169.stdout:1/198: dread d4/f1b [0,4194304] 0 2024-03-04T13:42:44.031 INFO:tasks.workunit.client.1.smithi169.stdout:9/225: mkdir d7/d2d/d48 0 2024-03-04T13:42:44.034 INFO:tasks.workunit.client.1.smithi169.stdout:1/199: truncate d4/d1d/f34 208133 0 2024-03-04T13:42:44.036 INFO:tasks.workunit.client.1.smithi169.stdout:9/226: chown d7/d25/c2c 5466683 1 2024-03-04T13:42:44.040 INFO:tasks.workunit.client.1.smithi169.stdout:1/200: dread - d4/d5/d36/d35/d16/d3a/f3f zero size 2024-03-04T13:42:44.043 INFO:tasks.workunit.client.1.smithi169.stdout:3/172: dread d7/f3b [0,4194304] 0 2024-03-04T13:42:44.043 INFO:tasks.workunit.client.1.smithi169.stdout:5/248: dwrite d1/d16/f3a [0,4194304] 0 2024-03-04T13:42:44.046 INFO:tasks.workunit.client.1.smithi169.stdout:8/191: chown d7/d10/d14/c38 8 1 2024-03-04T13:42:44.047 INFO:tasks.workunit.client.1.smithi169.stdout:3/173: write d7/de/d14/f2a [108096,44184] 0 2024-03-04T13:42:44.049 INFO:tasks.workunit.client.1.smithi169.stdout:3/174: stat d7/de/d14/d13/d1a/c3d 0 2024-03-04T13:42:44.050 INFO:tasks.workunit.client.1.smithi169.stdout:4/159: write d4/f7 [3616626,9395] 0 2024-03-04T13:42:44.053 INFO:tasks.workunit.client.1.smithi169.stdout:3/175: write d7/de/d14/d18/d21/f31 [4984189,51301] 0 2024-03-04T13:42:44.055 INFO:tasks.workunit.client.1.smithi169.stdout:3/176: read d7/fb [1641990,62468] 0 2024-03-04T13:42:44.056 INFO:tasks.workunit.client.1.smithi169.stdout:7/272: write d2/f1f [533399,28382] 0 2024-03-04T13:42:44.058 INFO:tasks.workunit.client.1.smithi169.stdout:1/201: symlink d4/d5/d36/d35/d16/d3a/l42 0 2024-03-04T13:42:44.059 INFO:tasks.workunit.client.1.smithi169.stdout:2/214: creat d1/d5/d22/f55 x:0 0 0 2024-03-04T13:42:44.061 INFO:tasks.workunit.client.1.smithi169.stdout:7/273: chown d2/f1f 833775048 1 2024-03-04T13:42:44.063 INFO:tasks.workunit.client.1.smithi169.stdout:8/192: rmdir d7/d10/d24 39 2024-03-04T13:42:44.069 INFO:tasks.workunit.client.1.smithi169.stdout:9/227: mkdir d7/d49 0 2024-03-04T13:42:44.073 INFO:tasks.workunit.client.1.smithi169.stdout:5/249: symlink d1/d16/d2f/d39/l45 0 2024-03-04T13:42:44.073 INFO:tasks.workunit.client.1.smithi169.stdout:6/174: chown d0/f30 357617 1 2024-03-04T13:42:44.074 INFO:tasks.workunit.client.1.smithi169.stdout:5/250: read d1/f14 [6603785,6414] 0 2024-03-04T13:42:44.076 INFO:tasks.workunit.client.1.smithi169.stdout:8/193: dread d7/d10/d17/f20 [0,4194304] 0 2024-03-04T13:42:44.077 INFO:tasks.workunit.client.1.smithi169.stdout:5/251: readlink d1/d16/d2a/l32 0 2024-03-04T13:42:44.082 INFO:tasks.workunit.client.1.smithi169.stdout:6/175: dread - d0/f3b zero size 2024-03-04T13:42:44.084 INFO:tasks.workunit.client.1.smithi169.stdout:5/252: fdatasync d1/d16/d17/d1b/f34 0 2024-03-04T13:42:44.089 INFO:tasks.workunit.client.1.smithi169.stdout:5/253: mkdir d1/d16/d2a/d46 0 2024-03-04T13:42:44.092 INFO:tasks.workunit.client.1.smithi169.stdout:5/254: fdatasync d1/d16/d17/d1b/f44 0 2024-03-04T13:42:44.092 INFO:tasks.workunit.client.1.smithi169.stdout:5/255: chown d1/d16 953 1 2024-03-04T13:42:44.094 INFO:tasks.workunit.client.1.smithi169.stdout:8/194: rename d7/d10/d24/c3f to d7/c41 0 2024-03-04T13:42:44.094 INFO:tasks.workunit.client.1.smithi169.stdout:1/202: truncate d4/d5/d36/d35/df/f28 5380973 0 2024-03-04T13:42:44.098 INFO:tasks.workunit.client.1.smithi169.stdout:6/176: creat d0/f3d x:0 0 0 2024-03-04T13:42:44.098 INFO:tasks.workunit.client.1.smithi169.stdout:5/256: mknod d1/d23/c47 0 2024-03-04T13:42:44.098 INFO:tasks.workunit.client.1.smithi169.stdout:1/203: write d4/d5/f38 [152405,87943] 0 2024-03-04T13:42:44.099 INFO:tasks.workunit.client.1.smithi169.stdout:1/204: chown d4 645 1 2024-03-04T13:42:44.105 INFO:tasks.workunit.client.1.smithi169.stdout:6/177: mkdir d0/d32/d3e 0 2024-03-04T13:42:44.107 INFO:tasks.workunit.client.1.smithi169.stdout:3/177: dwrite d7/f1d [0,4194304] 0 2024-03-04T13:42:44.109 INFO:tasks.workunit.client.1.smithi169.stdout:5/257: symlink d1/d16/d17/l48 0 2024-03-04T13:42:44.109 INFO:tasks.workunit.client.1.smithi169.stdout:8/195: link d7/d10/d17/l2f d7/l42 0 2024-03-04T13:42:44.109 INFO:tasks.workunit.client.1.smithi169.stdout:9/228: dread d7/dc/d12/d32/d42/f43 [0,4194304] 0 2024-03-04T13:42:44.113 INFO:tasks.workunit.client.1.smithi169.stdout:3/178: chown d7/de/l29 0 1 2024-03-04T13:42:44.116 INFO:tasks.workunit.client.1.smithi169.stdout:5/258: mknod d1/d16/d2f/d39/c49 0 2024-03-04T13:42:44.125 INFO:tasks.workunit.client.1.smithi169.stdout:5/259: chown d1/c21 1684 1 2024-03-04T13:42:44.125 INFO:tasks.workunit.client.1.smithi169.stdout:1/205: dwrite d4/d5/f38 [0,4194304] 0 2024-03-04T13:42:44.126 INFO:tasks.workunit.client.1.smithi169.stdout:6/178: dread d0/f15 [0,4194304] 0 2024-03-04T13:42:44.138 INFO:tasks.workunit.client.1.smithi169.stdout:9/229: dwrite d7/d2d/f40 [0,4194304] 0 2024-03-04T13:42:44.148 INFO:tasks.workunit.client.1.smithi169.stdout:6/179: dread d0/f28 [0,4194304] 0 2024-03-04T13:42:44.151 INFO:tasks.workunit.client.1.smithi169.stdout:1/206: rename d4/d5/d36/d35/d16/d3a/l42 to d4/d5/l43 0 2024-03-04T13:42:44.153 INFO:tasks.workunit.client.1.smithi169.stdout:9/230: symlink d7/l4a 0 2024-03-04T13:42:44.155 INFO:tasks.workunit.client.1.smithi169.stdout:1/207: write d4/f20 [4766549,47864] 0 2024-03-04T13:42:44.158 INFO:tasks.workunit.client.1.smithi169.stdout:6/180: creat d0/d32/d3a/f3f x:0 0 0 2024-03-04T13:42:44.159 INFO:tasks.workunit.client.1.smithi169.stdout:3/179: dread d7/f9 [0,4194304] 0 2024-03-04T13:42:44.161 INFO:tasks.workunit.client.1.smithi169.stdout:1/208: symlink d4/d5/d36/d35/d16/d3a/l44 0 2024-03-04T13:42:44.164 INFO:tasks.workunit.client.1.smithi169.stdout:6/181: mkdir d0/d3c/d40 0 2024-03-04T13:42:44.170 INFO:tasks.workunit.client.1.smithi169.stdout:3/180: truncate d7/f36 81046 0 2024-03-04T13:42:44.171 INFO:tasks.workunit.client.1.smithi169.stdout:6/182: truncate d0/d32/d3a/f3f 684719 0 2024-03-04T13:42:44.171 INFO:tasks.workunit.client.1.smithi169.stdout:8/196: dwrite d7/d10/d17/f2a [0,4194304] 0 2024-03-04T13:42:44.175 INFO:tasks.workunit.client.1.smithi169.stdout:5/260: dwrite d1/d16/d17/f18 [0,4194304] 0 2024-03-04T13:42:44.177 INFO:tasks.workunit.client.1.smithi169.stdout:3/181: write d7/de/d14/d18/f27 [557880,84050] 0 2024-03-04T13:42:44.183 INFO:tasks.workunit.client.1.smithi169.stdout:1/209: dread d4/d1d/f40 [0,4194304] 0 2024-03-04T13:42:44.183 INFO:tasks.workunit.client.1.smithi169.stdout:8/197: rename d7/d10/d24/d3e/c1f to d7/d10/d24/d3e/c43 0 2024-03-04T13:42:44.186 INFO:tasks.workunit.client.1.smithi169.stdout:8/198: dread - d7/d12/f3a zero size 2024-03-04T13:42:44.191 INFO:tasks.workunit.client.1.smithi169.stdout:5/261: creat d1/f4a x:0 0 0 2024-03-04T13:42:44.192 INFO:tasks.workunit.client.1.smithi169.stdout:5/262: fdatasync d1/d16/d2a/f3e 0 2024-03-04T13:42:44.194 INFO:tasks.workunit.client.1.smithi169.stdout:9/231: dwrite d7/dc/d12/d32/d42/f46 [0,4194304] 0 2024-03-04T13:42:44.196 INFO:tasks.workunit.client.1.smithi169.stdout:5/263: stat d1/d16/d17/d1b/f29 0 2024-03-04T13:42:44.198 INFO:tasks.workunit.client.1.smithi169.stdout:9/232: write d7/d2d/f36 [306751,98290] 0 2024-03-04T13:42:44.204 INFO:tasks.workunit.client.1.smithi169.stdout:9/233: write d7/d2d/f40 [4091297,90137] 0 2024-03-04T13:42:44.209 INFO:tasks.workunit.client.1.smithi169.stdout:1/210: dread d4/d5/f38 [0,4194304] 0 2024-03-04T13:42:44.213 INFO:tasks.workunit.client.1.smithi169.stdout:1/211: rmdir d4/d5/d36 39 2024-03-04T13:42:44.219 INFO:tasks.workunit.client.1.smithi169.stdout:6/183: truncate d0/f37 1573461 0 2024-03-04T13:42:44.220 INFO:tasks.workunit.client.1.smithi169.stdout:1/212: mknod d4/d5/d36/d35/df/c45 0 2024-03-04T13:42:44.220 INFO:tasks.workunit.client.1.smithi169.stdout:3/182: dread d7/de/d14/d18/f27 [0,4194304] 0 2024-03-04T13:42:44.225 INFO:tasks.workunit.client.1.smithi169.stdout:1/213: readlink d4/d5/l43 0 2024-03-04T13:42:44.226 INFO:tasks.workunit.client.1.smithi169.stdout:9/234: dread d7/d2d/f36 [0,4194304] 0 2024-03-04T13:42:44.227 INFO:tasks.workunit.client.1.smithi169.stdout:5/264: write d1/f24 [2772662,81598] 0 2024-03-04T13:42:44.229 INFO:tasks.workunit.client.1.smithi169.stdout:3/183: mknod d7/de/d14/d13/d1a/d2d/c3e 0 2024-03-04T13:42:44.231 INFO:tasks.workunit.client.1.smithi169.stdout:5/265: chown d1/d16/d17/l48 15210103 1 2024-03-04T13:42:44.232 INFO:tasks.workunit.client.1.smithi169.stdout:3/184: fsync d7/de/d14/d18/d21/d2c/f3a 0 2024-03-04T13:42:44.232 INFO:tasks.workunit.client.1.smithi169.stdout:5/266: stat d1/lc 0 2024-03-04T13:42:44.235 INFO:tasks.workunit.client.1.smithi169.stdout:3/185: dread d7/f36 [0,4194304] 0 2024-03-04T13:42:44.236 INFO:tasks.workunit.client.1.smithi169.stdout:1/214: creat d4/d5/d36/d35/df/f46 x:0 0 0 2024-03-04T13:42:44.240 INFO:tasks.workunit.client.1.smithi169.stdout:8/199: dread d7/d10/d24/f3c [0,4194304] 0 2024-03-04T13:42:44.242 INFO:tasks.workunit.client.1.smithi169.stdout:1/215: truncate d4/d5/d36/d35/df/f46 711646 0 2024-03-04T13:42:44.243 INFO:tasks.workunit.client.1.smithi169.stdout:9/235: link d7/dc/d12/d20/c2a d7/d25/c4b 0 2024-03-04T13:42:44.247 INFO:tasks.workunit.client.1.smithi169.stdout:1/216: readlink d4/d5/d36/d35/d16/d3a/l44 0 2024-03-04T13:42:44.257 INFO:tasks.workunit.client.1.smithi169.stdout:5/267: dwrite d1/f8 [0,4194304] 0 2024-03-04T13:42:44.260 INFO:tasks.workunit.client.1.smithi169.stdout:9/236: mkdir d7/dc/d3e/d4c 0 2024-03-04T13:42:44.263 INFO:tasks.workunit.client.1.smithi169.stdout:3/186: getdents d7/de/d2b 0 2024-03-04T13:42:44.266 INFO:tasks.workunit.client.1.smithi169.stdout:3/187: truncate d7/fb 4981495 0 2024-03-04T13:42:44.269 INFO:tasks.workunit.client.1.smithi169.stdout:9/237: symlink d7/dc/d12/d32/d42/l4d 0 2024-03-04T13:42:44.278 INFO:tasks.workunit.client.1.smithi169.stdout:3/188: mkdir d7/de/d14/d18/d21/d3f 0 2024-03-04T13:42:44.280 INFO:tasks.workunit.client.1.smithi169.stdout:6/184: dread d0/f12 [0,4194304] 0 2024-03-04T13:42:44.283 INFO:tasks.workunit.client.1.smithi169.stdout:5/268: creat d1/d16/d17/f4b x:0 0 0 2024-03-04T13:42:44.291 INFO:tasks.workunit.client.1.smithi169.stdout:1/217: rename d4/d5/d36/d35/df/c29 to d4/c47 0 2024-03-04T13:42:44.292 INFO:tasks.workunit.client.1.smithi169.stdout:5/269: creat d1/d16/d17/d28/f4c x:0 0 0 2024-03-04T13:42:44.294 INFO:tasks.workunit.client.1.smithi169.stdout:8/200: dwrite d7/d10/d17/f31 [0,4194304] 0 2024-03-04T13:42:44.295 INFO:tasks.workunit.client.1.smithi169.stdout:6/185: symlink d0/d3c/d40/l41 0 2024-03-04T13:42:44.297 INFO:tasks.workunit.client.1.smithi169.stdout:1/218: write d4/d5/d36/d35/d16/d3a/f3f [380083,116453] 0 2024-03-04T13:42:44.303 INFO:tasks.workunit.client.1.smithi169.stdout:8/201: fsync d7/ff 0 2024-03-04T13:42:44.307 INFO:tasks.workunit.client.1.smithi169.stdout:5/270: link d1/d16/d17/d28/f2c d1/d16/d17/d28/f4d 0 2024-03-04T13:42:44.308 INFO:tasks.workunit.client.1.smithi169.stdout:9/238: dread d7/fe [0,4194304] 0 2024-03-04T13:42:44.309 INFO:tasks.workunit.client.1.smithi169.stdout:3/189: dread f4 [0,4194304] 0 2024-03-04T13:42:44.310 INFO:tasks.workunit.client.1.smithi169.stdout:9/239: chown d7/d2d 11334335 1 2024-03-04T13:42:44.311 INFO:tasks.workunit.client.1.smithi169.stdout:3/190: stat d7/fb 0 2024-03-04T13:42:44.314 INFO:tasks.workunit.client.1.smithi169.stdout:3/191: chown d7/c38 3 1 2024-03-04T13:42:44.315 INFO:tasks.workunit.client.1.smithi169.stdout:9/240: dread d7/d25/f3d [0,4194304] 0 2024-03-04T13:42:44.316 INFO:tasks.workunit.client.1.smithi169.stdout:5/271: creat d1/d16/d2a/d46/f4e x:0 0 0 2024-03-04T13:42:44.316 INFO:tasks.workunit.client.1.smithi169.stdout:3/192: readlink d7/de/l17 0 2024-03-04T13:42:44.318 INFO:tasks.workunit.client.1.smithi169.stdout:9/241: readlink d7/dc/l28 0 2024-03-04T13:42:44.320 INFO:tasks.workunit.client.1.smithi169.stdout:8/202: rename d7/d10/d14 to d7/d10/d17/d44 0 2024-03-04T13:42:44.320 INFO:tasks.workunit.client.1.smithi169.stdout:3/193: dread - d7/de/d14/d18/f30 zero size 2024-03-04T13:42:44.322 INFO:tasks.workunit.client.1.smithi169.stdout:8/203: chown d7/d10/d17/f31 320864 1 2024-03-04T13:42:44.324 INFO:tasks.workunit.client.1.smithi169.stdout:6/186: dwrite d0/f38 [0,4194304] 0 2024-03-04T13:42:44.325 INFO:tasks.workunit.client.1.smithi169.stdout:3/194: read - d7/de/d14/d13/d1a/d2d/f37 zero size 2024-03-04T13:42:44.328 INFO:tasks.workunit.client.1.smithi169.stdout:3/195: truncate d7/f33 153870 0 2024-03-04T13:42:44.332 INFO:tasks.workunit.client.1.smithi169.stdout:9/242: write d7/fe [2349261,1166] 0 2024-03-04T13:42:44.333 INFO:tasks.workunit.client.1.smithi169.stdout:3/196: write d7/de/d14/d18/f30 [965165,6607] 0 2024-03-04T13:42:44.339 INFO:tasks.workunit.client.1.smithi169.stdout:6/187: chown d0/f16 14 1 2024-03-04T13:42:44.341 INFO:tasks.workunit.client.1.smithi169.stdout:6/188: chown d0/d32/d3a 0 1 2024-03-04T13:42:44.342 INFO:tasks.workunit.client.1.smithi169.stdout:9/243: rename d7/dc/d3e/d4c to d7/dc/d12/d20/d4e 0 2024-03-04T13:42:44.344 INFO:tasks.workunit.client.1.smithi169.stdout:9/244: chown d7/dc/d12/d20 26 1 2024-03-04T13:42:44.350 INFO:tasks.workunit.client.1.smithi169.stdout:9/245: dread d7/d27/f2f [0,4194304] 0 2024-03-04T13:42:44.351 INFO:tasks.workunit.client.1.smithi169.stdout:1/219: dwrite d4/f1b [0,4194304] 0 2024-03-04T13:42:44.355 INFO:tasks.workunit.client.1.smithi169.stdout:5/272: write d1/f14 [34020,103003] 0 2024-03-04T13:42:44.358 INFO:tasks.workunit.client.1.smithi169.stdout:5/273: stat d1/d16/d17/d28 0 2024-03-04T13:42:44.360 INFO:tasks.workunit.client.1.smithi169.stdout:5/274: truncate d1/d23/f40 913506 0 2024-03-04T13:42:44.361 INFO:tasks.workunit.client.1.smithi169.stdout:8/204: truncate d7/d10/d24/f3c 2063414 0 2024-03-04T13:42:44.364 INFO:tasks.workunit.client.1.smithi169.stdout:8/205: write d7/d10/d24/f2c [3277095,70279] 0 2024-03-04T13:42:44.369 INFO:tasks.workunit.client.1.smithi169.stdout:5/275: link d1/d16/d17/f1a d1/d16/d2f/d38/f4f 0 2024-03-04T13:42:44.374 INFO:tasks.workunit.client.1.smithi169.stdout:6/189: dwrite d0/f29 [0,4194304] 0 2024-03-04T13:42:44.374 INFO:tasks.workunit.client.1.smithi169.stdout:9/246: truncate d7/d27/f2b 5577130 0 2024-03-04T13:42:44.378 INFO:tasks.workunit.client.1.smithi169.stdout:5/276: symlink d1/d16/d17/d28/l50 0 2024-03-04T13:42:44.379 INFO:tasks.workunit.client.1.smithi169.stdout:6/190: creat d0/d32/d3a/f42 x:0 0 0 2024-03-04T13:42:44.386 INFO:tasks.workunit.client.1.smithi169.stdout:5/277: truncate d1/d16/d17/d1b/f34 998871 0 2024-03-04T13:42:44.389 INFO:tasks.workunit.client.1.smithi169.stdout:5/278: write d1/d16/d2a/f3e [268212,20291] 0 2024-03-04T13:42:44.395 INFO:tasks.workunit.client.1.smithi169.stdout:9/247: getdents d7/d27 0 2024-03-04T13:42:44.399 INFO:tasks.workunit.client.1.smithi169.stdout:9/248: mkdir d7/dc/d12/d32/d42/d4f 0 2024-03-04T13:42:44.405 INFO:tasks.workunit.client.1.smithi169.stdout:6/191: dwrite d0/f9 [0,4194304] 0 2024-03-04T13:42:44.409 INFO:tasks.workunit.client.1.smithi169.stdout:8/206: dread d7/ff [0,4194304] 0 2024-03-04T13:42:44.410 INFO:tasks.workunit.client.1.smithi169.stdout:6/192: mkdir d0/d32/d3a/d43 0 2024-03-04T13:42:44.413 INFO:tasks.workunit.client.1.smithi169.stdout:6/193: write d0/f28 [294578,30824] 0 2024-03-04T13:42:44.415 INFO:tasks.workunit.client.1.smithi169.stdout:9/249: dwrite d7/f11 [0,4194304] 0 2024-03-04T13:42:44.415 INFO:tasks.workunit.client.1.smithi169.stdout:8/207: unlink d7/d10/f3b 0 2024-03-04T13:42:44.420 INFO:tasks.workunit.client.1.smithi169.stdout:6/194: mknod d0/d3c/d40/c44 0 2024-03-04T13:42:44.422 INFO:tasks.workunit.client.1.smithi169.stdout:8/208: fsync d7/ff 0 2024-03-04T13:42:44.425 INFO:tasks.workunit.client.1.smithi169.stdout:9/250: unlink d7/dc/d12/d32/f3b 0 2024-03-04T13:42:44.427 INFO:tasks.workunit.client.1.smithi169.stdout:9/251: readlink d7/dc/l44 0 2024-03-04T13:42:44.428 INFO:tasks.workunit.client.1.smithi169.stdout:6/195: write d0/f2b [902223,73352] 0 2024-03-04T13:42:44.430 INFO:tasks.workunit.client.1.smithi169.stdout:8/209: unlink d7/d10/d24/f2c 0 2024-03-04T13:42:44.433 INFO:tasks.workunit.client.1.smithi169.stdout:8/210: chown d7/d10/d22/f25 1 1 2024-03-04T13:42:44.442 INFO:tasks.workunit.client.1.smithi169.stdout:9/252: dread d7/dc/fd [4194304,4194304] 0 2024-03-04T13:42:44.442 INFO:tasks.workunit.client.1.smithi169.stdout:8/211: link d7/d10/d17/f31 d7/d10/d24/d3e/f45 0 2024-03-04T13:42:44.446 INFO:tasks.workunit.client.1.smithi169.stdout:8/212: mkdir d7/d10/d22/d46 0 2024-03-04T13:42:44.448 INFO:tasks.workunit.client.1.smithi169.stdout:9/253: getdents d7/d49 0 2024-03-04T13:42:44.451 INFO:tasks.workunit.client.1.smithi169.stdout:6/196: dwrite d0/f35 [0,4194304] 0 2024-03-04T13:42:44.455 INFO:tasks.workunit.client.1.smithi169.stdout:8/213: write d7/d10/d17/f29 [3457258,29720] 0 2024-03-04T13:42:44.461 INFO:tasks.workunit.client.1.smithi169.stdout:8/214: mknod d7/d12/c47 0 2024-03-04T13:42:44.463 INFO:tasks.workunit.client.1.smithi169.stdout:9/254: rename d7/d27/f2b to d7/dc/d12/d32/d42/d4f/f50 0 2024-03-04T13:42:44.465 INFO:tasks.workunit.client.1.smithi169.stdout:6/197: creat d0/d32/d3a/d43/f45 x:0 0 0 2024-03-04T13:42:44.468 INFO:tasks.workunit.client.1.smithi169.stdout:6/198: write d0/d32/d3a/d43/f45 [851967,102681] 0 2024-03-04T13:42:44.469 INFO:tasks.workunit.client.1.smithi169.stdout:9/255: mknod d7/d25/c51 0 2024-03-04T13:42:44.473 INFO:tasks.workunit.client.1.smithi169.stdout:8/215: rename d7/c41 to d7/d10/d24/d3e/c48 0 2024-03-04T13:42:44.476 INFO:tasks.workunit.client.1.smithi169.stdout:9/256: mknod d7/dc/d12/d20/d4e/c52 0 2024-03-04T13:42:44.477 INFO:tasks.workunit.client.1.smithi169.stdout:8/216: readlink d7/d12/l26 0 2024-03-04T13:42:44.478 INFO:tasks.workunit.client.1.smithi169.stdout:8/217: stat d7/d10 0 2024-03-04T13:42:44.483 INFO:tasks.workunit.client.1.smithi169.stdout:9/257: mknod d7/dc/d12/c53 0 2024-03-04T13:42:44.484 INFO:tasks.workunit.client.1.smithi169.stdout:9/258: stat d7/d2d/f40 0 2024-03-04T13:42:44.484 INFO:tasks.workunit.client.1.smithi169.stdout:8/218: unlink d7/d10/d17/c2b 0 2024-03-04T13:42:44.485 INFO:tasks.workunit.client.1.smithi169.stdout:6/199: dread d0/d32/d3a/d43/f45 [0,4194304] 0 2024-03-04T13:42:44.487 INFO:tasks.workunit.client.1.smithi169.stdout:9/259: mkdir d7/d27/d54 0 2024-03-04T13:42:44.488 INFO:tasks.workunit.client.1.smithi169.stdout:6/200: chown d0/f1b 148358426 1 2024-03-04T13:42:44.489 INFO:tasks.workunit.client.1.smithi169.stdout:8/219: mkdir d7/d10/d22/d46/d49 0 2024-03-04T13:42:44.492 INFO:tasks.workunit.client.1.smithi169.stdout:6/201: truncate d0/f15 1786563 0 2024-03-04T13:42:44.496 INFO:tasks.workunit.client.1.smithi169.stdout:8/220: rename d7/d12/f1c to d7/d10/d17/d44/d36/f4a 0 2024-03-04T13:42:44.497 INFO:tasks.workunit.client.1.smithi169.stdout:6/202: mknod d0/d3c/d40/c46 0 2024-03-04T13:42:44.503 INFO:tasks.workunit.client.1.smithi169.stdout:8/221: write d7/d10/d17/d44/d36/f4a [3517800,83587] 0 2024-03-04T13:42:44.507 INFO:tasks.workunit.client.1.smithi169.stdout:8/222: mknod d7/d10/d22/c4b 0 2024-03-04T13:42:44.508 INFO:tasks.workunit.client.1.smithi169.stdout:9/260: dwrite d7/dc/d12/d32/d42/f46 [4194304,4194304] 0 2024-03-04T13:42:44.508 INFO:tasks.workunit.client.1.smithi169.stdout:6/203: dread d0/d32/d3a/f21 [0,4194304] 0 2024-03-04T13:42:44.508 INFO:tasks.workunit.client.1.smithi169.stdout:8/223: readlink d7/ld 0 2024-03-04T13:42:44.518 INFO:tasks.workunit.client.1.smithi169.stdout:6/204: getdents d0/d32/d3a/d43 0 2024-03-04T13:42:44.521 INFO:tasks.workunit.client.1.smithi169.stdout:6/205: stat d0/f3b 0 2024-03-04T13:42:44.525 INFO:tasks.workunit.client.1.smithi169.stdout:6/206: truncate d0/d32/d3a/f42 704315 0 2024-03-04T13:42:44.526 INFO:tasks.workunit.client.1.smithi169.stdout:9/261: truncate f6 7694437 0 2024-03-04T13:42:44.528 INFO:tasks.workunit.client.1.smithi169.stdout:8/224: dwrite d7/d10/d17/f31 [0,4194304] 0 2024-03-04T13:42:44.530 INFO:tasks.workunit.client.1.smithi169.stdout:9/262: symlink d7/d27/l55 0 2024-03-04T13:42:44.537 INFO:tasks.workunit.client.1.smithi169.stdout:9/263: write d7/d27/f2f [459563,120752] 0 2024-03-04T13:42:44.541 INFO:tasks.workunit.client.1.smithi169.stdout:6/207: dread d0/f9 [0,4194304] 0 2024-03-04T13:42:44.549 INFO:tasks.workunit.client.1.smithi169.stdout:6/208: rename d0/d3c/d40/c46 to d0/d3c/d40/c47 0 2024-03-04T13:42:44.550 INFO:tasks.workunit.client.1.smithi169.stdout:9/264: dread d7/fe [0,4194304] 0 2024-03-04T13:42:44.552 INFO:tasks.workunit.client.1.smithi169.stdout:6/209: stat d0/d32/d3a/d43 0 2024-03-04T13:42:44.554 INFO:tasks.workunit.client.1.smithi169.stdout:9/265: mkdir d7/d25/d47/d56 0 2024-03-04T13:42:44.556 INFO:tasks.workunit.client.1.smithi169.stdout:8/225: dwrite d7/ff [0,4194304] 0 2024-03-04T13:42:44.557 INFO:tasks.workunit.client.1.smithi169.stdout:8/226: chown d7/d10/d17 87669710 1 2024-03-04T13:42:44.558 INFO:tasks.workunit.client.1.smithi169.stdout:6/210: mknod d0/d32/d3e/c48 0 2024-03-04T13:42:44.561 INFO:tasks.workunit.client.1.smithi169.stdout:8/227: write d7/d10/d22/f25 [80874,41868] 0 2024-03-04T13:42:44.562 INFO:tasks.workunit.client.1.smithi169.stdout:6/211: readlink d0/lc 0 2024-03-04T13:42:44.563 INFO:tasks.workunit.client.1.smithi169.stdout:6/212: read - d0/f16 zero size 2024-03-04T13:42:44.564 INFO:tasks.workunit.client.1.smithi169.stdout:8/228: fdatasync d7/d10/d17/d44/d36/f4a 0 2024-03-04T13:42:44.569 INFO:tasks.workunit.client.1.smithi169.stdout:6/213: symlink d0/d3c/d40/l49 0 2024-03-04T13:42:44.570 INFO:tasks.workunit.client.1.smithi169.stdout:9/266: write d7/dc/f1e [351122,116940] 0 2024-03-04T13:42:44.586 INFO:tasks.workunit.client.1.smithi169.stdout:8/229: dwrite d7/d10/d17/f20 [0,4194304] 0 2024-03-04T13:42:44.591 INFO:tasks.workunit.client.1.smithi169.stdout:8/230: mkdir d7/d10/d17/d44/d4c 0 2024-03-04T13:42:44.591 INFO:tasks.workunit.client.1.smithi169.stdout:6/214: dwrite d0/f9 [0,4194304] 0 2024-03-04T13:42:44.595 INFO:tasks.workunit.client.1.smithi169.stdout:6/215: write d0/f35 [1116929,45597] 0 2024-03-04T13:42:44.600 INFO:tasks.workunit.client.1.smithi169.stdout:9/267: dwrite d7/dc/d12/d32/d42/f46 [4194304,4194304] 0 2024-03-04T13:42:44.600 INFO:tasks.workunit.client.1.smithi169.stdout:6/216: chown d0/d32/d3a/l22 683 1 2024-03-04T13:42:44.601 INFO:tasks.workunit.client.1.smithi169.stdout:9/268: write d7/fb [1087616,107030] 0 2024-03-04T13:42:44.603 INFO:tasks.workunit.client.1.smithi169.stdout:6/217: stat d0/d3c/d40/c47 0 2024-03-04T13:42:44.604 INFO:tasks.workunit.client.1.smithi169.stdout:9/269: chown d7/dc/c14 237883701 1 2024-03-04T13:42:44.613 INFO:tasks.workunit.client.1.smithi169.stdout:6/218: creat d0/f4a x:0 0 0 2024-03-04T13:42:44.614 INFO:tasks.workunit.client.1.smithi169.stdout:8/231: dwrite d7/d10/d17/f29 [0,4194304] 0 2024-03-04T13:42:44.621 INFO:tasks.workunit.client.1.smithi169.stdout:6/219: creat d0/f4b x:0 0 0 2024-03-04T13:42:44.621 INFO:tasks.workunit.client.1.smithi169.stdout:8/232: creat d7/f4d x:0 0 0 2024-03-04T13:42:44.624 INFO:tasks.workunit.client.1.smithi169.stdout:9/270: dwrite d7/f11 [0,4194304] 0 2024-03-04T13:42:44.624 INFO:tasks.workunit.client.1.smithi169.stdout:6/220: mknod d0/d3c/c4c 0 2024-03-04T13:42:44.629 INFO:tasks.workunit.client.1.smithi169.stdout:4/160: sync 2024-03-04T13:42:44.629 INFO:tasks.workunit.client.1.smithi169.stdout:0/176: sync 2024-03-04T13:42:44.632 INFO:tasks.workunit.client.1.smithi169.stdout:6/221: creat d0/d3c/d40/f4d x:0 0 0 2024-03-04T13:42:44.634 INFO:tasks.workunit.client.1.smithi169.stdout:4/161: mknod d4/d11/d2e/d21/c3a 0 2024-03-04T13:42:44.635 INFO:tasks.workunit.client.1.smithi169.stdout:4/162: chown d4/d11/d23 4530 1 2024-03-04T13:42:44.638 INFO:tasks.workunit.client.1.smithi169.stdout:8/233: rmdir d7/d10/d2d 0 2024-03-04T13:42:44.639 INFO:tasks.workunit.client.1.smithi169.stdout:0/177: mknod d5/d9/dc/d1c/c43 0 2024-03-04T13:42:44.641 INFO:tasks.workunit.client.1.smithi169.stdout:4/163: mkdir d4/d11/d3b 0 2024-03-04T13:42:44.642 INFO:tasks.workunit.client.1.smithi169.stdout:8/234: write d7/d12/f15 [3380250,13903] 0 2024-03-04T13:42:44.647 INFO:tasks.workunit.client.1.smithi169.stdout:9/271: dread d7/dc/d12/d20/f24 [0,4194304] 0 2024-03-04T13:42:44.649 INFO:tasks.workunit.client.1.smithi169.stdout:4/164: creat d4/d1c/d25/f3c x:0 0 0 2024-03-04T13:42:44.652 INFO:tasks.workunit.client.1.smithi169.stdout:9/272: mknod d7/dc/c57 0 2024-03-04T13:42:44.654 INFO:tasks.workunit.client.1.smithi169.stdout:4/165: write d4/fa [272633,44447] 0 2024-03-04T13:42:44.659 INFO:tasks.workunit.client.1.smithi169.stdout:9/273: creat d7/dc/d12/d32/d42/f58 x:0 0 0 2024-03-04T13:42:44.661 INFO:tasks.workunit.client.1.smithi169.stdout:8/235: link d7/d10/d22/c4b d7/d10/c4e 0 2024-03-04T13:42:44.662 INFO:tasks.workunit.client.1.smithi169.stdout:6/222: dread d0/f1 [0,4194304] 0 2024-03-04T13:42:44.664 INFO:tasks.workunit.client.1.smithi169.stdout:0/178: dwrite d5/d9/f42 [0,4194304] 0 2024-03-04T13:42:44.675 INFO:tasks.workunit.client.1.smithi169.stdout:8/236: creat d7/d10/d17/d44/f4f x:0 0 0 2024-03-04T13:42:44.678 INFO:tasks.workunit.client.1.smithi169.stdout:8/237: write d7/d10/d17/d44/f4f [328894,124553] 0 2024-03-04T13:42:44.680 INFO:tasks.workunit.client.1.smithi169.stdout:6/223: write d0/d32/d3a/d43/f45 [874098,34572] 0 2024-03-04T13:42:44.681 INFO:tasks.workunit.client.1.smithi169.stdout:6/224: fdatasync d0/f35 0 2024-03-04T13:42:44.693 INFO:tasks.workunit.client.1.smithi169.stdout:0/179: link d5/d9/dc/d12/d15/d19/f2e d5/d9/dc/d12/f44 0 2024-03-04T13:42:44.694 INFO:tasks.workunit.client.1.smithi169.stdout:0/180: fdatasync d5/d9/f2d 0 2024-03-04T13:42:44.696 INFO:tasks.workunit.client.1.smithi169.stdout:4/166: dread d4/d11/d2e/d21/f29 [0,4194304] 0 2024-03-04T13:42:44.700 INFO:tasks.workunit.client.1.smithi169.stdout:6/225: creat d0/f4e x:0 0 0 2024-03-04T13:42:44.701 INFO:tasks.workunit.client.1.smithi169.stdout:6/226: write d0/f4b [851291,116902] 0 2024-03-04T13:42:44.703 INFO:tasks.workunit.client.1.smithi169.stdout:6/227: chown d0/d32/d3a/f2a 1 1 2024-03-04T13:42:44.706 INFO:tasks.workunit.client.1.smithi169.stdout:4/167: rename d4/d11/d2e/d21/c3a to d4/d11/d2e/d39/c3d 0 2024-03-04T13:42:44.713 INFO:tasks.workunit.client.1.smithi169.stdout:4/168: write d4/f9 [838901,23997] 0 2024-03-04T13:42:44.718 INFO:tasks.workunit.client.1.smithi169.stdout:8/238: write d7/d10/d24/f3c [3101652,52495] 0 2024-03-04T13:42:44.719 INFO:tasks.workunit.client.1.smithi169.stdout:4/169: write d4/d1c/d25/f33 [1021090,88421] 0 2024-03-04T13:42:44.720 INFO:tasks.workunit.client.1.smithi169.stdout:0/181: dwrite d5/d9/dc/d12/d15/d19/f20 [0,4194304] 0 2024-03-04T13:42:44.720 INFO:tasks.workunit.client.1.smithi169.stdout:9/274: dwrite d7/d25/f3d [0,4194304] 0 2024-03-04T13:42:44.721 INFO:tasks.workunit.client.1.smithi169.stdout:2/215: sync 2024-03-04T13:42:44.723 INFO:tasks.workunit.client.1.smithi169.stdout:2/216: fsync d1/f50 0 2024-03-04T13:42:44.726 INFO:tasks.workunit.client.1.smithi169.stdout:8/239: creat d7/d10/d24/f50 x:0 0 0 2024-03-04T13:42:44.729 INFO:tasks.workunit.client.1.smithi169.stdout:2/217: chown d1/c15 875616 1 2024-03-04T13:42:44.730 INFO:tasks.workunit.client.1.smithi169.stdout:6/228: truncate d0/d32/d3a/f3f 612887 0 2024-03-04T13:42:44.731 INFO:tasks.workunit.client.1.smithi169.stdout:0/182: rename l4 to d5/d9/dc/l45 0 2024-03-04T13:42:44.731 INFO:tasks.workunit.client.1.smithi169.stdout:8/240: write d7/d12/f15 [654528,8580] 0 2024-03-04T13:42:44.733 INFO:tasks.workunit.client.1.smithi169.stdout:2/218: symlink d1/d5/d19/d3c/l56 0 2024-03-04T13:42:44.733 INFO:tasks.workunit.client.1.smithi169.stdout:0/183: fsync d5/d9/dc/d1c/f36 0 2024-03-04T13:42:44.736 INFO:tasks.workunit.client.1.smithi169.stdout:8/241: write d7/d10/d17/d44/f4f [996512,5271] 0 2024-03-04T13:42:44.738 INFO:tasks.workunit.client.1.smithi169.stdout:6/229: creat d0/d32/d3a/f4f x:0 0 0 2024-03-04T13:42:44.740 INFO:tasks.workunit.client.1.smithi169.stdout:2/219: write d1/f3 [4388451,85987] 0 2024-03-04T13:42:44.741 INFO:tasks.workunit.client.1.smithi169.stdout:0/184: mkdir d5/d9/dc/d12/d15/d46 0 2024-03-04T13:42:44.746 INFO:tasks.workunit.client.1.smithi169.stdout:8/242: rmdir d7/d10/d17/d44 39 2024-03-04T13:42:44.749 INFO:tasks.workunit.client.1.smithi169.stdout:0/185: truncate d5/d9/f2d 704561 0 2024-03-04T13:42:44.751 INFO:tasks.workunit.client.1.smithi169.stdout:9/275: dwrite d7/d2d/f36 [0,4194304] 0 2024-03-04T13:42:44.758 INFO:tasks.workunit.client.1.smithi169.stdout:0/186: chown d5/d9/dc/d1c/f2c 2 1 2024-03-04T13:42:44.761 INFO:tasks.workunit.client.1.smithi169.stdout:4/170: truncate d4/d11/d23/f35 853819 0 2024-03-04T13:42:44.761 INFO:tasks.workunit.client.1.smithi169.stdout:0/187: read d5/f8 [3264602,82601] 0 2024-03-04T13:42:44.768 INFO:tasks.workunit.client.1.smithi169.stdout:6/230: link d0/f4e d0/d32/d3a/d43/f50 0 2024-03-04T13:42:44.770 INFO:tasks.workunit.client.1.smithi169.stdout:0/188: write d5/d9/dc/d1c/f2c [3491367,129933] 0 2024-03-04T13:42:44.883 INFO:tasks.workunit.client.1.smithi169.stdout:6/231: creat d0/d3c/d40/f51 x:0 0 0 2024-03-04T13:42:44.883 INFO:tasks.workunit.client.1.smithi169.stdout:0/189: chown d5/d9/d3d 4028 1 2024-03-04T13:42:44.883 INFO:tasks.workunit.client.1.smithi169.stdout:4/171: link d4/d1c/d25/f3c d4/d11/d2e/d21/f3e 0 2024-03-04T13:42:44.884 INFO:tasks.workunit.client.1.smithi169.stdout:2/220: dread d1/d5/d19/f27 [0,4194304] 0 2024-03-04T13:42:44.885 INFO:tasks.workunit.client.1.smithi169.stdout:0/190: chown d5/d9/d39 446047341 1 2024-03-04T13:42:44.885 INFO:tasks.workunit.client.1.smithi169.stdout:8/243: dwrite d7/d10/d24/f3c [0,4194304] 0 2024-03-04T13:42:44.886 INFO:tasks.workunit.client.1.smithi169.stdout:6/232: dread - d0/f1b zero size 2024-03-04T13:42:44.890 INFO:tasks.workunit.client.1.smithi169.stdout:0/191: symlink d5/d9/dc/l47 0 2024-03-04T13:42:44.891 INFO:tasks.workunit.client.1.smithi169.stdout:4/172: creat d4/d11/d3b/f3f x:0 0 0 2024-03-04T13:42:44.893 INFO:tasks.workunit.client.1.smithi169.stdout:2/221: mknod d1/d5/d22/c57 0 2024-03-04T13:42:44.893 INFO:tasks.workunit.client.1.smithi169.stdout:0/192: write d5/d9/d39/f3a [4848377,100175] 0 2024-03-04T13:42:44.896 INFO:tasks.workunit.client.1.smithi169.stdout:6/233: mkdir d0/d3c/d40/d52 0 2024-03-04T13:42:44.900 INFO:tasks.workunit.client.1.smithi169.stdout:2/222: write d1/d5/d19/f3f [9474,116085] 0 2024-03-04T13:42:44.900 INFO:tasks.workunit.client.1.smithi169.stdout:4/173: truncate d4/d1c/d25/f3c 128311 0 2024-03-04T13:42:44.902 INFO:tasks.workunit.client.1.smithi169.stdout:0/193: creat d5/d9/d3d/f48 x:0 0 0 2024-03-04T13:42:44.902 INFO:tasks.workunit.client.1.smithi169.stdout:6/234: fdatasync d0/d32/d3a/f21 0 2024-03-04T13:42:44.904 INFO:tasks.workunit.client.1.smithi169.stdout:0/194: readlink d5/d9/dc/l47 0 2024-03-04T13:42:44.909 INFO:tasks.workunit.client.1.smithi169.stdout:8/244: dwrite d7/d10/d24/f3c [0,4194304] 0 2024-03-04T13:42:44.911 INFO:tasks.workunit.client.1.smithi169.stdout:8/245: write d7/ff [410589,120264] 0 2024-03-04T13:42:44.911 INFO:tasks.workunit.client.1.smithi169.stdout:4/174: mknod d4/d11/d2e/d21/c40 0 2024-03-04T13:42:44.913 INFO:tasks.workunit.client.1.smithi169.stdout:2/223: creat d1/d5/d19/d3c/f58 x:0 0 0 2024-03-04T13:42:44.924 INFO:tasks.workunit.client.1.smithi169.stdout:2/224: mknod d1/d5/d22/d45/d4a/c59 0 2024-03-04T13:42:44.931 INFO:tasks.workunit.client.1.smithi169.stdout:4/175: rename d4/d11/d2e/d21/f3e to d4/f41 0 2024-03-04T13:42:44.932 INFO:tasks.workunit.client.1.smithi169.stdout:0/195: dwrite d5/d9/dc/d1c/f36 [0,4194304] 0 2024-03-04T13:42:44.934 INFO:tasks.workunit.client.1.smithi169.stdout:1/220: sync 2024-03-04T13:42:44.934 INFO:tasks.workunit.client.1.smithi169.stdout:5/279: sync 2024-03-04T13:42:44.934 INFO:tasks.workunit.client.1.smithi169.stdout:3/197: sync 2024-03-04T13:42:44.936 INFO:tasks.workunit.client.1.smithi169.stdout:2/225: mknod d1/d5/d22/d45/c5a 0 2024-03-04T13:42:44.936 INFO:tasks.workunit.client.1.smithi169.stdout:7/274: sync 2024-03-04T13:42:44.939 INFO:tasks.workunit.client.1.smithi169.stdout:3/198: fdatasync d7/de/d14/f2a 0 2024-03-04T13:42:44.940 INFO:tasks.workunit.client.1.smithi169.stdout:0/196: rename d5 to d5/d9/dc/d12/d15/d46/d49 22 2024-03-04T13:42:44.940 INFO:tasks.workunit.client.1.smithi169.stdout:5/280: rmdir d1/d16/d17 39 2024-03-04T13:42:44.943 INFO:tasks.workunit.client.1.smithi169.stdout:8/246: dwrite d7/f4d [0,4194304] 0 2024-03-04T13:42:44.946 INFO:tasks.workunit.client.1.smithi169.stdout:2/226: write d1/d5/d19/d1e/f43 [658697,28084] 0 2024-03-04T13:42:44.950 INFO:tasks.workunit.client.1.smithi169.stdout:8/247: read d7/d10/d24/d3e/f33 [43809,48306] 0 2024-03-04T13:42:44.950 INFO:tasks.workunit.client.1.smithi169.stdout:5/281: readlink d1/d16/d17/d28/l43 0 2024-03-04T13:42:44.952 INFO:tasks.workunit.client.1.smithi169.stdout:2/227: write d1/d5/d19/d3c/f58 [970520,126034] 0 2024-03-04T13:42:44.956 INFO:tasks.workunit.client.1.smithi169.stdout:7/275: rename d2/d15/l20 to d2/de/d37/l5f 0 2024-03-04T13:42:44.959 INFO:tasks.workunit.client.1.smithi169.stdout:7/276: stat d2/d15/l51 0 2024-03-04T13:42:44.959 INFO:tasks.workunit.client.1.smithi169.stdout:3/199: mknod d7/de/d2b/d39/c40 0 2024-03-04T13:42:44.959 INFO:tasks.workunit.client.1.smithi169.stdout:2/228: chown d1/d5/d19/d1e/f24 3619892 1 2024-03-04T13:42:44.961 INFO:tasks.workunit.client.1.smithi169.stdout:2/229: readlink d1/d5/d19/l1f 0 2024-03-04T13:42:44.966 INFO:tasks.workunit.client.1.smithi169.stdout:0/197: fsync d5/d9/dc/d1c/f36 0 2024-03-04T13:42:44.967 INFO:tasks.workunit.client.1.smithi169.stdout:1/221: rename d4/d13/d26/l32 to d4/d5/d36/d35/d16/d3a/l48 0 2024-03-04T13:42:44.979 INFO:tasks.workunit.client.1.smithi169.stdout:0/198: dread d5/d9/dc/d1c/f23 [0,4194304] 0 2024-03-04T13:42:44.988 INFO:tasks.workunit.client.1.smithi169.stdout:0/199: mknod d5/d9/dc/d1c/c4a 0 2024-03-04T13:42:44.991 INFO:tasks.workunit.client.1.smithi169.stdout:0/200: fsync d5/d9/dc/d12/d15/d19/f32 0 2024-03-04T13:42:44.992 INFO:tasks.workunit.client.1.smithi169.stdout:8/248: rename d7/d10/d24/d3e/c48 to d7/c51 0 2024-03-04T13:42:44.995 INFO:tasks.workunit.client.1.smithi169.stdout:9/276: sync 2024-03-04T13:42:45.001 INFO:tasks.workunit.client.1.smithi169.stdout:6/235: dwrite d0/f4e [0,4194304] 0 2024-03-04T13:42:45.002 INFO:tasks.workunit.client.1.smithi169.stdout:3/200: rename d7/de/d14/d13/d1a/d2d/f2e to d7/de/d14/d18/d21/d3f/f41 0 2024-03-04T13:42:45.010 INFO:tasks.workunit.client.1.smithi169.stdout:4/176: dwrite d4/f15 [0,4194304] 0 2024-03-04T13:42:45.013 INFO:tasks.workunit.client.1.smithi169.stdout:8/249: dwrite d7/d10/d17/f27 [4194304,4194304] 0 2024-03-04T13:42:45.019 INFO:tasks.workunit.client.1.smithi169.stdout:0/201: dwrite d5/d9/dc/d12/d15/d19/f28 [0,4194304] 0 2024-03-04T13:42:45.022 INFO:tasks.workunit.client.1.smithi169.stdout:6/236: mknod d0/d3c/c53 0 2024-03-04T13:42:45.023 INFO:tasks.workunit.client.1.smithi169.stdout:5/282: truncate d1/d16/f20 4379765 0 2024-03-04T13:42:45.025 INFO:tasks.workunit.client.1.smithi169.stdout:3/201: symlink d7/de/d14/d18/d21/d3f/l42 0 2024-03-04T13:42:45.042 INFO:tasks.workunit.client.1.smithi169.stdout:0/202: write d5/f8 [4642868,90768] 0 2024-03-04T13:42:45.046 INFO:tasks.workunit.client.1.smithi169.stdout:6/237: chown d0/l11 2268377 1 2024-03-04T13:42:45.049 INFO:tasks.workunit.client.1.smithi169.stdout:3/202: mknod d7/de/d2b/d39/c43 0 2024-03-04T13:42:45.049 INFO:tasks.workunit.client.1.smithi169.stdout:2/230: dwrite d1/d5/f8 [4194304,4194304] 0 2024-03-04T13:42:45.050 INFO:tasks.workunit.client.1.smithi169.stdout:1/222: rename d4/d5/d36/d35/df/f24 to d4/d5/f49 0 2024-03-04T13:42:45.052 INFO:tasks.workunit.client.1.smithi169.stdout:7/277: dwrite d2/de/d3e/d41/f59 [0,4194304] 0 2024-03-04T13:42:45.060 INFO:tasks.workunit.client.1.smithi169.stdout:6/238: dread d0/f28 [0,4194304] 0 2024-03-04T13:42:45.062 INFO:tasks.workunit.client.1.smithi169.stdout:1/223: write d4/d1d/f34 [202435,91538] 0 2024-03-04T13:42:45.062 INFO:tasks.workunit.client.1.smithi169.stdout:8/250: symlink d7/d10/d17/d44/d4c/l52 0 2024-03-04T13:42:45.074 INFO:tasks.workunit.client.1.smithi169.stdout:5/283: dwrite d1/d16/d17/d28/f4c [0,4194304] 0 2024-03-04T13:42:45.075 INFO:tasks.workunit.client.1.smithi169.stdout:2/231: rename d1/d5/c2d to d1/d5/d19/d3c/c5b 0 2024-03-04T13:42:45.080 INFO:tasks.workunit.client.1.smithi169.stdout:4/177: dwrite d4/f9 [0,4194304] 0 2024-03-04T13:42:45.080 INFO:tasks.workunit.client.1.smithi169.stdout:7/278: chown d2/c31 216190 1 2024-03-04T13:42:45.085 INFO:tasks.workunit.client.1.smithi169.stdout:6/239: mknod d0/d32/d3a/d43/c54 0 2024-03-04T13:42:45.085 INFO:tasks.workunit.client.1.smithi169.stdout:8/251: creat d7/d10/d22/d46/f53 x:0 0 0 2024-03-04T13:42:45.088 INFO:tasks.workunit.client.1.smithi169.stdout:9/277: dwrite d7/dc/d12/d20/f24 [0,4194304] 0 2024-03-04T13:42:45.089 INFO:tasks.workunit.client.1.smithi169.stdout:8/252: write d7/d10/d17/d44/d36/f4a [807883,122028] 0 2024-03-04T13:42:45.094 INFO:tasks.workunit.client.1.smithi169.stdout:7/279: unlink d2/f1c 0 2024-03-04T13:42:45.100 INFO:tasks.workunit.client.1.smithi169.stdout:2/232: creat d1/d5/d19/d1e/f5c x:0 0 0 2024-03-04T13:42:45.103 INFO:tasks.workunit.client.1.smithi169.stdout:2/233: fsync d1/d5/d19/f47 0 2024-03-04T13:42:45.106 INFO:tasks.workunit.client.1.smithi169.stdout:8/253: rename d7/d12/c47 to d7/d10/d22/d46/d49/c54 0 2024-03-04T13:42:45.111 INFO:tasks.workunit.client.1.smithi169.stdout:9/278: dread d7/d2d/f36 [0,4194304] 0 2024-03-04T13:42:45.115 INFO:tasks.workunit.client.1.smithi169.stdout:9/279: write d7/d27/f2f [744266,15090] 0 2024-03-04T13:42:45.116 INFO:tasks.workunit.client.1.smithi169.stdout:9/280: readlink d7/d25/l37 0 2024-03-04T13:42:45.117 INFO:tasks.workunit.client.1.smithi169.stdout:4/178: mknod d4/d11/d2e/c42 0 2024-03-04T13:42:45.123 INFO:tasks.workunit.client.1.smithi169.stdout:2/234: creat d1/d5/d22/d45/d4a/f5d x:0 0 0 2024-03-04T13:42:45.126 INFO:tasks.workunit.client.1.smithi169.stdout:7/280: rename d2/d15/d54/f5e to d2/d28/f60 0 2024-03-04T13:42:45.126 INFO:tasks.workunit.client.1.smithi169.stdout:8/254: rmdir d7/d10/d17/d44 39 2024-03-04T13:42:45.126 INFO:tasks.workunit.client.1.smithi169.stdout:3/203: link d7/c38 d7/de/d14/d13/d1a/d2d/c44 0 2024-03-04T13:42:45.126 INFO:tasks.workunit.client.1.smithi169.stdout:2/235: chown d1/d5/d22/d3b/f4c 37209 1 2024-03-04T13:42:45.131 INFO:tasks.workunit.client.1.smithi169.stdout:1/224: getdents d4/d5/d36 0 2024-03-04T13:42:45.133 INFO:tasks.workunit.client.1.smithi169.stdout:4/179: fdatasync d4/d11/f30 0 2024-03-04T13:42:45.136 INFO:tasks.workunit.client.1.smithi169.stdout:2/236: chown d1/d5/d22/d3b/d41/c49 6362549 1 2024-03-04T13:42:45.136 INFO:tasks.workunit.client.1.smithi169.stdout:1/225: chown d4/d13/l18 968095532 1 2024-03-04T13:42:45.136 INFO:tasks.workunit.client.1.smithi169.stdout:9/281: mkdir d7/dc/d12/d59 0 2024-03-04T13:42:45.138 INFO:tasks.workunit.client.1.smithi169.stdout:4/180: chown d4/d11/l2a 20428 1 2024-03-04T13:42:45.139 INFO:tasks.workunit.client.1.smithi169.stdout:1/226: dread - d4/d13/f37 zero size 2024-03-04T13:42:45.140 INFO:tasks.workunit.client.1.smithi169.stdout:7/281: mknod d2/de/d3e/d41/d5b/c61 0 2024-03-04T13:42:45.147 INFO:tasks.workunit.client.1.smithi169.stdout:9/282: creat d7/d27/f5a x:0 0 0 2024-03-04T13:42:45.150 INFO:tasks.workunit.client.1.smithi169.stdout:6/240: dwrite d0/fb [0,4194304] 0 2024-03-04T13:42:45.167 INFO:tasks.workunit.client.1.smithi169.stdout:7/282: dwrite d2/de/f53 [0,4194304] 0 2024-03-04T13:42:45.168 INFO:tasks.workunit.client.1.smithi169.stdout:4/181: dwrite d4/f9 [4194304,4194304] 0 2024-03-04T13:42:45.172 INFO:tasks.workunit.client.1.smithi169.stdout:3/204: write d7/f3b [3121884,40293] 0 2024-03-04T13:42:45.176 INFO:tasks.workunit.client.1.smithi169.stdout:3/205: write d7/f25 [1408452,51369] 0 2024-03-04T13:42:45.177 INFO:tasks.workunit.client.1.smithi169.stdout:6/241: rename d0/d32/d3a/c34 to d0/d32/c55 0 2024-03-04T13:42:45.179 INFO:tasks.workunit.client.1.smithi169.stdout:8/255: truncate d7/d10/d17/f27 3040128 0 2024-03-04T13:42:45.183 INFO:tasks.workunit.client.1.smithi169.stdout:9/283: dwrite d7/dc/f1e [0,4194304] 0 2024-03-04T13:42:45.187 INFO:tasks.workunit.client.1.smithi169.stdout:2/237: link d1/d5/d22/d45/d4a/f53 d1/d5/d19/d1e/f5e 0 2024-03-04T13:42:45.189 INFO:tasks.workunit.client.1.smithi169.stdout:7/283: creat d2/de/d3e/f62 x:0 0 0 2024-03-04T13:42:45.191 INFO:tasks.workunit.client.1.smithi169.stdout:2/238: dread - d1/d5/d19/d1e/f5e zero size 2024-03-04T13:42:45.193 INFO:tasks.workunit.client.1.smithi169.stdout:6/242: readlink d0/l1a 0 2024-03-04T13:42:45.194 INFO:tasks.workunit.client.1.smithi169.stdout:2/239: chown d1/d5/d22/d45/d4a/c59 195736 1 2024-03-04T13:42:45.197 INFO:tasks.workunit.client.1.smithi169.stdout:3/206: rename d7/cc to d7/de/d24/c45 0 2024-03-04T13:42:45.198 INFO:tasks.workunit.client.1.smithi169.stdout:6/243: chown d0/l11 15 1 2024-03-04T13:42:45.200 INFO:tasks.workunit.client.1.smithi169.stdout:3/207: fdatasync d7/de/d14/d13/d1a/d2d/f37 0 2024-03-04T13:42:45.202 INFO:tasks.workunit.client.1.smithi169.stdout:9/284: rmdir d7/d27 39 2024-03-04T13:42:45.212 INFO:tasks.workunit.client.1.smithi169.stdout:7/284: mknod d2/de/d37/c63 0 2024-03-04T13:42:45.212 INFO:tasks.workunit.client.1.smithi169.stdout:7/285: fdatasync d2/de/ff 0 2024-03-04T13:42:45.213 INFO:tasks.workunit.client.1.smithi169.stdout:2/240: fsync d1/f51 0 2024-03-04T13:42:45.218 INFO:tasks.workunit.client.1.smithi169.stdout:1/227: dwrite d4/d1d/f40 [8388608,4194304] 0 2024-03-04T13:42:45.221 INFO:tasks.workunit.client.1.smithi169.stdout:3/208: creat d7/de/d14/d18/f46 x:0 0 0 2024-03-04T13:42:45.223 INFO:tasks.workunit.client.1.smithi169.stdout:3/209: write d7/f25 [2092822,45773] 0 2024-03-04T13:42:45.224 INFO:tasks.workunit.client.1.smithi169.stdout:9/285: creat d7/dc/d12/d20/d4e/f5b x:0 0 0 2024-03-04T13:42:45.229 INFO:tasks.workunit.client.1.smithi169.stdout:9/286: read d7/dc/f1d [2879467,99492] 0 2024-03-04T13:42:45.232 INFO:tasks.workunit.client.1.smithi169.stdout:6/244: mkdir d0/d3c/d40/d52/d56 0 2024-03-04T13:42:45.234 INFO:tasks.workunit.client.1.smithi169.stdout:2/241: creat d1/d5/d19/d3c/f5f x:0 0 0 2024-03-04T13:42:45.235 INFO:tasks.workunit.client.1.smithi169.stdout:2/242: readlink d1/l38 0 2024-03-04T13:42:45.240 INFO:tasks.workunit.client.1.smithi169.stdout:1/228: dwrite d4/d5/d36/d35/df/f1c [4194304,4194304] 0 2024-03-04T13:42:45.242 INFO:tasks.workunit.client.1.smithi169.stdout:3/210: mkdir d7/de/d14/d47 0 2024-03-04T13:42:45.247 INFO:tasks.workunit.client.1.smithi169.stdout:3/211: chown d7/de/d14/d13/d1a/d2d/f37 59249 1 2024-03-04T13:42:45.247 INFO:tasks.workunit.client.1.smithi169.stdout:7/286: fsync d2/f22 0 2024-03-04T13:42:45.250 INFO:tasks.workunit.client.1.smithi169.stdout:9/287: mkdir d7/dc/d12/d32/d5c 0 2024-03-04T13:42:45.251 INFO:tasks.workunit.client.1.smithi169.stdout:7/287: fdatasync d2/d15/f24 0 2024-03-04T13:42:45.254 INFO:tasks.workunit.client.1.smithi169.stdout:2/243: creat d1/d5/d19/d36/f60 x:0 0 0 2024-03-04T13:42:45.254 INFO:tasks.workunit.client.1.smithi169.stdout:4/182: dwrite d4/d11/d23/f35 [0,4194304] 0 2024-03-04T13:42:45.266 INFO:tasks.workunit.client.1.smithi169.stdout:3/212: creat d7/de/d14/f48 x:0 0 0 2024-03-04T13:42:45.268 INFO:tasks.workunit.client.1.smithi169.stdout:3/213: read - d7/de/d14/d13/d1a/d2d/f37 zero size 2024-03-04T13:42:45.271 INFO:tasks.workunit.client.1.smithi169.stdout:9/288: fdatasync d7/d2d/f36 0 2024-03-04T13:42:45.274 INFO:tasks.workunit.client.1.smithi169.stdout:7/288: symlink d2/de/d3e/d41/l64 0 2024-03-04T13:42:45.274 INFO:tasks.workunit.client.1.smithi169.stdout:1/229: truncate d4/f25 82089 0 2024-03-04T13:42:45.278 INFO:tasks.workunit.client.1.smithi169.stdout:1/230: dread - d4/d13/f37 zero size 2024-03-04T13:42:45.278 INFO:tasks.workunit.client.1.smithi169.stdout:1/231: readlink d4/d5/l43 0 2024-03-04T13:42:45.278 INFO:tasks.workunit.client.1.smithi169.stdout:4/183: mknod d4/d11/d2e/d39/c43 0 2024-03-04T13:42:45.278 INFO:tasks.workunit.client.1.smithi169.stdout:7/289: read d2/de/d37/f3f [116629,86196] 0 2024-03-04T13:42:45.283 INFO:tasks.workunit.client.1.smithi169.stdout:6/245: link d0/d3c/c53 d0/d3c/d40/d52/c57 0 2024-03-04T13:42:45.288 INFO:tasks.workunit.client.1.smithi169.stdout:2/244: rename d1/d5/d19/d1e/d29/c2a to d1/d5/d22/d45/c61 0 2024-03-04T13:42:45.288 INFO:tasks.workunit.client.1.smithi169.stdout:4/184: mknod d4/d11/d23/c44 0 2024-03-04T13:42:45.290 INFO:tasks.workunit.client.1.smithi169.stdout:2/245: chown d1/c2c 3 1 2024-03-04T13:42:45.291 INFO:tasks.workunit.client.1.smithi169.stdout:3/214: unlink d7/de/d14/d13/l1f 0 2024-03-04T13:42:45.306 INFO:tasks.workunit.client.1.smithi169.stdout:2/246: creat d1/d5/d22/d45/f62 x:0 0 0 2024-03-04T13:42:45.310 INFO:tasks.workunit.client.1.smithi169.stdout:4/185: creat d4/d11/d2e/d37/f45 x:0 0 0 2024-03-04T13:42:45.312 INFO:tasks.workunit.client.1.smithi169.stdout:1/232: dwrite d4/d13/f37 [0,4194304] 0 2024-03-04T13:42:45.312 INFO:tasks.workunit.client.1.smithi169.stdout:9/289: dwrite d7/d27/f2f [0,4194304] 0 2024-03-04T13:42:45.314 INFO:tasks.workunit.client.1.smithi169.stdout:6/246: dwrite d0/d3c/d40/f4d [0,4194304] 0 2024-03-04T13:42:45.315 INFO:tasks.workunit.client.1.smithi169.stdout:4/186: truncate d4/d11/f1b 985257 0 2024-03-04T13:42:45.316 INFO:tasks.workunit.client.1.smithi169.stdout:2/247: write d1/d5/d19/d3c/f58 [84197,48219] 0 2024-03-04T13:42:45.322 INFO:tasks.workunit.client.1.smithi169.stdout:7/290: dwrite d2/d28/f60 [0,4194304] 0 2024-03-04T13:42:45.322 INFO:tasks.workunit.client.1.smithi169.stdout:4/187: creat d4/d11/d23/f46 x:0 0 0 2024-03-04T13:42:45.327 INFO:tasks.workunit.client.1.smithi169.stdout:4/188: write d4/f15 [1054590,83465] 0 2024-03-04T13:42:45.327 INFO:tasks.workunit.client.1.smithi169.stdout:6/247: creat d0/d32/f58 x:0 0 0 2024-03-04T13:42:45.331 INFO:tasks.workunit.client.1.smithi169.stdout:7/291: chown d2/de/d3e/d41/l64 340 1 2024-03-04T13:42:45.332 INFO:tasks.workunit.client.1.smithi169.stdout:1/233: mknod d4/d5/d36/d35/c4a 0 2024-03-04T13:42:45.335 INFO:tasks.workunit.client.1.smithi169.stdout:1/234: chown d4/d5/l14 2849276 1 2024-03-04T13:42:45.339 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:42:45 smithi107 ceph-mon[28739]: pgmap v44: 65 pgs: 65 active+clean; 586 MiB data, 3.4 GiB used, 533 GiB / 536 GiB avail; 5.7 MiB/s rd, 64 MiB/s wr, 230 op/s 2024-03-04T13:42:45.342 INFO:tasks.workunit.client.1.smithi169.stdout:6/248: mknod d0/d32/d3a/d43/c59 0 2024-03-04T13:42:45.345 INFO:tasks.workunit.client.1.smithi169.stdout:9/290: creat d7/d27/d54/f5d x:0 0 0 2024-03-04T13:42:45.347 INFO:tasks.workunit.client.1.smithi169.stdout:0/203: sync 2024-03-04T13:42:45.348 INFO:tasks.workunit.client.1.smithi169.stdout:7/292: mknod d2/de/d37/c65 0 2024-03-04T13:42:45.353 INFO:tasks.workunit.client.1.smithi169.stdout:4/189: symlink d4/l47 0 2024-03-04T13:42:45.354 INFO:tasks.workunit.client.1.smithi169.stdout:6/249: stat d0/c2e 0 2024-03-04T13:42:45.355 INFO:tasks.workunit.client.1.smithi169.stdout:4/190: chown d4 19 1 2024-03-04T13:42:45.356 INFO:tasks.workunit.client.1.smithi169.stdout:2/248: dwrite d1/d5/d22/d45/d4a/f5d [0,4194304] 0 2024-03-04T13:42:45.357 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:42:45 smithi169 ceph-mon[36313]: pgmap v44: 65 pgs: 65 active+clean; 586 MiB data, 3.4 GiB used, 533 GiB / 536 GiB avail; 5.7 MiB/s rd, 64 MiB/s wr, 230 op/s 2024-03-04T13:42:45.357 INFO:tasks.workunit.client.1.smithi169.stdout:7/293: write d2/fa [193381,124252] 0 2024-03-04T13:42:45.358 INFO:tasks.workunit.client.1.smithi169.stdout:1/235: mknod d4/d5/c4b 0 2024-03-04T13:42:45.363 INFO:tasks.workunit.client.1.smithi169.stdout:6/250: dread - d0/f1d zero size 2024-03-04T13:42:45.363 INFO:tasks.workunit.client.1.smithi169.stdout:0/204: symlink d5/d9/dc/d12/l4b 0 2024-03-04T13:42:45.366 INFO:tasks.workunit.client.1.smithi169.stdout:6/251: write d0/f35 [4103403,100115] 0 2024-03-04T13:42:45.367 INFO:tasks.workunit.client.1.smithi169.stdout:0/205: readlink d5/d9/dc/d12/l4b 0 2024-03-04T13:42:45.370 INFO:tasks.workunit.client.1.smithi169.stdout:0/206: fdatasync d5/d9/dc/d12/d15/d19/f20 0 2024-03-04T13:42:45.383 INFO:tasks.workunit.client.1.smithi169.stdout:9/291: getdents d7/dc 0 2024-03-04T13:42:45.387 INFO:tasks.workunit.client.1.smithi169.stdout:2/249: mkdir d1/d5/d19/d63 0 2024-03-04T13:42:45.390 INFO:tasks.workunit.client.1.smithi169.stdout:0/207: dwrite d5/d9/dc/d12/d15/d19/f20 [0,4194304] 0 2024-03-04T13:42:45.391 INFO:tasks.workunit.client.1.smithi169.stdout:3/215: fsync d7/fb 0 2024-03-04T13:42:45.392 INFO:tasks.workunit.client.1.smithi169.stdout:9/292: symlink d7/dc/d12/d32/d42/l5e 0 2024-03-04T13:42:45.396 INFO:tasks.workunit.client.1.smithi169.stdout:0/208: mkdir d5/d4c 0 2024-03-04T13:42:45.401 INFO:tasks.workunit.client.1.smithi169.stdout:6/252: getdents d0/d32/d3a 0 2024-03-04T13:42:45.404 INFO:tasks.workunit.client.1.smithi169.stdout:9/293: creat d7/d25/d3c/f5f x:0 0 0 2024-03-04T13:42:45.404 INFO:tasks.workunit.client.1.smithi169.stdout:6/253: fsync d0/d32/f58 0 2024-03-04T13:42:45.407 INFO:tasks.workunit.client.1.smithi169.stdout:9/294: chown d7/d2d/l2e 3 1 2024-03-04T13:42:45.408 INFO:tasks.workunit.client.1.smithi169.stdout:6/254: write d0/f37 [3082745,92817] 0 2024-03-04T13:42:45.408 INFO:tasks.workunit.client.1.smithi169.stdout:2/250: rmdir d1/d5/d42 0 2024-03-04T13:42:45.410 INFO:tasks.workunit.client.1.smithi169.stdout:3/216: creat d7/f49 x:0 0 0 2024-03-04T13:42:45.412 INFO:tasks.workunit.client.1.smithi169.stdout:9/295: fdatasync d7/dc/d12/d32/d42/f43 0 2024-03-04T13:42:45.413 INFO:tasks.workunit.client.1.smithi169.stdout:3/217: chown d7/ld 7810280 1 2024-03-04T13:42:45.415 INFO:tasks.workunit.client.1.smithi169.stdout:6/255: chown d0/ce 3181765 1 2024-03-04T13:42:45.420 INFO:tasks.workunit.client.1.smithi169.stdout:4/191: write d4/d1c/d25/f3c [623765,62715] 0 2024-03-04T13:42:45.421 INFO:tasks.workunit.client.1.smithi169.stdout:6/256: mknod d0/d32/d3a/d43/c5a 0 2024-03-04T13:42:45.422 INFO:tasks.workunit.client.1.smithi169.stdout:7/294: truncate d2/de/f4a 775843 0 2024-03-04T13:42:45.425 INFO:tasks.workunit.client.1.smithi169.stdout:3/218: dread d7/de/d14/d18/f30 [0,4194304] 0 2024-03-04T13:42:45.426 INFO:tasks.workunit.client.1.smithi169.stdout:0/209: link d5/d9/dc/l45 d5/d9/dc/l4d 0 2024-03-04T13:42:45.431 INFO:tasks.workunit.client.1.smithi169.stdout:9/296: dwrite d7/dc/f1e [0,4194304] 0 2024-03-04T13:42:45.432 INFO:tasks.workunit.client.1.smithi169.stdout:7/295: unlink d2/d15/f24 0 2024-03-04T13:42:45.435 INFO:tasks.workunit.client.1.smithi169.stdout:6/257: truncate d0/f8 4361540 0 2024-03-04T13:42:45.437 INFO:tasks.workunit.client.1.smithi169.stdout:0/210: unlink d5/d9/dc/d1c/c22 0 2024-03-04T13:42:45.440 INFO:tasks.workunit.client.1.smithi169.stdout:0/211: stat d5/d9/dc/c10 0 2024-03-04T13:42:45.448 INFO:tasks.workunit.client.1.smithi169.stdout:7/296: mknod d2/d28/c66 0 2024-03-04T13:42:45.450 INFO:tasks.workunit.client.1.smithi169.stdout:9/297: dread d7/fe [0,4194304] 0 2024-03-04T13:42:45.453 INFO:tasks.workunit.client.1.smithi169.stdout:6/258: dread d0/f38 [0,4194304] 0 2024-03-04T13:42:45.456 INFO:tasks.workunit.client.1.smithi169.stdout:3/219: dwrite d7/de/d14/d18/f27 [0,4194304] 0 2024-03-04T13:42:45.459 INFO:tasks.workunit.client.1.smithi169.stdout:9/298: truncate d7/dc/d12/d32/d42/f58 495662 0 2024-03-04T13:42:45.462 INFO:tasks.workunit.client.1.smithi169.stdout:9/299: read - d7/dc/d12/d20/d4e/f5b zero size 2024-03-04T13:42:45.464 INFO:tasks.workunit.client.1.smithi169.stdout:3/220: chown d7/de/d14/d13/d1a/d2d/f37 104928 1 2024-03-04T13:42:45.466 INFO:tasks.workunit.client.1.smithi169.stdout:6/259: link d0/d32/d3a/l25 d0/d3c/l5b 0 2024-03-04T13:42:45.467 INFO:tasks.workunit.client.1.smithi169.stdout:9/300: mknod d7/dc/d12/d32/d42/c60 0 2024-03-04T13:42:45.484 INFO:tasks.workunit.client.1.smithi169.stdout:6/260: dwrite d0/f4a [0,4194304] 0 2024-03-04T13:42:45.487 INFO:tasks.workunit.client.1.smithi169.stdout:4/192: truncate d4/f7 7178465 0 2024-03-04T13:42:45.498 INFO:tasks.workunit.client.1.smithi169.stdout:9/301: dwrite d7/d2d/f36 [0,4194304] 0 2024-03-04T13:42:45.504 INFO:tasks.workunit.client.1.smithi169.stdout:0/212: dread d5/d9/dc/d12/d15/d19/f2e [0,4194304] 0 2024-03-04T13:42:45.507 INFO:tasks.workunit.client.1.smithi169.stdout:3/221: dwrite d7/de/d14/d18/f27 [0,4194304] 0 2024-03-04T13:42:45.511 INFO:tasks.workunit.client.1.smithi169.stdout:5/284: sync 2024-03-04T13:42:45.511 INFO:tasks.workunit.client.1.smithi169.stdout:8/256: sync 2024-03-04T13:42:45.512 INFO:tasks.workunit.client.1.smithi169.stdout:6/261: getdents d0/d32/d3e 0 2024-03-04T13:42:45.513 INFO:tasks.workunit.client.1.smithi169.stdout:9/302: getdents d7/dc/d12/d20/d4e 0 2024-03-04T13:42:45.514 INFO:tasks.workunit.client.1.smithi169.stdout:3/222: stat f6 0 2024-03-04T13:42:45.514 INFO:tasks.workunit.client.1.smithi169.stdout:0/213: truncate d5/d9/f2d 1539857 0 2024-03-04T13:42:45.514 INFO:tasks.workunit.client.1.smithi169.stdout:7/297: dwrite d2/f22 [0,4194304] 0 2024-03-04T13:42:45.517 INFO:tasks.workunit.client.1.smithi169.stdout:0/214: dread - d5/d9/d3d/f41 zero size 2024-03-04T13:42:45.530 INFO:tasks.workunit.client.1.smithi169.stdout:4/193: dwrite d4/f15 [4194304,4194304] 0 2024-03-04T13:42:45.534 INFO:tasks.workunit.client.1.smithi169.stdout:8/257: mkdir d7/d55 0 2024-03-04T13:42:45.534 INFO:tasks.workunit.client.1.smithi169.stdout:3/223: creat d7/de/d14/d13/f4a x:0 0 0 2024-03-04T13:42:45.535 INFO:tasks.workunit.client.1.smithi169.stdout:0/215: creat d5/d4c/f4e x:0 0 0 2024-03-04T13:42:45.536 INFO:tasks.workunit.client.1.smithi169.stdout:7/298: symlink d2/de/l67 0 2024-03-04T13:42:45.536 INFO:tasks.workunit.client.1.smithi169.stdout:9/303: mknod d7/d25/d47/c61 0 2024-03-04T13:42:45.538 INFO:tasks.workunit.client.1.smithi169.stdout:8/258: mknod d7/d12/c56 0 2024-03-04T13:42:45.539 INFO:tasks.workunit.client.1.smithi169.stdout:7/299: fsync d2/fa 0 2024-03-04T13:42:45.542 INFO:tasks.workunit.client.1.smithi169.stdout:7/300: write d2/fa [1295589,93014] 0 2024-03-04T13:42:45.542 INFO:tasks.workunit.client.1.smithi169.stdout:6/262: link d0/f4b d0/d3c/d40/f5c 0 2024-03-04T13:42:45.545 INFO:tasks.workunit.client.1.smithi169.stdout:9/304: write d7/fe [2325013,46857] 0 2024-03-04T13:42:45.556 INFO:tasks.workunit.client.1.smithi169.stdout:5/285: dread d1/d16/d2f/f33 [0,4194304] 0 2024-03-04T13:42:45.557 INFO:tasks.workunit.client.1.smithi169.stdout:0/216: mknod d5/d4c/c4f 0 2024-03-04T13:42:45.559 INFO:tasks.workunit.client.1.smithi169.stdout:8/259: stat d7/d10/d17/d44/d4c 0 2024-03-04T13:42:45.560 INFO:tasks.workunit.client.1.smithi169.stdout:8/260: chown d7/d10/d22/d46 119661704 1 2024-03-04T13:42:45.566 INFO:tasks.workunit.client.1.smithi169.stdout:7/301: mkdir d2/d28/d68 0 2024-03-04T13:42:45.571 INFO:tasks.workunit.client.1.smithi169.stdout:9/305: read d7/dc/d12/d32/d42/d4f/f50 [4023043,77275] 0 2024-03-04T13:42:45.572 INFO:tasks.workunit.client.1.smithi169.stdout:3/224: link d7/f3b d7/de/d2b/d39/f4b 0 2024-03-04T13:42:45.574 INFO:tasks.workunit.client.1.smithi169.stdout:9/306: chown d7/f11 2246137 1 2024-03-04T13:42:45.575 INFO:tasks.workunit.client.1.smithi169.stdout:8/261: creat d7/d10/d24/d3e/f57 x:0 0 0 2024-03-04T13:42:45.576 INFO:tasks.workunit.client.1.smithi169.stdout:6/263: mkdir d0/d5d 0 2024-03-04T13:42:45.578 INFO:tasks.workunit.client.1.smithi169.stdout:0/217: dwrite d5/d9/dc/d12/d15/d19/f2f [0,4194304] 0 2024-03-04T13:42:45.579 INFO:tasks.workunit.client.1.smithi169.stdout:4/194: getdents d4 0 2024-03-04T13:42:45.579 INFO:tasks.workunit.client.1.smithi169.stdout:5/286: dwrite d1/d16/d17/d28/f41 [0,4194304] 0 2024-03-04T13:42:45.583 INFO:tasks.workunit.client.1.smithi169.stdout:7/302: mkdir d2/de/d3e/d41/d69 0 2024-03-04T13:42:45.583 INFO:tasks.workunit.client.1.smithi169.stdout:7/303: dread - d2/d28/f56 zero size 2024-03-04T13:42:45.590 INFO:tasks.workunit.client.1.smithi169.stdout:5/287: dread - d1/d16/f31 zero size 2024-03-04T13:42:45.595 INFO:tasks.workunit.client.1.smithi169.stdout:3/225: chown d7/de/d14/d13/d1a/l1c 8199 1 2024-03-04T13:42:45.595 INFO:tasks.workunit.client.1.smithi169.stdout:6/264: mkdir d0/d32/d3a/d5e 0 2024-03-04T13:42:45.598 INFO:tasks.workunit.client.1.smithi169.stdout:9/307: dread d7/fe [0,4194304] 0 2024-03-04T13:42:45.601 INFO:tasks.workunit.client.1.smithi169.stdout:9/308: chown d7/d25/d47/c61 113560893 1 2024-03-04T13:42:45.603 INFO:tasks.workunit.client.1.smithi169.stdout:7/304: dread d2/f34 [0,4194304] 0 2024-03-04T13:42:45.604 INFO:tasks.workunit.client.1.smithi169.stdout:0/218: dread d5/d9/f42 [0,4194304] 0 2024-03-04T13:42:45.604 INFO:tasks.workunit.client.1.smithi169.stdout:4/195: rename d4/d1c/d25 to d4/d11/d2e/d39/d48 0 2024-03-04T13:42:45.611 INFO:tasks.workunit.client.1.smithi169.stdout:0/219: write d5/d9/dc/d1c/f36 [3638872,112404] 0 2024-03-04T13:42:45.613 INFO:tasks.workunit.client.1.smithi169.stdout:6/265: write d0/f12 [1840736,21722] 0 2024-03-04T13:42:45.619 INFO:tasks.workunit.client.1.smithi169.stdout:7/305: mknod d2/de/d3e/d41/c6a 0 2024-03-04T13:42:45.621 INFO:tasks.workunit.client.1.smithi169.stdout:9/309: dread d7/dc/d12/d32/d42/f43 [0,4194304] 0 2024-03-04T13:42:45.623 INFO:tasks.workunit.client.1.smithi169.stdout:4/196: dwrite d4/d1c/f2c [0,4194304] 0 2024-03-04T13:42:45.623 INFO:tasks.workunit.client.1.smithi169.stdout:1/236: sync 2024-03-04T13:42:45.627 INFO:tasks.workunit.client.1.smithi169.stdout:9/310: write d7/d25/f3d [247291,23710] 0 2024-03-04T13:42:45.627 INFO:tasks.workunit.client.1.smithi169.stdout:1/237: stat d4/d5/l43 0 2024-03-04T13:42:45.628 INFO:tasks.workunit.client.1.smithi169.stdout:4/197: chown d4/d11/d2e/d39/d48 25285 1 2024-03-04T13:42:45.629 INFO:tasks.workunit.client.1.smithi169.stdout:3/226: truncate d7/f3b 1517332 0 2024-03-04T13:42:45.631 INFO:tasks.workunit.client.1.smithi169.stdout:0/220: symlink d5/d9/d3d/l50 0 2024-03-04T13:42:45.634 INFO:tasks.workunit.client.1.smithi169.stdout:3/227: chown d7/de/d14/d13/d1a/d2d 1380 1 2024-03-04T13:42:45.634 INFO:tasks.workunit.client.1.smithi169.stdout:1/238: write d4/d5/d36/d35/d16/d3a/f3f [750059,113489] 0 2024-03-04T13:42:45.640 INFO:tasks.workunit.client.1.smithi169.stdout:9/311: creat d7/d25/d3c/f62 x:0 0 0 2024-03-04T13:42:45.642 INFO:tasks.workunit.client.1.smithi169.stdout:8/262: link d7/d10/d24/l28 d7/l58 0 2024-03-04T13:42:45.643 INFO:tasks.workunit.client.1.smithi169.stdout:8/263: fdatasync d7/d10/f1e 0 2024-03-04T13:42:45.646 INFO:tasks.workunit.client.1.smithi169.stdout:1/239: dread d4/d1d/f23 [0,4194304] 0 2024-03-04T13:42:45.649 INFO:tasks.workunit.client.1.smithi169.stdout:1/240: fsync d4/d5/d36/d35/df/f2b 0 2024-03-04T13:42:45.651 INFO:tasks.workunit.client.1.smithi169.stdout:4/198: mknod d4/c49 0 2024-03-04T13:42:45.653 INFO:tasks.workunit.client.1.smithi169.stdout:5/288: dwrite d1/f14 [0,4194304] 0 2024-03-04T13:42:45.656 INFO:tasks.workunit.client.1.smithi169.stdout:0/221: creat d5/d4c/f51 x:0 0 0 2024-03-04T13:42:45.665 INFO:tasks.workunit.client.1.smithi169.stdout:8/264: dwrite d7/d10/d22/f25 [0,4194304] 0 2024-03-04T13:42:45.666 INFO:tasks.workunit.client.1.smithi169.stdout:7/306: mkdir d2/d6b 0 2024-03-04T13:42:45.668 INFO:tasks.workunit.client.1.smithi169.stdout:9/312: mknod d7/d25/c63 0 2024-03-04T13:42:45.669 INFO:tasks.workunit.client.1.smithi169.stdout:9/313: stat d7/d25/d3c 0 2024-03-04T13:42:45.675 INFO:tasks.workunit.client.1.smithi169.stdout:5/289: mkdir d1/d16/d17/d2d/d51 0 2024-03-04T13:42:45.675 INFO:tasks.workunit.client.1.smithi169.stdout:0/222: rmdir d5/d9/dc/d1c 39 2024-03-04T13:42:45.683 INFO:tasks.workunit.client.1.smithi169.stdout:9/314: mkdir d7/d25/d64 0 2024-03-04T13:42:45.688 INFO:tasks.workunit.client.1.smithi169.stdout:9/315: chown d7/d25/d3c/f5f 514305533 1 2024-03-04T13:42:45.692 INFO:tasks.workunit.client.1.smithi169.stdout:3/228: dwrite d7/de/d14/d18/f30 [0,4194304] 0 2024-03-04T13:42:45.693 INFO:tasks.workunit.client.1.smithi169.stdout:7/307: dwrite d2/de/d3e/f62 [0,4194304] 0 2024-03-04T13:42:45.696 INFO:tasks.workunit.client.1.smithi169.stdout:0/223: dread - d5/d9/dc/d1c/f33 zero size 2024-03-04T13:42:45.698 INFO:tasks.workunit.client.1.smithi169.stdout:2/251: sync 2024-03-04T13:42:45.703 INFO:tasks.workunit.client.1.smithi169.stdout:2/252: readlink d1/d5/l2b 0 2024-03-04T13:42:45.703 INFO:tasks.workunit.client.1.smithi169.stdout:8/265: dwrite d7/d10/d24/f3c [0,4194304] 0 2024-03-04T13:42:45.703 INFO:tasks.workunit.client.1.smithi169.stdout:7/308: write d2/f2d [1351178,115708] 0 2024-03-04T13:42:45.703 INFO:tasks.workunit.client.1.smithi169.stdout:6/266: truncate d0/f8 72684 0 2024-03-04T13:42:45.707 INFO:tasks.workunit.client.1.smithi169.stdout:1/241: link d4/l2a d4/d1d/l4c 0 2024-03-04T13:42:45.709 INFO:tasks.workunit.client.1.smithi169.stdout:3/229: rename d7/de/d14/d18/d21/d2c to d7/de/d14/d18/d21/d2c/d4c 22 2024-03-04T13:42:45.710 INFO:tasks.workunit.client.1.smithi169.stdout:1/242: write d4/f20 [2869420,825] 0 2024-03-04T13:42:45.710 INFO:tasks.workunit.client.1.smithi169.stdout:6/267: write d0/f37 [5104547,54794] 0 2024-03-04T13:42:45.713 INFO:tasks.workunit.client.1.smithi169.stdout:1/243: fdatasync d4/d1d/f34 0 2024-03-04T13:42:45.720 INFO:tasks.workunit.client.1.smithi169.stdout:0/224: creat d5/d9/dc/d12/d15/d19/f52 x:0 0 0 2024-03-04T13:42:45.729 INFO:tasks.workunit.client.1.smithi169.stdout:2/253: mkdir d1/d5/d19/d36/d64 0 2024-03-04T13:42:45.735 INFO:tasks.workunit.client.1.smithi169.stdout:7/309: write d2/de/d3e/f62 [5026733,63244] 0 2024-03-04T13:42:45.740 INFO:tasks.workunit.client.1.smithi169.stdout:4/199: truncate d4/f7 2852186 0 2024-03-04T13:42:45.743 INFO:tasks.workunit.client.1.smithi169.stdout:9/316: creat d7/d2d/d48/f65 x:0 0 0 2024-03-04T13:42:45.747 INFO:tasks.workunit.client.1.smithi169.stdout:6/268: unlink d0/d32/d3a/d43/f50 0 2024-03-04T13:42:45.748 INFO:tasks.workunit.client.1.smithi169.stdout:5/290: dwrite d1/d16/d17/f1a [0,4194304] 0 2024-03-04T13:42:45.749 INFO:tasks.workunit.client.1.smithi169.stdout:6/269: fsync d0/d32/d3a/f42 0 2024-03-04T13:42:45.755 INFO:tasks.workunit.client.1.smithi169.stdout:1/244: creat d4/d5/d36/d35/d16/f4d x:0 0 0 2024-03-04T13:42:45.759 INFO:tasks.workunit.client.1.smithi169.stdout:2/254: mknod d1/d5/d22/d3b/c65 0 2024-03-04T13:42:45.761 INFO:tasks.workunit.client.1.smithi169.stdout:4/200: readlink d4/l10 0 2024-03-04T13:42:45.764 INFO:tasks.workunit.client.1.smithi169.stdout:8/266: symlink d7/d10/l59 0 2024-03-04T13:42:45.768 INFO:tasks.workunit.client.1.smithi169.stdout:1/245: dwrite d4/d5/d36/d35/df/f2b [0,4194304] 0 2024-03-04T13:42:45.772 INFO:tasks.workunit.client.1.smithi169.stdout:5/291: chown d1/d16/d17/d1b/c22 56 1 2024-03-04T13:42:45.773 INFO:tasks.workunit.client.1.smithi169.stdout:6/270: mknod d0/d32/d3a/c5f 0 2024-03-04T13:42:45.777 INFO:tasks.workunit.client.1.smithi169.stdout:7/310: mknod d2/d6b/c6c 0 2024-03-04T13:42:45.779 INFO:tasks.workunit.client.1.smithi169.stdout:2/255: mkdir d1/d5/d22/d45/d66 0 2024-03-04T13:42:45.783 INFO:tasks.workunit.client.1.smithi169.stdout:3/230: rename d7/l8 to d7/l4d 0 2024-03-04T13:42:45.785 INFO:tasks.workunit.client.1.smithi169.stdout:3/231: dread - d7/de/d14/d13/f4a zero size 2024-03-04T13:42:45.788 INFO:tasks.workunit.client.1.smithi169.stdout:1/246: mkdir d4/d5/d2d/d4e 0 2024-03-04T13:42:45.789 INFO:tasks.workunit.client.1.smithi169.stdout:3/232: write d7/de/d14/d13/d1a/d2d/f37 [961257,74853] 0 2024-03-04T13:42:45.793 INFO:tasks.workunit.client.1.smithi169.stdout:6/271: chown d0/d3c/d40/d52/c57 1 1 2024-03-04T13:42:45.794 INFO:tasks.workunit.client.1.smithi169.stdout:6/272: fdatasync d0/f16 0 2024-03-04T13:42:45.795 INFO:tasks.workunit.client.1.smithi169.stdout:0/225: creat d5/d9/dc/d12/f53 x:0 0 0 2024-03-04T13:42:45.795 INFO:tasks.workunit.client.1.smithi169.stdout:7/311: unlink d2/de/d37/c65 0 2024-03-04T13:42:45.797 INFO:tasks.workunit.client.1.smithi169.stdout:2/256: dwrite d1/d5/d19/d1e/f24 [0,4194304] 0 2024-03-04T13:42:45.799 INFO:tasks.workunit.client.1.smithi169.stdout:6/273: dread d0/f28 [0,4194304] 0 2024-03-04T13:42:45.801 INFO:tasks.workunit.client.1.smithi169.stdout:7/312: read d2/de/d3e/d41/f49 [2040368,57661] 0 2024-03-04T13:42:45.805 INFO:tasks.workunit.client.1.smithi169.stdout:8/267: rename d7/l42 to d7/d10/d22/d46/l5a 0 2024-03-04T13:42:45.806 INFO:tasks.workunit.client.1.smithi169.stdout:9/317: rmdir d7/dc/d12/d32/d5c 0 2024-03-04T13:42:45.808 INFO:tasks.workunit.client.1.smithi169.stdout:8/268: write d7/f4d [3123515,56613] 0 2024-03-04T13:42:45.809 INFO:tasks.workunit.client.1.smithi169.stdout:5/292: mknod d1/d16/d17/d2d/d51/c52 0 2024-03-04T13:42:45.810 INFO:tasks.workunit.client.1.smithi169.stdout:1/247: mkdir d4/d13/d26/d4f 0 2024-03-04T13:42:45.814 INFO:tasks.workunit.client.1.smithi169.stdout:0/226: creat d5/d9/dc/d1c/f54 x:0 0 0 2024-03-04T13:42:45.816 INFO:tasks.workunit.client.1.smithi169.stdout:3/233: dwrite f6 [4194304,4194304] 0 2024-03-04T13:42:45.816 INFO:tasks.workunit.client.1.smithi169.stdout:2/257: creat d1/d5/f67 x:0 0 0 2024-03-04T13:42:45.827 INFO:tasks.workunit.client.1.smithi169.stdout:6/274: dwrite d0/f29 [0,4194304] 0 2024-03-04T13:42:45.830 INFO:tasks.workunit.client.1.smithi169.stdout:9/318: mkdir d7/d2d/d48/d66 0 2024-03-04T13:42:45.833 INFO:tasks.workunit.client.1.smithi169.stdout:5/293: symlink d1/d16/d17/d28/l53 0 2024-03-04T13:42:45.835 INFO:tasks.workunit.client.1.smithi169.stdout:7/313: dwrite d2/de/f53 [0,4194304] 0 2024-03-04T13:42:45.850 INFO:tasks.workunit.client.1.smithi169.stdout:8/269: rename d7/d10/d22/d46/f53 to d7/d10/d22/f5b 0 2024-03-04T13:42:45.853 INFO:tasks.workunit.client.1.smithi169.stdout:3/234: symlink d7/de/d14/l4e 0 2024-03-04T13:42:45.860 INFO:tasks.workunit.client.1.smithi169.stdout:7/314: dwrite d2/f34 [0,4194304] 0 2024-03-04T13:42:45.862 INFO:tasks.workunit.client.1.smithi169.stdout:2/258: unlink d1/d5/d19/d3c/l46 0 2024-03-04T13:42:45.867 INFO:tasks.workunit.client.1.smithi169.stdout:6/275: unlink d0/d32/f33 0 2024-03-04T13:42:45.875 INFO:tasks.workunit.client.1.smithi169.stdout:1/248: link d4/d1d/f40 d4/d5/d36/f50 0 2024-03-04T13:42:45.880 INFO:tasks.workunit.client.1.smithi169.stdout:8/270: mkdir d7/d10/d24/d5c 0 2024-03-04T13:42:45.890 INFO:tasks.workunit.client.1.smithi169.stdout:5/294: dread d1/f24 [0,4194304] 0 2024-03-04T13:42:45.892 INFO:tasks.workunit.client.1.smithi169.stdout:3/235: creat d7/de/d14/f4f x:0 0 0 2024-03-04T13:42:45.893 INFO:tasks.workunit.client.1.smithi169.stdout:3/236: chown d7/de 13636 1 2024-03-04T13:42:45.897 INFO:tasks.workunit.client.1.smithi169.stdout:7/315: rename d2/de/d3e/d41/c6a to d2/d6b/c6d 0 2024-03-04T13:42:45.904 INFO:tasks.workunit.client.1.smithi169.stdout:6/276: symlink d0/d3c/d40/d52/l60 0 2024-03-04T13:42:45.912 INFO:tasks.workunit.client.1.smithi169.stdout:8/271: readlink d7/d10/l39 0 2024-03-04T13:42:45.916 INFO:tasks.workunit.client.1.smithi169.stdout:8/272: write d7/d10/d22/f25 [4145214,7610] 0 2024-03-04T13:42:45.919 INFO:tasks.workunit.client.1.smithi169.stdout:9/319: dread d7/dc/fd [0,4194304] 0 2024-03-04T13:42:45.922 INFO:tasks.workunit.client.1.smithi169.stdout:3/237: rmdir d7/de/d2b/d39 39 2024-03-04T13:42:45.926 INFO:tasks.workunit.client.1.smithi169.stdout:5/295: rename d1/d16/d2a/d46/f4e to d1/d16/d17/d2d/f54 0 2024-03-04T13:42:45.927 INFO:tasks.workunit.client.1.smithi169.stdout:7/316: creat d2/d15/d17/f6e x:0 0 0 2024-03-04T13:42:45.930 INFO:tasks.workunit.client.1.smithi169.stdout:7/317: read d2/de/f53 [1852783,126300] 0 2024-03-04T13:42:45.931 INFO:tasks.workunit.client.1.smithi169.stdout:2/259: truncate d1/d5/d19/f2e 525691 0 2024-03-04T13:42:45.933 INFO:tasks.workunit.client.1.smithi169.stdout:0/227: rmdir d5/d9/dc/d1c 39 2024-03-04T13:42:45.940 INFO:tasks.workunit.client.1.smithi169.stdout:8/273: mkdir d7/d10/d17/d44/d5d 0 2024-03-04T13:42:45.942 INFO:tasks.workunit.client.1.smithi169.stdout:8/274: chown d7/d10/d24/d5c 214457 1 2024-03-04T13:42:45.943 INFO:tasks.workunit.client.1.smithi169.stdout:9/320: mkdir d7/d2d/d48/d67 0 2024-03-04T13:42:45.944 INFO:tasks.workunit.client.1.smithi169.stdout:8/275: fsync d7/f4d 0 2024-03-04T13:42:45.946 INFO:tasks.workunit.client.1.smithi169.stdout:8/276: chown d7/d10/d17/d44/d36/l3d 119227694 1 2024-03-04T13:42:45.948 INFO:tasks.workunit.client.1.smithi169.stdout:8/277: fsync d7/d10/d22/f30 0 2024-03-04T13:42:45.953 INFO:tasks.workunit.client.1.smithi169.stdout:5/296: mknod d1/d16/d17/d2d/c55 0 2024-03-04T13:42:45.959 INFO:tasks.workunit.client.1.smithi169.stdout:2/260: truncate d1/d5/d22/f32 4173694 0 2024-03-04T13:42:45.966 INFO:tasks.workunit.client.1.smithi169.stdout:3/238: dread f5 [0,4194304] 0 2024-03-04T13:42:45.966 INFO:tasks.workunit.client.1.smithi169.stdout:7/318: dread d2/de/f1e [0,4194304] 0 2024-03-04T13:42:45.967 INFO:tasks.workunit.client.1.smithi169.stdout:9/321: symlink d7/d27/l68 0 2024-03-04T13:42:45.970 INFO:tasks.workunit.client.1.smithi169.stdout:3/239: chown d7/de/d14/d18/d21/d3f/l42 118001 1 2024-03-04T13:42:45.971 INFO:tasks.workunit.client.1.smithi169.stdout:8/278: dwrite d7/d12/f15 [4194304,4194304] 0 2024-03-04T13:42:45.972 INFO:tasks.workunit.client.1.smithi169.stdout:3/240: chown d7/de/d14/d18/d21/d2c/f3a 2 1 2024-03-04T13:42:45.974 INFO:tasks.workunit.client.1.smithi169.stdout:3/241: fdatasync d7/de/d14/d18/d21/d2c/f3a 0 2024-03-04T13:42:45.979 INFO:tasks.workunit.client.1.smithi169.stdout:1/249: fsync d4/d1d/f40 0 2024-03-04T13:42:45.982 INFO:tasks.workunit.client.1.smithi169.stdout:3/242: dread d7/de/d14/d13/d1a/d2d/f37 [0,4194304] 0 2024-03-04T13:42:45.985 INFO:tasks.workunit.client.1.smithi169.stdout:3/243: chown d7/de/d14/d13/d1a/d2d 6899 1 2024-03-04T13:42:45.987 INFO:tasks.workunit.client.1.smithi169.stdout:7/319: mkdir d2/d15/d17/d6f 0 2024-03-04T13:42:45.991 INFO:tasks.workunit.client.1.smithi169.stdout:8/279: rename d7/d10/d17/f29 to d7/d12/f5e 0 2024-03-04T13:42:45.999 INFO:tasks.workunit.client.1.smithi169.stdout:2/261: mkdir d1/d5/d19/d68 0 2024-03-04T13:42:46.002 INFO:tasks.workunit.client.1.smithi169.stdout:3/244: dread d7/f36 [0,4194304] 0 2024-03-04T13:42:46.005 INFO:tasks.workunit.client.1.smithi169.stdout:7/320: mkdir d2/d28/d70 0 2024-03-04T13:42:46.005 INFO:tasks.workunit.client.1.smithi169.stdout:3/245: write d7/de/d14/d18/f30 [3472117,39295] 0 2024-03-04T13:42:46.008 INFO:tasks.workunit.client.1.smithi169.stdout:8/280: creat d7/d10/d24/d3e/f5f x:0 0 0 2024-03-04T13:42:46.008 INFO:tasks.workunit.client.1.smithi169.stdout:7/321: write d2/f22 [2379896,18633] 0 2024-03-04T13:42:46.009 INFO:tasks.workunit.client.1.smithi169.stdout:0/228: getdents d5/d9/dc/d12/d15 0 2024-03-04T13:42:46.010 INFO:tasks.workunit.client.1.smithi169.stdout:7/322: chown d2/f34 43970174 1 2024-03-04T13:42:46.013 INFO:tasks.workunit.client.1.smithi169.stdout:1/250: creat d4/d13/d26/d4f/f51 x:0 0 0 2024-03-04T13:42:46.016 INFO:tasks.workunit.client.1.smithi169.stdout:2/262: dread d1/d5/d22/d45/d4a/f5d [0,4194304] 0 2024-03-04T13:42:46.021 INFO:tasks.workunit.client.1.smithi169.stdout:3/246: creat d7/de/d14/d13/f50 x:0 0 0 2024-03-04T13:42:46.023 INFO:tasks.workunit.client.1.smithi169.stdout:3/247: dread - d7/de/d14/d18/f46 zero size 2024-03-04T13:42:46.024 INFO:tasks.workunit.client.1.smithi169.stdout:8/281: rmdir d7/d10/d22/d46/d49 39 2024-03-04T13:42:46.027 INFO:tasks.workunit.client.1.smithi169.stdout:7/323: creat d2/de/d3e/d41/f71 x:0 0 0 2024-03-04T13:42:46.034 INFO:tasks.workunit.client.1.smithi169.stdout:3/248: creat d7/de/d14/d18/d21/d2c/f51 x:0 0 0 2024-03-04T13:42:46.038 INFO:tasks.workunit.client.1.smithi169.stdout:2/263: mkdir d1/d5/d19/d69 0 2024-03-04T13:42:46.041 INFO:tasks.workunit.client.1.smithi169.stdout:2/264: write d1/d5/d19/d36/f4b [27332,48994] 0 2024-03-04T13:42:46.043 INFO:tasks.workunit.client.1.smithi169.stdout:2/265: chown d1/d5/l20 0 1 2024-03-04T13:42:46.044 INFO:tasks.workunit.client.1.smithi169.stdout:1/251: creat d4/d5/d36/d35/f52 x:0 0 0 2024-03-04T13:42:46.052 INFO:tasks.workunit.client.1.smithi169.stdout:2/266: fdatasync d1/d5/d19/d1e/f5e 0 2024-03-04T13:42:46.056 INFO:tasks.workunit.client.1.smithi169.stdout:3/249: creat d7/de/d2b/f52 x:0 0 0 2024-03-04T13:42:46.057 INFO:tasks.workunit.client.1.smithi169.stdout:3/250: chown d7/de/d24 7 1 2024-03-04T13:42:46.057 INFO:tasks.workunit.client.1.smithi169.stdout:7/324: getdents d2/de/d3e/d41/d5b 0 2024-03-04T13:42:46.058 INFO:tasks.workunit.client.1.smithi169.stdout:4/201: truncate d4/f7 567820 0 2024-03-04T13:42:46.060 INFO:tasks.workunit.client.1.smithi169.stdout:9/322: write f6 [1673292,101989] 0 2024-03-04T13:42:46.067 INFO:tasks.workunit.client.1.smithi169.stdout:7/325: creat d2/d15/d17/d6f/f72 x:0 0 0 2024-03-04T13:42:46.069 INFO:tasks.workunit.client.1.smithi169.stdout:7/326: readlink d2/d28/l2e 0 2024-03-04T13:42:46.072 INFO:tasks.workunit.client.1.smithi169.stdout:4/202: link d4/fa d4/d11/d2e/f4a 0 2024-03-04T13:42:46.074 INFO:tasks.workunit.client.1.smithi169.stdout:2/267: dwrite d1/f3d [0,4194304] 0 2024-03-04T13:42:46.076 INFO:tasks.workunit.client.1.smithi169.stdout:2/268: read - d1/d5/d22/d3b/f4c zero size 2024-03-04T13:42:46.078 INFO:tasks.workunit.client.1.smithi169.stdout:2/269: fsync d1/f50 0 2024-03-04T13:42:46.080 INFO:tasks.workunit.client.1.smithi169.stdout:7/327: creat d2/d28/d70/f73 x:0 0 0 2024-03-04T13:42:46.082 INFO:tasks.workunit.client.1.smithi169.stdout:3/251: dwrite d7/de/d14/d13/f50 [0,4194304] 0 2024-03-04T13:42:46.086 INFO:tasks.workunit.client.1.smithi169.stdout:8/282: truncate d7/d12/f5e 2910593 0 2024-03-04T13:42:46.086 INFO:tasks.workunit.client.1.smithi169.stdout:3/252: chown d7/de/d14/d13 79665155 1 2024-03-04T13:42:46.088 INFO:tasks.workunit.client.1.smithi169.stdout:8/283: dread - d7/d12/f3a zero size 2024-03-04T13:42:46.090 INFO:tasks.workunit.client.1.smithi169.stdout:2/270: rmdir d1/d5/d22 39 2024-03-04T13:42:46.099 INFO:tasks.workunit.client.1.smithi169.stdout:3/253: mkdir d7/d53 0 2024-03-04T13:42:46.108 INFO:tasks.workunit.client.1.smithi169.stdout:9/323: dread d7/dc/f1d [0,4194304] 0 2024-03-04T13:42:46.109 INFO:tasks.workunit.client.1.smithi169.stdout:4/203: dread d4/d1c/f2c [0,4194304] 0 2024-03-04T13:42:46.111 INFO:tasks.workunit.client.1.smithi169.stdout:2/271: creat d1/d5/d19/f6a x:0 0 0 2024-03-04T13:42:46.116 INFO:tasks.workunit.client.1.smithi169.stdout:2/272: write d1/d5/f9 [4779867,24964] 0 2024-03-04T13:42:46.118 INFO:tasks.workunit.client.1.smithi169.stdout:7/328: dwrite d2/d15/d17/f1d [0,4194304] 0 2024-03-04T13:42:46.119 INFO:tasks.workunit.client.1.smithi169.stdout:1/252: dwrite d4/d1d/f23 [0,4194304] 0 2024-03-04T13:42:46.121 INFO:tasks.workunit.client.1.smithi169.stdout:9/324: creat d7/d25/d47/d56/f69 x:0 0 0 2024-03-04T13:42:46.124 INFO:tasks.workunit.client.1.smithi169.stdout:9/325: write d7/dc/d12/d32/d42/f58 [520977,63227] 0 2024-03-04T13:42:46.124 INFO:tasks.workunit.client.1.smithi169.stdout:1/253: write d4/d5/d36/d35/d16/d3a/f3f [499054,44356] 0 2024-03-04T13:42:46.126 INFO:tasks.workunit.client.1.smithi169.stdout:2/273: write d1/d5/d22/d45/f62 [170752,19346] 0 2024-03-04T13:42:46.126 INFO:tasks.workunit.client.1.smithi169.stdout:9/326: write d7/dc/fd [752120,53273] 0 2024-03-04T13:42:46.132 INFO:tasks.workunit.client.1.smithi169.stdout:8/284: truncate d7/d10/d24/f3c 1462357 0 2024-03-04T13:42:46.137 INFO:tasks.workunit.client.1.smithi169.stdout:1/254: creat d4/d1d/f53 x:0 0 0 2024-03-04T13:42:46.143 INFO:tasks.workunit.client.1.smithi169.stdout:9/327: rmdir d7/d25/d47 39 2024-03-04T13:42:46.151 INFO:tasks.workunit.client.1.smithi169.stdout:7/329: dwrite d2/de/f1e [0,4194304] 0 2024-03-04T13:42:46.151 INFO:tasks.workunit.client.1.smithi169.stdout:1/255: mkdir d4/d13/d26/d54 0 2024-03-04T13:42:46.155 INFO:tasks.workunit.client.1.smithi169.stdout:3/254: dwrite f5 [0,4194304] 0 2024-03-04T13:42:46.157 INFO:tasks.workunit.client.1.smithi169.stdout:9/328: mknod d7/d2d/c6a 0 2024-03-04T13:42:46.157 INFO:tasks.workunit.client.1.smithi169.stdout:7/330: write d2/fa [1764753,23288] 0 2024-03-04T13:42:46.158 INFO:tasks.workunit.client.1.smithi169.stdout:1/256: dread - d4/d13/d26/f2e zero size 2024-03-04T13:42:46.158 INFO:tasks.workunit.client.1.smithi169.stdout:8/285: dread d7/d12/f15 [4194304,4194304] 0 2024-03-04T13:42:46.162 INFO:tasks.workunit.client.1.smithi169.stdout:3/255: chown d7/de/d14/d18/d21/d2c/f3a 1 1 2024-03-04T13:42:46.165 INFO:tasks.workunit.client.1.smithi169.stdout:1/257: fsync d4/d5/d36/d35/d16/d3a/f3f 0 2024-03-04T13:42:46.165 INFO:tasks.workunit.client.1.smithi169.stdout:7/331: mknod d2/de/d37/c74 0 2024-03-04T13:42:46.169 INFO:tasks.workunit.client.1.smithi169.stdout:2/274: write d1/d5/d22/f32 [2073227,121774] 0 2024-03-04T13:42:46.173 INFO:tasks.workunit.client.1.smithi169.stdout:9/329: creat d7/f6b x:0 0 0 2024-03-04T13:42:46.174 INFO:tasks.workunit.client.1.smithi169.stdout:2/275: write d1/f3 [9167236,26132] 0 2024-03-04T13:42:46.176 INFO:tasks.workunit.client.1.smithi169.stdout:7/332: rename d2/de/d37/c45 to d2/de/d3e/d41/c75 0 2024-03-04T13:42:46.176 INFO:tasks.workunit.client.1.smithi169.stdout:3/256: symlink d7/de/d14/d47/l54 0 2024-03-04T13:42:46.179 INFO:tasks.workunit.client.1.smithi169.stdout:2/276: write d1/d5/d19/d36/f4b [14162,125635] 0 2024-03-04T13:42:46.179 INFO:tasks.workunit.client.1.smithi169.stdout:3/257: readlink d7/de/d14/d13/d1a/l1c 0 2024-03-04T13:42:46.180 INFO:tasks.workunit.client.1.smithi169.stdout:1/258: symlink d4/d1d/l55 0 2024-03-04T13:42:46.183 INFO:tasks.workunit.client.1.smithi169.stdout:1/259: write d4/f20 [2756306,24112] 0 2024-03-04T13:42:46.183 INFO:tasks.workunit.client.1.smithi169.stdout:3/258: truncate d7/de/d14/f48 913442 0 2024-03-04T13:42:46.184 INFO:tasks.workunit.client.1.smithi169.stdout:9/330: unlink d7/d25/d3c/l38 0 2024-03-04T13:42:46.186 INFO:tasks.workunit.client.1.smithi169.stdout:7/333: creat d2/d6b/f76 x:0 0 0 2024-03-04T13:42:46.187 INFO:tasks.workunit.client.1.smithi169.stdout:1/260: dread - d4/d5/d36/d35/d16/f4d zero size 2024-03-04T13:42:46.188 INFO:tasks.workunit.client.1.smithi169.stdout:9/331: fdatasync d7/dc/d12/d32/d42/f58 0 2024-03-04T13:42:46.189 INFO:tasks.workunit.client.1.smithi169.stdout:2/277: rmdir d1/d5/d19 39 2024-03-04T13:42:46.192 INFO:tasks.workunit.client.1.smithi169.stdout:1/261: read d4/f1b [2437368,110607] 0 2024-03-04T13:42:46.195 INFO:tasks.workunit.client.1.smithi169.stdout:1/262: write d4/d13/d26/d4f/f51 [658340,122188] 0 2024-03-04T13:42:46.198 INFO:tasks.workunit.client.1.smithi169.stdout:7/334: dread d2/f2d [0,4194304] 0 2024-03-04T13:42:46.199 INFO:tasks.workunit.client.1.smithi169.stdout:7/335: chown d2/f7 86197 1 2024-03-04T13:42:46.200 INFO:tasks.workunit.client.1.smithi169.stdout:2/278: chown d1/d5/d19/f6a 36 1 2024-03-04T13:42:46.201 INFO:tasks.workunit.client.1.smithi169.stdout:2/279: chown d1/l17 3691 1 2024-03-04T13:42:46.203 INFO:tasks.workunit.client.1.smithi169.stdout:9/332: creat d7/dc/d3e/f6c x:0 0 0 2024-03-04T13:42:46.205 INFO:tasks.workunit.client.1.smithi169.stdout:3/259: symlink d7/de/d2b/d39/l55 0 2024-03-04T13:42:46.207 INFO:tasks.workunit.client.1.smithi169.stdout:8/286: link d7/d10/d22/c4b d7/d10/d17/c60 0 2024-03-04T13:42:46.211 INFO:tasks.workunit.client.1.smithi169.stdout:2/280: write d1/f40 [725503,117563] 0 2024-03-04T13:42:46.211 INFO:tasks.workunit.client.1.smithi169.stdout:1/263: getdents d4/d13/d26/d54 0 2024-03-04T13:42:46.212 INFO:tasks.workunit.client.1.smithi169.stdout:3/260: mkdir d7/de/d14/d13/d1a/d2d/d56 0 2024-03-04T13:42:46.215 INFO:tasks.workunit.client.1.smithi169.stdout:1/264: readlink d4/d1d/l4c 0 2024-03-04T13:42:46.219 INFO:tasks.workunit.client.1.smithi169.stdout:3/261: symlink d7/de/l57 0 2024-03-04T13:42:46.222 INFO:tasks.workunit.client.1.smithi169.stdout:1/265: truncate d4/d5/d36/d35/f52 89445 0 2024-03-04T13:42:46.222 INFO:tasks.workunit.client.1.smithi169.stdout:7/336: getdents d2 0 2024-03-04T13:42:46.223 INFO:tasks.workunit.client.1.smithi169.stdout:3/262: chown d7/de/d14/d18 54739 1 2024-03-04T13:42:46.227 INFO:tasks.workunit.client.1.smithi169.stdout:3/263: write d7/de/d14/d13/f50 [4881589,41929] 0 2024-03-04T13:42:46.228 INFO:tasks.workunit.client.1.smithi169.stdout:8/287: link d7/d10/d17/l2f d7/d10/d24/d3e/l61 0 2024-03-04T13:42:46.230 INFO:tasks.workunit.client.1.smithi169.stdout:7/337: symlink d2/d15/d17/l77 0 2024-03-04T13:42:46.231 INFO:tasks.workunit.client.1.smithi169.stdout:9/333: dwrite d7/dc/f1d [0,4194304] 0 2024-03-04T13:42:46.232 INFO:tasks.workunit.client.1.smithi169.stdout:8/288: dread - d7/d10/d24/d3e/f57 zero size 2024-03-04T13:42:46.232 INFO:tasks.workunit.client.1.smithi169.stdout:3/264: chown d7/de/d2b/d39/c43 1583479139 1 2024-03-04T13:42:46.236 INFO:tasks.workunit.client.1.smithi169.stdout:1/266: creat d4/d5/d36/d35/f56 x:0 0 0 2024-03-04T13:42:46.239 INFO:tasks.workunit.client.1.smithi169.stdout:2/281: dwrite d1/d5/d19/d1e/f5c [0,4194304] 0 2024-03-04T13:42:46.242 INFO:tasks.workunit.client.1.smithi169.stdout:2/282: stat d1/d5/d19/d1e/f5c 0 2024-03-04T13:42:46.242 INFO:tasks.workunit.client.1.smithi169.stdout:8/289: mkdir d7/d10/d17/d44/d36/d62 0 2024-03-04T13:42:46.242 INFO:tasks.workunit.client.1.smithi169.stdout:1/267: unlink d4/c30 0 2024-03-04T13:42:46.245 INFO:tasks.workunit.client.1.smithi169.stdout:1/268: write d4/d1d/f53 [811827,96652] 0 2024-03-04T13:42:46.245 INFO:tasks.workunit.client.1.smithi169.stdout:8/290: write d7/f4d [2059133,120898] 0 2024-03-04T13:42:46.246 INFO:tasks.workunit.client.1.smithi169.stdout:3/265: mknod d7/de/d14/d13/d1a/d2d/d56/c58 0 2024-03-04T13:42:46.247 INFO:tasks.workunit.client.1.smithi169.stdout:2/283: write d1/d5/d19/d1e/f54 [2272239,32804] 0 2024-03-04T13:42:46.248 INFO:tasks.workunit.client.1.smithi169.stdout:1/269: dread - d4/d5/d36/d35/d16/f4d zero size 2024-03-04T13:42:46.249 INFO:tasks.workunit.client.1.smithi169.stdout:9/334: getdents d7/dc/d12/d32 0 2024-03-04T13:42:46.250 INFO:tasks.workunit.client.1.smithi169.stdout:2/284: dread - d1/d5/f67 zero size 2024-03-04T13:42:46.251 INFO:tasks.workunit.client.1.smithi169.stdout:1/270: fdatasync d4/d13/f37 0 2024-03-04T13:42:46.251 INFO:tasks.workunit.client.1.smithi169.stdout:8/291: symlink d7/l63 0 2024-03-04T13:42:46.253 INFO:tasks.workunit.client.1.smithi169.stdout:1/271: readlink d4/d5/l31 0 2024-03-04T13:42:46.255 INFO:tasks.workunit.client.1.smithi169.stdout:1/272: dread - d4/d5/d36/d35/f56 zero size 2024-03-04T13:42:46.256 INFO:tasks.workunit.client.1.smithi169.stdout:3/266: rename d7/f3b to d7/de/d14/d47/f59 0 2024-03-04T13:42:46.264 INFO:tasks.workunit.client.1.smithi169.stdout:1/273: creat d4/d5/f57 x:0 0 0 2024-03-04T13:42:46.266 INFO:tasks.workunit.client.1.smithi169.stdout:3/267: unlink d7/de/d14/d13/d1a/d2d/d56/c58 0 2024-03-04T13:42:46.267 INFO:tasks.workunit.client.1.smithi169.stdout:9/335: dread d7/dc/d12/d32/d42/f43 [0,4194304] 0 2024-03-04T13:42:46.269 INFO:tasks.workunit.client.1.smithi169.stdout:9/336: fdatasync d7/d2d/f36 0 2024-03-04T13:42:46.270 INFO:tasks.workunit.client.1.smithi169.stdout:3/268: mkdir d7/de/d14/d13/d5a 0 2024-03-04T13:42:46.272 INFO:tasks.workunit.client.1.smithi169.stdout:3/269: chown d7/de/d14/l10 196808 1 2024-03-04T13:42:46.282 INFO:tasks.workunit.client.1.smithi169.stdout:1/274: dwrite d4/d13/d26/f2e [0,4194304] 0 2024-03-04T13:42:46.284 INFO:tasks.workunit.client.1.smithi169.stdout:9/337: dread d7/d2d/f36 [0,4194304] 0 2024-03-04T13:42:46.287 INFO:tasks.workunit.client.1.smithi169.stdout:1/275: unlink d4/d13/l3e 0 2024-03-04T13:42:46.291 INFO:tasks.workunit.client.1.smithi169.stdout:9/338: unlink d7/d2d/l39 0 2024-03-04T13:42:46.295 INFO:tasks.workunit.client.1.smithi169.stdout:9/339: creat d7/dc/d12/d59/f6d x:0 0 0 2024-03-04T13:42:46.296 INFO:tasks.workunit.client.1.smithi169.stdout:9/340: write d7/d2d/f40 [1115422,11710] 0 2024-03-04T13:42:46.300 INFO:tasks.workunit.client.1.smithi169.stdout:9/341: creat d7/d25/d47/d56/f6e x:0 0 0 2024-03-04T13:42:46.304 INFO:tasks.workunit.client.1.smithi169.stdout:9/342: getdents d7/d25/d47 0 2024-03-04T13:42:46.306 INFO:tasks.workunit.client.1.smithi169.stdout:8/292: dread d7/d10/d24/d3e/f45 [0,4194304] 0 2024-03-04T13:42:46.310 INFO:tasks.workunit.client.1.smithi169.stdout:8/293: mknod d7/d10/d17/d44/d36/c64 0 2024-03-04T13:42:46.314 INFO:tasks.workunit.client.1.smithi169.stdout:8/294: unlink d7/d10/d24/d3e/c43 0 2024-03-04T13:42:46.319 INFO:tasks.workunit.client.1.smithi169.stdout:9/343: dwrite d7/d2d/f36 [0,4194304] 0 2024-03-04T13:42:46.321 INFO:tasks.workunit.client.1.smithi169.stdout:9/344: write d7/d27/f5a [471815,72791] 0 2024-03-04T13:42:46.325 INFO:tasks.workunit.client.1.smithi169.stdout:9/345: creat d7/d2d/d48/f6f x:0 0 0 2024-03-04T13:42:46.328 INFO:tasks.workunit.client.1.smithi169.stdout:9/346: write d7/dc/d12/d20/f24 [449117,127904] 0 2024-03-04T13:42:46.333 INFO:tasks.workunit.client.1.smithi169.stdout:9/347: rename d7/dc/c14 to d7/dc/d12/d32/c70 0 2024-03-04T13:42:46.339 INFO:tasks.workunit.client.1.smithi169.stdout:8/295: dwrite d7/d10/d24/f3c [0,4194304] 0 2024-03-04T13:42:46.344 INFO:tasks.workunit.client.1.smithi169.stdout:8/296: symlink d7/d10/d17/d44/d36/d62/l65 0 2024-03-04T13:42:46.353 INFO:tasks.workunit.client.1.smithi169.stdout:8/297: truncate d7/d10/d22/f25 1238589 0 2024-03-04T13:42:46.359 INFO:tasks.workunit.client.1.smithi169.stdout:8/298: unlink d7/d10/d17/d44/d36/l3d 0 2024-03-04T13:42:46.362 INFO:tasks.workunit.client.1.smithi169.stdout:8/299: getdents d7/d10/d17/d44/d5d 0 2024-03-04T13:42:46.362 INFO:tasks.workunit.client.1.smithi169.stdout:8/300: readlink d7/l63 0 2024-03-04T13:42:46.369 INFO:tasks.workunit.client.1.smithi169.stdout:8/301: dread d7/d12/f5e [0,4194304] 0 2024-03-04T13:42:46.376 INFO:tasks.workunit.client.1.smithi169.stdout:8/302: link d7/d10/d17/l2f d7/d10/d24/d5c/l66 0 2024-03-04T13:42:46.387 INFO:tasks.workunit.client.1.smithi169.stdout:8/303: dwrite d7/d10/f1e [0,4194304] 0 2024-03-04T13:42:46.388 INFO:tasks.workunit.client.1.smithi169.stdout:8/304: fsync d7/d10/d24/d3e/f5f 0 2024-03-04T13:42:46.396 INFO:tasks.workunit.client.1.smithi169.stdout:8/305: link d7/l58 d7/l67 0 2024-03-04T13:42:46.412 INFO:tasks.workunit.client.1.smithi169.stdout:8/306: dread d7/d10/d24/d3e/f33 [0,4194304] 0 2024-03-04T13:42:46.414 INFO:tasks.workunit.client.1.smithi169.stdout:8/307: fdatasync d7/d10/d24/d3e/f40 0 2024-03-04T13:42:46.423 INFO:tasks.workunit.client.1.smithi169.stdout:8/308: dread d7/d10/d22/f25 [0,4194304] 0 2024-03-04T13:42:46.437 INFO:tasks.workunit.client.1.smithi169.stdout:8/309: dwrite d7/d10/d22/f30 [0,4194304] 0 2024-03-04T13:42:46.451 INFO:tasks.workunit.client.1.smithi169.stdout:8/310: dwrite d7/d12/f3a [0,4194304] 0 2024-03-04T13:42:46.453 INFO:tasks.workunit.client.1.smithi169.stdout:8/311: dread - d7/d10/d22/f5b zero size 2024-03-04T13:42:46.457 INFO:tasks.workunit.client.1.smithi169.stdout:8/312: getdents d7/d10/d24/d5c 0 2024-03-04T13:42:46.461 INFO:tasks.workunit.client.1.smithi169.stdout:8/313: rename d7/d12/f3a to d7/d10/d24/d3e/f68 0 2024-03-04T13:42:46.464 INFO:tasks.workunit.client.1.smithi169.stdout:8/314: mknod d7/d10/d24/d5c/c69 0 2024-03-04T13:42:46.468 INFO:tasks.workunit.client.1.smithi169.stdout:8/315: rmdir d7/d55 0 2024-03-04T13:42:46.471 INFO:tasks.workunit.client.1.smithi169.stdout:8/316: getdents d7/d10/d22/d46 0 2024-03-04T13:42:46.536 INFO:tasks.workunit.client.1.smithi169.stdout:8/317: write d7/d10/d22/f30 [4502133,17591] 0 2024-03-04T13:42:46.541 INFO:tasks.workunit.client.1.smithi169.stdout:8/318: creat d7/f6a x:0 0 0 2024-03-04T13:42:46.546 INFO:tasks.workunit.client.1.smithi169.stdout:8/319: rename d7/d10/d24/d3e/f5f to d7/d10/d24/f6b 0 2024-03-04T13:42:46.565 INFO:tasks.workunit.client.1.smithi169.stdout:8/320: dwrite d7/d10/d17/d44/d36/f4a [0,4194304] 0 2024-03-04T13:42:46.574 INFO:tasks.workunit.client.1.smithi169.stdout:8/321: getdents d7/d10 0 2024-03-04T13:42:46.576 INFO:tasks.workunit.client.1.smithi169.stdout:8/322: write d7/f6a [173800,103190] 0 2024-03-04T13:42:46.577 INFO:tasks.workunit.client.1.smithi169.stdout:8/323: write d7/f6a [441139,87651] 0 2024-03-04T13:42:46.582 INFO:tasks.workunit.client.1.smithi169.stdout:8/324: mknod d7/d10/d17/d44/d36/d62/c6c 0 2024-03-04T13:42:46.583 INFO:tasks.workunit.client.1.smithi169.stdout:8/325: dread - d7/d10/d24/d3e/f40 zero size 2024-03-04T13:42:46.586 INFO:tasks.workunit.client.1.smithi169.stdout:8/326: creat d7/d10/d24/d3e/f6d x:0 0 0 2024-03-04T13:42:46.589 INFO:tasks.workunit.client.1.smithi169.stdout:8/327: creat d7/d10/d22/d46/f6e x:0 0 0 2024-03-04T13:42:46.594 INFO:tasks.workunit.client.1.smithi169.stdout:8/328: mknod d7/d10/d24/d3e/c6f 0 2024-03-04T13:42:46.599 INFO:tasks.workunit.client.1.smithi169.stdout:8/329: mkdir d7/d10/d17/d44/d36/d62/d70 0 2024-03-04T13:42:46.602 INFO:tasks.workunit.client.1.smithi169.stdout:8/330: truncate d7/d10/d24/d3e/f57 754074 0 2024-03-04T13:42:46.605 INFO:tasks.workunit.client.1.smithi169.stdout:8/331: creat d7/d10/d24/f71 x:0 0 0 2024-03-04T13:42:46.617 INFO:tasks.workunit.client.1.smithi169.stdout:8/332: truncate d7/d10/d24/f3c 3532896 0 2024-03-04T13:42:46.622 INFO:tasks.workunit.client.1.smithi169.stdout:8/333: rename d7/d10/d17/d44/d4c/l52 to d7/d10/d17/d44/d4c/l72 0 2024-03-04T13:42:46.645 INFO:tasks.workunit.client.1.smithi169.stdout:8/334: dwrite d7/d10/d24/d3e/f45 [0,4194304] 0 2024-03-04T13:42:46.651 INFO:tasks.workunit.client.1.smithi169.stdout:8/335: rename d7/d10/d17 to d7/d10/d17/d44/d73 22 2024-03-04T13:42:46.653 INFO:tasks.workunit.client.1.smithi169.stdout:8/336: readlink d7/d10/d24/d5c/l66 0 2024-03-04T13:42:46.668 INFO:tasks.workunit.client.1.smithi169.stdout:6/277: sync 2024-03-04T13:42:46.673 INFO:tasks.workunit.client.1.smithi169.stdout:6/278: symlink d0/d32/d3a/d5e/l61 0 2024-03-04T13:42:46.676 INFO:tasks.workunit.client.1.smithi169.stdout:6/279: creat d0/d3c/d40/d52/d56/f62 x:0 0 0 2024-03-04T13:42:46.678 INFO:tasks.workunit.client.1.smithi169.stdout:6/280: chown d0/ld 270804 1 2024-03-04T13:42:46.692 INFO:tasks.workunit.client.1.smithi169.stdout:6/281: dwrite d0/d3c/d40/f51 [0,4194304] 0 2024-03-04T13:42:46.694 INFO:tasks.workunit.client.1.smithi169.stdout:6/282: chown d0/f12 54240 1 2024-03-04T13:42:46.698 INFO:tasks.workunit.client.1.smithi169.stdout:8/337: dread d7/d10/d17/f20 [0,4194304] 0 2024-03-04T13:42:46.704 INFO:tasks.workunit.client.1.smithi169.stdout:6/283: truncate d0/f1 2089564 0 2024-03-04T13:42:46.705 INFO:tasks.workunit.client.1.smithi169.stdout:8/338: fdatasync d7/d10/d24/f3c 0 2024-03-04T13:42:46.708 INFO:tasks.workunit.client.1.smithi169.stdout:6/284: mkdir d0/d63 0 2024-03-04T13:42:46.724 INFO:tasks.workunit.client.1.smithi169.stdout:8/339: dwrite d7/d10/d17/f2a [4194304,4194304] 0 2024-03-04T13:42:46.728 INFO:tasks.workunit.client.1.smithi169.stdout:8/340: chown d7/d10/d22/c4b 63320510 1 2024-03-04T13:42:46.729 INFO:tasks.workunit.client.1.smithi169.stdout:8/341: chown d7/ca 211242748 1 2024-03-04T13:42:46.732 INFO:tasks.workunit.client.1.smithi169.stdout:8/342: write d7/d10/d17/f20 [3497973,54707] 0 2024-03-04T13:42:46.738 INFO:tasks.workunit.client.1.smithi169.stdout:8/343: creat d7/d10/d17/f74 x:0 0 0 2024-03-04T13:42:46.739 INFO:tasks.workunit.client.1.smithi169.stdout:6/285: dread d0/f4e [0,4194304] 0 2024-03-04T13:42:46.741 INFO:tasks.workunit.client.1.smithi169.stdout:8/344: mkdir d7/d10/d22/d75 0 2024-03-04T13:42:46.744 INFO:tasks.workunit.client.1.smithi169.stdout:6/286: rename d0/ca to d0/d63/c64 0 2024-03-04T13:42:46.745 INFO:tasks.workunit.client.1.smithi169.stdout:8/345: mknod d7/d10/d17/d44/d36/d62/d70/c76 0 2024-03-04T13:42:46.748 INFO:tasks.workunit.client.1.smithi169.stdout:8/346: dread - d7/d10/d17/f74 zero size 2024-03-04T13:42:46.748 INFO:tasks.workunit.client.1.smithi169.stdout:6/287: rename d0/f37 to d0/d32/d3a/f65 0 2024-03-04T13:42:46.750 INFO:tasks.workunit.client.1.smithi169.stdout:8/347: mknod d7/d10/d17/d44/c77 0 2024-03-04T13:42:46.751 INFO:tasks.workunit.client.1.smithi169.stdout:6/288: creat d0/d32/d3e/f66 x:0 0 0 2024-03-04T13:42:46.752 INFO:tasks.workunit.client.1.smithi169.stdout:8/348: chown d7/l63 16251 1 2024-03-04T13:42:46.753 INFO:tasks.workunit.client.1.smithi169.stdout:6/289: fdatasync d0/f29 0 2024-03-04T13:42:46.754 INFO:tasks.workunit.client.1.smithi169.stdout:8/349: truncate d7/d10/d24/f50 479238 0 2024-03-04T13:42:46.757 INFO:tasks.workunit.client.1.smithi169.stdout:8/350: write d7/d10/d24/d3e/f40 [567817,47706] 0 2024-03-04T13:42:46.793 INFO:tasks.workunit.client.1.smithi169.stdout:8/351: truncate d7/d12/f5e 4080971 0 2024-03-04T13:42:46.799 INFO:tasks.workunit.client.1.smithi169.stdout:8/352: link d7/ld d7/d10/d24/d3e/l78 0 2024-03-04T13:42:46.818 INFO:tasks.workunit.client.1.smithi169.stdout:8/353: dread d7/d10/d17/f2a [0,4194304] 0 2024-03-04T13:42:46.822 INFO:tasks.workunit.client.1.smithi169.stdout:8/354: mkdir d7/d10/d22/d75/d79 0 2024-03-04T13:42:46.825 INFO:tasks.workunit.client.1.smithi169.stdout:8/355: read d7/d10/d24/d3e/f40 [229246,49045] 0 2024-03-04T13:42:46.826 INFO:tasks.workunit.client.1.smithi169.stdout:8/356: stat d7/d12/l32 0 2024-03-04T13:42:46.828 INFO:tasks.workunit.client.1.smithi169.stdout:8/357: chown d7/ca 17904 1 2024-03-04T13:42:46.834 INFO:tasks.workunit.client.1.smithi169.stdout:8/358: getdents d7/d10/d17/d44/d36/d62/d70 0 2024-03-04T13:42:46.840 INFO:tasks.workunit.client.1.smithi169.stdout:8/359: dread d7/d10/d24/d3e/f40 [0,4194304] 0 2024-03-04T13:42:46.858 INFO:tasks.workunit.client.1.smithi169.stdout:8/360: dwrite d7/f4d [0,4194304] 0 2024-03-04T13:42:46.883 INFO:tasks.workunit.client.1.smithi169.stdout:8/361: dwrite d7/d10/d24/f6b [0,4194304] 0 2024-03-04T13:42:46.890 INFO:tasks.workunit.client.1.smithi169.stdout:8/362: creat d7/d10/d24/f7a x:0 0 0 2024-03-04T13:42:46.892 INFO:tasks.workunit.client.1.smithi169.stdout:3/270: sync 2024-03-04T13:42:46.892 INFO:tasks.workunit.client.1.smithi169.stdout:4/204: sync 2024-03-04T13:42:46.892 INFO:tasks.workunit.client.1.smithi169.stdout:7/338: sync 2024-03-04T13:42:46.892 INFO:tasks.workunit.client.1.smithi169.stdout:2/285: sync 2024-03-04T13:42:46.893 INFO:tasks.workunit.client.1.smithi169.stdout:6/290: sync 2024-03-04T13:42:46.893 INFO:tasks.workunit.client.1.smithi169.stdout:5/297: sync 2024-03-04T13:42:46.894 INFO:tasks.workunit.client.1.smithi169.stdout:1/276: sync 2024-03-04T13:42:46.894 INFO:tasks.workunit.client.1.smithi169.stdout:9/348: sync 2024-03-04T13:42:46.894 INFO:tasks.workunit.client.1.smithi169.stdout:0/229: sync 2024-03-04T13:42:46.895 INFO:tasks.workunit.client.1.smithi169.stdout:7/339: chown d2/fa 320 1 2024-03-04T13:42:46.898 INFO:tasks.workunit.client.1.smithi169.stdout:4/205: stat d4/d1c/c1f 0 2024-03-04T13:42:46.900 INFO:tasks.workunit.client.1.smithi169.stdout:1/277: unlink d4/d1d/f53 0 2024-03-04T13:42:46.901 INFO:tasks.workunit.client.1.smithi169.stdout:7/340: read d2/de/ff [3292900,89647] 0 2024-03-04T13:42:46.903 INFO:tasks.workunit.client.1.smithi169.stdout:6/291: rename d0/d32/d3e/c48 to d0/d63/c67 0 2024-03-04T13:42:46.904 INFO:tasks.workunit.client.1.smithi169.stdout:5/298: symlink d1/d16/d2a/d46/l56 0 2024-03-04T13:42:46.907 INFO:tasks.workunit.client.1.smithi169.stdout:5/299: write d1/d16/f3b [98438,110210] 0 2024-03-04T13:42:46.912 INFO:tasks.workunit.client.1.smithi169.stdout:5/300: chown d1/d16/d17/d2d/d51 335868 1 2024-03-04T13:42:46.912 INFO:tasks.workunit.client.1.smithi169.stdout:7/341: stat d2/de/f4a 0 2024-03-04T13:42:46.913 INFO:tasks.workunit.client.1.smithi169.stdout:6/292: creat d0/d3c/f68 x:0 0 0 2024-03-04T13:42:46.916 INFO:tasks.workunit.client.1.smithi169.stdout:8/363: dwrite d7/f4d [0,4194304] 0 2024-03-04T13:42:46.918 INFO:tasks.workunit.client.1.smithi169.stdout:0/230: symlink d5/d9/dc/l55 0 2024-03-04T13:42:46.919 INFO:tasks.workunit.client.1.smithi169.stdout:5/301: write d1/d16/f3a [3263444,40571] 0 2024-03-04T13:42:46.921 INFO:tasks.workunit.client.1.smithi169.stdout:2/286: dwrite d1/f3 [0,4194304] 0 2024-03-04T13:42:46.931 INFO:tasks.workunit.client.1.smithi169.stdout:4/206: unlink d4/d11/d2e/d39/d48/f33 0 2024-03-04T13:42:46.931 INFO:tasks.workunit.client.1.smithi169.stdout:3/271: link d7/de/d14/d13/d1a/l1c d7/de/d2b/d39/l5b 0 2024-03-04T13:42:46.932 INFO:tasks.workunit.client.1.smithi169.stdout:9/349: link d7/dc/d12/d59/f6d d7/f71 0 2024-03-04T13:42:46.933 INFO:tasks.workunit.client.1.smithi169.stdout:0/231: symlink d5/d9/l56 0 2024-03-04T13:42:46.941 INFO:tasks.workunit.client.1.smithi169.stdout:6/293: truncate d0/f8 1034040 0 2024-03-04T13:42:46.943 INFO:tasks.workunit.client.1.smithi169.stdout:4/207: creat d4/d11/f4b x:0 0 0 2024-03-04T13:42:46.946 INFO:tasks.workunit.client.1.smithi169.stdout:2/287: creat d1/d5/d19/d36/f6b x:0 0 0 2024-03-04T13:42:46.947 INFO:tasks.workunit.client.1.smithi169.stdout:3/272: mkdir d7/de/d14/d18/d21/d2c/d5c 0 2024-03-04T13:42:46.949 INFO:tasks.workunit.client.1.smithi169.stdout:2/288: write d1/d5/d22/d3b/f52 [187749,107314] 0 2024-03-04T13:42:46.951 INFO:tasks.workunit.client.1.smithi169.stdout:0/232: write d5/d9/dc/d1c/f2c [1772190,94674] 0 2024-03-04T13:42:46.960 INFO:tasks.workunit.client.1.smithi169.stdout:6/294: unlink d0/f1b 0 2024-03-04T13:42:46.961 INFO:tasks.workunit.client.1.smithi169.stdout:4/208: creat d4/d11/d2e/d39/d48/f4c x:0 0 0 2024-03-04T13:42:46.963 INFO:tasks.workunit.client.1.smithi169.stdout:1/278: dread d4/d5/d36/f50 [0,4194304] 0 2024-03-04T13:42:46.967 INFO:tasks.workunit.client.1.smithi169.stdout:6/295: write d0/d32/d3a/f42 [459515,118221] 0 2024-03-04T13:42:46.975 INFO:tasks.workunit.client.1.smithi169.stdout:0/233: dwrite d5/d4c/f51 [0,4194304] 0 2024-03-04T13:42:46.978 INFO:tasks.workunit.client.1.smithi169.stdout:3/273: dwrite f5 [4194304,4194304] 0 2024-03-04T13:42:46.981 INFO:tasks.workunit.client.1.smithi169.stdout:4/209: symlink d4/d11/d3b/l4d 0 2024-03-04T13:42:46.985 INFO:tasks.workunit.client.1.smithi169.stdout:4/210: read d4/d11/f1b [797765,78656] 0 2024-03-04T13:42:46.987 INFO:tasks.workunit.client.1.smithi169.stdout:2/289: link d1/d5/d19/d1e/d29/f39 d1/d5/d19/d3c/f6c 0 2024-03-04T13:42:46.990 INFO:tasks.workunit.client.1.smithi169.stdout:4/211: chown d4/d11/f30 0 1 2024-03-04T13:42:46.993 INFO:tasks.workunit.client.1.smithi169.stdout:6/296: dwrite d0/f16 [0,4194304] 0 2024-03-04T13:42:47.006 INFO:tasks.workunit.client.1.smithi169.stdout:0/234: creat d5/d9/dc/d12/d15/d46/f57 x:0 0 0 2024-03-04T13:42:47.007 INFO:tasks.workunit.client.1.smithi169.stdout:3/274: dwrite d7/de/d14/d13/f50 [0,4194304] 0 2024-03-04T13:42:47.008 INFO:tasks.workunit.client.1.smithi169.stdout:2/290: mkdir d1/d6d 0 2024-03-04T13:42:47.010 INFO:tasks.workunit.client.1.smithi169.stdout:0/235: stat d5/d9/d39/c3f 0 2024-03-04T13:42:47.013 INFO:tasks.workunit.client.1.smithi169.stdout:0/236: dread - d5/d9/dc/d1c/f54 zero size 2024-03-04T13:42:47.014 INFO:tasks.workunit.client.1.smithi169.stdout:9/350: link d7/d2d/c35 d7/d25/c72 0 2024-03-04T13:42:47.019 INFO:tasks.workunit.client.1.smithi169.stdout:2/291: symlink d1/l6e 0 2024-03-04T13:42:47.022 INFO:tasks.workunit.client.1.smithi169.stdout:3/275: rmdir d7/de/d14/d18/d21/d3f 39 2024-03-04T13:42:47.026 INFO:tasks.workunit.client.1.smithi169.stdout:3/276: truncate d7/de/d14/f20 782137 0 2024-03-04T13:42:47.027 INFO:tasks.workunit.client.1.smithi169.stdout:4/212: dread d4/fa [0,4194304] 0 2024-03-04T13:42:47.029 INFO:tasks.workunit.client.1.smithi169.stdout:6/297: truncate d0/f29 2251965 0 2024-03-04T13:42:47.030 INFO:tasks.workunit.client.1.smithi169.stdout:3/277: write d7/f33 [95500,5817] 0 2024-03-04T13:42:47.031 INFO:tasks.workunit.client.1.smithi169.stdout:0/237: creat d5/d9/d39/f58 x:0 0 0 2024-03-04T13:42:47.033 INFO:tasks.workunit.client.1.smithi169.stdout:1/279: getdents d4/d13/d26/d4f 0 2024-03-04T13:42:47.038 INFO:tasks.workunit.client.1.smithi169.stdout:4/213: truncate d4/d11/d23/f35 4430380 0 2024-03-04T13:42:47.040 INFO:tasks.workunit.client.1.smithi169.stdout:6/298: creat d0/d3c/d40/f69 x:0 0 0 2024-03-04T13:42:47.047 INFO:tasks.workunit.client.1.smithi169.stdout:9/351: dwrite d7/dc/d12/d32/d42/f58 [0,4194304] 0 2024-03-04T13:42:47.048 INFO:tasks.workunit.client.1.smithi169.stdout:1/280: dread d4/d5/d36/d35/df/f46 [0,4194304] 0 2024-03-04T13:42:47.049 INFO:tasks.workunit.client.1.smithi169.stdout:6/299: write d0/d32/d3a/f42 [1601678,2604] 0 2024-03-04T13:42:47.052 INFO:tasks.workunit.client.1.smithi169.stdout:4/214: fsync d4/f41 0 2024-03-04T13:42:47.054 INFO:tasks.workunit.client.1.smithi169.stdout:4/215: stat d4/f41 0 2024-03-04T13:42:47.054 INFO:tasks.workunit.client.1.smithi169.stdout:9/352: mknod d7/d27/c73 0 2024-03-04T13:42:47.056 INFO:tasks.workunit.client.1.smithi169.stdout:1/281: write d4/d5/d36/d35/f52 [207745,30587] 0 2024-03-04T13:42:47.057 INFO:tasks.workunit.client.1.smithi169.stdout:3/278: dwrite d7/de/d14/f2a [0,4194304] 0 2024-03-04T13:42:47.058 INFO:tasks.workunit.client.1.smithi169.stdout:7/342: truncate d2/de/d3e/d41/f49 3577207 0 2024-03-04T13:42:47.060 INFO:tasks.workunit.client.1.smithi169.stdout:6/300: mknod d0/d32/d3a/d5e/c6a 0 2024-03-04T13:42:47.061 INFO:tasks.workunit.client.1.smithi169.stdout:4/216: write d4/d11/d2e/d39/d48/f3c [51702,78817] 0 2024-03-04T13:42:47.061 INFO:tasks.workunit.client.1.smithi169.stdout:0/238: dread d5/d9/f42 [0,4194304] 0 2024-03-04T13:42:47.062 INFO:tasks.workunit.client.1.smithi169.stdout:1/282: chown d4/d5/l31 239 1 2024-03-04T13:42:47.066 INFO:tasks.workunit.client.1.smithi169.stdout:4/217: write d4/d11/d2e/d37/f45 [785475,16440] 0 2024-03-04T13:42:47.068 INFO:tasks.workunit.client.1.smithi169.stdout:0/239: chown d5/d9/dc/d12/d15/f40 503955 1 2024-03-04T13:42:47.073 INFO:tasks.workunit.client.1.smithi169.stdout:5/302: truncate d1/d16/d17/d28/f4d 3407225 0 2024-03-04T13:42:47.076 INFO:tasks.workunit.client.1.smithi169.stdout:2/292: dread d1/d5/d19/d1e/f54 [0,4194304] 0 2024-03-04T13:42:47.086 INFO:tasks.workunit.client.1.smithi169.stdout:7/343: mkdir d2/de/d3e/d41/d69/d78 0 2024-03-04T13:42:47.088 INFO:tasks.workunit.client.1.smithi169.stdout:7/344: write d2/d28/f60 [2076096,23482] 0 2024-03-04T13:42:47.089 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:42:46 smithi107 ceph-mon[28739]: pgmap v45: 65 pgs: 65 active+clean; 586 MiB data, 3.4 GiB used, 533 GiB / 536 GiB avail; 5.4 MiB/s rd, 63 MiB/s wr, 151 op/s 2024-03-04T13:42:47.091 INFO:tasks.workunit.client.1.smithi169.stdout:4/218: dread d4/d11/f1b [0,4194304] 0 2024-03-04T13:42:47.097 INFO:tasks.workunit.client.1.smithi169.stdout:9/353: dread d7/d2d/f40 [0,4194304] 0 2024-03-04T13:42:47.099 INFO:tasks.workunit.client.1.smithi169.stdout:6/301: link d0/d3c/d40/f5c d0/d32/f6b 0 2024-03-04T13:42:47.100 INFO:tasks.workunit.client.1.smithi169.stdout:9/354: write f6 [6999046,63529] 0 2024-03-04T13:42:47.101 INFO:tasks.workunit.client.1.smithi169.stdout:6/302: fsync d0/f2b 0 2024-03-04T13:42:47.103 INFO:tasks.workunit.client.1.smithi169.stdout:9/355: chown d7/d25/c2c 456 1 2024-03-04T13:42:47.103 INFO:tasks.workunit.client.1.smithi169.stdout:1/283: rename d4/d5/c4b to d4/d5/d36/d35/c58 0 2024-03-04T13:42:47.105 INFO:tasks.workunit.client.1.smithi169.stdout:2/293: mknod d1/d5/d19/d63/c6f 0 2024-03-04T13:42:47.106 INFO:tasks.workunit.client.1.smithi169.stdout:3/279: getdents d7/de/d14/d13 0 2024-03-04T13:42:47.107 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:42:46 smithi169 ceph-mon[36313]: pgmap v45: 65 pgs: 65 active+clean; 586 MiB data, 3.4 GiB used, 533 GiB / 536 GiB avail; 5.4 MiB/s rd, 63 MiB/s wr, 151 op/s 2024-03-04T13:42:47.108 INFO:tasks.workunit.client.1.smithi169.stdout:4/219: creat d4/d11/d2e/d39/d48/f4e x:0 0 0 2024-03-04T13:42:47.109 INFO:tasks.workunit.client.1.smithi169.stdout:2/294: write d1/d5/d22/f55 [503974,120926] 0 2024-03-04T13:42:47.113 INFO:tasks.workunit.client.1.smithi169.stdout:8/364: sync 2024-03-04T13:42:47.113 INFO:tasks.workunit.client.1.smithi169.stdout:6/303: symlink d0/d32/d3e/l6c 0 2024-03-04T13:42:47.114 INFO:tasks.workunit.client.1.smithi169.stdout:9/356: mknod d7/dc/d3e/c74 0 2024-03-04T13:42:47.116 INFO:tasks.workunit.client.1.smithi169.stdout:9/357: dread - d7/dc/d12/d59/f6d zero size 2024-03-04T13:42:47.117 INFO:tasks.workunit.client.1.smithi169.stdout:0/240: rename d5/d9/d39 to d5/d9/d3d/d59 0 2024-03-04T13:42:47.118 INFO:tasks.workunit.client.1.smithi169.stdout:9/358: dread - d7/d2d/d48/f65 zero size 2024-03-04T13:42:47.119 INFO:tasks.workunit.client.1.smithi169.stdout:1/284: creat d4/d5/d36/f59 x:0 0 0 2024-03-04T13:42:47.120 INFO:tasks.workunit.client.1.smithi169.stdout:0/241: chown d5/d9/d3d/l50 1508927 1 2024-03-04T13:42:47.120 INFO:tasks.workunit.client.1.smithi169.stdout:9/359: readlink d7/dc/lf 0 2024-03-04T13:42:47.128 INFO:tasks.workunit.client.1.smithi169.stdout:4/220: truncate d4/fa 2774060 0 2024-03-04T13:42:47.129 INFO:tasks.workunit.client.1.smithi169.stdout:1/285: truncate d4/d5/d36/d35/f56 349018 0 2024-03-04T13:42:47.129 INFO:tasks.workunit.client.1.smithi169.stdout:0/242: chown d5/d9/dc/d12/d15/d19/f2e 262026 1 2024-03-04T13:42:47.135 INFO:tasks.workunit.client.1.smithi169.stdout:8/365: dread d7/d10/d17/f2a [4194304,4194304] 0 2024-03-04T13:42:47.135 INFO:tasks.workunit.client.1.smithi169.stdout:0/243: write d5/d9/d3d/d59/f58 [633197,41201] 0 2024-03-04T13:42:47.140 INFO:tasks.workunit.client.1.smithi169.stdout:2/295: rename d1/d5/d19/f27 to d1/d5/d22/d45/d4a/f70 0 2024-03-04T13:42:47.141 INFO:tasks.workunit.client.1.smithi169.stdout:7/345: dread d2/de/d37/f3f [0,4194304] 0 2024-03-04T13:42:47.142 INFO:tasks.workunit.client.1.smithi169.stdout:3/280: creat d7/d53/f5d x:0 0 0 2024-03-04T13:42:47.144 INFO:tasks.workunit.client.1.smithi169.stdout:3/281: chown f6 11 1 2024-03-04T13:42:47.146 INFO:tasks.workunit.client.1.smithi169.stdout:4/221: symlink d4/d11/l4f 0 2024-03-04T13:42:47.147 INFO:tasks.workunit.client.1.smithi169.stdout:5/303: fsync d1/d23/f40 0 2024-03-04T13:42:47.149 INFO:tasks.workunit.client.1.smithi169.stdout:4/222: fdatasync d4/d11/d2e/d39/d48/f4e 0 2024-03-04T13:42:47.149 INFO:tasks.workunit.client.1.smithi169.stdout:2/296: unlink d1/d5/d22/l26 0 2024-03-04T13:42:47.152 INFO:tasks.workunit.client.1.smithi169.stdout:9/360: creat d7/d2d/d48/d66/f75 x:0 0 0 2024-03-04T13:42:47.153 INFO:tasks.workunit.client.1.smithi169.stdout:3/282: truncate d7/de/d14/d13/d1a/f28 187992 0 2024-03-04T13:42:47.155 INFO:tasks.workunit.client.1.smithi169.stdout:7/346: creat d2/d28/d70/f79 x:0 0 0 2024-03-04T13:42:47.155 INFO:tasks.workunit.client.1.smithi169.stdout:4/223: dread - d4/d11/f4b zero size 2024-03-04T13:42:47.159 INFO:tasks.workunit.client.1.smithi169.stdout:6/304: getdents d0/d3c/d40 0 2024-03-04T13:42:47.160 INFO:tasks.workunit.client.1.smithi169.stdout:8/366: dread d7/d10/d24/d3e/f68 [0,4194304] 0 2024-03-04T13:42:47.162 INFO:tasks.workunit.client.1.smithi169.stdout:9/361: write d7/d2d/d48/f6f [133383,77872] 0 2024-03-04T13:42:47.163 INFO:tasks.workunit.client.1.smithi169.stdout:1/286: dwrite d4/d5/d36/d35/d16/d3a/f3f [0,4194304] 0 2024-03-04T13:42:47.163 INFO:tasks.workunit.client.1.smithi169.stdout:7/347: chown d2/de/d3e/l44 2726014 1 2024-03-04T13:42:47.169 INFO:tasks.workunit.client.1.smithi169.stdout:5/304: link d1/c21 d1/d23/c57 0 2024-03-04T13:42:47.171 INFO:tasks.workunit.client.1.smithi169.stdout:2/297: dread d1/d5/d22/d45/f62 [0,4194304] 0 2024-03-04T13:42:47.172 INFO:tasks.workunit.client.1.smithi169.stdout:9/362: write d7/d25/d47/d56/f6e [292702,10267] 0 2024-03-04T13:42:47.173 INFO:tasks.workunit.client.1.smithi169.stdout:2/298: stat d1/d5/l20 0 2024-03-04T13:42:47.176 INFO:tasks.workunit.client.1.smithi169.stdout:6/305: creat d0/d32/d3a/d5e/f6d x:0 0 0 2024-03-04T13:42:47.178 INFO:tasks.workunit.client.1.smithi169.stdout:2/299: write d1/d5/d19/d1e/f43 [545651,6582] 0 2024-03-04T13:42:47.182 INFO:tasks.workunit.client.1.smithi169.stdout:4/224: rmdir d4/d11/d2e/d39/d48 39 2024-03-04T13:42:47.183 INFO:tasks.workunit.client.1.smithi169.stdout:0/244: dread d5/d9/dc/d12/d15/d19/f2e [0,4194304] 0 2024-03-04T13:42:47.185 INFO:tasks.workunit.client.1.smithi169.stdout:7/348: mkdir d2/de/d3e/d41/d5b/d7a 0 2024-03-04T13:42:47.188 INFO:tasks.workunit.client.1.smithi169.stdout:0/245: truncate d5/d9/dc/d12/d15/d19/f32 215488 0 2024-03-04T13:42:47.190 INFO:tasks.workunit.client.1.smithi169.stdout:9/363: creat d7/d27/f76 x:0 0 0 2024-03-04T13:42:47.190 INFO:tasks.workunit.client.1.smithi169.stdout:3/283: write d7/f36 [602205,57885] 0 2024-03-04T13:42:47.192 INFO:tasks.workunit.client.1.smithi169.stdout:0/246: dread d5/d9/f2d [0,4194304] 0 2024-03-04T13:42:47.193 INFO:tasks.workunit.client.1.smithi169.stdout:8/367: creat d7/d10/d17/d44/d5d/f7b x:0 0 0 2024-03-04T13:42:47.195 INFO:tasks.workunit.client.1.smithi169.stdout:0/247: chown d5/d9/d3d/f41 7 1 2024-03-04T13:42:47.196 INFO:tasks.workunit.client.1.smithi169.stdout:2/300: fdatasync d1/d5/d22/d45/d4a/f5d 0 2024-03-04T13:42:47.198 INFO:tasks.workunit.client.1.smithi169.stdout:7/349: fdatasync d2/de/d3e/d41/f4c 0 2024-03-04T13:42:47.201 INFO:tasks.workunit.client.1.smithi169.stdout:0/248: write d5/d9/dc/d12/d15/d19/f28 [3413652,90749] 0 2024-03-04T13:42:47.204 INFO:tasks.workunit.client.1.smithi169.stdout:3/284: mkdir d7/de/d14/d13/d1a/d2d/d56/d5e 0 2024-03-04T13:42:47.204 INFO:tasks.workunit.client.1.smithi169.stdout:7/350: read d2/de/f1e [2667466,102770] 0 2024-03-04T13:42:47.204 INFO:tasks.workunit.client.1.smithi169.stdout:8/368: dread d7/d10/d24/d3e/f33 [0,4194304] 0 2024-03-04T13:42:47.204 INFO:tasks.workunit.client.1.smithi169.stdout:1/287: creat d4/d5/f5a x:0 0 0 2024-03-04T13:42:47.205 INFO:tasks.workunit.client.1.smithi169.stdout:1/288: chown d4/d13 281 1 2024-03-04T13:42:47.212 INFO:tasks.workunit.client.1.smithi169.stdout:6/306: link d0/f12 d0/d32/d3a/d43/f6e 0 2024-03-04T13:42:47.217 INFO:tasks.workunit.client.1.smithi169.stdout:5/305: write d1/d16/f3b [1825389,63006] 0 2024-03-04T13:42:47.222 INFO:tasks.workunit.client.1.smithi169.stdout:7/351: mknod d2/de/d3e/c7b 0 2024-03-04T13:42:47.223 INFO:tasks.workunit.client.1.smithi169.stdout:9/364: dwrite d7/d2d/f40 [4194304,4194304] 0 2024-03-04T13:42:47.225 INFO:tasks.workunit.client.1.smithi169.stdout:1/289: dwrite d4/d1d/f23 [0,4194304] 0 2024-03-04T13:42:47.228 INFO:tasks.workunit.client.1.smithi169.stdout:7/352: write d2/de/d3e/d41/f59 [71709,82394] 0 2024-03-04T13:42:47.230 INFO:tasks.workunit.client.1.smithi169.stdout:7/353: fsync d2/d15/d17/f6e 0 2024-03-04T13:42:47.231 INFO:tasks.workunit.client.1.smithi169.stdout:6/307: creat d0/d3c/d40/f6f x:0 0 0 2024-03-04T13:42:47.233 INFO:tasks.workunit.client.1.smithi169.stdout:6/308: chown d0/l11 2188079 1 2024-03-04T13:42:47.236 INFO:tasks.workunit.client.1.smithi169.stdout:5/306: symlink d1/d16/d2a/l58 0 2024-03-04T13:42:47.237 INFO:tasks.workunit.client.1.smithi169.stdout:5/307: stat d1/f42 0 2024-03-04T13:42:47.241 INFO:tasks.workunit.client.1.smithi169.stdout:4/225: getdents d4/d11/d2e 0 2024-03-04T13:42:47.242 INFO:tasks.workunit.client.1.smithi169.stdout:7/354: unlink d2/de/c1a 0 2024-03-04T13:42:47.245 INFO:tasks.workunit.client.1.smithi169.stdout:6/309: mkdir d0/d32/d70 0 2024-03-04T13:42:47.248 INFO:tasks.workunit.client.1.smithi169.stdout:2/301: truncate d1/f3d 3027873 0 2024-03-04T13:42:47.253 INFO:tasks.workunit.client.1.smithi169.stdout:5/308: symlink d1/d16/d17/d1b/l59 0 2024-03-04T13:42:47.253 INFO:tasks.workunit.client.1.smithi169.stdout:1/290: mkdir d4/d5/d36/d35/d5b 0 2024-03-04T13:42:47.253 INFO:tasks.workunit.client.1.smithi169.stdout:3/285: write d7/de/d14/d47/f59 [31167,80087] 0 2024-03-04T13:42:47.257 INFO:tasks.workunit.client.1.smithi169.stdout:4/226: creat d4/d1c/f50 x:0 0 0 2024-03-04T13:42:47.258 INFO:tasks.workunit.client.1.smithi169.stdout:6/310: dread d0/d32/d3a/d43/f45 [0,4194304] 0 2024-03-04T13:42:47.259 INFO:tasks.workunit.client.1.smithi169.stdout:8/369: rename d7/d10/d22/d46/l5a to d7/d10/d22/d46/d49/l7c 0 2024-03-04T13:42:47.263 INFO:tasks.workunit.client.1.smithi169.stdout:9/365: creat d7/f77 x:0 0 0 2024-03-04T13:42:47.264 INFO:tasks.workunit.client.1.smithi169.stdout:1/291: write d4/d5/d36/d35/d16/d3a/f3f [134160,86718] 0 2024-03-04T13:42:47.264 INFO:tasks.workunit.client.1.smithi169.stdout:5/309: mkdir d1/d16/d17/d2d/d5a 0 2024-03-04T13:42:47.264 INFO:tasks.workunit.client.1.smithi169.stdout:4/227: write d4/d11/d23/f35 [2403300,110645] 0 2024-03-04T13:42:47.266 INFO:tasks.workunit.client.1.smithi169.stdout:3/286: fsync d7/de/d14/d47/f59 0 2024-03-04T13:42:47.266 INFO:tasks.workunit.client.1.smithi169.stdout:9/366: write d7/d25/d3c/f62 [168851,90755] 0 2024-03-04T13:42:47.272 INFO:tasks.workunit.client.1.smithi169.stdout:8/370: readlink d7/d10/d17/l2f 0 2024-03-04T13:42:47.273 INFO:tasks.workunit.client.1.smithi169.stdout:1/292: rmdir d4/d1d 39 2024-03-04T13:42:47.275 INFO:tasks.workunit.client.1.smithi169.stdout:2/302: dwrite d1/f3 [0,4194304] 0 2024-03-04T13:42:47.276 INFO:tasks.workunit.client.1.smithi169.stdout:9/367: symlink d7/dc/d12/d32/d42/l78 0 2024-03-04T13:42:47.277 INFO:tasks.workunit.client.1.smithi169.stdout:5/310: dread d1/d16/d2f/f33 [0,4194304] 0 2024-03-04T13:42:47.280 INFO:tasks.workunit.client.1.smithi169.stdout:4/228: unlink d4/l47 0 2024-03-04T13:42:47.282 INFO:tasks.workunit.client.1.smithi169.stdout:2/303: write d1/d5/d19/d1e/f24 [4351691,81567] 0 2024-03-04T13:42:47.283 INFO:tasks.workunit.client.1.smithi169.stdout:3/287: mkdir d7/de/d14/d13/d1a/d2d/d56/d5e/d5f 0 2024-03-04T13:42:47.288 INFO:tasks.workunit.client.1.smithi169.stdout:3/288: truncate d7/de/d2b/d39/f4b 2083988 0 2024-03-04T13:42:47.293 INFO:tasks.workunit.client.1.smithi169.stdout:9/368: creat d7/dc/f79 x:0 0 0 2024-03-04T13:42:47.297 INFO:tasks.workunit.client.1.smithi169.stdout:9/369: write d7/dc/f79 [59452,93288] 0 2024-03-04T13:42:47.299 INFO:tasks.workunit.client.1.smithi169.stdout:5/311: creat d1/d16/d2f/d38/f5b x:0 0 0 2024-03-04T13:42:47.303 INFO:tasks.workunit.client.1.smithi169.stdout:2/304: chown d1/d5/d22/d45/c61 8118 1 2024-03-04T13:42:47.303 INFO:tasks.workunit.client.1.smithi169.stdout:8/371: write d7/d10/d22/d46/f6e [690185,36107] 0 2024-03-04T13:42:47.305 INFO:tasks.workunit.client.1.smithi169.stdout:2/305: readlink d1/l6e 0 2024-03-04T13:42:47.305 INFO:tasks.workunit.client.1.smithi169.stdout:3/289: symlink d7/de/d2b/d39/l60 0 2024-03-04T13:42:47.306 INFO:tasks.workunit.client.1.smithi169.stdout:1/293: dwrite d4/d5/f49 [0,4194304] 0 2024-03-04T13:42:47.312 INFO:tasks.workunit.client.1.smithi169.stdout:8/372: creat d7/d10/d17/d44/d4c/f7d x:0 0 0 2024-03-04T13:42:47.313 INFO:tasks.workunit.client.1.smithi169.stdout:4/229: truncate d4/d11/d2e/d39/d48/f4e 273584 0 2024-03-04T13:42:47.315 INFO:tasks.workunit.client.1.smithi169.stdout:2/306: mknod d1/d5/d22/d3b/c71 0 2024-03-04T13:42:47.318 INFO:tasks.workunit.client.1.smithi169.stdout:9/370: creat d7/f7a x:0 0 0 2024-03-04T13:42:47.318 INFO:tasks.workunit.client.1.smithi169.stdout:5/312: dwrite d1/d16/f3a [0,4194304] 0 2024-03-04T13:42:47.320 INFO:tasks.workunit.client.1.smithi169.stdout:9/371: read - d7/f6b zero size 2024-03-04T13:42:47.322 INFO:tasks.workunit.client.1.smithi169.stdout:8/373: symlink d7/d10/l7e 0 2024-03-04T13:42:47.327 INFO:tasks.workunit.client.1.smithi169.stdout:6/311: dwrite d0/f8 [0,4194304] 0 2024-03-04T13:42:47.328 INFO:tasks.workunit.client.1.smithi169.stdout:5/313: symlink d1/d16/d2a/l5c 0 2024-03-04T13:42:47.328 INFO:tasks.workunit.client.1.smithi169.stdout:2/307: fdatasync d1/d5/d19/d1e/f54 0 2024-03-04T13:42:47.329 INFO:tasks.workunit.client.1.smithi169.stdout:6/312: dread - d0/f3d zero size 2024-03-04T13:42:47.332 INFO:tasks.workunit.client.1.smithi169.stdout:6/313: chown d0/d3c/d40/d52/d56/f62 26954 1 2024-03-04T13:42:47.342 INFO:tasks.workunit.client.1.smithi169.stdout:1/294: getdents d4/d5/d36/d35/df 0 2024-03-04T13:42:47.342 INFO:tasks.workunit.client.1.smithi169.stdout:9/372: mknod d7/dc/d12/c7b 0 2024-03-04T13:42:47.343 INFO:tasks.workunit.client.1.smithi169.stdout:6/314: creat d0/d3c/d40/d52/d56/f71 x:0 0 0 2024-03-04T13:42:47.344 INFO:tasks.workunit.client.1.smithi169.stdout:5/314: write d1/d16/d17/d2d/f54 [510963,123925] 0 2024-03-04T13:42:47.350 INFO:tasks.workunit.client.1.smithi169.stdout:1/295: unlink d4/d5/d36/d35/c4a 0 2024-03-04T13:42:47.352 INFO:tasks.workunit.client.1.smithi169.stdout:6/315: unlink d0/d32/d3a/l20 0 2024-03-04T13:42:47.354 INFO:tasks.workunit.client.1.smithi169.stdout:6/316: chown d0/f28 13039449 1 2024-03-04T13:42:47.355 INFO:tasks.workunit.client.1.smithi169.stdout:6/317: dread - d0/f3b zero size 2024-03-04T13:42:47.357 INFO:tasks.workunit.client.1.smithi169.stdout:5/315: mkdir d1/d16/d17/d2d/d5a/d5d 0 2024-03-04T13:42:47.360 INFO:tasks.workunit.client.1.smithi169.stdout:5/316: write d1/d16/f20 [3753400,81892] 0 2024-03-04T13:42:47.360 INFO:tasks.workunit.client.1.smithi169.stdout:1/296: rename d4/d5/d36/d35/f52 to d4/d5/d36/d35/d5b/f5c 0 2024-03-04T13:42:47.364 INFO:tasks.workunit.client.1.smithi169.stdout:1/297: rename d4/d5/d36 to d4/d5/d36/d35/d16/d3a/d5d 22 2024-03-04T13:42:47.368 INFO:tasks.workunit.client.1.smithi169.stdout:9/373: dwrite d7/d25/d47/d56/f6e [0,4194304] 0 2024-03-04T13:42:47.371 INFO:tasks.workunit.client.1.smithi169.stdout:2/308: dread d1/d5/f9 [0,4194304] 0 2024-03-04T13:42:47.371 INFO:tasks.workunit.client.1.smithi169.stdout:5/317: write d1/f24 [4332468,129751] 0 2024-03-04T13:42:47.374 INFO:tasks.workunit.client.1.smithi169.stdout:6/318: truncate d0/f12 3283621 0 2024-03-04T13:42:47.375 INFO:tasks.workunit.client.1.smithi169.stdout:9/374: unlink d7/dc/l44 0 2024-03-04T13:42:47.380 INFO:tasks.workunit.client.1.smithi169.stdout:6/319: mknod d0/d32/d3a/d5e/c72 0 2024-03-04T13:42:47.382 INFO:tasks.workunit.client.1.smithi169.stdout:5/318: rename d1/d16/d17/d1b/l59 to d1/d16/d17/d1b/l5e 0 2024-03-04T13:42:47.383 INFO:tasks.workunit.client.1.smithi169.stdout:1/298: dread d4/d1d/f34 [0,4194304] 0 2024-03-04T13:42:47.389 INFO:tasks.workunit.client.1.smithi169.stdout:6/320: mknod d0/d32/d3a/d5e/c73 0 2024-03-04T13:42:47.392 INFO:tasks.workunit.client.1.smithi169.stdout:4/230: truncate d4/f15 972198 0 2024-03-04T13:42:47.393 INFO:tasks.workunit.client.1.smithi169.stdout:5/319: creat d1/d16/d2f/d39/f5f x:0 0 0 2024-03-04T13:42:47.394 INFO:tasks.workunit.client.1.smithi169.stdout:4/231: stat d4/d11/d23/f35 0 2024-03-04T13:42:47.396 INFO:tasks.workunit.client.1.smithi169.stdout:8/374: dwrite d7/d12/f15 [0,4194304] 0 2024-03-04T13:42:47.403 INFO:tasks.workunit.client.1.smithi169.stdout:1/299: mknod d4/d5/d36/d35/c5e 0 2024-03-04T13:42:47.406 INFO:tasks.workunit.client.1.smithi169.stdout:8/375: mknod d7/d12/c7f 0 2024-03-04T13:42:47.417 INFO:tasks.workunit.client.1.smithi169.stdout:1/300: symlink d4/d13/d26/l5f 0 2024-03-04T13:42:47.418 INFO:tasks.workunit.client.1.smithi169.stdout:9/375: write d7/dc/d12/d59/f6d [955329,47375] 0 2024-03-04T13:42:47.421 INFO:tasks.workunit.client.1.smithi169.stdout:4/232: dwrite d4/d11/d2e/d37/f45 [0,4194304] 0 2024-03-04T13:42:47.427 INFO:tasks.workunit.client.1.smithi169.stdout:6/321: dread d0/f38 [0,4194304] 0 2024-03-04T13:42:47.431 INFO:tasks.workunit.client.1.smithi169.stdout:2/309: dwrite d1/d5/d19/d3c/f6c [0,4194304] 0 2024-03-04T13:42:47.432 INFO:tasks.workunit.client.1.smithi169.stdout:6/322: truncate d0/f16 5133496 0 2024-03-04T13:42:47.432 INFO:tasks.workunit.client.1.smithi169.stdout:5/320: truncate d1/d16/f3b 292107 0 2024-03-04T13:42:47.435 INFO:tasks.workunit.client.1.smithi169.stdout:4/233: unlink d4/d11/d2e/f31 0 2024-03-04T13:42:47.439 INFO:tasks.workunit.client.1.smithi169.stdout:9/376: symlink d7/dc/d12/l7c 0 2024-03-04T13:42:47.445 INFO:tasks.workunit.client.1.smithi169.stdout:2/310: creat d1/d5/d22/d45/f72 x:0 0 0 2024-03-04T13:42:47.445 INFO:tasks.workunit.client.1.smithi169.stdout:8/376: dwrite d7/d10/d17/f2a [4194304,4194304] 0 2024-03-04T13:42:47.451 INFO:tasks.workunit.client.1.smithi169.stdout:5/321: mkdir d1/d60 0 2024-03-04T13:42:47.456 INFO:tasks.workunit.client.1.smithi169.stdout:9/377: mknod d7/dc/d12/d32/d42/c7d 0 2024-03-04T13:42:47.456 INFO:tasks.workunit.client.1.smithi169.stdout:6/323: truncate d0/f2f 2607365 0 2024-03-04T13:42:47.463 INFO:tasks.workunit.client.1.smithi169.stdout:4/234: fdatasync d4/d1c/f2c 0 2024-03-04T13:42:47.463 INFO:tasks.workunit.client.1.smithi169.stdout:5/322: rmdir d1/d16/d17/d2d 39 2024-03-04T13:42:47.466 INFO:tasks.workunit.client.1.smithi169.stdout:9/378: creat d7/dc/d12/d32/d42/f7e x:0 0 0 2024-03-04T13:42:47.466 INFO:tasks.workunit.client.1.smithi169.stdout:8/377: unlink d7/d10/l59 0 2024-03-04T13:42:47.470 INFO:tasks.workunit.client.1.smithi169.stdout:4/235: fsync d4/d1c/f2c 0 2024-03-04T13:42:47.471 INFO:tasks.workunit.client.1.smithi169.stdout:5/323: readlink d1/d16/d17/d28/l50 0 2024-03-04T13:42:47.472 INFO:tasks.workunit.client.1.smithi169.stdout:1/301: dwrite d4/f25 [0,4194304] 0 2024-03-04T13:42:47.474 INFO:tasks.workunit.client.1.smithi169.stdout:9/379: creat d7/d2d/d48/f7f x:0 0 0 2024-03-04T13:42:47.475 INFO:tasks.workunit.client.1.smithi169.stdout:5/324: fdatasync d1/d16/d17/f1a 0 2024-03-04T13:42:47.475 INFO:tasks.workunit.client.1.smithi169.stdout:4/236: readlink d4/d11/d23/l36 0 2024-03-04T13:42:47.480 INFO:tasks.workunit.client.1.smithi169.stdout:4/237: stat d4/d1c 0 2024-03-04T13:42:47.497 INFO:tasks.workunit.client.1.smithi169.stdout:4/238: dread d4/f9 [0,4194304] 0 2024-03-04T13:42:47.507 INFO:tasks.workunit.client.1.smithi169.stdout:4/239: chown d4/d1c 6473 1 2024-03-04T13:42:47.513 INFO:tasks.workunit.client.1.smithi169.stdout:2/311: dwrite d1/d5/d19/d1e/f5e [0,4194304] 0 2024-03-04T13:42:47.513 INFO:tasks.workunit.client.1.smithi169.stdout:5/325: dwrite d1/d16/f31 [0,4194304] 0 2024-03-04T13:42:47.529 INFO:tasks.workunit.client.1.smithi169.stdout:6/324: dwrite d0/f4b [0,4194304] 0 2024-03-04T13:42:47.532 INFO:tasks.workunit.client.1.smithi169.stdout:8/378: dwrite d7/d10/d24/f7a [0,4194304] 0 2024-03-04T13:42:47.536 INFO:tasks.workunit.client.1.smithi169.stdout:9/380: dwrite d7/d2d/d48/f65 [0,4194304] 0 2024-03-04T13:42:47.537 INFO:tasks.workunit.client.1.smithi169.stdout:1/302: dwrite d4/d5/d36/f50 [0,4194304] 0 2024-03-04T13:42:47.540 INFO:tasks.workunit.client.1.smithi169.stdout:5/326: dread d1/d23/f40 [0,4194304] 0 2024-03-04T13:42:47.544 INFO:tasks.workunit.client.1.smithi169.stdout:8/379: rmdir d7/d10/d17/d44/d36/d62/d70 39 2024-03-04T13:42:47.547 INFO:tasks.workunit.client.1.smithi169.stdout:6/325: unlink d0/f12 0 2024-03-04T13:42:47.554 INFO:tasks.workunit.client.1.smithi169.stdout:4/240: dwrite d4/d1c/f2c [0,4194304] 0 2024-03-04T13:42:47.560 INFO:tasks.workunit.client.1.smithi169.stdout:4/241: fsync d4/d11/f4b 0 2024-03-04T13:42:47.562 INFO:tasks.workunit.client.1.smithi169.stdout:8/380: rename d7/d12/c56 to d7/d10/d22/d75/d79/c80 0 2024-03-04T13:42:47.564 INFO:tasks.workunit.client.1.smithi169.stdout:5/327: getdents d1/d16/d17/d2d/d5a 0 2024-03-04T13:42:47.566 INFO:tasks.workunit.client.1.smithi169.stdout:9/381: dwrite d7/dc/d12/d20/f24 [0,4194304] 0 2024-03-04T13:42:47.568 INFO:tasks.workunit.client.1.smithi169.stdout:5/328: stat d1/d16/d17/f4b 0 2024-03-04T13:42:47.574 INFO:tasks.workunit.client.1.smithi169.stdout:2/312: dwrite d1/d5/d19/d1e/d29/f39 [0,4194304] 0 2024-03-04T13:42:47.575 INFO:tasks.workunit.client.1.smithi169.stdout:8/381: chown d7/d10/d17/d44/d36/f4a 2 1 2024-03-04T13:42:47.584 INFO:tasks.workunit.client.1.smithi169.stdout:1/303: dwrite d4/d5/f5a [0,4194304] 0 2024-03-04T13:42:47.595 INFO:tasks.workunit.client.1.smithi169.stdout:2/313: creat d1/d5/d22/d45/d66/f73 x:0 0 0 2024-03-04T13:42:47.600 INFO:tasks.workunit.client.1.smithi169.stdout:2/314: write d1/f3 [3212995,42354] 0 2024-03-04T13:42:47.601 INFO:tasks.workunit.client.1.smithi169.stdout:1/304: creat d4/d13/d26/f60 x:0 0 0 2024-03-04T13:42:47.603 INFO:tasks.workunit.client.1.smithi169.stdout:0/249: sync 2024-03-04T13:42:47.608 INFO:tasks.workunit.client.1.smithi169.stdout:7/355: sync 2024-03-04T13:42:47.608 INFO:tasks.workunit.client.1.smithi169.stdout:3/290: sync 2024-03-04T13:42:47.610 INFO:tasks.workunit.client.1.smithi169.stdout:4/242: write d4/d11/d2e/f4a [3554381,26811] 0 2024-03-04T13:42:47.610 INFO:tasks.workunit.client.1.smithi169.stdout:5/329: dwrite d1/f8 [0,4194304] 0 2024-03-04T13:42:47.621 INFO:tasks.workunit.client.1.smithi169.stdout:9/382: dwrite d7/fb [0,4194304] 0 2024-03-04T13:42:47.621 INFO:tasks.workunit.client.1.smithi169.stdout:6/326: dwrite d0/d32/d3a/f3f [0,4194304] 0 2024-03-04T13:42:47.621 INFO:tasks.workunit.client.1.smithi169.stdout:0/250: write d5/d9/dc/d12/f53 [238587,15874] 0 2024-03-04T13:42:47.624 INFO:tasks.workunit.client.1.smithi169.stdout:1/305: dread - d4/d5/d36/d35/d16/f4d zero size 2024-03-04T13:42:47.625 INFO:tasks.workunit.client.1.smithi169.stdout:2/315: write d1/d5/f8 [8123068,43620] 0 2024-03-04T13:42:47.626 INFO:tasks.workunit.client.1.smithi169.stdout:3/291: read d7/de/d14/f20 [419183,76947] 0 2024-03-04T13:42:47.628 INFO:tasks.workunit.client.1.smithi169.stdout:7/356: creat d2/d15/d54/f7c x:0 0 0 2024-03-04T13:42:47.633 INFO:tasks.workunit.client.1.smithi169.stdout:4/243: chown d4/d1c/c1f 442006 1 2024-03-04T13:42:47.638 INFO:tasks.workunit.client.1.smithi169.stdout:6/327: chown d0/d32/d3a/l22 258482055 1 2024-03-04T13:42:47.646 INFO:tasks.workunit.client.1.smithi169.stdout:4/244: creat d4/d11/d2e/f51 x:0 0 0 2024-03-04T13:42:47.647 INFO:tasks.workunit.client.1.smithi169.stdout:7/357: mknod d2/c7d 0 2024-03-04T13:42:47.655 INFO:tasks.workunit.client.1.smithi169.stdout:3/292: mknod d7/de/d14/d18/d21/d3f/c61 0 2024-03-04T13:42:47.658 INFO:tasks.workunit.client.1.smithi169.stdout:4/245: write d4/d11/d2e/d39/d48/f3c [1529872,113310] 0 2024-03-04T13:42:47.663 INFO:tasks.workunit.client.1.smithi169.stdout:5/330: dwrite d1/d16/d17/d28/f41 [0,4194304] 0 2024-03-04T13:42:47.668 INFO:tasks.workunit.client.1.smithi169.stdout:0/251: getdents d5/d9/dc/d12 0 2024-03-04T13:42:47.670 INFO:tasks.workunit.client.1.smithi169.stdout:3/293: chown d7/de/d2b/d39/l60 6648 1 2024-03-04T13:42:47.681 INFO:tasks.workunit.client.1.smithi169.stdout:3/294: write f5 [893920,26048] 0 2024-03-04T13:42:47.684 INFO:tasks.workunit.client.1.smithi169.stdout:5/331: symlink d1/d16/d2a/d46/l61 0 2024-03-04T13:42:47.696 INFO:tasks.workunit.client.1.smithi169.stdout:9/383: dwrite d7/f71 [0,4194304] 0 2024-03-04T13:42:47.699 INFO:tasks.workunit.client.1.smithi169.stdout:5/332: fdatasync d1/d16/d2f/f33 0 2024-03-04T13:42:47.702 INFO:tasks.workunit.client.1.smithi169.stdout:7/358: mkdir d2/de/d3e/d7e 0 2024-03-04T13:42:47.706 INFO:tasks.workunit.client.1.smithi169.stdout:8/382: dread d7/d10/f1e [4194304,4194304] 0 2024-03-04T13:42:47.710 INFO:tasks.workunit.client.1.smithi169.stdout:5/333: readlink d1/d16/d17/d28/l53 0 2024-03-04T13:42:47.711 INFO:tasks.workunit.client.1.smithi169.stdout:9/384: write d7/d2d/d48/f65 [5197363,6298] 0 2024-03-04T13:42:47.717 INFO:tasks.workunit.client.1.smithi169.stdout:7/359: readlink d2/de/d3e/d41/d5b/l5c 0 2024-03-04T13:42:47.724 INFO:tasks.workunit.client.1.smithi169.stdout:5/334: write d1/f24 [3070358,8183] 0 2024-03-04T13:42:47.725 INFO:tasks.workunit.client.1.smithi169.stdout:2/316: dwrite d1/d5/d19/f2e [0,4194304] 0 2024-03-04T13:42:47.728 INFO:tasks.workunit.client.1.smithi169.stdout:1/306: dwrite d4/d5/d36/d35/df/f46 [0,4194304] 0 2024-03-04T13:42:47.730 INFO:tasks.workunit.client.1.smithi169.stdout:4/246: dwrite d4/d11/d23/f35 [0,4194304] 0 2024-03-04T13:42:47.731 INFO:tasks.workunit.client.1.smithi169.stdout:8/383: write d7/d10/d24/d3e/f57 [392258,13152] 0 2024-03-04T13:42:47.735 INFO:tasks.workunit.client.1.smithi169.stdout:3/295: link d7/de/d2b/d39/f4b d7/de/d14/d18/d21/f62 0 2024-03-04T13:42:47.735 INFO:tasks.workunit.client.1.smithi169.stdout:0/252: dwrite d5/d9/dc/d12/f53 [0,4194304] 0 2024-03-04T13:42:47.746 INFO:tasks.workunit.client.1.smithi169.stdout:4/247: symlink d4/d1c/l52 0 2024-03-04T13:42:47.752 INFO:tasks.workunit.client.1.smithi169.stdout:5/335: unlink d1/d16/d17/d28/l43 0 2024-03-04T13:42:47.760 INFO:tasks.workunit.client.1.smithi169.stdout:7/360: rename d2/d6b/c6c to d2/d28/c7f 0 2024-03-04T13:42:47.761 INFO:tasks.workunit.client.1.smithi169.stdout:0/253: write d5/d9/dc/d12/d15/d19/f32 [862054,109916] 0 2024-03-04T13:42:47.767 INFO:tasks.workunit.client.1.smithi169.stdout:9/385: dwrite d7/d2d/d48/f7f [0,4194304] 0 2024-03-04T13:42:47.773 INFO:tasks.workunit.client.1.smithi169.stdout:1/307: dwrite d4/d13/d26/d4f/f51 [0,4194304] 0 2024-03-04T13:42:47.776 INFO:tasks.workunit.client.1.smithi169.stdout:4/248: creat d4/d11/d2e/d21/f53 x:0 0 0 2024-03-04T13:42:47.786 INFO:tasks.workunit.client.1.smithi169.stdout:3/296: dwrite d7/de/d14/f2a [0,4194304] 0 2024-03-04T13:42:47.786 INFO:tasks.workunit.client.1.smithi169.stdout:2/317: link d1/d5/d22/d45/d4a/f5d d1/d5/d19/d1e/f74 0 2024-03-04T13:42:47.798 INFO:tasks.workunit.client.1.smithi169.stdout:9/386: rmdir d7 39 2024-03-04T13:42:47.809 INFO:tasks.workunit.client.1.smithi169.stdout:4/249: dwrite d4/d11/d3b/f3f [0,4194304] 0 2024-03-04T13:42:47.810 INFO:tasks.workunit.client.1.smithi169.stdout:1/308: dwrite d4/d5/d36/d35/f56 [0,4194304] 0 2024-03-04T13:42:47.810 INFO:tasks.workunit.client.1.smithi169.stdout:0/254: creat d5/d9/dc/d1c/f5a x:0 0 0 2024-03-04T13:42:47.814 INFO:tasks.workunit.client.1.smithi169.stdout:3/297: dwrite d7/de/d14/d13/d1a/f28 [0,4194304] 0 2024-03-04T13:42:47.815 INFO:tasks.workunit.client.1.smithi169.stdout:8/384: rename d7/d10/d17/d44/d36/d62/d70 to d7/d10/d22/d75/d81 0 2024-03-04T13:42:47.823 INFO:tasks.workunit.client.1.smithi169.stdout:7/361: symlink d2/d28/d68/l80 0 2024-03-04T13:42:47.829 INFO:tasks.workunit.client.1.smithi169.stdout:2/318: creat d1/f75 x:0 0 0 2024-03-04T13:42:47.837 INFO:tasks.workunit.client.1.smithi169.stdout:2/319: write d1/d5/d19/d1e/d29/f3e [855148,109727] 0 2024-03-04T13:42:47.844 INFO:tasks.workunit.client.1.smithi169.stdout:1/309: dread d4/d5/d36/d35/df/f1c [0,4194304] 0 2024-03-04T13:42:47.844 INFO:tasks.workunit.client.1.smithi169.stdout:0/255: dwrite d5/d9/d3d/f41 [0,4194304] 0 2024-03-04T13:42:47.861 INFO:tasks.workunit.client.1.smithi169.stdout:9/387: creat d7/dc/f80 x:0 0 0 2024-03-04T13:42:47.866 INFO:tasks.workunit.client.1.smithi169.stdout:4/250: symlink d4/d11/d2e/d39/l54 0 2024-03-04T13:42:47.871 INFO:tasks.workunit.client.1.smithi169.stdout:8/385: symlink d7/d10/l82 0 2024-03-04T13:42:47.877 INFO:tasks.workunit.client.1.smithi169.stdout:7/362: creat d2/d15/d17/f81 x:0 0 0 2024-03-04T13:42:47.879 INFO:tasks.workunit.client.1.smithi169.stdout:7/363: write d2/de/f2a [5062454,73593] 0 2024-03-04T13:42:47.880 INFO:tasks.workunit.client.1.smithi169.stdout:9/388: symlink d7/dc/d12/d32/d42/l81 0 2024-03-04T13:42:47.885 INFO:tasks.workunit.client.1.smithi169.stdout:2/320: getdents d1/d5/d19/d69 0 2024-03-04T13:42:47.887 INFO:tasks.workunit.client.1.smithi169.stdout:2/321: readlink d1/l11 0 2024-03-04T13:42:47.891 INFO:tasks.workunit.client.1.smithi169.stdout:6/328: sync 2024-03-04T13:42:47.895 INFO:tasks.workunit.client.1.smithi169.stdout:3/298: dwrite d7/de/d2b/d39/f4b [0,4194304] 0 2024-03-04T13:42:47.901 INFO:tasks.workunit.client.1.smithi169.stdout:9/389: creat d7/dc/d12/d20/f82 x:0 0 0 2024-03-04T13:42:47.917 INFO:tasks.workunit.client.1.smithi169.stdout:1/310: truncate d4/d5/d36/f50 3592185 0 2024-03-04T13:42:47.920 INFO:tasks.workunit.client.1.smithi169.stdout:1/311: rename d4 to d4/d5/d36/d35/d5b/d61 22 2024-03-04T13:42:47.920 INFO:tasks.workunit.client.1.smithi169.stdout:2/322: dwrite d1/d5/d22/f55 [0,4194304] 0 2024-03-04T13:42:47.921 INFO:tasks.workunit.client.1.smithi169.stdout:7/364: write d2/de/f4a [1343282,85612] 0 2024-03-04T13:42:47.924 INFO:tasks.workunit.client.1.smithi169.stdout:4/251: write d4/f15 [1076006,2896] 0 2024-03-04T13:42:47.934 INFO:tasks.workunit.client.1.smithi169.stdout:3/299: mkdir d7/de/d14/d47/d63 0 2024-03-04T13:42:47.937 INFO:tasks.workunit.client.1.smithi169.stdout:5/336: sync 2024-03-04T13:42:47.941 INFO:tasks.workunit.client.1.smithi169.stdout:8/386: creat d7/d10/d22/d46/f83 x:0 0 0 2024-03-04T13:42:47.943 INFO:tasks.workunit.client.1.smithi169.stdout:0/256: getdents d5/d9/d3d 0 2024-03-04T13:42:47.945 INFO:tasks.workunit.client.1.smithi169.stdout:1/312: symlink d4/d13/l62 0 2024-03-04T13:42:47.948 INFO:tasks.workunit.client.1.smithi169.stdout:2/323: dwrite d1/d5/d19/f2e [0,4194304] 0 2024-03-04T13:42:47.951 INFO:tasks.workunit.client.1.smithi169.stdout:2/324: stat d1/d5/f4e 0 2024-03-04T13:42:47.952 INFO:tasks.workunit.client.1.smithi169.stdout:4/252: symlink d4/d11/d2e/l55 0 2024-03-04T13:42:47.953 INFO:tasks.workunit.client.1.smithi169.stdout:7/365: write d2/d28/f4b [878140,92486] 0 2024-03-04T13:42:47.956 INFO:tasks.workunit.client.1.smithi169.stdout:7/366: fsync d2/d15/d54/f7c 0 2024-03-04T13:42:47.958 INFO:tasks.workunit.client.1.smithi169.stdout:5/337: mkdir d1/d16/d2a/d62 0 2024-03-04T13:42:47.964 INFO:tasks.workunit.client.1.smithi169.stdout:8/387: readlink d7/l67 0 2024-03-04T13:42:47.970 INFO:tasks.workunit.client.1.smithi169.stdout:8/388: write d7/d10/d17/f2a [1861404,13229] 0 2024-03-04T13:42:47.974 INFO:tasks.workunit.client.1.smithi169.stdout:5/338: dread d1/d16/d2a/f3e [0,4194304] 0 2024-03-04T13:42:47.974 INFO:tasks.workunit.client.1.smithi169.stdout:4/253: rmdir d4/d11/d2e/d37 39 2024-03-04T13:42:47.979 INFO:tasks.workunit.client.1.smithi169.stdout:0/257: dwrite d5/d9/dc/d1c/f5a [0,4194304] 0 2024-03-04T13:42:47.981 INFO:tasks.workunit.client.1.smithi169.stdout:3/300: creat d7/de/d14/d13/d1a/d2d/d56/d5e/d5f/f64 x:0 0 0 2024-03-04T13:42:47.983 INFO:tasks.workunit.client.1.smithi169.stdout:4/254: stat d4/d11/d2e/f51 0 2024-03-04T13:42:47.983 INFO:tasks.workunit.client.1.smithi169.stdout:5/339: read - d1/d16/d2f/d39/f5f zero size 2024-03-04T13:42:47.989 INFO:tasks.workunit.client.1.smithi169.stdout:7/367: fdatasync d2/de/d37/f3f 0 2024-03-04T13:42:47.992 INFO:tasks.workunit.client.1.smithi169.stdout:9/390: dwrite d7/d25/d47/d56/f6e [4194304,4194304] 0 2024-03-04T13:42:47.996 INFO:tasks.workunit.client.1.smithi169.stdout:9/391: truncate d7/f7a 768931 0 2024-03-04T13:42:48.000 INFO:tasks.workunit.client.1.smithi169.stdout:1/313: dread d4/d13/d26/f2e [0,4194304] 0 2024-03-04T13:42:48.006 INFO:tasks.workunit.client.1.smithi169.stdout:2/325: dread d1/f40 [0,4194304] 0 2024-03-04T13:42:48.009 INFO:tasks.workunit.client.1.smithi169.stdout:6/329: write d0/d32/d3a/d43/f6e [177305,61925] 0 2024-03-04T13:42:48.011 INFO:tasks.workunit.client.1.smithi169.stdout:2/326: write d1/d5/d19/d1e/f24 [1730128,72672] 0 2024-03-04T13:42:48.013 INFO:tasks.workunit.client.1.smithi169.stdout:0/258: write d5/f16 [4082962,111761] 0 2024-03-04T13:42:48.015 INFO:tasks.workunit.client.1.smithi169.stdout:2/327: fsync d1/d5/f4e 0 2024-03-04T13:42:48.021 INFO:tasks.workunit.client.1.smithi169.stdout:5/340: dwrite d1/d16/d17/d28/f4c [0,4194304] 0 2024-03-04T13:42:48.025 INFO:tasks.workunit.client.1.smithi169.stdout:7/368: rename d2/d28/l38 to d2/de/d3e/d41/d5b/l82 0 2024-03-04T13:42:48.028 INFO:tasks.workunit.client.1.smithi169.stdout:9/392: creat d7/d25/d47/f83 x:0 0 0 2024-03-04T13:42:48.031 INFO:tasks.workunit.client.1.smithi169.stdout:1/314: creat d4/d5/d36/d35/d16/d3a/f63 x:0 0 0 2024-03-04T13:42:48.033 INFO:tasks.workunit.client.1.smithi169.stdout:9/393: write d7/d25/d47/f83 [692627,121470] 0 2024-03-04T13:42:48.034 INFO:tasks.workunit.client.1.smithi169.stdout:6/330: fdatasync d0/f28 0 2024-03-04T13:42:48.038 INFO:tasks.workunit.client.1.smithi169.stdout:0/259: symlink d5/d9/dc/d12/d15/l5b 0 2024-03-04T13:42:48.043 INFO:tasks.workunit.client.1.smithi169.stdout:3/301: mkdir d7/de/d14/d13/d5a/d65 0 2024-03-04T13:42:48.050 INFO:tasks.workunit.client.1.smithi169.stdout:5/341: dread d1/d16/d2a/f3e [0,4194304] 0 2024-03-04T13:42:48.051 INFO:tasks.workunit.client.1.smithi169.stdout:9/394: mkdir d7/d27/d84 0 2024-03-04T13:42:48.052 INFO:tasks.workunit.client.1.smithi169.stdout:1/315: chown d4/d5/lc 1635689661 1 2024-03-04T13:42:48.054 INFO:tasks.workunit.client.1.smithi169.stdout:0/260: fdatasync d5/d9/dc/d12/d15/d19/f2e 0 2024-03-04T13:42:48.055 INFO:tasks.workunit.client.1.smithi169.stdout:9/395: write d7/d27/f5a [38770,56274] 0 2024-03-04T13:42:48.062 INFO:tasks.workunit.client.1.smithi169.stdout:3/302: mkdir d7/de/d14/d18/d21/d66 0 2024-03-04T13:42:48.065 INFO:tasks.workunit.client.1.smithi169.stdout:6/331: rename d0/d3c/d40/d52/d56 to d0/d32/d3a/d43/d74 0 2024-03-04T13:42:48.067 INFO:tasks.workunit.client.1.smithi169.stdout:8/389: dread d7/d12/f5e [0,4194304] 0 2024-03-04T13:42:48.068 INFO:tasks.workunit.client.1.smithi169.stdout:1/316: mknod d4/d13/d26/d4f/c64 0 2024-03-04T13:42:48.070 INFO:tasks.workunit.client.1.smithi169.stdout:5/342: mknod d1/d16/d17/d2d/d5a/c63 0 2024-03-04T13:42:48.071 INFO:tasks.workunit.client.1.smithi169.stdout:7/369: dwrite d2/f1f [0,4194304] 0 2024-03-04T13:42:48.076 INFO:tasks.workunit.client.1.smithi169.stdout:0/261: creat d5/d9/dc/d12/f5c x:0 0 0 2024-03-04T13:42:48.077 INFO:tasks.workunit.client.1.smithi169.stdout:4/255: link d4/cf d4/d11/d2e/d37/c56 0 2024-03-04T13:42:48.079 INFO:tasks.workunit.client.1.smithi169.stdout:6/332: creat d0/d3c/f75 x:0 0 0 2024-03-04T13:42:48.084 INFO:tasks.workunit.client.1.smithi169.stdout:5/343: symlink d1/d16/d17/d1b/l64 0 2024-03-04T13:42:48.085 INFO:tasks.workunit.client.1.smithi169.stdout:8/390: write d7/d10/f1e [3378727,106493] 0 2024-03-04T13:42:48.091 INFO:tasks.workunit.client.1.smithi169.stdout:0/262: mkdir d5/d9/d3d/d5d 0 2024-03-04T13:42:48.097 INFO:tasks.workunit.client.1.smithi169.stdout:4/256: write d4/f41 [745297,124477] 0 2024-03-04T13:42:48.099 INFO:tasks.workunit.client.1.smithi169.stdout:3/303: dwrite d7/de/d14/d47/f59 [0,4194304] 0 2024-03-04T13:42:48.104 INFO:tasks.workunit.client.1.smithi169.stdout:5/344: fsync d1/d16/f20 0 2024-03-04T13:42:48.107 INFO:tasks.workunit.client.1.smithi169.stdout:9/396: dwrite d7/dc/d3e/f6c [0,4194304] 0 2024-03-04T13:42:48.110 INFO:tasks.workunit.client.1.smithi169.stdout:7/370: dwrite d2/d28/f60 [0,4194304] 0 2024-03-04T13:42:48.122 INFO:tasks.workunit.client.1.smithi169.stdout:5/345: creat d1/f65 x:0 0 0 2024-03-04T13:42:48.135 INFO:tasks.workunit.client.1.smithi169.stdout:3/304: rename d7/de/d14/d18/d21/d2c to d7/de/d14/d13/d1a/d2d/d56/d5e/d67 0 2024-03-04T13:42:48.138 INFO:tasks.workunit.client.1.smithi169.stdout:0/263: symlink d5/d9/d3d/d5d/l5e 0 2024-03-04T13:42:48.138 INFO:tasks.workunit.client.1.smithi169.stdout:9/397: dwrite d7/d25/d47/d56/f6e [0,4194304] 0 2024-03-04T13:42:48.139 INFO:tasks.workunit.client.1.smithi169.stdout:6/333: link d0/d32/d3a/l2c d0/d3c/d40/d52/l76 0 2024-03-04T13:42:48.142 INFO:tasks.workunit.client.1.smithi169.stdout:9/398: readlink d7/dc/lf 0 2024-03-04T13:42:48.143 INFO:tasks.workunit.client.1.smithi169.stdout:3/305: stat d7/de/d14/d13/d5a/d65 0 2024-03-04T13:42:48.143 INFO:tasks.workunit.client.1.smithi169.stdout:0/264: readlink d5/l2b 0 2024-03-04T13:42:48.145 INFO:tasks.workunit.client.1.smithi169.stdout:5/346: mknod d1/d16/d17/d28/c66 0 2024-03-04T13:42:48.150 INFO:tasks.workunit.client.1.smithi169.stdout:3/306: fdatasync d7/fb 0 2024-03-04T13:42:48.152 INFO:tasks.workunit.client.1.smithi169.stdout:7/371: write d2/f2d [405104,86126] 0 2024-03-04T13:42:48.155 INFO:tasks.workunit.client.1.smithi169.stdout:1/317: dread d4/f1b [0,4194304] 0 2024-03-04T13:42:48.158 INFO:tasks.workunit.client.1.smithi169.stdout:1/318: chown d4/d5/f5a 1669 1 2024-03-04T13:42:48.163 INFO:tasks.workunit.client.1.smithi169.stdout:6/334: readlink d0/d3c/d40/d52/l76 0 2024-03-04T13:42:48.165 INFO:tasks.workunit.client.1.smithi169.stdout:5/347: mkdir d1/d16/d17/d2d/d5a/d67 0 2024-03-04T13:42:48.166 INFO:tasks.workunit.client.1.smithi169.stdout:5/348: readlink d1/lc 0 2024-03-04T13:42:48.166 INFO:tasks.workunit.client.1.smithi169.stdout:9/399: dwrite f6 [4194304,4194304] 0 2024-03-04T13:42:48.168 INFO:tasks.workunit.client.1.smithi169.stdout:3/307: rename d7/de/d2b/d39 to d7/de/d14/d18/d21/d3f/d68 0 2024-03-04T13:42:48.170 INFO:tasks.workunit.client.1.smithi169.stdout:1/319: mknod d4/d5/d36/d35/d16/c65 0 2024-03-04T13:42:48.173 INFO:tasks.workunit.client.1.smithi169.stdout:0/265: fsync d5/d9/dc/d12/d15/d19/f2e 0 2024-03-04T13:42:48.173 INFO:tasks.workunit.client.1.smithi169.stdout:1/320: write d4/d5/f57 [356472,102796] 0 2024-03-04T13:42:48.175 INFO:tasks.workunit.client.1.smithi169.stdout:5/349: chown d1/c12 28 1 2024-03-04T13:42:48.176 INFO:tasks.workunit.client.1.smithi169.stdout:8/391: dread d7/ff [0,4194304] 0 2024-03-04T13:42:48.178 INFO:tasks.workunit.client.1.smithi169.stdout:0/266: dread - d5/d9/dc/d12/d15/d46/f57 zero size 2024-03-04T13:42:48.179 INFO:tasks.workunit.client.1.smithi169.stdout:6/335: read d0/f15 [1678559,96464] 0 2024-03-04T13:42:48.181 INFO:tasks.workunit.client.1.smithi169.stdout:5/350: chown d1/d16/d17/d28/f2c 0 1 2024-03-04T13:42:48.181 INFO:tasks.workunit.client.1.smithi169.stdout:3/308: creat d7/de/d14/d18/d21/d3f/f69 x:0 0 0 2024-03-04T13:42:48.181 INFO:tasks.workunit.client.1.smithi169.stdout:1/321: chown d4/f25 480294 1 2024-03-04T13:42:48.183 INFO:tasks.workunit.client.1.smithi169.stdout:5/351: stat d1/f4a 0 2024-03-04T13:42:48.184 INFO:tasks.workunit.client.1.smithi169.stdout:6/336: chown d0/d3c/d40/c47 103 1 2024-03-04T13:42:48.185 INFO:tasks.workunit.client.1.smithi169.stdout:1/322: read d4/f1b [693531,73130] 0 2024-03-04T13:42:48.185 INFO:tasks.workunit.client.1.smithi169.stdout:0/267: dread - d5/d9/dc/d12/f5c zero size 2024-03-04T13:42:48.185 INFO:tasks.workunit.client.1.smithi169.stdout:4/257: dread d4/d11/f30 [0,4194304] 0 2024-03-04T13:42:48.186 INFO:tasks.workunit.client.1.smithi169.stdout:3/309: write d7/de/d14/d18/d21/f62 [3156745,28149] 0 2024-03-04T13:42:48.187 INFO:tasks.workunit.client.1.smithi169.stdout:8/392: rename d7/d12/l32 to d7/d10/d24/l84 0 2024-03-04T13:42:48.188 INFO:tasks.workunit.client.1.smithi169.stdout:5/352: readlink d1/d16/d17/d28/l50 0 2024-03-04T13:42:48.191 INFO:tasks.workunit.client.1.smithi169.stdout:6/337: chown d0/fb 10403 1 2024-03-04T13:42:48.195 INFO:tasks.workunit.client.1.smithi169.stdout:3/310: dread - d7/d53/f5d zero size 2024-03-04T13:42:48.203 INFO:tasks.workunit.client.1.smithi169.stdout:9/400: dwrite d7/f7a [0,4194304] 0 2024-03-04T13:42:48.206 INFO:tasks.workunit.client.1.smithi169.stdout:8/393: unlink d7/d10/d24/d5c/c69 0 2024-03-04T13:42:48.209 INFO:tasks.workunit.client.1.smithi169.stdout:4/258: creat d4/d11/d2e/d37/f57 x:0 0 0 2024-03-04T13:42:48.278 INFO:tasks.workunit.client.1.smithi169.stdout:8/394: creat d7/d10/d24/f85 x:0 0 0 2024-03-04T13:42:48.278 INFO:tasks.workunit.client.1.smithi169.stdout:2/328: sync 2024-03-04T13:42:48.278 INFO:tasks.workunit.client.1.smithi169.stdout:0/268: dwrite d5/d9/d3d/f48 [0,4194304] 0 2024-03-04T13:42:48.282 INFO:tasks.workunit.client.1.smithi169.stdout:8/395: dread d7/d10/d24/f50 [0,4194304] 0 2024-03-04T13:42:48.283 INFO:tasks.workunit.client.1.smithi169.stdout:6/338: write d0/f29 [1800563,16233] 0 2024-03-04T13:42:48.283 INFO:tasks.workunit.client.1.smithi169.stdout:4/259: mkdir d4/d1c/d58 0 2024-03-04T13:42:48.284 INFO:tasks.workunit.client.1.smithi169.stdout:1/323: link d4/c47 d4/d5/d36/d35/d16/d3a/c66 0 2024-03-04T13:42:48.284 INFO:tasks.workunit.client.1.smithi169.stdout:5/353: link d1/c7 d1/c68 0 2024-03-04T13:42:48.287 INFO:tasks.workunit.client.1.smithi169.stdout:2/329: mknod d1/d5/d19/d36/d64/c76 0 2024-03-04T13:42:48.291 INFO:tasks.workunit.client.1.smithi169.stdout:1/324: write d4/d5/d36/d35/d16/d3a/f63 [307886,59939] 0 2024-03-04T13:42:48.291 INFO:tasks.workunit.client.1.smithi169.stdout:2/330: chown d1/l38 53239 1 2024-03-04T13:42:48.293 INFO:tasks.workunit.client.1.smithi169.stdout:0/269: truncate d5/d9/dc/d12/d15/d19/f28 5039979 0 2024-03-04T13:42:48.293 INFO:tasks.workunit.client.1.smithi169.stdout:5/354: creat d1/d16/d17/d1b/f69 x:0 0 0 2024-03-04T13:42:48.300 INFO:tasks.workunit.client.1.smithi169.stdout:8/396: rename d7/d10/d17/d44/d36/d62/l65 to d7/d10/d22/d75/d81/l86 0 2024-03-04T13:42:48.301 INFO:tasks.workunit.client.1.smithi169.stdout:1/325: creat d4/d5/d36/f67 x:0 0 0 2024-03-04T13:42:48.304 INFO:tasks.workunit.client.1.smithi169.stdout:5/355: fdatasync d1/d16/d17/d1b/f29 0 2024-03-04T13:42:48.304 INFO:tasks.workunit.client.1.smithi169.stdout:9/401: dwrite d7/fe [0,4194304] 0 2024-03-04T13:42:48.308 INFO:tasks.workunit.client.1.smithi169.stdout:5/356: chown d1/d16/d2a/f3e 100 1 2024-03-04T13:42:48.312 INFO:tasks.workunit.client.1.smithi169.stdout:9/402: truncate d7/d25/f3d 4292870 0 2024-03-04T13:42:48.315 INFO:tasks.workunit.client.1.smithi169.stdout:5/357: write d1/d16/d17/d1b/f44 [709632,110809] 0 2024-03-04T13:42:48.318 INFO:tasks.workunit.client.1.smithi169.stdout:6/339: dwrite d0/f2b [0,4194304] 0 2024-03-04T13:42:48.322 INFO:tasks.workunit.client.1.smithi169.stdout:4/260: rmdir d4 39 2024-03-04T13:42:48.336 INFO:tasks.workunit.client.1.smithi169.stdout:1/326: creat d4/d5/d2d/d4e/f68 x:0 0 0 2024-03-04T13:42:48.338 INFO:tasks.workunit.client.1.smithi169.stdout:0/270: dread d5/d9/d3d/d59/f3a [0,4194304] 0 2024-03-04T13:42:48.340 INFO:tasks.workunit.client.1.smithi169.stdout:6/340: dwrite d0/d32/d3a/d5e/f6d [0,4194304] 0 2024-03-04T13:42:48.342 INFO:tasks.workunit.client.1.smithi169.stdout:6/341: fsync d0/d32/d3a/f42 0 2024-03-04T13:42:48.345 INFO:tasks.workunit.client.1.smithi169.stdout:6/342: write d0/f3b [294580,122088] 0 2024-03-04T13:42:48.349 INFO:tasks.workunit.client.1.smithi169.stdout:8/397: write d7/d10/d24/d3e/f45 [4958024,69301] 0 2024-03-04T13:42:48.356 INFO:tasks.workunit.client.1.smithi169.stdout:0/271: chown d5/d9/dc/d12/f44 103751397 1 2024-03-04T13:42:48.359 INFO:tasks.workunit.client.1.smithi169.stdout:2/331: getdents d1/d5/d22/d45/d66 0 2024-03-04T13:42:48.361 INFO:tasks.workunit.client.1.smithi169.stdout:9/403: rmdir d7/d2d/d48/d67 0 2024-03-04T13:42:48.362 INFO:tasks.workunit.client.1.smithi169.stdout:6/343: creat d0/d32/d3a/d5e/f77 x:0 0 0 2024-03-04T13:42:48.364 INFO:tasks.workunit.client.1.smithi169.stdout:4/261: dwrite d4/d11/d2e/f51 [0,4194304] 0 2024-03-04T13:42:48.364 INFO:tasks.workunit.client.1.smithi169.stdout:5/358: rmdir d1/d16/d17/d2d/d5a/d5d 0 2024-03-04T13:42:48.367 INFO:tasks.workunit.client.1.smithi169.stdout:0/272: mknod d5/d4c/c5f 0 2024-03-04T13:42:48.368 INFO:tasks.workunit.client.1.smithi169.stdout:9/404: write d7/dc/d3e/f41 [177120,28661] 0 2024-03-04T13:42:48.370 INFO:tasks.workunit.client.1.smithi169.stdout:2/332: unlink d1/d5/d22/d3b/d41/c49 0 2024-03-04T13:42:48.371 INFO:tasks.workunit.client.1.smithi169.stdout:9/405: write d7/d2d/d48/d66/f75 [658768,103300] 0 2024-03-04T13:42:48.375 INFO:tasks.workunit.client.1.smithi169.stdout:6/344: creat d0/d63/f78 x:0 0 0 2024-03-04T13:42:48.376 INFO:tasks.workunit.client.1.smithi169.stdout:9/406: chown d7/d25/c63 1 1 2024-03-04T13:42:48.377 INFO:tasks.workunit.client.1.smithi169.stdout:4/262: creat d4/d1c/f59 x:0 0 0 2024-03-04T13:42:48.385 INFO:tasks.workunit.client.1.smithi169.stdout:0/273: creat d5/d9/d3d/d59/f60 x:0 0 0 2024-03-04T13:42:48.388 INFO:tasks.workunit.client.1.smithi169.stdout:1/327: link d4/d5/c17 d4/d5/d36/c69 0 2024-03-04T13:42:48.389 INFO:tasks.workunit.client.1.smithi169.stdout:4/263: rmdir d4/d11/d2e/d37 39 2024-03-04T13:42:48.393 INFO:tasks.workunit.client.1.smithi169.stdout:6/345: getdents d0/d3c/d40 0 2024-03-04T13:42:48.394 INFO:tasks.workunit.client.1.smithi169.stdout:2/333: dwrite d1/d5/d19/d36/f4b [0,4194304] 0 2024-03-04T13:42:48.402 INFO:tasks.workunit.client.1.smithi169.stdout:9/407: getdents d7/dc/d12/d59 0 2024-03-04T13:42:48.403 INFO:tasks.workunit.client.1.smithi169.stdout:0/274: rename d5/d9/dc/d1c to d5/d9/dc/d12/d15/d19/d61 0 2024-03-04T13:42:48.407 INFO:tasks.workunit.client.1.smithi169.stdout:9/408: truncate d7/d25/d47/f83 1694767 0 2024-03-04T13:42:48.407 INFO:tasks.workunit.client.1.smithi169.stdout:8/398: dread d7/d10/d22/f30 [0,4194304] 0 2024-03-04T13:42:48.408 INFO:tasks.workunit.client.1.smithi169.stdout:1/328: symlink d4/d5/d36/d35/l6a 0 2024-03-04T13:42:48.409 INFO:tasks.workunit.client.1.smithi169.stdout:8/399: chown d7/ca 47924158 1 2024-03-04T13:42:48.412 INFO:tasks.workunit.client.1.smithi169.stdout:2/334: creat d1/d5/d19/d1e/d29/f77 x:0 0 0 2024-03-04T13:42:48.413 INFO:tasks.workunit.client.1.smithi169.stdout:1/329: chown d4/d13/d26/d4f/c64 56 1 2024-03-04T13:42:48.418 INFO:tasks.workunit.client.1.smithi169.stdout:7/372: sync 2024-03-04T13:42:48.418 INFO:tasks.workunit.client.1.smithi169.stdout:3/311: sync 2024-03-04T13:42:48.418 INFO:tasks.workunit.client.1.smithi169.stdout:6/346: dwrite d0/d32/d3a/d43/f6e [0,4194304] 0 2024-03-04T13:42:48.419 INFO:tasks.workunit.client.1.smithi169.stdout:4/264: creat d4/f5a x:0 0 0 2024-03-04T13:42:48.419 INFO:tasks.workunit.client.1.smithi169.stdout:5/359: write d1/d16/d17/d1b/f1e [1314107,47644] 0 2024-03-04T13:42:48.420 INFO:tasks.workunit.client.1.smithi169.stdout:8/400: unlink d7/d12/f15 0 2024-03-04T13:42:48.423 INFO:tasks.workunit.client.1.smithi169.stdout:7/373: write d2/d28/f60 [1277888,14975] 0 2024-03-04T13:42:48.426 INFO:tasks.workunit.client.1.smithi169.stdout:7/374: fsync d2/d15/d17/f5d 0 2024-03-04T13:42:48.440 INFO:tasks.workunit.client.1.smithi169.stdout:8/401: mkdir d7/d10/d22/d87 0 2024-03-04T13:42:48.440 INFO:tasks.workunit.client.1.smithi169.stdout:4/265: dwrite d4/fa [0,4194304] 0 2024-03-04T13:42:48.441 INFO:tasks.workunit.client.1.smithi169.stdout:8/402: fdatasync d7/d10/f1e 0 2024-03-04T13:42:48.444 INFO:tasks.workunit.client.1.smithi169.stdout:5/360: creat d1/d16/d17/d2d/f6a x:0 0 0 2024-03-04T13:42:48.448 INFO:tasks.workunit.client.1.smithi169.stdout:3/312: truncate d7/de/d14/d13/d1a/d2d/f37 1469788 0 2024-03-04T13:42:48.452 INFO:tasks.workunit.client.1.smithi169.stdout:7/375: symlink d2/d28/d68/l83 0 2024-03-04T13:42:48.454 INFO:tasks.workunit.client.1.smithi169.stdout:0/275: creat d5/d9/dc/d12/f62 x:0 0 0 2024-03-04T13:42:48.454 INFO:tasks.workunit.client.1.smithi169.stdout:7/376: truncate d2/d28/f60 4278918 0 2024-03-04T13:42:48.459 INFO:tasks.workunit.client.1.smithi169.stdout:9/409: creat d7/d2d/f85 x:0 0 0 2024-03-04T13:42:48.465 INFO:tasks.workunit.client.1.smithi169.stdout:4/266: unlink d4/d11/d2e/d39/d48/f4c 0 2024-03-04T13:42:48.469 INFO:tasks.workunit.client.1.smithi169.stdout:3/313: readlink d7/de/d14/d18/l26 0 2024-03-04T13:42:48.473 INFO:tasks.workunit.client.1.smithi169.stdout:2/335: rename d1/d5/d19/d36/f6b to d1/f78 0 2024-03-04T13:42:48.477 INFO:tasks.workunit.client.1.smithi169.stdout:7/377: dwrite d2/d15/d17/f6e [0,4194304] 0 2024-03-04T13:42:48.477 INFO:tasks.workunit.client.1.smithi169.stdout:9/410: symlink d7/dc/d12/d32/d42/d4f/l86 0 2024-03-04T13:42:48.478 INFO:tasks.workunit.client.1.smithi169.stdout:1/330: dwrite d4/d1d/f40 [0,4194304] 0 2024-03-04T13:42:48.482 INFO:tasks.workunit.client.1.smithi169.stdout:7/378: dread - d2/d15/d17/f81 zero size 2024-03-04T13:42:48.483 INFO:tasks.workunit.client.1.smithi169.stdout:1/331: chown d4/l2c 0 1 2024-03-04T13:42:48.484 INFO:tasks.workunit.client.1.smithi169.stdout:6/347: truncate d0/d3c/d40/f5c 84569 0 2024-03-04T13:42:48.485 INFO:tasks.workunit.client.1.smithi169.stdout:8/403: unlink d7/d10/d24/d3e/l61 0 2024-03-04T13:42:48.486 INFO:tasks.workunit.client.1.smithi169.stdout:4/267: chown d4/d11/d2e/d37 4114856 1 2024-03-04T13:42:48.489 INFO:tasks.workunit.client.1.smithi169.stdout:1/332: fsync d4/d5/d36/d35/d16/d3a/f63 0 2024-03-04T13:42:48.491 INFO:tasks.workunit.client.1.smithi169.stdout:8/404: chown d7/d10/d17/d44/d36 3668 1 2024-03-04T13:42:48.492 INFO:tasks.workunit.client.1.smithi169.stdout:4/268: fsync d4/d11/d2e/d37/f45 0 2024-03-04T13:42:48.496 INFO:tasks.workunit.client.1.smithi169.stdout:5/361: rename d1/f8 to d1/d16/d17/d2d/d5a/d67/f6b 0 2024-03-04T13:42:48.498 INFO:tasks.workunit.client.1.smithi169.stdout:9/411: rename d7/d27/d54/f5d to d7/d25/d47/f87 0 2024-03-04T13:42:48.503 INFO:tasks.workunit.client.1.smithi169.stdout:1/333: dread d4/d5/d36/d35/d16/d3a/f63 [0,4194304] 0 2024-03-04T13:42:48.505 INFO:tasks.workunit.client.1.smithi169.stdout:4/269: mknod d4/c5b 0 2024-03-04T13:42:48.505 INFO:tasks.workunit.client.1.smithi169.stdout:0/276: dwrite d5/d9/dc/d12/d15/d19/f2f [0,4194304] 0 2024-03-04T13:42:48.506 INFO:tasks.workunit.client.1.smithi169.stdout:4/270: write d4/d1c/f2c [2481985,77095] 0 2024-03-04T13:42:48.512 INFO:tasks.workunit.client.1.smithi169.stdout:2/336: readlink d1/d5/d19/d1e/d29/l2f 0 2024-03-04T13:42:48.516 INFO:tasks.workunit.client.1.smithi169.stdout:8/405: dread d7/d10/d17/d44/f4f [0,4194304] 0 2024-03-04T13:42:48.516 INFO:tasks.workunit.client.1.smithi169.stdout:2/337: write d1/d5/d19/d36/f60 [802347,35300] 0 2024-03-04T13:42:48.518 INFO:tasks.workunit.client.1.smithi169.stdout:7/379: write d2/de/d37/f3f [4902709,56198] 0 2024-03-04T13:42:48.522 INFO:tasks.workunit.client.1.smithi169.stdout:4/271: rmdir d4/d11/d2e 39 2024-03-04T13:42:48.525 INFO:tasks.workunit.client.1.smithi169.stdout:4/272: write d4/d11/f4b [784897,15957] 0 2024-03-04T13:42:48.528 INFO:tasks.workunit.client.1.smithi169.stdout:1/334: getdents d4/d13/d26/d54 0 2024-03-04T13:42:48.530 INFO:tasks.workunit.client.1.smithi169.stdout:5/362: write d1/d16/d17/d28/f2c [1010640,100978] 0 2024-03-04T13:42:48.534 INFO:tasks.workunit.client.1.smithi169.stdout:6/348: rename d0/d3c/d40/d52/l60 to d0/d32/d3a/d43/l79 0 2024-03-04T13:42:48.535 INFO:tasks.workunit.client.1.smithi169.stdout:6/349: fsync d0/f4a 0 2024-03-04T13:42:48.538 INFO:tasks.workunit.client.1.smithi169.stdout:9/412: rmdir d7/d27/d54 0 2024-03-04T13:42:48.541 INFO:tasks.workunit.client.1.smithi169.stdout:5/363: creat d1/d16/d2a/d46/f6c x:0 0 0 2024-03-04T13:42:48.541 INFO:tasks.workunit.client.1.smithi169.stdout:4/273: mkdir d4/d11/d2e/d37/d5c 0 2024-03-04T13:42:48.541 INFO:tasks.workunit.client.1.smithi169.stdout:8/406: rmdir d7/d10/d22/d75/d81 39 2024-03-04T13:42:48.542 INFO:tasks.workunit.client.1.smithi169.stdout:4/274: chown d4/f15 0 1 2024-03-04T13:42:48.545 INFO:tasks.workunit.client.1.smithi169.stdout:4/275: read d4/d11/d2e/d39/d48/f4e [191315,22685] 0 2024-03-04T13:42:48.546 INFO:tasks.workunit.client.1.smithi169.stdout:4/276: fsync d4/fa 0 2024-03-04T13:42:48.546 INFO:tasks.workunit.client.1.smithi169.stdout:0/277: getdents d5/d9/dc/d12 0 2024-03-04T13:42:48.548 INFO:tasks.workunit.client.1.smithi169.stdout:4/277: readlink d4/d11/l2a 0 2024-03-04T13:42:48.549 INFO:tasks.workunit.client.1.smithi169.stdout:9/413: creat d7/dc/d12/d32/f88 x:0 0 0 2024-03-04T13:42:48.551 INFO:tasks.workunit.client.1.smithi169.stdout:8/407: unlink d7/d10/d17/d44/d36/d62/c6c 0 2024-03-04T13:42:48.551 INFO:tasks.workunit.client.1.smithi169.stdout:4/278: fdatasync d4/d11/f30 0 2024-03-04T13:42:48.552 INFO:tasks.workunit.client.1.smithi169.stdout:2/338: truncate d1/d5/d19/f3f 145357 0 2024-03-04T13:42:48.554 INFO:tasks.workunit.client.1.smithi169.stdout:8/408: stat d7/d10/d17/f2a 0 2024-03-04T13:42:48.557 INFO:tasks.workunit.client.1.smithi169.stdout:1/335: creat d4/f6b x:0 0 0 2024-03-04T13:42:48.560 INFO:tasks.workunit.client.1.smithi169.stdout:6/350: dwrite d0/d32/d3a/f65 [0,4194304] 0 2024-03-04T13:42:48.562 INFO:tasks.workunit.client.1.smithi169.stdout:9/414: symlink d7/d2d/l89 0 2024-03-04T13:42:48.563 INFO:tasks.workunit.client.1.smithi169.stdout:2/339: symlink d1/l79 0 2024-03-04T13:42:48.570 INFO:tasks.workunit.client.1.smithi169.stdout:0/278: dwrite d5/d9/dc/d12/d15/d19/f20 [0,4194304] 0 2024-03-04T13:42:48.574 INFO:tasks.workunit.client.1.smithi169.stdout:6/351: mkdir d0/d7a 0 2024-03-04T13:42:48.578 INFO:tasks.workunit.client.1.smithi169.stdout:4/279: dwrite d4/d11/d23/f35 [0,4194304] 0 2024-03-04T13:42:48.580 INFO:tasks.workunit.client.1.smithi169.stdout:1/336: link d4/d5/f38 d4/d5/d2d/f6c 0 2024-03-04T13:42:48.585 INFO:tasks.workunit.client.1.smithi169.stdout:8/409: getdents d7/d10 0 2024-03-04T13:42:48.588 INFO:tasks.workunit.client.1.smithi169.stdout:9/415: dwrite d7/d2d/f36 [0,4194304] 0 2024-03-04T13:42:48.592 INFO:tasks.workunit.client.1.smithi169.stdout:2/340: rename d1/l6e to d1/d5/d19/d63/l7a 0 2024-03-04T13:42:48.593 INFO:tasks.workunit.client.1.smithi169.stdout:8/410: fdatasync d7/d10/d24/d3e/f40 0 2024-03-04T13:42:48.595 INFO:tasks.workunit.client.1.smithi169.stdout:4/280: rename d4/d11/d3b to d4/d11/d3b/d5d 22 2024-03-04T13:42:48.595 INFO:tasks.workunit.client.1.smithi169.stdout:0/279: dwrite d5/d9/dc/d12/d15/d19/f20 [0,4194304] 0 2024-03-04T13:42:48.595 INFO:tasks.workunit.client.1.smithi169.stdout:9/416: rmdir d7/dc/d12/d32/d42 39 2024-03-04T13:42:48.599 INFO:tasks.workunit.client.1.smithi169.stdout:0/280: dread - d5/d9/dc/d12/d15/d19/d61/f54 zero size 2024-03-04T13:42:48.599 INFO:tasks.workunit.client.1.smithi169.stdout:1/337: write d4/d5/d36/d35/df/f28 [5753563,77748] 0 2024-03-04T13:42:48.600 INFO:tasks.workunit.client.1.smithi169.stdout:9/417: mkdir d7/dc/d12/d8a 0 2024-03-04T13:42:48.603 INFO:tasks.workunit.client.1.smithi169.stdout:9/418: truncate d7/dc/f80 483751 0 2024-03-04T13:42:48.604 INFO:tasks.workunit.client.1.smithi169.stdout:0/281: readlink d5/d9/dc/l55 0 2024-03-04T13:42:48.606 INFO:tasks.workunit.client.1.smithi169.stdout:4/281: getdents d4/d11/d2e/d39 0 2024-03-04T13:42:48.608 INFO:tasks.workunit.client.1.smithi169.stdout:1/338: readlink d4/d5/d36/d35/d16/d3a/l48 0 2024-03-04T13:42:48.608 INFO:tasks.workunit.client.1.smithi169.stdout:4/282: write d4/f15 [404214,17114] 0 2024-03-04T13:42:48.611 INFO:tasks.workunit.client.1.smithi169.stdout:1/339: write d4/d13/d26/d4f/f51 [17575,108952] 0 2024-03-04T13:42:48.611 INFO:tasks.workunit.client.1.smithi169.stdout:0/282: creat d5/f63 x:0 0 0 2024-03-04T13:42:48.616 INFO:tasks.workunit.client.1.smithi169.stdout:8/411: truncate d7/d10/d17/f20 85565 0 2024-03-04T13:42:48.618 INFO:tasks.workunit.client.1.smithi169.stdout:2/341: truncate d1/d5/d19/d1e/d29/f39 133914 0 2024-03-04T13:42:48.618 INFO:tasks.workunit.client.1.smithi169.stdout:4/283: write d4/d11/d2e/d21/f29 [2973521,121746] 0 2024-03-04T13:42:48.619 INFO:tasks.workunit.client.1.smithi169.stdout:0/283: mkdir d5/d9/d3d/d59/d64 0 2024-03-04T13:42:48.620 INFO:tasks.workunit.client.1.smithi169.stdout:4/284: stat d4/d11/d3b/f3f 0 2024-03-04T13:42:48.625 INFO:tasks.workunit.client.1.smithi169.stdout:1/340: rename d4/d5/d36/d35/d16/d3a/f63 to d4/d13/d26/f6d 0 2024-03-04T13:42:48.631 INFO:tasks.workunit.client.1.smithi169.stdout:9/419: getdents d7/d25/d47 0 2024-03-04T13:42:48.632 INFO:tasks.workunit.client.1.smithi169.stdout:0/284: mkdir d5/d4c/d65 0 2024-03-04T13:42:48.633 INFO:tasks.workunit.client.1.smithi169.stdout:1/341: creat d4/d13/d26/d4f/f6e x:0 0 0 2024-03-04T13:42:48.636 INFO:tasks.workunit.client.1.smithi169.stdout:0/285: readlink d5/d9/dc/d12/d15/l5b 0 2024-03-04T13:42:48.642 INFO:tasks.workunit.client.1.smithi169.stdout:9/420: creat d7/d25/d64/f8b x:0 0 0 2024-03-04T13:42:48.644 INFO:tasks.workunit.client.1.smithi169.stdout:4/285: getdents d4 0 2024-03-04T13:42:48.646 INFO:tasks.workunit.client.1.smithi169.stdout:0/286: write d5/d9/dc/d12/d15/d19/f52 [586185,97798] 0 2024-03-04T13:42:48.649 INFO:tasks.workunit.client.1.smithi169.stdout:4/286: rmdir d4/d1c 39 2024-03-04T13:42:48.649 INFO:tasks.workunit.client.1.smithi169.stdout:2/342: dwrite d1/d5/f4f [0,4194304] 0 2024-03-04T13:42:48.656 INFO:tasks.workunit.client.1.smithi169.stdout:0/287: mkdir d5/d9/dc/d12/d66 0 2024-03-04T13:42:48.660 INFO:tasks.workunit.client.1.smithi169.stdout:1/342: rmdir d4/d5/d36 39 2024-03-04T13:42:48.662 INFO:tasks.workunit.client.1.smithi169.stdout:2/343: creat d1/d5/d19/d63/f7b x:0 0 0 2024-03-04T13:42:48.664 INFO:tasks.workunit.client.1.smithi169.stdout:1/343: readlink d4/d1d/l4c 0 2024-03-04T13:42:48.666 INFO:tasks.workunit.client.1.smithi169.stdout:8/412: dwrite d7/d10/d22/f5b [0,4194304] 0 2024-03-04T13:42:48.672 INFO:tasks.workunit.client.1.smithi169.stdout:9/421: truncate d7/dc/d12/d32/d42/f43 2917220 0 2024-03-04T13:42:48.675 INFO:tasks.workunit.client.1.smithi169.stdout:9/422: dread - d7/d25/d47/f87 zero size 2024-03-04T13:42:48.675 INFO:tasks.workunit.client.1.smithi169.stdout:4/287: unlink d4/d11/c1e 0 2024-03-04T13:42:48.675 INFO:tasks.workunit.client.1.smithi169.stdout:2/344: symlink d1/l7c 0 2024-03-04T13:42:48.682 INFO:tasks.workunit.client.1.smithi169.stdout:9/423: mknod d7/d25/d47/c8c 0 2024-03-04T13:42:48.683 INFO:tasks.workunit.client.1.smithi169.stdout:2/345: creat d1/d5/d22/d45/d4a/f7d x:0 0 0 2024-03-04T13:42:48.686 INFO:tasks.workunit.client.1.smithi169.stdout:9/424: dread - d7/dc/d12/d32/f88 zero size 2024-03-04T13:42:48.692 INFO:tasks.workunit.client.1.smithi169.stdout:8/413: dwrite d7/d12/f5e [0,4194304] 0 2024-03-04T13:42:48.699 INFO:tasks.workunit.client.1.smithi169.stdout:4/288: dwrite d4/d11/f4b [0,4194304] 0 2024-03-04T13:42:48.701 INFO:tasks.workunit.client.1.smithi169.stdout:0/288: write d5/d9/dc/d12/d15/d19/d61/f33 [575599,7897] 0 2024-03-04T13:42:48.705 INFO:tasks.workunit.client.1.smithi169.stdout:1/344: truncate d4/d5/d36/d35/df/f2b 521320 0 2024-03-04T13:42:48.708 INFO:tasks.workunit.client.1.smithi169.stdout:4/289: mkdir d4/d11/d2e/d21/d5e 0 2024-03-04T13:42:48.713 INFO:tasks.workunit.client.1.smithi169.stdout:9/425: dwrite d7/fb [0,4194304] 0 2024-03-04T13:42:48.715 INFO:tasks.workunit.client.1.smithi169.stdout:0/289: read d5/d9/dc/d12/d15/d19/f28 [3745337,94425] 0 2024-03-04T13:42:48.716 INFO:tasks.workunit.client.1.smithi169.stdout:1/345: mknod d4/d1d/c6f 0 2024-03-04T13:42:48.718 INFO:tasks.workunit.client.1.smithi169.stdout:8/414: rename d7/d10/d17/f20 to d7/d10/d17/d44/d5d/f88 0 2024-03-04T13:42:48.719 INFO:tasks.workunit.client.1.smithi169.stdout:1/346: readlink d4/d5/d36/d35/d16/d3a/l44 0 2024-03-04T13:42:48.721 INFO:tasks.workunit.client.1.smithi169.stdout:4/290: symlink d4/d11/d23/l5f 0 2024-03-04T13:42:48.726 INFO:tasks.workunit.client.1.smithi169.stdout:9/426: creat d7/dc/d12/d32/d42/d4f/f8d x:0 0 0 2024-03-04T13:42:48.726 INFO:tasks.workunit.client.1.smithi169.stdout:0/290: creat d5/d9/dc/d12/d15/d19/f67 x:0 0 0 2024-03-04T13:42:48.727 INFO:tasks.workunit.client.1.smithi169.stdout:8/415: symlink d7/d12/l89 0 2024-03-04T13:42:48.728 INFO:tasks.workunit.client.1.smithi169.stdout:0/291: fdatasync d5/d9/dc/d12/d15/d19/d61/f33 0 2024-03-04T13:42:48.729 INFO:tasks.workunit.client.1.smithi169.stdout:8/416: fsync d7/d10/d17/f74 0 2024-03-04T13:42:48.731 INFO:tasks.workunit.client.1.smithi169.stdout:9/427: symlink d7/d49/l8e 0 2024-03-04T13:42:48.734 INFO:tasks.workunit.client.1.smithi169.stdout:2/346: link d1/d5/d19/d3c/f6c d1/d5/d19/d69/f7e 0 2024-03-04T13:42:48.737 INFO:tasks.workunit.client.1.smithi169.stdout:8/417: chown d7/d10/d24/d3e/f68 8 1 2024-03-04T13:42:48.742 INFO:tasks.workunit.client.1.smithi169.stdout:9/428: read d7/dc/f1d [2805401,124946] 0 2024-03-04T13:42:48.742 INFO:tasks.workunit.client.1.smithi169.stdout:4/291: dread d4/f15 [0,4194304] 0 2024-03-04T13:42:48.743 INFO:tasks.workunit.client.1.smithi169.stdout:1/347: dwrite d4/d5/d36/f59 [0,4194304] 0 2024-03-04T13:42:48.747 INFO:tasks.workunit.client.1.smithi169.stdout:1/348: truncate d4/d13/d26/f2e 4809724 0 2024-03-04T13:42:48.748 INFO:tasks.workunit.client.1.smithi169.stdout:8/418: creat d7/d10/d17/d44/d36/f8a x:0 0 0 2024-03-04T13:42:48.751 INFO:tasks.workunit.client.1.smithi169.stdout:9/429: dread d7/d27/f5a [0,4194304] 0 2024-03-04T13:42:48.751 INFO:tasks.workunit.client.1.smithi169.stdout:4/292: readlink d4/l10 0 2024-03-04T13:42:48.756 INFO:tasks.workunit.client.1.smithi169.stdout:4/293: chown d4/d1c/c1f 325 1 2024-03-04T13:42:48.756 INFO:tasks.workunit.client.1.smithi169.stdout:2/347: dread d1/d5/d19/d36/f4b [0,4194304] 0 2024-03-04T13:42:48.758 INFO:tasks.workunit.client.1.smithi169.stdout:2/348: read - d1/d5/d22/d3b/f4d zero size 2024-03-04T13:42:48.766 INFO:tasks.workunit.client.1.smithi169.stdout:2/349: write d1/d5/d19/d1e/f5e [1471420,113907] 0 2024-03-04T13:42:48.770 INFO:tasks.workunit.client.1.smithi169.stdout:4/294: rename d4/d11/d2e/f4a to d4/d11/d2e/d39/f60 0 2024-03-04T13:42:48.775 INFO:tasks.workunit.client.1.smithi169.stdout:9/430: dwrite d7/dc/d12/d59/f6d [4194304,4194304] 0 2024-03-04T13:42:48.775 INFO:tasks.workunit.client.1.smithi169.stdout:8/419: dwrite d7/d10/d22/d46/f6e [0,4194304] 0 2024-03-04T13:42:48.776 INFO:tasks.workunit.client.1.smithi169.stdout:2/350: fdatasync d1/f40 0 2024-03-04T13:42:48.778 INFO:tasks.workunit.client.1.smithi169.stdout:8/420: stat d7/d10/d24 0 2024-03-04T13:42:48.780 INFO:tasks.workunit.client.1.smithi169.stdout:0/292: write d5/f37 [941299,28654] 0 2024-03-04T13:42:48.782 INFO:tasks.workunit.client.1.smithi169.stdout:8/421: read d7/d12/f5e [4072943,16950] 0 2024-03-04T13:42:48.788 INFO:tasks.workunit.client.1.smithi169.stdout:8/422: fsync d7/d10/d24/f6b 0 2024-03-04T13:42:48.820 INFO:tasks.workunit.client.1.smithi169.stdout:2/351: fsync d1/f75 0 2024-03-04T13:42:48.822 INFO:tasks.workunit.client.1.smithi169.stdout:9/431: mknod d7/dc/d12/d32/d42/c8f 0 2024-03-04T13:42:48.827 INFO:tasks.workunit.client.1.smithi169.stdout:2/352: write d1/d5/d19/d1e/f24 [904517,192] 0 2024-03-04T13:42:48.829 INFO:tasks.workunit.client.1.smithi169.stdout:0/293: rename d5/d4c/f4e to d5/d9/dc/d12/d15/d19/d61/f68 0 2024-03-04T13:42:48.833 INFO:tasks.workunit.client.1.smithi169.stdout:8/423: dread d7/d10/d24/f3c [0,4194304] 0 2024-03-04T13:42:48.843 INFO:tasks.workunit.client.1.smithi169.stdout:2/353: dread d1/d5/d19/d36/f4b [0,4194304] 0 2024-03-04T13:42:48.844 INFO:tasks.workunit.client.1.smithi169.stdout:0/294: dread d5/d9/dc/f1d [0,4194304] 0 2024-03-04T13:42:48.847 INFO:tasks.workunit.client.1.smithi169.stdout:9/432: read d7/dc/d12/d32/d42/f43 [1269807,29497] 0 2024-03-04T13:42:48.861 INFO:tasks.workunit.client.1.smithi169.stdout:0/295: read d5/d9/dc/d12/f53 [1551510,75101] 0 2024-03-04T13:42:48.871 INFO:tasks.workunit.client.1.smithi169.stdout:8/424: dwrite d7/d10/d24/d3e/f45 [0,4194304] 0 2024-03-04T13:42:48.875 INFO:tasks.workunit.client.1.smithi169.stdout:4/295: getdents d4/d11/d2e/d39 0 2024-03-04T13:42:48.884 INFO:tasks.workunit.client.1.smithi169.stdout:4/296: mkdir d4/d11/d2e/d39/d61 0 2024-03-04T13:42:48.885 INFO:tasks.workunit.client.1.smithi169.stdout:0/296: dread d5/d9/dc/d12/d15/d19/d61/f2c [0,4194304] 0 2024-03-04T13:42:48.902 INFO:tasks.workunit.client.1.smithi169.stdout:0/297: mkdir d5/d9/dc/d12/d15/d19/d69 0 2024-03-04T13:42:48.907 INFO:tasks.workunit.client.1.smithi169.stdout:2/354: dread d1/f3 [0,4194304] 0 2024-03-04T13:42:48.921 INFO:tasks.workunit.client.1.smithi169.stdout:2/355: dwrite d1/f75 [0,4194304] 0 2024-03-04T13:42:48.929 INFO:tasks.workunit.client.1.smithi169.stdout:4/297: symlink d4/d11/d2e/d37/d5c/l62 0 2024-03-04T13:42:48.931 INFO:tasks.workunit.client.1.smithi169.stdout:4/298: chown d4/d11/d23/f35 577507 1 2024-03-04T13:42:48.936 INFO:tasks.workunit.client.1.smithi169.stdout:2/356: write d1/d5/f9 [4959220,48206] 0 2024-03-04T13:42:48.938 INFO:tasks.workunit.client.1.smithi169.stdout:0/298: symlink d5/d9/dc/d12/d66/l6a 0 2024-03-04T13:42:48.941 INFO:tasks.workunit.client.1.smithi169.stdout:0/299: truncate d5/d9/d3d/d59/f58 1062988 0 2024-03-04T13:42:48.942 INFO:tasks.workunit.client.1.smithi169.stdout:2/357: creat d1/d5/d22/d45/d66/f7f x:0 0 0 2024-03-04T13:42:48.945 INFO:tasks.workunit.client.1.smithi169.stdout:2/358: fdatasync d1/d5/d19/d1e/d29/f3e 0 2024-03-04T13:42:48.947 INFO:tasks.workunit.client.1.smithi169.stdout:2/359: write d1/d5/f4f [1317560,18068] 0 2024-03-04T13:42:48.949 INFO:tasks.workunit.client.1.smithi169.stdout:0/300: fdatasync d5/d9/dc/d12/d15/d19/d61/f68 0 2024-03-04T13:42:48.958 INFO:tasks.workunit.client.1.smithi169.stdout:2/360: chown d1/d5/d19/d3c/f6c 6819261 1 2024-03-04T13:42:48.960 INFO:tasks.workunit.client.1.smithi169.stdout:2/361: chown d1/d5/d19/d68 1239 1 2024-03-04T13:42:48.966 INFO:tasks.workunit.client.1.smithi169.stdout:2/362: symlink d1/d5/d22/d3b/d41/l80 0 2024-03-04T13:42:48.970 INFO:tasks.workunit.client.1.smithi169.stdout:9/433: write d7/dc/d12/d32/d42/d4f/f50 [5040947,45797] 0 2024-03-04T13:42:49.009 INFO:tasks.workunit.client.1.smithi169.stdout:0/301: rename d5/d9/dc/d12/f3c to d5/d4c/d65/f6b 0 2024-03-04T13:42:49.011 INFO:tasks.workunit.client.1.smithi169.stdout:9/434: rename d7/d25/c51 to d7/d25/d64/c90 0 2024-03-04T13:42:49.013 INFO:tasks.workunit.client.1.smithi169.stdout:0/302: symlink d5/d9/dc/l6c 0 2024-03-04T13:42:49.016 INFO:tasks.workunit.client.1.smithi169.stdout:0/303: symlink d5/d9/d3d/d59/d64/l6d 0 2024-03-04T13:42:49.023 INFO:tasks.workunit.client.1.smithi169.stdout:4/299: mkdir d4/d11/d2e/d63 0 2024-03-04T13:42:49.025 INFO:tasks.workunit.client.1.smithi169.stdout:4/300: fdatasync d4/d11/d23/f46 0 2024-03-04T13:42:49.028 INFO:tasks.workunit.client.1.smithi169.stdout:9/435: dwrite d7/dc/d12/d32/d42/d4f/f8d [0,4194304] 0 2024-03-04T13:42:49.029 INFO:tasks.workunit.client.1.smithi169.stdout:9/436: dread - d7/dc/d12/d32/f88 zero size 2024-03-04T13:42:49.038 INFO:tasks.workunit.client.1.smithi169.stdout:8/425: link d7/d10/d17/f2a d7/f8b 0 2024-03-04T13:42:49.044 INFO:tasks.workunit.client.1.smithi169.stdout:8/426: rename d7/d10/d17/l19 to d7/d10/d22/d87/l8c 0 2024-03-04T13:42:49.049 INFO:tasks.workunit.client.1.smithi169.stdout:8/427: read d7/d10/d24/d3e/f33 [900039,91204] 0 2024-03-04T13:42:49.051 INFO:tasks.workunit.client.1.smithi169.stdout:9/437: dwrite d7/d2d/f40 [4194304,4194304] 0 2024-03-04T13:42:49.052 INFO:tasks.workunit.client.1.smithi169.stdout:4/301: dread d4/d11/d2e/d39/f60 [0,4194304] 0 2024-03-04T13:42:49.052 INFO:tasks.workunit.client.1.smithi169.stdout:0/304: link d5/d9/dc/d12/d15/d19/d61/f23 d5/d9/f6e 0 2024-03-04T13:42:49.054 INFO:tasks.workunit.client.1.smithi169.stdout:9/438: write d7/d2d/d48/f7f [3279771,92388] 0 2024-03-04T13:42:49.056 INFO:tasks.workunit.client.1.smithi169.stdout:9/439: stat d7/d25/f3d 0 2024-03-04T13:42:49.060 INFO:tasks.workunit.client.1.smithi169.stdout:0/305: rename d5/d9/dc/c10 to d5/d4c/d65/c6f 0 2024-03-04T13:42:49.062 INFO:tasks.workunit.client.1.smithi169.stdout:9/440: truncate d7/f10 1479324 0 2024-03-04T13:42:49.068 INFO:tasks.workunit.client.1.smithi169.stdout:8/428: dwrite d7/d10/d17/d44/d4c/f7d [0,4194304] 0 2024-03-04T13:42:49.070 INFO:tasks.workunit.client.1.smithi169.stdout:0/306: mknod d5/d9/dc/d12/d15/d19/d61/c70 0 2024-03-04T13:42:49.072 INFO:tasks.workunit.client.1.smithi169.stdout:9/441: symlink d7/d27/d84/l91 0 2024-03-04T13:42:49.076 INFO:tasks.workunit.client.1.smithi169.stdout:0/307: mkdir d5/d9/dc/d12/d15/d19/d61/d71 0 2024-03-04T13:42:49.076 INFO:tasks.workunit.client.1.smithi169.stdout:8/429: mknod d7/d10/c8d 0 2024-03-04T13:42:49.079 INFO:tasks.workunit.client.1.smithi169.stdout:4/302: dwrite d4/d11/d3b/f3f [0,4194304] 0 2024-03-04T13:42:49.083 INFO:tasks.workunit.client.1.smithi169.stdout:0/308: write d5/d9/d3d/d59/f58 [1426851,35726] 0 2024-03-04T13:42:49.085 INFO:tasks.workunit.client.1.smithi169.stdout:9/442: link d7/l30 d7/d49/l92 0 2024-03-04T13:42:49.086 INFO:tasks.workunit.client.1.smithi169.stdout:9/443: fdatasync d7/f6b 0 2024-03-04T13:42:49.093 INFO:tasks.workunit.client.1.smithi169.stdout:9/444: chown d7/d2d/d48/d66 2728 1 2024-03-04T13:42:49.097 INFO:tasks.workunit.client.1.smithi169.stdout:9/445: unlink d7/f77 0 2024-03-04T13:42:49.101 INFO:tasks.workunit.client.1.smithi169.stdout:0/309: dread d5/d9/dc/d12/d15/d19/d61/f21 [0,4194304] 0 2024-03-04T13:42:49.102 INFO:tasks.workunit.client.1.smithi169.stdout:4/303: dread d4/f9 [0,4194304] 0 2024-03-04T13:42:49.117 INFO:tasks.workunit.client.1.smithi169.stdout:9/446: dwrite d7/dc/f80 [0,4194304] 0 2024-03-04T13:42:49.121 INFO:tasks.workunit.client.1.smithi169.stdout:4/304: dread d4/fa [0,4194304] 0 2024-03-04T13:42:49.124 INFO:tasks.workunit.client.1.smithi169.stdout:4/305: read - d4/d1c/f59 zero size 2024-03-04T13:42:49.125 INFO:tasks.workunit.client.1.smithi169.stdout:9/447: creat d7/dc/d12/d8a/f93 x:0 0 0 2024-03-04T13:42:49.126 INFO:tasks.workunit.client.1.smithi169.stdout:0/310: dwrite d5/f63 [0,4194304] 0 2024-03-04T13:42:49.128 INFO:tasks.workunit.client.1.smithi169.stdout:9/448: unlink d7/dc/l45 0 2024-03-04T13:42:49.131 INFO:tasks.workunit.client.1.smithi169.stdout:4/306: read d4/d11/d2e/d37/f45 [944378,28224] 0 2024-03-04T13:42:49.138 INFO:tasks.workunit.client.1.smithi169.stdout:9/449: link d7/d2d/f40 d7/dc/d12/d32/f94 0 2024-03-04T13:42:49.142 INFO:tasks.workunit.client.1.smithi169.stdout:8/430: dwrite d7/d10/d17/f27 [0,4194304] 0 2024-03-04T13:42:49.147 INFO:tasks.workunit.client.1.smithi169.stdout:0/311: dwrite d5/d9/dc/d12/d15/d19/f2e [4194304,4194304] 0 2024-03-04T13:42:49.148 INFO:tasks.workunit.client.1.smithi169.stdout:4/307: dread d4/d11/d3b/f3f [0,4194304] 0 2024-03-04T13:42:49.150 INFO:tasks.workunit.client.1.smithi169.stdout:0/312: readlink d5/d9/dc/l4d 0 2024-03-04T13:42:49.153 INFO:tasks.workunit.client.1.smithi169.stdout:4/308: write d4/d11/d2e/d21/f29 [3008936,122172] 0 2024-03-04T13:42:49.154 INFO:tasks.workunit.client.1.smithi169.stdout:0/313: mknod d5/d9/dc/d12/c72 0 2024-03-04T13:42:49.155 INFO:tasks.workunit.client.1.smithi169.stdout:4/309: chown d4/le 176 1 2024-03-04T13:42:49.155 INFO:tasks.workunit.client.1.smithi169.stdout:3/314: sync 2024-03-04T13:42:49.163 INFO:tasks.workunit.client.1.smithi169.stdout:4/310: mkdir d4/d11/d2e/d39/d61/d64 0 2024-03-04T13:42:49.172 INFO:tasks.workunit.client.1.smithi169.stdout:3/315: write d7/de/d24/f3c [1009148,63024] 0 2024-03-04T13:42:49.173 INFO:tasks.workunit.client.1.smithi169.stdout:0/314: dwrite d5/d9/d3d/d59/f3a [0,4194304] 0 2024-03-04T13:42:49.176 INFO:tasks.workunit.client.1.smithi169.stdout:3/316: readlink d7/de/d14/d18/l26 0 2024-03-04T13:42:49.182 INFO:tasks.workunit.client.1.smithi169.stdout:3/317: read d7/de/d14/d13/f50 [1314380,54916] 0 2024-03-04T13:42:49.182 INFO:tasks.workunit.client.1.smithi169.stdout:0/315: link d5/d9/d3d/l50 d5/d9/l73 0 2024-03-04T13:42:49.183 INFO:tasks.workunit.client.1.smithi169.stdout:3/318: rename d7/de to d7/de/d6a 22 2024-03-04T13:42:49.186 INFO:tasks.workunit.client.1.smithi169.stdout:3/319: creat d7/de/d14/f6b x:0 0 0 2024-03-04T13:42:49.191 INFO:tasks.workunit.client.1.smithi169.stdout:3/320: getdents d7/de/d14/d13/d1a/d2d/d56/d5e 0 2024-03-04T13:42:49.193 INFO:tasks.workunit.client.1.smithi169.stdout:3/321: chown d7/de/d14/d18/d21/f62 825467512 1 2024-03-04T13:42:49.197 INFO:tasks.workunit.client.1.smithi169.stdout:3/322: mkdir d7/de/d14/d47/d63/d6c 0 2024-03-04T13:42:49.201 INFO:tasks.workunit.client.1.smithi169.stdout:3/323: link d7/de/d14/d13/d1a/c3d d7/d53/c6d 0 2024-03-04T13:42:49.206 INFO:tasks.workunit.client.1.smithi169.stdout:3/324: creat d7/de/d2b/f6e x:0 0 0 2024-03-04T13:42:49.209 INFO:tasks.workunit.client.1.smithi169.stdout:3/325: getdents d7/de/d14/d13/d1a 0 2024-03-04T13:42:49.211 INFO:tasks.workunit.client.1.smithi169.stdout:3/326: mknod d7/de/d2b/c6f 0 2024-03-04T13:42:49.216 INFO:tasks.workunit.client.1.smithi169.stdout:0/316: dread d5/d9/dc/d12/d15/d19/d61/f23 [0,4194304] 0 2024-03-04T13:42:49.220 INFO:tasks.workunit.client.1.smithi169.stdout:0/317: creat d5/d9/dc/d12/d15/f74 x:0 0 0 2024-03-04T13:42:49.223 INFO:tasks.workunit.client.1.smithi169.stdout:0/318: unlink d5/d9/dc/d12/d15/d19/f2f 0 2024-03-04T13:42:49.226 INFO:tasks.workunit.client.1.smithi169.stdout:0/319: symlink d5/d9/dc/d12/l75 0 2024-03-04T13:42:49.227 INFO:tasks.workunit.client.1.smithi169.stdout:0/320: stat d5/d9/dc/d12/d15/d19/d61/c43 0 2024-03-04T13:42:49.229 INFO:tasks.workunit.client.1.smithi169.stdout:0/321: mknod d5/d4c/c76 0 2024-03-04T13:42:49.245 INFO:tasks.workunit.client.1.smithi169.stdout:0/322: dwrite d5/d9/dc/d12/d15/d19/d61/f54 [0,4194304] 0 2024-03-04T13:42:49.245 INFO:tasks.workunit.client.1.smithi169.stdout:3/327: dread d7/de/d14/f2a [0,4194304] 0 2024-03-04T13:42:49.249 INFO:tasks.workunit.client.1.smithi169.stdout:0/323: truncate d5/d9/dc/d12/f5c 40651 0 2024-03-04T13:42:49.255 INFO:tasks.workunit.client.1.smithi169.stdout:3/328: rename d7/de/d14/d18/f46 to d7/f70 0 2024-03-04T13:42:49.257 INFO:tasks.workunit.client.1.smithi169.stdout:0/324: mkdir d5/d9/dc/d12/d15/d19/d77 0 2024-03-04T13:42:49.259 INFO:tasks.workunit.client.1.smithi169.stdout:3/329: write d7/f70 [332587,96482] 0 2024-03-04T13:42:49.261 INFO:tasks.workunit.client.1.smithi169.stdout:0/325: creat d5/d9/dc/d12/d66/f78 x:0 0 0 2024-03-04T13:42:49.261 INFO:tasks.workunit.client.1.smithi169.stdout:3/330: fdatasync d7/de/d14/d18/d21/f62 0 2024-03-04T13:42:49.263 INFO:tasks.workunit.client.1.smithi169.stdout:0/326: creat d5/d4c/f79 x:0 0 0 2024-03-04T13:42:49.267 INFO:tasks.workunit.client.1.smithi169.stdout:0/327: mknod d5/d9/d3d/d59/c7a 0 2024-03-04T13:42:49.277 INFO:tasks.workunit.client.1.smithi169.stdout:3/331: dwrite d7/de/d14/d13/d1a/d2d/d56/d5e/d5f/f64 [0,4194304] 0 2024-03-04T13:42:49.284 INFO:tasks.workunit.client.1.smithi169.stdout:3/332: mkdir d7/de/d14/d13/d5a/d71 0 2024-03-04T13:42:49.288 INFO:tasks.workunit.client.1.smithi169.stdout:3/333: symlink d7/de/d14/d47/d63/d6c/l72 0 2024-03-04T13:42:49.290 INFO:tasks.workunit.client.1.smithi169.stdout:3/334: mknod d7/de/d14/d13/d5a/d71/c73 0 2024-03-04T13:42:49.293 INFO:tasks.workunit.client.1.smithi169.stdout:3/335: rmdir d7/de/d14/d13/d1a/d2d 39 2024-03-04T13:42:49.310 INFO:tasks.workunit.client.1.smithi169.stdout:3/336: dwrite f4 [0,4194304] 0 2024-03-04T13:42:49.313 INFO:tasks.workunit.client.1.smithi169.stdout:3/337: fdatasync d7/de/d14/d13/f50 0 2024-03-04T13:42:49.316 INFO:tasks.workunit.client.1.smithi169.stdout:3/338: mknod d7/d53/c74 0 2024-03-04T13:42:49.320 INFO:tasks.workunit.client.1.smithi169.stdout:3/339: creat d7/de/d14/d18/d21/d66/f75 x:0 0 0 2024-03-04T13:42:49.322 INFO:tasks.workunit.client.1.smithi169.stdout:3/340: chown d7/de/d14/d13/d1a/d2d/d56/d5e 3859 1 2024-03-04T13:42:49.324 INFO:tasks.workunit.client.1.smithi169.stdout:3/341: symlink d7/d53/l76 0 2024-03-04T13:42:49.328 INFO:tasks.workunit.client.1.smithi169.stdout:3/342: dread d7/de/d14/d13/f15 [0,4194304] 0 2024-03-04T13:42:49.329 INFO:tasks.workunit.client.1.smithi169.stdout:3/343: dread - d7/de/d14/f6b zero size 2024-03-04T13:42:49.332 INFO:tasks.workunit.client.1.smithi169.stdout:3/344: creat d7/de/d14/d47/d63/d6c/f77 x:0 0 0 2024-03-04T13:42:49.340 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:42:49 smithi107 ceph-mon[28739]: pgmap v46: 65 pgs: 65 active+clean; 783 MiB data, 3.9 GiB used, 533 GiB / 536 GiB avail; 14 MiB/s rd, 106 MiB/s wr, 207 op/s 2024-03-04T13:42:49.357 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:42:49 smithi169 ceph-mon[36313]: pgmap v46: 65 pgs: 65 active+clean; 783 MiB data, 3.9 GiB used, 533 GiB / 536 GiB avail; 14 MiB/s rd, 106 MiB/s wr, 207 op/s 2024-03-04T13:42:49.373 INFO:tasks.workunit.client.1.smithi169.stdout:3/345: dread f5 [4194304,4194304] 0 2024-03-04T13:42:49.375 INFO:tasks.workunit.client.1.smithi169.stdout:3/346: fsync d7/de/d14/f48 0 2024-03-04T13:42:49.392 INFO:tasks.workunit.client.1.smithi169.stdout:7/380: sync 2024-03-04T13:42:49.392 INFO:tasks.workunit.client.1.smithi169.stdout:5/364: sync 2024-03-04T13:42:49.392 INFO:tasks.workunit.client.1.smithi169.stdout:1/349: sync 2024-03-04T13:42:49.392 INFO:tasks.workunit.client.1.smithi169.stdout:6/352: sync 2024-03-04T13:42:49.392 INFO:tasks.workunit.client.1.smithi169.stdout:2/363: sync 2024-03-04T13:42:49.396 INFO:tasks.workunit.client.1.smithi169.stdout:6/353: dread - d0/d32/d3a/d5e/f77 zero size 2024-03-04T13:42:49.431 INFO:tasks.workunit.client.1.smithi169.stdout:3/347: readlink d7/de/d14/d18/d21/d3f/d68/l5b 0 2024-03-04T13:42:49.434 INFO:tasks.workunit.client.1.smithi169.stdout:5/365: creat d1/d16/d17/d2d/d51/f6d x:0 0 0 2024-03-04T13:42:49.435 INFO:tasks.workunit.client.1.smithi169.stdout:2/364: mknod d1/d5/d22/d3b/d41/c81 0 2024-03-04T13:42:49.438 INFO:tasks.workunit.client.1.smithi169.stdout:1/350: mkdir d4/d5/d36/d35/d70 0 2024-03-04T13:42:49.441 INFO:tasks.workunit.client.1.smithi169.stdout:7/381: link d2/d28/l55 d2/de/d3e/d41/d5b/d7a/l84 0 2024-03-04T13:42:49.444 INFO:tasks.workunit.client.1.smithi169.stdout:6/354: creat d0/d32/d3a/d43/f7b x:0 0 0 2024-03-04T13:42:49.446 INFO:tasks.workunit.client.1.smithi169.stdout:3/348: mkdir d7/de/d14/d13/d1a/d78 0 2024-03-04T13:42:49.448 INFO:tasks.workunit.client.1.smithi169.stdout:5/366: creat d1/d16/d17/d2d/d51/f6e x:0 0 0 2024-03-04T13:42:49.449 INFO:tasks.workunit.client.1.smithi169.stdout:2/365: rmdir d1/d5/d19/d69 39 2024-03-04T13:42:49.453 INFO:tasks.workunit.client.1.smithi169.stdout:2/366: truncate d1/d5/d22/d3b/f4d 287765 0 2024-03-04T13:42:49.456 INFO:tasks.workunit.client.1.smithi169.stdout:1/351: rename d4/d13 to d4/d5/d2d/d4e/d71 0 2024-03-04T13:42:49.458 INFO:tasks.workunit.client.1.smithi169.stdout:7/382: fsync d2/de/f39 0 2024-03-04T13:42:49.461 INFO:tasks.workunit.client.1.smithi169.stdout:1/352: dread d4/d5/d2d/d4e/d71/d26/f6d [0,4194304] 0 2024-03-04T13:42:49.463 INFO:tasks.workunit.client.1.smithi169.stdout:3/349: mknod d7/de/d14/d18/d21/d3f/d68/c79 0 2024-03-04T13:42:49.467 INFO:tasks.workunit.client.1.smithi169.stdout:1/353: write d4/d5/d36/d35/d16/d3a/f3f [2610751,31794] 0 2024-03-04T13:42:49.471 INFO:tasks.workunit.client.1.smithi169.stdout:7/383: mkdir d2/d28/d85 0 2024-03-04T13:42:49.479 INFO:tasks.workunit.client.1.smithi169.stdout:2/367: dwrite d1/d5/d22/d3b/f52 [0,4194304] 0 2024-03-04T13:42:49.483 INFO:tasks.workunit.client.1.smithi169.stdout:7/384: rename d2/de/d37/c63 to d2/d6b/c86 0 2024-03-04T13:42:49.485 INFO:tasks.workunit.client.1.smithi169.stdout:1/354: creat d4/d5/d2d/d4e/d71/d26/d54/f72 x:0 0 0 2024-03-04T13:42:49.489 INFO:tasks.workunit.client.1.smithi169.stdout:3/350: dwrite d7/de/d14/d13/d1a/d2d/d56/d5e/d67/f3a [0,4194304] 0 2024-03-04T13:42:49.490 INFO:tasks.workunit.client.1.smithi169.stdout:7/385: creat d2/d28/d70/f87 x:0 0 0 2024-03-04T13:42:49.490 INFO:tasks.workunit.client.1.smithi169.stdout:1/355: fsync d4/d5/d2d/d4e/d71/d26/d4f/f51 0 2024-03-04T13:42:49.495 INFO:tasks.workunit.client.1.smithi169.stdout:3/351: creat d7/de/d14/d18/f7a x:0 0 0 2024-03-04T13:42:49.497 INFO:tasks.workunit.client.1.smithi169.stdout:3/352: truncate d7/f36 1382684 0 2024-03-04T13:42:49.497 INFO:tasks.workunit.client.1.smithi169.stdout:1/356: symlink d4/d5/d36/d35/d5b/l73 0 2024-03-04T13:42:49.499 INFO:tasks.workunit.client.1.smithi169.stdout:7/386: link d2/d15/d17/d6f/f72 d2/de/d3e/d41/d69/f88 0 2024-03-04T13:42:49.501 INFO:tasks.workunit.client.1.smithi169.stdout:7/387: dread - d2/d28/d70/f79 zero size 2024-03-04T13:42:49.501 INFO:tasks.workunit.client.1.smithi169.stdout:1/357: creat d4/d5/d2d/f74 x:0 0 0 2024-03-04T13:42:49.502 INFO:tasks.workunit.client.1.smithi169.stdout:6/355: write d0/f30 [468403,112833] 0 2024-03-04T13:42:49.503 INFO:tasks.workunit.client.1.smithi169.stdout:3/353: symlink d7/de/d2b/l7b 0 2024-03-04T13:42:49.503 INFO:tasks.workunit.client.1.smithi169.stdout:1/358: fdatasync d4/d1d/f40 0 2024-03-04T13:42:49.503 INFO:tasks.workunit.client.1.smithi169.stdout:5/367: truncate d1/d16/d17/d28/f4d 146236 0 2024-03-04T13:42:49.509 INFO:tasks.workunit.client.1.smithi169.stdout:3/354: stat d7/c38 0 2024-03-04T13:42:49.510 INFO:tasks.workunit.client.1.smithi169.stdout:6/356: chown d0/d32/d3a/d43/c59 8 1 2024-03-04T13:42:49.511 INFO:tasks.workunit.client.1.smithi169.stdout:3/355: chown d7/de/d14/l23 4722 1 2024-03-04T13:42:49.511 INFO:tasks.workunit.client.1.smithi169.stdout:1/359: mkdir d4/d5/d36/d75 0 2024-03-04T13:42:49.515 INFO:tasks.workunit.client.1.smithi169.stdout:1/360: write d4/d5/d2d/d4e/f68 [1023068,14868] 0 2024-03-04T13:42:49.517 INFO:tasks.workunit.client.1.smithi169.stdout:3/356: write d7/de/d14/d18/d21/d3f/d68/f4b [2964729,120604] 0 2024-03-04T13:42:49.522 INFO:tasks.workunit.client.1.smithi169.stdout:5/368: creat d1/d16/d17/f6f x:0 0 0 2024-03-04T13:42:49.523 INFO:tasks.workunit.client.1.smithi169.stdout:7/388: getdents d2/d15/d17 0 2024-03-04T13:42:49.525 INFO:tasks.workunit.client.1.smithi169.stdout:7/389: fsync d2/d6b/f76 0 2024-03-04T13:42:49.526 INFO:tasks.workunit.client.1.smithi169.stdout:3/357: creat d7/de/d14/d13/d1a/d2d/d56/f7c x:0 0 0 2024-03-04T13:42:49.527 INFO:tasks.workunit.client.1.smithi169.stdout:7/390: chown d2/de 2531 1 2024-03-04T13:42:49.529 INFO:tasks.workunit.client.1.smithi169.stdout:1/361: rename d4/d1d/c6f to d4/d5/d36/d35/df/c76 0 2024-03-04T13:42:49.532 INFO:tasks.workunit.client.1.smithi169.stdout:2/368: dread d1/f51 [0,4194304] 0 2024-03-04T13:42:49.532 INFO:tasks.workunit.client.1.smithi169.stdout:5/369: creat d1/d16/d2a/f70 x:0 0 0 2024-03-04T13:42:49.533 INFO:tasks.workunit.client.1.smithi169.stdout:3/358: write d7/de/d14/d18/d21/f62 [827960,64117] 0 2024-03-04T13:42:49.535 INFO:tasks.workunit.client.1.smithi169.stdout:1/362: read d4/d5/d36/f59 [2255415,77155] 0 2024-03-04T13:42:49.535 INFO:tasks.workunit.client.1.smithi169.stdout:6/357: dwrite d0/f29 [0,4194304] 0 2024-03-04T13:42:49.538 INFO:tasks.workunit.client.1.smithi169.stdout:7/391: mkdir d2/de/d3e/d41/d89 0 2024-03-04T13:42:49.541 INFO:tasks.workunit.client.1.smithi169.stdout:7/392: read d2/f1f [312178,48499] 0 2024-03-04T13:42:49.548 INFO:tasks.workunit.client.1.smithi169.stdout:3/359: unlink d7/de/d14/d13/f50 0 2024-03-04T13:42:49.549 INFO:tasks.workunit.client.1.smithi169.stdout:3/360: fsync d7/de/d14/d18/f27 0 2024-03-04T13:42:49.550 INFO:tasks.workunit.client.1.smithi169.stdout:1/363: unlink d4/d5/d2d/d4e/d71/d26/d4f/c64 0 2024-03-04T13:42:49.552 INFO:tasks.workunit.client.1.smithi169.stdout:6/358: mknod d0/d32/d3a/c7c 0 2024-03-04T13:42:49.558 INFO:tasks.workunit.client.1.smithi169.stdout:7/393: mkdir d2/de/d8a 0 2024-03-04T13:42:49.561 INFO:tasks.workunit.client.1.smithi169.stdout:7/394: dread - d2/d28/d70/f79 zero size 2024-03-04T13:42:49.564 INFO:tasks.workunit.client.1.smithi169.stdout:8/431: sync 2024-03-04T13:42:49.564 INFO:tasks.workunit.client.1.smithi169.stdout:9/450: sync 2024-03-04T13:42:49.566 INFO:tasks.workunit.client.1.smithi169.stdout:3/361: dwrite d7/f49 [0,4194304] 0 2024-03-04T13:42:49.569 INFO:tasks.workunit.client.1.smithi169.stdout:1/364: unlink d4/d5/d2d/l2f 0 2024-03-04T13:42:49.573 INFO:tasks.workunit.client.1.smithi169.stdout:6/359: creat d0/d3c/d40/f7d x:0 0 0 2024-03-04T13:42:49.575 INFO:tasks.workunit.client.1.smithi169.stdout:5/370: creat d1/d16/d17/f71 x:0 0 0 2024-03-04T13:42:49.579 INFO:tasks.workunit.client.1.smithi169.stdout:7/395: creat d2/de/d3e/d41/f8b x:0 0 0 2024-03-04T13:42:49.580 INFO:tasks.workunit.client.1.smithi169.stdout:8/432: chown d7/d10/d17/d44/d5d/f88 87162005 1 2024-03-04T13:42:49.581 INFO:tasks.workunit.client.1.smithi169.stdout:9/451: unlink d7/dc/d12/d32/d42/c8f 0 2024-03-04T13:42:49.584 INFO:tasks.workunit.client.1.smithi169.stdout:6/360: rmdir d0/d32 39 2024-03-04T13:42:49.585 INFO:tasks.workunit.client.1.smithi169.stdout:2/369: dwrite d1/f30 [0,4194304] 0 2024-03-04T13:42:49.588 INFO:tasks.workunit.client.1.smithi169.stdout:6/361: truncate d0/d3c/f75 552840 0 2024-03-04T13:42:49.588 INFO:tasks.workunit.client.1.smithi169.stdout:2/370: read - d1/d5/d19/d63/f7b zero size 2024-03-04T13:42:49.588 INFO:tasks.workunit.client.1.smithi169.stdout:3/362: mkdir d7/de/d2b/d7d 0 2024-03-04T13:42:49.589 INFO:tasks.workunit.client.1.smithi169.stdout:8/433: write d7/d10/d24/f6b [4195551,39616] 0 2024-03-04T13:42:49.592 INFO:tasks.workunit.client.1.smithi169.stdout:8/434: chown d7/d10/d24 1423479390 1 2024-03-04T13:42:49.593 INFO:tasks.workunit.client.1.smithi169.stdout:2/371: chown d1/d5/d22/d45/d4a/f5d 99 1 2024-03-04T13:42:49.596 INFO:tasks.workunit.client.1.smithi169.stdout:8/435: write d7/d10/d17/f27 [1783880,15914] 0 2024-03-04T13:42:49.597 INFO:tasks.workunit.client.1.smithi169.stdout:6/362: getdents d0/d3c/d40/d52 0 2024-03-04T13:42:49.606 INFO:tasks.workunit.client.1.smithi169.stdout:8/436: unlink d7/d10/d17/d44/d5d/f7b 0 2024-03-04T13:42:49.614 INFO:tasks.workunit.client.1.smithi169.stdout:5/371: dread d1/d16/d2f/d38/f4f [0,4194304] 0 2024-03-04T13:42:49.619 INFO:tasks.workunit.client.1.smithi169.stdout:5/372: mkdir d1/d16/d2f/d72 0 2024-03-04T13:42:49.625 INFO:tasks.workunit.client.1.smithi169.stdout:2/372: truncate d1/f3 7625613 0 2024-03-04T13:42:49.630 INFO:tasks.workunit.client.1.smithi169.stdout:6/363: dwrite d0/d32/d3a/d43/d74/f71 [0,4194304] 0 2024-03-04T13:42:49.634 INFO:tasks.workunit.client.1.smithi169.stdout:2/373: creat d1/d5/d19/d1e/f82 x:0 0 0 2024-03-04T13:42:49.637 INFO:tasks.workunit.client.1.smithi169.stdout:3/363: dread d7/de/d14/d18/d21/d3f/d68/f4b [0,4194304] 0 2024-03-04T13:42:49.638 INFO:tasks.workunit.client.1.smithi169.stdout:6/364: read d0/f15 [1383034,56767] 0 2024-03-04T13:42:49.638 INFO:tasks.workunit.client.1.smithi169.stdout:5/373: rename d1/d16/d17/d1b/l5e to d1/d16/d2f/d39/l73 0 2024-03-04T13:42:49.639 INFO:tasks.workunit.client.1.smithi169.stdout:8/437: dwrite d7/d10/d24/d3e/f57 [0,4194304] 0 2024-03-04T13:42:49.645 INFO:tasks.workunit.client.1.smithi169.stdout:5/374: write d1/d16/d17/d1b/f69 [947429,37725] 0 2024-03-04T13:42:49.646 INFO:tasks.workunit.client.1.smithi169.stdout:5/375: readlink d1/d16/l1d 0 2024-03-04T13:42:49.647 INFO:tasks.workunit.client.1.smithi169.stdout:3/364: unlink d7/de/d14/d18/d21/d3f/d68/c79 0 2024-03-04T13:42:49.648 INFO:tasks.workunit.client.1.smithi169.stdout:6/365: creat d0/d32/d3a/d43/d74/f7e x:0 0 0 2024-03-04T13:42:49.653 INFO:tasks.workunit.client.1.smithi169.stdout:3/365: chown d7/de/d14/d18/f7a 143827646 1 2024-03-04T13:42:49.661 INFO:tasks.workunit.client.1.smithi169.stdout:7/396: dread d2/d28/f33 [0,4194304] 0 2024-03-04T13:42:49.661 INFO:tasks.workunit.client.1.smithi169.stdout:8/438: symlink d7/d10/d22/d46/l8e 0 2024-03-04T13:42:49.663 INFO:tasks.workunit.client.1.smithi169.stdout:5/376: dread d1/d16/d17/d2d/f54 [0,4194304] 0 2024-03-04T13:42:49.663 INFO:tasks.workunit.client.1.smithi169.stdout:2/374: dwrite d1/f75 [0,4194304] 0 2024-03-04T13:42:49.668 INFO:tasks.workunit.client.1.smithi169.stdout:2/375: chown d1/d5/d19/d1e/d29/l2f 225746 1 2024-03-04T13:42:49.669 INFO:tasks.workunit.client.1.smithi169.stdout:3/366: mknod d7/de/d2b/c7e 0 2024-03-04T13:42:49.671 INFO:tasks.workunit.client.1.smithi169.stdout:8/439: fdatasync d7/d10/d24/f3c 0 2024-03-04T13:42:49.673 INFO:tasks.workunit.client.1.smithi169.stdout:6/366: link d0/l11 d0/d32/d3a/d5e/l7f 0 2024-03-04T13:42:49.674 INFO:tasks.workunit.client.1.smithi169.stdout:5/377: mkdir d1/d16/d17/d1b/d74 0 2024-03-04T13:42:49.675 INFO:tasks.workunit.client.1.smithi169.stdout:9/452: dread d7/dc/d12/d32/d42/f58 [0,4194304] 0 2024-03-04T13:42:49.677 INFO:tasks.workunit.client.1.smithi169.stdout:7/397: rename d2/de/c36 to d2/de/d3e/d41/c8c 0 2024-03-04T13:42:49.677 INFO:tasks.workunit.client.1.smithi169.stdout:2/376: symlink d1/d5/d19/d1e/l83 0 2024-03-04T13:42:49.679 INFO:tasks.workunit.client.1.smithi169.stdout:2/377: dread - d1/d5/d19/f6a zero size 2024-03-04T13:42:49.680 INFO:tasks.workunit.client.1.smithi169.stdout:3/367: creat d7/de/d14/f7f x:0 0 0 2024-03-04T13:42:49.681 INFO:tasks.workunit.client.1.smithi169.stdout:8/440: rmdir d7/d10/d22 39 2024-03-04T13:42:49.681 INFO:tasks.workunit.client.1.smithi169.stdout:8/441: fdatasync d7/f4d 0 2024-03-04T13:42:49.682 INFO:tasks.workunit.client.1.smithi169.stdout:6/367: symlink d0/d32/d3a/l80 0 2024-03-04T13:42:49.686 INFO:tasks.workunit.client.1.smithi169.stdout:9/453: mkdir d7/dc/d95 0 2024-03-04T13:42:49.687 INFO:tasks.workunit.client.1.smithi169.stdout:5/378: unlink d1/d16/d2f/d38/f5b 0 2024-03-04T13:42:49.691 INFO:tasks.workunit.client.1.smithi169.stdout:7/398: mknod d2/c8d 0 2024-03-04T13:42:49.695 INFO:tasks.workunit.client.1.smithi169.stdout:8/442: dwrite d7/d12/f5e [0,4194304] 0 2024-03-04T13:42:49.711 INFO:tasks.workunit.client.1.smithi169.stdout:0/328: sync 2024-03-04T13:42:49.711 INFO:tasks.workunit.client.1.smithi169.stdout:4/311: sync 2024-03-04T13:42:49.712 INFO:tasks.workunit.client.1.smithi169.stdout:0/329: readlink d5/l2b 0 2024-03-04T13:42:49.725 INFO:tasks.workunit.client.1.smithi169.stdout:0/330: dwrite d5/d9/d3d/f41 [4194304,4194304] 0 2024-03-04T13:42:49.750 INFO:tasks.workunit.client.1.smithi169.stdout:5/379: creat d1/f75 x:0 0 0 2024-03-04T13:42:49.756 INFO:tasks.workunit.client.1.smithi169.stdout:9/454: dread d7/d27/f5a [0,4194304] 0 2024-03-04T13:42:49.758 INFO:tasks.workunit.client.1.smithi169.stdout:0/331: write d5/d9/dc/d12/f44 [1077933,7131] 0 2024-03-04T13:42:49.759 INFO:tasks.workunit.client.1.smithi169.stdout:2/378: link d1/d5/d22/d45/d66/f73 d1/f84 0 2024-03-04T13:42:49.761 INFO:tasks.workunit.client.1.smithi169.stdout:5/380: mkdir d1/d76 0 2024-03-04T13:42:49.765 INFO:tasks.workunit.client.1.smithi169.stdout:4/312: creat d4/d11/d2e/d39/d48/f65 x:0 0 0 2024-03-04T13:42:49.768 INFO:tasks.workunit.client.1.smithi169.stdout:9/455: truncate d7/dc/d12/d32/d42/f43 1976715 0 2024-03-04T13:42:49.773 INFO:tasks.workunit.client.1.smithi169.stdout:6/368: getdents d0/d32/d3a/d43 0 2024-03-04T13:42:49.773 INFO:tasks.workunit.client.1.smithi169.stdout:9/456: read d7/d27/f2f [2954360,79113] 0 2024-03-04T13:42:49.775 INFO:tasks.workunit.client.1.smithi169.stdout:7/399: symlink d2/de/d3e/d41/d69/d78/l8e 0 2024-03-04T13:42:49.777 INFO:tasks.workunit.client.1.smithi169.stdout:2/379: rename d1/d5/d19/c48 to d1/d5/d22/d3b/d41/c85 0 2024-03-04T13:42:49.778 INFO:tasks.workunit.client.1.smithi169.stdout:8/443: symlink d7/d10/d24/l8f 0 2024-03-04T13:42:49.780 INFO:tasks.workunit.client.1.smithi169.stdout:0/332: dwrite d5/d9/dc/d12/f44 [0,4194304] 0 2024-03-04T13:42:49.780 INFO:tasks.workunit.client.1.smithi169.stdout:4/313: dread d4/fa [0,4194304] 0 2024-03-04T13:42:49.788 INFO:tasks.workunit.client.1.smithi169.stdout:9/457: rmdir d7/d25/d3c 39 2024-03-04T13:42:49.794 INFO:tasks.workunit.client.1.smithi169.stdout:4/314: symlink d4/d11/d2e/d37/l66 0 2024-03-04T13:42:49.800 INFO:tasks.workunit.client.1.smithi169.stdout:9/458: write d7/dc/d12/d8a/f93 [91601,59755] 0 2024-03-04T13:42:49.804 INFO:tasks.workunit.client.1.smithi169.stdout:0/333: write d5/d9/f42 [3558234,126002] 0 2024-03-04T13:42:49.807 INFO:tasks.workunit.client.1.smithi169.stdout:6/369: dwrite d0/d32/d3a/f65 [4194304,4194304] 0 2024-03-04T13:42:49.807 INFO:tasks.workunit.client.1.smithi169.stdout:7/400: unlink d2/d28/d70/f73 0 2024-03-04T13:42:49.814 INFO:tasks.workunit.client.1.smithi169.stdout:4/315: mkdir d4/d11/d3b/d67 0 2024-03-04T13:42:49.816 INFO:tasks.workunit.client.1.smithi169.stdout:6/370: truncate d0/d3c/f68 543033 0 2024-03-04T13:42:49.820 INFO:tasks.workunit.client.1.smithi169.stdout:2/380: dwrite d1/d5/d19/f47 [0,4194304] 0 2024-03-04T13:42:49.820 INFO:tasks.workunit.client.1.smithi169.stdout:9/459: symlink d7/d25/d64/l96 0 2024-03-04T13:42:49.820 INFO:tasks.workunit.client.1.smithi169.stdout:8/444: dwrite d7/d10/d24/d3e/f68 [0,4194304] 0 2024-03-04T13:42:49.824 INFO:tasks.workunit.client.1.smithi169.stdout:6/371: write d0/d32/d3a/f4f [351030,112457] 0 2024-03-04T13:42:49.824 INFO:tasks.workunit.client.1.smithi169.stdout:9/460: write d7/dc/d12/d20/f24 [4887065,44804] 0 2024-03-04T13:42:49.827 INFO:tasks.workunit.client.1.smithi169.stdout:6/372: chown d0/ce 7827057 1 2024-03-04T13:42:49.829 INFO:tasks.workunit.client.1.smithi169.stdout:2/381: dread - d1/d5/d19/f6a zero size 2024-03-04T13:42:49.834 INFO:tasks.workunit.client.1.smithi169.stdout:2/382: chown d1/d5/f8 229706 1 2024-03-04T13:42:49.834 INFO:tasks.workunit.client.1.smithi169.stdout:0/334: readlink d5/d9/dc/d12/l4b 0 2024-03-04T13:42:49.835 INFO:tasks.workunit.client.1.smithi169.stdout:4/316: rmdir d4/d1c 39 2024-03-04T13:42:49.846 INFO:tasks.workunit.client.1.smithi169.stdout:0/335: fsync d5/d9/dc/d12/d15/d19/d61/f5a 0 2024-03-04T13:42:49.847 INFO:tasks.workunit.client.1.smithi169.stdout:2/383: chown d1/d5/d19/d1e/d29/f77 254 1 2024-03-04T13:42:49.848 INFO:tasks.workunit.client.1.smithi169.stdout:6/373: stat d0/c1f 0 2024-03-04T13:42:49.848 INFO:tasks.workunit.client.1.smithi169.stdout:5/381: dwrite d1/d16/f20 [0,4194304] 0 2024-03-04T13:42:49.850 INFO:tasks.workunit.client.1.smithi169.stdout:0/336: truncate d5/d4c/f79 947618 0 2024-03-04T13:42:49.852 INFO:tasks.workunit.client.1.smithi169.stdout:5/382: write d1/d16/d17/f4b [93206,43106] 0 2024-03-04T13:42:49.852 INFO:tasks.workunit.client.1.smithi169.stdout:4/317: truncate d4/f9 805304 0 2024-03-04T13:42:49.856 INFO:tasks.workunit.client.1.smithi169.stdout:8/445: chown d7/d10/d17/l2f 14535846 1 2024-03-04T13:42:49.859 INFO:tasks.workunit.client.1.smithi169.stdout:2/384: readlink d1/l11 0 2024-03-04T13:42:49.861 INFO:tasks.workunit.client.1.smithi169.stdout:2/385: stat d1/d5/d19/d1e/d29/f3e 0 2024-03-04T13:42:49.866 INFO:tasks.workunit.client.1.smithi169.stdout:5/383: rename d1/f42 to d1/d16/d2f/d72/f77 0 2024-03-04T13:42:49.869 INFO:tasks.workunit.client.1.smithi169.stdout:8/446: creat d7/d10/d24/d5c/f90 x:0 0 0 2024-03-04T13:42:49.870 INFO:tasks.workunit.client.1.smithi169.stdout:8/447: fsync d7/d10/d17/f74 0 2024-03-04T13:42:49.873 INFO:tasks.workunit.client.1.smithi169.stdout:8/448: write d7/f6a [1351708,10632] 0 2024-03-04T13:42:49.880 INFO:tasks.workunit.client.1.smithi169.stdout:2/386: unlink d1/d5/d19/d1e/l83 0 2024-03-04T13:42:49.889 INFO:tasks.workunit.client.1.smithi169.stdout:7/401: link d2/d15/c16 d2/de/c8f 0 2024-03-04T13:42:49.891 INFO:tasks.workunit.client.1.smithi169.stdout:8/449: fsync d7/d10/d24/d3e/f33 0 2024-03-04T13:42:49.899 INFO:tasks.workunit.client.1.smithi169.stdout:4/318: creat d4/d11/d2e/d39/f68 x:0 0 0 2024-03-04T13:42:49.900 INFO:tasks.workunit.client.1.smithi169.stdout:0/337: truncate d5/d9/dc/f1d 2077569 0 2024-03-04T13:42:49.907 INFO:tasks.workunit.client.1.smithi169.stdout:7/402: stat d2/c31 0 2024-03-04T13:42:49.908 INFO:tasks.workunit.client.1.smithi169.stdout:6/374: dwrite d0/d32/f6b [0,4194304] 0 2024-03-04T13:42:49.910 INFO:tasks.workunit.client.1.smithi169.stdout:2/387: rename d1/d5/d19/d1e/f54 to d1/d6d/f86 0 2024-03-04T13:42:49.922 INFO:tasks.workunit.client.1.smithi169.stdout:0/338: dwrite d5/d9/dc/d12/d15/d19/d61/f54 [0,4194304] 0 2024-03-04T13:42:49.934 INFO:tasks.workunit.client.1.smithi169.stdout:1/365: sync 2024-03-04T13:42:49.940 INFO:tasks.workunit.client.1.smithi169.stdout:5/384: link d1/c21 d1/d16/d17/d28/c78 0 2024-03-04T13:42:49.940 INFO:tasks.workunit.client.1.smithi169.stdout:6/375: creat d0/d32/f81 x:0 0 0 2024-03-04T13:42:49.941 INFO:tasks.workunit.client.1.smithi169.stdout:0/339: symlink d5/d9/dc/d12/d15/d19/l7b 0 2024-03-04T13:42:49.946 INFO:tasks.workunit.client.1.smithi169.stdout:8/450: getdents d7/d10/d17/d44/d36/d62 0 2024-03-04T13:42:49.948 INFO:tasks.workunit.client.1.smithi169.stdout:0/340: unlink d5/f63 0 2024-03-04T13:42:49.949 INFO:tasks.workunit.client.1.smithi169.stdout:4/319: link d4/d11/d2e/d39/d48/f4e d4/f69 0 2024-03-04T13:42:49.954 INFO:tasks.workunit.client.1.smithi169.stdout:6/376: rmdir d0/d32/d3a/d43/d74 39 2024-03-04T13:42:49.958 INFO:tasks.workunit.client.1.smithi169.stdout:8/451: rename d7/d10/d17/d44/d36/c64 to d7/d10/d24/d5c/c91 0 2024-03-04T13:42:49.962 INFO:tasks.workunit.client.1.smithi169.stdout:1/366: dwrite d4/d5/d2d/d4e/d71/d26/f2e [0,4194304] 0 2024-03-04T13:42:49.964 INFO:tasks.workunit.client.1.smithi169.stdout:1/367: chown d4/l2c 47558 1 2024-03-04T13:42:49.966 INFO:tasks.workunit.client.1.smithi169.stdout:5/385: creat d1/d16/d17/f79 x:0 0 0 2024-03-04T13:42:49.969 INFO:tasks.workunit.client.1.smithi169.stdout:4/320: dread d4/d11/f4b [0,4194304] 0 2024-03-04T13:42:49.970 INFO:tasks.workunit.client.1.smithi169.stdout:8/452: symlink d7/d10/l92 0 2024-03-04T13:42:49.973 INFO:tasks.workunit.client.1.smithi169.stdout:6/377: rmdir d0/d5d 0 2024-03-04T13:42:49.973 INFO:tasks.workunit.client.1.smithi169.stdout:0/341: dwrite d5/d9/dc/d12/d15/d19/f67 [0,4194304] 0 2024-03-04T13:42:49.976 INFO:tasks.workunit.client.1.smithi169.stdout:4/321: dread - d4/d1c/f50 zero size 2024-03-04T13:42:49.978 INFO:tasks.workunit.client.1.smithi169.stdout:8/453: mkdir d7/d10/d17/d44/d4c/d93 0 2024-03-04T13:42:49.979 INFO:tasks.workunit.client.1.smithi169.stdout:8/454: readlink d7/d10/d24/l8f 0 2024-03-04T13:42:49.982 INFO:tasks.workunit.client.1.smithi169.stdout:0/342: creat d5/d9/d3d/d5d/f7c x:0 0 0 2024-03-04T13:42:49.982 INFO:tasks.workunit.client.1.smithi169.stdout:8/455: write d7/d10/d24/d3e/f68 [1237568,77901] 0 2024-03-04T13:42:49.986 INFO:tasks.workunit.client.1.smithi169.stdout:0/343: fsync d5/d9/dc/d12/f5c 0 2024-03-04T13:42:49.987 INFO:tasks.workunit.client.1.smithi169.stdout:7/403: dwrite d2/d15/d17/f1d [4194304,4194304] 0 2024-03-04T13:42:49.988 INFO:tasks.workunit.client.1.smithi169.stdout:1/368: rename d4/d5/d36/d35/c5e to d4/c77 0 2024-03-04T13:42:49.991 INFO:tasks.workunit.client.1.smithi169.stdout:6/378: symlink d0/d7a/l82 0 2024-03-04T13:42:49.997 INFO:tasks.workunit.client.1.smithi169.stdout:8/456: mknod d7/d12/c94 0 2024-03-04T13:42:49.997 INFO:tasks.workunit.client.1.smithi169.stdout:5/386: link d1/d16/d2a/d46/l56 d1/d16/d17/d1b/l7a 0 2024-03-04T13:42:49.999 INFO:tasks.workunit.client.1.smithi169.stdout:1/369: dread d4/d1d/f34 [0,4194304] 0 2024-03-04T13:42:50.000 INFO:tasks.workunit.client.1.smithi169.stdout:0/344: mknod d5/d9/dc/c7d 0 2024-03-04T13:42:50.007 INFO:tasks.workunit.client.1.smithi169.stdout:8/457: write d7/d10/d17/d44/d4c/f7d [422157,16161] 0 2024-03-04T13:42:50.008 INFO:tasks.workunit.client.1.smithi169.stdout:1/370: write d4/d5/d36/d35/f56 [1722625,71882] 0 2024-03-04T13:42:50.011 INFO:tasks.workunit.client.1.smithi169.stdout:5/387: symlink d1/d76/l7b 0 2024-03-04T13:42:50.015 INFO:tasks.workunit.client.1.smithi169.stdout:7/404: dwrite d2/d15/d17/f6e [0,4194304] 0 2024-03-04T13:42:50.020 INFO:tasks.workunit.client.1.smithi169.stdout:7/405: stat d2/de/d3e/d41/f71 0 2024-03-04T13:42:50.021 INFO:tasks.workunit.client.1.smithi169.stdout:7/406: fdatasync d2/d15/d54/f7c 0 2024-03-04T13:42:50.024 INFO:tasks.workunit.client.1.smithi169.stdout:5/388: fdatasync d1/f4a 0 2024-03-04T13:42:50.026 INFO:tasks.workunit.client.1.smithi169.stdout:6/379: link d0/d32/d3a/c2d d0/c83 0 2024-03-04T13:42:50.029 INFO:tasks.workunit.client.1.smithi169.stdout:1/371: mkdir d4/d5/d36/d35/d70/d78 0 2024-03-04T13:42:50.030 INFO:tasks.workunit.client.1.smithi169.stdout:5/389: dread - d1/f4a zero size 2024-03-04T13:42:50.033 INFO:tasks.workunit.client.1.smithi169.stdout:8/458: dwrite d7/f4d [0,4194304] 0 2024-03-04T13:42:50.033 INFO:tasks.workunit.client.1.smithi169.stdout:0/345: dread d5/d9/dc/d12/d15/d19/d61/f21 [0,4194304] 0 2024-03-04T13:42:50.034 INFO:tasks.workunit.client.1.smithi169.stdout:5/390: chown d1/d16/d2a/l2e 507659 1 2024-03-04T13:42:50.035 INFO:tasks.workunit.client.1.smithi169.stdout:7/407: mknod d2/de/d3e/d41/d69/d78/c90 0 2024-03-04T13:42:50.037 INFO:tasks.workunit.client.1.smithi169.stdout:5/391: write d1/d16/f3b [934541,35136] 0 2024-03-04T13:42:50.040 INFO:tasks.workunit.client.1.smithi169.stdout:5/392: chown d1/lf 1365 1 2024-03-04T13:42:50.041 INFO:tasks.workunit.client.1.smithi169.stdout:1/372: creat d4/d5/d2d/d4e/d71/d26/f79 x:0 0 0 2024-03-04T13:42:50.046 INFO:tasks.workunit.client.1.smithi169.stdout:6/380: write d0/d32/d3a/d43/f6e [1807791,7275] 0 2024-03-04T13:42:50.048 INFO:tasks.workunit.client.1.smithi169.stdout:5/393: rename d1/d16/d17/f1a to d1/d16/d2f/d72/f7c 0 2024-03-04T13:42:50.053 INFO:tasks.workunit.client.1.smithi169.stdout:0/346: getdents d5/d9/d3d/d5d 0 2024-03-04T13:42:50.054 INFO:tasks.workunit.client.1.smithi169.stdout:5/394: creat d1/f7d x:0 0 0 2024-03-04T13:42:50.056 INFO:tasks.workunit.client.1.smithi169.stdout:6/381: symlink d0/d32/d3a/d43/d74/l84 0 2024-03-04T13:42:50.057 INFO:tasks.workunit.client.1.smithi169.stdout:5/395: chown d1/d16/d2f/d38 0 1 2024-03-04T13:42:50.060 INFO:tasks.workunit.client.1.smithi169.stdout:7/408: dread d2/d28/f4b [0,4194304] 0 2024-03-04T13:42:50.061 INFO:tasks.workunit.client.1.smithi169.stdout:8/459: dwrite d7/d12/f5e [4194304,4194304] 0 2024-03-04T13:42:50.064 INFO:tasks.workunit.client.1.smithi169.stdout:6/382: symlink d0/l85 0 2024-03-04T13:42:50.064 INFO:tasks.workunit.client.1.smithi169.stdout:5/396: mknod d1/d16/d2f/d72/c7e 0 2024-03-04T13:42:50.066 INFO:tasks.workunit.client.1.smithi169.stdout:1/373: write d4/f1b [133518,94776] 0 2024-03-04T13:42:50.067 INFO:tasks.workunit.client.1.smithi169.stdout:7/409: unlink d2/de/f39 0 2024-03-04T13:42:50.069 INFO:tasks.workunit.client.1.smithi169.stdout:7/410: readlink d2/d28/l2e 0 2024-03-04T13:42:50.069 INFO:tasks.workunit.client.1.smithi169.stdout:8/460: creat d7/d10/d24/d3e/f95 x:0 0 0 2024-03-04T13:42:50.076 INFO:tasks.workunit.client.1.smithi169.stdout:6/383: rename d0/f3d to d0/d3c/f86 0 2024-03-04T13:42:50.076 INFO:tasks.workunit.client.1.smithi169.stdout:0/347: link d5/d9/dc/d12/c72 d5/d9/dc/d12/d66/c7e 0 2024-03-04T13:42:50.077 INFO:tasks.workunit.client.1.smithi169.stdout:6/384: write d0/fb [2465512,66264] 0 2024-03-04T13:42:50.081 INFO:tasks.workunit.client.1.smithi169.stdout:0/348: truncate d5/d9/dc/d12/d15/d19/f20 5115729 0 2024-03-04T13:42:50.085 INFO:tasks.workunit.client.1.smithi169.stdout:5/397: rename d1/d16/d2a/d46/l56 to d1/d16/d17/l7f 0 2024-03-04T13:42:50.087 INFO:tasks.workunit.client.1.smithi169.stdout:1/374: rename d4/d5/lc to d4/d5/d36/d35/d5b/l7a 0 2024-03-04T13:42:50.088 INFO:tasks.workunit.client.1.smithi169.stdout:5/398: chown d1/d16/d17/d1b/f44 95896 1 2024-03-04T13:42:50.089 INFO:tasks.workunit.client.1.smithi169.stdout:7/411: link d2/d6b/c86 d2/de/d3e/c91 0 2024-03-04T13:42:50.090 INFO:tasks.workunit.client.1.smithi169.stdout:5/399: write d1/f7d [464725,82104] 0 2024-03-04T13:42:50.092 INFO:tasks.workunit.client.1.smithi169.stdout:6/385: rename d0/f28 to d0/d7a/f87 0 2024-03-04T13:42:50.092 INFO:tasks.workunit.client.1.smithi169.stdout:5/400: dread - d1/d16/d17/f6f zero size 2024-03-04T13:42:50.093 INFO:tasks.workunit.client.1.smithi169.stdout:1/375: rmdir d4/d5/d2d/d4e/d71/d26/d54 39 2024-03-04T13:42:50.096 INFO:tasks.workunit.client.1.smithi169.stdout:7/412: creat d2/de/d3e/d41/d69/f92 x:0 0 0 2024-03-04T13:42:50.097 INFO:tasks.workunit.client.1.smithi169.stdout:5/401: creat d1/f80 x:0 0 0 2024-03-04T13:42:50.099 INFO:tasks.workunit.client.1.smithi169.stdout:6/386: creat d0/d32/d3a/f88 x:0 0 0 2024-03-04T13:42:50.103 INFO:tasks.workunit.client.1.smithi169.stdout:5/402: unlink d1/d23/c47 0 2024-03-04T13:42:50.104 INFO:tasks.workunit.client.1.smithi169.stdout:5/403: readlink d1/l11 0 2024-03-04T13:42:50.107 INFO:tasks.workunit.client.1.smithi169.stdout:3/368: sync 2024-03-04T13:42:50.110 INFO:tasks.workunit.client.1.smithi169.stdout:1/376: rename d4/d1d/f23 to d4/d5/d2d/d4e/d71/d26/f7b 0 2024-03-04T13:42:50.111 INFO:tasks.workunit.client.1.smithi169.stdout:5/404: mkdir d1/d81 0 2024-03-04T13:42:50.113 INFO:tasks.workunit.client.1.smithi169.stdout:5/405: read - d1/d16/d2f/d39/f5f zero size 2024-03-04T13:42:50.116 INFO:tasks.workunit.client.1.smithi169.stdout:7/413: rmdir d2/de/d3e/d41 39 2024-03-04T13:42:50.121 INFO:tasks.workunit.client.1.smithi169.stdout:1/377: truncate d4/d5/f38 3905351 0 2024-03-04T13:42:50.122 INFO:tasks.workunit.client.1.smithi169.stdout:6/387: getdents d0/d32/d3a/d43/d74 0 2024-03-04T13:42:50.125 INFO:tasks.workunit.client.1.smithi169.stdout:1/378: write d4/f1b [2768742,114276] 0 2024-03-04T13:42:50.125 INFO:tasks.workunit.client.1.smithi169.stdout:6/388: chown d0/d32/d3a/f42 6 1 2024-03-04T13:42:50.126 INFO:tasks.workunit.client.1.smithi169.stdout:7/414: truncate d2/de/d3e/d41/f8b 801012 0 2024-03-04T13:42:50.131 INFO:tasks.workunit.client.1.smithi169.stdout:3/369: write d7/de/d14/d13/d1a/d2d/f37 [179662,113517] 0 2024-03-04T13:42:50.133 INFO:tasks.workunit.client.1.smithi169.stdout:5/406: dwrite d1/d16/d2a/f70 [0,4194304] 0 2024-03-04T13:42:50.140 INFO:tasks.workunit.client.1.smithi169.stdout:5/407: creat d1/d16/d17/d2d/d5a/d67/f82 x:0 0 0 2024-03-04T13:42:50.143 INFO:tasks.workunit.client.1.smithi169.stdout:5/408: write d1/d23/f40 [1928661,22887] 0 2024-03-04T13:42:50.150 INFO:tasks.workunit.client.1.smithi169.stdout:5/409: symlink d1/d16/d17/d2d/d51/l83 0 2024-03-04T13:42:50.151 INFO:tasks.workunit.client.1.smithi169.stdout:6/389: dwrite d0/d32/d3a/f27 [0,4194304] 0 2024-03-04T13:42:50.152 INFO:tasks.workunit.client.1.smithi169.stdout:5/410: stat d1/d16/d2f/d72/f7c 0 2024-03-04T13:42:50.158 INFO:tasks.workunit.client.1.smithi169.stdout:5/411: write d1/d16/d17/d28/f4c [434304,80923] 0 2024-03-04T13:42:50.158 INFO:tasks.workunit.client.1.smithi169.stdout:6/390: creat d0/d32/f89 x:0 0 0 2024-03-04T13:42:50.158 INFO:tasks.workunit.client.1.smithi169.stdout:3/370: dwrite d7/de/d14/d13/d1a/d2d/d56/d5e/d67/f3a [0,4194304] 0 2024-03-04T13:42:50.163 INFO:tasks.workunit.client.1.smithi169.stdout:5/412: write d1/d16/d17/d1b/f25 [3301032,3211] 0 2024-03-04T13:42:50.164 INFO:tasks.workunit.client.1.smithi169.stdout:6/391: creat d0/f8a x:0 0 0 2024-03-04T13:42:50.168 INFO:tasks.workunit.client.1.smithi169.stdout:3/371: creat d7/de/d14/d13/d5a/f80 x:0 0 0 2024-03-04T13:42:50.169 INFO:tasks.workunit.client.1.smithi169.stdout:5/413: mknod d1/d16/d2a/d46/c84 0 2024-03-04T13:42:50.173 INFO:tasks.workunit.client.1.smithi169.stdout:6/392: link d0/d32/d3a/f23 d0/d3c/f8b 0 2024-03-04T13:42:50.174 INFO:tasks.workunit.client.1.smithi169.stdout:5/414: read d1/f24 [2050291,11406] 0 2024-03-04T13:42:50.237 INFO:tasks.workunit.client.1.smithi169.stdout:3/372: truncate d7/de/d14/d13/f4a 285700 0 2024-03-04T13:42:50.237 INFO:tasks.workunit.client.1.smithi169.stdout:7/415: dread d2/de/f2a [0,4194304] 0 2024-03-04T13:42:50.237 INFO:tasks.workunit.client.1.smithi169.stdout:5/415: write d1/d16/d17/d2d/d51/f6d [884829,49748] 0 2024-03-04T13:42:50.237 INFO:tasks.workunit.client.1.smithi169.stdout:3/373: rmdir d7/de/d14/d47 39 2024-03-04T13:42:50.237 INFO:tasks.workunit.client.1.smithi169.stdout:5/416: write d1/d16/d17/f79 [669742,90649] 0 2024-03-04T13:42:50.238 INFO:tasks.workunit.client.1.smithi169.stdout:7/416: creat d2/d28/f93 x:0 0 0 2024-03-04T13:42:50.238 INFO:tasks.workunit.client.1.smithi169.stdout:3/374: read d7/de/d14/d13/f4a [103217,40402] 0 2024-03-04T13:42:50.238 INFO:tasks.workunit.client.1.smithi169.stdout:6/393: truncate d0/d32/d3a/f23 158753 0 2024-03-04T13:42:50.238 INFO:tasks.workunit.client.1.smithi169.stdout:5/417: creat d1/d23/f85 x:0 0 0 2024-03-04T13:42:50.238 INFO:tasks.workunit.client.1.smithi169.stdout:1/379: dread d4/d5/d2d/d4e/d71/d26/f7b [0,4194304] 0 2024-03-04T13:42:50.238 INFO:tasks.workunit.client.1.smithi169.stdout:3/375: write d7/f33 [176601,29894] 0 2024-03-04T13:42:50.238 INFO:tasks.workunit.client.1.smithi169.stdout:6/394: chown d0/d32/d3a/f21 3913 1 2024-03-04T13:42:50.238 INFO:tasks.workunit.client.1.smithi169.stdout:3/376: chown d7/f9 17598972 1 2024-03-04T13:42:50.238 INFO:tasks.workunit.client.1.smithi169.stdout:1/380: symlink d4/d5/d36/d35/df/l7c 0 2024-03-04T13:42:50.238 INFO:tasks.workunit.client.1.smithi169.stdout:6/395: link d0/d63/f78 d0/d3c/d40/d52/f8c 0 2024-03-04T13:42:50.238 INFO:tasks.workunit.client.1.smithi169.stdout:5/418: dwrite d1/d16/d17/f79 [0,4194304] 0 2024-03-04T13:42:50.238 INFO:tasks.workunit.client.1.smithi169.stdout:1/381: truncate d4/d5/d36/d35/df/f1c 2451864 0 2024-03-04T13:42:50.282 INFO:tasks.workunit.client.1.smithi169.stdout:7/417: truncate d2/f22 3324747 0 2024-03-04T13:42:50.283 INFO:tasks.workunit.client.1.smithi169.stdout:7/418: stat d2/de/l47 0 2024-03-04T13:42:50.285 INFO:tasks.workunit.client.1.smithi169.stdout:7/419: chown d2/d15/l51 2004381 1 2024-03-04T13:42:50.286 INFO:tasks.workunit.client.1.smithi169.stdout:7/420: stat d2/de 0 2024-03-04T13:42:50.289 INFO:tasks.workunit.client.1.smithi169.stdout:6/396: dwrite d0/f35 [0,4194304] 0 2024-03-04T13:42:50.293 INFO:tasks.workunit.client.1.smithi169.stdout:7/421: creat d2/de/d3e/d41/d5b/d7a/f94 x:0 0 0 2024-03-04T13:42:50.300 INFO:tasks.workunit.client.1.smithi169.stdout:7/422: unlink d2/d15/c16 0 2024-03-04T13:42:50.306 INFO:tasks.workunit.client.1.smithi169.stdout:6/397: dwrite d0/d32/d3a/f4f [0,4194304] 0 2024-03-04T13:42:50.312 INFO:tasks.workunit.client.1.smithi169.stdout:6/398: mknod d0/d32/d3a/d5e/c8d 0 2024-03-04T13:42:50.317 INFO:tasks.workunit.client.1.smithi169.stdout:7/423: dwrite d2/de/d37/f3f [0,4194304] 0 2024-03-04T13:42:50.319 INFO:tasks.workunit.client.1.smithi169.stdout:5/419: dread d1/d16/d17/d28/f4c [0,4194304] 0 2024-03-04T13:42:50.323 INFO:tasks.workunit.client.1.smithi169.stdout:6/399: link d0/d32/d3a/d5e/l61 d0/d32/d3a/d43/d74/l8e 0 2024-03-04T13:42:50.324 INFO:tasks.workunit.client.1.smithi169.stdout:5/420: truncate d1/d16/d17/d1b/f69 2003324 0 2024-03-04T13:42:50.326 INFO:tasks.workunit.client.1.smithi169.stdout:7/424: creat d2/f95 x:0 0 0 2024-03-04T13:42:50.326 INFO:tasks.workunit.client.1.smithi169.stdout:5/421: chown d1/d16/d2a/l32 68 1 2024-03-04T13:42:50.327 INFO:tasks.workunit.client.1.smithi169.stdout:6/400: read d0/f1 [51798,23467] 0 2024-03-04T13:42:50.331 INFO:tasks.workunit.client.1.smithi169.stdout:5/422: fdatasync d1/f4a 0 2024-03-04T13:42:50.331 INFO:tasks.workunit.client.1.smithi169.stdout:6/401: chown d0/d32/d3a/d43/d74/f71 26 1 2024-03-04T13:42:50.336 INFO:tasks.workunit.client.1.smithi169.stdout:6/402: truncate d0/d3c/d40/f6f 650075 0 2024-03-04T13:42:50.347 INFO:tasks.workunit.client.1.smithi169.stdout:7/425: dwrite d2/d28/f56 [0,4194304] 0 2024-03-04T13:42:50.348 INFO:tasks.workunit.client.1.smithi169.stdout:9/461: sync 2024-03-04T13:42:50.351 INFO:tasks.workunit.client.1.smithi169.stdout:7/426: dread - d2/d28/d70/f87 zero size 2024-03-04T13:42:50.352 INFO:tasks.workunit.client.1.smithi169.stdout:5/423: dwrite d1/d16/d17/d2d/f54 [0,4194304] 0 2024-03-04T13:42:50.361 INFO:tasks.workunit.client.1.smithi169.stdout:9/462: mknod d7/d2d/c97 0 2024-03-04T13:42:50.361 INFO:tasks.workunit.client.1.smithi169.stdout:5/424: mknod d1/d16/d17/d2d/d5a/c86 0 2024-03-04T13:42:50.364 INFO:tasks.workunit.client.1.smithi169.stdout:9/463: write d7/dc/f1e [1224808,67441] 0 2024-03-04T13:42:50.366 INFO:tasks.workunit.client.1.smithi169.stdout:6/403: truncate d0/f16 472662 0 2024-03-04T13:42:50.366 INFO:tasks.workunit.client.1.smithi169.stdout:9/464: stat d7/dc/c57 0 2024-03-04T13:42:50.369 INFO:tasks.workunit.client.1.smithi169.stdout:5/425: stat d1/d16/d17/d28/c78 0 2024-03-04T13:42:50.373 INFO:tasks.workunit.client.1.smithi169.stdout:9/465: fsync d7/f33 0 2024-03-04T13:42:50.377 INFO:tasks.workunit.client.1.smithi169.stdout:7/427: fsync d2/de/d3e/d41/f8b 0 2024-03-04T13:42:50.380 INFO:tasks.workunit.client.1.smithi169.stdout:9/466: rename d7/dc/d12/d20/d4e/f5b to d7/d25/d64/f98 0 2024-03-04T13:42:50.383 INFO:tasks.workunit.client.1.smithi169.stdout:7/428: readlink d2/de/l27 0 2024-03-04T13:42:50.392 INFO:tasks.workunit.client.1.smithi169.stdout:5/426: dwrite d1/d16/d17/d2d/f54 [0,4194304] 0 2024-03-04T13:42:50.394 INFO:tasks.workunit.client.1.smithi169.stdout:6/404: write d0/d32/d3a/f4f [4364838,115940] 0 2024-03-04T13:42:50.398 INFO:tasks.workunit.client.1.smithi169.stdout:6/405: creat d0/d32/f8f x:0 0 0 2024-03-04T13:42:50.398 INFO:tasks.workunit.client.1.smithi169.stdout:5/427: chown d1/d16/d17/f4b 19715 1 2024-03-04T13:42:50.401 INFO:tasks.workunit.client.1.smithi169.stdout:5/428: stat d1/d16/d17/d28/f4c 0 2024-03-04T13:42:50.404 INFO:tasks.workunit.client.1.smithi169.stdout:6/406: symlink d0/d32/d3e/l90 0 2024-03-04T13:42:50.405 INFO:tasks.workunit.client.1.smithi169.stdout:5/429: stat d1/d16/d17/d2d/d5a/d67 0 2024-03-04T13:42:50.407 INFO:tasks.workunit.client.1.smithi169.stdout:6/407: chown d0/f2b 15 1 2024-03-04T13:42:50.407 INFO:tasks.workunit.client.1.smithi169.stdout:7/429: dwrite d2/d15/d17/f5d [0,4194304] 0 2024-03-04T13:42:50.411 INFO:tasks.workunit.client.1.smithi169.stdout:6/408: read d0/d7a/f87 [383290,39974] 0 2024-03-04T13:42:50.412 INFO:tasks.workunit.client.1.smithi169.stdout:9/467: dwrite d7/f10 [0,4194304] 0 2024-03-04T13:42:50.413 INFO:tasks.workunit.client.1.smithi169.stdout:6/409: readlink d0/d32/l36 0 2024-03-04T13:42:50.417 INFO:tasks.workunit.client.1.smithi169.stdout:6/410: write d0/d32/d3a/f88 [323047,107015] 0 2024-03-04T13:42:50.418 INFO:tasks.workunit.client.1.smithi169.stdout:5/430: symlink d1/d16/d17/d2d/d51/l87 0 2024-03-04T13:42:50.419 INFO:tasks.workunit.client.1.smithi169.stdout:7/430: symlink d2/de/d3e/d7e/l96 0 2024-03-04T13:42:50.420 INFO:tasks.workunit.client.1.smithi169.stdout:9/468: mknod d7/d27/d84/c99 0 2024-03-04T13:42:50.424 INFO:tasks.workunit.client.1.smithi169.stdout:7/431: mknod d2/d15/d17/c97 0 2024-03-04T13:42:50.427 INFO:tasks.workunit.client.1.smithi169.stdout:6/411: rename d0/d3c/d40/f69 to d0/d3c/d40/d52/f91 0 2024-03-04T13:42:50.429 INFO:tasks.workunit.client.1.smithi169.stdout:6/412: write d0/f8 [2753434,39403] 0 2024-03-04T13:42:50.429 INFO:tasks.workunit.client.1.smithi169.stdout:5/431: symlink d1/d16/d17/d1b/d74/l88 0 2024-03-04T13:42:50.430 INFO:tasks.workunit.client.1.smithi169.stdout:7/432: write d2/f2d [901211,52842] 0 2024-03-04T13:42:50.432 INFO:tasks.workunit.client.1.smithi169.stdout:7/433: readlink d2/d28/l2e 0 2024-03-04T13:42:50.434 INFO:tasks.workunit.client.1.smithi169.stdout:5/432: creat d1/d16/d17/d2d/f89 x:0 0 0 2024-03-04T13:42:50.437 INFO:tasks.workunit.client.1.smithi169.stdout:5/433: chown d1/d16/d17/d2d/d51/f6e 179 1 2024-03-04T13:42:50.443 INFO:tasks.workunit.client.1.smithi169.stdout:6/413: write d0/f1 [176601,37255] 0 2024-03-04T13:42:50.446 INFO:tasks.workunit.client.1.smithi169.stdout:7/434: write d2/de/d3e/d41/f49 [3743017,11319] 0 2024-03-04T13:42:50.446 INFO:tasks.workunit.client.1.smithi169.stdout:6/414: dread - d0/d3c/d40/d52/f8c zero size 2024-03-04T13:42:50.452 INFO:tasks.workunit.client.1.smithi169.stdout:6/415: mknod d0/d3c/d40/c92 0 2024-03-04T13:42:50.453 INFO:tasks.workunit.client.1.smithi169.stdout:7/435: creat d2/d28/d70/f98 x:0 0 0 2024-03-04T13:42:50.453 INFO:tasks.workunit.client.1.smithi169.stdout:5/434: dwrite d1/f65 [0,4194304] 0 2024-03-04T13:42:50.460 INFO:tasks.workunit.client.1.smithi169.stdout:5/435: mkdir d1/d16/d17/d2d/d5a/d8a 0 2024-03-04T13:42:50.464 INFO:tasks.workunit.client.1.smithi169.stdout:7/436: write d2/d15/d17/d6f/f72 [287229,20612] 0 2024-03-04T13:42:50.465 INFO:tasks.workunit.client.1.smithi169.stdout:5/436: creat d1/d81/f8b x:0 0 0 2024-03-04T13:42:50.475 INFO:tasks.workunit.client.1.smithi169.stdout:7/437: write d2/d28/f33 [3687762,91389] 0 2024-03-04T13:42:50.477 INFO:tasks.workunit.client.1.smithi169.stdout:7/438: dread - d2/d15/d17/f81 zero size 2024-03-04T13:42:50.479 INFO:tasks.workunit.client.1.smithi169.stdout:7/439: dread - d2/de/d3e/d41/d69/f92 zero size 2024-03-04T13:42:50.480 INFO:tasks.workunit.client.1.smithi169.stdout:6/416: dread d0/d32/d3a/d43/f6e [0,4194304] 0 2024-03-04T13:42:50.481 INFO:tasks.workunit.client.1.smithi169.stdout:5/437: dwrite d1/f65 [0,4194304] 0 2024-03-04T13:42:50.481 INFO:tasks.workunit.client.1.smithi169.stdout:6/417: dread - d0/f8a zero size 2024-03-04T13:42:50.481 INFO:tasks.workunit.client.1.smithi169.stdout:7/440: chown d2/d28/f33 41 1 2024-03-04T13:42:50.483 INFO:tasks.workunit.client.1.smithi169.stdout:6/418: dread - d0/d3c/d40/f7d zero size 2024-03-04T13:42:50.487 INFO:tasks.workunit.client.1.smithi169.stdout:7/441: rename d2/de/d3e/d41/d5b/c61 to d2/d15/d17/d6f/c99 0 2024-03-04T13:42:50.490 INFO:tasks.workunit.client.1.smithi169.stdout:6/419: unlink d0/d32/d3a/d43/d74/l84 0 2024-03-04T13:42:50.492 INFO:tasks.workunit.client.1.smithi169.stdout:6/420: readlink d0/d32/d3e/l90 0 2024-03-04T13:42:50.492 INFO:tasks.workunit.client.1.smithi169.stdout:7/442: write d2/de/ff [1675473,53898] 0 2024-03-04T13:42:50.494 INFO:tasks.workunit.client.1.smithi169.stdout:5/438: dread d1/d16/f20 [0,4194304] 0 2024-03-04T13:42:50.495 INFO:tasks.workunit.client.1.smithi169.stdout:7/443: mknod d2/d15/d17/c9a 0 2024-03-04T13:42:50.495 INFO:tasks.workunit.client.1.smithi169.stdout:5/439: chown d1/f75 16840 1 2024-03-04T13:42:50.497 INFO:tasks.workunit.client.1.smithi169.stdout:6/421: mkdir d0/d32/d3a/d43/d74/d93 0 2024-03-04T13:42:50.498 INFO:tasks.workunit.client.1.smithi169.stdout:7/444: write d2/d28/f93 [84482,48210] 0 2024-03-04T13:42:50.502 INFO:tasks.workunit.client.1.smithi169.stdout:6/422: symlink d0/d32/d70/l94 0 2024-03-04T13:42:50.502 INFO:tasks.workunit.client.1.smithi169.stdout:7/445: write d2/f1f [5185035,5735] 0 2024-03-04T13:42:50.505 INFO:tasks.workunit.client.1.smithi169.stdout:6/423: mkdir d0/d3c/d95 0 2024-03-04T13:42:50.505 INFO:tasks.workunit.client.1.smithi169.stdout:7/446: stat d2/de/d3e/d41/d5b/l5c 0 2024-03-04T13:42:50.507 INFO:tasks.workunit.client.1.smithi169.stdout:6/424: chown d0/c19 563975 1 2024-03-04T13:42:50.511 INFO:tasks.workunit.client.1.smithi169.stdout:6/425: write d0/d32/d3a/d43/d74/f62 [346804,40231] 0 2024-03-04T13:42:50.514 INFO:tasks.workunit.client.1.smithi169.stdout:6/426: write d0/f30 [673167,38324] 0 2024-03-04T13:42:50.514 INFO:tasks.workunit.client.1.smithi169.stdout:5/440: dwrite d1/f65 [4194304,4194304] 0 2024-03-04T13:42:50.516 INFO:tasks.workunit.client.1.smithi169.stdout:5/441: dread - d1/d16/d17/d2d/f6a zero size 2024-03-04T13:42:50.519 INFO:tasks.workunit.client.1.smithi169.stdout:6/427: creat d0/d3c/d95/f96 x:0 0 0 2024-03-04T13:42:50.520 INFO:tasks.workunit.client.1.smithi169.stdout:5/442: chown d1/d16/d17/d2d/d5a 3 1 2024-03-04T13:42:50.522 INFO:tasks.workunit.client.1.smithi169.stdout:6/428: chown d0/d3c/d40/c47 0 1 2024-03-04T13:42:50.524 INFO:tasks.workunit.client.1.smithi169.stdout:7/447: dwrite d2/f2d [0,4194304] 0 2024-03-04T13:42:50.524 INFO:tasks.workunit.client.1.smithi169.stdout:5/443: rename d1/d16/f3a to d1/f8c 0 2024-03-04T13:42:50.527 INFO:tasks.workunit.client.1.smithi169.stdout:4/322: sync 2024-03-04T13:42:50.527 INFO:tasks.workunit.client.1.smithi169.stdout:2/388: sync 2024-03-04T13:42:50.529 INFO:tasks.workunit.client.1.smithi169.stdout:6/429: readlink d0/d32/d3a/d43/d74/l8e 0 2024-03-04T13:42:50.534 INFO:tasks.workunit.client.1.smithi169.stdout:5/444: unlink d1/d16/d17/d1b/f34 0 2024-03-04T13:42:50.535 INFO:tasks.workunit.client.1.smithi169.stdout:6/430: dread - d0/d32/f58 zero size 2024-03-04T13:42:50.537 INFO:tasks.workunit.client.1.smithi169.stdout:5/445: fsync d1/d16/d17/d1b/f1e 0 2024-03-04T13:42:50.543 INFO:tasks.workunit.client.1.smithi169.stdout:6/431: mknod d0/d32/c97 0 2024-03-04T13:42:50.562 INFO:tasks.workunit.client.1.smithi169.stdout:4/323: dwrite d4/f5a [0,4194304] 0 2024-03-04T13:42:50.562 INFO:tasks.workunit.client.1.smithi169.stdout:2/389: dwrite d1/f50 [0,4194304] 0 2024-03-04T13:42:50.571 INFO:tasks.workunit.client.1.smithi169.stdout:5/446: dwrite d1/d16/d17/d2d/f6a [0,4194304] 0 2024-03-04T13:42:50.579 INFO:tasks.workunit.client.1.smithi169.stdout:6/432: dwrite d0/d3c/d40/f6f [0,4194304] 0 2024-03-04T13:42:50.579 INFO:tasks.workunit.client.1.smithi169.stdout:2/390: dread - d1/d5/d19/d1e/f82 zero size 2024-03-04T13:42:50.584 INFO:tasks.workunit.client.1.smithi169.stdout:5/447: read d1/d16/d17/f79 [4116132,85756] 0 2024-03-04T13:42:50.586 INFO:tasks.workunit.client.1.smithi169.stdout:9/469: sync 2024-03-04T13:42:50.587 INFO:tasks.workunit.client.1.smithi169.stdout:0/349: sync 2024-03-04T13:42:50.587 INFO:tasks.workunit.client.1.smithi169.stdout:5/448: read - d1/d16/d2a/d46/f6c zero size 2024-03-04T13:42:50.587 INFO:tasks.workunit.client.1.smithi169.stdout:3/377: sync 2024-03-04T13:42:50.587 INFO:tasks.workunit.client.1.smithi169.stdout:1/382: sync 2024-03-04T13:42:50.587 INFO:tasks.workunit.client.1.smithi169.stdout:8/461: sync 2024-03-04T13:42:50.589 INFO:tasks.workunit.client.1.smithi169.stdout:2/391: creat d1/d6d/f87 x:0 0 0 2024-03-04T13:42:50.593 INFO:tasks.workunit.client.1.smithi169.stdout:7/448: dwrite d2/d15/d17/f42 [0,4194304] 0 2024-03-04T13:42:50.596 INFO:tasks.workunit.client.1.smithi169.stdout:8/462: read - d7/d10/d17/d44/d36/f8a zero size 2024-03-04T13:42:50.597 INFO:tasks.workunit.client.1.smithi169.stdout:9/470: write d7/d27/f76 [485794,45165] 0 2024-03-04T13:42:50.598 INFO:tasks.workunit.client.1.smithi169.stdout:5/449: truncate d1/f7d 847359 0 2024-03-04T13:42:50.601 INFO:tasks.workunit.client.1.smithi169.stdout:3/378: write d7/de/d14/d18/f30 [1410552,112474] 0 2024-03-04T13:42:50.603 INFO:tasks.workunit.client.1.smithi169.stdout:4/324: write d4/f69 [192993,63196] 0 2024-03-04T13:42:50.604 INFO:tasks.workunit.client.1.smithi169.stdout:2/392: read d1/d5/f4f [3551201,76049] 0 2024-03-04T13:42:50.609 INFO:tasks.workunit.client.1.smithi169.stdout:5/450: mknod d1/d81/c8d 0 2024-03-04T13:42:50.611 INFO:tasks.workunit.client.1.smithi169.stdout:6/433: write d0/f9 [2695203,69220] 0 2024-03-04T13:42:50.611 INFO:tasks.workunit.client.1.smithi169.stdout:3/379: fdatasync d7/f1d 0 2024-03-04T13:42:50.615 INFO:tasks.workunit.client.1.smithi169.stdout:8/463: mkdir d7/d10/d22/d87/d96 0 2024-03-04T13:42:50.618 INFO:tasks.workunit.client.1.smithi169.stdout:7/449: getdents d2/de/d8a 0 2024-03-04T13:42:50.619 INFO:tasks.workunit.client.1.smithi169.stdout:3/380: dread - d7/de/d14/d18/f7a zero size 2024-03-04T13:42:50.622 INFO:tasks.workunit.client.1.smithi169.stdout:1/383: rename d4/d5/d2d/d4e/f68 to d4/f7d 0 2024-03-04T13:42:50.625 INFO:tasks.workunit.client.1.smithi169.stdout:1/384: chown d4/d5/d36/d75 10223496 1 2024-03-04T13:42:50.625 INFO:tasks.workunit.client.1.smithi169.stdout:6/434: mkdir d0/d3c/d40/d52/d98 0 2024-03-04T13:42:50.626 INFO:tasks.workunit.client.1.smithi169.stdout:2/393: creat d1/d5/d19/d68/f88 x:0 0 0 2024-03-04T13:42:50.629 INFO:tasks.workunit.client.1.smithi169.stdout:4/325: dwrite d4/f69 [0,4194304] 0 2024-03-04T13:42:50.633 INFO:tasks.workunit.client.1.smithi169.stdout:2/394: truncate d1/d5/d22/d3b/f4c 851963 0 2024-03-04T13:42:50.637 INFO:tasks.workunit.client.1.smithi169.stdout:7/450: creat d2/d15/d17/d6f/f9b x:0 0 0 2024-03-04T13:42:50.638 INFO:tasks.workunit.client.1.smithi169.stdout:3/381: rename d7/de/d14/f2a to d7/de/d14/d13/d1a/d2d/d56/f81 0 2024-03-04T13:42:50.638 INFO:tasks.workunit.client.1.smithi169.stdout:9/471: link d7/d27/d84/c99 d7/c9a 0 2024-03-04T13:42:50.641 INFO:tasks.workunit.client.1.smithi169.stdout:9/472: stat d7/dc/f1d 0 2024-03-04T13:42:50.641 INFO:tasks.workunit.client.1.smithi169.stdout:2/395: creat d1/d5/d22/d45/f89 x:0 0 0 2024-03-04T13:42:50.641 INFO:tasks.workunit.client.1.smithi169.stdout:8/464: dread d7/d10/d24/d3e/f57 [0,4194304] 0 2024-03-04T13:42:50.642 INFO:tasks.workunit.client.1.smithi169.stdout:3/382: write d7/d53/f5d [347688,109996] 0 2024-03-04T13:42:50.643 INFO:tasks.workunit.client.1.smithi169.stdout:4/326: mknod d4/d1c/d58/c6a 0 2024-03-04T13:42:50.644 INFO:tasks.workunit.client.1.smithi169.stdout:4/327: stat d4/d11/d2e/d39 0 2024-03-04T13:42:50.645 INFO:tasks.workunit.client.1.smithi169.stdout:6/435: symlink d0/l99 0 2024-03-04T13:42:50.646 INFO:tasks.workunit.client.1.smithi169.stdout:1/385: mkdir d4/d5/d2d/d4e/d71/d26/d7e 0 2024-03-04T13:42:50.649 INFO:tasks.workunit.client.1.smithi169.stdout:7/451: unlink d2/d15/d54/f7c 0 2024-03-04T13:42:50.650 INFO:tasks.workunit.client.1.smithi169.stdout:5/451: link d1/d16/d2a/d46/c84 d1/d16/d17/d2d/c8e 0 2024-03-04T13:42:50.653 INFO:tasks.workunit.client.1.smithi169.stdout:8/465: unlink d7/d10/d22/f5b 0 2024-03-04T13:42:50.654 INFO:tasks.workunit.client.1.smithi169.stdout:2/396: mkdir d1/d5/d19/d1e/d29/d8a 0 2024-03-04T13:42:50.657 INFO:tasks.workunit.client.1.smithi169.stdout:9/473: rename d7/d2d/d48/f6f to d7/dc/f9b 0 2024-03-04T13:42:50.659 INFO:tasks.workunit.client.1.smithi169.stdout:6/436: symlink d0/d32/d3e/l9a 0 2024-03-04T13:42:50.660 INFO:tasks.workunit.client.1.smithi169.stdout:3/383: creat d7/de/d14/d18/d21/d3f/f82 x:0 0 0 2024-03-04T13:42:50.663 INFO:tasks.workunit.client.1.smithi169.stdout:1/386: creat d4/d5/d36/d35/d5b/f7f x:0 0 0 2024-03-04T13:42:50.665 INFO:tasks.workunit.client.1.smithi169.stdout:8/466: symlink d7/d10/d17/d44/l97 0 2024-03-04T13:42:50.667 INFO:tasks.workunit.client.1.smithi169.stdout:2/397: mknod d1/d5/d22/d45/d66/c8b 0 2024-03-04T13:42:50.667 INFO:tasks.workunit.client.1.smithi169.stdout:5/452: write d1/f14 [169126,120103] 0 2024-03-04T13:42:50.669 INFO:tasks.workunit.client.1.smithi169.stdout:9/474: creat d7/d2d/d48/f9c x:0 0 0 2024-03-04T13:42:50.670 INFO:tasks.workunit.client.1.smithi169.stdout:6/437: mknod d0/d32/c9b 0 2024-03-04T13:42:50.672 INFO:tasks.workunit.client.1.smithi169.stdout:7/452: dwrite d2/d6b/f76 [0,4194304] 0 2024-03-04T13:42:50.672 INFO:tasks.workunit.client.1.smithi169.stdout:1/387: truncate d4/f6b 1042392 0 2024-03-04T13:42:50.673 INFO:tasks.workunit.client.1.smithi169.stdout:3/384: creat d7/de/d14/d13/d5a/f83 x:0 0 0 2024-03-04T13:42:50.673 INFO:tasks.workunit.client.1.smithi169.stdout:5/453: dread - d1/f80 zero size 2024-03-04T13:42:50.674 INFO:tasks.workunit.client.1.smithi169.stdout:2/398: write d1/d6d/f87 [98420,36382] 0 2024-03-04T13:42:50.682 INFO:tasks.workunit.client.1.smithi169.stdout:7/453: write d2/de/d3e/d41/f8b [11086,101222] 0 2024-03-04T13:42:50.685 INFO:tasks.workunit.client.1.smithi169.stdout:9/475: creat d7/dc/d12/d32/d42/f9d x:0 0 0 2024-03-04T13:42:50.686 INFO:tasks.workunit.client.1.smithi169.stdout:1/388: mknod d4/d5/d36/d35/df/c80 0 2024-03-04T13:42:50.687 INFO:tasks.workunit.client.1.smithi169.stdout:8/467: creat d7/d10/d17/f98 x:0 0 0 2024-03-04T13:42:50.689 INFO:tasks.workunit.client.1.smithi169.stdout:6/438: unlink d0/d32/d3a/d43/c54 0 2024-03-04T13:42:50.689 INFO:tasks.workunit.client.1.smithi169.stdout:4/328: dwrite d4/d11/d2e/d39/f60 [0,4194304] 0 2024-03-04T13:42:50.690 INFO:tasks.workunit.client.1.smithi169.stdout:1/389: fsync d4/d5/f49 0 2024-03-04T13:42:50.692 INFO:tasks.workunit.client.1.smithi169.stdout:6/439: chown d0/d3c/d40/f7d 107904 1 2024-03-04T13:42:50.694 INFO:tasks.workunit.client.1.smithi169.stdout:5/454: mknod d1/d16/c8f 0 2024-03-04T13:42:50.694 INFO:tasks.workunit.client.1.smithi169.stdout:4/329: write d4/d11/d2e/d39/d48/f3c [1768364,89196] 0 2024-03-04T13:42:50.695 INFO:tasks.workunit.client.1.smithi169.stdout:8/468: read d7/d10/d24/f7a [1550058,22901] 0 2024-03-04T13:42:50.700 INFO:tasks.workunit.client.1.smithi169.stdout:8/469: dread - d7/d10/d17/d44/d36/f8a zero size 2024-03-04T13:42:50.702 INFO:tasks.workunit.client.1.smithi169.stdout:5/455: truncate d1/d23/f85 76927 0 2024-03-04T13:42:50.708 INFO:tasks.workunit.client.1.smithi169.stdout:1/390: mkdir d4/d5/d36/d35/d16/d81 0 2024-03-04T13:42:50.712 INFO:tasks.workunit.client.1.smithi169.stdout:1/391: chown d4/d5/d2d/d4e/d71/d26/d4f/f6e 23420 1 2024-03-04T13:42:50.712 INFO:tasks.workunit.client.1.smithi169.stdout:9/476: creat d7/d25/d3c/f9e x:0 0 0 2024-03-04T13:42:50.712 INFO:tasks.workunit.client.1.smithi169.stdout:4/330: mkdir d4/d11/d2e/d39/d61/d6b 0 2024-03-04T13:42:50.713 INFO:tasks.workunit.client.1.smithi169.stdout:7/454: symlink d2/de/d37/l9c 0 2024-03-04T13:42:50.716 INFO:tasks.workunit.client.1.smithi169.stdout:1/392: truncate d4/f25 5058070 0 2024-03-04T13:42:50.716 INFO:tasks.workunit.client.1.smithi169.stdout:7/455: dread - d2/d28/d70/f98 zero size 2024-03-04T13:42:50.717 INFO:tasks.workunit.client.1.smithi169.stdout:4/331: write d4/d11/d2e/d39/d48/f3c [1989790,104051] 0 2024-03-04T13:42:50.719 INFO:tasks.workunit.client.1.smithi169.stdout:3/385: mkdir d7/de/d14/d13/d1a/d2d/d56/d5e/d67/d5c/d84 0 2024-03-04T13:42:50.721 INFO:tasks.workunit.client.1.smithi169.stdout:5/456: write d1/d16/d2f/d72/f7c [5125248,42835] 0 2024-03-04T13:42:50.721 INFO:tasks.workunit.client.1.smithi169.stdout:2/399: rename d1/d5/f8 to d1/d5/d19/f8c 0 2024-03-04T13:42:50.724 INFO:tasks.workunit.client.1.smithi169.stdout:1/393: write d4/d5/f57 [160467,63913] 0 2024-03-04T13:42:50.724 INFO:tasks.workunit.client.1.smithi169.stdout:8/470: dread d7/f6a [0,4194304] 0 2024-03-04T13:42:50.725 INFO:tasks.workunit.client.1.smithi169.stdout:9/477: dread d7/dc/f79 [0,4194304] 0 2024-03-04T13:42:50.733 INFO:tasks.workunit.client.1.smithi169.stdout:3/386: fsync d7/de/d14/d13/f4a 0 2024-03-04T13:42:50.734 INFO:tasks.workunit.client.1.smithi169.stdout:4/332: write d4/d11/d2e/f51 [3800079,122401] 0 2024-03-04T13:42:50.736 INFO:tasks.workunit.client.1.smithi169.stdout:1/394: write d4/f7d [62171,120244] 0 2024-03-04T13:42:50.737 INFO:tasks.workunit.client.1.smithi169.stdout:8/471: symlink d7/d10/d24/d3e/l99 0 2024-03-04T13:42:50.741 INFO:tasks.workunit.client.1.smithi169.stdout:8/472: fdatasync d7/d10/d24/d3e/f95 0 2024-03-04T13:42:50.741 INFO:tasks.workunit.client.1.smithi169.stdout:3/387: dread d7/de/d14/d13/f4a [0,4194304] 0 2024-03-04T13:42:50.741 INFO:tasks.workunit.client.1.smithi169.stdout:4/333: write d4/d1c/f2c [1397548,65644] 0 2024-03-04T13:42:50.743 INFO:tasks.workunit.client.1.smithi169.stdout:6/440: dread d0/d3c/d40/f51 [0,4194304] 0 2024-03-04T13:42:50.748 INFO:tasks.workunit.client.1.smithi169.stdout:7/456: link d2/c31 d2/de/d3e/d41/d5b/d7a/c9d 0 2024-03-04T13:42:50.749 INFO:tasks.workunit.client.1.smithi169.stdout:8/473: write d7/d10/d24/f6b [1971192,1875] 0 2024-03-04T13:42:50.750 INFO:tasks.workunit.client.1.smithi169.stdout:9/478: link d7/d25/l37 d7/d49/l9f 0 2024-03-04T13:42:50.757 INFO:tasks.workunit.client.1.smithi169.stdout:3/388: truncate d7/de/d14/f20 1619120 0 2024-03-04T13:42:50.759 INFO:tasks.workunit.client.1.smithi169.stdout:5/457: dwrite d1/d16/d17/d1b/f29 [0,4194304] 0 2024-03-04T13:42:50.762 INFO:tasks.workunit.client.1.smithi169.stdout:4/334: mkdir d4/d11/d2e/d6c 0 2024-03-04T13:42:50.766 INFO:tasks.workunit.client.1.smithi169.stdout:7/457: symlink d2/de/d3e/d41/d5b/l9e 0 2024-03-04T13:42:50.769 INFO:tasks.workunit.client.1.smithi169.stdout:2/400: dwrite d1/d5/d22/d45/f72 [0,4194304] 0 2024-03-04T13:42:50.773 INFO:tasks.workunit.client.1.smithi169.stdout:9/479: symlink d7/d49/la0 0 2024-03-04T13:42:50.773 INFO:tasks.workunit.client.1.smithi169.stdout:8/474: link d7/d10/d24/f71 d7/d10/d17/d44/d5d/f9a 0 2024-03-04T13:42:50.773 INFO:tasks.workunit.client.1.smithi169.stdout:7/458: readlink d2/de/l67 0 2024-03-04T13:42:50.774 INFO:tasks.workunit.client.1.smithi169.stdout:1/395: dwrite d4/d5/d36/d35/d16/d3a/f3f [0,4194304] 0 2024-03-04T13:42:50.779 INFO:tasks.workunit.client.1.smithi169.stdout:2/401: rmdir d1/d5/d19 39 2024-03-04T13:42:50.784 INFO:tasks.workunit.client.1.smithi169.stdout:5/458: mknod d1/d16/d17/d1b/d74/c90 0 2024-03-04T13:42:50.784 INFO:tasks.workunit.client.1.smithi169.stdout:9/480: fsync d7/d27/f5a 0 2024-03-04T13:42:50.785 INFO:tasks.workunit.client.1.smithi169.stdout:0/350: sync 2024-03-04T13:42:50.791 INFO:tasks.workunit.client.1.smithi169.stdout:1/396: fdatasync d4/d5/d2d/d4e/d71/d26/d54/f72 0 2024-03-04T13:42:50.793 INFO:tasks.workunit.client.1.smithi169.stdout:2/402: write d1/d5/d19/f47 [3048774,25055] 0 2024-03-04T13:42:50.794 INFO:tasks.workunit.client.1.smithi169.stdout:0/351: write d5/f37 [1455363,110003] 0 2024-03-04T13:42:50.803 INFO:tasks.workunit.client.1.smithi169.stdout:8/475: link d7/d12/f5e d7/f9b 0 2024-03-04T13:42:50.806 INFO:tasks.workunit.client.1.smithi169.stdout:9/481: mknod d7/dc/d12/ca1 0 2024-03-04T13:42:50.810 INFO:tasks.workunit.client.1.smithi169.stdout:0/352: unlink d5/d9/dc/l4d 0 2024-03-04T13:42:50.813 INFO:tasks.workunit.client.1.smithi169.stdout:0/353: chown d5/l17 1597904 1 2024-03-04T13:42:50.815 INFO:tasks.workunit.client.1.smithi169.stdout:5/459: dwrite d1/d16/d17/d1b/f29 [0,4194304] 0 2024-03-04T13:42:50.817 INFO:tasks.workunit.client.1.smithi169.stdout:0/354: fdatasync d5/d9/dc/d12/f5c 0 2024-03-04T13:42:50.818 INFO:tasks.workunit.client.1.smithi169.stdout:5/460: chown d1/f7d 1220364713 1 2024-03-04T13:42:50.819 INFO:tasks.workunit.client.1.smithi169.stdout:1/397: mkdir d4/d5/d2d/d4e/d71/d82 0 2024-03-04T13:42:50.824 INFO:tasks.workunit.client.1.smithi169.stdout:4/335: truncate d4/d11/f1b 106799 0 2024-03-04T13:42:50.824 INFO:tasks.workunit.client.1.smithi169.stdout:2/403: mknod d1/d5/d22/d45/c8d 0 2024-03-04T13:42:50.827 INFO:tasks.workunit.client.1.smithi169.stdout:7/459: write d2/f22 [3004387,28216] 0 2024-03-04T13:42:50.835 INFO:tasks.workunit.client.1.smithi169.stdout:5/461: mkdir d1/d81/d91 0 2024-03-04T13:42:50.838 INFO:tasks.workunit.client.1.smithi169.stdout:6/441: dread d0/f8 [0,4194304] 0 2024-03-04T13:42:50.839 INFO:tasks.workunit.client.1.smithi169.stdout:5/462: write d1/d16/d17/d1b/f25 [2162448,69969] 0 2024-03-04T13:42:50.841 INFO:tasks.workunit.client.1.smithi169.stdout:0/355: mkdir d5/d9/d3d/d59/d64/d7f 0 2024-03-04T13:42:50.842 INFO:tasks.workunit.client.1.smithi169.stdout:6/442: stat d0/d3c/d40/d52/f91 0 2024-03-04T13:42:50.843 INFO:tasks.workunit.client.1.smithi169.stdout:2/404: creat d1/d5/d19/d63/f8e x:0 0 0 2024-03-04T13:42:50.848 INFO:tasks.workunit.client.1.smithi169.stdout:4/336: symlink d4/d11/d2e/d39/d48/l6d 0 2024-03-04T13:42:50.850 INFO:tasks.workunit.client.1.smithi169.stdout:4/337: stat d4/d11/d2e/d21/c40 0 2024-03-04T13:42:50.853 INFO:tasks.workunit.client.1.smithi169.stdout:7/460: creat d2/de/d3e/d41/d5b/f9f x:0 0 0 2024-03-04T13:42:50.853 INFO:tasks.workunit.client.1.smithi169.stdout:3/389: dread f5 [0,4194304] 0 2024-03-04T13:42:50.892 INFO:tasks.workunit.client.1.smithi169.stdout:1/398: truncate d4/d5/d36/d35/df/f2b 1031100 0 2024-03-04T13:42:50.894 INFO:tasks.workunit.client.1.smithi169.stdout:5/463: write d1/f24 [5138373,30316] 0 2024-03-04T13:42:50.896 INFO:tasks.workunit.client.1.smithi169.stdout:0/356: readlink d5/d9/l73 0 2024-03-04T13:42:50.899 INFO:tasks.workunit.client.1.smithi169.stdout:5/464: fsync d1/f24 0 2024-03-04T13:42:50.909 INFO:tasks.workunit.client.1.smithi169.stdout:3/390: rename d7/de/d14 to d7/de/d2b/d7d/d85 0 2024-03-04T13:42:50.913 INFO:tasks.workunit.client.1.smithi169.stdout:9/482: truncate d7/dc/f80 925845 0 2024-03-04T13:42:50.995 INFO:tasks.workunit.client.1.smithi169.stdout:0/357: symlink d5/d9/d3d/l80 0 2024-03-04T13:42:50.995 INFO:tasks.workunit.client.1.smithi169.stdout:6/443: fdatasync d0/d32/d3a/d43/f45 0 2024-03-04T13:42:50.996 INFO:tasks.workunit.client.1.smithi169.stdout:4/338: mkdir d4/d11/d2e/d21/d5e/d6e 0 2024-03-04T13:42:50.997 INFO:tasks.workunit.client.1.smithi169.stdout:7/461: symlink d2/de/d3e/la0 0 2024-03-04T13:42:50.998 INFO:tasks.workunit.client.1.smithi169.stdout:9/483: write d7/d2d/d48/f9c [240090,118706] 0 2024-03-04T13:42:51.000 INFO:tasks.workunit.client.1.smithi169.stdout:6/444: truncate d0/f8a 893266 0 2024-03-04T13:42:51.002 INFO:tasks.workunit.client.1.smithi169.stdout:1/399: link d4/d5/d36/d35/d16/c65 d4/d5/d36/d35/d5b/c83 0 2024-03-04T13:42:51.004 INFO:tasks.workunit.client.1.smithi169.stdout:9/484: mknod d7/d49/ca2 0 2024-03-04T13:42:51.005 INFO:tasks.workunit.client.1.smithi169.stdout:9/485: chown d7/d2d/d48 2093471 1 2024-03-04T13:42:51.007 INFO:tasks.workunit.client.1.smithi169.stdout:6/445: unlink d0/ld 0 2024-03-04T13:42:51.009 INFO:tasks.workunit.client.1.smithi169.stdout:9/486: write d7/dc/d12/d59/f6d [3770259,50162] 0 2024-03-04T13:42:51.009 INFO:tasks.workunit.client.1.smithi169.stdout:2/405: getdents d1/d5 0 2024-03-04T13:42:51.011 INFO:tasks.workunit.client.1.smithi169.stdout:4/339: getdents d4/d11/d2e/d21/d5e/d6e 0 2024-03-04T13:42:51.012 INFO:tasks.workunit.client.1.smithi169.stdout:1/400: creat d4/d5/d2d/d4e/f84 x:0 0 0 2024-03-04T13:42:51.014 INFO:tasks.workunit.client.1.smithi169.stdout:6/446: truncate d0/d3c/d95/f96 31046 0 2024-03-04T13:42:51.015 INFO:tasks.workunit.client.1.smithi169.stdout:9/487: chown d7/d25/d3c/f9e 0 1 2024-03-04T13:42:51.019 INFO:tasks.workunit.client.1.smithi169.stdout:0/358: dread d5/d9/dc/d12/f44 [0,4194304] 0 2024-03-04T13:42:51.021 INFO:tasks.workunit.client.1.smithi169.stdout:6/447: unlink d0/d32/d3a/d43/f6e 0 2024-03-04T13:42:51.022 INFO:tasks.workunit.client.1.smithi169.stdout:7/462: dwrite d2/de/d3e/d41/f8b [0,4194304] 0 2024-03-04T13:42:51.025 INFO:tasks.workunit.client.1.smithi169.stdout:4/340: symlink d4/d11/d2e/d39/l6f 0 2024-03-04T13:42:51.030 INFO:tasks.workunit.client.1.smithi169.stdout:0/359: creat d5/d9/d3d/d59/d64/f81 x:0 0 0 2024-03-04T13:42:51.031 INFO:tasks.workunit.client.1.smithi169.stdout:2/406: link d1/d6d/f87 d1/d5/d22/d45/f8f 0 2024-03-04T13:42:51.033 INFO:tasks.workunit.client.1.smithi169.stdout:6/448: stat d0/d63/c64 0 2024-03-04T13:42:51.035 INFO:tasks.workunit.client.1.smithi169.stdout:2/407: chown d1/d5/d19/d3c/f5f 19712 1 2024-03-04T13:42:51.035 INFO:tasks.workunit.client.1.smithi169.stdout:7/463: creat d2/d28/fa1 x:0 0 0 2024-03-04T13:42:51.037 INFO:tasks.workunit.client.1.smithi169.stdout:3/391: truncate d7/de/d2b/d7d/d85/d18/f30 459159 0 2024-03-04T13:42:51.039 INFO:tasks.workunit.client.1.smithi169.stdout:1/401: dwrite d4/d5/d36/d35/df/f46 [0,4194304] 0 2024-03-04T13:42:51.040 INFO:tasks.workunit.client.1.smithi169.stdout:7/464: dread - d2/de/d3e/d41/d5b/d7a/f94 zero size 2024-03-04T13:42:51.043 INFO:tasks.workunit.client.1.smithi169.stdout:4/341: symlink d4/d11/d2e/d63/l70 0 2024-03-04T13:42:51.044 INFO:tasks.workunit.client.1.smithi169.stdout:1/402: truncate d4/d1d/f34 534686 0 2024-03-04T13:42:51.044 INFO:tasks.workunit.client.1.smithi169.stdout:2/408: rmdir d1/d5/d19/d63 39 2024-03-04T13:42:51.046 INFO:tasks.workunit.client.1.smithi169.stdout:1/403: chown d4/d5/d36/d35/d70/d78 16649506 1 2024-03-04T13:42:51.048 INFO:tasks.workunit.client.1.smithi169.stdout:3/392: mkdir d7/de/d2b/d7d/d85/d18/d21/d3f/d86 0 2024-03-04T13:42:51.053 INFO:tasks.workunit.client.1.smithi169.stdout:1/404: rmdir d4/d5/d36/d35/d70 39 2024-03-04T13:42:51.054 INFO:tasks.workunit.client.1.smithi169.stdout:2/409: dread d1/d5/d19/d69/f7e [0,4194304] 0 2024-03-04T13:42:51.055 INFO:tasks.workunit.client.1.smithi169.stdout:4/342: getdents d4/d11/d2e/d21/d5e/d6e 0 2024-03-04T13:42:51.055 INFO:tasks.workunit.client.1.smithi169.stdout:1/405: readlink d4/d5/l14 0 2024-03-04T13:42:51.058 INFO:tasks.workunit.client.1.smithi169.stdout:4/343: chown d4/d1c/d58/c6a 227 1 2024-03-04T13:42:51.060 INFO:tasks.workunit.client.1.smithi169.stdout:4/344: readlink d4/d11/l4f 0 2024-03-04T13:42:51.061 INFO:tasks.workunit.client.1.smithi169.stdout:2/410: write d1/d5/d19/f47 [4289770,79654] 0 2024-03-04T13:42:51.062 INFO:tasks.workunit.client.1.smithi169.stdout:1/406: read d4/d5/d36/d35/f56 [245763,76106] 0 2024-03-04T13:42:51.070 INFO:tasks.workunit.client.1.smithi169.stdout:7/465: write d2/f7 [3678424,43232] 0 2024-03-04T13:42:51.071 INFO:tasks.workunit.client.1.smithi169.stdout:9/488: dwrite d7/dc/d12/d32/d42/f46 [4194304,4194304] 0 2024-03-04T13:42:51.075 INFO:tasks.workunit.client.1.smithi169.stdout:3/393: truncate d7/de/d2b/d7d/d85/d13/d1a/d2d/f37 808290 0 2024-03-04T13:42:51.078 INFO:tasks.workunit.client.1.smithi169.stdout:7/466: symlink d2/de/d3e/d41/d89/la2 0 2024-03-04T13:42:51.082 INFO:tasks.workunit.client.1.smithi169.stdout:3/394: mknod d7/de/d2b/d7d/d85/d47/c87 0 2024-03-04T13:42:51.082 INFO:tasks.workunit.client.1.smithi169.stdout:7/467: mknod d2/de/d3e/d41/d5b/ca3 0 2024-03-04T13:42:51.085 INFO:tasks.workunit.client.1.smithi169.stdout:7/468: write d2/de/d37/f3f [739643,115823] 0 2024-03-04T13:42:51.090 INFO:tasks.workunit.client.1.smithi169.stdout:3/395: creat d7/de/d2b/d7d/d85/d47/d63/d6c/f88 x:0 0 0 2024-03-04T13:42:51.090 INFO:tasks.workunit.client.1.smithi169.stdout:9/489: read d7/f71 [7350693,56320] 0 2024-03-04T13:42:51.090 INFO:tasks.workunit.client.1.smithi169.stdout:7/469: unlink d2/f1f 0 2024-03-04T13:42:51.092 INFO:tasks.workunit.client.1.smithi169.stdout:9/490: chown d7/d2d/c6a 252628109 1 2024-03-04T13:42:51.094 INFO:tasks.workunit.client.1.smithi169.stdout:1/407: truncate d4/d5/d36/d35/f56 2903342 0 2024-03-04T13:42:51.094 INFO:tasks.workunit.client.1.smithi169.stdout:4/345: truncate d4/fa 954214 0 2024-03-04T13:42:51.096 INFO:tasks.workunit.client.1.smithi169.stdout:3/396: dread - d7/de/d2b/d7d/d85/d18/d21/d66/f75 zero size 2024-03-04T13:42:51.097 INFO:tasks.workunit.client.1.smithi169.stdout:4/346: read d4/d11/f4b [1187807,121320] 0 2024-03-04T13:42:51.099 INFO:tasks.workunit.client.1.smithi169.stdout:0/360: dread d5/d9/d3d/f48 [0,4194304] 0 2024-03-04T13:42:51.101 INFO:tasks.workunit.client.1.smithi169.stdout:7/470: unlink d2/d28/d68/l83 0 2024-03-04T13:42:51.103 INFO:tasks.workunit.client.1.smithi169.stdout:9/491: write d7/dc/f1d [907165,86661] 0 2024-03-04T13:42:51.106 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:42:51 smithi169 ceph-mon[36313]: pgmap v47: 65 pgs: 65 active+clean; 877 MiB data, 4.2 GiB used, 532 GiB / 536 GiB avail; 19 MiB/s rd, 118 MiB/s wr, 213 op/s 2024-03-04T13:42:51.108 INFO:tasks.workunit.client.1.smithi169.stdout:0/361: read - d5/d9/dc/d12/d15/d46/f57 zero size 2024-03-04T13:42:51.108 INFO:tasks.workunit.client.1.smithi169.stdout:1/408: getdents d4/d5/d2d/d4e/d71/d26 0 2024-03-04T13:42:51.112 INFO:tasks.workunit.client.1.smithi169.stdout:4/347: rename d4/d1c/l52 to d4/d1c/d58/l71 0 2024-03-04T13:42:51.113 INFO:tasks.workunit.client.1.smithi169.stdout:7/471: mknod d2/de/d3e/d41/d89/ca4 0 2024-03-04T13:42:51.115 INFO:tasks.workunit.client.1.smithi169.stdout:6/449: dread d0/d32/d3a/d43/d74/f71 [0,4194304] 0 2024-03-04T13:42:51.119 INFO:tasks.workunit.client.1.smithi169.stdout:0/362: mkdir d5/d9/dc/d12/d15/d46/d82 0 2024-03-04T13:42:51.120 INFO:tasks.workunit.client.1.smithi169.stdout:1/409: mknod d4/d5/d2d/d4e/d71/d26/d54/c85 0 2024-03-04T13:42:51.122 INFO:tasks.workunit.client.1.smithi169.stdout:4/348: mkdir d4/d11/d2e/d21/d5e/d72 0 2024-03-04T13:42:51.126 INFO:tasks.workunit.client.1.smithi169.stdout:4/349: write d4/d11/d2e/d21/f29 [6693051,122163] 0 2024-03-04T13:42:51.126 INFO:tasks.workunit.client.1.smithi169.stdout:0/363: write d5/d9/dc/d12/d15/d19/d61/f68 [255663,96874] 0 2024-03-04T13:42:51.128 INFO:tasks.workunit.client.1.smithi169.stdout:3/397: dread d7/de/d2b/d7d/d85/d18/d21/d3f/d68/f4b [0,4194304] 0 2024-03-04T13:42:51.129 INFO:tasks.workunit.client.1.smithi169.stdout:7/472: rename d2/d28/d68/l80 to d2/de/d3e/d41/la5 0 2024-03-04T13:42:51.130 INFO:tasks.workunit.client.1.smithi169.stdout:3/398: stat d7/de/d2b/f52 0 2024-03-04T13:42:51.137 INFO:tasks.workunit.client.1.smithi169.stdout:4/350: unlink d4/d11/d2e/d39/l6f 0 2024-03-04T13:42:51.139 INFO:tasks.workunit.client.1.smithi169.stdout:4/351: chown d4/d11/f30 817336 1 2024-03-04T13:42:51.140 INFO:tasks.workunit.client.1.smithi169.stdout:7/473: write d2/d28/f4b [1437522,129945] 0 2024-03-04T13:42:51.142 INFO:tasks.workunit.client.1.smithi169.stdout:3/399: creat d7/de/d24/f89 x:0 0 0 2024-03-04T13:42:51.143 INFO:tasks.workunit.client.1.smithi169.stdout:1/410: mkdir d4/d5/d86 0 2024-03-04T13:42:51.144 INFO:tasks.workunit.client.1.smithi169.stdout:9/492: truncate d7/dc/f1d 834782 0 2024-03-04T13:42:51.145 INFO:tasks.workunit.client.1.smithi169.stdout:0/364: mknod d5/d9/dc/d12/d15/d46/d82/c83 0 2024-03-04T13:42:51.153 INFO:tasks.workunit.client.1.smithi169.stdout:9/493: mkdir d7/dc/d3e/da3 0 2024-03-04T13:42:51.154 INFO:tasks.workunit.client.1.smithi169.stdout:0/365: write d5/d9/f6e [331328,22567] 0 2024-03-04T13:42:51.158 INFO:tasks.workunit.client.1.smithi169.stdout:6/450: dwrite d0/f16 [0,4194304] 0 2024-03-04T13:42:51.161 INFO:tasks.workunit.client.1.smithi169.stdout:7/474: creat d2/d15/fa6 x:0 0 0 2024-03-04T13:42:51.162 INFO:tasks.workunit.client.1.smithi169.stdout:3/400: creat d7/de/d2b/d7d/d85/d18/d21/f8a x:0 0 0 2024-03-04T13:42:51.163 INFO:tasks.workunit.client.1.smithi169.stdout:6/451: rename d0/d32 to d0/d32/d3e/d9c 22 2024-03-04T13:42:51.171 INFO:tasks.workunit.client.1.smithi169.stdout:0/366: symlink d5/d9/dc/d12/l84 0 2024-03-04T13:42:51.171 INFO:tasks.workunit.client.1.smithi169.stdout:3/401: rmdir d7/de/d2b/d7d 39 2024-03-04T13:42:51.171 INFO:tasks.workunit.client.1.smithi169.stdout:0/367: stat d5/d9/dc/d12/d15/d19/d61/f5a 0 2024-03-04T13:42:51.172 INFO:tasks.workunit.client.1.smithi169.stdout:6/452: creat d0/d3c/d40/f9d x:0 0 0 2024-03-04T13:42:51.174 INFO:tasks.workunit.client.1.smithi169.stdout:7/475: creat d2/de/d3e/d7e/fa7 x:0 0 0 2024-03-04T13:42:51.184 INFO:tasks.workunit.client.1.smithi169.stdout:3/402: mknod d7/de/d2b/d7d/d85/d18/d21/d66/c8b 0 2024-03-04T13:42:51.188 INFO:tasks.workunit.client.1.smithi169.stdout:3/403: fdatasync d7/f33 0 2024-03-04T13:42:51.192 INFO:tasks.workunit.client.1.smithi169.stdout:1/411: dwrite d4/d5/d36/f50 [0,4194304] 0 2024-03-04T13:42:51.192 INFO:tasks.workunit.client.1.smithi169.stdout:4/352: dwrite d4/f7 [0,4194304] 0 2024-03-04T13:42:51.199 INFO:tasks.workunit.client.1.smithi169.stdout:7/476: dwrite d2/d15/d17/d6f/f72 [0,4194304] 0 2024-03-04T13:42:51.200 INFO:tasks.workunit.client.1.smithi169.stdout:4/353: mkdir d4/d11/d2e/d37/d73 0 2024-03-04T13:42:51.200 INFO:tasks.workunit.client.1.smithi169.stdout:6/453: truncate d0/d3c/d40/f6f 4190023 0 2024-03-04T13:42:51.203 INFO:tasks.workunit.client.1.smithi169.stdout:1/412: read d4/d5/d36/f59 [4033688,35728] 0 2024-03-04T13:42:51.205 INFO:tasks.workunit.client.1.smithi169.stdout:7/477: mknod d2/de/d3e/d41/d69/d78/ca8 0 2024-03-04T13:42:51.205 INFO:tasks.workunit.client.1.smithi169.stdout:4/354: fdatasync d4/d11/f4b 0 2024-03-04T13:42:51.206 INFO:tasks.workunit.client.1.smithi169.stdout:1/413: readlink d4/d5/d2d/d4e/d71/l62 0 2024-03-04T13:42:51.209 INFO:tasks.workunit.client.1.smithi169.stdout:0/368: dwrite d5/d9/dc/d12/f44 [4194304,4194304] 0 2024-03-04T13:42:51.210 INFO:tasks.workunit.client.1.smithi169.stdout:3/404: getdents d7/de/d2b/d7d/d85/d18/d21/d66 0 2024-03-04T13:42:51.218 INFO:tasks.workunit.client.1.smithi169.stdout:3/405: creat d7/d53/f8c x:0 0 0 2024-03-04T13:42:51.220 INFO:tasks.workunit.client.1.smithi169.stdout:1/414: rename d4/d5/d36/f50 to d4/d5/d36/d35/d5b/f87 0 2024-03-04T13:42:51.224 INFO:tasks.workunit.client.1.smithi169.stdout:4/355: getdents d4/d11/d2e/d21 0 2024-03-04T13:42:51.225 INFO:tasks.workunit.client.1.smithi169.stdout:1/415: write d4/f7d [338058,21812] 0 2024-03-04T13:42:51.225 INFO:tasks.workunit.client.1.smithi169.stdout:3/406: readlink d7/de/d2b/d7d/d85/d18/l26 0 2024-03-04T13:42:51.230 INFO:tasks.workunit.client.1.smithi169.stdout:3/407: fdatasync d7/fb 0 2024-03-04T13:42:51.235 INFO:tasks.workunit.client.1.smithi169.stdout:7/478: dwrite d2/de/d3e/d41/f71 [0,4194304] 0 2024-03-04T13:42:51.253 INFO:tasks.workunit.client.1.smithi169.stdout:8/476: sync 2024-03-04T13:42:51.256 INFO:tasks.workunit.client.1.smithi169.stdout:4/356: mkdir d4/d11/d2e/d21/d5e/d72/d74 0 2024-03-04T13:42:51.259 INFO:tasks.workunit.client.1.smithi169.stdout:4/357: chown d4/c49 134606945 1 2024-03-04T13:42:51.260 INFO:tasks.workunit.client.1.smithi169.stdout:7/479: creat d2/de/d3e/d41/fa9 x:0 0 0 2024-03-04T13:42:51.262 INFO:tasks.workunit.client.1.smithi169.stdout:8/477: mknod d7/d10/d17/d44/d4c/c9c 0 2024-03-04T13:42:51.263 INFO:tasks.workunit.client.1.smithi169.stdout:1/416: dwrite d4/d5/d36/d35/df/f1c [0,4194304] 0 2024-03-04T13:42:51.263 INFO:tasks.workunit.client.1.smithi169.stdout:3/408: symlink d7/de/d2b/d7d/d85/d18/d21/d3f/d86/l8d 0 2024-03-04T13:42:51.267 INFO:tasks.workunit.client.1.smithi169.stdout:1/417: creat d4/d5/d36/f88 x:0 0 0 2024-03-04T13:42:51.268 INFO:tasks.workunit.client.1.smithi169.stdout:3/409: write d7/f36 [783024,126638] 0 2024-03-04T13:42:51.269 INFO:tasks.workunit.client.1.smithi169.stdout:1/418: fdatasync d4/d5/d36/f88 0 2024-03-04T13:42:51.271 INFO:tasks.workunit.client.1.smithi169.stdout:7/480: unlink d2/de/l58 0 2024-03-04T13:42:51.272 INFO:tasks.workunit.client.1.smithi169.stdout:0/369: dread d5/f8 [0,4194304] 0 2024-03-04T13:42:51.274 INFO:tasks.workunit.client.1.smithi169.stdout:8/478: getdents d7/d10/d22/d75/d79 0 2024-03-04T13:42:51.275 INFO:tasks.workunit.client.1.smithi169.stdout:0/370: symlink d5/l85 0 2024-03-04T13:42:51.278 INFO:tasks.workunit.client.1.smithi169.stdout:3/410: truncate d7/de/d2b/d7d/d85/f4f 847253 0 2024-03-04T13:42:51.279 INFO:tasks.workunit.client.1.smithi169.stdout:7/481: mknod d2/de/d3e/d41/d69/d78/caa 0 2024-03-04T13:42:51.283 INFO:tasks.workunit.client.1.smithi169.stdout:0/371: symlink d5/d4c/d65/l86 0 2024-03-04T13:42:51.283 INFO:tasks.workunit.client.1.smithi169.stdout:8/479: read d7/d10/d24/d3e/f33 [992367,103116] 0 2024-03-04T13:42:51.284 INFO:tasks.workunit.client.1.smithi169.stdout:4/358: dwrite d4/d11/d2e/d21/f53 [0,4194304] 0 2024-03-04T13:42:51.288 INFO:tasks.workunit.client.1.smithi169.stdout:1/419: rename d4/d5/d36/d35/df/c45 to d4/d5/d36/d35/d70/d78/c89 0 2024-03-04T13:42:51.292 INFO:tasks.workunit.client.1.smithi169.stdout:8/480: stat d7/d10/f1e 0 2024-03-04T13:42:51.293 INFO:tasks.workunit.client.1.smithi169.stdout:0/372: symlink d5/d9/dc/d12/d15/l87 0 2024-03-04T13:42:51.294 INFO:tasks.workunit.client.1.smithi169.stdout:0/373: chown d5/d4c 502311 1 2024-03-04T13:42:51.296 INFO:tasks.workunit.client.1.smithi169.stdout:8/481: chown d7/d10/d17/d44/c77 0 1 2024-03-04T13:42:51.300 INFO:tasks.workunit.client.1.smithi169.stdout:4/359: getdents d4/d11/d2e/d39/d61/d64 0 2024-03-04T13:42:51.305 INFO:tasks.workunit.client.1.smithi169.stdout:3/411: dread d7/de/d2b/d7d/d85/f20 [0,4194304] 0 2024-03-04T13:42:51.312 INFO:tasks.workunit.client.1.smithi169.stdout:0/374: link d5/d9/dc/d12/d15/d19/d61/f2c d5/d9/dc/d12/d15/d19/d77/f88 0 2024-03-04T13:42:51.315 INFO:tasks.workunit.client.1.smithi169.stdout:4/360: dwrite d4/f7 [0,4194304] 0 2024-03-04T13:42:51.320 INFO:tasks.workunit.client.1.smithi169.stdout:4/361: rename d4/d11/d2e/d21/d5e/d72 to d4/d1c/d75 0 2024-03-04T13:42:51.321 INFO:tasks.workunit.client.1.smithi169.stdout:3/412: dread d7/de/d2b/d7d/d85/d13/d1a/d2d/d56/d5e/d67/f3a [0,4194304] 0 2024-03-04T13:42:51.321 INFO:tasks.workunit.client.1.smithi169.stdout:3/413: chown d7 1224456714 1 2024-03-04T13:42:51.322 INFO:tasks.workunit.client.1.smithi169.stdout:4/362: fdatasync d4/d11/d2e/d21/f29 0 2024-03-04T13:42:51.322 INFO:tasks.workunit.client.1.smithi169.stdout:7/482: truncate d2/d15/d17/f6e 547181 0 2024-03-04T13:42:51.324 INFO:tasks.workunit.client.1.smithi169.stdout:1/420: truncate d4/d1d/f40 12364 0 2024-03-04T13:42:51.325 INFO:tasks.workunit.client.1.smithi169.stdout:8/482: write d7/d10/d24/f7a [2687328,54898] 0 2024-03-04T13:42:51.327 INFO:tasks.workunit.client.1.smithi169.stdout:3/414: write d7/de/d2b/d7d/d85/d13/d1a/d2d/d56/f7c [245356,14580] 0 2024-03-04T13:42:51.332 INFO:tasks.workunit.client.1.smithi169.stdout:0/375: write d5/d9/dc/d12/d15/d19/d61/f2c [399527,20891] 0 2024-03-04T13:42:51.335 INFO:tasks.workunit.client.1.smithi169.stdout:3/415: creat d7/de/d2b/d7d/d85/d13/d5a/d71/f8e x:0 0 0 2024-03-04T13:42:51.336 INFO:tasks.workunit.client.1.smithi169.stdout:3/416: readlink d7/de/l57 0 2024-03-04T13:42:51.338 INFO:tasks.workunit.client.1.smithi169.stdout:3/417: fdatasync d7/de/d2b/d7d/d85/d47/f59 0 2024-03-04T13:42:51.339 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:42:51 smithi107 ceph-mon[28739]: pgmap v47: 65 pgs: 65 active+clean; 877 MiB data, 4.2 GiB used, 532 GiB / 536 GiB avail; 19 MiB/s rd, 118 MiB/s wr, 213 op/s 2024-03-04T13:42:51.341 INFO:tasks.workunit.client.1.smithi169.stdout:3/418: creat d7/de/d24/f8f x:0 0 0 2024-03-04T13:42:51.346 INFO:tasks.workunit.client.1.smithi169.stdout:8/483: dwrite d7/f6a [0,4194304] 0 2024-03-04T13:42:51.351 INFO:tasks.workunit.client.1.smithi169.stdout:3/419: rename d7/de/d2b/d7d/d85/f20 to d7/de/d2b/d7d/d85/d13/d1a/d2d/d56/f90 0 2024-03-04T13:42:51.355 INFO:tasks.workunit.client.1.smithi169.stdout:8/484: fsync d7/d10/d24/f3c 0 2024-03-04T13:42:51.356 INFO:tasks.workunit.client.1.smithi169.stdout:7/483: truncate d2/fa 1259918 0 2024-03-04T13:42:51.357 INFO:tasks.workunit.client.1.smithi169.stdout:8/485: chown d7/d10/d22/d75/d81 63 1 2024-03-04T13:42:51.364 INFO:tasks.workunit.client.1.smithi169.stdout:8/486: write d7/d10/d17/d44/f4f [1594419,77470] 0 2024-03-04T13:42:51.365 INFO:tasks.workunit.client.1.smithi169.stdout:7/484: unlink d2/de/d3e/c91 0 2024-03-04T13:42:51.370 INFO:tasks.workunit.client.1.smithi169.stdout:7/485: creat d2/de/d3e/d41/d69/fab x:0 0 0 2024-03-04T13:42:51.374 INFO:tasks.workunit.client.1.smithi169.stdout:7/486: chown d2/c31 29 1 2024-03-04T13:42:51.376 INFO:tasks.workunit.client.1.smithi169.stdout:7/487: creat d2/d15/d54/fac x:0 0 0 2024-03-04T13:42:51.387 INFO:tasks.workunit.client.1.smithi169.stdout:3/420: dread f6 [0,4194304] 0 2024-03-04T13:42:51.391 INFO:tasks.workunit.client.1.smithi169.stdout:7/488: dwrite d2/de/d3e/d41/d5b/d7a/f94 [0,4194304] 0 2024-03-04T13:42:51.392 INFO:tasks.workunit.client.1.smithi169.stdout:3/421: write d7/de/d2b/d7d/d85/d47/d63/d6c/f88 [372484,19467] 0 2024-03-04T13:42:51.394 INFO:tasks.workunit.client.1.smithi169.stdout:3/422: readlink d7/de/d2b/d7d/d85/d47/l54 0 2024-03-04T13:42:51.395 INFO:tasks.workunit.client.1.smithi169.stdout:3/423: fsync d7/f36 0 2024-03-04T13:42:51.402 INFO:tasks.workunit.client.1.smithi169.stdout:7/489: mkdir d2/d15/dad 0 2024-03-04T13:42:51.402 INFO:tasks.workunit.client.1.smithi169.stdout:3/424: mknod d7/de/d2b/d7d/d85/d13/d1a/d78/c91 0 2024-03-04T13:42:51.403 INFO:tasks.workunit.client.1.smithi169.stdout:7/490: fdatasync d2/d28/f4b 0 2024-03-04T13:42:51.404 INFO:tasks.workunit.client.1.smithi169.stdout:3/425: write d7/f70 [1060693,70988] 0 2024-03-04T13:42:51.405 INFO:tasks.workunit.client.1.smithi169.stdout:7/491: dread - d2/de/d3e/d41/fa9 zero size 2024-03-04T13:42:51.411 INFO:tasks.workunit.client.1.smithi169.stdout:7/492: creat d2/d15/d17/d6f/fae x:0 0 0 2024-03-04T13:42:51.415 INFO:tasks.workunit.client.1.smithi169.stdout:7/493: rename d2/c7d to d2/d15/d54/caf 0 2024-03-04T13:42:51.418 INFO:tasks.workunit.client.1.smithi169.stdout:7/494: readlink d2/de/d37/l5f 0 2024-03-04T13:42:51.420 INFO:tasks.workunit.client.1.smithi169.stdout:7/495: chown d2/d15/d17/d6f/f72 112 1 2024-03-04T13:42:51.421 INFO:tasks.workunit.client.1.smithi169.stdout:7/496: stat d2/d28/c66 0 2024-03-04T13:42:51.422 INFO:tasks.workunit.client.1.smithi169.stdout:3/426: dwrite d7/de/d2b/d7d/d85/d18/d21/d3f/d68/f4b [0,4194304] 0 2024-03-04T13:42:51.423 INFO:tasks.workunit.client.1.smithi169.stdout:7/497: read - d2/d15/d17/d6f/fae zero size 2024-03-04T13:42:51.424 INFO:tasks.workunit.client.1.smithi169.stdout:3/427: fdatasync d7/de/d2b/f52 0 2024-03-04T13:42:51.426 INFO:tasks.workunit.client.1.smithi169.stdout:7/498: chown d2/de/d3e/d41/d69/fab 102413 1 2024-03-04T13:42:51.430 INFO:tasks.workunit.client.1.smithi169.stdout:3/428: mknod d7/de/d2b/d7d/d85/d47/c92 0 2024-03-04T13:42:51.432 INFO:tasks.workunit.client.1.smithi169.stdout:3/429: mknod d7/d53/c93 0 2024-03-04T13:42:51.433 INFO:tasks.workunit.client.1.smithi169.stdout:3/430: chown d7/de/d2b/d7d 6 1 2024-03-04T13:42:51.435 INFO:tasks.workunit.client.1.smithi169.stdout:3/431: chown d7/de/d2b/d7d/d85/d13/d1a/d2d/d56/d5e 117 1 2024-03-04T13:42:51.436 INFO:tasks.workunit.client.1.smithi169.stdout:3/432: readlink d7/d53/l76 0 2024-03-04T13:42:51.439 INFO:tasks.workunit.client.1.smithi169.stdout:3/433: mknod d7/de/d2b/d7d/d85/d18/d21/d3f/d68/c94 0 2024-03-04T13:42:51.440 INFO:tasks.workunit.client.1.smithi169.stdout:7/499: dwrite d2/d15/d17/f1d [4194304,4194304] 0 2024-03-04T13:42:51.442 INFO:tasks.workunit.client.1.smithi169.stdout:7/500: write d2/d28/fa1 [498283,75819] 0 2024-03-04T13:42:51.446 INFO:tasks.workunit.client.1.smithi169.stdout:3/434: mkdir d7/de/d2b/d7d/d85/d13/d1a/d2d/d56/d5e/d67/d5c/d84/d95 0 2024-03-04T13:42:51.448 INFO:tasks.workunit.client.1.smithi169.stdout:7/501: dread d2/d28/f93 [0,4194304] 0 2024-03-04T13:42:51.450 INFO:tasks.workunit.client.1.smithi169.stdout:7/502: readlink d2/d28/l55 0 2024-03-04T13:42:51.452 INFO:tasks.workunit.client.1.smithi169.stdout:3/435: creat d7/de/d2b/d7d/d85/d13/d1a/d2d/d56/d5e/d67/d5c/d84/f96 x:0 0 0 2024-03-04T13:42:51.454 INFO:tasks.workunit.client.1.smithi169.stdout:7/503: creat d2/de/d3e/d41/d69/fb0 x:0 0 0 2024-03-04T13:42:51.458 INFO:tasks.workunit.client.1.smithi169.stdout:3/436: getdents d7/de 0 2024-03-04T13:42:51.462 INFO:tasks.workunit.client.1.smithi169.stdout:3/437: read d7/de/d2b/d7d/d85/d18/d21/f31 [2618027,56211] 0 2024-03-04T13:42:51.470 INFO:tasks.workunit.client.1.smithi169.stdout:3/438: creat d7/de/d2b/d7d/d85/d13/d1a/d2d/d56/d5e/d67/d5c/d84/d95/f97 x:0 0 0 2024-03-04T13:42:51.478 INFO:tasks.workunit.client.1.smithi169.stdout:5/465: sync 2024-03-04T13:42:51.484 INFO:tasks.workunit.client.1.smithi169.stdout:7/504: dwrite d2/de/f1e [0,4194304] 0 2024-03-04T13:42:51.488 INFO:tasks.workunit.client.1.smithi169.stdout:3/439: dwrite d7/de/d2b/d7d/d85/d13/d1a/f28 [0,4194304] 0 2024-03-04T13:42:51.490 INFO:tasks.workunit.client.1.smithi169.stdout:3/440: write f4 [4816278,160] 0 2024-03-04T13:42:51.496 INFO:tasks.workunit.client.1.smithi169.stdout:3/441: mkdir d7/de/d2b/d7d/d85/d13/d1a/d98 0 2024-03-04T13:42:51.499 INFO:tasks.workunit.client.1.smithi169.stdout:3/442: rmdir d7/de/d2b/d7d/d85 39 2024-03-04T13:42:51.503 INFO:tasks.workunit.client.1.smithi169.stdout:3/443: write d7/de/d2b/d7d/d85/d13/d1a/d2d/d56/d5e/d67/f51 [799304,19966] 0 2024-03-04T13:42:51.505 INFO:tasks.workunit.client.1.smithi169.stdout:3/444: chown d7/f49 15147006 1 2024-03-04T13:42:51.523 INFO:tasks.workunit.client.1.smithi169.stdout:3/445: write d7/de/d2b/d7d/d85/d13/f4a [137490,125063] 0 2024-03-04T13:42:51.526 INFO:tasks.workunit.client.1.smithi169.stdout:3/446: chown d7/de/d2b/d7d/d85/d18/d21/d66/f75 291046 1 2024-03-04T13:42:51.526 INFO:tasks.workunit.client.1.smithi169.stdout:5/466: dread d1/d16/d2a/f70 [0,4194304] 0 2024-03-04T13:42:51.526 INFO:tasks.workunit.client.1.smithi169.stdout:5/467: chown d1/d23 17817 1 2024-03-04T13:42:51.529 INFO:tasks.workunit.client.1.smithi169.stdout:5/468: creat d1/d81/d91/f92 x:0 0 0 2024-03-04T13:42:51.530 INFO:tasks.workunit.client.1.smithi169.stdout:3/447: creat d7/de/d2b/d7d/d85/d18/d21/d3f/d86/f99 x:0 0 0 2024-03-04T13:42:51.532 INFO:tasks.workunit.client.1.smithi169.stdout:5/469: write d1/d16/d17/d2d/d51/f6d [1696603,28065] 0 2024-03-04T13:42:51.535 INFO:tasks.workunit.client.1.smithi169.stdout:5/470: write d1/d16/d17/d2d/f54 [2554175,97098] 0 2024-03-04T13:42:51.538 INFO:tasks.workunit.client.1.smithi169.stdout:3/448: rename d7/de/d2b/d7d/d85/d13/d5a/d71 to d7/de/d2b/d7d/d85/d13/d1a/d98/d9a 0 2024-03-04T13:42:51.540 INFO:tasks.workunit.client.1.smithi169.stdout:5/471: rename d1/d16/d17/d2d/d5a/c63 to d1/d16/d2f/d39/c93 0 2024-03-04T13:42:51.542 INFO:tasks.workunit.client.1.smithi169.stdout:3/449: rmdir d7/de/d2b/d7d/d85/d47 39 2024-03-04T13:42:51.542 INFO:tasks.workunit.client.1.smithi169.stdout:5/472: write d1/f14 [4530752,56885] 0 2024-03-04T13:42:51.545 INFO:tasks.workunit.client.1.smithi169.stdout:3/450: readlink d7/de/d2b/d7d/d85/d18/d21/d3f/d86/l8d 0 2024-03-04T13:42:51.546 INFO:tasks.workunit.client.1.smithi169.stdout:5/473: creat d1/d81/d91/f94 x:0 0 0 2024-03-04T13:42:51.548 INFO:tasks.workunit.client.1.smithi169.stdout:5/474: mknod d1/d16/d17/d2d/d51/c95 0 2024-03-04T13:42:51.550 INFO:tasks.workunit.client.1.smithi169.stdout:5/475: dread - d1/d16/d17/f6f zero size 2024-03-04T13:42:51.551 INFO:tasks.workunit.client.1.smithi169.stdout:3/451: creat d7/de/d2b/d7d/d85/d13/d5a/d65/f9b x:0 0 0 2024-03-04T13:42:51.552 INFO:tasks.workunit.client.1.smithi169.stdout:5/476: mknod d1/d81/d91/c96 0 2024-03-04T13:42:51.552 INFO:tasks.workunit.client.1.smithi169.stdout:3/452: chown d7/de/d2b/d7d/d85/f6b 219238563 1 2024-03-04T13:42:51.554 INFO:tasks.workunit.client.1.smithi169.stdout:5/477: write d1/d16/d2f/f33 [1747092,65290] 0 2024-03-04T13:42:51.555 INFO:tasks.workunit.client.1.smithi169.stdout:3/453: symlink d7/de/d24/l9c 0 2024-03-04T13:42:51.558 INFO:tasks.workunit.client.1.smithi169.stdout:5/478: creat d1/d16/d17/d2d/d5a/d67/f97 x:0 0 0 2024-03-04T13:42:51.558 INFO:tasks.workunit.client.1.smithi169.stdout:3/454: truncate d7/de/d2b/f52 870506 0 2024-03-04T13:42:51.559 INFO:tasks.workunit.client.1.smithi169.stdout:5/479: chown d1/ld 12376 1 2024-03-04T13:42:51.561 INFO:tasks.workunit.client.1.smithi169.stdout:5/480: write d1/d16/d17/f6f [278857,88597] 0 2024-03-04T13:42:51.562 INFO:tasks.workunit.client.1.smithi169.stdout:3/455: creat d7/de/d2b/d7d/d85/d47/f9d x:0 0 0 2024-03-04T13:42:51.565 INFO:tasks.workunit.client.1.smithi169.stdout:3/456: write d7/de/d2b/d7d/d85/d18/d21/d66/f75 [1010819,23598] 0 2024-03-04T13:42:51.568 INFO:tasks.workunit.client.1.smithi169.stdout:3/457: write d7/de/d2b/d7d/d85/d13/d1a/d98/d9a/f8e [37733,27178] 0 2024-03-04T13:42:51.572 INFO:tasks.workunit.client.1.smithi169.stdout:3/458: symlink d7/de/d2b/d7d/l9e 0 2024-03-04T13:42:51.576 INFO:tasks.workunit.client.1.smithi169.stdout:5/481: dwrite d1/d81/d91/f94 [0,4194304] 0 2024-03-04T13:42:51.580 INFO:tasks.workunit.client.1.smithi169.stdout:5/482: creat d1/d81/f98 x:0 0 0 2024-03-04T13:42:51.583 INFO:tasks.workunit.client.1.smithi169.stdout:5/483: write d1/d16/f31 [4534339,122965] 0 2024-03-04T13:42:51.584 INFO:tasks.workunit.client.1.smithi169.stdout:5/484: write d1/f7d [594202,1788] 0 2024-03-04T13:42:51.589 INFO:tasks.workunit.client.1.smithi169.stdout:5/485: mkdir d1/d16/d17/d2d/d5a/d8a/d99 0 2024-03-04T13:42:51.591 INFO:tasks.workunit.client.1.smithi169.stdout:5/486: chown d1/d16/d17/d2d/d5a/d67 17153118 1 2024-03-04T13:42:51.593 INFO:tasks.workunit.client.1.smithi169.stdout:3/459: fsync d7/de/d2b/d7d/d85/d47/f9d 0 2024-03-04T13:42:51.595 INFO:tasks.workunit.client.1.smithi169.stdout:5/487: creat d1/d16/d17/d2d/f9a x:0 0 0 2024-03-04T13:42:51.595 INFO:tasks.workunit.client.1.smithi169.stdout:3/460: stat d7/de/d2b/d7d/d85/d18/d21/f8a 0 2024-03-04T13:42:51.597 INFO:tasks.workunit.client.1.smithi169.stdout:3/461: chown d7/de/d24/l34 13737572 1 2024-03-04T13:42:51.598 INFO:tasks.workunit.client.1.smithi169.stdout:5/488: mknod d1/d16/d17/d1b/d74/c9b 0 2024-03-04T13:42:51.599 INFO:tasks.workunit.client.1.smithi169.stdout:3/462: chown d7/de/d2b/d7d/d85/d13/d1a/d2d/c3e 120910883 1 2024-03-04T13:42:51.600 INFO:tasks.workunit.client.1.smithi169.stdout:5/489: chown d1/d16/d17/d2d/d51/c95 9173052 1 2024-03-04T13:42:51.601 INFO:tasks.workunit.client.1.smithi169.stdout:3/463: fsync d7/de/d2b/f6e 0 2024-03-04T13:42:51.603 INFO:tasks.workunit.client.1.smithi169.stdout:3/464: fsync d7/de/d2b/d7d/d85/d18/d21/f31 0 2024-03-04T13:42:51.614 INFO:tasks.workunit.client.1.smithi169.stdout:5/490: dwrite d1/f7d [0,4194304] 0 2024-03-04T13:42:51.616 INFO:tasks.workunit.client.1.smithi169.stdout:5/491: chown d1/d16/d2f/d38 454704188 1 2024-03-04T13:42:51.618 INFO:tasks.workunit.client.1.smithi169.stdout:5/492: stat d1/d16/l1d 0 2024-03-04T13:42:51.621 INFO:tasks.workunit.client.1.smithi169.stdout:3/465: dwrite d7/de/d2b/f6e [0,4194304] 0 2024-03-04T13:42:51.625 INFO:tasks.workunit.client.1.smithi169.stdout:5/493: symlink d1/d81/l9c 0 2024-03-04T13:42:51.631 INFO:tasks.workunit.client.1.smithi169.stdout:3/466: truncate d7/de/d2b/d7d/d85/d13/d1a/d2d/d56/f7c 798136 0 2024-03-04T13:42:51.631 INFO:tasks.workunit.client.1.smithi169.stdout:5/494: symlink d1/d81/d91/l9d 0 2024-03-04T13:42:51.636 INFO:tasks.workunit.client.1.smithi169.stdout:3/467: creat d7/de/d2b/d7d/d85/d18/d21/f9f x:0 0 0 2024-03-04T13:42:51.643 INFO:tasks.workunit.client.1.smithi169.stdout:3/468: rename d7/de/d2b/d7d/d85/d18/d21/l35 to d7/de/d2b/d7d/d85/d13/d1a/d98/d9a/la0 0 2024-03-04T13:42:51.647 INFO:tasks.workunit.client.1.smithi169.stdout:3/469: mkdir d7/de/d2b/d7d/d85/d13/d5a/da1 0 2024-03-04T13:42:51.648 INFO:tasks.workunit.client.1.smithi169.stdout:5/495: dwrite d1/d16/d17/d2d/d5a/d67/f97 [0,4194304] 0 2024-03-04T13:42:51.650 INFO:tasks.workunit.client.1.smithi169.stdout:2/411: sync 2024-03-04T13:42:51.652 INFO:tasks.workunit.client.1.smithi169.stdout:5/496: mkdir d1/d16/d2a/d46/d9e 0 2024-03-04T13:42:51.656 INFO:tasks.workunit.client.1.smithi169.stdout:3/470: write d7/de/d2b/d7d/d85/d13/d1a/d2d/d56/d5e/d67/f3a [3431020,111269] 0 2024-03-04T13:42:51.661 INFO:tasks.workunit.client.1.smithi169.stdout:3/471: creat d7/de/d2b/d7d/d85/d18/fa2 x:0 0 0 2024-03-04T13:42:51.663 INFO:tasks.workunit.client.1.smithi169.stdout:5/497: rmdir d1/d16/d17/d2d/d5a/d8a/d99 0 2024-03-04T13:42:51.666 INFO:tasks.workunit.client.1.smithi169.stdout:3/472: creat d7/de/d2b/d7d/d85/d13/d1a/d2d/d56/fa3 x:0 0 0 2024-03-04T13:42:51.668 INFO:tasks.workunit.client.1.smithi169.stdout:3/473: readlink d7/de/d2b/d7d/d85/l23 0 2024-03-04T13:42:51.677 INFO:tasks.workunit.client.1.smithi169.stdout:5/498: dwrite d1/f80 [0,4194304] 0 2024-03-04T13:42:51.689 INFO:tasks.workunit.client.1.smithi169.stdout:2/412: dread d1/d5/d19/d1e/f74 [0,4194304] 0 2024-03-04T13:42:51.692 INFO:tasks.workunit.client.1.smithi169.stdout:5/499: dwrite d1/d16/d2f/d38/f4f [0,4194304] 0 2024-03-04T13:42:51.692 INFO:tasks.workunit.client.1.smithi169.stdout:2/413: write d1/f78 [890000,6759] 0 2024-03-04T13:42:51.694 INFO:tasks.workunit.client.1.smithi169.stdout:2/414: stat d1/d5/d19/d36/f60 0 2024-03-04T13:42:51.695 INFO:tasks.workunit.client.1.smithi169.stdout:5/500: write d1/f65 [1172478,110349] 0 2024-03-04T13:42:51.700 INFO:tasks.workunit.client.1.smithi169.stdout:5/501: chown d1/d16/d17/d2d/d51/f6d 24 1 2024-03-04T13:42:51.787 INFO:tasks.workunit.client.1.smithi169.stdout:5/502: rename d1/d16/d17/f6f to d1/d76/f9f 0 2024-03-04T13:42:51.791 INFO:tasks.workunit.client.1.smithi169.stdout:5/503: rmdir d1/d76 39 2024-03-04T13:42:51.793 INFO:tasks.workunit.client.1.smithi169.stdout:2/415: dwrite d1/d5/d19/d3c/f6c [0,4194304] 0 2024-03-04T13:42:51.828 INFO:tasks.workunit.client.1.smithi169.stdout:9/494: sync 2024-03-04T13:42:51.833 INFO:tasks.workunit.client.1.smithi169.stdout:5/504: truncate d1/d16/d2f/f33 2479963 0 2024-03-04T13:42:51.841 INFO:tasks.workunit.client.1.smithi169.stdout:5/505: rename d1/d16/d17/d2d/f54 to d1/d60/fa0 0 2024-03-04T13:42:51.844 INFO:tasks.workunit.client.1.smithi169.stdout:5/506: write d1/d16/d17/d28/f41 [949208,119650] 0 2024-03-04T13:42:51.851 INFO:tasks.workunit.client.1.smithi169.stdout:2/416: dwrite d1/d5/d19/d69/f7e [0,4194304] 0 2024-03-04T13:42:51.852 INFO:tasks.workunit.client.1.smithi169.stdout:2/417: readlink d1/d5/l2b 0 2024-03-04T13:42:51.853 INFO:tasks.workunit.client.1.smithi169.stdout:9/495: getdents d7/d2d/d48/d66 0 2024-03-04T13:42:51.855 INFO:tasks.workunit.client.1.smithi169.stdout:9/496: readlink d7/dc/d12/d32/d42/l4d 0 2024-03-04T13:42:51.857 INFO:tasks.workunit.client.1.smithi169.stdout:5/507: dread d1/d16/d17/f79 [0,4194304] 0 2024-03-04T13:42:51.859 INFO:tasks.workunit.client.1.smithi169.stdout:2/418: creat d1/d5/d22/d45/f90 x:0 0 0 2024-03-04T13:42:51.860 INFO:tasks.workunit.client.1.smithi169.stdout:2/419: chown d1/d5/d22/c57 8137248 1 2024-03-04T13:42:51.862 INFO:tasks.workunit.client.1.smithi169.stdout:9/497: unlink d7/d25/c63 0 2024-03-04T13:42:51.866 INFO:tasks.workunit.client.1.smithi169.stdout:5/508: mknod d1/d81/ca1 0 2024-03-04T13:42:51.868 INFO:tasks.workunit.client.1.smithi169.stdout:5/509: fdatasync d1/d16/d17/d2d/f9a 0 2024-03-04T13:42:51.871 INFO:tasks.workunit.client.1.smithi169.stdout:9/498: symlink d7/dc/d12/d8a/la4 0 2024-03-04T13:42:51.873 INFO:tasks.workunit.client.1.smithi169.stdout:9/499: chown d7/d25/d47/c61 34146318 1 2024-03-04T13:42:51.882 INFO:tasks.workunit.client.1.smithi169.stdout:2/420: getdents d1/d5/d22 0 2024-03-04T13:42:51.884 INFO:tasks.workunit.client.1.smithi169.stdout:5/510: link d1/ld d1/d16/d17/d2d/d51/la2 0 2024-03-04T13:42:51.884 INFO:tasks.workunit.client.1.smithi169.stdout:9/500: dread d7/dc/f1d [0,4194304] 0 2024-03-04T13:42:51.886 INFO:tasks.workunit.client.1.smithi169.stdout:2/421: chown d1/c3a 50834754 1 2024-03-04T13:42:51.888 INFO:tasks.workunit.client.1.smithi169.stdout:2/422: fdatasync d1/d5/d22/d45/d4a/f7d 0 2024-03-04T13:42:51.892 INFO:tasks.workunit.client.1.smithi169.stdout:2/423: creat d1/d5/d22/d45/f91 x:0 0 0 2024-03-04T13:42:51.894 INFO:tasks.workunit.client.1.smithi169.stdout:2/424: write d1/d5/d19/d68/f88 [728877,123270] 0 2024-03-04T13:42:51.896 INFO:tasks.workunit.client.1.smithi169.stdout:5/511: link d1/d16/d17/d2d/d51/c52 d1/d16/d17/ca3 0 2024-03-04T13:42:51.898 INFO:tasks.workunit.client.1.smithi169.stdout:2/425: unlink d1/d5/d19/d36/c37 0 2024-03-04T13:42:51.902 INFO:tasks.workunit.client.1.smithi169.stdout:5/512: creat d1/d16/d17/fa4 x:0 0 0 2024-03-04T13:42:51.904 INFO:tasks.workunit.client.1.smithi169.stdout:5/513: stat d1/d16/d2f/d72/f77 0 2024-03-04T13:42:51.905 INFO:tasks.workunit.client.1.smithi169.stdout:5/514: dread - d1/d16/d17/fa4 zero size 2024-03-04T13:42:51.908 INFO:tasks.workunit.client.1.smithi169.stdout:5/515: chown d1/d16/d17/d1b/d74/c9b 32370 1 2024-03-04T13:42:51.910 INFO:tasks.workunit.client.1.smithi169.stdout:4/363: sync 2024-03-04T13:42:51.910 INFO:tasks.workunit.client.1.smithi169.stdout:6/454: sync 2024-03-04T13:42:51.923 INFO:tasks.workunit.client.1.smithi169.stdout:4/364: write d4/d11/d23/f35 [2553269,82731] 0 2024-03-04T13:42:51.928 INFO:tasks.workunit.client.1.smithi169.stdout:4/365: read d4/f15 [124822,60995] 0 2024-03-04T13:42:51.931 INFO:tasks.workunit.client.1.smithi169.stdout:2/426: dread d1/d5/d22/d3b/f52 [0,4194304] 0 2024-03-04T13:42:51.935 INFO:tasks.workunit.client.1.smithi169.stdout:5/516: dwrite d1/d16/d17/d28/f4d [0,4194304] 0 2024-03-04T13:42:51.940 INFO:tasks.workunit.client.1.smithi169.stdout:2/427: creat d1/d5/d22/d45/d4a/f92 x:0 0 0 2024-03-04T13:42:51.943 INFO:tasks.workunit.client.1.smithi169.stdout:9/501: dwrite d7/dc/f23 [0,4194304] 0 2024-03-04T13:42:51.946 INFO:tasks.workunit.client.1.smithi169.stdout:4/366: unlink d4/l22 0 2024-03-04T13:42:51.948 INFO:tasks.workunit.client.1.smithi169.stdout:4/367: write d4/f9 [4002296,20858] 0 2024-03-04T13:42:51.951 INFO:tasks.workunit.client.1.smithi169.stdout:2/428: unlink d1/f78 0 2024-03-04T13:42:51.955 INFO:tasks.workunit.client.1.smithi169.stdout:9/502: write d7/d25/d47/f87 [667198,14021] 0 2024-03-04T13:42:51.961 INFO:tasks.workunit.client.1.smithi169.stdout:6/455: dwrite d0/d3c/d40/d52/f91 [0,4194304] 0 2024-03-04T13:42:51.967 INFO:tasks.workunit.client.1.smithi169.stdout:5/517: dwrite d1/d16/d17/d2d/d51/f6d [0,4194304] 0 2024-03-04T13:42:51.974 INFO:tasks.workunit.client.1.smithi169.stdout:4/368: link d4/d11/d2e/d21/f53 d4/d11/d2e/d21/d5e/f76 0 2024-03-04T13:42:51.978 INFO:tasks.workunit.client.1.smithi169.stdout:9/503: mknod d7/dc/d12/d32/ca5 0 2024-03-04T13:42:51.986 INFO:tasks.workunit.client.1.smithi169.stdout:2/429: link d1/f40 d1/d5/d19/d1e/d29/d8a/f93 0 2024-03-04T13:42:51.988 INFO:tasks.workunit.client.1.smithi169.stdout:5/518: creat d1/d16/d17/d2d/d5a/d8a/fa5 x:0 0 0 2024-03-04T13:42:51.990 INFO:tasks.workunit.client.1.smithi169.stdout:5/519: chown d1/d16/d17/f71 144594719 1 2024-03-04T13:42:51.991 INFO:tasks.workunit.client.1.smithi169.stdout:4/369: mkdir d4/d11/d23/d77 0 2024-03-04T13:42:51.994 INFO:tasks.workunit.client.1.smithi169.stdout:4/370: chown d4/l16 193301815 1 2024-03-04T13:42:51.995 INFO:tasks.workunit.client.1.smithi169.stdout:5/520: read d1/d16/f31 [2241008,31647] 0 2024-03-04T13:42:52.001 INFO:tasks.workunit.client.1.smithi169.stdout:5/521: truncate d1/d16/d2a/f3e 1221349 0 2024-03-04T13:42:52.006 INFO:tasks.workunit.client.1.smithi169.stdout:5/522: symlink d1/d16/d2a/la6 0 2024-03-04T13:42:52.009 INFO:tasks.workunit.client.1.smithi169.stdout:5/523: symlink d1/d16/la7 0 2024-03-04T13:42:52.012 INFO:tasks.workunit.client.1.smithi169.stdout:6/456: dread d0/f4e [0,4194304] 0 2024-03-04T13:42:52.014 INFO:tasks.workunit.client.1.smithi169.stdout:6/457: chown d0/d32/d3a/d5e/l61 10 1 2024-03-04T13:42:52.014 INFO:tasks.workunit.client.1.smithi169.stdout:4/371: dread d4/d11/d2e/d39/d48/f3c [0,4194304] 0 2024-03-04T13:42:52.015 INFO:tasks.workunit.client.1.smithi169.stdout:2/430: truncate d1/d5/d19/d1e/f43 533225 0 2024-03-04T13:42:52.018 INFO:tasks.workunit.client.1.smithi169.stdout:2/431: dread - d1/d5/d22/d45/d66/f73 zero size 2024-03-04T13:42:52.018 INFO:tasks.workunit.client.1.smithi169.stdout:4/372: mkdir d4/d11/d2e/d6c/d78 0 2024-03-04T13:42:52.022 INFO:tasks.workunit.client.1.smithi169.stdout:6/458: truncate d0/d32/d3a/d43/f45 798606 0 2024-03-04T13:42:52.027 INFO:tasks.workunit.client.1.smithi169.stdout:2/432: link d1/f40 d1/d5/d22/f94 0 2024-03-04T13:42:52.034 INFO:tasks.workunit.client.1.smithi169.stdout:2/433: getdents d1/d5/d19/d1e/d29 0 2024-03-04T13:42:52.038 INFO:tasks.workunit.client.1.smithi169.stdout:6/459: dwrite d0/d32/d3a/f27 [0,4194304] 0 2024-03-04T13:42:52.039 INFO:tasks.workunit.client.1.smithi169.stdout:4/373: truncate d4/f15 224948 0 2024-03-04T13:42:52.041 INFO:tasks.workunit.client.1.smithi169.stdout:2/434: creat d1/d5/d19/d36/d64/f95 x:0 0 0 2024-03-04T13:42:52.049 INFO:tasks.workunit.client.1.smithi169.stdout:2/435: rename d1/d5/c33 to d1/d5/d19/d36/d64/c96 0 2024-03-04T13:42:52.053 INFO:tasks.workunit.client.1.smithi169.stdout:4/374: creat d4/d11/f79 x:0 0 0 2024-03-04T13:42:52.059 INFO:tasks.workunit.client.1.smithi169.stdout:2/436: fdatasync d1/d5/d22/f32 0 2024-03-04T13:42:52.066 INFO:tasks.workunit.client.1.smithi169.stdout:2/437: creat d1/d5/d22/d45/d4a/f97 x:0 0 0 2024-03-04T13:42:52.072 INFO:tasks.workunit.client.1.smithi169.stdout:2/438: mknod d1/d5/d22/d45/d66/c98 0 2024-03-04T13:42:52.073 INFO:tasks.workunit.client.1.smithi169.stdout:2/439: fdatasync d1/d5/d22/d45/f91 0 2024-03-04T13:42:52.075 INFO:tasks.workunit.client.1.smithi169.stdout:6/460: dread d0/f29 [0,4194304] 0 2024-03-04T13:42:52.133 INFO:tasks.workunit.client.1.smithi169.stdout:1/421: sync 2024-03-04T13:42:52.133 INFO:tasks.workunit.client.1.smithi169.stdout:8/487: sync 2024-03-04T13:42:52.133 INFO:tasks.workunit.client.1.smithi169.stdout:3/474: sync 2024-03-04T13:42:52.134 INFO:tasks.workunit.client.1.smithi169.stdout:0/376: sync 2024-03-04T13:42:52.134 INFO:tasks.workunit.client.1.smithi169.stdout:7/505: sync 2024-03-04T13:42:52.141 INFO:tasks.workunit.client.1.smithi169.stdout:3/475: fdatasync d7/de/d2b/f52 0 2024-03-04T13:42:52.142 INFO:tasks.workunit.client.1.smithi169.stdout:8/488: stat d7/d10/d17/f98 0 2024-03-04T13:42:52.143 INFO:tasks.workunit.client.1.smithi169.stdout:7/506: unlink d2/de/f1e 0 2024-03-04T13:42:52.145 INFO:tasks.workunit.client.1.smithi169.stdout:8/489: chown d7/d10/c8d 408 1 2024-03-04T13:42:52.150 INFO:tasks.workunit.client.1.smithi169.stdout:0/377: dread d5/d9/dc/d12/d15/d19/d61/f33 [0,4194304] 0 2024-03-04T13:42:52.150 INFO:tasks.workunit.client.1.smithi169.stdout:3/476: mkdir d7/de/d2b/d7d/d85/d13/d5a/d65/da4 0 2024-03-04T13:42:52.153 INFO:tasks.workunit.client.1.smithi169.stdout:7/507: chown d2/d6b/c6d 272 1 2024-03-04T13:42:52.154 INFO:tasks.workunit.client.1.smithi169.stdout:0/378: chown d5/c25 1 1 2024-03-04T13:42:52.156 INFO:tasks.workunit.client.1.smithi169.stdout:1/422: rename d4/d5/d36/d35/d16/d3a/c66 to d4/d5/d2d/d4e/d71/c8a 0 2024-03-04T13:42:52.159 INFO:tasks.workunit.client.1.smithi169.stdout:0/379: readlink d5/d9/d3d/l80 0 2024-03-04T13:42:52.164 INFO:tasks.workunit.client.1.smithi169.stdout:7/508: write d2/de/d3e/d41/d69/fab [972752,2486] 0 2024-03-04T13:42:52.164 INFO:tasks.workunit.client.1.smithi169.stdout:6/461: dwrite d0/d32/d3a/d5e/f77 [0,4194304] 0 2024-03-04T13:42:52.166 INFO:tasks.workunit.client.1.smithi169.stdout:7/509: stat d2/de/d3e/d41/d5b/f9f 0 2024-03-04T13:42:52.166 INFO:tasks.workunit.client.1.smithi169.stdout:0/380: dread - d5/d9/dc/d12/d66/f78 zero size 2024-03-04T13:42:52.172 INFO:tasks.workunit.client.1.smithi169.stdout:6/462: mkdir d0/d3c/d95/d9e 0 2024-03-04T13:42:52.172 INFO:tasks.workunit.client.1.smithi169.stdout:1/423: chown d4/d5/d2d/d4e/d71/c8a 16512 1 2024-03-04T13:42:52.173 INFO:tasks.workunit.client.1.smithi169.stdout:2/440: dread d1/f51 [0,4194304] 0 2024-03-04T13:42:52.173 INFO:tasks.workunit.client.1.smithi169.stdout:1/424: readlink d4/d5/l14 0 2024-03-04T13:42:52.174 INFO:tasks.workunit.client.1.smithi169.stdout:8/490: link d7/d10/d17/f2a d7/d10/d22/d46/f9d 0 2024-03-04T13:42:52.177 INFO:tasks.workunit.client.1.smithi169.stdout:1/425: dread d4/d5/d36/d35/d5b/f87 [0,4194304] 0 2024-03-04T13:42:52.178 INFO:tasks.workunit.client.1.smithi169.stdout:0/381: truncate d5/d9/dc/d12/d15/f40 473451 0 2024-03-04T13:42:52.178 INFO:tasks.workunit.client.1.smithi169.stdout:1/426: chown d4/d5/d2d/d4e/d71/d26/d54 66 1 2024-03-04T13:42:52.179 INFO:tasks.workunit.client.1.smithi169.stdout:6/463: mknod d0/d3c/d40/c9f 0 2024-03-04T13:42:52.181 INFO:tasks.workunit.client.1.smithi169.stdout:1/427: chown d4/d5/d36/d35/d5b 110977338 1 2024-03-04T13:42:52.183 INFO:tasks.workunit.client.1.smithi169.stdout:8/491: mkdir d7/d10/d22/d75/d79/d9e 0 2024-03-04T13:42:52.184 INFO:tasks.workunit.client.1.smithi169.stdout:6/464: fdatasync d0/f29 0 2024-03-04T13:42:52.186 INFO:tasks.workunit.client.1.smithi169.stdout:8/492: readlink d7/d10/d22/d46/l8e 0 2024-03-04T13:42:52.186 INFO:tasks.workunit.client.1.smithi169.stdout:6/465: chown d0/l99 171385 1 2024-03-04T13:42:52.188 INFO:tasks.workunit.client.1.smithi169.stdout:3/477: truncate d7/de/d2b/d7d/d85/d47/f59 82276 0 2024-03-04T13:42:52.189 INFO:tasks.workunit.client.1.smithi169.stdout:0/382: symlink d5/d9/dc/d12/d15/d19/d77/l89 0 2024-03-04T13:42:52.196 INFO:tasks.workunit.client.1.smithi169.stdout:6/466: rename d0/d32/f8f to d0/d32/d3a/d43/d74/fa0 0 2024-03-04T13:42:52.198 INFO:tasks.workunit.client.1.smithi169.stdout:7/510: dwrite d2/de/d3e/f62 [0,4194304] 0 2024-03-04T13:42:52.203 INFO:tasks.workunit.client.1.smithi169.stdout:1/428: dwrite d4/d5/d2d/f74 [0,4194304] 0 2024-03-04T13:42:52.207 INFO:tasks.workunit.client.1.smithi169.stdout:2/441: dwrite d1/d5/d22/f32 [0,4194304] 0 2024-03-04T13:42:52.212 INFO:tasks.workunit.client.1.smithi169.stdout:3/478: mknod d7/de/d2b/d7d/d85/d13/d5a/da1/ca5 0 2024-03-04T13:42:52.214 INFO:tasks.workunit.client.1.smithi169.stdout:1/429: truncate d4/d5/d2d/d4e/f84 175799 0 2024-03-04T13:42:52.221 INFO:tasks.workunit.client.1.smithi169.stdout:6/467: getdents d0/d32/d70 0 2024-03-04T13:42:52.223 INFO:tasks.workunit.client.1.smithi169.stdout:8/493: dwrite d7/d10/d24/f3c [0,4194304] 0 2024-03-04T13:42:52.228 INFO:tasks.workunit.client.1.smithi169.stdout:2/442: rename d1/d5/d19/d1e/d29/d8a to d1/d5/d22/d3b/d99 0 2024-03-04T13:42:52.229 INFO:tasks.workunit.client.1.smithi169.stdout:6/468: mknod d0/d3c/d40/ca1 0 2024-03-04T13:42:52.230 INFO:tasks.workunit.client.1.smithi169.stdout:3/479: read - d7/de/d2b/d7d/d85/d13/d1a/d2d/d56/d5e/d67/d5c/d84/f96 zero size 2024-03-04T13:42:52.232 INFO:tasks.workunit.client.1.smithi169.stdout:1/430: getdents d4/d5/d2d/d4e/d71/d82 0 2024-03-04T13:42:52.236 INFO:tasks.workunit.client.1.smithi169.stdout:8/494: creat d7/f9f x:0 0 0 2024-03-04T13:42:52.238 INFO:tasks.workunit.client.1.smithi169.stdout:6/469: fsync d0/d3c/f86 0 2024-03-04T13:42:52.239 INFO:tasks.workunit.client.1.smithi169.stdout:0/383: dwrite d5/d9/dc/d12/d15/f40 [0,4194304] 0 2024-03-04T13:42:52.248 INFO:tasks.workunit.client.1.smithi169.stdout:3/480: creat d7/de/d24/fa6 x:0 0 0 2024-03-04T13:42:52.249 INFO:tasks.workunit.client.1.smithi169.stdout:6/470: unlink d0/d32/d70/l94 0 2024-03-04T13:42:52.249 INFO:tasks.workunit.client.1.smithi169.stdout:2/443: creat d1/d5/d22/d45/d4a/f9a x:0 0 0 2024-03-04T13:42:52.253 INFO:tasks.workunit.client.1.smithi169.stdout:8/495: creat d7/d10/d17/d44/d4c/fa0 x:0 0 0 2024-03-04T13:42:52.254 INFO:tasks.workunit.client.1.smithi169.stdout:7/511: dwrite d2/de/f2a [0,4194304] 0 2024-03-04T13:42:52.255 INFO:tasks.workunit.client.1.smithi169.stdout:7/512: chown d2/d15/d17 247 1 2024-03-04T13:42:52.258 INFO:tasks.workunit.client.1.smithi169.stdout:0/384: creat d5/d9/dc/d12/d15/d46/f8a x:0 0 0 2024-03-04T13:42:52.260 INFO:tasks.workunit.client.1.smithi169.stdout:3/481: write d7/de/d2b/d7d/d85/d18/d21/d3f/f69 [396953,37363] 0 2024-03-04T13:42:52.260 INFO:tasks.workunit.client.1.smithi169.stdout:8/496: fdatasync d7/d10/d17/d44/f4f 0 2024-03-04T13:42:52.261 INFO:tasks.workunit.client.1.smithi169.stdout:7/513: mknod d2/d28/d68/cb1 0 2024-03-04T13:42:52.262 INFO:tasks.workunit.client.1.smithi169.stdout:6/471: rename d0/lc to d0/d32/d3a/d43/la2 0 2024-03-04T13:42:52.265 INFO:tasks.workunit.client.1.smithi169.stdout:2/444: dread d1/d5/d19/f3f [0,4194304] 0 2024-03-04T13:42:52.267 INFO:tasks.workunit.client.1.smithi169.stdout:8/497: fdatasync d7/d10/d24/f7a 0 2024-03-04T13:42:52.267 INFO:tasks.workunit.client.1.smithi169.stdout:1/431: dwrite d4/d5/d2d/d4e/d71/d26/d4f/f51 [0,4194304] 0 2024-03-04T13:42:52.270 INFO:tasks.workunit.client.1.smithi169.stdout:6/472: write d0/d32/f58 [904594,19431] 0 2024-03-04T13:42:52.271 INFO:tasks.workunit.client.1.smithi169.stdout:3/482: creat d7/de/d2b/d7d/d85/d13/d5a/da1/fa7 x:0 0 0 2024-03-04T13:42:52.275 INFO:tasks.workunit.client.1.smithi169.stdout:7/514: rmdir d2/d28/d85 0 2024-03-04T13:42:52.278 INFO:tasks.workunit.client.1.smithi169.stdout:8/498: write d7/d10/d24/d3e/f95 [536089,116955] 0 2024-03-04T13:42:52.279 INFO:tasks.workunit.client.1.smithi169.stdout:1/432: write d4/d5/d36/d35/d16/d3a/f3f [3033896,81845] 0 2024-03-04T13:42:52.280 INFO:tasks.workunit.client.1.smithi169.stdout:6/473: symlink d0/d32/d70/la3 0 2024-03-04T13:42:52.284 INFO:tasks.workunit.client.1.smithi169.stdout:3/483: write d7/fb [3504024,91553] 0 2024-03-04T13:42:52.285 INFO:tasks.workunit.client.1.smithi169.stdout:2/445: mknod d1/d5/d19/d63/c9b 0 2024-03-04T13:42:52.290 INFO:tasks.workunit.client.1.smithi169.stdout:8/499: chown d7/d10/l82 77347 1 2024-03-04T13:42:52.294 INFO:tasks.workunit.client.1.smithi169.stdout:6/474: creat d0/d32/d3a/d43/d74/fa4 x:0 0 0 2024-03-04T13:42:52.301 INFO:tasks.workunit.client.1.smithi169.stdout:2/446: rename d1/d5/d22/d45/d66/f73 to d1/d5/d22/d3b/f9c 0 2024-03-04T13:42:52.304 INFO:tasks.workunit.client.1.smithi169.stdout:0/385: dread d5/d9/dc/d12/d15/d19/d77/f88 [0,4194304] 0 2024-03-04T13:42:52.306 INFO:tasks.workunit.client.1.smithi169.stdout:3/484: symlink d7/de/d2b/d7d/d85/d18/d21/la8 0 2024-03-04T13:42:52.309 INFO:tasks.workunit.client.1.smithi169.stdout:7/515: dwrite d2/de/ff [0,4194304] 0 2024-03-04T13:42:52.316 INFO:tasks.workunit.client.1.smithi169.stdout:8/500: dread d7/d10/d24/f3c [0,4194304] 0 2024-03-04T13:42:52.316 INFO:tasks.workunit.client.1.smithi169.stdout:2/447: symlink d1/d6d/l9d 0 2024-03-04T13:42:52.317 INFO:tasks.workunit.client.1.smithi169.stdout:7/516: write d2/de/d3e/d41/d69/fab [1429029,43550] 0 2024-03-04T13:42:52.320 INFO:tasks.workunit.client.1.smithi169.stdout:3/485: mknod d7/de/d2b/d7d/d85/d13/d1a/d2d/d56/d5e/d67/ca9 0 2024-03-04T13:42:52.321 INFO:tasks.workunit.client.1.smithi169.stdout:7/517: stat d2/de/d3e/d41/d5b/l9e 0 2024-03-04T13:42:52.323 INFO:tasks.workunit.client.1.smithi169.stdout:7/518: readlink d2/de/l67 0 2024-03-04T13:42:52.323 INFO:tasks.workunit.client.1.smithi169.stdout:6/475: link d0/d3c/d40/c44 d0/d3c/ca5 0 2024-03-04T13:42:52.326 INFO:tasks.workunit.client.1.smithi169.stdout:9/504: sync 2024-03-04T13:42:52.326 INFO:tasks.workunit.client.1.smithi169.stdout:5/524: sync 2024-03-04T13:42:52.328 INFO:tasks.workunit.client.1.smithi169.stdout:1/433: dread d4/f1b [0,4194304] 0 2024-03-04T13:42:52.332 INFO:tasks.workunit.client.1.smithi169.stdout:2/448: unlink d1/d5/d22/f55 0 2024-03-04T13:42:52.337 INFO:tasks.workunit.client.1.smithi169.stdout:6/476: mkdir d0/d32/d3a/d43/d74/da6 0 2024-03-04T13:42:52.338 INFO:tasks.workunit.client.1.smithi169.stdout:8/501: dwrite d7/d10/d24/d3e/f68 [0,4194304] 0 2024-03-04T13:42:52.338 INFO:tasks.workunit.client.1.smithi169.stdout:6/477: stat d0/f30 0 2024-03-04T13:42:52.342 INFO:tasks.workunit.client.1.smithi169.stdout:9/505: mkdir d7/dc/d12/d20/da6 0 2024-03-04T13:42:52.348 INFO:tasks.workunit.client.1.smithi169.stdout:2/449: write d1/f30 [4923651,28773] 0 2024-03-04T13:42:52.348 INFO:tasks.workunit.client.1.smithi169.stdout:3/486: creat d7/de/d2b/d7d/d85/d13/d5a/d65/da4/faa x:0 0 0 2024-03-04T13:42:52.348 INFO:tasks.workunit.client.1.smithi169.stdout:0/386: dread d5/d9/dc/d12/d15/d19/f2e [0,4194304] 0 2024-03-04T13:42:52.348 INFO:tasks.workunit.client.1.smithi169.stdout:8/502: unlink d7/d10/d24/d3e/f6d 0 2024-03-04T13:42:52.350 INFO:tasks.workunit.client.1.smithi169.stdout:9/506: creat d7/d25/d64/fa7 x:0 0 0 2024-03-04T13:42:52.354 INFO:tasks.workunit.client.1.smithi169.stdout:0/387: chown d5/d9/dc/f1d 236 1 2024-03-04T13:42:52.355 INFO:tasks.workunit.client.1.smithi169.stdout:1/434: symlink d4/d5/d2d/d4e/d71/d26/d7e/l8b 0 2024-03-04T13:42:52.356 INFO:tasks.workunit.client.1.smithi169.stdout:1/435: stat d4/d5/d36 0 2024-03-04T13:42:52.358 INFO:tasks.workunit.client.1.smithi169.stdout:7/519: rename d2/de/d37/c74 to d2/d15/cb2 0 2024-03-04T13:42:52.360 INFO:tasks.workunit.client.1.smithi169.stdout:8/503: creat d7/d10/d17/d44/d36/d62/fa1 x:0 0 0 2024-03-04T13:42:52.361 INFO:tasks.workunit.client.1.smithi169.stdout:2/450: mkdir d1/d5/d22/d9e 0 2024-03-04T13:42:52.362 INFO:tasks.workunit.client.1.smithi169.stdout:0/388: write d5/d9/dc/f1d [2160516,76103] 0 2024-03-04T13:42:52.365 INFO:tasks.workunit.client.1.smithi169.stdout:0/389: chown d5/d9/dc/d12/d15/d19/f20 485246 1 2024-03-04T13:42:52.367 INFO:tasks.workunit.client.1.smithi169.stdout:5/525: getdents d1/d16/d17/d1b/d74 0 2024-03-04T13:42:52.368 INFO:tasks.workunit.client.1.smithi169.stdout:0/390: write d5/d9/dc/d12/d15/d19/f67 [1893982,99470] 0 2024-03-04T13:42:52.370 INFO:tasks.workunit.client.1.smithi169.stdout:6/478: rename d0/f8a to d0/d3c/d95/fa7 0 2024-03-04T13:42:52.374 INFO:tasks.workunit.client.1.smithi169.stdout:1/436: unlink d4/d5/f38 0 2024-03-04T13:42:52.375 INFO:tasks.workunit.client.1.smithi169.stdout:2/451: symlink d1/d5/d19/d68/l9f 0 2024-03-04T13:42:52.379 INFO:tasks.workunit.client.1.smithi169.stdout:5/526: creat d1/d16/d17/d2d/d51/fa8 x:0 0 0 2024-03-04T13:42:52.382 INFO:tasks.workunit.client.1.smithi169.stdout:8/504: rename d7/d10/d22/d87 to d7/d10/d22/d75/d81/da2 0 2024-03-04T13:42:52.383 INFO:tasks.workunit.client.1.smithi169.stdout:3/487: write d7/f1d [2158580,48662] 0 2024-03-04T13:42:52.388 INFO:tasks.workunit.client.1.smithi169.stdout:5/527: unlink d1/d16/d17/d2d/c3d 0 2024-03-04T13:42:52.395 INFO:tasks.workunit.client.1.smithi169.stdout:8/505: creat d7/d10/d17/d44/d4c/fa3 x:0 0 0 2024-03-04T13:42:52.397 INFO:tasks.workunit.client.1.smithi169.stdout:5/528: creat d1/d16/d2f/d38/fa9 x:0 0 0 2024-03-04T13:42:52.398 INFO:tasks.workunit.client.1.smithi169.stdout:0/391: write d5/d9/f2d [2450173,108201] 0 2024-03-04T13:42:52.398 INFO:tasks.workunit.client.1.smithi169.stdout:8/506: write d7/d10/d22/d46/f6e [1525123,5309] 0 2024-03-04T13:42:52.408 INFO:tasks.workunit.client.1.smithi169.stdout:6/479: truncate d0/f2f 292293 0 2024-03-04T13:42:52.416 INFO:tasks.workunit.client.1.smithi169.stdout:3/488: dwrite f5 [4194304,4194304] 0 2024-03-04T13:42:52.418 INFO:tasks.workunit.client.1.smithi169.stdout:2/452: dwrite d1/d5/d19/f3f [0,4194304] 0 2024-03-04T13:42:52.429 INFO:tasks.workunit.client.1.smithi169.stdout:0/392: dwrite d5/f37 [0,4194304] 0 2024-03-04T13:42:52.430 INFO:tasks.workunit.client.1.smithi169.stdout:6/480: creat d0/d3c/d40/d52/fa8 x:0 0 0 2024-03-04T13:42:52.434 INFO:tasks.workunit.client.1.smithi169.stdout:6/481: write d0/d3c/d40/d52/f91 [4703817,68215] 0 2024-03-04T13:42:52.443 INFO:tasks.workunit.client.1.smithi169.stdout:2/453: mkdir d1/d5/d19/da0 0 2024-03-04T13:42:52.450 INFO:tasks.workunit.client.1.smithi169.stdout:5/529: dread d1/f65 [4194304,4194304] 0 2024-03-04T13:42:52.454 INFO:tasks.workunit.client.1.smithi169.stdout:5/530: fdatasync d1/d16/d17/f71 0 2024-03-04T13:42:52.455 INFO:tasks.workunit.client.1.smithi169.stdout:2/454: creat d1/d5/d19/fa1 x:0 0 0 2024-03-04T13:42:52.461 INFO:tasks.workunit.client.1.smithi169.stdout:6/482: dread d0/d32/d3a/f42 [0,4194304] 0 2024-03-04T13:42:52.467 INFO:tasks.workunit.client.1.smithi169.stdout:6/483: mknod d0/d3c/d40/ca9 0 2024-03-04T13:42:52.468 INFO:tasks.workunit.client.1.smithi169.stdout:0/393: dwrite d5/d9/dc/d12/d15/d46/f57 [0,4194304] 0 2024-03-04T13:42:52.468 INFO:tasks.workunit.client.1.smithi169.stdout:8/507: dread d7/f8b [0,4194304] 0 2024-03-04T13:42:52.470 INFO:tasks.workunit.client.1.smithi169.stdout:3/489: dwrite d7/de/d2b/d7d/d85/d13/d1a/d2d/f37 [0,4194304] 0 2024-03-04T13:42:52.473 INFO:tasks.workunit.client.1.smithi169.stdout:8/508: readlink d7/d10/d24/l8f 0 2024-03-04T13:42:52.475 INFO:tasks.workunit.client.1.smithi169.stdout:6/484: symlink d0/d7a/laa 0 2024-03-04T13:42:52.476 INFO:tasks.workunit.client.1.smithi169.stdout:2/455: link d1/d5/d22/d45/c61 d1/d5/d19/d1e/d29/ca2 0 2024-03-04T13:42:52.481 INFO:tasks.workunit.client.1.smithi169.stdout:6/485: write d0/d32/d3a/f3f [2489179,61142] 0 2024-03-04T13:42:52.483 INFO:tasks.workunit.client.1.smithi169.stdout:5/531: dwrite d1/d16/d17/d2d/d5a/d67/f6b [0,4194304] 0 2024-03-04T13:42:52.485 INFO:tasks.workunit.client.1.smithi169.stdout:8/509: write d7/ff [3114039,1587] 0 2024-03-04T13:42:52.490 INFO:tasks.workunit.client.1.smithi169.stdout:6/486: write d0/d32/d3a/f65 [564612,90299] 0 2024-03-04T13:42:52.501 INFO:tasks.workunit.client.1.smithi169.stdout:3/490: unlink d7/de/d2b/d7d/d85/d18/d21/f62 0 2024-03-04T13:42:52.501 INFO:tasks.workunit.client.1.smithi169.stdout:6/487: creat d0/d3c/d95/fab x:0 0 0 2024-03-04T13:42:52.505 INFO:tasks.workunit.client.1.smithi169.stdout:5/532: fsync d1/d16/d17/d1b/f69 0 2024-03-04T13:42:52.507 INFO:tasks.workunit.client.1.smithi169.stdout:0/394: dwrite d5/d9/d3d/d59/f3a [0,4194304] 0 2024-03-04T13:42:52.510 INFO:tasks.workunit.client.1.smithi169.stdout:3/491: read - d7/de/d2b/d7d/d85/d18/fa2 zero size 2024-03-04T13:42:52.515 INFO:tasks.workunit.client.1.smithi169.stdout:2/456: dwrite d1/d5/d22/d45/f72 [0,4194304] 0 2024-03-04T13:42:52.515 INFO:tasks.workunit.client.1.smithi169.stdout:3/492: read d7/f49 [3305678,89626] 0 2024-03-04T13:42:52.518 INFO:tasks.workunit.client.1.smithi169.stdout:6/488: dread d0/f1 [0,4194304] 0 2024-03-04T13:42:52.521 INFO:tasks.workunit.client.1.smithi169.stdout:5/533: creat d1/d16/d2f/faa x:0 0 0 2024-03-04T13:42:52.522 INFO:tasks.workunit.client.1.smithi169.stdout:2/457: write d1/d5/d19/f47 [4800305,80283] 0 2024-03-04T13:42:52.524 INFO:tasks.workunit.client.1.smithi169.stdout:0/395: mknod d5/d9/dc/d12/d15/d19/d77/c8b 0 2024-03-04T13:42:52.526 INFO:tasks.workunit.client.1.smithi169.stdout:0/396: stat d5/d4c/c5f 0 2024-03-04T13:42:52.527 INFO:tasks.workunit.client.1.smithi169.stdout:8/510: dwrite d7/d10/d24/d3e/f33 [0,4194304] 0 2024-03-04T13:42:52.530 INFO:tasks.workunit.client.1.smithi169.stdout:6/489: mkdir d0/d3c/d40/d52/dac 0 2024-03-04T13:42:52.530 INFO:tasks.workunit.client.1.smithi169.stdout:2/458: stat d1/d5/d19/d1e/f82 0 2024-03-04T13:42:52.539 INFO:tasks.workunit.client.1.smithi169.stdout:8/511: write d7/d10/d24/d5c/f90 [236036,38333] 0 2024-03-04T13:42:52.544 INFO:tasks.workunit.client.1.smithi169.stdout:2/459: rename d1/d5/f4f to d1/d5/d19/d36/fa3 0 2024-03-04T13:42:52.549 INFO:tasks.workunit.client.1.smithi169.stdout:0/397: mkdir d5/d9/dc/d12/d15/d19/d61/d71/d8c 0 2024-03-04T13:42:52.553 INFO:tasks.workunit.client.1.smithi169.stdout:5/534: dread d1/d16/d17/d2d/d5a/d67/f6b [0,4194304] 0 2024-03-04T13:42:52.557 INFO:tasks.workunit.client.1.smithi169.stdout:0/398: readlink d5/d9/dc/l45 0 2024-03-04T13:42:52.559 INFO:tasks.workunit.client.1.smithi169.stdout:6/490: dwrite d0/d3c/d40/f9d [0,4194304] 0 2024-03-04T13:42:52.559 INFO:tasks.workunit.client.1.smithi169.stdout:5/535: dread - d1/d16/d17/d2d/d51/f6e zero size 2024-03-04T13:42:52.559 INFO:tasks.workunit.client.1.smithi169.stdout:4/375: sync 2024-03-04T13:42:52.560 INFO:tasks.workunit.client.1.smithi169.stdout:2/460: creat d1/d5/d22/fa4 x:0 0 0 2024-03-04T13:42:52.566 INFO:tasks.workunit.client.1.smithi169.stdout:3/493: dwrite d7/de/d2b/d7d/d85/f4f [0,4194304] 0 2024-03-04T13:42:52.573 INFO:tasks.workunit.client.1.smithi169.stdout:0/399: mknod d5/d4c/d65/c8d 0 2024-03-04T13:42:52.574 INFO:tasks.workunit.client.1.smithi169.stdout:6/491: symlink d0/d63/lad 0 2024-03-04T13:42:52.576 INFO:tasks.workunit.client.1.smithi169.stdout:5/536: mkdir d1/d16/d2f/d72/dab 0 2024-03-04T13:42:52.583 INFO:tasks.workunit.client.1.smithi169.stdout:6/492: symlink d0/lae 0 2024-03-04T13:42:52.585 INFO:tasks.workunit.client.1.smithi169.stdout:5/537: dread d1/d23/f85 [0,4194304] 0 2024-03-04T13:42:52.586 INFO:tasks.workunit.client.1.smithi169.stdout:0/400: link d5/d9/dc/f1d d5/d4c/d65/f8e 0 2024-03-04T13:42:52.587 INFO:tasks.workunit.client.1.smithi169.stdout:6/493: chown d0/d3c/d40/f51 185702 1 2024-03-04T13:42:52.589 INFO:tasks.workunit.client.1.smithi169.stdout:4/376: dwrite d4/f7 [4194304,4194304] 0 2024-03-04T13:42:52.591 INFO:tasks.workunit.client.1.smithi169.stdout:3/494: dwrite d7/fb [0,4194304] 0 2024-03-04T13:42:52.593 INFO:tasks.workunit.client.1.smithi169.stdout:4/377: dread - d4/d11/f79 zero size 2024-03-04T13:42:52.595 INFO:tasks.workunit.client.1.smithi169.stdout:6/494: truncate d0/d7a/f87 1409440 0 2024-03-04T13:42:52.597 INFO:tasks.workunit.client.1.smithi169.stdout:8/512: dread d7/ff [0,4194304] 0 2024-03-04T13:42:52.599 INFO:tasks.workunit.client.1.smithi169.stdout:6/495: fsync d0/d3c/f68 0 2024-03-04T13:42:52.606 INFO:tasks.workunit.client.1.smithi169.stdout:3/495: symlink d7/de/d2b/d7d/d85/d47/d63/d6c/lab 0 2024-03-04T13:42:52.609 INFO:tasks.workunit.client.1.smithi169.stdout:2/461: dwrite d1/d5/d22/d45/d4a/f5d [0,4194304] 0 2024-03-04T13:42:52.613 INFO:tasks.workunit.client.1.smithi169.stdout:6/496: unlink d0/c1f 0 2024-03-04T13:42:52.615 INFO:tasks.workunit.client.1.smithi169.stdout:4/378: mknod d4/d11/d2e/d39/c7a 0 2024-03-04T13:42:52.615 INFO:tasks.workunit.client.1.smithi169.stdout:5/538: write d1/d16/d2a/f3e [1105363,99259] 0 2024-03-04T13:42:52.618 INFO:tasks.workunit.client.1.smithi169.stdout:2/462: symlink d1/d6d/la5 0 2024-03-04T13:42:52.619 INFO:tasks.workunit.client.1.smithi169.stdout:3/496: creat d7/de/d2b/d7d/d85/d13/d1a/d2d/d56/d5e/d67/d5c/d84/fac x:0 0 0 2024-03-04T13:42:52.622 INFO:tasks.workunit.client.1.smithi169.stdout:4/379: symlink d4/d1c/d58/l7b 0 2024-03-04T13:42:52.622 INFO:tasks.workunit.client.1.smithi169.stdout:8/513: write d7/d10/d24/f71 [980065,18472] 0 2024-03-04T13:42:52.627 INFO:tasks.workunit.client.1.smithi169.stdout:3/497: symlink d7/de/d2b/d7d/d85/d13/d1a/d2d/d56/d5e/d67/d5c/d84/d95/lad 0 2024-03-04T13:42:52.628 INFO:tasks.workunit.client.1.smithi169.stdout:4/380: mknod d4/d11/d2e/d39/d61/d64/c7c 0 2024-03-04T13:42:52.632 INFO:tasks.workunit.client.1.smithi169.stdout:3/498: chown d7/de/d2b/d7d/d85/d13/d1a/d2d/d56/d5e/d67/d5c/d84/d95 20 1 2024-03-04T13:42:52.635 INFO:tasks.workunit.client.1.smithi169.stdout:4/381: rename d4/d11/d23/c44 to d4/d11/d2e/d39/d61/d6b/c7d 0 2024-03-04T13:42:52.641 INFO:tasks.workunit.client.1.smithi169.stdout:4/382: unlink d4/d11/f4b 0 2024-03-04T13:42:52.652 INFO:tasks.workunit.client.1.smithi169.stdout:3/499: dwrite d7/de/d2b/d7d/d85/d18/d21/d3f/d86/f99 [0,4194304] 0 2024-03-04T13:42:52.657 INFO:tasks.workunit.client.1.smithi169.stdout:2/463: dread d1/d5/d19/f8c [0,4194304] 0 2024-03-04T13:42:52.657 INFO:tasks.workunit.client.1.smithi169.stdout:4/383: dwrite d4/f7 [4194304,4194304] 0 2024-03-04T13:42:52.660 INFO:tasks.workunit.client.1.smithi169.stdout:5/539: dread d1/d16/d17/d2d/f6a [0,4194304] 0 2024-03-04T13:42:52.665 INFO:tasks.workunit.client.1.smithi169.stdout:5/540: chown d1/d16/d17/d2d/d51/f6d 531 1 2024-03-04T13:42:52.667 INFO:tasks.workunit.client.1.smithi169.stdout:3/500: rename d7/de/d2b/d7d/d85/d18/d21/f8a to d7/de/fae 0 2024-03-04T13:42:52.671 INFO:tasks.workunit.client.1.smithi169.stdout:4/384: mkdir d4/d11/d3b/d67/d7e 0 2024-03-04T13:42:52.674 INFO:tasks.workunit.client.1.smithi169.stdout:5/541: creat d1/d16/d17/d1b/d74/fac x:0 0 0 2024-03-04T13:42:52.678 INFO:tasks.workunit.client.1.smithi169.stdout:4/385: unlink d4/d11/d2e/d21/f53 0 2024-03-04T13:42:52.681 INFO:tasks.workunit.client.1.smithi169.stdout:5/542: symlink d1/d16/d2f/d72/dab/lad 0 2024-03-04T13:42:52.681 INFO:tasks.workunit.client.1.smithi169.stdout:4/386: write d4/f7 [8618117,17853] 0 2024-03-04T13:42:52.682 INFO:tasks.workunit.client.1.smithi169.stdout:3/501: dread d7/de/d2b/d7d/d85/d13/d1a/d2d/d56/f90 [0,4194304] 0 2024-03-04T13:42:52.684 INFO:tasks.workunit.client.1.smithi169.stdout:2/464: dwrite d1/d5/d22/d45/f89 [0,4194304] 0 2024-03-04T13:42:52.689 INFO:tasks.workunit.client.1.smithi169.stdout:5/543: truncate d1/d16/d17/d1b/f29 5165559 0 2024-03-04T13:42:52.690 INFO:tasks.workunit.client.1.smithi169.stdout:3/502: creat d7/de/d2b/d7d/d85/d18/d21/d3f/d86/faf x:0 0 0 2024-03-04T13:42:52.693 INFO:tasks.workunit.client.1.smithi169.stdout:3/503: dread - d7/de/d2b/d7d/d85/d13/d5a/da1/fa7 zero size 2024-03-04T13:42:52.712 INFO:tasks.workunit.client.1.smithi169.stdout:3/504: dwrite d7/de/d2b/d7d/d85/d13/d1a/d2d/d56/d5e/d5f/f64 [0,4194304] 0 2024-03-04T13:42:52.712 INFO:tasks.workunit.client.1.smithi169.stdout:4/387: dwrite d4/fa [0,4194304] 0 2024-03-04T13:42:52.717 INFO:tasks.workunit.client.1.smithi169.stdout:4/388: write d4/d11/d2e/d37/f45 [3156618,115083] 0 2024-03-04T13:42:52.718 INFO:tasks.workunit.client.1.smithi169.stdout:2/465: dread d1/f40 [0,4194304] 0 2024-03-04T13:42:52.731 INFO:tasks.workunit.client.1.smithi169.stdout:2/466: dread d1/d5/d19/f3f [0,4194304] 0 2024-03-04T13:42:52.742 INFO:tasks.workunit.client.1.smithi169.stdout:2/467: dwrite d1/d5/d19/d36/f60 [0,4194304] 0 2024-03-04T13:42:52.801 INFO:tasks.workunit.client.1.smithi169.stdout:3/505: rmdir d7/de/d2b/d7d/d85/d13/d1a/d78 39 2024-03-04T13:42:52.826 INFO:tasks.workunit.client.1.smithi169.stdout:4/389: dwrite d4/f41 [0,4194304] 0 2024-03-04T13:42:52.833 INFO:tasks.workunit.client.1.smithi169.stdout:3/506: dwrite f6 [0,4194304] 0 2024-03-04T13:42:52.833 INFO:tasks.workunit.client.1.smithi169.stdout:2/468: dwrite d1/d5/f67 [0,4194304] 0 2024-03-04T13:42:52.841 INFO:tasks.workunit.client.1.smithi169.stdout:3/507: creat d7/de/d2b/d7d/d85/d18/fb0 x:0 0 0 2024-03-04T13:42:52.857 INFO:tasks.workunit.client.1.smithi169.stdout:3/508: dwrite d7/f36 [0,4194304] 0 2024-03-04T13:42:52.863 INFO:tasks.workunit.client.1.smithi169.stdout:3/509: mknod d7/de/d2b/d7d/d85/cb1 0 2024-03-04T13:42:52.875 INFO:tasks.workunit.client.1.smithi169.stdout:4/390: truncate d4/d1c/f2c 2548432 0 2024-03-04T13:42:52.882 INFO:tasks.workunit.client.1.smithi169.stdout:2/469: truncate d1/d5/d19/d36/f60 512314 0 2024-03-04T13:42:52.882 INFO:tasks.workunit.client.1.smithi169.stdout:4/391: dread d4/f15 [0,4194304] 0 2024-03-04T13:42:52.894 INFO:tasks.workunit.client.1.smithi169.stdout:2/470: link d1/d5/d22/d3b/d99/f93 d1/d5/d22/d3b/d99/fa6 0 2024-03-04T13:42:52.895 INFO:tasks.workunit.client.1.smithi169.stdout:4/392: link d4/d11/d2e/d63/l70 d4/d1c/d58/l7f 0 2024-03-04T13:42:52.900 INFO:tasks.workunit.client.1.smithi169.stdout:4/393: unlink d4/d11/d2e/d21/c40 0 2024-03-04T13:42:52.902 INFO:tasks.workunit.client.1.smithi169.stdout:4/394: write d4/d1c/f59 [415241,30649] 0 2024-03-04T13:42:52.905 INFO:tasks.workunit.client.1.smithi169.stdout:4/395: write d4/d11/d2e/d21/f29 [3739117,40828] 0 2024-03-04T13:42:52.907 INFO:tasks.workunit.client.1.smithi169.stdout:3/510: truncate d7/de/d2b/d7d/d85/d18/d21/d3f/d86/f99 3157833 0 2024-03-04T13:42:52.909 INFO:tasks.workunit.client.0.smithi107.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-03-04T13:42:52.910 INFO:tasks.workunit.client.1.smithi169.stdout:3/511: chown d7/de/d2b/d7d/d85/d13/d1a/d2d/f37 1 1 2024-03-04T13:42:52.911 INFO:tasks.workunit.client.1.smithi169.stdout:4/396: link d4/c49 d4/d11/d3b/c80 0 2024-03-04T13:42:52.915 INFO:tasks.workunit.client.1.smithi169.stdout:3/512: write d7/de/d2b/d7d/d85/d47/f9d [310685,20046] 0 2024-03-04T13:42:52.916 INFO:tasks.workunit.client.1.smithi169.stdout:4/397: mkdir d4/d1c/d75/d81 0 2024-03-04T13:42:52.923 INFO:tasks.workunit.client.1.smithi169.stdout:2/471: dread d1/d5/d19/d1e/d29/f3e [0,4194304] 0 2024-03-04T13:42:52.927 INFO:tasks.workunit.client.0.smithi107.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-03-04T13:42:52.927 INFO:tasks.workunit.client.0.smithi107.stderr:+ make 2024-03-04T13:42:52.930 INFO:tasks.workunit.client.1.smithi169.stdout:3/513: creat d7/de/d2b/d7d/d85/d18/d21/d3f/d68/fb2 x:0 0 0 2024-03-04T13:42:52.933 INFO:tasks.workunit.client.1.smithi169.stdout:2/472: unlink d1/d5/d22/d45/f91 0 2024-03-04T13:42:52.936 INFO:tasks.workunit.client.1.smithi169.stdout:2/473: chown d1/d5/d22/d3b/d41/c81 1863565601 1 2024-03-04T13:42:52.944 INFO:tasks.workunit.client.1.smithi169.stdout:4/398: dread d4/f9 [0,4194304] 0 2024-03-04T13:42:52.944 INFO:tasks.workunit.client.1.smithi169.stdout:2/474: rename d1/d5/d22/d45/d4a/f9a to d1/d5/d19/d68/fa7 0 2024-03-04T13:42:52.949 INFO:tasks.workunit.client.1.smithi169.stdout:4/399: write d4/d11/d3b/f3f [4821454,10111] 0 2024-03-04T13:42:52.950 INFO:tasks.workunit.client.1.smithi169.stdout:2/475: symlink d1/la8 0 2024-03-04T13:42:52.950 INFO:tasks.workunit.client.1.smithi169.stdout:3/514: dread d7/f9 [0,4194304] 0 2024-03-04T13:42:52.951 INFO:tasks.workunit.client.1.smithi169.stdout:3/515: chown d7/de/d24 62 1 2024-03-04T13:42:52.953 INFO:tasks.workunit.client.1.smithi169.stdout:3/516: fsync d7/de/d2b/d7d/d85/f6b 0 2024-03-04T13:42:52.955 INFO:tasks.workunit.client.1.smithi169.stdout:4/400: creat d4/d11/d3b/d67/d7e/f82 x:0 0 0 2024-03-04T13:42:52.955 INFO:tasks.workunit.client.1.smithi169.stdout:3/517: dread - d7/de/d2b/d7d/d85/f7f zero size 2024-03-04T13:42:52.961 INFO:tasks.workunit.client.1.smithi169.stdout:3/518: creat d7/d53/fb3 x:0 0 0 2024-03-04T13:42:52.970 INFO:tasks.workunit.client.1.smithi169.stdout:2/476: write d1/f40 [4068063,10152] 0 2024-03-04T13:42:52.975 INFO:tasks.workunit.client.1.smithi169.stdout:4/401: dread d4/f69 [0,4194304] 0 2024-03-04T13:42:52.977 INFO:tasks.workunit.client.1.smithi169.stdout:4/402: stat d4/d11/d3b/d67/d7e 0 2024-03-04T13:42:52.978 INFO:tasks.workunit.client.1.smithi169.stdout:3/519: dwrite d7/de/d24/f3c [0,4194304] 0 2024-03-04T13:42:52.980 INFO:tasks.workunit.client.1.smithi169.stdout:4/403: chown d4/d11/d2e/d39/d48/l2b 17859 1 2024-03-04T13:42:52.981 INFO:tasks.workunit.client.1.smithi169.stdout:4/404: write d4/d11/d2e/f51 [3536535,9001] 0 2024-03-04T13:42:52.984 INFO:tasks.workunit.client.1.smithi169.stdout:3/520: creat d7/de/d2b/fb4 x:0 0 0 2024-03-04T13:42:52.986 INFO:tasks.workunit.client.1.smithi169.stdout:9/507: sync 2024-03-04T13:42:52.988 INFO:tasks.workunit.client.1.smithi169.stdout:3/521: rmdir d7/de/d2b/d7d/d85/d47/d63 39 2024-03-04T13:42:52.993 INFO:tasks.workunit.client.1.smithi169.stdout:3/522: chown d7/de/d2b/d7d/d85/d13/d1a/d2d/d56/f7c 2572677 1 2024-03-04T13:42:52.994 INFO:tasks.workunit.client.1.smithi169.stdout:9/508: creat d7/d25/fa8 x:0 0 0 2024-03-04T13:42:52.995 INFO:tasks.workunit.client.1.smithi169.stdout:3/523: dread - d7/de/d2b/d7d/d85/f7f zero size 2024-03-04T13:42:52.996 INFO:tasks.workunit.client.1.smithi169.stdout:4/405: truncate d4/fa 3784795 0 2024-03-04T13:42:52.999 INFO:tasks.workunit.client.1.smithi169.stdout:2/477: dread d1/f75 [0,4194304] 0 2024-03-04T13:42:53.000 INFO:tasks.workunit.client.1.smithi169.stdout:9/509: rename d7/d27/l55 to d7/dc/d95/la9 0 2024-03-04T13:42:53.000 INFO:tasks.workunit.client.1.smithi169.stdout:4/406: rmdir d4/d11/d2e/d6c 39 2024-03-04T13:42:53.002 INFO:tasks.workunit.client.1.smithi169.stdout:2/478: read d1/d5/d22/d3b/d99/f93 [3165720,121899] 0 2024-03-04T13:42:53.003 INFO:tasks.workunit.client.1.smithi169.stdout:3/524: write d7/de/d2b/d7d/d85/d47/d63/d6c/f88 [880474,116268] 0 2024-03-04T13:42:53.004 INFO:tasks.workunit.client.1.smithi169.stdout:4/407: creat d4/d11/d3b/f83 x:0 0 0 2024-03-04T13:42:53.006 INFO:tasks.workunit.client.1.smithi169.stdout:9/510: rename d7/dc/d12/c53 to d7/d25/d64/caa 0 2024-03-04T13:42:53.008 INFO:tasks.workunit.client.0.smithi107.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-03-04T13:42:53.008 INFO:tasks.workunit.client.1.smithi169.stdout:9/511: chown d7/d25/d64 765 1 2024-03-04T13:42:53.009 INFO:tasks.workunit.client.1.smithi169.stdout:3/525: symlink d7/de/d2b/d7d/d85/lb5 0 2024-03-04T13:42:53.011 INFO:tasks.workunit.client.1.smithi169.stdout:9/512: creat d7/dc/d3e/fab x:0 0 0 2024-03-04T13:42:53.022 INFO:tasks.workunit.client.1.smithi169.stdout:3/526: getdents d7/de/d2b/d7d/d85/d13/d1a/d2d/d56/d5e/d67/d5c/d84/d95 0 2024-03-04T13:42:53.023 INFO:tasks.workunit.client.1.smithi169.stdout:9/513: rename d7/d25/d47/d56/f69 to d7/d25/fac 0 2024-03-04T13:42:53.023 INFO:tasks.workunit.client.1.smithi169.stdout:2/479: dwrite d1/d5/d19/d68/f88 [0,4194304] 0 2024-03-04T13:42:53.025 INFO:tasks.workunit.client.1.smithi169.stdout:4/408: dwrite d4/f41 [4194304,4194304] 0 2024-03-04T13:42:53.030 INFO:tasks.workunit.client.1.smithi169.stdout:9/514: rmdir d7/d25/d3c 39 2024-03-04T13:42:53.030 INFO:tasks.workunit.client.1.smithi169.stdout:4/409: dread - d4/d11/d2e/d37/f57 zero size 2024-03-04T13:42:53.031 INFO:tasks.workunit.client.1.smithi169.stdout:2/480: mknod d1/d5/d22/d3b/d99/ca9 0 2024-03-04T13:42:53.031 INFO:tasks.workunit.client.1.smithi169.stdout:9/515: rmdir d7/dc/d95 39 2024-03-04T13:42:53.034 INFO:tasks.workunit.client.1.smithi169.stdout:9/516: creat d7/dc/d3e/da3/fad x:0 0 0 2024-03-04T13:42:53.035 INFO:tasks.workunit.client.1.smithi169.stdout:2/481: rename d1/d5/l35 to d1/d5/d22/d45/laa 0 2024-03-04T13:42:53.038 INFO:tasks.workunit.client.1.smithi169.stdout:9/517: symlink d7/dc/d12/d59/lae 0 2024-03-04T13:42:53.038 INFO:tasks.workunit.client.1.smithi169.stdout:9/518: readlink d7/l19 0 2024-03-04T13:42:53.055 INFO:tasks.workunit.client.1.smithi169.stdout:2/482: dwrite d1/d5/d22/d45/d66/f7f [0,4194304] 0 2024-03-04T13:42:53.059 INFO:tasks.workunit.client.1.smithi169.stdout:2/483: mknod d1/d5/d22/d3b/d99/cab 0 2024-03-04T13:42:53.068 INFO:tasks.workunit.client.1.smithi169.stdout:9/519: dread d7/fe [0,4194304] 0 2024-03-04T13:42:53.069 INFO:tasks.workunit.client.1.smithi169.stdout:9/520: dread - d7/d2d/f85 zero size 2024-03-04T13:42:53.072 INFO:tasks.workunit.client.1.smithi169.stdout:2/484: dwrite d1/d5/f9 [0,4194304] 0 2024-03-04T13:42:53.076 INFO:tasks.workunit.client.1.smithi169.stdout:2/485: mknod d1/d5/d22/d45/d4a/cac 0 2024-03-04T13:42:53.076 INFO:tasks.workunit.client.1.smithi169.stdout:2/486: readlink d1/l79 0 2024-03-04T13:42:53.079 INFO:tasks.workunit.client.1.smithi169.stdout:2/487: creat d1/d5/fad x:0 0 0 2024-03-04T13:42:53.084 INFO:tasks.workunit.client.1.smithi169.stdout:2/488: rmdir d1/d5/d19/da0 0 2024-03-04T13:42:53.089 INFO:tasks.workunit.client.1.smithi169.stdout:2/489: link d1/d5/d22/d45/f72 d1/d5/d19/d63/fae 0 2024-03-04T13:42:53.091 INFO:tasks.workunit.client.1.smithi169.stdout:2/490: write d1/d5/d19/d1e/d29/f39 [4224036,118106] 0 2024-03-04T13:42:53.092 INFO:tasks.workunit.client.1.smithi169.stdout:2/491: chown d1/d5/fe 431880 1 2024-03-04T13:42:53.095 INFO:tasks.workunit.client.1.smithi169.stdout:2/492: mkdir d1/d5/d19/d36/daf 0 2024-03-04T13:42:53.098 INFO:tasks.workunit.client.1.smithi169.stdout:2/493: unlink d1/d5/d22/d45/d66/f7f 0 2024-03-04T13:42:53.100 INFO:tasks.workunit.client.1.smithi169.stdout:2/494: write d1/d5/d19/d36/f4b [1060708,56615] 0 2024-03-04T13:42:53.102 INFO:tasks.workunit.client.1.smithi169.stdout:2/495: write d1/d5/d22/f32 [2857350,85907] 0 2024-03-04T13:42:53.289 INFO:tasks.workunit.client.1.smithi169.stdout:1/437: sync 2024-03-04T13:42:53.290 INFO:tasks.workunit.client.1.smithi169.stdout:7/520: sync 2024-03-04T13:42:53.291 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:42:53 smithi169 ceph-mon[36313]: pgmap v48: 65 pgs: 65 active+clean; 877 MiB data, 4.2 GiB used, 532 GiB / 536 GiB avail; 19 MiB/s rd, 117 MiB/s wr, 209 op/s 2024-03-04T13:42:53.292 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:42:53 smithi169 ceph-mon[36313]: from='mgr.24331 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:42:53.299 INFO:tasks.workunit.client.1.smithi169.stdout:1/438: mknod d4/d5/d36/d35/d16/d3a/c8c 0 2024-03-04T13:42:53.320 INFO:tasks.workunit.client.1.smithi169.stdout:7/521: dwrite d2/de/d3e/d41/f4c [0,4194304] 0 2024-03-04T13:42:53.323 INFO:tasks.workunit.client.1.smithi169.stdout:7/522: symlink d2/d28/lb3 0 2024-03-04T13:42:53.340 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:42:53 smithi107 ceph-mon[28739]: pgmap v48: 65 pgs: 65 active+clean; 877 MiB data, 4.2 GiB used, 532 GiB / 536 GiB avail; 19 MiB/s rd, 117 MiB/s wr, 209 op/s 2024-03-04T13:42:53.340 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:42:53 smithi107 ceph-mon[28739]: from='mgr.24331 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:42:53.355 INFO:tasks.workunit.client.1.smithi169.stdout:7/523: dwrite d2/de/d3e/d41/f71 [0,4194304] 0 2024-03-04T13:42:53.366 INFO:tasks.workunit.client.1.smithi169.stdout:7/524: rename d2/d28/f56 to d2/de/d3e/d41/d69/d78/fb4 0 2024-03-04T13:42:53.371 INFO:tasks.workunit.client.1.smithi169.stdout:7/525: mknod d2/de/d3e/cb5 0 2024-03-04T13:42:53.374 INFO:tasks.workunit.client.1.smithi169.stdout:7/526: mknod d2/d15/d17/d6f/cb6 0 2024-03-04T13:42:53.376 INFO:tasks.workunit.client.1.smithi169.stdout:7/527: write d2/d28/d70/f79 [982352,73692] 0 2024-03-04T13:42:53.379 INFO:tasks.workunit.client.1.smithi169.stdout:7/528: symlink d2/d28/d70/lb7 0 2024-03-04T13:42:53.382 INFO:tasks.workunit.client.1.smithi169.stdout:7/529: write d2/d28/d70/f98 [13026,4939] 0 2024-03-04T13:42:53.385 INFO:tasks.workunit.client.1.smithi169.stdout:7/530: mkdir d2/d28/db8 0 2024-03-04T13:42:53.392 INFO:tasks.workunit.client.1.smithi169.stdout:7/531: truncate d2/de/d3e/d41/f5a 4776477 0 2024-03-04T13:42:53.395 INFO:tasks.workunit.client.1.smithi169.stdout:0/401: sync 2024-03-04T13:42:53.395 INFO:tasks.workunit.client.1.smithi169.stdout:6/497: sync 2024-03-04T13:42:53.400 INFO:tasks.workunit.client.1.smithi169.stdout:6/498: read d0/f35 [2990140,100166] 0 2024-03-04T13:42:53.405 INFO:tasks.workunit.client.1.smithi169.stdout:6/499: mknod d0/d3c/d95/caf 0 2024-03-04T13:42:53.408 INFO:tasks.workunit.client.1.smithi169.stdout:6/500: chown d0/d3c/d95/caf 51186521 1 2024-03-04T13:42:53.413 INFO:tasks.workunit.client.1.smithi169.stdout:6/501: creat d0/fb0 x:0 0 0 2024-03-04T13:42:53.416 INFO:tasks.workunit.client.1.smithi169.stdout:7/532: dwrite d2/de/d3e/d41/f8b [0,4194304] 0 2024-03-04T13:42:53.418 INFO:tasks.workunit.client.1.smithi169.stdout:6/502: rmdir d0/d32/d3e 39 2024-03-04T13:42:53.420 INFO:tasks.workunit.client.1.smithi169.stdout:7/533: fdatasync d2/d15/d17/f81 0 2024-03-04T13:42:53.423 INFO:tasks.workunit.client.1.smithi169.stdout:6/503: symlink d0/d32/d3a/lb1 0 2024-03-04T13:42:53.425 INFO:tasks.workunit.client.1.smithi169.stdout:7/534: unlink d2/d28/f93 0 2024-03-04T13:42:53.425 INFO:tasks.workunit.client.1.smithi169.stdout:0/402: dwrite d5/d9/dc/d12/d15/d19/d61/f54 [0,4194304] 0 2024-03-04T13:42:53.425 INFO:tasks.workunit.client.1.smithi169.stdout:6/504: fsync d0/d3c/d95/fab 0 2024-03-04T13:42:53.433 INFO:tasks.workunit.client.1.smithi169.stdout:0/403: mknod d5/d9/d3d/c8f 0 2024-03-04T13:42:53.433 INFO:tasks.workunit.client.1.smithi169.stdout:7/535: creat d2/d15/fb9 x:0 0 0 2024-03-04T13:42:53.437 INFO:tasks.workunit.client.1.smithi169.stdout:0/404: mkdir d5/d9/dc/d12/d15/d19/d61/d71/d8c/d90 0 2024-03-04T13:42:53.437 INFO:tasks.workunit.client.1.smithi169.stdout:7/536: write d2/d15/d17/d6f/f9b [231801,60309] 0 2024-03-04T13:42:53.445 INFO:tasks.workunit.client.1.smithi169.stdout:6/505: dread d0/d32/d3a/d43/d74/f62 [0,4194304] 0 2024-03-04T13:42:53.447 INFO:tasks.workunit.client.1.smithi169.stdout:6/506: write d0/f30 [932760,104107] 0 2024-03-04T13:42:53.451 INFO:tasks.workunit.client.1.smithi169.stdout:0/405: dread d5/d9/dc/d12/d15/d19/f32 [0,4194304] 0 2024-03-04T13:42:53.452 INFO:tasks.workunit.client.1.smithi169.stdout:8/514: sync 2024-03-04T13:42:53.452 INFO:tasks.workunit.client.1.smithi169.stdout:1/439: sync 2024-03-04T13:42:53.452 INFO:tasks.workunit.client.1.smithi169.stdout:3/527: sync 2024-03-04T13:42:53.452 INFO:tasks.workunit.client.1.smithi169.stdout:2/496: sync 2024-03-04T13:42:53.452 INFO:tasks.workunit.client.1.smithi169.stdout:4/410: sync 2024-03-04T13:42:53.453 INFO:tasks.workunit.client.1.smithi169.stdout:9/521: sync 2024-03-04T13:42:53.453 INFO:tasks.workunit.client.1.smithi169.stdout:5/544: sync 2024-03-04T13:42:53.454 INFO:tasks.workunit.client.1.smithi169.stdout:6/507: mknod d0/d3c/d40/d52/d98/cb2 0 2024-03-04T13:42:53.456 INFO:tasks.workunit.client.1.smithi169.stdout:8/515: dread - d7/f9f zero size 2024-03-04T13:42:53.458 INFO:tasks.workunit.client.1.smithi169.stdout:7/537: dwrite d2/d28/f33 [0,4194304] 0 2024-03-04T13:42:53.459 INFO:tasks.workunit.client.1.smithi169.stdout:1/440: read d4/f20 [4463070,42943] 0 2024-03-04T13:42:53.460 INFO:tasks.workunit.client.1.smithi169.stdout:0/406: mknod d5/d9/d3d/c91 0 2024-03-04T13:42:53.463 INFO:tasks.workunit.client.1.smithi169.stdout:6/508: rename d0/f38 to d0/d3c/d95/fb3 0 2024-03-04T13:42:53.467 INFO:tasks.workunit.client.1.smithi169.stdout:2/497: creat d1/d5/d22/d45/fb0 x:0 0 0 2024-03-04T13:42:53.475 INFO:tasks.workunit.client.1.smithi169.stdout:5/545: rename d1/d76/f9f to d1/d60/fae 0 2024-03-04T13:42:53.483 INFO:tasks.workunit.client.1.smithi169.stdout:7/538: symlink d2/d15/dad/lba 0 2024-03-04T13:42:53.484 INFO:tasks.workunit.client.1.smithi169.stdout:6/509: rename d0/f1d to d0/d3c/d40/d52/fb4 0 2024-03-04T13:42:53.486 INFO:tasks.workunit.client.1.smithi169.stdout:7/539: write d2/d6b/f76 [3778652,130365] 0 2024-03-04T13:42:53.491 INFO:tasks.workunit.client.1.smithi169.stdout:6/510: rename d0/d3c/d40/d52/d98 to d0/d3c/d40/d52/d98/db5 22 2024-03-04T13:42:53.493 INFO:tasks.workunit.client.1.smithi169.stdout:3/528: write d7/de/d2b/d7d/d85/d18/f30 [235975,4873] 0 2024-03-04T13:42:53.497 INFO:tasks.workunit.client.1.smithi169.stdout:9/522: dread d7/dc/fd [4194304,4194304] 0 2024-03-04T13:42:53.503 INFO:tasks.workunit.client.1.smithi169.stdout:5/546: link d1/d16/d17/l7f d1/d60/laf 0 2024-03-04T13:42:53.509 INFO:tasks.workunit.client.1.smithi169.stdout:4/411: dwrite d4/d11/d2e/d39/f60 [0,4194304] 0 2024-03-04T13:42:53.514 INFO:tasks.workunit.client.1.smithi169.stdout:6/511: link d0/f2b d0/d3c/d40/d52/d98/fb6 0 2024-03-04T13:42:53.515 INFO:tasks.workunit.client.1.smithi169.stdout:3/529: chown d7/de/d2b/d7d/d85/d47/c87 6718814 1 2024-03-04T13:42:53.522 INFO:tasks.workunit.client.1.smithi169.stdout:0/407: dwrite d5/d9/dc/f1d [0,4194304] 0 2024-03-04T13:42:53.525 INFO:tasks.workunit.client.1.smithi169.stdout:1/441: dwrite d4/d5/d2d/d4e/d71/d26/d4f/f51 [0,4194304] 0 2024-03-04T13:42:53.527 INFO:tasks.workunit.client.1.smithi169.stdout:8/516: dwrite d7/d10/d22/d46/f9d [4194304,4194304] 0 2024-03-04T13:42:53.530 INFO:tasks.workunit.client.1.smithi169.stdout:4/412: creat d4/d11/d3b/d67/f84 x:0 0 0 2024-03-04T13:42:53.531 INFO:tasks.workunit.client.1.smithi169.stdout:5/547: creat d1/fb0 x:0 0 0 2024-03-04T13:42:53.532 INFO:tasks.workunit.client.1.smithi169.stdout:7/540: dread d2/de/d3e/d41/f71 [0,4194304] 0 2024-03-04T13:42:53.536 INFO:tasks.workunit.client.1.smithi169.stdout:2/498: dwrite d1/d6d/f87 [0,4194304] 0 2024-03-04T13:42:53.536 INFO:tasks.workunit.client.1.smithi169.stdout:0/408: truncate d5/d9/d3d/f41 8408222 0 2024-03-04T13:42:53.537 INFO:tasks.workunit.client.1.smithi169.stdout:7/541: chown d2/d15/d17/d6f 62 1 2024-03-04T13:42:53.542 INFO:tasks.workunit.client.1.smithi169.stdout:3/530: rename d7/de/fae to d7/de/d2b/d7d/d85/d13/d1a/d2d/fb6 0 2024-03-04T13:42:53.545 INFO:tasks.workunit.client.1.smithi169.stdout:5/548: mkdir d1/d81/db1 0 2024-03-04T13:42:53.545 INFO:tasks.workunit.client.1.smithi169.stdout:8/517: mkdir d7/d10/d22/d75/d81/da4 0 2024-03-04T13:42:53.545 INFO:tasks.workunit.client.1.smithi169.stdout:6/512: creat d0/d32/d3e/fb7 x:0 0 0 2024-03-04T13:42:53.549 INFO:tasks.workunit.client.1.smithi169.stdout:2/499: creat d1/d5/d19/d36/d64/fb1 x:0 0 0 2024-03-04T13:42:53.550 INFO:tasks.workunit.client.1.smithi169.stdout:4/413: rmdir d4/d11/d3b/d67 39 2024-03-04T13:42:53.552 INFO:tasks.workunit.client.1.smithi169.stdout:4/414: chown d4/f15 2 1 2024-03-04T13:42:53.553 INFO:tasks.workunit.client.1.smithi169.stdout:3/531: dread d7/d53/f5d [0,4194304] 0 2024-03-04T13:42:53.554 INFO:tasks.workunit.client.1.smithi169.stdout:1/442: mknod d4/d5/d36/d35/c8d 0 2024-03-04T13:42:53.557 INFO:tasks.workunit.client.1.smithi169.stdout:5/549: creat d1/d76/fb2 x:0 0 0 2024-03-04T13:42:53.558 INFO:tasks.workunit.client.1.smithi169.stdout:1/443: readlink d4/d5/d2d/d4e/d71/d26/d7e/l8b 0 2024-03-04T13:42:53.559 INFO:tasks.workunit.client.1.smithi169.stdout:5/550: chown d1/d16/f20 0 1 2024-03-04T13:42:53.560 INFO:tasks.workunit.client.1.smithi169.stdout:6/513: chown d0/d32/d3a/d43/f45 2776 1 2024-03-04T13:42:53.562 INFO:tasks.workunit.client.1.smithi169.stdout:4/415: chown d4/cf 4626366 1 2024-03-04T13:42:53.562 INFO:tasks.workunit.client.1.smithi169.stdout:5/551: chown d1/c12 695318580 1 2024-03-04T13:42:53.564 INFO:tasks.workunit.client.1.smithi169.stdout:2/500: symlink d1/d5/d22/d3b/lb2 0 2024-03-04T13:42:53.567 INFO:tasks.workunit.client.1.smithi169.stdout:3/532: creat d7/de/d2b/d7d/d85/d13/d5a/da1/fb7 x:0 0 0 2024-03-04T13:42:53.569 INFO:tasks.workunit.client.1.smithi169.stdout:0/409: write d5/d9/dc/d12/f53 [3026361,115164] 0 2024-03-04T13:42:53.571 INFO:tasks.workunit.client.1.smithi169.stdout:7/542: rmdir d2/de/d8a 0 2024-03-04T13:42:53.572 INFO:tasks.workunit.client.1.smithi169.stdout:4/416: rename d4/d11/d2e/d21/f29 to d4/d1c/d75/d81/f85 0 2024-03-04T13:42:53.579 INFO:tasks.workunit.client.1.smithi169.stdout:7/543: mkdir d2/de/d3e/d7e/dbb 0 2024-03-04T13:42:53.583 INFO:tasks.workunit.client.1.smithi169.stdout:3/533: mkdir d7/db8 0 2024-03-04T13:42:53.587 INFO:tasks.workunit.client.1.smithi169.stdout:0/410: rename d5/d9/d3d/f41 to d5/d4c/d65/f92 0 2024-03-04T13:42:53.588 INFO:tasks.workunit.client.1.smithi169.stdout:4/417: mkdir d4/d11/d2e/d37/d5c/d86 0 2024-03-04T13:42:53.590 INFO:tasks.workunit.client.1.smithi169.stdout:3/534: fsync d7/de/d2b/d7d/d85/d13/d1a/d2d/d56/fa3 0 2024-03-04T13:42:53.591 INFO:tasks.workunit.client.1.smithi169.stdout:1/444: dwrite d4/f1b [0,4194304] 0 2024-03-04T13:42:53.593 INFO:tasks.workunit.client.1.smithi169.stdout:5/552: getdents d1/d81 0 2024-03-04T13:42:53.594 INFO:tasks.workunit.client.1.smithi169.stdout:6/514: link d0/d32/d3a/l25 d0/d3c/d40/d52/dac/lb8 0 2024-03-04T13:42:53.597 INFO:tasks.workunit.client.1.smithi169.stdout:6/515: chown d0/d63/f78 53 1 2024-03-04T13:42:53.599 INFO:tasks.workunit.client.1.smithi169.stdout:1/445: creat d4/d1d/f8e x:0 0 0 2024-03-04T13:42:53.600 INFO:tasks.workunit.client.1.smithi169.stdout:4/418: fsync d4/d11/d3b/d67/f84 0 2024-03-04T13:42:53.601 INFO:tasks.workunit.client.1.smithi169.stdout:6/516: write d0/d3c/d95/fa7 [1767462,115630] 0 2024-03-04T13:42:53.601 INFO:tasks.workunit.client.1.smithi169.stdout:3/535: truncate d7/de/d2b/d7d/d85/d18/d21/f31 541958 0 2024-03-04T13:42:53.602 INFO:tasks.workunit.client.1.smithi169.stdout:0/411: symlink d5/d9/dc/d12/d15/d19/d69/l93 0 2024-03-04T13:42:53.605 INFO:tasks.workunit.client.1.smithi169.stdout:2/501: rename d1/f3 to d1/d5/d19/d36/daf/fb3 0 2024-03-04T13:42:53.608 INFO:tasks.workunit.client.1.smithi169.stdout:1/446: write d4/d5/d36/d35/df/f46 [2574538,103828] 0 2024-03-04T13:42:53.612 INFO:tasks.workunit.client.1.smithi169.stdout:6/517: rmdir d0/d32/d70 39 2024-03-04T13:42:53.617 INFO:tasks.workunit.client.1.smithi169.stdout:2/502: unlink d1/d5/d22/fa4 0 2024-03-04T13:42:53.617 INFO:tasks.workunit.client.1.smithi169.stdout:8/518: dread d7/d10/d22/d46/f6e [0,4194304] 0 2024-03-04T13:42:53.622 INFO:tasks.workunit.client.1.smithi169.stdout:7/544: dwrite d2/d28/f33 [0,4194304] 0 2024-03-04T13:42:53.624 INFO:tasks.workunit.client.1.smithi169.stdout:6/518: mkdir d0/d32/d3e/db9 0 2024-03-04T13:42:53.624 INFO:tasks.workunit.client.1.smithi169.stdout:2/503: stat d1/d5/d19/d63/c6f 0 2024-03-04T13:42:53.630 INFO:tasks.workunit.client.1.smithi169.stdout:5/553: dwrite d1/d16/d2f/faa [0,4194304] 0 2024-03-04T13:42:53.633 INFO:tasks.workunit.client.1.smithi169.stdout:6/519: creat d0/d63/fba x:0 0 0 2024-03-04T13:42:53.635 INFO:tasks.workunit.client.1.smithi169.stdout:9/523: sync 2024-03-04T13:42:53.637 INFO:tasks.workunit.client.1.smithi169.stdout:8/519: mkdir d7/d10/d22/d46/da5 0 2024-03-04T13:42:53.637 INFO:tasks.workunit.client.1.smithi169.stdout:4/419: dread d4/f7 [0,4194304] 0 2024-03-04T13:42:53.639 INFO:tasks.workunit.client.1.smithi169.stdout:2/504: creat d1/d5/d19/d36/d64/fb4 x:0 0 0 2024-03-04T13:42:53.639 INFO:tasks.workunit.client.1.smithi169.stdout:0/412: dwrite d5/d9/f2d [0,4194304] 0 2024-03-04T13:42:53.639 INFO:tasks.workunit.client.1.smithi169.stdout:7/545: symlink d2/de/d3e/d41/lbc 0 2024-03-04T13:42:53.640 INFO:tasks.workunit.client.1.smithi169.stdout:6/520: truncate d0/f15 1692998 0 2024-03-04T13:42:53.648 INFO:tasks.workunit.client.1.smithi169.stdout:6/521: chown d0/d32/d3a/f27 912953 1 2024-03-04T13:42:53.649 INFO:tasks.workunit.client.1.smithi169.stdout:3/536: dwrite d7/de/d24/f89 [0,4194304] 0 2024-03-04T13:42:53.650 INFO:tasks.workunit.client.1.smithi169.stdout:9/524: creat d7/dc/d12/d20/da6/faf x:0 0 0 2024-03-04T13:42:53.654 INFO:tasks.workunit.client.1.smithi169.stdout:4/420: creat d4/d11/d2e/d37/f87 x:0 0 0 2024-03-04T13:42:53.654 INFO:tasks.workunit.client.1.smithi169.stdout:8/520: mkdir d7/d10/d22/d46/da5/da6 0 2024-03-04T13:42:53.660 INFO:tasks.workunit.client.1.smithi169.stdout:0/413: rename d5/d9/dc/d12/d15/d46/d82 to d5/d9/dc/d12/d15/d19/d77/d94 0 2024-03-04T13:42:53.669 INFO:tasks.workunit.client.1.smithi169.stdout:1/447: dread d4/d5/d36/f59 [0,4194304] 0 2024-03-04T13:42:53.669 INFO:tasks.workunit.client.1.smithi169.stdout:5/554: getdents d1/d16/d17/d2d/d51 0 2024-03-04T13:42:53.673 INFO:tasks.workunit.client.1.smithi169.stdout:4/421: symlink d4/l88 0 2024-03-04T13:42:53.679 INFO:tasks.workunit.client.1.smithi169.stdout:3/537: getdents d7/de/d2b/d7d/d85/d13/d1a/d2d 0 2024-03-04T13:42:53.680 INFO:tasks.workunit.client.1.smithi169.stdout:5/555: creat d1/d16/d2f/d39/fb3 x:0 0 0 2024-03-04T13:42:53.681 INFO:tasks.workunit.client.1.smithi169.stdout:4/422: dread - d4/d11/d2e/d39/f68 zero size 2024-03-04T13:42:53.682 INFO:tasks.workunit.client.1.smithi169.stdout:1/448: mkdir d4/d5/d36/d35/d8f 0 2024-03-04T13:42:53.683 INFO:tasks.workunit.client.1.smithi169.stdout:3/538: unlink d7/f9 0 2024-03-04T13:42:53.686 INFO:tasks.workunit.client.1.smithi169.stdout:2/505: dwrite d1/d5/d19/d69/f7e [0,4194304] 0 2024-03-04T13:42:53.688 INFO:tasks.workunit.client.1.smithi169.stdout:6/522: dread d0/d3c/d40/f9d [0,4194304] 0 2024-03-04T13:42:53.691 INFO:tasks.workunit.client.1.smithi169.stdout:7/546: dwrite d2/f22 [0,4194304] 0 2024-03-04T13:42:53.693 INFO:tasks.workunit.client.1.smithi169.stdout:5/556: rename d1/f14 to d1/d16/d2a/d62/fb4 0 2024-03-04T13:42:53.696 INFO:tasks.workunit.client.1.smithi169.stdout:9/525: dwrite d7/dc/d3e/da3/fad [0,4194304] 0 2024-03-04T13:42:53.696 INFO:tasks.workunit.client.1.smithi169.stdout:2/506: dread - d1/d5/d22/d45/d4a/f7d zero size 2024-03-04T13:42:53.701 INFO:tasks.workunit.client.1.smithi169.stdout:1/449: read d4/d5/d2d/d4e/d71/d26/f7b [1123607,69817] 0 2024-03-04T13:42:53.703 INFO:tasks.workunit.client.1.smithi169.stdout:7/547: fsync d2/d15/d17/f81 0 2024-03-04T13:42:53.704 INFO:tasks.workunit.client.1.smithi169.stdout:0/414: dwrite d5/d4c/f51 [0,4194304] 0 2024-03-04T13:42:53.706 INFO:tasks.workunit.client.1.smithi169.stdout:1/450: dread - d4/d1d/f8e zero size 2024-03-04T13:42:53.707 INFO:tasks.workunit.client.1.smithi169.stdout:6/523: read d0/f15 [824468,52595] 0 2024-03-04T13:42:53.708 INFO:tasks.workunit.client.1.smithi169.stdout:5/557: truncate d1/d16/d17/d28/f4c 4654163 0 2024-03-04T13:42:53.713 INFO:tasks.workunit.client.1.smithi169.stdout:7/548: symlink d2/de/d3e/d41/lbd 0 2024-03-04T13:42:53.716 INFO:tasks.workunit.client.1.smithi169.stdout:7/549: chown d2/d15/fb9 14 1 2024-03-04T13:42:53.718 INFO:tasks.workunit.client.1.smithi169.stdout:1/451: unlink d4/d5/d36/d35/d16/c65 0 2024-03-04T13:42:53.719 INFO:tasks.workunit.client.1.smithi169.stdout:4/423: link d4/cf d4/d11/d2e/d21/c89 0 2024-03-04T13:42:53.720 INFO:tasks.workunit.client.1.smithi169.stdout:6/524: getdents d0/d32/d3a/d43/d74/d93 0 2024-03-04T13:42:53.722 INFO:tasks.workunit.client.1.smithi169.stdout:9/526: creat d7/d2d/fb0 x:0 0 0 2024-03-04T13:42:53.729 INFO:tasks.workunit.client.1.smithi169.stdout:1/452: unlink d4/d5/d36/d35/c12 0 2024-03-04T13:42:53.732 INFO:tasks.workunit.client.1.smithi169.stdout:9/527: creat d7/dc/d3e/fb1 x:0 0 0 2024-03-04T13:42:53.734 INFO:tasks.workunit.client.1.smithi169.stdout:2/507: dread d1/d5/d19/f3f [0,4194304] 0 2024-03-04T13:42:53.734 INFO:tasks.workunit.client.1.smithi169.stdout:3/539: dwrite d7/de/d2b/d7d/d85/d13/d1a/d2d/d56/d5e/d67/f3a [0,4194304] 0 2024-03-04T13:42:53.737 INFO:tasks.workunit.client.1.smithi169.stdout:4/424: rename d4/d11/d23/f35 to d4/d11/d2e/d63/f8a 0 2024-03-04T13:42:53.738 INFO:tasks.workunit.client.1.smithi169.stdout:1/453: chown d4/d5/d36/f59 3882960 1 2024-03-04T13:42:53.739 INFO:tasks.workunit.client.1.smithi169.stdout:9/528: stat d7/d2d/d48/d66 0 2024-03-04T13:42:53.743 INFO:tasks.workunit.client.1.smithi169.stdout:5/558: dwrite d1/f8c [0,4194304] 0 2024-03-04T13:42:53.745 INFO:tasks.workunit.client.1.smithi169.stdout:4/425: chown d4/d11/d3b/d67/f84 514896 1 2024-03-04T13:42:53.748 INFO:tasks.workunit.client.1.smithi169.stdout:0/415: dwrite d5/f37 [4194304,4194304] 0 2024-03-04T13:42:53.748 INFO:tasks.workunit.client.1.smithi169.stdout:7/550: truncate d2/de/d3e/d41/d69/fab 669110 0 2024-03-04T13:42:53.750 INFO:tasks.workunit.client.1.smithi169.stdout:9/529: write d7/dc/d3e/da3/fad [3504789,105726] 0 2024-03-04T13:42:53.751 INFO:tasks.workunit.client.1.smithi169.stdout:6/525: dwrite d0/d3c/f86 [0,4194304] 0 2024-03-04T13:42:53.754 INFO:tasks.workunit.client.1.smithi169.stdout:3/540: write d7/de/d2b/d7d/d85/d13/d1a/d2d/d56/d5e/d67/d5c/d84/f96 [823580,88598] 0 2024-03-04T13:42:53.756 INFO:tasks.workunit.client.1.smithi169.stdout:9/530: dread - d7/d25/d64/f8b zero size 2024-03-04T13:42:53.760 INFO:tasks.workunit.client.1.smithi169.stdout:5/559: symlink d1/d16/d2a/lb5 0 2024-03-04T13:42:53.761 INFO:tasks.workunit.client.1.smithi169.stdout:4/426: mkdir d4/d11/d2e/d39/d48/d8b 0 2024-03-04T13:42:53.764 INFO:tasks.workunit.client.1.smithi169.stdout:7/551: mknod d2/de/d3e/d41/d69/cbe 0 2024-03-04T13:42:53.769 INFO:tasks.workunit.client.1.smithi169.stdout:4/427: chown d4/d11/d2e/d39/d48 221866965 1 2024-03-04T13:42:53.771 INFO:tasks.workunit.client.1.smithi169.stdout:3/541: mknod d7/de/d2b/d7d/d85/d13/d1a/d2d/d56/d5e/d5f/cb9 0 2024-03-04T13:42:53.774 INFO:tasks.workunit.client.1.smithi169.stdout:5/560: symlink d1/d16/d17/d2d/lb6 0 2024-03-04T13:42:53.775 INFO:tasks.workunit.client.1.smithi169.stdout:1/454: dread d4/d5/d36/d35/d5b/f87 [0,4194304] 0 2024-03-04T13:42:53.778 INFO:tasks.workunit.client.1.smithi169.stdout:6/526: rename d0/d32/d3a/d43/c59 to d0/d3c/d40/d52/cbb 0 2024-03-04T13:42:53.779 INFO:tasks.workunit.client.1.smithi169.stdout:0/416: dwrite d5/d9/f42 [0,4194304] 0 2024-03-04T13:42:53.779 INFO:tasks.workunit.client.1.smithi169.stdout:7/552: write d2/de/d3e/d41/d69/d78/fb4 [2146871,25570] 0 2024-03-04T13:42:53.780 INFO:tasks.workunit.client.1.smithi169.stdout:1/455: fdatasync d4/f7d 0 2024-03-04T13:42:53.780 INFO:tasks.workunit.client.1.smithi169.stdout:4/428: creat d4/d11/d2e/d37/d5c/f8c x:0 0 0 2024-03-04T13:42:53.784 INFO:tasks.workunit.client.1.smithi169.stdout:6/527: dread - d0/d32/d3a/d43/d74/f7e zero size 2024-03-04T13:42:53.784 INFO:tasks.workunit.client.1.smithi169.stdout:9/531: dread d7/f33 [0,4194304] 0 2024-03-04T13:42:53.786 INFO:tasks.workunit.client.1.smithi169.stdout:3/542: truncate d7/de/d2b/d7d/d85/d18/fa2 449843 0 2024-03-04T13:42:53.792 INFO:tasks.workunit.client.1.smithi169.stdout:4/429: dread - d4/d11/d3b/d67/f84 zero size 2024-03-04T13:42:53.794 INFO:tasks.workunit.client.1.smithi169.stdout:7/553: read d2/de/d37/f3f [896462,29690] 0 2024-03-04T13:42:53.794 INFO:tasks.workunit.client.1.smithi169.stdout:3/543: write d7/de/d2b/d7d/d85/d13/f4a [826909,84220] 0 2024-03-04T13:42:53.799 INFO:tasks.workunit.client.1.smithi169.stdout:7/554: write d2/de/d3e/d7e/fa7 [206230,112054] 0 2024-03-04T13:42:53.801 INFO:tasks.workunit.client.1.smithi169.stdout:5/561: dwrite d1/d81/f98 [0,4194304] 0 2024-03-04T13:42:53.805 INFO:tasks.workunit.client.1.smithi169.stdout:5/562: write d1/d16/d2f/d38/f4f [4068002,74723] 0 2024-03-04T13:42:53.819 INFO:tasks.workunit.client.1.smithi169.stdout:5/563: dwrite d1/d16/d17/d2d/d51/f6d [0,4194304] 0 2024-03-04T13:42:53.845 INFO:tasks.workunit.client.1.smithi169.stdout:0/417: mkdir d5/d4c/d65/d95 0 2024-03-04T13:42:53.847 INFO:tasks.workunit.client.1.smithi169.stdout:1/456: fsync d4/d5/d36/d35/df/f1c 0 2024-03-04T13:42:53.852 INFO:tasks.workunit.client.1.smithi169.stdout:9/532: mknod d7/dc/d12/d8a/cb2 0 2024-03-04T13:42:53.858 INFO:tasks.workunit.client.1.smithi169.stdout:8/521: sync 2024-03-04T13:42:53.861 INFO:tasks.workunit.client.1.smithi169.stdout:8/522: fsync d7/d10/d24/d3e/f40 0 2024-03-04T13:42:53.864 INFO:tasks.workunit.client.1.smithi169.stdout:1/457: dwrite d4/d5/d36/d35/df/f1c [0,4194304] 0 2024-03-04T13:42:53.866 INFO:tasks.workunit.client.1.smithi169.stdout:3/544: rename d7/de/d2b/d7d/d85/f48 to d7/de/d2b/d7d/d85/d47/d63/d6c/fba 0 2024-03-04T13:42:53.869 INFO:tasks.workunit.client.1.smithi169.stdout:0/418: dread d5/d9/dc/d12/f44 [4194304,4194304] 0 2024-03-04T13:42:53.875 INFO:tasks.workunit.client.1.smithi169.stdout:9/533: mknod d7/d25/d47/cb3 0 2024-03-04T13:42:53.879 INFO:tasks.workunit.client.1.smithi169.stdout:1/458: dwrite d4/f25 [0,4194304] 0 2024-03-04T13:42:53.882 INFO:tasks.workunit.client.1.smithi169.stdout:1/459: write d4/d5/f49 [4338607,76577] 0 2024-03-04T13:42:53.886 INFO:tasks.workunit.client.1.smithi169.stdout:1/460: stat d4/d1d/l4c 0 2024-03-04T13:42:53.886 INFO:tasks.workunit.client.1.smithi169.stdout:2/508: sync 2024-03-04T13:42:53.890 INFO:tasks.workunit.client.1.smithi169.stdout:2/509: dread - d1/d5/d19/d36/d64/fb1 zero size 2024-03-04T13:42:53.895 INFO:tasks.workunit.client.1.smithi169.stdout:0/419: mknod d5/d9/d3d/d59/d64/c96 0 2024-03-04T13:42:53.899 INFO:tasks.workunit.client.1.smithi169.stdout:0/420: read d5/f37 [3084431,7449] 0 2024-03-04T13:42:53.900 INFO:tasks.workunit.client.1.smithi169.stdout:9/534: creat d7/dc/d12/d20/da6/fb4 x:0 0 0 2024-03-04T13:42:53.902 INFO:tasks.workunit.client.1.smithi169.stdout:2/510: dread d1/d5/d22/d3b/d99/fa6 [0,4194304] 0 2024-03-04T13:42:53.903 INFO:tasks.workunit.client.1.smithi169.stdout:4/430: creat d4/d11/f8d x:0 0 0 2024-03-04T13:42:53.905 INFO:tasks.workunit.client.1.smithi169.stdout:4/431: chown d4/d11/d3b/d67/f84 936 1 2024-03-04T13:42:53.909 INFO:tasks.workunit.client.1.smithi169.stdout:3/545: getdents d7/de/d2b/d7d/d85/d18/d21/d66 0 2024-03-04T13:42:53.923 INFO:tasks.workunit.client.1.smithi169.stdout:1/461: chown d4/d5/d36/d35/d5b/l7a 1 1 2024-03-04T13:42:53.926 INFO:tasks.workunit.client.1.smithi169.stdout:1/462: write d4/d5/d36/f88 [551284,23895] 0 2024-03-04T13:42:53.927 INFO:tasks.workunit.client.1.smithi169.stdout:2/511: dwrite d1/d5/d19/d1e/f24 [4194304,4194304] 0 2024-03-04T13:42:53.927 INFO:tasks.workunit.client.1.smithi169.stdout:9/535: dread d7/f10 [0,4194304] 0 2024-03-04T13:42:53.929 INFO:tasks.workunit.client.1.smithi169.stdout:1/463: write d4/d1d/f40 [698608,83162] 0 2024-03-04T13:42:53.932 INFO:tasks.workunit.client.1.smithi169.stdout:2/512: fdatasync d1/d5/d19/d1e/d29/f3e 0 2024-03-04T13:42:53.940 INFO:tasks.workunit.client.1.smithi169.stdout:0/421: creat d5/d9/dc/d12/d15/d19/d61/d71/d8c/d90/f97 x:0 0 0 2024-03-04T13:42:53.940 INFO:tasks.workunit.client.1.smithi169.stdout:4/432: dwrite d4/d1c/d75/d81/f85 [0,4194304] 0 2024-03-04T13:42:53.944 INFO:tasks.workunit.client.1.smithi169.stdout:0/422: chown d5/d9/dc/d12/d15/d19/d77/d94/c83 22560 1 2024-03-04T13:42:53.947 INFO:tasks.workunit.client.1.smithi169.stdout:0/423: readlink d5/d9/dc/d12/d15/l87 0 2024-03-04T13:42:53.948 INFO:tasks.workunit.client.1.smithi169.stdout:3/546: dwrite d7/f25 [0,4194304] 0 2024-03-04T13:42:53.954 INFO:tasks.workunit.client.1.smithi169.stdout:3/547: chown d7/de/d2b/d7d/d85/d18/d21/la8 32136830 1 2024-03-04T13:42:53.989 INFO:tasks.workunit.client.1.smithi169.stdout:8/523: creat d7/d10/d22/fa7 x:0 0 0 2024-03-04T13:42:54.002 INFO:tasks.workunit.client.1.smithi169.stdout:7/555: unlink d2/d28/f60 0 2024-03-04T13:42:54.004 INFO:tasks.workunit.client.1.smithi169.stdout:7/556: fsync d2/d15/fa6 0 2024-03-04T13:42:54.006 INFO:tasks.workunit.client.1.smithi169.stdout:1/464: rename d4/d5/d36/c3b to d4/d5/d2d/d4e/d71/d26/d4f/c90 0 2024-03-04T13:42:54.009 INFO:tasks.workunit.client.1.smithi169.stdout:0/424: rmdir d5/d9/dc/d12/d15/d19/d77 39 2024-03-04T13:42:54.013 INFO:tasks.workunit.client.1.smithi169.stdout:6/528: link d0/f4a d0/d3c/fbc 0 2024-03-04T13:42:54.016 INFO:tasks.workunit.client.1.smithi169.stdout:8/524: creat d7/d10/d17/d44/d36/fa8 x:0 0 0 2024-03-04T13:42:54.018 INFO:tasks.workunit.client.1.smithi169.stdout:7/557: creat d2/d15/d17/fbf x:0 0 0 2024-03-04T13:42:54.021 INFO:tasks.workunit.client.1.smithi169.stdout:1/465: symlink d4/d5/d36/d35/d70/d78/l91 0 2024-03-04T13:42:54.023 INFO:tasks.workunit.client.1.smithi169.stdout:1/466: read d4/d5/d36/d35/d5b/f87 [2142585,27639] 0 2024-03-04T13:42:54.023 INFO:tasks.workunit.client.1.smithi169.stdout:0/425: mkdir d5/d9/dc/d12/d15/d46/d98 0 2024-03-04T13:42:54.027 INFO:tasks.workunit.client.1.smithi169.stdout:1/467: read d4/d5/d2d/d4e/d71/d26/d4f/f51 [2820250,53281] 0 2024-03-04T13:42:54.028 INFO:tasks.workunit.client.1.smithi169.stdout:6/529: truncate d0/d32/d3a/d43/f7b 662891 0 2024-03-04T13:42:54.028 INFO:tasks.workunit.client.1.smithi169.stdout:2/513: link d1/d5/d22/d45/d66/c8b d1/d5/d19/d1e/d29/cb5 0 2024-03-04T13:42:54.029 INFO:tasks.workunit.client.1.smithi169.stdout:3/548: link d7/de/d2b/d7d/d85/d18/d21/d3f/d86/l8d d7/de/d2b/d7d/d85/d13/d1a/d2d/d56/d5e/d5f/lbb 0 2024-03-04T13:42:54.031 INFO:tasks.workunit.client.1.smithi169.stdout:1/468: readlink d4/d5/d36/d35/d16/d3a/l44 0 2024-03-04T13:42:54.032 INFO:tasks.workunit.client.1.smithi169.stdout:0/426: symlink d5/d4c/d65/l99 0 2024-03-04T13:42:54.033 INFO:tasks.workunit.client.1.smithi169.stdout:5/564: link d1/d16/d17/d1b/c22 d1/d16/d17/d2d/cb7 0 2024-03-04T13:42:54.035 INFO:tasks.workunit.client.1.smithi169.stdout:8/525: dwrite d7/d10/d17/d44/f4f [0,4194304] 0 2024-03-04T13:42:54.038 INFO:tasks.workunit.client.1.smithi169.stdout:6/530: mkdir d0/d3c/d95/dbd 0 2024-03-04T13:42:54.041 INFO:tasks.workunit.client.1.smithi169.stdout:2/514: symlink d1/d5/d22/d45/d4a/lb6 0 2024-03-04T13:42:54.043 INFO:tasks.workunit.client.1.smithi169.stdout:2/515: fsync d1/d5/d19/d63/f8e 0 2024-03-04T13:42:54.045 INFO:tasks.workunit.client.1.smithi169.stdout:0/427: symlink d5/d9/dc/d12/d66/l9a 0 2024-03-04T13:42:54.045 INFO:tasks.workunit.client.1.smithi169.stdout:1/469: fdatasync d4/d5/d36/d35/d5b/f87 0 2024-03-04T13:42:54.047 INFO:tasks.workunit.client.1.smithi169.stdout:5/565: mkdir d1/d16/d2f/d72/db8 0 2024-03-04T13:42:54.051 INFO:tasks.workunit.client.1.smithi169.stdout:3/549: mknod d7/de/d2b/d7d/d85/d47/cbc 0 2024-03-04T13:42:54.054 INFO:tasks.workunit.client.1.smithi169.stdout:2/516: symlink d1/d5/d19/d63/lb7 0 2024-03-04T13:42:54.054 INFO:tasks.workunit.client.1.smithi169.stdout:3/550: readlink d7/de/d2b/d7d/d85/d18/d21/la8 0 2024-03-04T13:42:54.057 INFO:tasks.workunit.client.1.smithi169.stdout:8/526: mknod d7/d10/d22/d46/d49/ca9 0 2024-03-04T13:42:54.058 INFO:tasks.workunit.client.1.smithi169.stdout:5/566: creat d1/d81/d91/fb9 x:0 0 0 2024-03-04T13:42:54.059 INFO:tasks.workunit.client.1.smithi169.stdout:2/517: mkdir d1/d5/d19/d3c/db8 0 2024-03-04T13:42:54.060 INFO:tasks.workunit.client.1.smithi169.stdout:1/470: write d4/d5/d2d/d4e/d71/d26/f7b [2799793,12075] 0 2024-03-04T13:42:54.061 INFO:tasks.workunit.client.1.smithi169.stdout:8/527: read - d7/d10/d17/d44/d4c/fa0 zero size 2024-03-04T13:42:54.063 INFO:tasks.workunit.client.1.smithi169.stdout:5/567: write d1/d16/d17/d2d/d5a/d67/f82 [311035,27850] 0 2024-03-04T13:42:54.063 INFO:tasks.workunit.client.1.smithi169.stdout:0/428: dread d5/d9/f42 [0,4194304] 0 2024-03-04T13:42:54.071 INFO:tasks.workunit.client.1.smithi169.stdout:5/568: dread d1/d16/d17/f4b [0,4194304] 0 2024-03-04T13:42:54.074 INFO:tasks.workunit.client.1.smithi169.stdout:6/531: dread d0/d3c/d95/fb3 [0,4194304] 0 2024-03-04T13:42:54.075 INFO:tasks.workunit.client.1.smithi169.stdout:0/429: creat d5/d9/dc/d12/d15/d19/d77/f9b x:0 0 0 2024-03-04T13:42:54.080 INFO:tasks.workunit.client.1.smithi169.stdout:6/532: fdatasync d0/d3c/d40/f4d 0 2024-03-04T13:42:54.081 INFO:tasks.workunit.client.1.smithi169.stdout:0/430: chown d5/d9/d3d/d59/d64/c96 2325 1 2024-03-04T13:42:54.087 INFO:tasks.workunit.client.1.smithi169.stdout:9/536: write d7/dc/d12/d32/d42/f7e [813315,55568] 0 2024-03-04T13:42:54.088 INFO:tasks.workunit.client.1.smithi169.stdout:1/471: dread d4/d5/d36/d35/d5b/f87 [0,4194304] 0 2024-03-04T13:42:54.090 INFO:tasks.workunit.client.1.smithi169.stdout:2/518: dwrite d1/d5/d19/d36/f4b [0,4194304] 0 2024-03-04T13:42:54.091 INFO:tasks.workunit.client.1.smithi169.stdout:3/551: dread d7/de/d2b/d7d/d85/d13/d1a/d2d/f37 [0,4194304] 0 2024-03-04T13:42:54.091 INFO:tasks.workunit.client.1.smithi169.stdout:9/537: readlink d7/dc/d12/d32/d42/l81 0 2024-03-04T13:42:54.094 INFO:tasks.workunit.client.1.smithi169.stdout:9/538: chown d7/d27/f2f 0 1 2024-03-04T13:42:54.098 INFO:tasks.workunit.client.0.smithi107.stderr:++ readlink -f fsstress 2024-03-04T13:42:54.098 INFO:tasks.workunit.client.1.smithi169.stdout:3/552: dread - d7/de/d2b/d7d/d85/d13/d5a/f80 zero size 2024-03-04T13:42:54.102 INFO:tasks.workunit.client.1.smithi169.stdout:3/553: chown d7/de/d2b/d7d/d85/d18/d21/d3f/d68/c94 508027327 1 2024-03-04T13:42:54.102 INFO:tasks.workunit.client.0.smithi107.stderr:+ BIN=/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-03-04T13:42:54.102 INFO:tasks.workunit.client.0.smithi107.stderr:+ popd 2024-03-04T13:42:54.103 INFO:tasks.workunit.client.1.smithi169.stdout:1/472: rmdir d4/d5/d2d/d4e/d71/d26/d4f 39 2024-03-04T13:42:54.106 INFO:tasks.workunit.client.0.smithi107.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-03-04T13:42:54.106 INFO:tasks.workunit.client.0.smithi107.stderr:+ popd 2024-03-04T13:42:54.107 INFO:tasks.workunit.client.1.smithi169.stdout:2/519: mknod d1/d5/d22/d3b/d99/cb9 0 2024-03-04T13:42:54.107 INFO:tasks.workunit.client.0.smithi107.stdout:~/cephtest/mnt.0/client.0/tmp 2024-03-04T13:42:54.108 INFO:tasks.workunit.client.0.smithi107.stderr:++ mktemp -d -p . 2024-03-04T13:42:54.112 INFO:tasks.workunit.client.1.smithi169.stdout:5/569: dread d1/d16/d17/d2d/d5a/d67/f97 [0,4194304] 0 2024-03-04T13:42:54.113 INFO:tasks.workunit.client.0.smithi107.stderr:+ T=./tmp.BZKaCoIoCT 2024-03-04T13:42:54.113 INFO:tasks.workunit.client.0.smithi107.stderr:+ /home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.BZKaCoIoCT -l 1 -n 1000 -p 10 -v 2024-03-04T13:42:54.114 INFO:tasks.workunit.client.1.smithi169.stdout:0/431: getdents d5/d9/dc/d12/d15/d19/d77/d94 0 2024-03-04T13:42:54.115 INFO:tasks.workunit.client.1.smithi169.stdout:3/554: truncate d7/de/d2b/d7d/d85/d18/d21/d3f/d68/fb2 555776 0 2024-03-04T13:42:54.120 INFO:tasks.workunit.client.1.smithi169.stdout:1/473: symlink d4/d5/d2d/d4e/d71/d26/l92 0 2024-03-04T13:42:54.123 INFO:tasks.workunit.client.1.smithi169.stdout:7/558: dwrite d2/de/d3e/d41/d69/fab [0,4194304] 0 2024-03-04T13:42:54.124 INFO:tasks.workunit.client.1.smithi169.stdout:2/520: link d1/d5/d19/d1e/d29/f39 d1/d5/d22/d3b/d41/fba 0 2024-03-04T13:42:54.128 INFO:tasks.workunit.client.1.smithi169.stdout:7/559: dread - d2/d28/d70/f87 zero size 2024-03-04T13:42:54.129 INFO:tasks.workunit.client.1.smithi169.stdout:6/533: dread d0/d3c/fbc [0,4194304] 0 2024-03-04T13:42:54.129 INFO:tasks.workunit.client.0.smithi107.stdout:seed = 1709689199 2024-03-04T13:42:54.130 INFO:tasks.workunit.client.1.smithi169.stdout:5/570: creat d1/d16/d2a/d62/fba x:0 0 0 2024-03-04T13:42:54.134 INFO:tasks.workunit.client.1.smithi169.stdout:3/555: mknod d7/de/d2b/d7d/d85/cbd 0 2024-03-04T13:42:54.137 INFO:tasks.workunit.client.1.smithi169.stdout:7/560: mkdir d2/dc0 0 2024-03-04T13:42:54.137 INFO:tasks.workunit.client.1.smithi169.stdout:1/474: getdents d4/d5/d36/d35/d16/d81 0 2024-03-04T13:42:54.142 INFO:tasks.workunit.client.1.smithi169.stdout:5/571: mkdir d1/d16/d2f/d38/dbb 0 2024-03-04T13:42:54.143 INFO:tasks.workunit.client.0.smithi107.stdout:0/0: truncate - no filename 2024-03-04T13:42:54.146 INFO:tasks.workunit.client.1.smithi169.stdout:9/539: dwrite d7/f7a [0,4194304] 0 2024-03-04T13:42:54.147 INFO:tasks.workunit.client.1.smithi169.stdout:2/521: dwrite d1/d6d/f87 [0,4194304] 0 2024-03-04T13:42:54.148 INFO:tasks.workunit.client.1.smithi169.stdout:6/534: mkdir d0/d32/d3a/d43/d74/d93/dbe 0 2024-03-04T13:42:54.150 INFO:tasks.workunit.client.0.smithi107.stdout:2/0: stat - no entries 2024-03-04T13:42:54.150 INFO:tasks.workunit.client.0.smithi107.stdout:2/1: rmdir - no directory 2024-03-04T13:42:54.151 INFO:tasks.workunit.client.0.smithi107.stdout:0/1: creat f0 x:0 0 0 2024-03-04T13:42:54.152 INFO:tasks.workunit.client.1.smithi169.stdout:5/572: unlink d1/f65 0 2024-03-04T13:42:54.152 INFO:tasks.workunit.client.0.smithi107.stdout:0/2: write f0 [460313,89390] 0 2024-03-04T13:42:54.153 INFO:tasks.workunit.client.0.smithi107.stdout:1/0: symlink l0 0 2024-03-04T13:42:54.154 INFO:tasks.workunit.client.1.smithi169.stdout:2/522: read - d1/d5/d19/f6a zero size 2024-03-04T13:42:54.157 INFO:tasks.workunit.client.0.smithi107.stdout:3/0: rmdir - no directory 2024-03-04T13:42:54.157 INFO:tasks.workunit.client.0.smithi107.stdout:3/1: chown . 229579 1 2024-03-04T13:42:54.157 INFO:tasks.workunit.client.0.smithi107.stdout:3/2: dread - no filename 2024-03-04T13:42:54.157 INFO:tasks.workunit.client.0.smithi107.stdout:3/3: write - no filename 2024-03-04T13:42:54.158 INFO:tasks.workunit.client.0.smithi107.stdout:3/4: dwrite - no filename 2024-03-04T13:42:54.158 INFO:tasks.workunit.client.0.smithi107.stdout:3/5: write - no filename 2024-03-04T13:42:54.160 INFO:tasks.workunit.client.1.smithi169.stdout:9/540: rmdir d7/d25/d47/d56 39 2024-03-04T13:42:54.162 INFO:tasks.workunit.client.0.smithi107.stdout:1/1: mkdir d1 0 2024-03-04T13:42:54.162 INFO:tasks.workunit.client.0.smithi107.stdout:1/2: dwrite - no filename 2024-03-04T13:42:54.163 INFO:tasks.workunit.client.0.smithi107.stdout:1/3: dwrite - no filename 2024-03-04T13:42:54.168 INFO:tasks.workunit.client.1.smithi169.stdout:9/541: readlink d7/dc/d12/d32/d42/l4d 0 2024-03-04T13:42:54.170 INFO:tasks.workunit.client.1.smithi169.stdout:7/561: dwrite d2/d15/d54/fac [0,4194304] 0 2024-03-04T13:42:54.170 INFO:tasks.workunit.client.1.smithi169.stdout:2/523: mkdir d1/d5/d22/d3b/d41/dbb 0 2024-03-04T13:42:54.170 INFO:tasks.workunit.client.1.smithi169.stdout:3/556: dwrite d7/de/d2b/d7d/d85/d18/d21/f9f [0,4194304] 0 2024-03-04T13:42:54.174 INFO:tasks.workunit.client.0.smithi107.stdout:2/2: creat f0 x:0 0 0 2024-03-04T13:42:54.176 INFO:tasks.workunit.client.1.smithi169.stdout:5/573: write d1/d16/d17/d1b/f25 [4639716,78866] 0 2024-03-04T13:42:54.176 INFO:tasks.workunit.client.1.smithi169.stdout:9/542: write d7/dc/d12/d20/f24 [5148515,107828] 0 2024-03-04T13:42:54.178 INFO:tasks.workunit.client.1.smithi169.stdout:5/574: fdatasync d1/f75 0 2024-03-04T13:42:54.182 INFO:tasks.workunit.client.0.smithi107.stdout:2/3: fdatasync f0 0 2024-03-04T13:42:54.183 INFO:tasks.workunit.client.1.smithi169.stdout:2/524: unlink d1/d5/f67 0 2024-03-04T13:42:54.183 INFO:tasks.workunit.client.0.smithi107.stdout:3/6: symlink l0 0 2024-03-04T13:42:54.183 INFO:tasks.workunit.client.0.smithi107.stdout:3/7: read - no filename 2024-03-04T13:42:54.185 INFO:tasks.workunit.client.1.smithi169.stdout:1/475: dwrite d4/d5/d36/f67 [0,4194304] 0 2024-03-04T13:42:54.185 INFO:tasks.workunit.client.1.smithi169.stdout:9/543: creat d7/dc/d12/d32/d42/d4f/fb5 x:0 0 0 2024-03-04T13:42:54.187 INFO:tasks.workunit.client.0.smithi107.stdout:5/0: rename - no filename 2024-03-04T13:42:54.190 INFO:tasks.workunit.client.0.smithi107.stdout:0/3: dread f0 [0,4194304] 0 2024-03-04T13:42:54.191 INFO:tasks.workunit.client.0.smithi107.stdout:3/8: mkdir d1 0 2024-03-04T13:42:54.191 INFO:tasks.workunit.client.1.smithi169.stdout:7/562: rename d2/fa to d2/d6b/fc1 0 2024-03-04T13:42:54.192 INFO:tasks.workunit.client.1.smithi169.stdout:2/525: mkdir d1/d6d/dbc 0 2024-03-04T13:42:54.194 INFO:tasks.workunit.client.0.smithi107.stdout:6/0: readlink - no filename 2024-03-04T13:42:54.194 INFO:tasks.workunit.client.0.smithi107.stdout:6/1: dwrite - no filename 2024-03-04T13:42:54.194 INFO:tasks.workunit.client.0.smithi107.stdout:6/2: write - no filename 2024-03-04T13:42:54.195 INFO:tasks.workunit.client.1.smithi169.stdout:1/476: mkdir d4/d5/d36/d35/d8f/d93 0 2024-03-04T13:42:54.197 INFO:tasks.workunit.client.0.smithi107.stdout:4/0: mkdir d0 0 2024-03-04T13:42:54.197 INFO:tasks.workunit.client.0.smithi107.stdout:4/1: readlink - no filename 2024-03-04T13:42:54.197 INFO:tasks.workunit.client.1.smithi169.stdout:2/526: creat d1/d5/d19/d63/fbd x:0 0 0 2024-03-04T13:42:54.198 INFO:tasks.workunit.client.0.smithi107.stdout:0/4: symlink l1 0 2024-03-04T13:42:54.200 INFO:tasks.workunit.client.1.smithi169.stdout:7/563: unlink d2/d15/d17/d6f/fae 0 2024-03-04T13:42:54.200 INFO:tasks.workunit.client.1.smithi169.stdout:2/527: chown d1/d5/d22/d3b/d41/c81 234846685 1 2024-03-04T13:42:54.200 INFO:tasks.workunit.client.1.smithi169.stdout:5/575: dwrite d1/d81/d91/fb9 [0,4194304] 0 2024-03-04T13:42:54.202 INFO:tasks.workunit.client.0.smithi107.stdout:5/1: creat f0 x:0 0 0 2024-03-04T13:42:54.203 INFO:tasks.workunit.client.1.smithi169.stdout:7/564: stat d2/de/d3e/d41/d69/d78 0 2024-03-04T13:42:54.205 INFO:tasks.workunit.client.0.smithi107.stdout:4/2: rename d0 to d0/d1 22 2024-03-04T13:42:54.206 INFO:tasks.workunit.client.0.smithi107.stdout:5/2: chown f0 443 1 2024-03-04T13:42:54.207 INFO:tasks.workunit.client.1.smithi169.stdout:7/565: write d2/d15/d17/f5d [193942,73585] 0 2024-03-04T13:42:54.207 INFO:tasks.workunit.client.0.smithi107.stdout:7/0: dwrite - no filename 2024-03-04T13:42:54.207 INFO:tasks.workunit.client.0.smithi107.stdout:7/1: write - no filename 2024-03-04T13:42:54.207 INFO:tasks.workunit.client.0.smithi107.stdout:7/2: rmdir - no directory 2024-03-04T13:42:54.207 INFO:tasks.workunit.client.0.smithi107.stdout:7/3: write - no filename 2024-03-04T13:42:54.207 INFO:tasks.workunit.client.0.smithi107.stdout:7/4: stat - no entries 2024-03-04T13:42:54.208 INFO:tasks.workunit.client.0.smithi107.stdout:7/5: truncate - no filename 2024-03-04T13:42:54.208 INFO:tasks.workunit.client.0.smithi107.stdout:7/6: dwrite - no filename 2024-03-04T13:42:54.208 INFO:tasks.workunit.client.0.smithi107.stdout:7/7: write - no filename 2024-03-04T13:42:54.208 INFO:tasks.workunit.client.0.smithi107.stdout:3/9: symlink d1/l2 0 2024-03-04T13:42:54.208 INFO:tasks.workunit.client.0.smithi107.stdout:3/10: fsync - no filename 2024-03-04T13:42:54.209 INFO:tasks.workunit.client.0.smithi107.stdout:3/11: dread - no filename 2024-03-04T13:42:54.209 INFO:tasks.workunit.client.0.smithi107.stdout:3/12: dread - no filename 2024-03-04T13:42:54.209 INFO:tasks.workunit.client.0.smithi107.stdout:4/3: chown d0 9 1 2024-03-04T13:42:54.209 INFO:tasks.workunit.client.0.smithi107.stdout:4/4: dread - no filename 2024-03-04T13:42:54.209 INFO:tasks.workunit.client.0.smithi107.stdout:4/5: readlink - no filename 2024-03-04T13:42:54.210 INFO:tasks.workunit.client.1.smithi169.stdout:7/566: write d2/d28/d70/f98 [329180,41434] 0 2024-03-04T13:42:54.211 INFO:tasks.workunit.client.0.smithi107.stdout:3/13: stat d1/l2 0 2024-03-04T13:42:54.215 INFO:tasks.workunit.client.0.smithi107.stdout:0/5: rename l1 to l2 0 2024-03-04T13:42:54.217 INFO:tasks.workunit.client.0.smithi107.stdout:6/3: creat f0 x:0 0 0 2024-03-04T13:42:54.218 INFO:tasks.workunit.client.1.smithi169.stdout:2/528: creat d1/d6d/dbc/fbe x:0 0 0 2024-03-04T13:42:54.220 INFO:tasks.workunit.client.0.smithi107.stdout:0/6: truncate f0 623344 0 2024-03-04T13:42:54.220 INFO:tasks.workunit.client.0.smithi107.stdout:2/4: dwrite f0 [0,4194304] 0 2024-03-04T13:42:54.221 INFO:tasks.workunit.client.0.smithi107.stdout:8/0: fsync - no filename 2024-03-04T13:42:54.221 INFO:tasks.workunit.client.0.smithi107.stdout:8/1: rename - no filename 2024-03-04T13:42:54.221 INFO:tasks.workunit.client.0.smithi107.stdout:8/2: write - no filename 2024-03-04T13:42:54.222 INFO:tasks.workunit.client.1.smithi169.stdout:1/477: dwrite d4/f25 [4194304,4194304] 0 2024-03-04T13:42:54.222 INFO:tasks.workunit.client.1.smithi169.stdout:2/529: write d1/d5/d19/d1e/d29/f3e [694911,71004] 0 2024-03-04T13:42:54.223 INFO:tasks.workunit.client.1.smithi169.stdout:4/433: sync 2024-03-04T13:42:54.225 INFO:tasks.workunit.client.1.smithi169.stdout:2/530: chown d1/d5/d19/d1e/f5c 0 1 2024-03-04T13:42:54.226 INFO:tasks.workunit.client.0.smithi107.stdout:5/3: creat f1 x:0 0 0 2024-03-04T13:42:54.227 INFO:tasks.workunit.client.0.smithi107.stdout:5/4: dread - f1 zero size 2024-03-04T13:42:54.228 INFO:tasks.workunit.client.1.smithi169.stdout:7/567: read d2/fd [3395129,33667] 0 2024-03-04T13:42:54.229 INFO:tasks.workunit.client.0.smithi107.stdout:3/14: creat d1/f3 x:0 0 0 2024-03-04T13:42:54.230 INFO:tasks.workunit.client.0.smithi107.stdout:6/4: creat f1 x:0 0 0 2024-03-04T13:42:54.230 INFO:tasks.workunit.client.1.smithi169.stdout:5/576: link d1/d16/d2f/d72/f77 d1/d16/d2a/d46/fbc 0 2024-03-04T13:42:54.232 INFO:tasks.workunit.client.0.smithi107.stdout:6/5: write f0 [860637,39598] 0 2024-03-04T13:42:54.233 INFO:tasks.workunit.client.0.smithi107.stdout:9/0: dread - no filename 2024-03-04T13:42:54.233 INFO:tasks.workunit.client.0.smithi107.stdout:9/1: truncate - no filename 2024-03-04T13:42:54.233 INFO:tasks.workunit.client.0.smithi107.stdout:9/2: link - no file 2024-03-04T13:42:54.234 INFO:tasks.workunit.client.1.smithi169.stdout:5/577: fdatasync d1/d16/d17/d2d/d5a/d67/f82 0 2024-03-04T13:42:54.234 INFO:tasks.workunit.client.0.smithi107.stdout:9/3: chown . 66610 1 2024-03-04T13:42:54.235 INFO:tasks.workunit.client.0.smithi107.stdout:0/7: mknod c3 0 2024-03-04T13:42:54.235 INFO:tasks.workunit.client.1.smithi169.stdout:5/578: readlink d1/lf 0 2024-03-04T13:42:54.237 INFO:tasks.workunit.client.1.smithi169.stdout:2/531: chown d1/d5/d19/d1e/f43 64353 1 2024-03-04T13:42:54.237 INFO:tasks.workunit.client.0.smithi107.stdout:0/8: read f0 [251095,45224] 0 2024-03-04T13:42:54.237 INFO:tasks.workunit.client.0.smithi107.stdout:0/9: rmdir - no directory 2024-03-04T13:42:54.238 INFO:tasks.workunit.client.1.smithi169.stdout:4/434: rename d4/d11/d3b/l4d to d4/d11/d2e/d37/d5c/l8e 0 2024-03-04T13:42:54.239 INFO:tasks.workunit.client.0.smithi107.stdout:2/5: creat f1 x:0 0 0 2024-03-04T13:42:54.244 INFO:tasks.workunit.client.0.smithi107.stdout:7/8: creat f0 x:0 0 0 2024-03-04T13:42:54.245 INFO:tasks.workunit.client.0.smithi107.stdout:7/9: fdatasync f0 0 2024-03-04T13:42:54.246 INFO:tasks.workunit.client.0.smithi107.stdout:4/6: creat d0/f2 x:0 0 0 2024-03-04T13:42:54.247 INFO:tasks.workunit.client.0.smithi107.stdout:4/7: chown d0/f2 12984158 1 2024-03-04T13:42:54.248 INFO:tasks.workunit.client.0.smithi107.stdout:4/8: dread - d0/f2 zero size 2024-03-04T13:42:54.248 INFO:tasks.workunit.client.0.smithi107.stdout:5/5: symlink l2 0 2024-03-04T13:42:54.249 INFO:tasks.workunit.client.1.smithi169.stdout:1/478: dwrite d4/d5/d36/d35/df/f1c [0,4194304] 0 2024-03-04T13:42:54.250 INFO:tasks.workunit.client.0.smithi107.stdout:5/6: chown f0 198018873 1 2024-03-04T13:42:54.251 INFO:tasks.workunit.client.0.smithi107.stdout:5/7: dread - f1 zero size 2024-03-04T13:42:54.251 INFO:tasks.workunit.client.0.smithi107.stdout:3/15: symlink d1/l4 0 2024-03-04T13:42:54.252 INFO:tasks.workunit.client.0.smithi107.stdout:3/16: rename d1 to d1/d5 22 2024-03-04T13:42:54.254 INFO:tasks.workunit.client.1.smithi169.stdout:4/435: mkdir d4/d11/d2e/d37/d5c/d86/d8f 0 2024-03-04T13:42:54.255 INFO:tasks.workunit.client.0.smithi107.stdout:3/17: write d1/f3 [103844,60314] 0 2024-03-04T13:42:54.255 INFO:tasks.workunit.client.0.smithi107.stdout:6/6: mknod c2 0 2024-03-04T13:42:54.256 INFO:tasks.workunit.client.1.smithi169.stdout:4/436: chown d4/l88 25608 1 2024-03-04T13:42:54.259 INFO:tasks.workunit.client.0.smithi107.stdout:3/18: fdatasync d1/f3 0 2024-03-04T13:42:54.260 INFO:tasks.workunit.client.1.smithi169.stdout:1/479: mknod d4/d5/d36/d35/d70/d78/c94 0 2024-03-04T13:42:54.260 INFO:tasks.workunit.client.1.smithi169.stdout:9/544: dread d7/dc/d12/d59/f6d [0,4194304] 0 2024-03-04T13:42:54.262 INFO:tasks.workunit.client.1.smithi169.stdout:2/532: dwrite d1/d5/d19/d68/fa7 [0,4194304] 0 2024-03-04T13:42:54.264 INFO:tasks.workunit.client.1.smithi169.stdout:5/579: rename d1/d16/d2f/d38 to d1/d16/dbd 0 2024-03-04T13:42:54.265 INFO:tasks.workunit.client.1.smithi169.stdout:4/437: rmdir d4/d11/d2e 39 2024-03-04T13:42:54.265 INFO:tasks.workunit.client.1.smithi169.stdout:5/580: stat d1/d16/d17/l48 0 2024-03-04T13:42:54.266 INFO:tasks.workunit.client.1.smithi169.stdout:5/581: stat d1/f75 0 2024-03-04T13:42:54.267 INFO:tasks.workunit.client.0.smithi107.stdout:7/10: mknod c1 0 2024-03-04T13:42:54.268 INFO:tasks.workunit.client.0.smithi107.stdout:8/3: symlink l0 0 2024-03-04T13:42:54.268 INFO:tasks.workunit.client.0.smithi107.stdout:7/11: dread - f0 zero size 2024-03-04T13:42:54.271 INFO:tasks.workunit.client.0.smithi107.stdout:4/9: rename d0/f2 to d0/f3 0 2024-03-04T13:42:54.271 INFO:tasks.workunit.client.0.smithi107.stdout:2/6: creat f2 x:0 0 0 2024-03-04T13:42:54.272 INFO:tasks.workunit.client.0.smithi107.stdout:5/8: symlink l3 0 2024-03-04T13:42:54.273 INFO:tasks.workunit.client.0.smithi107.stdout:5/9: readlink l3 0 2024-03-04T13:42:54.274 INFO:tasks.workunit.client.1.smithi169.stdout:9/545: creat d7/dc/d95/fb6 x:0 0 0 2024-03-04T13:42:54.274 INFO:tasks.workunit.client.0.smithi107.stdout:2/7: dread - f2 zero size 2024-03-04T13:42:54.275 INFO:tasks.workunit.client.0.smithi107.stdout:7/12: truncate f0 230298 0 2024-03-04T13:42:54.275 INFO:tasks.workunit.client.0.smithi107.stdout:6/7: rename f1 to f3 0 2024-03-04T13:42:54.276 INFO:tasks.workunit.client.1.smithi169.stdout:5/582: creat d1/d76/fbe x:0 0 0 2024-03-04T13:42:54.277 INFO:tasks.workunit.client.0.smithi107.stdout:7/13: stat f0 0 2024-03-04T13:42:54.278 INFO:tasks.workunit.client.1.smithi169.stdout:5/583: chown d1/d60/fae 123335 1 2024-03-04T13:42:54.281 INFO:tasks.workunit.client.1.smithi169.stdout:5/584: chown d1/d16/d17/d1b/d74/l88 237940600 1 2024-03-04T13:42:54.281 INFO:tasks.workunit.client.0.smithi107.stdout:8/4: rename l0 to l1 0 2024-03-04T13:42:54.282 INFO:tasks.workunit.client.0.smithi107.stdout:3/19: creat d1/f6 x:0 0 0 2024-03-04T13:42:54.285 INFO:tasks.workunit.client.0.smithi107.stdout:3/20: write d1/f6 [660448,46536] 0 2024-03-04T13:42:54.286 INFO:tasks.workunit.client.0.smithi107.stdout:4/10: symlink d0/l4 0 2024-03-04T13:42:54.287 INFO:tasks.workunit.client.1.smithi169.stdout:7/568: dread d2/fd [0,4194304] 0 2024-03-04T13:42:54.289 INFO:tasks.workunit.client.0.smithi107.stdout:2/8: mknod c3 0 2024-03-04T13:42:54.290 INFO:tasks.workunit.client.1.smithi169.stdout:5/585: creat d1/d16/d17/d2d/d51/fbf x:0 0 0 2024-03-04T13:42:54.291 INFO:tasks.workunit.client.0.smithi107.stdout:3/21: write d1/f3 [594881,113543] 0 2024-03-04T13:42:54.293 INFO:tasks.workunit.client.0.smithi107.stdout:6/8: creat f4 x:0 0 0 2024-03-04T13:42:54.293 INFO:tasks.workunit.client.1.smithi169.stdout:5/586: dread - d1/d16/d17/d1b/d74/fac zero size 2024-03-04T13:42:54.294 INFO:tasks.workunit.client.0.smithi107.stdout:7/14: symlink l2 0 2024-03-04T13:42:54.294 INFO:tasks.workunit.client.0.smithi107.stdout:7/15: rmdir - no directory 2024-03-04T13:42:54.294 INFO:tasks.workunit.client.1.smithi169.stdout:1/480: link d4/d5/d2d/d4e/d71/l62 d4/l95 0 2024-03-04T13:42:54.295 INFO:tasks.workunit.client.1.smithi169.stdout:1/481: chown d4/d5/d36/d35/l3d 14120625 1 2024-03-04T13:42:54.296 INFO:tasks.workunit.client.1.smithi169.stdout:5/587: write d1/d16/d17/d2d/d51/f6e [807422,78780] 0 2024-03-04T13:42:54.298 INFO:tasks.workunit.client.0.smithi107.stdout:0/10: dwrite f0 [0,4194304] 0 2024-03-04T13:42:54.311 INFO:tasks.workunit.client.0.smithi107.stdout:6/9: symlink l5 0 2024-03-04T13:42:54.314 INFO:tasks.workunit.client.0.smithi107.stdout:7/16: rename l2 to l3 0 2024-03-04T13:42:54.316 INFO:tasks.workunit.client.1.smithi169.stdout:5/588: dread d1/d16/d2a/f3e [0,4194304] 0 2024-03-04T13:42:54.319 INFO:tasks.workunit.client.0.smithi107.stdout:2/9: link f1 f4 0 2024-03-04T13:42:54.319 INFO:tasks.workunit.client.0.smithi107.stdout:0/11: dread f0 [0,4194304] 0 2024-03-04T13:42:54.320 INFO:tasks.workunit.client.0.smithi107.stdout:6/10: creat f6 x:0 0 0 2024-03-04T13:42:54.322 INFO:tasks.workunit.client.0.smithi107.stdout:2/10: write f2 [447259,98378] 0 2024-03-04T13:42:54.325 INFO:tasks.workunit.client.0.smithi107.stdout:2/11: write f0 [1337002,35862] 0 2024-03-04T13:42:54.325 INFO:tasks.workunit.client.1.smithi169.stdout:2/533: dwrite d1/d5/d22/d45/f62 [0,4194304] 0 2024-03-04T13:42:54.328 INFO:tasks.workunit.client.0.smithi107.stdout:4/11: getdents d0 0 2024-03-04T13:42:54.329 INFO:tasks.workunit.client.0.smithi107.stdout:7/17: link f0 f4 0 2024-03-04T13:42:54.330 INFO:tasks.workunit.client.0.smithi107.stdout:4/12: dread - d0/f3 zero size 2024-03-04T13:42:54.331 INFO:tasks.workunit.client.0.smithi107.stdout:6/11: mknod c7 0 2024-03-04T13:42:54.331 INFO:tasks.workunit.client.1.smithi169.stdout:4/438: dread d4/d11/d2e/d21/d5e/f76 [0,4194304] 0 2024-03-04T13:42:54.332 INFO:tasks.workunit.client.0.smithi107.stdout:6/12: stat f0 0 2024-03-04T13:42:54.335 INFO:tasks.workunit.client.0.smithi107.stdout:0/12: dread f0 [0,4194304] 0 2024-03-04T13:42:54.335 INFO:tasks.workunit.client.0.smithi107.stdout:8/5: chown l1 145 1 2024-03-04T13:42:54.336 INFO:tasks.workunit.client.0.smithi107.stdout:8/6: chown l1 577869 1 2024-03-04T13:42:54.336 INFO:tasks.workunit.client.0.smithi107.stdout:8/7: dwrite - no filename 2024-03-04T13:42:54.336 INFO:tasks.workunit.client.0.smithi107.stdout:8/8: read - no filename 2024-03-04T13:42:54.336 INFO:tasks.workunit.client.0.smithi107.stdout:8/9: rmdir - no directory 2024-03-04T13:42:54.338 INFO:tasks.workunit.client.1.smithi169.stdout:5/589: creat d1/fc0 x:0 0 0 2024-03-04T13:42:54.339 INFO:tasks.workunit.client.1.smithi169.stdout:5/590: readlink d1/d23/l37 0 2024-03-04T13:42:54.342 INFO:tasks.workunit.client.0.smithi107.stdout:4/13: mknod d0/c5 0 2024-03-04T13:42:54.344 INFO:tasks.workunit.client.1.smithi169.stdout:7/569: truncate d2/de/d3e/f62 335890 0 2024-03-04T13:42:54.346 INFO:tasks.workunit.client.1.smithi169.stdout:1/482: write d4/d5/d2d/d4e/d71/f37 [1561718,21690] 0 2024-03-04T13:42:54.346 INFO:tasks.workunit.client.0.smithi107.stdout:0/13: readlink l2 0 2024-03-04T13:42:54.347 INFO:tasks.workunit.client.0.smithi107.stdout:0/14: rmdir - no directory 2024-03-04T13:42:54.347 INFO:tasks.workunit.client.1.smithi169.stdout:5/591: mkdir d1/d81/dc1 0 2024-03-04T13:42:54.348 INFO:tasks.workunit.client.0.smithi107.stdout:8/10: creat f2 x:0 0 0 2024-03-04T13:42:54.351 INFO:tasks.workunit.client.0.smithi107.stdout:8/11: dread - f2 zero size 2024-03-04T13:42:54.351 INFO:tasks.workunit.client.0.smithi107.stdout:6/13: link f0 f8 0 2024-03-04T13:42:54.352 INFO:tasks.workunit.client.1.smithi169.stdout:7/570: read d2/d15/d17/d6f/f72 [555380,86706] 0 2024-03-04T13:42:54.354 INFO:tasks.workunit.client.1.smithi169.stdout:7/571: chown d2/de/d3e/c7b 168594 1 2024-03-04T13:42:54.354 INFO:tasks.workunit.client.0.smithi107.stdout:4/14: creat d0/f6 x:0 0 0 2024-03-04T13:42:54.355 INFO:tasks.workunit.client.1.smithi169.stdout:2/534: getdents d1/d5/d22/d3b 0 2024-03-04T13:42:54.356 INFO:tasks.workunit.client.1.smithi169.stdout:7/572: fdatasync d2/d28/f4b 0 2024-03-04T13:42:54.356 INFO:tasks.workunit.client.0.smithi107.stdout:0/15: creat f4 x:0 0 0 2024-03-04T13:42:54.357 INFO:tasks.workunit.client.0.smithi107.stdout:4/15: chown d0/c5 5659 1 2024-03-04T13:42:54.357 INFO:tasks.workunit.client.0.smithi107.stdout:0/16: truncate f4 404716 0 2024-03-04T13:42:54.358 INFO:tasks.workunit.client.0.smithi107.stdout:4/16: chown d0/f6 6187628 1 2024-03-04T13:42:54.358 INFO:tasks.workunit.client.1.smithi169.stdout:2/535: write d1/d5/d22/d45/f90 [255954,131049] 0 2024-03-04T13:42:54.359 INFO:tasks.workunit.client.0.smithi107.stdout:8/12: creat f3 x:0 0 0 2024-03-04T13:42:54.360 INFO:tasks.workunit.client.0.smithi107.stdout:8/13: read - f3 zero size 2024-03-04T13:42:54.361 INFO:tasks.workunit.client.0.smithi107.stdout:6/14: mknod c9 0 2024-03-04T13:42:54.362 INFO:tasks.workunit.client.1.smithi169.stdout:1/483: creat d4/d5/d36/d35/d16/d81/f96 x:0 0 0 2024-03-04T13:42:54.363 INFO:tasks.workunit.client.0.smithi107.stdout:8/14: write f3 [133769,107603] 0 2024-03-04T13:42:54.364 INFO:tasks.workunit.client.1.smithi169.stdout:1/484: write d4/f6b [1571759,16613] 0 2024-03-04T13:42:54.366 INFO:tasks.workunit.client.0.smithi107.stdout:0/17: symlink l5 0 2024-03-04T13:42:54.367 INFO:tasks.workunit.client.1.smithi169.stdout:4/439: truncate d4/f5a 1270117 0 2024-03-04T13:42:54.367 INFO:tasks.workunit.client.1.smithi169.stdout:1/485: stat d4/d5/d36/d35/l6a 0 2024-03-04T13:42:54.373 INFO:tasks.workunit.client.1.smithi169.stdout:5/592: dwrite d1/d16/d17/f4b [0,4194304] 0 2024-03-04T13:42:54.374 INFO:tasks.workunit.client.0.smithi107.stdout:6/15: creat fa x:0 0 0 2024-03-04T13:42:54.374 INFO:tasks.workunit.client.1.smithi169.stdout:5/593: dread - d1/f75 zero size 2024-03-04T13:42:54.375 INFO:tasks.workunit.client.0.smithi107.stdout:0/18: write f4 [1156166,66802] 0 2024-03-04T13:42:54.376 INFO:tasks.workunit.client.1.smithi169.stdout:5/594: fsync d1/d16/dbd/fa9 0 2024-03-04T13:42:54.376 INFO:tasks.workunit.client.1.smithi169.stdout:7/573: rename d2/d6b/fc1 to d2/de/d3e/d41/d69/fc2 0 2024-03-04T13:42:54.376 INFO:tasks.workunit.client.0.smithi107.stdout:8/15: creat f4 x:0 0 0 2024-03-04T13:42:54.376 INFO:tasks.workunit.client.0.smithi107.stdout:0/19: rmdir - no directory 2024-03-04T13:42:54.385 INFO:tasks.workunit.client.1.smithi169.stdout:7/574: write d2/de/d3e/d41/d69/f88 [900299,11377] 0 2024-03-04T13:42:54.389 INFO:tasks.workunit.client.0.smithi107.stdout:7/18: dwrite f4 [0,4194304] 0 2024-03-04T13:42:54.389 INFO:tasks.workunit.client.0.smithi107.stdout:8/16: read f3 [92959,77124] 0 2024-03-04T13:42:54.389 INFO:tasks.workunit.client.1.smithi169.stdout:2/536: write d1/d5/d19/d1e/d29/f39 [4341961,24070] 0 2024-03-04T13:42:54.391 INFO:tasks.workunit.client.1.smithi169.stdout:5/595: mknod d1/d16/d2a/d46/d9e/cc2 0 2024-03-04T13:42:54.394 INFO:tasks.workunit.client.0.smithi107.stdout:2/12: dwrite f1 [0,4194304] 0 2024-03-04T13:42:54.398 INFO:tasks.workunit.client.1.smithi169.stdout:2/537: symlink d1/d5/d22/d45/d4a/lbf 0 2024-03-04T13:42:54.400 INFO:tasks.workunit.client.1.smithi169.stdout:5/596: truncate d1/d16/d17/f18 4226274 0 2024-03-04T13:42:54.401 INFO:tasks.workunit.client.1.smithi169.stdout:1/486: getdents d4/d5/d36/d35/d8f 0 2024-03-04T13:42:54.402 INFO:tasks.workunit.client.0.smithi107.stdout:7/19: creat f5 x:0 0 0 2024-03-04T13:42:54.404 INFO:tasks.workunit.client.0.smithi107.stdout:2/13: rename c3 to c5 0 2024-03-04T13:42:54.405 INFO:tasks.workunit.client.1.smithi169.stdout:1/487: fsync d4/d5/d36/f88 0 2024-03-04T13:42:54.408 INFO:tasks.workunit.client.1.smithi169.stdout:2/538: readlink d1/d5/d22/l44 0 2024-03-04T13:42:54.409 INFO:tasks.workunit.client.1.smithi169.stdout:7/575: read d2/d15/d17/f42 [228311,129969] 0 2024-03-04T13:42:54.412 INFO:tasks.workunit.client.0.smithi107.stdout:2/14: creat f6 x:0 0 0 2024-03-04T13:42:54.414 INFO:tasks.workunit.client.0.smithi107.stdout:8/17: link f4 f5 0 2024-03-04T13:42:54.414 INFO:tasks.workunit.client.0.smithi107.stdout:8/18: rmdir - no directory 2024-03-04T13:42:54.415 INFO:tasks.workunit.client.0.smithi107.stdout:8/19: stat f5 0 2024-03-04T13:42:54.415 INFO:tasks.workunit.client.1.smithi169.stdout:1/488: creat d4/d5/d36/d35/d8f/f97 x:0 0 0 2024-03-04T13:42:54.416 INFO:tasks.workunit.client.0.smithi107.stdout:6/16: dread f8 [0,4194304] 0 2024-03-04T13:42:54.416 INFO:tasks.workunit.client.1.smithi169.stdout:4/440: write d4/f5a [605484,8249] 0 2024-03-04T13:42:54.419 INFO:tasks.workunit.client.1.smithi169.stdout:4/441: readlink d4/d1c/d58/l7b 0 2024-03-04T13:42:54.421 INFO:tasks.workunit.client.0.smithi107.stdout:2/15: write f4 [3923002,99306] 0 2024-03-04T13:42:54.432 INFO:tasks.workunit.client.1.smithi169.stdout:2/539: dwrite d1/d5/d19/d36/f4b [0,4194304] 0 2024-03-04T13:42:54.433 INFO:tasks.workunit.client.1.smithi169.stdout:1/489: creat d4/d5/d2d/d4e/d71/d26/f98 x:0 0 0 2024-03-04T13:42:54.433 INFO:tasks.workunit.client.1.smithi169.stdout:7/576: dwrite d2/d28/fa1 [0,4194304] 0 2024-03-04T13:42:54.433 INFO:tasks.workunit.client.0.smithi107.stdout:6/17: write f4 [720601,54269] 0 2024-03-04T13:42:54.433 INFO:tasks.workunit.client.0.smithi107.stdout:0/20: dwrite f4 [0,4194304] 0 2024-03-04T13:42:54.436 INFO:tasks.workunit.client.0.smithi107.stdout:0/21: chown f4 10342 1 2024-03-04T13:42:54.439 INFO:tasks.workunit.client.0.smithi107.stdout:7/20: getdents . 0 2024-03-04T13:42:54.443 INFO:tasks.workunit.client.0.smithi107.stdout:6/18: rename f4 to fb 0 2024-03-04T13:42:54.447 INFO:tasks.workunit.client.0.smithi107.stdout:7/21: write f0 [4415129,62269] 0 2024-03-04T13:42:54.449 INFO:tasks.workunit.client.0.smithi107.stdout:7/22: readlink l3 0 2024-03-04T13:42:54.450 INFO:tasks.workunit.client.1.smithi169.stdout:4/442: dwrite d4/d11/d2e/d63/f8a [0,4194304] 0 2024-03-04T13:42:54.451 INFO:tasks.workunit.client.0.smithi107.stdout:7/23: dread - f5 zero size 2024-03-04T13:42:54.452 INFO:tasks.workunit.client.1.smithi169.stdout:5/597: dread d1/d16/d17/d28/f4d [0,4194304] 0 2024-03-04T13:42:54.454 INFO:tasks.workunit.client.1.smithi169.stdout:4/443: fsync d4/d11/d23/f46 0 2024-03-04T13:42:54.454 INFO:tasks.workunit.client.1.smithi169.stdout:7/577: creat d2/de/d3e/d41/fc3 x:0 0 0 2024-03-04T13:42:54.455 INFO:tasks.workunit.client.0.smithi107.stdout:7/24: fdatasync f5 0 2024-03-04T13:42:54.460 INFO:tasks.workunit.client.1.smithi169.stdout:5/598: fdatasync d1/d16/d17/d2d/d51/fa8 0 2024-03-04T13:42:54.460 INFO:tasks.workunit.client.1.smithi169.stdout:2/540: creat d1/fc0 x:0 0 0 2024-03-04T13:42:54.460 INFO:tasks.workunit.client.0.smithi107.stdout:7/25: chown f5 0 1 2024-03-04T13:42:54.461 INFO:tasks.workunit.client.0.smithi107.stdout:0/22: link l5 l6 0 2024-03-04T13:42:54.461 INFO:tasks.workunit.client.1.smithi169.stdout:1/490: link d4/l2c d4/d5/d36/d75/l99 0 2024-03-04T13:42:54.464 INFO:tasks.workunit.client.0.smithi107.stdout:8/20: dwrite f5 [0,4194304] 0 2024-03-04T13:42:54.465 INFO:tasks.workunit.client.0.smithi107.stdout:2/16: dwrite f2 [0,4194304] 0 2024-03-04T13:42:54.465 INFO:tasks.workunit.client.0.smithi107.stdout:7/26: read - f5 zero size 2024-03-04T13:42:54.466 INFO:tasks.workunit.client.1.smithi169.stdout:5/599: dread d1/d16/f20 [4194304,4194304] 0 2024-03-04T13:42:54.467 INFO:tasks.workunit.client.1.smithi169.stdout:2/541: symlink d1/d5/d19/d3c/lc1 0 2024-03-04T13:42:54.467 INFO:tasks.workunit.client.1.smithi169.stdout:7/578: creat d2/de/d3e/d41/d89/fc4 x:0 0 0 2024-03-04T13:42:54.468 INFO:tasks.workunit.client.1.smithi169.stdout:1/491: creat d4/d5/d36/d35/d8f/f9a x:0 0 0 2024-03-04T13:42:54.470 INFO:tasks.workunit.client.1.smithi169.stdout:5/600: read d1/d81/d91/f94 [1247082,79434] 0 2024-03-04T13:42:54.470 INFO:tasks.workunit.client.0.smithi107.stdout:7/27: mkdir d6 0 2024-03-04T13:42:54.471 INFO:tasks.workunit.client.1.smithi169.stdout:5/601: chown d1/d16/d2a 1139877 1 2024-03-04T13:42:54.474 INFO:tasks.workunit.client.0.smithi107.stdout:8/21: mkdir d6 0 2024-03-04T13:42:54.478 INFO:tasks.workunit.client.1.smithi169.stdout:5/602: rename d1/lc to d1/d16/d17/d2d/lc3 0 2024-03-04T13:42:54.479 INFO:tasks.workunit.client.0.smithi107.stdout:7/28: read f0 [2171371,23697] 0 2024-03-04T13:42:54.481 INFO:tasks.workunit.client.1.smithi169.stdout:1/492: unlink d4/d5/d36/d35/d5b/f5c 0 2024-03-04T13:42:54.484 INFO:tasks.workunit.client.1.smithi169.stdout:2/542: dwrite d1/d5/d22/d45/d4a/f92 [0,4194304] 0 2024-03-04T13:42:54.490 INFO:tasks.workunit.client.0.smithi107.stdout:6/19: getdents . 0 2024-03-04T13:42:54.494 INFO:tasks.workunit.client.0.smithi107.stdout:8/22: symlink d6/l7 0 2024-03-04T13:42:54.495 INFO:tasks.workunit.client.1.smithi169.stdout:1/493: link d4/d5/d2d/d4e/f84 d4/d5/d36/d35/d8f/d93/f9b 0 2024-03-04T13:42:54.498 INFO:tasks.workunit.client.0.smithi107.stdout:7/29: creat d6/f7 x:0 0 0 2024-03-04T13:42:54.498 INFO:tasks.workunit.client.0.smithi107.stdout:8/23: chown l1 27 1 2024-03-04T13:42:54.501 INFO:tasks.workunit.client.1.smithi169.stdout:5/603: dwrite d1/d60/fa0 [0,4194304] 0 2024-03-04T13:42:54.503 INFO:tasks.workunit.client.0.smithi107.stdout:6/20: dread fb [0,4194304] 0 2024-03-04T13:42:54.503 INFO:tasks.workunit.client.0.smithi107.stdout:2/17: dwrite f2 [0,4194304] 0 2024-03-04T13:42:54.504 INFO:tasks.workunit.client.0.smithi107.stdout:0/23: truncate f0 962428 0 2024-03-04T13:42:54.506 INFO:tasks.workunit.client.1.smithi169.stdout:1/494: truncate d4/d5/d2d/f6c 2675237 0 2024-03-04T13:42:54.508 INFO:tasks.workunit.client.0.smithi107.stdout:7/30: mknod d6/c8 0 2024-03-04T13:42:54.510 INFO:tasks.workunit.client.0.smithi107.stdout:8/24: creat d6/f8 x:0 0 0 2024-03-04T13:42:54.512 INFO:tasks.workunit.client.0.smithi107.stdout:7/31: truncate f5 374458 0 2024-03-04T13:42:54.513 INFO:tasks.workunit.client.0.smithi107.stdout:8/25: truncate f3 692005 0 2024-03-04T13:42:54.517 INFO:tasks.workunit.client.1.smithi169.stdout:5/604: unlink d1/d16/d2a/f70 0 2024-03-04T13:42:54.518 INFO:tasks.workunit.client.0.smithi107.stdout:0/24: creat f7 x:0 0 0 2024-03-04T13:42:54.520 INFO:tasks.workunit.client.0.smithi107.stdout:8/26: symlink d6/l9 0 2024-03-04T13:42:54.521 INFO:tasks.workunit.client.0.smithi107.stdout:0/25: creat f8 x:0 0 0 2024-03-04T13:42:54.522 INFO:tasks.workunit.client.1.smithi169.stdout:7/579: dread d2/de/f53 [0,4194304] 0 2024-03-04T13:42:54.524 INFO:tasks.workunit.client.0.smithi107.stdout:8/27: dread - f2 zero size 2024-03-04T13:42:54.525 INFO:tasks.workunit.client.1.smithi169.stdout:5/605: getdents d1/d23 0 2024-03-04T13:42:54.528 INFO:tasks.workunit.client.0.smithi107.stdout:8/28: mkdir d6/da 0 2024-03-04T13:42:54.528 INFO:tasks.workunit.client.0.smithi107.stdout:0/26: link l2 l9 0 2024-03-04T13:42:54.529 INFO:tasks.workunit.client.0.smithi107.stdout:8/29: rename d6/da to d6/da/db 22 2024-03-04T13:42:54.530 INFO:tasks.workunit.client.1.smithi169.stdout:5/606: rename d1/d16/d17/d1b/d74/c90 to d1/d16/d2f/d39/cc4 0 2024-03-04T13:42:54.531 INFO:tasks.workunit.client.1.smithi169.stdout:5/607: chown d1/d16/d17/d2d/d5a/d8a 2577 1 2024-03-04T13:42:54.532 INFO:tasks.workunit.client.1.smithi169.stdout:5/608: write d1/d16/d17/d2d/d51/fa8 [47684,125411] 0 2024-03-04T13:42:54.535 INFO:tasks.workunit.client.0.smithi107.stdout:2/18: dread f2 [0,4194304] 0 2024-03-04T13:42:54.535 INFO:tasks.workunit.client.1.smithi169.stdout:5/609: rmdir d1/d16/d17/d2d/d5a/d67 39 2024-03-04T13:42:54.538 INFO:tasks.workunit.client.1.smithi169.stdout:5/610: symlink d1/lc5 0 2024-03-04T13:42:54.541 INFO:tasks.workunit.client.1.smithi169.stdout:5/611: read - d1/d16/d17/d2d/d5a/d8a/fa5 zero size 2024-03-04T13:42:54.544 INFO:tasks.workunit.client.0.smithi107.stdout:7/32: rmdir d6 39 2024-03-04T13:42:54.544 INFO:tasks.workunit.client.1.smithi169.stdout:2/543: dread d1/d5/d19/f2e [0,4194304] 0 2024-03-04T13:42:54.545 INFO:tasks.workunit.client.1.smithi169.stdout:5/612: creat d1/d16/d17/d1b/d74/fc6 x:0 0 0 2024-03-04T13:42:54.550 INFO:tasks.workunit.client.1.smithi169.stdout:5/613: symlink d1/d16/d2a/d46/lc7 0 2024-03-04T13:42:54.553 INFO:tasks.workunit.client.1.smithi169.stdout:1/495: dwrite d4/d5/d36/d35/df/f28 [0,4194304] 0 2024-03-04T13:42:54.554 INFO:tasks.workunit.client.0.smithi107.stdout:6/21: dwrite fb [0,4194304] 0 2024-03-04T13:42:54.556 INFO:tasks.workunit.client.0.smithi107.stdout:7/33: mknod d6/c9 0 2024-03-04T13:42:54.559 INFO:tasks.workunit.client.0.smithi107.stdout:8/30: dwrite f5 [0,4194304] 0 2024-03-04T13:42:54.560 INFO:tasks.workunit.client.1.smithi169.stdout:2/544: write d1/d5/fe [3673780,28552] 0 2024-03-04T13:42:54.561 INFO:tasks.workunit.client.0.smithi107.stdout:7/34: read f5 [355287,71770] 0 2024-03-04T13:42:54.563 INFO:tasks.workunit.client.0.smithi107.stdout:0/27: dwrite f8 [0,4194304] 0 2024-03-04T13:42:54.564 INFO:tasks.workunit.client.0.smithi107.stdout:6/22: creat fc x:0 0 0 2024-03-04T13:42:54.565 INFO:tasks.workunit.client.0.smithi107.stdout:8/31: write f4 [48981,99020] 0 2024-03-04T13:42:54.568 INFO:tasks.workunit.client.0.smithi107.stdout:7/35: chown l3 0 1 2024-03-04T13:42:54.568 INFO:tasks.workunit.client.0.smithi107.stdout:6/23: fsync f0 0 2024-03-04T13:42:54.569 INFO:tasks.workunit.client.1.smithi169.stdout:7/580: dread d2/d28/f33 [0,4194304] 0 2024-03-04T13:42:54.572 INFO:tasks.workunit.client.0.smithi107.stdout:2/19: dwrite f4 [0,4194304] 0 2024-03-04T13:42:54.572 INFO:tasks.workunit.client.1.smithi169.stdout:2/545: rename d1/c15 to d1/d5/d22/d45/cc2 0 2024-03-04T13:42:54.574 INFO:tasks.workunit.client.0.smithi107.stdout:0/28: mkdir da 0 2024-03-04T13:42:54.574 INFO:tasks.workunit.client.0.smithi107.stdout:6/24: write fa [466152,49343] 0 2024-03-04T13:42:54.577 INFO:tasks.workunit.client.0.smithi107.stdout:7/36: creat d6/fa x:0 0 0 2024-03-04T13:42:54.577 INFO:tasks.workunit.client.0.smithi107.stdout:7/37: readlink l3 0 2024-03-04T13:42:54.578 INFO:tasks.workunit.client.1.smithi169.stdout:2/546: write d1/fc0 [993073,79669] 0 2024-03-04T13:42:54.580 INFO:tasks.workunit.client.1.smithi169.stdout:5/614: dwrite d1/d16/d2f/d39/f5f [0,4194304] 0 2024-03-04T13:42:54.588 INFO:tasks.workunit.client.1.smithi169.stdout:2/547: truncate d1/d5/d22/d45/d4a/f97 675153 0 2024-03-04T13:42:54.590 INFO:tasks.workunit.client.0.smithi107.stdout:6/25: creat fd x:0 0 0 2024-03-04T13:42:54.590 INFO:tasks.workunit.client.1.smithi169.stdout:2/548: chown d1/d5/d22/d3b 9473 1 2024-03-04T13:42:54.591 INFO:tasks.workunit.client.0.smithi107.stdout:6/26: dread - fc zero size 2024-03-04T13:42:54.595 INFO:tasks.workunit.client.1.smithi169.stdout:1/496: dwrite d4/d5/d36/d35/d16/f4d [0,4194304] 0 2024-03-04T13:42:54.595 INFO:tasks.workunit.client.1.smithi169.stdout:2/549: read - d1/d5/d22/d45/fb0 zero size 2024-03-04T13:42:54.599 INFO:tasks.workunit.client.0.smithi107.stdout:7/38: symlink d6/lb 0 2024-03-04T13:42:54.600 INFO:tasks.workunit.client.0.smithi107.stdout:0/29: creat da/fb x:0 0 0 2024-03-04T13:42:54.601 INFO:tasks.workunit.client.0.smithi107.stdout:7/39: readlink d6/lb 0 2024-03-04T13:42:54.602 INFO:tasks.workunit.client.0.smithi107.stdout:6/27: creat fe x:0 0 0 2024-03-04T13:42:54.602 INFO:tasks.workunit.client.1.smithi169.stdout:5/615: symlink d1/d16/d17/lc8 0 2024-03-04T13:42:54.603 INFO:tasks.workunit.client.0.smithi107.stdout:8/32: link l1 d6/lc 0 2024-03-04T13:42:54.604 INFO:tasks.workunit.client.0.smithi107.stdout:2/20: fsync f1 0 2024-03-04T13:42:54.604 INFO:tasks.workunit.client.0.smithi107.stdout:0/30: mkdir da/dc 0 2024-03-04T13:42:54.606 INFO:tasks.workunit.client.0.smithi107.stdout:2/21: write f4 [2516954,108773] 0 2024-03-04T13:42:54.607 INFO:tasks.workunit.client.0.smithi107.stdout:2/22: rmdir - no directory 2024-03-04T13:42:54.607 INFO:tasks.workunit.client.0.smithi107.stdout:2/23: fsync f0 0 2024-03-04T13:42:54.607 INFO:tasks.workunit.client.0.smithi107.stdout:2/24: readlink - no filename 2024-03-04T13:42:54.607 INFO:tasks.workunit.client.0.smithi107.stdout:2/25: fsync f6 0 2024-03-04T13:42:54.608 INFO:tasks.workunit.client.0.smithi107.stdout:7/40: creat d6/fc x:0 0 0 2024-03-04T13:42:54.613 INFO:tasks.workunit.client.0.smithi107.stdout:6/28: mkdir df 0 2024-03-04T13:42:54.618 INFO:tasks.workunit.client.0.smithi107.stdout:8/33: creat d6/fd x:0 0 0 2024-03-04T13:42:54.619 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:42:54 smithi107 ceph-mon[28739]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' 2024-03-04T13:42:54.619 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:42:54 smithi107 ceph-mon[28739]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' 2024-03-04T13:42:54.619 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:42:54 smithi107 ceph-mon[28739]: from='mgr.24331 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-04T13:42:54.619 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:42:54 smithi107 ceph-mon[28739]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-04T13:42:54.619 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:42:54 smithi107 ceph-mon[28739]: from='mgr.24331 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-04T13:42:54.620 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:42:54 smithi107 ceph-mon[28739]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-04T13:42:54.620 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:42:54 smithi107 ceph-mon[28739]: from='mgr.24331 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-04T13:42:54.620 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:42:54 smithi107 ceph-mon[28739]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-04T13:42:54.620 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:42:54 smithi107 ceph-mon[28739]: from='mgr.24331 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:42:54.620 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:42:54 smithi107 ceph-mon[28739]: Upgrade: Need to upgrade myself (mgr.smithi169.eihczr) 2024-03-04T13:42:54.620 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:42:54 smithi107 ceph-mon[28739]: Upgrade: Need to upgrade myself (mgr.smithi169.eihczr) 2024-03-04T13:42:54.620 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:42:54 smithi107 ceph-mon[28739]: pgmap v49: 65 pgs: 65 active+clean; 1.2 GiB data, 5.1 GiB used, 531 GiB / 536 GiB avail; 39 MiB/s rd, 174 MiB/s wr, 346 op/s 2024-03-04T13:42:54.621 INFO:tasks.workunit.client.1.smithi169.stdout:1/497: dwrite d4/d5/d2d/d4e/d71/f37 [4194304,4194304] 0 2024-03-04T13:42:54.621 INFO:tasks.workunit.client.0.smithi107.stdout:2/26: dread f0 [0,4194304] 0 2024-03-04T13:42:54.627 INFO:tasks.workunit.client.1.smithi169.stdout:7/581: link d2/de/d3e/d41/d69/fc2 d2/d28/d70/fc5 0 2024-03-04T13:42:54.628 INFO:tasks.workunit.client.0.smithi107.stdout:7/41: symlink d6/ld 0 2024-03-04T13:42:54.629 INFO:tasks.workunit.client.1.smithi169.stdout:2/550: write d1/d5/d19/d3c/f5f [651423,34351] 0 2024-03-04T13:42:54.632 INFO:tasks.workunit.client.1.smithi169.stdout:7/582: write d2/de/d3e/d41/f8b [2307632,115563] 0 2024-03-04T13:42:54.636 INFO:tasks.workunit.client.1.smithi169.stdout:7/583: stat d2/d28/d70/f98 0 2024-03-04T13:42:54.636 INFO:tasks.workunit.client.1.smithi169.stdout:5/616: dread d1/d16/d2f/f33 [0,4194304] 0 2024-03-04T13:42:54.636 INFO:tasks.workunit.client.1.smithi169.stdout:7/584: dread - d2/de/d3e/d41/d69/fb0 zero size 2024-03-04T13:42:54.637 INFO:tasks.workunit.client.1.smithi169.stdout:1/498: chown d4/d5/d2d/d4e/d71/d26/l5f 13199 1 2024-03-04T13:42:54.638 INFO:tasks.workunit.client.0.smithi107.stdout:6/29: mknod df/c10 0 2024-03-04T13:42:54.639 INFO:tasks.workunit.client.0.smithi107.stdout:0/31: write f0 [1243724,127158] 0 2024-03-04T13:42:54.640 INFO:tasks.workunit.client.1.smithi169.stdout:2/551: mknod d1/d5/d19/d36/d64/cc3 0 2024-03-04T13:42:54.641 INFO:tasks.workunit.client.0.smithi107.stdout:6/30: write fc [173091,84887] 0 2024-03-04T13:42:54.643 INFO:tasks.workunit.client.0.smithi107.stdout:6/31: chown df 0 1 2024-03-04T13:42:54.645 INFO:tasks.workunit.client.0.smithi107.stdout:2/27: truncate f4 376367 0 2024-03-04T13:42:54.646 INFO:tasks.workunit.client.1.smithi169.stdout:2/552: read d1/d5/d22/d45/f72 [871446,106611] 0 2024-03-04T13:42:54.647 INFO:tasks.workunit.client.1.smithi169.stdout:5/617: getdents d1/d16/d2a/d46/d9e 0 2024-03-04T13:42:54.648 INFO:tasks.workunit.client.0.smithi107.stdout:2/28: fsync f0 0 2024-03-04T13:42:54.648 INFO:tasks.workunit.client.1.smithi169.stdout:7/585: read d2/de/d3e/d41/d69/d78/fb4 [2139415,107391] 0 2024-03-04T13:42:54.649 INFO:tasks.workunit.client.0.smithi107.stdout:0/32: mkdir da/dc/dd 0 2024-03-04T13:42:54.649 INFO:tasks.workunit.client.1.smithi169.stdout:2/553: chown d1/d5/d19/d68/l9f 5222206 1 2024-03-04T13:42:54.649 INFO:tasks.workunit.client.0.smithi107.stdout:6/32: creat df/f11 x:0 0 0 2024-03-04T13:42:54.650 INFO:tasks.workunit.client.1.smithi169.stdout:7/586: fsync d2/de/d3e/d41/fa9 0 2024-03-04T13:42:54.651 INFO:tasks.workunit.client.0.smithi107.stdout:7/42: dwrite d6/fa [0,4194304] 0 2024-03-04T13:42:54.652 INFO:tasks.workunit.client.1.smithi169.stdout:5/618: mknod d1/d16/cc9 0 2024-03-04T13:42:54.654 INFO:tasks.workunit.client.1.smithi169.stdout:7/587: write d2/de/d3e/d41/d69/f92 [91671,898] 0 2024-03-04T13:42:54.656 INFO:tasks.workunit.client.1.smithi169.stdout:1/499: dwrite d4/f6b [0,4194304] 0 2024-03-04T13:42:54.658 INFO:tasks.workunit.client.0.smithi107.stdout:6/33: mknod df/c12 0 2024-03-04T13:42:54.659 INFO:tasks.workunit.client.0.smithi107.stdout:2/29: dread f1 [0,4194304] 0 2024-03-04T13:42:54.659 INFO:tasks.workunit.client.0.smithi107.stdout:8/34: dwrite f3 [0,4194304] 0 2024-03-04T13:42:54.660 INFO:tasks.workunit.client.0.smithi107.stdout:6/34: chown f6 18 1 2024-03-04T13:42:54.662 INFO:tasks.workunit.client.1.smithi169.stdout:8/528: sync 2024-03-04T13:42:54.667 INFO:tasks.workunit.client.0.smithi107.stdout:0/33: getdents da/dc/dd 0 2024-03-04T13:42:54.667 INFO:tasks.workunit.client.0.smithi107.stdout:0/34: chown da 4714 1 2024-03-04T13:42:54.669 INFO:tasks.workunit.client.1.smithi169.stdout:2/554: dwrite d1/d5/d19/d63/fbd [0,4194304] 0 2024-03-04T13:42:54.670 INFO:tasks.workunit.client.0.smithi107.stdout:6/35: mknod df/c13 0 2024-03-04T13:42:54.671 INFO:tasks.workunit.client.1.smithi169.stdout:1/500: creat d4/d5/d36/d35/d16/d3a/f9c x:0 0 0 2024-03-04T13:42:54.673 INFO:tasks.workunit.client.1.smithi169.stdout:2/555: write d1/d5/d19/d36/d64/f95 [213628,11603] 0 2024-03-04T13:42:54.674 INFO:tasks.workunit.client.1.smithi169.stdout:7/588: unlink d2/de/f4a 0 2024-03-04T13:42:54.676 INFO:tasks.workunit.client.1.smithi169.stdout:7/589: stat d2/de/d3e/d41/d69/fab 0 2024-03-04T13:42:54.677 INFO:tasks.workunit.client.0.smithi107.stdout:6/36: mkdir df/d14 0 2024-03-04T13:42:54.679 INFO:tasks.workunit.client.0.smithi107.stdout:6/37: chown df 48787554 1 2024-03-04T13:42:54.680 INFO:tasks.workunit.client.1.smithi169.stdout:5/619: link d1/d16/d17/d2d/d5a/d67/f82 d1/d81/dc1/fca 0 2024-03-04T13:42:54.681 INFO:tasks.workunit.client.0.smithi107.stdout:0/35: mkdir da/de 0 2024-03-04T13:42:54.682 INFO:tasks.workunit.client.0.smithi107.stdout:6/38: fsync f6 0 2024-03-04T13:42:54.682 INFO:tasks.workunit.client.0.smithi107.stdout:7/43: dread d6/fa [0,4194304] 0 2024-03-04T13:42:54.683 INFO:tasks.workunit.client.1.smithi169.stdout:2/556: chown d1/d5/d22/d45/c61 629252 1 2024-03-04T13:42:54.684 INFO:tasks.workunit.client.0.smithi107.stdout:2/30: getdents . 0 2024-03-04T13:42:54.684 INFO:tasks.workunit.client.1.smithi169.stdout:7/590: mknod d2/d15/dad/cc6 0 2024-03-04T13:42:54.685 INFO:tasks.workunit.client.0.smithi107.stdout:0/36: read f0 [137416,69110] 0 2024-03-04T13:42:54.686 INFO:tasks.workunit.client.0.smithi107.stdout:2/31: dread - f6 zero size 2024-03-04T13:42:54.688 INFO:tasks.workunit.client.1.smithi169.stdout:1/501: mknod d4/d5/d36/d35/d5b/c9d 0 2024-03-04T13:42:54.689 INFO:tasks.workunit.client.1.smithi169.stdout:1/502: chown d4/d5/d36/d35/df 83833 1 2024-03-04T13:42:54.691 INFO:tasks.workunit.client.1.smithi169.stdout:8/529: creat d7/d10/d17/faa x:0 0 0 2024-03-04T13:42:54.691 INFO:tasks.workunit.client.0.smithi107.stdout:7/44: creat d6/fe x:0 0 0 2024-03-04T13:42:54.692 INFO:tasks.workunit.client.1.smithi169.stdout:2/557: write d1/f75 [2600823,121137] 0 2024-03-04T13:42:54.693 INFO:tasks.workunit.client.0.smithi107.stdout:8/35: dwrite f2 [0,4194304] 0 2024-03-04T13:42:54.693 INFO:tasks.workunit.client.1.smithi169.stdout:5/620: symlink d1/d81/db1/lcb 0 2024-03-04T13:42:54.694 INFO:tasks.workunit.client.0.smithi107.stdout:8/36: chown d6/da 7651368 1 2024-03-04T13:42:54.696 INFO:tasks.workunit.client.0.smithi107.stdout:2/32: rename f6 to f7 0 2024-03-04T13:42:54.697 INFO:tasks.workunit.client.1.smithi169.stdout:8/530: rmdir d7/d10/d22/d75/d79 39 2024-03-04T13:42:54.698 INFO:tasks.workunit.client.0.smithi107.stdout:7/45: fsync d6/fa 0 2024-03-04T13:42:54.699 INFO:tasks.workunit.client.1.smithi169.stdout:1/503: dread d4/d5/f57 [0,4194304] 0 2024-03-04T13:42:54.699 INFO:tasks.workunit.client.0.smithi107.stdout:2/33: chown f0 1 1 2024-03-04T13:42:54.705 INFO:tasks.workunit.client.0.smithi107.stdout:0/37: dread f4 [0,4194304] 0 2024-03-04T13:42:54.706 INFO:tasks.workunit.client.1.smithi169.stdout:5/621: mkdir d1/d16/dbd/dbb/dcc 0 2024-03-04T13:42:54.706 INFO:tasks.workunit.client.1.smithi169.stdout:0/432: sync 2024-03-04T13:42:54.706 INFO:tasks.workunit.client.1.smithi169.stdout:6/535: sync 2024-03-04T13:42:54.706 INFO:tasks.workunit.client.0.smithi107.stdout:7/46: mknod d6/cf 0 2024-03-04T13:42:54.712 INFO:tasks.workunit.client.0.smithi107.stdout:6/39: rmdir df 39 2024-03-04T13:42:54.712 INFO:tasks.workunit.client.1.smithi169.stdout:2/558: dwrite d1/d5/d22/d45/d4a/f92 [4194304,4194304] 0 2024-03-04T13:42:54.715 INFO:tasks.workunit.client.1.smithi169.stdout:2/559: readlink d1/d5/d19/d3c/l56 0 2024-03-04T13:42:54.719 INFO:tasks.workunit.client.1.smithi169.stdout:5/622: creat d1/d16/d2a/fcd x:0 0 0 2024-03-04T13:42:54.719 INFO:tasks.workunit.client.1.smithi169.stdout:2/560: write d1/d5/d19/d69/f7e [2904830,120787] 0 2024-03-04T13:42:54.720 INFO:tasks.workunit.client.1.smithi169.stdout:6/536: mknod d0/d32/d3a/d43/d74/cbf 0 2024-03-04T13:42:54.724 INFO:tasks.workunit.client.1.smithi169.stdout:1/504: creat d4/d5/d36/d35/d8f/d93/f9e x:0 0 0 2024-03-04T13:42:54.725 INFO:tasks.workunit.client.0.smithi107.stdout:2/34: truncate f0 479815 0 2024-03-04T13:42:54.725 INFO:tasks.workunit.client.0.smithi107.stdout:7/47: dwrite d6/fc [0,4194304] 0 2024-03-04T13:42:54.726 INFO:tasks.workunit.client.1.smithi169.stdout:8/531: dread d7/f4d [0,4194304] 0 2024-03-04T13:42:54.727 INFO:tasks.workunit.client.1.smithi169.stdout:0/433: mknod d5/d9/dc/d12/d15/d19/d61/d71/d8c/c9c 0 2024-03-04T13:42:54.729 INFO:tasks.workunit.client.1.smithi169.stdout:1/505: write d4/d1d/f34 [395817,17885] 0 2024-03-04T13:42:54.731 INFO:tasks.workunit.client.1.smithi169.stdout:7/591: dread d2/d15/d17/f42 [0,4194304] 0 2024-03-04T13:42:54.732 INFO:tasks.workunit.client.0.smithi107.stdout:8/37: getdents d6 0 2024-03-04T13:42:54.733 INFO:tasks.workunit.client.0.smithi107.stdout:8/38: chown f3 51 1 2024-03-04T13:42:54.733 INFO:tasks.workunit.client.0.smithi107.stdout:8/39: chown d6/da 5720266 1 2024-03-04T13:42:54.734 INFO:tasks.workunit.client.1.smithi169.stdout:7/592: chown d2/d15/d17/d6f 2989700 1 2024-03-04T13:42:54.735 INFO:tasks.workunit.client.0.smithi107.stdout:2/35: rename c5 to c8 0 2024-03-04T13:42:54.735 INFO:tasks.workunit.client.0.smithi107.stdout:0/38: dwrite f4 [0,4194304] 0 2024-03-04T13:42:54.736 INFO:tasks.workunit.client.0.smithi107.stdout:0/39: read f4 [1384823,57411] 0 2024-03-04T13:42:54.737 INFO:tasks.workunit.client.1.smithi169.stdout:7/593: chown d2/d15/fb9 88 1 2024-03-04T13:42:54.737 INFO:tasks.workunit.client.0.smithi107.stdout:0/40: truncate f7 99203 0 2024-03-04T13:42:54.739 INFO:tasks.workunit.client.0.smithi107.stdout:0/41: truncate f7 709776 0 2024-03-04T13:42:54.740 INFO:tasks.workunit.client.1.smithi169.stdout:7/594: write d2/de/ff [1899499,96019] 0 2024-03-04T13:42:54.741 INFO:tasks.workunit.client.0.smithi107.stdout:7/48: unlink d6/cf 0 2024-03-04T13:42:54.744 INFO:tasks.workunit.client.1.smithi169.stdout:2/561: dread - d1/d5/d22/d3b/f9c zero size 2024-03-04T13:42:54.745 INFO:tasks.workunit.client.0.smithi107.stdout:6/40: creat df/f15 x:0 0 0 2024-03-04T13:42:54.748 INFO:tasks.workunit.client.1.smithi169.stdout:2/562: chown d1/d5/d22/f32 1363747 1 2024-03-04T13:42:54.748 INFO:tasks.workunit.client.0.smithi107.stdout:8/40: creat d6/fe x:0 0 0 2024-03-04T13:42:54.751 INFO:tasks.workunit.client.1.smithi169.stdout:5/623: mkdir d1/d16/d2f/d72/db8/dce 0 2024-03-04T13:42:54.752 INFO:tasks.workunit.client.1.smithi169.stdout:0/434: dwrite d5/d9/dc/d12/d15/f74 [0,4194304] 0 2024-03-04T13:42:54.753 INFO:tasks.workunit.client.1.smithi169.stdout:1/506: creat d4/d5/d36/f9f x:0 0 0 2024-03-04T13:42:54.758 INFO:tasks.workunit.client.1.smithi169.stdout:6/537: mknod d0/cc0 0 2024-03-04T13:42:54.762 INFO:tasks.workunit.client.0.smithi107.stdout:7/49: unlink l3 0 2024-03-04T13:42:54.764 INFO:tasks.workunit.client.0.smithi107.stdout:6/41: mkdir df/d16 0 2024-03-04T13:42:54.765 INFO:tasks.workunit.client.0.smithi107.stdout:7/50: write d6/fa [198779,18742] 0 2024-03-04T13:42:54.766 INFO:tasks.workunit.client.0.smithi107.stdout:7/51: chown f0 7863 1 2024-03-04T13:42:54.767 INFO:tasks.workunit.client.0.smithi107.stdout:7/52: rename d6 to d6/d10 22 2024-03-04T13:42:54.769 INFO:tasks.workunit.client.1.smithi169.stdout:1/507: mknod d4/d5/d36/d35/df/ca0 0 2024-03-04T13:42:54.770 INFO:tasks.workunit.client.0.smithi107.stdout:0/42: creat da/dc/dd/ff x:0 0 0 2024-03-04T13:42:54.773 INFO:tasks.workunit.client.0.smithi107.stdout:8/41: link d6/fd d6/ff 0 2024-03-04T13:42:54.774 INFO:tasks.workunit.client.0.smithi107.stdout:7/53: mknod d6/c11 0 2024-03-04T13:42:54.776 INFO:tasks.workunit.client.1.smithi169.stdout:2/563: getdents d1/d5/d19/d3c/db8 0 2024-03-04T13:42:54.778 INFO:tasks.workunit.client.0.smithi107.stdout:0/43: unlink f7 0 2024-03-04T13:42:54.778 INFO:tasks.workunit.client.1.smithi169.stdout:3/557: sync 2024-03-04T13:42:54.778 INFO:tasks.workunit.client.1.smithi169.stdout:9/546: sync 2024-03-04T13:42:54.779 INFO:tasks.workunit.client.1.smithi169.stdout:4/444: sync 2024-03-04T13:42:54.779 INFO:tasks.workunit.client.0.smithi107.stdout:2/36: dwrite f1 [0,4194304] 0 2024-03-04T13:42:54.784 INFO:tasks.workunit.client.1.smithi169.stdout:4/445: truncate d4/d11/d2e/d37/d5c/f8c 666081 0 2024-03-04T13:42:54.784 INFO:tasks.workunit.client.0.smithi107.stdout:8/42: mknod d6/da/c10 0 2024-03-04T13:42:54.786 INFO:tasks.workunit.client.0.smithi107.stdout:8/43: readlink d6/l9 0 2024-03-04T13:42:54.786 INFO:tasks.workunit.client.0.smithi107.stdout:6/42: dwrite f0 [0,4194304] 0 2024-03-04T13:42:54.787 INFO:tasks.workunit.client.1.smithi169.stdout:6/538: creat d0/d3c/d40/d52/dac/fc1 x:0 0 0 2024-03-04T13:42:54.788 INFO:tasks.workunit.client.0.smithi107.stdout:8/44: chown d6/fd 13 1 2024-03-04T13:42:54.789 INFO:tasks.workunit.client.1.smithi169.stdout:7/595: link d2/de/d37/f3f d2/dc0/fc7 0 2024-03-04T13:42:54.790 INFO:tasks.workunit.client.0.smithi107.stdout:8/45: read - d6/ff zero size 2024-03-04T13:42:54.790 INFO:tasks.workunit.client.0.smithi107.stdout:8/46: stat f5 0 2024-03-04T13:42:54.791 INFO:tasks.workunit.client.0.smithi107.stdout:0/44: mknod da/dc/c10 0 2024-03-04T13:42:54.792 INFO:tasks.workunit.client.0.smithi107.stdout:8/47: write f2 [848684,38627] 0 2024-03-04T13:42:54.793 INFO:tasks.workunit.client.0.smithi107.stdout:8/48: readlink d6/l9 0 2024-03-04T13:42:54.794 INFO:tasks.workunit.client.1.smithi169.stdout:8/532: creat d7/d10/d17/d44/d36/d62/fab x:0 0 0 2024-03-04T13:42:54.795 INFO:tasks.workunit.client.0.smithi107.stdout:2/37: write f2 [3756087,92681] 0 2024-03-04T13:42:54.795 INFO:tasks.workunit.client.1.smithi169.stdout:0/435: dread d5/d9/dc/d12/d15/d46/f57 [0,4194304] 0 2024-03-04T13:42:54.796 INFO:tasks.workunit.client.1.smithi169.stdout:3/558: dread d7/de/d2b/d7d/d85/d18/f30 [0,4194304] 0 2024-03-04T13:42:54.797 INFO:tasks.workunit.client.1.smithi169.stdout:2/564: rename d1/d5/d19/d1e/f82 to d1/d5/d22/d45/fc4 0 2024-03-04T13:42:54.798 INFO:tasks.workunit.client.1.smithi169.stdout:8/533: fsync d7/d10/d24/d3e/f68 0 2024-03-04T13:42:54.801 INFO:tasks.workunit.client.1.smithi169.stdout:5/624: link d1/d16/d2a/d46/lc7 d1/d16/d17/d1b/d74/lcf 0 2024-03-04T13:42:54.805 INFO:tasks.workunit.client.1.smithi169.stdout:9/547: unlink d7/dc/d12/d20/d4e/c52 0 2024-03-04T13:42:54.805 INFO:tasks.workunit.client.1.smithi169.stdout:4/446: dwrite d4/f41 [4194304,4194304] 0 2024-03-04T13:42:54.807 INFO:tasks.workunit.client.0.smithi107.stdout:0/45: write f4 [3690294,101864] 0 2024-03-04T13:42:54.808 INFO:tasks.workunit.client.1.smithi169.stdout:3/559: stat d7/de/d2b/d7d/d85/d13/d1a/d2d/d56/d5e/d67/d5c/d84/f96 0 2024-03-04T13:42:54.808 INFO:tasks.workunit.client.1.smithi169.stdout:9/548: chown d7/dc/d12/d20/da6/fb4 9674078 1 2024-03-04T13:42:54.812 INFO:tasks.workunit.client.0.smithi107.stdout:8/49: unlink d6/f8 0 2024-03-04T13:42:54.812 INFO:tasks.workunit.client.1.smithi169.stdout:4/447: truncate d4/d11/d2e/d39/d48/f3c 9206843 0 2024-03-04T13:42:54.813 INFO:tasks.workunit.client.0.smithi107.stdout:8/50: chown f5 7221231 1 2024-03-04T13:42:54.814 INFO:tasks.workunit.client.0.smithi107.stdout:8/51: readlink d6/l7 0 2024-03-04T13:42:54.814 INFO:tasks.workunit.client.0.smithi107.stdout:8/52: fdatasync f2 0 2024-03-04T13:42:54.816 INFO:tasks.workunit.client.1.smithi169.stdout:7/596: creat d2/de/d3e/d41/d69/fc8 x:0 0 0 2024-03-04T13:42:54.817 INFO:tasks.workunit.client.1.smithi169.stdout:4/448: truncate d4/d11/d2e/d39/f68 787520 0 2024-03-04T13:42:54.818 INFO:tasks.workunit.client.0.smithi107.stdout:2/38: dread f0 [0,4194304] 0 2024-03-04T13:42:54.822 INFO:tasks.workunit.client.1.smithi169.stdout:2/565: rename d1/c3a to d1/d5/d19/d68/cc5 0 2024-03-04T13:42:54.824 INFO:tasks.workunit.client.1.smithi169.stdout:8/534: mkdir d7/d10/d22/d75/d81/da2/dac 0 2024-03-04T13:42:54.824 INFO:tasks.workunit.client.0.smithi107.stdout:8/53: creat d6/da/f11 x:0 0 0 2024-03-04T13:42:54.826 INFO:tasks.workunit.client.0.smithi107.stdout:8/54: readlink d6/l7 0 2024-03-04T13:42:54.826 INFO:tasks.workunit.client.0.smithi107.stdout:2/39: mknod c9 0 2024-03-04T13:42:54.826 INFO:tasks.workunit.client.0.smithi107.stdout:8/55: chown d6/l7 6995723 1 2024-03-04T13:42:54.827 INFO:tasks.workunit.client.0.smithi107.stdout:2/40: read - f7 zero size 2024-03-04T13:42:54.827 INFO:tasks.workunit.client.1.smithi169.stdout:5/625: dread d1/d16/d17/d1b/f69 [0,4194304] 0 2024-03-04T13:42:54.830 INFO:tasks.workunit.client.1.smithi169.stdout:9/549: creat d7/d25/d64/fb7 x:0 0 0 2024-03-04T13:42:54.830 INFO:tasks.workunit.client.1.smithi169.stdout:3/560: mkdir d7/de/d2b/d7d/d85/d13/d1a/d2d/dbe 0 2024-03-04T13:42:54.831 INFO:tasks.workunit.client.0.smithi107.stdout:6/43: rmdir df/d14 0 2024-03-04T13:42:54.831 INFO:tasks.workunit.client.0.smithi107.stdout:2/41: chown f1 30849 1 2024-03-04T13:42:54.831 INFO:tasks.workunit.client.0.smithi107.stdout:0/46: getdents da/de 0 2024-03-04T13:42:54.840 INFO:tasks.workunit.client.1.smithi169.stdout:4/449: unlink d4/d11/f8d 0 2024-03-04T13:42:54.840 INFO:tasks.workunit.client.0.smithi107.stdout:8/56: rmdir d6/da 39 2024-03-04T13:42:54.842 INFO:tasks.workunit.client.1.smithi169.stdout:2/566: mknod d1/d5/d19/d63/cc6 0 2024-03-04T13:42:54.844 INFO:tasks.workunit.client.0.smithi107.stdout:2/42: dwrite f1 [0,4194304] 0 2024-03-04T13:42:54.845 INFO:tasks.workunit.client.1.smithi169.stdout:8/535: symlink d7/d12/lad 0 2024-03-04T13:42:54.847 INFO:tasks.workunit.client.1.smithi169.stdout:1/508: getdents d4/d5/d36/d35/d16/d81 0 2024-03-04T13:42:54.850 INFO:tasks.workunit.client.1.smithi169.stdout:5/626: unlink d1/d81/d91/f94 0 2024-03-04T13:42:54.854 INFO:tasks.workunit.client.1.smithi169.stdout:9/550: mknod d7/dc/d3e/da3/cb8 0 2024-03-04T13:42:54.857 INFO:tasks.workunit.client.1.smithi169.stdout:3/561: unlink d7/de/d2b/d7d/d85/d18/fa2 0 2024-03-04T13:42:54.857 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:42:54 smithi169 ceph-mon[36313]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' 2024-03-04T13:42:54.857 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:42:54 smithi169 ceph-mon[36313]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' 2024-03-04T13:42:54.857 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:42:54 smithi169 ceph-mon[36313]: from='mgr.24331 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-04T13:42:54.857 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:42:54 smithi169 ceph-mon[36313]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-04T13:42:54.857 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:42:54 smithi169 ceph-mon[36313]: from='mgr.24331 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-04T13:42:54.857 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:42:54 smithi169 ceph-mon[36313]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-04T13:42:54.857 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:42:54 smithi169 ceph-mon[36313]: from='mgr.24331 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-04T13:42:54.857 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:42:54 smithi169 ceph-mon[36313]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-04T13:42:54.858 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:42:54 smithi169 ceph-mon[36313]: from='mgr.24331 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:42:54.858 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:42:54 smithi169 ceph-mon[36313]: Upgrade: Need to upgrade myself (mgr.smithi169.eihczr) 2024-03-04T13:42:54.858 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:42:54 smithi169 ceph-mon[36313]: Upgrade: Need to upgrade myself (mgr.smithi169.eihczr) 2024-03-04T13:42:54.858 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:42:54 smithi169 ceph-mon[36313]: pgmap v49: 65 pgs: 65 active+clean; 1.2 GiB data, 5.1 GiB used, 531 GiB / 536 GiB avail; 39 MiB/s rd, 174 MiB/s wr, 346 op/s 2024-03-04T13:42:54.858 INFO:tasks.workunit.client.1.smithi169.stdout:0/436: dwrite d5/d9/dc/d12/f62 [0,4194304] 0 2024-03-04T13:42:54.862 INFO:tasks.workunit.client.1.smithi169.stdout:4/450: symlink d4/d1c/d75/d81/l90 0 2024-03-04T13:42:54.871 INFO:tasks.workunit.client.1.smithi169.stdout:2/567: mknod d1/d6d/cc7 0 2024-03-04T13:42:54.872 INFO:tasks.workunit.client.1.smithi169.stdout:3/562: dread d7/de/d2b/d7d/d85/d13/d1a/d2d/f37 [0,4194304] 0 2024-03-04T13:42:54.877 INFO:tasks.workunit.client.1.smithi169.stdout:8/536: mknod d7/d10/d17/d44/d4c/cae 0 2024-03-04T13:42:54.878 INFO:tasks.workunit.client.1.smithi169.stdout:8/537: dread - d7/f9f zero size 2024-03-04T13:42:54.884 INFO:tasks.workunit.client.1.smithi169.stdout:2/568: dread d1/f51 [4194304,4194304] 0 2024-03-04T13:42:54.884 INFO:tasks.workunit.client.1.smithi169.stdout:6/539: getdents d0/d32/d3a/d43 0 2024-03-04T13:42:54.886 INFO:tasks.workunit.client.1.smithi169.stdout:4/451: symlink d4/d11/d3b/d67/d7e/l91 0 2024-03-04T13:42:54.887 INFO:tasks.workunit.client.1.smithi169.stdout:2/569: read d1/d5/d19/f2e [3393395,104285] 0 2024-03-04T13:42:54.893 INFO:tasks.workunit.client.1.smithi169.stdout:7/597: rename d2/d15/fa6 to d2/fc9 0 2024-03-04T13:42:54.900 INFO:tasks.workunit.client.1.smithi169.stdout:1/509: symlink d4/d5/d2d/d4e/d71/d82/la1 0 2024-03-04T13:42:54.902 INFO:tasks.workunit.client.1.smithi169.stdout:1/510: fdatasync d4/d5/d36/d35/df/f28 0 2024-03-04T13:42:54.905 INFO:tasks.workunit.client.1.smithi169.stdout:0/437: dread d5/f8 [0,4194304] 0 2024-03-04T13:42:54.908 INFO:tasks.workunit.client.1.smithi169.stdout:2/570: creat d1/d5/d19/d69/fc8 x:0 0 0 2024-03-04T13:42:54.910 INFO:tasks.workunit.client.1.smithi169.stdout:8/538: dread d7/d10/d22/d46/f6e [0,4194304] 0 2024-03-04T13:42:54.910 INFO:tasks.workunit.client.1.smithi169.stdout:5/627: rename d1/d16/d17/d1b/d74/l88 to d1/d16/d17/d2d/d5a/d67/ld0 0 2024-03-04T13:42:54.912 INFO:tasks.workunit.client.1.smithi169.stdout:7/598: symlink d2/de/d37/lca 0 2024-03-04T13:42:54.914 INFO:tasks.workunit.client.1.smithi169.stdout:3/563: mknod d7/de/d2b/d7d/d85/d13/d1a/cbf 0 2024-03-04T13:42:54.917 INFO:tasks.workunit.client.1.smithi169.stdout:9/551: creat d7/dc/d12/d32/fb9 x:0 0 0 2024-03-04T13:42:54.918 INFO:tasks.workunit.client.1.smithi169.stdout:3/564: read d7/f33 [12936,70348] 0 2024-03-04T13:42:54.921 INFO:tasks.workunit.client.1.smithi169.stdout:2/571: fsync d1/d5/d22/d3b/f52 0 2024-03-04T13:42:54.921 INFO:tasks.workunit.client.1.smithi169.stdout:9/552: chown d7/dc/d12/d32/d42/f9d 1196 1 2024-03-04T13:42:54.929 INFO:tasks.workunit.client.1.smithi169.stdout:3/565: write d7/de/d2b/d7d/d85/d18/f7a [112977,40675] 0 2024-03-04T13:42:54.935 INFO:tasks.workunit.client.1.smithi169.stdout:9/553: stat d7/dc/d12/d32/d42/c60 0 2024-03-04T13:42:54.940 INFO:tasks.workunit.client.1.smithi169.stdout:8/539: dwrite d7/d10/d17/d44/d5d/f9a [0,4194304] 0 2024-03-04T13:42:54.944 INFO:tasks.workunit.client.1.smithi169.stdout:5/628: dread d1/d16/d17/d2d/d5a/d67/f6b [0,4194304] 0 2024-03-04T13:42:54.947 INFO:tasks.workunit.client.1.smithi169.stdout:7/599: dwrite d2/d15/d17/d6f/f72 [4194304,4194304] 0 2024-03-04T13:42:54.958 INFO:tasks.workunit.client.1.smithi169.stdout:4/452: truncate d4/f7 1153417 0 2024-03-04T13:42:54.958 INFO:tasks.workunit.client.1.smithi169.stdout:3/566: dwrite d7/de/d2b/d7d/d85/d13/d5a/f83 [0,4194304] 0 2024-03-04T13:42:54.964 INFO:tasks.workunit.client.1.smithi169.stdout:4/453: dread d4/d11/d2e/d39/f68 [0,4194304] 0 2024-03-04T13:42:54.972 INFO:tasks.workunit.client.0.smithi107.stdout:6/44: unlink df/c10 0 2024-03-04T13:42:54.973 INFO:tasks.workunit.client.1.smithi169.stdout:6/540: link d0/d32/d3e/l90 d0/d32/d70/lc2 0 2024-03-04T13:42:54.975 INFO:tasks.workunit.client.0.smithi107.stdout:6/45: write fd [736409,48580] 0 2024-03-04T13:42:54.976 INFO:tasks.workunit.client.1.smithi169.stdout:0/438: rename d5/d9/dc/d12/c3e to d5/d9/dc/d12/d15/d46/d98/c9d 0 2024-03-04T13:42:54.976 INFO:tasks.workunit.client.1.smithi169.stdout:2/572: dread - d1/d5/f4e zero size 2024-03-04T13:42:54.976 INFO:tasks.workunit.client.0.smithi107.stdout:2/43: mkdir da 0 2024-03-04T13:42:54.979 INFO:tasks.workunit.client.0.smithi107.stdout:0/47: creat da/de/f11 x:0 0 0 2024-03-04T13:42:54.980 INFO:tasks.workunit.client.1.smithi169.stdout:1/511: creat d4/d5/d36/d35/fa2 x:0 0 0 2024-03-04T13:42:54.980 INFO:tasks.workunit.client.1.smithi169.stdout:9/554: rename d7/d49/l8e to d7/d49/lba 0 2024-03-04T13:42:54.980 INFO:tasks.workunit.client.1.smithi169.stdout:2/573: dread - d1/d5/d22/d45/fc4 zero size 2024-03-04T13:42:54.980 INFO:tasks.workunit.client.1.smithi169.stdout:8/540: mknod d7/d10/d22/d75/d81/da2/caf 0 2024-03-04T13:42:54.981 INFO:tasks.workunit.client.1.smithi169.stdout:5/629: unlink d1/d81/d91/l9d 0 2024-03-04T13:42:54.982 INFO:tasks.workunit.client.1.smithi169.stdout:8/541: read - d7/f9f zero size 2024-03-04T13:42:54.982 INFO:tasks.workunit.client.0.smithi107.stdout:0/48: chown c3 181033 1 2024-03-04T13:42:54.983 INFO:tasks.workunit.client.1.smithi169.stdout:9/555: dread - d7/dc/d12/d32/fb9 zero size 2024-03-04T13:42:54.985 INFO:tasks.workunit.client.0.smithi107.stdout:0/49: fsync da/dc/dd/ff 0 2024-03-04T13:42:54.987 INFO:tasks.workunit.client.1.smithi169.stdout:3/567: creat d7/de/d2b/d7d/d85/d13/d1a/d2d/fc0 x:0 0 0 2024-03-04T13:42:54.989 INFO:tasks.workunit.client.1.smithi169.stdout:6/541: mknod d0/d63/cc3 0 2024-03-04T13:42:54.991 INFO:tasks.workunit.client.1.smithi169.stdout:1/512: creat d4/d5/d2d/d4e/fa3 x:0 0 0 2024-03-04T13:42:54.991 INFO:tasks.workunit.client.1.smithi169.stdout:8/542: dread d7/d10/d24/d3e/f40 [0,4194304] 0 2024-03-04T13:42:54.991 INFO:tasks.workunit.client.1.smithi169.stdout:3/568: chown d7/de/d2b/d7d/d85/d13/d1a/d2d/fb6 464845 1 2024-03-04T13:42:54.994 INFO:tasks.workunit.client.1.smithi169.stdout:1/513: stat d4/d5/d36/d35/d16/f4d 0 2024-03-04T13:42:54.996 INFO:tasks.workunit.client.1.smithi169.stdout:2/574: truncate d1/d5/d19/f3f 5210693 0 2024-03-04T13:42:54.996 INFO:tasks.workunit.client.0.smithi107.stdout:6/46: dread f8 [0,4194304] 0 2024-03-04T13:42:54.998 INFO:tasks.workunit.client.0.smithi107.stdout:0/50: link l5 da/l12 0 2024-03-04T13:42:55.001 INFO:tasks.workunit.client.1.smithi169.stdout:4/454: rename d4/d11/d3b/f3f to d4/d11/d2e/d21/d5e/d6e/f92 0 2024-03-04T13:42:55.003 INFO:tasks.workunit.client.0.smithi107.stdout:0/51: symlink da/dc/l13 0 2024-03-04T13:42:55.005 INFO:tasks.workunit.client.1.smithi169.stdout:7/600: creat d2/de/fcb x:0 0 0 2024-03-04T13:42:55.008 INFO:tasks.workunit.client.1.smithi169.stdout:8/543: mknod d7/d10/d17/d44/d36/cb0 0 2024-03-04T13:42:55.009 INFO:tasks.workunit.client.1.smithi169.stdout:7/601: truncate d2/d15/d17/d6f/f72 8903720 0 2024-03-04T13:42:55.009 INFO:tasks.workunit.client.1.smithi169.stdout:9/556: dread d7/dc/d12/d32/d42/f43 [0,4194304] 0 2024-03-04T13:42:55.010 INFO:tasks.workunit.client.1.smithi169.stdout:0/439: dread d5/d9/dc/d12/d15/d19/f2e [4194304,4194304] 0 2024-03-04T13:42:55.011 INFO:tasks.workunit.client.0.smithi107.stdout:0/52: readlink l9 0 2024-03-04T13:42:55.012 INFO:tasks.workunit.client.1.smithi169.stdout:9/557: dread - d7/d25/d64/fa7 zero size 2024-03-04T13:42:55.013 INFO:tasks.workunit.client.1.smithi169.stdout:1/514: fsync d4/d5/d36/d35/d5b/f7f 0 2024-03-04T13:42:55.013 INFO:tasks.workunit.client.1.smithi169.stdout:9/558: chown d7/f7a 6278613 1 2024-03-04T13:42:55.014 INFO:tasks.workunit.client.0.smithi107.stdout:6/47: rmdir df/d16 0 2024-03-04T13:42:55.015 INFO:tasks.workunit.client.1.smithi169.stdout:2/575: mknod d1/d6d/cc9 0 2024-03-04T13:42:55.016 INFO:tasks.workunit.client.0.smithi107.stdout:0/53: readlink l6 0 2024-03-04T13:42:55.018 INFO:tasks.workunit.client.0.smithi107.stdout:0/54: chown da/dc/c10 28 1 2024-03-04T13:42:55.019 INFO:tasks.workunit.client.1.smithi169.stdout:6/542: rename d0/d63/cc3 to d0/d32/d3a/d43/d74/cc4 0 2024-03-04T13:42:55.020 INFO:tasks.workunit.client.1.smithi169.stdout:6/543: read - d0/d32/d3a/d43/d74/f7e zero size 2024-03-04T13:42:55.023 INFO:tasks.workunit.client.1.smithi169.stdout:3/569: mkdir d7/de/d2b/d7d/d85/d13/d1a/d78/dc1 0 2024-03-04T13:42:55.025 INFO:tasks.workunit.client.1.smithi169.stdout:3/570: chown d7/de/d2b/d7d/d85/d13/d5a/da1 0 1 2024-03-04T13:42:55.026 INFO:tasks.workunit.client.0.smithi107.stdout:0/55: link f8 da/dc/f14 0 2024-03-04T13:42:55.027 INFO:tasks.workunit.client.0.smithi107.stdout:6/48: dread f8 [0,4194304] 0 2024-03-04T13:42:55.027 INFO:tasks.workunit.client.1.smithi169.stdout:1/515: write d4/d5/f57 [1035003,92770] 0 2024-03-04T13:42:55.029 INFO:tasks.workunit.client.1.smithi169.stdout:2/576: unlink d1/d5/d22/d45/d4a/f97 0 2024-03-04T13:42:55.034 INFO:tasks.workunit.client.1.smithi169.stdout:8/544: creat d7/d10/d22/d75/d79/fb1 x:0 0 0 2024-03-04T13:42:55.035 INFO:tasks.workunit.client.1.smithi169.stdout:6/544: rmdir d0/d3c/d40 39 2024-03-04T13:42:55.036 INFO:tasks.workunit.client.1.smithi169.stdout:0/440: fsync d5/d4c/d65/f6b 0 2024-03-04T13:42:55.039 INFO:tasks.workunit.client.1.smithi169.stdout:7/602: link d2/d28/fa1 d2/de/d3e/d41/d5b/d7a/fcc 0 2024-03-04T13:42:55.040 INFO:tasks.workunit.client.0.smithi107.stdout:0/56: dread f8 [0,4194304] 0 2024-03-04T13:42:55.044 INFO:tasks.workunit.client.1.smithi169.stdout:1/516: symlink d4/d5/d36/d35/df/la4 0 2024-03-04T13:42:55.045 INFO:tasks.workunit.client.0.smithi107.stdout:0/57: unlink da/fb 0 2024-03-04T13:42:55.049 INFO:tasks.workunit.client.1.smithi169.stdout:4/455: rmdir d4/d11/d2e/d39/d48/d8b 0 2024-03-04T13:42:55.049 INFO:tasks.workunit.client.1.smithi169.stdout:8/545: dwrite d7/d10/d24/f7a [0,4194304] 0 2024-03-04T13:42:55.057 INFO:tasks.workunit.client.1.smithi169.stdout:7/603: mknod d2/de/d3e/d41/d69/ccd 0 2024-03-04T13:42:55.059 INFO:tasks.workunit.client.1.smithi169.stdout:1/517: dwrite d4/d5/d36/d35/fa2 [0,4194304] 0 2024-03-04T13:42:55.059 INFO:tasks.workunit.client.1.smithi169.stdout:7/604: chown d2/de/d3e/d41/lbc 28 1 2024-03-04T13:42:55.060 INFO:tasks.workunit.client.1.smithi169.stdout:9/559: rename d7/d27/l68 to d7/d2d/lbb 0 2024-03-04T13:42:55.063 INFO:tasks.workunit.client.1.smithi169.stdout:7/605: chown d2/de/d3e/d41/d69/fab 9 1 2024-03-04T13:42:55.064 INFO:tasks.workunit.client.1.smithi169.stdout:1/518: fsync d4/d5/d36/d35/d8f/f9a 0 2024-03-04T13:42:55.066 INFO:tasks.workunit.client.0.smithi107.stdout:8/57: write d6/ff [115375,18963] 0 2024-03-04T13:42:55.066 INFO:tasks.workunit.client.1.smithi169.stdout:7/606: chown d2/de/fcb 303 1 2024-03-04T13:42:55.066 INFO:tasks.workunit.client.1.smithi169.stdout:6/545: creat d0/d3c/d95/dbd/fc5 x:0 0 0 2024-03-04T13:42:55.067 INFO:tasks.workunit.client.1.smithi169.stdout:6/546: read - d0/d63/fba zero size 2024-03-04T13:42:55.068 INFO:tasks.workunit.client.0.smithi107.stdout:8/58: readlink d6/l9 0 2024-03-04T13:42:55.069 INFO:tasks.workunit.client.1.smithi169.stdout:6/547: stat d0/d32/d3a/d43/d74/da6 0 2024-03-04T13:42:55.070 INFO:tasks.workunit.client.0.smithi107.stdout:0/58: getdents da/de 0 2024-03-04T13:42:55.070 INFO:tasks.workunit.client.0.smithi107.stdout:8/59: chown f4 1 1 2024-03-04T13:42:55.072 INFO:tasks.workunit.client.1.smithi169.stdout:3/571: rename d7/de/d2b/d7d/d85/d18/c2f to d7/de/d2b/d7d/d85/d13/d1a/d2d/d56/d5e/d67/d5c/cc2 0 2024-03-04T13:42:55.074 INFO:tasks.workunit.client.1.smithi169.stdout:9/560: mkdir d7/dc/d3e/da3/dbc 0 2024-03-04T13:42:55.076 INFO:tasks.workunit.client.1.smithi169.stdout:3/572: chown d7/de/d2b/d7d/d85/d18/d21/d3f/l42 13000024 1 2024-03-04T13:42:55.077 INFO:tasks.workunit.client.1.smithi169.stdout:4/456: truncate d4/d11/f1b 413827 0 2024-03-04T13:42:55.078 INFO:tasks.workunit.client.0.smithi107.stdout:8/60: chown d6/lc 1 1 2024-03-04T13:42:55.079 INFO:tasks.workunit.client.0.smithi107.stdout:2/44: write f0 [20320,128142] 0 2024-03-04T13:42:55.080 INFO:tasks.workunit.client.1.smithi169.stdout:4/457: chown d4/d11/d2e/d39/d61/d64 0 1 2024-03-04T13:42:55.080 INFO:tasks.workunit.client.0.smithi107.stdout:0/59: symlink da/dc/dd/l15 0 2024-03-04T13:42:55.081 INFO:tasks.workunit.client.1.smithi169.stdout:1/519: mkdir d4/d5/d2d/d4e/da5 0 2024-03-04T13:42:55.081 INFO:tasks.workunit.client.0.smithi107.stdout:8/61: write f2 [4035795,13142] 0 2024-03-04T13:42:55.085 INFO:tasks.workunit.client.1.smithi169.stdout:5/630: write d1/d23/f85 [243008,60377] 0 2024-03-04T13:42:55.087 INFO:tasks.workunit.client.0.smithi107.stdout:0/60: write f0 [1503178,73434] 0 2024-03-04T13:42:55.088 INFO:tasks.workunit.client.1.smithi169.stdout:3/573: creat d7/de/d2b/d7d/d85/d13/d1a/d2d/d56/fc3 x:0 0 0 2024-03-04T13:42:55.088 INFO:tasks.workunit.client.1.smithi169.stdout:5/631: dread - d1/d16/d17/d2d/d51/fbf zero size 2024-03-04T13:42:55.091 INFO:tasks.workunit.client.1.smithi169.stdout:4/458: symlink d4/d11/d2e/d37/d5c/l93 0 2024-03-04T13:42:55.093 INFO:tasks.workunit.client.1.smithi169.stdout:4/459: dread - d4/d11/d3b/d67/f84 zero size 2024-03-04T13:42:55.094 INFO:tasks.workunit.client.1.smithi169.stdout:1/520: readlink d4/l95 0 2024-03-04T13:42:55.097 INFO:tasks.workunit.client.0.smithi107.stdout:0/61: write da/de/f11 [880288,130645] 0 2024-03-04T13:42:55.098 INFO:tasks.workunit.client.0.smithi107.stdout:8/62: symlink d6/da/l12 0 2024-03-04T13:42:55.100 INFO:tasks.workunit.client.1.smithi169.stdout:3/574: dread d7/d53/f5d [0,4194304] 0 2024-03-04T13:42:55.105 INFO:tasks.workunit.client.1.smithi169.stdout:1/521: creat d4/d5/d2d/d4e/fa6 x:0 0 0 2024-03-04T13:42:55.110 INFO:tasks.workunit.client.0.smithi107.stdout:2/45: dwrite f2 [4194304,4194304] 0 2024-03-04T13:42:55.110 INFO:tasks.workunit.client.1.smithi169.stdout:7/607: getdents d2/de/d3e/d41 0 2024-03-04T13:42:55.111 INFO:tasks.workunit.client.1.smithi169.stdout:6/548: getdents d0/d32 0 2024-03-04T13:42:55.112 INFO:tasks.workunit.client.0.smithi107.stdout:2/46: read f1 [1505635,80319] 0 2024-03-04T13:42:55.113 INFO:tasks.workunit.client.1.smithi169.stdout:7/608: stat d2/d15/d54/c57 0 2024-03-04T13:42:55.115 INFO:tasks.workunit.client.1.smithi169.stdout:6/549: chown d0/d32/d3a/l2c 2841430 1 2024-03-04T13:42:55.115 INFO:tasks.workunit.client.0.smithi107.stdout:2/47: dread f0 [0,4194304] 0 2024-03-04T13:42:55.116 INFO:tasks.workunit.client.1.smithi169.stdout:4/460: rename d4/d11/d2e/d39/c3d to d4/d1c/d75/c94 0 2024-03-04T13:42:55.118 INFO:tasks.workunit.client.0.smithi107.stdout:8/63: dwrite f2 [0,4194304] 0 2024-03-04T13:42:55.126 INFO:tasks.workunit.client.1.smithi169.stdout:5/632: creat d1/d16/d17/d2d/d5a/fd1 x:0 0 0 2024-03-04T13:42:55.126 INFO:tasks.workunit.client.1.smithi169.stdout:1/522: dwrite d4/f6b [0,4194304] 0 2024-03-04T13:42:55.127 INFO:tasks.workunit.client.1.smithi169.stdout:8/546: write d7/d10/d24/f85 [642737,117081] 0 2024-03-04T13:42:55.129 INFO:tasks.workunit.client.1.smithi169.stdout:4/461: rmdir d4/d11/d2e/d21 39 2024-03-04T13:42:55.129 INFO:tasks.workunit.client.1.smithi169.stdout:2/577: dwrite d1/d5/f4e [0,4194304] 0 2024-03-04T13:42:55.134 INFO:tasks.workunit.client.1.smithi169.stdout:6/550: dread d0/d32/d3a/d43/d74/f62 [0,4194304] 0 2024-03-04T13:42:55.136 INFO:tasks.workunit.client.1.smithi169.stdout:1/523: write d4/d1d/f8e [594858,96050] 0 2024-03-04T13:42:55.137 INFO:tasks.workunit.client.0.smithi107.stdout:8/64: dwrite f3 [0,4194304] 0 2024-03-04T13:42:55.138 INFO:tasks.workunit.client.1.smithi169.stdout:0/441: truncate d5/d9/dc/d12/f62 2327921 0 2024-03-04T13:42:55.139 INFO:tasks.workunit.client.1.smithi169.stdout:3/575: rmdir d7/de/d2b/d7d/d85/d18 39 2024-03-04T13:42:55.141 INFO:tasks.workunit.client.1.smithi169.stdout:1/524: dread - d4/d5/d36/d35/d5b/f7f zero size 2024-03-04T13:42:55.142 INFO:tasks.workunit.client.0.smithi107.stdout:2/48: creat da/fb x:0 0 0 2024-03-04T13:42:55.144 INFO:tasks.workunit.client.0.smithi107.stdout:2/49: write f2 [6860520,88991] 0 2024-03-04T13:42:55.146 INFO:tasks.workunit.client.1.smithi169.stdout:8/547: mknod d7/d10/d17/d44/d5d/cb2 0 2024-03-04T13:42:55.146 INFO:tasks.workunit.client.0.smithi107.stdout:8/65: rmdir d6/da 39 2024-03-04T13:42:55.146 INFO:tasks.workunit.client.1.smithi169.stdout:1/525: write d4/d5/d36/d35/d5b/f87 [1092738,43144] 0 2024-03-04T13:42:55.148 INFO:tasks.workunit.client.1.smithi169.stdout:9/561: write d7/dc/d3e/f6c [2154726,29354] 0 2024-03-04T13:42:55.151 INFO:tasks.workunit.client.1.smithi169.stdout:4/462: fsync d4/d11/d2e/d39/d48/f65 0 2024-03-04T13:42:55.155 INFO:tasks.workunit.client.0.smithi107.stdout:2/50: fdatasync f0 0 2024-03-04T13:42:55.157 INFO:tasks.workunit.client.1.smithi169.stdout:4/463: write d4/d11/f79 [228480,88104] 0 2024-03-04T13:42:55.158 INFO:tasks.workunit.client.1.smithi169.stdout:6/551: rename d0/f4b to d0/d3c/d95/dbd/fc6 0 2024-03-04T13:42:55.161 INFO:tasks.workunit.client.0.smithi107.stdout:8/66: symlink d6/da/l13 0 2024-03-04T13:42:55.163 INFO:tasks.workunit.client.0.smithi107.stdout:2/51: mknod da/cc 0 2024-03-04T13:42:55.165 INFO:tasks.workunit.client.0.smithi107.stdout:8/67: symlink d6/da/l14 0 2024-03-04T13:42:55.167 INFO:tasks.workunit.client.0.smithi107.stdout:8/68: chown d6/lc 1 1 2024-03-04T13:42:55.167 INFO:tasks.workunit.client.1.smithi169.stdout:8/548: mknod d7/d10/d22/d75/d81/da2/cb3 0 2024-03-04T13:42:55.168 INFO:tasks.workunit.client.0.smithi107.stdout:0/62: dwrite da/dc/f14 [4194304,4194304] 0 2024-03-04T13:42:55.168 INFO:tasks.workunit.client.0.smithi107.stdout:8/69: dread - d6/da/f11 zero size 2024-03-04T13:42:55.168 INFO:tasks.workunit.client.0.smithi107.stdout:0/63: readlink da/dc/dd/l15 0 2024-03-04T13:42:55.168 INFO:tasks.workunit.client.1.smithi169.stdout:2/578: creat d1/d5/d19/d3c/db8/fca x:0 0 0 2024-03-04T13:42:55.169 INFO:tasks.workunit.client.0.smithi107.stdout:8/70: chown f2 0 1 2024-03-04T13:42:55.170 INFO:tasks.workunit.client.0.smithi107.stdout:8/71: write d6/fd [800235,7054] 0 2024-03-04T13:42:55.170 INFO:tasks.workunit.client.1.smithi169.stdout:4/464: symlink d4/d11/d2e/d39/d61/l95 0 2024-03-04T13:42:55.172 INFO:tasks.workunit.client.0.smithi107.stdout:8/72: readlink d6/da/l12 0 2024-03-04T13:42:55.173 INFO:tasks.workunit.client.1.smithi169.stdout:8/549: truncate d7/d10/d22/fa7 1010312 0 2024-03-04T13:42:55.175 INFO:tasks.workunit.client.0.smithi107.stdout:8/73: rename d6/da to d6/da/d15 22 2024-03-04T13:42:55.177 INFO:tasks.workunit.client.1.smithi169.stdout:0/442: link d5/d4c/d65/f92 d5/d9/dc/d12/d15/f9e 0 2024-03-04T13:42:55.179 INFO:tasks.workunit.client.0.smithi107.stdout:0/64: dread da/de/f11 [0,4194304] 0 2024-03-04T13:42:55.181 INFO:tasks.workunit.client.1.smithi169.stdout:5/633: link d1/c68 d1/d16/d2a/d62/cd2 0 2024-03-04T13:42:55.186 INFO:tasks.workunit.client.1.smithi169.stdout:4/465: rmdir d4/d11/d3b/d67 39 2024-03-04T13:42:55.188 INFO:tasks.workunit.client.0.smithi107.stdout:0/65: rename da/dc/dd/l15 to da/dc/l16 0 2024-03-04T13:42:55.189 INFO:tasks.workunit.client.1.smithi169.stdout:6/552: rename d0/d3c/d95/fab to d0/d3c/d40/d52/dac/fc7 0 2024-03-04T13:42:55.194 INFO:tasks.workunit.client.0.smithi107.stdout:8/74: dwrite d6/da/f11 [0,4194304] 0 2024-03-04T13:42:55.456 INFO:tasks.workunit.client.1.smithi169.stdout:3/576: unlink d7/de/d2b/d7d/d85/d18/d21/d3f/d86/f99 0 2024-03-04T13:42:55.460 INFO:tasks.workunit.client.1.smithi169.stdout:7/609: truncate d2/d15/d17/f1d 460023 0 2024-03-04T13:42:55.464 INFO:tasks.workunit.client.1.smithi169.stdout:7/610: write d2/de/fcb [718072,82184] 0 2024-03-04T13:42:55.464 INFO:tasks.workunit.client.0.smithi107.stdout:0/66: mkdir da/dc/dd/d17 0 2024-03-04T13:42:55.465 INFO:tasks.workunit.client.1.smithi169.stdout:0/443: fdatasync d5/d9/dc/d12/d15/d19/f52 0 2024-03-04T13:42:55.469 INFO:tasks.workunit.client.0.smithi107.stdout:2/52: getdents da 0 2024-03-04T13:42:55.470 INFO:tasks.workunit.client.1.smithi169.stdout:2/579: mkdir d1/d5/d22/d9e/dcb 0 2024-03-04T13:42:55.470 INFO:tasks.workunit.client.1.smithi169.stdout:0/444: read d5/f8 [823612,65515] 0 2024-03-04T13:42:55.473 INFO:tasks.workunit.client.0.smithi107.stdout:2/53: truncate f0 1214837 0 2024-03-04T13:42:55.476 INFO:tasks.workunit.client.0.smithi107.stdout:8/75: mkdir d6/d16 0 2024-03-04T13:42:55.478 INFO:tasks.workunit.client.0.smithi107.stdout:8/76: chown d6/l9 7187 1 2024-03-04T13:42:55.479 INFO:tasks.workunit.client.0.smithi107.stdout:0/67: dread da/de/f11 [0,4194304] 0 2024-03-04T13:42:55.481 INFO:tasks.workunit.client.0.smithi107.stdout:8/77: chown d6/fe 14 1 2024-03-04T13:42:55.484 INFO:tasks.workunit.client.0.smithi107.stdout:8/78: fsync d6/fd 0 2024-03-04T13:42:55.485 INFO:tasks.workunit.client.0.smithi107.stdout:2/54: creat da/fd x:0 0 0 2024-03-04T13:42:55.492 INFO:tasks.workunit.client.0.smithi107.stdout:0/68: rename l2 to da/l18 0 2024-03-04T13:42:55.499 INFO:tasks.workunit.client.1.smithi169.stdout:8/550: creat d7/d10/d22/d46/da5/da6/fb4 x:0 0 0 2024-03-04T13:42:55.505 INFO:tasks.workunit.client.0.smithi107.stdout:2/55: symlink da/le 0 2024-03-04T13:42:55.505 INFO:tasks.workunit.client.1.smithi169.stdout:8/551: chown d7/d10/d22/d46/f83 208 1 2024-03-04T13:42:55.514 INFO:tasks.workunit.client.0.smithi107.stdout:0/69: unlink da/dc/f14 0 2024-03-04T13:42:55.515 INFO:tasks.workunit.client.0.smithi107.stdout:8/79: dread d6/da/f11 [0,4194304] 0 2024-03-04T13:42:55.516 INFO:tasks.workunit.client.0.smithi107.stdout:8/80: chown d6/fe 20703464 1 2024-03-04T13:42:55.517 INFO:tasks.workunit.client.1.smithi169.stdout:5/634: symlink d1/d16/dbd/dbb/dcc/ld3 0 2024-03-04T13:42:55.518 INFO:tasks.workunit.client.0.smithi107.stdout:0/70: truncate f0 1577300 0 2024-03-04T13:42:55.519 INFO:tasks.workunit.client.0.smithi107.stdout:8/81: chown d6/lc 44224545 1 2024-03-04T13:42:55.519 INFO:tasks.workunit.client.0.smithi107.stdout:2/56: mkdir da/df 0 2024-03-04T13:42:55.533 INFO:tasks.workunit.client.0.smithi107.stdout:2/57: unlink da/le 0 2024-03-04T13:42:55.535 INFO:tasks.workunit.client.0.smithi107.stdout:8/82: mknod d6/c17 0 2024-03-04T13:42:55.535 INFO:tasks.workunit.client.1.smithi169.stdout:6/553: write d0/d3c/d95/fb3 [575481,7801] 0 2024-03-04T13:42:55.539 INFO:tasks.workunit.client.0.smithi107.stdout:8/83: write f2 [3976020,129496] 0 2024-03-04T13:42:55.540 INFO:tasks.workunit.client.0.smithi107.stdout:0/71: rename da/l12 to da/dc/dd/l19 0 2024-03-04T13:42:55.544 INFO:tasks.workunit.client.1.smithi169.stdout:7/611: truncate d2/dc0/fc7 515314 0 2024-03-04T13:42:55.546 INFO:tasks.workunit.client.1.smithi169.stdout:5/635: rmdir d1/d16/d2f 39 2024-03-04T13:42:55.548 INFO:tasks.workunit.client.1.smithi169.stdout:0/445: creat d5/d9/d3d/d59/d64/d7f/f9f x:0 0 0 2024-03-04T13:42:55.549 INFO:tasks.workunit.client.1.smithi169.stdout:5/636: truncate d1/d81/f8b 490423 0 2024-03-04T13:42:55.550 INFO:tasks.workunit.client.1.smithi169.stdout:2/580: rename d1/d5/d22/d3b/d41/c85 to d1/d5/d19/d36/ccc 0 2024-03-04T13:42:55.553 INFO:tasks.workunit.client.1.smithi169.stdout:2/581: write d1/d5/d19/d1e/f24 [9425215,102289] 0 2024-03-04T13:42:55.554 INFO:tasks.workunit.client.0.smithi107.stdout:0/72: rename da/dc/dd/d17 to da/dc/d1a 0 2024-03-04T13:42:55.555 INFO:tasks.workunit.client.1.smithi169.stdout:3/577: creat d7/de/d2b/d7d/d85/fc4 x:0 0 0 2024-03-04T13:42:55.559 INFO:tasks.workunit.client.1.smithi169.stdout:6/554: symlink d0/d3c/d95/dbd/lc8 0 2024-03-04T13:42:55.560 INFO:tasks.workunit.client.0.smithi107.stdout:0/73: creat da/dc/f1b x:0 0 0 2024-03-04T13:42:55.561 INFO:tasks.workunit.client.1.smithi169.stdout:4/466: truncate d4/d11/d2e/f51 422364 0 2024-03-04T13:42:55.564 INFO:tasks.workunit.client.1.smithi169.stdout:0/446: mknod d5/d9/d3d/d59/d64/d7f/ca0 0 2024-03-04T13:42:55.566 INFO:tasks.workunit.client.0.smithi107.stdout:8/84: dread d6/da/f11 [0,4194304] 0 2024-03-04T13:42:55.567 INFO:tasks.workunit.client.0.smithi107.stdout:2/58: dread f4 [0,4194304] 0 2024-03-04T13:42:55.569 INFO:tasks.workunit.client.0.smithi107.stdout:0/74: write da/de/f11 [1255711,30316] 0 2024-03-04T13:42:55.573 INFO:tasks.workunit.client.1.smithi169.stdout:9/562: sync 2024-03-04T13:42:55.573 INFO:tasks.workunit.client.1.smithi169.stdout:1/526: sync 2024-03-04T13:42:55.574 INFO:tasks.workunit.client.1.smithi169.stdout:5/637: symlink d1/d81/dc1/ld4 0 2024-03-04T13:42:55.582 INFO:tasks.workunit.client.1.smithi169.stdout:7/612: creat d2/d15/fce x:0 0 0 2024-03-04T13:42:55.583 INFO:tasks.workunit.client.0.smithi107.stdout:8/85: mknod d6/c18 0 2024-03-04T13:42:55.583 INFO:tasks.workunit.client.1.smithi169.stdout:6/555: write d0/d32/d3a/d43/d74/f71 [3255832,54532] 0 2024-03-04T13:42:55.585 INFO:tasks.workunit.client.1.smithi169.stdout:3/578: dwrite d7/d53/f8c [0,4194304] 0 2024-03-04T13:42:55.586 INFO:tasks.workunit.client.0.smithi107.stdout:0/75: truncate f8 4790874 0 2024-03-04T13:42:55.587 INFO:tasks.workunit.client.1.smithi169.stdout:2/582: dwrite d1/d5/d22/d3b/f52 [4194304,4194304] 0 2024-03-04T13:42:55.591 INFO:tasks.workunit.client.1.smithi169.stdout:5/638: symlink d1/d76/ld5 0 2024-03-04T13:42:55.592 INFO:tasks.workunit.client.0.smithi107.stdout:8/86: creat d6/da/f19 x:0 0 0 2024-03-04T13:42:55.595 INFO:tasks.workunit.client.1.smithi169.stdout:8/552: dwrite d7/d10/d17/d44/d36/f4a [0,4194304] 0 2024-03-04T13:42:55.595 INFO:tasks.workunit.client.0.smithi107.stdout:0/76: creat da/de/f1c x:0 0 0 2024-03-04T13:42:55.597 INFO:tasks.workunit.client.0.smithi107.stdout:2/59: dwrite da/fd [0,4194304] 0 2024-03-04T13:42:55.599 INFO:tasks.workunit.client.0.smithi107.stdout:2/60: truncate f7 987741 0 2024-03-04T13:42:55.600 INFO:tasks.workunit.client.0.smithi107.stdout:2/61: dread - da/fb zero size 2024-03-04T13:42:55.602 INFO:tasks.workunit.client.1.smithi169.stdout:8/553: chown d7/f6a 67064382 1 2024-03-04T13:42:55.604 INFO:tasks.workunit.client.1.smithi169.stdout:4/467: dwrite d4/fa [4194304,4194304] 0 2024-03-04T13:42:55.606 INFO:tasks.workunit.client.1.smithi169.stdout:7/613: rmdir d2/de/d3e 39 2024-03-04T13:42:55.608 INFO:tasks.workunit.client.1.smithi169.stdout:7/614: stat d2/d15/d54/fac 0 2024-03-04T13:42:55.609 INFO:tasks.workunit.client.0.smithi107.stdout:0/77: unlink c3 0 2024-03-04T13:42:55.614 INFO:tasks.workunit.client.0.smithi107.stdout:0/78: write da/dc/f1b [193722,71983] 0 2024-03-04T13:42:55.615 INFO:tasks.workunit.client.1.smithi169.stdout:1/527: dread d4/d1d/f8e [0,4194304] 0 2024-03-04T13:42:55.617 INFO:tasks.workunit.client.1.smithi169.stdout:6/556: mknod d0/d3c/d40/d52/d98/cc9 0 2024-03-04T13:42:55.618 INFO:tasks.workunit.client.0.smithi107.stdout:8/87: creat d6/d16/f1a x:0 0 0 2024-03-04T13:42:55.620 INFO:tasks.workunit.client.0.smithi107.stdout:8/88: chown d6/da/l12 248 1 2024-03-04T13:42:55.620 INFO:tasks.workunit.client.1.smithi169.stdout:6/557: truncate d0/d32/f89 477760 0 2024-03-04T13:42:55.623 INFO:tasks.workunit.client.1.smithi169.stdout:9/563: dwrite d7/dc/d12/d20/da6/fb4 [0,4194304] 0 2024-03-04T13:42:55.629 INFO:tasks.workunit.client.1.smithi169.stdout:0/447: truncate d5/d9/dc/d12/d15/d19/f20 25282 0 2024-03-04T13:42:55.632 INFO:tasks.workunit.client.1.smithi169.stdout:8/554: symlink d7/d10/d17/d44/lb5 0 2024-03-04T13:42:55.637 INFO:tasks.workunit.client.0.smithi107.stdout:8/89: link d6/fd d6/da/f1b 0 2024-03-04T13:42:55.637 INFO:tasks.workunit.client.0.smithi107.stdout:0/79: dread da/de/f11 [0,4194304] 0 2024-03-04T13:42:55.639 INFO:tasks.workunit.client.1.smithi169.stdout:5/639: mknod d1/cd6 0 2024-03-04T13:42:55.640 INFO:tasks.workunit.client.0.smithi107.stdout:2/62: getdents da 0 2024-03-04T13:42:55.643 INFO:tasks.workunit.client.0.smithi107.stdout:0/80: creat da/dc/f1d x:0 0 0 2024-03-04T13:42:55.644 INFO:tasks.workunit.client.1.smithi169.stdout:9/564: symlink d7/dc/d12/d20/d4e/lbd 0 2024-03-04T13:42:55.645 INFO:tasks.workunit.client.1.smithi169.stdout:9/565: chown d7/d27/c73 7775 1 2024-03-04T13:42:55.646 INFO:tasks.workunit.client.0.smithi107.stdout:2/63: write da/fd [2660995,102746] 0 2024-03-04T13:42:55.648 INFO:tasks.workunit.client.1.smithi169.stdout:0/448: chown d5/d9/dc/d12/f62 531111 1 2024-03-04T13:42:55.649 INFO:tasks.workunit.client.0.smithi107.stdout:8/90: dread d6/ff [0,4194304] 0 2024-03-04T13:42:55.650 INFO:tasks.workunit.client.1.smithi169.stdout:4/468: fsync d4/d11/d2e/f51 0 2024-03-04T13:42:55.650 INFO:tasks.workunit.client.0.smithi107.stdout:8/91: chown d6/da/l12 3475 1 2024-03-04T13:42:55.651 INFO:tasks.workunit.client.0.smithi107.stdout:0/81: mkdir da/dc/dd/d1e 0 2024-03-04T13:42:55.652 INFO:tasks.workunit.client.1.smithi169.stdout:7/615: mkdir d2/de/dcf 0 2024-03-04T13:42:55.653 INFO:tasks.workunit.client.0.smithi107.stdout:2/64: creat da/f10 x:0 0 0 2024-03-04T13:42:55.654 INFO:tasks.workunit.client.1.smithi169.stdout:1/528: mknod d4/d5/d36/d35/d5b/ca7 0 2024-03-04T13:42:55.654 INFO:tasks.workunit.client.1.smithi169.stdout:5/640: dread d1/d16/d17/d1b/f25 [0,4194304] 0 2024-03-04T13:42:55.655 INFO:tasks.workunit.client.1.smithi169.stdout:6/558: mkdir d0/d32/d3e/db9/dca 0 2024-03-04T13:42:55.656 INFO:tasks.workunit.client.1.smithi169.stdout:1/529: fdatasync d4/d5/d36/d35/d8f/d93/f9e 0 2024-03-04T13:42:55.658 INFO:tasks.workunit.client.0.smithi107.stdout:8/92: write d6/ff [1318499,102861] 0 2024-03-04T13:42:55.659 INFO:tasks.workunit.client.0.smithi107.stdout:8/93: stat f5 0 2024-03-04T13:42:55.660 INFO:tasks.workunit.client.1.smithi169.stdout:7/616: dread d2/d28/d70/f79 [0,4194304] 0 2024-03-04T13:42:55.661 INFO:tasks.workunit.client.0.smithi107.stdout:0/82: dread f4 [0,4194304] 0 2024-03-04T13:42:55.662 INFO:tasks.workunit.client.1.smithi169.stdout:0/449: mknod d5/d9/dc/d12/d15/ca1 0 2024-03-04T13:42:55.663 INFO:tasks.workunit.client.1.smithi169.stdout:4/469: symlink d4/d1c/l96 0 2024-03-04T13:42:55.663 INFO:tasks.workunit.client.1.smithi169.stdout:0/450: chown d5/d4c/d65/l86 466887 1 2024-03-04T13:42:55.664 INFO:tasks.workunit.client.1.smithi169.stdout:4/470: fdatasync d4/d11/d2e/d39/f60 0 2024-03-04T13:42:55.665 INFO:tasks.workunit.client.1.smithi169.stdout:3/579: write d7/de/d2b/d7d/d85/d47/d63/d6c/f77 [567648,8001] 0 2024-03-04T13:42:55.666 INFO:tasks.workunit.client.0.smithi107.stdout:0/83: creat da/dc/dd/f1f x:0 0 0 2024-03-04T13:42:55.670 INFO:tasks.workunit.client.1.smithi169.stdout:2/583: getdents d1/d5/d22/d45/d66 0 2024-03-04T13:42:55.672 INFO:tasks.workunit.client.0.smithi107.stdout:0/84: unlink da/dc/f1b 0 2024-03-04T13:42:55.673 INFO:tasks.workunit.client.1.smithi169.stdout:1/530: mkdir d4/d5/d36/d35/df/da8 0 2024-03-04T13:42:55.675 INFO:tasks.workunit.client.1.smithi169.stdout:6/559: rename d0/d32/d3a/d5e/c73 to d0/d32/d3a/d43/ccb 0 2024-03-04T13:42:55.678 INFO:tasks.workunit.client.1.smithi169.stdout:8/555: symlink d7/d10/d17/d44/d36/d62/lb6 0 2024-03-04T13:42:55.680 INFO:tasks.workunit.client.1.smithi169.stdout:7/617: creat d2/de/d37/fd0 x:0 0 0 2024-03-04T13:42:55.683 INFO:tasks.workunit.client.1.smithi169.stdout:4/471: fdatasync d4/f69 0 2024-03-04T13:42:55.685 INFO:tasks.workunit.client.1.smithi169.stdout:5/641: rmdir d1/d16/d2f 39 2024-03-04T13:42:55.686 INFO:tasks.workunit.client.1.smithi169.stdout:5/642: chown d1/d81/db1 0 1 2024-03-04T13:42:55.687 INFO:tasks.workunit.client.1.smithi169.stdout:2/584: creat d1/d5/d22/d45/d4a/fcd x:0 0 0 2024-03-04T13:42:55.688 INFO:tasks.workunit.client.1.smithi169.stdout:1/531: mkdir d4/d5/d36/da9 0 2024-03-04T13:42:55.690 INFO:tasks.workunit.client.1.smithi169.stdout:1/532: readlink d4/d5/d2d/d4e/d71/d26/l5f 0 2024-03-04T13:42:55.694 INFO:tasks.workunit.client.1.smithi169.stdout:0/451: link d5/d9/f42 d5/d9/dc/d12/d15/d19/d77/d94/fa2 0 2024-03-04T13:42:55.695 INFO:tasks.workunit.client.1.smithi169.stdout:4/472: fdatasync d4/f15 0 2024-03-04T13:42:55.698 INFO:tasks.workunit.client.1.smithi169.stdout:5/643: creat d1/d60/fd7 x:0 0 0 2024-03-04T13:42:55.704 INFO:tasks.workunit.client.1.smithi169.stdout:1/533: rename d4/d5/d36/d35/l6a to d4/d5/d2d/d4e/d71/d82/laa 0 2024-03-04T13:42:55.706 INFO:tasks.workunit.client.1.smithi169.stdout:8/556: dwrite d7/d10/d22/d46/f6e [0,4194304] 0 2024-03-04T13:42:55.709 INFO:tasks.workunit.client.0.smithi107.stdout:1/4: sync 2024-03-04T13:42:55.710 INFO:tasks.workunit.client.1.smithi169.stdout:4/473: fdatasync d4/d11/d3b/d67/f84 0 2024-03-04T13:42:55.710 INFO:tasks.workunit.client.1.smithi169.stdout:5/644: creat d1/d16/d17/d28/fd8 x:0 0 0 2024-03-04T13:42:55.715 INFO:tasks.workunit.client.0.smithi107.stdout:8/94: dwrite d6/da/f19 [0,4194304] 0 2024-03-04T13:42:55.718 INFO:tasks.workunit.client.0.smithi107.stdout:8/95: truncate f3 4482246 0 2024-03-04T13:42:55.719 INFO:tasks.workunit.client.0.smithi107.stdout:0/85: dread da/de/f11 [0,4194304] 0 2024-03-04T13:42:55.719 INFO:tasks.workunit.client.0.smithi107.stdout:8/96: stat d6/da 0 2024-03-04T13:42:55.720 INFO:tasks.workunit.client.1.smithi169.stdout:0/452: rename d5/d9/dc/d12/d15/d19/d61/f36 to d5/d9/dc/d12/d15/d19/d77/fa3 0 2024-03-04T13:42:55.720 INFO:tasks.workunit.client.1.smithi169.stdout:9/566: dwrite d7/d25/d3c/f5f [0,4194304] 0 2024-03-04T13:42:55.721 INFO:tasks.workunit.client.1.smithi169.stdout:5/645: dread d1/d16/d17/d1b/f69 [0,4194304] 0 2024-03-04T13:42:55.723 INFO:tasks.workunit.client.0.smithi107.stdout:1/5: getdents d1 0 2024-03-04T13:42:55.725 INFO:tasks.workunit.client.1.smithi169.stdout:0/453: chown d5/d9/dc/d12/d15/d19/d77/f9b 552832 1 2024-03-04T13:42:55.725 INFO:tasks.workunit.client.0.smithi107.stdout:0/86: chown da/dc/dd/ff 8207 1 2024-03-04T13:42:55.725 INFO:tasks.workunit.client.1.smithi169.stdout:3/580: getdents d7/de/d2b/d7d/d85/d13/d1a/d98/d9a 0 2024-03-04T13:42:55.733 INFO:tasks.workunit.client.1.smithi169.stdout:4/474: creat d4/d1c/d58/f97 x:0 0 0 2024-03-04T13:42:55.734 INFO:tasks.workunit.client.0.smithi107.stdout:8/97: dread f2 [0,4194304] 0 2024-03-04T13:42:55.735 INFO:tasks.workunit.client.1.smithi169.stdout:7/618: dwrite d2/d28/fa1 [0,4194304] 0 2024-03-04T13:42:55.736 INFO:tasks.workunit.client.1.smithi169.stdout:6/560: write d0/d3c/d95/dbd/fc6 [4423688,16398] 0 2024-03-04T13:42:55.736 INFO:tasks.workunit.client.0.smithi107.stdout:8/98: write d6/ff [2451749,16880] 0 2024-03-04T13:42:55.737 INFO:tasks.workunit.client.1.smithi169.stdout:9/567: mkdir d7/dc/d12/d32/d42/d4f/dbe 0 2024-03-04T13:42:55.738 INFO:tasks.workunit.client.0.smithi107.stdout:8/99: dread - d6/fe zero size 2024-03-04T13:42:55.738 INFO:tasks.workunit.client.0.smithi107.stdout:1/6: mknod d1/c2 0 2024-03-04T13:42:55.740 INFO:tasks.workunit.client.0.smithi107.stdout:8/100: truncate d6/fe 533467 0 2024-03-04T13:42:55.741 INFO:tasks.workunit.client.1.smithi169.stdout:4/475: truncate d4/d11/d3b/f83 712941 0 2024-03-04T13:42:55.741 INFO:tasks.workunit.client.1.smithi169.stdout:7/619: truncate d2/f22 4905836 0 2024-03-04T13:42:55.741 INFO:tasks.workunit.client.1.smithi169.stdout:9/568: write d7/d2d/f85 [293338,21892] 0 2024-03-04T13:42:55.748 INFO:tasks.workunit.client.0.smithi107.stdout:8/101: fdatasync d6/ff 0 2024-03-04T13:42:55.749 INFO:tasks.workunit.client.1.smithi169.stdout:1/534: link d4/d5/d36/d35/df/f2b d4/d5/d36/d35/d70/fab 0 2024-03-04T13:42:55.749 INFO:tasks.workunit.client.1.smithi169.stdout:2/585: dwrite d1/d5/d19/d63/fae [0,4194304] 0 2024-03-04T13:42:55.755 INFO:tasks.workunit.client.1.smithi169.stdout:4/476: mkdir d4/d11/d3b/d67/d98 0 2024-03-04T13:42:55.756 INFO:tasks.workunit.client.1.smithi169.stdout:6/561: chown d0/d32/d3a/d43/d74/fa0 12768 1 2024-03-04T13:42:55.758 INFO:tasks.workunit.client.1.smithi169.stdout:1/535: symlink d4/d5/d36/d35/d70/lac 0 2024-03-04T13:42:55.758 INFO:tasks.workunit.client.1.smithi169.stdout:5/646: getdents d1/d76 0 2024-03-04T13:42:55.761 INFO:tasks.workunit.client.0.smithi107.stdout:8/102: dread d6/da/f19 [0,4194304] 0 2024-03-04T13:42:55.762 INFO:tasks.workunit.client.1.smithi169.stdout:0/454: getdents d5/d9/dc/d12/d15/d19/d77/d94 0 2024-03-04T13:42:55.762 INFO:tasks.workunit.client.0.smithi107.stdout:8/103: fsync d6/fe 0 2024-03-04T13:42:55.763 INFO:tasks.workunit.client.1.smithi169.stdout:9/569: creat d7/d25/fbf x:0 0 0 2024-03-04T13:42:55.764 INFO:tasks.workunit.client.1.smithi169.stdout:2/586: mkdir d1/d5/d19/d36/d64/dce 0 2024-03-04T13:42:55.764 INFO:tasks.workunit.client.0.smithi107.stdout:1/7: getdents d1 0 2024-03-04T13:42:55.764 INFO:tasks.workunit.client.0.smithi107.stdout:1/8: dread - no filename 2024-03-04T13:42:55.764 INFO:tasks.workunit.client.0.smithi107.stdout:1/9: dwrite - no filename 2024-03-04T13:42:55.765 INFO:tasks.workunit.client.1.smithi169.stdout:6/562: fsync d0/d3c/d40/f4d 0 2024-03-04T13:42:55.765 INFO:tasks.workunit.client.0.smithi107.stdout:8/104: chown f4 872 1 2024-03-04T13:42:55.766 INFO:tasks.workunit.client.1.smithi169.stdout:5/647: fdatasync d1/d16/d17/d2d/d5a/d67/f82 0 2024-03-04T13:42:55.767 INFO:tasks.workunit.client.1.smithi169.stdout:0/455: fdatasync d5/d9/d3d/d5d/f7c 0 2024-03-04T13:42:55.769 INFO:tasks.workunit.client.1.smithi169.stdout:0/456: fdatasync d5/d9/d3d/d59/d64/d7f/f9f 0 2024-03-04T13:42:55.769 INFO:tasks.workunit.client.1.smithi169.stdout:1/536: rename d4/d5/d36/d35/df/f1c to d4/d5/d2d/d4e/d71/d82/fad 0 2024-03-04T13:42:55.771 INFO:tasks.workunit.client.0.smithi107.stdout:1/10: mkdir d1/d3 0 2024-03-04T13:42:55.771 INFO:tasks.workunit.client.0.smithi107.stdout:1/11: rename d1 to d1/d4 22 2024-03-04T13:42:55.772 INFO:tasks.workunit.client.0.smithi107.stdout:1/12: chown d1/d3 72134 1 2024-03-04T13:42:55.773 INFO:tasks.workunit.client.1.smithi169.stdout:0/457: dread - d5/d9/dc/d12/d15/d19/d61/d71/d8c/d90/f97 zero size 2024-03-04T13:42:55.776 INFO:tasks.workunit.client.1.smithi169.stdout:9/570: symlink d7/dc/d12/d20/d4e/lc0 0 2024-03-04T13:42:55.777 INFO:tasks.workunit.client.0.smithi107.stdout:1/13: unlink l0 0 2024-03-04T13:42:55.777 INFO:tasks.workunit.client.0.smithi107.stdout:1/14: fdatasync - no filename 2024-03-04T13:42:55.779 INFO:tasks.workunit.client.1.smithi169.stdout:4/477: dread d4/f7 [0,4194304] 0 2024-03-04T13:42:55.780 INFO:tasks.workunit.client.1.smithi169.stdout:1/537: chown d4/d5/d2d/d4e/d71/d26/f79 1 1 2024-03-04T13:42:55.780 INFO:tasks.workunit.client.0.smithi107.stdout:8/105: getdents d6/d16 0 2024-03-04T13:42:55.782 INFO:tasks.workunit.client.0.smithi107.stdout:1/15: unlink d1/c2 0 2024-03-04T13:42:55.782 INFO:tasks.workunit.client.0.smithi107.stdout:1/16: chown d1 8 1 2024-03-04T13:42:55.785 INFO:tasks.workunit.client.0.smithi107.stdout:8/106: rename d6/ff to d6/d16/f1c 0 2024-03-04T13:42:55.786 INFO:tasks.workunit.client.0.smithi107.stdout:8/107: readlink d6/l9 0 2024-03-04T13:42:55.788 INFO:tasks.workunit.client.1.smithi169.stdout:6/563: link d0/d32/d3a/f42 d0/d32/d70/fcc 0 2024-03-04T13:42:55.789 INFO:tasks.workunit.client.1.smithi169.stdout:6/564: stat d0/d3c/d40/d52/d98/fb6 0 2024-03-04T13:42:55.789 INFO:tasks.workunit.client.1.smithi169.stdout:2/587: creat d1/d5/d19/fcf x:0 0 0 2024-03-04T13:42:55.790 INFO:tasks.workunit.client.0.smithi107.stdout:1/17: creat d1/d3/f5 x:0 0 0 2024-03-04T13:42:55.790 INFO:tasks.workunit.client.0.smithi107.stdout:1/18: readlink - no filename 2024-03-04T13:42:55.791 INFO:tasks.workunit.client.1.smithi169.stdout:4/478: rename d4/d11/d23/f46 to d4/d11/d2e/d37/d5c/d86/f99 0 2024-03-04T13:42:55.793 INFO:tasks.workunit.client.1.smithi169.stdout:9/571: mknod d7/cc1 0 2024-03-04T13:42:55.795 INFO:tasks.workunit.client.1.smithi169.stdout:6/565: unlink d0/d32/d3a/l80 0 2024-03-04T13:42:55.798 INFO:tasks.workunit.client.1.smithi169.stdout:0/458: creat d5/d9/dc/d12/fa4 x:0 0 0 2024-03-04T13:42:55.800 INFO:tasks.workunit.client.1.smithi169.stdout:0/459: write d5/d9/dc/d12/d15/d19/d61/f54 [3366145,18347] 0 2024-03-04T13:42:55.801 INFO:tasks.workunit.client.1.smithi169.stdout:6/566: rename d0/d32/d3a/f2a to d0/d3c/d95/dbd/fcd 0 2024-03-04T13:42:55.804 INFO:tasks.workunit.client.1.smithi169.stdout:9/572: symlink d7/dc/d3e/da3/dbc/lc2 0 2024-03-04T13:42:55.806 INFO:tasks.workunit.client.1.smithi169.stdout:9/573: dread - d7/d25/d3c/f9e zero size 2024-03-04T13:42:55.806 INFO:tasks.workunit.client.1.smithi169.stdout:4/479: symlink d4/d11/d2e/d6c/l9a 0 2024-03-04T13:42:55.808 INFO:tasks.workunit.client.1.smithi169.stdout:9/574: fdatasync d7/d25/d3c/f5f 0 2024-03-04T13:42:55.811 INFO:tasks.workunit.client.1.smithi169.stdout:4/480: dread d4/d11/d2e/d37/d5c/f8c [0,4194304] 0 2024-03-04T13:42:55.813 INFO:tasks.workunit.client.1.smithi169.stdout:4/481: truncate d4/d1c/f50 113586 0 2024-03-04T13:42:55.816 INFO:tasks.workunit.client.1.smithi169.stdout:0/460: unlink d5/d9/dc/d12/d15/d19/d77/fa3 0 2024-03-04T13:42:55.816 INFO:tasks.workunit.client.1.smithi169.stdout:2/588: write d1/d5/d22/d45/d4a/f70 [255400,73153] 0 2024-03-04T13:42:55.819 INFO:tasks.workunit.client.1.smithi169.stdout:6/567: rename d0/d32/d3a/d43/ccb to d0/d3c/d40/d52/d98/cce 0 2024-03-04T13:42:55.819 INFO:tasks.workunit.client.1.smithi169.stdout:9/575: dread d7/dc/d12/d32/d42/f7e [0,4194304] 0 2024-03-04T13:42:55.819 INFO:tasks.workunit.client.0.smithi107.stdout:1/19: dwrite d1/d3/f5 [0,4194304] 0 2024-03-04T13:42:55.828 INFO:tasks.workunit.client.1.smithi169.stdout:2/589: mknod d1/d5/d22/d45/d4a/cd0 0 2024-03-04T13:42:55.829 INFO:tasks.workunit.client.1.smithi169.stdout:0/461: mkdir d5/d9/dc/d12/da5 0 2024-03-04T13:42:55.832 INFO:tasks.workunit.client.1.smithi169.stdout:0/462: chown d5/d4c/f79 1 1 2024-03-04T13:42:55.835 INFO:tasks.workunit.client.1.smithi169.stdout:9/576: dread d7/f71 [4194304,4194304] 0 2024-03-04T13:42:55.839 INFO:tasks.workunit.client.1.smithi169.stdout:6/568: dwrite d0/d32/d3a/f4f [4194304,4194304] 0 2024-03-04T13:42:55.839 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:42:55 smithi107 ceph-mon[28739]: mgrmap e25: smithi169.eihczr(active, since 92s), standbys: smithi107.wpzgwl 2024-03-04T13:42:55.842 INFO:tasks.workunit.client.1.smithi169.stdout:2/590: getdents d1/d5/d19/d36 0 2024-03-04T13:42:55.843 INFO:tasks.workunit.client.1.smithi169.stdout:2/591: fsync d1/d5/fad 0 2024-03-04T13:42:55.844 INFO:tasks.workunit.client.1.smithi169.stdout:9/577: mkdir d7/d27/dc3 0 2024-03-04T13:42:55.847 INFO:tasks.workunit.client.1.smithi169.stdout:6/569: symlink d0/d7a/lcf 0 2024-03-04T13:42:55.848 INFO:tasks.workunit.client.1.smithi169.stdout:4/482: dwrite d4/d1c/f2c [0,4194304] 0 2024-03-04T13:42:55.849 INFO:tasks.workunit.client.0.smithi107.stdout:1/20: fdatasync d1/d3/f5 0 2024-03-04T13:42:55.851 INFO:tasks.workunit.client.1.smithi169.stdout:6/570: write d0/d32/f81 [4939,91680] 0 2024-03-04T13:42:55.851 INFO:tasks.workunit.client.1.smithi169.stdout:2/592: unlink d1/d5/d19/fcf 0 2024-03-04T13:42:55.852 INFO:tasks.workunit.client.1.smithi169.stdout:0/463: creat d5/d9/dc/fa6 x:0 0 0 2024-03-04T13:42:55.853 INFO:tasks.workunit.client.0.smithi107.stdout:1/21: write d1/d3/f5 [2607783,109709] 0 2024-03-04T13:42:55.854 INFO:tasks.workunit.client.1.smithi169.stdout:2/593: stat d1/d5/d22/d45/d4a/lb6 0 2024-03-04T13:42:55.856 INFO:tasks.workunit.client.0.smithi107.stdout:1/22: rename d1 to d1/d3/d6 22 2024-03-04T13:42:55.856 INFO:tasks.workunit.client.1.smithi169.stdout:9/578: rename d7/dc/c1a to d7/dc/d12/d20/d4e/cc4 0 2024-03-04T13:42:55.858 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:42:55 smithi169 ceph-mon[36313]: mgrmap e25: smithi169.eihczr(active, since 92s), standbys: smithi107.wpzgwl 2024-03-04T13:42:55.865 INFO:tasks.workunit.client.1.smithi169.stdout:4/483: mknod d4/d11/d2e/d39/c9b 0 2024-03-04T13:42:55.866 INFO:tasks.workunit.client.1.smithi169.stdout:2/594: creat d1/d6d/dbc/fd1 x:0 0 0 2024-03-04T13:42:55.867 INFO:tasks.workunit.client.1.smithi169.stdout:4/484: chown d4 48 1 2024-03-04T13:42:55.868 INFO:tasks.workunit.client.1.smithi169.stdout:9/579: creat d7/dc/d12/d20/da6/fc5 x:0 0 0 2024-03-04T13:42:55.869 INFO:tasks.workunit.client.1.smithi169.stdout:6/571: link d0/d63/c64 d0/d32/d3a/d5e/cd0 0 2024-03-04T13:42:55.871 INFO:tasks.workunit.client.0.smithi107.stdout:1/23: dread d1/d3/f5 [0,4194304] 0 2024-03-04T13:42:55.871 INFO:tasks.workunit.client.0.smithi107.stdout:1/24: chown d1 11223 1 2024-03-04T13:42:55.876 INFO:tasks.workunit.client.1.smithi169.stdout:6/572: unlink d0/d7a/laa 0 2024-03-04T13:42:55.876 INFO:tasks.workunit.client.1.smithi169.stdout:0/464: dwrite d5/d9/dc/d12/f53 [0,4194304] 0 2024-03-04T13:42:55.876 INFO:tasks.workunit.client.1.smithi169.stdout:2/595: dread d1/d5/d19/d1e/f43 [0,4194304] 0 2024-03-04T13:42:55.881 INFO:tasks.workunit.client.1.smithi169.stdout:4/485: link d4/d1c/f59 d4/d11/d3b/d67/d7e/f9c 0 2024-03-04T13:42:55.887 INFO:tasks.workunit.client.1.smithi169.stdout:9/580: rename d7/d27/d84/c99 to d7/dc/d12/d59/cc6 0 2024-03-04T13:42:55.889 INFO:tasks.workunit.client.1.smithi169.stdout:0/465: symlink d5/d9/d3d/d59/la7 0 2024-03-04T13:42:55.890 INFO:tasks.workunit.client.0.smithi107.stdout:9/4: sync 2024-03-04T13:42:55.891 INFO:tasks.workunit.client.0.smithi107.stdout:9/5: chown . 62 1 2024-03-04T13:42:55.892 INFO:tasks.workunit.client.1.smithi169.stdout:2/596: dread d1/d5/d22/d3b/d41/fba [4194304,4194304] 0 2024-03-04T13:42:55.892 INFO:tasks.workunit.client.1.smithi169.stdout:4/486: mkdir d4/d1c/d58/d9d 0 2024-03-04T13:42:55.898 INFO:tasks.workunit.client.1.smithi169.stdout:9/581: link d7/dc/d3e/fb1 d7/dc/d12/d20/da6/fc7 0 2024-03-04T13:42:55.900 INFO:tasks.workunit.client.1.smithi169.stdout:4/487: symlink d4/d11/d3b/d67/l9e 0 2024-03-04T13:42:55.905 INFO:tasks.workunit.client.1.smithi169.stdout:2/597: creat d1/d5/d22/d3b/fd2 x:0 0 0 2024-03-04T13:42:55.907 INFO:tasks.workunit.client.1.smithi169.stdout:2/598: fsync d1/d5/d19/d63/fbd 0 2024-03-04T13:42:55.908 INFO:tasks.workunit.client.1.smithi169.stdout:0/466: unlink d5/d9/dc/d12/l38 0 2024-03-04T13:42:55.909 INFO:tasks.workunit.client.1.smithi169.stdout:4/488: dread d4/d1c/f59 [0,4194304] 0 2024-03-04T13:42:55.920 INFO:tasks.workunit.client.1.smithi169.stdout:2/599: dread d1/d5/d22/d45/f72 [0,4194304] 0 2024-03-04T13:42:55.927 INFO:tasks.workunit.client.1.smithi169.stdout:9/582: dwrite d7/dc/d12/d20/da6/fc7 [0,4194304] 0 2024-03-04T13:42:55.928 INFO:tasks.workunit.client.1.smithi169.stdout:4/489: rmdir d4/d11/d2e/d21 39 2024-03-04T13:42:55.936 INFO:tasks.workunit.client.1.smithi169.stdout:4/490: write d4/d11/f79 [1149356,73214] 0 2024-03-04T13:42:55.944 INFO:tasks.workunit.client.1.smithi169.stdout:6/573: write d0/d3c/d40/f4d [1256101,43112] 0 2024-03-04T13:42:55.946 INFO:tasks.workunit.client.1.smithi169.stdout:8/557: sync 2024-03-04T13:42:55.961 INFO:tasks.workunit.client.1.smithi169.stdout:2/600: dread d1/d5/d22/d45/f89 [0,4194304] 0 2024-03-04T13:42:55.970 INFO:tasks.workunit.client.0.smithi107.stdout:1/25: mknod d1/d3/c7 0 2024-03-04T13:42:55.974 INFO:tasks.workunit.client.0.smithi107.stdout:9/6: creat f0 x:0 0 0 2024-03-04T13:42:55.979 INFO:tasks.workunit.client.0.smithi107.stdout:9/7: symlink l1 0 2024-03-04T13:42:55.980 INFO:tasks.workunit.client.0.smithi107.stdout:3/22: sync 2024-03-04T13:42:55.980 INFO:tasks.workunit.client.0.smithi107.stdout:5/10: sync 2024-03-04T13:42:55.981 INFO:tasks.workunit.client.0.smithi107.stdout:4/17: sync 2024-03-04T13:42:55.982 INFO:tasks.workunit.client.0.smithi107.stdout:9/8: chown f0 82037 1 2024-03-04T13:42:55.984 INFO:tasks.workunit.client.0.smithi107.stdout:4/18: dread - d0/f3 zero size 2024-03-04T13:42:55.987 INFO:tasks.workunit.client.0.smithi107.stdout:5/11: mknod c4 0 2024-03-04T13:42:55.988 INFO:tasks.workunit.client.0.smithi107.stdout:4/19: dread - d0/f3 zero size 2024-03-04T13:42:55.990 INFO:tasks.workunit.client.0.smithi107.stdout:3/23: symlink d1/l7 0 2024-03-04T13:42:55.990 INFO:tasks.workunit.client.1.smithi169.stdout:4/491: mknod d4/d11/d2e/d37/c9f 0 2024-03-04T13:42:55.993 INFO:tasks.workunit.client.1.smithi169.stdout:4/492: write d4/d11/d2e/d37/f45 [1748177,34103] 0 2024-03-04T13:42:55.993 INFO:tasks.workunit.client.0.smithi107.stdout:9/9: creat f2 x:0 0 0 2024-03-04T13:42:55.995 INFO:tasks.workunit.client.1.smithi169.stdout:4/493: read d4/f9 [996619,66022] 0 2024-03-04T13:42:55.995 INFO:tasks.workunit.client.1.smithi169.stdout:8/558: unlink d7/d10/d24/d5c/l66 0 2024-03-04T13:42:55.995 INFO:tasks.workunit.client.0.smithi107.stdout:5/12: truncate f0 666050 0 2024-03-04T13:42:55.997 INFO:tasks.workunit.client.1.smithi169.stdout:2/601: mknod d1/d5/cd3 0 2024-03-04T13:42:55.997 INFO:tasks.workunit.client.0.smithi107.stdout:3/24: write d1/f3 [1750768,93568] 0 2024-03-04T13:42:55.999 INFO:tasks.workunit.client.0.smithi107.stdout:4/20: mkdir d0/d7 0 2024-03-04T13:42:55.999 INFO:tasks.workunit.client.0.smithi107.stdout:5/13: readlink l2 0 2024-03-04T13:42:56.003 INFO:tasks.workunit.client.0.smithi107.stdout:3/25: fsync d1/f6 0 2024-03-04T13:42:56.003 INFO:tasks.workunit.client.1.smithi169.stdout:6/574: fdatasync d0/d3c/d95/dbd/fcd 0 2024-03-04T13:42:56.004 INFO:tasks.workunit.client.0.smithi107.stdout:5/14: mkdir d5 0 2024-03-04T13:42:56.007 INFO:tasks.workunit.client.1.smithi169.stdout:4/494: creat d4/d11/d3b/d67/fa0 x:0 0 0 2024-03-04T13:42:56.010 INFO:tasks.workunit.client.0.smithi107.stdout:3/26: truncate d1/f6 1509914 0 2024-03-04T13:42:56.011 INFO:tasks.workunit.client.0.smithi107.stdout:4/21: rename d0/l4 to d0/l8 0 2024-03-04T13:42:56.013 INFO:tasks.workunit.client.0.smithi107.stdout:1/26: dread d1/d3/f5 [0,4194304] 0 2024-03-04T13:42:56.016 INFO:tasks.workunit.client.0.smithi107.stdout:5/15: rename f0 to d5/f6 0 2024-03-04T13:42:56.017 INFO:tasks.workunit.client.0.smithi107.stdout:4/22: mkdir d0/d9 0 2024-03-04T13:42:56.018 INFO:tasks.workunit.client.0.smithi107.stdout:1/27: mkdir d1/d8 0 2024-03-04T13:42:56.018 INFO:tasks.workunit.client.0.smithi107.stdout:5/16: write f1 [164584,92644] 0 2024-03-04T13:42:56.020 INFO:tasks.workunit.client.0.smithi107.stdout:5/17: chown d5 3731740 1 2024-03-04T13:42:56.025 INFO:tasks.workunit.client.0.smithi107.stdout:5/18: creat d5/f7 x:0 0 0 2024-03-04T13:42:56.028 INFO:tasks.workunit.client.0.smithi107.stdout:4/23: link d0/f6 d0/d7/fa 0 2024-03-04T13:42:56.032 INFO:tasks.workunit.client.0.smithi107.stdout:4/24: fdatasync d0/f6 0 2024-03-04T13:42:56.036 INFO:tasks.workunit.client.0.smithi107.stdout:9/10: getdents . 0 2024-03-04T13:42:56.038 INFO:tasks.workunit.client.0.smithi107.stdout:8/108: sync 2024-03-04T13:42:56.038 INFO:tasks.workunit.client.0.smithi107.stdout:0/87: sync 2024-03-04T13:42:56.038 INFO:tasks.workunit.client.0.smithi107.stdout:7/54: sync 2024-03-04T13:42:56.039 INFO:tasks.workunit.client.0.smithi107.stdout:2/65: sync 2024-03-04T13:42:56.039 INFO:tasks.workunit.client.0.smithi107.stdout:6/49: sync 2024-03-04T13:42:56.039 INFO:tasks.workunit.client.0.smithi107.stdout:5/19: dread d5/f6 [0,4194304] 0 2024-03-04T13:42:56.043 INFO:tasks.workunit.client.0.smithi107.stdout:3/27: dwrite d1/f3 [0,4194304] 0 2024-03-04T13:42:56.046 INFO:tasks.workunit.client.0.smithi107.stdout:9/11: write f0 [94275,110006] 0 2024-03-04T13:42:56.046 INFO:tasks.workunit.client.0.smithi107.stdout:1/28: write d1/d3/f5 [4276557,20018] 0 2024-03-04T13:42:56.046 INFO:tasks.workunit.client.0.smithi107.stdout:0/88: fdatasync da/de/f1c 0 2024-03-04T13:42:56.046 INFO:tasks.workunit.client.0.smithi107.stdout:8/109: symlink d6/da/l1d 0 2024-03-04T13:42:56.048 INFO:tasks.workunit.client.0.smithi107.stdout:6/50: rmdir df 39 2024-03-04T13:42:56.048 INFO:tasks.workunit.client.0.smithi107.stdout:3/28: stat d1 0 2024-03-04T13:42:56.049 INFO:tasks.workunit.client.0.smithi107.stdout:2/66: mknod da/df/c11 0 2024-03-04T13:42:56.051 INFO:tasks.workunit.client.0.smithi107.stdout:7/55: write d6/fc [4449161,35988] 0 2024-03-04T13:42:56.053 INFO:tasks.workunit.client.0.smithi107.stdout:5/20: dread d5/f6 [0,4194304] 0 2024-03-04T13:42:56.055 INFO:tasks.workunit.client.0.smithi107.stdout:9/12: creat f3 x:0 0 0 2024-03-04T13:42:56.056 INFO:tasks.workunit.client.0.smithi107.stdout:0/89: write da/dc/f1d [722125,7423] 0 2024-03-04T13:42:56.063 INFO:tasks.workunit.client.0.smithi107.stdout:5/21: dread - d5/f7 zero size 2024-03-04T13:42:56.063 INFO:tasks.workunit.client.0.smithi107.stdout:1/29: write d1/d3/f5 [2589975,96067] 0 2024-03-04T13:42:56.064 INFO:tasks.workunit.client.0.smithi107.stdout:9/13: dread - f2 zero size 2024-03-04T13:42:56.064 INFO:tasks.workunit.client.0.smithi107.stdout:7/56: creat d6/f12 x:0 0 0 2024-03-04T13:42:56.064 INFO:tasks.workunit.client.0.smithi107.stdout:8/110: write d6/d16/f1a [339479,127196] 0 2024-03-04T13:42:56.064 INFO:tasks.workunit.client.0.smithi107.stdout:6/51: write df/f15 [180002,106141] 0 2024-03-04T13:42:56.068 INFO:tasks.workunit.client.1.smithi169.stdout:2/602: dwrite d1/d6d/f87 [4194304,4194304] 0 2024-03-04T13:42:56.068 INFO:tasks.workunit.client.1.smithi169.stdout:8/559: rmdir d7/d10/d24/d5c 39 2024-03-04T13:42:56.071 INFO:tasks.workunit.client.0.smithi107.stdout:5/22: mkdir d5/d8 0 2024-03-04T13:42:56.071 INFO:tasks.workunit.client.1.smithi169.stdout:6/575: truncate d0/d3c/f8b 274167 0 2024-03-04T13:42:56.077 INFO:tasks.workunit.client.1.smithi169.stdout:8/560: mknod d7/d10/d22/d75/d81/cb7 0 2024-03-04T13:42:56.081 INFO:tasks.workunit.client.1.smithi169.stdout:6/576: creat d0/d32/d70/fd1 x:0 0 0 2024-03-04T13:42:56.082 INFO:tasks.workunit.client.0.smithi107.stdout:6/52: dread - df/f11 zero size 2024-03-04T13:42:56.083 INFO:tasks.workunit.client.1.smithi169.stdout:8/561: write d7/d10/d17/d44/d4c/fa3 [918273,85069] 0 2024-03-04T13:42:56.085 INFO:tasks.workunit.client.1.smithi169.stdout:2/603: truncate d1/d5/d19/d1e/f5c 1913091 0 2024-03-04T13:42:56.085 INFO:tasks.workunit.client.1.smithi169.stdout:9/583: dwrite d7/dc/d3e/f41 [0,4194304] 0 2024-03-04T13:42:56.089 INFO:tasks.workunit.client.1.smithi169.stdout:2/604: dread d1/d5/d19/d69/f7e [4194304,4194304] 0 2024-03-04T13:42:56.089 INFO:tasks.workunit.client.1.smithi169.stdout:4/495: link d4/d11/d2e/d37/d5c/l8e d4/d11/d3b/la1 0 2024-03-04T13:42:56.091 INFO:tasks.workunit.client.1.smithi169.stdout:2/605: readlink d1/d5/d22/d3b/lb2 0 2024-03-04T13:42:56.092 INFO:tasks.workunit.client.1.smithi169.stdout:4/496: read d4/d11/d2e/d39/f60 [5032606,28385] 0 2024-03-04T13:42:56.092 INFO:tasks.workunit.client.1.smithi169.stdout:8/562: truncate d7/d10/d24/f50 1319784 0 2024-03-04T13:42:56.094 INFO:tasks.workunit.client.1.smithi169.stdout:9/584: creat d7/dc/d12/d20/d4e/fc8 x:0 0 0 2024-03-04T13:42:56.094 INFO:tasks.workunit.client.1.smithi169.stdout:8/563: chown d7/d10/d24/f85 38 1 2024-03-04T13:42:56.095 INFO:tasks.workunit.client.1.smithi169.stdout:6/577: mknod d0/d32/d3a/d43/cd2 0 2024-03-04T13:42:56.098 INFO:tasks.workunit.client.0.smithi107.stdout:7/57: write f5 [609487,95063] 0 2024-03-04T13:42:56.098 INFO:tasks.workunit.client.1.smithi169.stdout:8/564: stat d7/d10/d17/c60 0 2024-03-04T13:42:56.098 INFO:tasks.workunit.client.1.smithi169.stdout:2/606: mknod d1/d5/d22/d45/d66/cd4 0 2024-03-04T13:42:56.098 INFO:tasks.workunit.client.1.smithi169.stdout:2/607: fsync d1/d5/d22/d45/d4a/f7d 0 2024-03-04T13:42:56.099 INFO:tasks.workunit.client.1.smithi169.stdout:4/497: symlink d4/la2 0 2024-03-04T13:42:56.099 INFO:tasks.workunit.client.1.smithi169.stdout:8/565: read - d7/d10/d17/d44/d36/d62/fab zero size 2024-03-04T13:42:56.099 INFO:tasks.workunit.client.1.smithi169.stdout:6/578: creat d0/d32/d3a/d43/d74/fd3 x:0 0 0 2024-03-04T13:42:56.099 INFO:tasks.workunit.client.0.smithi107.stdout:3/29: truncate d1/f3 3188009 0 2024-03-04T13:42:56.101 INFO:tasks.workunit.client.1.smithi169.stdout:6/579: read d0/d32/f81 [63598,25395] 0 2024-03-04T13:42:56.104 INFO:tasks.workunit.client.1.smithi169.stdout:6/580: write d0/f29 [3020105,75037] 0 2024-03-04T13:42:56.107 INFO:tasks.workunit.client.0.smithi107.stdout:6/53: mkdir df/d17 0 2024-03-04T13:42:56.108 INFO:tasks.workunit.client.1.smithi169.stdout:8/566: chown d7/d10/d22/d75/d81/cb7 3274920 1 2024-03-04T13:42:56.110 INFO:tasks.workunit.client.0.smithi107.stdout:4/25: dwrite d0/f6 [0,4194304] 0 2024-03-04T13:42:56.111 INFO:tasks.workunit.client.0.smithi107.stdout:3/30: chown d1/f6 112 1 2024-03-04T13:42:56.115 INFO:tasks.workunit.client.1.smithi169.stdout:9/585: dwrite d7/d25/d3c/f9e [0,4194304] 0 2024-03-04T13:42:56.115 INFO:tasks.workunit.client.0.smithi107.stdout:6/54: chown c7 1594 1 2024-03-04T13:42:56.119 INFO:tasks.workunit.client.0.smithi107.stdout:8/111: dread d6/d16/f1c [0,4194304] 0 2024-03-04T13:42:56.121 INFO:tasks.workunit.client.1.smithi169.stdout:2/608: dwrite d1/d5/d22/d45/d4a/f7d [0,4194304] 0 2024-03-04T13:42:56.122 INFO:tasks.workunit.client.0.smithi107.stdout:6/55: read - df/f11 zero size 2024-03-04T13:42:56.122 INFO:tasks.workunit.client.0.smithi107.stdout:7/58: mkdir d6/d13 0 2024-03-04T13:42:56.123 INFO:tasks.workunit.client.1.smithi169.stdout:8/567: creat d7/d10/d22/d75/d81/da2/dac/fb8 x:0 0 0 2024-03-04T13:42:56.128 INFO:tasks.workunit.client.0.smithi107.stdout:0/90: dread f8 [0,4194304] 0 2024-03-04T13:42:56.129 INFO:tasks.workunit.client.1.smithi169.stdout:2/609: chown d1/d5/d22/d45/d4a/lbf 2 1 2024-03-04T13:42:56.129 INFO:tasks.workunit.client.1.smithi169.stdout:9/586: creat d7/dc/d3e/da3/fc9 x:0 0 0 2024-03-04T13:42:56.131 INFO:tasks.workunit.client.0.smithi107.stdout:4/26: creat d0/fb x:0 0 0 2024-03-04T13:42:56.134 INFO:tasks.workunit.client.0.smithi107.stdout:2/67: dwrite da/fb [0,4194304] 0 2024-03-04T13:42:56.137 INFO:tasks.workunit.client.1.smithi169.stdout:8/568: mkdir d7/d10/d22/d75/db9 0 2024-03-04T13:42:56.142 INFO:tasks.workunit.client.1.smithi169.stdout:9/587: truncate d7/d2d/fb0 881716 0 2024-03-04T13:42:56.142 INFO:tasks.workunit.client.1.smithi169.stdout:4/498: dwrite d4/f7 [0,4194304] 0 2024-03-04T13:42:56.142 INFO:tasks.workunit.client.1.smithi169.stdout:8/569: write d7/f6a [172071,11253] 0 2024-03-04T13:42:56.143 INFO:tasks.workunit.client.0.smithi107.stdout:3/31: rename d1/l4 to d1/l8 0 2024-03-04T13:42:56.143 INFO:tasks.workunit.client.0.smithi107.stdout:1/30: dread d1/d3/f5 [0,4194304] 0 2024-03-04T13:42:56.144 INFO:tasks.workunit.client.0.smithi107.stdout:9/14: dwrite f0 [0,4194304] 0 2024-03-04T13:42:56.144 INFO:tasks.workunit.client.1.smithi169.stdout:2/610: mkdir d1/d5/d19/d36/d64/dce/dd5 0 2024-03-04T13:42:56.148 INFO:tasks.workunit.client.0.smithi107.stdout:2/68: symlink da/df/l12 0 2024-03-04T13:42:56.149 INFO:tasks.workunit.client.1.smithi169.stdout:8/570: chown d7/d10/d17/d44/d4c/d93 14890543 1 2024-03-04T13:42:56.149 INFO:tasks.workunit.client.1.smithi169.stdout:6/581: dwrite d0/d3c/d95/dbd/fc5 [0,4194304] 0 2024-03-04T13:42:56.153 INFO:tasks.workunit.client.0.smithi107.stdout:5/23: dwrite f1 [0,4194304] 0 2024-03-04T13:42:56.157 INFO:tasks.workunit.client.0.smithi107.stdout:2/69: chown da/fd 11523314 1 2024-03-04T13:42:56.157 INFO:tasks.workunit.client.0.smithi107.stdout:5/24: fsync d5/f7 0 2024-03-04T13:42:56.157 INFO:tasks.workunit.client.1.smithi169.stdout:3/581: sync 2024-03-04T13:42:56.158 INFO:tasks.workunit.client.0.smithi107.stdout:9/15: write f0 [4057141,14027] 0 2024-03-04T13:42:56.162 INFO:tasks.workunit.client.1.smithi169.stdout:8/571: creat d7/d10/d22/d75/db9/fba x:0 0 0 2024-03-04T13:42:56.164 INFO:tasks.workunit.client.1.smithi169.stdout:6/582: creat d0/d32/d3a/d5e/fd4 x:0 0 0 2024-03-04T13:42:56.165 INFO:tasks.workunit.client.0.smithi107.stdout:9/16: write f3 [581094,123108] 0 2024-03-04T13:42:56.165 INFO:tasks.workunit.client.1.smithi169.stdout:4/499: truncate d4/d11/d2e/d37/d5c/f8c 988797 0 2024-03-04T13:42:56.166 INFO:tasks.workunit.client.0.smithi107.stdout:9/17: stat l1 0 2024-03-04T13:42:56.166 INFO:tasks.workunit.client.0.smithi107.stdout:6/56: mknod df/d17/c18 0 2024-03-04T13:42:56.168 INFO:tasks.workunit.client.0.smithi107.stdout:3/32: creat d1/f9 x:0 0 0 2024-03-04T13:42:56.169 INFO:tasks.workunit.client.1.smithi169.stdout:3/582: symlink d7/de/d2b/d7d/d85/d13/d1a/d2d/d56/d5e/d67/lc5 0 2024-03-04T13:42:56.173 INFO:tasks.workunit.client.1.smithi169.stdout:3/583: chown d7/de/d2b/d7d/d85/d13/d5a/f83 1665 1 2024-03-04T13:42:56.173 INFO:tasks.workunit.client.0.smithi107.stdout:5/25: creat d5/f9 x:0 0 0 2024-03-04T13:42:56.173 INFO:tasks.workunit.client.0.smithi107.stdout:9/18: write f2 [950980,96751] 0 2024-03-04T13:42:56.175 INFO:tasks.workunit.client.0.smithi107.stdout:8/112: dread f3 [0,4194304] 0 2024-03-04T13:42:56.177 INFO:tasks.workunit.client.0.smithi107.stdout:2/70: creat da/df/f13 x:0 0 0 2024-03-04T13:42:56.178 INFO:tasks.workunit.client.1.smithi169.stdout:3/584: rmdir d7/de/d2b/d7d/d85/d47/d63 39 2024-03-04T13:42:56.179 INFO:tasks.workunit.client.0.smithi107.stdout:7/59: dwrite d6/f12 [0,4194304] 0 2024-03-04T13:42:56.179 INFO:tasks.workunit.client.0.smithi107.stdout:4/27: mkdir d0/d7/dc 0 2024-03-04T13:42:56.181 INFO:tasks.workunit.client.1.smithi169.stdout:8/572: link d7/d10/d17/f98 d7/d10/d17/d44/d5d/fbb 0 2024-03-04T13:42:56.183 INFO:tasks.workunit.client.0.smithi107.stdout:6/57: fdatasync f8 0 2024-03-04T13:42:56.184 INFO:tasks.workunit.client.1.smithi169.stdout:4/500: rename d4/d11/l4f to d4/d11/la3 0 2024-03-04T13:42:56.186 INFO:tasks.workunit.client.1.smithi169.stdout:6/583: getdents d0/d32/d3a/d43/d74 0 2024-03-04T13:42:56.187 INFO:tasks.workunit.client.1.smithi169.stdout:4/501: write d4/d1c/f2c [246128,110086] 0 2024-03-04T13:42:56.190 INFO:tasks.workunit.client.1.smithi169.stdout:3/585: creat d7/db8/fc6 x:0 0 0 2024-03-04T13:42:56.191 INFO:tasks.workunit.client.0.smithi107.stdout:8/113: creat d6/da/f1e x:0 0 0 2024-03-04T13:42:56.193 INFO:tasks.workunit.client.0.smithi107.stdout:1/31: rmdir d1/d8 0 2024-03-04T13:42:56.195 INFO:tasks.workunit.client.0.smithi107.stdout:0/91: link da/l18 da/dc/d1a/l20 0 2024-03-04T13:42:56.195 INFO:tasks.workunit.client.1.smithi169.stdout:8/573: symlink d7/d10/d22/d75/lbc 0 2024-03-04T13:42:56.196 INFO:tasks.workunit.client.1.smithi169.stdout:6/584: rmdir d0/d32/d3a/d43/d74/d93 39 2024-03-04T13:42:56.197 INFO:tasks.workunit.client.0.smithi107.stdout:6/58: creat df/f19 x:0 0 0 2024-03-04T13:42:56.198 INFO:tasks.workunit.client.0.smithi107.stdout:6/59: truncate fc 438000 0 2024-03-04T13:42:56.198 INFO:tasks.workunit.client.0.smithi107.stdout:2/71: symlink da/l14 0 2024-03-04T13:42:56.199 INFO:tasks.workunit.client.1.smithi169.stdout:8/574: stat d7/d10/d22/d46/l8e 0 2024-03-04T13:42:56.202 INFO:tasks.workunit.client.1.smithi169.stdout:4/502: symlink d4/d11/d2e/d63/la4 0 2024-03-04T13:42:56.202 INFO:tasks.workunit.client.1.smithi169.stdout:2/611: write d1/d5/d22/d3b/f4d [623660,69410] 0 2024-03-04T13:42:56.205 INFO:tasks.workunit.client.0.smithi107.stdout:9/19: dread f2 [0,4194304] 0 2024-03-04T13:42:56.207 INFO:tasks.workunit.client.1.smithi169.stdout:4/503: write d4/d11/d2e/d63/f8a [2108866,13748] 0 2024-03-04T13:42:56.207 INFO:tasks.workunit.client.0.smithi107.stdout:5/26: mkdir d5/d8/da 0 2024-03-04T13:42:56.209 INFO:tasks.workunit.client.0.smithi107.stdout:7/60: mkdir d6/d13/d14 0 2024-03-04T13:42:56.210 INFO:tasks.workunit.client.0.smithi107.stdout:8/114: mkdir d6/d16/d1f 0 2024-03-04T13:42:56.212 INFO:tasks.workunit.client.1.smithi169.stdout:7/620: sync 2024-03-04T13:42:56.214 INFO:tasks.workunit.client.0.smithi107.stdout:1/32: rmdir d1/d3 39 2024-03-04T13:42:56.215 INFO:tasks.workunit.client.1.smithi169.stdout:9/588: dread d7/dc/f1e [0,4194304] 0 2024-03-04T13:42:56.216 INFO:tasks.workunit.client.1.smithi169.stdout:8/575: symlink d7/d10/d22/d75/lbd 0 2024-03-04T13:42:56.217 INFO:tasks.workunit.client.1.smithi169.stdout:3/586: dwrite d7/de/d2b/d7d/d85/d18/d21/d3f/d68/fb2 [0,4194304] 0 2024-03-04T13:42:56.218 INFO:tasks.workunit.client.0.smithi107.stdout:9/20: dread f3 [0,4194304] 0 2024-03-04T13:42:56.219 INFO:tasks.workunit.client.0.smithi107.stdout:2/72: creat da/df/f15 x:0 0 0 2024-03-04T13:42:56.222 INFO:tasks.workunit.client.0.smithi107.stdout:6/60: mknod df/d17/c1a 0 2024-03-04T13:42:56.223 INFO:tasks.workunit.client.1.smithi169.stdout:8/576: write d7/f8b [3873392,78279] 0 2024-03-04T13:42:56.223 INFO:tasks.workunit.client.1.smithi169.stdout:9/589: write d7/d25/d64/fb7 [771439,78683] 0 2024-03-04T13:42:56.223 INFO:tasks.workunit.client.1.smithi169.stdout:4/504: rename d4/d11/d23 to d4/d11/d2e/d6c/d78/da5 0 2024-03-04T13:42:56.224 INFO:tasks.workunit.client.0.smithi107.stdout:2/73: write da/fd [4712750,98594] 0 2024-03-04T13:42:56.226 INFO:tasks.workunit.client.1.smithi169.stdout:4/505: fdatasync d4/f5a 0 2024-03-04T13:42:56.233 INFO:tasks.workunit.client.0.smithi107.stdout:0/92: rename l5 to da/l21 0 2024-03-04T13:42:56.234 INFO:tasks.workunit.client.1.smithi169.stdout:3/587: dread d7/de/d2b/d7d/d85/d47/f9d [0,4194304] 0 2024-03-04T13:42:56.239 INFO:tasks.workunit.client.0.smithi107.stdout:4/28: dwrite d0/f3 [0,4194304] 0 2024-03-04T13:42:56.240 INFO:tasks.workunit.client.0.smithi107.stdout:9/21: mkdir d4 0 2024-03-04T13:42:56.241 INFO:tasks.workunit.client.0.smithi107.stdout:1/33: chown d1/d3/c7 1002 1 2024-03-04T13:42:56.241 INFO:tasks.workunit.client.1.smithi169.stdout:9/590: creat d7/dc/d12/d20/d4e/fca x:0 0 0 2024-03-04T13:42:56.242 INFO:tasks.workunit.client.0.smithi107.stdout:5/27: dwrite d5/f9 [0,4194304] 0 2024-03-04T13:42:56.247 INFO:tasks.workunit.client.1.smithi169.stdout:4/506: rmdir d4/d11/d2e/d39/d61/d64 39 2024-03-04T13:42:56.249 INFO:tasks.workunit.client.0.smithi107.stdout:3/33: getdents d1 0 2024-03-04T13:42:56.250 INFO:tasks.workunit.client.1.smithi169.stdout:7/621: rename d2/d15/d17/c97 to d2/de/d3e/d41/cd1 0 2024-03-04T13:42:56.254 INFO:tasks.workunit.client.0.smithi107.stdout:2/74: symlink da/df/l16 0 2024-03-04T13:42:56.258 INFO:tasks.workunit.client.0.smithi107.stdout:8/115: mknod d6/c20 0 2024-03-04T13:42:56.258 INFO:tasks.workunit.client.0.smithi107.stdout:4/29: fsync d0/f3 0 2024-03-04T13:42:56.259 INFO:tasks.workunit.client.1.smithi169.stdout:8/577: dwrite d7/d10/d24/d3e/f45 [0,4194304] 0 2024-03-04T13:42:56.260 INFO:tasks.workunit.client.1.smithi169.stdout:2/612: link d1/d5/d19/d68/cc5 d1/d5/d19/d36/daf/cd6 0 2024-03-04T13:42:56.261 INFO:tasks.workunit.client.1.smithi169.stdout:4/507: stat d4/d11/f1b 0 2024-03-04T13:42:56.262 INFO:tasks.workunit.client.0.smithi107.stdout:0/93: mknod da/dc/c22 0 2024-03-04T13:42:56.263 INFO:tasks.workunit.client.0.smithi107.stdout:1/34: dread d1/d3/f5 [0,4194304] 0 2024-03-04T13:42:56.265 INFO:tasks.workunit.client.1.smithi169.stdout:7/622: truncate d2/d28/d70/f79 137021 0 2024-03-04T13:42:56.266 INFO:tasks.workunit.client.0.smithi107.stdout:9/22: write f3 [383874,44065] 0 2024-03-04T13:42:56.271 INFO:tasks.workunit.client.1.smithi169.stdout:8/578: readlink l4 0 2024-03-04T13:42:56.271 INFO:tasks.workunit.client.0.smithi107.stdout:5/28: creat d5/d8/fb x:0 0 0 2024-03-04T13:42:56.276 INFO:tasks.workunit.client.0.smithi107.stdout:2/75: mknod da/df/c17 0 2024-03-04T13:42:56.276 INFO:tasks.workunit.client.0.smithi107.stdout:7/61: truncate d6/f12 3699878 0 2024-03-04T13:42:56.278 INFO:tasks.workunit.client.1.smithi169.stdout:7/623: symlink d2/de/d3e/d41/d5b/ld2 0 2024-03-04T13:42:56.278 INFO:tasks.workunit.client.0.smithi107.stdout:2/76: dread - da/df/f15 zero size 2024-03-04T13:42:56.280 INFO:tasks.workunit.client.0.smithi107.stdout:4/30: dread d0/d7/fa [0,4194304] 0 2024-03-04T13:42:56.281 INFO:tasks.workunit.client.0.smithi107.stdout:8/116: unlink d6/da/c10 0 2024-03-04T13:42:56.281 INFO:tasks.workunit.client.0.smithi107.stdout:2/77: readlink da/df/l16 0 2024-03-04T13:42:56.282 INFO:tasks.workunit.client.1.smithi169.stdout:8/579: chown d7/d10/l39 67 1 2024-03-04T13:42:56.284 INFO:tasks.workunit.client.0.smithi107.stdout:9/23: dread f0 [0,4194304] 0 2024-03-04T13:42:56.285 INFO:tasks.workunit.client.1.smithi169.stdout:3/588: getdents d7/de/d2b/d7d/d85/d18/d21 0 2024-03-04T13:42:56.286 INFO:tasks.workunit.client.0.smithi107.stdout:6/61: truncate fb 1150870 0 2024-03-04T13:42:56.286 INFO:tasks.workunit.client.0.smithi107.stdout:9/24: chown f0 1 1 2024-03-04T13:42:56.287 INFO:tasks.workunit.client.1.smithi169.stdout:4/508: symlink d4/d11/d2e/d21/la6 0 2024-03-04T13:42:56.288 INFO:tasks.workunit.client.1.smithi169.stdout:2/613: dwrite d1/d5/d22/d45/f8f [4194304,4194304] 0 2024-03-04T13:42:56.288 INFO:tasks.workunit.client.1.smithi169.stdout:9/591: dread d7/dc/d3e/f6c [0,4194304] 0 2024-03-04T13:42:56.289 INFO:tasks.workunit.client.0.smithi107.stdout:5/29: dwrite d5/f9 [0,4194304] 0 2024-03-04T13:42:56.289 INFO:tasks.workunit.client.0.smithi107.stdout:5/30: chown c4 1276555830 1 2024-03-04T13:42:56.297 INFO:tasks.workunit.client.1.smithi169.stdout:8/580: mkdir d7/d10/d22/d75/db9/dbe 0 2024-03-04T13:42:56.297 INFO:tasks.workunit.client.1.smithi169.stdout:3/589: truncate d7/f49 1627329 0 2024-03-04T13:42:56.300 INFO:tasks.workunit.client.0.smithi107.stdout:7/62: symlink d6/d13/l15 0 2024-03-04T13:42:56.302 INFO:tasks.workunit.client.1.smithi169.stdout:2/614: symlink d1/d5/d19/d3c/db8/ld7 0 2024-03-04T13:42:56.304 INFO:tasks.workunit.client.0.smithi107.stdout:8/117: write f3 [769005,112247] 0 2024-03-04T13:42:56.305 INFO:tasks.workunit.client.0.smithi107.stdout:0/94: dwrite da/de/f11 [0,4194304] 0 2024-03-04T13:42:56.305 INFO:tasks.workunit.client.0.smithi107.stdout:4/31: symlink d0/ld 0 2024-03-04T13:42:56.306 INFO:tasks.workunit.client.1.smithi169.stdout:3/590: chown d7/de/d2b/d7d/d85/d47/d63 21552 1 2024-03-04T13:42:56.308 INFO:tasks.workunit.client.0.smithi107.stdout:8/118: read f5 [562619,47191] 0 2024-03-04T13:42:56.308 INFO:tasks.workunit.client.1.smithi169.stdout:8/581: creat d7/d10/d22/d75/d81/da2/fbf x:0 0 0 2024-03-04T13:42:56.309 INFO:tasks.workunit.client.1.smithi169.stdout:4/509: dwrite d4/d11/d2e/d37/f45 [0,4194304] 0 2024-03-04T13:42:56.310 INFO:tasks.workunit.client.0.smithi107.stdout:4/32: write d0/f3 [2492594,49016] 0 2024-03-04T13:42:56.311 INFO:tasks.workunit.client.0.smithi107.stdout:8/119: chown f2 19179825 1 2024-03-04T13:42:56.311 INFO:tasks.workunit.client.0.smithi107.stdout:6/62: mkdir df/d1b 0 2024-03-04T13:42:56.316 INFO:tasks.workunit.client.0.smithi107.stdout:9/25: unlink f2 0 2024-03-04T13:42:56.317 INFO:tasks.workunit.client.1.smithi169.stdout:4/510: chown d4/d11/d2e/d63 359 1 2024-03-04T13:42:56.317 INFO:tasks.workunit.client.0.smithi107.stdout:8/120: truncate d6/d16/f1c 2609388 0 2024-03-04T13:42:56.319 INFO:tasks.workunit.client.0.smithi107.stdout:5/31: mkdir d5/dc 0 2024-03-04T13:42:56.320 INFO:tasks.workunit.client.1.smithi169.stdout:4/511: read d4/d1c/f59 [288284,32416] 0 2024-03-04T13:42:56.321 INFO:tasks.workunit.client.0.smithi107.stdout:1/35: dwrite d1/d3/f5 [0,4194304] 0 2024-03-04T13:42:56.323 INFO:tasks.workunit.client.1.smithi169.stdout:4/512: write d4/f9 [1513285,38786] 0 2024-03-04T13:42:56.323 INFO:tasks.workunit.client.0.smithi107.stdout:5/32: chown f1 0 1 2024-03-04T13:42:56.325 INFO:tasks.workunit.client.0.smithi107.stdout:5/33: chown d5/f7 9718 1 2024-03-04T13:42:56.327 INFO:tasks.workunit.client.1.smithi169.stdout:9/592: link d7/dc/f1e d7/dc/d12/d20/da6/fcb 0 2024-03-04T13:42:56.328 INFO:tasks.workunit.client.0.smithi107.stdout:3/34: truncate d1/f3 2697016 0 2024-03-04T13:42:56.329 INFO:tasks.workunit.client.0.smithi107.stdout:1/36: dread d1/d3/f5 [4194304,4194304] 0 2024-03-04T13:42:56.329 INFO:tasks.workunit.client.0.smithi107.stdout:1/37: readlink - no filename 2024-03-04T13:42:56.331 INFO:tasks.workunit.client.0.smithi107.stdout:7/63: mkdir d6/d13/d16 0 2024-03-04T13:42:56.333 INFO:tasks.workunit.client.0.smithi107.stdout:1/38: write d1/d3/f5 [4825816,16029] 0 2024-03-04T13:42:56.334 INFO:tasks.workunit.client.0.smithi107.stdout:7/64: dread - d6/fe zero size 2024-03-04T13:42:56.335 INFO:tasks.workunit.client.0.smithi107.stdout:2/78: link da/f10 da/f18 0 2024-03-04T13:42:56.337 INFO:tasks.workunit.client.1.smithi169.stdout:7/624: dwrite d2/de/d3e/d41/f5a [0,4194304] 0 2024-03-04T13:42:56.337 INFO:tasks.workunit.client.1.smithi169.stdout:3/591: dwrite d7/f25 [4194304,4194304] 0 2024-03-04T13:42:56.338 INFO:tasks.workunit.client.0.smithi107.stdout:6/63: dwrite f8 [0,4194304] 0 2024-03-04T13:42:56.340 INFO:tasks.workunit.client.0.smithi107.stdout:1/39: chown d1/d3 904689 1 2024-03-04T13:42:56.340 INFO:tasks.workunit.client.0.smithi107.stdout:0/95: fdatasync f8 0 2024-03-04T13:42:56.345 INFO:tasks.workunit.client.1.smithi169.stdout:9/593: symlink d7/d27/lcc 0 2024-03-04T13:42:56.345 INFO:tasks.workunit.client.0.smithi107.stdout:0/96: truncate da/dc/dd/f1f 189764 0 2024-03-04T13:42:56.348 INFO:tasks.workunit.client.0.smithi107.stdout:8/121: symlink d6/d16/l21 0 2024-03-04T13:42:56.349 INFO:tasks.workunit.client.1.smithi169.stdout:7/625: mkdir d2/de/d3e/d41/d5b/d7a/dd3 0 2024-03-04T13:42:56.350 INFO:tasks.workunit.client.1.smithi169.stdout:2/615: dwrite d1/d5/d19/d69/f7e [0,4194304] 0 2024-03-04T13:42:56.352 INFO:tasks.workunit.client.1.smithi169.stdout:7/626: readlink d2/de/d3e/d41/d5b/l5c 0 2024-03-04T13:42:56.354 INFO:tasks.workunit.client.1.smithi169.stdout:7/627: fdatasync d2/d15/fce 0 2024-03-04T13:42:56.358 INFO:tasks.workunit.client.1.smithi169.stdout:9/594: symlink d7/lcd 0 2024-03-04T13:42:56.360 INFO:tasks.workunit.client.1.smithi169.stdout:9/595: chown d7/d2d/f85 50007 1 2024-03-04T13:42:56.362 INFO:tasks.workunit.client.0.smithi107.stdout:3/35: creat d1/fa x:0 0 0 2024-03-04T13:42:56.363 INFO:tasks.workunit.client.1.smithi169.stdout:3/592: link d7/de/d2b/d7d/d85/d47/d63/d6c/f77 d7/de/d2b/d7d/d85/d18/d21/d3f/fc7 0 2024-03-04T13:42:56.364 INFO:tasks.workunit.client.0.smithi107.stdout:4/33: mkdir d0/d9/de 0 2024-03-04T13:42:56.364 INFO:tasks.workunit.client.0.smithi107.stdout:7/65: mkdir d6/d17 0 2024-03-04T13:42:56.364 INFO:tasks.workunit.client.0.smithi107.stdout:8/122: dread d6/d16/f1a [0,4194304] 0 2024-03-04T13:42:56.367 INFO:tasks.workunit.client.1.smithi169.stdout:7/628: creat d2/de/d37/fd4 x:0 0 0 2024-03-04T13:42:56.368 INFO:tasks.workunit.client.0.smithi107.stdout:4/34: truncate d0/fb 317648 0 2024-03-04T13:42:56.369 INFO:tasks.workunit.client.0.smithi107.stdout:8/123: chown d6/c17 9944 1 2024-03-04T13:42:56.369 INFO:tasks.workunit.client.0.smithi107.stdout:9/26: mknod d4/c5 0 2024-03-04T13:42:56.369 INFO:tasks.workunit.client.0.smithi107.stdout:4/35: readlink d0/ld 0 2024-03-04T13:42:56.371 INFO:tasks.workunit.client.0.smithi107.stdout:5/34: symlink d5/d8/da/ld 0 2024-03-04T13:42:56.373 INFO:tasks.workunit.client.1.smithi169.stdout:9/596: rmdir d7/dc/d12/d32/d42/d4f/dbe 0 2024-03-04T13:42:56.373 INFO:tasks.workunit.client.1.smithi169.stdout:3/593: creat d7/de/d2b/d7d/d85/d13/d1a/d2d/d56/d5e/d5f/fc8 x:0 0 0 2024-03-04T13:42:56.374 INFO:tasks.workunit.client.1.smithi169.stdout:7/629: mkdir d2/d28/db8/dd5 0 2024-03-04T13:42:56.375 INFO:tasks.workunit.client.0.smithi107.stdout:5/35: write d5/d8/fb [252784,35109] 0 2024-03-04T13:42:56.375 INFO:tasks.workunit.client.1.smithi169.stdout:7/630: stat d2/d28/d70/f98 0 2024-03-04T13:42:56.376 INFO:tasks.workunit.client.0.smithi107.stdout:3/36: mknod d1/cb 0 2024-03-04T13:42:56.378 INFO:tasks.workunit.client.1.smithi169.stdout:9/597: creat d7/dc/d12/d20/d4e/fce x:0 0 0 2024-03-04T13:42:56.380 INFO:tasks.workunit.client.1.smithi169.stdout:0/467: sync 2024-03-04T13:42:56.380 INFO:tasks.workunit.client.1.smithi169.stdout:1/538: sync 2024-03-04T13:42:56.380 INFO:tasks.workunit.client.1.smithi169.stdout:5/648: sync 2024-03-04T13:42:56.382 INFO:tasks.workunit.client.1.smithi169.stdout:7/631: mkdir d2/de/d3e/d41/d89/dd6 0 2024-03-04T13:42:56.383 INFO:tasks.workunit.client.1.smithi169.stdout:5/649: stat d1/d81/f8b 0 2024-03-04T13:42:56.383 INFO:tasks.workunit.client.1.smithi169.stdout:3/594: truncate d7/de/d2b/d7d/d85/d13/f15 140825 0 2024-03-04T13:42:56.383 INFO:tasks.workunit.client.0.smithi107.stdout:8/124: dread d6/d16/f1a [0,4194304] 0 2024-03-04T13:42:56.386 INFO:tasks.workunit.client.1.smithi169.stdout:1/539: write d4/d5/d36/d35/df/f28 [4477615,37618] 0 2024-03-04T13:42:56.387 INFO:tasks.workunit.client.0.smithi107.stdout:8/125: write d6/fe [209475,76779] 0 2024-03-04T13:42:56.389 INFO:tasks.workunit.client.0.smithi107.stdout:5/36: fsync d5/f6 0 2024-03-04T13:42:56.394 INFO:tasks.workunit.client.1.smithi169.stdout:5/650: chown d1/d81/f8b 77600 1 2024-03-04T13:42:56.395 INFO:tasks.workunit.client.0.smithi107.stdout:9/27: dread f0 [0,4194304] 0 2024-03-04T13:42:56.396 INFO:tasks.workunit.client.0.smithi107.stdout:9/28: fdatasync f3 0 2024-03-04T13:42:56.397 INFO:tasks.workunit.client.0.smithi107.stdout:0/97: link da/dc/l13 da/l23 0 2024-03-04T13:42:56.397 INFO:tasks.workunit.client.0.smithi107.stdout:9/29: readlink l1 0 2024-03-04T13:42:56.398 INFO:tasks.workunit.client.0.smithi107.stdout:9/30: rename d4 to d4/d6 22 2024-03-04T13:42:56.399 INFO:tasks.workunit.client.1.smithi169.stdout:9/598: rename d7/dc/d3e/c74 to d7/dc/d12/d59/ccf 0 2024-03-04T13:42:56.400 INFO:tasks.workunit.client.0.smithi107.stdout:6/64: fdatasync f8 0 2024-03-04T13:42:56.400 INFO:tasks.workunit.client.0.smithi107.stdout:4/36: mkdir d0/d7/dc/df 0 2024-03-04T13:42:56.401 INFO:tasks.workunit.client.0.smithi107.stdout:6/65: dread - df/f11 zero size 2024-03-04T13:42:56.403 INFO:tasks.workunit.client.1.smithi169.stdout:9/599: write d7/dc/d12/d20/d4e/fca [1037960,82691] 0 2024-03-04T13:42:56.403 INFO:tasks.workunit.client.1.smithi169.stdout:0/468: creat d5/d9/dc/d12/d15/d19/d61/d71/fa8 x:0 0 0 2024-03-04T13:42:56.405 INFO:tasks.workunit.client.1.smithi169.stdout:3/595: mknod d7/de/d2b/d7d/d85/d13/d1a/d2d/d56/d5e/d5f/cc9 0 2024-03-04T13:42:56.409 INFO:tasks.workunit.client.1.smithi169.stdout:7/632: dwrite d2/de/ff [4194304,4194304] 0 2024-03-04T13:42:56.409 INFO:tasks.workunit.client.0.smithi107.stdout:1/40: getdents d1 0 2024-03-04T13:42:56.410 INFO:tasks.workunit.client.0.smithi107.stdout:2/79: write f4 [4121472,55186] 0 2024-03-04T13:42:56.414 INFO:tasks.workunit.client.0.smithi107.stdout:4/37: dread d0/d7/fa [0,4194304] 0 2024-03-04T13:42:56.415 INFO:tasks.workunit.client.0.smithi107.stdout:5/37: creat d5/d8/fe x:0 0 0 2024-03-04T13:42:56.416 INFO:tasks.workunit.client.0.smithi107.stdout:2/80: write f2 [5914220,31786] 0 2024-03-04T13:42:56.418 INFO:tasks.workunit.client.1.smithi169.stdout:0/469: fdatasync d5/d9/dc/d12/f62 0 2024-03-04T13:42:56.419 INFO:tasks.workunit.client.1.smithi169.stdout:8/582: dwrite d7/d10/d22/f30 [0,4194304] 0 2024-03-04T13:42:56.419 INFO:tasks.workunit.client.1.smithi169.stdout:2/616: dread d1/d5/d19/d68/f88 [0,4194304] 0 2024-03-04T13:42:56.425 INFO:tasks.workunit.client.0.smithi107.stdout:0/98: creat da/dc/dd/f24 x:0 0 0 2024-03-04T13:42:56.426 INFO:tasks.workunit.client.1.smithi169.stdout:2/617: fdatasync d1/d5/d22/d45/f8f 0 2024-03-04T13:42:56.427 INFO:tasks.workunit.client.0.smithi107.stdout:9/31: fsync f0 0 2024-03-04T13:42:56.428 INFO:tasks.workunit.client.0.smithi107.stdout:2/81: truncate da/df/f15 281895 0 2024-03-04T13:42:56.429 INFO:tasks.workunit.client.1.smithi169.stdout:7/633: creat d2/d6b/fd7 x:0 0 0 2024-03-04T13:42:56.429 INFO:tasks.workunit.client.0.smithi107.stdout:6/66: dwrite fc [0,4194304] 0 2024-03-04T13:42:56.432 INFO:tasks.workunit.client.1.smithi169.stdout:7/634: read - d2/de/d37/fd4 zero size 2024-03-04T13:42:56.433 INFO:tasks.workunit.client.1.smithi169.stdout:2/618: write d1/f75 [3029389,52309] 0 2024-03-04T13:42:56.439 INFO:tasks.workunit.client.0.smithi107.stdout:0/99: truncate da/de/f1c 436487 0 2024-03-04T13:42:56.444 INFO:tasks.workunit.client.1.smithi169.stdout:5/651: dread d1/f24 [4194304,4194304] 0 2024-03-04T13:42:56.446 INFO:tasks.workunit.client.0.smithi107.stdout:1/41: write d1/d3/f5 [3938697,13972] 0 2024-03-04T13:42:56.446 INFO:tasks.workunit.client.0.smithi107.stdout:3/37: rmdir d1 39 2024-03-04T13:42:56.448 INFO:tasks.workunit.client.1.smithi169.stdout:0/470: dwrite d5/d9/dc/d12/fa4 [0,4194304] 0 2024-03-04T13:42:56.452 INFO:tasks.workunit.client.0.smithi107.stdout:0/100: write da/dc/dd/ff [648127,23212] 0 2024-03-04T13:42:56.453 INFO:tasks.workunit.client.1.smithi169.stdout:9/600: dread d7/f7a [0,4194304] 0 2024-03-04T13:42:56.453 INFO:tasks.workunit.client.0.smithi107.stdout:9/32: mknod d4/c7 0 2024-03-04T13:42:56.456 INFO:tasks.workunit.client.1.smithi169.stdout:1/540: link d4/f25 d4/d5/d36/d35/fae 0 2024-03-04T13:42:56.458 INFO:tasks.workunit.client.0.smithi107.stdout:2/82: dread f4 [0,4194304] 0 2024-03-04T13:42:56.460 INFO:tasks.workunit.client.1.smithi169.stdout:1/541: fdatasync d4/d5/d2d/d4e/d71/f37 0 2024-03-04T13:42:56.464 INFO:tasks.workunit.client.1.smithi169.stdout:3/596: dread d7/de/d2b/f6e [0,4194304] 0 2024-03-04T13:42:56.465 INFO:tasks.workunit.client.0.smithi107.stdout:4/38: dwrite d0/fb [0,4194304] 0 2024-03-04T13:42:56.466 INFO:tasks.workunit.client.1.smithi169.stdout:5/652: mknod d1/d16/d2a/cd9 0 2024-03-04T13:42:56.466 INFO:tasks.workunit.client.1.smithi169.stdout:3/597: dread - d7/de/d2b/d7d/d85/fc4 zero size 2024-03-04T13:42:56.468 INFO:tasks.workunit.client.1.smithi169.stdout:0/471: creat d5/d9/d3d/d59/d64/d7f/fa9 x:0 0 0 2024-03-04T13:42:56.472 INFO:tasks.workunit.client.0.smithi107.stdout:9/33: creat d4/f8 x:0 0 0 2024-03-04T13:42:56.475 INFO:tasks.workunit.client.0.smithi107.stdout:7/66: dwrite f4 [4194304,4194304] 0 2024-03-04T13:42:56.475 INFO:tasks.workunit.client.0.smithi107.stdout:0/101: creat da/de/f25 x:0 0 0 2024-03-04T13:42:56.475 INFO:tasks.workunit.client.1.smithi169.stdout:9/601: symlink d7/dc/d12/d20/da6/ld0 0 2024-03-04T13:42:56.478 INFO:tasks.workunit.client.0.smithi107.stdout:0/102: truncate da/dc/dd/f1f 486763 0 2024-03-04T13:42:56.478 INFO:tasks.workunit.client.1.smithi169.stdout:5/653: dread d1/d16/d17/d2d/d51/fa8 [0,4194304] 0 2024-03-04T13:42:56.479 INFO:tasks.workunit.client.0.smithi107.stdout:6/67: dread f8 [0,4194304] 0 2024-03-04T13:42:56.480 INFO:tasks.workunit.client.1.smithi169.stdout:2/619: dwrite d1/d5/d22/d45/fc4 [0,4194304] 0 2024-03-04T13:42:56.481 INFO:tasks.workunit.client.0.smithi107.stdout:8/126: fsync d6/da/f1b 0 2024-03-04T13:42:56.486 INFO:tasks.workunit.client.0.smithi107.stdout:4/39: write d0/d7/fa [1863609,69700] 0 2024-03-04T13:42:56.487 INFO:tasks.workunit.client.0.smithi107.stdout:4/40: chown d0 62055 1 2024-03-04T13:42:56.488 INFO:tasks.workunit.client.1.smithi169.stdout:7/635: creat d2/d15/fd8 x:0 0 0 2024-03-04T13:42:56.492 INFO:tasks.workunit.client.1.smithi169.stdout:0/472: dwrite d5/d9/dc/fa6 [0,4194304] 0 2024-03-04T13:42:56.494 INFO:tasks.workunit.client.0.smithi107.stdout:2/83: dread f2 [0,4194304] 0 2024-03-04T13:42:56.494 INFO:tasks.workunit.client.0.smithi107.stdout:7/67: creat d6/d13/f18 x:0 0 0 2024-03-04T13:42:56.497 INFO:tasks.workunit.client.0.smithi107.stdout:2/84: read - da/df/f13 zero size 2024-03-04T13:42:56.498 INFO:tasks.workunit.client.0.smithi107.stdout:2/85: truncate da/df/f15 424966 0 2024-03-04T13:42:56.499 INFO:tasks.workunit.client.0.smithi107.stdout:9/34: creat d4/f9 x:0 0 0 2024-03-04T13:42:56.502 INFO:tasks.workunit.client.0.smithi107.stdout:9/35: truncate d4/f9 483273 0 2024-03-04T13:42:56.502 INFO:tasks.workunit.client.1.smithi169.stdout:1/542: mkdir d4/d5/d2d/d4e/d71/d26/daf 0 2024-03-04T13:42:56.503 INFO:tasks.workunit.client.1.smithi169.stdout:8/583: dwrite d7/d10/d17/d44/d5d/f88 [0,4194304] 0 2024-03-04T13:42:56.510 INFO:tasks.workunit.client.0.smithi107.stdout:6/68: rename df/f11 to df/d1b/f1c 0 2024-03-04T13:42:56.511 INFO:tasks.workunit.client.0.smithi107.stdout:6/69: fdatasync f0 0 2024-03-04T13:42:56.512 INFO:tasks.workunit.client.1.smithi169.stdout:2/620: creat d1/d5/d19/d1e/d29/fd8 x:0 0 0 2024-03-04T13:42:56.513 INFO:tasks.workunit.client.0.smithi107.stdout:6/70: chown fa 1 1 2024-03-04T13:42:56.514 INFO:tasks.workunit.client.0.smithi107.stdout:6/71: chown df/d17 31 1 2024-03-04T13:42:56.515 INFO:tasks.workunit.client.0.smithi107.stdout:4/41: mkdir d0/d9/d10 0 2024-03-04T13:42:56.521 INFO:tasks.workunit.client.0.smithi107.stdout:3/38: link d1/f9 d1/fc 0 2024-03-04T13:42:56.525 INFO:tasks.workunit.client.1.smithi169.stdout:5/654: dwrite d1/d16/d17/d28/f4c [0,4194304] 0 2024-03-04T13:42:56.526 INFO:tasks.workunit.client.0.smithi107.stdout:2/86: mknod da/df/c19 0 2024-03-04T13:42:56.526 INFO:tasks.workunit.client.0.smithi107.stdout:3/39: chown d1/l2 12 1 2024-03-04T13:42:56.530 INFO:tasks.workunit.client.0.smithi107.stdout:1/42: truncate d1/d3/f5 3752612 0 2024-03-04T13:42:56.531 INFO:tasks.workunit.client.0.smithi107.stdout:3/40: chown d1/cb 2096 1 2024-03-04T13:42:56.534 INFO:tasks.workunit.client.0.smithi107.stdout:0/103: rename da/dc/d1a to da/de/d26 0 2024-03-04T13:42:56.537 INFO:tasks.workunit.client.1.smithi169.stdout:2/621: mkdir d1/d5/d19/d68/dd9 0 2024-03-04T13:42:56.538 INFO:tasks.workunit.client.0.smithi107.stdout:7/68: truncate d6/f12 3605257 0 2024-03-04T13:42:56.539 INFO:tasks.workunit.client.1.smithi169.stdout:7/636: fdatasync d2/dc0/fc7 0 2024-03-04T13:42:56.542 INFO:tasks.workunit.client.0.smithi107.stdout:3/41: creat d1/fd x:0 0 0 2024-03-04T13:42:56.544 INFO:tasks.workunit.client.0.smithi107.stdout:9/36: dwrite d4/f8 [0,4194304] 0 2024-03-04T13:42:56.545 INFO:tasks.workunit.client.1.smithi169.stdout:2/622: mkdir d1/d5/d19/d36/d64/dce/dda 0 2024-03-04T13:42:56.547 INFO:tasks.workunit.client.1.smithi169.stdout:2/623: fdatasync d1/d5/d19/d3c/f5f 0 2024-03-04T13:42:56.550 INFO:tasks.workunit.client.1.smithi169.stdout:8/584: creat d7/d10/d22/d46/fc0 x:0 0 0 2024-03-04T13:42:56.552 INFO:tasks.workunit.client.1.smithi169.stdout:1/543: rename d4/d5/d36/d35/df/l7c to d4/d5/d36/d35/lb0 0 2024-03-04T13:42:56.555 INFO:tasks.workunit.client.0.smithi107.stdout:3/42: mkdir d1/de 0 2024-03-04T13:42:56.556 INFO:tasks.workunit.client.0.smithi107.stdout:4/42: dwrite d0/d7/fa [0,4194304] 0 2024-03-04T13:42:56.562 INFO:tasks.workunit.client.1.smithi169.stdout:0/473: getdents d5/d9/dc/d12/d15/d19/d77 0 2024-03-04T13:42:56.563 INFO:tasks.workunit.client.1.smithi169.stdout:2/624: dwrite d1/d5/d19/d3c/f5f [0,4194304] 0 2024-03-04T13:42:56.564 INFO:tasks.workunit.client.0.smithi107.stdout:2/87: rename c8 to da/df/c1a 0 2024-03-04T13:42:56.565 INFO:tasks.workunit.client.0.smithi107.stdout:6/72: link fb df/d1b/f1d 0 2024-03-04T13:42:56.568 INFO:tasks.workunit.client.0.smithi107.stdout:7/69: dwrite d6/fa [0,4194304] 0 2024-03-04T13:42:56.569 INFO:tasks.workunit.client.0.smithi107.stdout:4/43: write d0/fb [2891149,97574] 0 2024-03-04T13:42:56.570 INFO:tasks.workunit.client.1.smithi169.stdout:5/655: getdents d1/d16/d2a/d62 0 2024-03-04T13:42:56.570 INFO:tasks.workunit.client.0.smithi107.stdout:7/70: fdatasync d6/f7 0 2024-03-04T13:42:56.577 INFO:tasks.workunit.client.1.smithi169.stdout:5/656: write d1/d16/d17/d1b/d74/fac [385743,50242] 0 2024-03-04T13:42:56.577 INFO:tasks.workunit.client.1.smithi169.stdout:1/544: mknod d4/d5/cb1 0 2024-03-04T13:42:56.577 INFO:tasks.workunit.client.0.smithi107.stdout:2/88: mknod da/c1b 0 2024-03-04T13:42:56.579 INFO:tasks.workunit.client.0.smithi107.stdout:9/37: dread f0 [0,4194304] 0 2024-03-04T13:42:56.580 INFO:tasks.workunit.client.1.smithi169.stdout:5/657: write d1/d16/d17/d2d/d5a/d8a/fa5 [791834,67257] 0 2024-03-04T13:42:56.580 INFO:tasks.workunit.client.0.smithi107.stdout:6/73: rename fc to df/d17/f1e 0 2024-03-04T13:42:56.583 INFO:tasks.workunit.client.1.smithi169.stdout:3/598: truncate d7/de/d2b/d7d/d85/f4f 3881674 0 2024-03-04T13:42:56.584 INFO:tasks.workunit.client.1.smithi169.stdout:7/637: link d2/de/d3e/d41/cd1 d2/dc0/cd9 0 2024-03-04T13:42:56.587 INFO:tasks.workunit.client.0.smithi107.stdout:0/104: rmdir da/de/d26 39 2024-03-04T13:42:56.589 INFO:tasks.workunit.client.1.smithi169.stdout:0/474: dwrite d5/d9/d3d/d59/d64/d7f/fa9 [0,4194304] 0 2024-03-04T13:42:56.591 INFO:tasks.workunit.client.0.smithi107.stdout:3/43: dread d1/f6 [0,4194304] 0 2024-03-04T13:42:56.593 INFO:tasks.workunit.client.1.smithi169.stdout:1/545: fsync d4/d1d/f40 0 2024-03-04T13:42:56.593 INFO:tasks.workunit.client.1.smithi169.stdout:2/625: symlink d1/d5/d22/d9e/dcb/ldb 0 2024-03-04T13:42:56.596 INFO:tasks.workunit.client.1.smithi169.stdout:2/626: readlink d1/d5/d19/d1e/d29/l2f 0 2024-03-04T13:42:56.599 INFO:tasks.workunit.client.1.smithi169.stdout:2/627: stat d1/d5/d22/d3b/lb2 0 2024-03-04T13:42:56.599 INFO:tasks.workunit.client.0.smithi107.stdout:9/38: rmdir d4 39 2024-03-04T13:42:56.601 INFO:tasks.workunit.client.1.smithi169.stdout:5/658: mknod d1/d16/d2a/d46/d9e/cda 0 2024-03-04T13:42:56.601 INFO:tasks.workunit.client.0.smithi107.stdout:9/39: stat f3 0 2024-03-04T13:42:56.603 INFO:tasks.workunit.client.0.smithi107.stdout:2/89: creat da/f1c x:0 0 0 2024-03-04T13:42:56.603 INFO:tasks.workunit.client.1.smithi169.stdout:5/659: chown d1/d16/d17/d28/f41 310 1 2024-03-04T13:42:56.604 INFO:tasks.workunit.client.1.smithi169.stdout:7/638: creat d2/de/d3e/d41/d69/d78/fda x:0 0 0 2024-03-04T13:42:56.605 INFO:tasks.workunit.client.1.smithi169.stdout:6/585: sync 2024-03-04T13:42:56.607 INFO:tasks.workunit.client.0.smithi107.stdout:7/71: mkdir d6/d13/d16/d19 0 2024-03-04T13:42:56.608 INFO:tasks.workunit.client.1.smithi169.stdout:7/639: read d2/de/d3e/d41/d69/d78/fb4 [3555454,83711] 0 2024-03-04T13:42:56.609 INFO:tasks.workunit.client.1.smithi169.stdout:3/599: rename d7/de/d2b/d7d/d85/d18/d21/d3f/d86/faf to d7/de/d2b/d7d/d85/d18/d21/fca 0 2024-03-04T13:42:56.610 INFO:tasks.workunit.client.1.smithi169.stdout:7/640: truncate d2/f7 4607737 0 2024-03-04T13:42:56.612 INFO:tasks.workunit.client.1.smithi169.stdout:1/546: creat d4/d5/d2d/d4e/d71/d82/fb2 x:0 0 0 2024-03-04T13:42:56.613 INFO:tasks.workunit.client.0.smithi107.stdout:0/105: dwrite da/de/f11 [4194304,4194304] 0 2024-03-04T13:42:56.616 INFO:tasks.workunit.client.1.smithi169.stdout:5/660: creat d1/d81/fdb x:0 0 0 2024-03-04T13:42:56.619 INFO:tasks.workunit.client.1.smithi169.stdout:5/661: dread - d1/d16/d17/d28/fd8 zero size 2024-03-04T13:42:56.619 INFO:tasks.workunit.client.1.smithi169.stdout:8/585: write d7/d10/d24/f50 [1450296,13161] 0 2024-03-04T13:42:56.620 INFO:tasks.workunit.client.0.smithi107.stdout:3/44: write d1/f6 [452646,116464] 0 2024-03-04T13:42:56.620 INFO:tasks.workunit.client.1.smithi169.stdout:6/586: creat d0/d3c/d40/d52/d98/fd5 x:0 0 0 2024-03-04T13:42:56.622 INFO:tasks.workunit.client.0.smithi107.stdout:3/45: write d1/fa [596212,113417] 0 2024-03-04T13:42:56.626 INFO:tasks.workunit.client.1.smithi169.stdout:0/475: rename d5/d4c/c76 to d5/d9/dc/d12/d15/d46/caa 0 2024-03-04T13:42:56.627 INFO:tasks.workunit.client.0.smithi107.stdout:1/43: write d1/d3/f5 [1815773,76087] 0 2024-03-04T13:42:56.628 INFO:tasks.workunit.client.0.smithi107.stdout:2/90: mknod da/df/c1d 0 2024-03-04T13:42:56.637 INFO:tasks.workunit.client.1.smithi169.stdout:2/628: unlink d1/d5/d19/d36/daf/fb3 0 2024-03-04T13:42:56.639 INFO:tasks.workunit.client.1.smithi169.stdout:5/662: mkdir d1/d16/d17/d1b/d74/ddc 0 2024-03-04T13:42:56.640 INFO:tasks.workunit.client.1.smithi169.stdout:5/663: dread - d1/d16/d2a/fcd zero size 2024-03-04T13:42:56.641 INFO:tasks.workunit.client.0.smithi107.stdout:0/106: creat da/f27 x:0 0 0 2024-03-04T13:42:56.642 INFO:tasks.workunit.client.0.smithi107.stdout:4/44: truncate d0/fb 2169077 0 2024-03-04T13:42:56.643 INFO:tasks.workunit.client.0.smithi107.stdout:9/40: mkdir d4/da 0 2024-03-04T13:42:56.645 INFO:tasks.workunit.client.0.smithi107.stdout:9/41: stat d4/c5 0 2024-03-04T13:42:56.646 INFO:tasks.workunit.client.1.smithi169.stdout:0/476: creat d5/d9/dc/d12/d15/d19/d77/d94/fab x:0 0 0 2024-03-04T13:42:56.647 INFO:tasks.workunit.client.0.smithi107.stdout:1/44: mkdir d1/d3/d9 0 2024-03-04T13:42:56.648 INFO:tasks.workunit.client.1.smithi169.stdout:9/602: sync 2024-03-04T13:42:56.648 INFO:tasks.workunit.client.1.smithi169.stdout:4/513: sync 2024-03-04T13:42:56.648 INFO:tasks.workunit.client.1.smithi169.stdout:7/641: symlink d2/ldb 0 2024-03-04T13:42:56.650 INFO:tasks.workunit.client.0.smithi107.stdout:2/91: creat da/df/f1e x:0 0 0 2024-03-04T13:42:56.651 INFO:tasks.workunit.client.1.smithi169.stdout:7/642: write d2/d15/fb9 [928597,1358] 0 2024-03-04T13:42:56.651 INFO:tasks.workunit.client.0.smithi107.stdout:2/92: chown da/f18 1 1 2024-03-04T13:42:56.652 INFO:tasks.workunit.client.1.smithi169.stdout:3/600: creat d7/de/d2b/d7d/d85/d13/d1a/d2d/dbe/fcb x:0 0 0 2024-03-04T13:42:56.653 INFO:tasks.workunit.client.1.smithi169.stdout:3/601: write f6 [2353679,18133] 0 2024-03-04T13:42:56.654 INFO:tasks.workunit.client.0.smithi107.stdout:3/46: dread d1/f3 [0,4194304] 0 2024-03-04T13:42:56.654 INFO:tasks.workunit.client.1.smithi169.stdout:1/547: creat d4/d5/d2d/d4e/d71/d26/d4f/fb3 x:0 0 0 2024-03-04T13:42:56.655 INFO:tasks.workunit.client.0.smithi107.stdout:6/74: truncate fd 558035 0 2024-03-04T13:42:56.658 INFO:tasks.workunit.client.0.smithi107.stdout:4/45: dread d0/f3 [0,4194304] 0 2024-03-04T13:42:56.658 INFO:tasks.workunit.client.1.smithi169.stdout:2/629: rename d1/d5/d19/d1e/d29/ca2 to d1/d5/d19/d36/cdc 0 2024-03-04T13:42:56.659 INFO:tasks.workunit.client.0.smithi107.stdout:3/47: truncate d1/f9 480078 0 2024-03-04T13:42:56.661 INFO:tasks.workunit.client.0.smithi107.stdout:0/107: creat da/dc/f28 x:0 0 0 2024-03-04T13:42:56.661 INFO:tasks.workunit.client.1.smithi169.stdout:8/586: mkdir d7/d10/d17/d44/d36/d62/dc1 0 2024-03-04T13:42:56.663 INFO:tasks.workunit.client.0.smithi107.stdout:3/48: write d1/f6 [1371752,98396] 0 2024-03-04T13:42:56.664 INFO:tasks.workunit.client.1.smithi169.stdout:0/477: dwrite d5/d9/dc/d12/d15/d19/d61/f2c [0,4194304] 0 2024-03-04T13:42:56.666 INFO:tasks.workunit.client.0.smithi107.stdout:2/93: fsync da/fb 0 2024-03-04T13:42:56.668 INFO:tasks.workunit.client.1.smithi169.stdout:9/603: unlink d7/d25/d64/f98 0 2024-03-04T13:42:56.671 INFO:tasks.workunit.client.0.smithi107.stdout:6/75: symlink df/d17/l1f 0 2024-03-04T13:42:56.672 INFO:tasks.workunit.client.0.smithi107.stdout:1/45: dwrite d1/d3/f5 [0,4194304] 0 2024-03-04T13:42:56.675 INFO:tasks.workunit.client.1.smithi169.stdout:1/548: mkdir d4/d5/d36/d35/d16/d81/db4 0 2024-03-04T13:42:56.677 INFO:tasks.workunit.client.0.smithi107.stdout:6/76: write fe [995432,9866] 0 2024-03-04T13:42:56.679 INFO:tasks.workunit.client.1.smithi169.stdout:4/514: rename d4/la2 to d4/d11/d2e/d21/la7 0 2024-03-04T13:42:56.680 INFO:tasks.workunit.client.1.smithi169.stdout:1/549: dread - d4/d5/d2d/d4e/fa6 zero size 2024-03-04T13:42:56.684 INFO:tasks.workunit.client.0.smithi107.stdout:7/72: truncate f5 258829 0 2024-03-04T13:42:56.685 INFO:tasks.workunit.client.0.smithi107.stdout:2/94: rename da/df/c1d to da/df/c1f 0 2024-03-04T13:42:56.685 INFO:tasks.workunit.client.1.smithi169.stdout:5/664: write d1/f4a [174752,97187] 0 2024-03-04T13:42:56.686 INFO:tasks.workunit.client.0.smithi107.stdout:1/46: mknod d1/d3/ca 0 2024-03-04T13:42:56.688 INFO:tasks.workunit.client.0.smithi107.stdout:2/95: chown c9 531 1 2024-03-04T13:42:56.689 INFO:tasks.workunit.client.0.smithi107.stdout:4/46: dwrite d0/f3 [0,4194304] 0 2024-03-04T13:42:56.690 INFO:tasks.workunit.client.0.smithi107.stdout:0/108: mkdir da/de/d26/d29 0 2024-03-04T13:42:56.691 INFO:tasks.workunit.client.1.smithi169.stdout:8/587: chown d7/d10/d24/d5c 2 1 2024-03-04T13:42:56.692 INFO:tasks.workunit.client.0.smithi107.stdout:3/49: mknod d1/de/cf 0 2024-03-04T13:42:56.693 INFO:tasks.workunit.client.0.smithi107.stdout:1/47: write d1/d3/f5 [5112181,42870] 0 2024-03-04T13:42:56.693 INFO:tasks.workunit.client.1.smithi169.stdout:8/588: chown d7/d10/d22/d46/da5 1793010309 1 2024-03-04T13:42:56.694 INFO:tasks.workunit.client.1.smithi169.stdout:9/604: fdatasync d7/d25/f3d 0 2024-03-04T13:42:56.697 INFO:tasks.workunit.client.0.smithi107.stdout:0/109: chown da/dc/c10 1317538 1 2024-03-04T13:42:56.698 INFO:tasks.workunit.client.1.smithi169.stdout:8/589: write d7/d10/d22/d46/f6e [353984,123027] 0 2024-03-04T13:42:56.698 INFO:tasks.workunit.client.1.smithi169.stdout:1/550: dwrite d4/d5/d36/d35/fa2 [4194304,4194304] 0 2024-03-04T13:42:56.698 INFO:tasks.workunit.client.1.smithi169.stdout:7/643: rename d2/d15/l3b to d2/d28/d68/ldc 0 2024-03-04T13:42:56.702 INFO:tasks.workunit.client.0.smithi107.stdout:0/110: chown da/de/f11 28822432 1 2024-03-04T13:42:56.702 INFO:tasks.workunit.client.0.smithi107.stdout:6/77: fsync df/d1b/f1d 0 2024-03-04T13:42:56.707 INFO:tasks.workunit.client.1.smithi169.stdout:3/602: link d7/de/d2b/d7d/d85/d13/d5a/da1/fa7 d7/de/d2b/d7d/d85/d13/d5a/fcc 0 2024-03-04T13:42:56.709 INFO:tasks.workunit.client.0.smithi107.stdout:2/96: creat da/f20 x:0 0 0 2024-03-04T13:42:56.711 INFO:tasks.workunit.client.1.smithi169.stdout:2/630: symlink d1/d5/d19/d68/dd9/ldd 0 2024-03-04T13:42:56.713 INFO:tasks.workunit.client.1.smithi169.stdout:2/631: readlink d1/d5/d22/d3b/lb2 0 2024-03-04T13:42:56.719 INFO:tasks.workunit.client.1.smithi169.stdout:6/587: getdents d0/d32/d70 0 2024-03-04T13:42:56.721 INFO:tasks.workunit.client.1.smithi169.stdout:0/478: symlink d5/d9/dc/d12/da5/lac 0 2024-03-04T13:42:56.722 INFO:tasks.workunit.client.1.smithi169.stdout:6/588: dread - d0/d32/d3a/d43/d74/fa4 zero size 2024-03-04T13:42:56.726 INFO:tasks.workunit.client.1.smithi169.stdout:9/605: fsync d7/fe 0 2024-03-04T13:42:56.726 INFO:tasks.workunit.client.0.smithi107.stdout:6/78: unlink c7 0 2024-03-04T13:42:56.726 INFO:tasks.workunit.client.0.smithi107.stdout:0/111: write f4 [602307,127043] 0 2024-03-04T13:42:56.728 INFO:tasks.workunit.client.0.smithi107.stdout:7/73: dread f4 [4194304,4194304] 0 2024-03-04T13:42:56.728 INFO:tasks.workunit.client.1.smithi169.stdout:9/606: chown d7/d25/fa8 155185140 1 2024-03-04T13:42:56.730 INFO:tasks.workunit.client.0.smithi107.stdout:4/47: unlink d0/fb 0 2024-03-04T13:42:56.730 INFO:tasks.workunit.client.0.smithi107.stdout:1/48: dwrite d1/d3/f5 [0,4194304] 0 2024-03-04T13:42:56.731 INFO:tasks.workunit.client.1.smithi169.stdout:5/665: dwrite d1/d60/fae [0,4194304] 0 2024-03-04T13:42:56.731 INFO:tasks.workunit.client.0.smithi107.stdout:4/48: stat d0/d9/d10 0 2024-03-04T13:42:56.732 INFO:tasks.workunit.client.0.smithi107.stdout:0/112: write da/dc/f1d [80615,100399] 0 2024-03-04T13:42:56.734 INFO:tasks.workunit.client.0.smithi107.stdout:0/113: fsync da/f27 0 2024-03-04T13:42:56.739 INFO:tasks.workunit.client.1.smithi169.stdout:3/603: mknod d7/de/d2b/d7d/d85/d13/d5a/ccd 0 2024-03-04T13:42:56.741 INFO:tasks.workunit.client.0.smithi107.stdout:6/79: write df/d1b/f1c [69503,58062] 0 2024-03-04T13:42:56.747 INFO:tasks.workunit.client.0.smithi107.stdout:7/74: creat d6/f1a x:0 0 0 2024-03-04T13:42:56.748 INFO:tasks.workunit.client.1.smithi169.stdout:4/515: dwrite d4/d11/d2e/d39/d48/f65 [0,4194304] 0 2024-03-04T13:42:56.750 INFO:tasks.workunit.client.0.smithi107.stdout:1/49: mknod d1/cb 0 2024-03-04T13:42:56.753 INFO:tasks.workunit.client.1.smithi169.stdout:9/607: fsync d7/dc/f9b 0 2024-03-04T13:42:56.754 INFO:tasks.workunit.client.1.smithi169.stdout:8/590: mknod d7/d10/d22/d75/d81/da4/cc2 0 2024-03-04T13:42:56.754 INFO:tasks.workunit.client.0.smithi107.stdout:4/49: rename d0/f6 to d0/d9/f11 0 2024-03-04T13:42:56.757 INFO:tasks.workunit.client.1.smithi169.stdout:0/479: symlink d5/d9/dc/lad 0 2024-03-04T13:42:56.760 INFO:tasks.workunit.client.1.smithi169.stdout:7/644: dread d2/de/f53 [0,4194304] 0 2024-03-04T13:42:56.760 INFO:tasks.workunit.client.1.smithi169.stdout:3/604: creat d7/de/d2b/d7d/d85/d13/d1a/d2d/fce x:0 0 0 2024-03-04T13:42:56.766 INFO:tasks.workunit.client.1.smithi169.stdout:1/551: creat d4/d5/d36/d35/d5b/fb5 x:0 0 0 2024-03-04T13:42:56.766 INFO:tasks.workunit.client.0.smithi107.stdout:4/50: rmdir d0/d7 39 2024-03-04T13:42:56.767 INFO:tasks.workunit.client.1.smithi169.stdout:1/552: readlink d4/d5/d36/d35/d16/d3a/l44 0 2024-03-04T13:42:56.771 INFO:tasks.workunit.client.0.smithi107.stdout:0/114: rmdir da/de/d26/d29 0 2024-03-04T13:42:56.773 INFO:tasks.workunit.client.1.smithi169.stdout:1/553: truncate d4/d5/d36/d35/d5b/fb5 407469 0 2024-03-04T13:42:56.773 INFO:tasks.workunit.client.1.smithi169.stdout:8/591: unlink d7/ca 0 2024-03-04T13:42:56.774 INFO:tasks.workunit.client.1.smithi169.stdout:2/632: dread d1/d5/d19/d1e/d29/f3e [0,4194304] 0 2024-03-04T13:42:56.775 INFO:tasks.workunit.client.1.smithi169.stdout:0/480: mknod d5/cae 0 2024-03-04T13:42:56.778 INFO:tasks.workunit.client.0.smithi107.stdout:3/50: dwrite d1/f9 [0,4194304] 0 2024-03-04T13:42:56.779 INFO:tasks.workunit.client.1.smithi169.stdout:7/645: creat d2/de/d3e/d7e/fdd x:0 0 0 2024-03-04T13:42:56.780 INFO:tasks.workunit.client.0.smithi107.stdout:0/115: creat da/f2a x:0 0 0 2024-03-04T13:42:56.780 INFO:tasks.workunit.client.1.smithi169.stdout:4/516: symlink d4/d11/d3b/d67/d98/la8 0 2024-03-04T13:42:56.781 INFO:tasks.workunit.client.0.smithi107.stdout:2/97: truncate f2 3461583 0 2024-03-04T13:42:56.783 INFO:tasks.workunit.client.0.smithi107.stdout:1/50: chown d1 545 1 2024-03-04T13:42:56.786 INFO:tasks.workunit.client.1.smithi169.stdout:5/666: rename d1/d16/d17/d1b/f25 to d1/d16/d17/d2d/d5a/fdd 0 2024-03-04T13:42:56.791 INFO:tasks.workunit.client.1.smithi169.stdout:0/481: creat d5/d4c/d65/faf x:0 0 0 2024-03-04T13:42:56.792 INFO:tasks.workunit.client.1.smithi169.stdout:7/646: stat d2/d15/d54/caf 0 2024-03-04T13:42:56.793 INFO:tasks.workunit.client.0.smithi107.stdout:7/75: dwrite d6/fc [0,4194304] 0 2024-03-04T13:42:56.795 INFO:tasks.workunit.client.1.smithi169.stdout:4/517: mkdir d4/d11/d2e/d21/da9 0 2024-03-04T13:42:56.796 INFO:tasks.workunit.client.0.smithi107.stdout:0/116: symlink da/de/d26/l2b 0 2024-03-04T13:42:56.796 INFO:tasks.workunit.client.0.smithi107.stdout:2/98: write f7 [1055626,74327] 0 2024-03-04T13:42:56.797 INFO:tasks.workunit.client.0.smithi107.stdout:7/76: write d6/d13/f18 [1012063,75120] 0 2024-03-04T13:42:56.797 INFO:tasks.workunit.client.1.smithi169.stdout:4/518: read d4/d11/d3b/f83 [427509,114133] 0 2024-03-04T13:42:56.801 INFO:tasks.workunit.client.1.smithi169.stdout:1/554: creat d4/d5/d2d/d4e/d71/d26/d7e/fb6 x:0 0 0 2024-03-04T13:42:56.806 INFO:tasks.workunit.client.0.smithi107.stdout:4/51: dwrite d0/f3 [0,4194304] 0 2024-03-04T13:42:56.807 INFO:tasks.workunit.client.0.smithi107.stdout:3/51: dread d1/fc [0,4194304] 0 2024-03-04T13:42:56.813 INFO:tasks.workunit.client.0.smithi107.stdout:3/52: write d1/fd [448901,37274] 0 2024-03-04T13:42:56.813 INFO:tasks.workunit.client.1.smithi169.stdout:3/605: rename d7/de/d2b/d7d/d85/d13/l1e to d7/de/d2b/d7d/d85/d18/d21/d66/lcf 0 2024-03-04T13:42:56.814 INFO:tasks.workunit.client.1.smithi169.stdout:9/608: dwrite d7/dc/f1e [0,4194304] 0 2024-03-04T13:42:56.816 INFO:tasks.workunit.client.0.smithi107.stdout:2/99: creat da/f21 x:0 0 0 2024-03-04T13:42:56.819 INFO:tasks.workunit.client.0.smithi107.stdout:0/117: truncate f8 172234 0 2024-03-04T13:42:56.819 INFO:tasks.workunit.client.0.smithi107.stdout:7/77: unlink d6/lb 0 2024-03-04T13:42:56.822 INFO:tasks.workunit.client.0.smithi107.stdout:1/51: dwrite d1/d3/f5 [0,4194304] 0 2024-03-04T13:42:56.824 INFO:tasks.workunit.client.0.smithi107.stdout:1/52: chown d1/d3/f5 12899 1 2024-03-04T13:42:56.824 INFO:tasks.workunit.client.1.smithi169.stdout:2/633: getdents d1/d5/d19/d3c 0 2024-03-04T13:42:56.824 INFO:tasks.workunit.client.1.smithi169.stdout:4/519: symlink d4/laa 0 2024-03-04T13:42:56.825 INFO:tasks.workunit.client.1.smithi169.stdout:5/667: rename d1/d16/d17/d2d/d51/f6e to d1/d16/d2f/fde 0 2024-03-04T13:42:56.825 INFO:tasks.workunit.client.1.smithi169.stdout:0/482: write d5/d9/dc/d12/d15/d19/d61/f21 [1741075,22642] 0 2024-03-04T13:42:56.827 INFO:tasks.workunit.client.1.smithi169.stdout:7/647: write d2/de/f53 [2727866,120] 0 2024-03-04T13:42:56.830 INFO:tasks.workunit.client.0.smithi107.stdout:4/52: symlink d0/d9/d10/l12 0 2024-03-04T13:42:56.833 INFO:tasks.workunit.client.0.smithi107.stdout:3/53: rename l0 to d1/l10 0 2024-03-04T13:42:56.835 INFO:tasks.workunit.client.1.smithi169.stdout:9/609: mkdir d7/dc/d12/d20/da6/dd1 0 2024-03-04T13:42:56.836 INFO:tasks.workunit.client.1.smithi169.stdout:1/555: write d4/d5/d2d/d4e/f84 [1080037,4397] 0 2024-03-04T13:42:56.837 INFO:tasks.workunit.client.1.smithi169.stdout:9/610: fsync d7/dc/f9b 0 2024-03-04T13:42:56.837 INFO:tasks.workunit.client.1.smithi169.stdout:8/592: dwrite d7/d10/d22/f25 [0,4194304] 0 2024-03-04T13:42:56.840 INFO:tasks.workunit.client.1.smithi169.stdout:9/611: stat d7/d25/d64 0 2024-03-04T13:42:56.843 INFO:tasks.workunit.client.1.smithi169.stdout:8/593: chown d7/f9b 1715 1 2024-03-04T13:42:56.846 INFO:tasks.workunit.client.1.smithi169.stdout:9/612: stat d7/dc/d12/d20/da6 0 2024-03-04T13:42:56.846 INFO:tasks.workunit.client.0.smithi107.stdout:7/78: symlink d6/l1b 0 2024-03-04T13:42:56.847 INFO:tasks.workunit.client.1.smithi169.stdout:1/556: write d4/d5/d36/d35/d16/d3a/f9c [498649,96661] 0 2024-03-04T13:42:56.848 INFO:tasks.workunit.client.1.smithi169.stdout:2/634: mkdir d1/d5/d22/d45/dde 0 2024-03-04T13:42:56.848 INFO:tasks.workunit.client.1.smithi169.stdout:8/594: write d7/f8b [9408004,128253] 0 2024-03-04T13:42:56.848 INFO:tasks.workunit.client.0.smithi107.stdout:7/79: fsync d6/fa 0 2024-03-04T13:42:56.849 INFO:tasks.workunit.client.1.smithi169.stdout:4/520: symlink d4/d11/d3b/d67/d98/lab 0 2024-03-04T13:42:56.851 INFO:tasks.workunit.client.1.smithi169.stdout:5/668: fsync d1/d81/dc1/fca 0 2024-03-04T13:42:56.854 INFO:tasks.workunit.client.1.smithi169.stdout:4/521: stat d4/d11/d2e/d39/d48/f4e 0 2024-03-04T13:42:56.858 INFO:tasks.workunit.client.1.smithi169.stdout:3/606: mknod d7/de/d2b/d7d/d85/d18/d21/d3f/d86/cd0 0 2024-03-04T13:42:56.858 INFO:tasks.workunit.client.1.smithi169.stdout:2/635: mknod d1/d6d/cdf 0 2024-03-04T13:42:56.859 INFO:tasks.workunit.client.1.smithi169.stdout:8/595: write d7/d10/d22/f25 [141332,114290] 0 2024-03-04T13:42:56.861 INFO:tasks.workunit.client.1.smithi169.stdout:7/648: dwrite d2/de/d3e/d41/f5a [4194304,4194304] 0 2024-03-04T13:42:56.867 INFO:tasks.workunit.client.1.smithi169.stdout:2/636: readlink d1/d5/d22/d45/d4a/lb6 0 2024-03-04T13:42:56.869 INFO:tasks.workunit.client.0.smithi107.stdout:4/53: dwrite d0/d9/f11 [0,4194304] 0 2024-03-04T13:42:56.869 INFO:tasks.workunit.client.1.smithi169.stdout:7/649: write d2/d15/fce [885854,23749] 0 2024-03-04T13:42:56.869 INFO:tasks.workunit.client.1.smithi169.stdout:3/607: write d7/de/d2b/d7d/d85/d13/d5a/da1/fb7 [109482,48855] 0 2024-03-04T13:42:56.882 INFO:tasks.workunit.client.1.smithi169.stdout:0/483: dread d5/d9/dc/d12/f62 [0,4194304] 0 2024-03-04T13:42:56.887 INFO:tasks.workunit.client.1.smithi169.stdout:0/484: fsync d5/d9/d3d/d59/d64/f81 0 2024-03-04T13:42:56.888 INFO:tasks.workunit.client.1.smithi169.stdout:3/608: mknod d7/de/d2b/d7d/d85/d13/d1a/d2d/d56/d5e/d67/d5c/d84/d95/cd1 0 2024-03-04T13:42:56.889 INFO:tasks.workunit.client.0.smithi107.stdout:4/54: dwrite d0/d9/f11 [0,4194304] 0 2024-03-04T13:42:56.889 INFO:tasks.workunit.client.0.smithi107.stdout:3/54: dwrite d1/f3 [0,4194304] 0 2024-03-04T13:42:56.893 INFO:tasks.workunit.client.1.smithi169.stdout:9/613: getdents d7/dc/d12/d20/da6 0 2024-03-04T13:42:56.894 INFO:tasks.workunit.client.1.smithi169.stdout:7/650: symlink d2/d28/db8/dd5/lde 0 2024-03-04T13:42:56.897 INFO:tasks.workunit.client.1.smithi169.stdout:0/485: rmdir d5/d9/d3d/d59/d64/d7f 39 2024-03-04T13:42:56.898 INFO:tasks.workunit.client.1.smithi169.stdout:0/486: readlink d5/d9/dc/l6c 0 2024-03-04T13:42:56.900 INFO:tasks.workunit.client.1.smithi169.stdout:6/589: sync 2024-03-04T13:42:56.900 INFO:tasks.workunit.client.1.smithi169.stdout:7/651: chown d2/d15/d17/d6f/f72 2320 1 2024-03-04T13:42:56.903 INFO:tasks.workunit.client.1.smithi169.stdout:1/557: dwrite d4/f25 [0,4194304] 0 2024-03-04T13:42:56.915 INFO:tasks.workunit.client.1.smithi169.stdout:8/596: rename d7/ld to d7/d10/d17/d44/d36/d62/lc3 0 2024-03-04T13:42:56.915 INFO:tasks.workunit.client.1.smithi169.stdout:4/522: link d4/d11/d2e/d63/la4 d4/d11/d2e/d21/d5e/lac 0 2024-03-04T13:42:56.921 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:42:56 smithi107 ceph-mon[28739]: pgmap v50: 65 pgs: 65 active+clean; 1.2 GiB data, 5.1 GiB used, 531 GiB / 536 GiB avail; 34 MiB/s rd, 119 MiB/s wr, 258 op/s 2024-03-04T13:42:56.922 INFO:tasks.workunit.client.1.smithi169.stdout:2/637: rename d1/d5/d19/d3c/db8/ld7 to d1/d5/d22/d45/le0 0 2024-03-04T13:42:56.933 INFO:tasks.workunit.client.1.smithi169.stdout:7/652: dread d2/de/d3e/f62 [0,4194304] 0 2024-03-04T13:42:56.934 INFO:tasks.workunit.client.1.smithi169.stdout:2/638: rmdir d1/d5/d22/d45 39 2024-03-04T13:42:56.940 INFO:tasks.workunit.client.1.smithi169.stdout:8/597: mknod d7/d10/d22/d46/cc4 0 2024-03-04T13:42:56.941 INFO:tasks.workunit.client.1.smithi169.stdout:9/614: dwrite d7/d25/d64/fa7 [0,4194304] 0 2024-03-04T13:42:56.942 INFO:tasks.workunit.client.1.smithi169.stdout:3/609: dwrite d7/de/d2b/d7d/d85/d13/d1a/d2d/dbe/fcb [0,4194304] 0 2024-03-04T13:42:56.944 INFO:tasks.workunit.client.1.smithi169.stdout:0/487: dread d5/d9/dc/d12/d15/d19/d77/f88 [0,4194304] 0 2024-03-04T13:42:56.945 INFO:tasks.workunit.client.1.smithi169.stdout:5/669: dread d1/d16/dbd/f4f [0,4194304] 0 2024-03-04T13:42:56.949 INFO:tasks.workunit.client.1.smithi169.stdout:9/615: write d7/dc/d12/d20/da6/fc5 [314798,66192] 0 2024-03-04T13:42:56.956 INFO:tasks.workunit.client.1.smithi169.stdout:5/670: dread d1/d16/d17/d2d/d5a/d8a/fa5 [0,4194304] 0 2024-03-04T13:42:56.961 INFO:tasks.workunit.client.1.smithi169.stdout:6/590: write d0/d3c/d95/dbd/fcd [978638,26606] 0 2024-03-04T13:42:56.969 INFO:tasks.workunit.client.1.smithi169.stdout:5/671: dread d1/d16/d2a/f3e [0,4194304] 0 2024-03-04T13:42:56.971 INFO:tasks.workunit.client.1.smithi169.stdout:4/523: write d4/d11/d2e/d39/f68 [1275020,123174] 0 2024-03-04T13:42:56.974 INFO:tasks.workunit.client.1.smithi169.stdout:4/524: stat d4/fa 0 2024-03-04T13:42:56.986 INFO:tasks.workunit.client.1.smithi169.stdout:1/558: getdents d4/d5/d36/d35/d16 0 2024-03-04T13:42:56.988 INFO:tasks.workunit.client.1.smithi169.stdout:3/610: chown d7/de/l29 21 1 2024-03-04T13:42:56.989 INFO:tasks.workunit.client.1.smithi169.stdout:8/598: fsync d7/d10/d24/f3c 0 2024-03-04T13:42:56.989 INFO:tasks.workunit.client.0.smithi107.stdout:5/38: sync 2024-03-04T13:42:56.989 INFO:tasks.workunit.client.0.smithi107.stdout:9/42: sync 2024-03-04T13:42:56.989 INFO:tasks.workunit.client.0.smithi107.stdout:6/80: sync 2024-03-04T13:42:56.989 INFO:tasks.workunit.client.0.smithi107.stdout:8/127: sync 2024-03-04T13:42:56.989 INFO:tasks.workunit.client.0.smithi107.stdout:5/39: chown d5 0 1 2024-03-04T13:42:56.990 INFO:tasks.workunit.client.0.smithi107.stdout:6/81: fsync df/f15 0 2024-03-04T13:42:56.990 INFO:tasks.workunit.client.0.smithi107.stdout:5/40: fdatasync d5/f9 0 2024-03-04T13:42:56.992 INFO:tasks.workunit.client.1.smithi169.stdout:2/639: rename d1/d5/d19/d36 to d1/d5/d19/d3c/db8/de1 0 2024-03-04T13:42:56.992 INFO:tasks.workunit.client.1.smithi169.stdout:7/653: rename d2/de/d3e/d41/d5b/d7a to d2/de/d3e/d41/d5b/d7a/dd3/ddf 22 2024-03-04T13:42:56.994 INFO:tasks.workunit.client.0.smithi107.stdout:6/82: dread df/f15 [0,4194304] 0 2024-03-04T13:42:56.994 INFO:tasks.workunit.client.1.smithi169.stdout:7/654: write d2/de/ff [5164219,75365] 0 2024-03-04T13:42:56.996 INFO:tasks.workunit.client.1.smithi169.stdout:7/655: readlink d2/d15/l52 0 2024-03-04T13:42:56.997 INFO:tasks.workunit.client.1.smithi169.stdout:7/656: dread - d2/de/d3e/d41/d69/fc8 zero size 2024-03-04T13:42:56.998 INFO:tasks.workunit.client.1.smithi169.stdout:5/672: mknod d1/d16/d2f/d39/cdf 0 2024-03-04T13:42:57.004 INFO:tasks.workunit.client.0.smithi107.stdout:5/41: dwrite d5/f6 [0,4194304] 0 2024-03-04T13:42:57.005 INFO:tasks.workunit.client.1.smithi169.stdout:2/640: mknod d1/d6d/dbc/ce2 0 2024-03-04T13:42:57.005 INFO:tasks.workunit.client.0.smithi107.stdout:5/42: write d5/d8/fe [651093,84567] 0 2024-03-04T13:42:57.012 INFO:tasks.workunit.client.1.smithi169.stdout:6/591: mknod d0/cd6 0 2024-03-04T13:42:57.013 INFO:tasks.workunit.client.1.smithi169.stdout:6/592: chown d0/d3c/d40/l49 401456 1 2024-03-04T13:42:57.016 INFO:tasks.workunit.client.1.smithi169.stdout:5/673: symlink d1/d16/dbd/dbb/dcc/le0 0 2024-03-04T13:42:57.017 INFO:tasks.workunit.client.1.smithi169.stdout:1/559: symlink d4/d5/d36/da9/lb7 0 2024-03-04T13:42:57.026 INFO:tasks.workunit.client.1.smithi169.stdout:7/657: mknod d2/de/d3e/ce0 0 2024-03-04T13:42:57.031 INFO:tasks.workunit.client.1.smithi169.stdout:3/611: rename d7/de/d24/l9c to d7/de/d2b/ld2 0 2024-03-04T13:42:57.032 INFO:tasks.workunit.client.1.smithi169.stdout:5/674: truncate d1/f24 5662569 0 2024-03-04T13:42:57.037 INFO:tasks.workunit.client.1.smithi169.stdout:0/488: getdents d5/d9/dc/d12/d15/d19/d69 0 2024-03-04T13:42:57.039 INFO:tasks.workunit.client.1.smithi169.stdout:7/658: creat d2/d15/d54/fe1 x:0 0 0 2024-03-04T13:42:57.042 INFO:tasks.workunit.client.1.smithi169.stdout:8/599: rename d7/d10/d22/d75/d81/da2/caf to d7/d10/d22/d75/d81/cc5 0 2024-03-04T13:42:57.046 INFO:tasks.workunit.client.1.smithi169.stdout:1/560: truncate d4/d5/d36/d35/d5b/f87 954951 0 2024-03-04T13:42:57.048 INFO:tasks.workunit.client.1.smithi169.stdout:3/612: creat d7/de/d2b/d7d/d85/d13/d5a/d65/da4/fd3 x:0 0 0 2024-03-04T13:42:57.054 INFO:tasks.workunit.client.1.smithi169.stdout:5/675: rename d1/d16/d2a/lb5 to d1/d76/le1 0 2024-03-04T13:42:57.055 INFO:tasks.workunit.client.1.smithi169.stdout:4/525: dwrite d4/d11/f30 [0,4194304] 0 2024-03-04T13:42:57.056 INFO:tasks.workunit.client.1.smithi169.stdout:5/676: read - d1/d81/fdb zero size 2024-03-04T13:42:57.057 INFO:tasks.workunit.client.1.smithi169.stdout:5/677: chown d1/d16/d17/d2d 6616623 1 2024-03-04T13:42:57.059 INFO:tasks.workunit.client.1.smithi169.stdout:8/600: mkdir d7/d10/d22/d46/da5/da6/dc6 0 2024-03-04T13:42:57.067 INFO:tasks.workunit.client.1.smithi169.stdout:0/489: mkdir d5/d4c/d65/d95/db0 0 2024-03-04T13:42:57.070 INFO:tasks.workunit.client.1.smithi169.stdout:7/659: link d2/de/d3e/d41/d69/f92 d2/de/d3e/d7e/fe2 0 2024-03-04T13:42:57.071 INFO:tasks.workunit.client.1.smithi169.stdout:7/660: dread - d2/d6b/fd7 zero size 2024-03-04T13:42:57.073 INFO:tasks.workunit.client.1.smithi169.stdout:4/526: unlink d4/d11/d2e/d6c/d78/da5/l36 0 2024-03-04T13:42:57.073 INFO:tasks.workunit.client.1.smithi169.stdout:7/661: read d2/de/ff [3721627,86411] 0 2024-03-04T13:42:57.076 INFO:tasks.workunit.client.1.smithi169.stdout:7/662: truncate d2/de/d3e/d41/f5a 9243745 0 2024-03-04T13:42:57.080 INFO:tasks.workunit.client.1.smithi169.stdout:8/601: rename d7/d10/c8d to d7/d10/d22/d75/d81/da2/d96/cc7 0 2024-03-04T13:42:57.083 INFO:tasks.workunit.client.1.smithi169.stdout:0/490: mknod d5/d9/dc/d12/d15/d19/d77/d94/cb1 0 2024-03-04T13:42:57.085 INFO:tasks.workunit.client.1.smithi169.stdout:2/641: truncate d1/d5/d22/f32 3229374 0 2024-03-04T13:42:57.087 INFO:tasks.workunit.client.1.smithi169.stdout:6/593: dwrite d0/d3c/d40/d52/d98/fb6 [0,4194304] 0 2024-03-04T13:42:57.091 INFO:tasks.workunit.client.1.smithi169.stdout:9/616: sync 2024-03-04T13:42:57.091 INFO:tasks.workunit.client.1.smithi169.stdout:4/527: symlink d4/d11/d2e/d21/lad 0 2024-03-04T13:42:57.094 INFO:tasks.workunit.client.1.smithi169.stdout:9/617: chown d7/dc/d12/d32/d42/c60 35811796 1 2024-03-04T13:42:57.094 INFO:tasks.workunit.client.1.smithi169.stdout:4/528: write d4/f5a [1328504,63027] 0 2024-03-04T13:42:57.098 INFO:tasks.workunit.client.1.smithi169.stdout:7/663: dwrite d2/d15/d54/fe1 [0,4194304] 0 2024-03-04T13:42:57.098 INFO:tasks.workunit.client.1.smithi169.stdout:9/618: dread - d7/dc/d12/d32/d42/d4f/fb5 zero size 2024-03-04T13:42:57.099 INFO:tasks.workunit.client.1.smithi169.stdout:7/664: stat d2/d15 0 2024-03-04T13:42:57.100 INFO:tasks.workunit.client.1.smithi169.stdout:1/561: fsync d4/d5/d2d/d4e/d71/d82/fad 0 2024-03-04T13:42:57.103 INFO:tasks.workunit.client.1.smithi169.stdout:1/562: write d4/d1d/f34 [852094,38799] 0 2024-03-04T13:42:57.107 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:42:56 smithi169 ceph-mon[36313]: pgmap v50: 65 pgs: 65 active+clean; 1.2 GiB data, 5.1 GiB used, 531 GiB / 536 GiB avail; 34 MiB/s rd, 119 MiB/s wr, 258 op/s 2024-03-04T13:42:57.114 INFO:tasks.workunit.client.0.smithi107.stdout:7/80: mkdir d6/d13/d16/d1c 0 2024-03-04T13:42:57.116 INFO:tasks.workunit.client.0.smithi107.stdout:1/53: symlink d1/lc 0 2024-03-04T13:42:57.119 INFO:tasks.workunit.client.1.smithi169.stdout:9/619: mkdir d7/d25/d3c/dd2 0 2024-03-04T13:42:57.121 INFO:tasks.workunit.client.1.smithi169.stdout:8/602: mknod d7/d10/d17/d44/d4c/d93/cc8 0 2024-03-04T13:42:57.122 INFO:tasks.workunit.client.1.smithi169.stdout:3/613: dwrite d7/de/d24/f8f [0,4194304] 0 2024-03-04T13:42:57.124 INFO:tasks.workunit.client.1.smithi169.stdout:8/603: chown d7/d10/d22/d75/lbd 127831 1 2024-03-04T13:42:57.125 INFO:tasks.workunit.client.1.smithi169.stdout:3/614: write d7/de/d24/f8f [2148866,108022] 0 2024-03-04T13:42:57.128 INFO:tasks.workunit.client.0.smithi107.stdout:3/55: mkdir d1/de/d11 0 2024-03-04T13:42:57.128 INFO:tasks.workunit.client.1.smithi169.stdout:8/604: write d7/d10/d22/d75/d81/da2/dac/fb8 [833909,51791] 0 2024-03-04T13:42:57.129 INFO:tasks.workunit.client.1.smithi169.stdout:2/642: creat d1/d5/d22/d45/d4a/fe3 x:0 0 0 2024-03-04T13:42:57.133 INFO:tasks.workunit.client.1.smithi169.stdout:5/678: getdents d1/d16/d2f/d72/db8 0 2024-03-04T13:42:57.134 INFO:tasks.workunit.client.0.smithi107.stdout:8/128: creat d6/d16/f22 x:0 0 0 2024-03-04T13:42:57.135 INFO:tasks.workunit.client.1.smithi169.stdout:3/615: dread d7/f33 [0,4194304] 0 2024-03-04T13:42:57.138 INFO:tasks.workunit.client.1.smithi169.stdout:3/616: dread - d7/de/d2b/d7d/d85/d13/d5a/fcc zero size 2024-03-04T13:42:57.138 INFO:tasks.workunit.client.0.smithi107.stdout:9/43: unlink d4/c5 0 2024-03-04T13:42:57.139 INFO:tasks.workunit.client.0.smithi107.stdout:9/44: stat f0 0 2024-03-04T13:42:57.142 INFO:tasks.workunit.client.0.smithi107.stdout:6/83: creat df/f20 x:0 0 0 2024-03-04T13:42:57.143 INFO:tasks.workunit.client.0.smithi107.stdout:7/81: dwrite d6/f7 [0,4194304] 0 2024-03-04T13:42:57.143 INFO:tasks.workunit.client.0.smithi107.stdout:6/84: chown f6 2061 1 2024-03-04T13:42:57.144 INFO:tasks.workunit.client.1.smithi169.stdout:9/620: mknod d7/dc/d12/d8a/cd3 0 2024-03-04T13:42:57.144 INFO:tasks.workunit.client.0.smithi107.stdout:7/82: chown d6/f7 8 1 2024-03-04T13:42:57.146 INFO:tasks.workunit.client.0.smithi107.stdout:0/118: dread f8 [0,4194304] 0 2024-03-04T13:42:57.148 INFO:tasks.workunit.client.1.smithi169.stdout:7/665: mkdir d2/de/d3e/d7e/dbb/de3 0 2024-03-04T13:42:57.149 INFO:tasks.workunit.client.1.smithi169.stdout:7/666: read - d2/de/d3e/d41/fc3 zero size 2024-03-04T13:42:57.150 INFO:tasks.workunit.client.0.smithi107.stdout:2/100: getdents da 0 2024-03-04T13:42:57.151 INFO:tasks.workunit.client.0.smithi107.stdout:4/55: rename d0/c5 to d0/d7/dc/c13 0 2024-03-04T13:42:57.152 INFO:tasks.workunit.client.0.smithi107.stdout:4/56: chown d0/d7/dc 1370520 1 2024-03-04T13:42:57.153 INFO:tasks.workunit.client.1.smithi169.stdout:2/643: write d1/d5/d22/d45/f72 [509895,38519] 0 2024-03-04T13:42:57.154 INFO:tasks.workunit.client.1.smithi169.stdout:1/563: symlink d4/d5/d2d/d4e/d71/d26/d54/lb8 0 2024-03-04T13:42:57.157 INFO:tasks.workunit.client.1.smithi169.stdout:5/679: unlink d1/lf 0 2024-03-04T13:42:57.158 INFO:tasks.workunit.client.1.smithi169.stdout:5/680: write d1/d60/fd7 [644546,81995] 0 2024-03-04T13:42:57.161 INFO:tasks.workunit.client.0.smithi107.stdout:6/85: dwrite f0 [0,4194304] 0 2024-03-04T13:42:57.164 INFO:tasks.workunit.client.0.smithi107.stdout:7/83: mkdir d6/d1d 0 2024-03-04T13:42:57.165 INFO:tasks.workunit.client.0.smithi107.stdout:5/43: mkdir d5/dc/df 0 2024-03-04T13:42:57.167 INFO:tasks.workunit.client.0.smithi107.stdout:0/119: creat da/dc/dd/f2c x:0 0 0 2024-03-04T13:42:57.168 INFO:tasks.workunit.client.1.smithi169.stdout:3/617: creat d7/de/d2b/d7d/d85/d13/d5a/d65/fd4 x:0 0 0 2024-03-04T13:42:57.169 INFO:tasks.workunit.client.0.smithi107.stdout:0/120: write da/dc/dd/f2c [1024437,19994] 0 2024-03-04T13:42:57.170 INFO:tasks.workunit.client.1.smithi169.stdout:9/621: mknod d7/dc/d12/d32/d42/cd4 0 2024-03-04T13:42:57.172 INFO:tasks.workunit.client.1.smithi169.stdout:4/529: write d4/d11/d2e/d37/d5c/f8c [1281270,79765] 0 2024-03-04T13:42:57.172 INFO:tasks.workunit.client.1.smithi169.stdout:1/564: dwrite d4/d5/d36/d35/d8f/f9a [0,4194304] 0 2024-03-04T13:42:57.173 INFO:tasks.workunit.client.0.smithi107.stdout:2/101: rename da/df/l16 to da/df/l22 0 2024-03-04T13:42:57.175 INFO:tasks.workunit.client.1.smithi169.stdout:8/605: mkdir d7/d10/dc9 0 2024-03-04T13:42:57.176 INFO:tasks.workunit.client.0.smithi107.stdout:3/56: mkdir d1/de/d11/d12 0 2024-03-04T13:42:57.178 INFO:tasks.workunit.client.1.smithi169.stdout:8/606: stat d7/d10/d22/d75/lbc 0 2024-03-04T13:42:57.185 INFO:tasks.workunit.client.1.smithi169.stdout:5/681: readlink d1/d76/le1 0 2024-03-04T13:42:57.187 INFO:tasks.workunit.client.0.smithi107.stdout:4/57: rename d0/d7 to d0/d9/de/d14 0 2024-03-04T13:42:57.187 INFO:tasks.workunit.client.0.smithi107.stdout:8/129: creat d6/d16/d1f/f23 x:0 0 0 2024-03-04T13:42:57.187 INFO:tasks.workunit.client.0.smithi107.stdout:9/45: symlink d4/da/lb 0 2024-03-04T13:42:57.188 INFO:tasks.workunit.client.0.smithi107.stdout:9/46: read f0 [3198916,30214] 0 2024-03-04T13:42:57.188 INFO:tasks.workunit.client.0.smithi107.stdout:6/86: mkdir df/d21 0 2024-03-04T13:42:57.189 INFO:tasks.workunit.client.1.smithi169.stdout:6/594: link d0/d3c/d40/d52/l76 d0/d3c/ld7 0 2024-03-04T13:42:57.190 INFO:tasks.workunit.client.0.smithi107.stdout:4/58: write d0/d9/f11 [3868627,90620] 0 2024-03-04T13:42:57.191 INFO:tasks.workunit.client.1.smithi169.stdout:4/530: dread d4/f7 [0,4194304] 0 2024-03-04T13:42:57.191 INFO:tasks.workunit.client.0.smithi107.stdout:6/87: write f0 [4965823,41819] 0 2024-03-04T13:42:57.192 INFO:tasks.workunit.client.0.smithi107.stdout:5/44: symlink d5/l10 0 2024-03-04T13:42:57.194 INFO:tasks.workunit.client.1.smithi169.stdout:3/618: dwrite d7/de/d2b/d7d/d85/d13/f4a [0,4194304] 0 2024-03-04T13:42:57.194 INFO:tasks.workunit.client.0.smithi107.stdout:6/88: chown c9 25 1 2024-03-04T13:42:57.195 INFO:tasks.workunit.client.0.smithi107.stdout:2/102: dwrite da/df/f15 [0,4194304] 0 2024-03-04T13:42:57.196 INFO:tasks.workunit.client.0.smithi107.stdout:5/45: stat d5/d8/da/ld 0 2024-03-04T13:42:57.198 INFO:tasks.workunit.client.1.smithi169.stdout:4/531: fdatasync d4/d11/d2e/d39/d48/f4e 0 2024-03-04T13:42:57.200 INFO:tasks.workunit.client.1.smithi169.stdout:0/491: sync 2024-03-04T13:42:57.200 INFO:tasks.workunit.client.1.smithi169.stdout:4/532: chown d4/d11/d2e/d39/d48 7191 1 2024-03-04T13:42:57.200 INFO:tasks.workunit.client.0.smithi107.stdout:5/46: dread d5/d8/fb [0,4194304] 0 2024-03-04T13:42:57.202 INFO:tasks.workunit.client.1.smithi169.stdout:0/492: write d5/d9/d3d/d5d/f7c [696970,125608] 0 2024-03-04T13:42:57.204 INFO:tasks.workunit.client.0.smithi107.stdout:4/59: dread d0/f3 [0,4194304] 0 2024-03-04T13:42:57.206 INFO:tasks.workunit.client.0.smithi107.stdout:4/60: write d0/d9/de/d14/fa [1803989,102062] 0 2024-03-04T13:42:57.213 INFO:tasks.workunit.client.1.smithi169.stdout:9/622: creat d7/d25/d64/fd5 x:0 0 0 2024-03-04T13:42:57.217 INFO:tasks.workunit.client.1.smithi169.stdout:7/667: write d2/f2d [5044206,96436] 0 2024-03-04T13:42:57.217 INFO:tasks.workunit.client.1.smithi169.stdout:2/644: symlink d1/le4 0 2024-03-04T13:42:57.220 INFO:tasks.workunit.client.0.smithi107.stdout:4/61: fdatasync d0/d9/de/d14/fa 0 2024-03-04T13:42:57.222 INFO:tasks.workunit.client.0.smithi107.stdout:4/62: write d0/f3 [15602,91378] 0 2024-03-04T13:42:57.223 INFO:tasks.workunit.client.1.smithi169.stdout:2/645: write d1/d5/d22/d45/f62 [4140771,120007] 0 2024-03-04T13:42:57.227 INFO:tasks.workunit.client.1.smithi169.stdout:4/533: dwrite d4/f69 [0,4194304] 0 2024-03-04T13:42:57.227 INFO:tasks.workunit.client.0.smithi107.stdout:9/47: fsync f0 0 2024-03-04T13:42:57.229 INFO:tasks.workunit.client.1.smithi169.stdout:4/534: write d4/f41 [6485357,19484] 0 2024-03-04T13:42:57.230 INFO:tasks.workunit.client.1.smithi169.stdout:4/535: stat d4/f7 0 2024-03-04T13:42:57.233 INFO:tasks.workunit.client.1.smithi169.stdout:5/682: symlink d1/d16/dbd/dbb/dcc/le2 0 2024-03-04T13:42:57.233 INFO:tasks.workunit.client.0.smithi107.stdout:4/63: fsync d0/f3 0 2024-03-04T13:42:57.234 INFO:tasks.workunit.client.1.smithi169.stdout:4/536: stat d4/d11/d2e/d39/d61 0 2024-03-04T13:42:57.237 INFO:tasks.workunit.client.0.smithi107.stdout:7/84: mkdir d6/d13/d16/d19/d1e 0 2024-03-04T13:42:57.238 INFO:tasks.workunit.client.1.smithi169.stdout:1/565: truncate d4/d5/d2d/d4e/d71/d26/f60 480654 0 2024-03-04T13:42:57.240 INFO:tasks.workunit.client.0.smithi107.stdout:6/89: symlink df/l22 0 2024-03-04T13:42:57.242 INFO:tasks.workunit.client.0.smithi107.stdout:6/90: chown df/f15 125367 1 2024-03-04T13:42:57.244 INFO:tasks.workunit.client.0.smithi107.stdout:9/48: dwrite d4/f8 [0,4194304] 0 2024-03-04T13:42:57.245 INFO:tasks.workunit.client.0.smithi107.stdout:6/91: read f0 [4803436,29008] 0 2024-03-04T13:42:57.250 INFO:tasks.workunit.client.0.smithi107.stdout:2/103: unlink da/l14 0 2024-03-04T13:42:57.250 INFO:tasks.workunit.client.0.smithi107.stdout:1/54: link d1/d3/c7 d1/d3/cd 0 2024-03-04T13:42:57.250 INFO:tasks.workunit.client.0.smithi107.stdout:5/47: rename d5/d8/fe to d5/d8/f11 0 2024-03-04T13:42:57.252 INFO:tasks.workunit.client.0.smithi107.stdout:6/92: read df/d1b/f1c [123147,77135] 0 2024-03-04T13:42:57.253 INFO:tasks.workunit.client.1.smithi169.stdout:9/623: dread d7/dc/fd [4194304,4194304] 0 2024-03-04T13:42:57.253 INFO:tasks.workunit.client.1.smithi169.stdout:4/537: stat d4/d11/d2e/d21/c89 0 2024-03-04T13:42:57.253 INFO:tasks.workunit.client.0.smithi107.stdout:3/57: creat d1/de/d11/d12/f13 x:0 0 0 2024-03-04T13:42:57.255 INFO:tasks.workunit.client.0.smithi107.stdout:8/130: dwrite f4 [0,4194304] 0 2024-03-04T13:42:57.255 INFO:tasks.workunit.client.0.smithi107.stdout:6/93: truncate df/f20 138467 0 2024-03-04T13:42:57.257 INFO:tasks.workunit.client.0.smithi107.stdout:8/131: dread - d6/d16/f22 zero size 2024-03-04T13:42:57.258 INFO:tasks.workunit.client.1.smithi169.stdout:5/683: creat d1/d16/dbd/dbb/fe3 x:0 0 0 2024-03-04T13:42:57.261 INFO:tasks.workunit.client.1.smithi169.stdout:8/607: link d7/ff d7/d10/d17/d44/d36/d62/fca 0 2024-03-04T13:42:57.266 INFO:tasks.workunit.client.1.smithi169.stdout:8/608: chown d7/d10/d22/d46/f83 15582 1 2024-03-04T13:42:57.267 INFO:tasks.workunit.client.1.smithi169.stdout:6/595: creat d0/d32/d3a/d43/d74/fd8 x:0 0 0 2024-03-04T13:42:57.268 INFO:tasks.workunit.client.0.smithi107.stdout:4/64: rmdir d0/d9/de 39 2024-03-04T13:42:57.269 INFO:tasks.workunit.client.1.smithi169.stdout:3/619: write d7/f49 [1107894,25170] 0 2024-03-04T13:42:57.272 INFO:tasks.workunit.client.1.smithi169.stdout:6/596: chown d0/d32/d3a/f3f 10819027 1 2024-03-04T13:42:57.274 INFO:tasks.workunit.client.1.smithi169.stdout:9/624: creat d7/dc/d12/d32/d42/fd6 x:0 0 0 2024-03-04T13:42:57.276 INFO:tasks.workunit.client.1.smithi169.stdout:7/668: dwrite d2/de/d3e/d41/d69/f92 [0,4194304] 0 2024-03-04T13:42:57.276 INFO:tasks.workunit.client.1.smithi169.stdout:4/538: dwrite d4/d11/d2e/d39/f60 [0,4194304] 0 2024-03-04T13:42:57.282 INFO:tasks.workunit.client.1.smithi169.stdout:6/597: truncate d0/d3c/d95/dbd/fc6 5024303 0 2024-03-04T13:42:57.283 INFO:tasks.workunit.client.1.smithi169.stdout:2/646: truncate d1/f40 1173692 0 2024-03-04T13:42:57.284 INFO:tasks.workunit.client.0.smithi107.stdout:2/104: symlink da/l23 0 2024-03-04T13:42:57.286 INFO:tasks.workunit.client.1.smithi169.stdout:8/609: dread d7/d10/d24/d3e/f95 [0,4194304] 0 2024-03-04T13:42:57.289 INFO:tasks.workunit.client.0.smithi107.stdout:5/48: rename f1 to d5/dc/df/f12 0 2024-03-04T13:42:57.291 INFO:tasks.workunit.client.0.smithi107.stdout:3/58: creat d1/f14 x:0 0 0 2024-03-04T13:42:57.295 INFO:tasks.workunit.client.0.smithi107.stdout:6/94: creat df/d17/f23 x:0 0 0 2024-03-04T13:42:57.298 INFO:tasks.workunit.client.0.smithi107.stdout:8/132: creat d6/d16/f24 x:0 0 0 2024-03-04T13:42:57.299 INFO:tasks.workunit.client.1.smithi169.stdout:1/566: write d4/d5/d2d/d4e/d71/d26/f60 [550640,86380] 0 2024-03-04T13:42:57.302 INFO:tasks.workunit.client.0.smithi107.stdout:3/59: dread d1/f9 [0,4194304] 0 2024-03-04T13:42:57.303 INFO:tasks.workunit.client.0.smithi107.stdout:7/85: creat d6/d13/d16/d1c/f1f x:0 0 0 2024-03-04T13:42:57.303 INFO:tasks.workunit.client.0.smithi107.stdout:3/60: chown d1/fc 14221055 1 2024-03-04T13:42:57.304 INFO:tasks.workunit.client.1.smithi169.stdout:7/669: dwrite d2/d6b/fd7 [0,4194304] 0 2024-03-04T13:42:57.310 INFO:tasks.workunit.client.1.smithi169.stdout:5/684: dread d1/d81/f98 [0,4194304] 0 2024-03-04T13:42:57.310 INFO:tasks.workunit.client.1.smithi169.stdout:6/598: rename d0/d32/d3a/f88 to d0/d3c/d40/d52/dac/fd9 0 2024-03-04T13:42:57.313 INFO:tasks.workunit.client.1.smithi169.stdout:8/610: mkdir d7/d10/d22/d46/da5/dcb 0 2024-03-04T13:42:57.313 INFO:tasks.workunit.client.1.smithi169.stdout:7/670: read d2/de/d3e/d41/d5b/d7a/fcc [1358439,22629] 0 2024-03-04T13:42:57.314 INFO:tasks.workunit.client.1.smithi169.stdout:5/685: truncate d1/d23/f85 1196317 0 2024-03-04T13:42:57.317 INFO:tasks.workunit.client.1.smithi169.stdout:4/539: link d4/d1c/c1f d4/d11/d2e/d39/d48/cae 0 2024-03-04T13:42:57.319 INFO:tasks.workunit.client.0.smithi107.stdout:5/49: fsync d5/d8/f11 0 2024-03-04T13:42:57.321 INFO:tasks.workunit.client.1.smithi169.stdout:4/540: read d4/d11/d2e/d39/d48/f4e [3991935,19852] 0 2024-03-04T13:42:57.323 INFO:tasks.workunit.client.0.smithi107.stdout:6/95: truncate f8 1439402 0 2024-03-04T13:42:57.327 INFO:tasks.workunit.client.0.smithi107.stdout:7/86: symlink d6/d13/d16/d19/l20 0 2024-03-04T13:42:57.328 INFO:tasks.workunit.client.0.smithi107.stdout:9/49: link d4/c7 d4/da/cc 0 2024-03-04T13:42:57.329 INFO:tasks.workunit.client.0.smithi107.stdout:2/105: write da/f18 [457866,60504] 0 2024-03-04T13:42:57.329 INFO:tasks.workunit.client.0.smithi107.stdout:9/50: rename d4 to d4/dd 22 2024-03-04T13:42:57.330 INFO:tasks.workunit.client.0.smithi107.stdout:1/55: creat d1/fe x:0 0 0 2024-03-04T13:42:57.331 INFO:tasks.workunit.client.0.smithi107.stdout:9/51: write d4/f8 [1648225,18684] 0 2024-03-04T13:42:57.334 INFO:tasks.workunit.client.1.smithi169.stdout:9/625: getdents d7/dc/d12/d32/d42/d4f 0 2024-03-04T13:42:57.336 INFO:tasks.workunit.client.1.smithi169.stdout:9/626: write d7/dc/d3e/f41 [85245,61612] 0 2024-03-04T13:42:57.342 INFO:tasks.workunit.client.0.smithi107.stdout:4/65: rmdir d0/d9/de/d14/dc/df 0 2024-03-04T13:42:57.342 INFO:tasks.workunit.client.0.smithi107.stdout:8/133: dwrite d6/da/f19 [4194304,4194304] 0 2024-03-04T13:42:57.343 INFO:tasks.workunit.client.0.smithi107.stdout:7/87: getdents d6/d1d 0 2024-03-04T13:42:57.344 INFO:tasks.workunit.client.1.smithi169.stdout:2/647: chown d1/d5/d22/f94 10735033 1 2024-03-04T13:42:57.344 INFO:tasks.workunit.client.0.smithi107.stdout:4/66: write d0/f3 [4156857,6553] 0 2024-03-04T13:42:57.345 INFO:tasks.workunit.client.1.smithi169.stdout:5/686: unlink d1/d16/d2a/d46/fbc 0 2024-03-04T13:42:57.346 INFO:tasks.workunit.client.0.smithi107.stdout:8/134: write d6/da/f1e [893230,47252] 0 2024-03-04T13:42:57.349 INFO:tasks.workunit.client.1.smithi169.stdout:3/620: link d7/de/d2b/d7d/d85/d13/d1a/c3d d7/de/d2b/d7d/d85/d13/d1a/d2d/d56/d5e/d67/d5c/d84/d95/cd5 0 2024-03-04T13:42:57.350 INFO:tasks.workunit.client.0.smithi107.stdout:8/135: chown d6/c17 175161 1 2024-03-04T13:42:57.350 INFO:tasks.workunit.client.0.smithi107.stdout:4/67: write d0/d9/de/d14/fa [3481730,14222] 0 2024-03-04T13:42:57.352 INFO:tasks.workunit.client.1.smithi169.stdout:7/671: creat d2/de/d3e/d41/d5b/d7a/dd3/fe4 x:0 0 0 2024-03-04T13:42:57.353 INFO:tasks.workunit.client.0.smithi107.stdout:9/52: write f0 [3715557,107195] 0 2024-03-04T13:42:57.355 INFO:tasks.workunit.client.1.smithi169.stdout:7/672: fsync d2/de/d3e/d41/d69/fb0 0 2024-03-04T13:42:57.356 INFO:tasks.workunit.client.0.smithi107.stdout:7/88: mknod d6/d13/c21 0 2024-03-04T13:42:57.357 INFO:tasks.workunit.client.0.smithi107.stdout:6/96: dwrite fb [0,4194304] 0 2024-03-04T13:42:57.359 INFO:tasks.workunit.client.0.smithi107.stdout:4/68: write d0/d9/de/d14/fa [4373258,46722] 0 2024-03-04T13:42:57.360 INFO:tasks.workunit.client.0.smithi107.stdout:7/89: readlink d6/d13/l15 0 2024-03-04T13:42:57.361 INFO:tasks.workunit.client.1.smithi169.stdout:1/567: rename d4/d5/d36/d35/d70/d78/c94 to d4/d5/d36/d35/d5b/cb9 0 2024-03-04T13:42:57.364 INFO:tasks.workunit.client.1.smithi169.stdout:6/599: creat d0/d3c/fda x:0 0 0 2024-03-04T13:42:57.366 INFO:tasks.workunit.client.1.smithi169.stdout:0/493: sync 2024-03-04T13:42:57.369 INFO:tasks.workunit.client.1.smithi169.stdout:8/611: creat d7/d10/fcc x:0 0 0 2024-03-04T13:42:57.370 INFO:tasks.workunit.client.1.smithi169.stdout:4/541: truncate d4/d11/d2e/d39/f68 1255025 0 2024-03-04T13:42:57.370 INFO:tasks.workunit.client.1.smithi169.stdout:0/494: fdatasync d5/d9/d3d/d59/d64/f81 0 2024-03-04T13:42:57.373 INFO:tasks.workunit.client.1.smithi169.stdout:3/621: truncate d7/de/d2b/d7d/d85/d13/d1a/d98/d9a/f8e 810663 0 2024-03-04T13:42:57.373 INFO:tasks.workunit.client.1.smithi169.stdout:9/627: write d7/dc/f79 [243045,100971] 0 2024-03-04T13:42:57.376 INFO:tasks.workunit.client.1.smithi169.stdout:7/673: unlink d2/de/d37/f3f 0 2024-03-04T13:42:57.380 INFO:tasks.workunit.client.1.smithi169.stdout:2/648: rename d1/d6d/f87 to d1/d5/d22/d45/d4a/fe5 0 2024-03-04T13:42:57.382 INFO:tasks.workunit.client.1.smithi169.stdout:2/649: write d1/fc0 [229409,7085] 0 2024-03-04T13:42:57.384 INFO:tasks.workunit.client.1.smithi169.stdout:2/650: readlink d1/la8 0 2024-03-04T13:42:57.387 INFO:tasks.workunit.client.1.smithi169.stdout:5/687: mkdir d1/d16/d17/d1b/d74/ddc/de4 0 2024-03-04T13:42:57.388 INFO:tasks.workunit.client.1.smithi169.stdout:8/612: symlink d7/d10/d17/d44/d4c/d93/lcd 0 2024-03-04T13:42:57.389 INFO:tasks.workunit.client.1.smithi169.stdout:3/622: dwrite d7/f49 [0,4194304] 0 2024-03-04T13:42:57.391 INFO:tasks.workunit.client.1.smithi169.stdout:1/568: mkdir d4/d5/d36/d75/dba 0 2024-03-04T13:42:57.400 INFO:tasks.workunit.client.1.smithi169.stdout:6/600: dread d0/d32/d3a/d43/f45 [0,4194304] 0 2024-03-04T13:42:57.400 INFO:tasks.workunit.client.1.smithi169.stdout:1/569: write d4/d5/d36/d35/d16/d3a/f9c [817221,63967] 0 2024-03-04T13:42:57.400 INFO:tasks.workunit.client.1.smithi169.stdout:9/628: symlink d7/d25/ld7 0 2024-03-04T13:42:57.404 INFO:tasks.workunit.client.1.smithi169.stdout:6/601: fdatasync d0/d32/d3a/d5e/fd4 0 2024-03-04T13:42:57.404 INFO:tasks.workunit.client.1.smithi169.stdout:5/688: creat d1/d16/dbd/dbb/dcc/fe5 x:0 0 0 2024-03-04T13:42:57.406 INFO:tasks.workunit.client.1.smithi169.stdout:8/613: fsync d7/d10/d17/d44/d5d/fbb 0 2024-03-04T13:42:57.407 INFO:tasks.workunit.client.0.smithi107.stdout:1/56: rmdir d1/d3/d9 0 2024-03-04T13:42:57.408 INFO:tasks.workunit.client.0.smithi107.stdout:0/121: sync 2024-03-04T13:42:57.413 INFO:tasks.workunit.client.1.smithi169.stdout:7/674: dread d2/d15/d17/f42 [0,4194304] 0 2024-03-04T13:42:57.413 INFO:tasks.workunit.client.1.smithi169.stdout:9/629: mknod d7/dc/d12/d8a/cd8 0 2024-03-04T13:42:57.413 INFO:tasks.workunit.client.0.smithi107.stdout:2/106: getdents da 0 2024-03-04T13:42:57.415 INFO:tasks.workunit.client.0.smithi107.stdout:0/122: chown da/f27 3814533 1 2024-03-04T13:42:57.415 INFO:tasks.workunit.client.0.smithi107.stdout:4/69: creat d0/d9/d10/f15 x:0 0 0 2024-03-04T13:42:57.416 INFO:tasks.workunit.client.1.smithi169.stdout:1/570: creat d4/d5/d36/d75/dba/fbb x:0 0 0 2024-03-04T13:42:57.417 INFO:tasks.workunit.client.0.smithi107.stdout:9/53: dread d4/f8 [0,4194304] 0 2024-03-04T13:42:57.420 INFO:tasks.workunit.client.1.smithi169.stdout:4/542: dwrite d4/f15 [0,4194304] 0 2024-03-04T13:42:57.423 INFO:tasks.workunit.client.1.smithi169.stdout:0/495: dread d5/d9/d3d/f48 [0,4194304] 0 2024-03-04T13:42:57.424 INFO:tasks.workunit.client.0.smithi107.stdout:8/136: dwrite d6/d16/f24 [0,4194304] 0 2024-03-04T13:42:57.425 INFO:tasks.workunit.client.0.smithi107.stdout:6/97: truncate fd 1083461 0 2024-03-04T13:42:57.429 INFO:tasks.workunit.client.1.smithi169.stdout:3/623: rename d7/de/d2b/d7d/d85/d47/c87 to d7/cd6 0 2024-03-04T13:42:57.431 INFO:tasks.workunit.client.0.smithi107.stdout:2/107: creat da/df/f24 x:0 0 0 2024-03-04T13:42:57.432 INFO:tasks.workunit.client.0.smithi107.stdout:7/90: dwrite d6/fe [0,4194304] 0 2024-03-04T13:42:57.432 INFO:tasks.workunit.client.1.smithi169.stdout:8/614: mknod d7/d10/d22/d75/d81/da2/d96/cce 0 2024-03-04T13:42:57.437 INFO:tasks.workunit.client.1.smithi169.stdout:6/602: write d0/d3c/d40/f51 [2789977,85520] 0 2024-03-04T13:42:57.437 INFO:tasks.workunit.client.1.smithi169.stdout:2/651: getdents d1 0 2024-03-04T13:42:57.440 INFO:tasks.workunit.client.0.smithi107.stdout:1/57: dread d1/d3/f5 [0,4194304] 0 2024-03-04T13:42:57.442 INFO:tasks.workunit.client.1.smithi169.stdout:4/543: dwrite d4/f5a [0,4194304] 0 2024-03-04T13:42:57.443 INFO:tasks.workunit.client.0.smithi107.stdout:1/58: dread - d1/fe zero size 2024-03-04T13:42:57.443 INFO:tasks.workunit.client.0.smithi107.stdout:8/137: mkdir d6/da/d25 0 2024-03-04T13:42:57.445 INFO:tasks.workunit.client.0.smithi107.stdout:6/98: mkdir df/d17/d24 0 2024-03-04T13:42:57.446 INFO:tasks.workunit.client.0.smithi107.stdout:2/108: mknod da/c25 0 2024-03-04T13:42:57.449 INFO:tasks.workunit.client.0.smithi107.stdout:7/91: unlink d6/f1a 0 2024-03-04T13:42:57.449 INFO:tasks.workunit.client.1.smithi169.stdout:5/689: getdents d1/d16/d2f/d72/db8/dce 0 2024-03-04T13:42:57.452 INFO:tasks.workunit.client.0.smithi107.stdout:0/123: rename da/l21 to da/dc/dd/l2d 0 2024-03-04T13:42:57.455 INFO:tasks.workunit.client.1.smithi169.stdout:3/624: mkdir d7/de/d2b/d7d/d85/d13/d1a/d2d/d56/d5e/d67/dd7 0 2024-03-04T13:42:57.457 INFO:tasks.workunit.client.0.smithi107.stdout:0/124: chown da/f2a 2484 1 2024-03-04T13:42:57.459 INFO:tasks.workunit.client.0.smithi107.stdout:9/54: dwrite d4/f8 [4194304,4194304] 0 2024-03-04T13:42:57.463 INFO:tasks.workunit.client.0.smithi107.stdout:0/125: write da/de/f11 [8452763,763] 0 2024-03-04T13:42:57.464 INFO:tasks.workunit.client.1.smithi169.stdout:2/652: dread - d1/d5/d19/d63/f7b zero size 2024-03-04T13:42:57.465 INFO:tasks.workunit.client.0.smithi107.stdout:4/70: rename d0/ld to d0/l16 0 2024-03-04T13:42:57.466 INFO:tasks.workunit.client.1.smithi169.stdout:6/603: fsync d0/f35 0 2024-03-04T13:42:57.469 INFO:tasks.workunit.client.1.smithi169.stdout:0/496: dwrite d5/d9/dc/d12/d15/d19/d61/f2c [0,4194304] 0 2024-03-04T13:42:57.471 INFO:tasks.workunit.client.1.smithi169.stdout:0/497: fdatasync d5/d9/f2d 0 2024-03-04T13:42:57.472 INFO:tasks.workunit.client.0.smithi107.stdout:1/59: dread d1/d3/f5 [0,4194304] 0 2024-03-04T13:42:57.473 INFO:tasks.workunit.client.0.smithi107.stdout:9/55: rename d4/c7 to d4/ce 0 2024-03-04T13:42:57.473 INFO:tasks.workunit.client.0.smithi107.stdout:1/60: chown d1 33645 1 2024-03-04T13:42:57.473 INFO:tasks.workunit.client.0.smithi107.stdout:8/138: dwrite f4 [0,4194304] 0 2024-03-04T13:42:57.473 INFO:tasks.workunit.client.0.smithi107.stdout:9/56: readlink l1 0 2024-03-04T13:42:57.474 INFO:tasks.workunit.client.0.smithi107.stdout:2/109: dread da/fb [0,4194304] 0 2024-03-04T13:42:57.475 INFO:tasks.workunit.client.1.smithi169.stdout:5/690: mknod d1/d16/d2a/d62/ce6 0 2024-03-04T13:42:57.475 INFO:tasks.workunit.client.0.smithi107.stdout:8/139: readlink d6/l7 0 2024-03-04T13:42:57.476 INFO:tasks.workunit.client.0.smithi107.stdout:2/110: chown da 4233378 1 2024-03-04T13:42:57.478 INFO:tasks.workunit.client.1.smithi169.stdout:9/630: creat d7/d25/fd9 x:0 0 0 2024-03-04T13:42:57.479 INFO:tasks.workunit.client.0.smithi107.stdout:0/126: mknod da/dc/c2e 0 2024-03-04T13:42:57.484 INFO:tasks.workunit.client.0.smithi107.stdout:9/57: creat d4/ff x:0 0 0 2024-03-04T13:42:57.485 INFO:tasks.workunit.client.0.smithi107.stdout:1/61: fdatasync d1/d3/f5 0 2024-03-04T13:42:57.486 INFO:tasks.workunit.client.1.smithi169.stdout:7/675: link d2/d15/d54/caf d2/de/d3e/d41/d69/d78/ce5 0 2024-03-04T13:42:57.486 INFO:tasks.workunit.client.1.smithi169.stdout:8/615: mknod d7/d10/d17/d44/d36/d62/dc1/ccf 0 2024-03-04T13:42:57.487 INFO:tasks.workunit.client.0.smithi107.stdout:0/127: dread f8 [0,4194304] 0 2024-03-04T13:42:57.487 INFO:tasks.workunit.client.0.smithi107.stdout:1/62: fdatasync d1/d3/f5 0 2024-03-04T13:42:57.487 INFO:tasks.workunit.client.1.smithi169.stdout:6/604: symlink d0/d3c/d40/d52/ldb 0 2024-03-04T13:42:57.491 INFO:tasks.workunit.client.1.smithi169.stdout:0/498: mkdir d5/d9/dc/d12/d15/d19/d77/db2 0 2024-03-04T13:42:57.491 INFO:tasks.workunit.client.1.smithi169.stdout:7/676: write d2/de/f2a [3565202,8149] 0 2024-03-04T13:42:57.491 INFO:tasks.workunit.client.1.smithi169.stdout:3/625: unlink d7/de/d24/c45 0 2024-03-04T13:42:57.494 INFO:tasks.workunit.client.1.smithi169.stdout:5/691: creat d1/d16/dbd/dbb/dcc/fe7 x:0 0 0 2024-03-04T13:42:57.495 INFO:tasks.workunit.client.0.smithi107.stdout:9/58: symlink d4/da/l10 0 2024-03-04T13:42:57.495 INFO:tasks.workunit.client.0.smithi107.stdout:4/71: dwrite d0/d9/d10/f15 [0,4194304] 0 2024-03-04T13:42:57.496 INFO:tasks.workunit.client.0.smithi107.stdout:9/59: fsync f0 0 2024-03-04T13:42:57.499 INFO:tasks.workunit.client.1.smithi169.stdout:9/631: dwrite d7/dc/d3e/da3/fc9 [0,4194304] 0 2024-03-04T13:42:57.500 INFO:tasks.workunit.client.1.smithi169.stdout:8/616: truncate d7/d10/d22/d46/da5/da6/fb4 490584 0 2024-03-04T13:42:57.503 INFO:tasks.workunit.client.1.smithi169.stdout:8/617: fsync d7/d10/d22/d46/da5/da6/fb4 0 2024-03-04T13:42:57.504 INFO:tasks.workunit.client.0.smithi107.stdout:1/63: mkdir d1/d3/df 0 2024-03-04T13:42:57.505 INFO:tasks.workunit.client.1.smithi169.stdout:1/571: link d4/d5/d2d/d4e/d71/d26/d54/c85 d4/d5/d36/d35/d5b/cbc 0 2024-03-04T13:42:57.508 INFO:tasks.workunit.client.1.smithi169.stdout:1/572: stat d4/d5/d36/d35/df/ca0 0 2024-03-04T13:42:57.510 INFO:tasks.workunit.client.0.smithi107.stdout:8/140: link d6/d16/f22 d6/f26 0 2024-03-04T13:42:57.512 INFO:tasks.workunit.client.1.smithi169.stdout:6/605: creat d0/d32/d70/fdc x:0 0 0 2024-03-04T13:42:57.517 INFO:tasks.workunit.client.1.smithi169.stdout:7/677: dwrite d2/de/d3e/d41/f8b [4194304,4194304] 0 2024-03-04T13:42:57.517 INFO:tasks.workunit.client.1.smithi169.stdout:8/618: read d7/d12/f5e [3334602,86777] 0 2024-03-04T13:42:57.518 INFO:tasks.workunit.client.1.smithi169.stdout:3/626: dwrite d7/fb [0,4194304] 0 2024-03-04T13:42:57.521 INFO:tasks.workunit.client.0.smithi107.stdout:4/72: dwrite d0/f3 [0,4194304] 0 2024-03-04T13:42:57.524 INFO:tasks.workunit.client.0.smithi107.stdout:6/99: truncate df/d17/f1e 1651325 0 2024-03-04T13:42:57.525 INFO:tasks.workunit.client.0.smithi107.stdout:9/60: dwrite f3 [0,4194304] 0 2024-03-04T13:42:57.526 INFO:tasks.workunit.client.0.smithi107.stdout:6/100: fsync fa 0 2024-03-04T13:42:57.539 INFO:tasks.workunit.client.1.smithi169.stdout:5/692: creat d1/d16/d17/d2d/d5a/d8a/fe8 x:0 0 0 2024-03-04T13:42:57.545 INFO:tasks.workunit.client.1.smithi169.stdout:4/544: getdents d4/d1c/d75 0 2024-03-04T13:42:57.548 INFO:tasks.workunit.client.1.smithi169.stdout:8/619: dwrite d7/d10/d22/d75/db9/fba [0,4194304] 0 2024-03-04T13:42:57.555 INFO:tasks.workunit.client.1.smithi169.stdout:6/606: mknod d0/d3c/d40/cdd 0 2024-03-04T13:42:57.557 INFO:tasks.workunit.client.1.smithi169.stdout:7/678: creat d2/d28/fe6 x:0 0 0 2024-03-04T13:42:57.559 INFO:tasks.workunit.client.1.smithi169.stdout:1/573: truncate d4/d5/d2d/d4e/d71/d26/d4f/f51 3440961 0 2024-03-04T13:42:57.561 INFO:tasks.workunit.client.1.smithi169.stdout:1/574: fsync d4/d5/d36/d35/d8f/f9a 0 2024-03-04T13:42:57.562 INFO:tasks.workunit.client.1.smithi169.stdout:3/627: unlink d7/de/d2b/d7d/d85/d13/d5a/d65/da4/faa 0 2024-03-04T13:42:57.564 INFO:tasks.workunit.client.1.smithi169.stdout:2/653: getdents d1/d5 0 2024-03-04T13:42:57.565 INFO:tasks.workunit.client.1.smithi169.stdout:1/575: dread - d4/d5/d36/f9f zero size 2024-03-04T13:42:57.567 INFO:tasks.workunit.client.1.smithi169.stdout:9/632: mknod d7/d2d/cda 0 2024-03-04T13:42:57.568 INFO:tasks.workunit.client.1.smithi169.stdout:3/628: chown d7/de/d2b/d7d/d85/d18/d21/d3f/c61 0 1 2024-03-04T13:42:57.569 INFO:tasks.workunit.client.0.smithi107.stdout:3/61: sync 2024-03-04T13:42:57.570 INFO:tasks.workunit.client.1.smithi169.stdout:5/693: rmdir d1/d23 39 2024-03-04T13:42:57.571 INFO:tasks.workunit.client.0.smithi107.stdout:8/141: link f4 d6/d16/f27 0 2024-03-04T13:42:57.571 INFO:tasks.workunit.client.0.smithi107.stdout:9/61: rmdir d4/da 39 2024-03-04T13:42:57.572 INFO:tasks.workunit.client.0.smithi107.stdout:6/101: unlink df/c12 0 2024-03-04T13:42:57.575 INFO:tasks.workunit.client.1.smithi169.stdout:0/499: write d5/d9/dc/d12/d15/d19/d61/f5a [4076024,20938] 0 2024-03-04T13:42:57.575 INFO:tasks.workunit.client.1.smithi169.stdout:3/629: chown d7/de/d2b/d7d/d85/d13/d1a/d2d/d56/d5e/d67/lc5 5 1 2024-03-04T13:42:57.575 INFO:tasks.workunit.client.1.smithi169.stdout:6/607: creat d0/d3c/d40/fde x:0 0 0 2024-03-04T13:42:57.576 INFO:tasks.workunit.client.0.smithi107.stdout:6/102: dread - f3 zero size 2024-03-04T13:42:57.576 INFO:tasks.workunit.client.0.smithi107.stdout:1/64: dwrite d1/d3/f5 [0,4194304] 0 2024-03-04T13:42:57.576 INFO:tasks.workunit.client.0.smithi107.stdout:3/62: chown d1/de/d11/d12 6 1 2024-03-04T13:42:57.577 INFO:tasks.workunit.client.1.smithi169.stdout:5/694: read d1/d16/d17/d1b/f29 [4521460,79407] 0 2024-03-04T13:42:57.578 INFO:tasks.workunit.client.0.smithi107.stdout:4/73: symlink d0/l17 0 2024-03-04T13:42:57.578 INFO:tasks.workunit.client.0.smithi107.stdout:7/92: dwrite f5 [0,4194304] 0 2024-03-04T13:42:57.580 INFO:tasks.workunit.client.0.smithi107.stdout:8/142: fdatasync f5 0 2024-03-04T13:42:57.582 INFO:tasks.workunit.client.1.smithi169.stdout:5/695: truncate d1/d16/d2f/d39/fb3 39743 0 2024-03-04T13:42:57.586 INFO:tasks.workunit.client.1.smithi169.stdout:5/696: chown d1/l11 1963 1 2024-03-04T13:42:57.589 INFO:tasks.workunit.client.1.smithi169.stdout:9/633: dwrite d7/d25/d64/fa7 [0,4194304] 0 2024-03-04T13:42:57.594 INFO:tasks.workunit.client.0.smithi107.stdout:6/103: dread fb [0,4194304] 0 2024-03-04T13:42:57.595 INFO:tasks.workunit.client.1.smithi169.stdout:8/620: symlink d7/d10/d22/ld0 0 2024-03-04T13:42:57.597 INFO:tasks.workunit.client.1.smithi169.stdout:0/500: mknod d5/d9/cb3 0 2024-03-04T13:42:57.597 INFO:tasks.workunit.client.0.smithi107.stdout:7/93: unlink d6/l1b 0 2024-03-04T13:42:57.600 INFO:tasks.workunit.client.1.smithi169.stdout:4/545: dwrite d4/d11/d2e/d37/d5c/d86/f99 [0,4194304] 0 2024-03-04T13:42:57.601 INFO:tasks.workunit.client.0.smithi107.stdout:9/62: mkdir d4/da/d11 0 2024-03-04T13:42:57.602 INFO:tasks.workunit.client.1.smithi169.stdout:6/608: creat d0/d3c/d40/d52/dac/fdf x:0 0 0 2024-03-04T13:42:57.603 INFO:tasks.workunit.client.0.smithi107.stdout:3/63: dwrite d1/f3 [0,4194304] 0 2024-03-04T13:42:57.604 INFO:tasks.workunit.client.0.smithi107.stdout:9/63: chown d4/f9 0 1 2024-03-04T13:42:57.606 INFO:tasks.workunit.client.1.smithi169.stdout:4/546: write d4/d11/d3b/d67/fa0 [349834,14785] 0 2024-03-04T13:42:57.607 INFO:tasks.workunit.client.1.smithi169.stdout:1/576: symlink d4/d5/d36/d35/d5b/lbd 0 2024-03-04T13:42:57.608 INFO:tasks.workunit.client.0.smithi107.stdout:9/64: dread - d4/ff zero size 2024-03-04T13:42:57.609 INFO:tasks.workunit.client.1.smithi169.stdout:4/547: dread d4/d1c/f50 [0,4194304] 0 2024-03-04T13:42:57.610 INFO:tasks.workunit.client.0.smithi107.stdout:1/65: mkdir d1/d10 0 2024-03-04T13:42:57.613 INFO:tasks.workunit.client.0.smithi107.stdout:4/74: dwrite d0/d9/f11 [0,4194304] 0 2024-03-04T13:42:57.614 INFO:tasks.workunit.client.1.smithi169.stdout:9/634: unlink d7/c1b 0 2024-03-04T13:42:57.614 INFO:tasks.workunit.client.0.smithi107.stdout:1/66: truncate d1/fe 359259 0 2024-03-04T13:42:57.615 INFO:tasks.workunit.client.0.smithi107.stdout:1/67: fdatasync d1/fe 0 2024-03-04T13:42:57.616 INFO:tasks.workunit.client.1.smithi169.stdout:7/679: creat d2/de/fe7 x:0 0 0 2024-03-04T13:42:57.619 INFO:tasks.workunit.client.1.smithi169.stdout:7/680: chown d2/de/d3e/d41/f5a 0 1 2024-03-04T13:42:57.622 INFO:tasks.workunit.client.1.smithi169.stdout:7/681: write d2/de/d3e/d41/fa9 [889668,75395] 0 2024-03-04T13:42:57.624 INFO:tasks.workunit.client.0.smithi107.stdout:9/65: dread d4/f8 [4194304,4194304] 0 2024-03-04T13:42:57.625 INFO:tasks.workunit.client.1.smithi169.stdout:3/630: write d7/de/d2b/d7d/d85/d18/f27 [1355202,7487] 0 2024-03-04T13:42:57.625 INFO:tasks.workunit.client.0.smithi107.stdout:9/66: readlink l1 0 2024-03-04T13:42:57.626 INFO:tasks.workunit.client.0.smithi107.stdout:6/104: mknod df/d21/c25 0 2024-03-04T13:42:57.629 INFO:tasks.workunit.client.1.smithi169.stdout:3/631: stat d7/ld 0 2024-03-04T13:42:57.629 INFO:tasks.workunit.client.1.smithi169.stdout:4/548: dread d4/f5a [0,4194304] 0 2024-03-04T13:42:57.631 INFO:tasks.workunit.client.0.smithi107.stdout:7/94: mkdir d6/d13/d16/d19/d1e/d22 0 2024-03-04T13:42:57.631 INFO:tasks.workunit.client.1.smithi169.stdout:2/654: link d1/d5/d19/d63/f7b d1/d5/d19/d3c/db8/de1/fe6 0 2024-03-04T13:42:57.632 INFO:tasks.workunit.client.1.smithi169.stdout:4/549: stat d4/d11/d2e/d6c/d78/da5/d77 0 2024-03-04T13:42:57.635 INFO:tasks.workunit.client.0.smithi107.stdout:1/68: rename d1/d3/f5 to d1/d3/f11 0 2024-03-04T13:42:57.636 INFO:tasks.workunit.client.1.smithi169.stdout:9/635: creat d7/dc/d12/d20/da6/fdb x:0 0 0 2024-03-04T13:42:57.638 INFO:tasks.workunit.client.0.smithi107.stdout:3/64: link d1/f3 d1/de/d11/d12/f15 0 2024-03-04T13:42:57.638 INFO:tasks.workunit.client.0.smithi107.stdout:4/75: dwrite d0/d9/d10/f15 [4194304,4194304] 0 2024-03-04T13:42:57.639 INFO:tasks.workunit.client.0.smithi107.stdout:1/69: write d1/fe [608178,64570] 0 2024-03-04T13:42:57.645 INFO:tasks.workunit.client.0.smithi107.stdout:6/105: rename df/d1b/f1c to df/d21/f26 0 2024-03-04T13:42:57.645 INFO:tasks.workunit.client.0.smithi107.stdout:5/50: sync 2024-03-04T13:42:57.646 INFO:tasks.workunit.client.0.smithi107.stdout:3/65: rename d1/de/d11 to d1/de/d11/d12/d16 22 2024-03-04T13:42:57.646 INFO:tasks.workunit.client.0.smithi107.stdout:1/70: dread d1/d3/f11 [4194304,4194304] 0 2024-03-04T13:42:57.647 INFO:tasks.workunit.client.0.smithi107.stdout:8/143: getdents d6/d16/d1f 0 2024-03-04T13:42:57.647 INFO:tasks.workunit.client.0.smithi107.stdout:9/67: dwrite d4/ff [0,4194304] 0 2024-03-04T13:42:57.648 INFO:tasks.workunit.client.0.smithi107.stdout:6/106: chown df/d17/l1f 194 1 2024-03-04T13:42:57.649 INFO:tasks.workunit.client.1.smithi169.stdout:3/632: mkdir d7/de/d2b/d7d/d85/d13/d5a/dd8 0 2024-03-04T13:42:57.657 INFO:tasks.workunit.client.1.smithi169.stdout:2/655: dwrite d1/d5/d22/d45/d4a/f5d [0,4194304] 0 2024-03-04T13:42:57.661 INFO:tasks.workunit.client.1.smithi169.stdout:8/621: rename d7/d10/d22/ld0 to d7/d10/d17/d44/d36/d62/ld1 0 2024-03-04T13:42:57.664 INFO:tasks.workunit.client.0.smithi107.stdout:3/66: dread d1/de/d11/d12/f15 [0,4194304] 0 2024-03-04T13:42:57.667 INFO:tasks.workunit.client.1.smithi169.stdout:6/609: dwrite d0/d32/d70/fcc [0,4194304] 0 2024-03-04T13:42:57.668 INFO:tasks.workunit.client.0.smithi107.stdout:1/71: rmdir d1/d3 39 2024-03-04T13:42:57.668 INFO:tasks.workunit.client.0.smithi107.stdout:8/144: dwrite f3 [0,4194304] 0 2024-03-04T13:42:57.670 INFO:tasks.workunit.client.1.smithi169.stdout:6/610: chown d0/d32/d3a/d43/d74/fd8 14708928 1 2024-03-04T13:42:57.672 INFO:tasks.workunit.client.0.smithi107.stdout:5/51: rename l3 to d5/d8/l13 0 2024-03-04T13:42:57.673 INFO:tasks.workunit.client.1.smithi169.stdout:9/636: symlink d7/dc/d12/d32/d42/ldc 0 2024-03-04T13:42:57.675 INFO:tasks.workunit.client.0.smithi107.stdout:9/68: dwrite f0 [0,4194304] 0 2024-03-04T13:42:57.676 INFO:tasks.workunit.client.1.smithi169.stdout:7/682: chown d2/dc0/fc7 277 1 2024-03-04T13:42:57.678 INFO:tasks.workunit.client.0.smithi107.stdout:4/76: mkdir d0/d9/d18 0 2024-03-04T13:42:57.678 INFO:tasks.workunit.client.0.smithi107.stdout:9/69: read d4/ff [3661971,55895] 0 2024-03-04T13:42:57.678 INFO:tasks.workunit.client.0.smithi107.stdout:3/67: symlink d1/de/d11/l17 0 2024-03-04T13:42:57.679 INFO:tasks.workunit.client.0.smithi107.stdout:8/145: symlink d6/d16/d1f/l28 0 2024-03-04T13:42:57.680 INFO:tasks.workunit.client.0.smithi107.stdout:9/70: fsync d4/f9 0 2024-03-04T13:42:57.680 INFO:tasks.workunit.client.0.smithi107.stdout:8/146: stat d6/da/l13 0 2024-03-04T13:42:57.685 INFO:tasks.workunit.client.1.smithi169.stdout:8/622: creat d7/d10/d17/d44/fd2 x:0 0 0 2024-03-04T13:42:57.687 INFO:tasks.workunit.client.1.smithi169.stdout:0/501: getdents d5/d9/dc/d12/da5 0 2024-03-04T13:42:57.690 INFO:tasks.workunit.client.0.smithi107.stdout:6/107: rename df/d21 to df/d17/d24/d27 0 2024-03-04T13:42:57.693 INFO:tasks.workunit.client.0.smithi107.stdout:4/77: readlink d0/l16 0 2024-03-04T13:42:57.693 INFO:tasks.workunit.client.0.smithi107.stdout:6/108: write fe [270833,95052] 0 2024-03-04T13:42:57.694 INFO:tasks.workunit.client.1.smithi169.stdout:9/637: rmdir d7/d2d/d48/d66 39 2024-03-04T13:42:57.697 INFO:tasks.workunit.client.0.smithi107.stdout:3/68: mkdir d1/de/d18 0 2024-03-04T13:42:57.701 INFO:tasks.workunit.client.1.smithi169.stdout:0/502: creat d5/d9/dc/d12/d15/d46/fb4 x:0 0 0 2024-03-04T13:42:57.702 INFO:tasks.workunit.client.1.smithi169.stdout:4/550: dwrite d4/f7 [0,4194304] 0 2024-03-04T13:42:57.706 INFO:tasks.workunit.client.0.smithi107.stdout:8/147: dwrite d6/fe [0,4194304] 0 2024-03-04T13:42:57.706 INFO:tasks.workunit.client.1.smithi169.stdout:4/551: dread - d4/d11/d2e/d37/f87 zero size 2024-03-04T13:42:57.707 INFO:tasks.workunit.client.0.smithi107.stdout:4/78: creat d0/d9/d10/f19 x:0 0 0 2024-03-04T13:42:57.708 INFO:tasks.workunit.client.1.smithi169.stdout:4/552: chown d4/d11/d2e/d21/d5e/f76 5 1 2024-03-04T13:42:57.710 INFO:tasks.workunit.client.1.smithi169.stdout:1/577: rename d4/d5/d2d/d4e/f84 to d4/d5/d36/d35/d5b/fbe 0 2024-03-04T13:42:57.712 INFO:tasks.workunit.client.0.smithi107.stdout:5/52: dwrite d5/dc/df/f12 [0,4194304] 0 2024-03-04T13:42:57.713 INFO:tasks.workunit.client.1.smithi169.stdout:1/578: chown d4/d5/l31 1698 1 2024-03-04T13:42:57.713 INFO:tasks.workunit.client.0.smithi107.stdout:3/69: creat d1/de/f19 x:0 0 0 2024-03-04T13:42:57.715 INFO:tasks.workunit.client.1.smithi169.stdout:1/579: fsync d4/d5/d36/d35/d16/d3a/f9c 0 2024-03-04T13:42:57.717 INFO:tasks.workunit.client.1.smithi169.stdout:0/503: truncate d5/d9/dc/d12/f62 305105 0 2024-03-04T13:42:57.719 INFO:tasks.workunit.client.1.smithi169.stdout:4/553: creat d4/d1c/d58/faf x:0 0 0 2024-03-04T13:42:57.722 INFO:tasks.workunit.client.1.smithi169.stdout:4/554: write d4/f9 [905290,104418] 0 2024-03-04T13:42:57.723 INFO:tasks.workunit.client.0.smithi107.stdout:5/53: write d5/f9 [4826824,94062] 0 2024-03-04T13:42:57.724 INFO:tasks.workunit.client.1.smithi169.stdout:3/633: rename d7/de/d2b/d7d/d85/d13/d5a/d65/fd4 to d7/de/d2b/d7d/d85/d18/d21/d66/fd9 0 2024-03-04T13:42:57.725 INFO:tasks.workunit.client.1.smithi169.stdout:7/683: truncate d2/de/d3e/d41/f71 2137481 0 2024-03-04T13:42:57.727 INFO:tasks.workunit.client.1.smithi169.stdout:6/611: getdents d0/d3c/d40/d52/dac 0 2024-03-04T13:42:57.730 INFO:tasks.workunit.client.1.smithi169.stdout:8/623: dread d7/d10/d17/d44/d4c/f7d [0,4194304] 0 2024-03-04T13:42:57.732 INFO:tasks.workunit.client.0.smithi107.stdout:3/70: mknod d1/de/d18/c1a 0 2024-03-04T13:42:57.734 INFO:tasks.workunit.client.0.smithi107.stdout:5/54: write d5/d8/f11 [1126132,93394] 0 2024-03-04T13:42:57.739 INFO:tasks.workunit.client.1.smithi169.stdout:2/656: rename d1/d5/d22/d9e to d1/d5/d22/d45/d4a/de7 0 2024-03-04T13:42:57.740 INFO:tasks.workunit.client.0.smithi107.stdout:6/109: dwrite f6 [0,4194304] 0 2024-03-04T13:42:57.741 INFO:tasks.workunit.client.0.smithi107.stdout:9/71: dwrite d4/f8 [0,4194304] 0 2024-03-04T13:42:57.742 INFO:tasks.workunit.client.1.smithi169.stdout:7/684: creat d2/de/d3e/d41/fe8 x:0 0 0 2024-03-04T13:42:57.743 INFO:tasks.workunit.client.0.smithi107.stdout:8/148: dwrite d6/fe [0,4194304] 0 2024-03-04T13:42:57.746 INFO:tasks.workunit.client.0.smithi107.stdout:3/71: symlink d1/l1b 0 2024-03-04T13:42:57.748 INFO:tasks.workunit.client.0.smithi107.stdout:3/72: stat d1/de/d11/d12 0 2024-03-04T13:42:57.749 INFO:tasks.workunit.client.1.smithi169.stdout:3/634: creat d7/db8/fda x:0 0 0 2024-03-04T13:42:57.750 INFO:tasks.workunit.client.0.smithi107.stdout:9/72: fsync d4/ff 0 2024-03-04T13:42:57.752 INFO:tasks.workunit.client.1.smithi169.stdout:8/624: chown d7/d10/d24/d3e/l78 199387680 1 2024-03-04T13:42:57.755 INFO:tasks.workunit.client.1.smithi169.stdout:8/625: write d7/d10/d24/f71 [1991542,97715] 0 2024-03-04T13:42:57.756 INFO:tasks.workunit.client.0.smithi107.stdout:9/73: readlink d4/da/lb 0 2024-03-04T13:42:57.756 INFO:tasks.workunit.client.0.smithi107.stdout:6/110: creat df/d17/f28 x:0 0 0 2024-03-04T13:42:57.760 INFO:tasks.workunit.client.1.smithi169.stdout:8/626: fsync d7/d10/d17/d44/d4c/fa3 0 2024-03-04T13:42:57.760 INFO:tasks.workunit.client.1.smithi169.stdout:9/638: rename d7/dc/d12/d32/d42/l78 to d7/dc/d12/d8a/ldd 0 2024-03-04T13:42:57.762 INFO:tasks.workunit.client.0.smithi107.stdout:4/79: truncate d0/d9/de/d14/fa 357415 0 2024-03-04T13:42:57.762 INFO:tasks.workunit.client.1.smithi169.stdout:2/657: stat d1/d5/d22/d45/cc2 0 2024-03-04T13:42:57.765 INFO:tasks.workunit.client.0.smithi107.stdout:3/73: rename d1/fd to d1/de/d11/f1c 0 2024-03-04T13:42:57.766 INFO:tasks.workunit.client.0.smithi107.stdout:3/74: write d1/fa [952188,81915] 0 2024-03-04T13:42:57.766 INFO:tasks.workunit.client.1.smithi169.stdout:2/658: write d1/d5/d22/d45/d4a/f70 [3522750,91044] 0 2024-03-04T13:42:57.767 INFO:tasks.workunit.client.0.smithi107.stdout:3/75: readlink d1/l8 0 2024-03-04T13:42:57.767 INFO:tasks.workunit.client.0.smithi107.stdout:9/74: rmdir d4/da 39 2024-03-04T13:42:57.767 INFO:tasks.workunit.client.0.smithi107.stdout:9/75: fdatasync f3 0 2024-03-04T13:42:57.769 INFO:tasks.workunit.client.0.smithi107.stdout:9/76: read d4/f9 [332883,62357] 0 2024-03-04T13:42:57.770 INFO:tasks.workunit.client.0.smithi107.stdout:3/76: truncate d1/de/d11/f1c 1247460 0 2024-03-04T13:42:57.770 INFO:tasks.workunit.client.1.smithi169.stdout:3/635: creat d7/de/d2b/d7d/d85/d13/d1a/d78/fdb x:0 0 0 2024-03-04T13:42:57.771 INFO:tasks.workunit.client.0.smithi107.stdout:3/77: fdatasync d1/de/f19 0 2024-03-04T13:42:57.771 INFO:tasks.workunit.client.1.smithi169.stdout:4/555: dread d4/d11/d2e/d63/f8a [0,4194304] 0 2024-03-04T13:42:57.773 INFO:tasks.workunit.client.0.smithi107.stdout:0/128: sync 2024-03-04T13:42:57.774 INFO:tasks.workunit.client.0.smithi107.stdout:2/111: sync 2024-03-04T13:42:57.774 INFO:tasks.workunit.client.1.smithi169.stdout:3/636: write d7/de/d2b/d7d/d85/d13/d1a/d2d/d56/d5e/d5f/fc8 [530978,13021] 0 2024-03-04T13:42:57.776 INFO:tasks.workunit.client.1.smithi169.stdout:9/639: creat d7/d27/fde x:0 0 0 2024-03-04T13:42:57.778 INFO:tasks.workunit.client.0.smithi107.stdout:2/112: write da/f1c [932073,90053] 0 2024-03-04T13:42:57.779 INFO:tasks.workunit.client.0.smithi107.stdout:6/111: rename fa to df/f29 0 2024-03-04T13:42:57.780 INFO:tasks.workunit.client.0.smithi107.stdout:6/112: write df/d17/f23 [572328,88690] 0 2024-03-04T13:42:57.782 INFO:tasks.workunit.client.0.smithi107.stdout:8/149: creat d6/f29 x:0 0 0 2024-03-04T13:42:57.783 INFO:tasks.workunit.client.1.smithi169.stdout:1/580: write d4/d5/d2d/f6c [2817897,72662] 0 2024-03-04T13:42:57.783 INFO:tasks.workunit.client.0.smithi107.stdout:8/150: dread - d6/f29 zero size 2024-03-04T13:42:57.788 INFO:tasks.workunit.client.1.smithi169.stdout:6/612: link d0/d32/d3a/d43/f7b d0/d32/d70/fe0 0 2024-03-04T13:42:57.790 INFO:tasks.workunit.client.1.smithi169.stdout:3/637: dread d7/de/d2b/d7d/d85/d13/d1a/d2d/dbe/fcb [0,4194304] 0 2024-03-04T13:42:57.792 INFO:tasks.workunit.client.1.smithi169.stdout:2/659: creat d1/d5/d19/d68/fe8 x:0 0 0 2024-03-04T13:42:57.793 INFO:tasks.workunit.client.0.smithi107.stdout:0/129: dwrite da/f27 [0,4194304] 0 2024-03-04T13:42:57.793 INFO:tasks.workunit.client.0.smithi107.stdout:8/151: dread d6/da/f1b [0,4194304] 0 2024-03-04T13:42:57.794 INFO:tasks.workunit.client.0.smithi107.stdout:6/113: dread f6 [0,4194304] 0 2024-03-04T13:42:57.795 INFO:tasks.workunit.client.1.smithi169.stdout:4/556: creat d4/d11/d2e/d37/d5c/fb0 x:0 0 0 2024-03-04T13:42:57.796 INFO:tasks.workunit.client.0.smithi107.stdout:6/114: chown df/d17/f23 13635 1 2024-03-04T13:42:57.796 INFO:tasks.workunit.client.0.smithi107.stdout:5/55: truncate d5/d8/f11 277931 0 2024-03-04T13:42:57.798 INFO:tasks.workunit.client.1.smithi169.stdout:7/685: write d2/de/d3e/d41/d5b/f9f [294171,46611] 0 2024-03-04T13:42:57.801 INFO:tasks.workunit.client.0.smithi107.stdout:7/95: sync 2024-03-04T13:42:57.801 INFO:tasks.workunit.client.0.smithi107.stdout:2/113: creat da/f26 x:0 0 0 2024-03-04T13:42:57.801 INFO:tasks.workunit.client.0.smithi107.stdout:1/72: sync 2024-03-04T13:42:57.802 INFO:tasks.workunit.client.0.smithi107.stdout:7/96: readlink d6/ld 0 2024-03-04T13:42:57.802 INFO:tasks.workunit.client.1.smithi169.stdout:5/697: sync 2024-03-04T13:42:57.802 INFO:tasks.workunit.client.1.smithi169.stdout:9/640: creat d7/dc/d12/d20/da6/fdf x:0 0 0 2024-03-04T13:42:57.807 INFO:tasks.workunit.client.1.smithi169.stdout:8/627: dread d7/d10/d17/d44/d5d/f9a [0,4194304] 0 2024-03-04T13:42:57.809 INFO:tasks.workunit.client.0.smithi107.stdout:3/78: link d1/f9 d1/de/f1d 0 2024-03-04T13:42:57.812 INFO:tasks.workunit.client.0.smithi107.stdout:0/130: dwrite da/dc/dd/ff [0,4194304] 0 2024-03-04T13:42:57.813 INFO:tasks.workunit.client.1.smithi169.stdout:6/613: mknod d0/d32/d3e/db9/ce1 0 2024-03-04T13:42:57.814 INFO:tasks.workunit.client.0.smithi107.stdout:0/131: write da/dc/dd/ff [484485,103361] 0 2024-03-04T13:42:57.815 INFO:tasks.workunit.client.1.smithi169.stdout:4/557: dwrite d4/d11/d2e/f51 [0,4194304] 0 2024-03-04T13:42:57.816 INFO:tasks.workunit.client.1.smithi169.stdout:3/638: write d7/de/d2b/f6e [4571861,10628] 0 2024-03-04T13:42:57.819 INFO:tasks.workunit.client.0.smithi107.stdout:4/80: truncate d0/d9/d10/f15 2245485 0 2024-03-04T13:42:57.822 INFO:tasks.workunit.client.0.smithi107.stdout:5/56: mkdir d5/d8/d14 0 2024-03-04T13:42:57.824 INFO:tasks.workunit.client.1.smithi169.stdout:4/558: truncate d4/d11/d2e/d37/d5c/fb0 1010810 0 2024-03-04T13:42:57.827 INFO:tasks.workunit.client.0.smithi107.stdout:2/114: mkdir da/df/d27 0 2024-03-04T13:42:57.827 INFO:tasks.workunit.client.1.smithi169.stdout:0/504: rename d5/d9/dc/d12/d15/ca1 to d5/d9/dc/cb5 0 2024-03-04T13:42:57.830 INFO:tasks.workunit.client.1.smithi169.stdout:0/505: readlink d5/d9/dc/d12/d15/d19/l7b 0 2024-03-04T13:42:57.831 INFO:tasks.workunit.client.1.smithi169.stdout:7/686: creat d2/de/d3e/d41/fe9 x:0 0 0 2024-03-04T13:42:57.833 INFO:tasks.workunit.client.0.smithi107.stdout:0/132: dwrite da/dc/dd/f24 [0,4194304] 0 2024-03-04T13:42:57.834 INFO:tasks.workunit.client.1.smithi169.stdout:7/687: chown d2/de/d3e/d41/d69 793268 1 2024-03-04T13:42:57.834 INFO:tasks.workunit.client.0.smithi107.stdout:9/77: mkdir d4/da/d11/d12 0 2024-03-04T13:42:57.836 INFO:tasks.workunit.client.0.smithi107.stdout:0/133: truncate da/de/f1c 631009 0 2024-03-04T13:42:57.836 INFO:tasks.workunit.client.1.smithi169.stdout:8/628: unlink d7/d10/d17/d44/d5d/f9a 0 2024-03-04T13:42:57.837 INFO:tasks.workunit.client.0.smithi107.stdout:3/79: mkdir d1/d1e 0 2024-03-04T13:42:57.839 INFO:tasks.workunit.client.0.smithi107.stdout:8/152: mkdir d6/da/d25/d2a 0 2024-03-04T13:42:57.841 INFO:tasks.workunit.client.1.smithi169.stdout:1/581: dread d4/d5/d2d/d4e/d71/d26/f2e [0,4194304] 0 2024-03-04T13:42:57.841 INFO:tasks.workunit.client.0.smithi107.stdout:8/153: write d6/d16/f24 [3573374,41030] 0 2024-03-04T13:42:57.842 INFO:tasks.workunit.client.0.smithi107.stdout:8/154: dread - d6/f29 zero size 2024-03-04T13:42:57.844 INFO:tasks.workunit.client.1.smithi169.stdout:6/614: symlink d0/d32/d3e/db9/le2 0 2024-03-04T13:42:57.845 INFO:tasks.workunit.client.1.smithi169.stdout:1/582: write d4/d5/d36/d35/d8f/f97 [677387,102563] 0 2024-03-04T13:42:57.857 INFO:tasks.workunit.client.1.smithi169.stdout:3/639: rename d7/de/d2b/d7d/d85/d47/c92 to d7/de/d2b/d7d/d85/d13/d1a/d98/cdc 0 2024-03-04T13:42:57.860 INFO:tasks.workunit.client.1.smithi169.stdout:0/506: mknod d5/d4c/cb6 0 2024-03-04T13:42:57.866 INFO:tasks.workunit.client.1.smithi169.stdout:7/688: write d2/de/d3e/d41/d69/d78/fb4 [1338886,4534] 0 2024-03-04T13:42:57.869 INFO:tasks.workunit.client.1.smithi169.stdout:2/660: truncate d1/d5/d19/d1e/f24 4443097 0 2024-03-04T13:42:57.870 INFO:tasks.workunit.client.1.smithi169.stdout:2/661: readlink d1/d5/d19/d1e/d29/l2f 0 2024-03-04T13:42:57.871 INFO:tasks.workunit.client.1.smithi169.stdout:2/662: stat d1/f50 0 2024-03-04T13:42:57.874 INFO:tasks.workunit.client.1.smithi169.stdout:2/663: fdatasync d1/d5/fe 0 2024-03-04T13:42:57.874 INFO:tasks.workunit.client.1.smithi169.stdout:6/615: chown d0/c83 4608957 1 2024-03-04T13:42:57.876 INFO:tasks.workunit.client.1.smithi169.stdout:6/616: fsync d0/d32/d3a/f42 0 2024-03-04T13:42:57.879 INFO:tasks.workunit.client.1.smithi169.stdout:1/583: rename d4/d5/d36/d35/d5b/l7a to d4/d5/d36/d75/dba/lbf 0 2024-03-04T13:42:57.882 INFO:tasks.workunit.client.1.smithi169.stdout:3/640: dread - d7/de/d2b/d7d/d85/d18/d21/fca zero size 2024-03-04T13:42:57.886 INFO:tasks.workunit.client.1.smithi169.stdout:5/698: creat d1/d16/d17/fe9 x:0 0 0 2024-03-04T13:42:57.889 INFO:tasks.workunit.client.1.smithi169.stdout:3/641: dread d7/de/d2b/f52 [0,4194304] 0 2024-03-04T13:42:57.890 INFO:tasks.workunit.client.1.smithi169.stdout:9/641: creat d7/d25/d47/fe0 x:0 0 0 2024-03-04T13:42:57.891 INFO:tasks.workunit.client.1.smithi169.stdout:7/689: mkdir d2/d28/db8/dd5/dea 0 2024-03-04T13:42:57.893 INFO:tasks.workunit.client.0.smithi107.stdout:6/115: truncate df/d17/d24/d27/f26 95383 0 2024-03-04T13:42:57.893 INFO:tasks.workunit.client.0.smithi107.stdout:5/57: truncate d5/d8/fb 1285506 0 2024-03-04T13:42:57.893 INFO:tasks.workunit.client.1.smithi169.stdout:1/584: dread d4/d5/d36/d35/d5b/f87 [0,4194304] 0 2024-03-04T13:42:57.894 INFO:tasks.workunit.client.0.smithi107.stdout:9/78: unlink d4/f9 0 2024-03-04T13:42:57.895 INFO:tasks.workunit.client.0.smithi107.stdout:0/134: readlink da/l18 0 2024-03-04T13:42:57.896 INFO:tasks.workunit.client.0.smithi107.stdout:8/155: creat d6/d16/d1f/f2b x:0 0 0 2024-03-04T13:42:57.896 INFO:tasks.workunit.client.1.smithi169.stdout:7/690: fdatasync d2/de/d3e/d41/d5b/f9f 0 2024-03-04T13:42:57.897 INFO:tasks.workunit.client.0.smithi107.stdout:7/97: truncate f4 4461197 0 2024-03-04T13:42:57.897 INFO:tasks.workunit.client.0.smithi107.stdout:9/79: read d4/ff [3676181,113870] 0 2024-03-04T13:42:57.897 INFO:tasks.workunit.client.0.smithi107.stdout:5/58: rename d5/d8/da to d5/d8/da/d15 22 2024-03-04T13:42:57.898 INFO:tasks.workunit.client.0.smithi107.stdout:0/135: write da/f2a [783173,70524] 0 2024-03-04T13:42:57.899 INFO:tasks.workunit.client.1.smithi169.stdout:8/629: dwrite d7/d10/d17/f27 [0,4194304] 0 2024-03-04T13:42:57.905 INFO:tasks.workunit.client.1.smithi169.stdout:8/630: write d7/d10/d17/d44/f4f [3187934,73336] 0 2024-03-04T13:42:57.906 INFO:tasks.workunit.client.0.smithi107.stdout:8/156: dread d6/d16/f1c [0,4194304] 0 2024-03-04T13:42:57.906 INFO:tasks.workunit.client.1.smithi169.stdout:2/664: dread d1/d5/d19/d3c/db8/de1/f4b [0,4194304] 0 2024-03-04T13:42:57.912 INFO:tasks.workunit.client.1.smithi169.stdout:6/617: rmdir d0 39 2024-03-04T13:42:57.914 INFO:tasks.workunit.client.1.smithi169.stdout:5/699: dwrite d1/d60/fae [0,4194304] 0 2024-03-04T13:42:57.922 INFO:tasks.workunit.client.1.smithi169.stdout:3/642: creat d7/de/d2b/d7d/d85/d13/d5a/da1/fdd x:0 0 0 2024-03-04T13:42:57.930 INFO:tasks.workunit.client.1.smithi169.stdout:4/559: getdents d4/d11/d2e/d39/d48 0 2024-03-04T13:42:57.932 INFO:tasks.workunit.client.1.smithi169.stdout:7/691: mkdir d2/de/d3e/d41/d69/deb 0 2024-03-04T13:42:57.948 INFO:tasks.workunit.client.1.smithi169.stdout:5/700: symlink d1/d16/d2f/lea 0 2024-03-04T13:42:57.951 INFO:tasks.workunit.client.1.smithi169.stdout:4/560: dread d4/d11/f79 [0,4194304] 0 2024-03-04T13:42:57.951 INFO:tasks.workunit.client.1.smithi169.stdout:1/585: mknod d4/d5/d36/d35/d16/d81/db4/cc0 0 2024-03-04T13:42:57.954 INFO:tasks.workunit.client.1.smithi169.stdout:3/643: truncate d7/de/d2b/d7d/d85/d13/d5a/da1/fa7 948765 0 2024-03-04T13:42:57.954 INFO:tasks.workunit.client.1.smithi169.stdout:8/631: dread d7/d10/d24/d3e/f68 [0,4194304] 0 2024-03-04T13:42:57.956 INFO:tasks.workunit.client.1.smithi169.stdout:1/586: write d4/d5/d36/d35/d16/d81/f96 [1030384,7856] 0 2024-03-04T13:42:57.957 INFO:tasks.workunit.client.1.smithi169.stdout:9/642: dwrite d7/d25/d3c/f62 [0,4194304] 0 2024-03-04T13:42:57.957 INFO:tasks.workunit.client.1.smithi169.stdout:7/692: dwrite d2/d15/d17/f81 [0,4194304] 0 2024-03-04T13:42:57.959 INFO:tasks.workunit.client.1.smithi169.stdout:1/587: stat d4/d5/d36/d35 0 2024-03-04T13:42:58.002 INFO:tasks.workunit.client.1.smithi169.stdout:5/701: read d1/d16/d17/f18 [2140148,27797] 0 2024-03-04T13:42:58.005 INFO:tasks.workunit.client.1.smithi169.stdout:2/665: dwrite d1/d5/d19/d1e/f43 [0,4194304] 0 2024-03-04T13:42:58.006 INFO:tasks.workunit.client.1.smithi169.stdout:2/666: fsync d1/d5/d22/d45/f90 0 2024-03-04T13:42:58.008 INFO:tasks.workunit.client.1.smithi169.stdout:2/667: chown d1/d5/d19/l23 11300 1 2024-03-04T13:42:58.011 INFO:tasks.workunit.client.1.smithi169.stdout:2/668: dread - d1/d5/d22/d45/d4a/fcd zero size 2024-03-04T13:42:58.012 INFO:tasks.workunit.client.1.smithi169.stdout:8/632: chown d7/d10/d22/d75/d81/da2/d96/cc7 110 1 2024-03-04T13:42:58.016 INFO:tasks.workunit.client.1.smithi169.stdout:5/702: symlink d1/d16/d17/d2d/d51/leb 0 2024-03-04T13:42:58.024 INFO:tasks.workunit.client.1.smithi169.stdout:7/693: truncate d2/d28/d70/fc5 1060354 0 2024-03-04T13:42:58.028 INFO:tasks.workunit.client.1.smithi169.stdout:8/633: mkdir d7/d10/d17/d44/d36/dd3 0 2024-03-04T13:42:58.029 INFO:tasks.workunit.client.1.smithi169.stdout:8/634: chown d7 3148890 1 2024-03-04T13:42:58.034 INFO:tasks.workunit.client.1.smithi169.stdout:5/703: mknod d1/d60/cec 0 2024-03-04T13:42:58.035 INFO:tasks.workunit.client.1.smithi169.stdout:5/704: chown d1/d16/d17/fe9 10866145 1 2024-03-04T13:42:58.037 INFO:tasks.workunit.client.0.smithi107.stdout:8/157: sync 2024-03-04T13:42:58.037 INFO:tasks.workunit.client.0.smithi107.stdout:3/80: sync 2024-03-04T13:42:58.039 INFO:tasks.workunit.client.0.smithi107.stdout:8/158: dread - d6/d16/d1f/f23 zero size 2024-03-04T13:42:58.039 INFO:tasks.workunit.client.1.smithi169.stdout:7/694: unlink d2/fc9 0 2024-03-04T13:42:58.041 INFO:tasks.workunit.client.0.smithi107.stdout:8/159: read d6/da/f1b [430853,81864] 0 2024-03-04T13:42:58.041 INFO:tasks.workunit.client.1.smithi169.stdout:2/669: mkdir d1/d5/d19/de9 0 2024-03-04T13:42:58.046 INFO:tasks.workunit.client.1.smithi169.stdout:6/618: link d0/d32/d3a/d43/la2 d0/d3c/d95/le3 0 2024-03-04T13:42:58.047 INFO:tasks.workunit.client.1.smithi169.stdout:6/619: chown d0/d3c/d40/f5c 97584677 1 2024-03-04T13:42:58.050 INFO:tasks.workunit.client.1.smithi169.stdout:7/695: mkdir d2/de/d3e/d41/dec 0 2024-03-04T13:42:58.052 INFO:tasks.workunit.client.1.smithi169.stdout:7/696: dread - d2/de/d3e/d41/d69/d78/fda zero size 2024-03-04T13:42:58.058 INFO:tasks.workunit.client.1.smithi169.stdout:2/670: mknod d1/d5/d22/cea 0 2024-03-04T13:42:58.059 INFO:tasks.workunit.client.0.smithi107.stdout:2/115: getdents da/df/d27 0 2024-03-04T13:42:58.059 INFO:tasks.workunit.client.1.smithi169.stdout:8/635: dwrite d7/d10/d24/d3e/f95 [0,4194304] 0 2024-03-04T13:42:58.060 INFO:tasks.workunit.client.1.smithi169.stdout:7/697: mkdir d2/d28/d70/ded 0 2024-03-04T13:42:58.062 INFO:tasks.workunit.client.0.smithi107.stdout:2/116: dread - da/f20 zero size 2024-03-04T13:42:58.064 INFO:tasks.workunit.client.1.smithi169.stdout:6/620: mknod d0/d32/d3a/d43/d74/ce4 0 2024-03-04T13:42:58.064 INFO:tasks.workunit.client.1.smithi169.stdout:8/636: truncate d7/d10/d24/f7a 4776477 0 2024-03-04T13:42:58.066 INFO:tasks.workunit.client.1.smithi169.stdout:5/705: dwrite d1/d16/dbd/f4f [0,4194304] 0 2024-03-04T13:42:58.068 INFO:tasks.workunit.client.0.smithi107.stdout:2/117: readlink da/df/l12 0 2024-03-04T13:42:58.069 INFO:tasks.workunit.client.1.smithi169.stdout:2/671: symlink d1/d5/d19/d69/leb 0 2024-03-04T13:42:58.073 INFO:tasks.workunit.client.1.smithi169.stdout:7/698: rename d2/de/d37/lca to d2/d15/d54/lee 0 2024-03-04T13:42:58.076 INFO:tasks.workunit.client.0.smithi107.stdout:6/116: symlink df/d17/d24/l2a 0 2024-03-04T13:42:58.078 INFO:tasks.workunit.client.1.smithi169.stdout:2/672: fdatasync d1/d5/d22/d45/d4a/fe3 0 2024-03-04T13:42:58.080 INFO:tasks.workunit.client.1.smithi169.stdout:5/706: dread d1/d60/fd7 [0,4194304] 0 2024-03-04T13:42:58.081 INFO:tasks.workunit.client.1.smithi169.stdout:2/673: write d1/d6d/dbc/fbe [265935,125770] 0 2024-03-04T13:42:58.085 INFO:tasks.workunit.client.1.smithi169.stdout:8/637: dread d7/d10/d17/f27 [0,4194304] 0 2024-03-04T13:42:58.086 INFO:tasks.workunit.client.0.smithi107.stdout:5/59: chown d5/d8/fb 2 1 2024-03-04T13:42:58.087 INFO:tasks.workunit.client.1.smithi169.stdout:2/674: truncate d1/d5/fad 668309 0 2024-03-04T13:42:58.087 INFO:tasks.workunit.client.0.smithi107.stdout:0/136: rmdir da/de/d26 39 2024-03-04T13:42:58.089 INFO:tasks.workunit.client.1.smithi169.stdout:8/638: chown d7/d10/d22/d75/d81/da2/dac 7308 1 2024-03-04T13:42:58.089 INFO:tasks.workunit.client.0.smithi107.stdout:0/137: chown da/dc/f28 3940 1 2024-03-04T13:42:58.091 INFO:tasks.workunit.client.0.smithi107.stdout:0/138: read - da/de/f25 zero size 2024-03-04T13:42:58.094 INFO:tasks.workunit.client.1.smithi169.stdout:7/699: dwrite d2/d15/d54/fac [0,4194304] 0 2024-03-04T13:42:58.095 INFO:tasks.workunit.client.0.smithi107.stdout:8/160: creat d6/da/f2c x:0 0 0 2024-03-04T13:42:58.095 INFO:tasks.workunit.client.1.smithi169.stdout:5/707: link d1/c27 d1/d81/ced 0 2024-03-04T13:42:58.096 INFO:tasks.workunit.client.1.smithi169.stdout:7/700: write d2/d15/fd8 [895551,76817] 0 2024-03-04T13:42:58.096 INFO:tasks.workunit.client.1.smithi169.stdout:2/675: mknod d1/d5/d19/de9/cec 0 2024-03-04T13:42:58.100 INFO:tasks.workunit.client.0.smithi107.stdout:6/117: creat df/d17/d24/f2b x:0 0 0 2024-03-04T13:42:58.102 INFO:tasks.workunit.client.1.smithi169.stdout:7/701: creat d2/de/d3e/d7e/fef x:0 0 0 2024-03-04T13:42:58.103 INFO:tasks.workunit.client.1.smithi169.stdout:5/708: symlink d1/d16/dbd/dbb/lee 0 2024-03-04T13:42:58.104 INFO:tasks.workunit.client.1.smithi169.stdout:5/709: read - d1/d81/fdb zero size 2024-03-04T13:42:58.107 INFO:tasks.workunit.client.1.smithi169.stdout:9/643: write d7/d2d/f40 [9088249,8737] 0 2024-03-04T13:42:58.110 INFO:tasks.workunit.client.0.smithi107.stdout:8/161: mkdir d6/da/d25/d2d 0 2024-03-04T13:42:58.113 INFO:tasks.workunit.client.1.smithi169.stdout:5/710: creat d1/d16/d2f/fef x:0 0 0 2024-03-04T13:42:58.114 INFO:tasks.workunit.client.0.smithi107.stdout:6/118: mkdir df/d1b/d2c 0 2024-03-04T13:42:58.114 INFO:tasks.workunit.client.1.smithi169.stdout:5/711: stat d1/d16/d17/d2d/d51/c95 0 2024-03-04T13:42:58.115 INFO:tasks.workunit.client.1.smithi169.stdout:9/644: rmdir d7/d2d/d48 39 2024-03-04T13:42:58.116 INFO:tasks.workunit.client.0.smithi107.stdout:3/81: dwrite d1/de/d11/d12/f15 [4194304,4194304] 0 2024-03-04T13:42:58.117 INFO:tasks.workunit.client.1.smithi169.stdout:9/645: fdatasync d7/dc/d12/d20/da6/fb4 0 2024-03-04T13:42:58.118 INFO:tasks.workunit.client.0.smithi107.stdout:9/80: rmdir d4/da/d11/d12 0 2024-03-04T13:42:58.119 INFO:tasks.workunit.client.0.smithi107.stdout:0/139: unlink l6 0 2024-03-04T13:42:58.120 INFO:tasks.workunit.client.1.smithi169.stdout:5/712: creat d1/d81/db1/ff0 x:0 0 0 2024-03-04T13:42:58.126 INFO:tasks.workunit.client.1.smithi169.stdout:2/676: dread d1/d5/d19/d3c/db8/de1/fa3 [0,4194304] 0 2024-03-04T13:42:58.130 INFO:tasks.workunit.client.1.smithi169.stdout:1/588: dwrite d4/d5/d36/d35/d70/fab [0,4194304] 0 2024-03-04T13:42:58.133 INFO:tasks.workunit.client.0.smithi107.stdout:6/119: creat df/d17/d24/f2d x:0 0 0 2024-03-04T13:42:58.134 INFO:tasks.workunit.client.1.smithi169.stdout:9/646: unlink d7/d49/l9f 0 2024-03-04T13:42:58.134 INFO:tasks.workunit.client.0.smithi107.stdout:6/120: read df/d1b/f1d [446832,81228] 0 2024-03-04T13:42:58.137 INFO:tasks.workunit.client.0.smithi107.stdout:6/121: write df/d17/d24/f2d [279211,113069] 0 2024-03-04T13:42:58.141 INFO:tasks.workunit.client.1.smithi169.stdout:1/589: read d4/d5/d36/d35/df/f46 [3375244,86715] 0 2024-03-04T13:42:58.141 INFO:tasks.workunit.client.1.smithi169.stdout:9/647: dread - d7/f6b zero size 2024-03-04T13:42:58.145 INFO:tasks.workunit.client.0.smithi107.stdout:2/118: getdents da/df 0 2024-03-04T13:42:58.146 INFO:tasks.workunit.client.0.smithi107.stdout:8/162: mknod d6/c2e 0 2024-03-04T13:42:58.147 INFO:tasks.workunit.client.0.smithi107.stdout:2/119: write da/df/f15 [2825633,34713] 0 2024-03-04T13:42:58.148 INFO:tasks.workunit.client.0.smithi107.stdout:2/120: stat f7 0 2024-03-04T13:42:58.149 INFO:tasks.workunit.client.0.smithi107.stdout:2/121: readlink da/df/l12 0 2024-03-04T13:42:58.149 INFO:tasks.workunit.client.1.smithi169.stdout:3/644: dwrite d7/de/d2b/d7d/d85/d13/d5a/da1/fa7 [0,4194304] 0 2024-03-04T13:42:58.149 INFO:tasks.workunit.client.1.smithi169.stdout:5/713: dwrite d1/d16/d17/d2d/d5a/fdd [0,4194304] 0 2024-03-04T13:42:58.149 INFO:tasks.workunit.client.0.smithi107.stdout:6/122: mkdir df/d2e 0 2024-03-04T13:42:58.154 INFO:tasks.workunit.client.0.smithi107.stdout:9/81: truncate d4/ff 3153347 0 2024-03-04T13:42:58.154 INFO:tasks.workunit.client.0.smithi107.stdout:9/82: readlink l1 0 2024-03-04T13:42:58.156 INFO:tasks.workunit.client.1.smithi169.stdout:9/648: unlink d7/d49/la0 0 2024-03-04T13:42:58.160 INFO:tasks.workunit.client.0.smithi107.stdout:5/60: read d5/d8/f11 [260748,43325] 0 2024-03-04T13:42:58.163 INFO:tasks.workunit.client.0.smithi107.stdout:1/73: truncate d1/d3/f11 3648499 0 2024-03-04T13:42:58.165 INFO:tasks.workunit.client.1.smithi169.stdout:3/645: dread d7/de/d2b/d7d/d85/d47/d63/d6c/f88 [0,4194304] 0 2024-03-04T13:42:58.167 INFO:tasks.workunit.client.0.smithi107.stdout:8/163: fsync d6/d16/f27 0 2024-03-04T13:42:58.168 INFO:tasks.workunit.client.1.smithi169.stdout:5/714: symlink d1/d16/d17/d1b/lf1 0 2024-03-04T13:42:58.172 INFO:tasks.workunit.client.0.smithi107.stdout:8/164: write d6/fe [1948991,73076] 0 2024-03-04T13:42:58.175 INFO:tasks.workunit.client.0.smithi107.stdout:9/83: dwrite f0 [0,4194304] 0 2024-03-04T13:42:58.177 INFO:tasks.workunit.client.0.smithi107.stdout:8/165: chown d6/da/d25/d2d 2097271 1 2024-03-04T13:42:58.179 INFO:tasks.workunit.client.1.smithi169.stdout:5/715: write d1/d16/d2f/fde [943862,119386] 0 2024-03-04T13:42:58.180 INFO:tasks.workunit.client.1.smithi169.stdout:3/646: dread d7/de/d2b/d7d/d85/d13/f15 [0,4194304] 0 2024-03-04T13:42:58.180 INFO:tasks.workunit.client.0.smithi107.stdout:7/98: read f4 [2465860,16883] 0 2024-03-04T13:42:58.185 INFO:tasks.workunit.client.0.smithi107.stdout:4/81: dwrite d0/d9/de/d14/fa [0,4194304] 0 2024-03-04T13:42:58.188 INFO:tasks.workunit.client.1.smithi169.stdout:7/702: write d2/d15/d17/f1d [1281786,29531] 0 2024-03-04T13:42:58.190 INFO:tasks.workunit.client.1.smithi169.stdout:5/716: creat d1/d16/d17/d2d/ff2 x:0 0 0 2024-03-04T13:42:58.196 INFO:tasks.workunit.client.1.smithi169.stdout:1/590: dread d4/d5/d2d/d4e/d71/d26/f7b [0,4194304] 0 2024-03-04T13:42:58.197 INFO:tasks.workunit.client.1.smithi169.stdout:7/703: creat d2/de/d3e/d41/d5b/ff0 x:0 0 0 2024-03-04T13:42:58.199 INFO:tasks.workunit.client.0.smithi107.stdout:0/140: getdents da/dc/dd 0 2024-03-04T13:42:58.201 INFO:tasks.workunit.client.1.smithi169.stdout:3/647: mkdir d7/de/d2b/d7d/d85/d13/d1a/d2d/d56/d5e/d67/dd7/dde 0 2024-03-04T13:42:58.202 INFO:tasks.workunit.client.1.smithi169.stdout:3/648: dread - d7/d53/fb3 zero size 2024-03-04T13:42:58.202 INFO:tasks.workunit.client.0.smithi107.stdout:4/82: dwrite d0/d9/f11 [0,4194304] 0 2024-03-04T13:42:58.203 INFO:tasks.workunit.client.1.smithi169.stdout:5/717: symlink d1/d16/d2a/d46/d9e/lf3 0 2024-03-04T13:42:58.205 INFO:tasks.workunit.client.0.smithi107.stdout:8/166: mknod d6/d16/d1f/c2f 0 2024-03-04T13:42:58.206 INFO:tasks.workunit.client.1.smithi169.stdout:1/591: rmdir d4/d5/d36/d35/d70 39 2024-03-04T13:42:58.208 INFO:tasks.workunit.client.1.smithi169.stdout:3/649: dread d7/fb [4194304,4194304] 0 2024-03-04T13:42:58.208 INFO:tasks.workunit.client.0.smithi107.stdout:9/84: creat d4/da/d11/f13 x:0 0 0 2024-03-04T13:42:58.208 INFO:tasks.workunit.client.1.smithi169.stdout:1/592: write d4/f1b [3223392,48903] 0 2024-03-04T13:42:58.212 INFO:tasks.workunit.client.1.smithi169.stdout:7/704: rename d2/de/d3e/ce0 to d2/de/d3e/d41/d69/cf1 0 2024-03-04T13:42:58.212 INFO:tasks.workunit.client.1.smithi169.stdout:0/507: sync 2024-03-04T13:42:58.212 INFO:tasks.workunit.client.1.smithi169.stdout:4/561: sync 2024-03-04T13:42:58.215 INFO:tasks.workunit.client.1.smithi169.stdout:7/705: dread - d2/de/d37/fd0 zero size 2024-03-04T13:42:58.216 INFO:tasks.workunit.client.1.smithi169.stdout:5/718: mknod d1/d16/dbd/dbb/cf4 0 2024-03-04T13:42:58.219 INFO:tasks.workunit.client.0.smithi107.stdout:3/82: dwrite d1/f9 [0,4194304] 0 2024-03-04T13:42:58.221 INFO:tasks.workunit.client.0.smithi107.stdout:2/122: link da/df/l22 da/l28 0 2024-03-04T13:42:58.225 INFO:tasks.workunit.client.0.smithi107.stdout:2/123: write da/f21 [706594,16786] 0 2024-03-04T13:42:58.227 INFO:tasks.workunit.client.0.smithi107.stdout:0/141: rmdir da/dc 39 2024-03-04T13:42:58.227 INFO:tasks.workunit.client.0.smithi107.stdout:3/83: truncate d1/fa 1137555 0 2024-03-04T13:42:58.229 INFO:tasks.workunit.client.0.smithi107.stdout:8/167: dwrite d6/da/f19 [4194304,4194304] 0 2024-03-04T13:42:58.229 INFO:tasks.workunit.client.0.smithi107.stdout:4/83: dwrite d0/d9/de/d14/fa [0,4194304] 0 2024-03-04T13:42:58.231 INFO:tasks.workunit.client.0.smithi107.stdout:5/61: write d5/d8/f11 [883066,9561] 0 2024-03-04T13:42:58.236 INFO:tasks.workunit.client.1.smithi169.stdout:0/508: dwrite d5/d9/dc/d12/d15/d19/d61/f21 [0,4194304] 0 2024-03-04T13:42:58.237 INFO:tasks.workunit.client.1.smithi169.stdout:1/593: rename d4/d5/d36/d35/d8f/f9a to d4/d5/d2d/d4e/fc1 0 2024-03-04T13:42:58.243 INFO:tasks.workunit.client.1.smithi169.stdout:9/649: dwrite d7/dc/d12/d8a/f93 [0,4194304] 0 2024-03-04T13:42:58.249 INFO:tasks.workunit.client.0.smithi107.stdout:6/123: dwrite fd [0,4194304] 0 2024-03-04T13:42:58.254 INFO:tasks.workunit.client.0.smithi107.stdout:8/168: dread d6/da/f11 [0,4194304] 0 2024-03-04T13:42:58.255 INFO:tasks.workunit.client.1.smithi169.stdout:4/562: mknod d4/d11/d3b/cb1 0 2024-03-04T13:42:58.257 INFO:tasks.workunit.client.1.smithi169.stdout:5/719: symlink d1/lf5 0 2024-03-04T13:42:58.258 INFO:tasks.workunit.client.0.smithi107.stdout:2/124: mkdir da/d29 0 2024-03-04T13:42:58.258 INFO:tasks.workunit.client.0.smithi107.stdout:8/169: chown d6/d16/d1f/l28 2794 1 2024-03-04T13:42:58.259 INFO:tasks.workunit.client.0.smithi107.stdout:3/84: stat d1/l10 0 2024-03-04T13:42:58.260 INFO:tasks.workunit.client.1.smithi169.stdout:4/563: write d4/d11/d3b/d67/f84 [795389,91570] 0 2024-03-04T13:42:58.260 INFO:tasks.workunit.client.0.smithi107.stdout:2/125: fsync da/df/f1e 0 2024-03-04T13:42:58.263 INFO:tasks.workunit.client.1.smithi169.stdout:0/509: read - d5/d9/dc/d12/d66/f78 zero size 2024-03-04T13:42:58.266 INFO:tasks.workunit.client.0.smithi107.stdout:4/84: readlink d0/l8 0 2024-03-04T13:42:58.267 INFO:tasks.workunit.client.0.smithi107.stdout:5/62: creat d5/d8/da/f16 x:0 0 0 2024-03-04T13:42:58.267 INFO:tasks.workunit.client.1.smithi169.stdout:1/594: rename d4/d5/d36/d35/d16/d3a/f3f to d4/d5/d36/d35/df/fc2 0 2024-03-04T13:42:58.268 INFO:tasks.workunit.client.1.smithi169.stdout:3/650: write d7/de/d2b/d7d/d85/d18/d21/d3f/f69 [1151925,72776] 0 2024-03-04T13:42:58.269 INFO:tasks.workunit.client.0.smithi107.stdout:9/85: mknod d4/c14 0 2024-03-04T13:42:58.272 INFO:tasks.workunit.client.1.smithi169.stdout:1/595: chown d4/d5/d36/d35/df/c80 325563 1 2024-03-04T13:42:58.272 INFO:tasks.workunit.client.1.smithi169.stdout:7/706: dwrite d2/d15/d17/f42 [0,4194304] 0 2024-03-04T13:42:58.274 INFO:tasks.workunit.client.1.smithi169.stdout:3/651: dread - d7/de/d2b/d7d/d85/d13/d5a/da1/fdd zero size 2024-03-04T13:42:58.276 INFO:tasks.workunit.client.0.smithi107.stdout:7/99: link d6/ld d6/d13/d14/l23 0 2024-03-04T13:42:58.276 INFO:tasks.workunit.client.1.smithi169.stdout:1/596: read d4/d5/d2d/d4e/d71/d26/f60 [591543,3578] 0 2024-03-04T13:42:58.277 INFO:tasks.workunit.client.1.smithi169.stdout:0/510: dread - d5/d9/d3d/d59/d64/d7f/f9f zero size 2024-03-04T13:42:58.279 INFO:tasks.workunit.client.1.smithi169.stdout:7/707: truncate d2/de/d3e/d7e/fef 623378 0 2024-03-04T13:42:58.279 INFO:tasks.workunit.client.1.smithi169.stdout:4/564: rename d4/d11/f30 to d4/d1c/fb2 0 2024-03-04T13:42:58.281 INFO:tasks.workunit.client.0.smithi107.stdout:6/124: fdatasync f3 0 2024-03-04T13:42:58.282 INFO:tasks.workunit.client.0.smithi107.stdout:1/74: dwrite d1/d3/f11 [0,4194304] 0 2024-03-04T13:42:58.285 INFO:tasks.workunit.client.0.smithi107.stdout:8/170: rename d6/d16/d1f/l28 to d6/da/l30 0 2024-03-04T13:42:58.287 INFO:tasks.workunit.client.1.smithi169.stdout:1/597: mknod d4/d5/d2d/d4e/d71/d26/d4f/cc3 0 2024-03-04T13:42:58.288 INFO:tasks.workunit.client.1.smithi169.stdout:5/720: dwrite d1/d23/f85 [0,4194304] 0 2024-03-04T13:42:58.290 INFO:tasks.workunit.client.1.smithi169.stdout:7/708: unlink d2/d15/fd8 0 2024-03-04T13:42:58.292 INFO:tasks.workunit.client.1.smithi169.stdout:4/565: creat d4/d1c/d75/fb3 x:0 0 0 2024-03-04T13:42:58.299 INFO:tasks.workunit.client.1.smithi169.stdout:5/721: creat d1/d16/d17/d28/ff6 x:0 0 0 2024-03-04T13:42:58.301 INFO:tasks.workunit.client.1.smithi169.stdout:4/566: symlink d4/d1c/lb4 0 2024-03-04T13:42:58.301 INFO:tasks.workunit.client.1.smithi169.stdout:0/511: mknod d5/d9/dc/d12/d15/d19/d77/db2/cb7 0 2024-03-04T13:42:58.302 INFO:tasks.workunit.client.0.smithi107.stdout:1/75: write d1/d3/f11 [314517,32752] 0 2024-03-04T13:42:58.302 INFO:tasks.workunit.client.0.smithi107.stdout:9/86: dread d4/f8 [4194304,4194304] 0 2024-03-04T13:42:58.305 INFO:tasks.workunit.client.0.smithi107.stdout:3/85: dwrite d1/de/f19 [0,4194304] 0 2024-03-04T13:42:58.306 INFO:tasks.workunit.client.1.smithi169.stdout:0/512: creat d5/d9/fb8 x:0 0 0 2024-03-04T13:42:58.306 INFO:tasks.workunit.client.1.smithi169.stdout:3/652: link d7/de/d2b/d7d/d85/d18/d21/d3f/d68/c43 d7/de/d2b/d7d/d85/d13/d5a/d65/da4/cdf 0 2024-03-04T13:42:58.307 INFO:tasks.workunit.client.1.smithi169.stdout:5/722: write d1/d81/f98 [2035078,111193] 0 2024-03-04T13:42:58.308 INFO:tasks.workunit.client.1.smithi169.stdout:1/598: link d4/f1b d4/d5/fc4 0 2024-03-04T13:42:58.309 INFO:tasks.workunit.client.0.smithi107.stdout:2/126: dwrite da/f10 [0,4194304] 0 2024-03-04T13:42:58.309 INFO:tasks.workunit.client.0.smithi107.stdout:4/85: creat d0/d9/f1a x:0 0 0 2024-03-04T13:42:58.309 INFO:tasks.workunit.client.1.smithi169.stdout:4/567: truncate d4/d11/d3b/f83 1259139 0 2024-03-04T13:42:58.310 INFO:tasks.workunit.client.0.smithi107.stdout:0/142: getdents da 0 2024-03-04T13:42:58.310 INFO:tasks.workunit.client.1.smithi169.stdout:5/723: readlink d1/d16/dbd/dbb/dcc/le2 0 2024-03-04T13:42:58.315 INFO:tasks.workunit.client.1.smithi169.stdout:4/568: fdatasync d4/d11/f1b 0 2024-03-04T13:42:58.315 INFO:tasks.workunit.client.0.smithi107.stdout:1/76: rename d1 to d1/d12 22 2024-03-04T13:42:58.317 INFO:tasks.workunit.client.0.smithi107.stdout:2/127: readlink da/l28 0 2024-03-04T13:42:58.318 INFO:tasks.workunit.client.1.smithi169.stdout:3/653: dread d7/de/d2b/d7d/d85/d47/d63/d6c/f88 [0,4194304] 0 2024-03-04T13:42:58.318 INFO:tasks.workunit.client.0.smithi107.stdout:8/171: link d6/l7 d6/da/d25/l31 0 2024-03-04T13:42:58.322 INFO:tasks.workunit.client.1.smithi169.stdout:0/513: read d5/d4c/d65/f92 [5125617,65549] 0 2024-03-04T13:42:58.325 INFO:tasks.workunit.client.0.smithi107.stdout:0/143: creat da/f2f x:0 0 0 2024-03-04T13:42:58.328 INFO:tasks.workunit.client.1.smithi169.stdout:4/569: rename d4/d11/d2e/d37/c9f to d4/d11/d2e/d37/d73/cb5 0 2024-03-04T13:42:58.329 INFO:tasks.workunit.client.1.smithi169.stdout:3/654: truncate d7/de/d24/fa6 385952 0 2024-03-04T13:42:58.329 INFO:tasks.workunit.client.0.smithi107.stdout:6/125: dwrite df/d17/f28 [0,4194304] 0 2024-03-04T13:42:58.331 INFO:tasks.workunit.client.1.smithi169.stdout:4/570: write d4/f41 [6486495,53971] 0 2024-03-04T13:42:58.333 INFO:tasks.workunit.client.0.smithi107.stdout:4/86: mknod d0/d9/c1b 0 2024-03-04T13:42:58.336 INFO:tasks.workunit.client.0.smithi107.stdout:1/77: mknod d1/d3/c13 0 2024-03-04T13:42:58.338 INFO:tasks.workunit.client.0.smithi107.stdout:4/87: read d0/d9/de/d14/fa [2920231,87665] 0 2024-03-04T13:42:58.340 INFO:tasks.workunit.client.1.smithi169.stdout:5/724: dread d1/d16/d17/d1b/f1e [0,4194304] 0 2024-03-04T13:42:58.342 INFO:tasks.workunit.client.1.smithi169.stdout:5/725: dread - d1/f75 zero size 2024-03-04T13:42:58.342 INFO:tasks.workunit.client.0.smithi107.stdout:9/87: dread f0 [0,4194304] 0 2024-03-04T13:42:58.344 INFO:tasks.workunit.client.0.smithi107.stdout:3/86: dwrite d1/de/d11/d12/f15 [0,4194304] 0 2024-03-04T13:42:58.345 INFO:tasks.workunit.client.0.smithi107.stdout:9/88: write d4/da/d11/f13 [371560,82122] 0 2024-03-04T13:42:58.348 INFO:tasks.workunit.client.0.smithi107.stdout:2/128: dread f2 [0,4194304] 0 2024-03-04T13:42:58.349 INFO:tasks.workunit.client.1.smithi169.stdout:4/571: dread d4/f15 [0,4194304] 0 2024-03-04T13:42:58.351 INFO:tasks.workunit.client.1.smithi169.stdout:9/650: write d7/dc/d12/d32/d42/f7e [432327,121262] 0 2024-03-04T13:42:58.356 INFO:tasks.workunit.client.1.smithi169.stdout:5/726: creat d1/d16/dbd/dbb/ff7 x:0 0 0 2024-03-04T13:42:58.358 INFO:tasks.workunit.client.0.smithi107.stdout:4/88: creat d0/d9/d10/f1c x:0 0 0 2024-03-04T13:42:58.363 INFO:tasks.workunit.client.0.smithi107.stdout:2/129: rename da/c25 to da/c2a 0 2024-03-04T13:42:58.364 INFO:tasks.workunit.client.1.smithi169.stdout:9/651: readlink d7/l13 0 2024-03-04T13:42:58.366 INFO:tasks.workunit.client.0.smithi107.stdout:6/126: truncate df/d17/f1e 1551854 0 2024-03-04T13:42:58.370 INFO:tasks.workunit.client.0.smithi107.stdout:5/63: truncate d5/f9 4756929 0 2024-03-04T13:42:58.370 INFO:tasks.workunit.client.0.smithi107.stdout:9/89: dwrite d4/da/d11/f13 [0,4194304] 0 2024-03-04T13:42:58.374 INFO:tasks.workunit.client.0.smithi107.stdout:5/64: dread - d5/d8/da/f16 zero size 2024-03-04T13:42:58.376 INFO:tasks.workunit.client.1.smithi169.stdout:3/655: dread f6 [4194304,4194304] 0 2024-03-04T13:42:58.377 INFO:tasks.workunit.client.0.smithi107.stdout:7/100: truncate f5 2927062 0 2024-03-04T13:42:58.378 INFO:tasks.workunit.client.0.smithi107.stdout:4/89: creat d0/d9/de/d14/f1d x:0 0 0 2024-03-04T13:42:58.378 INFO:tasks.workunit.client.0.smithi107.stdout:5/65: write d5/f6 [1316898,90805] 0 2024-03-04T13:42:58.379 INFO:tasks.workunit.client.0.smithi107.stdout:2/130: creat da/f2b x:0 0 0 2024-03-04T13:42:58.382 INFO:tasks.workunit.client.0.smithi107.stdout:8/172: getdents d6/da 0 2024-03-04T13:42:58.383 INFO:tasks.workunit.client.0.smithi107.stdout:5/66: read d5/d8/f11 [874961,117976] 0 2024-03-04T13:42:58.383 INFO:tasks.workunit.client.0.smithi107.stdout:3/87: dwrite d1/de/f19 [4194304,4194304] 0 2024-03-04T13:42:58.384 INFO:tasks.workunit.client.0.smithi107.stdout:3/88: read d1/f9 [2330788,105318] 0 2024-03-04T13:42:58.385 INFO:tasks.workunit.client.0.smithi107.stdout:6/127: rename df/d17/l1f to df/d1b/l2f 0 2024-03-04T13:42:58.388 INFO:tasks.workunit.client.1.smithi169.stdout:3/656: creat d7/de/d2b/d7d/d85/d13/d5a/fe0 x:0 0 0 2024-03-04T13:42:58.388 INFO:tasks.workunit.client.0.smithi107.stdout:6/128: dread df/f20 [0,4194304] 0 2024-03-04T13:42:58.389 INFO:tasks.workunit.client.1.smithi169.stdout:4/572: link d4/d11/d3b/la1 d4/d11/d2e/d6c/d78/da5/d77/lb6 0 2024-03-04T13:42:58.392 INFO:tasks.workunit.client.1.smithi169.stdout:9/652: link d7/d27/c73 d7/d49/ce1 0 2024-03-04T13:42:58.393 INFO:tasks.workunit.client.0.smithi107.stdout:2/131: symlink da/df/l2c 0 2024-03-04T13:42:58.393 INFO:tasks.workunit.client.0.smithi107.stdout:4/90: write d0/d9/de/d14/fa [3393019,130016] 0 2024-03-04T13:42:58.393 INFO:tasks.workunit.client.1.smithi169.stdout:5/727: getdents d1/d16/d17/d1b/d74/ddc 0 2024-03-04T13:42:58.394 INFO:tasks.workunit.client.0.smithi107.stdout:2/132: stat da/l28 0 2024-03-04T13:42:58.394 INFO:tasks.workunit.client.0.smithi107.stdout:8/173: unlink d6/da/l13 0 2024-03-04T13:42:58.397 INFO:tasks.workunit.client.1.smithi169.stdout:3/657: rmdir d7/de/d2b/d7d/d85/d13/d1a/d2d 39 2024-03-04T13:42:58.399 INFO:tasks.workunit.client.0.smithi107.stdout:5/67: creat d5/d8/f17 x:0 0 0 2024-03-04T13:42:58.402 INFO:tasks.workunit.client.1.smithi169.stdout:4/573: link d4/d1c/d58/faf d4/d11/d2e/d21/fb7 0 2024-03-04T13:42:58.402 INFO:tasks.workunit.client.0.smithi107.stdout:9/90: dwrite f0 [4194304,4194304] 0 2024-03-04T13:42:58.403 INFO:tasks.workunit.client.1.smithi169.stdout:4/574: chown d4/d1c/f50 315962 1 2024-03-04T13:42:58.404 INFO:tasks.workunit.client.1.smithi169.stdout:9/653: symlink d7/dc/d12/d20/da6/dd1/le2 0 2024-03-04T13:42:58.405 INFO:tasks.workunit.client.1.smithi169.stdout:4/575: dread - d4/d11/d3b/d67/d7e/f82 zero size 2024-03-04T13:42:58.407 INFO:tasks.workunit.client.0.smithi107.stdout:5/68: symlink d5/dc/l18 0 2024-03-04T13:42:58.409 INFO:tasks.workunit.client.1.smithi169.stdout:3/658: creat d7/de/d2b/d7d/d85/d13/d1a/d2d/d56/d5e/d67/d5c/d84/fe1 x:0 0 0 2024-03-04T13:42:58.410 INFO:tasks.workunit.client.1.smithi169.stdout:9/654: symlink d7/dc/d12/d20/da6/dd1/le3 0 2024-03-04T13:42:58.413 INFO:tasks.workunit.client.0.smithi107.stdout:4/91: mkdir d0/d9/d1e 0 2024-03-04T13:42:58.415 INFO:tasks.workunit.client.1.smithi169.stdout:5/728: dwrite d1/d16/d2a/d62/fb4 [0,4194304] 0 2024-03-04T13:42:58.416 INFO:tasks.workunit.client.0.smithi107.stdout:9/91: rename f3 to d4/da/f15 0 2024-03-04T13:42:58.418 INFO:tasks.workunit.client.0.smithi107.stdout:2/133: creat da/d29/f2d x:0 0 0 2024-03-04T13:42:58.418 INFO:tasks.workunit.client.1.smithi169.stdout:9/655: creat d7/dc/d12/d32/d42/fe4 x:0 0 0 2024-03-04T13:42:58.419 INFO:tasks.workunit.client.0.smithi107.stdout:8/174: symlink d6/da/d25/d2a/l32 0 2024-03-04T13:42:58.420 INFO:tasks.workunit.client.0.smithi107.stdout:2/134: write da/df/f13 [262921,25868] 0 2024-03-04T13:42:58.420 INFO:tasks.workunit.client.0.smithi107.stdout:8/175: stat d6/d16/d1f/f23 0 2024-03-04T13:42:58.420 INFO:tasks.workunit.client.0.smithi107.stdout:2/135: chown da 3059580 1 2024-03-04T13:42:58.421 INFO:tasks.workunit.client.0.smithi107.stdout:2/136: dread - da/f26 zero size 2024-03-04T13:42:58.422 INFO:tasks.workunit.client.0.smithi107.stdout:5/69: unlink l2 0 2024-03-04T13:42:58.423 INFO:tasks.workunit.client.0.smithi107.stdout:4/92: read d0/d9/d10/f15 [1042359,47700] 0 2024-03-04T13:42:58.424 INFO:tasks.workunit.client.0.smithi107.stdout:5/70: truncate d5/f7 869069 0 2024-03-04T13:42:58.425 INFO:tasks.workunit.client.0.smithi107.stdout:5/71: fdatasync d5/dc/df/f12 0 2024-03-04T13:42:58.426 INFO:tasks.workunit.client.1.smithi169.stdout:9/656: creat d7/dc/d12/d20/d4e/fe5 x:0 0 0 2024-03-04T13:42:58.427 INFO:tasks.workunit.client.0.smithi107.stdout:5/72: chown d5/f6 207534422 1 2024-03-04T13:42:58.428 INFO:tasks.workunit.client.0.smithi107.stdout:5/73: rename d5/d8 to d5/d8/da/d19 22 2024-03-04T13:42:58.429 INFO:tasks.workunit.client.0.smithi107.stdout:8/176: rmdir d6/da/d25 39 2024-03-04T13:42:58.429 INFO:tasks.workunit.client.0.smithi107.stdout:2/137: rename da/f21 to da/f2e 0 2024-03-04T13:42:58.430 INFO:tasks.workunit.client.1.smithi169.stdout:4/576: dwrite d4/fa [0,4194304] 0 2024-03-04T13:42:58.430 INFO:tasks.workunit.client.1.smithi169.stdout:9/657: chown d7/l13 0 1 2024-03-04T13:42:58.430 INFO:tasks.workunit.client.0.smithi107.stdout:5/74: stat c4 0 2024-03-04T13:42:58.432 INFO:tasks.workunit.client.0.smithi107.stdout:2/138: write da/df/f24 [967589,122354] 0 2024-03-04T13:42:58.432 INFO:tasks.workunit.client.1.smithi169.stdout:6/621: sync 2024-03-04T13:42:58.432 INFO:tasks.workunit.client.0.smithi107.stdout:5/75: truncate d5/d8/f17 164245 0 2024-03-04T13:42:58.435 INFO:tasks.workunit.client.1.smithi169.stdout:9/658: write d7/dc/f1e [1417637,35825] 0 2024-03-04T13:42:58.436 INFO:tasks.workunit.client.1.smithi169.stdout:3/659: dwrite d7/de/d24/f8f [0,4194304] 0 2024-03-04T13:42:58.439 INFO:tasks.workunit.client.1.smithi169.stdout:4/577: mkdir d4/d11/d2e/d37/d5c/d86/db8 0 2024-03-04T13:42:58.442 INFO:tasks.workunit.client.1.smithi169.stdout:9/659: mkdir d7/d27/d84/de6 0 2024-03-04T13:42:58.444 INFO:tasks.workunit.client.1.smithi169.stdout:5/729: dwrite d1/d76/fbe [0,4194304] 0 2024-03-04T13:42:58.445 INFO:tasks.workunit.client.1.smithi169.stdout:3/660: fsync d7/de/d2b/d7d/d85/d13/d1a/d2d/d56/d5e/d67/f3a 0 2024-03-04T13:42:58.449 INFO:tasks.workunit.client.0.smithi107.stdout:9/92: dwrite d4/f8 [4194304,4194304] 0 2024-03-04T13:42:58.450 INFO:tasks.workunit.client.0.smithi107.stdout:4/93: dwrite d0/d9/f1a [0,4194304] 0 2024-03-04T13:42:58.451 INFO:tasks.workunit.client.1.smithi169.stdout:9/660: rename d7/dc/l1c to d7/d27/d84/le7 0 2024-03-04T13:42:58.453 INFO:tasks.workunit.client.1.smithi169.stdout:3/661: mkdir d7/de/d2b/d7d/d85/d13/d1a/d2d/dbe/de2 0 2024-03-04T13:42:58.454 INFO:tasks.workunit.client.1.smithi169.stdout:4/578: creat d4/d11/d2e/d39/fb9 x:0 0 0 2024-03-04T13:42:58.457 INFO:tasks.workunit.client.0.smithi107.stdout:8/177: dwrite d6/f29 [0,4194304] 0 2024-03-04T13:42:58.457 INFO:tasks.workunit.client.1.smithi169.stdout:9/661: rename d7/lcd to d7/d49/le8 0 2024-03-04T13:42:58.458 INFO:tasks.workunit.client.1.smithi169.stdout:9/662: stat d7/l13 0 2024-03-04T13:42:58.458 INFO:tasks.workunit.client.0.smithi107.stdout:9/93: chown d4/ff 124696 1 2024-03-04T13:42:58.459 INFO:tasks.workunit.client.1.smithi169.stdout:4/579: write d4/d11/d3b/d67/d7e/f82 [473811,34037] 0 2024-03-04T13:42:58.460 INFO:tasks.workunit.client.1.smithi169.stdout:3/662: mknod d7/de/d2b/d7d/d85/d18/d21/ce3 0 2024-03-04T13:42:58.461 INFO:tasks.workunit.client.0.smithi107.stdout:8/178: symlink d6/d16/l33 0 2024-03-04T13:42:58.462 INFO:tasks.workunit.client.0.smithi107.stdout:4/94: truncate d0/d9/d10/f1c 582328 0 2024-03-04T13:42:58.465 INFO:tasks.workunit.client.0.smithi107.stdout:4/95: write d0/d9/de/d14/fa [1764076,129769] 0 2024-03-04T13:42:58.472 INFO:tasks.workunit.client.1.smithi169.stdout:4/580: fdatasync d4/d11/d2e/d63/f8a 0 2024-03-04T13:42:58.474 INFO:tasks.workunit.client.1.smithi169.stdout:4/581: stat d4/d1c/d58/l7b 0 2024-03-04T13:42:58.498 INFO:tasks.workunit.client.0.smithi107.stdout:4/96: getdents d0/d9/d10 0 2024-03-04T13:42:58.499 INFO:tasks.workunit.client.1.smithi169.stdout:0/514: write d5/d9/dc/d12/d15/d19/f20 [741409,81666] 0 2024-03-04T13:42:58.500 INFO:tasks.workunit.client.0.smithi107.stdout:0/144: rename da/dc/l16 to da/dc/l30 0 2024-03-04T13:42:58.503 INFO:tasks.workunit.client.0.smithi107.stdout:0/145: chown da/dc/dd/f2c 461829 1 2024-03-04T13:42:58.504 INFO:tasks.workunit.client.0.smithi107.stdout:4/97: dread d0/d9/d10/f1c [0,4194304] 0 2024-03-04T13:42:58.505 INFO:tasks.workunit.client.1.smithi169.stdout:4/582: dread d4/d1c/fb2 [0,4194304] 0 2024-03-04T13:42:58.506 INFO:tasks.workunit.client.0.smithi107.stdout:0/146: chown da/dc/dd/f1f 7432512 1 2024-03-04T13:42:58.507 INFO:tasks.workunit.client.1.smithi169.stdout:4/583: fsync d4/d1c/d75/fb3 0 2024-03-04T13:42:58.509 INFO:tasks.workunit.client.1.smithi169.stdout:4/584: fdatasync d4/d1c/f2c 0 2024-03-04T13:42:58.513 INFO:tasks.workunit.client.0.smithi107.stdout:6/129: getdents df/d17 0 2024-03-04T13:42:58.514 INFO:tasks.workunit.client.0.smithi107.stdout:6/130: chown df/d17/d24/l2a 1659240 1 2024-03-04T13:42:58.520 INFO:tasks.workunit.client.0.smithi107.stdout:0/147: chown da/de/d26/l2b 1 1 2024-03-04T13:42:58.521 INFO:tasks.workunit.client.0.smithi107.stdout:6/131: mkdir df/d17/d24/d30 0 2024-03-04T13:42:58.521 INFO:tasks.workunit.client.0.smithi107.stdout:4/98: dread d0/d9/de/d14/fa [0,4194304] 0 2024-03-04T13:42:58.521 INFO:tasks.workunit.client.1.smithi169.stdout:4/585: symlink d4/d11/d2e/d39/d61/lba 0 2024-03-04T13:42:58.521 INFO:tasks.workunit.client.1.smithi169.stdout:4/586: dread - d4/d1c/d58/faf zero size 2024-03-04T13:42:58.532 INFO:tasks.workunit.client.1.smithi169.stdout:4/587: rename d4/d11/d2e/d21/d5e to d4/d11/d3b/d67/dbb 0 2024-03-04T13:42:58.534 INFO:tasks.workunit.client.1.smithi169.stdout:0/515: dread d5/d9/dc/d12/d15/f40 [0,4194304] 0 2024-03-04T13:42:58.536 INFO:tasks.workunit.client.1.smithi169.stdout:4/588: mkdir d4/d1c/d58/d9d/dbc 0 2024-03-04T13:42:58.536 INFO:tasks.workunit.client.0.smithi107.stdout:0/148: dwrite da/dc/f28 [0,4194304] 0 2024-03-04T13:42:58.538 INFO:tasks.workunit.client.1.smithi169.stdout:4/589: readlink d4/d11/d2e/d6c/d78/da5/l5f 0 2024-03-04T13:42:58.543 INFO:tasks.workunit.client.0.smithi107.stdout:0/149: rename da/de/f25 to da/f31 0 2024-03-04T13:42:58.547 INFO:tasks.workunit.client.0.smithi107.stdout:0/150: creat da/f32 x:0 0 0 2024-03-04T13:42:58.548 INFO:tasks.workunit.client.0.smithi107.stdout:2/139: truncate da/df/f15 2757424 0 2024-03-04T13:42:58.551 INFO:tasks.workunit.client.1.smithi169.stdout:4/590: dread d4/d11/d2e/d37/d5c/f8c [0,4194304] 0 2024-03-04T13:42:58.552 INFO:tasks.workunit.client.0.smithi107.stdout:0/151: creat da/dc/f33 x:0 0 0 2024-03-04T13:42:58.552 INFO:tasks.workunit.client.0.smithi107.stdout:0/152: fdatasync f0 0 2024-03-04T13:42:58.554 INFO:tasks.workunit.client.0.smithi107.stdout:0/153: dread - da/dc/f33 zero size 2024-03-04T13:42:58.555 INFO:tasks.workunit.client.1.smithi169.stdout:6/622: dwrite d0/d63/f78 [0,4194304] 0 2024-03-04T13:42:58.559 INFO:tasks.workunit.client.0.smithi107.stdout:9/94: truncate d4/da/f15 3997212 0 2024-03-04T13:42:58.559 INFO:tasks.workunit.client.0.smithi107.stdout:8/179: write d6/d16/f27 [3630830,25759] 0 2024-03-04T13:42:58.561 INFO:tasks.workunit.client.1.smithi169.stdout:5/730: dwrite d1/d16/d2a/d46/f6c [0,4194304] 0 2024-03-04T13:42:58.562 INFO:tasks.workunit.client.1.smithi169.stdout:9/663: truncate d7/dc/fd 4673359 0 2024-03-04T13:42:58.563 INFO:tasks.workunit.client.0.smithi107.stdout:2/140: dwrite da/fb [0,4194304] 0 2024-03-04T13:42:58.564 INFO:tasks.workunit.client.0.smithi107.stdout:2/141: read f2 [3300089,43506] 0 2024-03-04T13:42:58.566 INFO:tasks.workunit.client.1.smithi169.stdout:9/664: write d7/d25/d3c/f62 [2278961,50339] 0 2024-03-04T13:42:58.566 INFO:tasks.workunit.client.1.smithi169.stdout:6/623: symlink d0/d63/le5 0 2024-03-04T13:42:58.570 INFO:tasks.workunit.client.1.smithi169.stdout:5/731: chown d1/d16/d2f/d72/dab/lad 33046934 1 2024-03-04T13:42:58.570 INFO:tasks.workunit.client.0.smithi107.stdout:5/76: write d5/f9 [1821217,15836] 0 2024-03-04T13:42:58.573 INFO:tasks.workunit.client.0.smithi107.stdout:5/77: fsync d5/f6 0 2024-03-04T13:42:58.574 INFO:tasks.workunit.client.0.smithi107.stdout:5/78: stat d5/d8/da 0 2024-03-04T13:42:58.576 INFO:tasks.workunit.client.1.smithi169.stdout:4/591: dwrite d4/f7 [0,4194304] 0 2024-03-04T13:42:58.576 INFO:tasks.workunit.client.0.smithi107.stdout:5/79: chown d5/d8/d14 275276857 1 2024-03-04T13:42:58.579 INFO:tasks.workunit.client.0.smithi107.stdout:0/154: dwrite f4 [0,4194304] 0 2024-03-04T13:42:58.579 INFO:tasks.workunit.client.1.smithi169.stdout:9/665: creat d7/d49/fe9 x:0 0 0 2024-03-04T13:42:58.590 INFO:tasks.workunit.client.0.smithi107.stdout:5/80: dread d5/dc/df/f12 [0,4194304] 0 2024-03-04T13:42:58.591 INFO:tasks.workunit.client.1.smithi169.stdout:6/624: getdents d0/d3c/d95 0 2024-03-04T13:42:58.592 INFO:tasks.workunit.client.1.smithi169.stdout:6/625: fsync d0/d32/f6b 0 2024-03-04T13:42:58.593 INFO:tasks.workunit.client.0.smithi107.stdout:6/132: truncate df/d1b/f1d 1977948 0 2024-03-04T13:42:58.597 INFO:tasks.workunit.client.0.smithi107.stdout:0/155: mkdir da/dc/d34 0 2024-03-04T13:42:58.600 INFO:tasks.workunit.client.0.smithi107.stdout:8/180: getdents d6 0 2024-03-04T13:42:58.602 INFO:tasks.workunit.client.1.smithi169.stdout:5/732: symlink d1/d16/d17/d2d/d5a/lf8 0 2024-03-04T13:42:58.605 INFO:tasks.workunit.client.1.smithi169.stdout:6/626: truncate d0/f1 636109 0 2024-03-04T13:42:58.607 INFO:tasks.workunit.client.1.smithi169.stdout:6/627: truncate d0/d3c/d40/f9d 4633877 0 2024-03-04T13:42:58.610 INFO:tasks.workunit.client.0.smithi107.stdout:5/81: getdents d5/d8/d14 0 2024-03-04T13:42:58.612 INFO:tasks.workunit.client.1.smithi169.stdout:5/733: mknod d1/d16/d17/d2d/d5a/d67/cf9 0 2024-03-04T13:42:58.612 INFO:tasks.workunit.client.1.smithi169.stdout:6/628: dread d0/d32/f81 [0,4194304] 0 2024-03-04T13:42:58.612 INFO:tasks.workunit.client.0.smithi107.stdout:5/82: stat d5/d8/da/f16 0 2024-03-04T13:42:58.612 INFO:tasks.workunit.client.0.smithi107.stdout:2/142: link da/c2a da/c2f 0 2024-03-04T13:42:58.613 INFO:tasks.workunit.client.0.smithi107.stdout:5/83: fdatasync d5/f7 0 2024-03-04T13:42:58.614 INFO:tasks.workunit.client.1.smithi169.stdout:4/592: rename d4/d11/d2e/d21/la6 to d4/d11/d2e/d39/lbd 0 2024-03-04T13:42:58.615 INFO:tasks.workunit.client.0.smithi107.stdout:5/84: write d5/f9 [196224,128316] 0 2024-03-04T13:42:58.617 INFO:tasks.workunit.client.0.smithi107.stdout:9/95: truncate d4/da/d11/f13 4091474 0 2024-03-04T13:42:58.620 INFO:tasks.workunit.client.0.smithi107.stdout:5/85: dread d5/f7 [0,4194304] 0 2024-03-04T13:42:58.620 INFO:tasks.workunit.client.0.smithi107.stdout:2/143: dread da/f1c [0,4194304] 0 2024-03-04T13:42:58.622 INFO:tasks.workunit.client.1.smithi169.stdout:5/734: mkdir d1/d16/d2a/d46/d9e/dfa 0 2024-03-04T13:42:58.625 INFO:tasks.workunit.client.1.smithi169.stdout:5/735: write d1/d16/d17/d2d/ff2 [158664,97957] 0 2024-03-04T13:42:58.628 INFO:tasks.workunit.client.1.smithi169.stdout:9/666: link d7/d2d/lbb d7/dc/d12/lea 0 2024-03-04T13:42:58.629 INFO:tasks.workunit.client.0.smithi107.stdout:9/96: rename d4/da/lb to d4/da/l16 0 2024-03-04T13:42:58.632 INFO:tasks.workunit.client.1.smithi169.stdout:9/667: read d7/dc/d12/d59/f6d [7844025,19811] 0 2024-03-04T13:42:58.632 INFO:tasks.workunit.client.1.smithi169.stdout:5/736: symlink d1/d16/d17/d2d/d5a/d67/lfb 0 2024-03-04T13:42:58.632 INFO:tasks.workunit.client.0.smithi107.stdout:2/144: creat da/f30 x:0 0 0 2024-03-04T13:42:58.633 INFO:tasks.workunit.client.1.smithi169.stdout:6/629: dread d0/d3c/d95/fb3 [0,4194304] 0 2024-03-04T13:42:58.634 INFO:tasks.workunit.client.0.smithi107.stdout:8/181: dread f2 [0,4194304] 0 2024-03-04T13:42:58.637 INFO:tasks.workunit.client.1.smithi169.stdout:2/677: sync 2024-03-04T13:42:58.637 INFO:tasks.workunit.client.1.smithi169.stdout:8/639: sync 2024-03-04T13:42:58.639 INFO:tasks.workunit.client.1.smithi169.stdout:5/737: mknod d1/d81/cfc 0 2024-03-04T13:42:58.640 INFO:tasks.workunit.client.1.smithi169.stdout:5/738: readlink d1/d81/l9c 0 2024-03-04T13:42:58.641 INFO:tasks.workunit.client.0.smithi107.stdout:0/156: creat da/de/f35 x:0 0 0 2024-03-04T13:42:58.642 INFO:tasks.workunit.client.0.smithi107.stdout:0/157: dread - da/f2f zero size 2024-03-04T13:42:58.643 INFO:tasks.workunit.client.1.smithi169.stdout:9/668: symlink d7/dc/d12/d20/leb 0 2024-03-04T13:42:58.644 INFO:tasks.workunit.client.0.smithi107.stdout:9/97: creat d4/da/d11/f17 x:0 0 0 2024-03-04T13:42:58.644 INFO:tasks.workunit.client.1.smithi169.stdout:6/630: mknod d0/d32/d3e/db9/ce6 0 2024-03-04T13:42:58.647 INFO:tasks.workunit.client.1.smithi169.stdout:2/678: unlink d1/d5/d19/d3c/f5f 0 2024-03-04T13:42:58.647 INFO:tasks.workunit.client.0.smithi107.stdout:6/133: dwrite df/d17/f1e [0,4194304] 0 2024-03-04T13:42:58.649 INFO:tasks.workunit.client.1.smithi169.stdout:5/739: mknod d1/d81/db1/cfd 0 2024-03-04T13:42:58.650 INFO:tasks.workunit.client.0.smithi107.stdout:8/182: creat d6/d16/d1f/f34 x:0 0 0 2024-03-04T13:42:58.655 INFO:tasks.workunit.client.1.smithi169.stdout:6/631: symlink d0/d3c/d40/le7 0 2024-03-04T13:42:58.655 INFO:tasks.workunit.client.0.smithi107.stdout:0/158: dread f4 [0,4194304] 0 2024-03-04T13:42:58.658 INFO:tasks.workunit.client.1.smithi169.stdout:6/632: truncate d0/d32/d70/fdc 999913 0 2024-03-04T13:42:58.661 INFO:tasks.workunit.client.1.smithi169.stdout:5/740: mkdir d1/d16/d17/d28/dfe 0 2024-03-04T13:42:58.664 INFO:tasks.workunit.client.1.smithi169.stdout:6/633: creat d0/d32/fe8 x:0 0 0 2024-03-04T13:42:58.664 INFO:tasks.workunit.client.1.smithi169.stdout:2/679: dwrite d1/d5/d19/d3c/f6c [4194304,4194304] 0 2024-03-04T13:42:58.665 INFO:tasks.workunit.client.1.smithi169.stdout:5/741: write d1/d16/d17/d1b/d74/fc6 [306432,86950] 0 2024-03-04T13:42:58.666 INFO:tasks.workunit.client.0.smithi107.stdout:5/86: link d5/l10 d5/d8/d14/l1a 0 2024-03-04T13:42:58.666 INFO:tasks.workunit.client.0.smithi107.stdout:6/134: dwrite df/d17/f23 [0,4194304] 0 2024-03-04T13:42:58.668 INFO:tasks.workunit.client.0.smithi107.stdout:6/135: chown df/f20 438 1 2024-03-04T13:42:58.670 INFO:tasks.workunit.client.0.smithi107.stdout:6/136: write f3 [538854,6308] 0 2024-03-04T13:42:58.672 INFO:tasks.workunit.client.0.smithi107.stdout:9/98: symlink d4/l18 0 2024-03-04T13:42:58.674 INFO:tasks.workunit.client.1.smithi169.stdout:6/634: mknod d0/d32/d3a/d5e/ce9 0 2024-03-04T13:42:58.675 INFO:tasks.workunit.client.1.smithi169.stdout:6/635: chown d0/fb 1274777 1 2024-03-04T13:42:58.675 INFO:tasks.workunit.client.1.smithi169.stdout:2/680: symlink d1/d5/d22/d45/led 0 2024-03-04T13:42:58.676 INFO:tasks.workunit.client.0.smithi107.stdout:0/159: dwrite da/dc/f1d [0,4194304] 0 2024-03-04T13:42:58.679 INFO:tasks.workunit.client.0.smithi107.stdout:9/99: chown d4/da/l10 72 1 2024-03-04T13:42:58.680 INFO:tasks.workunit.client.1.smithi169.stdout:2/681: chown d1/d5/d19/d1e/f43 40 1 2024-03-04T13:42:58.683 INFO:tasks.workunit.client.0.smithi107.stdout:2/145: getdents da/df 0 2024-03-04T13:42:58.690 INFO:tasks.workunit.client.1.smithi169.stdout:5/742: dwrite d1/d76/fb2 [0,4194304] 0 2024-03-04T13:42:58.691 INFO:tasks.workunit.client.0.smithi107.stdout:8/183: rename d6/d16/d1f/f2b to d6/da/f35 0 2024-03-04T13:42:58.695 INFO:tasks.workunit.client.1.smithi169.stdout:6/636: truncate d0/d3c/d95/dbd/fc6 4680230 0 2024-03-04T13:42:58.702 INFO:tasks.workunit.client.1.smithi169.stdout:6/637: dread d0/d32/d70/fdc [0,4194304] 0 2024-03-04T13:42:58.706 INFO:tasks.workunit.client.1.smithi169.stdout:9/669: dwrite d7/d2d/d48/f7f [0,4194304] 0 2024-03-04T13:42:58.708 INFO:tasks.workunit.client.0.smithi107.stdout:0/160: symlink da/l36 0 2024-03-04T13:42:58.708 INFO:tasks.workunit.client.0.smithi107.stdout:6/137: write f0 [1297818,74251] 0 2024-03-04T13:42:58.710 INFO:tasks.workunit.client.1.smithi169.stdout:5/743: rename d1/d16/d17/d28/ff6 to d1/d16/d17/d2d/d5a/d67/fff 0 2024-03-04T13:42:58.713 INFO:tasks.workunit.client.0.smithi107.stdout:8/184: dwrite d6/d16/d1f/f34 [0,4194304] 0 2024-03-04T13:42:58.714 INFO:tasks.workunit.client.0.smithi107.stdout:0/161: stat da/l18 0 2024-03-04T13:42:58.714 INFO:tasks.workunit.client.0.smithi107.stdout:5/87: dread d5/f9 [0,4194304] 0 2024-03-04T13:42:58.715 INFO:tasks.workunit.client.0.smithi107.stdout:8/185: readlink d6/d16/l21 0 2024-03-04T13:42:58.716 INFO:tasks.workunit.client.0.smithi107.stdout:0/162: fdatasync da/de/f35 0 2024-03-04T13:42:58.716 INFO:tasks.workunit.client.0.smithi107.stdout:9/100: creat d4/f19 x:0 0 0 2024-03-04T13:42:58.717 INFO:tasks.workunit.client.1.smithi169.stdout:5/744: creat d1/d76/f100 x:0 0 0 2024-03-04T13:42:58.718 INFO:tasks.workunit.client.1.smithi169.stdout:9/670: mkdir d7/d2d/d48/d66/dec 0 2024-03-04T13:42:58.718 INFO:tasks.workunit.client.0.smithi107.stdout:8/186: truncate d6/f26 153132 0 2024-03-04T13:42:58.719 INFO:tasks.workunit.client.1.smithi169.stdout:2/682: write d1/d5/d19/d3c/db8/de1/fa3 [796857,73393] 0 2024-03-04T13:42:58.720 INFO:tasks.workunit.client.1.smithi169.stdout:5/745: write d1/d81/db1/ff0 [469505,128417] 0 2024-03-04T13:42:58.720 INFO:tasks.workunit.client.0.smithi107.stdout:0/163: write da/dc/dd/f24 [1696468,128716] 0 2024-03-04T13:42:58.724 INFO:tasks.workunit.client.0.smithi107.stdout:8/187: write d6/d16/f1c [1103997,63909] 0 2024-03-04T13:42:58.724 INFO:tasks.workunit.client.1.smithi169.stdout:9/671: write d7/d25/d64/fd5 [353060,15035] 0 2024-03-04T13:42:58.724 INFO:tasks.workunit.client.1.smithi169.stdout:5/746: creat d1/d16/d2a/f101 x:0 0 0 2024-03-04T13:42:58.724 INFO:tasks.workunit.client.0.smithi107.stdout:0/164: fdatasync da/f32 0 2024-03-04T13:42:58.725 INFO:tasks.workunit.client.1.smithi169.stdout:2/683: creat d1/d5/d19/d3c/db8/fee x:0 0 0 2024-03-04T13:42:58.725 INFO:tasks.workunit.client.0.smithi107.stdout:9/101: symlink d4/l1a 0 2024-03-04T13:42:58.727 INFO:tasks.workunit.client.1.smithi169.stdout:9/672: chown d7/dc/d12/d32/d42/d4f/fb5 368 1 2024-03-04T13:42:58.728 INFO:tasks.workunit.client.0.smithi107.stdout:5/88: read d5/d8/fb [835312,105664] 0 2024-03-04T13:42:58.729 INFO:tasks.workunit.client.1.smithi169.stdout:2/684: mknod d1/d5/d19/d1e/d29/cef 0 2024-03-04T13:42:58.732 INFO:tasks.workunit.client.1.smithi169.stdout:9/673: mknod d7/dc/d12/d59/ced 0 2024-03-04T13:42:58.733 INFO:tasks.workunit.client.0.smithi107.stdout:5/89: write d5/d8/f17 [97399,49240] 0 2024-03-04T13:42:58.733 INFO:tasks.workunit.client.0.smithi107.stdout:9/102: creat d4/da/f1b x:0 0 0 2024-03-04T13:42:58.735 INFO:tasks.workunit.client.1.smithi169.stdout:9/674: chown d7/dc/d12/d32/d42/f43 1718 1 2024-03-04T13:42:58.737 INFO:tasks.workunit.client.0.smithi107.stdout:6/138: dwrite df/d17/f23 [0,4194304] 0 2024-03-04T13:42:58.737 INFO:tasks.workunit.client.0.smithi107.stdout:2/146: write f2 [2275008,64672] 0 2024-03-04T13:42:58.739 INFO:tasks.workunit.client.0.smithi107.stdout:2/147: chown f7 0 1 2024-03-04T13:42:58.739 INFO:tasks.workunit.client.0.smithi107.stdout:5/90: rmdir d5/d8 39 2024-03-04T13:42:58.740 INFO:tasks.workunit.client.0.smithi107.stdout:6/139: write df/d17/d24/f2d [957437,86267] 0 2024-03-04T13:42:58.743 INFO:tasks.workunit.client.0.smithi107.stdout:0/165: dread da/dc/f1d [0,4194304] 0 2024-03-04T13:42:58.745 INFO:tasks.workunit.client.0.smithi107.stdout:0/166: readlink da/l18 0 2024-03-04T13:42:58.745 INFO:tasks.workunit.client.0.smithi107.stdout:5/91: chown d5/d8/da/ld 4631 1 2024-03-04T13:42:58.747 INFO:tasks.workunit.client.0.smithi107.stdout:2/148: mknod da/df/c31 0 2024-03-04T13:42:58.748 INFO:tasks.workunit.client.0.smithi107.stdout:9/103: dread f0 [4194304,4194304] 0 2024-03-04T13:42:58.750 INFO:tasks.workunit.client.0.smithi107.stdout:0/167: stat da/dc/l13 0 2024-03-04T13:42:58.750 INFO:tasks.workunit.client.0.smithi107.stdout:5/92: dread d5/f7 [0,4194304] 0 2024-03-04T13:42:58.750 INFO:tasks.workunit.client.1.smithi169.stdout:2/685: dwrite d1/d5/d22/d45/f62 [4194304,4194304] 0 2024-03-04T13:42:58.752 INFO:tasks.workunit.client.0.smithi107.stdout:5/93: dread - d5/d8/da/f16 zero size 2024-03-04T13:42:58.753 INFO:tasks.workunit.client.1.smithi169.stdout:9/675: getdents d7/d25/d47/d56 0 2024-03-04T13:42:58.753 INFO:tasks.workunit.client.0.smithi107.stdout:0/168: write da/dc/dd/ff [1615194,81604] 0 2024-03-04T13:42:58.754 INFO:tasks.workunit.client.1.smithi169.stdout:6/638: dwrite d0/d3c/d40/d52/fa8 [0,4194304] 0 2024-03-04T13:42:58.755 INFO:tasks.workunit.client.0.smithi107.stdout:5/94: write d5/f6 [4071332,19589] 0 2024-03-04T13:42:58.755 INFO:tasks.workunit.client.0.smithi107.stdout:9/104: chown d4/da/l16 2998 1 2024-03-04T13:42:58.756 INFO:tasks.workunit.client.1.smithi169.stdout:2/686: write d1/d5/d22/d3b/f52 [8396501,124943] 0 2024-03-04T13:42:58.756 INFO:tasks.workunit.client.0.smithi107.stdout:2/149: mknod da/df/d27/c32 0 2024-03-04T13:42:58.758 INFO:tasks.workunit.client.0.smithi107.stdout:5/95: chown d5/f6 12551333 1 2024-03-04T13:42:58.764 INFO:tasks.workunit.client.1.smithi169.stdout:2/687: dread d1/d6d/dbc/fbe [0,4194304] 0 2024-03-04T13:42:58.766 INFO:tasks.workunit.client.0.smithi107.stdout:2/150: mkdir da/df/d33 0 2024-03-04T13:42:58.767 INFO:tasks.workunit.client.0.smithi107.stdout:6/140: dwrite df/f29 [0,4194304] 0 2024-03-04T13:42:58.767 INFO:tasks.workunit.client.0.smithi107.stdout:2/151: chown da/df/c17 5631 1 2024-03-04T13:42:58.770 INFO:tasks.workunit.client.1.smithi169.stdout:6/639: creat d0/d32/d3a/d5e/fea x:0 0 0 2024-03-04T13:42:58.773 INFO:tasks.workunit.client.1.smithi169.stdout:6/640: write d0/d32/d70/fcc [2676626,99083] 0 2024-03-04T13:42:58.774 INFO:tasks.workunit.client.1.smithi169.stdout:5/747: dwrite d1/d16/d17/f71 [0,4194304] 0 2024-03-04T13:42:58.775 INFO:tasks.workunit.client.1.smithi169.stdout:5/748: fdatasync d1/fc0 0 2024-03-04T13:42:58.778 INFO:tasks.workunit.client.1.smithi169.stdout:9/676: rename d7/dc/d12/d32/d42/f7e to d7/dc/d12/d32/d42/d4f/fee 0 2024-03-04T13:42:58.784 INFO:tasks.workunit.client.1.smithi169.stdout:6/641: creat d0/d3c/d40/d52/dac/feb x:0 0 0 2024-03-04T13:42:58.785 INFO:tasks.workunit.client.0.smithi107.stdout:5/96: truncate d5/dc/df/f12 2780943 0 2024-03-04T13:42:58.787 INFO:tasks.workunit.client.0.smithi107.stdout:6/141: dwrite f0 [0,4194304] 0 2024-03-04T13:42:58.804 INFO:tasks.workunit.client.1.smithi169.stdout:5/749: mknod d1/d16/d17/d2d/d5a/d8a/c102 0 2024-03-04T13:42:58.825 INFO:tasks.workunit.client.1.smithi169.stdout:1/599: sync 2024-03-04T13:42:58.825 INFO:tasks.workunit.client.1.smithi169.stdout:7/709: sync 2024-03-04T13:42:58.827 INFO:tasks.workunit.client.1.smithi169.stdout:1/600: chown d4/d5/d36/d75/dba/fbb 9316348 1 2024-03-04T13:42:58.830 INFO:tasks.workunit.client.0.smithi107.stdout:9/105: creat d4/f1c x:0 0 0 2024-03-04T13:42:58.830 INFO:tasks.workunit.client.1.smithi169.stdout:7/710: dread d2/de/d3e/d41/fa9 [0,4194304] 0 2024-03-04T13:42:58.836 INFO:tasks.workunit.client.1.smithi169.stdout:7/711: dread d2/de/d3e/d41/d5b/f9f [0,4194304] 0 2024-03-04T13:42:58.836 INFO:tasks.workunit.client.0.smithi107.stdout:6/142: dread df/d17/d24/d27/f26 [0,4194304] 0 2024-03-04T13:42:58.840 INFO:tasks.workunit.client.0.smithi107.stdout:0/169: link da/dc/dd/l19 da/dc/d34/l37 0 2024-03-04T13:42:58.842 INFO:tasks.workunit.client.0.smithi107.stdout:0/170: write da/f2f [542186,25213] 0 2024-03-04T13:42:58.849 INFO:tasks.workunit.client.0.smithi107.stdout:9/106: symlink d4/l1d 0 2024-03-04T13:42:58.850 INFO:tasks.workunit.client.0.smithi107.stdout:9/107: read f0 [5388810,17319] 0 2024-03-04T13:42:58.850 INFO:tasks.workunit.client.0.smithi107.stdout:5/97: link d5/dc/l18 d5/d8/l1b 0 2024-03-04T13:42:58.854 INFO:tasks.workunit.client.1.smithi169.stdout:9/677: rename d7/d25/fd9 to d7/d2d/d48/fef 0 2024-03-04T13:42:58.856 INFO:tasks.workunit.client.0.smithi107.stdout:0/171: rename da/dc/dd/ff to da/dc/d34/f38 0 2024-03-04T13:42:58.857 INFO:tasks.workunit.client.1.smithi169.stdout:6/642: creat d0/d32/d3a/d43/d74/d93/dbe/fec x:0 0 0 2024-03-04T13:42:58.858 INFO:tasks.workunit.client.0.smithi107.stdout:6/143: mkdir df/d1b/d2c/d31 0 2024-03-04T13:42:58.859 INFO:tasks.workunit.client.0.smithi107.stdout:0/172: write da/de/f11 [5471398,68191] 0 2024-03-04T13:42:58.864 INFO:tasks.workunit.client.0.smithi107.stdout:5/98: unlink c4 0 2024-03-04T13:42:58.871 INFO:tasks.workunit.client.1.smithi169.stdout:1/601: getdents d4/d5/d36/d35 0 2024-03-04T13:42:58.872 INFO:tasks.workunit.client.0.smithi107.stdout:0/173: mknod da/dc/c39 0 2024-03-04T13:42:58.873 INFO:tasks.workunit.client.0.smithi107.stdout:0/174: read - da/f31 zero size 2024-03-04T13:42:58.875 INFO:tasks.workunit.client.0.smithi107.stdout:9/108: mknod d4/c1e 0 2024-03-04T13:42:58.876 INFO:tasks.workunit.client.1.smithi169.stdout:7/712: getdents d2/de 0 2024-03-04T13:42:58.876 INFO:tasks.workunit.client.0.smithi107.stdout:5/99: symlink d5/dc/l1c 0 2024-03-04T13:42:58.877 INFO:tasks.workunit.client.1.smithi169.stdout:7/713: chown d2/de 0 1 2024-03-04T13:42:58.879 INFO:tasks.workunit.client.1.smithi169.stdout:7/714: write d2/d28/fe6 [814181,79956] 0 2024-03-04T13:42:58.881 INFO:tasks.workunit.client.1.smithi169.stdout:9/678: getdents d7/dc/d12/d20 0 2024-03-04T13:42:58.882 INFO:tasks.workunit.client.0.smithi107.stdout:6/144: dwrite df/f15 [0,4194304] 0 2024-03-04T13:42:58.884 INFO:tasks.workunit.client.0.smithi107.stdout:6/145: chown df/d17/d24/d30 51 1 2024-03-04T13:42:58.885 INFO:tasks.workunit.client.0.smithi107.stdout:9/109: mknod d4/da/c1f 0 2024-03-04T13:42:58.886 INFO:tasks.workunit.client.0.smithi107.stdout:5/100: truncate d5/d8/f11 1753592 0 2024-03-04T13:42:58.888 INFO:tasks.workunit.client.1.smithi169.stdout:9/679: write d7/d2d/d48/fef [901249,95762] 0 2024-03-04T13:42:58.889 INFO:tasks.workunit.client.0.smithi107.stdout:0/175: getdents da/dc/dd/d1e 0 2024-03-04T13:42:58.895 INFO:tasks.workunit.client.0.smithi107.stdout:9/110: mkdir d4/da/d11/d20 0 2024-03-04T13:42:58.897 INFO:tasks.workunit.client.0.smithi107.stdout:9/111: read - d4/f19 zero size 2024-03-04T13:42:58.899 INFO:tasks.workunit.client.1.smithi169.stdout:7/715: dwrite d2/de/d3e/d41/d69/fab [4194304,4194304] 0 2024-03-04T13:42:58.905 INFO:tasks.workunit.client.1.smithi169.stdout:9/680: dwrite d7/dc/d12/d20/da6/fb4 [0,4194304] 0 2024-03-04T13:42:58.908 INFO:tasks.workunit.client.1.smithi169.stdout:9/681: chown d7/dc/d12/d32/d42/fd6 0 1 2024-03-04T13:42:58.915 INFO:tasks.workunit.client.0.smithi107.stdout:0/176: dwrite da/de/f11 [4194304,4194304] 0 2024-03-04T13:42:58.915 INFO:tasks.workunit.client.0.smithi107.stdout:6/146: dwrite f0 [0,4194304] 0 2024-03-04T13:42:58.921 INFO:tasks.workunit.client.0.smithi107.stdout:0/177: write da/dc/dd/f24 [216311,41582] 0 2024-03-04T13:42:58.933 INFO:tasks.workunit.client.0.smithi107.stdout:6/147: dread df/d17/f1e [0,4194304] 0 2024-03-04T13:42:58.935 INFO:tasks.workunit.client.1.smithi169.stdout:3/663: sync 2024-03-04T13:42:58.935 INFO:tasks.workunit.client.1.smithi169.stdout:0/516: sync 2024-03-04T13:42:58.936 INFO:tasks.workunit.client.0.smithi107.stdout:6/148: dread - df/d17/d24/f2b zero size 2024-03-04T13:42:58.937 INFO:tasks.workunit.client.1.smithi169.stdout:9/682: dwrite d7/dc/d3e/da3/fc9 [4194304,4194304] 0 2024-03-04T13:42:58.937 INFO:tasks.workunit.client.0.smithi107.stdout:6/149: stat df/d1b 0 2024-03-04T13:42:58.939 INFO:tasks.workunit.client.1.smithi169.stdout:5/750: truncate d1/d16/d17/f71 2779562 0 2024-03-04T13:42:58.940 INFO:tasks.workunit.client.1.smithi169.stdout:9/683: chown d7/dc/d12/d32/d42 4 1 2024-03-04T13:42:58.941 INFO:tasks.workunit.client.1.smithi169.stdout:0/517: creat d5/d4c/d65/d95/fb9 x:0 0 0 2024-03-04T13:42:58.943 INFO:tasks.workunit.client.0.smithi107.stdout:6/150: symlink df/d1b/d2c/l32 0 2024-03-04T13:42:58.945 INFO:tasks.workunit.client.0.smithi107.stdout:6/151: write f8 [2258387,99492] 0 2024-03-04T13:42:58.948 INFO:tasks.workunit.client.1.smithi169.stdout:5/751: mkdir d1/d16/d2a/d103 0 2024-03-04T13:42:58.949 INFO:tasks.workunit.client.1.smithi169.stdout:0/518: write d5/d9/dc/d12/d15/d19/d61/d71/d8c/d90/f97 [834533,11426] 0 2024-03-04T13:42:58.954 INFO:tasks.workunit.client.1.smithi169.stdout:7/716: dread d2/de/ff [4194304,4194304] 0 2024-03-04T13:42:58.960 INFO:tasks.workunit.client.0.smithi107.stdout:6/152: dread df/d17/f1e [0,4194304] 0 2024-03-04T13:42:58.965 INFO:tasks.workunit.client.1.smithi169.stdout:6/643: write d0/d3c/f68 [1564257,95298] 0 2024-03-04T13:42:58.970 INFO:tasks.workunit.client.0.smithi107.stdout:6/153: dread fd [0,4194304] 0 2024-03-04T13:42:58.974 INFO:tasks.workunit.client.0.smithi107.stdout:6/154: mknod df/d17/d24/d27/c33 0 2024-03-04T13:42:58.976 INFO:tasks.workunit.client.0.smithi107.stdout:6/155: read df/d17/d24/f2d [32354,16970] 0 2024-03-04T13:42:58.977 INFO:tasks.workunit.client.0.smithi107.stdout:4/99: sync 2024-03-04T13:42:58.977 INFO:tasks.workunit.client.0.smithi107.stdout:7/101: sync 2024-03-04T13:42:58.977 INFO:tasks.workunit.client.0.smithi107.stdout:1/78: sync 2024-03-04T13:42:58.977 INFO:tasks.workunit.client.0.smithi107.stdout:3/89: sync 2024-03-04T13:42:58.977 INFO:tasks.workunit.client.0.smithi107.stdout:6/156: fsync df/d17/d24/f2b 0 2024-03-04T13:42:58.979 INFO:tasks.workunit.client.0.smithi107.stdout:1/79: chown d1/d3/ca 1 1 2024-03-04T13:42:58.979 INFO:tasks.workunit.client.1.smithi169.stdout:1/602: dwrite d4/d5/d36/d35/f56 [0,4194304] 0 2024-03-04T13:42:58.980 INFO:tasks.workunit.client.1.smithi169.stdout:1/603: chown d4/d5/d36/d75 17509538 1 2024-03-04T13:42:58.985 INFO:tasks.workunit.client.1.smithi169.stdout:3/664: creat d7/de/d2b/d7d/d85/d18/d21/fe4 x:0 0 0 2024-03-04T13:42:58.987 INFO:tasks.workunit.client.0.smithi107.stdout:3/90: dread d1/f6 [0,4194304] 0 2024-03-04T13:42:58.988 INFO:tasks.workunit.client.0.smithi107.stdout:3/91: stat d1/l10 0 2024-03-04T13:42:58.988 INFO:tasks.workunit.client.1.smithi169.stdout:9/684: mknod d7/d25/cf0 0 2024-03-04T13:42:58.990 INFO:tasks.workunit.client.1.smithi169.stdout:9/685: write d7/dc/d12/d20/d4e/fe5 [605557,120491] 0 2024-03-04T13:42:58.991 INFO:tasks.workunit.client.0.smithi107.stdout:6/157: rename df/d17/c1a to df/d1b/d2c/d31/c34 0 2024-03-04T13:42:58.993 INFO:tasks.workunit.client.1.smithi169.stdout:0/519: truncate d5/d9/dc/d12/d15/d19/d61/f33 53142 0 2024-03-04T13:42:58.996 INFO:tasks.workunit.client.1.smithi169.stdout:6/644: creat d0/d32/d3a/d5e/fed x:0 0 0 2024-03-04T13:42:58.996 INFO:tasks.workunit.client.0.smithi107.stdout:1/80: dwrite d1/d3/f11 [0,4194304] 0 2024-03-04T13:42:58.998 INFO:tasks.workunit.client.0.smithi107.stdout:4/100: dwrite d0/d9/f11 [0,4194304] 0 2024-03-04T13:42:58.999 INFO:tasks.workunit.client.0.smithi107.stdout:1/81: readlink d1/lc 0 2024-03-04T13:42:59.002 INFO:tasks.workunit.client.1.smithi169.stdout:7/717: creat d2/de/d3e/d41/dec/ff2 x:0 0 0 2024-03-04T13:42:59.002 INFO:tasks.workunit.client.0.smithi107.stdout:1/82: write d1/fe [1021215,88592] 0 2024-03-04T13:42:59.003 INFO:tasks.workunit.client.0.smithi107.stdout:3/92: truncate d1/de/f1d 3356386 0 2024-03-04T13:42:59.004 INFO:tasks.workunit.client.1.smithi169.stdout:7/718: fdatasync d2/de/d3e/d41/d69/fc8 0 2024-03-04T13:42:59.006 INFO:tasks.workunit.client.0.smithi107.stdout:3/93: chown d1/f3 6071 1 2024-03-04T13:42:59.010 INFO:tasks.workunit.client.1.smithi169.stdout:1/604: creat d4/d5/d2d/d4e/da5/fc5 x:0 0 0 2024-03-04T13:42:59.010 INFO:tasks.workunit.client.1.smithi169.stdout:3/665: mknod d7/de/d2b/d7d/d85/d13/d1a/d78/dc1/ce5 0 2024-03-04T13:42:59.015 INFO:tasks.workunit.client.0.smithi107.stdout:9/112: dwrite d4/da/f15 [0,4194304] 0 2024-03-04T13:42:59.017 INFO:tasks.workunit.client.1.smithi169.stdout:6/645: dread d0/f9 [4194304,4194304] 0 2024-03-04T13:42:59.019 INFO:tasks.workunit.client.0.smithi107.stdout:3/94: mkdir d1/de/d18/d1f 0 2024-03-04T13:42:59.019 INFO:tasks.workunit.client.1.smithi169.stdout:0/520: dread d5/d9/dc/d12/d15/d46/f57 [0,4194304] 0 2024-03-04T13:42:59.020 INFO:tasks.workunit.client.0.smithi107.stdout:9/113: truncate d4/da/f1b 338000 0 2024-03-04T13:42:59.021 INFO:tasks.workunit.client.0.smithi107.stdout:3/95: write d1/de/f19 [3502074,37680] 0 2024-03-04T13:42:59.022 INFO:tasks.workunit.client.0.smithi107.stdout:1/83: dwrite d1/d3/f11 [0,4194304] 0 2024-03-04T13:42:59.022 INFO:tasks.workunit.client.0.smithi107.stdout:9/114: write d4/f1c [630819,66025] 0 2024-03-04T13:42:59.024 INFO:tasks.workunit.client.1.smithi169.stdout:0/521: readlink d5/d9/d3d/l80 0 2024-03-04T13:42:59.027 INFO:tasks.workunit.client.1.smithi169.stdout:0/522: readlink d5/d9/d3d/l50 0 2024-03-04T13:42:59.027 INFO:tasks.workunit.client.1.smithi169.stdout:9/686: rename d7/d25/d47 to d7/dc/df1 0 2024-03-04T13:42:59.030 INFO:tasks.workunit.client.1.smithi169.stdout:0/523: write d5/d4c/d65/faf [632200,88528] 0 2024-03-04T13:42:59.030 INFO:tasks.workunit.client.1.smithi169.stdout:9/687: fsync d7/dc/d12/d32/d42/d4f/fb5 0 2024-03-04T13:42:59.031 INFO:tasks.workunit.client.0.smithi107.stdout:6/158: mknod df/d2e/c35 0 2024-03-04T13:42:59.031 INFO:tasks.workunit.client.1.smithi169.stdout:7/719: dwrite d2/de/d3e/d41/fc3 [0,4194304] 0 2024-03-04T13:42:59.034 INFO:tasks.workunit.client.0.smithi107.stdout:0/178: truncate da/de/f1c 606823 0 2024-03-04T13:42:59.035 INFO:tasks.workunit.client.0.smithi107.stdout:0/179: stat da/l36 0 2024-03-04T13:42:59.037 INFO:tasks.workunit.client.1.smithi169.stdout:6/646: mkdir d0/d32/d3a/d43/d74/d93/dee 0 2024-03-04T13:42:59.040 INFO:tasks.workunit.client.1.smithi169.stdout:6/647: fdatasync d0/d32/d3a/d5e/fd4 0 2024-03-04T13:42:59.040 INFO:tasks.workunit.client.1.smithi169.stdout:3/666: truncate d7/de/d2b/d7d/d85/d18/d21/d3f/f82 323445 0 2024-03-04T13:42:59.041 INFO:tasks.workunit.client.0.smithi107.stdout:1/84: dwrite d1/d3/f11 [0,4194304] 0 2024-03-04T13:42:59.047 INFO:tasks.workunit.client.1.smithi169.stdout:0/524: mkdir d5/d4c/d65/d95/dba 0 2024-03-04T13:42:59.048 INFO:tasks.workunit.client.1.smithi169.stdout:7/720: fsync d2/de/d3e/f62 0 2024-03-04T13:42:59.051 INFO:tasks.workunit.client.0.smithi107.stdout:9/115: unlink d4/ce 0 2024-03-04T13:42:59.053 INFO:tasks.workunit.client.1.smithi169.stdout:6/648: mkdir d0/d32/d3a/def 0 2024-03-04T13:42:59.053 INFO:tasks.workunit.client.1.smithi169.stdout:6/649: chown d0 15816 1 2024-03-04T13:42:59.054 INFO:tasks.workunit.client.0.smithi107.stdout:3/96: creat d1/de/d18/d1f/f20 x:0 0 0 2024-03-04T13:42:59.055 INFO:tasks.workunit.client.0.smithi107.stdout:3/97: dread - d1/de/d18/d1f/f20 zero size 2024-03-04T13:42:59.057 INFO:tasks.workunit.client.0.smithi107.stdout:3/98: write d1/de/f19 [9270534,15935] 0 2024-03-04T13:42:59.060 INFO:tasks.workunit.client.0.smithi107.stdout:3/99: write d1/f3 [106660,31680] 0 2024-03-04T13:42:59.061 INFO:tasks.workunit.client.0.smithi107.stdout:3/100: chown d1/de/cf 5 1 2024-03-04T13:42:59.062 INFO:tasks.workunit.client.0.smithi107.stdout:1/85: symlink d1/d3/df/l14 0 2024-03-04T13:42:59.066 INFO:tasks.workunit.client.0.smithi107.stdout:0/180: creat da/f3a x:0 0 0 2024-03-04T13:42:59.069 INFO:tasks.workunit.client.1.smithi169.stdout:2/688: sync 2024-03-04T13:42:59.069 INFO:tasks.workunit.client.1.smithi169.stdout:4/593: sync 2024-03-04T13:42:59.070 INFO:tasks.workunit.client.1.smithi169.stdout:8/640: sync 2024-03-04T13:42:59.072 INFO:tasks.workunit.client.1.smithi169.stdout:1/605: link d4/d5/d36/d35/d5b/f87 d4/d5/fc6 0 2024-03-04T13:42:59.072 INFO:tasks.workunit.client.1.smithi169.stdout:8/641: write d7/d10/d24/f3c [2539334,64887] 0 2024-03-04T13:42:59.074 INFO:tasks.workunit.client.0.smithi107.stdout:3/101: mkdir d1/d21 0 2024-03-04T13:42:59.075 INFO:tasks.workunit.client.0.smithi107.stdout:3/102: fdatasync d1/de/f19 0 2024-03-04T13:42:59.075 INFO:tasks.workunit.client.0.smithi107.stdout:1/86: dwrite d1/fe [0,4194304] 0 2024-03-04T13:42:59.077 INFO:tasks.workunit.client.0.smithi107.stdout:3/103: write d1/f14 [581623,62312] 0 2024-03-04T13:42:59.078 INFO:tasks.workunit.client.0.smithi107.stdout:3/104: readlink d1/l2 0 2024-03-04T13:42:59.079 INFO:tasks.workunit.client.1.smithi169.stdout:7/721: rename d2/de/fe7 to d2/ff3 0 2024-03-04T13:42:59.081 INFO:tasks.workunit.client.1.smithi169.stdout:4/594: mknod d4/d11/d2e/d37/d73/cbe 0 2024-03-04T13:42:59.083 INFO:tasks.workunit.client.1.smithi169.stdout:2/689: creat d1/d5/d19/d3c/db8/de1/daf/ff0 x:0 0 0 2024-03-04T13:42:59.084 INFO:tasks.workunit.client.0.smithi107.stdout:0/181: mknod da/dc/dd/c3b 0 2024-03-04T13:42:59.086 INFO:tasks.workunit.client.1.smithi169.stdout:3/667: dread d7/de/d24/f3c [0,4194304] 0 2024-03-04T13:42:59.087 INFO:tasks.workunit.client.1.smithi169.stdout:9/688: dread d7/dc/d3e/f41 [0,4194304] 0 2024-03-04T13:42:59.087 INFO:tasks.workunit.client.1.smithi169.stdout:5/752: sync 2024-03-04T13:42:59.087 INFO:tasks.workunit.client.0.smithi107.stdout:9/116: creat d4/f21 x:0 0 0 2024-03-04T13:42:59.090 INFO:tasks.workunit.client.1.smithi169.stdout:5/753: chown d1/d16/d2a/f101 7753064 1 2024-03-04T13:42:59.092 INFO:tasks.workunit.client.0.smithi107.stdout:1/87: mknod d1/d3/df/c15 0 2024-03-04T13:42:59.092 INFO:tasks.workunit.client.1.smithi169.stdout:3/668: truncate d7/de/d2b/d7d/d85/d13/d5a/d65/da4/fd3 55276 0 2024-03-04T13:42:59.095 INFO:tasks.workunit.client.1.smithi169.stdout:6/650: rename d0/d32/d3a/d43/f7b to d0/d3c/d40/ff0 0 2024-03-04T13:42:59.098 INFO:tasks.workunit.client.1.smithi169.stdout:4/595: creat d4/d11/d2e/d6c/d78/da5/fbf x:0 0 0 2024-03-04T13:42:59.100 INFO:tasks.workunit.client.0.smithi107.stdout:3/105: creat d1/d1e/f22 x:0 0 0 2024-03-04T13:42:59.102 INFO:tasks.workunit.client.1.smithi169.stdout:9/689: rmdir d7/dc/d3e/da3/dbc 39 2024-03-04T13:42:59.102 INFO:tasks.workunit.client.0.smithi107.stdout:5/101: dwrite d5/d8/f11 [0,4194304] 0 2024-03-04T13:42:59.104 INFO:tasks.workunit.client.0.smithi107.stdout:5/102: chown d5/f7 3 1 2024-03-04T13:42:59.104 INFO:tasks.workunit.client.1.smithi169.stdout:1/606: mknod d4/d5/d36/d35/d5b/cc7 0 2024-03-04T13:42:59.106 INFO:tasks.workunit.client.0.smithi107.stdout:7/102: write f5 [1237677,47916] 0 2024-03-04T13:42:59.106 INFO:tasks.workunit.client.1.smithi169.stdout:1/607: fdatasync d4/d5/d2d/d4e/d71/d26/f98 0 2024-03-04T13:42:59.108 INFO:tasks.workunit.client.0.smithi107.stdout:7/103: chown d6/d17 115 1 2024-03-04T13:42:59.110 INFO:tasks.workunit.client.0.smithi107.stdout:9/117: mknod d4/c22 0 2024-03-04T13:42:59.110 INFO:tasks.workunit.client.0.smithi107.stdout:1/88: dwrite d1/d3/f11 [0,4194304] 0 2024-03-04T13:42:59.113 INFO:tasks.workunit.client.0.smithi107.stdout:5/103: symlink d5/d8/da/l1d 0 2024-03-04T13:42:59.113 INFO:tasks.workunit.client.0.smithi107.stdout:1/89: stat d1/lc 0 2024-03-04T13:42:59.113 INFO:tasks.workunit.client.0.smithi107.stdout:3/106: creat d1/f23 x:0 0 0 2024-03-04T13:42:59.114 INFO:tasks.workunit.client.1.smithi169.stdout:2/690: rename d1/d5/d19/d63/c9b to d1/d6d/cf1 0 2024-03-04T13:42:59.114 INFO:tasks.workunit.client.1.smithi169.stdout:5/754: dread d1/d16/d17/d1b/f44 [0,4194304] 0 2024-03-04T13:42:59.115 INFO:tasks.workunit.client.0.smithi107.stdout:3/107: fsync d1/f14 0 2024-03-04T13:42:59.115 INFO:tasks.workunit.client.0.smithi107.stdout:0/182: creat da/f3c x:0 0 0 2024-03-04T13:42:59.115 INFO:tasks.workunit.client.0.smithi107.stdout:1/90: fsync d1/fe 0 2024-03-04T13:42:59.118 INFO:tasks.workunit.client.0.smithi107.stdout:1/91: write d1/d3/f11 [5138324,67116] 0 2024-03-04T13:42:59.121 INFO:tasks.workunit.client.0.smithi107.stdout:0/183: chown da/dc/c2e 0 1 2024-03-04T13:42:59.121 INFO:tasks.workunit.client.0.smithi107.stdout:6/159: getdents df/d2e 0 2024-03-04T13:42:59.124 INFO:tasks.workunit.client.0.smithi107.stdout:0/184: dread da/dc/dd/f1f [0,4194304] 0 2024-03-04T13:42:59.124 INFO:tasks.workunit.client.1.smithi169.stdout:8/642: rmdir d7/d10/dc9 0 2024-03-04T13:42:59.127 INFO:tasks.workunit.client.0.smithi107.stdout:0/185: read da/dc/dd/f24 [2198018,4631] 0 2024-03-04T13:42:59.129 INFO:tasks.workunit.client.0.smithi107.stdout:3/108: mknod d1/de/d11/c24 0 2024-03-04T13:42:59.131 INFO:tasks.workunit.client.0.smithi107.stdout:0/186: dread - da/dc/f33 zero size 2024-03-04T13:42:59.131 INFO:tasks.workunit.client.1.smithi169.stdout:0/525: write d5/d9/d3d/d59/f60 [1000765,58249] 0 2024-03-04T13:42:59.134 INFO:tasks.workunit.client.1.smithi169.stdout:6/651: unlink d0/d63/c67 0 2024-03-04T13:42:59.135 INFO:tasks.workunit.client.0.smithi107.stdout:9/118: dwrite d4/f19 [0,4194304] 0 2024-03-04T13:42:59.135 INFO:tasks.workunit.client.0.smithi107.stdout:7/104: dwrite d6/f7 [0,4194304] 0 2024-03-04T13:42:59.137 INFO:tasks.workunit.client.1.smithi169.stdout:5/755: creat d1/d16/dbd/dbb/f104 x:0 0 0 2024-03-04T13:42:59.139 INFO:tasks.workunit.client.1.smithi169.stdout:4/596: mkdir d4/d11/d2e/d6c/d78/da5/d77/dc0 0 2024-03-04T13:42:59.141 INFO:tasks.workunit.client.0.smithi107.stdout:8/188: sync 2024-03-04T13:42:59.141 INFO:tasks.workunit.client.0.smithi107.stdout:2/152: sync 2024-03-04T13:42:59.141 INFO:tasks.workunit.client.0.smithi107.stdout:6/160: dread fb [0,4194304] 0 2024-03-04T13:42:59.142 INFO:tasks.workunit.client.1.smithi169.stdout:9/690: symlink d7/dc/d3e/da3/dbc/lf2 0 2024-03-04T13:42:59.144 INFO:tasks.workunit.client.1.smithi169.stdout:9/691: truncate d7/dc/d12/d20/da6/fc5 433890 0 2024-03-04T13:42:59.150 INFO:tasks.workunit.client.1.smithi169.stdout:8/643: truncate d7/d10/f1e 2780284 0 2024-03-04T13:42:59.150 INFO:tasks.workunit.client.0.smithi107.stdout:1/92: mkdir d1/d10/d16 0 2024-03-04T13:42:59.152 INFO:tasks.workunit.client.1.smithi169.stdout:1/608: unlink d4/d5/d36/d35/df/c76 0 2024-03-04T13:42:59.154 INFO:tasks.workunit.client.0.smithi107.stdout:0/187: rename da/dc/c2e to da/dc/dd/d1e/c3d 0 2024-03-04T13:42:59.155 INFO:tasks.workunit.client.0.smithi107.stdout:0/188: dread - da/f32 zero size 2024-03-04T13:42:59.155 INFO:tasks.workunit.client.1.smithi169.stdout:3/669: creat d7/de/d2b/d7d/fe6 x:0 0 0 2024-03-04T13:42:59.157 INFO:tasks.workunit.client.1.smithi169.stdout:0/526: mkdir d5/d9/dc/d12/d15/d19/d61/d71/dbb 0 2024-03-04T13:42:59.160 INFO:tasks.workunit.client.0.smithi107.stdout:2/153: mkdir da/d34 0 2024-03-04T13:42:59.161 INFO:tasks.workunit.client.0.smithi107.stdout:2/154: chown da/df/l2c 1052455 1 2024-03-04T13:42:59.161 INFO:tasks.workunit.client.1.smithi169.stdout:6/652: write d0/d3c/d40/d52/dac/fc7 [313728,117117] 0 2024-03-04T13:42:59.163 INFO:tasks.workunit.client.1.smithi169.stdout:5/756: mkdir d1/d16/d2a/d46/d9e/d105 0 2024-03-04T13:42:59.164 INFO:tasks.workunit.client.1.smithi169.stdout:6/653: chown d0/d32/d3a/d43/d74/f62 642501 1 2024-03-04T13:42:59.164 INFO:tasks.workunit.client.1.smithi169.stdout:4/597: rmdir d4/d11/d3b/d67/d98 39 2024-03-04T13:42:59.167 INFO:tasks.workunit.client.0.smithi107.stdout:1/93: mkdir d1/d3/d17 0 2024-03-04T13:42:59.169 INFO:tasks.workunit.client.1.smithi169.stdout:9/692: symlink d7/dc/d12/d20/da6/lf3 0 2024-03-04T13:42:59.170 INFO:tasks.workunit.client.1.smithi169.stdout:9/693: write d7/d49/fe9 [436439,74256] 0 2024-03-04T13:42:59.172 INFO:tasks.workunit.client.1.smithi169.stdout:8/644: truncate d7/d10/d22/d46/f83 888374 0 2024-03-04T13:42:59.173 INFO:tasks.workunit.client.1.smithi169.stdout:8/645: fsync d7/d10/d22/d46/da5/da6/fb4 0 2024-03-04T13:42:59.174 INFO:tasks.workunit.client.0.smithi107.stdout:0/189: rename da/de to da/dc/dd/d3e 0 2024-03-04T13:42:59.178 INFO:tasks.workunit.client.0.smithi107.stdout:0/190: dread f8 [0,4194304] 0 2024-03-04T13:42:59.180 INFO:tasks.workunit.client.1.smithi169.stdout:7/722: dwrite d2/de/d3e/d41/d69/fab [8388608,4194304] 0 2024-03-04T13:42:59.181 INFO:tasks.workunit.client.0.smithi107.stdout:1/94: dwrite d1/d3/f11 [0,4194304] 0 2024-03-04T13:42:59.181 INFO:tasks.workunit.client.1.smithi169.stdout:2/691: rename d1/d5/d22/d45/d4a/lb6 to d1/d5/d22/lf2 0 2024-03-04T13:42:59.185 INFO:tasks.workunit.client.0.smithi107.stdout:5/104: write d5/dc/df/f12 [2605534,94635] 0 2024-03-04T13:42:59.196 INFO:tasks.workunit.client.0.smithi107.stdout:1/95: dread d1/fe [0,4194304] 0 2024-03-04T13:42:59.197 INFO:tasks.workunit.client.0.smithi107.stdout:2/155: mknod da/df/d33/c35 0 2024-03-04T13:42:59.198 INFO:tasks.workunit.client.0.smithi107.stdout:1/96: write d1/fe [1524210,20369] 0 2024-03-04T13:42:59.199 INFO:tasks.workunit.client.1.smithi169.stdout:3/670: truncate d7/de/d24/fa6 657572 0 2024-03-04T13:42:59.203 INFO:tasks.workunit.client.0.smithi107.stdout:8/189: unlink d6/da/d25/l31 0 2024-03-04T13:42:59.206 INFO:tasks.workunit.client.1.smithi169.stdout:4/598: dwrite d4/f15 [0,4194304] 0 2024-03-04T13:42:59.206 INFO:tasks.workunit.client.0.smithi107.stdout:5/105: mknod d5/dc/c1e 0 2024-03-04T13:42:59.210 INFO:tasks.workunit.client.0.smithi107.stdout:0/191: unlink da/dc/d34/l37 0 2024-03-04T13:42:59.211 INFO:tasks.workunit.client.1.smithi169.stdout:9/694: mknod d7/d2d/d48/cf4 0 2024-03-04T13:42:59.213 INFO:tasks.workunit.client.1.smithi169.stdout:8/646: mkdir d7/dd4 0 2024-03-04T13:42:59.213 INFO:tasks.workunit.client.0.smithi107.stdout:0/192: stat da/dc/dd/d3e/f11 0 2024-03-04T13:42:59.214 INFO:tasks.workunit.client.1.smithi169.stdout:7/723: dread d2/d6b/fd7 [0,4194304] 0 2024-03-04T13:42:59.216 INFO:tasks.workunit.client.1.smithi169.stdout:2/692: dread - d1/d5/d19/d3c/db8/de1/fe6 zero size 2024-03-04T13:42:59.219 INFO:tasks.workunit.client.1.smithi169.stdout:1/609: rename d4/l2c to d4/d5/d36/d35/d16/lc8 0 2024-03-04T13:42:59.222 INFO:tasks.workunit.client.0.smithi107.stdout:7/105: truncate d6/fe 3956474 0 2024-03-04T13:42:59.222 INFO:tasks.workunit.client.0.smithi107.stdout:8/190: dwrite d6/d16/f27 [4194304,4194304] 0 2024-03-04T13:42:59.223 INFO:tasks.workunit.client.1.smithi169.stdout:1/610: dread - d4/d5/d2d/d4e/da5/fc5 zero size 2024-03-04T13:42:59.226 INFO:tasks.workunit.client.1.smithi169.stdout:3/671: creat d7/de/d2b/d7d/d85/d13/d5a/d65/da4/fe7 x:0 0 0 2024-03-04T13:42:59.228 INFO:tasks.workunit.client.1.smithi169.stdout:2/693: dread d1/f40 [0,4194304] 0 2024-03-04T13:42:59.228 INFO:tasks.workunit.client.0.smithi107.stdout:5/106: creat d5/d8/f1f x:0 0 0 2024-03-04T13:42:59.233 INFO:tasks.workunit.client.1.smithi169.stdout:2/694: chown d1/d5/d19/d3c/db8/de1 682665585 1 2024-03-04T13:42:59.234 INFO:tasks.workunit.client.0.smithi107.stdout:0/193: dwrite da/f3c [0,4194304] 0 2024-03-04T13:42:59.234 INFO:tasks.workunit.client.0.smithi107.stdout:1/97: symlink d1/d3/d17/l18 0 2024-03-04T13:42:59.236 INFO:tasks.workunit.client.1.smithi169.stdout:7/724: dwrite d2/de/d3e/d41/f8b [0,4194304] 0 2024-03-04T13:42:59.238 INFO:tasks.workunit.client.0.smithi107.stdout:2/156: dwrite f4 [4194304,4194304] 0 2024-03-04T13:42:59.243 INFO:tasks.workunit.client.1.smithi169.stdout:9/695: dread d7/dc/d12/d32/d42/d4f/f8d [0,4194304] 0 2024-03-04T13:42:59.243 INFO:tasks.workunit.client.1.smithi169.stdout:3/672: rmdir d7/de/d2b/d7d/d85 39 2024-03-04T13:42:59.246 INFO:tasks.workunit.client.1.smithi169.stdout:6/654: getdents d0/d3c/d95/dbd 0 2024-03-04T13:42:59.247 INFO:tasks.workunit.client.0.smithi107.stdout:9/119: getdents d4/da 0 2024-03-04T13:42:59.247 INFO:tasks.workunit.client.0.smithi107.stdout:8/191: symlink d6/da/l36 0 2024-03-04T13:42:59.248 INFO:tasks.workunit.client.0.smithi107.stdout:8/192: fdatasync f4 0 2024-03-04T13:42:59.250 INFO:tasks.workunit.client.0.smithi107.stdout:9/120: write d4/da/f15 [2226330,79003] 0 2024-03-04T13:42:59.250 INFO:tasks.workunit.client.0.smithi107.stdout:8/193: write f4 [1778381,114128] 0 2024-03-04T13:42:59.253 INFO:tasks.workunit.client.0.smithi107.stdout:9/121: chown d4/da/l10 37053398 1 2024-03-04T13:42:59.255 INFO:tasks.workunit.client.1.smithi169.stdout:1/611: dwrite d4/d5/f57 [0,4194304] 0 2024-03-04T13:42:59.255 INFO:tasks.workunit.client.1.smithi169.stdout:7/725: creat d2/de/d3e/d41/dec/ff4 x:0 0 0 2024-03-04T13:42:59.255 INFO:tasks.workunit.client.0.smithi107.stdout:7/106: rename d6/d13/d14/l23 to d6/d13/d16/d19/l24 0 2024-03-04T13:42:59.260 INFO:tasks.workunit.client.1.smithi169.stdout:7/726: readlink d2/d15/d17/l19 0 2024-03-04T13:42:59.261 INFO:tasks.workunit.client.1.smithi169.stdout:9/696: chown d7/l30 67318839 1 2024-03-04T13:42:59.261 INFO:tasks.workunit.client.1.smithi169.stdout:2/695: dwrite d1/d5/d19/d3c/db8/de1/fa3 [0,4194304] 0 2024-03-04T13:42:59.262 INFO:tasks.workunit.client.1.smithi169.stdout:4/599: dwrite d4/d1c/fb2 [0,4194304] 0 2024-03-04T13:42:59.263 INFO:tasks.workunit.client.1.smithi169.stdout:1/612: chown d4/d5/d36/d35/d16/d3a/l44 24322563 1 2024-03-04T13:42:59.264 INFO:tasks.workunit.client.0.smithi107.stdout:1/98: mkdir d1/d19 0 2024-03-04T13:42:59.265 INFO:tasks.workunit.client.1.smithi169.stdout:9/697: chown d7/dc/d12/d20/d4e/fc8 172953 1 2024-03-04T13:42:59.270 INFO:tasks.workunit.client.1.smithi169.stdout:4/600: chown d4/d11/d2e/d21 424 1 2024-03-04T13:42:59.271 INFO:tasks.workunit.client.1.smithi169.stdout:5/757: rename d1/f75 to d1/d16/d17/d1b/f106 0 2024-03-04T13:42:59.272 INFO:tasks.workunit.client.0.smithi107.stdout:2/157: link f7 da/d34/f36 0 2024-03-04T13:42:59.272 INFO:tasks.workunit.client.0.smithi107.stdout:9/122: dwrite d4/f1c [0,4194304] 0 2024-03-04T13:42:59.274 INFO:tasks.workunit.client.1.smithi169.stdout:2/696: stat d1/d5/d19/d1e/f5c 0 2024-03-04T13:42:59.275 INFO:tasks.workunit.client.0.smithi107.stdout:8/194: symlink d6/d16/d1f/l37 0 2024-03-04T13:42:59.280 INFO:tasks.workunit.client.1.smithi169.stdout:6/655: creat d0/d32/d3e/db9/dca/ff1 x:0 0 0 2024-03-04T13:42:59.286 INFO:tasks.workunit.client.0.smithi107.stdout:5/107: creat d5/f20 x:0 0 0 2024-03-04T13:42:59.288 INFO:tasks.workunit.client.0.smithi107.stdout:5/108: read d5/d8/fb [28832,43934] 0 2024-03-04T13:42:59.289 INFO:tasks.workunit.client.0.smithi107.stdout:1/99: unlink d1/d3/c13 0 2024-03-04T13:42:59.289 INFO:tasks.workunit.client.0.smithi107.stdout:7/107: dread d6/d13/f18 [0,4194304] 0 2024-03-04T13:42:59.290 INFO:tasks.workunit.client.0.smithi107.stdout:1/100: write d1/d3/f11 [3557960,3085] 0 2024-03-04T13:42:59.291 INFO:tasks.workunit.client.0.smithi107.stdout:7/108: write d6/fa [4145245,78515] 0 2024-03-04T13:42:59.294 INFO:tasks.workunit.client.0.smithi107.stdout:2/158: rename da/f2b to da/df/f37 0 2024-03-04T13:42:59.296 INFO:tasks.workunit.client.1.smithi169.stdout:1/613: dwrite d4/d5/d36/f88 [0,4194304] 0 2024-03-04T13:42:59.296 INFO:tasks.workunit.client.0.smithi107.stdout:2/159: write f0 [82288,49584] 0 2024-03-04T13:42:59.298 INFO:tasks.workunit.client.0.smithi107.stdout:2/160: truncate da/f26 348209 0 2024-03-04T13:42:59.301 INFO:tasks.workunit.client.1.smithi169.stdout:3/673: dwrite f5 [0,4194304] 0 2024-03-04T13:42:59.303 INFO:tasks.workunit.client.0.smithi107.stdout:1/101: dread d1/d3/f11 [0,4194304] 0 2024-03-04T13:42:59.306 INFO:tasks.workunit.client.0.smithi107.stdout:8/195: mknod d6/da/d25/c38 0 2024-03-04T13:42:59.306 INFO:tasks.workunit.client.0.smithi107.stdout:8/196: dread - d6/da/f35 zero size 2024-03-04T13:42:59.308 INFO:tasks.workunit.client.1.smithi169.stdout:6/656: creat d0/d3c/d40/d52/dac/ff2 x:0 0 0 2024-03-04T13:42:59.309 INFO:tasks.workunit.client.1.smithi169.stdout:5/758: mkdir d1/d16/d107 0 2024-03-04T13:42:59.317 INFO:tasks.workunit.client.1.smithi169.stdout:4/601: rename d4/f9 to d4/d11/d2e/d39/d61/fc1 0 2024-03-04T13:42:59.321 INFO:tasks.workunit.client.0.smithi107.stdout:4/101: sync 2024-03-04T13:42:59.321 INFO:tasks.workunit.client.0.smithi107.stdout:7/109: fdatasync f4 0 2024-03-04T13:42:59.322 INFO:tasks.workunit.client.1.smithi169.stdout:5/759: mkdir d1/d16/d2f/d108 0 2024-03-04T13:42:59.324 INFO:tasks.workunit.client.0.smithi107.stdout:7/110: chown d6/d13/d16/d19/l20 26271925 1 2024-03-04T13:42:59.324 INFO:tasks.workunit.client.1.smithi169.stdout:3/674: chown d7/de/d2b/d7d/d85/f4f 245 1 2024-03-04T13:42:59.326 INFO:tasks.workunit.client.0.smithi107.stdout:7/111: write f5 [21475,1789] 0 2024-03-04T13:42:59.327 INFO:tasks.workunit.client.1.smithi169.stdout:9/698: dwrite d7/d27/f2f [0,4194304] 0 2024-03-04T13:42:59.327 INFO:tasks.workunit.client.0.smithi107.stdout:2/161: unlink da/df/l22 0 2024-03-04T13:42:59.329 INFO:tasks.workunit.client.1.smithi169.stdout:6/657: stat d0/d3c/d40/d52/dac/lb8 0 2024-03-04T13:42:59.332 INFO:tasks.workunit.client.1.smithi169.stdout:1/614: rename d4/d5/d2d/d4e/d71/d26/f60 to d4/d5/d36/d75/dba/fc9 0 2024-03-04T13:42:59.339 INFO:tasks.workunit.client.0.smithi107.stdout:1/102: mknod d1/d3/c1a 0 2024-03-04T13:42:59.339 INFO:tasks.workunit.client.0.smithi107.stdout:8/197: creat d6/f39 x:0 0 0 2024-03-04T13:42:59.340 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:42:59 smithi107 ceph-mon[28739]: pgmap v51: 65 pgs: 65 active+clean; 1.5 GiB data, 6.0 GiB used, 530 GiB / 536 GiB avail; 47 MiB/s rd, 178 MiB/s wr, 374 op/s 2024-03-04T13:42:59.340 INFO:tasks.workunit.client.1.smithi169.stdout:1/615: truncate d4/d5/d36/d35/d16/d3a/f9c 1191554 0 2024-03-04T13:42:59.340 INFO:tasks.workunit.client.1.smithi169.stdout:2/697: write d1/f3d [3828452,15815] 0 2024-03-04T13:42:59.344 INFO:tasks.workunit.client.1.smithi169.stdout:6/658: dread - d0/d3c/d40/f7d zero size 2024-03-04T13:42:59.349 INFO:tasks.workunit.client.1.smithi169.stdout:6/659: write d0/d32/d3a/d43/d74/d93/dbe/fec [451516,120444] 0 2024-03-04T13:42:59.354 INFO:tasks.workunit.client.0.smithi107.stdout:1/103: rename d1/d19 to d1/d10/d16/d1b 0 2024-03-04T13:42:59.356 INFO:tasks.workunit.client.0.smithi107.stdout:9/123: truncate d4/f1c 1627993 0 2024-03-04T13:42:59.356 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:42:59 smithi169 ceph-mon[36313]: pgmap v51: 65 pgs: 65 active+clean; 1.5 GiB data, 6.0 GiB used, 530 GiB / 536 GiB avail; 47 MiB/s rd, 178 MiB/s wr, 374 op/s 2024-03-04T13:42:59.358 INFO:tasks.workunit.client.1.smithi169.stdout:4/602: write d4/d1c/d75/d81/f85 [4180305,63506] 0 2024-03-04T13:42:59.360 INFO:tasks.workunit.client.0.smithi107.stdout:5/109: creat d5/f21 x:0 0 0 2024-03-04T13:42:59.362 INFO:tasks.workunit.client.1.smithi169.stdout:5/760: creat d1/d16/d17/f109 x:0 0 0 2024-03-04T13:42:59.364 INFO:tasks.workunit.client.1.smithi169.stdout:3/675: write d7/fb [4797624,110840] 0 2024-03-04T13:42:59.364 INFO:tasks.workunit.client.0.smithi107.stdout:7/112: symlink d6/d17/l25 0 2024-03-04T13:42:59.368 INFO:tasks.workunit.client.1.smithi169.stdout:6/660: rename d0/d32/d3a/c1e to d0/d32/d3e/db9/cf3 0 2024-03-04T13:42:59.370 INFO:tasks.workunit.client.1.smithi169.stdout:9/699: creat d7/d2d/d48/ff5 x:0 0 0 2024-03-04T13:42:59.370 INFO:tasks.workunit.client.1.smithi169.stdout:2/698: dwrite d1/d5/d19/f8c [0,4194304] 0 2024-03-04T13:42:59.371 INFO:tasks.workunit.client.1.smithi169.stdout:6/661: write d0/d32/fe8 [864127,2571] 0 2024-03-04T13:42:59.375 INFO:tasks.workunit.client.0.smithi107.stdout:4/102: dwrite d0/d9/d10/f1c [0,4194304] 0 2024-03-04T13:42:59.377 INFO:tasks.workunit.client.1.smithi169.stdout:1/616: write d4/d5/d2d/d4e/d71/d82/fad [1737767,81612] 0 2024-03-04T13:42:59.380 INFO:tasks.workunit.client.1.smithi169.stdout:3/676: symlink d7/de/d2b/d7d/d85/d13/d1a/d2d/d56/d5e/d67/d5c/d84/le8 0 2024-03-04T13:42:59.381 INFO:tasks.workunit.client.0.smithi107.stdout:5/110: mkdir d5/dc/d22 0 2024-03-04T13:42:59.384 INFO:tasks.workunit.client.0.smithi107.stdout:8/198: write d6/da/f1b [3128926,36562] 0 2024-03-04T13:42:59.395 INFO:tasks.workunit.client.1.smithi169.stdout:3/677: dwrite d7/f25 [0,4194304] 0 2024-03-04T13:42:59.401 INFO:tasks.workunit.client.1.smithi169.stdout:3/678: truncate d7/de/d2b/d7d/d85/d13/d1a/d2d/fb6 136980 0 2024-03-04T13:42:59.403 INFO:tasks.workunit.client.1.smithi169.stdout:6/662: unlink d0/d3c/d95/f96 0 2024-03-04T13:42:59.408 INFO:tasks.workunit.client.1.smithi169.stdout:5/761: rename d1/d81/ced to d1/d16/d2f/d72/c10a 0 2024-03-04T13:42:59.409 INFO:tasks.workunit.client.1.smithi169.stdout:1/617: rmdir d4/d5/d36/d35/d16/d81 39 2024-03-04T13:42:59.411 INFO:tasks.workunit.client.1.smithi169.stdout:4/603: truncate d4/f15 1976705 0 2024-03-04T13:42:59.416 INFO:tasks.workunit.client.1.smithi169.stdout:9/700: symlink d7/lf6 0 2024-03-04T13:42:59.417 INFO:tasks.workunit.client.1.smithi169.stdout:3/679: symlink d7/de/d2b/d7d/d85/d13/d1a/d78/dc1/le9 0 2024-03-04T13:42:59.419 INFO:tasks.workunit.client.1.smithi169.stdout:6/663: mknod d0/d3c/d40/d52/d98/cf4 0 2024-03-04T13:42:59.420 INFO:tasks.workunit.client.1.smithi169.stdout:9/701: truncate d7/dc/d12/d20/da6/fb4 5208141 0 2024-03-04T13:42:59.422 INFO:tasks.workunit.client.0.smithi107.stdout:7/113: dwrite f4 [0,4194304] 0 2024-03-04T13:42:59.422 INFO:tasks.workunit.client.1.smithi169.stdout:9/702: fsync d7/d2d/d48/f7f 0 2024-03-04T13:42:59.423 INFO:tasks.workunit.client.0.smithi107.stdout:9/124: write d4/da/d11/f13 [2971539,34118] 0 2024-03-04T13:42:59.423 INFO:tasks.workunit.client.1.smithi169.stdout:2/699: dread d1/d5/d19/d1e/d29/f3e [0,4194304] 0 2024-03-04T13:42:59.424 INFO:tasks.workunit.client.1.smithi169.stdout:1/618: readlink d4/d5/d2d/d4e/d71/d82/laa 0 2024-03-04T13:42:59.425 INFO:tasks.workunit.client.1.smithi169.stdout:3/680: dread - d7/de/d2b/d7d/d85/f7f zero size 2024-03-04T13:42:59.426 INFO:tasks.workunit.client.1.smithi169.stdout:1/619: stat d4/d5/d36/d35/f56 0 2024-03-04T13:42:59.429 INFO:tasks.workunit.client.0.smithi107.stdout:1/104: getdents d1/d3 0 2024-03-04T13:42:59.429 INFO:tasks.workunit.client.1.smithi169.stdout:2/700: rmdir d1/d5/d19/d3c 39 2024-03-04T13:42:59.431 INFO:tasks.workunit.client.0.smithi107.stdout:3/109: sync 2024-03-04T13:42:59.431 INFO:tasks.workunit.client.0.smithi107.stdout:9/125: unlink f0 0 2024-03-04T13:42:59.432 INFO:tasks.workunit.client.1.smithi169.stdout:5/762: getdents d1/d16/d17/d1b/d74/ddc 0 2024-03-04T13:42:59.433 INFO:tasks.workunit.client.0.smithi107.stdout:8/199: dread d6/d16/f27 [4194304,4194304] 0 2024-03-04T13:42:59.433 INFO:tasks.workunit.client.0.smithi107.stdout:3/110: read d1/de/f19 [7296886,63626] 0 2024-03-04T13:42:59.435 INFO:tasks.workunit.client.0.smithi107.stdout:9/126: fdatasync d4/da/d11/f17 0 2024-03-04T13:42:59.436 INFO:tasks.workunit.client.0.smithi107.stdout:3/111: write d1/fa [893438,130863] 0 2024-03-04T13:42:59.437 INFO:tasks.workunit.client.1.smithi169.stdout:0/527: sync 2024-03-04T13:42:59.440 INFO:tasks.workunit.client.0.smithi107.stdout:7/114: creat d6/d13/d16/d19/f26 x:0 0 0 2024-03-04T13:42:59.441 INFO:tasks.workunit.client.1.smithi169.stdout:9/703: symlink d7/d25/lf7 0 2024-03-04T13:42:59.442 INFO:tasks.workunit.client.1.smithi169.stdout:9/704: fsync d7/d49/fe9 0 2024-03-04T13:42:59.443 INFO:tasks.workunit.client.0.smithi107.stdout:5/111: link d5/dc/c1e d5/d8/d14/c23 0 2024-03-04T13:42:59.445 INFO:tasks.workunit.client.1.smithi169.stdout:2/701: mkdir d1/d5/d22/d45/df3 0 2024-03-04T13:42:59.449 INFO:tasks.workunit.client.1.smithi169.stdout:5/763: rename d1/d16/d17/d28/c2b to d1/d16/d2a/d46/d9e/c10b 0 2024-03-04T13:42:59.450 INFO:tasks.workunit.client.0.smithi107.stdout:4/103: getdents d0 0 2024-03-04T13:42:59.452 INFO:tasks.workunit.client.0.smithi107.stdout:3/112: dwrite d1/d1e/f22 [0,4194304] 0 2024-03-04T13:42:59.452 INFO:tasks.workunit.client.1.smithi169.stdout:3/681: mknod d7/de/d2b/d7d/d85/cea 0 2024-03-04T13:42:59.453 INFO:tasks.workunit.client.1.smithi169.stdout:5/764: chown d1/d16/d2f/fde 67705412 1 2024-03-04T13:42:59.455 INFO:tasks.workunit.client.0.smithi107.stdout:7/115: rmdir d6/d13 39 2024-03-04T13:42:59.456 INFO:tasks.workunit.client.1.smithi169.stdout:4/604: getdents d4/d11/d2e/d37/d5c/d86 0 2024-03-04T13:42:59.458 INFO:tasks.workunit.client.0.smithi107.stdout:7/116: write f0 [1388092,126516] 0 2024-03-04T13:42:59.459 INFO:tasks.workunit.client.1.smithi169.stdout:0/528: dwrite d5/d9/dc/d12/d15/d46/fb4 [0,4194304] 0 2024-03-04T13:42:59.460 INFO:tasks.workunit.client.0.smithi107.stdout:9/127: mkdir d4/da/d11/d20/d23 0 2024-03-04T13:42:59.463 INFO:tasks.workunit.client.0.smithi107.stdout:7/117: write d6/fc [80392,91359] 0 2024-03-04T13:42:59.467 INFO:tasks.workunit.client.0.smithi107.stdout:7/118: read f4 [1108118,102279] 0 2024-03-04T13:42:59.467 INFO:tasks.workunit.client.0.smithi107.stdout:9/128: write d4/da/d11/f13 [4744966,65276] 0 2024-03-04T13:42:59.468 INFO:tasks.workunit.client.0.smithi107.stdout:5/112: creat d5/f24 x:0 0 0 2024-03-04T13:42:59.471 INFO:tasks.workunit.client.1.smithi169.stdout:1/620: dwrite d4/d5/d36/d35/d8f/d93/f9b [0,4194304] 0 2024-03-04T13:42:59.472 INFO:tasks.workunit.client.1.smithi169.stdout:9/705: dread d7/d25/d3c/f9e [0,4194304] 0 2024-03-04T13:42:59.473 INFO:tasks.workunit.client.0.smithi107.stdout:5/113: truncate d5/d8/f17 834940 0 2024-03-04T13:42:59.473 INFO:tasks.workunit.client.0.smithi107.stdout:4/104: dwrite d0/d9/f1a [0,4194304] 0 2024-03-04T13:42:59.473 INFO:tasks.workunit.client.0.smithi107.stdout:5/114: chown d5 475 1 2024-03-04T13:42:59.476 INFO:tasks.workunit.client.1.smithi169.stdout:3/682: dread d7/de/d24/f3c [0,4194304] 0 2024-03-04T13:42:59.476 INFO:tasks.workunit.client.0.smithi107.stdout:3/113: fsync d1/de/f1d 0 2024-03-04T13:42:59.483 INFO:tasks.workunit.client.0.smithi107.stdout:1/105: getdents d1 0 2024-03-04T13:42:59.485 INFO:tasks.workunit.client.0.smithi107.stdout:1/106: write d1/fe [2988963,46325] 0 2024-03-04T13:42:59.490 INFO:tasks.workunit.client.1.smithi169.stdout:1/621: unlink d4/d5/d36/d35/fae 0 2024-03-04T13:42:59.491 INFO:tasks.workunit.client.1.smithi169.stdout:5/765: dwrite d1/d16/d17/d1b/f106 [0,4194304] 0 2024-03-04T13:42:59.495 INFO:tasks.workunit.client.0.smithi107.stdout:5/115: unlink d5/d8/da/l1d 0 2024-03-04T13:42:59.500 INFO:tasks.workunit.client.1.smithi169.stdout:3/683: rmdir d7/de/d2b/d7d/d85/d13/d1a/d2d/d56/d5e/d67/dd7 39 2024-03-04T13:42:59.503 INFO:tasks.workunit.client.1.smithi169.stdout:2/702: getdents d1/d5/d19/d3c/db8/de1/d64/dce/dda 0 2024-03-04T13:42:59.506 INFO:tasks.workunit.client.1.smithi169.stdout:2/703: readlink d1/d6d/l9d 0 2024-03-04T13:42:59.507 INFO:tasks.workunit.client.1.smithi169.stdout:9/706: dread d7/dc/df1/f87 [0,4194304] 0 2024-03-04T13:42:59.509 INFO:tasks.workunit.client.1.smithi169.stdout:2/704: dread - d1/d5/d19/d68/fe8 zero size 2024-03-04T13:42:59.509 INFO:tasks.workunit.client.0.smithi107.stdout:1/107: truncate d1/d3/f11 3499498 0 2024-03-04T13:42:59.514 INFO:tasks.workunit.client.1.smithi169.stdout:5/766: dwrite d1/f4a [0,4194304] 0 2024-03-04T13:42:59.516 INFO:tasks.workunit.client.1.smithi169.stdout:3/684: unlink d7/de/d2b/d7d/d85/d13/f15 0 2024-03-04T13:42:59.517 INFO:tasks.workunit.client.0.smithi107.stdout:5/116: rmdir d5/dc 39 2024-03-04T13:42:59.518 INFO:tasks.workunit.client.0.smithi107.stdout:5/117: read - d5/d8/da/f16 zero size 2024-03-04T13:42:59.519 INFO:tasks.workunit.client.1.smithi169.stdout:4/605: getdents d4/d11 0 2024-03-04T13:42:59.519 INFO:tasks.workunit.client.0.smithi107.stdout:4/105: write d0/d9/d10/f15 [2360878,95163] 0 2024-03-04T13:42:59.521 INFO:tasks.workunit.client.0.smithi107.stdout:3/114: getdents d1/d21 0 2024-03-04T13:42:59.521 INFO:tasks.workunit.client.1.smithi169.stdout:3/685: readlink d7/de/d2b/d7d/d85/d13/d1a/d2d/d56/d5e/d67/d5c/d84/le8 0 2024-03-04T13:42:59.521 INFO:tasks.workunit.client.0.smithi107.stdout:3/115: readlink d1/l2 0 2024-03-04T13:42:59.522 INFO:tasks.workunit.client.0.smithi107.stdout:9/129: creat d4/f24 x:0 0 0 2024-03-04T13:42:59.524 INFO:tasks.workunit.client.0.smithi107.stdout:9/130: chown d4/da/f1b 7026 1 2024-03-04T13:42:59.524 INFO:tasks.workunit.client.1.smithi169.stdout:9/707: mkdir d7/d49/df8 0 2024-03-04T13:42:59.525 INFO:tasks.workunit.client.0.smithi107.stdout:1/108: symlink d1/d10/d16/l1c 0 2024-03-04T13:42:59.528 INFO:tasks.workunit.client.1.smithi169.stdout:2/705: rename d1/f84 to d1/d5/d19/d3c/db8/de1/d64/ff4 0 2024-03-04T13:42:59.530 INFO:tasks.workunit.client.1.smithi169.stdout:0/529: getdents d5/d9/dc/d12/d15/d46 0 2024-03-04T13:42:59.532 INFO:tasks.workunit.client.1.smithi169.stdout:9/708: mkdir d7/dc/d12/d8a/df9 0 2024-03-04T13:42:59.534 INFO:tasks.workunit.client.0.smithi107.stdout:3/116: symlink d1/l25 0 2024-03-04T13:42:59.534 INFO:tasks.workunit.client.0.smithi107.stdout:3/117: chown d1/l8 1 1 2024-03-04T13:42:59.535 INFO:tasks.workunit.client.0.smithi107.stdout:5/118: dwrite d5/f24 [0,4194304] 0 2024-03-04T13:42:59.535 INFO:tasks.workunit.client.1.smithi169.stdout:4/606: dwrite d4/d11/d2e/d63/f8a [0,4194304] 0 2024-03-04T13:42:59.536 INFO:tasks.workunit.client.0.smithi107.stdout:9/131: rename d4/f1c to d4/da/f25 0 2024-03-04T13:42:59.538 INFO:tasks.workunit.client.0.smithi107.stdout:9/132: write d4/f24 [503131,94996] 0 2024-03-04T13:42:59.539 INFO:tasks.workunit.client.1.smithi169.stdout:5/767: unlink d1/d16/d17/d2d/lc3 0 2024-03-04T13:42:59.539 INFO:tasks.workunit.client.1.smithi169.stdout:4/607: chown d4/d11/d2e/d37/d5c/fb0 594 1 2024-03-04T13:42:59.539 INFO:tasks.workunit.client.0.smithi107.stdout:1/109: rmdir d1/d10 39 2024-03-04T13:42:59.543 INFO:tasks.workunit.client.1.smithi169.stdout:4/608: chown d4/d1c/d75/d81/f85 0 1 2024-03-04T13:42:59.544 INFO:tasks.workunit.client.1.smithi169.stdout:9/709: creat d7/dc/d12/d20/d4e/ffa x:0 0 0 2024-03-04T13:42:59.545 INFO:tasks.workunit.client.0.smithi107.stdout:4/106: mkdir d0/d9/d1f 0 2024-03-04T13:42:59.545 INFO:tasks.workunit.client.0.smithi107.stdout:3/118: mkdir d1/d1e/d26 0 2024-03-04T13:42:59.548 INFO:tasks.workunit.client.1.smithi169.stdout:5/768: rename d1/d16/d17/d2d/lb6 to d1/d16/d17/d2d/d5a/d67/l10c 0 2024-03-04T13:42:59.549 INFO:tasks.workunit.client.1.smithi169.stdout:4/609: dread d4/d11/d3b/d67/d7e/f82 [0,4194304] 0 2024-03-04T13:42:59.555 INFO:tasks.workunit.client.0.smithi107.stdout:7/119: dwrite d6/fe [0,4194304] 0 2024-03-04T13:42:59.557 INFO:tasks.workunit.client.0.smithi107.stdout:5/119: mknod d5/d8/d14/c25 0 2024-03-04T13:42:59.559 INFO:tasks.workunit.client.1.smithi169.stdout:4/610: dread d4/d11/f79 [0,4194304] 0 2024-03-04T13:42:59.561 INFO:tasks.workunit.client.1.smithi169.stdout:5/769: creat d1/d16/dbd/dbb/dcc/f10d x:0 0 0 2024-03-04T13:42:59.564 INFO:tasks.workunit.client.1.smithi169.stdout:4/611: write d4/d11/d2e/d39/fb9 [528915,130964] 0 2024-03-04T13:42:59.564 INFO:tasks.workunit.client.0.smithi107.stdout:3/119: dwrite d1/f23 [0,4194304] 0 2024-03-04T13:42:59.565 INFO:tasks.workunit.client.0.smithi107.stdout:3/120: readlink d1/l1b 0 2024-03-04T13:42:59.569 INFO:tasks.workunit.client.1.smithi169.stdout:9/710: symlink d7/dc/d3e/da3/lfb 0 2024-03-04T13:42:59.570 INFO:tasks.workunit.client.0.smithi107.stdout:3/121: write d1/f3 [50669,88723] 0 2024-03-04T13:42:59.573 INFO:tasks.workunit.client.1.smithi169.stdout:4/612: dread d4/d1c/f59 [0,4194304] 0 2024-03-04T13:42:59.575 INFO:tasks.workunit.client.1.smithi169.stdout:9/711: chown d7/dc/df1/c61 3026 1 2024-03-04T13:42:59.580 INFO:tasks.workunit.client.1.smithi169.stdout:2/706: dread d1/d5/d22/d45/d4a/f5d [0,4194304] 0 2024-03-04T13:42:59.582 INFO:tasks.workunit.client.0.smithi107.stdout:1/110: rmdir d1/d3 39 2024-03-04T13:42:59.586 INFO:tasks.workunit.client.1.smithi169.stdout:4/613: mknod d4/d1c/d58/cc2 0 2024-03-04T13:42:59.588 INFO:tasks.workunit.client.0.smithi107.stdout:4/107: link d0/d9/f11 d0/d9/d1e/f20 0 2024-03-04T13:42:59.588 INFO:tasks.workunit.client.0.smithi107.stdout:7/120: dread f5 [0,4194304] 0 2024-03-04T13:42:59.589 INFO:tasks.workunit.client.1.smithi169.stdout:0/530: dread d5/f37 [4194304,4194304] 0 2024-03-04T13:42:59.589 INFO:tasks.workunit.client.0.smithi107.stdout:9/133: truncate d4/f19 1936880 0 2024-03-04T13:42:59.590 INFO:tasks.workunit.client.1.smithi169.stdout:5/770: mkdir d1/d16/d2a/d46/d9e/d105/d10e 0 2024-03-04T13:42:59.590 INFO:tasks.workunit.client.0.smithi107.stdout:7/121: write d6/fc [4877905,88929] 0 2024-03-04T13:42:59.591 INFO:tasks.workunit.client.0.smithi107.stdout:7/122: fsync d6/f7 0 2024-03-04T13:42:59.593 INFO:tasks.workunit.client.0.smithi107.stdout:1/111: rmdir d1 39 2024-03-04T13:42:59.595 INFO:tasks.workunit.client.0.smithi107.stdout:4/108: dread d0/d9/d10/f15 [0,4194304] 0 2024-03-04T13:42:59.595 INFO:tasks.workunit.client.1.smithi169.stdout:4/614: rename d4/d11/d2e/d63/f8a to d4/d11/d2e/d63/fc3 0 2024-03-04T13:42:59.596 INFO:tasks.workunit.client.1.smithi169.stdout:5/771: creat d1/d16/d2f/f10f x:0 0 0 2024-03-04T13:42:59.596 INFO:tasks.workunit.client.1.smithi169.stdout:4/615: stat d4/d11/d2e/d39/d48/f65 0 2024-03-04T13:42:59.598 INFO:tasks.workunit.client.1.smithi169.stdout:9/712: mknod d7/dc/df1/cfc 0 2024-03-04T13:42:59.599 INFO:tasks.workunit.client.1.smithi169.stdout:2/707: mkdir d1/d5/d19/d3c/db8/de1/d64/dce/dd5/df5 0 2024-03-04T13:42:59.602 INFO:tasks.workunit.client.1.smithi169.stdout:0/531: unlink d5/d9/dc/d12/d15/d46/d98/c9d 0 2024-03-04T13:42:59.603 INFO:tasks.workunit.client.0.smithi107.stdout:6/161: sync 2024-03-04T13:42:59.604 INFO:tasks.workunit.client.0.smithi107.stdout:4/109: mkdir d0/d9/de/d14/dc/d21 0 2024-03-04T13:42:59.606 INFO:tasks.workunit.client.0.smithi107.stdout:4/110: readlink d0/d9/d10/l12 0 2024-03-04T13:42:59.606 INFO:tasks.workunit.client.1.smithi169.stdout:9/713: unlink d7/dc/d12/d20/da6/fcb 0 2024-03-04T13:42:59.607 INFO:tasks.workunit.client.0.smithi107.stdout:7/123: mknod d6/d13/d16/d19/d1e/c27 0 2024-03-04T13:42:59.609 INFO:tasks.workunit.client.0.smithi107.stdout:4/111: truncate d0/d9/de/d14/f1d 139891 0 2024-03-04T13:42:59.610 INFO:tasks.workunit.client.1.smithi169.stdout:7/727: sync 2024-03-04T13:42:59.610 INFO:tasks.workunit.client.1.smithi169.stdout:8/647: sync 2024-03-04T13:42:59.611 INFO:tasks.workunit.client.0.smithi107.stdout:1/112: chown d1/d10/d16/l1c 2076961 1 2024-03-04T13:42:59.612 INFO:tasks.workunit.client.1.smithi169.stdout:0/532: mknod d5/d4c/d65/cbc 0 2024-03-04T13:42:59.613 INFO:tasks.workunit.client.1.smithi169.stdout:7/728: write d2/de/d3e/d41/d89/fc4 [273692,103017] 0 2024-03-04T13:42:59.613 INFO:tasks.workunit.client.0.smithi107.stdout:6/162: unlink df/f20 0 2024-03-04T13:42:59.615 INFO:tasks.workunit.client.1.smithi169.stdout:5/772: creat d1/d16/d17/d28/dfe/f110 x:0 0 0 2024-03-04T13:42:59.622 INFO:tasks.workunit.client.1.smithi169.stdout:9/714: symlink d7/dc/d3e/lfd 0 2024-03-04T13:42:59.623 INFO:tasks.workunit.client.1.smithi169.stdout:0/533: dread d5/f37 [4194304,4194304] 0 2024-03-04T13:42:59.625 INFO:tasks.workunit.client.0.smithi107.stdout:6/163: mkdir df/d2e/d36 0 2024-03-04T13:42:59.626 INFO:tasks.workunit.client.1.smithi169.stdout:8/648: creat d7/d10/d17/d44/fd5 x:0 0 0 2024-03-04T13:42:59.628 INFO:tasks.workunit.client.0.smithi107.stdout:7/124: mkdir d6/d1d/d28 0 2024-03-04T13:42:59.631 INFO:tasks.workunit.client.1.smithi169.stdout:7/729: write d2/de/d37/fd4 [142222,37581] 0 2024-03-04T13:42:59.631 INFO:tasks.workunit.client.0.smithi107.stdout:7/125: truncate d6/d13/f18 1520356 0 2024-03-04T13:42:59.633 INFO:tasks.workunit.client.1.smithi169.stdout:2/708: link d1/d5/d19/d3c/l56 d1/d5/d19/d1e/d29/lf6 0 2024-03-04T13:42:59.634 INFO:tasks.workunit.client.0.smithi107.stdout:9/134: dwrite d4/da/f25 [0,4194304] 0 2024-03-04T13:42:59.635 INFO:tasks.workunit.client.1.smithi169.stdout:9/715: symlink d7/d2d/d48/d66/lfe 0 2024-03-04T13:42:59.640 INFO:tasks.workunit.client.0.smithi107.stdout:7/126: creat d6/d1d/f29 x:0 0 0 2024-03-04T13:42:59.640 INFO:tasks.workunit.client.1.smithi169.stdout:8/649: chown d7/d10/d22/d46/f83 8580 1 2024-03-04T13:42:59.641 INFO:tasks.workunit.client.1.smithi169.stdout:0/534: dwrite d5/d9/f2d [0,4194304] 0 2024-03-04T13:42:59.642 INFO:tasks.workunit.client.1.smithi169.stdout:2/709: chown d1/d5/d22/d45/d4a/de7/dcb/ldb 206852693 1 2024-03-04T13:42:59.648 INFO:tasks.workunit.client.1.smithi169.stdout:5/773: symlink d1/d16/d17/l111 0 2024-03-04T13:42:59.650 INFO:tasks.workunit.client.0.smithi107.stdout:4/112: write d0/d9/d1e/f20 [1073525,106014] 0 2024-03-04T13:42:59.652 INFO:tasks.workunit.client.1.smithi169.stdout:4/616: dwrite d4/d11/d2e/d37/f57 [0,4194304] 0 2024-03-04T13:42:59.655 INFO:tasks.workunit.client.1.smithi169.stdout:0/535: chown d5/d9/dc/d12/d15/d19/d61/d71/dbb 1101701 1 2024-03-04T13:42:59.658 INFO:tasks.workunit.client.1.smithi169.stdout:8/650: unlink d7/d10/d24/f71 0 2024-03-04T13:42:59.658 INFO:tasks.workunit.client.0.smithi107.stdout:9/135: creat d4/f26 x:0 0 0 2024-03-04T13:42:59.662 INFO:tasks.workunit.client.1.smithi169.stdout:9/716: mkdir d7/dc/d12/d8a/df9/dff 0 2024-03-04T13:42:59.662 INFO:tasks.workunit.client.0.smithi107.stdout:6/164: dread f6 [0,4194304] 0 2024-03-04T13:42:59.664 INFO:tasks.workunit.client.0.smithi107.stdout:4/113: mknod d0/d9/d18/c22 0 2024-03-04T13:42:59.666 INFO:tasks.workunit.client.1.smithi169.stdout:4/617: mknod d4/d11/d2e/d6c/cc4 0 2024-03-04T13:42:59.667 INFO:tasks.workunit.client.1.smithi169.stdout:9/717: chown d7/dc/d12/d32/d42/f46 24529 1 2024-03-04T13:42:59.669 INFO:tasks.workunit.client.1.smithi169.stdout:7/730: getdents d2/d28/d70 0 2024-03-04T13:42:59.671 INFO:tasks.workunit.client.0.smithi107.stdout:4/114: symlink d0/d9/de/d14/l23 0 2024-03-04T13:42:59.674 INFO:tasks.workunit.client.0.smithi107.stdout:7/127: dwrite d6/d13/f18 [0,4194304] 0 2024-03-04T13:42:59.675 INFO:tasks.workunit.client.0.smithi107.stdout:1/113: dwrite d1/d3/f11 [4194304,4194304] 0 2024-03-04T13:42:59.676 INFO:tasks.workunit.client.1.smithi169.stdout:7/731: write d2/de/d3e/d7e/fe2 [1900588,71580] 0 2024-03-04T13:42:59.678 INFO:tasks.workunit.client.1.smithi169.stdout:2/710: dwrite d1/d5/d22/d45/d4a/f92 [0,4194304] 0 2024-03-04T13:42:59.679 INFO:tasks.workunit.client.1.smithi169.stdout:4/618: creat d4/d11/d3b/d67/dbb/d6e/fc5 x:0 0 0 2024-03-04T13:42:59.679 INFO:tasks.workunit.client.1.smithi169.stdout:5/774: dwrite d1/d16/dbd/dbb/ff7 [0,4194304] 0 2024-03-04T13:42:59.683 INFO:tasks.workunit.client.0.smithi107.stdout:9/136: dread d4/f8 [4194304,4194304] 0 2024-03-04T13:42:59.683 INFO:tasks.workunit.client.1.smithi169.stdout:4/619: chown d4/d11/d2e/d21/c89 392170705 1 2024-03-04T13:42:59.684 INFO:tasks.workunit.client.0.smithi107.stdout:6/165: dread df/f15 [0,4194304] 0 2024-03-04T13:42:59.689 INFO:tasks.workunit.client.1.smithi169.stdout:0/536: getdents d5/d9/dc/d12/d15/d19/d61 0 2024-03-04T13:42:59.692 INFO:tasks.workunit.client.1.smithi169.stdout:2/711: chown d1/la8 279120 1 2024-03-04T13:42:59.695 INFO:tasks.workunit.client.1.smithi169.stdout:4/620: dread d4/d11/d2e/d63/fc3 [4194304,4194304] 0 2024-03-04T13:42:59.698 INFO:tasks.workunit.client.1.smithi169.stdout:8/651: write d7/d10/d24/d3e/f57 [3796963,21407] 0 2024-03-04T13:42:59.702 INFO:tasks.workunit.client.1.smithi169.stdout:8/652: write d7/d10/d17/d44/d36/d62/fab [5240,128811] 0 2024-03-04T13:42:59.704 INFO:tasks.workunit.client.0.smithi107.stdout:1/114: mknod d1/d3/d17/c1d 0 2024-03-04T13:42:59.705 INFO:tasks.workunit.client.1.smithi169.stdout:0/537: mknod d5/cbd 0 2024-03-04T13:42:59.707 INFO:tasks.workunit.client.1.smithi169.stdout:9/718: creat d7/d2d/d48/d66/dec/f100 x:0 0 0 2024-03-04T13:42:59.708 INFO:tasks.workunit.client.1.smithi169.stdout:0/538: chown d5/d9/dc/d12/da5 1484296445 1 2024-03-04T13:42:59.708 INFO:tasks.workunit.client.0.smithi107.stdout:9/137: read d4/f19 [902884,9084] 0 2024-03-04T13:42:59.712 INFO:tasks.workunit.client.0.smithi107.stdout:7/128: symlink d6/d13/d16/d19/d1e/d22/l2a 0 2024-03-04T13:42:59.713 INFO:tasks.workunit.client.0.smithi107.stdout:1/115: creat d1/d10/d16/d1b/f1e x:0 0 0 2024-03-04T13:42:59.715 INFO:tasks.workunit.client.0.smithi107.stdout:1/116: fdatasync d1/d10/d16/d1b/f1e 0 2024-03-04T13:42:59.715 INFO:tasks.workunit.client.1.smithi169.stdout:8/653: mknod d7/d10/d24/d3e/cd6 0 2024-03-04T13:42:59.719 INFO:tasks.workunit.client.1.smithi169.stdout:0/539: creat d5/d9/dc/d12/d15/d19/d61/d71/fbe x:0 0 0 2024-03-04T13:42:59.720 INFO:tasks.workunit.client.0.smithi107.stdout:1/117: mknod d1/c1f 0 2024-03-04T13:42:59.723 INFO:tasks.workunit.client.1.smithi169.stdout:2/712: dwrite d1/d5/d19/d68/fe8 [0,4194304] 0 2024-03-04T13:42:59.723 INFO:tasks.workunit.client.1.smithi169.stdout:7/732: dwrite d2/de/d3e/d41/d5b/f9f [0,4194304] 0 2024-03-04T13:42:59.725 INFO:tasks.workunit.client.1.smithi169.stdout:6/664: sync 2024-03-04T13:42:59.726 INFO:tasks.workunit.client.0.smithi107.stdout:1/118: mkdir d1/d10/d20 0 2024-03-04T13:42:59.727 INFO:tasks.workunit.client.1.smithi169.stdout:8/654: write d7/d10/d22/d75/d81/da2/fbf [22151,14826] 0 2024-03-04T13:42:59.730 INFO:tasks.workunit.client.0.smithi107.stdout:7/129: dread f4 [0,4194304] 0 2024-03-04T13:42:59.732 INFO:tasks.workunit.client.0.smithi107.stdout:1/119: symlink d1/d10/d16/l21 0 2024-03-04T13:42:59.733 INFO:tasks.workunit.client.1.smithi169.stdout:6/665: fsync d0/d3c/d95/dbd/fcd 0 2024-03-04T13:42:59.734 INFO:tasks.workunit.client.1.smithi169.stdout:5/775: dwrite d1/d16/d17/fe9 [0,4194304] 0 2024-03-04T13:42:59.736 INFO:tasks.workunit.client.1.smithi169.stdout:7/733: chown d2/de/d3e/d41/d89/la2 953 1 2024-03-04T13:42:59.737 INFO:tasks.workunit.client.1.smithi169.stdout:9/719: write d7/d25/fa8 [99000,67442] 0 2024-03-04T13:42:59.738 INFO:tasks.workunit.client.0.smithi107.stdout:1/120: mknod d1/d10/c22 0 2024-03-04T13:42:59.739 INFO:tasks.workunit.client.0.smithi107.stdout:9/138: dwrite d4/da/d11/f13 [0,4194304] 0 2024-03-04T13:42:59.741 INFO:tasks.workunit.client.1.smithi169.stdout:4/621: dwrite d4/d11/d2e/d63/fc3 [0,4194304] 0 2024-03-04T13:42:59.744 INFO:tasks.workunit.client.0.smithi107.stdout:9/139: chown d4/da/d11/f13 23515 1 2024-03-04T13:42:59.744 INFO:tasks.workunit.client.1.smithi169.stdout:8/655: dread d7/d10/d24/d3e/f40 [0,4194304] 0 2024-03-04T13:42:59.744 INFO:tasks.workunit.client.1.smithi169.stdout:6/666: readlink d0/d32/d3a/d43/la2 0 2024-03-04T13:42:59.748 INFO:tasks.workunit.client.1.smithi169.stdout:5/776: creat d1/d16/dbd/f112 x:0 0 0 2024-03-04T13:42:59.749 INFO:tasks.workunit.client.0.smithi107.stdout:6/166: dwrite df/d17/d24/f2d [0,4194304] 0 2024-03-04T13:42:59.750 INFO:tasks.workunit.client.1.smithi169.stdout:4/622: write d4/d1c/d75/fb3 [630218,13413] 0 2024-03-04T13:42:59.752 INFO:tasks.workunit.client.1.smithi169.stdout:5/777: dread - d1/fb0 zero size 2024-03-04T13:42:59.752 INFO:tasks.workunit.client.1.smithi169.stdout:6/667: write d0/d32/d3a/f4f [16106,58552] 0 2024-03-04T13:42:59.754 INFO:tasks.workunit.client.1.smithi169.stdout:9/720: mkdir d7/dc/d3e/da3/d101 0 2024-03-04T13:42:59.754 INFO:tasks.workunit.client.1.smithi169.stdout:4/623: fsync d4/d11/d2e/d37/f57 0 2024-03-04T13:42:59.758 INFO:tasks.workunit.client.0.smithi107.stdout:6/167: creat df/d1b/f37 x:0 0 0 2024-03-04T13:42:59.758 INFO:tasks.workunit.client.1.smithi169.stdout:6/668: rmdir d0 39 2024-03-04T13:42:59.761 INFO:tasks.workunit.client.0.smithi107.stdout:1/121: dread d1/d3/f11 [4194304,4194304] 0 2024-03-04T13:42:59.762 INFO:tasks.workunit.client.0.smithi107.stdout:9/140: getdents d4/da 0 2024-03-04T13:42:59.766 INFO:tasks.workunit.client.0.smithi107.stdout:6/168: mkdir df/d38 0 2024-03-04T13:42:59.768 INFO:tasks.workunit.client.0.smithi107.stdout:9/141: stat d4/f26 0 2024-03-04T13:42:59.771 INFO:tasks.workunit.client.1.smithi169.stdout:4/624: rename d4/d11/d2e/d39/d48/cae to d4/d11/d2e/d6c/d78/da5/d77/dc0/cc6 0 2024-03-04T13:42:59.775 INFO:tasks.workunit.client.1.smithi169.stdout:8/656: dread d7/d10/d24/f85 [0,4194304] 0 2024-03-04T13:42:59.776 INFO:tasks.workunit.client.1.smithi169.stdout:6/669: symlink d0/d32/d3a/d43/d74/da6/lf5 0 2024-03-04T13:42:59.779 INFO:tasks.workunit.client.0.smithi107.stdout:1/122: truncate d1/fe 4697141 0 2024-03-04T13:42:59.779 INFO:tasks.workunit.client.0.smithi107.stdout:6/169: creat df/d1b/d2c/d31/f39 x:0 0 0 2024-03-04T13:42:59.780 INFO:tasks.workunit.client.1.smithi169.stdout:5/778: dwrite d1/d16/d2f/f33 [0,4194304] 0 2024-03-04T13:42:59.780 INFO:tasks.workunit.client.1.smithi169.stdout:9/721: dread d7/d2d/f40 [8388608,4194304] 0 2024-03-04T13:42:59.781 INFO:tasks.workunit.client.0.smithi107.stdout:1/123: chown d1/d10 2 1 2024-03-04T13:42:59.781 INFO:tasks.workunit.client.0.smithi107.stdout:7/130: dwrite d6/d13/d16/d1c/f1f [0,4194304] 0 2024-03-04T13:42:59.786 INFO:tasks.workunit.client.0.smithi107.stdout:1/124: dread - d1/d10/d16/d1b/f1e zero size 2024-03-04T13:42:59.788 INFO:tasks.workunit.client.1.smithi169.stdout:8/657: truncate d7/d10/d17/f74 855518 0 2024-03-04T13:42:59.788 INFO:tasks.workunit.client.0.smithi107.stdout:7/131: rename d6/d13/d16 to d6/d13/d16/d2b 22 2024-03-04T13:42:59.789 INFO:tasks.workunit.client.0.smithi107.stdout:6/170: symlink df/d1b/d2c/l3a 0 2024-03-04T13:42:59.793 INFO:tasks.workunit.client.0.smithi107.stdout:1/125: creat d1/d10/d16/d1b/f23 x:0 0 0 2024-03-04T13:42:59.794 INFO:tasks.workunit.client.1.smithi169.stdout:4/625: link d4/d11/d2e/d6c/d78/da5/l5f d4/d11/d2e/d6c/d78/da5/d77/dc0/lc7 0 2024-03-04T13:42:59.795 INFO:tasks.workunit.client.0.smithi107.stdout:1/126: read d1/d3/f11 [7002370,106213] 0 2024-03-04T13:42:59.796 INFO:tasks.workunit.client.1.smithi169.stdout:8/658: symlink d7/d10/d22/d75/db9/dbe/ld7 0 2024-03-04T13:42:59.797 INFO:tasks.workunit.client.0.smithi107.stdout:6/171: dread fb [0,4194304] 0 2024-03-04T13:42:59.799 INFO:tasks.workunit.client.1.smithi169.stdout:4/626: symlink d4/d1c/lc8 0 2024-03-04T13:42:59.800 INFO:tasks.workunit.client.0.smithi107.stdout:9/142: dread d4/f8 [4194304,4194304] 0 2024-03-04T13:42:59.800 INFO:tasks.workunit.client.0.smithi107.stdout:1/127: unlink d1/d3/f11 0 2024-03-04T13:42:59.801 INFO:tasks.workunit.client.1.smithi169.stdout:9/722: dread d7/d25/d3c/f9e [0,4194304] 0 2024-03-04T13:42:59.803 INFO:tasks.workunit.client.1.smithi169.stdout:1/622: sync 2024-03-04T13:42:59.803 INFO:tasks.workunit.client.1.smithi169.stdout:3/686: sync 2024-03-04T13:42:59.806 INFO:tasks.workunit.client.1.smithi169.stdout:7/734: dread d2/d15/d17/f42 [0,4194304] 0 2024-03-04T13:42:59.808 INFO:tasks.workunit.client.0.smithi107.stdout:8/200: sync 2024-03-04T13:42:59.809 INFO:tasks.workunit.client.0.smithi107.stdout:2/162: sync 2024-03-04T13:42:59.809 INFO:tasks.workunit.client.0.smithi107.stdout:0/194: sync 2024-03-04T13:42:59.809 INFO:tasks.workunit.client.0.smithi107.stdout:6/172: mkdir df/d17/d24/d30/d3b 0 2024-03-04T13:42:59.810 INFO:tasks.workunit.client.0.smithi107.stdout:8/201: fdatasync d6/da/f19 0 2024-03-04T13:42:59.811 INFO:tasks.workunit.client.1.smithi169.stdout:1/623: rmdir d4/d5/d2d 39 2024-03-04T13:42:59.811 INFO:tasks.workunit.client.1.smithi169.stdout:3/687: write d7/de/d2b/d7d/d85/d13/d1a/d2d/d56/f81 [6177413,66587] 0 2024-03-04T13:42:59.811 INFO:tasks.workunit.client.0.smithi107.stdout:0/195: read da/f3c [1179210,53710] 0 2024-03-04T13:42:59.814 INFO:tasks.workunit.client.1.smithi169.stdout:9/723: mknod d7/d49/df8/c102 0 2024-03-04T13:42:59.821 INFO:tasks.workunit.client.1.smithi169.stdout:8/659: truncate d7/d10/d17/d44/d4c/fa3 501649 0 2024-03-04T13:42:59.822 INFO:tasks.workunit.client.0.smithi107.stdout:6/173: dread df/f29 [0,4194304] 0 2024-03-04T13:42:59.826 INFO:tasks.workunit.client.1.smithi169.stdout:6/670: dread d0/d3c/d40/f4d [0,4194304] 0 2024-03-04T13:42:59.827 INFO:tasks.workunit.client.1.smithi169.stdout:3/688: symlink d7/de/d2b/d7d/d85/d13/d5a/da1/leb 0 2024-03-04T13:42:59.830 INFO:tasks.workunit.client.0.smithi107.stdout:8/202: rename d6/d16/f1a to d6/d16/d1f/f3a 0 2024-03-04T13:42:59.830 INFO:tasks.workunit.client.0.smithi107.stdout:7/132: write d6/f12 [4006361,39582] 0 2024-03-04T13:42:59.831 INFO:tasks.workunit.client.0.smithi107.stdout:8/203: readlink d6/d16/l21 0 2024-03-04T13:42:59.833 INFO:tasks.workunit.client.0.smithi107.stdout:9/143: link d4/c1e d4/da/c27 0 2024-03-04T13:42:59.835 INFO:tasks.workunit.client.1.smithi169.stdout:5/779: dread d1/d81/d91/fb9 [0,4194304] 0 2024-03-04T13:42:59.837 INFO:tasks.workunit.client.1.smithi169.stdout:4/627: write d4/d11/f1b [1119542,57291] 0 2024-03-04T13:42:59.837 INFO:tasks.workunit.client.0.smithi107.stdout:1/128: getdents d1/d3 0 2024-03-04T13:42:59.839 INFO:tasks.workunit.client.1.smithi169.stdout:5/780: truncate d1/d16/f20 5660316 0 2024-03-04T13:42:59.840 INFO:tasks.workunit.client.0.smithi107.stdout:2/163: chown da/df/d27/c32 1767797 1 2024-03-04T13:42:59.841 INFO:tasks.workunit.client.0.smithi107.stdout:2/164: readlink da/df/l12 0 2024-03-04T13:42:59.843 INFO:tasks.workunit.client.1.smithi169.stdout:8/660: symlink d7/d10/d24/d3e/ld8 0 2024-03-04T13:42:59.845 INFO:tasks.workunit.client.0.smithi107.stdout:6/174: rmdir df/d17 39 2024-03-04T13:42:59.845 INFO:tasks.workunit.client.1.smithi169.stdout:3/689: dwrite d7/de/d2b/d7d/d85/d13/d1a/d2d/d56/d5e/d67/f3a [0,4194304] 0 2024-03-04T13:42:59.846 INFO:tasks.workunit.client.0.smithi107.stdout:6/175: write df/d1b/f37 [524321,40000] 0 2024-03-04T13:42:59.849 INFO:tasks.workunit.client.0.smithi107.stdout:6/176: chown df/d1b/f37 272 1 2024-03-04T13:42:59.849 INFO:tasks.workunit.client.0.smithi107.stdout:6/177: read df/f29 [2903803,56112] 0 2024-03-04T13:42:59.851 INFO:tasks.workunit.client.1.smithi169.stdout:6/671: rename d0/d32/d3e/db9 to d0/d3c/d40/df6 0 2024-03-04T13:42:59.852 INFO:tasks.workunit.client.0.smithi107.stdout:6/178: read df/d1b/f37 [116761,86879] 0 2024-03-04T13:42:59.852 INFO:tasks.workunit.client.1.smithi169.stdout:3/690: read d7/de/d2b/f6e [3426176,28617] 0 2024-03-04T13:42:59.856 INFO:tasks.workunit.client.1.smithi169.stdout:6/672: write d0/d3c/d40/fde [903769,36805] 0 2024-03-04T13:42:59.856 INFO:tasks.workunit.client.1.smithi169.stdout:4/628: dwrite d4/d11/d2e/d37/d5c/fb0 [0,4194304] 0 2024-03-04T13:42:59.858 INFO:tasks.workunit.client.0.smithi107.stdout:8/204: creat d6/da/d25/f3b x:0 0 0 2024-03-04T13:42:59.860 INFO:tasks.workunit.client.0.smithi107.stdout:8/205: readlink d6/da/l30 0 2024-03-04T13:42:59.861 INFO:tasks.workunit.client.0.smithi107.stdout:8/206: readlink d6/da/l30 0 2024-03-04T13:42:59.862 INFO:tasks.workunit.client.1.smithi169.stdout:6/673: write d0/f35 [445961,28431] 0 2024-03-04T13:42:59.862 INFO:tasks.workunit.client.1.smithi169.stdout:4/629: dread - d4/d1c/d58/faf zero size 2024-03-04T13:42:59.863 INFO:tasks.workunit.client.1.smithi169.stdout:5/781: creat d1/d16/d2f/d72/db8/f113 x:0 0 0 2024-03-04T13:42:59.864 INFO:tasks.workunit.client.0.smithi107.stdout:5/120: sync 2024-03-04T13:42:59.865 INFO:tasks.workunit.client.1.smithi169.stdout:4/630: fdatasync d4/d11/d3b/d67/f84 0 2024-03-04T13:42:59.865 INFO:tasks.workunit.client.1.smithi169.stdout:1/624: rmdir d4/d5/d36/d35/d8f/d93 39 2024-03-04T13:42:59.866 INFO:tasks.workunit.client.1.smithi169.stdout:6/674: stat d0/d32/d3a/d43/d74/d93 0 2024-03-04T13:42:59.870 INFO:tasks.workunit.client.0.smithi107.stdout:7/133: mkdir d6/d1d/d28/d2c 0 2024-03-04T13:42:59.872 INFO:tasks.workunit.client.0.smithi107.stdout:7/134: chown d6/d13 76940 1 2024-03-04T13:42:59.872 INFO:tasks.workunit.client.1.smithi169.stdout:7/735: getdents d2/d28/db8/dd5 0 2024-03-04T13:42:59.872 INFO:tasks.workunit.client.0.smithi107.stdout:0/196: link da/dc/dd/d3e/d26/l20 da/dc/dd/d1e/l3f 0 2024-03-04T13:42:59.873 INFO:tasks.workunit.client.1.smithi169.stdout:5/782: write d1/d16/d2f/d39/f5f [5079312,43575] 0 2024-03-04T13:42:59.873 INFO:tasks.workunit.client.0.smithi107.stdout:1/129: mkdir d1/d10/d20/d24 0 2024-03-04T13:42:59.875 INFO:tasks.workunit.client.1.smithi169.stdout:9/724: dwrite d7/dc/d12/d32/d42/d4f/f50 [4194304,4194304] 0 2024-03-04T13:42:59.875 INFO:tasks.workunit.client.0.smithi107.stdout:1/130: dread - d1/d10/d16/d1b/f1e zero size 2024-03-04T13:42:59.878 INFO:tasks.workunit.client.0.smithi107.stdout:5/121: dread d5/d8/f17 [0,4194304] 0 2024-03-04T13:42:59.879 INFO:tasks.workunit.client.0.smithi107.stdout:8/207: dread f2 [0,4194304] 0 2024-03-04T13:42:59.881 INFO:tasks.workunit.client.1.smithi169.stdout:8/661: dwrite d7/d10/d24/f3c [0,4194304] 0 2024-03-04T13:42:59.881 INFO:tasks.workunit.client.1.smithi169.stdout:4/631: stat d4/d11/d2e/d39/d61/d64/c7c 0 2024-03-04T13:42:59.882 INFO:tasks.workunit.client.0.smithi107.stdout:0/197: write da/f3c [2326978,67905] 0 2024-03-04T13:42:59.883 INFO:tasks.workunit.client.0.smithi107.stdout:5/122: write d5/d8/f11 [616082,118756] 0 2024-03-04T13:42:59.884 INFO:tasks.workunit.client.0.smithi107.stdout:9/144: getdents d4/da/d11/d20/d23 0 2024-03-04T13:42:59.885 INFO:tasks.workunit.client.0.smithi107.stdout:1/131: truncate d1/d10/d16/d1b/f1e 686374 0 2024-03-04T13:42:59.887 INFO:tasks.workunit.client.0.smithi107.stdout:9/145: readlink d4/l18 0 2024-03-04T13:42:59.890 INFO:tasks.workunit.client.0.smithi107.stdout:7/135: unlink d6/d13/d16/d19/l24 0 2024-03-04T13:42:59.890 INFO:tasks.workunit.client.0.smithi107.stdout:9/146: dread - d4/f21 zero size 2024-03-04T13:42:59.891 INFO:tasks.workunit.client.1.smithi169.stdout:5/783: creat d1/d16/d2f/d72/dab/f114 x:0 0 0 2024-03-04T13:42:59.896 INFO:tasks.workunit.client.0.smithi107.stdout:7/136: read d6/fc [2597116,81221] 0 2024-03-04T13:42:59.897 INFO:tasks.workunit.client.0.smithi107.stdout:6/179: dwrite f6 [0,4194304] 0 2024-03-04T13:42:59.898 INFO:tasks.workunit.client.1.smithi169.stdout:9/725: mknod d7/d25/d3c/c103 0 2024-03-04T13:42:59.902 INFO:tasks.workunit.client.1.smithi169.stdout:8/662: write d7/d10/d17/f27 [1315611,73570] 0 2024-03-04T13:42:59.904 INFO:tasks.workunit.client.0.smithi107.stdout:2/165: getdents da/d34 0 2024-03-04T13:42:59.908 INFO:tasks.workunit.client.1.smithi169.stdout:8/663: chown d7/d10/d17/d44/d5d/f88 944377 1 2024-03-04T13:42:59.910 INFO:tasks.workunit.client.0.smithi107.stdout:8/208: dwrite d6/da/f1b [0,4194304] 0 2024-03-04T13:42:59.913 INFO:tasks.workunit.client.0.smithi107.stdout:8/209: fsync d6/da/f2c 0 2024-03-04T13:42:59.916 INFO:tasks.workunit.client.1.smithi169.stdout:6/675: dwrite d0/d32/d3a/f27 [4194304,4194304] 0 2024-03-04T13:42:59.916 INFO:tasks.workunit.client.1.smithi169.stdout:7/736: rename d2/de/d3e/d41/d69/fc2 to d2/de/ff5 0 2024-03-04T13:42:59.917 INFO:tasks.workunit.client.1.smithi169.stdout:4/632: rename d4/d11/d3b/d67/dbb/d6e to d4/d11/d3b/d67/dbb/d6e/dc9 22 2024-03-04T13:42:59.917 INFO:tasks.workunit.client.0.smithi107.stdout:6/180: rename l5 to df/d1b/l3c 0 2024-03-04T13:42:59.920 INFO:tasks.workunit.client.0.smithi107.stdout:0/198: mknod da/c40 0 2024-03-04T13:42:59.922 INFO:tasks.workunit.client.1.smithi169.stdout:4/633: write d4/d11/d2e/d37/d5c/d86/f99 [4083500,13824] 0 2024-03-04T13:42:59.923 INFO:tasks.workunit.client.1.smithi169.stdout:1/625: dwrite d4/d5/d2d/d4e/d71/d26/f2e [0,4194304] 0 2024-03-04T13:42:59.925 INFO:tasks.workunit.client.0.smithi107.stdout:5/123: creat d5/dc/d22/f26 x:0 0 0 2024-03-04T13:42:59.927 INFO:tasks.workunit.client.0.smithi107.stdout:9/147: dwrite d4/f24 [0,4194304] 0 2024-03-04T13:42:59.931 INFO:tasks.workunit.client.1.smithi169.stdout:1/626: mknod d4/d5/d2d/d4e/da5/cca 0 2024-03-04T13:42:59.932 INFO:tasks.workunit.client.0.smithi107.stdout:3/122: sync 2024-03-04T13:42:59.932 INFO:tasks.workunit.client.0.smithi107.stdout:4/115: sync 2024-03-04T13:42:59.932 INFO:tasks.workunit.client.0.smithi107.stdout:8/210: rmdir d6/da/d25 39 2024-03-04T13:42:59.934 INFO:tasks.workunit.client.0.smithi107.stdout:2/166: mkdir da/d38 0 2024-03-04T13:42:59.935 INFO:tasks.workunit.client.1.smithi169.stdout:9/726: rename d7/d25/d64/fb7 to d7/dc/d3e/da3/f104 0 2024-03-04T13:42:59.935 INFO:tasks.workunit.client.0.smithi107.stdout:1/132: link d1/d10/d16/l1c d1/d10/d20/l25 0 2024-03-04T13:42:59.936 INFO:tasks.workunit.client.1.smithi169.stdout:4/634: mkdir d4/d11/d2e/d37/d5c/d86/d8f/dca 0 2024-03-04T13:42:59.940 INFO:tasks.workunit.client.1.smithi169.stdout:4/635: dread - d4/d11/d3b/d67/dbb/d6e/fc5 zero size 2024-03-04T13:42:59.940 INFO:tasks.workunit.client.1.smithi169.stdout:9/727: symlink d7/d2d/d48/d66/l105 0 2024-03-04T13:42:59.943 INFO:tasks.workunit.client.1.smithi169.stdout:5/784: truncate d1/d16/d17/d2d/d5a/d67/f97 3102021 0 2024-03-04T13:42:59.944 INFO:tasks.workunit.client.0.smithi107.stdout:3/123: fdatasync d1/de/d11/d12/f13 0 2024-03-04T13:42:59.944 INFO:tasks.workunit.client.0.smithi107.stdout:1/133: fdatasync d1/d10/d16/d1b/f23 0 2024-03-04T13:42:59.944 INFO:tasks.workunit.client.0.smithi107.stdout:7/137: dwrite d6/f7 [0,4194304] 0 2024-03-04T13:42:59.945 INFO:tasks.workunit.client.1.smithi169.stdout:8/664: rename d7/d10/d22/d46/d49/ca9 to d7/d10/d24/cd9 0 2024-03-04T13:42:59.947 INFO:tasks.workunit.client.0.smithi107.stdout:2/167: chown da/df/l12 64055471 1 2024-03-04T13:42:59.951 INFO:tasks.workunit.client.0.smithi107.stdout:8/211: symlink d6/d16/l3c 0 2024-03-04T13:42:59.951 INFO:tasks.workunit.client.0.smithi107.stdout:9/148: write d4/da/f25 [1508723,107048] 0 2024-03-04T13:42:59.954 INFO:tasks.workunit.client.0.smithi107.stdout:6/181: readlink df/d1b/l3c 0 2024-03-04T13:42:59.959 INFO:tasks.workunit.client.1.smithi169.stdout:6/676: dread d0/d3c/d40/d52/dac/fc7 [0,4194304] 0 2024-03-04T13:42:59.962 INFO:tasks.workunit.client.1.smithi169.stdout:6/677: fsync d0/d3c/d40/d52/f8c 0 2024-03-04T13:42:59.962 INFO:tasks.workunit.client.0.smithi107.stdout:1/134: unlink d1/d10/c22 0 2024-03-04T13:42:59.963 INFO:tasks.workunit.client.1.smithi169.stdout:9/728: creat d7/dc/d12/d59/f106 x:0 0 0 2024-03-04T13:42:59.964 INFO:tasks.workunit.client.0.smithi107.stdout:0/199: write da/dc/dd/f24 [536216,27627] 0 2024-03-04T13:42:59.964 INFO:tasks.workunit.client.1.smithi169.stdout:5/785: creat d1/d16/d2f/d39/f115 x:0 0 0 2024-03-04T13:42:59.965 INFO:tasks.workunit.client.0.smithi107.stdout:4/116: dwrite d0/d9/de/d14/fa [0,4194304] 0 2024-03-04T13:42:59.965 INFO:tasks.workunit.client.0.smithi107.stdout:7/138: creat d6/d13/d16/d1c/f2d x:0 0 0 2024-03-04T13:42:59.968 INFO:tasks.workunit.client.0.smithi107.stdout:3/124: symlink d1/de/d18/d1f/l27 0 2024-03-04T13:42:59.971 INFO:tasks.workunit.client.0.smithi107.stdout:4/117: dread d0/d9/de/d14/f1d [0,4194304] 0 2024-03-04T13:42:59.972 INFO:tasks.workunit.client.0.smithi107.stdout:9/149: creat d4/da/d11/d20/f28 x:0 0 0 2024-03-04T13:42:59.972 INFO:tasks.workunit.client.0.smithi107.stdout:4/118: chown d0/f3 908763 1 2024-03-04T13:42:59.974 INFO:tasks.workunit.client.1.smithi169.stdout:4/636: dwrite d4/f7 [0,4194304] 0 2024-03-04T13:42:59.976 INFO:tasks.workunit.client.0.smithi107.stdout:4/119: truncate d0/d9/d10/f19 45829 0 2024-03-04T13:42:59.976 INFO:tasks.workunit.client.1.smithi169.stdout:4/637: write d4/d11/d2e/d37/f57 [284828,112137] 0 2024-03-04T13:42:59.978 INFO:tasks.workunit.client.0.smithi107.stdout:1/135: mknod d1/d3/df/c26 0 2024-03-04T13:42:59.979 INFO:tasks.workunit.client.0.smithi107.stdout:4/120: fdatasync d0/f3 0 2024-03-04T13:42:59.980 INFO:tasks.workunit.client.0.smithi107.stdout:0/200: rmdir da/dc/dd/d3e 39 2024-03-04T13:42:59.983 INFO:tasks.workunit.client.0.smithi107.stdout:0/201: dread - da/f31 zero size 2024-03-04T13:42:59.986 INFO:tasks.workunit.client.0.smithi107.stdout:4/121: truncate d0/d9/f1a 4750122 0 2024-03-04T13:42:59.993 INFO:tasks.workunit.client.0.smithi107.stdout:3/125: dwrite d1/de/f19 [4194304,4194304] 0 2024-03-04T13:42:59.993 INFO:tasks.workunit.client.0.smithi107.stdout:9/150: dread d4/da/f25 [0,4194304] 0 2024-03-04T13:42:59.993 INFO:tasks.workunit.client.0.smithi107.stdout:5/124: link d5/d8/l1b d5/d8/l27 0 2024-03-04T13:42:59.998 INFO:tasks.workunit.client.0.smithi107.stdout:1/136: mknod d1/d10/d16/d1b/c27 0 2024-03-04T13:42:59.998 INFO:tasks.workunit.client.0.smithi107.stdout:0/202: rmdir da/dc/d34 39 2024-03-04T13:43:00.001 INFO:tasks.workunit.client.0.smithi107.stdout:5/125: mknod d5/d8/d14/c28 0 2024-03-04T13:43:00.001 INFO:tasks.workunit.client.0.smithi107.stdout:9/151: symlink d4/da/d11/d20/l29 0 2024-03-04T13:43:00.003 INFO:tasks.workunit.client.1.smithi169.stdout:8/665: mkdir d7/d10/d24/d3e/dda 0 2024-03-04T13:43:00.003 INFO:tasks.workunit.client.0.smithi107.stdout:3/126: rename d1/de/d18/c1a to d1/de/d18/c28 0 2024-03-04T13:43:00.006 INFO:tasks.workunit.client.1.smithi169.stdout:6/678: creat d0/d3c/d95/dbd/ff7 x:0 0 0 2024-03-04T13:43:00.006 INFO:tasks.workunit.client.1.smithi169.stdout:8/666: write d7/d10/d17/d44/d36/d62/fab [1164203,123000] 0 2024-03-04T13:43:00.009 INFO:tasks.workunit.client.0.smithi107.stdout:6/182: getdents df/d17/d24/d27 0 2024-03-04T13:43:00.009 INFO:tasks.workunit.client.1.smithi169.stdout:5/786: creat d1/d16/d2f/d72/f116 x:0 0 0 2024-03-04T13:43:00.015 INFO:tasks.workunit.client.0.smithi107.stdout:3/127: mkdir d1/d29 0 2024-03-04T13:43:00.015 INFO:tasks.workunit.client.0.smithi107.stdout:4/122: link d0/l8 d0/d9/d1e/l24 0 2024-03-04T13:43:00.018 INFO:tasks.workunit.client.1.smithi169.stdout:7/737: rename d2/de/d3e/c7b to d2/dc0/cf6 0 2024-03-04T13:43:00.021 INFO:tasks.workunit.client.0.smithi107.stdout:0/203: mknod da/dc/dd/d3e/d26/c41 0 2024-03-04T13:43:00.022 INFO:tasks.workunit.client.1.smithi169.stdout:8/667: mkdir d7/d10/d22/d46/da5/da6/ddb 0 2024-03-04T13:43:00.023 INFO:tasks.workunit.client.1.smithi169.stdout:8/668: read d7/d10/d22/d46/f83 [744293,73920] 0 2024-03-04T13:43:00.028 INFO:tasks.workunit.client.1.smithi169.stdout:9/729: rename d7/dc/d12/d20/da6/fdb to d7/d2d/d48/d66/dec/f107 0 2024-03-04T13:43:00.030 INFO:tasks.workunit.client.0.smithi107.stdout:4/123: rename d0/f3 to d0/d9/d10/f25 0 2024-03-04T13:43:00.032 INFO:tasks.workunit.client.0.smithi107.stdout:1/137: dwrite d1/d10/d16/d1b/f23 [0,4194304] 0 2024-03-04T13:43:00.032 INFO:tasks.workunit.client.0.smithi107.stdout:5/126: dwrite d5/f20 [0,4194304] 0 2024-03-04T13:43:00.033 INFO:tasks.workunit.client.1.smithi169.stdout:5/787: link d1/d16/d2a/cd9 d1/d16/d2a/d46/d9e/c117 0 2024-03-04T13:43:00.034 INFO:tasks.workunit.client.1.smithi169.stdout:5/788: fsync d1/d16/d17/f109 0 2024-03-04T13:43:00.037 INFO:tasks.workunit.client.0.smithi107.stdout:3/128: dread d1/f9 [0,4194304] 0 2024-03-04T13:43:00.037 INFO:tasks.workunit.client.0.smithi107.stdout:6/183: dread fd [0,4194304] 0 2024-03-04T13:43:00.040 INFO:tasks.workunit.client.1.smithi169.stdout:8/669: dwrite d7/d10/d22/d46/fc0 [0,4194304] 0 2024-03-04T13:43:00.042 INFO:tasks.workunit.client.0.smithi107.stdout:8/212: truncate d6/d16/f1c 1233164 0 2024-03-04T13:43:00.043 INFO:tasks.workunit.client.1.smithi169.stdout:8/670: creat d7/dd4/fdc x:0 0 0 2024-03-04T13:43:00.045 INFO:tasks.workunit.client.0.smithi107.stdout:0/204: dread f0 [0,4194304] 0 2024-03-04T13:43:00.049 INFO:tasks.workunit.client.0.smithi107.stdout:2/168: truncate da/fd 610146 0 2024-03-04T13:43:00.049 INFO:tasks.workunit.client.0.smithi107.stdout:9/152: dwrite d4/da/f25 [0,4194304] 0 2024-03-04T13:43:00.054 INFO:tasks.workunit.client.0.smithi107.stdout:3/129: rename d1/de/d11/d12/f13 to d1/d29/f2a 0 2024-03-04T13:43:00.055 INFO:tasks.workunit.client.0.smithi107.stdout:5/127: link d5/dc/d22/f26 d5/f29 0 2024-03-04T13:43:00.063 INFO:tasks.workunit.client.0.smithi107.stdout:5/128: chown d5/dc/d22 111379922 1 2024-03-04T13:43:00.064 INFO:tasks.workunit.client.0.smithi107.stdout:6/184: chown df/d17/d24/d27/c25 434 1 2024-03-04T13:43:00.070 INFO:tasks.workunit.client.0.smithi107.stdout:3/130: unlink d1/de/d11/f1c 0 2024-03-04T13:43:00.071 INFO:tasks.workunit.client.0.smithi107.stdout:7/139: dwrite f4 [0,4194304] 0 2024-03-04T13:43:00.071 INFO:tasks.workunit.client.0.smithi107.stdout:6/185: readlink df/d1b/d2c/l3a 0 2024-03-04T13:43:00.072 INFO:tasks.workunit.client.0.smithi107.stdout:0/205: creat da/dc/d34/f42 x:0 0 0 2024-03-04T13:43:00.077 INFO:tasks.workunit.client.0.smithi107.stdout:8/213: rename d6/da/f1e to d6/f3d 0 2024-03-04T13:43:00.079 INFO:tasks.workunit.client.0.smithi107.stdout:5/129: mkdir d5/d8/d2a 0 2024-03-04T13:43:00.079 INFO:tasks.workunit.client.0.smithi107.stdout:0/206: fsync da/f3c 0 2024-03-04T13:43:00.083 INFO:tasks.workunit.client.1.smithi169.stdout:4/638: dwrite d4/f69 [0,4194304] 0 2024-03-04T13:43:00.084 INFO:tasks.workunit.client.0.smithi107.stdout:3/131: creat d1/f2b x:0 0 0 2024-03-04T13:43:00.085 INFO:tasks.workunit.client.0.smithi107.stdout:1/138: dwrite d1/d10/d16/d1b/f1e [0,4194304] 0 2024-03-04T13:43:00.088 INFO:tasks.workunit.client.0.smithi107.stdout:9/153: rename d4 to d4/da/d2a 22 2024-03-04T13:43:00.090 INFO:tasks.workunit.client.0.smithi107.stdout:6/186: rename df/d17 to df/d17/d24/d3d 22 2024-03-04T13:43:00.091 INFO:tasks.workunit.client.1.smithi169.stdout:9/730: rename d7/d2d/d48/d66/dec/f107 to d7/d27/dc3/f108 0 2024-03-04T13:43:00.091 INFO:tasks.workunit.client.0.smithi107.stdout:1/139: write d1/d10/d16/d1b/f23 [1839005,119487] 0 2024-03-04T13:43:00.092 INFO:tasks.workunit.client.1.smithi169.stdout:6/679: write d0/d3c/d40/d52/fb4 [730511,113072] 0 2024-03-04T13:43:00.098 INFO:tasks.workunit.client.1.smithi169.stdout:6/680: fdatasync d0/d32/d3a/d43/f45 0 2024-03-04T13:43:00.099 INFO:tasks.workunit.client.1.smithi169.stdout:8/671: dread d7/ff [0,4194304] 0 2024-03-04T13:43:00.100 INFO:tasks.workunit.client.0.smithi107.stdout:5/130: write d5/dc/d22/f26 [398172,85335] 0 2024-03-04T13:43:00.104 INFO:tasks.workunit.client.1.smithi169.stdout:0/540: sync 2024-03-04T13:43:00.108 INFO:tasks.workunit.client.1.smithi169.stdout:9/731: rename d7/dc/f79 to d7/dc/d12/d20/d4e/f109 0 2024-03-04T13:43:00.111 INFO:tasks.workunit.client.0.smithi107.stdout:9/154: readlink d4/da/l10 0 2024-03-04T13:43:00.114 INFO:tasks.workunit.client.0.smithi107.stdout:3/132: dwrite d1/d1e/f22 [0,4194304] 0 2024-03-04T13:43:00.114 INFO:tasks.workunit.client.0.smithi107.stdout:6/187: dread f6 [0,4194304] 0 2024-03-04T13:43:00.117 INFO:tasks.workunit.client.0.smithi107.stdout:6/188: read df/d17/d24/d27/f26 [74226,46648] 0 2024-03-04T13:43:00.119 INFO:tasks.workunit.client.0.smithi107.stdout:6/189: write f0 [4838690,42232] 0 2024-03-04T13:43:00.123 INFO:tasks.workunit.client.1.smithi169.stdout:5/789: dread d1/d16/d17/d28/f4c [0,4194304] 0 2024-03-04T13:43:00.126 INFO:tasks.workunit.client.0.smithi107.stdout:1/140: fdatasync d1/d10/d16/d1b/f1e 0 2024-03-04T13:43:00.127 INFO:tasks.workunit.client.1.smithi169.stdout:9/732: rename d7/dc/d12/d8a/df9/dff to d7/d25/d64/d10a 0 2024-03-04T13:43:00.127 INFO:tasks.workunit.client.0.smithi107.stdout:1/141: chown d1 16279542 1 2024-03-04T13:43:00.129 INFO:tasks.workunit.client.1.smithi169.stdout:4/639: getdents d4/d11/d2e/d6c/d78/da5/d77 0 2024-03-04T13:43:00.129 INFO:tasks.workunit.client.1.smithi169.stdout:9/733: dread - d7/dc/d12/d32/d42/fd6 zero size 2024-03-04T13:43:00.130 INFO:tasks.workunit.client.0.smithi107.stdout:4/124: read d0/d9/de/d14/f1d [104026,112524] 0 2024-03-04T13:43:00.131 INFO:tasks.workunit.client.0.smithi107.stdout:2/169: write da/fd [854228,104710] 0 2024-03-04T13:43:00.133 INFO:tasks.workunit.client.1.smithi169.stdout:6/681: truncate d0/d32/d70/fe0 1241829 0 2024-03-04T13:43:00.136 INFO:tasks.workunit.client.0.smithi107.stdout:3/133: fdatasync d1/f9 0 2024-03-04T13:43:00.137 INFO:tasks.workunit.client.1.smithi169.stdout:8/672: write d7/d10/d17/d44/d36/f8a [787458,92228] 0 2024-03-04T13:43:00.140 INFO:tasks.workunit.client.1.smithi169.stdout:0/541: write d5/d9/dc/d12/d15/f9e [5301743,10980] 0 2024-03-04T13:43:00.142 INFO:tasks.workunit.client.1.smithi169.stdout:9/734: mkdir d7/dc/d12/d32/d42/d10b 0 2024-03-04T13:43:00.144 INFO:tasks.workunit.client.1.smithi169.stdout:0/542: chown d5/d9/dc/d12/d15/d19/d61/f33 28638 1 2024-03-04T13:43:00.146 INFO:tasks.workunit.client.1.smithi169.stdout:9/735: fsync d7/dc/d12/d32/d42/d4f/f50 0 2024-03-04T13:43:00.146 INFO:tasks.workunit.client.1.smithi169.stdout:5/790: rename d1/ld to d1/d16/d17/d1b/d74/ddc/l118 0 2024-03-04T13:43:00.146 INFO:tasks.workunit.client.0.smithi107.stdout:9/155: write d4/ff [3399204,10106] 0 2024-03-04T13:43:00.147 INFO:tasks.workunit.client.0.smithi107.stdout:7/140: getdents d6/d13/d14 0 2024-03-04T13:43:00.147 INFO:tasks.workunit.client.0.smithi107.stdout:4/125: dread d0/d9/d10/f1c [0,4194304] 0 2024-03-04T13:43:00.150 INFO:tasks.workunit.client.0.smithi107.stdout:5/131: mkdir d5/d2b 0 2024-03-04T13:43:00.150 INFO:tasks.workunit.client.0.smithi107.stdout:4/126: readlink d0/d9/de/d14/l23 0 2024-03-04T13:43:00.150 INFO:tasks.workunit.client.0.smithi107.stdout:7/141: read d6/d13/d16/d1c/f1f [915021,27924] 0 2024-03-04T13:43:00.152 INFO:tasks.workunit.client.0.smithi107.stdout:1/142: mknod d1/d10/d16/c28 0 2024-03-04T13:43:00.154 INFO:tasks.workunit.client.0.smithi107.stdout:2/170: truncate da/df/f15 3768233 0 2024-03-04T13:43:00.160 INFO:tasks.workunit.client.0.smithi107.stdout:3/134: dwrite d1/f3 [4194304,4194304] 0 2024-03-04T13:43:00.161 INFO:tasks.workunit.client.1.smithi169.stdout:6/682: rename d0/d3c/d40/d52/fb4 to d0/d32/d3a/d43/d74/d93/dbe/ff8 0 2024-03-04T13:43:00.161 INFO:tasks.workunit.client.1.smithi169.stdout:0/543: mknod d5/d4c/d65/d95/dba/cbf 0 2024-03-04T13:43:00.161 INFO:tasks.workunit.client.1.smithi169.stdout:4/640: dread d4/fa [4194304,4194304] 0 2024-03-04T13:43:00.164 INFO:tasks.workunit.client.0.smithi107.stdout:0/207: getdents da/dc 0 2024-03-04T13:43:00.164 INFO:tasks.workunit.client.0.smithi107.stdout:8/214: mkdir d6/da/d25/d2d/d3e 0 2024-03-04T13:43:00.164 INFO:tasks.workunit.client.0.smithi107.stdout:9/156: dwrite d4/f26 [0,4194304] 0 2024-03-04T13:43:00.164 INFO:tasks.workunit.client.1.smithi169.stdout:5/791: symlink d1/d16/d17/l119 0 2024-03-04T13:43:00.166 INFO:tasks.workunit.client.0.smithi107.stdout:6/190: mkdir df/d38/d3e 0 2024-03-04T13:43:00.169 INFO:tasks.workunit.client.0.smithi107.stdout:5/132: readlink d5/d8/l13 0 2024-03-04T13:43:00.169 INFO:tasks.workunit.client.1.smithi169.stdout:8/673: getdents d7/d10/d22/d46/da5 0 2024-03-04T13:43:00.174 INFO:tasks.workunit.client.1.smithi169.stdout:6/683: getdents d0/d3c/d95/d9e 0 2024-03-04T13:43:00.179 INFO:tasks.workunit.client.1.smithi169.stdout:9/736: link d7/d2d/d48/d66/f75 d7/dc/d12/d59/f10c 0 2024-03-04T13:43:00.180 INFO:tasks.workunit.client.1.smithi169.stdout:9/737: stat d7/dc/d3e/da3 0 2024-03-04T13:43:00.180 INFO:tasks.workunit.client.1.smithi169.stdout:6/684: rmdir d0/d32/d3e 39 2024-03-04T13:43:00.182 INFO:tasks.workunit.client.0.smithi107.stdout:8/215: creat d6/da/d25/f3f x:0 0 0 2024-03-04T13:43:00.182 INFO:tasks.workunit.client.0.smithi107.stdout:2/171: dread - da/f30 zero size 2024-03-04T13:43:00.187 INFO:tasks.workunit.client.0.smithi107.stdout:9/157: unlink d4/da/f25 0 2024-03-04T13:43:00.190 INFO:tasks.workunit.client.1.smithi169.stdout:5/792: link d1/d16/d2a/d46/l61 d1/d16/dbd/dbb/l11a 0 2024-03-04T13:43:00.190 INFO:tasks.workunit.client.0.smithi107.stdout:2/172: chown f2 5896 1 2024-03-04T13:43:00.190 INFO:tasks.workunit.client.0.smithi107.stdout:1/143: unlink d1/d10/d20/l25 0 2024-03-04T13:43:00.193 INFO:tasks.workunit.client.1.smithi169.stdout:9/738: unlink d7/d27/lcc 0 2024-03-04T13:43:00.193 INFO:tasks.workunit.client.0.smithi107.stdout:3/135: dwrite d1/de/d18/d1f/f20 [0,4194304] 0 2024-03-04T13:43:00.194 INFO:tasks.workunit.client.0.smithi107.stdout:6/191: dread df/d17/f1e [0,4194304] 0 2024-03-04T13:43:00.196 INFO:tasks.workunit.client.0.smithi107.stdout:3/136: chown d1/de/d11/d12 33333 1 2024-03-04T13:43:00.196 INFO:tasks.workunit.client.1.smithi169.stdout:6/685: rename d0/d32/f58 to d0/d63/ff9 0 2024-03-04T13:43:00.198 INFO:tasks.workunit.client.0.smithi107.stdout:6/192: write df/d17/d24/f2b [4733,15003] 0 2024-03-04T13:43:00.201 INFO:tasks.workunit.client.0.smithi107.stdout:6/193: chown f8 3856 1 2024-03-04T13:43:00.201 INFO:tasks.workunit.client.0.smithi107.stdout:9/158: readlink d4/da/l16 0 2024-03-04T13:43:00.202 INFO:tasks.workunit.client.1.smithi169.stdout:0/544: dwrite d5/f16 [4194304,4194304] 0 2024-03-04T13:43:00.204 INFO:tasks.workunit.client.1.smithi169.stdout:4/641: dwrite d4/d11/f79 [0,4194304] 0 2024-03-04T13:43:00.208 INFO:tasks.workunit.client.0.smithi107.stdout:2/173: dread f0 [0,4194304] 0 2024-03-04T13:43:00.209 INFO:tasks.workunit.client.1.smithi169.stdout:9/739: symlink d7/dc/d12/d20/d4e/l10d 0 2024-03-04T13:43:00.209 INFO:tasks.workunit.client.1.smithi169.stdout:5/793: dread d1/d16/d17/d28/f4c [0,4194304] 0 2024-03-04T13:43:00.209 INFO:tasks.workunit.client.1.smithi169.stdout:0/545: chown d5/d9/d3d/d59/d64/d7f/ca0 12558 1 2024-03-04T13:43:00.211 INFO:tasks.workunit.client.0.smithi107.stdout:4/127: dwrite d0/d9/de/d14/f1d [0,4194304] 0 2024-03-04T13:43:00.211 INFO:tasks.workunit.client.0.smithi107.stdout:2/174: chown da/df/c19 32758 1 2024-03-04T13:43:00.212 INFO:tasks.workunit.client.1.smithi169.stdout:8/674: truncate d7/ff 1894077 0 2024-03-04T13:43:00.213 INFO:tasks.workunit.client.0.smithi107.stdout:2/175: readlink da/df/l2c 0 2024-03-04T13:43:00.213 INFO:tasks.workunit.client.1.smithi169.stdout:2/713: sync 2024-03-04T13:43:00.214 INFO:tasks.workunit.client.1.smithi169.stdout:3/691: sync 2024-03-04T13:43:00.214 INFO:tasks.workunit.client.0.smithi107.stdout:0/208: dwrite da/dc/d34/f38 [0,4194304] 0 2024-03-04T13:43:00.215 INFO:tasks.workunit.client.0.smithi107.stdout:0/209: stat da/dc/c10 0 2024-03-04T13:43:00.216 INFO:tasks.workunit.client.1.smithi169.stdout:3/692: dread - d7/de/d2b/d7d/d85/d13/d5a/fe0 zero size 2024-03-04T13:43:00.218 INFO:tasks.workunit.client.1.smithi169.stdout:6/686: rmdir d0/d3c/d40/df6/dca 39 2024-03-04T13:43:00.222 INFO:tasks.workunit.client.0.smithi107.stdout:3/137: rename d1/l7 to d1/l2c 0 2024-03-04T13:43:00.223 INFO:tasks.workunit.client.0.smithi107.stdout:6/194: fsync fd 0 2024-03-04T13:43:00.223 INFO:tasks.workunit.client.1.smithi169.stdout:3/693: write d7/de/d2b/d7d/d85/d13/d5a/d65/da4/fe7 [695787,129097] 0 2024-03-04T13:43:00.224 INFO:tasks.workunit.client.1.smithi169.stdout:5/794: stat d1/d16/d2a/d46/d9e/c117 0 2024-03-04T13:43:00.225 INFO:tasks.workunit.client.0.smithi107.stdout:7/142: dread d6/fc [0,4194304] 0 2024-03-04T13:43:00.225 INFO:tasks.workunit.client.0.smithi107.stdout:6/195: rename df/d1b/d2c to df/d1b/d2c/d3f 22 2024-03-04T13:43:00.226 INFO:tasks.workunit.client.1.smithi169.stdout:6/687: truncate d0/d32/d3a/d43/d74/f7e 14219 0 2024-03-04T13:43:00.227 INFO:tasks.workunit.client.1.smithi169.stdout:0/546: creat d5/d9/dc/d12/d15/d19/d61/d71/d8c/d90/fc0 x:0 0 0 2024-03-04T13:43:00.228 INFO:tasks.workunit.client.0.smithi107.stdout:7/143: write f0 [4901620,97505] 0 2024-03-04T13:43:00.228 INFO:tasks.workunit.client.1.smithi169.stdout:5/795: chown d1/d16/d17/d28/f4d 60 1 2024-03-04T13:43:00.229 INFO:tasks.workunit.client.0.smithi107.stdout:7/144: fsync d6/d1d/f29 0 2024-03-04T13:43:00.231 INFO:tasks.workunit.client.1.smithi169.stdout:8/675: unlink d7/d12/f5e 0 2024-03-04T13:43:00.232 INFO:tasks.workunit.client.0.smithi107.stdout:9/159: creat d4/da/d11/f2b x:0 0 0 2024-03-04T13:43:00.233 INFO:tasks.workunit.client.0.smithi107.stdout:5/133: truncate d5/d8/f11 1658157 0 2024-03-04T13:43:00.236 INFO:tasks.workunit.client.1.smithi169.stdout:5/796: chown d1/d81/db1/cfd 25 1 2024-03-04T13:43:00.237 INFO:tasks.workunit.client.1.smithi169.stdout:0/547: write d5/d9/d3d/d59/f60 [344792,99852] 0 2024-03-04T13:43:00.238 INFO:tasks.workunit.client.1.smithi169.stdout:3/694: fsync d7/de/d2b/f6e 0 2024-03-04T13:43:00.240 INFO:tasks.workunit.client.1.smithi169.stdout:4/642: dwrite d4/d11/d2e/d39/d48/f3c [8388608,4194304] 0 2024-03-04T13:43:00.245 INFO:tasks.workunit.client.0.smithi107.stdout:2/176: write da/f18 [971208,75325] 0 2024-03-04T13:43:00.245 INFO:tasks.workunit.client.1.smithi169.stdout:8/676: dread d7/d10/d22/d46/f83 [0,4194304] 0 2024-03-04T13:43:00.246 INFO:tasks.workunit.client.0.smithi107.stdout:1/144: mknod d1/c29 0 2024-03-04T13:43:00.250 INFO:tasks.workunit.client.1.smithi169.stdout:4/643: write d4/d11/d2e/d39/fb9 [478991,116532] 0 2024-03-04T13:43:00.251 INFO:tasks.workunit.client.0.smithi107.stdout:6/196: dread df/d17/f23 [0,4194304] 0 2024-03-04T13:43:00.251 INFO:tasks.workunit.client.0.smithi107.stdout:3/138: dwrite d1/f3 [8388608,4194304] 0 2024-03-04T13:43:00.256 INFO:tasks.workunit.client.1.smithi169.stdout:2/714: dread d1/d6d/dbc/fbe [0,4194304] 0 2024-03-04T13:43:00.257 INFO:tasks.workunit.client.1.smithi169.stdout:1/627: sync 2024-03-04T13:43:00.257 INFO:tasks.workunit.client.1.smithi169.stdout:7/738: sync 2024-03-04T13:43:00.258 INFO:tasks.workunit.client.0.smithi107.stdout:3/139: read - d1/d29/f2a zero size 2024-03-04T13:43:00.266 INFO:tasks.workunit.client.1.smithi169.stdout:2/715: chown d1/f30 21646 1 2024-03-04T13:43:00.268 INFO:tasks.workunit.client.1.smithi169.stdout:8/677: rename d7/d10/d22/d75/d79/d9e to d7/d10/d17/d44/d5d/ddd 0 2024-03-04T13:43:00.269 INFO:tasks.workunit.client.0.smithi107.stdout:5/134: write d5/f9 [2719445,38049] 0 2024-03-04T13:43:00.270 INFO:tasks.workunit.client.0.smithi107.stdout:6/197: mkdir df/d17/d24/d40 0 2024-03-04T13:43:00.271 INFO:tasks.workunit.client.1.smithi169.stdout:4/644: read d4/d11/d3b/d67/d7e/f9c [192586,99794] 0 2024-03-04T13:43:00.271 INFO:tasks.workunit.client.1.smithi169.stdout:3/695: dwrite d7/db8/fc6 [0,4194304] 0 2024-03-04T13:43:00.272 INFO:tasks.workunit.client.1.smithi169.stdout:6/688: write d0/d3c/d40/d52/dac/fc1 [235041,122287] 0 2024-03-04T13:43:00.273 INFO:tasks.workunit.client.0.smithi107.stdout:4/128: rmdir d0/d9/d1f 0 2024-03-04T13:43:00.274 INFO:tasks.workunit.client.0.smithi107.stdout:9/160: mkdir d4/da/d11/d20/d23/d2c 0 2024-03-04T13:43:00.275 INFO:tasks.workunit.client.0.smithi107.stdout:3/140: mkdir d1/d2d 0 2024-03-04T13:43:00.277 INFO:tasks.workunit.client.1.smithi169.stdout:9/740: dread d7/dc/d12/d32/d42/f43 [0,4194304] 0 2024-03-04T13:43:00.279 INFO:tasks.workunit.client.0.smithi107.stdout:2/177: mkdir da/d39 0 2024-03-04T13:43:00.281 INFO:tasks.workunit.client.0.smithi107.stdout:9/161: write d4/f21 [1020381,99123] 0 2024-03-04T13:43:00.283 INFO:tasks.workunit.client.0.smithi107.stdout:5/135: symlink d5/d8/l2c 0 2024-03-04T13:43:00.283 INFO:tasks.workunit.client.1.smithi169.stdout:2/716: rmdir d1/d5/d19/d1e/d29 39 2024-03-04T13:43:00.284 INFO:tasks.workunit.client.0.smithi107.stdout:1/145: dread d1/fe [0,4194304] 0 2024-03-04T13:43:00.285 INFO:tasks.workunit.client.1.smithi169.stdout:9/741: chown d7/d25/d3c/f5f 1071 1 2024-03-04T13:43:00.286 INFO:tasks.workunit.client.0.smithi107.stdout:6/198: write f6 [4946649,58779] 0 2024-03-04T13:43:00.287 INFO:tasks.workunit.client.1.smithi169.stdout:7/739: mknod d2/dc0/cf7 0 2024-03-04T13:43:00.287 INFO:tasks.workunit.client.0.smithi107.stdout:4/129: write d0/d9/d10/f25 [1708075,90615] 0 2024-03-04T13:43:00.289 INFO:tasks.workunit.client.0.smithi107.stdout:3/141: rename d1/de/d11/l17 to d1/de/d11/l2e 0 2024-03-04T13:43:00.290 INFO:tasks.workunit.client.0.smithi107.stdout:0/210: dwrite da/f2a [0,4194304] 0 2024-03-04T13:43:00.290 INFO:tasks.workunit.client.0.smithi107.stdout:2/178: mkdir da/d38/d3a 0 2024-03-04T13:43:00.290 INFO:tasks.workunit.client.1.smithi169.stdout:4/645: chown d4/d1c/d58/l71 0 1 2024-03-04T13:43:00.291 INFO:tasks.workunit.client.1.smithi169.stdout:8/678: truncate d7/d10/d17/d44/fd5 250983 0 2024-03-04T13:43:00.291 INFO:tasks.workunit.client.0.smithi107.stdout:6/199: rename df/d2e to df/d2e/d41 22 2024-03-04T13:43:00.294 INFO:tasks.workunit.client.0.smithi107.stdout:2/179: dread - da/df/f1e zero size 2024-03-04T13:43:00.296 INFO:tasks.workunit.client.1.smithi169.stdout:0/548: dwrite d5/d9/dc/d12/d15/d19/d61/f23 [4194304,4194304] 0 2024-03-04T13:43:00.298 INFO:tasks.workunit.client.0.smithi107.stdout:5/136: mkdir d5/d8/d14/d2d 0 2024-03-04T13:43:00.298 INFO:tasks.workunit.client.0.smithi107.stdout:4/130: unlink d0/d9/d10/f15 0 2024-03-04T13:43:00.298 INFO:tasks.workunit.client.1.smithi169.stdout:0/549: stat d5/d9/dc/d12/l4b 0 2024-03-04T13:43:00.303 INFO:tasks.workunit.client.0.smithi107.stdout:6/200: mknod df/d38/c42 0 2024-03-04T13:43:00.304 INFO:tasks.workunit.client.0.smithi107.stdout:0/211: symlink da/dc/dd/d3e/d26/l43 0 2024-03-04T13:43:00.305 INFO:tasks.workunit.client.0.smithi107.stdout:6/201: write f8 [1228381,56293] 0 2024-03-04T13:43:00.306 INFO:tasks.workunit.client.1.smithi169.stdout:2/717: dread d1/d5/d19/f47 [4194304,4194304] 0 2024-03-04T13:43:00.310 INFO:tasks.workunit.client.0.smithi107.stdout:4/131: stat d0/l17 0 2024-03-04T13:43:00.311 INFO:tasks.workunit.client.1.smithi169.stdout:4/646: creat d4/d11/d2e/d21/da9/fcb x:0 0 0 2024-03-04T13:43:00.312 INFO:tasks.workunit.client.0.smithi107.stdout:6/202: write df/d17/d24/f2b [406740,77769] 0 2024-03-04T13:43:00.312 INFO:tasks.workunit.client.0.smithi107.stdout:4/132: chown d0/d9/d18 130519 1 2024-03-04T13:43:00.319 INFO:tasks.workunit.client.0.smithi107.stdout:6/203: stat df/d17/d24/d40 0 2024-03-04T13:43:00.319 INFO:tasks.workunit.client.0.smithi107.stdout:4/133: write d0/d9/d10/f25 [3628691,11795] 0 2024-03-04T13:43:00.320 INFO:tasks.workunit.client.1.smithi169.stdout:5/797: truncate d1/d16/dbd/f4f 996948 0 2024-03-04T13:43:00.321 INFO:tasks.workunit.client.0.smithi107.stdout:1/146: dread d1/fe [0,4194304] 0 2024-03-04T13:43:00.325 INFO:tasks.workunit.client.0.smithi107.stdout:2/180: dwrite da/f18 [0,4194304] 0 2024-03-04T13:43:00.325 INFO:tasks.workunit.client.1.smithi169.stdout:3/696: dwrite d7/de/d2b/d7d/d85/d13/d1a/d2d/dbe/fcb [0,4194304] 0 2024-03-04T13:43:00.330 INFO:tasks.workunit.client.1.smithi169.stdout:6/689: dread d0/f2b [0,4194304] 0 2024-03-04T13:43:00.332 INFO:tasks.workunit.client.1.smithi169.stdout:7/740: dwrite d2/ff3 [0,4194304] 0 2024-03-04T13:43:00.332 INFO:tasks.workunit.client.1.smithi169.stdout:4/647: symlink d4/d11/d3b/d67/lcc 0 2024-03-04T13:43:00.336 INFO:tasks.workunit.client.1.smithi169.stdout:5/798: creat d1/d81/d91/f11b x:0 0 0 2024-03-04T13:43:00.338 INFO:tasks.workunit.client.1.smithi169.stdout:8/679: dread d7/ff [0,4194304] 0 2024-03-04T13:43:00.339 INFO:tasks.workunit.client.0.smithi107.stdout:1/147: mknod d1/d3/c2a 0 2024-03-04T13:43:00.340 INFO:tasks.workunit.client.1.smithi169.stdout:3/697: chown d7/de/d2b/d7d/d85/d18/d21/d66/f75 130560151 1 2024-03-04T13:43:00.340 INFO:tasks.workunit.client.1.smithi169.stdout:0/550: dwrite d5/d4c/d65/f8e [0,4194304] 0 2024-03-04T13:43:00.341 INFO:tasks.workunit.client.1.smithi169.stdout:9/742: write d7/d2d/d48/f9c [33609,24717] 0 2024-03-04T13:43:00.341 INFO:tasks.workunit.client.0.smithi107.stdout:2/181: readlink da/l23 0 2024-03-04T13:43:00.341 INFO:tasks.workunit.client.1.smithi169.stdout:7/741: stat d2/de/d3e/d41/f49 0 2024-03-04T13:43:00.343 INFO:tasks.workunit.client.0.smithi107.stdout:1/148: read d1/fe [2433511,118042] 0 2024-03-04T13:43:00.349 INFO:tasks.workunit.client.0.smithi107.stdout:4/134: dwrite d0/d9/d10/f19 [0,4194304] 0 2024-03-04T13:43:00.352 INFO:tasks.workunit.client.1.smithi169.stdout:6/690: mkdir d0/d3c/d40/d52/d98/dfa 0 2024-03-04T13:43:00.353 INFO:tasks.workunit.client.1.smithi169.stdout:5/799: symlink d1/d16/d2f/l11c 0 2024-03-04T13:43:00.356 INFO:tasks.workunit.client.1.smithi169.stdout:1/628: dwrite d4/d5/d2d/d4e/d71/d26/d54/f72 [0,4194304] 0 2024-03-04T13:43:00.356 INFO:tasks.workunit.client.1.smithi169.stdout:2/718: write d1/f51 [602747,127560] 0 2024-03-04T13:43:00.357 INFO:tasks.workunit.client.1.smithi169.stdout:5/800: write d1/f4a [1451419,11526] 0 2024-03-04T13:43:00.358 INFO:tasks.workunit.client.0.smithi107.stdout:9/162: dread d4/ff [0,4194304] 0 2024-03-04T13:43:00.363 INFO:tasks.workunit.client.0.smithi107.stdout:4/135: rmdir d0 39 2024-03-04T13:43:00.364 INFO:tasks.workunit.client.1.smithi169.stdout:9/743: creat d7/d2d/d48/d66/f10e x:0 0 0 2024-03-04T13:43:00.366 INFO:tasks.workunit.client.0.smithi107.stdout:0/212: write da/dc/dd/d3e/f1c [1518032,93832] 0 2024-03-04T13:43:00.369 INFO:tasks.workunit.client.1.smithi169.stdout:3/698: creat d7/de/d2b/d7d/d85/d13/d1a/d98/d9a/fec x:0 0 0 2024-03-04T13:43:00.370 INFO:tasks.workunit.client.1.smithi169.stdout:4/648: mkdir d4/d11/d2e/dcd 0 2024-03-04T13:43:00.370 INFO:tasks.workunit.client.0.smithi107.stdout:2/182: rename da/l28 to da/d34/l3b 0 2024-03-04T13:43:00.375 INFO:tasks.workunit.client.0.smithi107.stdout:8/216: sync 2024-03-04T13:43:00.375 INFO:tasks.workunit.client.0.smithi107.stdout:0/213: truncate da/dc/dd/f24 4936942 0 2024-03-04T13:43:00.376 INFO:tasks.workunit.client.0.smithi107.stdout:1/149: dwrite d1/d10/d16/d1b/f1e [0,4194304] 0 2024-03-04T13:43:00.376 INFO:tasks.workunit.client.0.smithi107.stdout:7/145: sync 2024-03-04T13:43:00.380 INFO:tasks.workunit.client.1.smithi169.stdout:5/801: creat d1/d23/f11d x:0 0 0 2024-03-04T13:43:00.384 INFO:tasks.workunit.client.1.smithi169.stdout:7/742: dwrite d2/de/d3e/d41/dec/ff4 [0,4194304] 0 2024-03-04T13:43:00.386 INFO:tasks.workunit.client.0.smithi107.stdout:9/163: dread d4/da/d11/f13 [0,4194304] 0 2024-03-04T13:43:00.389 INFO:tasks.workunit.client.1.smithi169.stdout:1/629: creat d4/d5/d2d/d4e/d71/d82/fcb x:0 0 0 2024-03-04T13:43:00.389 INFO:tasks.workunit.client.1.smithi169.stdout:2/719: dread - d1/d5/d19/d3c/db8/de1/d64/ff4 zero size 2024-03-04T13:43:00.390 INFO:tasks.workunit.client.0.smithi107.stdout:6/204: dwrite df/d1b/f37 [0,4194304] 0 2024-03-04T13:43:00.391 INFO:tasks.workunit.client.1.smithi169.stdout:9/744: symlink d7/d27/l10f 0 2024-03-04T13:43:00.394 INFO:tasks.workunit.client.1.smithi169.stdout:1/630: chown d4/d5/d36/da9/lb7 3479260 1 2024-03-04T13:43:00.395 INFO:tasks.workunit.client.0.smithi107.stdout:4/136: creat d0/d9/de/d14/dc/f26 x:0 0 0 2024-03-04T13:43:00.395 INFO:tasks.workunit.client.0.smithi107.stdout:6/205: chown df/d17/d24/l2a 3657 1 2024-03-04T13:43:00.398 INFO:tasks.workunit.client.0.smithi107.stdout:2/183: dwrite f2 [0,4194304] 0 2024-03-04T13:43:00.402 INFO:tasks.workunit.client.0.smithi107.stdout:1/150: symlink d1/d10/l2b 0 2024-03-04T13:43:00.405 INFO:tasks.workunit.client.1.smithi169.stdout:3/699: read d7/de/d2b/d7d/d85/d18/f30 [394600,90317] 0 2024-03-04T13:43:00.408 INFO:tasks.workunit.client.0.smithi107.stdout:7/146: symlink d6/d13/d16/d19/l2e 0 2024-03-04T13:43:00.413 INFO:tasks.workunit.client.1.smithi169.stdout:6/691: creat d0/d3c/d95/d9e/ffb x:0 0 0 2024-03-04T13:43:00.417 INFO:tasks.workunit.client.1.smithi169.stdout:5/802: dread - d1/d16/d17/d2d/f9a zero size 2024-03-04T13:43:00.419 INFO:tasks.workunit.client.0.smithi107.stdout:2/184: unlink da/df/d27/c32 0 2024-03-04T13:43:00.421 INFO:tasks.workunit.client.1.smithi169.stdout:2/720: mkdir d1/d5/d22/d45/d4a/de7/dcb/df7 0 2024-03-04T13:43:00.421 INFO:tasks.workunit.client.0.smithi107.stdout:1/151: creat d1/d10/d20/f2c x:0 0 0 2024-03-04T13:43:00.422 INFO:tasks.workunit.client.0.smithi107.stdout:1/152: dread - d1/d10/d20/f2c zero size 2024-03-04T13:43:00.424 INFO:tasks.workunit.client.1.smithi169.stdout:3/700: fdatasync d7/de/d2b/d7d/d85/d47/d63/d6c/f77 0 2024-03-04T13:43:00.425 INFO:tasks.workunit.client.1.smithi169.stdout:4/649: write d4/d11/d3b/d67/d7e/f82 [1069034,87215] 0 2024-03-04T13:43:00.426 INFO:tasks.workunit.client.0.smithi107.stdout:8/217: creat d6/da/d25/d2d/d3e/f40 x:0 0 0 2024-03-04T13:43:00.427 INFO:tasks.workunit.client.0.smithi107.stdout:9/164: symlink d4/da/d11/d20/d23/d2c/l2d 0 2024-03-04T13:43:00.429 INFO:tasks.workunit.client.1.smithi169.stdout:4/650: write d4/d1c/d75/d81/f85 [1242183,101276] 0 2024-03-04T13:43:00.430 INFO:tasks.workunit.client.1.smithi169.stdout:0/551: truncate d5/d9/d3d/d59/d64/d7f/fa9 735012 0 2024-03-04T13:43:00.433 INFO:tasks.workunit.client.0.smithi107.stdout:4/137: link d0/d9/d10/f1c d0/d9/de/d14/dc/f27 0 2024-03-04T13:43:00.435 INFO:tasks.workunit.client.1.smithi169.stdout:5/803: rmdir d1/d76 39 2024-03-04T13:43:00.439 INFO:tasks.workunit.client.0.smithi107.stdout:7/147: write d6/fc [3889627,109638] 0 2024-03-04T13:43:00.441 INFO:tasks.workunit.client.0.smithi107.stdout:1/153: write d1/fe [4980769,91736] 0 2024-03-04T13:43:00.442 INFO:tasks.workunit.client.0.smithi107.stdout:7/148: dread - d6/d13/d16/d1c/f2d zero size 2024-03-04T13:43:00.444 INFO:tasks.workunit.client.1.smithi169.stdout:2/721: truncate d1/d5/d19/d68/fa7 4728049 0 2024-03-04T13:43:00.447 INFO:tasks.workunit.client.0.smithi107.stdout:0/214: link da/dc/dd/d3e/f11 da/dc/dd/d3e/f44 0 2024-03-04T13:43:00.447 INFO:tasks.workunit.client.0.smithi107.stdout:2/185: write f0 [469123,52525] 0 2024-03-04T13:43:00.449 INFO:tasks.workunit.client.1.smithi169.stdout:9/745: write d7/dc/d12/d59/f10c [228729,106547] 0 2024-03-04T13:43:00.450 INFO:tasks.workunit.client.1.smithi169.stdout:6/692: readlink d0/d32/d3e/l90 0 2024-03-04T13:43:00.451 INFO:tasks.workunit.client.0.smithi107.stdout:4/138: rmdir d0/d9/d1e 39 2024-03-04T13:43:00.453 INFO:tasks.workunit.client.1.smithi169.stdout:8/680: rmdir d7/d10/d24/d3e/dda 0 2024-03-04T13:43:00.455 INFO:tasks.workunit.client.1.smithi169.stdout:4/651: creat d4/d11/d2e/d6c/d78/da5/fce x:0 0 0 2024-03-04T13:43:00.457 INFO:tasks.workunit.client.1.smithi169.stdout:0/552: mkdir d5/d9/dc/d12/d15/d46/d98/dc1 0 2024-03-04T13:43:00.458 INFO:tasks.workunit.client.0.smithi107.stdout:8/218: chown d6/d16/d1f/f3a 163567 1 2024-03-04T13:43:00.459 INFO:tasks.workunit.client.1.smithi169.stdout:5/804: unlink d1/f7d 0 2024-03-04T13:43:00.461 INFO:tasks.workunit.client.0.smithi107.stdout:7/149: symlink d6/d13/d16/d19/d1e/l2f 0 2024-03-04T13:43:00.461 INFO:tasks.workunit.client.1.smithi169.stdout:0/553: write d5/d9/d3d/d59/d64/f81 [75429,89839] 0 2024-03-04T13:43:00.462 INFO:tasks.workunit.client.1.smithi169.stdout:5/805: dread - d1/d16/d17/f109 zero size 2024-03-04T13:43:00.464 INFO:tasks.workunit.client.0.smithi107.stdout:6/206: getdents df/d1b/d2c/d31 0 2024-03-04T13:43:00.465 INFO:tasks.workunit.client.0.smithi107.stdout:7/150: chown d6/d13/l15 0 1 2024-03-04T13:43:00.466 INFO:tasks.workunit.client.0.smithi107.stdout:6/207: dread - df/f19 zero size 2024-03-04T13:43:00.466 INFO:tasks.workunit.client.1.smithi169.stdout:9/746: creat d7/d27/f110 x:0 0 0 2024-03-04T13:43:00.468 INFO:tasks.workunit.client.0.smithi107.stdout:6/208: fsync df/d1b/d2c/d31/f39 0 2024-03-04T13:43:00.469 INFO:tasks.workunit.client.0.smithi107.stdout:2/186: dwrite da/f18 [4194304,4194304] 0 2024-03-04T13:43:00.471 INFO:tasks.workunit.client.1.smithi169.stdout:8/681: dwrite d7/d10/d17/d44/f4f [0,4194304] 0 2024-03-04T13:43:00.472 INFO:tasks.workunit.client.0.smithi107.stdout:1/154: creat d1/d10/d20/d24/f2d x:0 0 0 2024-03-04T13:43:00.472 INFO:tasks.workunit.client.0.smithi107.stdout:8/219: readlink d6/l7 0 2024-03-04T13:43:00.473 INFO:tasks.workunit.client.1.smithi169.stdout:2/722: dread d1/d5/d22/d45/f89 [0,4194304] 0 2024-03-04T13:43:00.475 INFO:tasks.workunit.client.0.smithi107.stdout:4/139: dread d0/d9/d10/f25 [0,4194304] 0 2024-03-04T13:43:00.476 INFO:tasks.workunit.client.0.smithi107.stdout:0/215: unlink da/f27 0 2024-03-04T13:43:00.482 INFO:tasks.workunit.client.1.smithi169.stdout:4/652: mknod d4/d1c/d58/d9d/ccf 0 2024-03-04T13:43:00.482 INFO:tasks.workunit.client.0.smithi107.stdout:9/165: truncate d4/f8 5706932 0 2024-03-04T13:43:00.485 INFO:tasks.workunit.client.1.smithi169.stdout:5/806: rmdir d1/d16/dbd/dbb 39 2024-03-04T13:43:00.487 INFO:tasks.workunit.client.0.smithi107.stdout:7/151: symlink d6/d17/l30 0 2024-03-04T13:43:00.489 INFO:tasks.workunit.client.1.smithi169.stdout:0/554: symlink d5/d9/dc/d12/d66/lc2 0 2024-03-04T13:43:00.490 INFO:tasks.workunit.client.0.smithi107.stdout:8/220: mknod d6/d16/c41 0 2024-03-04T13:43:00.491 INFO:tasks.workunit.client.1.smithi169.stdout:9/747: mkdir d7/dc/d12/d20/da6/d111 0 2024-03-04T13:43:00.493 INFO:tasks.workunit.client.1.smithi169.stdout:6/693: rename d0/d3c/d40/ff0 to d0/d32/d3a/d5e/ffc 0 2024-03-04T13:43:00.494 INFO:tasks.workunit.client.1.smithi169.stdout:6/694: write d0/d32/fe8 [439063,38639] 0 2024-03-04T13:43:00.495 INFO:tasks.workunit.client.0.smithi107.stdout:0/216: creat da/dc/d34/f45 x:0 0 0 2024-03-04T13:43:00.496 INFO:tasks.workunit.client.1.smithi169.stdout:4/653: truncate d4/d1c/d58/faf 429160 0 2024-03-04T13:43:00.498 INFO:tasks.workunit.client.1.smithi169.stdout:1/631: link d4/f20 d4/d5/d36/d35/d5b/fcc 0 2024-03-04T13:43:00.500 INFO:tasks.workunit.client.1.smithi169.stdout:3/701: getdents d7/d53 0 2024-03-04T13:43:00.500 INFO:tasks.workunit.client.0.smithi107.stdout:6/209: getdents df/d17/d24/d40 0 2024-03-04T13:43:00.500 INFO:tasks.workunit.client.0.smithi107.stdout:9/166: read d4/da/f1b [54147,83979] 0 2024-03-04T13:43:00.502 INFO:tasks.workunit.client.1.smithi169.stdout:4/654: truncate d4/d11/d2e/f51 4739897 0 2024-03-04T13:43:00.503 INFO:tasks.workunit.client.1.smithi169.stdout:4/655: readlink d4/le 0 2024-03-04T13:43:00.504 INFO:tasks.workunit.client.1.smithi169.stdout:0/555: mkdir d5/d9/dc/d12/d66/dc3 0 2024-03-04T13:43:00.505 INFO:tasks.workunit.client.1.smithi169.stdout:4/656: fsync d4/d1c/f2c 0 2024-03-04T13:43:00.505 INFO:tasks.workunit.client.0.smithi107.stdout:7/152: mkdir d6/d13/d31 0 2024-03-04T13:43:00.513 INFO:tasks.workunit.client.0.smithi107.stdout:3/142: sync 2024-03-04T13:43:00.513 INFO:tasks.workunit.client.0.smithi107.stdout:5/137: sync 2024-03-04T13:43:00.513 INFO:tasks.workunit.client.1.smithi169.stdout:2/723: creat d1/d5/d22/d3b/d41/dbb/ff8 x:0 0 0 2024-03-04T13:43:00.515 INFO:tasks.workunit.client.0.smithi107.stdout:9/167: dread d4/f24 [0,4194304] 0 2024-03-04T13:43:00.516 INFO:tasks.workunit.client.1.smithi169.stdout:8/682: dread d7/d10/d17/d44/d4c/f7d [0,4194304] 0 2024-03-04T13:43:00.516 INFO:tasks.workunit.client.1.smithi169.stdout:6/695: mknod d0/d3c/d40/d52/d98/cfd 0 2024-03-04T13:43:00.517 INFO:tasks.workunit.client.0.smithi107.stdout:0/217: rmdir da/dc 39 2024-03-04T13:43:00.517 INFO:tasks.workunit.client.1.smithi169.stdout:2/724: read d1/d5/d19/d63/fbd [2008924,107738] 0 2024-03-04T13:43:00.518 INFO:tasks.workunit.client.0.smithi107.stdout:4/140: creat d0/d9/de/d14/dc/d21/f28 x:0 0 0 2024-03-04T13:43:00.519 INFO:tasks.workunit.client.1.smithi169.stdout:5/807: mknod d1/d16/d107/c11e 0 2024-03-04T13:43:00.519 INFO:tasks.workunit.client.1.smithi169.stdout:1/632: dwrite d4/d1d/f34 [0,4194304] 0 2024-03-04T13:43:00.524 INFO:tasks.workunit.client.1.smithi169.stdout:3/702: mkdir d7/db8/ded 0 2024-03-04T13:43:00.524 INFO:tasks.workunit.client.0.smithi107.stdout:2/187: link da/df/c1a da/d39/c3c 0 2024-03-04T13:43:00.529 INFO:tasks.workunit.client.0.smithi107.stdout:8/221: truncate d6/d16/f1c 1570855 0 2024-03-04T13:43:00.530 INFO:tasks.workunit.client.0.smithi107.stdout:7/153: creat d6/d1d/f32 x:0 0 0 2024-03-04T13:43:00.533 INFO:tasks.workunit.client.1.smithi169.stdout:8/683: chown d7/d10/d22/d75/d81/cc5 770884232 1 2024-03-04T13:43:00.535 INFO:tasks.workunit.client.0.smithi107.stdout:1/155: link d1/d10/l2b d1/d10/d20/l2e 0 2024-03-04T13:43:00.536 INFO:tasks.workunit.client.0.smithi107.stdout:5/138: creat d5/dc/df/f2e x:0 0 0 2024-03-04T13:43:00.539 INFO:tasks.workunit.client.0.smithi107.stdout:1/156: write d1/d10/d20/f2c [518326,31982] 0 2024-03-04T13:43:00.540 INFO:tasks.workunit.client.1.smithi169.stdout:5/808: mknod d1/d16/d2f/d72/dab/c11f 0 2024-03-04T13:43:00.543 INFO:tasks.workunit.client.0.smithi107.stdout:1/157: write d1/d10/d20/f2c [646133,74315] 0 2024-03-04T13:43:00.544 INFO:tasks.workunit.client.0.smithi107.stdout:1/158: chown d1/d10 178 1 2024-03-04T13:43:00.546 INFO:tasks.workunit.client.0.smithi107.stdout:4/141: mkdir d0/d9/d18/d29 0 2024-03-04T13:43:00.547 INFO:tasks.workunit.client.0.smithi107.stdout:1/159: chown d1/d3/c7 217590137 1 2024-03-04T13:43:00.547 INFO:tasks.workunit.client.1.smithi169.stdout:9/748: dwrite d7/dc/d12/d32/d42/f46 [4194304,4194304] 0 2024-03-04T13:43:00.551 INFO:tasks.workunit.client.0.smithi107.stdout:3/143: dread d1/de/f1d [0,4194304] 0 2024-03-04T13:43:00.551 INFO:tasks.workunit.client.0.smithi107.stdout:1/160: write d1/d10/d16/d1b/f23 [245920,55902] 0 2024-03-04T13:43:00.552 INFO:tasks.workunit.client.1.smithi169.stdout:8/684: dwrite d7/d10/d22/d75/db9/fba [4194304,4194304] 0 2024-03-04T13:43:00.552 INFO:tasks.workunit.client.1.smithi169.stdout:3/703: rename d7/de/d2b/d7d/d85/d18/d21/d3f/d68/l55 to d7/db8/ded/lee 0 2024-03-04T13:43:00.554 INFO:tasks.workunit.client.1.smithi169.stdout:3/704: stat d7/de/d2b/d7d/d85/d18/d21/d66/lcf 0 2024-03-04T13:43:00.555 INFO:tasks.workunit.client.0.smithi107.stdout:2/188: mknod da/df/c3d 0 2024-03-04T13:43:00.557 INFO:tasks.workunit.client.0.smithi107.stdout:2/189: chown da/d34/f36 63723 1 2024-03-04T13:43:00.559 INFO:tasks.workunit.client.1.smithi169.stdout:1/633: write d4/d5/d36/f67 [5158719,91802] 0 2024-03-04T13:43:00.560 INFO:tasks.workunit.client.0.smithi107.stdout:7/154: truncate d6/d13/d16/d1c/f1f 2835271 0 2024-03-04T13:43:00.561 INFO:tasks.workunit.client.1.smithi169.stdout:0/556: link d5/d9/dc/d12/f5c d5/d9/d3d/d59/d64/d7f/fc4 0 2024-03-04T13:43:00.567 INFO:tasks.workunit.client.1.smithi169.stdout:5/809: mkdir d1/d16/d17/d28/dfe/d120 0 2024-03-04T13:43:00.567 INFO:tasks.workunit.client.0.smithi107.stdout:9/168: symlink d4/da/l2e 0 2024-03-04T13:43:00.568 INFO:tasks.workunit.client.1.smithi169.stdout:9/749: symlink d7/d2d/d48/d66/dec/l112 0 2024-03-04T13:43:00.570 INFO:tasks.workunit.client.1.smithi169.stdout:6/696: dwrite d0/f3b [0,4194304] 0 2024-03-04T13:43:00.571 INFO:tasks.workunit.client.1.smithi169.stdout:8/685: mknod d7/d10/d17/d44/d36/d62/dc1/cde 0 2024-03-04T13:43:00.573 INFO:tasks.workunit.client.1.smithi169.stdout:2/725: dwrite d1/d5/d19/d3c/db8/de1/f60 [0,4194304] 0 2024-03-04T13:43:00.573 INFO:tasks.workunit.client.0.smithi107.stdout:3/144: dwrite d1/f3 [0,4194304] 0 2024-03-04T13:43:00.573 INFO:tasks.workunit.client.0.smithi107.stdout:6/210: creat df/f43 x:0 0 0 2024-03-04T13:43:00.576 INFO:tasks.workunit.client.1.smithi169.stdout:0/557: symlink d5/d9/dc/d12/d66/lc5 0 2024-03-04T13:43:00.577 INFO:tasks.workunit.client.1.smithi169.stdout:8/686: write d7/d10/d22/d46/da5/da6/fb4 [329864,29573] 0 2024-03-04T13:43:00.577 INFO:tasks.workunit.client.1.smithi169.stdout:1/634: chown d4/d5/d36/d35/d16/f4d 8902 1 2024-03-04T13:43:00.578 INFO:tasks.workunit.client.1.smithi169.stdout:6/697: read d0/d32/d3a/d43/d74/d93/dbe/fec [283903,78667] 0 2024-03-04T13:43:00.579 INFO:tasks.workunit.client.0.smithi107.stdout:2/190: mkdir da/df/d3e 0 2024-03-04T13:43:00.582 INFO:tasks.workunit.client.0.smithi107.stdout:7/155: dwrite f4 [4194304,4194304] 0 2024-03-04T13:43:00.586 INFO:tasks.workunit.client.0.smithi107.stdout:5/139: rename d5/d8/l1b to d5/d8/d2a/l2f 0 2024-03-04T13:43:00.589 INFO:tasks.workunit.client.1.smithi169.stdout:4/657: truncate d4/d11/d2e/d63/fc3 2436025 0 2024-03-04T13:43:00.592 INFO:tasks.workunit.client.0.smithi107.stdout:5/140: read d5/f9 [4557878,64814] 0 2024-03-04T13:43:00.594 INFO:tasks.workunit.client.1.smithi169.stdout:2/726: rename d1/la8 to d1/lf9 0 2024-03-04T13:43:00.600 INFO:tasks.workunit.client.1.smithi169.stdout:7/743: sync 2024-03-04T13:43:00.604 INFO:tasks.workunit.client.1.smithi169.stdout:0/558: read d5/d9/dc/d12/f24 [1841019,109139] 0 2024-03-04T13:43:00.609 INFO:tasks.workunit.client.0.smithi107.stdout:9/169: dwrite d4/ff [0,4194304] 0 2024-03-04T13:43:00.612 INFO:tasks.workunit.client.1.smithi169.stdout:9/750: mknod d7/c113 0 2024-03-04T13:43:00.614 INFO:tasks.workunit.client.1.smithi169.stdout:4/658: dwrite d4/f7 [0,4194304] 0 2024-03-04T13:43:00.616 INFO:tasks.workunit.client.0.smithi107.stdout:5/141: dwrite d5/dc/d22/f26 [0,4194304] 0 2024-03-04T13:43:00.616 INFO:tasks.workunit.client.1.smithi169.stdout:6/698: creat d0/d3c/d95/ffe x:0 0 0 2024-03-04T13:43:00.618 INFO:tasks.workunit.client.0.smithi107.stdout:3/145: write d1/d29/f2a [801926,96458] 0 2024-03-04T13:43:00.620 INFO:tasks.workunit.client.0.smithi107.stdout:6/211: write df/f29 [4067312,72016] 0 2024-03-04T13:43:00.628 INFO:tasks.workunit.client.0.smithi107.stdout:7/156: rename d6/fc to d6/d1d/f33 0 2024-03-04T13:43:00.629 INFO:tasks.workunit.client.1.smithi169.stdout:8/687: dwrite d7/d10/d24/d3e/f57 [0,4194304] 0 2024-03-04T13:43:00.632 INFO:tasks.workunit.client.0.smithi107.stdout:8/222: truncate f5 6431610 0 2024-03-04T13:43:00.632 INFO:tasks.workunit.client.1.smithi169.stdout:8/688: chown d7/d10/l82 1833422 1 2024-03-04T13:43:00.633 INFO:tasks.workunit.client.1.smithi169.stdout:1/635: dwrite d4/d5/d36/d35/fa2 [0,4194304] 0 2024-03-04T13:43:00.633 INFO:tasks.workunit.client.1.smithi169.stdout:2/727: creat d1/d5/d22/d3b/ffa x:0 0 0 2024-03-04T13:43:00.635 INFO:tasks.workunit.client.0.smithi107.stdout:7/157: truncate d6/d13/d16/d19/f26 807675 0 2024-03-04T13:43:00.635 INFO:tasks.workunit.client.1.smithi169.stdout:5/810: dread d1/d16/d2f/faa [0,4194304] 0 2024-03-04T13:43:00.636 INFO:tasks.workunit.client.0.smithi107.stdout:7/158: stat f0 0 2024-03-04T13:43:00.638 INFO:tasks.workunit.client.1.smithi169.stdout:7/744: mknod d2/de/d3e/d41/d5b/d7a/cf8 0 2024-03-04T13:43:00.640 INFO:tasks.workunit.client.0.smithi107.stdout:4/142: creat d0/f2a x:0 0 0 2024-03-04T13:43:00.647 INFO:tasks.workunit.client.0.smithi107.stdout:9/170: rmdir d4/da 39 2024-03-04T13:43:00.651 INFO:tasks.workunit.client.1.smithi169.stdout:3/705: getdents d7/de/d2b/d7d/d85/d18/d21/d3f/d86 0 2024-03-04T13:43:00.651 INFO:tasks.workunit.client.0.smithi107.stdout:2/191: mknod da/c3f 0 2024-03-04T13:43:00.651 INFO:tasks.workunit.client.0.smithi107.stdout:0/218: truncate da/dc/dd/d3e/f1c 1026157 0 2024-03-04T13:43:00.653 INFO:tasks.workunit.client.0.smithi107.stdout:1/161: truncate d1/d10/d16/d1b/f23 862859 0 2024-03-04T13:43:00.654 INFO:tasks.workunit.client.0.smithi107.stdout:3/146: creat d1/d1e/f2f x:0 0 0 2024-03-04T13:43:00.655 INFO:tasks.workunit.client.1.smithi169.stdout:9/751: creat d7/d2d/d48/d66/f114 x:0 0 0 2024-03-04T13:43:00.659 INFO:tasks.workunit.client.1.smithi169.stdout:4/659: creat d4/d1c/d58/d9d/fd0 x:0 0 0 2024-03-04T13:43:00.659 INFO:tasks.workunit.client.0.smithi107.stdout:7/159: dwrite d6/f12 [0,4194304] 0 2024-03-04T13:43:00.661 INFO:tasks.workunit.client.1.smithi169.stdout:1/636: dwrite d4/d5/d2d/d4e/fa3 [0,4194304] 0 2024-03-04T13:43:00.665 INFO:tasks.workunit.client.0.smithi107.stdout:8/223: write d6/fe [4284459,80444] 0 2024-03-04T13:43:00.667 INFO:tasks.workunit.client.0.smithi107.stdout:8/224: fdatasync d6/d16/f24 0 2024-03-04T13:43:00.668 INFO:tasks.workunit.client.0.smithi107.stdout:8/225: chown d6/d16/d1f/f34 1 1 2024-03-04T13:43:00.676 INFO:tasks.workunit.client.1.smithi169.stdout:5/811: rmdir d1/d16/d17/d2d/d5a/d67 39 2024-03-04T13:43:00.686 INFO:tasks.workunit.client.1.smithi169.stdout:6/699: creat d0/d32/d3a/def/fff x:0 0 0 2024-03-04T13:43:00.690 INFO:tasks.workunit.client.0.smithi107.stdout:1/162: mknod d1/d3/d17/c2f 0 2024-03-04T13:43:00.690 INFO:tasks.workunit.client.0.smithi107.stdout:3/147: dwrite d1/d1e/f2f [0,4194304] 0 2024-03-04T13:43:00.692 INFO:tasks.workunit.client.1.smithi169.stdout:5/812: unlink d1/fc0 0 2024-03-04T13:43:00.693 INFO:tasks.workunit.client.0.smithi107.stdout:6/212: creat df/d17/d24/d40/f44 x:0 0 0 2024-03-04T13:43:00.699 INFO:tasks.workunit.client.0.smithi107.stdout:6/213: write df/f29 [2348177,98200] 0 2024-03-04T13:43:00.700 INFO:tasks.workunit.client.1.smithi169.stdout:0/559: link d5/d9/dc/d12/d15/d19/d77/db2/cb7 d5/d9/dc/d12/d15/d46/d98/dc1/cc6 0 2024-03-04T13:43:00.701 INFO:tasks.workunit.client.0.smithi107.stdout:2/192: creat da/df/d3e/f40 x:0 0 0 2024-03-04T13:43:00.701 INFO:tasks.workunit.client.0.smithi107.stdout:5/142: dread d5/f6 [0,4194304] 0 2024-03-04T13:43:00.702 INFO:tasks.workunit.client.1.smithi169.stdout:4/660: unlink d4/d1c/d58/faf 0 2024-03-04T13:43:00.705 INFO:tasks.workunit.client.0.smithi107.stdout:0/219: creat da/dc/dd/f46 x:0 0 0 2024-03-04T13:43:00.706 INFO:tasks.workunit.client.0.smithi107.stdout:3/148: mknod d1/de/d11/d12/c30 0 2024-03-04T13:43:00.707 INFO:tasks.workunit.client.1.smithi169.stdout:3/706: dread d7/de/d24/f8f [0,4194304] 0 2024-03-04T13:43:00.709 INFO:tasks.workunit.client.0.smithi107.stdout:9/171: dwrite d4/da/d11/f17 [0,4194304] 0 2024-03-04T13:43:00.710 INFO:tasks.workunit.client.1.smithi169.stdout:6/700: truncate d0/d3c/d40/df6/dca/ff1 982636 0 2024-03-04T13:43:00.711 INFO:tasks.workunit.client.1.smithi169.stdout:2/728: creat d1/d5/d19/d1e/ffb x:0 0 0 2024-03-04T13:43:00.714 INFO:tasks.workunit.client.1.smithi169.stdout:1/637: mkdir d4/dcd 0 2024-03-04T13:43:00.716 INFO:tasks.workunit.client.1.smithi169.stdout:2/729: read d1/d5/d22/d3b/f4d [175433,114473] 0 2024-03-04T13:43:00.716 INFO:tasks.workunit.client.0.smithi107.stdout:6/214: fsync df/d17/f1e 0 2024-03-04T13:43:00.717 INFO:tasks.workunit.client.1.smithi169.stdout:8/689: dread d7/d10/d22/f25 [0,4194304] 0 2024-03-04T13:43:00.719 INFO:tasks.workunit.client.1.smithi169.stdout:5/813: truncate d1/d60/fa0 924100 0 2024-03-04T13:43:00.722 INFO:tasks.workunit.client.0.smithi107.stdout:5/143: truncate d5/d8/fb 1918708 0 2024-03-04T13:43:00.727 INFO:tasks.workunit.client.1.smithi169.stdout:4/661: rmdir d4/d11/d2e/d39/d61 39 2024-03-04T13:43:00.729 INFO:tasks.workunit.client.0.smithi107.stdout:3/149: unlink d1/d1e/f2f 0 2024-03-04T13:43:00.730 INFO:tasks.workunit.client.1.smithi169.stdout:7/745: dwrite d2/de/d3e/d41/f49 [4194304,4194304] 0 2024-03-04T13:43:00.730 INFO:tasks.workunit.client.1.smithi169.stdout:4/662: truncate d4/d11/f1b 1513411 0 2024-03-04T13:43:00.731 INFO:tasks.workunit.client.0.smithi107.stdout:7/160: dread d6/d13/d16/d1c/f1f [0,4194304] 0 2024-03-04T13:43:00.733 INFO:tasks.workunit.client.0.smithi107.stdout:6/215: dwrite f8 [0,4194304] 0 2024-03-04T13:43:00.733 INFO:tasks.workunit.client.1.smithi169.stdout:9/752: dwrite d7/dc/f23 [0,4194304] 0 2024-03-04T13:43:00.735 INFO:tasks.workunit.client.1.smithi169.stdout:7/746: dread - d2/de/d3e/d41/fe8 zero size 2024-03-04T13:43:00.735 INFO:tasks.workunit.client.0.smithi107.stdout:6/216: readlink df/d1b/d2c/l3a 0 2024-03-04T13:43:00.735 INFO:tasks.workunit.client.0.smithi107.stdout:2/193: rename da/df/f37 to da/d34/f41 0 2024-03-04T13:43:00.736 INFO:tasks.workunit.client.1.smithi169.stdout:3/707: symlink d7/de/d2b/d7d/d85/d18/d21/d3f/d86/lef 0 2024-03-04T13:43:00.737 INFO:tasks.workunit.client.0.smithi107.stdout:6/217: write df/d1b/d2c/d31/f39 [31901,21477] 0 2024-03-04T13:43:00.739 INFO:tasks.workunit.client.0.smithi107.stdout:6/218: dread - df/d17/d24/d40/f44 zero size 2024-03-04T13:43:00.745 INFO:tasks.workunit.client.0.smithi107.stdout:7/161: dread f5 [0,4194304] 0 2024-03-04T13:43:00.746 INFO:tasks.workunit.client.0.smithi107.stdout:6/219: truncate df/f43 192970 0 2024-03-04T13:43:00.747 INFO:tasks.workunit.client.0.smithi107.stdout:9/172: mkdir d4/da/d11/d20/d2f 0 2024-03-04T13:43:00.747 INFO:tasks.workunit.client.0.smithi107.stdout:6/220: stat df/l22 0 2024-03-04T13:43:00.749 INFO:tasks.workunit.client.0.smithi107.stdout:4/143: link d0/d9/de/d14/dc/f26 d0/d9/d1e/f2b 0 2024-03-04T13:43:00.750 INFO:tasks.workunit.client.1.smithi169.stdout:5/814: dwrite d1/d16/d17/d1b/f29 [0,4194304] 0 2024-03-04T13:43:00.751 INFO:tasks.workunit.client.0.smithi107.stdout:5/144: mknod d5/dc/d22/c30 0 2024-03-04T13:43:00.754 INFO:tasks.workunit.client.1.smithi169.stdout:2/730: truncate d1/d5/d19/d3c/db8/de1/fe6 72677 0 2024-03-04T13:43:00.755 INFO:tasks.workunit.client.0.smithi107.stdout:5/145: truncate d5/d8/da/f16 790502 0 2024-03-04T13:43:00.757 INFO:tasks.workunit.client.1.smithi169.stdout:2/731: fsync d1/d5/d22/d3b/ffa 0 2024-03-04T13:43:00.757 INFO:tasks.workunit.client.0.smithi107.stdout:3/150: creat d1/d1e/f31 x:0 0 0 2024-03-04T13:43:00.758 INFO:tasks.workunit.client.0.smithi107.stdout:0/220: mkdir da/dc/dd/d3e/d47 0 2024-03-04T13:43:00.759 INFO:tasks.workunit.client.1.smithi169.stdout:2/732: dread - d1/d5/d19/d3c/db8/de1/d64/ff4 zero size 2024-03-04T13:43:00.764 INFO:tasks.workunit.client.0.smithi107.stdout:1/163: fsync d1/d10/d16/d1b/f23 0 2024-03-04T13:43:00.767 INFO:tasks.workunit.client.0.smithi107.stdout:2/194: symlink da/d34/l42 0 2024-03-04T13:43:00.771 INFO:tasks.workunit.client.0.smithi107.stdout:6/221: readlink df/d1b/l3c 0 2024-03-04T13:43:00.774 INFO:tasks.workunit.client.1.smithi169.stdout:4/663: dread d4/d11/d3b/d67/dbb/d6e/f92 [0,4194304] 0 2024-03-04T13:43:00.775 INFO:tasks.workunit.client.0.smithi107.stdout:3/151: dwrite d1/d1e/f31 [0,4194304] 0 2024-03-04T13:43:00.775 INFO:tasks.workunit.client.1.smithi169.stdout:4/664: chown d4/d11 2057170372 1 2024-03-04T13:43:00.777 INFO:tasks.workunit.client.0.smithi107.stdout:9/173: creat d4/da/d11/d20/f30 x:0 0 0 2024-03-04T13:43:00.780 INFO:tasks.workunit.client.0.smithi107.stdout:4/144: creat d0/d9/de/d14/dc/d21/f2c x:0 0 0 2024-03-04T13:43:00.781 INFO:tasks.workunit.client.1.smithi169.stdout:5/815: fsync d1/fb0 0 2024-03-04T13:43:00.785 INFO:tasks.workunit.client.0.smithi107.stdout:5/146: dread d5/d8/f17 [0,4194304] 0 2024-03-04T13:43:00.785 INFO:tasks.workunit.client.1.smithi169.stdout:8/690: write d7/d10/d24/d5c/f90 [1060824,123082] 0 2024-03-04T13:43:00.785 INFO:tasks.workunit.client.1.smithi169.stdout:6/701: creat d0/f100 x:0 0 0 2024-03-04T13:43:00.792 INFO:tasks.workunit.client.0.smithi107.stdout:8/226: sync 2024-03-04T13:43:00.794 INFO:tasks.workunit.client.1.smithi169.stdout:1/638: unlink d4/d5/d36/d35/df/fc2 0 2024-03-04T13:43:00.798 INFO:tasks.workunit.client.1.smithi169.stdout:0/560: getdents d5/d9/dc 0 2024-03-04T13:43:00.798 INFO:tasks.workunit.client.0.smithi107.stdout:5/147: dwrite d5/f29 [0,4194304] 0 2024-03-04T13:43:00.798 INFO:tasks.workunit.client.0.smithi107.stdout:7/162: symlink d6/d1d/d28/d2c/l34 0 2024-03-04T13:43:00.801 INFO:tasks.workunit.client.0.smithi107.stdout:7/163: read - d6/d1d/f29 zero size 2024-03-04T13:43:00.801 INFO:tasks.workunit.client.1.smithi169.stdout:5/816: dwrite d1/d16/d2f/f33 [0,4194304] 0 2024-03-04T13:43:00.804 INFO:tasks.workunit.client.0.smithi107.stdout:5/148: dread d5/d8/f17 [0,4194304] 0 2024-03-04T13:43:00.804 INFO:tasks.workunit.client.0.smithi107.stdout:9/174: mkdir d4/da/d11/d20/d31 0 2024-03-04T13:43:00.808 INFO:tasks.workunit.client.0.smithi107.stdout:7/164: truncate d6/d13/d16/d1c/f2d 536177 0 2024-03-04T13:43:00.809 INFO:tasks.workunit.client.1.smithi169.stdout:1/639: dread d4/d1d/f40 [0,4194304] 0 2024-03-04T13:43:00.809 INFO:tasks.workunit.client.1.smithi169.stdout:8/691: mknod d7/d10/d17/d44/d36/cdf 0 2024-03-04T13:43:00.813 INFO:tasks.workunit.client.0.smithi107.stdout:0/221: mkdir da/dc/dd/d3e/d48 0 2024-03-04T13:43:00.815 INFO:tasks.workunit.client.1.smithi169.stdout:2/733: mknod d1/d5/d22/d45/d4a/de7/dcb/df7/cfc 0 2024-03-04T13:43:00.817 INFO:tasks.workunit.client.1.smithi169.stdout:3/708: dwrite d7/f70 [0,4194304] 0 2024-03-04T13:43:00.818 INFO:tasks.workunit.client.0.smithi107.stdout:7/165: dread f4 [4194304,4194304] 0 2024-03-04T13:43:00.819 INFO:tasks.workunit.client.0.smithi107.stdout:1/164: mknod d1/c30 0 2024-03-04T13:43:00.826 INFO:tasks.workunit.client.1.smithi169.stdout:8/692: dread d7/d10/d22/d75/d81/da2/dac/fb8 [0,4194304] 0 2024-03-04T13:43:00.828 INFO:tasks.workunit.client.0.smithi107.stdout:6/222: creat df/d2e/d36/f45 x:0 0 0 2024-03-04T13:43:00.829 INFO:tasks.workunit.client.1.smithi169.stdout:9/753: dwrite d7/dc/f80 [0,4194304] 0 2024-03-04T13:43:00.829 INFO:tasks.workunit.client.0.smithi107.stdout:4/145: write d0/d9/d1e/f2b [955936,46471] 0 2024-03-04T13:43:00.837 INFO:tasks.workunit.client.0.smithi107.stdout:1/165: dwrite d1/d10/d16/d1b/f1e [0,4194304] 0 2024-03-04T13:43:00.841 INFO:tasks.workunit.client.0.smithi107.stdout:6/223: dread df/d1b/f37 [0,4194304] 0 2024-03-04T13:43:00.842 INFO:tasks.workunit.client.0.smithi107.stdout:5/149: unlink d5/f21 0 2024-03-04T13:43:00.844 INFO:tasks.workunit.client.1.smithi169.stdout:7/747: getdents d2/de/d3e/d41/d89 0 2024-03-04T13:43:00.847 INFO:tasks.workunit.client.1.smithi169.stdout:1/640: creat d4/d5/d36/fce x:0 0 0 2024-03-04T13:43:00.850 INFO:tasks.workunit.client.0.smithi107.stdout:2/195: dwrite da/f1c [0,4194304] 0 2024-03-04T13:43:00.856 INFO:tasks.workunit.client.0.smithi107.stdout:7/166: rename f5 to d6/d13/d16/d1c/f35 0 2024-03-04T13:43:00.856 INFO:tasks.workunit.client.1.smithi169.stdout:3/709: chown d7/cd6 0 1 2024-03-04T13:43:00.857 INFO:tasks.workunit.client.0.smithi107.stdout:7/167: chown d6/d1d/f32 523907 1 2024-03-04T13:43:00.858 INFO:tasks.workunit.client.0.smithi107.stdout:5/150: dwrite d5/d8/f1f [0,4194304] 0 2024-03-04T13:43:00.863 INFO:tasks.workunit.client.1.smithi169.stdout:9/754: unlink d7/d27/f2f 0 2024-03-04T13:43:00.864 INFO:tasks.workunit.client.0.smithi107.stdout:3/152: link d1/de/f1d d1/de/f32 0 2024-03-04T13:43:00.865 INFO:tasks.workunit.client.1.smithi169.stdout:9/755: fdatasync d7/d2d/d48/d66/f10e 0 2024-03-04T13:43:00.866 INFO:tasks.workunit.client.1.smithi169.stdout:5/817: creat d1/d16/dbd/dbb/f121 x:0 0 0 2024-03-04T13:43:00.867 INFO:tasks.workunit.client.1.smithi169.stdout:7/748: truncate d2/d15/d17/fbf 904710 0 2024-03-04T13:43:00.870 INFO:tasks.workunit.client.0.smithi107.stdout:9/175: mkdir d4/d32 0 2024-03-04T13:43:00.872 INFO:tasks.workunit.client.0.smithi107.stdout:6/224: write df/f15 [2923256,8601] 0 2024-03-04T13:43:00.875 INFO:tasks.workunit.client.1.smithi169.stdout:3/710: dread d7/de/d2b/d7d/d85/d18/d21/d3f/f82 [0,4194304] 0 2024-03-04T13:43:00.876 INFO:tasks.workunit.client.1.smithi169.stdout:9/756: rmdir d7/d49 39 2024-03-04T13:43:00.879 INFO:tasks.workunit.client.1.smithi169.stdout:5/818: chown d1/c21 112114182 1 2024-03-04T13:43:00.880 INFO:tasks.workunit.client.1.smithi169.stdout:7/749: creat d2/de/d3e/d7e/ff9 x:0 0 0 2024-03-04T13:43:00.881 INFO:tasks.workunit.client.0.smithi107.stdout:7/168: mkdir d6/d1d/d36 0 2024-03-04T13:43:00.882 INFO:tasks.workunit.client.0.smithi107.stdout:7/169: chown d6/d13/d16/d19/d1e 5707 1 2024-03-04T13:43:00.883 INFO:tasks.workunit.client.1.smithi169.stdout:0/561: getdents d5/d9/dc/d12/d15/d19/d77/d94 0 2024-03-04T13:43:00.885 INFO:tasks.workunit.client.0.smithi107.stdout:7/170: write d6/d1d/f32 [975346,84880] 0 2024-03-04T13:43:00.886 INFO:tasks.workunit.client.1.smithi169.stdout:1/641: rename d4/d5/d2d/d4e/d71/d82/la1 to d4/d5/d36/d35/d8f/lcf 0 2024-03-04T13:43:00.887 INFO:tasks.workunit.client.0.smithi107.stdout:3/153: mkdir d1/de/d18/d33 0 2024-03-04T13:43:00.888 INFO:tasks.workunit.client.0.smithi107.stdout:7/171: readlink d6/d13/d16/d19/l20 0 2024-03-04T13:43:00.889 INFO:tasks.workunit.client.0.smithi107.stdout:9/176: fsync d4/da/f15 0 2024-03-04T13:43:00.889 INFO:tasks.workunit.client.0.smithi107.stdout:3/154: read d1/d1e/f22 [677004,8662] 0 2024-03-04T13:43:00.889 INFO:tasks.workunit.client.0.smithi107.stdout:7/172: stat d6/c9 0 2024-03-04T13:43:00.890 INFO:tasks.workunit.client.0.smithi107.stdout:6/225: rmdir df/d1b/d2c/d31 39 2024-03-04T13:43:00.892 INFO:tasks.workunit.client.1.smithi169.stdout:2/734: getdents d1/d5/d22/d45/d4a/de7/dcb 0 2024-03-04T13:43:00.892 INFO:tasks.workunit.client.0.smithi107.stdout:6/226: chown df/d17/d24/d40 9 1 2024-03-04T13:43:00.895 INFO:tasks.workunit.client.0.smithi107.stdout:6/227: readlink df/d1b/l3c 0 2024-03-04T13:43:00.895 INFO:tasks.workunit.client.0.smithi107.stdout:4/146: dread d0/d9/f1a [0,4194304] 0 2024-03-04T13:43:00.895 INFO:tasks.workunit.client.0.smithi107.stdout:0/222: rmdir da/dc/dd/d3e/d48 0 2024-03-04T13:43:00.897 INFO:tasks.workunit.client.0.smithi107.stdout:8/227: link l1 d6/da/l42 0 2024-03-04T13:43:00.900 INFO:tasks.workunit.client.1.smithi169.stdout:5/819: dwrite d1/d16/dbd/f112 [0,4194304] 0 2024-03-04T13:43:00.901 INFO:tasks.workunit.client.0.smithi107.stdout:1/166: creat d1/f31 x:0 0 0 2024-03-04T13:43:00.904 INFO:tasks.workunit.client.1.smithi169.stdout:1/642: write d4/d5/d36/d35/d8f/d93/f9b [2963061,78968] 0 2024-03-04T13:43:00.905 INFO:tasks.workunit.client.1.smithi169.stdout:0/562: dwrite d5/d4c/d65/faf [0,4194304] 0 2024-03-04T13:43:00.906 INFO:tasks.workunit.client.0.smithi107.stdout:7/173: mkdir d6/d1d/d28/d37 0 2024-03-04T13:43:00.906 INFO:tasks.workunit.client.1.smithi169.stdout:8/693: link d7/d10/d22/d75/d81/da2/l8c d7/d10/d17/d44/d36/d62/le0 0 2024-03-04T13:43:00.908 INFO:tasks.workunit.client.0.smithi107.stdout:9/177: dread d4/da/d11/f13 [0,4194304] 0 2024-03-04T13:43:00.909 INFO:tasks.workunit.client.0.smithi107.stdout:9/178: fdatasync d4/da/d11/d20/f28 0 2024-03-04T13:43:00.909 INFO:tasks.workunit.client.0.smithi107.stdout:3/155: rename d1/d1e/f31 to d1/de/d18/d33/f34 0 2024-03-04T13:43:00.910 INFO:tasks.workunit.client.1.smithi169.stdout:0/563: write d5/d9/dc/f1d [4934101,84514] 0 2024-03-04T13:43:00.912 INFO:tasks.workunit.client.0.smithi107.stdout:4/147: creat d0/d9/de/d14/dc/f2d x:0 0 0 2024-03-04T13:43:00.913 INFO:tasks.workunit.client.0.smithi107.stdout:2/196: dread f4 [0,4194304] 0 2024-03-04T13:43:00.914 INFO:tasks.workunit.client.1.smithi169.stdout:2/735: dwrite d1/f75 [0,4194304] 0 2024-03-04T13:43:00.915 INFO:tasks.workunit.client.0.smithi107.stdout:4/148: write d0/d9/d10/f19 [4316543,74453] 0 2024-03-04T13:43:00.916 INFO:tasks.workunit.client.1.smithi169.stdout:8/694: rename d7/d10/d22/d46 to d7/d10/d22/d46/da5/da6/dc6/de1 22 2024-03-04T13:43:00.916 INFO:tasks.workunit.client.1.smithi169.stdout:5/820: creat d1/d16/d107/f122 x:0 0 0 2024-03-04T13:43:00.918 INFO:tasks.workunit.client.0.smithi107.stdout:0/223: readlink da/dc/dd/l19 0 2024-03-04T13:43:00.919 INFO:tasks.workunit.client.0.smithi107.stdout:4/149: write d0/d9/d10/f19 [1518480,83452] 0 2024-03-04T13:43:00.920 INFO:tasks.workunit.client.1.smithi169.stdout:8/695: fdatasync d7/d10/d17/f2a 0 2024-03-04T13:43:00.923 INFO:tasks.workunit.client.1.smithi169.stdout:8/696: fsync d7/d10/d22/d75/d79/fb1 0 2024-03-04T13:43:00.929 INFO:tasks.workunit.client.0.smithi107.stdout:9/179: dwrite d4/da/d11/d20/f28 [0,4194304] 0 2024-03-04T13:43:00.930 INFO:tasks.workunit.client.0.smithi107.stdout:5/151: truncate d5/f6 480107 0 2024-03-04T13:43:00.934 INFO:tasks.workunit.client.1.smithi169.stdout:3/711: getdents d7/de/d2b/d7d/d85 0 2024-03-04T13:43:00.939 INFO:tasks.workunit.client.1.smithi169.stdout:3/712: chown d7/de/d2b/l7b 6864 1 2024-03-04T13:43:00.940 INFO:tasks.workunit.client.0.smithi107.stdout:2/197: write f4 [5824487,66839] 0 2024-03-04T13:43:00.942 INFO:tasks.workunit.client.0.smithi107.stdout:7/174: rename d6/d1d/d28 to d6/d13/d16/d19/d38 0 2024-03-04T13:43:00.943 INFO:tasks.workunit.client.1.smithi169.stdout:7/750: dwrite d2/d15/d17/fbf [0,4194304] 0 2024-03-04T13:43:00.945 INFO:tasks.workunit.client.1.smithi169.stdout:5/821: rename d1/d60/fd7 to d1/d16/dbd/dbb/dcc/f123 0 2024-03-04T13:43:00.952 INFO:tasks.workunit.client.0.smithi107.stdout:4/150: rmdir d0 39 2024-03-04T13:43:00.954 INFO:tasks.workunit.client.0.smithi107.stdout:0/224: fdatasync da/dc/dd/d3e/f11 0 2024-03-04T13:43:00.958 INFO:tasks.workunit.client.0.smithi107.stdout:9/180: mknod d4/da/d11/d20/d23/c33 0 2024-03-04T13:43:00.960 INFO:tasks.workunit.client.0.smithi107.stdout:7/175: dwrite d6/f7 [0,4194304] 0 2024-03-04T13:43:00.961 INFO:tasks.workunit.client.0.smithi107.stdout:7/176: chown f4 4474160 1 2024-03-04T13:43:00.972 INFO:tasks.workunit.client.0.smithi107.stdout:0/225: mknod da/dc/d34/c49 0 2024-03-04T13:43:00.977 INFO:tasks.workunit.client.0.smithi107.stdout:2/198: dread da/f10 [0,4194304] 0 2024-03-04T13:43:00.977 INFO:tasks.workunit.client.0.smithi107.stdout:2/199: read da/f18 [8249675,94898] 0 2024-03-04T13:43:00.977 INFO:tasks.workunit.client.1.smithi169.stdout:5/822: unlink d1/d16/f20 0 2024-03-04T13:43:00.977 INFO:tasks.workunit.client.1.smithi169.stdout:2/736: dwrite d1/d5/d19/f47 [4194304,4194304] 0 2024-03-04T13:43:00.978 INFO:tasks.workunit.client.1.smithi169.stdout:1/643: truncate d4/f6b 625176 0 2024-03-04T13:43:00.978 INFO:tasks.workunit.client.0.smithi107.stdout:5/152: symlink d5/d2b/l31 0 2024-03-04T13:43:00.979 INFO:tasks.workunit.client.1.smithi169.stdout:8/697: dwrite d7/d10/d24/d3e/f40 [0,4194304] 0 2024-03-04T13:43:00.981 INFO:tasks.workunit.client.1.smithi169.stdout:8/698: chown d7/d10/d17/c60 15 1 2024-03-04T13:43:00.987 INFO:tasks.workunit.client.0.smithi107.stdout:4/151: rename d0/d9/f1a to d0/d9/d1e/f2e 0 2024-03-04T13:43:00.988 INFO:tasks.workunit.client.1.smithi169.stdout:3/713: write d7/de/d2b/d7d/d85/d18/d21/d3f/fc7 [911341,69032] 0 2024-03-04T13:43:00.990 INFO:tasks.workunit.client.0.smithi107.stdout:0/226: creat da/dc/f4a x:0 0 0 2024-03-04T13:43:00.991 INFO:tasks.workunit.client.0.smithi107.stdout:1/167: getdents d1/d3/d17 0 2024-03-04T13:43:00.992 INFO:tasks.workunit.client.0.smithi107.stdout:5/153: dwrite d5/d8/da/f16 [0,4194304] 0 2024-03-04T13:43:00.996 INFO:tasks.workunit.client.0.smithi107.stdout:9/181: mkdir d4/d34 0 2024-03-04T13:43:00.996 INFO:tasks.workunit.client.0.smithi107.stdout:9/182: chown d4/da/d11 21346289 1 2024-03-04T13:43:01.001 INFO:tasks.workunit.client.1.smithi169.stdout:7/751: dwrite d2/d6b/f76 [0,4194304] 0 2024-03-04T13:43:01.013 INFO:tasks.workunit.client.0.smithi107.stdout:9/183: dwrite d4/da/d11/d20/f30 [0,4194304] 0 2024-03-04T13:43:01.014 INFO:tasks.workunit.client.1.smithi169.stdout:2/737: dread d1/d5/d19/d1e/f74 [0,4194304] 0 2024-03-04T13:43:01.016 INFO:tasks.workunit.client.0.smithi107.stdout:1/168: creat d1/d3/df/f32 x:0 0 0 2024-03-04T13:43:01.017 INFO:tasks.workunit.client.0.smithi107.stdout:4/152: mkdir d0/d2f 0 2024-03-04T13:43:01.022 INFO:tasks.workunit.client.0.smithi107.stdout:2/200: getdents da/df/d33 0 2024-03-04T13:43:01.027 INFO:tasks.workunit.client.0.smithi107.stdout:9/184: creat d4/d34/f35 x:0 0 0 2024-03-04T13:43:01.040 INFO:tasks.workunit.client.0.smithi107.stdout:0/227: truncate da/dc/dd/d3e/f44 2421257 0 2024-03-04T13:43:01.042 INFO:tasks.workunit.client.0.smithi107.stdout:9/185: getdents d4/da/d11/d20/d31 0 2024-03-04T13:43:01.043 INFO:tasks.workunit.client.1.smithi169.stdout:0/564: link d5/d9/dc/d12/d15/d19/d61/d71/d8c/c9c d5/d9/dc/d12/d15/d19/d61/d71/cc7 0 2024-03-04T13:43:01.043 INFO:tasks.workunit.client.0.smithi107.stdout:9/186: chown d4/f26 662930493 1 2024-03-04T13:43:01.044 INFO:tasks.workunit.client.0.smithi107.stdout:2/201: link da/fb da/df/d3e/f43 0 2024-03-04T13:43:01.045 INFO:tasks.workunit.client.1.smithi169.stdout:0/565: write d5/d4c/d65/d95/fb9 [723206,28398] 0 2024-03-04T13:43:01.046 INFO:tasks.workunit.client.0.smithi107.stdout:9/187: write d4/da/d11/d20/f28 [4696934,2485] 0 2024-03-04T13:43:01.048 INFO:tasks.workunit.client.0.smithi107.stdout:2/202: write da/f1c [853461,2226] 0 2024-03-04T13:43:01.049 INFO:tasks.workunit.client.0.smithi107.stdout:9/188: rename d4/da/d11 to d4/da/d11/d20/d23/d36 22 2024-03-04T13:43:01.053 INFO:tasks.workunit.client.0.smithi107.stdout:0/228: unlink da/dc/f1d 0 2024-03-04T13:43:01.053 INFO:tasks.workunit.client.1.smithi169.stdout:8/699: read d7/d10/d17/d44/d4c/fa3 [228024,75379] 0 2024-03-04T13:43:01.055 INFO:tasks.workunit.client.1.smithi169.stdout:8/700: write d7/d10/d24/d3e/f57 [2771457,59256] 0 2024-03-04T13:43:01.056 INFO:tasks.workunit.client.0.smithi107.stdout:2/203: write da/fb [177642,9976] 0 2024-03-04T13:43:01.056 INFO:tasks.workunit.client.0.smithi107.stdout:1/169: truncate d1/d10/d20/f2c 261708 0 2024-03-04T13:43:01.057 INFO:tasks.workunit.client.0.smithi107.stdout:2/204: dread - da/d34/f41 zero size 2024-03-04T13:43:01.059 INFO:tasks.workunit.client.0.smithi107.stdout:4/153: dread d0/d9/d10/f25 [0,4194304] 0 2024-03-04T13:43:01.066 INFO:tasks.workunit.client.1.smithi169.stdout:2/738: fsync d1/d5/d22/d45/f8f 0 2024-03-04T13:43:01.071 INFO:tasks.workunit.client.0.smithi107.stdout:0/229: mknod da/dc/d34/c4b 0 2024-03-04T13:43:01.071 INFO:tasks.workunit.client.0.smithi107.stdout:1/170: dwrite d1/d10/d20/d24/f2d [0,4194304] 0 2024-03-04T13:43:01.073 INFO:tasks.workunit.client.1.smithi169.stdout:4/665: sync 2024-03-04T13:43:01.082 INFO:tasks.workunit.client.0.smithi107.stdout:2/205: mkdir da/d44 0 2024-03-04T13:43:01.083 INFO:tasks.workunit.client.1.smithi169.stdout:3/714: dwrite d7/de/d2b/d7d/d85/d13/d5a/f80 [0,4194304] 0 2024-03-04T13:43:01.083 INFO:tasks.workunit.client.0.smithi107.stdout:4/154: mkdir d0/d30 0 2024-03-04T13:43:01.085 INFO:tasks.workunit.client.0.smithi107.stdout:9/189: getdents d4 0 2024-03-04T13:43:01.085 INFO:tasks.workunit.client.1.smithi169.stdout:3/715: chown d7/de/d2b/d7d/d85/d18 4513899 1 2024-03-04T13:43:01.085 INFO:tasks.workunit.client.1.smithi169.stdout:8/701: creat d7/d10/d17/d44/d36/d62/dc1/fe2 x:0 0 0 2024-03-04T13:43:01.086 INFO:tasks.workunit.client.0.smithi107.stdout:9/190: stat d4/da/d11/d20/l29 0 2024-03-04T13:43:01.086 INFO:tasks.workunit.client.1.smithi169.stdout:3/716: fdatasync d7/de/d2b/f52 0 2024-03-04T13:43:01.090 INFO:tasks.workunit.client.1.smithi169.stdout:5/823: link d1/d16/d2a/l58 d1/d60/l124 0 2024-03-04T13:43:01.090 INFO:tasks.workunit.client.0.smithi107.stdout:0/230: mknod da/c4c 0 2024-03-04T13:43:01.091 INFO:tasks.workunit.client.0.smithi107.stdout:4/155: chown d0/d9/de/d14/dc/f26 0 1 2024-03-04T13:43:01.091 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:00 smithi107 ceph-mon[28739]: Upgrade: Updating mgr.smithi107.wpzgwl 2024-03-04T13:43:01.091 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:00 smithi107 ceph-mon[28739]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' 2024-03-04T13:43:01.091 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:00 smithi107 ceph-mon[28739]: from='mgr.24331 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi107.wpzgwl", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-04T13:43:01.091 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:00 smithi107 ceph-mon[28739]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi107.wpzgwl", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-04T13:43:01.092 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:00 smithi107 ceph-mon[28739]: from='mgr.24331 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-04T13:43:01.092 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:00 smithi107 ceph-mon[28739]: from='mgr.24331 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:43:01.092 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:00 smithi107 ceph-mon[28739]: Deploying daemon mgr.smithi107.wpzgwl on smithi107 2024-03-04T13:43:01.092 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:00 smithi107 ceph-mon[28739]: pgmap v52: 65 pgs: 65 active+clean; 1.6 GiB data, 6.3 GiB used, 530 GiB / 536 GiB avail; 44 MiB/s rd, 154 MiB/s wr, 344 op/s 2024-03-04T13:43:01.093 INFO:tasks.workunit.client.0.smithi107.stdout:1/171: truncate d1/d10/d16/d1b/f23 711576 0 2024-03-04T13:43:01.094 INFO:tasks.workunit.client.0.smithi107.stdout:0/231: read da/f3c [604645,27168] 0 2024-03-04T13:43:01.094 INFO:tasks.workunit.client.0.smithi107.stdout:2/206: symlink da/d38/d3a/l45 0 2024-03-04T13:43:01.096 INFO:tasks.workunit.client.0.smithi107.stdout:2/207: write da/df/f13 [653654,94496] 0 2024-03-04T13:43:01.097 INFO:tasks.workunit.client.0.smithi107.stdout:1/172: mkdir d1/d3/df/d33 0 2024-03-04T13:43:01.097 INFO:tasks.workunit.client.1.smithi169.stdout:0/566: getdents d5/d9/dc/d12/d15/d19/d61 0 2024-03-04T13:43:01.101 INFO:tasks.workunit.client.1.smithi169.stdout:4/666: mkdir d4/d11/d2e/dcd/dd1 0 2024-03-04T13:43:01.101 INFO:tasks.workunit.client.1.smithi169.stdout:5/824: chown d1/d16/d2a/d62/fb4 2316 1 2024-03-04T13:43:01.102 INFO:tasks.workunit.client.0.smithi107.stdout:2/208: read - da/df/d3e/f40 zero size 2024-03-04T13:43:01.106 INFO:tasks.workunit.client.1.smithi169.stdout:1/644: truncate d4/d5/d2d/d4e/d71/d26/d54/f72 3651865 0 2024-03-04T13:43:01.107 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:00 smithi169 ceph-mon[36313]: Upgrade: Updating mgr.smithi107.wpzgwl 2024-03-04T13:43:01.107 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:00 smithi169 ceph-mon[36313]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' 2024-03-04T13:43:01.107 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:00 smithi169 ceph-mon[36313]: from='mgr.24331 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi107.wpzgwl", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-04T13:43:01.107 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:00 smithi169 ceph-mon[36313]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi107.wpzgwl", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-04T13:43:01.107 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:00 smithi169 ceph-mon[36313]: from='mgr.24331 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-04T13:43:01.107 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:00 smithi169 ceph-mon[36313]: from='mgr.24331 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:43:01.107 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:00 smithi169 ceph-mon[36313]: Deploying daemon mgr.smithi107.wpzgwl on smithi107 2024-03-04T13:43:01.108 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:00 smithi169 ceph-mon[36313]: pgmap v52: 65 pgs: 65 active+clean; 1.6 GiB data, 6.3 GiB used, 530 GiB / 536 GiB avail; 44 MiB/s rd, 154 MiB/s wr, 344 op/s 2024-03-04T13:43:01.108 INFO:tasks.workunit.client.0.smithi107.stdout:9/191: dwrite d4/da/d11/d20/f30 [0,4194304] 0 2024-03-04T13:43:01.109 INFO:tasks.workunit.client.1.smithi169.stdout:1/645: chown d4/d5/d2d/d4e/fa6 26 1 2024-03-04T13:43:01.110 INFO:tasks.workunit.client.0.smithi107.stdout:9/192: fsync d4/da/d11/f2b 0 2024-03-04T13:43:01.111 INFO:tasks.workunit.client.0.smithi107.stdout:9/193: chown d4/da/d11 74310549 1 2024-03-04T13:43:01.111 INFO:tasks.workunit.client.0.smithi107.stdout:0/232: getdents da/dc/dd/d3e/d47 0 2024-03-04T13:43:01.112 INFO:tasks.workunit.client.1.smithi169.stdout:7/752: getdents d2/de/d37 0 2024-03-04T13:43:01.112 INFO:tasks.workunit.client.1.smithi169.stdout:4/667: dread d4/f69 [0,4194304] 0 2024-03-04T13:43:01.114 INFO:tasks.workunit.client.0.smithi107.stdout:0/233: write da/f2a [3695762,51694] 0 2024-03-04T13:43:01.114 INFO:tasks.workunit.client.0.smithi107.stdout:1/173: dread d1/d10/d20/d24/f2d [0,4194304] 0 2024-03-04T13:43:01.115 INFO:tasks.workunit.client.0.smithi107.stdout:2/209: rename f4 to da/d39/f46 0 2024-03-04T13:43:01.117 INFO:tasks.workunit.client.0.smithi107.stdout:0/234: dread - da/dc/dd/d3e/f35 zero size 2024-03-04T13:43:01.117 INFO:tasks.workunit.client.0.smithi107.stdout:2/210: write da/df/f13 [122129,56051] 0 2024-03-04T13:43:01.120 INFO:tasks.workunit.client.1.smithi169.stdout:2/739: getdents d1/d5/d19/d68 0 2024-03-04T13:43:01.121 INFO:tasks.workunit.client.0.smithi107.stdout:4/156: write d0/d9/d10/f1c [4102964,36512] 0 2024-03-04T13:43:01.126 INFO:tasks.workunit.client.0.smithi107.stdout:1/174: dread d1/d10/d20/f2c [0,4194304] 0 2024-03-04T13:43:01.129 INFO:tasks.workunit.client.1.smithi169.stdout:7/753: readlink d2/d15/l2f 0 2024-03-04T13:43:01.130 INFO:tasks.workunit.client.1.smithi169.stdout:7/754: chown d2/de/d3e/d7e 7862 1 2024-03-04T13:43:01.131 INFO:tasks.workunit.client.1.smithi169.stdout:4/668: symlink d4/d11/d2e/d39/d48/ld2 0 2024-03-04T13:43:01.132 INFO:tasks.workunit.client.0.smithi107.stdout:2/211: dread - da/d29/f2d zero size 2024-03-04T13:43:01.134 INFO:tasks.workunit.client.1.smithi169.stdout:7/755: read d2/de/d3e/d41/d5b/f9f [664647,64619] 0 2024-03-04T13:43:01.134 INFO:tasks.workunit.client.0.smithi107.stdout:2/212: write f2 [304612,66454] 0 2024-03-04T13:43:01.134 INFO:tasks.workunit.client.0.smithi107.stdout:9/194: symlink d4/d32/l37 0 2024-03-04T13:43:01.135 INFO:tasks.workunit.client.1.smithi169.stdout:5/825: read d1/d16/d17/d28/f4c [2028452,11031] 0 2024-03-04T13:43:01.136 INFO:tasks.workunit.client.0.smithi107.stdout:4/157: dwrite d0/d9/de/d14/dc/f2d [0,4194304] 0 2024-03-04T13:43:01.136 INFO:tasks.workunit.client.0.smithi107.stdout:9/195: write d4/f21 [2110125,9617] 0 2024-03-04T13:43:01.137 INFO:tasks.workunit.client.0.smithi107.stdout:0/235: rmdir da/dc/d34 39 2024-03-04T13:43:01.139 INFO:tasks.workunit.client.1.smithi169.stdout:2/740: fsync d1/d5/d19/d69/fc8 0 2024-03-04T13:43:01.139 INFO:tasks.workunit.client.1.smithi169.stdout:8/702: link d7/d10/l7e d7/d10/d24/le3 0 2024-03-04T13:43:01.141 INFO:tasks.workunit.client.0.smithi107.stdout:1/175: creat d1/d3/d17/f34 x:0 0 0 2024-03-04T13:43:01.142 INFO:tasks.workunit.client.0.smithi107.stdout:1/176: chown d1/c29 0 1 2024-03-04T13:43:01.145 INFO:tasks.workunit.client.1.smithi169.stdout:3/717: getdents d7/de/d2b/d7d/d85/d13/d5a 0 2024-03-04T13:43:01.149 INFO:tasks.workunit.client.1.smithi169.stdout:2/741: dread d1/d5/d22/d45/f90 [0,4194304] 0 2024-03-04T13:43:01.151 INFO:tasks.workunit.client.1.smithi169.stdout:4/669: readlink d4/d11/d2e/d39/d61/lba 0 2024-03-04T13:43:01.153 INFO:tasks.workunit.client.1.smithi169.stdout:7/756: rename d2/de/d3e/d41/fa9 to d2/de/d3e/d41/ffa 0 2024-03-04T13:43:01.154 INFO:tasks.workunit.client.0.smithi107.stdout:4/158: dwrite d0/d9/f11 [0,4194304] 0 2024-03-04T13:43:01.154 INFO:tasks.workunit.client.0.smithi107.stdout:9/196: dwrite d4/f26 [0,4194304] 0 2024-03-04T13:43:01.155 INFO:tasks.workunit.client.1.smithi169.stdout:5/826: rename d1/c68 to d1/d16/d17/d1b/d74/ddc/de4/c125 0 2024-03-04T13:43:01.158 INFO:tasks.workunit.client.0.smithi107.stdout:0/236: rmdir da/dc/dd 39 2024-03-04T13:43:01.160 INFO:tasks.workunit.client.1.smithi169.stdout:8/703: creat d7/d10/d22/d46/d49/fe4 x:0 0 0 2024-03-04T13:43:01.163 INFO:tasks.workunit.client.0.smithi107.stdout:1/177: unlink d1/d3/c2a 0 2024-03-04T13:43:01.164 INFO:tasks.workunit.client.0.smithi107.stdout:2/213: dread f7 [0,4194304] 0 2024-03-04T13:43:01.165 INFO:tasks.workunit.client.1.smithi169.stdout:1/646: rmdir d4/d5/d86 0 2024-03-04T13:43:01.165 INFO:tasks.workunit.client.0.smithi107.stdout:1/178: dread d1/d10/d20/f2c [0,4194304] 0 2024-03-04T13:43:01.166 INFO:tasks.workunit.client.0.smithi107.stdout:4/159: readlink d0/l16 0 2024-03-04T13:43:01.167 INFO:tasks.workunit.client.0.smithi107.stdout:1/179: fdatasync d1/d3/d17/f34 0 2024-03-04T13:43:01.167 INFO:tasks.workunit.client.0.smithi107.stdout:4/160: chown d0/d9/d18/c22 6950 1 2024-03-04T13:43:01.169 INFO:tasks.workunit.client.0.smithi107.stdout:9/197: creat d4/da/d11/d20/d23/d2c/f38 x:0 0 0 2024-03-04T13:43:01.169 INFO:tasks.workunit.client.0.smithi107.stdout:4/161: write d0/d9/f11 [2228584,37821] 0 2024-03-04T13:43:01.171 INFO:tasks.workunit.client.0.smithi107.stdout:9/198: readlink d4/l1d 0 2024-03-04T13:43:01.172 INFO:tasks.workunit.client.0.smithi107.stdout:2/214: dread da/df/f13 [0,4194304] 0 2024-03-04T13:43:01.172 INFO:tasks.workunit.client.0.smithi107.stdout:4/162: chown d0/l16 88768 1 2024-03-04T13:43:01.172 INFO:tasks.workunit.client.0.smithi107.stdout:9/199: write d4/f26 [1299695,48252] 0 2024-03-04T13:43:01.174 INFO:tasks.workunit.client.0.smithi107.stdout:0/237: write da/dc/dd/f2c [467579,117931] 0 2024-03-04T13:43:01.174 INFO:tasks.workunit.client.1.smithi169.stdout:4/670: symlink d4/d11/d2e/d37/d5c/d86/ld3 0 2024-03-04T13:43:01.176 INFO:tasks.workunit.client.0.smithi107.stdout:0/238: fsync da/dc/dd/f46 0 2024-03-04T13:43:01.180 INFO:tasks.workunit.client.0.smithi107.stdout:0/239: dread f0 [0,4194304] 0 2024-03-04T13:43:01.182 INFO:tasks.workunit.client.0.smithi107.stdout:2/215: mknod da/df/d3e/c47 0 2024-03-04T13:43:01.183 INFO:tasks.workunit.client.1.smithi169.stdout:2/742: write d1/d5/d19/d3c/db8/fca [1028992,53889] 0 2024-03-04T13:43:01.183 INFO:tasks.workunit.client.1.smithi169.stdout:3/718: fsync d7/de/d2b/d7d/d85/d13/d1a/d98/d9a/f8e 0 2024-03-04T13:43:01.186 INFO:tasks.workunit.client.0.smithi107.stdout:9/200: truncate d4/da/f1b 1244945 0 2024-03-04T13:43:01.192 INFO:tasks.workunit.client.0.smithi107.stdout:0/240: read - da/dc/d34/f45 zero size 2024-03-04T13:43:01.192 INFO:tasks.workunit.client.1.smithi169.stdout:8/704: mknod d7/d10/ce5 0 2024-03-04T13:43:01.193 INFO:tasks.workunit.client.0.smithi107.stdout:2/216: dread da/f1c [0,4194304] 0 2024-03-04T13:43:01.193 INFO:tasks.workunit.client.0.smithi107.stdout:0/241: chown da/dc/dd/f46 62714 1 2024-03-04T13:43:01.193 INFO:tasks.workunit.client.0.smithi107.stdout:2/217: chown da/df 18162 1 2024-03-04T13:43:01.194 INFO:tasks.workunit.client.1.smithi169.stdout:8/705: dread - d7/d10/d17/f98 zero size 2024-03-04T13:43:01.194 INFO:tasks.workunit.client.1.smithi169.stdout:1/647: mkdir d4/d5/d2d/d4e/d71/d26/daf/dd0 0 2024-03-04T13:43:01.195 INFO:tasks.workunit.client.0.smithi107.stdout:4/163: mknod d0/d30/c31 0 2024-03-04T13:43:01.197 INFO:tasks.workunit.client.0.smithi107.stdout:9/201: mkdir d4/da/d11/d20/d39 0 2024-03-04T13:43:01.197 INFO:tasks.workunit.client.1.smithi169.stdout:7/757: write d2/de/ff5 [1862086,3176] 0 2024-03-04T13:43:01.198 INFO:tasks.workunit.client.0.smithi107.stdout:9/202: write d4/da/f15 [4058168,70169] 0 2024-03-04T13:43:01.200 INFO:tasks.workunit.client.0.smithi107.stdout:1/180: rename d1/d10/d16/d1b/c27 to d1/c35 0 2024-03-04T13:43:01.201 INFO:tasks.workunit.client.1.smithi169.stdout:5/827: link d1/d81/c8d d1/d16/d2a/d46/d9e/c126 0 2024-03-04T13:43:01.202 INFO:tasks.workunit.client.0.smithi107.stdout:2/218: write da/d34/f36 [58312,48494] 0 2024-03-04T13:43:01.202 INFO:tasks.workunit.client.0.smithi107.stdout:0/242: readlink da/dc/dd/d3e/d26/l20 0 2024-03-04T13:43:01.203 INFO:tasks.workunit.client.0.smithi107.stdout:0/243: readlink l9 0 2024-03-04T13:43:01.204 INFO:tasks.workunit.client.1.smithi169.stdout:3/719: rename d7/de/d2b/d7d/d85/d18/d21/d3f/d68/c40 to d7/de/d2b/d7d/d85/d18/d21/d3f/d68/cf0 0 2024-03-04T13:43:01.205 INFO:tasks.workunit.client.1.smithi169.stdout:4/671: dread d4/d1c/d75/d81/f85 [0,4194304] 0 2024-03-04T13:43:01.206 INFO:tasks.workunit.client.1.smithi169.stdout:1/648: fdatasync d4/d5/d36/d35/d5b/f87 0 2024-03-04T13:43:01.209 INFO:tasks.workunit.client.1.smithi169.stdout:2/743: symlink d1/d5/d22/d45/dde/lfd 0 2024-03-04T13:43:01.217 INFO:tasks.workunit.client.0.smithi107.stdout:9/203: creat d4/da/d11/d20/d39/f3a x:0 0 0 2024-03-04T13:43:01.217 INFO:tasks.workunit.client.1.smithi169.stdout:5/828: rename d1/d16/d17/d1b/d74/fc6 to d1/d60/f127 0 2024-03-04T13:43:01.217 INFO:tasks.workunit.client.1.smithi169.stdout:3/720: creat d7/db8/ded/ff1 x:0 0 0 2024-03-04T13:43:01.219 INFO:tasks.workunit.client.0.smithi107.stdout:8/228: sync 2024-03-04T13:43:01.219 INFO:tasks.workunit.client.0.smithi107.stdout:6/228: sync 2024-03-04T13:43:01.221 INFO:tasks.workunit.client.0.smithi107.stdout:2/219: symlink da/d44/l48 0 2024-03-04T13:43:01.223 INFO:tasks.workunit.client.0.smithi107.stdout:1/181: write d1/d10/d20/f2c [809001,77265] 0 2024-03-04T13:43:01.224 INFO:tasks.workunit.client.1.smithi169.stdout:2/744: read - d1/d5/d22/d45/fb0 zero size 2024-03-04T13:43:01.225 INFO:tasks.workunit.client.0.smithi107.stdout:4/164: truncate d0/d9/de/d14/dc/f2d 2071302 0 2024-03-04T13:43:01.228 INFO:tasks.workunit.client.1.smithi169.stdout:7/758: mkdir d2/de/dfb 0 2024-03-04T13:43:01.231 INFO:tasks.workunit.client.0.smithi107.stdout:0/244: creat da/dc/f4d x:0 0 0 2024-03-04T13:43:01.232 INFO:tasks.workunit.client.0.smithi107.stdout:9/204: mkdir d4/da/d11/d20/d31/d3b 0 2024-03-04T13:43:01.234 INFO:tasks.workunit.client.1.smithi169.stdout:1/649: creat d4/fd1 x:0 0 0 2024-03-04T13:43:01.234 INFO:tasks.workunit.client.0.smithi107.stdout:6/229: mkdir df/d1b/d2c/d46 0 2024-03-04T13:43:01.235 INFO:tasks.workunit.client.0.smithi107.stdout:1/182: dread d1/d10/d20/d24/f2d [0,4194304] 0 2024-03-04T13:43:01.236 INFO:tasks.workunit.client.1.smithi169.stdout:8/706: getdents d7/dd4 0 2024-03-04T13:43:01.236 INFO:tasks.workunit.client.1.smithi169.stdout:1/650: stat d4/d5/d36/d35/df/da8 0 2024-03-04T13:43:01.236 INFO:tasks.workunit.client.0.smithi107.stdout:1/183: dread - d1/f31 zero size 2024-03-04T13:43:01.236 INFO:tasks.workunit.client.1.smithi169.stdout:6/702: sync 2024-03-04T13:43:01.237 INFO:tasks.workunit.client.1.smithi169.stdout:9/757: sync 2024-03-04T13:43:01.237 INFO:tasks.workunit.client.0.smithi107.stdout:1/184: write d1/f31 [714588,23594] 0 2024-03-04T13:43:01.239 INFO:tasks.workunit.client.1.smithi169.stdout:4/672: dwrite d4/d11/d2e/d39/f60 [8388608,4194304] 0 2024-03-04T13:43:01.239 INFO:tasks.workunit.client.1.smithi169.stdout:1/651: readlink d4/d5/d2d/d4e/d71/l62 0 2024-03-04T13:43:01.241 INFO:tasks.workunit.client.1.smithi169.stdout:2/745: mknod d1/d5/d22/cfe 0 2024-03-04T13:43:01.242 INFO:tasks.workunit.client.1.smithi169.stdout:9/758: truncate d7/dc/df1/fe0 81360 0 2024-03-04T13:43:01.244 INFO:tasks.workunit.client.1.smithi169.stdout:4/673: write d4/d11/d2e/d39/f60 [6221600,119581] 0 2024-03-04T13:43:01.244 INFO:tasks.workunit.client.0.smithi107.stdout:2/220: mkdir da/d49 0 2024-03-04T13:43:01.248 INFO:tasks.workunit.client.1.smithi169.stdout:6/703: unlink d0/d3c/d40/df6/le2 0 2024-03-04T13:43:01.249 INFO:tasks.workunit.client.1.smithi169.stdout:4/674: chown d4/d11/d2e/d37/f45 0 1 2024-03-04T13:43:01.250 INFO:tasks.workunit.client.0.smithi107.stdout:9/205: unlink d4/f24 0 2024-03-04T13:43:01.254 INFO:tasks.workunit.client.0.smithi107.stdout:6/230: rename c2 to df/d1b/d2c/d46/c47 0 2024-03-04T13:43:01.261 INFO:tasks.workunit.client.1.smithi169.stdout:7/759: link d2/d6b/c6d d2/de/d37/cfc 0 2024-03-04T13:43:01.261 INFO:tasks.workunit.client.1.smithi169.stdout:2/746: fsync d1/d5/d19/d1e/d29/fd8 0 2024-03-04T13:43:01.261 INFO:tasks.workunit.client.1.smithi169.stdout:6/704: dread d0/d32/d3a/d43/d74/d93/dbe/ff8 [0,4194304] 0 2024-03-04T13:43:01.266 INFO:tasks.workunit.client.1.smithi169.stdout:1/652: dwrite d4/d5/d2d/d4e/d71/f37 [4194304,4194304] 0 2024-03-04T13:43:01.270 INFO:tasks.workunit.client.1.smithi169.stdout:1/653: stat d4/d5/d2d/d4e/d71/d82/fcb 0 2024-03-04T13:43:01.271 INFO:tasks.workunit.client.1.smithi169.stdout:4/675: dwrite d4/d11/f79 [4194304,4194304] 0 2024-03-04T13:43:01.278 INFO:tasks.workunit.client.1.smithi169.stdout:7/760: symlink d2/d28/d68/lfd 0 2024-03-04T13:43:01.282 INFO:tasks.workunit.client.1.smithi169.stdout:9/759: truncate d7/dc/f23 4009777 0 2024-03-04T13:43:01.284 INFO:tasks.workunit.client.1.smithi169.stdout:2/747: unlink d1/d5/d19/d3c/db8/fee 0 2024-03-04T13:43:01.285 INFO:tasks.workunit.client.0.smithi107.stdout:8/229: getdents d6/da/d25 0 2024-03-04T13:43:01.288 INFO:tasks.workunit.client.1.smithi169.stdout:8/707: link d7/d10/l7e d7/d10/d22/d75/d81/da4/le6 0 2024-03-04T13:43:01.288 INFO:tasks.workunit.client.1.smithi169.stdout:1/654: fdatasync d4/f7d 0 2024-03-04T13:43:01.290 INFO:tasks.workunit.client.0.smithi107.stdout:0/245: creat da/dc/f4e x:0 0 0 2024-03-04T13:43:01.290 INFO:tasks.workunit.client.1.smithi169.stdout:7/761: mknod d2/de/d3e/d7e/dbb/cfe 0 2024-03-04T13:43:01.293 INFO:tasks.workunit.client.1.smithi169.stdout:4/676: dwrite d4/d11/d3b/d67/fa0 [0,4194304] 0 2024-03-04T13:43:01.294 INFO:tasks.workunit.client.0.smithi107.stdout:4/165: getdents d0/d9/de/d14/dc 0 2024-03-04T13:43:01.295 INFO:tasks.workunit.client.0.smithi107.stdout:9/206: write d4/da/d11/f2b [1044282,18313] 0 2024-03-04T13:43:01.295 INFO:tasks.workunit.client.0.smithi107.stdout:0/246: write da/dc/d34/f38 [2593040,89114] 0 2024-03-04T13:43:01.298 INFO:tasks.workunit.client.0.smithi107.stdout:0/247: chown da/f2a 1013671485 1 2024-03-04T13:43:01.298 INFO:tasks.workunit.client.1.smithi169.stdout:9/760: symlink d7/dc/d12/d32/d42/d4f/l115 0 2024-03-04T13:43:01.299 INFO:tasks.workunit.client.0.smithi107.stdout:1/185: link d1/d3/d17/l18 d1/d10/d20/d24/l36 0 2024-03-04T13:43:01.306 INFO:tasks.workunit.client.1.smithi169.stdout:2/748: symlink d1/d5/d19/d3c/db8/de1/d64/lff 0 2024-03-04T13:43:01.308 INFO:tasks.workunit.client.0.smithi107.stdout:6/231: dwrite df/d1b/f1d [0,4194304] 0 2024-03-04T13:43:01.310 INFO:tasks.workunit.client.1.smithi169.stdout:2/749: chown d1/d5/d22/d3b/d99/fa6 449732 1 2024-03-04T13:43:01.317 INFO:tasks.workunit.client.1.smithi169.stdout:9/761: unlink d7/dc/d3e/da3/lfb 0 2024-03-04T13:43:01.318 INFO:tasks.workunit.client.1.smithi169.stdout:7/762: dwrite d2/ff3 [4194304,4194304] 0 2024-03-04T13:43:01.320 INFO:tasks.workunit.client.0.smithi107.stdout:1/186: dwrite d1/d10/d20/f2c [0,4194304] 0 2024-03-04T13:43:01.320 INFO:tasks.workunit.client.0.smithi107.stdout:9/207: creat d4/da/d11/d20/d31/f3c x:0 0 0 2024-03-04T13:43:01.322 INFO:tasks.workunit.client.0.smithi107.stdout:9/208: chown d4/da/l10 103 1 2024-03-04T13:43:01.323 INFO:tasks.workunit.client.0.smithi107.stdout:1/187: dread - d1/d3/df/f32 zero size 2024-03-04T13:43:01.323 INFO:tasks.workunit.client.1.smithi169.stdout:4/677: rename d4/d11/d3b/d67/dbb/d6e/f92 to d4/d11/d3b/d67/dbb/d6e/fd4 0 2024-03-04T13:43:01.328 INFO:tasks.workunit.client.1.smithi169.stdout:6/705: getdents d0/d63 0 2024-03-04T13:43:01.329 INFO:tasks.workunit.client.1.smithi169.stdout:8/708: write d7/d10/d17/faa [658474,2144] 0 2024-03-04T13:43:01.329 INFO:tasks.workunit.client.0.smithi107.stdout:2/221: link da/c1b da/d34/c4a 0 2024-03-04T13:43:01.331 INFO:tasks.workunit.client.0.smithi107.stdout:0/248: dread da/dc/dd/d3e/f1c [0,4194304] 0 2024-03-04T13:43:01.334 INFO:tasks.workunit.client.0.smithi107.stdout:0/249: read f8 [103354,6257] 0 2024-03-04T13:43:01.335 INFO:tasks.workunit.client.1.smithi169.stdout:2/750: rename d1/d5/d19/d3c/db8/de1/f60 to d1/d5/d22/d3b/f100 0 2024-03-04T13:43:01.335 INFO:tasks.workunit.client.0.smithi107.stdout:6/232: rmdir df/d1b/d2c 39 2024-03-04T13:43:01.335 INFO:tasks.workunit.client.1.smithi169.stdout:8/709: read d7/d10/d17/d44/d4c/f7d [145402,30790] 0 2024-03-04T13:43:01.336 INFO:tasks.workunit.client.1.smithi169.stdout:1/655: mknod d4/d5/d2d/cd2 0 2024-03-04T13:43:01.337 INFO:tasks.workunit.client.1.smithi169.stdout:9/762: symlink d7/dc/d12/d8a/l116 0 2024-03-04T13:43:01.338 INFO:tasks.workunit.client.0.smithi107.stdout:4/166: symlink d0/d2f/l32 0 2024-03-04T13:43:01.339 INFO:tasks.workunit.client.1.smithi169.stdout:7/763: mkdir d2/d15/d17/dff 0 2024-03-04T13:43:01.340 INFO:tasks.workunit.client.0.smithi107.stdout:4/167: chown d0/d9/de/d14/dc/c13 866123 1 2024-03-04T13:43:01.343 INFO:tasks.workunit.client.0.smithi107.stdout:1/188: dwrite d1/d10/d20/f2c [0,4194304] 0 2024-03-04T13:43:01.346 INFO:tasks.workunit.client.0.smithi107.stdout:1/189: readlink d1/d3/df/l14 0 2024-03-04T13:43:01.346 INFO:tasks.workunit.client.0.smithi107.stdout:0/250: stat da/dc/l30 0 2024-03-04T13:43:01.347 INFO:tasks.workunit.client.1.smithi169.stdout:9/763: creat d7/d27/f117 x:0 0 0 2024-03-04T13:43:01.348 INFO:tasks.workunit.client.0.smithi107.stdout:1/190: fdatasync d1/f31 0 2024-03-04T13:43:01.349 INFO:tasks.workunit.client.0.smithi107.stdout:8/230: getdents d6 0 2024-03-04T13:43:01.353 INFO:tasks.workunit.client.0.smithi107.stdout:9/209: creat d4/da/d11/d20/d2f/f3d x:0 0 0 2024-03-04T13:43:01.354 INFO:tasks.workunit.client.0.smithi107.stdout:4/168: dread d0/d9/de/d14/f1d [0,4194304] 0 2024-03-04T13:43:01.355 INFO:tasks.workunit.client.1.smithi169.stdout:7/764: creat d2/d15/dad/f100 x:0 0 0 2024-03-04T13:43:01.355 INFO:tasks.workunit.client.0.smithi107.stdout:9/210: fdatasync d4/da/d11/d20/f28 0 2024-03-04T13:43:01.356 INFO:tasks.workunit.client.1.smithi169.stdout:7/765: chown d2/de/d3e/cb5 242228649 1 2024-03-04T13:43:01.358 INFO:tasks.workunit.client.1.smithi169.stdout:0/567: sync 2024-03-04T13:43:01.358 INFO:tasks.workunit.client.0.smithi107.stdout:7/177: sync 2024-03-04T13:43:01.359 INFO:tasks.workunit.client.0.smithi107.stdout:5/154: sync 2024-03-04T13:43:01.359 INFO:tasks.workunit.client.0.smithi107.stdout:3/156: sync 2024-03-04T13:43:01.359 INFO:tasks.workunit.client.1.smithi169.stdout:6/706: link d0/d63/f78 d0/d63/f101 0 2024-03-04T13:43:01.363 INFO:tasks.workunit.client.1.smithi169.stdout:4/678: write d4/d11/d3b/d67/dbb/f76 [4691765,119747] 0 2024-03-04T13:43:01.363 INFO:tasks.workunit.client.1.smithi169.stdout:8/710: truncate d7/f9b 3158418 0 2024-03-04T13:43:01.364 INFO:tasks.workunit.client.0.smithi107.stdout:0/251: write f8 [349338,115493] 0 2024-03-04T13:43:01.367 INFO:tasks.workunit.client.0.smithi107.stdout:2/222: write da/f1c [64671,65132] 0 2024-03-04T13:43:01.369 INFO:tasks.workunit.client.0.smithi107.stdout:3/157: dread d1/fc [0,4194304] 0 2024-03-04T13:43:01.370 INFO:tasks.workunit.client.0.smithi107.stdout:1/191: rename d1/d10/d16/d1b/f1e to d1/d10/d16/f37 0 2024-03-04T13:43:01.371 INFO:tasks.workunit.client.1.smithi169.stdout:0/568: write d5/d9/dc/d12/d15/d46/f57 [4036951,89158] 0 2024-03-04T13:43:01.373 INFO:tasks.workunit.client.1.smithi169.stdout:0/569: readlink d5/d9/d3d/d59/l3b 0 2024-03-04T13:43:01.373 INFO:tasks.workunit.client.1.smithi169.stdout:7/766: rename d2/de/d3e/d41/d69/cbe to d2/d15/d17/c101 0 2024-03-04T13:43:01.374 INFO:tasks.workunit.client.0.smithi107.stdout:1/192: truncate d1/d3/df/f32 722654 0 2024-03-04T13:43:01.374 INFO:tasks.workunit.client.0.smithi107.stdout:4/169: dwrite d0/d9/de/d14/dc/d21/f2c [0,4194304] 0 2024-03-04T13:43:01.375 INFO:tasks.workunit.client.1.smithi169.stdout:2/751: write d1/d5/d19/f3f [4485575,22438] 0 2024-03-04T13:43:01.376 INFO:tasks.workunit.client.0.smithi107.stdout:1/193: dread d1/d3/df/f32 [0,4194304] 0 2024-03-04T13:43:01.381 INFO:tasks.workunit.client.0.smithi107.stdout:9/211: dread d4/da/f1b [0,4194304] 0 2024-03-04T13:43:01.384 INFO:tasks.workunit.client.1.smithi169.stdout:1/656: write d4/d5/d2d/f74 [688999,43507] 0 2024-03-04T13:43:01.386 INFO:tasks.workunit.client.0.smithi107.stdout:7/178: mkdir d6/d13/d16/d19/d39 0 2024-03-04T13:43:01.387 INFO:tasks.workunit.client.0.smithi107.stdout:7/179: readlink d6/d13/d16/d19/l2e 0 2024-03-04T13:43:01.388 INFO:tasks.workunit.client.0.smithi107.stdout:8/231: dwrite d6/d16/f22 [0,4194304] 0 2024-03-04T13:43:01.389 INFO:tasks.workunit.client.0.smithi107.stdout:2/223: chown da/c2f 5321 1 2024-03-04T13:43:01.389 INFO:tasks.workunit.client.1.smithi169.stdout:4/679: fdatasync d4/d11/d3b/d67/dbb/d6e/fd4 0 2024-03-04T13:43:01.399 INFO:tasks.workunit.client.1.smithi169.stdout:8/711: dwrite d7/d10/d17/faa [0,4194304] 0 2024-03-04T13:43:01.404 INFO:tasks.workunit.client.1.smithi169.stdout:9/764: creat d7/dc/f118 x:0 0 0 2024-03-04T13:43:01.404 INFO:tasks.workunit.client.1.smithi169.stdout:2/752: dread d1/d5/d19/f3f [4194304,4194304] 0 2024-03-04T13:43:01.404 INFO:tasks.workunit.client.1.smithi169.stdout:7/767: dwrite d2/de/d37/fd0 [0,4194304] 0 2024-03-04T13:43:01.409 INFO:tasks.workunit.client.1.smithi169.stdout:7/768: dread - d2/de/d3e/d7e/fdd zero size 2024-03-04T13:43:01.410 INFO:tasks.workunit.client.1.smithi169.stdout:1/657: chown d4/d5/d36/d75/dba/lbf 537 1 2024-03-04T13:43:01.415 INFO:tasks.workunit.client.1.smithi169.stdout:6/707: link d0/d32/d3a/d5e/ce9 d0/d63/c102 0 2024-03-04T13:43:01.416 INFO:tasks.workunit.client.1.smithi169.stdout:0/570: dwrite d5/d9/dc/d12/d15/f9e [8388608,4194304] 0 2024-03-04T13:43:01.416 INFO:tasks.workunit.client.0.smithi107.stdout:1/194: stat d1/d10/d20/d24/l36 0 2024-03-04T13:43:01.420 INFO:tasks.workunit.client.1.smithi169.stdout:4/680: fsync d4/d11/d2e/d39/d48/f4e 0 2024-03-04T13:43:01.421 INFO:tasks.workunit.client.1.smithi169.stdout:9/765: mknod d7/d27/dc3/c119 0 2024-03-04T13:43:01.424 INFO:tasks.workunit.client.1.smithi169.stdout:2/753: creat d1/d6d/f101 x:0 0 0 2024-03-04T13:43:01.425 INFO:tasks.workunit.client.1.smithi169.stdout:7/769: unlink d2/c31 0 2024-03-04T13:43:01.426 INFO:tasks.workunit.client.1.smithi169.stdout:4/681: chown d4/d11/d2e/d37/d73/cbe 98227 1 2024-03-04T13:43:01.427 INFO:tasks.workunit.client.1.smithi169.stdout:9/766: write d7/dc/d12/d32/d42/f46 [6125434,118879] 0 2024-03-04T13:43:01.429 INFO:tasks.workunit.client.1.smithi169.stdout:2/754: readlink d1/d5/d22/d3b/d41/l80 0 2024-03-04T13:43:01.429 INFO:tasks.workunit.client.0.smithi107.stdout:2/224: mknod da/d39/c4b 0 2024-03-04T13:43:01.430 INFO:tasks.workunit.client.0.smithi107.stdout:2/225: chown da/d44 66745903 1 2024-03-04T13:43:01.431 INFO:tasks.workunit.client.1.smithi169.stdout:1/658: symlink d4/d5/d36/d35/df/ld3 0 2024-03-04T13:43:01.431 INFO:tasks.workunit.client.0.smithi107.stdout:3/158: creat d1/d1e/d26/f35 x:0 0 0 2024-03-04T13:43:01.432 INFO:tasks.workunit.client.1.smithi169.stdout:6/708: truncate d0/d32/d3a/f21 2259615 0 2024-03-04T13:43:01.433 INFO:tasks.workunit.client.0.smithi107.stdout:8/232: dread d6/f29 [0,4194304] 0 2024-03-04T13:43:01.434 INFO:tasks.workunit.client.0.smithi107.stdout:5/155: creat d5/f32 x:0 0 0 2024-03-04T13:43:01.435 INFO:tasks.workunit.client.0.smithi107.stdout:5/156: fdatasync d5/f24 0 2024-03-04T13:43:01.436 INFO:tasks.workunit.client.0.smithi107.stdout:5/157: fsync d5/f29 0 2024-03-04T13:43:01.436 INFO:tasks.workunit.client.1.smithi169.stdout:9/767: stat d7/d2d/lbb 0 2024-03-04T13:43:01.437 INFO:tasks.workunit.client.1.smithi169.stdout:2/755: read d1/d5/d22/d45/d4a/f70 [1532006,62881] 0 2024-03-04T13:43:01.439 INFO:tasks.workunit.client.1.smithi169.stdout:0/571: mkdir d5/d9/dc/d12/d15/d46/d98/dc1/dc8 0 2024-03-04T13:43:01.439 INFO:tasks.workunit.client.1.smithi169.stdout:2/756: chown d1/d5/d19/d63/f8e 613 1 2024-03-04T13:43:01.444 INFO:tasks.workunit.client.0.smithi107.stdout:0/252: rename l9 to da/dc/dd/d3e/l4f 0 2024-03-04T13:43:01.447 INFO:tasks.workunit.client.0.smithi107.stdout:2/226: creat da/df/d3e/f4c x:0 0 0 2024-03-04T13:43:01.453 INFO:tasks.workunit.client.0.smithi107.stdout:1/195: mkdir d1/d38 0 2024-03-04T13:43:01.453 INFO:tasks.workunit.client.1.smithi169.stdout:9/768: unlink d7/fe 0 2024-03-04T13:43:01.454 INFO:tasks.workunit.client.0.smithi107.stdout:1/196: write d1/d10/d20/f2c [1181419,60588] 0 2024-03-04T13:43:01.455 INFO:tasks.workunit.client.1.smithi169.stdout:9/769: fdatasync d7/dc/d12/d20/d4e/fc8 0 2024-03-04T13:43:01.458 INFO:tasks.workunit.client.1.smithi169.stdout:1/659: creat d4/d5/d36/d35/d70/d78/fd4 x:0 0 0 2024-03-04T13:43:01.459 INFO:tasks.workunit.client.1.smithi169.stdout:7/770: dread d2/de/f53 [0,4194304] 0 2024-03-04T13:43:01.459 INFO:tasks.workunit.client.0.smithi107.stdout:7/180: truncate d6/d1d/f32 717017 0 2024-03-04T13:43:01.459 INFO:tasks.workunit.client.1.smithi169.stdout:1/660: readlink d4/d5/l14 0 2024-03-04T13:43:01.464 INFO:tasks.workunit.client.1.smithi169.stdout:0/572: creat d5/d9/dc/d12/d15/d19/d61/d71/d8c/fc9 x:0 0 0 2024-03-04T13:43:01.465 INFO:tasks.workunit.client.1.smithi169.stdout:0/573: chown d5/d9/l56 125014 1 2024-03-04T13:43:01.466 INFO:tasks.workunit.client.1.smithi169.stdout:4/682: truncate d4/d11/f79 3441950 0 2024-03-04T13:43:01.467 INFO:tasks.workunit.client.0.smithi107.stdout:5/158: mknod d5/dc/df/c33 0 2024-03-04T13:43:01.468 INFO:tasks.workunit.client.1.smithi169.stdout:7/771: dread d2/d15/d17/f1d [0,4194304] 0 2024-03-04T13:43:01.468 INFO:tasks.workunit.client.0.smithi107.stdout:7/181: dread d6/d1d/f33 [0,4194304] 0 2024-03-04T13:43:01.470 INFO:tasks.workunit.client.0.smithi107.stdout:6/233: rename df/d2e/d36/f45 to df/d1b/d2c/d31/f48 0 2024-03-04T13:43:01.472 INFO:tasks.workunit.client.0.smithi107.stdout:6/234: write df/d17/f28 [5179101,50718] 0 2024-03-04T13:43:01.476 INFO:tasks.workunit.client.0.smithi107.stdout:9/212: truncate d4/ff 986439 0 2024-03-04T13:43:01.476 INFO:tasks.workunit.client.0.smithi107.stdout:4/170: getdents d0/d9/de 0 2024-03-04T13:43:01.479 INFO:tasks.workunit.client.0.smithi107.stdout:9/213: truncate d4/da/d11/d20/d2f/f3d 93182 0 2024-03-04T13:43:01.480 INFO:tasks.workunit.client.1.smithi169.stdout:9/770: fsync d7/dc/df1/d56/f6e 0 2024-03-04T13:43:01.489 INFO:tasks.workunit.client.1.smithi169.stdout:6/709: creat d0/d32/d3a/d43/d74/f103 x:0 0 0 2024-03-04T13:43:01.489 INFO:tasks.workunit.client.0.smithi107.stdout:1/197: dread d1/d10/d16/d1b/f23 [0,4194304] 0 2024-03-04T13:43:01.492 INFO:tasks.workunit.client.0.smithi107.stdout:5/159: creat d5/d8/d14/f34 x:0 0 0 2024-03-04T13:43:01.493 INFO:tasks.workunit.client.1.smithi169.stdout:1/661: mknod d4/d5/d2d/d4e/cd5 0 2024-03-04T13:43:01.496 INFO:tasks.workunit.client.0.smithi107.stdout:7/182: symlink d6/d17/l3a 0 2024-03-04T13:43:01.497 INFO:tasks.workunit.client.1.smithi169.stdout:9/771: dwrite d7/dc/d12/d32/d42/fd6 [0,4194304] 0 2024-03-04T13:43:01.501 INFO:tasks.workunit.client.1.smithi169.stdout:0/574: fsync d5/d9/dc/d12/d15/d19/d77/f9b 0 2024-03-04T13:43:01.502 INFO:tasks.workunit.client.1.smithi169.stdout:9/772: dread - d7/dc/d12/d32/d42/fe4 zero size 2024-03-04T13:43:01.510 INFO:tasks.workunit.client.0.smithi107.stdout:7/183: dread d6/d13/d16/d1c/f35 [0,4194304] 0 2024-03-04T13:43:01.511 INFO:tasks.workunit.client.0.smithi107.stdout:0/253: symlink da/dc/dd/d3e/d47/l50 0 2024-03-04T13:43:01.511 INFO:tasks.workunit.client.1.smithi169.stdout:6/710: dwrite d0/f29 [4194304,4194304] 0 2024-03-04T13:43:01.515 INFO:tasks.workunit.client.0.smithi107.stdout:6/235: creat df/d2e/d36/f49 x:0 0 0 2024-03-04T13:43:01.518 INFO:tasks.workunit.client.0.smithi107.stdout:6/236: truncate df/d1b/d2c/d31/f48 282857 0 2024-03-04T13:43:01.518 INFO:tasks.workunit.client.0.smithi107.stdout:4/171: mkdir d0/d9/de/d14/d33 0 2024-03-04T13:43:01.523 INFO:tasks.workunit.client.1.smithi169.stdout:5/829: sync 2024-03-04T13:43:01.523 INFO:tasks.workunit.client.1.smithi169.stdout:3/721: sync 2024-03-04T13:43:01.523 INFO:tasks.workunit.client.0.smithi107.stdout:3/159: link d1/de/d11/d12/c30 d1/de/d11/d12/c36 0 2024-03-04T13:43:01.528 INFO:tasks.workunit.client.0.smithi107.stdout:1/198: symlink d1/d10/d20/d24/l39 0 2024-03-04T13:43:01.529 INFO:tasks.workunit.client.1.smithi169.stdout:2/757: rmdir d1/d5/d19/d3c/db8/de1/d64/dce/dda 0 2024-03-04T13:43:01.529 INFO:tasks.workunit.client.1.smithi169.stdout:3/722: truncate d7/de/d2b/d7d/d85/d13/d1a/d78/fdb 478169 0 2024-03-04T13:43:01.530 INFO:tasks.workunit.client.0.smithi107.stdout:5/160: creat d5/dc/df/f35 x:0 0 0 2024-03-04T13:43:01.530 INFO:tasks.workunit.client.1.smithi169.stdout:2/758: stat d1/d5/d22/d3b/f100 0 2024-03-04T13:43:01.532 INFO:tasks.workunit.client.1.smithi169.stdout:3/723: dread - d7/de/d2b/d7d/d85/d18/d21/fe4 zero size 2024-03-04T13:43:01.533 INFO:tasks.workunit.client.0.smithi107.stdout:5/161: truncate d5/dc/df/f35 157722 0 2024-03-04T13:43:01.533 INFO:tasks.workunit.client.0.smithi107.stdout:8/233: rename d6/d16/f1c to d6/d16/d1f/f43 0 2024-03-04T13:43:01.535 INFO:tasks.workunit.client.1.smithi169.stdout:0/575: creat d5/d9/dc/d12/d66/fca x:0 0 0 2024-03-04T13:43:01.537 INFO:tasks.workunit.client.0.smithi107.stdout:3/160: dread d1/de/d18/d33/f34 [0,4194304] 0 2024-03-04T13:43:01.537 INFO:tasks.workunit.client.0.smithi107.stdout:5/162: dread d5/f7 [0,4194304] 0 2024-03-04T13:43:01.538 INFO:tasks.workunit.client.0.smithi107.stdout:3/161: chown d1/de/d11 809263 1 2024-03-04T13:43:01.539 INFO:tasks.workunit.client.1.smithi169.stdout:4/683: dwrite d4/d11/d2e/d39/d48/f4e [0,4194304] 0 2024-03-04T13:43:01.539 INFO:tasks.workunit.client.0.smithi107.stdout:3/162: dread - d1/d1e/d26/f35 zero size 2024-03-04T13:43:01.542 INFO:tasks.workunit.client.1.smithi169.stdout:6/711: unlink d0/d32/d3a/d43/d74/l8e 0 2024-03-04T13:43:01.543 INFO:tasks.workunit.client.1.smithi169.stdout:8/712: sync 2024-03-04T13:43:01.543 INFO:tasks.workunit.client.1.smithi169.stdout:0/576: chown d5/d9/dc/d12/d15/d19/d61/d71/cc7 60028 1 2024-03-04T13:43:01.543 INFO:tasks.workunit.client.0.smithi107.stdout:9/214: creat d4/da/d11/d20/d31/d3b/f3e x:0 0 0 2024-03-04T13:43:01.543 INFO:tasks.workunit.client.0.smithi107.stdout:4/172: creat d0/d9/d10/f34 x:0 0 0 2024-03-04T13:43:01.545 INFO:tasks.workunit.client.1.smithi169.stdout:0/577: chown d5/d9/d3d/d59/d64/c96 1 1 2024-03-04T13:43:01.546 INFO:tasks.workunit.client.0.smithi107.stdout:9/215: write d4/da/d11/d20/d39/f3a [752775,51403] 0 2024-03-04T13:43:01.547 INFO:tasks.workunit.client.1.smithi169.stdout:7/772: stat d2/de/d3e/d41/d5b/d7a/dd3/fe4 0 2024-03-04T13:43:01.547 INFO:tasks.workunit.client.1.smithi169.stdout:0/578: readlink d5/d9/dc/d12/d15/l87 0 2024-03-04T13:43:01.549 INFO:tasks.workunit.client.0.smithi107.stdout:6/237: dread fd [0,4194304] 0 2024-03-04T13:43:01.550 INFO:tasks.workunit.client.0.smithi107.stdout:0/254: dread da/dc/dd/d3e/f44 [0,4194304] 0 2024-03-04T13:43:01.550 INFO:tasks.workunit.client.0.smithi107.stdout:2/227: rename da/d39/c4b to da/d29/c4d 0 2024-03-04T13:43:01.554 INFO:tasks.workunit.client.0.smithi107.stdout:5/163: symlink d5/dc/d22/l36 0 2024-03-04T13:43:01.556 INFO:tasks.workunit.client.0.smithi107.stdout:3/163: symlink d1/de/d11/l37 0 2024-03-04T13:43:01.559 INFO:tasks.workunit.client.0.smithi107.stdout:3/164: chown d1/de/d11/d12/c36 830 1 2024-03-04T13:43:01.560 INFO:tasks.workunit.client.1.smithi169.stdout:2/759: rename d1/d5/d19/de9/cec to d1/d5/d22/d45/d66/c102 0 2024-03-04T13:43:01.561 INFO:tasks.workunit.client.0.smithi107.stdout:6/238: rmdir df/d2e 39 2024-03-04T13:43:01.562 INFO:tasks.workunit.client.1.smithi169.stdout:9/773: dwrite d7/dc/d3e/fab [0,4194304] 0 2024-03-04T13:43:01.563 INFO:tasks.workunit.client.0.smithi107.stdout:2/228: dread f1 [4194304,4194304] 0 2024-03-04T13:43:01.565 INFO:tasks.workunit.client.0.smithi107.stdout:7/184: rename d6/f12 to d6/d1d/f3b 0 2024-03-04T13:43:01.567 INFO:tasks.workunit.client.1.smithi169.stdout:6/712: rmdir d0/d32 39 2024-03-04T13:43:01.569 INFO:tasks.workunit.client.0.smithi107.stdout:3/165: dread d1/f3 [8388608,4194304] 0 2024-03-04T13:43:01.570 INFO:tasks.workunit.client.1.smithi169.stdout:1/662: truncate d4/d5/d2d/d4e/d71/d26/f2e 4290272 0 2024-03-04T13:43:01.572 INFO:tasks.workunit.client.1.smithi169.stdout:1/663: stat d4/d1d/f8e 0 2024-03-04T13:43:01.572 INFO:tasks.workunit.client.0.smithi107.stdout:3/166: truncate d1/d29/f2a 1646256 0 2024-03-04T13:43:01.572 INFO:tasks.workunit.client.0.smithi107.stdout:1/199: truncate d1/d10/d16/f37 751694 0 2024-03-04T13:43:01.576 INFO:tasks.workunit.client.0.smithi107.stdout:4/173: link d0/d9/de/d14/dc/f27 d0/d9/d18/d29/f35 0 2024-03-04T13:43:01.576 INFO:tasks.workunit.client.1.smithi169.stdout:4/684: write d4/d11/d3b/d67/d7e/f9c [1373803,111081] 0 2024-03-04T13:43:01.578 INFO:tasks.workunit.client.1.smithi169.stdout:4/685: fdatasync d4/f41 0 2024-03-04T13:43:01.582 INFO:tasks.workunit.client.1.smithi169.stdout:8/713: rename d7/d10/d17/faa to d7/d10/d22/d75/d81/da2/dac/fe7 0 2024-03-04T13:43:01.582 INFO:tasks.workunit.client.0.smithi107.stdout:4/174: write d0/d9/de/d14/dc/d21/f2c [2416650,46298] 0 2024-03-04T13:43:01.584 INFO:tasks.workunit.client.0.smithi107.stdout:4/175: stat d0/d30/c31 0 2024-03-04T13:43:01.587 INFO:tasks.workunit.client.1.smithi169.stdout:9/774: symlink d7/d25/d64/d10a/l11a 0 2024-03-04T13:43:01.591 INFO:tasks.workunit.client.0.smithi107.stdout:4/176: dread d0/d9/de/d14/dc/f26 [0,4194304] 0 2024-03-04T13:43:01.592 INFO:tasks.workunit.client.0.smithi107.stdout:7/185: symlink d6/d13/d16/d19/l3c 0 2024-03-04T13:43:01.592 INFO:tasks.workunit.client.0.smithi107.stdout:2/229: dread da/df/f13 [0,4194304] 0 2024-03-04T13:43:01.593 INFO:tasks.workunit.client.0.smithi107.stdout:4/177: chown d0/d2f/l32 799669499 1 2024-03-04T13:43:01.593 INFO:tasks.workunit.client.0.smithi107.stdout:0/255: mknod da/dc/dd/c51 0 2024-03-04T13:43:01.595 INFO:tasks.workunit.client.0.smithi107.stdout:4/178: write d0/d9/de/d14/dc/d21/f28 [643613,123675] 0 2024-03-04T13:43:01.596 INFO:tasks.workunit.client.1.smithi169.stdout:3/724: write d7/de/d2b/d7d/d85/d47/f59 [380861,27211] 0 2024-03-04T13:43:01.598 INFO:tasks.workunit.client.0.smithi107.stdout:4/179: readlink d0/d9/de/d14/l23 0 2024-03-04T13:43:01.599 INFO:tasks.workunit.client.0.smithi107.stdout:4/180: readlink d0/d2f/l32 0 2024-03-04T13:43:01.603 INFO:tasks.workunit.client.0.smithi107.stdout:3/167: fsync d1/de/f32 0 2024-03-04T13:43:01.603 INFO:tasks.workunit.client.1.smithi169.stdout:0/579: dread d5/d9/dc/d12/f44 [4194304,4194304] 0 2024-03-04T13:43:01.605 INFO:tasks.workunit.client.1.smithi169.stdout:5/830: rename d1/d76/le1 to d1/d16/d17/d2d/d5a/d8a/l128 0 2024-03-04T13:43:01.607 INFO:tasks.workunit.client.0.smithi107.stdout:1/200: rename d1/d10/d16/d1b/f23 to d1/d3/df/f3a 0 2024-03-04T13:43:01.607 INFO:tasks.workunit.client.0.smithi107.stdout:9/216: dwrite d4/ff [0,4194304] 0 2024-03-04T13:43:01.607 INFO:tasks.workunit.client.1.smithi169.stdout:0/580: read d5/d9/dc/d12/d15/d19/d61/d71/d8c/d90/f97 [574676,49240] 0 2024-03-04T13:43:01.609 INFO:tasks.workunit.client.1.smithi169.stdout:7/773: write d2/de/d3e/d41/d5b/d7a/fcc [1395680,46988] 0 2024-03-04T13:43:01.610 INFO:tasks.workunit.client.1.smithi169.stdout:0/581: dread - d5/d9/dc/d12/d66/f78 zero size 2024-03-04T13:43:01.611 INFO:tasks.workunit.client.1.smithi169.stdout:7/774: readlink d2/d15/d17/l77 0 2024-03-04T13:43:01.611 INFO:tasks.workunit.client.1.smithi169.stdout:8/714: truncate d7/d10/d17/d44/d4c/fa3 776131 0 2024-03-04T13:43:01.611 INFO:tasks.workunit.client.1.smithi169.stdout:0/582: stat d5/d9/ce 0 2024-03-04T13:43:01.613 INFO:tasks.workunit.client.0.smithi107.stdout:5/164: truncate d5/dc/df/f35 127426 0 2024-03-04T13:43:01.615 INFO:tasks.workunit.client.1.smithi169.stdout:1/664: fsync d4/d5/d2d/d4e/d71/d26/f79 0 2024-03-04T13:43:01.620 INFO:tasks.workunit.client.0.smithi107.stdout:2/230: chown da/f20 1321 1 2024-03-04T13:43:01.622 INFO:tasks.workunit.client.0.smithi107.stdout:7/186: unlink d6/d13/d16/d19/d1e/d22/l2a 0 2024-03-04T13:43:01.624 INFO:tasks.workunit.client.0.smithi107.stdout:9/217: dread d4/ff [0,4194304] 0 2024-03-04T13:43:01.624 INFO:tasks.workunit.client.1.smithi169.stdout:5/831: chown d1/d60/fa0 49 1 2024-03-04T13:43:01.626 INFO:tasks.workunit.client.0.smithi107.stdout:8/234: getdents d6/d16/d1f 0 2024-03-04T13:43:01.627 INFO:tasks.workunit.client.1.smithi169.stdout:2/760: link d1/d5/d22/d3b/c65 d1/d5/d22/d3b/d41/c103 0 2024-03-04T13:43:01.628 INFO:tasks.workunit.client.0.smithi107.stdout:6/239: truncate df/d1b/d2c/d31/f48 122159 0 2024-03-04T13:43:01.629 INFO:tasks.workunit.client.0.smithi107.stdout:4/181: rmdir d0/d9/de/d14/dc 39 2024-03-04T13:43:01.633 INFO:tasks.workunit.client.1.smithi169.stdout:7/775: mknod d2/de/d3e/d41/d69/c102 0 2024-03-04T13:43:01.636 INFO:tasks.workunit.client.0.smithi107.stdout:3/168: write d1/f6 [1181097,79384] 0 2024-03-04T13:43:01.638 INFO:tasks.workunit.client.1.smithi169.stdout:4/686: truncate d4/d11/d2e/d37/d5c/d86/f99 3313762 0 2024-03-04T13:43:01.639 INFO:tasks.workunit.client.0.smithi107.stdout:1/201: truncate d1/d3/df/f32 1044031 0 2024-03-04T13:43:01.641 INFO:tasks.workunit.client.1.smithi169.stdout:1/665: creat d4/d5/d36/d35/df/fd6 x:0 0 0 2024-03-04T13:43:01.643 INFO:tasks.workunit.client.0.smithi107.stdout:5/165: creat d5/dc/df/f37 x:0 0 0 2024-03-04T13:43:01.645 INFO:tasks.workunit.client.0.smithi107.stdout:2/231: unlink da/df/l12 0 2024-03-04T13:43:01.647 INFO:tasks.workunit.client.1.smithi169.stdout:5/832: mkdir d1/d81/d91/d129 0 2024-03-04T13:43:01.650 INFO:tasks.workunit.client.0.smithi107.stdout:8/235: mknod d6/da/d25/d2d/d3e/c44 0 2024-03-04T13:43:01.651 INFO:tasks.workunit.client.0.smithi107.stdout:4/182: readlink d0/l17 0 2024-03-04T13:43:01.652 INFO:tasks.workunit.client.1.smithi169.stdout:4/687: dwrite d4/d11/d2e/d37/f57 [0,4194304] 0 2024-03-04T13:43:01.653 INFO:tasks.workunit.client.0.smithi107.stdout:6/240: fdatasync df/d17/f23 0 2024-03-04T13:43:01.653 INFO:tasks.workunit.client.0.smithi107.stdout:4/183: write d0/d9/de/d14/fa [3301503,24828] 0 2024-03-04T13:43:01.654 INFO:tasks.workunit.client.1.smithi169.stdout:4/688: chown d4/d11/d2e/d37/d5c 0 1 2024-03-04T13:43:01.654 INFO:tasks.workunit.client.0.smithi107.stdout:6/241: readlink df/d17/d24/l2a 0 2024-03-04T13:43:01.658 INFO:tasks.workunit.client.1.smithi169.stdout:3/725: creat d7/de/d2b/d7d/d85/d13/d1a/d2d/d56/d5e/d67/ff2 x:0 0 0 2024-03-04T13:43:01.663 INFO:tasks.workunit.client.1.smithi169.stdout:3/726: write d7/de/d2b/d7d/d85/d13/d5a/fcc [653659,107179] 0 2024-03-04T13:43:01.666 INFO:tasks.workunit.client.1.smithi169.stdout:9/775: rename d7/dc/d12/d59/ced to d7/d25/c11b 0 2024-03-04T13:43:01.682 INFO:tasks.workunit.client.0.smithi107.stdout:5/166: creat d5/d2b/f38 x:0 0 0 2024-03-04T13:43:01.682 INFO:tasks.workunit.client.0.smithi107.stdout:3/169: read d1/fc [2120028,20196] 0 2024-03-04T13:43:01.684 INFO:tasks.workunit.client.0.smithi107.stdout:8/236: dwrite d6/f39 [0,4194304] 0 2024-03-04T13:43:01.684 INFO:tasks.workunit.client.0.smithi107.stdout:0/256: creat da/dc/dd/d3e/f52 x:0 0 0 2024-03-04T13:43:01.685 INFO:tasks.workunit.client.0.smithi107.stdout:9/218: creat d4/d34/f3f x:0 0 0 2024-03-04T13:43:01.686 INFO:tasks.workunit.client.0.smithi107.stdout:3/170: chown d1/de/d11 4820 1 2024-03-04T13:43:01.688 INFO:tasks.workunit.client.0.smithi107.stdout:7/187: getdents d6/d13/d16/d19/d1e 0 2024-03-04T13:43:01.694 INFO:tasks.workunit.client.0.smithi107.stdout:7/188: chown d6/d13/d16 497 1 2024-03-04T13:43:01.694 INFO:tasks.workunit.client.0.smithi107.stdout:4/184: creat d0/d9/de/d14/f36 x:0 0 0 2024-03-04T13:43:01.694 INFO:tasks.workunit.client.0.smithi107.stdout:5/167: creat d5/d8/d2a/f39 x:0 0 0 2024-03-04T13:43:01.697 INFO:tasks.workunit.client.0.smithi107.stdout:9/219: symlink d4/da/d11/d20/d39/l40 0 2024-03-04T13:43:01.700 INFO:tasks.workunit.client.0.smithi107.stdout:6/242: getdents df 0 2024-03-04T13:43:01.703 INFO:tasks.workunit.client.0.smithi107.stdout:6/243: stat df/d17/f1e 0 2024-03-04T13:43:01.705 INFO:tasks.workunit.client.0.smithi107.stdout:0/257: rename da/dc/c39 to da/dc/dd/d3e/d47/c53 0 2024-03-04T13:43:01.709 INFO:tasks.workunit.client.0.smithi107.stdout:6/244: chown fd 85468424 1 2024-03-04T13:43:01.715 INFO:tasks.workunit.client.0.smithi107.stdout:9/220: getdents d4/da 0 2024-03-04T13:43:01.718 INFO:tasks.workunit.client.1.smithi169.stdout:8/715: link d7/d10/d24/l8f d7/d10/d17/d44/d4c/le8 0 2024-03-04T13:43:01.718 INFO:tasks.workunit.client.1.smithi169.stdout:0/583: link d5/d9/d3d/d59/d64/d7f/f9f d5/d9/dc/d12/d15/d19/fcb 0 2024-03-04T13:43:01.718 INFO:tasks.workunit.client.1.smithi169.stdout:4/689: creat d4/d11/d2e/d37/d5c/d86/d8f/fd5 x:0 0 0 2024-03-04T13:43:01.719 INFO:tasks.workunit.client.1.smithi169.stdout:1/666: mkdir d4/d5/d36/d35/d16/d81/db4/dd7 0 2024-03-04T13:43:01.719 INFO:tasks.workunit.client.0.smithi107.stdout:0/258: rmdir da/dc 39 2024-03-04T13:43:01.722 INFO:tasks.workunit.client.0.smithi107.stdout:2/232: dwrite da/f18 [0,4194304] 0 2024-03-04T13:43:01.723 INFO:tasks.workunit.client.0.smithi107.stdout:6/245: chown df/f15 19618 1 2024-03-04T13:43:01.724 INFO:tasks.workunit.client.1.smithi169.stdout:1/667: fdatasync d4/d5/d2d/d4e/d71/d82/fcb 0 2024-03-04T13:43:01.724 INFO:tasks.workunit.client.1.smithi169.stdout:8/716: fdatasync d7/d10/d24/d3e/f40 0 2024-03-04T13:43:01.724 INFO:tasks.workunit.client.0.smithi107.stdout:8/237: dwrite d6/fe [0,4194304] 0 2024-03-04T13:43:01.725 INFO:tasks.workunit.client.1.smithi169.stdout:2/761: link d1/d5/d19/d63/f8e d1/d5/d19/d3c/f104 0 2024-03-04T13:43:01.727 INFO:tasks.workunit.client.0.smithi107.stdout:3/171: dwrite d1/f14 [0,4194304] 0 2024-03-04T13:43:01.729 INFO:tasks.workunit.client.0.smithi107.stdout:7/189: dwrite d6/fa [0,4194304] 0 2024-03-04T13:43:01.730 INFO:tasks.workunit.client.1.smithi169.stdout:0/584: unlink d5/d9/dc/d12/d15/d19/l7b 0 2024-03-04T13:43:01.735 INFO:tasks.workunit.client.1.smithi169.stdout:4/690: creat d4/d1c/d58/d9d/dbc/fd6 x:0 0 0 2024-03-04T13:43:01.736 INFO:tasks.workunit.client.0.smithi107.stdout:9/221: rmdir d4/d34 39 2024-03-04T13:43:01.737 INFO:tasks.workunit.client.1.smithi169.stdout:8/717: mkdir d7/d10/d22/d75/de9 0 2024-03-04T13:43:01.737 INFO:tasks.workunit.client.1.smithi169.stdout:7/776: dwrite d2/d15/d17/d6f/f72 [4194304,4194304] 0 2024-03-04T13:43:01.738 INFO:tasks.workunit.client.1.smithi169.stdout:0/585: chown d5/d9/dc/d12/d15/d19/d61/d71/cc7 3225728 1 2024-03-04T13:43:01.740 INFO:tasks.workunit.client.1.smithi169.stdout:8/718: chown d7/d10/d24/d5c 534158 1 2024-03-04T13:43:01.740 INFO:tasks.workunit.client.0.smithi107.stdout:4/185: dwrite d0/d9/de/d14/dc/f26 [0,4194304] 0 2024-03-04T13:43:01.741 INFO:tasks.workunit.client.0.smithi107.stdout:8/238: creat d6/f45 x:0 0 0 2024-03-04T13:43:01.744 INFO:tasks.workunit.client.0.smithi107.stdout:3/172: symlink d1/de/d18/d1f/l38 0 2024-03-04T13:43:01.746 INFO:tasks.workunit.client.0.smithi107.stdout:4/186: write d0/d9/f11 [2480086,117256] 0 2024-03-04T13:43:01.746 INFO:tasks.workunit.client.1.smithi169.stdout:4/691: dread - d4/d11/d2e/d6c/d78/da5/fce zero size 2024-03-04T13:43:01.747 INFO:tasks.workunit.client.1.smithi169.stdout:3/727: link d7/de/d2b/d7d/d85/d13/d1a/d2d/d56/d5e/d67/d5c/d84/f96 d7/de/d2b/d7d/d85/d47/ff3 0 2024-03-04T13:43:01.748 INFO:tasks.workunit.client.1.smithi169.stdout:0/586: creat d5/d9/d3d/d59/d64/d7f/fcc x:0 0 0 2024-03-04T13:43:01.750 INFO:tasks.workunit.client.0.smithi107.stdout:0/259: truncate da/dc/f33 626123 0 2024-03-04T13:43:01.754 INFO:tasks.workunit.client.1.smithi169.stdout:9/776: dwrite d7/dc/d12/d32/d42/d4f/fb5 [0,4194304] 0 2024-03-04T13:43:01.755 INFO:tasks.workunit.client.0.smithi107.stdout:2/233: dread da/d39/f46 [4194304,4194304] 0 2024-03-04T13:43:01.757 INFO:tasks.workunit.client.1.smithi169.stdout:9/777: readlink d7/dc/d12/d20/da6/dd1/le2 0 2024-03-04T13:43:01.762 INFO:tasks.workunit.client.0.smithi107.stdout:9/222: write d4/da/d11/f13 [5763448,60086] 0 2024-03-04T13:43:01.763 INFO:tasks.workunit.client.0.smithi107.stdout:6/246: dwrite f6 [0,4194304] 0 2024-03-04T13:43:01.764 INFO:tasks.workunit.client.1.smithi169.stdout:0/587: creat d5/d4c/d65/d95/fcd x:0 0 0 2024-03-04T13:43:01.766 INFO:tasks.workunit.client.0.smithi107.stdout:0/260: mknod da/dc/dd/d3e/d47/c54 0 2024-03-04T13:43:01.768 INFO:tasks.workunit.client.1.smithi169.stdout:3/728: unlink d7/de/d2b/d7d/d85/d18/l26 0 2024-03-04T13:43:01.769 INFO:tasks.workunit.client.0.smithi107.stdout:0/261: write da/f32 [961643,31868] 0 2024-03-04T13:43:01.770 INFO:tasks.workunit.client.1.smithi169.stdout:2/762: getdents d1/d5/d22/d3b 0 2024-03-04T13:43:01.772 INFO:tasks.workunit.client.1.smithi169.stdout:3/729: dread - d7/de/d2b/d7d/d85/d18/d21/d3f/f41 zero size 2024-03-04T13:43:01.772 INFO:tasks.workunit.client.0.smithi107.stdout:7/190: dwrite f0 [0,4194304] 0 2024-03-04T13:43:01.774 INFO:tasks.workunit.client.0.smithi107.stdout:2/234: write da/d34/f41 [1034528,97974] 0 2024-03-04T13:43:01.775 INFO:tasks.workunit.client.0.smithi107.stdout:5/168: dread d5/dc/df/f12 [0,4194304] 0 2024-03-04T13:43:01.776 INFO:tasks.workunit.client.0.smithi107.stdout:4/187: creat d0/d9/f37 x:0 0 0 2024-03-04T13:43:01.778 INFO:tasks.workunit.client.1.smithi169.stdout:2/763: read d1/d5/d22/d3b/d99/f93 [33439,81909] 0 2024-03-04T13:43:01.780 INFO:tasks.workunit.client.0.smithi107.stdout:3/173: rmdir d1/d21 0 2024-03-04T13:43:01.783 INFO:tasks.workunit.client.1.smithi169.stdout:1/668: dread d4/d5/f5a [0,4194304] 0 2024-03-04T13:43:01.785 INFO:tasks.workunit.client.0.smithi107.stdout:6/247: dread df/d1b/f1d [0,4194304] 0 2024-03-04T13:43:01.791 INFO:tasks.workunit.client.1.smithi169.stdout:2/764: mkdir d1/d6d/dbc/d105 0 2024-03-04T13:43:01.791 INFO:tasks.workunit.client.0.smithi107.stdout:6/248: fsync df/d1b/d2c/d31/f39 0 2024-03-04T13:43:01.792 INFO:tasks.workunit.client.1.smithi169.stdout:7/777: truncate d2/f7 3484372 0 2024-03-04T13:43:01.792 INFO:tasks.workunit.client.0.smithi107.stdout:0/262: stat da/dc/dd/d3e/d26/l20 0 2024-03-04T13:43:01.796 INFO:tasks.workunit.client.1.smithi169.stdout:0/588: getdents d5/d9/dc/d12/da5 0 2024-03-04T13:43:01.797 INFO:tasks.workunit.client.1.smithi169.stdout:8/719: getdents d7/d10/d22/d75/d81/da2 0 2024-03-04T13:43:01.798 INFO:tasks.workunit.client.0.smithi107.stdout:4/188: symlink d0/d2f/l38 0 2024-03-04T13:43:01.802 INFO:tasks.workunit.client.1.smithi169.stdout:2/765: mkdir d1/d6d/dbc/d106 0 2024-03-04T13:43:01.803 INFO:tasks.workunit.client.1.smithi169.stdout:2/766: chown d1/d5/d19/f8c 142 1 2024-03-04T13:43:01.805 INFO:tasks.workunit.client.0.smithi107.stdout:6/249: dread fb [0,4194304] 0 2024-03-04T13:43:01.805 INFO:tasks.workunit.client.0.smithi107.stdout:9/223: dread d4/f21 [0,4194304] 0 2024-03-04T13:43:01.805 INFO:tasks.workunit.client.1.smithi169.stdout:3/730: link d7/de/d2b/d7d/d85/d13/d5a/fe0 d7/de/d2b/d7d/d85/d13/d1a/d98/d9a/ff4 0 2024-03-04T13:43:01.807 INFO:tasks.workunit.client.1.smithi169.stdout:2/767: read d1/d5/d19/f8c [7298304,2733] 0 2024-03-04T13:43:01.808 INFO:tasks.workunit.client.0.smithi107.stdout:3/174: write d1/de/d18/d33/f34 [1514799,125143] 0 2024-03-04T13:43:01.809 INFO:tasks.workunit.client.0.smithi107.stdout:5/169: dread d5/d8/fb [0,4194304] 0 2024-03-04T13:43:01.810 INFO:tasks.workunit.client.1.smithi169.stdout:4/692: dread d4/d11/d2e/d39/f60 [4194304,4194304] 0 2024-03-04T13:43:01.810 INFO:tasks.workunit.client.1.smithi169.stdout:7/778: rename d2/d28/c66 to d2/de/d3e/d41/d69/deb/c103 0 2024-03-04T13:43:01.810 INFO:tasks.workunit.client.1.smithi169.stdout:2/768: write d1/fc0 [2064796,114929] 0 2024-03-04T13:43:01.814 INFO:tasks.workunit.client.0.smithi107.stdout:8/239: unlink d6/da/l36 0 2024-03-04T13:43:01.824 INFO:tasks.workunit.client.1.smithi169.stdout:0/589: rename d5/d4c/cb6 to d5/d9/dc/d12/d15/d19/d77/cce 0 2024-03-04T13:43:01.826 INFO:tasks.workunit.client.1.smithi169.stdout:9/778: dread d7/dc/d3e/da3/fc9 [0,4194304] 0 2024-03-04T13:43:01.826 INFO:tasks.workunit.client.1.smithi169.stdout:5/833: sync 2024-03-04T13:43:01.826 INFO:tasks.workunit.client.1.smithi169.stdout:4/693: stat d4/d11/d3b/d67/dbb/lac 0 2024-03-04T13:43:01.826 INFO:tasks.workunit.client.1.smithi169.stdout:6/713: sync 2024-03-04T13:43:01.827 INFO:tasks.workunit.client.0.smithi107.stdout:4/189: creat d0/d2f/f39 x:0 0 0 2024-03-04T13:43:01.829 INFO:tasks.workunit.client.1.smithi169.stdout:1/669: dread d4/d5/d2d/d4e/d71/d26/d54/f72 [0,4194304] 0 2024-03-04T13:43:01.830 INFO:tasks.workunit.client.1.smithi169.stdout:5/834: dread - d1/fb0 zero size 2024-03-04T13:43:01.830 INFO:tasks.workunit.client.1.smithi169.stdout:9/779: write d7/d27/f110 [660867,99043] 0 2024-03-04T13:43:01.837 INFO:tasks.workunit.client.0.smithi107.stdout:9/224: creat d4/da/d11/d20/d23/f41 x:0 0 0 2024-03-04T13:43:01.839 INFO:tasks.workunit.client.0.smithi107.stdout:9/225: fsync d4/da/d11/d20/d31/f3c 0 2024-03-04T13:43:01.842 INFO:tasks.workunit.client.1.smithi169.stdout:3/731: dread d7/de/d2b/f6e [0,4194304] 0 2024-03-04T13:43:01.843 INFO:tasks.workunit.client.0.smithi107.stdout:9/226: dread d4/da/f1b [0,4194304] 0 2024-03-04T13:43:01.844 INFO:tasks.workunit.client.1.smithi169.stdout:0/590: creat d5/d9/dc/d12/d15/d19/d77/fcf x:0 0 0 2024-03-04T13:43:01.845 INFO:tasks.workunit.client.1.smithi169.stdout:3/732: chown d7/de/d2b/d7d/d85/d18/d21/d3f/d86/cd0 6 1 2024-03-04T13:43:01.846 INFO:tasks.workunit.client.0.smithi107.stdout:5/170: mknod d5/d2b/c3a 0 2024-03-04T13:43:01.847 INFO:tasks.workunit.client.0.smithi107.stdout:5/171: readlink d5/dc/d22/l36 0 2024-03-04T13:43:01.847 INFO:tasks.workunit.client.1.smithi169.stdout:4/694: fsync d4/f5a 0 2024-03-04T13:43:01.848 INFO:tasks.workunit.client.0.smithi107.stdout:5/172: dread - d5/d8/d14/f34 zero size 2024-03-04T13:43:01.849 INFO:tasks.workunit.client.0.smithi107.stdout:7/191: rmdir d6/d13/d31 0 2024-03-04T13:43:01.850 INFO:tasks.workunit.client.1.smithi169.stdout:4/695: write d4/d1c/fb2 [338093,97973] 0 2024-03-04T13:43:01.852 INFO:tasks.workunit.client.0.smithi107.stdout:6/250: write df/d17/d24/d27/f26 [242952,66950] 0 2024-03-04T13:43:01.853 INFO:tasks.workunit.client.1.smithi169.stdout:8/720: dread d7/d10/d24/d3e/f68 [0,4194304] 0 2024-03-04T13:43:01.853 INFO:tasks.workunit.client.1.smithi169.stdout:9/780: unlink d7/dc/d12/d32/d42/cd4 0 2024-03-04T13:43:01.854 INFO:tasks.workunit.client.0.smithi107.stdout:6/251: stat df/d17/d24/d30 0 2024-03-04T13:43:01.854 INFO:tasks.workunit.client.1.smithi169.stdout:5/835: fdatasync d1/d81/dc1/fca 0 2024-03-04T13:43:01.857 INFO:tasks.workunit.client.1.smithi169.stdout:8/721: chown d7/d10/d24/d5c 116818854 1 2024-03-04T13:43:01.858 INFO:tasks.workunit.client.1.smithi169.stdout:2/769: link d1/d5/d22/d3b/d41/dbb/ff8 d1/d5/d22/d45/f107 0 2024-03-04T13:43:01.858 INFO:tasks.workunit.client.1.smithi169.stdout:5/836: chown d1/d81/db1/ff0 93 1 2024-03-04T13:43:01.858 INFO:tasks.workunit.client.1.smithi169.stdout:7/779: symlink d2/de/dcf/l104 0 2024-03-04T13:43:01.861 INFO:tasks.workunit.client.1.smithi169.stdout:5/837: chown d1/d23/f85 8 1 2024-03-04T13:43:01.862 INFO:tasks.workunit.client.1.smithi169.stdout:2/770: write d1/d5/d22/d45/f90 [530666,94951] 0 2024-03-04T13:43:01.862 INFO:tasks.workunit.client.0.smithi107.stdout:2/235: truncate da/f2e 1287 0 2024-03-04T13:43:01.864 INFO:tasks.workunit.client.1.smithi169.stdout:5/838: fdatasync d1/d16/d2f/fde 0 2024-03-04T13:43:01.865 INFO:tasks.workunit.client.0.smithi107.stdout:9/227: fsync d4/d34/f3f 0 2024-03-04T13:43:01.865 INFO:tasks.workunit.client.1.smithi169.stdout:3/733: mkdir d7/de/d2b/d7d/d85/d13/df5 0 2024-03-04T13:43:01.866 INFO:tasks.workunit.client.0.smithi107.stdout:7/192: mkdir d6/d13/d16/d19/d1e/d3d 0 2024-03-04T13:43:01.867 INFO:tasks.workunit.client.0.smithi107.stdout:9/228: truncate d4/d34/f3f 345077 0 2024-03-04T13:43:01.867 INFO:tasks.workunit.client.0.smithi107.stdout:7/193: fsync d6/f7 0 2024-03-04T13:43:01.867 INFO:tasks.workunit.client.0.smithi107.stdout:8/240: mknod d6/d16/d1f/c46 0 2024-03-04T13:43:01.868 INFO:tasks.workunit.client.0.smithi107.stdout:7/194: chown d6/d13/d16/d19/d38 0 1 2024-03-04T13:43:01.869 INFO:tasks.workunit.client.1.smithi169.stdout:5/839: chown d1/d16/d17/d1b/d74/c9b 41837 1 2024-03-04T13:43:01.872 INFO:tasks.workunit.client.1.smithi169.stdout:0/591: dwrite d5/d9/dc/d12/d15/d19/d61/f5a [0,4194304] 0 2024-03-04T13:43:01.872 INFO:tasks.workunit.client.1.smithi169.stdout:9/781: rmdir d7/dc/d12/d20/da6/dd1 39 2024-03-04T13:43:01.874 INFO:tasks.workunit.client.1.smithi169.stdout:1/670: rename d4/dcd to d4/dd8 0 2024-03-04T13:43:01.887 INFO:tasks.workunit.client.1.smithi169.stdout:7/780: rename d2/de/d3e/d41/dec/ff2 to d2/de/d3e/d7e/dbb/de3/f105 0 2024-03-04T13:43:01.889 INFO:tasks.workunit.client.1.smithi169.stdout:3/734: creat d7/db8/ded/ff6 x:0 0 0 2024-03-04T13:43:01.889 INFO:tasks.workunit.client.1.smithi169.stdout:5/840: truncate d1/d16/d17/d2d/f89 64027 0 2024-03-04T13:43:01.891 INFO:tasks.workunit.client.1.smithi169.stdout:0/592: symlink d5/d9/d3d/d59/ld0 0 2024-03-04T13:43:01.896 INFO:tasks.workunit.client.1.smithi169.stdout:8/722: link d7/d10/fcc d7/fea 0 2024-03-04T13:43:01.896 INFO:tasks.workunit.client.1.smithi169.stdout:4/696: dread d4/d11/d2e/d37/d5c/fb0 [0,4194304] 0 2024-03-04T13:43:01.899 INFO:tasks.workunit.client.1.smithi169.stdout:4/697: write d4/d1c/f2c [2956862,107999] 0 2024-03-04T13:43:01.901 INFO:tasks.workunit.client.1.smithi169.stdout:0/593: rmdir d5/d4c/d65/d95 39 2024-03-04T13:43:01.902 INFO:tasks.workunit.client.1.smithi169.stdout:1/671: dwrite d4/d5/d2d/d4e/d71/d82/fcb [0,4194304] 0 2024-03-04T13:43:01.906 INFO:tasks.workunit.client.1.smithi169.stdout:2/771: write d1/d5/d22/d3b/f9c [641144,107042] 0 2024-03-04T13:43:01.906 INFO:tasks.workunit.client.1.smithi169.stdout:5/841: chown d1/d16/d17/d1b/d74/ddc/l118 32636492 1 2024-03-04T13:43:01.907 INFO:tasks.workunit.client.1.smithi169.stdout:0/594: chown d5/d4c/f51 7704 1 2024-03-04T13:43:01.908 INFO:tasks.workunit.client.1.smithi169.stdout:1/672: fdatasync d4/d5/d36/d75/dba/fbb 0 2024-03-04T13:43:01.911 INFO:tasks.workunit.client.1.smithi169.stdout:4/698: mknod d4/d11/d2e/d6c/cd7 0 2024-03-04T13:43:01.911 INFO:tasks.workunit.client.1.smithi169.stdout:2/772: fsync d1/d5/d22/d45/d4a/f7d 0 2024-03-04T13:43:01.912 INFO:tasks.workunit.client.1.smithi169.stdout:9/782: write d7/dc/d3e/da3/fad [2774274,76417] 0 2024-03-04T13:43:01.915 INFO:tasks.workunit.client.0.smithi107.stdout:2/236: creat da/d29/f4e x:0 0 0 2024-03-04T13:43:01.916 INFO:tasks.workunit.client.0.smithi107.stdout:0/263: rename da/dc/c10 to da/c55 0 2024-03-04T13:43:01.917 INFO:tasks.workunit.client.0.smithi107.stdout:2/237: readlink da/l23 0 2024-03-04T13:43:01.917 INFO:tasks.workunit.client.0.smithi107.stdout:4/190: write d0/d9/de/d14/dc/f2d [686971,89828] 0 2024-03-04T13:43:01.920 INFO:tasks.workunit.client.0.smithi107.stdout:0/264: chown da/dc/dd/d3e/d26/l2b 23931 1 2024-03-04T13:43:01.920 INFO:tasks.workunit.client.0.smithi107.stdout:3/175: getdents d1/de/d11 0 2024-03-04T13:43:01.921 INFO:tasks.workunit.client.1.smithi169.stdout:5/842: write d1/d16/dbd/fa9 [686117,109190] 0 2024-03-04T13:43:01.924 INFO:tasks.workunit.client.1.smithi169.stdout:0/595: mkdir d5/d9/d3d/d59/d64/d7f/dd1 0 2024-03-04T13:43:01.924 INFO:tasks.workunit.client.0.smithi107.stdout:3/176: write d1/d1e/d26/f35 [920280,45306] 0 2024-03-04T13:43:01.927 INFO:tasks.workunit.client.0.smithi107.stdout:7/195: mknod d6/d13/d16/d19/c3e 0 2024-03-04T13:43:01.927 INFO:tasks.workunit.client.0.smithi107.stdout:3/177: write d1/de/d18/d33/f34 [4613907,20337] 0 2024-03-04T13:43:01.928 INFO:tasks.workunit.client.0.smithi107.stdout:9/229: dread d4/da/d11/f13 [0,4194304] 0 2024-03-04T13:43:01.928 INFO:tasks.workunit.client.1.smithi169.stdout:8/723: truncate d7/d10/d24/d3e/f57 2327204 0 2024-03-04T13:43:01.929 INFO:tasks.workunit.client.0.smithi107.stdout:7/196: write d6/d13/f18 [637040,54585] 0 2024-03-04T13:43:01.932 INFO:tasks.workunit.client.0.smithi107.stdout:3/178: truncate d1/de/d18/d1f/f20 4630579 0 2024-03-04T13:43:01.934 INFO:tasks.workunit.client.1.smithi169.stdout:9/783: creat d7/dc/d95/f11c x:0 0 0 2024-03-04T13:43:01.934 INFO:tasks.workunit.client.1.smithi169.stdout:7/781: dwrite d2/de/d3e/d41/f4c [0,4194304] 0 2024-03-04T13:43:01.935 INFO:tasks.workunit.client.1.smithi169.stdout:1/673: symlink d4/d5/d2d/d4e/d71/d26/ld9 0 2024-03-04T13:43:01.938 INFO:tasks.workunit.client.0.smithi107.stdout:4/191: symlink d0/d9/d18/l3a 0 2024-03-04T13:43:01.939 INFO:tasks.workunit.client.1.smithi169.stdout:3/735: dwrite d7/de/d2b/d7d/d85/d13/d1a/d2d/d56/f7c [0,4194304] 0 2024-03-04T13:43:01.942 INFO:tasks.workunit.client.1.smithi169.stdout:8/724: chown d7/d10/d22/d75/d81/da4/cc2 110921 1 2024-03-04T13:43:01.943 INFO:tasks.workunit.client.1.smithi169.stdout:5/843: creat d1/d16/d2a/d62/f12a x:0 0 0 2024-03-04T13:43:01.945 INFO:tasks.workunit.client.0.smithi107.stdout:5/173: getdents d5/d2b 0 2024-03-04T13:43:01.946 INFO:tasks.workunit.client.1.smithi169.stdout:1/674: dread d4/d5/d36/d35/d5b/fb5 [0,4194304] 0 2024-03-04T13:43:01.950 INFO:tasks.workunit.client.0.smithi107.stdout:7/197: fdatasync d6/d13/d16/d1c/f1f 0 2024-03-04T13:43:01.951 INFO:tasks.workunit.client.1.smithi169.stdout:7/782: rmdir d2/de/d3e/d7e/dbb/de3 39 2024-03-04T13:43:01.951 INFO:tasks.workunit.client.1.smithi169.stdout:5/844: read - d1/d16/d17/d2d/d5a/d8a/fe8 zero size 2024-03-04T13:43:01.952 INFO:tasks.workunit.client.1.smithi169.stdout:8/725: chown d7/d10/d24/d3e/cd6 898 1 2024-03-04T13:43:01.952 INFO:tasks.workunit.client.1.smithi169.stdout:2/773: dwrite d1/d6d/f101 [0,4194304] 0 2024-03-04T13:43:01.952 INFO:tasks.workunit.client.0.smithi107.stdout:7/198: chown d6/d13/d16/d19/d38 0 1 2024-03-04T13:43:01.954 INFO:tasks.workunit.client.0.smithi107.stdout:8/241: dread d6/da/f11 [0,4194304] 0 2024-03-04T13:43:01.956 INFO:tasks.workunit.client.1.smithi169.stdout:2/774: readlink d1/d5/d22/d45/led 0 2024-03-04T13:43:01.956 INFO:tasks.workunit.client.1.smithi169.stdout:7/783: dread - d2/f95 zero size 2024-03-04T13:43:01.958 INFO:tasks.workunit.client.0.smithi107.stdout:8/242: read f3 [3469977,49811] 0 2024-03-04T13:43:01.959 INFO:tasks.workunit.client.1.smithi169.stdout:1/675: unlink d4/d5/d2d/d4e/d71/d82/fcb 0 2024-03-04T13:43:01.964 INFO:tasks.workunit.client.0.smithi107.stdout:4/192: rename d0/d9/de/d14/f36 to d0/d9/de/f3b 0 2024-03-04T13:43:01.965 INFO:tasks.workunit.client.0.smithi107.stdout:4/193: dread - d0/d9/f37 zero size 2024-03-04T13:43:01.966 INFO:tasks.workunit.client.0.smithi107.stdout:4/194: dread - d0/f2a zero size 2024-03-04T13:43:01.969 INFO:tasks.workunit.client.0.smithi107.stdout:5/174: creat d5/dc/f3b x:0 0 0 2024-03-04T13:43:01.977 INFO:tasks.workunit.client.1.smithi169.stdout:8/726: mkdir d7/d10/d22/d75/d81/da2/deb 0 2024-03-04T13:43:01.979 INFO:tasks.workunit.client.1.smithi169.stdout:0/596: dread d5/d9/dc/d12/f62 [0,4194304] 0 2024-03-04T13:43:01.981 INFO:tasks.workunit.client.0.smithi107.stdout:3/179: creat d1/d2d/f39 x:0 0 0 2024-03-04T13:43:01.982 INFO:tasks.workunit.client.1.smithi169.stdout:4/699: dread d4/d11/d2e/f51 [0,4194304] 0 2024-03-04T13:43:01.983 INFO:tasks.workunit.client.1.smithi169.stdout:5/845: rename d1/d16/d17/d2d/d5a/d67/cf9 to d1/d16/d2f/c12b 0 2024-03-04T13:43:01.989 INFO:tasks.workunit.client.0.smithi107.stdout:2/238: truncate da/f10 6404006 0 2024-03-04T13:43:01.990 INFO:tasks.workunit.client.0.smithi107.stdout:2/239: fsync da/f20 0 2024-03-04T13:43:01.992 INFO:tasks.workunit.client.0.smithi107.stdout:0/265: truncate da/f2a 192316 0 2024-03-04T13:43:01.995 INFO:tasks.workunit.client.0.smithi107.stdout:0/266: truncate da/dc/d34/f42 192733 0 2024-03-04T13:43:01.995 INFO:tasks.workunit.client.0.smithi107.stdout:4/195: rename d0/f2a to d0/d9/de/d14/dc/f3c 0 2024-03-04T13:43:01.996 INFO:tasks.workunit.client.0.smithi107.stdout:0/267: dread - da/dc/d34/f45 zero size 2024-03-04T13:43:01.996 INFO:tasks.workunit.client.1.smithi169.stdout:0/597: dwrite d5/d9/dc/d12/d15/d19/d61/f23 [4194304,4194304] 0 2024-03-04T13:43:01.996 INFO:tasks.workunit.client.0.smithi107.stdout:4/196: write d0/d9/f37 [330257,96297] 0 2024-03-04T13:43:01.997 INFO:tasks.workunit.client.1.smithi169.stdout:0/598: chown d5/d4c/d65 91711 1 2024-03-04T13:43:01.998 INFO:tasks.workunit.client.0.smithi107.stdout:5/175: symlink d5/d8/d2a/l3c 0 2024-03-04T13:43:01.999 INFO:tasks.workunit.client.1.smithi169.stdout:0/599: chown d5/d9/d3d/d59/d64/f81 3 1 2024-03-04T13:43:02.001 INFO:tasks.workunit.client.0.smithi107.stdout:3/180: unlink d1/f23 0 2024-03-04T13:43:02.006 INFO:tasks.workunit.client.1.smithi169.stdout:2/775: write d1/d5/d19/d68/fa7 [5527318,127287] 0 2024-03-04T13:43:02.006 INFO:tasks.workunit.client.1.smithi169.stdout:1/676: write d4/d5/d36/f59 [2073133,69526] 0 2024-03-04T13:43:02.007 INFO:tasks.workunit.client.0.smithi107.stdout:7/199: getdents d6/d13/d16/d19/d38/d37 0 2024-03-04T13:43:02.009 INFO:tasks.workunit.client.1.smithi169.stdout:1/677: stat d4/d1d/f34 0 2024-03-04T13:43:02.010 INFO:tasks.workunit.client.1.smithi169.stdout:9/784: dread d7/d25/d3c/f9e [0,4194304] 0 2024-03-04T13:43:02.012 INFO:tasks.workunit.client.0.smithi107.stdout:0/268: symlink da/dc/d34/l56 0 2024-03-04T13:43:02.015 INFO:tasks.workunit.client.0.smithi107.stdout:9/230: dwrite d4/f19 [0,4194304] 0 2024-03-04T13:43:02.016 INFO:tasks.workunit.client.0.smithi107.stdout:4/197: write d0/d9/de/f3b [1038736,45328] 0 2024-03-04T13:43:02.018 INFO:tasks.workunit.client.0.smithi107.stdout:5/176: creat d5/dc/f3d x:0 0 0 2024-03-04T13:43:02.020 INFO:tasks.workunit.client.0.smithi107.stdout:3/181: truncate d1/de/f1d 4050174 0 2024-03-04T13:43:02.020 INFO:tasks.workunit.client.0.smithi107.stdout:2/240: symlink da/d49/l4f 0 2024-03-04T13:43:02.020 INFO:tasks.workunit.client.0.smithi107.stdout:9/231: dread d4/da/d11/d20/d2f/f3d [0,4194304] 0 2024-03-04T13:43:02.020 INFO:tasks.workunit.client.0.smithi107.stdout:4/198: chown d0/d9/d18/d29/f35 4 1 2024-03-04T13:43:02.022 INFO:tasks.workunit.client.0.smithi107.stdout:5/177: fsync d5/f32 0 2024-03-04T13:43:02.023 INFO:tasks.workunit.client.0.smithi107.stdout:9/232: rename d4 to d4/da/d11/d42 22 2024-03-04T13:43:02.023 INFO:tasks.workunit.client.0.smithi107.stdout:3/182: chown d1/de/d11/l37 0 1 2024-03-04T13:43:02.025 INFO:tasks.workunit.client.0.smithi107.stdout:0/269: symlink da/dc/dd/d3e/l57 0 2024-03-04T13:43:02.025 INFO:tasks.workunit.client.0.smithi107.stdout:1/202: sync 2024-03-04T13:43:02.026 INFO:tasks.workunit.client.0.smithi107.stdout:2/241: stat da/df/c1a 0 2024-03-04T13:43:02.027 INFO:tasks.workunit.client.0.smithi107.stdout:2/242: chown da/d38 73754 1 2024-03-04T13:43:02.033 INFO:tasks.workunit.client.0.smithi107.stdout:4/199: rmdir d0/d9/de/d14 39 2024-03-04T13:43:02.036 INFO:tasks.workunit.client.0.smithi107.stdout:3/183: readlink d1/l2c 0 2024-03-04T13:43:02.037 INFO:tasks.workunit.client.0.smithi107.stdout:3/184: chown d1/d2d 1470 1 2024-03-04T13:43:02.038 INFO:tasks.workunit.client.0.smithi107.stdout:9/233: creat d4/d34/f43 x:0 0 0 2024-03-04T13:43:02.040 INFO:tasks.workunit.client.0.smithi107.stdout:1/203: unlink d1/fe 0 2024-03-04T13:43:02.040 INFO:tasks.workunit.client.0.smithi107.stdout:1/204: chown d1/d3/d17 1658449 1 2024-03-04T13:43:02.042 INFO:tasks.workunit.client.0.smithi107.stdout:0/270: creat da/dc/dd/d3e/d26/f58 x:0 0 0 2024-03-04T13:43:02.043 INFO:tasks.workunit.client.0.smithi107.stdout:2/243: unlink da/df/c31 0 2024-03-04T13:43:02.044 INFO:tasks.workunit.client.0.smithi107.stdout:2/244: chown da/f1c 739 1 2024-03-04T13:43:02.047 INFO:tasks.workunit.client.0.smithi107.stdout:3/185: mknod d1/d2d/c3a 0 2024-03-04T13:43:02.049 INFO:tasks.workunit.client.0.smithi107.stdout:3/186: write d1/d29/f2a [2499456,114695] 0 2024-03-04T13:43:02.049 INFO:tasks.workunit.client.0.smithi107.stdout:9/234: creat d4/d34/f44 x:0 0 0 2024-03-04T13:43:02.050 INFO:tasks.workunit.client.0.smithi107.stdout:8/243: dwrite d6/fd [0,4194304] 0 2024-03-04T13:43:02.050 INFO:tasks.workunit.client.1.smithi169.stdout:8/727: symlink d7/d10/d17/d44/lec 0 2024-03-04T13:43:02.052 INFO:tasks.workunit.client.0.smithi107.stdout:1/205: symlink d1/d10/l3b 0 2024-03-04T13:43:02.053 INFO:tasks.workunit.client.1.smithi169.stdout:8/728: dread - d7/d10/d17/d44/d36/d62/dc1/fe2 zero size 2024-03-04T13:43:02.054 INFO:tasks.workunit.client.0.smithi107.stdout:3/187: write d1/de/d18/d1f/f20 [2462403,8069] 0 2024-03-04T13:43:02.056 INFO:tasks.workunit.client.0.smithi107.stdout:9/235: read - d4/da/d11/d20/d23/d2c/f38 zero size 2024-03-04T13:43:02.060 INFO:tasks.workunit.client.1.smithi169.stdout:0/600: creat d5/d9/dc/d12/d15/d19/d61/d71/d8c/d90/fd2 x:0 0 0 2024-03-04T13:43:02.061 INFO:tasks.workunit.client.0.smithi107.stdout:3/188: write d1/de/f19 [6473829,112220] 0 2024-03-04T13:43:02.064 INFO:tasks.workunit.client.0.smithi107.stdout:4/200: dwrite d0/d9/d10/f25 [0,4194304] 0 2024-03-04T13:43:02.065 INFO:tasks.workunit.client.0.smithi107.stdout:8/244: write d6/da/f11 [4114334,12610] 0 2024-03-04T13:43:02.065 INFO:tasks.workunit.client.0.smithi107.stdout:2/245: dwrite da/df/f1e [0,4194304] 0 2024-03-04T13:43:02.065 INFO:tasks.workunit.client.1.smithi169.stdout:2/776: symlink d1/d5/d19/d3c/db8/de1/d64/dce/dd5/l108 0 2024-03-04T13:43:02.066 INFO:tasks.workunit.client.1.smithi169.stdout:9/785: stat d7/dc/d12/d8a/ldd 0 2024-03-04T13:43:02.070 INFO:tasks.workunit.client.1.smithi169.stdout:6/714: sync 2024-03-04T13:43:02.070 INFO:tasks.workunit.client.1.smithi169.stdout:9/786: chown d7/d2d/d48/d66/f114 3562 1 2024-03-04T13:43:02.072 INFO:tasks.workunit.client.1.smithi169.stdout:8/729: creat d7/d12/fed x:0 0 0 2024-03-04T13:43:02.073 INFO:tasks.workunit.client.1.smithi169.stdout:5/846: dwrite d1/d60/f127 [0,4194304] 0 2024-03-04T13:43:02.079 INFO:tasks.workunit.client.1.smithi169.stdout:0/601: creat d5/d9/dc/d12/d15/d19/d69/fd3 x:0 0 0 2024-03-04T13:43:02.095 INFO:tasks.workunit.client.1.smithi169.stdout:5/847: fdatasync d1/d16/d17/f79 0 2024-03-04T13:43:02.097 INFO:tasks.workunit.client.1.smithi169.stdout:2/777: dwrite d1/d5/d19/d3c/db8/fca [0,4194304] 0 2024-03-04T13:43:02.100 INFO:tasks.workunit.client.1.smithi169.stdout:4/700: link d4/d11/d2e/d37/d5c/l93 d4/d11/d2e/d39/d48/ld8 0 2024-03-04T13:43:02.104 INFO:tasks.workunit.client.1.smithi169.stdout:0/602: unlink d5/d9/dc/d12/d15/d19/d77/d94/cb1 0 2024-03-04T13:43:02.106 INFO:tasks.workunit.client.1.smithi169.stdout:6/715: creat d0/d32/d3a/d43/d74/d93/f104 x:0 0 0 2024-03-04T13:43:02.107 INFO:tasks.workunit.client.0.smithi107.stdout:5/178: getdents d5/d8/d2a 0 2024-03-04T13:43:02.108 INFO:tasks.workunit.client.0.smithi107.stdout:1/206: creat d1/d10/d20/d24/f3c x:0 0 0 2024-03-04T13:43:02.110 INFO:tasks.workunit.client.0.smithi107.stdout:2/246: rename f1 to da/df/d3e/f50 0 2024-03-04T13:43:02.110 INFO:tasks.workunit.client.0.smithi107.stdout:0/271: rename da/f32 to da/f59 0 2024-03-04T13:43:02.112 INFO:tasks.workunit.client.1.smithi169.stdout:8/730: symlink d7/d10/d24/lee 0 2024-03-04T13:43:02.113 INFO:tasks.workunit.client.0.smithi107.stdout:0/272: readlink da/dc/dd/d3e/d26/l2b 0 2024-03-04T13:43:02.113 INFO:tasks.workunit.client.0.smithi107.stdout:1/207: truncate d1/d3/d17/f34 860643 0 2024-03-04T13:43:02.114 INFO:tasks.workunit.client.1.smithi169.stdout:1/678: dread d4/d5/d36/d35/df/f46 [0,4194304] 0 2024-03-04T13:43:02.116 INFO:tasks.workunit.client.1.smithi169.stdout:5/848: mkdir d1/d16/d17/d1b/d74/d12c 0 2024-03-04T13:43:02.118 INFO:tasks.workunit.client.0.smithi107.stdout:8/245: dread d6/f3d [0,4194304] 0 2024-03-04T13:43:02.118 INFO:tasks.workunit.client.0.smithi107.stdout:5/179: symlink d5/d8/l3e 0 2024-03-04T13:43:02.119 INFO:tasks.workunit.client.0.smithi107.stdout:5/180: chown d5/d8/d14 522388 1 2024-03-04T13:43:02.119 INFO:tasks.workunit.client.1.smithi169.stdout:2/778: mknod d1/d6d/dbc/c109 0 2024-03-04T13:43:02.123 INFO:tasks.workunit.client.1.smithi169.stdout:0/603: rmdir d5/d9/dc/d12/d15/d46 39 2024-03-04T13:43:02.124 INFO:tasks.workunit.client.0.smithi107.stdout:2/247: dwrite da/fd [0,4194304] 0 2024-03-04T13:43:02.126 INFO:tasks.workunit.client.0.smithi107.stdout:4/201: rename d0/d9/de/d14/f1d to d0/d30/f3d 0 2024-03-04T13:43:02.131 INFO:tasks.workunit.client.0.smithi107.stdout:8/246: symlink d6/d16/l47 0 2024-03-04T13:43:02.134 INFO:tasks.workunit.client.1.smithi169.stdout:5/849: read d1/d16/d17/f71 [1315036,21420] 0 2024-03-04T13:43:02.135 INFO:tasks.workunit.client.0.smithi107.stdout:5/181: symlink d5/dc/df/l3f 0 2024-03-04T13:43:02.136 INFO:tasks.workunit.client.1.smithi169.stdout:1/679: mknod d4/d5/d36/d35/d70/d78/cda 0 2024-03-04T13:43:02.138 INFO:tasks.workunit.client.0.smithi107.stdout:2/248: symlink da/df/d27/l51 0 2024-03-04T13:43:02.139 INFO:tasks.workunit.client.0.smithi107.stdout:2/249: dread - da/d29/f4e zero size 2024-03-04T13:43:02.140 INFO:tasks.workunit.client.1.smithi169.stdout:2/779: mkdir d1/d5/d22/d45/d4a/de7/d10a 0 2024-03-04T13:43:02.140 INFO:tasks.workunit.client.0.smithi107.stdout:3/189: getdents d1/d1e/d26 0 2024-03-04T13:43:02.142 INFO:tasks.workunit.client.0.smithi107.stdout:3/190: write d1/de/d18/d33/f34 [2509623,83574] 0 2024-03-04T13:43:02.144 INFO:tasks.workunit.client.1.smithi169.stdout:3/736: sync 2024-03-04T13:43:02.144 INFO:tasks.workunit.client.0.smithi107.stdout:4/202: unlink d0/d2f/l32 0 2024-03-04T13:43:02.145 INFO:tasks.workunit.client.1.smithi169.stdout:6/716: mkdir d0/d105 0 2024-03-04T13:43:02.146 INFO:tasks.workunit.client.1.smithi169.stdout:9/787: dwrite f6 [4194304,4194304] 0 2024-03-04T13:43:02.146 INFO:tasks.workunit.client.0.smithi107.stdout:4/203: write d0/d9/de/d14/fa [1234500,10286] 0 2024-03-04T13:43:02.152 INFO:tasks.workunit.client.0.smithi107.stdout:4/204: dread d0/d9/de/d14/dc/d21/f28 [0,4194304] 0 2024-03-04T13:43:02.155 INFO:tasks.workunit.client.0.smithi107.stdout:3/191: rename d1/de/d18/d1f/l38 to d1/de/d11/l3b 0 2024-03-04T13:43:02.156 INFO:tasks.workunit.client.0.smithi107.stdout:3/192: stat d1/d2d/c3a 0 2024-03-04T13:43:02.157 INFO:tasks.workunit.client.0.smithi107.stdout:1/208: link d1/d3/df/l14 d1/d10/d20/l3d 0 2024-03-04T13:43:02.157 INFO:tasks.workunit.client.0.smithi107.stdout:5/182: rmdir d5/dc/df 39 2024-03-04T13:43:02.168 INFO:tasks.workunit.client.1.smithi169.stdout:5/850: write d1/d16/d17/d2d/d5a/d8a/fa5 [534158,40233] 0 2024-03-04T13:43:02.168 INFO:tasks.workunit.client.1.smithi169.stdout:3/737: dwrite d7/de/d2b/d7d/d85/d47/d63/d6c/f77 [0,4194304] 0 2024-03-04T13:43:02.168 INFO:tasks.workunit.client.0.smithi107.stdout:3/193: symlink d1/d2d/l3c 0 2024-03-04T13:43:02.172 INFO:tasks.workunit.client.1.smithi169.stdout:4/701: write d4/d11/d2e/d21/fb7 [357137,119309] 0 2024-03-04T13:43:02.175 INFO:tasks.workunit.client.0.smithi107.stdout:5/183: creat d5/d8/d2a/f40 x:0 0 0 2024-03-04T13:43:02.175 INFO:tasks.workunit.client.1.smithi169.stdout:1/680: mknod d4/d5/d36/d35/d16/d81/db4/dd7/cdb 0 2024-03-04T13:43:02.176 INFO:tasks.workunit.client.1.smithi169.stdout:4/702: read - d4/d11/d2e/d37/d5c/d86/d8f/fd5 zero size 2024-03-04T13:43:02.178 INFO:tasks.workunit.client.0.smithi107.stdout:3/194: creat d1/d1e/d26/f3d x:0 0 0 2024-03-04T13:43:02.180 INFO:tasks.workunit.client.1.smithi169.stdout:0/604: link d5/d4c/d65/f6b d5/d9/dc/d12/d15/d19/d61/d71/dbb/fd4 0 2024-03-04T13:43:02.180 INFO:tasks.workunit.client.1.smithi169.stdout:4/703: fdatasync d4/d11/d2e/d37/d5c/d86/d8f/fd5 0 2024-03-04T13:43:02.182 INFO:tasks.workunit.client.1.smithi169.stdout:8/731: link d7/d10/d17/d44/d36/d62/ld1 d7/d10/d17/d44/d5d/ddd/lef 0 2024-03-04T13:43:02.182 INFO:tasks.workunit.client.0.smithi107.stdout:8/247: dwrite f5 [0,4194304] 0 2024-03-04T13:43:02.185 INFO:tasks.workunit.client.1.smithi169.stdout:7/784: sync 2024-03-04T13:43:02.188 INFO:tasks.workunit.client.1.smithi169.stdout:2/780: creat d1/d5/d22/f10b x:0 0 0 2024-03-04T13:43:02.189 INFO:tasks.workunit.client.0.smithi107.stdout:4/205: link d0/l16 d0/d9/de/l3e 0 2024-03-04T13:43:02.189 INFO:tasks.workunit.client.0.smithi107.stdout:5/184: creat d5/dc/f41 x:0 0 0 2024-03-04T13:43:02.191 INFO:tasks.workunit.client.1.smithi169.stdout:6/717: creat d0/d3c/d40/df6/f106 x:0 0 0 2024-03-04T13:43:02.192 INFO:tasks.workunit.client.0.smithi107.stdout:5/185: readlink d5/dc/l1c 0 2024-03-04T13:43:02.192 INFO:tasks.workunit.client.0.smithi107.stdout:8/248: mknod d6/da/d25/d2a/c48 0 2024-03-04T13:43:02.193 INFO:tasks.workunit.client.0.smithi107.stdout:4/206: unlink d0/d9/d1e/f2e 0 2024-03-04T13:43:02.194 INFO:tasks.workunit.client.1.smithi169.stdout:4/704: symlink d4/d11/d2e/d6c/d78/da5/d77/ld9 0 2024-03-04T13:43:02.194 INFO:tasks.workunit.client.0.smithi107.stdout:5/186: truncate d5/dc/f3b 491772 0 2024-03-04T13:43:02.195 INFO:tasks.workunit.client.1.smithi169.stdout:7/785: mknod d2/d15/dad/c106 0 2024-03-04T13:43:02.196 INFO:tasks.workunit.client.1.smithi169.stdout:3/738: mkdir d7/de/d2b/d7d/d85/d13/d1a/d2d/d56/d5e/d67/dd7/df7 0 2024-03-04T13:43:02.196 INFO:tasks.workunit.client.1.smithi169.stdout:8/732: write d7/d10/d24/d3e/f95 [4317033,73949] 0 2024-03-04T13:43:02.197 INFO:tasks.workunit.client.0.smithi107.stdout:3/195: dread d1/f9 [0,4194304] 0 2024-03-04T13:43:02.199 INFO:tasks.workunit.client.0.smithi107.stdout:3/196: write d1/fa [971905,16482] 0 2024-03-04T13:43:02.199 INFO:tasks.workunit.client.0.smithi107.stdout:8/249: rename d6/d16/l21 to d6/da/d25/d2d/l49 0 2024-03-04T13:43:02.200 INFO:tasks.workunit.client.1.smithi169.stdout:2/781: creat d1/d5/d19/d3c/db8/f10c x:0 0 0 2024-03-04T13:43:02.202 INFO:tasks.workunit.client.0.smithi107.stdout:4/207: unlink d0/d9/de/d14/fa 0 2024-03-04T13:43:02.204 INFO:tasks.workunit.client.1.smithi169.stdout:9/788: getdents d7/d2d 0 2024-03-04T13:43:02.205 INFO:tasks.workunit.client.1.smithi169.stdout:9/789: chown d7/d2d/f36 252621 1 2024-03-04T13:43:02.207 INFO:tasks.workunit.client.1.smithi169.stdout:9/790: chown d7/dc/d12/d20/d4e/lc0 171097507 1 2024-03-04T13:43:02.207 INFO:tasks.workunit.client.0.smithi107.stdout:8/250: unlink d6/da/l1d 0 2024-03-04T13:43:02.211 INFO:tasks.workunit.client.1.smithi169.stdout:4/705: dread - d4/d1c/d58/f97 zero size 2024-03-04T13:43:02.212 INFO:tasks.workunit.client.0.smithi107.stdout:5/187: creat d5/f42 x:0 0 0 2024-03-04T13:43:02.214 INFO:tasks.workunit.client.1.smithi169.stdout:8/733: mknod d7/d10/d22/d75/d81/da4/cf0 0 2024-03-04T13:43:02.215 INFO:tasks.workunit.client.0.smithi107.stdout:5/188: chown d5/d8/d14/c28 51 1 2024-03-04T13:43:02.216 INFO:tasks.workunit.client.1.smithi169.stdout:3/739: read d7/de/d2b/d7d/d85/d13/d1a/d2d/d56/d5e/d67/f51 [27447,13773] 0 2024-03-04T13:43:02.216 INFO:tasks.workunit.client.1.smithi169.stdout:2/782: symlink d1/d5/d22/d45/d4a/de7/dcb/df7/l10d 0 2024-03-04T13:43:02.217 INFO:tasks.workunit.client.1.smithi169.stdout:3/740: fdatasync d7/db8/fc6 0 2024-03-04T13:43:02.218 INFO:tasks.workunit.client.1.smithi169.stdout:0/605: rmdir d5/d9/dc/d12/d66/dc3 0 2024-03-04T13:43:02.221 INFO:tasks.workunit.client.0.smithi107.stdout:5/189: mkdir d5/d8/d2a/d43 0 2024-03-04T13:43:02.222 INFO:tasks.workunit.client.1.smithi169.stdout:7/786: creat d2/d28/d70/ded/f107 x:0 0 0 2024-03-04T13:43:02.223 INFO:tasks.workunit.client.1.smithi169.stdout:4/706: write d4/d11/d2e/d37/d5c/f8c [1198659,5394] 0 2024-03-04T13:43:02.224 INFO:tasks.workunit.client.1.smithi169.stdout:1/681: getdents d4/d5/d36 0 2024-03-04T13:43:02.228 INFO:tasks.workunit.client.1.smithi169.stdout:2/783: creat d1/d5/d22/d45/d4a/de7/dcb/f10e x:0 0 0 2024-03-04T13:43:02.229 INFO:tasks.workunit.client.0.smithi107.stdout:8/251: dwrite d6/da/f35 [0,4194304] 0 2024-03-04T13:43:02.230 INFO:tasks.workunit.client.0.smithi107.stdout:8/252: chown d6/da 6 1 2024-03-04T13:43:02.231 INFO:tasks.workunit.client.1.smithi169.stdout:4/707: creat d4/d1c/d58/d9d/fda x:0 0 0 2024-03-04T13:43:02.232 INFO:tasks.workunit.client.1.smithi169.stdout:1/682: symlink d4/d5/d2d/d4e/da5/ldc 0 2024-03-04T13:43:02.234 INFO:tasks.workunit.client.1.smithi169.stdout:0/606: mknod d5/d9/d3d/d59/d64/cd5 0 2024-03-04T13:43:02.236 INFO:tasks.workunit.client.1.smithi169.stdout:2/784: creat d1/d5/d22/d3b/d99/f10f x:0 0 0 2024-03-04T13:43:02.237 INFO:tasks.workunit.client.0.smithi107.stdout:8/253: mknod d6/da/d25/d2d/d3e/c4a 0 2024-03-04T13:43:02.240 INFO:tasks.workunit.client.1.smithi169.stdout:8/734: creat d7/d10/d17/ff1 x:0 0 0 2024-03-04T13:43:02.240 INFO:tasks.workunit.client.1.smithi169.stdout:1/683: mknod d4/dd8/cdd 0 2024-03-04T13:43:02.241 INFO:tasks.workunit.client.1.smithi169.stdout:3/741: link d7/de/d2b/d7d/d85/d18/d21/d3f/d68/l60 d7/de/d2b/d7d/d85/d18/d21/d66/lf8 0 2024-03-04T13:43:02.246 INFO:tasks.workunit.client.1.smithi169.stdout:2/785: unlink d1/f3d 0 2024-03-04T13:43:02.251 INFO:tasks.workunit.client.0.smithi107.stdout:3/197: dread d1/d29/f2a [0,4194304] 0 2024-03-04T13:43:02.255 INFO:tasks.workunit.client.1.smithi169.stdout:0/607: rename d5/d9/dc/d12/d15/d19/d61/c43 to d5/d4c/d65/d95/dba/cd6 0 2024-03-04T13:43:02.260 INFO:tasks.workunit.client.1.smithi169.stdout:2/786: mknod d1/d5/d19/de9/c110 0 2024-03-04T13:43:02.261 INFO:tasks.workunit.client.1.smithi169.stdout:4/708: getdents d4/d11/d3b/d67 0 2024-03-04T13:43:02.263 INFO:tasks.workunit.client.1.smithi169.stdout:2/787: chown d1/fc0 0 1 2024-03-04T13:43:02.263 INFO:tasks.workunit.client.0.smithi107.stdout:3/198: link d1/d1e/d26/f35 d1/de/d11/f3e 0 2024-03-04T13:43:02.265 INFO:tasks.workunit.client.1.smithi169.stdout:8/735: truncate d7/d10/d24/d3e/f40 1419002 0 2024-03-04T13:43:02.266 INFO:tasks.workunit.client.1.smithi169.stdout:3/742: dwrite d7/de/d2b/d7d/d85/d18/d21/d3f/f69 [0,4194304] 0 2024-03-04T13:43:02.267 INFO:tasks.workunit.client.0.smithi107.stdout:1/209: dwrite d1/d10/d16/f37 [0,4194304] 0 2024-03-04T13:43:02.269 INFO:tasks.workunit.client.0.smithi107.stdout:1/210: write d1/d10/d16/f37 [3626185,128463] 0 2024-03-04T13:43:02.271 INFO:tasks.workunit.client.0.smithi107.stdout:3/199: fdatasync d1/de/d11/d12/f15 0 2024-03-04T13:43:02.273 INFO:tasks.workunit.client.0.smithi107.stdout:3/200: chown d1/de/f1d 210790253 1 2024-03-04T13:43:02.273 INFO:tasks.workunit.client.1.smithi169.stdout:5/851: dwrite d1/d16/d2a/d62/fb4 [0,4194304] 0 2024-03-04T13:43:02.276 INFO:tasks.workunit.client.0.smithi107.stdout:7/200: sync 2024-03-04T13:43:02.276 INFO:tasks.workunit.client.0.smithi107.stdout:6/252: sync 2024-03-04T13:43:02.277 INFO:tasks.workunit.client.1.smithi169.stdout:0/608: rmdir d5/d9/dc/d12/d15/d19/d77/d94 39 2024-03-04T13:43:02.284 INFO:tasks.workunit.client.1.smithi169.stdout:4/709: creat d4/d11/d2e/d63/fdb x:0 0 0 2024-03-04T13:43:02.285 INFO:tasks.workunit.client.1.smithi169.stdout:4/710: fdatasync d4/f69 0 2024-03-04T13:43:02.288 INFO:tasks.workunit.client.1.smithi169.stdout:5/852: unlink d1/d16/d17/d1b/d74/fac 0 2024-03-04T13:43:02.290 INFO:tasks.workunit.client.1.smithi169.stdout:4/711: chown d4/d1c/lb4 7421715 1 2024-03-04T13:43:02.290 INFO:tasks.workunit.client.0.smithi107.stdout:3/201: mkdir d1/de/d3f 0 2024-03-04T13:43:02.295 INFO:tasks.workunit.client.1.smithi169.stdout:8/736: mknod d7/d10/d17/d44/d36/d62/cf2 0 2024-03-04T13:43:02.301 INFO:tasks.workunit.client.0.smithi107.stdout:3/202: mknod d1/d2d/c40 0 2024-03-04T13:43:02.301 INFO:tasks.workunit.client.0.smithi107.stdout:4/208: write d0/d9/de/d14/dc/f27 [2993747,10165] 0 2024-03-04T13:43:02.302 INFO:tasks.workunit.client.1.smithi169.stdout:4/712: mkdir d4/d11/d3b/d67/dbb/d6e/ddc 0 2024-03-04T13:43:02.305 INFO:tasks.workunit.client.0.smithi107.stdout:6/253: mknod df/d1b/d2c/c4a 0 2024-03-04T13:43:02.306 INFO:tasks.workunit.client.1.smithi169.stdout:4/713: truncate d4/d11/d3b/d67/fa0 4771795 0 2024-03-04T13:43:02.307 INFO:tasks.workunit.client.0.smithi107.stdout:6/254: write df/d2e/d36/f49 [936979,98903] 0 2024-03-04T13:43:02.307 INFO:tasks.workunit.client.1.smithi169.stdout:8/737: mkdir d7/d10/d22/d75/db9/df3 0 2024-03-04T13:43:02.309 INFO:tasks.workunit.client.1.smithi169.stdout:2/788: dread d1/d5/d19/d63/fbd [0,4194304] 0 2024-03-04T13:43:02.309 INFO:tasks.workunit.client.1.smithi169.stdout:8/738: chown d7/ff 0 1 2024-03-04T13:43:02.311 INFO:tasks.workunit.client.0.smithi107.stdout:3/203: mkdir d1/d41 0 2024-03-04T13:43:02.311 INFO:tasks.workunit.client.1.smithi169.stdout:6/718: truncate d0/d32/d70/fcc 2384942 0 2024-03-04T13:43:02.314 INFO:tasks.workunit.client.1.smithi169.stdout:4/714: symlink d4/d11/d2e/d39/d61/d64/ldd 0 2024-03-04T13:43:02.318 INFO:tasks.workunit.client.0.smithi107.stdout:7/201: link d6/fe d6/f3f 0 2024-03-04T13:43:02.319 INFO:tasks.workunit.client.1.smithi169.stdout:8/739: dread - d7/d10/d17/d44/d36/fa8 zero size 2024-03-04T13:43:02.321 INFO:tasks.workunit.client.1.smithi169.stdout:6/719: fdatasync d0/f15 0 2024-03-04T13:43:02.321 INFO:tasks.workunit.client.0.smithi107.stdout:6/255: fsync df/d1b/f1d 0 2024-03-04T13:43:02.322 INFO:tasks.workunit.client.0.smithi107.stdout:3/204: mkdir d1/de/d18/d1f/d42 0 2024-03-04T13:43:02.325 INFO:tasks.workunit.client.0.smithi107.stdout:5/190: dwrite d5/f9 [0,4194304] 0 2024-03-04T13:43:02.332 INFO:tasks.workunit.client.1.smithi169.stdout:5/853: dread d1/f4a [0,4194304] 0 2024-03-04T13:43:02.334 INFO:tasks.workunit.client.1.smithi169.stdout:6/720: mknod d0/d3c/d40/d52/c107 0 2024-03-04T13:43:02.338 INFO:tasks.workunit.client.1.smithi169.stdout:6/721: stat d0/d32/d3a/d43/d74/fd8 0 2024-03-04T13:43:02.339 INFO:tasks.workunit.client.0.smithi107.stdout:3/205: dread d1/fa [0,4194304] 0 2024-03-04T13:43:02.340 INFO:tasks.workunit.client.1.smithi169.stdout:7/787: dwrite d2/de/d3e/d41/ffa [0,4194304] 0 2024-03-04T13:43:02.344 INFO:tasks.workunit.client.1.smithi169.stdout:1/684: dwrite d4/d5/d2d/d4e/d71/d82/fad [0,4194304] 0 2024-03-04T13:43:02.351 INFO:tasks.workunit.client.1.smithi169.stdout:7/788: creat d2/d28/d70/ded/f108 x:0 0 0 2024-03-04T13:43:02.352 INFO:tasks.workunit.client.0.smithi107.stdout:0/273: sync 2024-03-04T13:43:02.352 INFO:tasks.workunit.client.0.smithi107.stdout:9/236: sync 2024-03-04T13:43:02.352 INFO:tasks.workunit.client.0.smithi107.stdout:3/206: creat d1/de/d18/d1f/f43 x:0 0 0 2024-03-04T13:43:02.356 INFO:tasks.workunit.client.1.smithi169.stdout:0/609: write d5/d9/dc/d12/d15/d19/d61/f68 [525497,18778] 0 2024-03-04T13:43:02.358 INFO:tasks.workunit.client.0.smithi107.stdout:7/202: link d6/d13/d16/d1c/f1f d6/d13/d16/d19/d39/f40 0 2024-03-04T13:43:02.359 INFO:tasks.workunit.client.0.smithi107.stdout:6/256: creat df/d1b/f4b x:0 0 0 2024-03-04T13:43:02.359 INFO:tasks.workunit.client.1.smithi169.stdout:4/715: dread d4/f5a [0,4194304] 0 2024-03-04T13:43:02.362 INFO:tasks.workunit.client.1.smithi169.stdout:7/789: symlink d2/de/d3e/d41/l109 0 2024-03-04T13:43:02.365 INFO:tasks.workunit.client.1.smithi169.stdout:6/722: creat d0/d32/d3a/d43/f108 x:0 0 0 2024-03-04T13:43:02.366 INFO:tasks.workunit.client.0.smithi107.stdout:6/257: write df/f19 [226752,99703] 0 2024-03-04T13:43:02.366 INFO:tasks.workunit.client.0.smithi107.stdout:7/203: chown d6/d13/d16/d19/c3e 1088 1 2024-03-04T13:43:02.367 INFO:tasks.workunit.client.0.smithi107.stdout:9/237: unlink d4/l1d 0 2024-03-04T13:43:02.367 INFO:tasks.workunit.client.1.smithi169.stdout:0/610: rmdir d5 39 2024-03-04T13:43:02.368 INFO:tasks.workunit.client.1.smithi169.stdout:1/685: dread d4/d1d/f8e [0,4194304] 0 2024-03-04T13:43:02.369 INFO:tasks.workunit.client.1.smithi169.stdout:7/790: mkdir d2/d28/d10a 0 2024-03-04T13:43:02.371 INFO:tasks.workunit.client.0.smithi107.stdout:3/207: dread d1/fc [0,4194304] 0 2024-03-04T13:43:02.372 INFO:tasks.workunit.client.1.smithi169.stdout:6/723: creat d0/d3c/d40/f109 x:0 0 0 2024-03-04T13:43:02.373 INFO:tasks.workunit.client.0.smithi107.stdout:3/208: chown d1/l8 111199259 1 2024-03-04T13:43:02.373 INFO:tasks.workunit.client.0.smithi107.stdout:1/211: dwrite d1/d3/df/f3a [0,4194304] 0 2024-03-04T13:43:02.374 INFO:tasks.workunit.client.1.smithi169.stdout:1/686: truncate d4/d5/d36/d35/df/fd6 9107 0 2024-03-04T13:43:02.375 INFO:tasks.workunit.client.1.smithi169.stdout:2/789: write d1/d5/d19/d3c/f104 [224991,77068] 0 2024-03-04T13:43:02.377 INFO:tasks.workunit.client.0.smithi107.stdout:1/212: stat d1/d10/d20/d24 0 2024-03-04T13:43:02.377 INFO:tasks.workunit.client.0.smithi107.stdout:7/204: symlink d6/d13/d16/d19/d38/d2c/l41 0 2024-03-04T13:43:02.377 INFO:tasks.workunit.client.1.smithi169.stdout:0/611: chown d5/d4c/d65/f8e 413839 1 2024-03-04T13:43:02.378 INFO:tasks.workunit.client.0.smithi107.stdout:1/213: chown d1/d3/d17/c1d 831044 1 2024-03-04T13:43:02.381 INFO:tasks.workunit.client.0.smithi107.stdout:9/238: write d4/da/d11/f13 [4715271,89557] 0 2024-03-04T13:43:02.382 INFO:tasks.workunit.client.0.smithi107.stdout:4/209: truncate d0/d9/d10/f25 3238745 0 2024-03-04T13:43:02.382 INFO:tasks.workunit.client.0.smithi107.stdout:1/214: write d1/d10/d20/d24/f3c [1011685,105251] 0 2024-03-04T13:43:02.384 INFO:tasks.workunit.client.1.smithi169.stdout:8/740: write d7/d10/d24/f50 [1326705,78840] 0 2024-03-04T13:43:02.385 INFO:tasks.workunit.client.0.smithi107.stdout:1/215: write d1/d3/df/f3a [1433632,60947] 0 2024-03-04T13:43:02.386 INFO:tasks.workunit.client.1.smithi169.stdout:1/687: creat d4/d5/d36/d35/d16/d81/db4/fde x:0 0 0 2024-03-04T13:43:02.389 INFO:tasks.workunit.client.0.smithi107.stdout:6/258: rename df/d1b/l2f to df/d17/d24/d30/d3b/l4c 0 2024-03-04T13:43:02.391 INFO:tasks.workunit.client.1.smithi169.stdout:2/790: rename d1/d5/d22/d45/f107 to d1/d5/d19/d69/f111 0 2024-03-04T13:43:02.393 INFO:tasks.workunit.client.0.smithi107.stdout:3/209: creat d1/d2d/f44 x:0 0 0 2024-03-04T13:43:02.395 INFO:tasks.workunit.client.0.smithi107.stdout:3/210: write d1/d1e/d26/f3d [516952,116660] 0 2024-03-04T13:43:02.396 INFO:tasks.workunit.client.0.smithi107.stdout:7/205: creat d6/d13/d16/d19/d1e/f42 x:0 0 0 2024-03-04T13:43:02.398 INFO:tasks.workunit.client.1.smithi169.stdout:2/791: truncate d1/d5/d22/d3b/d99/f10f 603002 0 2024-03-04T13:43:02.399 INFO:tasks.workunit.client.1.smithi169.stdout:6/724: dwrite d0/d3c/d40/d52/dac/fdf [0,4194304] 0 2024-03-04T13:43:02.404 INFO:tasks.workunit.client.1.smithi169.stdout:2/792: dread - d1/d5/d22/d45/d4a/de7/dcb/f10e zero size 2024-03-04T13:43:02.406 INFO:tasks.workunit.client.0.smithi107.stdout:5/191: dwrite d5/f6 [0,4194304] 0 2024-03-04T13:43:02.407 INFO:tasks.workunit.client.0.smithi107.stdout:4/210: creat d0/d9/de/d14/dc/d21/f3f x:0 0 0 2024-03-04T13:43:02.408 INFO:tasks.workunit.client.0.smithi107.stdout:3/211: dread d1/fc [0,4194304] 0 2024-03-04T13:43:02.411 INFO:tasks.workunit.client.1.smithi169.stdout:0/612: rename d5/d9/dc/d12/d15/d19/d77/c8b to d5/d9/dc/d12/d15/d19/d61/d71/dbb/cd7 0 2024-03-04T13:43:02.414 INFO:tasks.workunit.client.1.smithi169.stdout:0/613: write d5/d9/dc/d12/d15/d19/d61/f2c [1682067,29311] 0 2024-03-04T13:43:02.417 INFO:tasks.workunit.client.1.smithi169.stdout:0/614: readlink d5/d9/dc/d12/l4b 0 2024-03-04T13:43:02.418 INFO:tasks.workunit.client.0.smithi107.stdout:1/216: creat d1/d10/d16/d1b/f3e x:0 0 0 2024-03-04T13:43:02.418 INFO:tasks.workunit.client.1.smithi169.stdout:2/793: fsync d1/d5/d19/f2e 0 2024-03-04T13:43:02.421 INFO:tasks.workunit.client.1.smithi169.stdout:7/791: truncate d2/de/d3e/d41/f4c 1729080 0 2024-03-04T13:43:02.426 INFO:tasks.workunit.client.0.smithi107.stdout:5/192: rmdir d5/d8/d14 39 2024-03-04T13:43:02.431 INFO:tasks.workunit.client.1.smithi169.stdout:6/725: creat d0/d32/d3a/d43/d74/d93/dee/f10a x:0 0 0 2024-03-04T13:43:02.432 INFO:tasks.workunit.client.0.smithi107.stdout:7/206: creat d6/d13/d16/d19/d1e/d3d/f43 x:0 0 0 2024-03-04T13:43:02.434 INFO:tasks.workunit.client.1.smithi169.stdout:8/741: dwrite d7/d10/d22/d46/f6e [0,4194304] 0 2024-03-04T13:43:02.436 INFO:tasks.workunit.client.0.smithi107.stdout:7/207: readlink d6/d17/l25 0 2024-03-04T13:43:02.438 INFO:tasks.workunit.client.1.smithi169.stdout:1/688: creat d4/d5/d2d/d4e/d71/d26/fdf x:0 0 0 2024-03-04T13:43:02.438 INFO:tasks.workunit.client.0.smithi107.stdout:9/239: dwrite d4/da/f1b [0,4194304] 0 2024-03-04T13:43:02.444 INFO:tasks.workunit.client.0.smithi107.stdout:3/212: dwrite d1/de/d18/d33/f34 [0,4194304] 0 2024-03-04T13:43:02.445 INFO:tasks.workunit.client.0.smithi107.stdout:2/250: sync 2024-03-04T13:43:02.445 INFO:tasks.workunit.client.0.smithi107.stdout:8/254: sync 2024-03-04T13:43:02.446 INFO:tasks.workunit.client.0.smithi107.stdout:6/259: dwrite df/d1b/f37 [4194304,4194304] 0 2024-03-04T13:43:02.449 INFO:tasks.workunit.client.1.smithi169.stdout:2/794: rename d1/d5/d22/d45/led to d1/d5/d19/d3c/db8/de1/d64/dce/l112 0 2024-03-04T13:43:02.452 INFO:tasks.workunit.client.1.smithi169.stdout:6/726: rmdir d0/d32/d3a/d5e 39 2024-03-04T13:43:02.452 INFO:tasks.workunit.client.0.smithi107.stdout:4/211: dwrite d0/d9/de/d14/dc/f26 [0,4194304] 0 2024-03-04T13:43:02.456 INFO:tasks.workunit.client.0.smithi107.stdout:6/260: fdatasync df/d17/d24/f2d 0 2024-03-04T13:43:02.456 INFO:tasks.workunit.client.0.smithi107.stdout:9/240: write d4/da/d11/d20/d31/f3c [98536,119730] 0 2024-03-04T13:43:02.457 INFO:tasks.workunit.client.0.smithi107.stdout:4/212: chown d0/d9 0 1 2024-03-04T13:43:02.458 INFO:tasks.workunit.client.0.smithi107.stdout:8/255: write d6/da/f1b [1957517,32695] 0 2024-03-04T13:43:02.459 INFO:tasks.workunit.client.0.smithi107.stdout:3/213: truncate d1/d2d/f39 915286 0 2024-03-04T13:43:02.460 INFO:tasks.workunit.client.0.smithi107.stdout:6/261: write df/d17/d24/d27/f26 [549623,22313] 0 2024-03-04T13:43:02.461 INFO:tasks.workunit.client.1.smithi169.stdout:1/689: unlink d4/d5/d36/d35/d5b/lbd 0 2024-03-04T13:43:02.462 INFO:tasks.workunit.client.0.smithi107.stdout:1/217: rename d1/d3/df/c15 to d1/c3f 0 2024-03-04T13:43:02.464 INFO:tasks.workunit.client.1.smithi169.stdout:0/615: rename d5/d9/dc/d12/d15/d19/d77/l89 to d5/d9/dc/d12/d15/d46/d98/dc1/dc8/ld8 0 2024-03-04T13:43:02.465 INFO:tasks.workunit.client.1.smithi169.stdout:8/742: symlink d7/d10/d22/d46/da5/dcb/lf4 0 2024-03-04T13:43:02.469 INFO:tasks.workunit.client.1.smithi169.stdout:0/616: write d5/d4c/d65/d95/fb9 [38768,25275] 0 2024-03-04T13:43:02.470 INFO:tasks.workunit.client.1.smithi169.stdout:7/792: dread d2/d15/d17/d6f/f72 [0,4194304] 0 2024-03-04T13:43:02.474 INFO:tasks.workunit.client.1.smithi169.stdout:0/617: write d5/d9/dc/d12/d15/d19/d61/f2c [5066746,96365] 0 2024-03-04T13:43:02.475 INFO:tasks.workunit.client.1.smithi169.stdout:6/727: rmdir d0/d32/d3a/d43 39 2024-03-04T13:43:02.480 INFO:tasks.workunit.client.1.smithi169.stdout:8/743: chown d7/d10/d24/d3e/c6f 30 1 2024-03-04T13:43:02.480 INFO:tasks.workunit.client.0.smithi107.stdout:0/274: sync 2024-03-04T13:43:02.480 INFO:tasks.workunit.client.0.smithi107.stdout:4/213: mknod d0/d2f/c40 0 2024-03-04T13:43:02.482 INFO:tasks.workunit.client.0.smithi107.stdout:0/275: chown da/dc/dd/d3e/d26 55980 1 2024-03-04T13:43:02.484 INFO:tasks.workunit.client.0.smithi107.stdout:2/251: dread da/f18 [0,4194304] 0 2024-03-04T13:43:02.484 INFO:tasks.workunit.client.0.smithi107.stdout:8/256: fsync d6/da/f1b 0 2024-03-04T13:43:02.484 INFO:tasks.workunit.client.0.smithi107.stdout:1/218: dwrite d1/d3/d17/f34 [0,4194304] 0 2024-03-04T13:43:02.487 INFO:tasks.workunit.client.1.smithi169.stdout:3/743: sync 2024-03-04T13:43:02.487 INFO:tasks.workunit.client.1.smithi169.stdout:9/791: sync 2024-03-04T13:43:02.488 INFO:tasks.workunit.client.1.smithi169.stdout:9/792: stat d7/f33 0 2024-03-04T13:43:02.488 INFO:tasks.workunit.client.0.smithi107.stdout:7/208: rename d6/c11 to d6/d13/d14/c44 0 2024-03-04T13:43:02.502 INFO:tasks.workunit.client.1.smithi169.stdout:1/690: creat d4/d5/d2d/d4e/d71/d26/daf/dd0/fe0 x:0 0 0 2024-03-04T13:43:02.504 INFO:tasks.workunit.client.0.smithi107.stdout:2/252: creat da/d44/f52 x:0 0 0 2024-03-04T13:43:02.505 INFO:tasks.workunit.client.1.smithi169.stdout:2/795: truncate d1/d5/d22/d45/d4a/f92 2057851 0 2024-03-04T13:43:02.506 INFO:tasks.workunit.client.0.smithi107.stdout:2/253: chown da/d39/c3c 122 1 2024-03-04T13:43:02.509 INFO:tasks.workunit.client.0.smithi107.stdout:1/219: mkdir d1/d10/d16/d1b/d40 0 2024-03-04T13:43:02.510 INFO:tasks.workunit.client.1.smithi169.stdout:3/744: unlink d7/de/d2b/d7d/d85/d13/d1a/d2d/d56/f7c 0 2024-03-04T13:43:02.512 INFO:tasks.workunit.client.1.smithi169.stdout:2/796: dread d1/d5/d19/d3c/db8/de1/d64/ff4 [0,4194304] 0 2024-03-04T13:43:02.512 INFO:tasks.workunit.client.0.smithi107.stdout:5/193: rename d5/d8/d14/f34 to d5/d8/d2a/f44 0 2024-03-04T13:43:02.516 INFO:tasks.workunit.client.0.smithi107.stdout:3/214: dwrite d1/d1e/f22 [4194304,4194304] 0 2024-03-04T13:43:02.519 INFO:tasks.workunit.client.1.smithi169.stdout:0/618: mknod d5/d9/dc/d12/d15/d19/d77/d94/cd9 0 2024-03-04T13:43:02.521 INFO:tasks.workunit.client.0.smithi107.stdout:3/215: read - d1/f2b zero size 2024-03-04T13:43:02.524 INFO:tasks.workunit.client.0.smithi107.stdout:6/262: dwrite df/d17/f23 [0,4194304] 0 2024-03-04T13:43:02.524 INFO:tasks.workunit.client.0.smithi107.stdout:9/241: creat d4/f45 x:0 0 0 2024-03-04T13:43:02.532 INFO:tasks.workunit.client.1.smithi169.stdout:3/745: mknod d7/de/d24/cf9 0 2024-03-04T13:43:02.533 INFO:tasks.workunit.client.1.smithi169.stdout:7/793: link d2/f95 d2/de/d37/f10b 0 2024-03-04T13:43:02.535 INFO:tasks.workunit.client.0.smithi107.stdout:3/216: rmdir d1/d1e/d26 39 2024-03-04T13:43:02.537 INFO:tasks.workunit.client.0.smithi107.stdout:4/214: creat d0/d9/de/f41 x:0 0 0 2024-03-04T13:43:02.537 INFO:tasks.workunit.client.0.smithi107.stdout:7/209: dwrite d6/d1d/f3b [0,4194304] 0 2024-03-04T13:43:02.537 INFO:tasks.workunit.client.0.smithi107.stdout:3/217: read d1/f3 [10176244,105087] 0 2024-03-04T13:43:02.539 INFO:tasks.workunit.client.0.smithi107.stdout:9/242: unlink d4/d34/f35 0 2024-03-04T13:43:02.540 INFO:tasks.workunit.client.0.smithi107.stdout:4/215: stat d0/d9/d18/c22 0 2024-03-04T13:43:02.541 INFO:tasks.workunit.client.0.smithi107.stdout:7/210: readlink d6/d13/d16/d19/l2e 0 2024-03-04T13:43:02.541 INFO:tasks.workunit.client.1.smithi169.stdout:9/793: rename d7/d25/cf0 to d7/c11d 0 2024-03-04T13:43:02.545 INFO:tasks.workunit.client.0.smithi107.stdout:1/220: rename d1/c35 to d1/d10/d20/d24/c41 0 2024-03-04T13:43:02.545 INFO:tasks.workunit.client.0.smithi107.stdout:7/211: readlink d6/d13/d16/d19/d1e/l2f 0 2024-03-04T13:43:02.547 INFO:tasks.workunit.client.0.smithi107.stdout:1/221: fsync d1/d10/d20/f2c 0 2024-03-04T13:43:02.547 INFO:tasks.workunit.client.0.smithi107.stdout:8/257: getdents d6/da/d25/d2a 0 2024-03-04T13:43:02.548 INFO:tasks.workunit.client.0.smithi107.stdout:6/263: dwrite fd [4194304,4194304] 0 2024-03-04T13:43:02.550 INFO:tasks.workunit.client.1.smithi169.stdout:7/794: symlink d2/d28/d68/l10c 0 2024-03-04T13:43:02.551 INFO:tasks.workunit.client.0.smithi107.stdout:9/243: mknod d4/d34/c46 0 2024-03-04T13:43:02.552 INFO:tasks.workunit.client.1.smithi169.stdout:4/716: sync 2024-03-04T13:43:02.552 INFO:tasks.workunit.client.1.smithi169.stdout:5/854: sync 2024-03-04T13:43:02.552 INFO:tasks.workunit.client.1.smithi169.stdout:7/795: chown d2/de/d3e/d41/la5 119 1 2024-03-04T13:43:02.555 INFO:tasks.workunit.client.0.smithi107.stdout:6/264: write df/d17/d24/f2b [1143026,39561] 0 2024-03-04T13:43:02.556 INFO:tasks.workunit.client.1.smithi169.stdout:2/797: truncate d1/d5/fe 701378 0 2024-03-04T13:43:02.556 INFO:tasks.workunit.client.1.smithi169.stdout:5/855: truncate d1/d16/d2f/d39/fb3 485980 0 2024-03-04T13:43:02.560 INFO:tasks.workunit.client.1.smithi169.stdout:1/691: link d4/d5/d36/d35/df/f46 d4/dd8/fe1 0 2024-03-04T13:43:02.564 INFO:tasks.workunit.client.0.smithi107.stdout:3/218: dwrite d1/de/d18/d33/f34 [0,4194304] 0 2024-03-04T13:43:02.566 INFO:tasks.workunit.client.0.smithi107.stdout:8/258: rmdir d6/da/d25/d2d 39 2024-03-04T13:43:02.567 INFO:tasks.workunit.client.1.smithi169.stdout:6/728: dwrite d0/d32/d3a/f21 [0,4194304] 0 2024-03-04T13:43:02.570 INFO:tasks.workunit.client.0.smithi107.stdout:1/222: symlink d1/d10/d20/d24/l42 0 2024-03-04T13:43:02.573 INFO:tasks.workunit.client.0.smithi107.stdout:4/216: mknod d0/d9/de/d14/d33/c42 0 2024-03-04T13:43:02.575 INFO:tasks.workunit.client.1.smithi169.stdout:8/744: dwrite d7/d10/d17/f31 [0,4194304] 0 2024-03-04T13:43:02.575 INFO:tasks.workunit.client.1.smithi169.stdout:4/717: dwrite d4/d11/d2e/d37/d5c/f8c [0,4194304] 0 2024-03-04T13:43:02.577 INFO:tasks.workunit.client.0.smithi107.stdout:9/244: mknod d4/da/d11/d20/c47 0 2024-03-04T13:43:02.581 INFO:tasks.workunit.client.1.smithi169.stdout:4/718: readlink d4/d11/d2e/d6c/l9a 0 2024-03-04T13:43:02.587 INFO:tasks.workunit.client.1.smithi169.stdout:0/619: dwrite d5/d9/dc/d12/d15/d19/d77/d94/fab [0,4194304] 0 2024-03-04T13:43:02.587 INFO:tasks.workunit.client.1.smithi169.stdout:4/719: readlink d4/d11/d2e/d37/d5c/d86/ld3 0 2024-03-04T13:43:02.587 INFO:tasks.workunit.client.1.smithi169.stdout:5/856: mknod d1/d16/dbd/dbb/c12d 0 2024-03-04T13:43:02.588 INFO:tasks.workunit.client.0.smithi107.stdout:8/259: dread f5 [0,4194304] 0 2024-03-04T13:43:02.589 INFO:tasks.workunit.client.1.smithi169.stdout:2/798: creat d1/d5/d22/d3b/f113 x:0 0 0 2024-03-04T13:43:02.590 INFO:tasks.workunit.client.1.smithi169.stdout:2/799: chown d1/d5/d22/d3b/c65 16 1 2024-03-04T13:43:02.593 INFO:tasks.workunit.client.0.smithi107.stdout:6/265: mknod df/d17/d24/d40/c4d 0 2024-03-04T13:43:02.594 INFO:tasks.workunit.client.1.smithi169.stdout:1/692: readlink d4/d5/d2d/d4e/d71/d26/l5f 0 2024-03-04T13:43:02.594 INFO:tasks.workunit.client.0.smithi107.stdout:6/266: chown c9 187 1 2024-03-04T13:43:02.596 INFO:tasks.workunit.client.1.smithi169.stdout:9/794: mkdir d7/d11e 0 2024-03-04T13:43:02.597 INFO:tasks.workunit.client.0.smithi107.stdout:3/219: creat d1/de/d11/f45 x:0 0 0 2024-03-04T13:43:02.599 INFO:tasks.workunit.client.1.smithi169.stdout:9/795: readlink d7/dc/d12/d20/da6/ld0 0 2024-03-04T13:43:02.599 INFO:tasks.workunit.client.1.smithi169.stdout:3/746: creat d7/de/d2b/d7d/ffa x:0 0 0 2024-03-04T13:43:02.600 INFO:tasks.workunit.client.0.smithi107.stdout:1/223: dwrite d1/d10/d16/d1b/f3e [0,4194304] 0 2024-03-04T13:43:02.602 INFO:tasks.workunit.client.0.smithi107.stdout:4/217: dwrite d0/d9/d1e/f2b [4194304,4194304] 0 2024-03-04T13:43:02.603 INFO:tasks.workunit.client.0.smithi107.stdout:8/260: rmdir d6/d16 39 2024-03-04T13:43:02.604 INFO:tasks.workunit.client.0.smithi107.stdout:6/267: unlink df/d17/d24/l2a 0 2024-03-04T13:43:02.608 INFO:tasks.workunit.client.1.smithi169.stdout:4/720: mkdir d4/d11/d2e/dcd/dde 0 2024-03-04T13:43:02.608 INFO:tasks.workunit.client.1.smithi169.stdout:9/796: read d7/d2d/d48/d66/f75 [540521,89198] 0 2024-03-04T13:43:02.608 INFO:tasks.workunit.client.0.smithi107.stdout:6/268: fsync df/f43 0 2024-03-04T13:43:02.608 INFO:tasks.workunit.client.0.smithi107.stdout:1/224: read d1/d3/df/f3a [223248,67173] 0 2024-03-04T13:43:02.608 INFO:tasks.workunit.client.0.smithi107.stdout:9/245: symlink d4/d32/l48 0 2024-03-04T13:43:02.610 INFO:tasks.workunit.client.0.smithi107.stdout:3/220: creat d1/de/d11/f46 x:0 0 0 2024-03-04T13:43:02.611 INFO:tasks.workunit.client.0.smithi107.stdout:6/269: write fe [1034544,119694] 0 2024-03-04T13:43:02.611 INFO:tasks.workunit.client.1.smithi169.stdout:0/620: mknod d5/d9/dc/d12/d66/cda 0 2024-03-04T13:43:02.618 INFO:tasks.workunit.client.0.smithi107.stdout:4/218: rmdir d0/d9/de/d14/dc 39 2024-03-04T13:43:02.620 INFO:tasks.workunit.client.1.smithi169.stdout:6/729: mkdir d0/d3c/d10b 0 2024-03-04T13:43:02.622 INFO:tasks.workunit.client.1.smithi169.stdout:6/730: readlink d0/d32/d70/lc2 0 2024-03-04T13:43:02.625 INFO:tasks.workunit.client.0.smithi107.stdout:9/246: symlink d4/d34/l49 0 2024-03-04T13:43:02.627 INFO:tasks.workunit.client.1.smithi169.stdout:5/857: creat d1/d81/d91/d129/f12e x:0 0 0 2024-03-04T13:43:02.627 INFO:tasks.workunit.client.1.smithi169.stdout:6/731: dread - d0/d3c/d95/d9e/ffb zero size 2024-03-04T13:43:02.628 INFO:tasks.workunit.client.0.smithi107.stdout:8/261: dread - d6/da/d25/f3f zero size 2024-03-04T13:43:02.631 INFO:tasks.workunit.client.1.smithi169.stdout:6/732: truncate d0/f35 4961801 0 2024-03-04T13:43:02.632 INFO:tasks.workunit.client.0.smithi107.stdout:7/212: dwrite d6/d13/d16/d1c/f1f [0,4194304] 0 2024-03-04T13:43:02.634 INFO:tasks.workunit.client.1.smithi169.stdout:7/796: link d2/d15/d17/c9a d2/de/d3e/d41/d69/deb/c10d 0 2024-03-04T13:43:02.634 INFO:tasks.workunit.client.1.smithi169.stdout:9/797: write d7/dc/d3e/da3/fc9 [8383065,35382] 0 2024-03-04T13:43:02.639 INFO:tasks.workunit.client.1.smithi169.stdout:1/693: unlink d4/d5/d36/d35/df/f46 0 2024-03-04T13:43:02.642 INFO:tasks.workunit.client.0.smithi107.stdout:9/247: rename d4/d34/f43 to d4/da/f4a 0 2024-03-04T13:43:02.643 INFO:tasks.workunit.client.1.smithi169.stdout:9/798: write d7/dc/f118 [172681,123923] 0 2024-03-04T13:43:02.643 INFO:tasks.workunit.client.0.smithi107.stdout:8/262: chown d6/d16/l47 239562 1 2024-03-04T13:43:02.644 INFO:tasks.workunit.client.0.smithi107.stdout:1/225: mkdir d1/d3/df/d33/d43 0 2024-03-04T13:43:02.646 INFO:tasks.workunit.client.0.smithi107.stdout:6/270: getdents df/d38/d3e 0 2024-03-04T13:43:02.647 INFO:tasks.workunit.client.1.smithi169.stdout:5/858: creat d1/d16/d2f/d72/dab/f12f x:0 0 0 2024-03-04T13:43:02.648 INFO:tasks.workunit.client.1.smithi169.stdout:9/799: dread d7/dc/df1/fe0 [0,4194304] 0 2024-03-04T13:43:02.651 INFO:tasks.workunit.client.1.smithi169.stdout:6/733: dread - d0/d32/d3a/d43/d74/fa0 zero size 2024-03-04T13:43:02.652 INFO:tasks.workunit.client.0.smithi107.stdout:3/221: link d1/de/d11/f46 d1/de/d18/f47 0 2024-03-04T13:43:02.654 INFO:tasks.workunit.client.0.smithi107.stdout:8/263: chown d6/c18 6834553 1 2024-03-04T13:43:02.656 INFO:tasks.workunit.client.0.smithi107.stdout:1/226: creat d1/d10/d20/d24/f44 x:0 0 0 2024-03-04T13:43:02.657 INFO:tasks.workunit.client.1.smithi169.stdout:2/800: link d1/d5/d19/d63/f7b d1/d5/d19/d3c/db8/f114 0 2024-03-04T13:43:02.658 INFO:tasks.workunit.client.0.smithi107.stdout:6/271: creat df/d38/f4e x:0 0 0 2024-03-04T13:43:02.662 INFO:tasks.workunit.client.1.smithi169.stdout:3/747: dread d7/de/d2b/d7d/d85/f4f [0,4194304] 0 2024-03-04T13:43:02.662 INFO:tasks.workunit.client.1.smithi169.stdout:1/694: symlink d4/d5/d36/d35/d8f/le2 0 2024-03-04T13:43:02.667 INFO:tasks.workunit.client.0.smithi107.stdout:8/264: dwrite d6/d16/d1f/f34 [4194304,4194304] 0 2024-03-04T13:43:02.669 INFO:tasks.workunit.client.0.smithi107.stdout:8/265: write d6/da/f35 [325662,118987] 0 2024-03-04T13:43:02.670 INFO:tasks.workunit.client.1.smithi169.stdout:7/797: dwrite d2/d28/d70/ded/f108 [0,4194304] 0 2024-03-04T13:43:02.670 INFO:tasks.workunit.client.1.smithi169.stdout:8/745: dread d7/d10/d22/f30 [0,4194304] 0 2024-03-04T13:43:02.673 INFO:tasks.workunit.client.1.smithi169.stdout:1/695: write d4/d5/d36/d35/d70/fab [1324442,14571] 0 2024-03-04T13:43:02.673 INFO:tasks.workunit.client.1.smithi169.stdout:5/859: rename d1/d16/d17/d2d/d5a/d67/fff to d1/d16/d2a/d46/f130 0 2024-03-04T13:43:02.679 INFO:tasks.workunit.client.1.smithi169.stdout:1/696: chown d4/d5/d36/d75/dba 3258222 1 2024-03-04T13:43:02.680 INFO:tasks.workunit.client.1.smithi169.stdout:7/798: write d2/de/d3e/d41/f49 [6046825,42307] 0 2024-03-04T13:43:02.685 INFO:tasks.workunit.client.1.smithi169.stdout:1/697: fdatasync d4/d5/d36/f59 0 2024-03-04T13:43:02.687 INFO:tasks.workunit.client.0.smithi107.stdout:7/213: link d6/d13/d16/d1c/f35 d6/f45 0 2024-03-04T13:43:02.693 INFO:tasks.workunit.client.0.smithi107.stdout:6/272: mkdir df/d38/d4f 0 2024-03-04T13:43:02.693 INFO:tasks.workunit.client.0.smithi107.stdout:4/219: dread d0/d9/f11 [0,4194304] 0 2024-03-04T13:43:02.693 INFO:tasks.workunit.client.1.smithi169.stdout:2/801: write d1/d6d/f86 [4463830,72267] 0 2024-03-04T13:43:02.701 INFO:tasks.workunit.client.1.smithi169.stdout:3/748: fdatasync d7/de/d2b/d7d/d85/d47/f9d 0 2024-03-04T13:43:02.706 INFO:tasks.workunit.client.1.smithi169.stdout:4/721: dread d4/d11/d2e/d37/f45 [0,4194304] 0 2024-03-04T13:43:02.706 INFO:tasks.workunit.client.1.smithi169.stdout:0/621: dread d5/d4c/f51 [0,4194304] 0 2024-03-04T13:43:02.707 INFO:tasks.workunit.client.1.smithi169.stdout:9/800: dread d7/dc/f80 [0,4194304] 0 2024-03-04T13:43:02.709 INFO:tasks.workunit.client.1.smithi169.stdout:4/722: dread - d4/d1c/d58/d9d/fda zero size 2024-03-04T13:43:02.714 INFO:tasks.workunit.client.1.smithi169.stdout:7/799: creat d2/d15/d17/f10e x:0 0 0 2024-03-04T13:43:02.715 INFO:tasks.workunit.client.0.smithi107.stdout:6/273: symlink df/d1b/d2c/d46/l50 0 2024-03-04T13:43:02.720 INFO:tasks.workunit.client.1.smithi169.stdout:1/698: mknod d4/d5/d2d/d4e/d71/d26/d4f/ce3 0 2024-03-04T13:43:02.721 INFO:tasks.workunit.client.0.smithi107.stdout:1/227: link d1/d3/cd d1/d10/d20/d24/c45 0 2024-03-04T13:43:02.722 INFO:tasks.workunit.client.0.smithi107.stdout:1/228: readlink d1/d10/d20/d24/l36 0 2024-03-04T13:43:02.723 INFO:tasks.workunit.client.1.smithi169.stdout:2/802: rename d1/d5/d22/d3b/d41/dbb to d1/d5/d19/d68/dd9/d115 0 2024-03-04T13:43:02.726 INFO:tasks.workunit.client.0.smithi107.stdout:4/220: dwrite d0/d9/de/d14/dc/f27 [0,4194304] 0 2024-03-04T13:43:02.726 INFO:tasks.workunit.client.0.smithi107.stdout:7/214: rename d6/ld to d6/d13/d16/l46 0 2024-03-04T13:43:02.727 INFO:tasks.workunit.client.1.smithi169.stdout:6/734: dwrite d0/d3c/d95/dbd/fc5 [4194304,4194304] 0 2024-03-04T13:43:02.728 INFO:tasks.workunit.client.0.smithi107.stdout:7/215: read d6/d1d/f33 [2440290,44555] 0 2024-03-04T13:43:02.729 INFO:tasks.workunit.client.0.smithi107.stdout:4/221: write d0/d9/d10/f1c [112882,87147] 0 2024-03-04T13:43:02.731 INFO:tasks.workunit.client.0.smithi107.stdout:6/274: mkdir df/d17/d24/d40/d51 0 2024-03-04T13:43:02.731 INFO:tasks.workunit.client.0.smithi107.stdout:3/222: truncate d1/d1e/f22 8349883 0 2024-03-04T13:43:02.732 INFO:tasks.workunit.client.1.smithi169.stdout:8/746: truncate d7/d10/d17/f2a 268328 0 2024-03-04T13:43:02.732 INFO:tasks.workunit.client.1.smithi169.stdout:8/747: chown d7/d12 14447439 1 2024-03-04T13:43:02.736 INFO:tasks.workunit.client.0.smithi107.stdout:6/275: write df/d17/f1e [5081760,100477] 0 2024-03-04T13:43:02.738 INFO:tasks.workunit.client.1.smithi169.stdout:8/748: chown d7/d10/d22/d46/da5/da6/dc6 317 1 2024-03-04T13:43:02.738 INFO:tasks.workunit.client.1.smithi169.stdout:8/749: stat d7/d10/d22/d75/d79/fb1 0 2024-03-04T13:43:02.738 INFO:tasks.workunit.client.1.smithi169.stdout:8/750: stat d7/l58 0 2024-03-04T13:43:02.741 INFO:tasks.workunit.client.0.smithi107.stdout:6/276: read f0 [246645,71600] 0 2024-03-04T13:43:02.742 INFO:tasks.workunit.client.1.smithi169.stdout:9/801: unlink d7/d2d/cda 0 2024-03-04T13:43:02.744 INFO:tasks.workunit.client.0.smithi107.stdout:9/248: link d4/f26 d4/da/f4b 0 2024-03-04T13:43:02.746 INFO:tasks.workunit.client.0.smithi107.stdout:6/277: read df/d2e/d36/f49 [226442,88989] 0 2024-03-04T13:43:02.747 INFO:tasks.workunit.client.0.smithi107.stdout:6/278: chown df/d17/d24/d27/f26 213 1 2024-03-04T13:43:02.748 INFO:tasks.workunit.client.0.smithi107.stdout:6/279: write df/d17/d24/d27/f26 [1617815,52176] 0 2024-03-04T13:43:02.750 INFO:tasks.workunit.client.0.smithi107.stdout:3/223: mkdir d1/de/d18/d33/d48 0 2024-03-04T13:43:02.751 INFO:tasks.workunit.client.0.smithi107.stdout:3/224: fsync d1/de/d11/f45 0 2024-03-04T13:43:02.754 INFO:tasks.workunit.client.0.smithi107.stdout:1/229: mkdir d1/d46 0 2024-03-04T13:43:02.761 INFO:tasks.workunit.client.1.smithi169.stdout:2/803: rename d1/d5/d22/d45/d4a/de7/dcb/f10e to d1/d5/d19/d3c/db8/de1/d64/dce/f116 0 2024-03-04T13:43:02.762 INFO:tasks.workunit.client.0.smithi107.stdout:0/276: sync 2024-03-04T13:43:02.766 INFO:tasks.workunit.client.1.smithi169.stdout:8/751: rmdir d7/d10/d22/d46/da5/dcb 39 2024-03-04T13:43:02.770 INFO:tasks.workunit.client.1.smithi169.stdout:3/749: dwrite d7/de/d2b/d7d/d85/d18/d21/d3f/d68/fb2 [0,4194304] 0 2024-03-04T13:43:02.774 INFO:tasks.workunit.client.1.smithi169.stdout:9/802: symlink d7/dc/df1/d56/l11f 0 2024-03-04T13:43:02.776 INFO:tasks.workunit.client.1.smithi169.stdout:3/750: dread - d7/de/d2b/d7d/d85/d18/d21/fe4 zero size 2024-03-04T13:43:02.776 INFO:tasks.workunit.client.0.smithi107.stdout:6/280: creat df/d17/f52 x:0 0 0 2024-03-04T13:43:02.778 INFO:tasks.workunit.client.0.smithi107.stdout:9/249: dwrite d4/da/f1b [0,4194304] 0 2024-03-04T13:43:02.778 INFO:tasks.workunit.client.1.smithi169.stdout:0/622: dwrite d5/d9/dc/d12/d15/d19/d77/f9b [0,4194304] 0 2024-03-04T13:43:02.778 INFO:tasks.workunit.client.1.smithi169.stdout:5/860: creat d1/d16/d17/d28/f131 x:0 0 0 2024-03-04T13:43:02.780 INFO:tasks.workunit.client.0.smithi107.stdout:6/281: write df/d1b/d2c/d31/f39 [298454,110364] 0 2024-03-04T13:43:02.780 INFO:tasks.workunit.client.0.smithi107.stdout:3/225: creat d1/de/d11/d12/f49 x:0 0 0 2024-03-04T13:43:02.782 INFO:tasks.workunit.client.0.smithi107.stdout:1/230: mkdir d1/d10/d16/d47 0 2024-03-04T13:43:02.783 INFO:tasks.workunit.client.0.smithi107.stdout:1/231: stat d1/d10/d20/d24 0 2024-03-04T13:43:02.784 INFO:tasks.workunit.client.0.smithi107.stdout:4/222: creat d0/d9/de/f43 x:0 0 0 2024-03-04T13:43:02.785 INFO:tasks.workunit.client.1.smithi169.stdout:6/735: unlink d0/d32/d3a/d5e/fea 0 2024-03-04T13:43:02.786 INFO:tasks.workunit.client.0.smithi107.stdout:4/223: chown d0/d9/d18 149823197 1 2024-03-04T13:43:02.789 INFO:tasks.workunit.client.1.smithi169.stdout:2/804: mkdir d1/d5/d19/d68/d117 0 2024-03-04T13:43:02.790 INFO:tasks.workunit.client.0.smithi107.stdout:7/216: link d6/f3f d6/d13/d16/d19/d38/f47 0 2024-03-04T13:43:02.790 INFO:tasks.workunit.client.0.smithi107.stdout:0/277: read da/dc/f28 [2190523,86450] 0 2024-03-04T13:43:02.791 INFO:tasks.workunit.client.1.smithi169.stdout:8/752: stat d7/d10/d17/d44/d4c/le8 0 2024-03-04T13:43:02.799 INFO:tasks.workunit.client.1.smithi169.stdout:0/623: dread d5/d4c/f51 [0,4194304] 0 2024-03-04T13:43:02.801 INFO:tasks.workunit.client.1.smithi169.stdout:3/751: rmdir d7/de/d2b/d7d/d85/d13/d5a 39 2024-03-04T13:43:02.802 INFO:tasks.workunit.client.1.smithi169.stdout:9/803: rmdir d7 39 2024-03-04T13:43:02.802 INFO:tasks.workunit.client.1.smithi169.stdout:8/753: write d7/d10/d22/d75/d81/da2/fbf [527257,96294] 0 2024-03-04T13:43:02.805 INFO:tasks.workunit.client.0.smithi107.stdout:7/217: creat d6/d13/d16/d1c/f48 x:0 0 0 2024-03-04T13:43:02.806 INFO:tasks.workunit.client.1.smithi169.stdout:7/800: truncate d2/d6b/f76 1440257 0 2024-03-04T13:43:02.806 INFO:tasks.workunit.client.0.smithi107.stdout:7/218: stat d6/d13/d14 0 2024-03-04T13:43:02.812 INFO:tasks.workunit.client.0.smithi107.stdout:0/278: creat da/dc/d34/f5a x:0 0 0 2024-03-04T13:43:02.812 INFO:tasks.workunit.client.1.smithi169.stdout:1/699: write d4/f25 [5376109,45664] 0 2024-03-04T13:43:02.812 INFO:tasks.workunit.client.1.smithi169.stdout:1/700: stat d4/d5/l14 0 2024-03-04T13:43:02.813 INFO:tasks.workunit.client.0.smithi107.stdout:3/226: symlink d1/de/d3f/l4a 0 2024-03-04T13:43:02.818 INFO:tasks.workunit.client.0.smithi107.stdout:7/219: rmdir d6/d13/d16/d19/d38/d2c 39 2024-03-04T13:43:02.820 INFO:tasks.workunit.client.0.smithi107.stdout:5/194: sync 2024-03-04T13:43:02.820 INFO:tasks.workunit.client.0.smithi107.stdout:2/254: sync 2024-03-04T13:43:02.820 INFO:tasks.workunit.client.0.smithi107.stdout:8/266: sync 2024-03-04T13:43:02.820 INFO:tasks.workunit.client.0.smithi107.stdout:4/224: dread d0/d9/de/d14/dc/f2d [0,4194304] 0 2024-03-04T13:43:02.821 INFO:tasks.workunit.client.1.smithi169.stdout:1/701: rmdir d4/d5/d36/da9 39 2024-03-04T13:43:02.822 INFO:tasks.workunit.client.0.smithi107.stdout:5/195: dread - d5/d8/d2a/f39 zero size 2024-03-04T13:43:02.824 INFO:tasks.workunit.client.0.smithi107.stdout:5/196: stat d5/d8/f1f 0 2024-03-04T13:43:02.824 INFO:tasks.workunit.client.0.smithi107.stdout:6/282: truncate df/f29 1646118 0 2024-03-04T13:43:02.825 INFO:tasks.workunit.client.0.smithi107.stdout:1/232: dwrite d1/d3/df/f3a [0,4194304] 0 2024-03-04T13:43:02.825 INFO:tasks.workunit.client.1.smithi169.stdout:0/624: dwrite d5/d9/fb8 [0,4194304] 0 2024-03-04T13:43:02.827 INFO:tasks.workunit.client.0.smithi107.stdout:6/283: fsync df/d17/d24/f2b 0 2024-03-04T13:43:02.827 INFO:tasks.workunit.client.1.smithi169.stdout:1/702: dread - d4/d5/d2d/d4e/da5/fc5 zero size 2024-03-04T13:43:02.829 INFO:tasks.workunit.client.1.smithi169.stdout:5/861: creat d1/d16/f132 x:0 0 0 2024-03-04T13:43:02.830 INFO:tasks.workunit.client.1.smithi169.stdout:3/752: write d7/de/d2b/d7d/d85/d13/d1a/d2d/fce [230808,97543] 0 2024-03-04T13:43:02.833 INFO:tasks.workunit.client.0.smithi107.stdout:2/255: unlink f2 0 2024-03-04T13:43:02.835 INFO:tasks.workunit.client.1.smithi169.stdout:6/736: dwrite d0/d3c/d40/d52/d98/fb6 [0,4194304] 0 2024-03-04T13:43:02.836 INFO:tasks.workunit.client.0.smithi107.stdout:2/256: read da/df/f13 [485838,126414] 0 2024-03-04T13:43:02.836 INFO:tasks.workunit.client.0.smithi107.stdout:8/267: stat d6/c2e 0 2024-03-04T13:43:02.837 INFO:tasks.workunit.client.0.smithi107.stdout:1/233: dread d1/f31 [0,4194304] 0 2024-03-04T13:43:02.840 INFO:tasks.workunit.client.0.smithi107.stdout:5/197: dread d5/f6 [0,4194304] 0 2024-03-04T13:43:02.842 INFO:tasks.workunit.client.0.smithi107.stdout:5/198: write d5/f9 [793410,6013] 0 2024-03-04T13:43:02.842 INFO:tasks.workunit.client.1.smithi169.stdout:7/801: symlink d2/de/d3e/d7e/dbb/de3/l10f 0 2024-03-04T13:43:02.842 INFO:tasks.workunit.client.1.smithi169.stdout:9/804: mkdir d7/d11e/d120 0 2024-03-04T13:43:02.846 INFO:tasks.workunit.client.1.smithi169.stdout:2/805: dread d1/d5/d19/d3c/f6c [4194304,4194304] 0 2024-03-04T13:43:02.848 INFO:tasks.workunit.client.1.smithi169.stdout:1/703: mkdir d4/d5/d36/d35/d16/d81/de4 0 2024-03-04T13:43:02.851 INFO:tasks.workunit.client.1.smithi169.stdout:1/704: readlink d4/d5/d36/d35/df/ld3 0 2024-03-04T13:43:02.852 INFO:tasks.workunit.client.1.smithi169.stdout:0/625: rename d5/d9/d3d/c8f to d5/d9/d3d/d5d/cdb 0 2024-03-04T13:43:02.856 INFO:tasks.workunit.client.1.smithi169.stdout:8/754: creat d7/d10/d22/d46/ff5 x:0 0 0 2024-03-04T13:43:02.856 INFO:tasks.workunit.client.0.smithi107.stdout:6/284: mknod df/d2e/d36/c53 0 2024-03-04T13:43:02.858 INFO:tasks.workunit.client.0.smithi107.stdout:7/220: creat d6/d13/d16/d19/d38/d2c/f49 x:0 0 0 2024-03-04T13:43:02.860 INFO:tasks.workunit.client.0.smithi107.stdout:7/221: dread - d6/d1d/f29 zero size 2024-03-04T13:43:02.862 INFO:tasks.workunit.client.0.smithi107.stdout:3/227: write d1/d1e/f22 [6489582,42005] 0 2024-03-04T13:43:02.863 INFO:tasks.workunit.client.1.smithi169.stdout:9/805: symlink d7/d25/d3c/l121 0 2024-03-04T13:43:02.864 INFO:tasks.workunit.client.0.smithi107.stdout:9/250: getdents d4/da/d11 0 2024-03-04T13:43:02.865 INFO:tasks.workunit.client.0.smithi107.stdout:3/228: write d1/de/d11/d12/f49 [515150,91216] 0 2024-03-04T13:43:02.866 INFO:tasks.workunit.client.0.smithi107.stdout:6/285: dread df/d17/d24/f2b [0,4194304] 0 2024-03-04T13:43:02.868 INFO:tasks.workunit.client.1.smithi169.stdout:6/737: rename d0/d63 to d0/d32/d3a/d43/d74/d93/d10c 0 2024-03-04T13:43:02.873 INFO:tasks.workunit.client.0.smithi107.stdout:0/279: rename da/dc/f33 to da/dc/dd/d3e/f5b 0 2024-03-04T13:43:02.875 INFO:tasks.workunit.client.0.smithi107.stdout:0/280: chown da/dc/dd/d3e/d26/l43 76653 1 2024-03-04T13:43:02.875 INFO:tasks.workunit.client.0.smithi107.stdout:1/234: symlink d1/d10/d20/d24/l48 0 2024-03-04T13:43:02.881 INFO:tasks.workunit.client.1.smithi169.stdout:7/802: dwrite d2/d15/d17/d6f/f72 [8388608,4194304] 0 2024-03-04T13:43:02.881 INFO:tasks.workunit.client.1.smithi169.stdout:0/626: dread d5/d9/dc/d12/d15/d19/d61/d71/d8c/d90/f97 [0,4194304] 0 2024-03-04T13:43:02.882 INFO:tasks.workunit.client.1.smithi169.stdout:6/738: symlink d0/d3c/d95/d9e/l10d 0 2024-03-04T13:43:02.884 INFO:tasks.workunit.client.1.smithi169.stdout:7/803: readlink d2/d15/l52 0 2024-03-04T13:43:02.885 INFO:tasks.workunit.client.1.smithi169.stdout:5/862: creat d1/d16/dbd/f133 x:0 0 0 2024-03-04T13:43:02.886 INFO:tasks.workunit.client.0.smithi107.stdout:7/222: unlink d6/d1d/f33 0 2024-03-04T13:43:02.889 INFO:tasks.workunit.client.0.smithi107.stdout:3/229: mkdir d1/de/d4b 0 2024-03-04T13:43:02.891 INFO:tasks.workunit.client.1.smithi169.stdout:0/627: stat d5/d9/dc/d12/da5 0 2024-03-04T13:43:02.893 INFO:tasks.workunit.client.0.smithi107.stdout:4/225: rename d0/l17 to d0/l44 0 2024-03-04T13:43:02.893 INFO:tasks.workunit.client.1.smithi169.stdout:2/806: link d1/d5/d22/d45/d4a/lbf d1/d5/d19/d1e/d29/l118 0 2024-03-04T13:43:02.894 INFO:tasks.workunit.client.1.smithi169.stdout:9/806: fdatasync d7/d2d/d48/fef 0 2024-03-04T13:43:02.895 INFO:tasks.workunit.client.1.smithi169.stdout:6/739: rmdir d0/d3c/d40 39 2024-03-04T13:43:02.896 INFO:tasks.workunit.client.1.smithi169.stdout:3/753: dread d7/de/d2b/d7d/d85/d13/d5a/da1/fa7 [0,4194304] 0 2024-03-04T13:43:02.899 INFO:tasks.workunit.client.1.smithi169.stdout:6/740: write d0/f29 [8844255,14561] 0 2024-03-04T13:43:02.902 INFO:tasks.workunit.client.1.smithi169.stdout:5/863: creat d1/d81/dc1/f134 x:0 0 0 2024-03-04T13:43:02.904 INFO:tasks.workunit.client.0.smithi107.stdout:9/251: rename d4/da/d11/d20/d31 to d4/da/d11/d4c 0 2024-03-04T13:43:02.904 INFO:tasks.workunit.client.1.smithi169.stdout:3/754: write d7/de/d2b/d7d/d85/d13/d5a/d65/da4/fe7 [567609,114277] 0 2024-03-04T13:43:02.907 INFO:tasks.workunit.client.0.smithi107.stdout:1/235: write d1/d10/d20/d24/f2d [2078890,120874] 0 2024-03-04T13:43:02.908 INFO:tasks.workunit.client.0.smithi107.stdout:1/236: fsync d1/d10/d20/d24/f44 0 2024-03-04T13:43:02.909 INFO:tasks.workunit.client.1.smithi169.stdout:3/755: readlink d7/de/d2b/d7d/d85/d18/d21/d66/lcf 0 2024-03-04T13:43:02.910 INFO:tasks.workunit.client.1.smithi169.stdout:0/628: unlink d5/d9/d3d/d59/f3a 0 2024-03-04T13:43:02.912 INFO:tasks.workunit.client.0.smithi107.stdout:7/223: creat d6/d13/d16/d19/d38/d2c/f4a x:0 0 0 2024-03-04T13:43:02.915 INFO:tasks.workunit.client.1.smithi169.stdout:0/629: fsync d5/f8 0 2024-03-04T13:43:02.916 INFO:tasks.workunit.client.1.smithi169.stdout:3/756: truncate d7/d53/fb3 603298 0 2024-03-04T13:43:02.917 INFO:tasks.workunit.client.0.smithi107.stdout:6/286: mknod df/d1b/c54 0 2024-03-04T13:43:02.919 INFO:tasks.workunit.client.1.smithi169.stdout:1/705: dread d4/f20 [0,4194304] 0 2024-03-04T13:43:02.919 INFO:tasks.workunit.client.1.smithi169.stdout:7/804: dwrite d2/de/d3e/d41/dec/ff4 [0,4194304] 0 2024-03-04T13:43:02.921 INFO:tasks.workunit.client.1.smithi169.stdout:4/723: sync 2024-03-04T13:43:02.922 INFO:tasks.workunit.client.1.smithi169.stdout:2/807: write d1/d5/d19/d3c/db8/de1/d64/dce/f116 [224669,59037] 0 2024-03-04T13:43:02.923 INFO:tasks.workunit.client.0.smithi107.stdout:4/226: creat d0/d9/de/d14/d33/f45 x:0 0 0 2024-03-04T13:43:02.924 INFO:tasks.workunit.client.1.smithi169.stdout:8/755: dread d7/d10/d17/f27 [0,4194304] 0 2024-03-04T13:43:02.924 INFO:tasks.workunit.client.1.smithi169.stdout:0/630: chown d5/d9/dc/d12/d15/d46/f8a 302758 1 2024-03-04T13:43:02.925 INFO:tasks.workunit.client.1.smithi169.stdout:9/807: dwrite d7/dc/d3e/da3/fc9 [4194304,4194304] 0 2024-03-04T13:43:02.926 INFO:tasks.workunit.client.0.smithi107.stdout:8/268: creat d6/da/d25/f4b x:0 0 0 2024-03-04T13:43:02.927 INFO:tasks.workunit.client.1.smithi169.stdout:6/741: truncate d0/d3c/d40/f109 445463 0 2024-03-04T13:43:02.927 INFO:tasks.workunit.client.0.smithi107.stdout:8/269: chown d6/d16/f24 2650 1 2024-03-04T13:43:02.931 INFO:tasks.workunit.client.0.smithi107.stdout:0/281: symlink da/dc/dd/l5c 0 2024-03-04T13:43:02.941 INFO:tasks.workunit.client.0.smithi107.stdout:9/252: mkdir d4/da/d11/d4c/d4d 0 2024-03-04T13:43:02.943 INFO:tasks.workunit.client.0.smithi107.stdout:4/227: dwrite d0/d9/d10/f1c [0,4194304] 0 2024-03-04T13:43:02.944 INFO:tasks.workunit.client.0.smithi107.stdout:9/253: chown d4/da/d11/d20/d23/d2c/f38 13 1 2024-03-04T13:43:02.944 INFO:tasks.workunit.client.0.smithi107.stdout:5/199: creat d5/d8/f45 x:0 0 0 2024-03-04T13:43:02.947 INFO:tasks.workunit.client.1.smithi169.stdout:7/805: creat d2/d15/dad/f110 x:0 0 0 2024-03-04T13:43:02.949 INFO:tasks.workunit.client.1.smithi169.stdout:7/806: fsync d2/d28/fe6 0 2024-03-04T13:43:02.950 INFO:tasks.workunit.client.0.smithi107.stdout:1/237: mknod d1/d10/c49 0 2024-03-04T13:43:02.950 INFO:tasks.workunit.client.1.smithi169.stdout:5/864: mknod d1/d16/d17/d28/c135 0 2024-03-04T13:43:02.953 INFO:tasks.workunit.client.0.smithi107.stdout:7/224: mknod d6/d13/d16/d19/d1e/d22/c4b 0 2024-03-04T13:43:02.954 INFO:tasks.workunit.client.1.smithi169.stdout:1/706: truncate d4/f7d 1977138 0 2024-03-04T13:43:02.955 INFO:tasks.workunit.client.0.smithi107.stdout:7/225: read d6/fa [3228752,39710] 0 2024-03-04T13:43:02.955 INFO:tasks.workunit.client.0.smithi107.stdout:2/257: dwrite da/f2e [0,4194304] 0 2024-03-04T13:43:02.955 INFO:tasks.workunit.client.0.smithi107.stdout:7/226: fsync f4 0 2024-03-04T13:43:02.957 INFO:tasks.workunit.client.0.smithi107.stdout:2/258: truncate da/d29/f2d 477801 0 2024-03-04T13:43:02.958 INFO:tasks.workunit.client.1.smithi169.stdout:2/808: creat d1/d6d/dbc/f119 x:0 0 0 2024-03-04T13:43:02.961 INFO:tasks.workunit.client.0.smithi107.stdout:1/238: dread d1/d3/d17/f34 [0,4194304] 0 2024-03-04T13:43:02.962 INFO:tasks.workunit.client.1.smithi169.stdout:6/742: rename d0/d3c/d40/df6/ce1 to d0/d7a/c10e 0 2024-03-04T13:43:02.962 INFO:tasks.workunit.client.1.smithi169.stdout:3/757: dread d7/f49 [0,4194304] 0 2024-03-04T13:43:02.964 INFO:tasks.workunit.client.1.smithi169.stdout:0/631: mknod d5/d4c/d65/d95/dba/cdc 0 2024-03-04T13:43:02.966 INFO:tasks.workunit.client.0.smithi107.stdout:6/287: unlink df/d38/c42 0 2024-03-04T13:43:02.967 INFO:tasks.workunit.client.1.smithi169.stdout:8/756: truncate d7/d10/d17/d44/d36/d62/fca 2848674 0 2024-03-04T13:43:02.969 INFO:tasks.workunit.client.1.smithi169.stdout:9/808: truncate d7/dc/d12/d32/d42/f58 1235541 0 2024-03-04T13:43:02.970 INFO:tasks.workunit.client.0.smithi107.stdout:8/270: write d6/fd [4010373,109557] 0 2024-03-04T13:43:02.977 INFO:tasks.workunit.client.0.smithi107.stdout:3/230: truncate d1/de/f19 3856559 0 2024-03-04T13:43:02.979 INFO:tasks.workunit.client.0.smithi107.stdout:3/231: chown d1/f14 359 1 2024-03-04T13:43:02.980 INFO:tasks.workunit.client.1.smithi169.stdout:2/809: rmdir d1/d5/d19/d1e/d29 39 2024-03-04T13:43:02.980 INFO:tasks.workunit.client.1.smithi169.stdout:7/807: rename d2/c48 to d2/d28/c111 0 2024-03-04T13:43:02.984 INFO:tasks.workunit.client.1.smithi169.stdout:6/743: symlink d0/d3c/d40/d52/l10f 0 2024-03-04T13:43:02.986 INFO:tasks.workunit.client.1.smithi169.stdout:0/632: unlink d5/d9/d3d/d59/f58 0 2024-03-04T13:43:02.989 INFO:tasks.workunit.client.0.smithi107.stdout:7/227: rename d6/c9 to d6/d13/d16/d19/d38/d37/c4c 0 2024-03-04T13:43:02.990 INFO:tasks.workunit.client.1.smithi169.stdout:6/744: dread d0/d32/d3a/d43/f45 [0,4194304] 0 2024-03-04T13:43:02.990 INFO:tasks.workunit.client.1.smithi169.stdout:3/758: mknod d7/de/d2b/d7d/d85/d18/d21/d3f/cfb 0 2024-03-04T13:43:02.991 INFO:tasks.workunit.client.0.smithi107.stdout:1/239: symlink d1/d3/d17/l4a 0 2024-03-04T13:43:02.993 INFO:tasks.workunit.client.1.smithi169.stdout:3/759: fsync d7/de/d2b/d7d/d85/d18/d21/d3f/f69 0 2024-03-04T13:43:02.994 INFO:tasks.workunit.client.1.smithi169.stdout:3/760: fsync d7/f70 0 2024-03-04T13:43:03.000 INFO:tasks.workunit.client.1.smithi169.stdout:9/809: creat d7/dc/df1/d56/f122 x:0 0 0 2024-03-04T13:43:03.002 INFO:tasks.workunit.client.0.smithi107.stdout:4/228: mknod d0/d9/d18/d29/c46 0 2024-03-04T13:43:03.002 INFO:tasks.workunit.client.1.smithi169.stdout:9/810: chown d7/d27/f76 0 1 2024-03-04T13:43:03.004 INFO:tasks.workunit.client.0.smithi107.stdout:8/271: truncate d6/d16/f27 7097509 0 2024-03-04T13:43:03.004 INFO:tasks.workunit.client.0.smithi107.stdout:5/200: dwrite d5/d8/f17 [0,4194304] 0 2024-03-04T13:43:03.005 INFO:tasks.workunit.client.1.smithi169.stdout:9/811: write d7/d25/d3c/f62 [1086831,99025] 0 2024-03-04T13:43:03.009 INFO:tasks.workunit.client.0.smithi107.stdout:3/232: mkdir d1/de/d11/d12/d4c 0 2024-03-04T13:43:03.009 INFO:tasks.workunit.client.1.smithi169.stdout:0/633: dwrite d5/d9/dc/d12/d15/d19/d61/d71/d8c/d90/fc0 [0,4194304] 0 2024-03-04T13:43:03.013 INFO:tasks.workunit.client.1.smithi169.stdout:0/634: chown d5/d9/dc/d12/d15/d46/d98/dc1 18702171 1 2024-03-04T13:43:03.021 INFO:tasks.workunit.client.0.smithi107.stdout:6/288: rename df/d2e/c35 to df/d17/c55 0 2024-03-04T13:43:03.021 INFO:tasks.workunit.client.1.smithi169.stdout:0/635: truncate d5/d9/dc/d12/d15/d19/d69/fd3 1003355 0 2024-03-04T13:43:03.022 INFO:tasks.workunit.client.0.smithi107.stdout:1/240: rename d1/d3/df/d33 to d1/d3/df/d33/d43/d4b 22 2024-03-04T13:43:03.022 INFO:tasks.workunit.client.0.smithi107.stdout:4/229: rename d0 to d0/d9/d10/d47 22 2024-03-04T13:43:03.026 INFO:tasks.workunit.client.0.smithi107.stdout:0/282: rename da/dc/dd/d3e/l57 to da/dc/dd/d3e/l5d 0 2024-03-04T13:43:03.027 INFO:tasks.workunit.client.0.smithi107.stdout:8/272: mkdir d6/d16/d1f/d4c 0 2024-03-04T13:43:03.028 INFO:tasks.workunit.client.0.smithi107.stdout:0/283: dread da/dc/dd/d3e/f5b [0,4194304] 0 2024-03-04T13:43:03.030 INFO:tasks.workunit.client.0.smithi107.stdout:5/201: write d5/dc/df/f2e [646737,73983] 0 2024-03-04T13:43:03.033 INFO:tasks.workunit.client.1.smithi169.stdout:3/761: dread f5 [4194304,4194304] 0 2024-03-04T13:43:03.036 INFO:tasks.workunit.client.1.smithi169.stdout:3/762: chown d7/de/d2b/d7d/d85/d13/d1a/d2d/fce 0 1 2024-03-04T13:43:03.040 INFO:tasks.workunit.client.0.smithi107.stdout:8/273: write d6/fe [1910398,55576] 0 2024-03-04T13:43:03.042 INFO:tasks.workunit.client.0.smithi107.stdout:0/284: chown da/f2a 4008 1 2024-03-04T13:43:03.046 INFO:tasks.workunit.client.0.smithi107.stdout:5/202: symlink d5/dc/d22/l46 0 2024-03-04T13:43:03.049 INFO:tasks.workunit.client.0.smithi107.stdout:6/289: mknod df/d1b/d2c/c56 0 2024-03-04T13:43:03.050 INFO:tasks.workunit.client.0.smithi107.stdout:7/228: link d6/d13/d16/d19/c3e d6/d13/c4d 0 2024-03-04T13:43:03.051 INFO:tasks.workunit.client.1.smithi169.stdout:3/763: dwrite d7/db8/ded/ff6 [0,4194304] 0 2024-03-04T13:43:03.053 INFO:tasks.workunit.client.0.smithi107.stdout:4/230: mknod d0/d9/c48 0 2024-03-04T13:43:03.053 INFO:tasks.workunit.client.0.smithi107.stdout:7/229: read - d6/d13/d16/d1c/f48 zero size 2024-03-04T13:43:03.056 INFO:tasks.workunit.client.0.smithi107.stdout:1/241: dread d1/d10/d20/d24/f2d [0,4194304] 0 2024-03-04T13:43:03.064 INFO:tasks.workunit.client.0.smithi107.stdout:0/285: symlink da/dc/dd/d3e/d47/l5e 0 2024-03-04T13:43:03.065 INFO:tasks.workunit.client.0.smithi107.stdout:5/203: mknod d5/d8/d2a/c47 0 2024-03-04T13:43:03.066 INFO:tasks.workunit.client.0.smithi107.stdout:3/233: truncate d1/d1e/f22 3465364 0 2024-03-04T13:43:03.075 INFO:tasks.workunit.client.0.smithi107.stdout:1/242: dread d1/d3/df/f32 [0,4194304] 0 2024-03-04T13:43:03.077 INFO:tasks.workunit.client.0.smithi107.stdout:0/286: creat da/dc/dd/d1e/f5f x:0 0 0 2024-03-04T13:43:03.078 INFO:tasks.workunit.client.0.smithi107.stdout:5/204: creat d5/d8/d2a/f48 x:0 0 0 2024-03-04T13:43:03.079 INFO:tasks.workunit.client.0.smithi107.stdout:8/274: write f3 [5463232,61623] 0 2024-03-04T13:43:03.079 INFO:tasks.workunit.client.1.smithi169.stdout:4/724: link d4/d11/d2e/d37/d73/cbe d4/d1c/d75/d81/cdf 0 2024-03-04T13:43:03.080 INFO:tasks.workunit.client.1.smithi169.stdout:7/808: truncate d2/de/d3e/f62 638725 0 2024-03-04T13:43:03.082 INFO:tasks.workunit.client.0.smithi107.stdout:4/231: mknod d0/d9/de/c49 0 2024-03-04T13:43:03.085 INFO:tasks.workunit.client.1.smithi169.stdout:6/745: creat d0/d32/d3e/f110 x:0 0 0 2024-03-04T13:43:03.086 INFO:tasks.workunit.client.1.smithi169.stdout:6/746: readlink d0/d32/d3e/l9a 0 2024-03-04T13:43:03.087 INFO:tasks.workunit.client.0.smithi107.stdout:0/287: symlink da/dc/dd/d1e/l60 0 2024-03-04T13:43:03.089 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:03 smithi107 ceph-mon[28739]: pgmap v53: 65 pgs: 65 active+clean; 1.6 GiB data, 6.3 GiB used, 530 GiB / 536 GiB avail; 39 MiB/s rd, 135 MiB/s wr, 280 op/s 2024-03-04T13:43:03.090 INFO:tasks.workunit.client.0.smithi107.stdout:5/205: dread d5/dc/df/f35 [0,4194304] 0 2024-03-04T13:43:03.093 INFO:tasks.workunit.client.0.smithi107.stdout:7/230: link d6/d13/d16/d1c/f35 d6/d13/d16/d19/d39/f4e 0 2024-03-04T13:43:03.095 INFO:tasks.workunit.client.0.smithi107.stdout:6/290: truncate df/d17/f1e 1760959 0 2024-03-04T13:43:03.096 INFO:tasks.workunit.client.0.smithi107.stdout:4/232: creat d0/d9/d10/f4a x:0 0 0 2024-03-04T13:43:03.097 INFO:tasks.workunit.client.0.smithi107.stdout:0/288: symlink da/dc/dd/d3e/d26/l61 0 2024-03-04T13:43:03.098 INFO:tasks.workunit.client.0.smithi107.stdout:4/233: write d0/d9/f37 [928014,102594] 0 2024-03-04T13:43:03.101 INFO:tasks.workunit.client.0.smithi107.stdout:4/234: write d0/d9/f37 [1956397,123102] 0 2024-03-04T13:43:03.102 INFO:tasks.workunit.client.1.smithi169.stdout:0/636: truncate d5/d9/dc/d12/f24 4361995 0 2024-03-04T13:43:03.104 INFO:tasks.workunit.client.0.smithi107.stdout:4/235: chown d0/d9/de/f43 0 1 2024-03-04T13:43:03.106 INFO:tasks.workunit.client.0.smithi107.stdout:6/291: unlink df/d1b/d2c/l3a 0 2024-03-04T13:43:03.109 INFO:tasks.workunit.client.0.smithi107.stdout:1/243: dwrite d1/d3/df/f32 [0,4194304] 0 2024-03-04T13:43:03.109 INFO:tasks.workunit.client.1.smithi169.stdout:1/707: rename d4/d5/d36/d35/d16/lc8 to d4/d5/d36/d35/d8f/d93/le5 0 2024-03-04T13:43:03.112 INFO:tasks.workunit.client.1.smithi169.stdout:2/810: mkdir d1/d5/d19/d11a 0 2024-03-04T13:43:03.113 INFO:tasks.workunit.client.0.smithi107.stdout:7/231: dwrite d6/d13/d16/d19/d38/d2c/f4a [0,4194304] 0 2024-03-04T13:43:03.113 INFO:tasks.workunit.client.0.smithi107.stdout:1/244: write d1/d3/df/f32 [656890,91640] 0 2024-03-04T13:43:03.113 INFO:tasks.workunit.client.0.smithi107.stdout:3/234: getdents d1/d2d 0 2024-03-04T13:43:03.116 INFO:tasks.workunit.client.0.smithi107.stdout:0/289: unlink da/f2f 0 2024-03-04T13:43:03.119 INFO:tasks.workunit.client.0.smithi107.stdout:8/275: creat d6/da/f4d x:0 0 0 2024-03-04T13:43:03.119 INFO:tasks.workunit.client.0.smithi107.stdout:0/290: truncate da/dc/f4d 576690 0 2024-03-04T13:43:03.120 INFO:tasks.workunit.client.1.smithi169.stdout:8/757: creat d7/d10/d22/d46/da5/ff6 x:0 0 0 2024-03-04T13:43:03.121 INFO:tasks.workunit.client.0.smithi107.stdout:8/276: write d6/da/f4d [281832,58203] 0 2024-03-04T13:43:03.122 INFO:tasks.workunit.client.1.smithi169.stdout:8/758: chown d7/cc 518 1 2024-03-04T13:43:03.126 INFO:tasks.workunit.client.1.smithi169.stdout:5/865: getdents d1 0 2024-03-04T13:43:03.126 INFO:tasks.workunit.client.1.smithi169.stdout:9/812: getdents d7/dc/d3e/da3/d101 0 2024-03-04T13:43:03.127 INFO:tasks.workunit.client.0.smithi107.stdout:6/292: mkdir df/d57 0 2024-03-04T13:43:03.129 INFO:tasks.workunit.client.1.smithi169.stdout:3/764: mknod d7/cfc 0 2024-03-04T13:43:03.130 INFO:tasks.workunit.client.1.smithi169.stdout:3/765: chown d7/de/d2b/d7d 7538414 1 2024-03-04T13:43:03.132 INFO:tasks.workunit.client.1.smithi169.stdout:1/708: symlink d4/d5/d36/d35/d70/le6 0 2024-03-04T13:43:03.134 INFO:tasks.workunit.client.1.smithi169.stdout:2/811: readlink d1/d5/d19/d1e/d29/l2f 0 2024-03-04T13:43:03.136 INFO:tasks.workunit.client.0.smithi107.stdout:1/245: unlink d1/c30 0 2024-03-04T13:43:03.138 INFO:tasks.workunit.client.0.smithi107.stdout:7/232: rmdir d6/d13/d14 39 2024-03-04T13:43:03.138 INFO:tasks.workunit.client.1.smithi169.stdout:2/812: dread d1/d5/d22/d3b/d99/f10f [0,4194304] 0 2024-03-04T13:43:03.139 INFO:tasks.workunit.client.1.smithi169.stdout:4/725: creat d4/d11/d2e/d37/d5c/d86/db8/fe0 x:0 0 0 2024-03-04T13:43:03.140 INFO:tasks.workunit.client.0.smithi107.stdout:7/233: write d6/d13/d16/d19/d1e/d3d/f43 [173538,70310] 0 2024-03-04T13:43:03.142 INFO:tasks.workunit.client.1.smithi169.stdout:6/747: symlink d0/l111 0 2024-03-04T13:43:03.143 INFO:tasks.workunit.client.0.smithi107.stdout:3/235: mkdir d1/de/d18/d33/d4d 0 2024-03-04T13:43:03.144 INFO:tasks.workunit.client.1.smithi169.stdout:8/759: unlink d7/d10/d17/d44/d4c/le8 0 2024-03-04T13:43:03.146 INFO:tasks.workunit.client.1.smithi169.stdout:4/726: dread d4/d11/d3b/d67/f84 [0,4194304] 0 2024-03-04T13:43:03.146 INFO:tasks.workunit.client.1.smithi169.stdout:5/866: symlink d1/d81/d91/d129/l136 0 2024-03-04T13:43:03.147 INFO:tasks.workunit.client.1.smithi169.stdout:8/760: dread - d7/d10/d22/d75/d79/fb1 zero size 2024-03-04T13:43:03.148 INFO:tasks.workunit.client.1.smithi169.stdout:9/813: dread - d7/dc/d12/d20/d4e/fce zero size 2024-03-04T13:43:03.149 INFO:tasks.workunit.client.1.smithi169.stdout:4/727: write d4/d1c/d58/d9d/fda [325518,45875] 0 2024-03-04T13:43:03.150 INFO:tasks.workunit.client.1.smithi169.stdout:9/814: chown d7/d25/d64 9742633 1 2024-03-04T13:43:03.153 INFO:tasks.workunit.client.0.smithi107.stdout:5/206: link d5/dc/df/f2e d5/f49 0 2024-03-04T13:43:03.153 INFO:tasks.workunit.client.1.smithi169.stdout:9/815: truncate d7/d27/dc3/f108 383164 0 2024-03-04T13:43:03.155 INFO:tasks.workunit.client.1.smithi169.stdout:1/709: creat d4/d1d/fe7 x:0 0 0 2024-03-04T13:43:03.157 INFO:tasks.workunit.client.0.smithi107.stdout:1/246: mkdir d1/d10/d20/d4c 0 2024-03-04T13:43:03.158 INFO:tasks.workunit.client.1.smithi169.stdout:2/813: mkdir d1/d5/d19/d3c/db8/de1/d64/dce/dd5/d11b 0 2024-03-04T13:43:03.159 INFO:tasks.workunit.client.1.smithi169.stdout:6/748: mkdir d0/d7a/d112 0 2024-03-04T13:43:03.160 INFO:tasks.workunit.client.0.smithi107.stdout:1/247: chown d1/d10/d16/f37 49 1 2024-03-04T13:43:03.161 INFO:tasks.workunit.client.1.smithi169.stdout:8/761: unlink d7/d10/d22/d75/db9/dbe/ld7 0 2024-03-04T13:43:03.163 INFO:tasks.workunit.client.1.smithi169.stdout:5/867: dwrite d1/d16/dbd/f133 [0,4194304] 0 2024-03-04T13:43:03.163 INFO:tasks.workunit.client.1.smithi169.stdout:4/728: mknod d4/d11/d2e/d6c/d78/da5/d77/ce1 0 2024-03-04T13:43:03.163 INFO:tasks.workunit.client.0.smithi107.stdout:8/277: mknod d6/da/d25/c4e 0 2024-03-04T13:43:03.165 INFO:tasks.workunit.client.1.smithi169.stdout:0/637: rename d5/d9/d3d/d5d/l5e to d5/d9/dc/d12/d15/d19/d77/ldd 0 2024-03-04T13:43:03.166 INFO:tasks.workunit.client.0.smithi107.stdout:4/236: creat d0/d9/de/d14/f4b x:0 0 0 2024-03-04T13:43:03.168 INFO:tasks.workunit.client.0.smithi107.stdout:4/237: fsync d0/d9/de/f43 0 2024-03-04T13:43:03.172 INFO:tasks.workunit.client.0.smithi107.stdout:7/234: symlink d6/d13/d14/l4f 0 2024-03-04T13:43:03.172 INFO:tasks.workunit.client.1.smithi169.stdout:6/749: write d0/d3c/d40/d52/dac/fc7 [1281852,22659] 0 2024-03-04T13:43:03.173 INFO:tasks.workunit.client.1.smithi169.stdout:5/868: rmdir d1/d16/d107 39 2024-03-04T13:43:03.173 INFO:tasks.workunit.client.0.smithi107.stdout:3/236: mknod d1/de/d18/d33/d48/c4e 0 2024-03-04T13:43:03.174 INFO:tasks.workunit.client.0.smithi107.stdout:0/291: creat da/f62 x:0 0 0 2024-03-04T13:43:03.174 INFO:tasks.workunit.client.1.smithi169.stdout:4/729: symlink d4/d11/le2 0 2024-03-04T13:43:03.176 INFO:tasks.workunit.client.0.smithi107.stdout:5/207: creat d5/d8/d2a/d43/f4a x:0 0 0 2024-03-04T13:43:03.177 INFO:tasks.workunit.client.1.smithi169.stdout:6/750: truncate d0/d32/d3a/d43/d74/d93/f104 598985 0 2024-03-04T13:43:03.177 INFO:tasks.workunit.client.1.smithi169.stdout:8/762: creat d7/d10/d17/d44/d4c/ff7 x:0 0 0 2024-03-04T13:43:03.177 INFO:tasks.workunit.client.0.smithi107.stdout:6/293: mknod df/d17/c58 0 2024-03-04T13:43:03.182 INFO:tasks.workunit.client.0.smithi107.stdout:7/235: chown d6/d13/d16/d19/c3e 21 1 2024-03-04T13:43:03.183 INFO:tasks.workunit.client.0.smithi107.stdout:3/237: chown d1/de/d11/l3b 998 1 2024-03-04T13:43:03.183 INFO:tasks.workunit.client.0.smithi107.stdout:7/236: chown f4 32552 1 2024-03-04T13:43:03.187 INFO:tasks.workunit.client.0.smithi107.stdout:1/248: dwrite d1/d10/d16/d1b/f3e [0,4194304] 0 2024-03-04T13:43:03.188 INFO:tasks.workunit.client.0.smithi107.stdout:8/278: dwrite d6/da/f19 [0,4194304] 0 2024-03-04T13:43:03.189 INFO:tasks.workunit.client.1.smithi169.stdout:8/763: rmdir d7/d10/d22/d46 39 2024-03-04T13:43:03.201 INFO:tasks.workunit.client.1.smithi169.stdout:6/751: symlink d0/d3c/d10b/l113 0 2024-03-04T13:43:03.203 INFO:tasks.workunit.client.0.smithi107.stdout:8/279: dread d6/da/f35 [0,4194304] 0 2024-03-04T13:43:03.206 INFO:tasks.workunit.client.1.smithi169.stdout:9/816: write d7/dc/d12/d20/da6/fdf [803959,118520] 0 2024-03-04T13:43:03.209 INFO:tasks.workunit.client.0.smithi107.stdout:3/238: rmdir d1/d1e/d26 39 2024-03-04T13:43:03.212 INFO:tasks.workunit.client.1.smithi169.stdout:8/764: dwrite d7/d10/d17/d44/d4c/ff7 [0,4194304] 0 2024-03-04T13:43:03.213 INFO:tasks.workunit.client.0.smithi107.stdout:0/292: rename da/dc/f4a to da/dc/dd/d3e/f63 0 2024-03-04T13:43:03.214 INFO:tasks.workunit.client.1.smithi169.stdout:8/765: readlink d7/d10/d17/d44/d4c/d93/lcd 0 2024-03-04T13:43:03.215 INFO:tasks.workunit.client.0.smithi107.stdout:0/293: write da/dc/d34/f38 [2908646,94615] 0 2024-03-04T13:43:03.220 INFO:tasks.workunit.client.0.smithi107.stdout:3/239: dread d1/de/f1d [0,4194304] 0 2024-03-04T13:43:03.221 INFO:tasks.workunit.client.0.smithi107.stdout:8/280: mknod d6/d16/d1f/c4f 0 2024-03-04T13:43:03.221 INFO:tasks.workunit.client.1.smithi169.stdout:1/710: truncate d4/d5/d2d/d4e/d71/d26/f7b 3884980 0 2024-03-04T13:43:03.222 INFO:tasks.workunit.client.0.smithi107.stdout:8/281: fdatasync d6/da/d25/f4b 0 2024-03-04T13:43:03.224 INFO:tasks.workunit.client.1.smithi169.stdout:5/869: write d1/d16/d2a/fcd [537292,120492] 0 2024-03-04T13:43:03.225 INFO:tasks.workunit.client.0.smithi107.stdout:1/249: mkdir d1/d3/df/d33/d43/d4d 0 2024-03-04T13:43:03.225 INFO:tasks.workunit.client.0.smithi107.stdout:5/208: write d5/dc/df/f35 [1021115,53452] 0 2024-03-04T13:43:03.229 INFO:tasks.workunit.client.0.smithi107.stdout:8/282: write d6/da/f11 [2702320,25108] 0 2024-03-04T13:43:03.229 INFO:tasks.workunit.client.1.smithi169.stdout:4/730: chown d4/d11/d2e/d37/c56 930293 1 2024-03-04T13:43:03.229 INFO:tasks.workunit.client.1.smithi169.stdout:6/752: rmdir d0/d3c/d40/d52/d98/dfa 0 2024-03-04T13:43:03.232 INFO:tasks.workunit.client.1.smithi169.stdout:4/731: write d4/f7 [449822,26222] 0 2024-03-04T13:43:03.232 INFO:tasks.workunit.client.1.smithi169.stdout:9/817: rename d7/d27/d84/le7 to d7/d25/d3c/dd2/l123 0 2024-03-04T13:43:03.236 INFO:tasks.workunit.client.0.smithi107.stdout:6/294: getdents df/d2e/d36 0 2024-03-04T13:43:03.237 INFO:tasks.workunit.client.0.smithi107.stdout:6/295: fsync fd 0 2024-03-04T13:43:03.238 INFO:tasks.workunit.client.1.smithi169.stdout:0/638: dwrite d5/d9/d3d/d59/d64/d7f/fc4 [0,4194304] 0 2024-03-04T13:43:03.238 INFO:tasks.workunit.client.1.smithi169.stdout:8/766: write d7/d10/d17/f98 [379004,112477] 0 2024-03-04T13:43:03.240 INFO:tasks.workunit.client.0.smithi107.stdout:0/294: write da/f31 [22556,112265] 0 2024-03-04T13:43:03.241 INFO:tasks.workunit.client.0.smithi107.stdout:7/237: link d6/d13/d16/d19/d38/d2c/f49 d6/d13/f50 0 2024-03-04T13:43:03.242 INFO:tasks.workunit.client.1.smithi169.stdout:0/639: chown d5/d9/d3d/d59/d64/cd5 0 1 2024-03-04T13:43:03.243 INFO:tasks.workunit.client.1.smithi169.stdout:5/870: dread d1/d16/d2f/fde [0,4194304] 0 2024-03-04T13:43:03.244 INFO:tasks.workunit.client.1.smithi169.stdout:0/640: write d5/d4c/d65/f92 [12961554,3230] 0 2024-03-04T13:43:03.250 INFO:tasks.workunit.client.0.smithi107.stdout:1/250: mkdir d1/d3/d4e 0 2024-03-04T13:43:03.254 INFO:tasks.workunit.client.0.smithi107.stdout:8/283: dread d6/d16/d1f/f34 [0,4194304] 0 2024-03-04T13:43:03.255 INFO:tasks.workunit.client.1.smithi169.stdout:4/732: write d4/d11/d3b/d67/dbb/d6e/fd4 [543205,58533] 0 2024-03-04T13:43:03.256 INFO:tasks.workunit.client.1.smithi169.stdout:1/711: unlink d4/d5/d36/d35/l3d 0 2024-03-04T13:43:03.259 INFO:tasks.workunit.client.1.smithi169.stdout:8/767: unlink d7/d10/d22/d75/lbc 0 2024-03-04T13:43:03.259 INFO:tasks.workunit.client.1.smithi169.stdout:1/712: chown d4/d5/l43 6021 1 2024-03-04T13:43:03.260 INFO:tasks.workunit.client.0.smithi107.stdout:1/251: dread d1/d3/df/f32 [0,4194304] 0 2024-03-04T13:43:03.260 INFO:tasks.workunit.client.0.smithi107.stdout:6/296: mknod df/d2e/d36/c59 0 2024-03-04T13:43:03.262 INFO:tasks.workunit.client.0.smithi107.stdout:1/252: chown d1/d3/d17/c1d 88454 1 2024-03-04T13:43:03.262 INFO:tasks.workunit.client.1.smithi169.stdout:8/768: stat d7/d10/d17/d44/d5d/f88 0 2024-03-04T13:43:03.264 INFO:tasks.workunit.client.0.smithi107.stdout:0/295: creat da/dc/dd/d3e/d26/f64 x:0 0 0 2024-03-04T13:43:03.269 INFO:tasks.workunit.client.1.smithi169.stdout:6/753: dwrite d0/d32/d3e/f66 [0,4194304] 0 2024-03-04T13:43:03.269 INFO:tasks.workunit.client.1.smithi169.stdout:0/641: rename d5/d9/d3d/d59/l3b to d5/d4c/d65/d95/lde 0 2024-03-04T13:43:03.271 INFO:tasks.workunit.client.1.smithi169.stdout:8/769: rename d7/d10 to d7/d10/d22/d75/db9/dbe/df8 22 2024-03-04T13:43:03.272 INFO:tasks.workunit.client.1.smithi169.stdout:6/754: fsync d0/d32/d3a/f4f 0 2024-03-04T13:43:03.275 INFO:tasks.workunit.client.1.smithi169.stdout:9/818: symlink d7/dc/df1/l124 0 2024-03-04T13:43:03.279 INFO:tasks.workunit.client.1.smithi169.stdout:1/713: dwrite d4/f25 [4194304,4194304] 0 2024-03-04T13:43:03.284 INFO:tasks.workunit.client.1.smithi169.stdout:5/871: mkdir d1/d16/d107/d137 0 2024-03-04T13:43:03.286 INFO:tasks.workunit.client.1.smithi169.stdout:0/642: creat d5/d9/dc/fdf x:0 0 0 2024-03-04T13:43:03.288 INFO:tasks.workunit.client.1.smithi169.stdout:8/770: read d7/f4d [3934049,70039] 0 2024-03-04T13:43:03.289 INFO:tasks.workunit.client.1.smithi169.stdout:6/755: rename d0/d32/d3a/l22 to d0/d3c/d40/d52/l114 0 2024-03-04T13:43:03.291 INFO:tasks.workunit.client.1.smithi169.stdout:9/819: mkdir d7/dc/d3e/d125 0 2024-03-04T13:43:03.293 INFO:tasks.workunit.client.1.smithi169.stdout:4/733: unlink d4/c5b 0 2024-03-04T13:43:03.294 INFO:tasks.workunit.client.1.smithi169.stdout:5/872: rmdir d1/d81/d91/d129 39 2024-03-04T13:43:03.295 INFO:tasks.workunit.client.1.smithi169.stdout:4/734: stat d4/d11/d2e/d39/c9b 0 2024-03-04T13:43:03.297 INFO:tasks.workunit.client.1.smithi169.stdout:9/820: creat d7/dc/df1/d56/f126 x:0 0 0 2024-03-04T13:43:03.300 INFO:tasks.workunit.client.1.smithi169.stdout:1/714: link d4/d5/d36/d35/d5b/c9d d4/d5/d36/d35/d16/d3a/ce8 0 2024-03-04T13:43:03.300 INFO:tasks.workunit.client.1.smithi169.stdout:5/873: write d1/d16/d2f/fef [938626,12140] 0 2024-03-04T13:43:03.300 INFO:tasks.workunit.client.1.smithi169.stdout:6/756: dread d0/d3c/d40/fde [0,4194304] 0 2024-03-04T13:43:03.300 INFO:tasks.workunit.client.1.smithi169.stdout:9/821: dread - d7/dc/df1/d56/f126 zero size 2024-03-04T13:43:03.301 INFO:tasks.workunit.client.1.smithi169.stdout:4/735: chown d4/d11/d2e/d37/d5c/d86/d8f/dca 310108 1 2024-03-04T13:43:03.301 INFO:tasks.workunit.client.1.smithi169.stdout:8/771: creat d7/d10/ff9 x:0 0 0 2024-03-04T13:43:03.303 INFO:tasks.workunit.client.1.smithi169.stdout:6/757: fdatasync d0/d3c/d40/d52/fa8 0 2024-03-04T13:43:03.305 INFO:tasks.workunit.client.1.smithi169.stdout:8/772: dread - d7/d10/d17/d44/d36/fa8 zero size 2024-03-04T13:43:03.307 INFO:tasks.workunit.client.1.smithi169.stdout:9/822: unlink d7/dc/f23 0 2024-03-04T13:43:03.309 INFO:tasks.workunit.client.1.smithi169.stdout:4/736: symlink d4/d1c/le3 0 2024-03-04T13:43:03.309 INFO:tasks.workunit.client.1.smithi169.stdout:9/823: dread - d7/dc/d12/d20/d4e/ffa zero size 2024-03-04T13:43:03.310 INFO:tasks.workunit.client.0.smithi107.stdout:3/240: mkdir d1/de/d18/d33/d4d/d4f 0 2024-03-04T13:43:03.310 INFO:tasks.workunit.client.1.smithi169.stdout:4/737: fsync d4/d11/d3b/d67/fa0 0 2024-03-04T13:43:03.312 INFO:tasks.workunit.client.0.smithi107.stdout:5/209: truncate d5/f20 131101 0 2024-03-04T13:43:03.312 INFO:tasks.workunit.client.1.smithi169.stdout:5/874: creat d1/d16/d17/d2d/d5a/d8a/f138 x:0 0 0 2024-03-04T13:43:03.313 INFO:tasks.workunit.client.0.smithi107.stdout:6/297: chown df/f43 1149 1 2024-03-04T13:43:03.318 INFO:tasks.workunit.client.0.smithi107.stdout:3/241: chown d1/d1e/d26/f3d 24031 1 2024-03-04T13:43:03.319 INFO:tasks.workunit.client.0.smithi107.stdout:0/296: mknod da/dc/c65 0 2024-03-04T13:43:03.320 INFO:tasks.workunit.client.0.smithi107.stdout:1/253: dread d1/d3/d17/f34 [0,4194304] 0 2024-03-04T13:43:03.321 INFO:tasks.workunit.client.0.smithi107.stdout:5/210: mkdir d5/d2b/d4b 0 2024-03-04T13:43:03.324 INFO:tasks.workunit.client.0.smithi107.stdout:8/284: getdents d6/d16/d1f 0 2024-03-04T13:43:03.324 INFO:tasks.workunit.client.1.smithi169.stdout:5/875: fsync d1/d16/dbd/dbb/dcc/fe7 0 2024-03-04T13:43:03.325 INFO:tasks.workunit.client.0.smithi107.stdout:0/297: rename da/dc/dd to da/dc/dd/d3e/d47/d66 22 2024-03-04T13:43:03.325 INFO:tasks.workunit.client.0.smithi107.stdout:3/242: creat d1/de/d18/d33/f50 x:0 0 0 2024-03-04T13:43:03.327 INFO:tasks.workunit.client.1.smithi169.stdout:0/643: symlink d5/d9/dc/d12/d15/d46/le0 0 2024-03-04T13:43:03.330 INFO:tasks.workunit.client.1.smithi169.stdout:8/773: getdents d7/d10/d22/d75/d81/da2/deb 0 2024-03-04T13:43:03.330 INFO:tasks.workunit.client.1.smithi169.stdout:0/644: dread - d5/d9/dc/d12/d15/d46/f8a zero size 2024-03-04T13:43:03.330 INFO:tasks.workunit.client.1.smithi169.stdout:9/824: dwrite d7/f71 [4194304,4194304] 0 2024-03-04T13:43:03.332 INFO:tasks.workunit.client.1.smithi169.stdout:1/715: creat d4/d5/d36/d35/d5b/fe9 x:0 0 0 2024-03-04T13:43:03.334 INFO:tasks.workunit.client.0.smithi107.stdout:5/211: creat d5/d8/d14/f4c x:0 0 0 2024-03-04T13:43:03.336 INFO:tasks.workunit.client.1.smithi169.stdout:4/738: mknod d4/d11/d2e/d21/da9/ce4 0 2024-03-04T13:43:03.338 INFO:tasks.workunit.client.1.smithi169.stdout:4/739: fsync d4/d1c/fb2 0 2024-03-04T13:43:03.338 INFO:tasks.workunit.client.1.smithi169.stdout:6/758: truncate d0/d32/d3a/f27 6850451 0 2024-03-04T13:43:03.340 INFO:tasks.workunit.client.0.smithi107.stdout:1/254: dwrite d1/d3/df/f3a [0,4194304] 0 2024-03-04T13:43:03.341 INFO:tasks.workunit.client.0.smithi107.stdout:3/243: rename d1/f9 to d1/d2d/f51 0 2024-03-04T13:43:03.341 INFO:tasks.workunit.client.1.smithi169.stdout:6/759: chown d0/d32/d70/la3 2396 1 2024-03-04T13:43:03.343 INFO:tasks.workunit.client.0.smithi107.stdout:0/298: creat da/dc/dd/d3e/d26/f67 x:0 0 0 2024-03-04T13:43:03.343 INFO:tasks.workunit.client.0.smithi107.stdout:3/244: chown d1/de/d11/f45 1066 1 2024-03-04T13:43:03.344 INFO:tasks.workunit.client.1.smithi169.stdout:5/876: dwrite d1/d16/d2a/d62/f12a [0,4194304] 0 2024-03-04T13:43:03.345 INFO:tasks.workunit.client.0.smithi107.stdout:0/299: dread - da/f62 zero size 2024-03-04T13:43:03.345 INFO:tasks.workunit.client.1.smithi169.stdout:8/774: write d7/d10/d22/d46/da5/da6/fb4 [153039,129813] 0 2024-03-04T13:43:03.346 INFO:tasks.workunit.client.1.smithi169.stdout:0/645: symlink d5/d9/dc/d12/d15/d46/d98/dc1/dc8/le1 0 2024-03-04T13:43:03.348 INFO:tasks.workunit.client.1.smithi169.stdout:1/716: mknod d4/d5/d36/d35/d8f/d93/cea 0 2024-03-04T13:43:03.351 INFO:tasks.workunit.client.1.smithi169.stdout:9/825: dread d7/dc/d12/d59/f6d [4194304,4194304] 0 2024-03-04T13:43:03.354 INFO:tasks.workunit.client.0.smithi107.stdout:5/212: mkdir d5/d8/d4d 0 2024-03-04T13:43:03.355 INFO:tasks.workunit.client.1.smithi169.stdout:6/760: mknod d0/d3c/d40/df6/dca/c115 0 2024-03-04T13:43:03.355 INFO:tasks.workunit.client.0.smithi107.stdout:6/298: getdents df/d1b/d2c/d31 0 2024-03-04T13:43:03.358 INFO:tasks.workunit.client.0.smithi107.stdout:3/245: creat d1/de/d18/d1f/d42/f52 x:0 0 0 2024-03-04T13:43:03.360 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:03 smithi169 ceph-mon[36313]: pgmap v53: 65 pgs: 65 active+clean; 1.6 GiB data, 6.3 GiB used, 530 GiB / 536 GiB avail; 39 MiB/s rd, 135 MiB/s wr, 280 op/s 2024-03-04T13:43:03.362 INFO:tasks.workunit.client.0.smithi107.stdout:6/299: creat df/d2e/f5a x:0 0 0 2024-03-04T13:43:03.362 INFO:tasks.workunit.client.1.smithi169.stdout:0/646: dread d5/d4c/f79 [0,4194304] 0 2024-03-04T13:43:03.364 INFO:tasks.workunit.client.0.smithi107.stdout:8/285: dwrite d6/d16/d1f/f3a [0,4194304] 0 2024-03-04T13:43:03.364 INFO:tasks.workunit.client.0.smithi107.stdout:0/300: getdents da/dc 0 2024-03-04T13:43:03.365 INFO:tasks.workunit.client.1.smithi169.stdout:8/775: mkdir d7/d10/d22/d46/dfa 0 2024-03-04T13:43:03.368 INFO:tasks.workunit.client.1.smithi169.stdout:5/877: read d1/d16/d17/d1b/f1e [800406,53968] 0 2024-03-04T13:43:03.368 INFO:tasks.workunit.client.1.smithi169.stdout:0/647: chown d5/d9/dc/d12/d15/d19/d61/f68 513752 1 2024-03-04T13:43:03.371 INFO:tasks.workunit.client.0.smithi107.stdout:1/255: dwrite d1/d10/d20/f2c [0,4194304] 0 2024-03-04T13:43:03.371 INFO:tasks.workunit.client.1.smithi169.stdout:9/826: creat d7/f127 x:0 0 0 2024-03-04T13:43:03.374 INFO:tasks.workunit.client.0.smithi107.stdout:5/213: dread d5/f6 [0,4194304] 0 2024-03-04T13:43:03.374 INFO:tasks.workunit.client.1.smithi169.stdout:1/717: dwrite d4/d1d/f40 [0,4194304] 0 2024-03-04T13:43:03.375 INFO:tasks.workunit.client.0.smithi107.stdout:3/246: symlink d1/de/l53 0 2024-03-04T13:43:03.376 INFO:tasks.workunit.client.1.smithi169.stdout:6/761: dread d0/d32/d3a/f21 [0,4194304] 0 2024-03-04T13:43:03.376 INFO:tasks.workunit.client.1.smithi169.stdout:9/827: chown d7/l4a 47 1 2024-03-04T13:43:03.381 INFO:tasks.workunit.client.1.smithi169.stdout:6/762: chown d0/d3c/f75 11 1 2024-03-04T13:43:03.382 INFO:tasks.workunit.client.1.smithi169.stdout:7/809: sync 2024-03-04T13:43:03.382 INFO:tasks.workunit.client.1.smithi169.stdout:2/814: sync 2024-03-04T13:43:03.384 INFO:tasks.workunit.client.1.smithi169.stdout:3/766: sync 2024-03-04T13:43:03.387 INFO:tasks.workunit.client.1.smithi169.stdout:1/718: truncate d4/d5/d36/d35/d8f/d93/f9e 788529 0 2024-03-04T13:43:03.387 INFO:tasks.workunit.client.0.smithi107.stdout:8/286: dwrite d6/da/f19 [8388608,4194304] 0 2024-03-04T13:43:03.387 INFO:tasks.workunit.client.1.smithi169.stdout:9/828: truncate d7/d2d/d48/d66/f10e 754602 0 2024-03-04T13:43:03.390 INFO:tasks.workunit.client.0.smithi107.stdout:9/254: sync 2024-03-04T13:43:03.390 INFO:tasks.workunit.client.0.smithi107.stdout:2/259: sync 2024-03-04T13:43:03.391 INFO:tasks.workunit.client.1.smithi169.stdout:7/810: write d2/de/d3e/d41/d69/d78/fda [352030,74624] 0 2024-03-04T13:43:03.392 INFO:tasks.workunit.client.0.smithi107.stdout:1/256: chown d1/d10/d20/l2e 78683 1 2024-03-04T13:43:03.393 INFO:tasks.workunit.client.1.smithi169.stdout:4/740: write d4/d11/d2e/d37/f45 [3627626,88247] 0 2024-03-04T13:43:03.397 INFO:tasks.workunit.client.1.smithi169.stdout:1/719: write d4/d5/d36/f88 [3266354,105825] 0 2024-03-04T13:43:03.399 INFO:tasks.workunit.client.1.smithi169.stdout:0/648: fsync d5/d9/dc/d12/f24 0 2024-03-04T13:43:03.402 INFO:tasks.workunit.client.1.smithi169.stdout:7/811: write d2/de/d3e/d7e/fe2 [4536953,21802] 0 2024-03-04T13:43:03.403 INFO:tasks.workunit.client.0.smithi107.stdout:8/287: dread d6/d16/d1f/f43 [0,4194304] 0 2024-03-04T13:43:03.406 INFO:tasks.workunit.client.0.smithi107.stdout:0/301: symlink da/l68 0 2024-03-04T13:43:03.407 INFO:tasks.workunit.client.0.smithi107.stdout:0/302: readlink da/dc/d34/l56 0 2024-03-04T13:43:03.411 INFO:tasks.workunit.client.1.smithi169.stdout:9/829: rename d7/dc to d7/dc/df1/d128 22 2024-03-04T13:43:03.412 INFO:tasks.workunit.client.0.smithi107.stdout:2/260: mkdir da/df/d33/d53 0 2024-03-04T13:43:03.415 INFO:tasks.workunit.client.0.smithi107.stdout:5/214: mknod d5/c4e 0 2024-03-04T13:43:03.415 INFO:tasks.workunit.client.1.smithi169.stdout:6/763: dwrite d0/d3c/d40/f9d [0,4194304] 0 2024-03-04T13:43:03.415 INFO:tasks.workunit.client.0.smithi107.stdout:3/247: mkdir d1/de/d11/d12/d4c/d54 0 2024-03-04T13:43:03.418 INFO:tasks.workunit.client.1.smithi169.stdout:6/764: readlink d0/d3c/d10b/l113 0 2024-03-04T13:43:03.418 INFO:tasks.workunit.client.1.smithi169.stdout:2/815: mkdir d1/d5/d19/d3c/db8/de1/d64/dce/dd5/d11b/d11c 0 2024-03-04T13:43:03.419 INFO:tasks.workunit.client.0.smithi107.stdout:0/303: mkdir da/dc/dd/d3e/d26/d69 0 2024-03-04T13:43:03.420 INFO:tasks.workunit.client.1.smithi169.stdout:1/720: truncate d4/d1d/f8e 165974 0 2024-03-04T13:43:03.422 INFO:tasks.workunit.client.0.smithi107.stdout:6/300: write df/f29 [680707,86323] 0 2024-03-04T13:43:03.424 INFO:tasks.workunit.client.0.smithi107.stdout:4/238: sync 2024-03-04T13:43:03.426 INFO:tasks.workunit.client.0.smithi107.stdout:2/261: rmdir da/d34 39 2024-03-04T13:43:03.427 INFO:tasks.workunit.client.1.smithi169.stdout:8/776: dwrite d7/f4d [0,4194304] 0 2024-03-04T13:43:03.429 INFO:tasks.workunit.client.0.smithi107.stdout:1/257: creat d1/d3/df/d33/d43/d4d/f4f x:0 0 0 2024-03-04T13:43:03.429 INFO:tasks.workunit.client.0.smithi107.stdout:9/255: dread d4/f8 [0,4194304] 0 2024-03-04T13:43:03.430 INFO:tasks.workunit.client.0.smithi107.stdout:1/258: chown d1/d3/df/d33/d43/d4d 0 1 2024-03-04T13:43:03.431 INFO:tasks.workunit.client.1.smithi169.stdout:4/741: symlink d4/d11/le5 0 2024-03-04T13:43:03.431 INFO:tasks.workunit.client.0.smithi107.stdout:1/259: stat d1/d3/d17/c2f 0 2024-03-04T13:43:03.432 INFO:tasks.workunit.client.0.smithi107.stdout:5/215: rename d5/d2b/c3a to d5/d8/d14/c4f 0 2024-03-04T13:43:03.434 INFO:tasks.workunit.client.1.smithi169.stdout:9/830: symlink d7/d27/l129 0 2024-03-04T13:43:03.434 INFO:tasks.workunit.client.1.smithi169.stdout:3/767: write d7/de/d2b/d7d/d85/d13/f4a [3996077,1574] 0 2024-03-04T13:43:03.436 INFO:tasks.workunit.client.0.smithi107.stdout:3/248: mknod d1/d1e/d26/c55 0 2024-03-04T13:43:03.437 INFO:tasks.workunit.client.1.smithi169.stdout:6/765: symlink d0/d32/d3e/l116 0 2024-03-04T13:43:03.438 INFO:tasks.workunit.client.0.smithi107.stdout:1/260: dread d1/d10/d20/d24/f3c [0,4194304] 0 2024-03-04T13:43:03.440 INFO:tasks.workunit.client.0.smithi107.stdout:0/304: mknod da/dc/dd/d3e/d26/c6a 0 2024-03-04T13:43:03.442 INFO:tasks.workunit.client.1.smithi169.stdout:7/812: creat d2/d28/db8/dd5/dea/f112 x:0 0 0 2024-03-04T13:43:03.443 INFO:tasks.workunit.client.0.smithi107.stdout:6/301: mkdir df/d2e/d36/d5b 0 2024-03-04T13:43:03.443 INFO:tasks.workunit.client.0.smithi107.stdout:4/239: mknod d0/d9/d10/c4c 0 2024-03-04T13:43:03.444 INFO:tasks.workunit.client.0.smithi107.stdout:0/305: chown da/dc/dd/d3e/d26/f67 288006832 1 2024-03-04T13:43:03.444 INFO:tasks.workunit.client.1.smithi169.stdout:2/816: symlink d1/d5/d19/d68/dd9/d115/l11d 0 2024-03-04T13:43:03.446 INFO:tasks.workunit.client.0.smithi107.stdout:9/256: readlink d4/da/l10 0 2024-03-04T13:43:03.446 INFO:tasks.workunit.client.1.smithi169.stdout:8/777: symlink d7/d10/d22/d46/d49/lfb 0 2024-03-04T13:43:03.447 INFO:tasks.workunit.client.1.smithi169.stdout:2/817: write d1/d5/d22/d45/d4a/f7d [4863234,109545] 0 2024-03-04T13:43:03.449 INFO:tasks.workunit.client.0.smithi107.stdout:5/216: creat d5/dc/d22/f50 x:0 0 0 2024-03-04T13:43:03.451 INFO:tasks.workunit.client.0.smithi107.stdout:8/288: creat d6/da/f50 x:0 0 0 2024-03-04T13:43:03.452 INFO:tasks.workunit.client.0.smithi107.stdout:3/249: creat d1/f56 x:0 0 0 2024-03-04T13:43:03.453 INFO:tasks.workunit.client.1.smithi169.stdout:7/813: mknod d2/de/d37/c113 0 2024-03-04T13:43:03.453 INFO:tasks.workunit.client.0.smithi107.stdout:8/289: stat d6/da/f11 0 2024-03-04T13:43:03.453 INFO:tasks.workunit.client.1.smithi169.stdout:6/766: truncate d0/d32/d3a/d43/d74/d93/d10c/ff9 402245 0 2024-03-04T13:43:03.454 INFO:tasks.workunit.client.1.smithi169.stdout:2/818: dread d1/d5/d22/f94 [0,4194304] 0 2024-03-04T13:43:03.456 INFO:tasks.workunit.client.0.smithi107.stdout:0/306: fsync f4 0 2024-03-04T13:43:03.457 INFO:tasks.workunit.client.0.smithi107.stdout:1/261: dwrite d1/d10/d20/f2c [0,4194304] 0 2024-03-04T13:43:03.458 INFO:tasks.workunit.client.0.smithi107.stdout:4/240: symlink d0/d9/de/d14/d33/l4d 0 2024-03-04T13:43:03.460 INFO:tasks.workunit.client.1.smithi169.stdout:0/649: getdents d5/d9/d3d 0 2024-03-04T13:43:03.461 INFO:tasks.workunit.client.0.smithi107.stdout:6/302: dwrite df/d17/f23 [0,4194304] 0 2024-03-04T13:43:03.461 INFO:tasks.workunit.client.0.smithi107.stdout:9/257: rmdir d4/da/d11 39 2024-03-04T13:43:03.463 INFO:tasks.workunit.client.1.smithi169.stdout:6/767: creat d0/d3c/d40/df6/dca/f117 x:0 0 0 2024-03-04T13:43:03.465 INFO:tasks.workunit.client.1.smithi169.stdout:4/742: dwrite d4/d11/d2e/d39/fb9 [0,4194304] 0 2024-03-04T13:43:03.465 INFO:tasks.workunit.client.1.smithi169.stdout:0/650: read - d5/d9/dc/d12/d15/d19/d61/d71/d8c/d90/fd2 zero size 2024-03-04T13:43:03.467 INFO:tasks.workunit.client.1.smithi169.stdout:2/819: mkdir d1/d5/d19/d3c/db8/d11e 0 2024-03-04T13:43:03.469 INFO:tasks.workunit.client.0.smithi107.stdout:3/250: creat d1/de/d11/d12/d4c/f57 x:0 0 0 2024-03-04T13:43:03.472 INFO:tasks.workunit.client.0.smithi107.stdout:3/251: readlink d1/l8 0 2024-03-04T13:43:03.475 INFO:tasks.workunit.client.0.smithi107.stdout:0/307: dread f0 [0,4194304] 0 2024-03-04T13:43:03.476 INFO:tasks.workunit.client.0.smithi107.stdout:8/290: unlink d6/da/d25/f3b 0 2024-03-04T13:43:03.477 INFO:tasks.workunit.client.1.smithi169.stdout:7/814: mkdir d2/d114 0 2024-03-04T13:43:03.482 INFO:tasks.workunit.client.1.smithi169.stdout:6/768: mkdir d0/d32/d3a/d43/d74/d93/dee/d118 0 2024-03-04T13:43:03.483 INFO:tasks.workunit.client.0.smithi107.stdout:2/262: rename da/c3f to da/c54 0 2024-03-04T13:43:03.483 INFO:tasks.workunit.client.0.smithi107.stdout:6/303: dwrite fb [4194304,4194304] 0 2024-03-04T13:43:03.484 INFO:tasks.workunit.client.1.smithi169.stdout:6/769: write d0/f3b [3657984,48914] 0 2024-03-04T13:43:03.485 INFO:tasks.workunit.client.0.smithi107.stdout:1/262: rename d1/d10/l2b to d1/d10/d16/d1b/l50 0 2024-03-04T13:43:03.485 INFO:tasks.workunit.client.0.smithi107.stdout:4/241: dwrite d0/d9/de/d14/dc/d21/f2c [0,4194304] 0 2024-03-04T13:43:03.486 INFO:tasks.workunit.client.1.smithi169.stdout:2/820: fdatasync d1/d5/d22/d3b/d99/f10f 0 2024-03-04T13:43:03.497 INFO:tasks.workunit.client.0.smithi107.stdout:5/217: truncate d5/f24 1353332 0 2024-03-04T13:43:03.500 INFO:tasks.workunit.client.0.smithi107.stdout:0/308: dwrite da/dc/d34/f45 [0,4194304] 0 2024-03-04T13:43:03.500 INFO:tasks.workunit.client.1.smithi169.stdout:7/815: fdatasync d2/de/d3e/d41/f5a 0 2024-03-04T13:43:03.504 INFO:tasks.workunit.client.1.smithi169.stdout:4/743: unlink d4/d11/d3b/d67/d7e/f82 0 2024-03-04T13:43:03.508 INFO:tasks.workunit.client.0.smithi107.stdout:6/304: dwrite f6 [0,4194304] 0 2024-03-04T13:43:03.508 INFO:tasks.workunit.client.0.smithi107.stdout:8/291: rmdir d6/d16/d1f 39 2024-03-04T13:43:03.510 INFO:tasks.workunit.client.0.smithi107.stdout:6/305: stat df/d1b/d2c/c56 0 2024-03-04T13:43:03.514 INFO:tasks.workunit.client.1.smithi169.stdout:6/770: rename d0/d3c/d95/fb3 to d0/d3c/d10b/f119 0 2024-03-04T13:43:03.519 INFO:tasks.workunit.client.1.smithi169.stdout:0/651: mknod d5/d4c/d65/d95/db0/ce2 0 2024-03-04T13:43:03.520 INFO:tasks.workunit.client.0.smithi107.stdout:2/263: rename da/df/c1f to da/d39/c55 0 2024-03-04T13:43:03.522 INFO:tasks.workunit.client.1.smithi169.stdout:2/821: truncate d1/d5/d19/f2e 2629131 0 2024-03-04T13:43:03.523 INFO:tasks.workunit.client.1.smithi169.stdout:2/822: chown d1/d5/d19/d68/d117 6795 1 2024-03-04T13:43:03.524 INFO:tasks.workunit.client.1.smithi169.stdout:1/721: truncate d4/d5/d36/f88 3703187 0 2024-03-04T13:43:03.525 INFO:tasks.workunit.client.1.smithi169.stdout:8/778: link d7/d10/d24/d3e/l78 d7/d10/d22/d75/d81/da2/d96/lfc 0 2024-03-04T13:43:03.527 INFO:tasks.workunit.client.1.smithi169.stdout:1/722: write d4/d5/d36/d35/d8f/d93/f9b [4101197,91103] 0 2024-03-04T13:43:03.535 INFO:tasks.workunit.client.1.smithi169.stdout:7/816: mkdir d2/d15/d17/d115 0 2024-03-04T13:43:03.542 INFO:tasks.workunit.client.0.smithi107.stdout:6/306: mkdir df/d17/d5c 0 2024-03-04T13:43:03.544 INFO:tasks.workunit.client.1.smithi169.stdout:2/823: mkdir d1/d5/d22/d45/d4a/d11f 0 2024-03-04T13:43:03.546 INFO:tasks.workunit.client.0.smithi107.stdout:1/263: rename d1/d10/d20/f2c to d1/d3/f51 0 2024-03-04T13:43:03.550 INFO:tasks.workunit.client.0.smithi107.stdout:2/264: creat da/d38/f56 x:0 0 0 2024-03-04T13:43:03.553 INFO:tasks.workunit.client.1.smithi169.stdout:8/779: write d7/d10/d22/d75/d81/da2/dac/fe7 [2927788,112793] 0 2024-03-04T13:43:03.554 INFO:tasks.workunit.client.1.smithi169.stdout:1/723: dwrite d4/d5/d36/d35/d5b/fbe [0,4194304] 0 2024-03-04T13:43:03.558 INFO:tasks.workunit.client.1.smithi169.stdout:8/780: stat d7/d10/d22/d75/d81/cc5 0 2024-03-04T13:43:03.562 INFO:tasks.workunit.client.1.smithi169.stdout:0/652: mknod d5/d9/dc/ce3 0 2024-03-04T13:43:03.566 INFO:tasks.workunit.client.0.smithi107.stdout:6/307: mkdir df/d17/d24/d30/d5d 0 2024-03-04T13:43:03.567 INFO:tasks.workunit.client.0.smithi107.stdout:6/308: dread - df/d1b/f4b zero size 2024-03-04T13:43:03.567 INFO:tasks.workunit.client.1.smithi169.stdout:4/744: truncate d4/d11/d2e/d37/d5c/d86/f99 697541 0 2024-03-04T13:43:03.568 INFO:tasks.workunit.client.0.smithi107.stdout:7/238: sync 2024-03-04T13:43:03.570 INFO:tasks.workunit.client.1.smithi169.stdout:4/745: fdatasync d4/d11/d3b/d67/f84 0 2024-03-04T13:43:03.571 INFO:tasks.workunit.client.1.smithi169.stdout:2/824: rename d1/d5/d19/d3c/db8/de1/d64/dce/dd5/d11b to d1/d5/d19/d3c/db8/de1/d64/dce/dd5/df5/d120 0 2024-03-04T13:43:03.573 INFO:tasks.workunit.client.0.smithi107.stdout:1/264: creat d1/d10/d20/d24/f52 x:0 0 0 2024-03-04T13:43:03.574 INFO:tasks.workunit.client.1.smithi169.stdout:1/724: fdatasync d4/d5/d36/d35/d5b/fcc 0 2024-03-04T13:43:03.575 INFO:tasks.workunit.client.0.smithi107.stdout:4/242: creat d0/d9/de/d14/f4e x:0 0 0 2024-03-04T13:43:03.577 INFO:tasks.workunit.client.1.smithi169.stdout:8/781: creat d7/d10/d17/d44/d4c/d93/ffd x:0 0 0 2024-03-04T13:43:03.583 INFO:tasks.workunit.client.0.smithi107.stdout:9/258: link d4/da/f4b d4/da/d11/d20/f4e 0 2024-03-04T13:43:03.584 INFO:tasks.workunit.client.0.smithi107.stdout:8/292: creat d6/d16/d1f/d4c/f51 x:0 0 0 2024-03-04T13:43:03.585 INFO:tasks.workunit.client.0.smithi107.stdout:6/309: chown df/f15 2018200 1 2024-03-04T13:43:03.588 INFO:tasks.workunit.client.0.smithi107.stdout:7/239: rmdir d6/d13/d16/d19/d1e/d3d 39 2024-03-04T13:43:03.592 INFO:tasks.workunit.client.0.smithi107.stdout:1/265: rename d1/d10/d20/d24/f3c to d1/d3/df/d33/d43/d4d/f53 0 2024-03-04T13:43:03.594 INFO:tasks.workunit.client.1.smithi169.stdout:1/725: stat d4/d5/d36/d75/l99 0 2024-03-04T13:43:03.597 INFO:tasks.workunit.client.1.smithi169.stdout:4/746: dread d4/d1c/f2c [0,4194304] 0 2024-03-04T13:43:03.597 INFO:tasks.workunit.client.0.smithi107.stdout:9/259: mknod d4/d34/c4f 0 2024-03-04T13:43:03.598 INFO:tasks.workunit.client.0.smithi107.stdout:9/260: truncate d4/f19 4970052 0 2024-03-04T13:43:03.599 INFO:tasks.workunit.client.1.smithi169.stdout:4/747: chown d4/d1c/l96 0 1 2024-03-04T13:43:03.599 INFO:tasks.workunit.client.1.smithi169.stdout:8/782: rename d7/d10/d17/d44/d5d/cb2 to d7/d10/d22/d75/d81/cfe 0 2024-03-04T13:43:03.600 INFO:tasks.workunit.client.0.smithi107.stdout:8/293: creat d6/d16/d1f/d4c/f52 x:0 0 0 2024-03-04T13:43:03.600 INFO:tasks.workunit.client.1.smithi169.stdout:8/783: chown d7/d10/d17/f98 32 1 2024-03-04T13:43:03.603 INFO:tasks.workunit.client.0.smithi107.stdout:5/218: write d5/f20 [781768,7457] 0 2024-03-04T13:43:03.608 INFO:tasks.workunit.client.1.smithi169.stdout:8/784: readlink d7/d10/d22/d75/d81/l86 0 2024-03-04T13:43:03.612 INFO:tasks.workunit.client.1.smithi169.stdout:7/817: dwrite d2/d28/d70/f98 [0,4194304] 0 2024-03-04T13:43:03.619 INFO:tasks.workunit.client.0.smithi107.stdout:6/310: truncate df/d17/d24/f2b 465932 0 2024-03-04T13:43:03.621 INFO:tasks.workunit.client.0.smithi107.stdout:7/240: mknod d6/d13/d16/d19/d1e/c51 0 2024-03-04T13:43:03.622 INFO:tasks.workunit.client.0.smithi107.stdout:7/241: readlink d6/d13/d16/d19/l20 0 2024-03-04T13:43:03.623 INFO:tasks.workunit.client.0.smithi107.stdout:6/311: write df/f43 [262362,21776] 0 2024-03-04T13:43:03.623 INFO:tasks.workunit.client.0.smithi107.stdout:1/266: mknod d1/d3/d4e/c54 0 2024-03-04T13:43:03.628 INFO:tasks.workunit.client.0.smithi107.stdout:9/261: write d4/da/f4a [292003,2644] 0 2024-03-04T13:43:03.631 INFO:tasks.workunit.client.0.smithi107.stdout:8/294: symlink d6/da/d25/d2d/l53 0 2024-03-04T13:43:03.632 INFO:tasks.workunit.client.0.smithi107.stdout:8/295: write d6/da/d25/f4b [393954,19789] 0 2024-03-04T13:43:03.634 INFO:tasks.workunit.client.0.smithi107.stdout:5/219: mknod d5/d2b/d4b/c51 0 2024-03-04T13:43:03.636 INFO:tasks.workunit.client.1.smithi169.stdout:2/825: dwrite d1/d5/d19/d3c/f58 [0,4194304] 0 2024-03-04T13:43:03.636 INFO:tasks.workunit.client.0.smithi107.stdout:1/267: creat d1/d3/df/d33/d43/f55 x:0 0 0 2024-03-04T13:43:03.639 INFO:tasks.workunit.client.1.smithi169.stdout:4/748: truncate d4/d1c/d75/d81/f85 2011247 0 2024-03-04T13:43:03.640 INFO:tasks.workunit.client.0.smithi107.stdout:4/243: getdents d0/d9/de/d14/dc 0 2024-03-04T13:43:03.641 INFO:tasks.workunit.client.1.smithi169.stdout:0/653: dwrite d5/d4c/d65/f6b [0,4194304] 0 2024-03-04T13:43:03.645 INFO:tasks.workunit.client.0.smithi107.stdout:9/262: truncate d4/f8 5889422 0 2024-03-04T13:43:03.646 INFO:tasks.workunit.client.1.smithi169.stdout:1/726: write d4/d1d/f8e [335148,71394] 0 2024-03-04T13:43:03.647 INFO:tasks.workunit.client.0.smithi107.stdout:9/263: write d4/da/d11/f17 [2228727,11529] 0 2024-03-04T13:43:03.650 INFO:tasks.workunit.client.1.smithi169.stdout:2/826: symlink d1/d5/d22/d3b/d99/l121 0 2024-03-04T13:43:03.651 INFO:tasks.workunit.client.1.smithi169.stdout:7/818: write d2/d28/f33 [2897940,129297] 0 2024-03-04T13:43:03.652 INFO:tasks.workunit.client.1.smithi169.stdout:2/827: chown d1/f30 1387879 1 2024-03-04T13:43:03.653 INFO:tasks.workunit.client.0.smithi107.stdout:5/220: mknod d5/d8/d2a/d43/c52 0 2024-03-04T13:43:03.654 INFO:tasks.workunit.client.1.smithi169.stdout:7/819: write d2/ff3 [7049277,81161] 0 2024-03-04T13:43:03.656 INFO:tasks.workunit.client.0.smithi107.stdout:5/221: truncate d5/d8/d2a/d43/f4a 482078 0 2024-03-04T13:43:03.657 INFO:tasks.workunit.client.0.smithi107.stdout:7/242: write d6/fa [4795956,113949] 0 2024-03-04T13:43:03.658 INFO:tasks.workunit.client.0.smithi107.stdout:6/312: mknod df/d17/d24/d40/d51/c5e 0 2024-03-04T13:43:03.658 INFO:tasks.workunit.client.0.smithi107.stdout:3/252: sync 2024-03-04T13:43:03.660 INFO:tasks.workunit.client.0.smithi107.stdout:3/253: dread - d1/f2b zero size 2024-03-04T13:43:03.661 INFO:tasks.workunit.client.0.smithi107.stdout:4/244: symlink d0/d9/d1e/l4f 0 2024-03-04T13:43:03.662 INFO:tasks.workunit.client.0.smithi107.stdout:4/245: dread - d0/d2f/f39 zero size 2024-03-04T13:43:03.663 INFO:tasks.workunit.client.1.smithi169.stdout:1/727: truncate d4/d5/fc4 2005618 0 2024-03-04T13:43:03.665 INFO:tasks.workunit.client.0.smithi107.stdout:9/264: symlink d4/da/d11/d20/d39/l50 0 2024-03-04T13:43:03.665 INFO:tasks.workunit.client.0.smithi107.stdout:4/246: dread - d0/d9/de/d14/dc/f3c zero size 2024-03-04T13:43:03.666 INFO:tasks.workunit.client.0.smithi107.stdout:1/268: getdents d1/d10/d20/d4c 0 2024-03-04T13:43:03.666 INFO:tasks.workunit.client.1.smithi169.stdout:7/820: symlink d2/de/l116 0 2024-03-04T13:43:03.671 INFO:tasks.workunit.client.0.smithi107.stdout:5/222: creat d5/d8/d2a/f53 x:0 0 0 2024-03-04T13:43:03.672 INFO:tasks.workunit.client.1.smithi169.stdout:7/821: symlink d2/d6b/l117 0 2024-03-04T13:43:03.673 INFO:tasks.workunit.client.0.smithi107.stdout:7/243: rmdir d6/d13/d16/d19/d38/d37 39 2024-03-04T13:43:03.675 INFO:tasks.workunit.client.1.smithi169.stdout:7/822: write d2/de/d3e/d41/d69/fc8 [24911,57319] 0 2024-03-04T13:43:03.675 INFO:tasks.workunit.client.0.smithi107.stdout:3/254: fdatasync d1/fa 0 2024-03-04T13:43:03.676 INFO:tasks.workunit.client.1.smithi169.stdout:4/749: truncate d4/d1c/f2c 2067418 0 2024-03-04T13:43:03.679 INFO:tasks.workunit.client.1.smithi169.stdout:7/823: write d2/d15/d17/fbf [3796274,130865] 0 2024-03-04T13:43:03.680 INFO:tasks.workunit.client.0.smithi107.stdout:8/296: creat d6/f54 x:0 0 0 2024-03-04T13:43:03.681 INFO:tasks.workunit.client.1.smithi169.stdout:0/654: write d5/d4c/d65/f8e [104947,31865] 0 2024-03-04T13:43:03.683 INFO:tasks.workunit.client.0.smithi107.stdout:9/265: mkdir d4/da/d11/d4c/d51 0 2024-03-04T13:43:03.684 INFO:tasks.workunit.client.1.smithi169.stdout:1/728: link d4/d5/d2d/d4e/d71/d26/f6d d4/d1d/feb 0 2024-03-04T13:43:03.687 INFO:tasks.workunit.client.0.smithi107.stdout:6/313: creat df/d38/f5f x:0 0 0 2024-03-04T13:43:03.687 INFO:tasks.workunit.client.0.smithi107.stdout:7/244: dread d6/d1d/f3b [0,4194304] 0 2024-03-04T13:43:03.687 INFO:tasks.workunit.client.0.smithi107.stdout:8/297: truncate d6/da/d25/f4b 547097 0 2024-03-04T13:43:03.691 INFO:tasks.workunit.client.0.smithi107.stdout:8/298: truncate d6/da/d25/f4b 663110 0 2024-03-04T13:43:03.693 INFO:tasks.workunit.client.0.smithi107.stdout:5/223: rename d5/d8/d4d to d5/d8/d14/d54 0 2024-03-04T13:43:03.693 INFO:tasks.workunit.client.0.smithi107.stdout:1/269: dread d1/d3/df/d33/d43/d4d/f53 [0,4194304] 0 2024-03-04T13:43:03.696 INFO:tasks.workunit.client.1.smithi169.stdout:2/828: dread d1/d5/f9 [0,4194304] 0 2024-03-04T13:43:03.697 INFO:tasks.workunit.client.1.smithi169.stdout:7/824: dread d2/de/fcb [0,4194304] 0 2024-03-04T13:43:03.698 INFO:tasks.workunit.client.0.smithi107.stdout:3/255: chown d1/de/f19 20 1 2024-03-04T13:43:03.699 INFO:tasks.workunit.client.1.smithi169.stdout:2/829: chown d1/d5/d19/d3c/db8/de1/ccc 1654 1 2024-03-04T13:43:03.701 INFO:tasks.workunit.client.1.smithi169.stdout:0/655: rename d5/d9/dc/d12/d15/d46 to d5/d9/d3d/d5d/de4 0 2024-03-04T13:43:03.701 INFO:tasks.workunit.client.0.smithi107.stdout:3/256: chown d1/de/d18/f47 54 1 2024-03-04T13:43:03.702 INFO:tasks.workunit.client.1.smithi169.stdout:0/656: dread - d5/d9/dc/fdf zero size 2024-03-04T13:43:03.704 INFO:tasks.workunit.client.0.smithi107.stdout:7/245: dwrite d6/d1d/f29 [0,4194304] 0 2024-03-04T13:43:03.705 INFO:tasks.workunit.client.1.smithi169.stdout:1/729: mknod d4/dd8/cec 0 2024-03-04T13:43:03.708 INFO:tasks.workunit.client.1.smithi169.stdout:2/830: rmdir d1/d5/d22/d45/dde 39 2024-03-04T13:43:03.712 INFO:tasks.workunit.client.1.smithi169.stdout:9/831: sync 2024-03-04T13:43:03.712 INFO:tasks.workunit.client.1.smithi169.stdout:5/878: sync 2024-03-04T13:43:03.713 INFO:tasks.workunit.client.0.smithi107.stdout:4/247: truncate d0/d9/de/f3b 430253 0 2024-03-04T13:43:03.714 INFO:tasks.workunit.client.0.smithi107.stdout:2/265: sync 2024-03-04T13:43:03.714 INFO:tasks.workunit.client.0.smithi107.stdout:0/309: sync 2024-03-04T13:43:03.715 INFO:tasks.workunit.client.1.smithi169.stdout:5/879: truncate d1/d16/dbd/dbb/f121 295922 0 2024-03-04T13:43:03.715 INFO:tasks.workunit.client.1.smithi169.stdout:7/825: link d2/d15/d17/d6f/f72 d2/d28/d10a/f118 0 2024-03-04T13:43:03.717 INFO:tasks.workunit.client.0.smithi107.stdout:0/310: write da/f3c [1883081,7404] 0 2024-03-04T13:43:03.721 INFO:tasks.workunit.client.0.smithi107.stdout:3/257: dwrite d1/d2d/f44 [0,4194304] 0 2024-03-04T13:43:03.722 INFO:tasks.workunit.client.1.smithi169.stdout:9/832: rename d7/dc/fd to d7/dc/d12/d59/f12a 0 2024-03-04T13:43:03.722 INFO:tasks.workunit.client.0.smithi107.stdout:9/266: fsync d4/da/f4b 0 2024-03-04T13:43:03.723 INFO:tasks.workunit.client.0.smithi107.stdout:8/299: dread d6/da/f11 [0,4194304] 0 2024-03-04T13:43:03.724 INFO:tasks.workunit.client.1.smithi169.stdout:5/880: mkdir d1/d16/d2a/d46/d9e/d139 0 2024-03-04T13:43:03.726 INFO:tasks.workunit.client.1.smithi169.stdout:7/826: symlink d2/d15/d54/l119 0 2024-03-04T13:43:03.728 INFO:tasks.workunit.client.0.smithi107.stdout:3/258: dread d1/d1e/d26/f3d [0,4194304] 0 2024-03-04T13:43:03.730 INFO:tasks.workunit.client.0.smithi107.stdout:4/248: rmdir d0 39 2024-03-04T13:43:03.732 INFO:tasks.workunit.client.1.smithi169.stdout:2/831: write d1/d6d/dbc/fbe [1264402,89323] 0 2024-03-04T13:43:03.738 INFO:tasks.workunit.client.0.smithi107.stdout:2/266: mknod da/df/d27/c57 0 2024-03-04T13:43:03.740 INFO:tasks.workunit.client.0.smithi107.stdout:2/267: write da/d29/f2d [1027252,52847] 0 2024-03-04T13:43:03.741 INFO:tasks.workunit.client.0.smithi107.stdout:1/270: mknod d1/d10/d16/d47/c56 0 2024-03-04T13:43:03.743 INFO:tasks.workunit.client.1.smithi169.stdout:7/827: dwrite d2/d28/d70/ded/f108 [0,4194304] 0 2024-03-04T13:43:03.745 INFO:tasks.workunit.client.1.smithi169.stdout:1/730: link d4/d5/d36/f9f d4/d5/d36/d35/d8f/d93/fed 0 2024-03-04T13:43:03.750 INFO:tasks.workunit.client.1.smithi169.stdout:0/657: getdents d5/d9/dc/d12 0 2024-03-04T13:43:03.750 INFO:tasks.workunit.client.1.smithi169.stdout:9/833: write d7/dc/d12/d20/f24 [3983994,8298] 0 2024-03-04T13:43:03.751 INFO:tasks.workunit.client.1.smithi169.stdout:0/658: write d5/d4c/d65/faf [2492132,85905] 0 2024-03-04T13:43:03.752 INFO:tasks.workunit.client.1.smithi169.stdout:0/659: stat d5/cae 0 2024-03-04T13:43:03.754 INFO:tasks.workunit.client.0.smithi107.stdout:9/267: fdatasync d4/da/f15 0 2024-03-04T13:43:03.755 INFO:tasks.workunit.client.0.smithi107.stdout:7/246: unlink d6/fe 0 2024-03-04T13:43:03.757 INFO:tasks.workunit.client.0.smithi107.stdout:6/314: link df/d1b/d2c/d31/c34 df/d17/d24/d40/d51/c60 0 2024-03-04T13:43:03.758 INFO:tasks.workunit.client.0.smithi107.stdout:7/247: write d6/d13/f18 [4894507,73153] 0 2024-03-04T13:43:03.760 INFO:tasks.workunit.client.1.smithi169.stdout:1/731: creat d4/d5/d2d/d4e/d71/d26/daf/fee x:0 0 0 2024-03-04T13:43:03.763 INFO:tasks.workunit.client.0.smithi107.stdout:5/224: creat d5/dc/f55 x:0 0 0 2024-03-04T13:43:03.765 INFO:tasks.workunit.client.1.smithi169.stdout:7/828: write d2/d15/d17/f81 [2039845,30586] 0 2024-03-04T13:43:03.767 INFO:tasks.workunit.client.1.smithi169.stdout:5/881: creat d1/d16/d17/d1b/f13a x:0 0 0 2024-03-04T13:43:03.768 INFO:tasks.workunit.client.1.smithi169.stdout:2/832: dwrite d1/d5/d22/d45/f72 [0,4194304] 0 2024-03-04T13:43:03.769 INFO:tasks.workunit.client.0.smithi107.stdout:2/268: mkdir da/df/d27/d58 0 2024-03-04T13:43:03.772 INFO:tasks.workunit.client.0.smithi107.stdout:0/311: mkdir da/dc/dd/d3e/d26/d69/d6b 0 2024-03-04T13:43:03.773 INFO:tasks.workunit.client.0.smithi107.stdout:4/249: dwrite d0/d9/de/d14/dc/f26 [0,4194304] 0 2024-03-04T13:43:03.775 INFO:tasks.workunit.client.0.smithi107.stdout:4/250: dread - d0/d9/d10/f4a zero size 2024-03-04T13:43:03.779 INFO:tasks.workunit.client.0.smithi107.stdout:5/225: dwrite d5/d8/d2a/f53 [0,4194304] 0 2024-03-04T13:43:03.779 INFO:tasks.workunit.client.0.smithi107.stdout:8/300: fsync f5 0 2024-03-04T13:43:03.780 INFO:tasks.workunit.client.0.smithi107.stdout:4/251: write d0/d9/de/d14/d33/f45 [221075,52160] 0 2024-03-04T13:43:03.782 INFO:tasks.workunit.client.1.smithi169.stdout:0/660: getdents d5/d9/d3d/d59/d64 0 2024-03-04T13:43:03.785 INFO:tasks.workunit.client.0.smithi107.stdout:8/301: truncate d6/da/f2c 528381 0 2024-03-04T13:43:03.788 INFO:tasks.workunit.client.1.smithi169.stdout:9/834: dwrite d7/dc/f1e [0,4194304] 0 2024-03-04T13:43:03.789 INFO:tasks.workunit.client.0.smithi107.stdout:7/248: mkdir d6/d17/d52 0 2024-03-04T13:43:03.791 INFO:tasks.workunit.client.0.smithi107.stdout:7/249: stat d6/d13/d16/d19/d1e/l2f 0 2024-03-04T13:43:03.794 INFO:tasks.workunit.client.1.smithi169.stdout:9/835: fsync d7/dc/d12/d20/da6/fdf 0 2024-03-04T13:43:03.809 INFO:tasks.workunit.client.0.smithi107.stdout:5/226: dwrite d5/d8/d2a/d43/f4a [0,4194304] 0 2024-03-04T13:43:03.811 INFO:tasks.workunit.client.1.smithi169.stdout:2/833: dread d1/f50 [0,4194304] 0 2024-03-04T13:43:03.827 INFO:tasks.workunit.client.1.smithi169.stdout:5/882: creat d1/d81/d91/f13b x:0 0 0 2024-03-04T13:43:03.852 INFO:tasks.workunit.client.1.smithi169.stdout:3/768: sync 2024-03-04T13:43:03.852 INFO:tasks.workunit.client.1.smithi169.stdout:6/771: sync 2024-03-04T13:43:03.852 INFO:tasks.workunit.client.1.smithi169.stdout:8/785: sync 2024-03-04T13:43:03.880 INFO:tasks.workunit.client.1.smithi169.stdout:7/829: write d2/f34 [327524,122234] 0 2024-03-04T13:43:03.885 INFO:tasks.workunit.client.1.smithi169.stdout:2/834: write d1/d5/d19/d1e/d29/f3e [3236184,110620] 0 2024-03-04T13:43:03.888 INFO:tasks.workunit.client.1.smithi169.stdout:2/835: chown d1/d5/d19/l1f 30906 1 2024-03-04T13:43:03.889 INFO:tasks.workunit.client.0.smithi107.stdout:2/269: chown da/d34/l42 4 1 2024-03-04T13:43:03.893 INFO:tasks.workunit.client.0.smithi107.stdout:1/271: creat d1/d10/d16/d1b/d40/f57 x:0 0 0 2024-03-04T13:43:03.902 INFO:tasks.workunit.client.0.smithi107.stdout:0/312: rename da/dc/d34/c49 to da/dc/dd/d3e/d26/d69/c6c 0 2024-03-04T13:43:03.904 INFO:tasks.workunit.client.0.smithi107.stdout:0/313: write da/dc/dd/d1e/f5f [578495,45634] 0 2024-03-04T13:43:03.905 INFO:tasks.workunit.client.1.smithi169.stdout:7/830: dwrite d2/de/d3e/d41/d69/fc8 [0,4194304] 0 2024-03-04T13:43:03.909 INFO:tasks.workunit.client.1.smithi169.stdout:7/831: chown d2/d28/lb3 0 1 2024-03-04T13:43:03.910 INFO:tasks.workunit.client.0.smithi107.stdout:6/315: mkdir df/d1b/d2c/d61 0 2024-03-04T13:43:03.914 INFO:tasks.workunit.client.1.smithi169.stdout:2/836: rename d1/d5/d19/d63/f8e to d1/d5/d22/d45/d4a/d11f/f122 0 2024-03-04T13:43:03.920 INFO:tasks.workunit.client.0.smithi107.stdout:4/252: creat d0/d30/f50 x:0 0 0 2024-03-04T13:43:03.928 INFO:tasks.workunit.client.0.smithi107.stdout:5/227: unlink d5/d8/da/ld 0 2024-03-04T13:43:03.928 INFO:tasks.workunit.client.1.smithi169.stdout:6/772: mknod d0/d32/d3a/d43/d74/c11a 0 2024-03-04T13:43:03.930 INFO:tasks.workunit.client.1.smithi169.stdout:2/837: creat d1/d5/d19/d3c/db8/de1/d64/dce/f123 x:0 0 0 2024-03-04T13:43:03.933 INFO:tasks.workunit.client.0.smithi107.stdout:3/259: link d1/de/d18/d1f/l27 d1/de/d11/d12/l58 0 2024-03-04T13:43:03.935 INFO:tasks.workunit.client.0.smithi107.stdout:2/270: mknod da/df/d33/c59 0 2024-03-04T13:43:03.936 INFO:tasks.workunit.client.1.smithi169.stdout:3/769: dread d7/de/d2b/d7d/d85/d18/d21/f9f [0,4194304] 0 2024-03-04T13:43:03.937 INFO:tasks.workunit.client.0.smithi107.stdout:1/272: symlink d1/d10/d20/l58 0 2024-03-04T13:43:03.937 INFO:tasks.workunit.client.1.smithi169.stdout:2/838: dread d1/d5/d22/d3b/d99/fa6 [0,4194304] 0 2024-03-04T13:43:03.938 INFO:tasks.workunit.client.0.smithi107.stdout:1/273: write d1/d10/d20/d24/f44 [239387,85991] 0 2024-03-04T13:43:03.946 INFO:tasks.workunit.client.0.smithi107.stdout:3/260: dwrite d1/f6 [0,4194304] 0 2024-03-04T13:43:03.948 INFO:tasks.workunit.client.0.smithi107.stdout:3/261: write d1/de/d18/d1f/f20 [2019397,2114] 0 2024-03-04T13:43:03.951 INFO:tasks.workunit.client.1.smithi169.stdout:6/773: link d0/d32/d3a/d5e/c8d d0/d7a/d112/c11b 0 2024-03-04T13:43:03.956 INFO:tasks.workunit.client.1.smithi169.stdout:6/774: truncate d0/d3c/d40/d52/dac/feb 271345 0 2024-03-04T13:43:03.960 INFO:tasks.workunit.client.0.smithi107.stdout:4/253: mknod d0/d9/d18/d29/c51 0 2024-03-04T13:43:03.961 INFO:tasks.workunit.client.0.smithi107.stdout:4/254: stat d0/d9/d18/d29/c46 0 2024-03-04T13:43:03.962 INFO:tasks.workunit.client.1.smithi169.stdout:4/750: sync 2024-03-04T13:43:03.966 INFO:tasks.workunit.client.1.smithi169.stdout:6/775: symlink d0/d7a/d112/l11c 0 2024-03-04T13:43:03.974 INFO:tasks.workunit.client.0.smithi107.stdout:5/228: creat d5/d8/d14/d2d/f56 x:0 0 0 2024-03-04T13:43:03.976 INFO:tasks.workunit.client.0.smithi107.stdout:4/255: dwrite d0/d9/de/d14/f4b [0,4194304] 0 2024-03-04T13:43:03.978 INFO:tasks.workunit.client.0.smithi107.stdout:4/256: read d0/d9/d1e/f2b [402468,22476] 0 2024-03-04T13:43:03.978 INFO:tasks.workunit.client.1.smithi169.stdout:6/776: symlink d0/d32/d3a/d43/d74/l11d 0 2024-03-04T13:43:03.979 INFO:tasks.workunit.client.0.smithi107.stdout:2/271: mknod da/df/d33/d53/c5a 0 2024-03-04T13:43:03.991 INFO:tasks.workunit.client.0.smithi107.stdout:6/316: link f0 df/d17/d24/d40/f62 0 2024-03-04T13:43:03.992 INFO:tasks.workunit.client.1.smithi169.stdout:9/836: sync 2024-03-04T13:43:03.992 INFO:tasks.workunit.client.1.smithi169.stdout:5/883: sync 2024-03-04T13:43:03.992 INFO:tasks.workunit.client.1.smithi169.stdout:1/732: sync 2024-03-04T13:43:03.993 INFO:tasks.workunit.client.0.smithi107.stdout:6/317: write df/d1b/f4b [110951,18168] 0 2024-03-04T13:43:03.997 INFO:tasks.workunit.client.0.smithi107.stdout:5/229: creat d5/d8/d2a/d43/f57 x:0 0 0 2024-03-04T13:43:03.998 INFO:tasks.workunit.client.1.smithi169.stdout:1/733: write d4/d5/d36/d75/dba/fbb [1002558,68702] 0 2024-03-04T13:43:03.998 INFO:tasks.workunit.client.1.smithi169.stdout:9/837: write d7/dc/d12/d32/d42/d4f/f50 [7884185,39912] 0 2024-03-04T13:43:04.002 INFO:tasks.workunit.client.1.smithi169.stdout:4/751: dread d4/d11/d2e/d39/d48/f4e [0,4194304] 0 2024-03-04T13:43:04.006 INFO:tasks.workunit.client.0.smithi107.stdout:4/257: unlink d0/d30/c31 0 2024-03-04T13:43:04.009 INFO:tasks.workunit.client.0.smithi107.stdout:2/272: fdatasync da/d39/f46 0 2024-03-04T13:43:04.010 INFO:tasks.workunit.client.0.smithi107.stdout:2/273: fsync da/df/d3e/f40 0 2024-03-04T13:43:04.024 INFO:tasks.workunit.client.0.smithi107.stdout:4/258: dread d0/d9/de/f3b [0,4194304] 0 2024-03-04T13:43:04.026 INFO:tasks.workunit.client.0.smithi107.stdout:4/259: chown d0/d9/de/d14/dc/f2d 0 1 2024-03-04T13:43:04.035 INFO:tasks.workunit.client.0.smithi107.stdout:5/230: creat d5/dc/f58 x:0 0 0 2024-03-04T13:43:04.041 INFO:tasks.workunit.client.0.smithi107.stdout:5/231: creat d5/dc/df/f59 x:0 0 0 2024-03-04T13:43:04.043 INFO:tasks.workunit.client.0.smithi107.stdout:5/232: chown d5/dc/f58 1125253 1 2024-03-04T13:43:04.045 INFO:tasks.workunit.client.1.smithi169.stdout:0/661: write d5/d9/dc/d12/fa4 [296417,48374] 0 2024-03-04T13:43:04.051 INFO:tasks.workunit.client.0.smithi107.stdout:5/233: mkdir d5/d8/d14/d5a 0 2024-03-04T13:43:04.055 INFO:tasks.workunit.client.0.smithi107.stdout:5/234: truncate d5/d8/d2a/f39 952673 0 2024-03-04T13:43:04.060 INFO:tasks.workunit.client.1.smithi169.stdout:0/662: dwrite d5/f16 [0,4194304] 0 2024-03-04T13:43:04.068 INFO:tasks.workunit.client.1.smithi169.stdout:1/734: symlink d4/d5/d2d/d4e/d71/lef 0 2024-03-04T13:43:04.068 INFO:tasks.workunit.client.1.smithi169.stdout:5/884: symlink d1/d16/d2f/d72/db8/l13c 0 2024-03-04T13:43:04.071 INFO:tasks.workunit.client.1.smithi169.stdout:1/735: dread - d4/d5/d2d/d4e/d71/d26/f98 zero size 2024-03-04T13:43:04.075 INFO:tasks.workunit.client.1.smithi169.stdout:4/752: fsync d4/d11/d2e/d37/d5c/d86/f99 0 2024-03-04T13:43:04.076 INFO:tasks.workunit.client.1.smithi169.stdout:5/885: unlink d1/lc5 0 2024-03-04T13:43:04.077 INFO:tasks.workunit.client.1.smithi169.stdout:4/753: truncate d4/d1c/d58/d9d/fda 1165224 0 2024-03-04T13:43:04.084 INFO:tasks.workunit.client.1.smithi169.stdout:4/754: creat d4/d11/d2e/d6c/d78/da5/d77/fe6 x:0 0 0 2024-03-04T13:43:04.090 INFO:tasks.workunit.client.1.smithi169.stdout:4/755: creat d4/d11/d3b/d67/d98/fe7 x:0 0 0 2024-03-04T13:43:04.091 INFO:tasks.workunit.client.1.smithi169.stdout:1/736: dwrite d4/d5/d2d/d4e/d71/d26/daf/fee [0,4194304] 0 2024-03-04T13:43:04.092 INFO:tasks.workunit.client.1.smithi169.stdout:4/756: stat d4/d11/d3b/d67/lcc 0 2024-03-04T13:43:04.098 INFO:tasks.workunit.client.1.smithi169.stdout:4/757: rmdir d4/d11/d2e/d37/d5c/d86 39 2024-03-04T13:43:04.101 INFO:tasks.workunit.client.1.smithi169.stdout:1/737: creat d4/d5/d36/d35/d5b/ff0 x:0 0 0 2024-03-04T13:43:04.102 INFO:tasks.workunit.client.1.smithi169.stdout:4/758: readlink d4/d11/d3b/la1 0 2024-03-04T13:43:04.104 INFO:tasks.workunit.client.0.smithi107.stdout:2/274: mknod da/c5b 0 2024-03-04T13:43:04.104 INFO:tasks.workunit.client.1.smithi169.stdout:2/839: write d1/d5/d22/d3b/d99/f10f [250202,68486] 0 2024-03-04T13:43:04.107 INFO:tasks.workunit.client.0.smithi107.stdout:6/318: rmdir df/d17 39 2024-03-04T13:43:04.110 INFO:tasks.workunit.client.0.smithi107.stdout:2/275: mkdir da/d49/d5c 0 2024-03-04T13:43:04.111 INFO:tasks.workunit.client.1.smithi169.stdout:1/738: mkdir d4/d5/d36/df1 0 2024-03-04T13:43:04.112 INFO:tasks.workunit.client.0.smithi107.stdout:6/319: chown df/d1b/d2c/d46 7 1 2024-03-04T13:43:04.113 INFO:tasks.workunit.client.0.smithi107.stdout:9/268: truncate d4/ff 360088 0 2024-03-04T13:43:04.113 INFO:tasks.workunit.client.1.smithi169.stdout:2/840: truncate d1/d5/d19/d3c/f104 1304569 0 2024-03-04T13:43:04.113 INFO:tasks.workunit.client.1.smithi169.stdout:3/770: dwrite d7/de/d2b/d7d/d85/d13/d5a/fe0 [0,4194304] 0 2024-03-04T13:43:04.117 INFO:tasks.workunit.client.1.smithi169.stdout:2/841: dread - d1/d5/d22/d3b/ffa zero size 2024-03-04T13:43:04.117 INFO:tasks.workunit.client.0.smithi107.stdout:6/320: chown df/d17/f28 138 1 2024-03-04T13:43:04.120 INFO:tasks.workunit.client.0.smithi107.stdout:3/262: write d1/de/d11/f3e [808599,90494] 0 2024-03-04T13:43:04.120 INFO:tasks.workunit.client.1.smithi169.stdout:4/759: dread d4/f69 [0,4194304] 0 2024-03-04T13:43:04.125 INFO:tasks.workunit.client.0.smithi107.stdout:3/263: dread - d1/f2b zero size 2024-03-04T13:43:04.125 INFO:tasks.workunit.client.1.smithi169.stdout:8/786: unlink d7/d10/d17/l2f 0 2024-03-04T13:43:04.126 INFO:tasks.workunit.client.1.smithi169.stdout:2/842: symlink d1/d5/d19/d3c/db8/de1/d64/l124 0 2024-03-04T13:43:04.130 INFO:tasks.workunit.client.1.smithi169.stdout:4/760: unlink d4/d11/d2e/d37/d5c/fb0 0 2024-03-04T13:43:04.130 INFO:tasks.workunit.client.0.smithi107.stdout:7/250: dwrite d6/d13/d16/d19/d39/f4e [0,4194304] 0 2024-03-04T13:43:04.133 INFO:tasks.workunit.client.0.smithi107.stdout:3/264: write d1/de/d11/d12/d4c/f57 [669408,129416] 0 2024-03-04T13:43:04.133 INFO:tasks.workunit.client.0.smithi107.stdout:7/251: stat d6/d1d/d36 0 2024-03-04T13:43:04.133 INFO:tasks.workunit.client.1.smithi169.stdout:2/843: read d1/d5/d19/f3f [769335,39857] 0 2024-03-04T13:43:04.134 INFO:tasks.workunit.client.1.smithi169.stdout:4/761: chown d4/d11/d2e/d6c/cc4 53543289 1 2024-03-04T13:43:04.135 INFO:tasks.workunit.client.0.smithi107.stdout:1/274: truncate d1/d10/d16/d1b/f3e 689476 0 2024-03-04T13:43:04.138 INFO:tasks.workunit.client.1.smithi169.stdout:8/787: fsync d7/d10/d17/d44/d36/f4a 0 2024-03-04T13:43:04.138 INFO:tasks.workunit.client.1.smithi169.stdout:1/739: dwrite d4/d5/d36/d35/d5b/f87 [4194304,4194304] 0 2024-03-04T13:43:04.138 INFO:tasks.workunit.client.0.smithi107.stdout:6/321: rmdir df/d38/d3e 0 2024-03-04T13:43:04.142 INFO:tasks.workunit.client.1.smithi169.stdout:4/762: readlink d4/d11/d2e/d37/l66 0 2024-03-04T13:43:04.143 INFO:tasks.workunit.client.0.smithi107.stdout:7/252: chown d6/d1d/f32 11679453 1 2024-03-04T13:43:04.143 INFO:tasks.workunit.client.0.smithi107.stdout:3/265: creat d1/de/d18/f59 x:0 0 0 2024-03-04T13:43:04.144 INFO:tasks.workunit.client.0.smithi107.stdout:3/266: fdatasync d1/f2b 0 2024-03-04T13:43:04.148 INFO:tasks.workunit.client.1.smithi169.stdout:4/763: stat d4/c12 0 2024-03-04T13:43:04.151 INFO:tasks.workunit.client.1.smithi169.stdout:1/740: dread - d4/d5/d2d/d4e/fa6 zero size 2024-03-04T13:43:04.152 INFO:tasks.workunit.client.0.smithi107.stdout:2/276: dwrite da/d29/f2d [0,4194304] 0 2024-03-04T13:43:04.152 INFO:tasks.workunit.client.1.smithi169.stdout:4/764: write d4/d11/d2e/d39/d48/f3c [3071507,53235] 0 2024-03-04T13:43:04.153 INFO:tasks.workunit.client.0.smithi107.stdout:2/277: stat da/d34/f41 0 2024-03-04T13:43:04.156 INFO:tasks.workunit.client.0.smithi107.stdout:2/278: chown da/df/d3e/f50 22051662 1 2024-03-04T13:43:04.157 INFO:tasks.workunit.client.1.smithi169.stdout:4/765: fdatasync d4/d11/d2e/d21/da9/fcb 0 2024-03-04T13:43:04.157 INFO:tasks.workunit.client.1.smithi169.stdout:2/844: dwrite d1/f30 [0,4194304] 0 2024-03-04T13:43:04.159 INFO:tasks.workunit.client.0.smithi107.stdout:8/302: rename d6/c17 to d6/d16/c55 0 2024-03-04T13:43:04.160 INFO:tasks.workunit.client.1.smithi169.stdout:2/845: readlink d1/d5/d22/d45/le0 0 2024-03-04T13:43:04.162 INFO:tasks.workunit.client.0.smithi107.stdout:2/279: write da/d29/f2d [2866674,17668] 0 2024-03-04T13:43:04.163 INFO:tasks.workunit.client.0.smithi107.stdout:2/280: chown da/df 104 1 2024-03-04T13:43:04.163 INFO:tasks.workunit.client.1.smithi169.stdout:8/788: rename d7/d10/d17/d44/d36/f4a to d7/d10/d24/fff 0 2024-03-04T13:43:04.165 INFO:tasks.workunit.client.0.smithi107.stdout:6/322: dwrite fb [4194304,4194304] 0 2024-03-04T13:43:04.167 INFO:tasks.workunit.client.1.smithi169.stdout:4/766: mkdir d4/d11/d2e/dcd/dd1/de8 0 2024-03-04T13:43:04.168 INFO:tasks.workunit.client.1.smithi169.stdout:4/767: chown d4/d11/d2e/d6c 7 1 2024-03-04T13:43:04.169 INFO:tasks.workunit.client.1.smithi169.stdout:3/771: truncate d7/de/d2b/d7d/d85/d13/d5a/da1/fdd 282826 0 2024-03-04T13:43:04.169 INFO:tasks.workunit.client.0.smithi107.stdout:8/303: creat d6/da/d25/d2d/d3e/f56 x:0 0 0 2024-03-04T13:43:04.169 INFO:tasks.workunit.client.0.smithi107.stdout:6/323: dread - df/d17/f52 zero size 2024-03-04T13:43:04.170 INFO:tasks.workunit.client.1.smithi169.stdout:4/768: readlink d4/d11/d3b/la1 0 2024-03-04T13:43:04.171 INFO:tasks.workunit.client.1.smithi169.stdout:8/789: creat d7/d10/d22/d75/d81/f100 x:0 0 0 2024-03-04T13:43:04.173 INFO:tasks.workunit.client.0.smithi107.stdout:1/275: dwrite d1/d3/d17/f34 [4194304,4194304] 0 2024-03-04T13:43:04.174 INFO:tasks.workunit.client.0.smithi107.stdout:2/281: mkdir da/d38/d3a/d5d 0 2024-03-04T13:43:04.175 INFO:tasks.workunit.client.0.smithi107.stdout:6/324: read - df/d17/f52 zero size 2024-03-04T13:43:04.175 INFO:tasks.workunit.client.1.smithi169.stdout:2/846: creat d1/d6d/dbc/d105/f125 x:0 0 0 2024-03-04T13:43:04.177 INFO:tasks.workunit.client.0.smithi107.stdout:2/282: chown da/df/c3d 164 1 2024-03-04T13:43:04.178 INFO:tasks.workunit.client.0.smithi107.stdout:6/325: write df/d17/d24/f2d [1550610,29362] 0 2024-03-04T13:43:04.179 INFO:tasks.workunit.client.0.smithi107.stdout:2/283: chown da/df/f1e 8457 1 2024-03-04T13:43:04.182 INFO:tasks.workunit.client.0.smithi107.stdout:0/314: rename da/dc/dd/d3e/d47/c53 to da/dc/d34/c6d 0 2024-03-04T13:43:04.182 INFO:tasks.workunit.client.0.smithi107.stdout:2/284: write da/df/d3e/f4c [265857,17646] 0 2024-03-04T13:43:04.182 INFO:tasks.workunit.client.1.smithi169.stdout:8/790: unlink d7/d10/d22/d75/d81/da2/dac/fe7 0 2024-03-04T13:43:04.185 INFO:tasks.workunit.client.1.smithi169.stdout:8/791: chown d7/d10/l39 30272 1 2024-03-04T13:43:04.191 INFO:tasks.workunit.client.1.smithi169.stdout:7/832: mknod d2/de/d3e/d7e/c11a 0 2024-03-04T13:43:04.194 INFO:tasks.workunit.client.0.smithi107.stdout:1/276: dwrite d1/d3/df/f3a [0,4194304] 0 2024-03-04T13:43:04.196 INFO:tasks.workunit.client.0.smithi107.stdout:5/235: link d5/d8/d14/f4c d5/d8/f5b 0 2024-03-04T13:43:04.197 INFO:tasks.workunit.client.1.smithi169.stdout:8/792: readlink d7/d10/d17/d44/d4c/l72 0 2024-03-04T13:43:04.199 INFO:tasks.workunit.client.0.smithi107.stdout:5/236: write d5/dc/f58 [378005,88987] 0 2024-03-04T13:43:04.199 INFO:tasks.workunit.client.0.smithi107.stdout:1/277: write d1/d3/df/d33/d43/d4d/f4f [764666,8762] 0 2024-03-04T13:43:04.200 INFO:tasks.workunit.client.0.smithi107.stdout:8/304: write d6/f3d [522773,33699] 0 2024-03-04T13:43:04.201 INFO:tasks.workunit.client.0.smithi107.stdout:0/315: stat da/dc/dd/d1e/c3d 0 2024-03-04T13:43:04.202 INFO:tasks.workunit.client.1.smithi169.stdout:3/772: link d7/de/d24/cf9 d7/de/d2b/d7d/d85/d13/d1a/d2d/d56/d5e/d67/dd7/cfd 0 2024-03-04T13:43:04.202 INFO:tasks.workunit.client.1.smithi169.stdout:4/769: dwrite d4/d11/d2e/d63/fdb [0,4194304] 0 2024-03-04T13:43:04.204 INFO:tasks.workunit.client.0.smithi107.stdout:5/237: write d5/dc/f58 [709245,11818] 0 2024-03-04T13:43:04.205 INFO:tasks.workunit.client.1.smithi169.stdout:4/770: fsync d4/f41 0 2024-03-04T13:43:04.206 INFO:tasks.workunit.client.0.smithi107.stdout:6/326: truncate f8 946214 0 2024-03-04T13:43:04.208 INFO:tasks.workunit.client.0.smithi107.stdout:2/285: dwrite da/d34/f36 [0,4194304] 0 2024-03-04T13:43:04.209 INFO:tasks.workunit.client.1.smithi169.stdout:4/771: chown d4/d11/d3b/la1 35253 1 2024-03-04T13:43:04.209 INFO:tasks.workunit.client.0.smithi107.stdout:8/305: read d6/fe [3213173,110700] 0 2024-03-04T13:43:04.209 INFO:tasks.workunit.client.0.smithi107.stdout:6/327: fdatasync df/d38/f4e 0 2024-03-04T13:43:04.213 INFO:tasks.workunit.client.0.smithi107.stdout:4/260: dwrite d0/d9/d1e/f20 [0,4194304] 0 2024-03-04T13:43:04.213 INFO:tasks.workunit.client.0.smithi107.stdout:1/278: rmdir d1/d10/d16/d1b/d40 39 2024-03-04T13:43:04.216 INFO:tasks.workunit.client.1.smithi169.stdout:2/847: link d1/d5/d19/d3c/db8/de1/f4b d1/d5/d19/d3c/db8/de1/f126 0 2024-03-04T13:43:04.226 INFO:tasks.workunit.client.1.smithi169.stdout:3/773: creat d7/de/d2b/d7d/d85/d13/d1a/d2d/d56/d5e/ffe x:0 0 0 2024-03-04T13:43:04.229 INFO:tasks.workunit.client.1.smithi169.stdout:3/774: dread - d7/db8/ded/ff1 zero size 2024-03-04T13:43:04.232 INFO:tasks.workunit.client.1.smithi169.stdout:4/772: creat d4/d11/d3b/d67/d7e/fe9 x:0 0 0 2024-03-04T13:43:04.233 INFO:tasks.workunit.client.1.smithi169.stdout:7/833: rmdir d2/d6b 39 2024-03-04T13:43:04.237 INFO:tasks.workunit.client.0.smithi107.stdout:2/286: readlink da/d34/l3b 0 2024-03-04T13:43:04.237 INFO:tasks.workunit.client.1.smithi169.stdout:2/848: creat d1/d5/d19/d63/f127 x:0 0 0 2024-03-04T13:43:04.242 INFO:tasks.workunit.client.1.smithi169.stdout:3/775: creat d7/de/d2b/d7d/d85/d13/d1a/d2d/d56/d5e/d5f/fff x:0 0 0 2024-03-04T13:43:04.247 INFO:tasks.workunit.client.0.smithi107.stdout:1/279: mkdir d1/d3/df/d33/d43/d59 0 2024-03-04T13:43:04.247 INFO:tasks.workunit.client.1.smithi169.stdout:6/777: dwrite d0/d32/d3a/d43/d74/d93/d10c/f101 [0,4194304] 0 2024-03-04T13:43:04.251 INFO:tasks.workunit.client.0.smithi107.stdout:0/316: mkdir da/dc/dd/d3e/d6e 0 2024-03-04T13:43:04.258 INFO:tasks.workunit.client.0.smithi107.stdout:6/328: unlink df/d17/f1e 0 2024-03-04T13:43:04.260 INFO:tasks.workunit.client.0.smithi107.stdout:6/329: write df/d38/f4e [431548,85925] 0 2024-03-04T13:43:04.261 INFO:tasks.workunit.client.0.smithi107.stdout:2/287: getdents da/d49/d5c 0 2024-03-04T13:43:04.262 INFO:tasks.workunit.client.0.smithi107.stdout:1/280: dwrite d1/d10/d20/d24/f52 [0,4194304] 0 2024-03-04T13:43:04.263 INFO:tasks.workunit.client.0.smithi107.stdout:8/306: creat d6/f57 x:0 0 0 2024-03-04T13:43:04.263 INFO:tasks.workunit.client.0.smithi107.stdout:2/288: chown da/df/l2c 820369554 1 2024-03-04T13:43:04.264 INFO:tasks.workunit.client.0.smithi107.stdout:5/238: rename d5/d8/d14/c4f to d5/dc/c5c 0 2024-03-04T13:43:04.265 INFO:tasks.workunit.client.0.smithi107.stdout:8/307: chown d6/da/d25/d2d/d3e/c4a 0 1 2024-03-04T13:43:04.266 INFO:tasks.workunit.client.0.smithi107.stdout:6/330: truncate df/f19 1273681 0 2024-03-04T13:43:04.267 INFO:tasks.workunit.client.0.smithi107.stdout:8/308: chown d6/da/f1b 565704096 1 2024-03-04T13:43:04.267 INFO:tasks.workunit.client.0.smithi107.stdout:1/281: creat d1/d3/d17/f5a x:0 0 0 2024-03-04T13:43:04.270 INFO:tasks.workunit.client.0.smithi107.stdout:5/239: chown d5/f24 81058 1 2024-03-04T13:43:04.270 INFO:tasks.workunit.client.0.smithi107.stdout:0/317: getdents da/dc/dd/d3e/d26 0 2024-03-04T13:43:04.271 INFO:tasks.workunit.client.0.smithi107.stdout:6/331: mkdir df/d17/d24/d40/d63 0 2024-03-04T13:43:04.284 INFO:tasks.workunit.client.0.smithi107.stdout:5/240: fdatasync d5/dc/df/f12 0 2024-03-04T13:43:04.284 INFO:tasks.workunit.client.0.smithi107.stdout:6/332: truncate df/d1b/d2c/d31/f39 909832 0 2024-03-04T13:43:04.284 INFO:tasks.workunit.client.0.smithi107.stdout:2/289: getdents da/d44 0 2024-03-04T13:43:04.285 INFO:tasks.workunit.client.0.smithi107.stdout:3/267: write d1/f3 [1282669,75438] 0 2024-03-04T13:43:04.285 INFO:tasks.workunit.client.0.smithi107.stdout:7/253: write d6/f3f [3110340,36364] 0 2024-03-04T13:43:04.287 INFO:tasks.workunit.client.0.smithi107.stdout:0/318: mknod da/dc/dd/d3e/d6e/c6f 0 2024-03-04T13:43:04.290 INFO:tasks.workunit.client.0.smithi107.stdout:5/241: mknod d5/dc/df/c5d 0 2024-03-04T13:43:04.291 INFO:tasks.workunit.client.0.smithi107.stdout:0/319: dread - da/dc/dd/d3e/f63 zero size 2024-03-04T13:43:04.291 INFO:tasks.workunit.client.0.smithi107.stdout:9/269: dwrite d4/da/f4b [0,4194304] 0 2024-03-04T13:43:04.293 INFO:tasks.workunit.client.1.smithi169.stdout:7/834: mknod d2/d28/db8/dd5/dea/c11b 0 2024-03-04T13:43:04.293 INFO:tasks.workunit.client.1.smithi169.stdout:2/849: symlink d1/d5/d19/d3c/db8/de1/d64/dce/dd5/l128 0 2024-03-04T13:43:04.293 INFO:tasks.workunit.client.1.smithi169.stdout:9/838: truncate d7/dc/d12/d20/f24 4266783 0 2024-03-04T13:43:04.294 INFO:tasks.workunit.client.0.smithi107.stdout:6/333: creat df/d57/f64 x:0 0 0 2024-03-04T13:43:04.294 INFO:tasks.workunit.client.0.smithi107.stdout:6/334: fsync df/d1b/f1d 0 2024-03-04T13:43:04.296 INFO:tasks.workunit.client.0.smithi107.stdout:7/254: read d6/d13/f18 [31817,90371] 0 2024-03-04T13:43:04.298 INFO:tasks.workunit.client.1.smithi169.stdout:8/793: getdents d7/d10/d17/d44/d36 0 2024-03-04T13:43:04.299 INFO:tasks.workunit.client.0.smithi107.stdout:7/255: write d6/d13/d16/d19/d1e/f42 [454447,46319] 0 2024-03-04T13:43:04.301 INFO:tasks.workunit.client.1.smithi169.stdout:9/839: mkdir d7/d27/d84/d12b 0 2024-03-04T13:43:04.304 INFO:tasks.workunit.client.1.smithi169.stdout:9/840: readlink d7/dc/d12/d20/leb 0 2024-03-04T13:43:04.314 INFO:tasks.workunit.client.1.smithi169.stdout:5/886: chown d1/d16/d17/d2d/f9a 961602960 1 2024-03-04T13:43:04.317 INFO:tasks.workunit.client.1.smithi169.stdout:8/794: dread d7/d10/d22/d46/da5/da6/fb4 [0,4194304] 0 2024-03-04T13:43:04.322 INFO:tasks.workunit.client.1.smithi169.stdout:7/835: dwrite d2/d15/d17/f81 [4194304,4194304] 0 2024-03-04T13:43:04.326 INFO:tasks.workunit.client.1.smithi169.stdout:9/841: mkdir d7/dc/d12/d32/d12c 0 2024-03-04T13:43:04.332 INFO:tasks.workunit.client.1.smithi169.stdout:2/850: dwrite d1/d5/d19/d3c/f104 [0,4194304] 0 2024-03-04T13:43:04.333 INFO:tasks.workunit.client.1.smithi169.stdout:5/887: mknod d1/d60/c13d 0 2024-03-04T13:43:04.334 INFO:tasks.workunit.client.0.smithi107.stdout:4/261: fsync d0/d9/f11 0 2024-03-04T13:43:04.335 INFO:tasks.workunit.client.0.smithi107.stdout:9/270: creat d4/d32/f52 x:0 0 0 2024-03-04T13:43:04.337 INFO:tasks.workunit.client.0.smithi107.stdout:2/290: link da/df/f13 da/df/d27/f5e 0 2024-03-04T13:43:04.338 INFO:tasks.workunit.client.1.smithi169.stdout:2/851: write d1/d5/d19/d68/fa7 [2838861,115898] 0 2024-03-04T13:43:04.340 INFO:tasks.workunit.client.0.smithi107.stdout:4/262: stat d0/d9/de/d14/dc/d21/f2c 0 2024-03-04T13:43:04.342 INFO:tasks.workunit.client.1.smithi169.stdout:3/776: dread d7/de/d2b/d7d/d85/d47/d63/d6c/f77 [0,4194304] 0 2024-03-04T13:43:04.342 INFO:tasks.workunit.client.1.smithi169.stdout:6/778: getdents d0/d32/d3a/def 0 2024-03-04T13:43:04.346 INFO:tasks.workunit.client.0.smithi107.stdout:5/242: dwrite d5/f32 [0,4194304] 0 2024-03-04T13:43:04.347 INFO:tasks.workunit.client.0.smithi107.stdout:8/309: truncate d6/d16/d1f/f3a 3536095 0 2024-03-04T13:43:04.349 INFO:tasks.workunit.client.1.smithi169.stdout:5/888: symlink d1/d16/d2a/d46/d9e/d105/l13e 0 2024-03-04T13:43:04.355 INFO:tasks.workunit.client.1.smithi169.stdout:9/842: symlink d7/dc/d12/d20/da6/dd1/l12d 0 2024-03-04T13:43:04.358 INFO:tasks.workunit.client.0.smithi107.stdout:7/256: creat d6/d13/d16/d19/d1e/f53 x:0 0 0 2024-03-04T13:43:04.360 INFO:tasks.workunit.client.1.smithi169.stdout:3/777: creat d7/de/d2b/d7d/d85/d13/d1a/d2d/d56/d5e/d5f/f100 x:0 0 0 2024-03-04T13:43:04.361 INFO:tasks.workunit.client.0.smithi107.stdout:4/263: creat d0/d9/de/d14/dc/f52 x:0 0 0 2024-03-04T13:43:04.363 INFO:tasks.workunit.client.0.smithi107.stdout:0/320: dwrite f4 [0,4194304] 0 2024-03-04T13:43:04.369 INFO:tasks.workunit.client.0.smithi107.stdout:2/291: mkdir da/df/d27/d58/d5f 0 2024-03-04T13:43:04.371 INFO:tasks.workunit.client.1.smithi169.stdout:2/852: symlink d1/d5/d19/d11a/l129 0 2024-03-04T13:43:04.372 INFO:tasks.workunit.client.1.smithi169.stdout:1/741: sync 2024-03-04T13:43:04.372 INFO:tasks.workunit.client.1.smithi169.stdout:0/663: sync 2024-03-04T13:43:04.373 INFO:tasks.workunit.client.0.smithi107.stdout:3/268: dread d1/f3 [4194304,4194304] 0 2024-03-04T13:43:04.373 INFO:tasks.workunit.client.0.smithi107.stdout:8/310: symlink d6/da/d25/d2d/l58 0 2024-03-04T13:43:04.378 INFO:tasks.workunit.client.0.smithi107.stdout:6/335: dwrite df/d2e/d36/f49 [0,4194304] 0 2024-03-04T13:43:04.381 INFO:tasks.workunit.client.1.smithi169.stdout:3/778: creat d7/de/d2b/d7d/d85/d13/d1a/d2d/dbe/f101 x:0 0 0 2024-03-04T13:43:04.383 INFO:tasks.workunit.client.0.smithi107.stdout:4/264: creat d0/d9/d18/f53 x:0 0 0 2024-03-04T13:43:04.387 INFO:tasks.workunit.client.0.smithi107.stdout:6/336: dread df/d2e/d36/f49 [0,4194304] 0 2024-03-04T13:43:04.389 INFO:tasks.workunit.client.0.smithi107.stdout:2/292: rmdir da/d39 39 2024-03-04T13:43:04.390 INFO:tasks.workunit.client.1.smithi169.stdout:8/795: dread - d7/d10/d22/d46/d49/fe4 zero size 2024-03-04T13:43:04.390 INFO:tasks.workunit.client.0.smithi107.stdout:6/337: truncate df/d17/d24/d40/f44 800728 0 2024-03-04T13:43:04.392 INFO:tasks.workunit.client.1.smithi169.stdout:9/843: creat d7/d27/d84/d12b/f12e x:0 0 0 2024-03-04T13:43:04.392 INFO:tasks.workunit.client.1.smithi169.stdout:5/889: write d1/d16/d2a/d46/f6c [10839,113755] 0 2024-03-04T13:43:04.394 INFO:tasks.workunit.client.0.smithi107.stdout:8/311: chown l1 81 1 2024-03-04T13:43:04.395 INFO:tasks.workunit.client.0.smithi107.stdout:8/312: readlink d6/d16/l47 0 2024-03-04T13:43:04.402 INFO:tasks.workunit.client.0.smithi107.stdout:4/265: dwrite d0/d9/de/d14/dc/f52 [0,4194304] 0 2024-03-04T13:43:04.402 INFO:tasks.workunit.client.1.smithi169.stdout:2/853: truncate d1/d6d/dbc/fd1 6090 0 2024-03-04T13:43:04.402 INFO:tasks.workunit.client.0.smithi107.stdout:0/321: symlink da/l70 0 2024-03-04T13:43:04.403 INFO:tasks.workunit.client.1.smithi169.stdout:5/890: chown d1/d16/d17/d2d/d5a/fd1 1906 1 2024-03-04T13:43:04.403 INFO:tasks.workunit.client.1.smithi169.stdout:3/779: symlink d7/de/d2b/d7d/d85/d13/d1a/d2d/d56/d5e/d67/d5c/l102 0 2024-03-04T13:43:04.404 INFO:tasks.workunit.client.0.smithi107.stdout:0/322: chown da/f3c 310 1 2024-03-04T13:43:04.406 INFO:tasks.workunit.client.0.smithi107.stdout:0/323: read - da/dc/d34/f5a zero size 2024-03-04T13:43:04.407 INFO:tasks.workunit.client.0.smithi107.stdout:6/338: mknod df/d17/d24/d27/c65 0 2024-03-04T13:43:04.407 INFO:tasks.workunit.client.1.smithi169.stdout:7/836: link d2/de/d3e/d41/d5b/l82 d2/dc0/l11c 0 2024-03-04T13:43:04.409 INFO:tasks.workunit.client.0.smithi107.stdout:2/293: dwrite da/d38/f56 [0,4194304] 0 2024-03-04T13:43:04.410 INFO:tasks.workunit.client.1.smithi169.stdout:9/844: dread d7/dc/d12/d59/f6d [4194304,4194304] 0 2024-03-04T13:43:04.410 INFO:tasks.workunit.client.1.smithi169.stdout:6/779: getdents d0/d32/d70 0 2024-03-04T13:43:04.411 INFO:tasks.workunit.client.0.smithi107.stdout:6/339: chown df/d17/c18 3783 1 2024-03-04T13:43:04.412 INFO:tasks.workunit.client.1.smithi169.stdout:3/780: dread - d7/de/d2b/d7d/d85/d13/d1a/d2d/d56/d5e/d5f/fff zero size 2024-03-04T13:43:04.415 INFO:tasks.workunit.client.1.smithi169.stdout:6/780: chown d0/d32/d3e/l9a 36531 1 2024-03-04T13:43:04.418 INFO:tasks.workunit.client.0.smithi107.stdout:4/266: symlink d0/d2f/l54 0 2024-03-04T13:43:04.418 INFO:tasks.workunit.client.1.smithi169.stdout:6/781: chown d0/d32/d3a/d43/d74/f103 56363 1 2024-03-04T13:43:04.421 INFO:tasks.workunit.client.1.smithi169.stdout:2/854: mknod d1/d5/d22/d45/d4a/c12a 0 2024-03-04T13:43:04.423 INFO:tasks.workunit.client.1.smithi169.stdout:7/837: fsync d2/de/d3e/d41/d5b/ff0 0 2024-03-04T13:43:04.423 INFO:tasks.workunit.client.1.smithi169.stdout:0/664: dwrite d5/d9/dc/d12/d15/f40 [0,4194304] 0 2024-03-04T13:43:04.423 INFO:tasks.workunit.client.0.smithi107.stdout:0/324: creat da/dc/dd/d3e/d26/f71 x:0 0 0 2024-03-04T13:43:04.423 INFO:tasks.workunit.client.0.smithi107.stdout:3/269: dread d1/d1e/f22 [0,4194304] 0 2024-03-04T13:43:04.425 INFO:tasks.workunit.client.0.smithi107.stdout:6/340: rmdir df/d17/d24 39 2024-03-04T13:43:04.425 INFO:tasks.workunit.client.0.smithi107.stdout:0/325: write da/dc/d34/f38 [4516361,36828] 0 2024-03-04T13:43:04.427 INFO:tasks.workunit.client.0.smithi107.stdout:0/326: write da/dc/dd/d3e/f35 [923681,28373] 0 2024-03-04T13:43:04.428 INFO:tasks.workunit.client.0.smithi107.stdout:5/243: link d5/dc/l18 d5/l5e 0 2024-03-04T13:43:04.429 INFO:tasks.workunit.client.1.smithi169.stdout:1/742: rename d4/d5/d2d/d4e/d71/d26/d4f/cc3 to d4/d5/d36/d35/cf2 0 2024-03-04T13:43:04.430 INFO:tasks.workunit.client.1.smithi169.stdout:9/845: creat d7/dc/d95/f12f x:0 0 0 2024-03-04T13:43:04.433 INFO:tasks.workunit.client.0.smithi107.stdout:3/270: creat d1/de/d18/d1f/d42/f5a x:0 0 0 2024-03-04T13:43:04.435 INFO:tasks.workunit.client.0.smithi107.stdout:7/257: truncate d6/d13/d16/d1c/f35 1199886 0 2024-03-04T13:43:04.440 INFO:tasks.workunit.client.0.smithi107.stdout:6/341: unlink fd 0 2024-03-04T13:43:04.443 INFO:tasks.workunit.client.1.smithi169.stdout:4/773: sync 2024-03-04T13:43:04.444 INFO:tasks.workunit.client.0.smithi107.stdout:7/258: unlink d6/d1d/f29 0 2024-03-04T13:43:04.446 INFO:tasks.workunit.client.1.smithi169.stdout:4/774: write d4/d11/d3b/d67/dbb/d6e/fc5 [652963,73816] 0 2024-03-04T13:43:04.446 INFO:tasks.workunit.client.0.smithi107.stdout:0/327: dwrite da/dc/f4e [0,4194304] 0 2024-03-04T13:43:04.450 INFO:tasks.workunit.client.1.smithi169.stdout:0/665: dwrite d5/d9/dc/d12/d15/d19/d61/f68 [0,4194304] 0 2024-03-04T13:43:04.451 INFO:tasks.workunit.client.0.smithi107.stdout:0/328: dread da/dc/d34/f42 [0,4194304] 0 2024-03-04T13:43:04.457 INFO:tasks.workunit.client.0.smithi107.stdout:0/329: chown da/f3c 98594 1 2024-03-04T13:43:04.457 INFO:tasks.workunit.client.0.smithi107.stdout:8/313: dwrite d6/d16/d1f/f23 [0,4194304] 0 2024-03-04T13:43:04.458 INFO:tasks.workunit.client.0.smithi107.stdout:0/330: write da/dc/f4e [2131669,37639] 0 2024-03-04T13:43:04.461 INFO:tasks.workunit.client.0.smithi107.stdout:6/342: rmdir df/d1b/d2c/d46 39 2024-03-04T13:43:04.461 INFO:tasks.workunit.client.0.smithi107.stdout:2/294: dread da/df/d3e/f50 [4194304,4194304] 0 2024-03-04T13:43:04.461 INFO:tasks.workunit.client.0.smithi107.stdout:0/331: stat da/dc/dd/l2d 0 2024-03-04T13:43:04.461 INFO:tasks.workunit.client.1.smithi169.stdout:5/891: write d1/d16/d17/d28/f41 [3355511,82382] 0 2024-03-04T13:43:04.463 INFO:tasks.workunit.client.0.smithi107.stdout:8/314: write d6/da/f50 [382296,88263] 0 2024-03-04T13:43:04.464 INFO:tasks.workunit.client.1.smithi169.stdout:8/796: getdents d7/d10/d22/d75/d81/da2/dac 0 2024-03-04T13:43:04.466 INFO:tasks.workunit.client.0.smithi107.stdout:2/295: chown da/d38/f56 579629 1 2024-03-04T13:43:04.466 INFO:tasks.workunit.client.0.smithi107.stdout:6/343: write df/d17/f52 [635079,31492] 0 2024-03-04T13:43:04.468 INFO:tasks.workunit.client.1.smithi169.stdout:3/781: dread d7/f1d [0,4194304] 0 2024-03-04T13:43:04.469 INFO:tasks.workunit.client.0.smithi107.stdout:2/296: fdatasync da/f20 0 2024-03-04T13:43:04.469 INFO:tasks.workunit.client.1.smithi169.stdout:7/838: mkdir d2/de/d11d 0 2024-03-04T13:43:04.479 INFO:tasks.workunit.client.0.smithi107.stdout:8/315: dread d6/da/f11 [0,4194304] 0 2024-03-04T13:43:04.479 INFO:tasks.workunit.client.1.smithi169.stdout:2/855: write d1/d5/d19/d69/fc8 [226250,12147] 0 2024-03-04T13:43:04.481 INFO:tasks.workunit.client.0.smithi107.stdout:0/332: dwrite f4 [0,4194304] 0 2024-03-04T13:43:04.483 INFO:tasks.workunit.client.1.smithi169.stdout:9/846: creat d7/d49/df8/f130 x:0 0 0 2024-03-04T13:43:04.484 INFO:tasks.workunit.client.0.smithi107.stdout:0/333: dread - da/dc/dd/d3e/d26/f71 zero size 2024-03-04T13:43:04.484 INFO:tasks.workunit.client.0.smithi107.stdout:2/297: rmdir da/d29 39 2024-03-04T13:43:04.485 INFO:tasks.workunit.client.1.smithi169.stdout:6/782: dread d0/f16 [0,4194304] 0 2024-03-04T13:43:04.486 INFO:tasks.workunit.client.0.smithi107.stdout:0/334: dread - da/dc/d34/f5a zero size 2024-03-04T13:43:04.497 INFO:tasks.workunit.client.1.smithi169.stdout:4/775: dwrite d4/d11/d2e/d39/f60 [4194304,4194304] 0 2024-03-04T13:43:04.498 INFO:tasks.workunit.client.1.smithi169.stdout:7/839: write d2/de/d3e/d41/d5b/f9f [1030828,34496] 0 2024-03-04T13:43:04.499 INFO:tasks.workunit.client.0.smithi107.stdout:0/335: mknod da/dc/dd/d3e/d26/c72 0 2024-03-04T13:43:04.502 INFO:tasks.workunit.client.0.smithi107.stdout:0/336: dread - da/dc/dd/d3e/f63 zero size 2024-03-04T13:43:04.502 INFO:tasks.workunit.client.1.smithi169.stdout:3/782: fsync d7/de/d2b/d7d/d85/d18/d21/d3f/f82 0 2024-03-04T13:43:04.502 INFO:tasks.workunit.client.1.smithi169.stdout:1/743: dwrite d4/d5/d2d/d4e/d71/d26/d4f/f6e [0,4194304] 0 2024-03-04T13:43:04.503 INFO:tasks.workunit.client.0.smithi107.stdout:3/271: truncate d1/de/d11/d12/d4c/f57 587589 0 2024-03-04T13:43:04.504 INFO:tasks.workunit.client.1.smithi169.stdout:4/776: fdatasync d4/d11/d3b/d67/dbb/d6e/fd4 0 2024-03-04T13:43:04.505 INFO:tasks.workunit.client.0.smithi107.stdout:2/298: dread da/df/d3e/f43 [0,4194304] 0 2024-03-04T13:43:04.508 INFO:tasks.workunit.client.1.smithi169.stdout:7/840: truncate d2/d28/d70/ded/f108 4201910 0 2024-03-04T13:43:04.509 INFO:tasks.workunit.client.0.smithi107.stdout:7/259: dread d6/d13/f18 [0,4194304] 0 2024-03-04T13:43:04.510 INFO:tasks.workunit.client.1.smithi169.stdout:0/666: mknod d5/d4c/d65/ce5 0 2024-03-04T13:43:04.513 INFO:tasks.workunit.client.1.smithi169.stdout:2/856: rmdir d1/d6d/dbc/d105 39 2024-03-04T13:43:04.516 INFO:tasks.workunit.client.1.smithi169.stdout:5/892: mkdir d1/d16/d13f 0 2024-03-04T13:43:04.517 INFO:tasks.workunit.client.0.smithi107.stdout:0/337: mkdir da/dc/dd/d3e/d6e/d73 0 2024-03-04T13:43:04.520 INFO:tasks.workunit.client.0.smithi107.stdout:2/299: creat da/d38/d3a/f60 x:0 0 0 2024-03-04T13:43:04.525 INFO:tasks.workunit.client.0.smithi107.stdout:6/344: link df/d1b/d2c/d46/l50 df/d1b/d2c/d31/l66 0 2024-03-04T13:43:04.526 INFO:tasks.workunit.client.0.smithi107.stdout:0/338: rename da/dc/dd/d3e/d47 to da/dc/dd/d3e/d6e/d73/d74 0 2024-03-04T13:43:04.529 INFO:tasks.workunit.client.0.smithi107.stdout:0/339: read - da/dc/dd/d3e/d26/f71 zero size 2024-03-04T13:43:04.531 INFO:tasks.workunit.client.0.smithi107.stdout:7/260: write d6/d13/d16/d19/d38/d2c/f4a [3472704,5653] 0 2024-03-04T13:43:04.531 INFO:tasks.workunit.client.0.smithi107.stdout:0/340: chown da/dc/d34/f5a 8385 1 2024-03-04T13:43:04.531 INFO:tasks.workunit.client.1.smithi169.stdout:7/841: mknod d2/d28/db8/dd5/dea/c11e 0 2024-03-04T13:43:04.533 INFO:tasks.workunit.client.0.smithi107.stdout:0/341: readlink da/l68 0 2024-03-04T13:43:04.533 INFO:tasks.workunit.client.0.smithi107.stdout:3/272: dwrite d1/de/d11/d12/f15 [4194304,4194304] 0 2024-03-04T13:43:04.537 INFO:tasks.workunit.client.0.smithi107.stdout:6/345: creat df/d17/d5c/f67 x:0 0 0 2024-03-04T13:43:04.540 INFO:tasks.workunit.client.1.smithi169.stdout:0/667: rename d5/d9/dc/d12/d15/d19/d61/d71/d8c/fc9 to d5/d9/d3d/d5d/de4/d98/dc1/fe6 0 2024-03-04T13:43:04.542 INFO:tasks.workunit.client.0.smithi107.stdout:0/342: fsync da/dc/dd/d3e/f63 0 2024-03-04T13:43:04.543 INFO:tasks.workunit.client.1.smithi169.stdout:8/797: rmdir d7/d10/d22/d46/da5/da6/ddb 0 2024-03-04T13:43:04.547 INFO:tasks.workunit.client.0.smithi107.stdout:8/316: dread d6/da/f19 [4194304,4194304] 0 2024-03-04T13:43:04.549 INFO:tasks.workunit.client.0.smithi107.stdout:8/317: write d6/d16/d1f/f23 [1514161,44654] 0 2024-03-04T13:43:04.551 INFO:tasks.workunit.client.0.smithi107.stdout:3/273: creat d1/de/d18/d33/d4d/f5b x:0 0 0 2024-03-04T13:43:04.553 INFO:tasks.workunit.client.0.smithi107.stdout:3/274: fdatasync d1/de/d18/d1f/f20 0 2024-03-04T13:43:04.553 INFO:tasks.workunit.client.0.smithi107.stdout:7/261: creat d6/d13/d16/d19/d39/f54 x:0 0 0 2024-03-04T13:43:04.559 INFO:tasks.workunit.client.0.smithi107.stdout:0/343: dwrite da/dc/dd/d3e/d26/f64 [0,4194304] 0 2024-03-04T13:43:04.564 INFO:tasks.workunit.client.1.smithi169.stdout:2/857: unlink d1/c2c 0 2024-03-04T13:43:04.568 INFO:tasks.workunit.client.1.smithi169.stdout:5/893: unlink d1/c21 0 2024-03-04T13:43:04.580 INFO:tasks.workunit.client.1.smithi169.stdout:9/847: link d7/dc/d3e/lfd d7/d25/d3c/l131 0 2024-03-04T13:43:04.581 INFO:tasks.workunit.client.1.smithi169.stdout:1/744: write d4/d5/d36/d35/d16/d81/f96 [1755040,48767] 0 2024-03-04T13:43:04.583 INFO:tasks.workunit.client.1.smithi169.stdout:8/798: creat d7/d10/d22/d75/d81/da2/f101 x:0 0 0 2024-03-04T13:43:04.584 INFO:tasks.workunit.client.1.smithi169.stdout:8/799: readlink d7/d10/l39 0 2024-03-04T13:43:04.585 INFO:tasks.workunit.client.0.smithi107.stdout:0/344: mknod da/dc/dd/d3e/d26/d69/c75 0 2024-03-04T13:43:04.587 INFO:tasks.workunit.client.0.smithi107.stdout:2/300: dwrite da/f18 [4194304,4194304] 0 2024-03-04T13:43:04.588 INFO:tasks.workunit.client.1.smithi169.stdout:3/783: rmdir d7/de/d2b/d7d/d85/d13/d1a/d2d/dbe/de2 0 2024-03-04T13:43:04.592 INFO:tasks.workunit.client.0.smithi107.stdout:7/262: dread d6/d13/d16/d19/d39/f40 [0,4194304] 0 2024-03-04T13:43:04.593 INFO:tasks.workunit.client.0.smithi107.stdout:7/263: write d6/fa [1352499,114284] 0 2024-03-04T13:43:04.596 INFO:tasks.workunit.client.1.smithi169.stdout:9/848: rmdir d7/dc/d12 39 2024-03-04T13:43:04.599 INFO:tasks.workunit.client.1.smithi169.stdout:7/842: dread d2/d15/d17/f5d [0,4194304] 0 2024-03-04T13:43:04.605 INFO:tasks.workunit.client.1.smithi169.stdout:6/783: getdents d0/d3c/d10b 0 2024-03-04T13:43:04.607 INFO:tasks.workunit.client.1.smithi169.stdout:4/777: dwrite d4/d1c/d75/d81/f85 [0,4194304] 0 2024-03-04T13:43:04.609 INFO:tasks.workunit.client.1.smithi169.stdout:6/784: stat d0/d3c/d40/d52/d98/cf4 0 2024-03-04T13:43:04.614 INFO:tasks.workunit.client.1.smithi169.stdout:8/800: fsync d7/d10/d24/d3e/f40 0 2024-03-04T13:43:04.614 INFO:tasks.workunit.client.1.smithi169.stdout:1/745: link d4/d5/d36/d35/d16/d3a/f9c d4/d5/d36/d35/d16/d81/ff3 0 2024-03-04T13:43:04.617 INFO:tasks.workunit.client.1.smithi169.stdout:0/668: dwrite d5/f37 [0,4194304] 0 2024-03-04T13:43:04.620 INFO:tasks.workunit.client.1.smithi169.stdout:2/858: rmdir d1/d5/d19/d68/d117 0 2024-03-04T13:43:04.620 INFO:tasks.workunit.client.1.smithi169.stdout:6/785: stat d0/d32/d3a/d5e/ffc 0 2024-03-04T13:43:04.622 INFO:tasks.workunit.client.1.smithi169.stdout:2/859: write d1/d5/d19/f47 [8313488,33789] 0 2024-03-04T13:43:04.625 INFO:tasks.workunit.client.1.smithi169.stdout:7/843: rmdir d2/d15/d17/d115 0 2024-03-04T13:43:04.628 INFO:tasks.workunit.client.1.smithi169.stdout:1/746: fdatasync d4/d5/d2d/d4e/d71/d26/d4f/f51 0 2024-03-04T13:43:04.633 INFO:tasks.workunit.client.1.smithi169.stdout:0/669: symlink d5/d9/dc/d12/d15/d19/le7 0 2024-03-04T13:43:04.641 INFO:tasks.workunit.client.1.smithi169.stdout:7/844: creat d2/de/d3e/d41/d5b/f11f x:0 0 0 2024-03-04T13:43:04.642 INFO:tasks.workunit.client.1.smithi169.stdout:2/860: truncate d1/d5/d22/d3b/d99/fa6 1615485 0 2024-03-04T13:43:04.643 INFO:tasks.workunit.client.1.smithi169.stdout:1/747: readlink d4/d5/d2d/d4e/d71/d26/l92 0 2024-03-04T13:43:04.644 INFO:tasks.workunit.client.1.smithi169.stdout:8/801: link d7/d10/d24/cd9 d7/d10/d22/d75/c102 0 2024-03-04T13:43:04.644 INFO:tasks.workunit.client.1.smithi169.stdout:4/778: dwrite d4/d11/d2e/d6c/d78/da5/d77/fe6 [0,4194304] 0 2024-03-04T13:43:04.648 INFO:tasks.workunit.client.1.smithi169.stdout:3/784: truncate d7/de/d24/f8f 320481 0 2024-03-04T13:43:04.651 INFO:tasks.workunit.client.1.smithi169.stdout:2/861: chown d1/d5/d19/d63/fbd 56668302 1 2024-03-04T13:43:04.652 INFO:tasks.workunit.client.1.smithi169.stdout:9/849: dwrite d7/dc/d12/d20/da6/fdf [0,4194304] 0 2024-03-04T13:43:04.655 INFO:tasks.workunit.client.1.smithi169.stdout:1/748: write d4/d5/d36/d35/d5b/fe9 [894351,97591] 0 2024-03-04T13:43:04.655 INFO:tasks.workunit.client.1.smithi169.stdout:6/786: write d0/f9 [5266744,58385] 0 2024-03-04T13:43:04.655 INFO:tasks.workunit.client.1.smithi169.stdout:8/802: mknod d7/d10/d22/d75/d81/da4/c103 0 2024-03-04T13:43:04.658 INFO:tasks.workunit.client.1.smithi169.stdout:0/670: rename d5/d4c/f51 to d5/d9/dc/d12/d15/d19/fe8 0 2024-03-04T13:43:04.662 INFO:tasks.workunit.client.1.smithi169.stdout:2/862: unlink d1/l79 0 2024-03-04T13:43:04.667 INFO:tasks.workunit.client.1.smithi169.stdout:6/787: write d0/d3c/d40/d52/dac/fc7 [161155,127150] 0 2024-03-04T13:43:04.668 INFO:tasks.workunit.client.0.smithi107.stdout:6/346: getdents df 0 2024-03-04T13:43:04.668 INFO:tasks.workunit.client.0.smithi107.stdout:2/301: fsync da/f10 0 2024-03-04T13:43:04.672 INFO:tasks.workunit.client.0.smithi107.stdout:2/302: write da/f30 [107333,74631] 0 2024-03-04T13:43:04.673 INFO:tasks.workunit.client.1.smithi169.stdout:9/850: rename d7/dc/d12/d20/d4e/cc4 to d7/dc/d3e/c132 0 2024-03-04T13:43:04.673 INFO:tasks.workunit.client.0.smithi107.stdout:7/264: creat d6/d17/f55 x:0 0 0 2024-03-04T13:43:04.673 INFO:tasks.workunit.client.1.smithi169.stdout:8/803: truncate d7/d10/d24/d3e/f68 4349827 0 2024-03-04T13:43:04.676 INFO:tasks.workunit.client.1.smithi169.stdout:2/863: chown d1/d5/d22/d3b/f52 792358 1 2024-03-04T13:43:04.676 INFO:tasks.workunit.client.0.smithi107.stdout:0/345: mknod da/c76 0 2024-03-04T13:43:04.677 INFO:tasks.workunit.client.0.smithi107.stdout:3/275: getdents d1/de/d18/d33/d4d 0 2024-03-04T13:43:04.678 INFO:tasks.workunit.client.1.smithi169.stdout:0/671: mknod d5/d9/dc/d12/d15/d19/d61/d71/d8c/d90/ce9 0 2024-03-04T13:43:04.679 INFO:tasks.workunit.client.0.smithi107.stdout:6/347: creat df/d1b/d2c/d46/f68 x:0 0 0 2024-03-04T13:43:04.679 INFO:tasks.workunit.client.1.smithi169.stdout:1/749: link d4/d1d/l55 d4/d5/d36/df1/lf4 0 2024-03-04T13:43:04.682 INFO:tasks.workunit.client.0.smithi107.stdout:8/318: dwrite d6/fd [0,4194304] 0 2024-03-04T13:43:04.684 INFO:tasks.workunit.client.0.smithi107.stdout:2/303: mknod da/df/d27/d58/c61 0 2024-03-04T13:43:04.686 INFO:tasks.workunit.client.0.smithi107.stdout:6/348: creat df/d57/f69 x:0 0 0 2024-03-04T13:43:04.686 INFO:tasks.workunit.client.1.smithi169.stdout:6/788: creat d0/d3c/d40/df6/f11e x:0 0 0 2024-03-04T13:43:04.686 INFO:tasks.workunit.client.1.smithi169.stdout:4/779: dwrite d4/d11/d3b/d67/dbb/d6e/fc5 [0,4194304] 0 2024-03-04T13:43:04.687 INFO:tasks.workunit.client.0.smithi107.stdout:6/349: dread - df/d57/f64 zero size 2024-03-04T13:43:04.688 INFO:tasks.workunit.client.1.smithi169.stdout:0/672: stat d5/d9/dc/d12/d15/d19/d61/d71/d8c/d90/f97 0 2024-03-04T13:43:04.690 INFO:tasks.workunit.client.1.smithi169.stdout:1/750: chown d4/d5/d2d/d4e/d71/lef 2 1 2024-03-04T13:43:04.691 INFO:tasks.workunit.client.1.smithi169.stdout:6/789: stat d0/d32/d3a/d5e/fed 0 2024-03-04T13:43:04.694 INFO:tasks.workunit.client.0.smithi107.stdout:8/319: rename d6/da/d25/d2d to d6/d16/d59 0 2024-03-04T13:43:04.696 INFO:tasks.workunit.client.1.smithi169.stdout:0/673: fsync d5/d9/dc/d12/d15/d19/d61/f23 0 2024-03-04T13:43:04.699 INFO:tasks.workunit.client.0.smithi107.stdout:0/346: dwrite da/dc/dd/d3e/d26/f71 [0,4194304] 0 2024-03-04T13:43:04.699 INFO:tasks.workunit.client.1.smithi169.stdout:2/864: dread d1/d5/f9 [0,4194304] 0 2024-03-04T13:43:04.700 INFO:tasks.workunit.client.0.smithi107.stdout:2/304: rename da/fb to da/df/d33/d53/f62 0 2024-03-04T13:43:04.700 INFO:tasks.workunit.client.1.smithi169.stdout:0/674: write d5/d9/dc/d12/d15/d19/d77/f88 [1030793,35838] 0 2024-03-04T13:43:04.702 INFO:tasks.workunit.client.0.smithi107.stdout:0/347: dread - da/dc/dd/d3e/d26/f67 zero size 2024-03-04T13:43:04.703 INFO:tasks.workunit.client.0.smithi107.stdout:8/320: unlink d6/d16/c41 0 2024-03-04T13:43:04.705 INFO:tasks.workunit.client.1.smithi169.stdout:1/751: mkdir d4/d5/d2d/d4e/d71/d26/d7e/df5 0 2024-03-04T13:43:04.710 INFO:tasks.workunit.client.1.smithi169.stdout:7/845: write d2/f95 [911891,52234] 0 2024-03-04T13:43:04.712 INFO:tasks.workunit.client.1.smithi169.stdout:1/752: chown d4/d5/d2d/d4e/d71/d26/d4f/ce3 3419300 1 2024-03-04T13:43:04.714 INFO:tasks.workunit.client.0.smithi107.stdout:7/265: fsync d6/d17/f55 0 2024-03-04T13:43:04.718 INFO:tasks.workunit.client.1.smithi169.stdout:0/675: mknod d5/d9/d3d/cea 0 2024-03-04T13:43:04.720 INFO:tasks.workunit.client.0.smithi107.stdout:8/321: rmdir d6/da/d25 39 2024-03-04T13:43:04.723 INFO:tasks.workunit.client.1.smithi169.stdout:4/780: dwrite d4/d11/d3b/d67/f84 [0,4194304] 0 2024-03-04T13:43:04.724 INFO:tasks.workunit.client.1.smithi169.stdout:2/865: mknod d1/d5/d22/d3b/d41/c12b 0 2024-03-04T13:43:04.727 INFO:tasks.workunit.client.0.smithi107.stdout:3/276: dread d1/fc [0,4194304] 0 2024-03-04T13:43:04.728 INFO:tasks.workunit.client.0.smithi107.stdout:6/350: mkdir df/d17/d24/d30/d5d/d6a 0 2024-03-04T13:43:04.731 INFO:tasks.workunit.client.0.smithi107.stdout:1/282: sync 2024-03-04T13:43:04.733 INFO:tasks.workunit.client.1.smithi169.stdout:3/785: dwrite d7/de/d2b/d7d/d85/d18/d21/d3f/f82 [0,4194304] 0 2024-03-04T13:43:04.737 INFO:tasks.workunit.client.0.smithi107.stdout:7/266: rmdir d6/d1d 39 2024-03-04T13:43:04.740 INFO:tasks.workunit.client.1.smithi169.stdout:9/851: dread d7/dc/d12/d59/f12a [4194304,4194304] 0 2024-03-04T13:43:04.741 INFO:tasks.workunit.client.1.smithi169.stdout:6/790: rename d0/d32/d3a/d43/d74/d93/d10c/ff9 to d0/d3c/d40/d52/dac/f11f 0 2024-03-04T13:43:04.745 INFO:tasks.workunit.client.1.smithi169.stdout:1/753: dread d4/d1d/f40 [0,4194304] 0 2024-03-04T13:43:04.745 INFO:tasks.workunit.client.0.smithi107.stdout:6/351: dwrite df/f43 [0,4194304] 0 2024-03-04T13:43:04.747 INFO:tasks.workunit.client.1.smithi169.stdout:8/804: dwrite d7/d10/fcc [0,4194304] 0 2024-03-04T13:43:04.747 INFO:tasks.workunit.client.0.smithi107.stdout:8/322: chown d6/da/d25/d2a 3 1 2024-03-04T13:43:04.749 INFO:tasks.workunit.client.0.smithi107.stdout:3/277: creat d1/d2d/f5c x:0 0 0 2024-03-04T13:43:04.755 INFO:tasks.workunit.client.0.smithi107.stdout:0/348: creat da/dc/dd/d3e/f77 x:0 0 0 2024-03-04T13:43:04.759 INFO:tasks.workunit.client.0.smithi107.stdout:8/323: creat d6/d16/d59/d3e/f5a x:0 0 0 2024-03-04T13:43:04.760 INFO:tasks.workunit.client.1.smithi169.stdout:6/791: symlink d0/d3c/d95/l120 0 2024-03-04T13:43:04.761 INFO:tasks.workunit.client.0.smithi107.stdout:3/278: creat d1/de/d18/d1f/d42/f5d x:0 0 0 2024-03-04T13:43:04.762 INFO:tasks.workunit.client.1.smithi169.stdout:9/852: symlink d7/dc/df1/d56/l133 0 2024-03-04T13:43:04.763 INFO:tasks.workunit.client.1.smithi169.stdout:6/792: chown d0/d3c/d40/d52/d98/cf4 33038 1 2024-03-04T13:43:04.766 INFO:tasks.workunit.client.1.smithi169.stdout:6/793: read - d0/d3c/d95/d9e/ffb zero size 2024-03-04T13:43:04.767 INFO:tasks.workunit.client.1.smithi169.stdout:1/754: mkdir d4/d5/d36/d75/dba/df6 0 2024-03-04T13:43:04.769 INFO:tasks.workunit.client.0.smithi107.stdout:7/267: mknod d6/d17/d52/c56 0 2024-03-04T13:43:04.769 INFO:tasks.workunit.client.1.smithi169.stdout:6/794: fdatasync d0/d3c/d40/d52/dac/fc7 0 2024-03-04T13:43:04.770 INFO:tasks.workunit.client.0.smithi107.stdout:6/352: dwrite df/d2e/d36/f49 [0,4194304] 0 2024-03-04T13:43:04.771 INFO:tasks.workunit.client.0.smithi107.stdout:8/324: rmdir d6/da 39 2024-03-04T13:43:04.772 INFO:tasks.workunit.client.1.smithi169.stdout:6/795: stat d0/d32/d3a/d43/d74/d93/dee 0 2024-03-04T13:43:04.773 INFO:tasks.workunit.client.0.smithi107.stdout:2/305: getdents da/d39 0 2024-03-04T13:43:04.775 INFO:tasks.workunit.client.1.smithi169.stdout:3/786: mkdir d7/de/d2b/d7d/d85/d18/d21/d103 0 2024-03-04T13:43:04.777 INFO:tasks.workunit.client.1.smithi169.stdout:9/853: dread - d7/dc/d12/d20/da6/faf zero size 2024-03-04T13:43:04.782 INFO:tasks.workunit.client.1.smithi169.stdout:5/894: sync 2024-03-04T13:43:04.785 INFO:tasks.workunit.client.0.smithi107.stdout:1/283: truncate d1/d3/df/f3a 367722 0 2024-03-04T13:43:04.787 INFO:tasks.workunit.client.1.smithi169.stdout:8/805: fdatasync d7/d10/d17/d44/d4c/fa3 0 2024-03-04T13:43:04.789 INFO:tasks.workunit.client.1.smithi169.stdout:2/866: write d1/d5/d19/d1e/f24 [2439215,4406] 0 2024-03-04T13:43:04.790 INFO:tasks.workunit.client.1.smithi169.stdout:1/755: mkdir d4/d5/d2d/d4e/d71/d82/df7 0 2024-03-04T13:43:04.793 INFO:tasks.workunit.client.1.smithi169.stdout:7/846: rename d2/de/d3e/d41/fc3 to d2/f120 0 2024-03-04T13:43:04.796 INFO:tasks.workunit.client.1.smithi169.stdout:0/676: link d5/d9/dc/d12/d15/d19/d77/d94/cd9 d5/d9/dc/d12/d15/d19/d61/d71/d8c/d90/ceb 0 2024-03-04T13:43:04.797 INFO:tasks.workunit.client.1.smithi169.stdout:4/781: dwrite d4/d11/d3b/f83 [0,4194304] 0 2024-03-04T13:43:04.803 INFO:tasks.workunit.client.0.smithi107.stdout:6/353: mkdir df/d17/d6b 0 2024-03-04T13:43:04.804 INFO:tasks.workunit.client.1.smithi169.stdout:9/854: symlink d7/dc/d12/d8a/l134 0 2024-03-04T13:43:04.808 INFO:tasks.workunit.client.1.smithi169.stdout:1/756: dread d4/d5/d36/d75/dba/fbb [0,4194304] 0 2024-03-04T13:43:04.811 INFO:tasks.workunit.client.0.smithi107.stdout:1/284: read d1/d10/d16/d1b/f3e [150794,103933] 0 2024-03-04T13:43:04.813 INFO:tasks.workunit.client.0.smithi107.stdout:3/279: rename d1/de/f1d to d1/d1e/f5e 0 2024-03-04T13:43:04.813 INFO:tasks.workunit.client.0.smithi107.stdout:0/349: getdents da/dc 0 2024-03-04T13:43:04.817 INFO:tasks.workunit.client.0.smithi107.stdout:3/280: chown d1/de/d18/d1f/f43 27 1 2024-03-04T13:43:04.817 INFO:tasks.workunit.client.0.smithi107.stdout:8/325: creat d6/f5b x:0 0 0 2024-03-04T13:43:04.818 INFO:tasks.workunit.client.0.smithi107.stdout:2/306: link da/d39/c55 da/df/d27/c63 0 2024-03-04T13:43:04.820 INFO:tasks.workunit.client.0.smithi107.stdout:8/326: fdatasync d6/f57 0 2024-03-04T13:43:04.820 INFO:tasks.workunit.client.1.smithi169.stdout:2/867: rmdir d1/d5/d19/d3c/db8/de1/d64/dce/dd5/df5/d120 39 2024-03-04T13:43:04.821 INFO:tasks.workunit.client.1.smithi169.stdout:3/787: write d7/de/d2b/d7d/d85/d13/d1a/d2d/d56/d5e/d67/d5c/d84/d95/f97 [1041565,12166] 0 2024-03-04T13:43:04.821 INFO:tasks.workunit.client.0.smithi107.stdout:9/271: sync 2024-03-04T13:43:04.821 INFO:tasks.workunit.client.0.smithi107.stdout:5/244: sync 2024-03-04T13:43:04.821 INFO:tasks.workunit.client.0.smithi107.stdout:4/267: sync 2024-03-04T13:43:04.824 INFO:tasks.workunit.client.0.smithi107.stdout:1/285: mkdir d1/d3/df/d33/d43/d5b 0 2024-03-04T13:43:04.825 INFO:tasks.workunit.client.0.smithi107.stdout:3/281: truncate d1/de/d11/f45 750169 0 2024-03-04T13:43:04.826 INFO:tasks.workunit.client.0.smithi107.stdout:6/354: dwrite df/d1b/f1d [4194304,4194304] 0 2024-03-04T13:43:04.826 INFO:tasks.workunit.client.1.smithi169.stdout:0/677: creat d5/d9/d3d/d5d/fec x:0 0 0 2024-03-04T13:43:04.827 INFO:tasks.workunit.client.0.smithi107.stdout:7/268: dwrite d6/f45 [0,4194304] 0 2024-03-04T13:43:04.827 INFO:tasks.workunit.client.1.smithi169.stdout:3/788: chown d7/de/d2b/d7d/d85/d13/d1a/l1c 1111 1 2024-03-04T13:43:04.828 INFO:tasks.workunit.client.1.smithi169.stdout:4/782: creat d4/d11/d2e/d37/d73/fea x:0 0 0 2024-03-04T13:43:04.829 INFO:tasks.workunit.client.1.smithi169.stdout:6/796: dwrite d0/d3c/d40/f7d [0,4194304] 0 2024-03-04T13:43:04.829 INFO:tasks.workunit.client.0.smithi107.stdout:2/307: creat da/d44/f64 x:0 0 0 2024-03-04T13:43:04.832 INFO:tasks.workunit.client.0.smithi107.stdout:8/327: creat d6/d16/d1f/f5c x:0 0 0 2024-03-04T13:43:04.833 INFO:tasks.workunit.client.1.smithi169.stdout:4/783: chown d4/d1c/lc8 40597 1 2024-03-04T13:43:04.834 INFO:tasks.workunit.client.1.smithi169.stdout:0/678: write d5/d9/dc/fdf [620207,10466] 0 2024-03-04T13:43:04.834 INFO:tasks.workunit.client.1.smithi169.stdout:1/757: dwrite d4/d5/d36/d35/fa2 [0,4194304] 0 2024-03-04T13:43:04.838 INFO:tasks.workunit.client.1.smithi169.stdout:6/797: dread - d0/d3c/d40/df6/dca/f117 zero size 2024-03-04T13:43:04.839 INFO:tasks.workunit.client.1.smithi169.stdout:3/789: stat d7/de/d2b/d7d/d85/d18/d21/d3f/d86/lef 0 2024-03-04T13:43:04.844 INFO:tasks.workunit.client.0.smithi107.stdout:4/268: creat d0/d9/d18/d29/f55 x:0 0 0 2024-03-04T13:43:04.845 INFO:tasks.workunit.client.1.smithi169.stdout:0/679: write d5/d9/dc/d12/d15/d19/d77/fcf [1047581,15019] 0 2024-03-04T13:43:04.845 INFO:tasks.workunit.client.0.smithi107.stdout:7/269: mknod d6/d13/d14/c57 0 2024-03-04T13:43:04.847 INFO:tasks.workunit.client.1.smithi169.stdout:3/790: write d7/de/d2b/d7d/d85/d18/d21/fe4 [881460,124674] 0 2024-03-04T13:43:04.848 INFO:tasks.workunit.client.1.smithi169.stdout:3/791: stat d7/f49 0 2024-03-04T13:43:04.850 INFO:tasks.workunit.client.0.smithi107.stdout:5/245: truncate d5/f49 106616 0 2024-03-04T13:43:04.852 INFO:tasks.workunit.client.0.smithi107.stdout:7/270: chown d6/d13/d16/d19/d38/d2c/l41 14349777 1 2024-03-04T13:43:04.853 INFO:tasks.workunit.client.1.smithi169.stdout:2/868: symlink d1/d5/d22/d45/d66/l12c 0 2024-03-04T13:43:04.856 INFO:tasks.workunit.client.0.smithi107.stdout:5/246: dread d5/dc/f3b [0,4194304] 0 2024-03-04T13:43:04.857 INFO:tasks.workunit.client.0.smithi107.stdout:6/355: dwrite df/d2e/d36/f49 [4194304,4194304] 0 2024-03-04T13:43:04.857 INFO:tasks.workunit.client.0.smithi107.stdout:9/272: link d4/da/d11/d4c/f3c d4/f53 0 2024-03-04T13:43:04.861 INFO:tasks.workunit.client.0.smithi107.stdout:4/269: dread d0/d9/de/f3b [0,4194304] 0 2024-03-04T13:43:04.862 INFO:tasks.workunit.client.1.smithi169.stdout:5/895: write d1/d16/d17/d2d/d51/fbf [125614,5938] 0 2024-03-04T13:43:04.863 INFO:tasks.workunit.client.1.smithi169.stdout:1/758: read d4/d5/d36/d75/dba/fc9 [72794,6081] 0 2024-03-04T13:43:04.864 INFO:tasks.workunit.client.0.smithi107.stdout:5/247: dread - d5/dc/df/f37 zero size 2024-03-04T13:43:04.865 INFO:tasks.workunit.client.1.smithi169.stdout:3/792: dread d7/de/d2b/d7d/d85/d18/d21/f9f [0,4194304] 0 2024-03-04T13:43:04.867 INFO:tasks.workunit.client.1.smithi169.stdout:0/680: symlink d5/d9/d3d/led 0 2024-03-04T13:43:04.868 INFO:tasks.workunit.client.0.smithi107.stdout:6/356: unlink df/d1b/f37 0 2024-03-04T13:43:04.870 INFO:tasks.workunit.client.0.smithi107.stdout:4/270: chown d0/d9/d1e/l4f 104492420 1 2024-03-04T13:43:04.870 INFO:tasks.workunit.client.0.smithi107.stdout:8/328: dwrite d6/f5b [0,4194304] 0 2024-03-04T13:43:04.871 INFO:tasks.workunit.client.0.smithi107.stdout:0/350: dread da/dc/f28 [0,4194304] 0 2024-03-04T13:43:04.872 INFO:tasks.workunit.client.1.smithi169.stdout:0/681: chown d5/d4c/d65/f8e 0 1 2024-03-04T13:43:04.874 INFO:tasks.workunit.client.1.smithi169.stdout:7/847: rename d2/d28/db8/dd5/dea/f112 to d2/de/d3e/d41/d5b/d7a/f121 0 2024-03-04T13:43:04.874 INFO:tasks.workunit.client.0.smithi107.stdout:0/351: chown da/dc/dd/d3e/d26/d69 25094 1 2024-03-04T13:43:04.875 INFO:tasks.workunit.client.0.smithi107.stdout:2/308: dwrite da/df/d33/d53/f62 [0,4194304] 0 2024-03-04T13:43:04.877 INFO:tasks.workunit.client.0.smithi107.stdout:7/271: dread d6/d13/d16/d1c/f35 [0,4194304] 0 2024-03-04T13:43:04.877 INFO:tasks.workunit.client.0.smithi107.stdout:2/309: chown da/df/c17 7 1 2024-03-04T13:43:04.878 INFO:tasks.workunit.client.0.smithi107.stdout:5/248: mkdir d5/d8/d2a/d43/d5f 0 2024-03-04T13:43:04.880 INFO:tasks.workunit.client.1.smithi169.stdout:8/806: getdents d7/d10/d17/d44/d4c 0 2024-03-04T13:43:04.880 INFO:tasks.workunit.client.0.smithi107.stdout:5/249: dread - d5/d8/d2a/d43/f57 zero size 2024-03-04T13:43:04.885 INFO:tasks.workunit.client.1.smithi169.stdout:1/759: unlink d4/d1d/fe7 0 2024-03-04T13:43:04.887 INFO:tasks.workunit.client.0.smithi107.stdout:9/273: fsync d4/f53 0 2024-03-04T13:43:04.889 INFO:tasks.workunit.client.1.smithi169.stdout:6/798: dread d0/d3c/d40/d52/d98/fb6 [0,4194304] 0 2024-03-04T13:43:04.890 INFO:tasks.workunit.client.0.smithi107.stdout:8/329: mknod d6/d16/d59/d3e/c5d 0 2024-03-04T13:43:04.892 INFO:tasks.workunit.client.0.smithi107.stdout:0/352: mkdir da/dc/dd/d3e/d6e/d73/d78 0 2024-03-04T13:43:04.892 INFO:tasks.workunit.client.1.smithi169.stdout:2/869: dwrite d1/d6d/dbc/f119 [0,4194304] 0 2024-03-04T13:43:04.894 INFO:tasks.workunit.client.1.smithi169.stdout:5/896: dwrite d1/d16/d17/d28/f41 [0,4194304] 0 2024-03-04T13:43:04.900 INFO:tasks.workunit.client.0.smithi107.stdout:9/274: dwrite d4/da/f1b [0,4194304] 0 2024-03-04T13:43:04.903 INFO:tasks.workunit.client.1.smithi169.stdout:7/848: mknod d2/d15/dad/c122 0 2024-03-04T13:43:04.903 INFO:tasks.workunit.client.1.smithi169.stdout:9/855: getdents d7/dc/d12/d32 0 2024-03-04T13:43:04.903 INFO:tasks.workunit.client.1.smithi169.stdout:4/784: truncate d4/d11/d2e/d37/f57 1266445 0 2024-03-04T13:43:04.907 INFO:tasks.workunit.client.1.smithi169.stdout:3/793: read d7/d53/f8c [2368101,37906] 0 2024-03-04T13:43:04.911 INFO:tasks.workunit.client.0.smithi107.stdout:6/357: creat df/d2e/d36/d5b/f6c x:0 0 0 2024-03-04T13:43:04.913 INFO:tasks.workunit.client.0.smithi107.stdout:6/358: chown df/d17/d24/d40/c4d 244717014 1 2024-03-04T13:43:04.913 INFO:tasks.workunit.client.1.smithi169.stdout:6/799: chown d0/d3c/d40/f6f 7308 1 2024-03-04T13:43:04.918 INFO:tasks.workunit.client.0.smithi107.stdout:0/353: symlink da/dc/d34/l79 0 2024-03-04T13:43:04.923 INFO:tasks.workunit.client.0.smithi107.stdout:9/275: rmdir d4/da/d11/d20/d23 39 2024-03-04T13:43:04.926 INFO:tasks.workunit.client.0.smithi107.stdout:7/272: link d6/f3f d6/d13/d16/f58 0 2024-03-04T13:43:04.928 INFO:tasks.workunit.client.0.smithi107.stdout:8/330: rmdir d6/d16 39 2024-03-04T13:43:04.929 INFO:tasks.workunit.client.0.smithi107.stdout:8/331: fsync d6/f39 0 2024-03-04T13:43:04.932 INFO:tasks.workunit.client.1.smithi169.stdout:0/682: truncate d5/d9/dc/d12/d15/d19/d61/f23 1275020 0 2024-03-04T13:43:04.937 INFO:tasks.workunit.client.1.smithi169.stdout:3/794: fsync d7/de/d2b/d7d/d85/f6b 0 2024-03-04T13:43:04.939 INFO:tasks.workunit.client.0.smithi107.stdout:6/359: mknod df/d17/d24/d27/c6d 0 2024-03-04T13:43:04.940 INFO:tasks.workunit.client.1.smithi169.stdout:3/795: dread - d7/de/d2b/d7d/d85/d13/d1a/d2d/dbe/f101 zero size 2024-03-04T13:43:04.941 INFO:tasks.workunit.client.1.smithi169.stdout:8/807: write d7/d10/d22/f30 [1609964,122645] 0 2024-03-04T13:43:04.944 INFO:tasks.workunit.client.1.smithi169.stdout:6/800: mkdir d0/d3c/d40/df6/dca/d121 0 2024-03-04T13:43:04.944 INFO:tasks.workunit.client.1.smithi169.stdout:8/808: write d7/d10/d24/f3c [4564395,3284] 0 2024-03-04T13:43:04.954 INFO:tasks.workunit.client.0.smithi107.stdout:2/310: dwrite da/df/f13 [0,4194304] 0 2024-03-04T13:43:04.954 INFO:tasks.workunit.client.1.smithi169.stdout:4/785: truncate d4/f69 1155182 0 2024-03-04T13:43:04.955 INFO:tasks.workunit.client.1.smithi169.stdout:7/849: write d2/de/d3e/d41/d5b/ff0 [1031768,41444] 0 2024-03-04T13:43:04.955 INFO:tasks.workunit.client.0.smithi107.stdout:4/271: getdents d0/d2f 0 2024-03-04T13:43:04.959 INFO:tasks.workunit.client.1.smithi169.stdout:0/683: rename d5/d9/dc/d12/d66/fca to d5/d9/d3d/d5d/de4/d98/dc1/dc8/fee 0 2024-03-04T13:43:04.962 INFO:tasks.workunit.client.1.smithi169.stdout:7/850: dread d2/f95 [0,4194304] 0 2024-03-04T13:43:04.962 INFO:tasks.workunit.client.0.smithi107.stdout:1/286: sync 2024-03-04T13:43:04.964 INFO:tasks.workunit.client.0.smithi107.stdout:3/282: sync 2024-03-04T13:43:04.965 INFO:tasks.workunit.client.0.smithi107.stdout:8/332: chown d6/d16/l33 132388615 1 2024-03-04T13:43:04.966 INFO:tasks.workunit.client.0.smithi107.stdout:3/283: write d1/f6 [785176,21788] 0 2024-03-04T13:43:04.969 INFO:tasks.workunit.client.1.smithi169.stdout:3/796: mkdir d7/de/d2b/d7d/d85/d13/d5a/d65/da4/d104 0 2024-03-04T13:43:04.970 INFO:tasks.workunit.client.0.smithi107.stdout:5/250: link d5/dc/l1c d5/d8/da/l60 0 2024-03-04T13:43:04.971 INFO:tasks.workunit.client.0.smithi107.stdout:3/284: write d1/de/d11/d12/f49 [215849,117823] 0 2024-03-04T13:43:04.971 INFO:tasks.workunit.client.1.smithi169.stdout:6/801: truncate d0/d3c/fbc 728672 0 2024-03-04T13:43:04.972 INFO:tasks.workunit.client.1.smithi169.stdout:9/856: dwrite d7/dc/df1/f87 [0,4194304] 0 2024-03-04T13:43:04.974 INFO:tasks.workunit.client.1.smithi169.stdout:8/809: creat d7/d10/d22/d75/db9/dbe/f104 x:0 0 0 2024-03-04T13:43:04.976 INFO:tasks.workunit.client.0.smithi107.stdout:3/285: chown d1/d2d/c40 372050629 1 2024-03-04T13:43:04.977 INFO:tasks.workunit.client.1.smithi169.stdout:8/810: readlink d7/d10/d17/l1b 0 2024-03-04T13:43:04.980 INFO:tasks.workunit.client.0.smithi107.stdout:3/286: chown d1/d29/f2a 180728 1 2024-03-04T13:43:04.980 INFO:tasks.workunit.client.0.smithi107.stdout:7/273: rename d6/d17/d52 to d6/d1d/d36/d59 0 2024-03-04T13:43:04.987 INFO:tasks.workunit.client.0.smithi107.stdout:7/274: chown d6/d13/d16/d19/d1e/f53 727 1 2024-03-04T13:43:04.987 INFO:tasks.workunit.client.0.smithi107.stdout:4/272: mknod d0/d9/de/d14/dc/d21/c56 0 2024-03-04T13:43:04.988 INFO:tasks.workunit.client.0.smithi107.stdout:7/275: write d6/fa [5917908,21229] 0 2024-03-04T13:43:04.991 INFO:tasks.workunit.client.1.smithi169.stdout:5/897: getdents d1/d16/d2f/d72/dab 0 2024-03-04T13:43:04.995 INFO:tasks.workunit.client.0.smithi107.stdout:5/251: dwrite d5/dc/df/f37 [0,4194304] 0 2024-03-04T13:43:04.996 INFO:tasks.workunit.client.1.smithi169.stdout:1/760: dwrite d4/d5/d36/d35/d8f/d93/f9e [0,4194304] 0 2024-03-04T13:43:04.998 INFO:tasks.workunit.client.0.smithi107.stdout:5/252: chown d5/d8/d2a/f53 5355 1 2024-03-04T13:43:05.003 INFO:tasks.workunit.client.0.smithi107.stdout:0/354: creat da/dc/dd/d3e/f7a x:0 0 0 2024-03-04T13:43:05.009 INFO:tasks.workunit.client.0.smithi107.stdout:3/287: creat d1/de/d18/f5f x:0 0 0 2024-03-04T13:43:05.010 INFO:tasks.workunit.client.1.smithi169.stdout:4/786: symlink d4/d1c/d75/d74/leb 0 2024-03-04T13:43:05.015 INFO:tasks.workunit.client.0.smithi107.stdout:7/276: fsync d6/d13/d16/f58 0 2024-03-04T13:43:05.018 INFO:tasks.workunit.client.0.smithi107.stdout:5/253: unlink d5/d8/l13 0 2024-03-04T13:43:05.020 INFO:tasks.workunit.client.0.smithi107.stdout:5/254: chown d5/d8/d2a 34853009 1 2024-03-04T13:43:05.023 INFO:tasks.workunit.client.0.smithi107.stdout:0/355: creat da/dc/dd/d3e/d6e/f7b x:0 0 0 2024-03-04T13:43:05.027 INFO:tasks.workunit.client.1.smithi169.stdout:8/811: dwrite d7/d10/d24/f85 [0,4194304] 0 2024-03-04T13:43:05.028 INFO:tasks.workunit.client.0.smithi107.stdout:9/276: truncate d4/da/d11/f13 1703497 0 2024-03-04T13:43:05.028 INFO:tasks.workunit.client.0.smithi107.stdout:3/288: dwrite d1/d2d/f44 [0,4194304] 0 2024-03-04T13:43:05.031 INFO:tasks.workunit.client.1.smithi169.stdout:0/684: write d5/d9/f42 [652324,44993] 0 2024-03-04T13:43:05.033 INFO:tasks.workunit.client.1.smithi169.stdout:0/685: fsync d5/d4c/d65/d95/fcd 0 2024-03-04T13:43:05.040 INFO:tasks.workunit.client.1.smithi169.stdout:6/802: write d0/d32/d3a/d5e/ffc [958443,113540] 0 2024-03-04T13:43:05.041 INFO:tasks.workunit.client.0.smithi107.stdout:1/287: write d1/d10/d16/d1b/f3e [1726068,3302] 0 2024-03-04T13:43:05.041 INFO:tasks.workunit.client.0.smithi107.stdout:6/360: getdents df/d1b/d2c/d31 0 2024-03-04T13:43:05.041 INFO:tasks.workunit.client.0.smithi107.stdout:7/277: write d6/d13/d16/d1c/f1f [1333196,16230] 0 2024-03-04T13:43:05.042 INFO:tasks.workunit.client.1.smithi169.stdout:3/797: write d7/de/d2b/d7d/d85/d13/d5a/da1/fb7 [661743,42503] 0 2024-03-04T13:43:05.048 INFO:tasks.workunit.client.0.smithi107.stdout:3/289: rename d1/de/d18/f47 to d1/d1e/d26/f60 0 2024-03-04T13:43:05.057 INFO:tasks.workunit.client.0.smithi107.stdout:2/311: dwrite da/df/f15 [4194304,4194304] 0 2024-03-04T13:43:05.067 INFO:tasks.workunit.client.0.smithi107.stdout:6/361: symlink df/d17/d24/d30/d5d/l6e 0 2024-03-04T13:43:05.067 INFO:tasks.workunit.client.0.smithi107.stdout:7/278: mknod d6/d13/d16/d19/d39/c5a 0 2024-03-04T13:43:05.070 INFO:tasks.workunit.client.0.smithi107.stdout:9/277: creat d4/da/d11/d4c/d4d/f54 x:0 0 0 2024-03-04T13:43:05.071 INFO:tasks.workunit.client.1.smithi169.stdout:8/812: dread d7/d10/d17/f31 [0,4194304] 0 2024-03-04T13:43:05.072 INFO:tasks.workunit.client.0.smithi107.stdout:3/290: mknod d1/d1e/d26/c61 0 2024-03-04T13:43:05.076 INFO:tasks.workunit.client.0.smithi107.stdout:9/278: readlink d4/d34/l49 0 2024-03-04T13:43:05.076 INFO:tasks.workunit.client.0.smithi107.stdout:8/333: getdents d6/da 0 2024-03-04T13:43:05.076 INFO:tasks.workunit.client.0.smithi107.stdout:6/362: mknod df/d57/c6f 0 2024-03-04T13:43:05.079 INFO:tasks.workunit.client.0.smithi107.stdout:7/279: creat d6/d1d/d36/f5b x:0 0 0 2024-03-04T13:43:05.079 INFO:tasks.workunit.client.0.smithi107.stdout:2/312: creat da/d49/f65 x:0 0 0 2024-03-04T13:43:05.079 INFO:tasks.workunit.client.0.smithi107.stdout:5/255: dread d5/f9 [0,4194304] 0 2024-03-04T13:43:05.081 INFO:tasks.workunit.client.0.smithi107.stdout:0/356: getdents da/dc 0 2024-03-04T13:43:05.084 INFO:tasks.workunit.client.0.smithi107.stdout:7/280: read d6/d1d/f32 [416282,69285] 0 2024-03-04T13:43:05.089 INFO:tasks.workunit.client.0.smithi107.stdout:6/363: creat df/d1b/d2c/d46/f70 x:0 0 0 2024-03-04T13:43:05.091 INFO:tasks.workunit.client.0.smithi107.stdout:2/313: symlink da/d49/l66 0 2024-03-04T13:43:05.091 INFO:tasks.workunit.client.0.smithi107.stdout:4/273: dwrite d0/d9/d10/f25 [0,4194304] 0 2024-03-04T13:43:05.094 INFO:tasks.workunit.client.0.smithi107.stdout:9/279: mknod d4/c55 0 2024-03-04T13:43:05.096 INFO:tasks.workunit.client.0.smithi107.stdout:2/314: write da/d38/d3a/f60 [318232,16669] 0 2024-03-04T13:43:05.096 INFO:tasks.workunit.client.0.smithi107.stdout:5/256: creat d5/dc/d22/f61 x:0 0 0 2024-03-04T13:43:05.101 INFO:tasks.workunit.client.1.smithi169.stdout:4/787: unlink d4/d1c/d75/d81/cdf 0 2024-03-04T13:43:05.104 INFO:tasks.workunit.client.1.smithi169.stdout:0/686: unlink d5/d9/dc/d12/f53 0 2024-03-04T13:43:05.107 INFO:tasks.workunit.client.1.smithi169.stdout:6/803: rename d0/d32/d3a/d5e to d0/d32/d3e/d122 0 2024-03-04T13:43:05.107 INFO:tasks.workunit.client.0.smithi107.stdout:8/334: rename d6/da/f4d to d6/d16/f5e 0 2024-03-04T13:43:05.109 INFO:tasks.workunit.client.1.smithi169.stdout:3/798: fsync d7/d53/f5d 0 2024-03-04T13:43:05.110 INFO:tasks.workunit.client.0.smithi107.stdout:9/280: read - d4/da/d11/d4c/d4d/f54 zero size 2024-03-04T13:43:05.111 INFO:tasks.workunit.client.0.smithi107.stdout:7/281: fdatasync d6/d13/d16/d19/d38/f47 0 2024-03-04T13:43:05.113 INFO:tasks.workunit.client.1.smithi169.stdout:5/898: creat d1/d60/f140 x:0 0 0 2024-03-04T13:43:05.113 INFO:tasks.workunit.client.0.smithi107.stdout:2/315: rmdir da/df/d33/d53 39 2024-03-04T13:43:05.115 INFO:tasks.workunit.client.1.smithi169.stdout:5/899: stat d1/d16/d17/d2d/d51/l87 0 2024-03-04T13:43:05.116 INFO:tasks.workunit.client.1.smithi169.stdout:5/900: readlink d1/d16/d2f/d72/dab/lad 0 2024-03-04T13:43:05.117 INFO:tasks.workunit.client.0.smithi107.stdout:1/288: dwrite d1/d3/f51 [0,4194304] 0 2024-03-04T13:43:05.117 INFO:tasks.workunit.client.0.smithi107.stdout:2/316: read f7 [310315,63188] 0 2024-03-04T13:43:05.118 INFO:tasks.workunit.client.1.smithi169.stdout:1/761: fdatasync d4/d1d/feb 0 2024-03-04T13:43:05.119 INFO:tasks.workunit.client.0.smithi107.stdout:6/364: rename df/d17/d24/d40/d51 to df/d2e/d36/d71 0 2024-03-04T13:43:05.120 INFO:tasks.workunit.client.0.smithi107.stdout:9/281: fsync d4/da/d11/f2b 0 2024-03-04T13:43:05.122 INFO:tasks.workunit.client.0.smithi107.stdout:6/365: write df/d17/d24/d40/f44 [1625201,102769] 0 2024-03-04T13:43:05.123 INFO:tasks.workunit.client.1.smithi169.stdout:6/804: dread d0/d3c/d40/d52/dac/fd9 [0,4194304] 0 2024-03-04T13:43:05.123 INFO:tasks.workunit.client.0.smithi107.stdout:3/291: dwrite d1/f14 [0,4194304] 0 2024-03-04T13:43:05.125 INFO:tasks.workunit.client.1.smithi169.stdout:6/805: stat d0/d32/d3a/d43/f108 0 2024-03-04T13:43:05.127 INFO:tasks.workunit.client.1.smithi169.stdout:6/806: write d0/d3c/d40/d52/f8c [1696780,48777] 0 2024-03-04T13:43:05.130 INFO:tasks.workunit.client.1.smithi169.stdout:8/813: symlink d7/d10/d22/d75/d81/da2/deb/l105 0 2024-03-04T13:43:05.130 INFO:tasks.workunit.client.0.smithi107.stdout:7/282: unlink d6/d13/d16/d19/l20 0 2024-03-04T13:43:05.130 INFO:tasks.workunit.client.0.smithi107.stdout:6/366: write f6 [2425344,125849] 0 2024-03-04T13:43:05.131 INFO:tasks.workunit.client.1.smithi169.stdout:3/799: unlink d7/de/d2b/d7d/d85/d13/d1a/f28 0 2024-03-04T13:43:05.139 INFO:tasks.workunit.client.0.smithi107.stdout:5/257: dwrite d5/f42 [0,4194304] 0 2024-03-04T13:43:05.143 INFO:tasks.workunit.client.1.smithi169.stdout:1/762: symlink d4/d5/d2d/d4e/d71/d26/d7e/lf8 0 2024-03-04T13:43:05.145 INFO:tasks.workunit.client.0.smithi107.stdout:1/289: dwrite d1/d3/d17/f5a [0,4194304] 0 2024-03-04T13:43:05.146 INFO:tasks.workunit.client.1.smithi169.stdout:7/851: truncate d2/d15/d17/f81 4308742 0 2024-03-04T13:43:05.149 INFO:tasks.workunit.client.0.smithi107.stdout:3/292: unlink d1/de/d11/l37 0 2024-03-04T13:43:05.155 INFO:tasks.workunit.client.0.smithi107.stdout:2/317: creat da/d49/d5c/f67 x:0 0 0 2024-03-04T13:43:05.156 INFO:tasks.workunit.client.1.smithi169.stdout:5/901: mkdir d1/d81/db1/d141 0 2024-03-04T13:43:05.158 INFO:tasks.workunit.client.0.smithi107.stdout:0/357: dwrite da/dc/f28 [0,4194304] 0 2024-03-04T13:43:05.160 INFO:tasks.workunit.client.1.smithi169.stdout:5/902: truncate d1/d81/d91/f11b 58848 0 2024-03-04T13:43:05.161 INFO:tasks.workunit.client.1.smithi169.stdout:9/857: sync 2024-03-04T13:43:05.161 INFO:tasks.workunit.client.1.smithi169.stdout:2/870: sync 2024-03-04T13:43:05.162 INFO:tasks.workunit.client.1.smithi169.stdout:1/763: dread d4/d5/d2d/d4e/d71/d26/d4f/f6e [0,4194304] 0 2024-03-04T13:43:05.164 INFO:tasks.workunit.client.1.smithi169.stdout:3/800: mkdir d7/de/d2b/d7d/d85/d13/d1a/d2d/d56/d5e/d67/d5c/d84/d105 0 2024-03-04T13:43:05.166 INFO:tasks.workunit.client.0.smithi107.stdout:0/358: read da/dc/d34/f38 [2283103,52319] 0 2024-03-04T13:43:05.166 INFO:tasks.workunit.client.1.smithi169.stdout:3/801: stat d7/de/d2b/d7d/d85/d47/d63/d6c/f88 0 2024-03-04T13:43:05.166 INFO:tasks.workunit.client.1.smithi169.stdout:3/802: readlink d7/ld 0 2024-03-04T13:43:05.168 INFO:tasks.workunit.client.0.smithi107.stdout:0/359: read - da/dc/dd/d3e/f63 zero size 2024-03-04T13:43:05.173 INFO:tasks.workunit.client.0.smithi107.stdout:5/258: fsync d5/dc/f3b 0 2024-03-04T13:43:05.177 INFO:tasks.workunit.client.1.smithi169.stdout:8/814: unlink d7/d10/d17/d44/d5d/ddd/lef 0 2024-03-04T13:43:05.178 INFO:tasks.workunit.client.0.smithi107.stdout:3/293: creat d1/de/d18/d1f/d42/f62 x:0 0 0 2024-03-04T13:43:05.179 INFO:tasks.workunit.client.0.smithi107.stdout:3/294: fdatasync d1/f56 0 2024-03-04T13:43:05.180 INFO:tasks.workunit.client.1.smithi169.stdout:8/815: write d7/d10/d22/d75/d81/da2/fbf [988834,3803] 0 2024-03-04T13:43:05.181 INFO:tasks.workunit.client.1.smithi169.stdout:5/903: mkdir d1/d16/d2a/d142 0 2024-03-04T13:43:05.183 INFO:tasks.workunit.client.0.smithi107.stdout:0/360: dwrite da/dc/f4e [0,4194304] 0 2024-03-04T13:43:05.184 INFO:tasks.workunit.client.0.smithi107.stdout:0/361: truncate da/f62 900267 0 2024-03-04T13:43:05.186 INFO:tasks.workunit.client.0.smithi107.stdout:0/362: chown da/dc/d34 15982 1 2024-03-04T13:43:05.188 INFO:tasks.workunit.client.0.smithi107.stdout:0/363: truncate da/f59 1538187 0 2024-03-04T13:43:05.188 INFO:tasks.workunit.client.0.smithi107.stdout:4/274: truncate d0/d9/d10/f1c 281647 0 2024-03-04T13:43:05.189 INFO:tasks.workunit.client.1.smithi169.stdout:4/788: write d4/f69 [1438601,28585] 0 2024-03-04T13:43:05.194 INFO:tasks.workunit.client.1.smithi169.stdout:7/852: mkdir d2/de/d123 0 2024-03-04T13:43:05.196 INFO:tasks.workunit.client.1.smithi169.stdout:7/853: stat d2/d114 0 2024-03-04T13:43:05.197 INFO:tasks.workunit.client.0.smithi107.stdout:8/335: dwrite d6/da/f2c [0,4194304] 0 2024-03-04T13:43:05.198 INFO:tasks.workunit.client.0.smithi107.stdout:3/295: dwrite d1/f56 [0,4194304] 0 2024-03-04T13:43:05.198 INFO:tasks.workunit.client.1.smithi169.stdout:3/803: truncate d7/de/d2b/d7d/d85/d13/d1a/d2d/d56/f81 1675712 0 2024-03-04T13:43:05.200 INFO:tasks.workunit.client.1.smithi169.stdout:3/804: chown d7/f1d 48 1 2024-03-04T13:43:05.202 INFO:tasks.workunit.client.1.smithi169.stdout:7/854: truncate d2/de/d3e/d7e/dbb/de3/f105 687067 0 2024-03-04T13:43:05.206 INFO:tasks.workunit.client.0.smithi107.stdout:5/259: mkdir d5/d8/d2a/d62 0 2024-03-04T13:43:05.210 INFO:tasks.workunit.client.1.smithi169.stdout:6/807: write d0/d32/d3a/f42 [2332224,84296] 0 2024-03-04T13:43:05.214 INFO:tasks.workunit.client.1.smithi169.stdout:5/904: symlink d1/d16/d2f/d39/l143 0 2024-03-04T13:43:05.216 INFO:tasks.workunit.client.0.smithi107.stdout:9/282: creat d4/da/d11/d20/f56 x:0 0 0 2024-03-04T13:43:05.216 INFO:tasks.workunit.client.1.smithi169.stdout:6/808: write d0/d32/d3a/d43/f108 [416605,49377] 0 2024-03-04T13:43:05.218 INFO:tasks.workunit.client.0.smithi107.stdout:6/367: truncate df/d2e/d36/f49 3248804 0 2024-03-04T13:43:05.223 INFO:tasks.workunit.client.0.smithi107.stdout:2/318: mkdir da/df/d33/d68 0 2024-03-04T13:43:05.225 INFO:tasks.workunit.client.0.smithi107.stdout:7/283: creat d6/d13/d16/d19/d1e/f5c x:0 0 0 2024-03-04T13:43:05.225 INFO:tasks.workunit.client.1.smithi169.stdout:2/871: dwrite d1/d5/d19/d1e/f5c [0,4194304] 0 2024-03-04T13:43:05.236 INFO:tasks.workunit.client.1.smithi169.stdout:8/816: mknod d7/d10/d22/d75/c106 0 2024-03-04T13:43:05.237 INFO:tasks.workunit.client.0.smithi107.stdout:1/290: truncate d1/d3/d17/f34 1735561 0 2024-03-04T13:43:05.240 INFO:tasks.workunit.client.0.smithi107.stdout:1/291: readlink d1/d3/d17/l18 0 2024-03-04T13:43:05.241 INFO:tasks.workunit.client.0.smithi107.stdout:3/296: rename d1/d2d/c3a to d1/de/d18/d33/c63 0 2024-03-04T13:43:05.242 INFO:tasks.workunit.client.1.smithi169.stdout:2/872: dread d1/f50 [0,4194304] 0 2024-03-04T13:43:05.242 INFO:tasks.workunit.client.0.smithi107.stdout:1/292: write d1/d3/df/d33/d43/f55 [286636,57227] 0 2024-03-04T13:43:05.243 INFO:tasks.workunit.client.1.smithi169.stdout:5/905: mknod d1/d16/d17/d1b/d74/ddc/de4/c144 0 2024-03-04T13:43:05.246 INFO:tasks.workunit.client.1.smithi169.stdout:0/687: dwrite d5/d9/dc/d12/d15/d19/f20 [0,4194304] 0 2024-03-04T13:43:05.246 INFO:tasks.workunit.client.1.smithi169.stdout:6/809: creat d0/d7a/d112/f123 x:0 0 0 2024-03-04T13:43:05.249 INFO:tasks.workunit.client.0.smithi107.stdout:5/260: unlink d5/dc/d22/f50 0 2024-03-04T13:43:05.253 INFO:tasks.workunit.client.1.smithi169.stdout:6/810: dread - d0/d32/d70/fd1 zero size 2024-03-04T13:43:05.254 INFO:tasks.workunit.client.0.smithi107.stdout:5/261: truncate d5/d8/d2a/d43/f4a 4624851 0 2024-03-04T13:43:05.254 INFO:tasks.workunit.client.0.smithi107.stdout:4/275: write d0/d9/de/d14/dc/f26 [1723306,98368] 0 2024-03-04T13:43:05.254 INFO:tasks.workunit.client.0.smithi107.stdout:9/283: dread d4/da/d11/d20/f30 [0,4194304] 0 2024-03-04T13:43:05.255 INFO:tasks.workunit.client.0.smithi107.stdout:6/368: mknod df/d2e/d36/c72 0 2024-03-04T13:43:05.263 INFO:tasks.workunit.client.0.smithi107.stdout:2/319: unlink da/fd 0 2024-03-04T13:43:05.263 INFO:tasks.workunit.client.0.smithi107.stdout:4/276: write d0/d30/f50 [821144,90275] 0 2024-03-04T13:43:05.266 INFO:tasks.workunit.client.0.smithi107.stdout:0/364: getdents da/dc/dd/d3e/d26/d69/d6b 0 2024-03-04T13:43:05.270 INFO:tasks.workunit.client.0.smithi107.stdout:0/365: dread - da/dc/dd/d3e/f63 zero size 2024-03-04T13:43:05.270 INFO:tasks.workunit.client.1.smithi169.stdout:0/688: dwrite d5/d9/dc/d12/d15/d19/d77/d94/fa2 [0,4194304] 0 2024-03-04T13:43:05.273 INFO:tasks.workunit.client.0.smithi107.stdout:3/297: symlink d1/de/d3f/l64 0 2024-03-04T13:43:05.274 INFO:tasks.workunit.client.1.smithi169.stdout:8/817: creat d7/d10/d22/d75/d81/f107 x:0 0 0 2024-03-04T13:43:05.276 INFO:tasks.workunit.client.1.smithi169.stdout:9/858: getdents d7/d27 0 2024-03-04T13:43:05.276 INFO:tasks.workunit.client.0.smithi107.stdout:9/284: dwrite d4/d34/f3f [0,4194304] 0 2024-03-04T13:43:05.277 INFO:tasks.workunit.client.0.smithi107.stdout:3/298: write d1/de/d18/d1f/d42/f5d [119184,73317] 0 2024-03-04T13:43:05.278 INFO:tasks.workunit.client.1.smithi169.stdout:9/859: stat d7/dc/d12/d32/f94 0 2024-03-04T13:43:05.279 INFO:tasks.workunit.client.0.smithi107.stdout:5/262: dwrite d5/d8/da/f16 [0,4194304] 0 2024-03-04T13:43:05.280 INFO:tasks.workunit.client.1.smithi169.stdout:1/764: getdents d4/d1d 0 2024-03-04T13:43:05.283 INFO:tasks.workunit.client.1.smithi169.stdout:7/855: write d2/de/d3e/d41/f4c [1732963,87644] 0 2024-03-04T13:43:05.284 INFO:tasks.workunit.client.1.smithi169.stdout:9/860: write d7/dc/d12/d32/d42/f46 [256171,12454] 0 2024-03-04T13:43:05.286 INFO:tasks.workunit.client.1.smithi169.stdout:2/873: chown d1/f40 13 1 2024-03-04T13:43:05.290 INFO:tasks.workunit.client.1.smithi169.stdout:4/789: dwrite d4/d11/d2e/d63/fc3 [0,4194304] 0 2024-03-04T13:43:05.295 INFO:tasks.workunit.client.0.smithi107.stdout:5/263: dread d5/f20 [0,4194304] 0 2024-03-04T13:43:05.296 INFO:tasks.workunit.client.1.smithi169.stdout:6/811: mkdir d0/d32/d3a/d43/d74/da6/d124 0 2024-03-04T13:43:05.297 INFO:tasks.workunit.client.0.smithi107.stdout:6/369: symlink df/d2e/d36/d5b/l73 0 2024-03-04T13:43:05.300 INFO:tasks.workunit.client.1.smithi169.stdout:3/805: creat d7/de/d2b/d7d/d85/d47/f106 x:0 0 0 2024-03-04T13:43:05.300 INFO:tasks.workunit.client.0.smithi107.stdout:6/370: write f6 [5669583,126773] 0 2024-03-04T13:43:05.304 INFO:tasks.workunit.client.0.smithi107.stdout:6/371: write fe [1500946,1622] 0 2024-03-04T13:43:05.314 INFO:tasks.workunit.client.0.smithi107.stdout:0/366: rename da/dc/dd/d3e/f5b to da/dc/f7c 0 2024-03-04T13:43:05.315 INFO:tasks.workunit.client.0.smithi107.stdout:1/293: getdents d1/d10/d20/d4c 0 2024-03-04T13:43:05.322 INFO:tasks.workunit.client.1.smithi169.stdout:9/861: read d7/d2d/f85 [209001,81423] 0 2024-03-04T13:43:05.323 INFO:tasks.workunit.client.0.smithi107.stdout:8/336: mkdir d6/da/d25/d5f 0 2024-03-04T13:43:05.324 INFO:tasks.workunit.client.1.smithi169.stdout:7/856: truncate d2/de/d37/f10b 1674337 0 2024-03-04T13:43:05.324 INFO:tasks.workunit.client.0.smithi107.stdout:8/337: chown d6/d16/d1f/c2f 40 1 2024-03-04T13:43:05.325 INFO:tasks.workunit.client.0.smithi107.stdout:3/299: mknod d1/de/d18/c65 0 2024-03-04T13:43:05.327 INFO:tasks.workunit.client.0.smithi107.stdout:1/294: dread d1/d3/f51 [0,4194304] 0 2024-03-04T13:43:05.330 INFO:tasks.workunit.client.1.smithi169.stdout:5/906: write d1/d16/d17/d28/dfe/f110 [528785,7075] 0 2024-03-04T13:43:05.331 INFO:tasks.workunit.client.1.smithi169.stdout:1/765: write d4/d5/d36/d75/dba/fc9 [725120,121600] 0 2024-03-04T13:43:05.332 INFO:tasks.workunit.client.1.smithi169.stdout:3/806: creat d7/de/d2b/d7d/d85/d13/d5a/da1/f107 x:0 0 0 2024-03-04T13:43:05.334 INFO:tasks.workunit.client.0.smithi107.stdout:2/320: creat da/d29/f69 x:0 0 0 2024-03-04T13:43:05.337 INFO:tasks.workunit.client.0.smithi107.stdout:4/277: link d0/d9/de/d14/dc/f2d d0/d9/d18/f57 0 2024-03-04T13:43:05.340 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:05 smithi107 ceph-mon[28739]: pgmap v54: 65 pgs: 65 active+clean; 2.1 GiB data, 7.7 GiB used, 529 GiB / 536 GiB avail; 69 MiB/s rd, 234 MiB/s wr, 469 op/s 2024-03-04T13:43:05.340 INFO:tasks.workunit.client.1.smithi169.stdout:8/818: dwrite d7/d10/d17/d44/d4c/fa3 [0,4194304] 0 2024-03-04T13:43:05.340 INFO:tasks.workunit.client.0.smithi107.stdout:9/285: symlink d4/da/l57 0 2024-03-04T13:43:05.342 INFO:tasks.workunit.client.1.smithi169.stdout:8/819: read - d7/d10/d17/d44/d36/fa8 zero size 2024-03-04T13:43:05.346 INFO:tasks.workunit.client.1.smithi169.stdout:4/790: dwrite d4/d11/d2e/d37/d5c/d86/db8/fe0 [0,4194304] 0 2024-03-04T13:43:05.347 INFO:tasks.workunit.client.0.smithi107.stdout:5/264: truncate d5/dc/df/f2e 1037439 0 2024-03-04T13:43:05.350 INFO:tasks.workunit.client.0.smithi107.stdout:4/278: dread d0/d9/d10/f25 [0,4194304] 0 2024-03-04T13:43:05.350 INFO:tasks.workunit.client.0.smithi107.stdout:5/265: write d5/d8/d14/d2d/f56 [892068,90028] 0 2024-03-04T13:43:05.352 INFO:tasks.workunit.client.0.smithi107.stdout:5/266: dread - d5/d8/f5b zero size 2024-03-04T13:43:05.353 INFO:tasks.workunit.client.1.smithi169.stdout:1/766: rename d4/d5/d36/d35/d16/d81/db4/cc0 to d4/d5/d36/d75/dba/cf9 0 2024-03-04T13:43:05.357 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:05 smithi169 ceph-mon[36313]: pgmap v54: 65 pgs: 65 active+clean; 2.1 GiB data, 7.7 GiB used, 529 GiB / 536 GiB avail; 69 MiB/s rd, 234 MiB/s wr, 469 op/s 2024-03-04T13:43:05.357 INFO:tasks.workunit.client.1.smithi169.stdout:1/767: chown d4/d5/d36/d35/d8f/d93/le5 1316 1 2024-03-04T13:43:05.360 INFO:tasks.workunit.client.1.smithi169.stdout:9/862: symlink d7/dc/d12/d32/d42/d10b/l135 0 2024-03-04T13:43:05.363 INFO:tasks.workunit.client.1.smithi169.stdout:0/689: truncate d5/d9/dc/d12/d15/d19/d77/d94/fab 4025692 0 2024-03-04T13:43:05.369 INFO:tasks.workunit.client.0.smithi107.stdout:0/367: write da/dc/dd/d3e/f11 [2673244,94299] 0 2024-03-04T13:43:05.371 INFO:tasks.workunit.client.1.smithi169.stdout:0/690: write d5/d4c/d65/f92 [7115461,62613] 0 2024-03-04T13:43:05.371 INFO:tasks.workunit.client.0.smithi107.stdout:8/338: unlink d6/c20 0 2024-03-04T13:43:05.373 INFO:tasks.workunit.client.1.smithi169.stdout:8/820: symlink d7/d10/d22/d75/db9/l108 0 2024-03-04T13:43:05.375 INFO:tasks.workunit.client.1.smithi169.stdout:4/791: dwrite d4/d1c/d58/d9d/fda [0,4194304] 0 2024-03-04T13:43:05.375 INFO:tasks.workunit.client.1.smithi169.stdout:6/812: creat d0/f125 x:0 0 0 2024-03-04T13:43:05.375 INFO:tasks.workunit.client.0.smithi107.stdout:3/300: mkdir d1/de/d11/d12/d4c/d54/d66 0 2024-03-04T13:43:05.376 INFO:tasks.workunit.client.0.smithi107.stdout:1/295: write d1/d3/df/f32 [2746704,39039] 0 2024-03-04T13:43:05.378 INFO:tasks.workunit.client.1.smithi169.stdout:0/691: fsync d5/d9/dc/d12/d15/d19/d77/f88 0 2024-03-04T13:43:05.381 INFO:tasks.workunit.client.0.smithi107.stdout:6/372: truncate df/d1b/f1d 6244267 0 2024-03-04T13:43:05.381 INFO:tasks.workunit.client.0.smithi107.stdout:7/284: sync 2024-03-04T13:43:05.382 INFO:tasks.workunit.client.0.smithi107.stdout:7/285: stat d6/d1d/d36/d59 0 2024-03-04T13:43:05.386 INFO:tasks.workunit.client.1.smithi169.stdout:1/768: dwrite d4/d5/d36/fce [0,4194304] 0 2024-03-04T13:43:05.386 INFO:tasks.workunit.client.1.smithi169.stdout:0/692: write d5/d9/dc/d12/d15/d19/f20 [1973063,97843] 0 2024-03-04T13:43:05.388 INFO:tasks.workunit.client.1.smithi169.stdout:0/693: readlink d5/l85 0 2024-03-04T13:43:05.389 INFO:tasks.workunit.client.1.smithi169.stdout:5/907: dread d1/d16/d17/d2d/d5a/d67/f97 [0,4194304] 0 2024-03-04T13:43:05.394 INFO:tasks.workunit.client.0.smithi107.stdout:2/321: rename da/df/d33/d53 to da/d49/d5c/d6a 0 2024-03-04T13:43:05.396 INFO:tasks.workunit.client.0.smithi107.stdout:8/339: creat d6/d16/d59/f60 x:0 0 0 2024-03-04T13:43:05.397 INFO:tasks.workunit.client.1.smithi169.stdout:7/857: write d2/d15/d17/f81 [3881113,855] 0 2024-03-04T13:43:05.397 INFO:tasks.workunit.client.0.smithi107.stdout:1/296: dwrite d1/d10/d16/d1b/f3e [0,4194304] 0 2024-03-04T13:43:05.397 INFO:tasks.workunit.client.1.smithi169.stdout:2/874: getdents d1/d5/d22/d45/d4a 0 2024-03-04T13:43:05.398 INFO:tasks.workunit.client.0.smithi107.stdout:9/286: link d4/d32/f52 d4/da/d11/d4c/d3b/f58 0 2024-03-04T13:43:05.401 INFO:tasks.workunit.client.0.smithi107.stdout:8/340: write d6/d16/d59/d3e/f56 [755592,31018] 0 2024-03-04T13:43:05.404 INFO:tasks.workunit.client.1.smithi169.stdout:9/863: symlink d7/dc/d12/d20/da6/d111/l136 0 2024-03-04T13:43:05.410 INFO:tasks.workunit.client.1.smithi169.stdout:9/864: write d7/d27/d84/d12b/f12e [573046,110859] 0 2024-03-04T13:43:05.410 INFO:tasks.workunit.client.1.smithi169.stdout:1/769: dread d4/d5/d36/d35/d5b/fe9 [0,4194304] 0 2024-03-04T13:43:05.414 INFO:tasks.workunit.client.0.smithi107.stdout:4/279: mknod d0/d9/c58 0 2024-03-04T13:43:05.416 INFO:tasks.workunit.client.0.smithi107.stdout:4/280: write d0/d9/f37 [1265727,124830] 0 2024-03-04T13:43:05.417 INFO:tasks.workunit.client.1.smithi169.stdout:4/792: mknod d4/cec 0 2024-03-04T13:43:05.418 INFO:tasks.workunit.client.0.smithi107.stdout:4/281: write d0/d9/d10/f19 [702720,46534] 0 2024-03-04T13:43:05.422 INFO:tasks.workunit.client.0.smithi107.stdout:4/282: chown d0/d9/de/d14/dc/d21/c56 7225 1 2024-03-04T13:43:05.422 INFO:tasks.workunit.client.1.smithi169.stdout:8/821: mknod d7/d10/d22/d46/d49/c109 0 2024-03-04T13:43:05.422 INFO:tasks.workunit.client.1.smithi169.stdout:0/694: read d5/d9/f2d [1012418,67276] 0 2024-03-04T13:43:05.424 INFO:tasks.workunit.client.0.smithi107.stdout:3/301: write d1/d1e/f22 [1587778,30618] 0 2024-03-04T13:43:05.429 INFO:tasks.workunit.client.0.smithi107.stdout:9/287: creat d4/d32/f59 x:0 0 0 2024-03-04T13:43:05.432 INFO:tasks.workunit.client.0.smithi107.stdout:1/297: mkdir d1/d10/d20/d24/d5c 0 2024-03-04T13:43:05.436 INFO:tasks.workunit.client.0.smithi107.stdout:8/341: creat d6/d16/d1f/d4c/f61 x:0 0 0 2024-03-04T13:43:05.442 INFO:tasks.workunit.client.1.smithi169.stdout:2/875: unlink d1/d5/d19/d1e/d29/lf6 0 2024-03-04T13:43:05.442 INFO:tasks.workunit.client.1.smithi169.stdout:1/770: dread d4/d5/d2d/d4e/fa3 [0,4194304] 0 2024-03-04T13:43:05.444 INFO:tasks.workunit.client.1.smithi169.stdout:2/876: fsync d1/d6d/f86 0 2024-03-04T13:43:05.445 INFO:tasks.workunit.client.0.smithi107.stdout:7/286: chown d6/d13/d16/d19/d38/d37/c4c 34 1 2024-03-04T13:43:05.447 INFO:tasks.workunit.client.0.smithi107.stdout:5/267: creat d5/d8/f63 x:0 0 0 2024-03-04T13:43:05.447 INFO:tasks.workunit.client.0.smithi107.stdout:7/287: write d6/d13/d16/d19/d1e/f42 [148206,96062] 0 2024-03-04T13:43:05.448 INFO:tasks.workunit.client.1.smithi169.stdout:8/822: rmdir d7/d10/d22/d75/d81/da2/deb 39 2024-03-04T13:43:05.448 INFO:tasks.workunit.client.1.smithi169.stdout:9/865: dwrite d7/f7a [0,4194304] 0 2024-03-04T13:43:05.450 INFO:tasks.workunit.client.0.smithi107.stdout:5/268: dread d5/f7 [0,4194304] 0 2024-03-04T13:43:05.450 INFO:tasks.workunit.client.1.smithi169.stdout:0/695: mkdir d5/d9/d3d/d5d/de4/def 0 2024-03-04T13:43:05.451 INFO:tasks.workunit.client.0.smithi107.stdout:2/322: mknod da/df/d27/d58/d5f/c6b 0 2024-03-04T13:43:05.454 INFO:tasks.workunit.client.0.smithi107.stdout:3/302: creat d1/de/d11/d12/f67 x:0 0 0 2024-03-04T13:43:05.455 INFO:tasks.workunit.client.1.smithi169.stdout:5/908: truncate d1/d60/f127 3435406 0 2024-03-04T13:43:05.456 INFO:tasks.workunit.client.0.smithi107.stdout:6/373: write df/d17/d24/f2b [1027027,121909] 0 2024-03-04T13:43:05.458 INFO:tasks.workunit.client.0.smithi107.stdout:8/342: rmdir d6/d16 39 2024-03-04T13:43:05.458 INFO:tasks.workunit.client.1.smithi169.stdout:4/793: mknod d4/d11/d2e/d39/ced 0 2024-03-04T13:43:05.462 INFO:tasks.workunit.client.0.smithi107.stdout:0/368: rename da/dc/dd/d3e/d26/c6a to da/c7d 0 2024-03-04T13:43:05.463 INFO:tasks.workunit.client.0.smithi107.stdout:4/283: rename d0 to d0/d9/d1e/d59 22 2024-03-04T13:43:05.463 INFO:tasks.workunit.client.0.smithi107.stdout:7/288: creat d6/d13/d14/f5d x:0 0 0 2024-03-04T13:43:05.464 INFO:tasks.workunit.client.1.smithi169.stdout:2/877: rmdir d1/d5/d19/d3c/db8/de1/d64/dce/dd5 39 2024-03-04T13:43:05.465 INFO:tasks.workunit.client.0.smithi107.stdout:0/369: fsync da/dc/dd/d3e/d26/f71 0 2024-03-04T13:43:05.465 INFO:tasks.workunit.client.0.smithi107.stdout:4/284: chown d0/d9/f37 1326 1 2024-03-04T13:43:05.466 INFO:tasks.workunit.client.0.smithi107.stdout:0/370: readlink da/l36 0 2024-03-04T13:43:05.468 INFO:tasks.workunit.client.0.smithi107.stdout:4/285: chown d0/d9/d18/d29/f55 4 1 2024-03-04T13:43:05.469 INFO:tasks.workunit.client.1.smithi169.stdout:7/858: truncate d2/de/d3e/d41/d69/f92 198998 0 2024-03-04T13:43:05.469 INFO:tasks.workunit.client.0.smithi107.stdout:0/371: fdatasync da/dc/dd/d3e/f63 0 2024-03-04T13:43:05.470 INFO:tasks.workunit.client.1.smithi169.stdout:7/859: chown d2/de/d3e/cb5 0 1 2024-03-04T13:43:05.471 INFO:tasks.workunit.client.0.smithi107.stdout:9/288: creat d4/da/d11/d4c/d51/f5a x:0 0 0 2024-03-04T13:43:05.474 INFO:tasks.workunit.client.0.smithi107.stdout:8/343: truncate d6/d16/d1f/f34 8835766 0 2024-03-04T13:43:05.474 INFO:tasks.workunit.client.1.smithi169.stdout:1/771: mkdir d4/d5/d2d/d4e/d71/d26/dfa 0 2024-03-04T13:43:05.475 INFO:tasks.workunit.client.1.smithi169.stdout:1/772: readlink d4/d5/l14 0 2024-03-04T13:43:05.476 INFO:tasks.workunit.client.0.smithi107.stdout:1/298: rename d1/d10/d20 to d1/d3/df/d33/d5d 0 2024-03-04T13:43:05.478 INFO:tasks.workunit.client.1.smithi169.stdout:7/860: read d2/de/d3e/d41/d69/fc8 [3187433,94790] 0 2024-03-04T13:43:05.484 INFO:tasks.workunit.client.0.smithi107.stdout:4/286: creat d0/d2f/f5a x:0 0 0 2024-03-04T13:43:05.484 INFO:tasks.workunit.client.1.smithi169.stdout:5/909: mknod d1/d23/c145 0 2024-03-04T13:43:05.488 INFO:tasks.workunit.client.1.smithi169.stdout:4/794: fdatasync d4/d1c/f2c 0 2024-03-04T13:43:05.490 INFO:tasks.workunit.client.0.smithi107.stdout:6/374: mkdir df/d38/d4f/d74 0 2024-03-04T13:43:05.491 INFO:tasks.workunit.client.0.smithi107.stdout:6/375: chown df/d17 5 1 2024-03-04T13:43:05.491 INFO:tasks.workunit.client.0.smithi107.stdout:3/303: dread d1/f3 [0,4194304] 0 2024-03-04T13:43:05.494 INFO:tasks.workunit.client.1.smithi169.stdout:2/878: creat d1/d5/d22/d45/d4a/de7/d10a/f12d x:0 0 0 2024-03-04T13:43:05.501 INFO:tasks.workunit.client.0.smithi107.stdout:7/289: symlink d6/l5e 0 2024-03-04T13:43:05.503 INFO:tasks.workunit.client.1.smithi169.stdout:8/823: write d7/d10/d17/f2a [1310851,115645] 0 2024-03-04T13:43:05.503 INFO:tasks.workunit.client.0.smithi107.stdout:4/287: dwrite d0/d9/d1e/f20 [0,4194304] 0 2024-03-04T13:43:05.507 INFO:tasks.workunit.client.0.smithi107.stdout:2/323: dread da/df/d3e/f50 [0,4194304] 0 2024-03-04T13:43:05.507 INFO:tasks.workunit.client.0.smithi107.stdout:4/288: chown d0/d9/d10/f4a 10724232 1 2024-03-04T13:43:05.508 INFO:tasks.workunit.client.0.smithi107.stdout:7/290: read d6/f45 [3246565,125973] 0 2024-03-04T13:43:05.508 INFO:tasks.workunit.client.0.smithi107.stdout:5/269: write d5/f49 [1552438,28061] 0 2024-03-04T13:43:05.509 INFO:tasks.workunit.client.0.smithi107.stdout:3/304: rename d1/de/f19 to d1/de/f68 0 2024-03-04T13:43:05.509 INFO:tasks.workunit.client.0.smithi107.stdout:7/291: chown d6/d13/d16/d19 1251 1 2024-03-04T13:43:05.512 INFO:tasks.workunit.client.0.smithi107.stdout:3/305: stat d1/d1e/d26/c61 0 2024-03-04T13:43:05.513 INFO:tasks.workunit.client.0.smithi107.stdout:9/289: dread d4/da/d11/f13 [0,4194304] 0 2024-03-04T13:43:05.514 INFO:tasks.workunit.client.0.smithi107.stdout:1/299: symlink d1/d3/df/d33/d43/d5b/l5e 0 2024-03-04T13:43:05.517 INFO:tasks.workunit.client.1.smithi169.stdout:4/795: rename d4/d11/d2e/d37 to d4/d11/d2e/dcd/dee 0 2024-03-04T13:43:05.520 INFO:tasks.workunit.client.0.smithi107.stdout:1/300: read d1/d3/df/d33/d5d/d24/f44 [329,96156] 0 2024-03-04T13:43:05.521 INFO:tasks.workunit.client.0.smithi107.stdout:6/376: dwrite df/d17/d24/d40/f44 [0,4194304] 0 2024-03-04T13:43:05.522 INFO:tasks.workunit.client.1.smithi169.stdout:1/773: dread d4/d5/d36/f88 [0,4194304] 0 2024-03-04T13:43:05.522 INFO:tasks.workunit.client.1.smithi169.stdout:2/879: mkdir d1/d5/d12e 0 2024-03-04T13:43:05.523 INFO:tasks.workunit.client.1.smithi169.stdout:9/866: link d7/d25/d64/c90 d7/dc/df1/d56/c137 0 2024-03-04T13:43:05.523 INFO:tasks.workunit.client.0.smithi107.stdout:1/301: write d1/d10/d16/f37 [4328640,129329] 0 2024-03-04T13:43:05.523 INFO:tasks.workunit.client.0.smithi107.stdout:0/372: rename da/dc/dd/d3e/d6e/d73/d74 to da/d7e 0 2024-03-04T13:43:05.528 INFO:tasks.workunit.client.0.smithi107.stdout:6/377: truncate df/d2e/f5a 574300 0 2024-03-04T13:43:05.528 INFO:tasks.workunit.client.1.smithi169.stdout:0/696: link d5/d9/d3d/d5d/cdb d5/d9/d3d/d5d/de4/d98/cf0 0 2024-03-04T13:43:05.529 INFO:tasks.workunit.client.0.smithi107.stdout:8/344: dwrite d6/fe [0,4194304] 0 2024-03-04T13:43:05.533 INFO:tasks.workunit.client.1.smithi169.stdout:9/867: dread d7/d2d/d48/f9c [0,4194304] 0 2024-03-04T13:43:05.535 INFO:tasks.workunit.client.1.smithi169.stdout:4/796: dread d4/d11/d3b/d67/d7e/f9c [0,4194304] 0 2024-03-04T13:43:05.538 INFO:tasks.workunit.client.1.smithi169.stdout:9/868: chown d7/dc/d12/d20/da6/dd1/l12d 4278090 1 2024-03-04T13:43:05.539 INFO:tasks.workunit.client.0.smithi107.stdout:7/292: symlink d6/d1d/d36/l5f 0 2024-03-04T13:43:05.543 INFO:tasks.workunit.client.1.smithi169.stdout:9/869: fdatasync d7/dc/d12/d20/da6/fdf 0 2024-03-04T13:43:05.547 INFO:tasks.workunit.client.0.smithi107.stdout:9/290: truncate d4/da/d11/d4c/f3c 1029125 0 2024-03-04T13:43:05.548 INFO:tasks.workunit.client.1.smithi169.stdout:7/861: dwrite d2/de/d3e/d41/fe9 [0,4194304] 0 2024-03-04T13:43:05.554 INFO:tasks.workunit.client.1.smithi169.stdout:2/880: mknod d1/d5/d19/d68/dd9/d115/c12f 0 2024-03-04T13:43:05.560 INFO:tasks.workunit.client.0.smithi107.stdout:0/373: rename da/dc/dd/d3e/f63 to da/d7e/f7f 0 2024-03-04T13:43:05.562 INFO:tasks.workunit.client.0.smithi107.stdout:3/306: mkdir d1/de/d18/d33/d4d/d4f/d69 0 2024-03-04T13:43:05.564 INFO:tasks.workunit.client.0.smithi107.stdout:0/374: dread da/dc/dd/f24 [4194304,4194304] 0 2024-03-04T13:43:05.564 INFO:tasks.workunit.client.0.smithi107.stdout:8/345: mknod d6/d16/d1f/d4c/c62 0 2024-03-04T13:43:05.565 INFO:tasks.workunit.client.0.smithi107.stdout:5/270: write d5/d8/fb [2903202,66187] 0 2024-03-04T13:43:05.566 INFO:tasks.workunit.client.0.smithi107.stdout:0/375: fdatasync da/dc/f4d 0 2024-03-04T13:43:05.567 INFO:tasks.workunit.client.0.smithi107.stdout:7/293: mkdir d6/d13/d16/d19/d60 0 2024-03-04T13:43:05.568 INFO:tasks.workunit.client.1.smithi169.stdout:7/862: mkdir d2/d28/d68/d124 0 2024-03-04T13:43:05.571 INFO:tasks.workunit.client.0.smithi107.stdout:7/294: dread - d6/d13/d16/d1c/f48 zero size 2024-03-04T13:43:05.571 INFO:tasks.workunit.client.0.smithi107.stdout:0/376: rename da/dc/dd to da/dc/dd/d3e/d6e/d73/d80 22 2024-03-04T13:43:05.571 INFO:tasks.workunit.client.1.smithi169.stdout:5/910: truncate d1/d16/d2a/d62/fb4 3321630 0 2024-03-04T13:43:05.575 INFO:tasks.workunit.client.0.smithi107.stdout:4/289: creat d0/d9/f5b x:0 0 0 2024-03-04T13:43:05.575 INFO:tasks.workunit.client.0.smithi107.stdout:7/295: write d6/d13/d16/d19/d39/f40 [1690538,81320] 0 2024-03-04T13:43:05.575 INFO:tasks.workunit.client.0.smithi107.stdout:9/291: mknod d4/da/d11/d4c/d51/c5b 0 2024-03-04T13:43:05.580 INFO:tasks.workunit.client.0.smithi107.stdout:4/290: truncate d0/d9/de/d14/f4e 33885 0 2024-03-04T13:43:05.580 INFO:tasks.workunit.client.0.smithi107.stdout:7/296: fsync d6/d13/d16/d19/f26 0 2024-03-04T13:43:05.580 INFO:tasks.workunit.client.1.smithi169.stdout:0/697: truncate d5/d9/dc/d12/f24 2245829 0 2024-03-04T13:43:05.581 INFO:tasks.workunit.client.1.smithi169.stdout:4/797: creat d4/d11/d2e/dcd/dee/d5c/d86/d8f/dca/fef x:0 0 0 2024-03-04T13:43:05.581 INFO:tasks.workunit.client.1.smithi169.stdout:9/870: mkdir d7/d25/d138 0 2024-03-04T13:43:05.582 INFO:tasks.workunit.client.1.smithi169.stdout:7/863: creat d2/d28/d70/ded/f125 x:0 0 0 2024-03-04T13:43:05.584 INFO:tasks.workunit.client.1.smithi169.stdout:4/798: read d4/d11/d3b/f83 [1091043,17075] 0 2024-03-04T13:43:05.584 INFO:tasks.workunit.client.0.smithi107.stdout:3/307: dwrite d1/f56 [0,4194304] 0 2024-03-04T13:43:05.587 INFO:tasks.workunit.client.0.smithi107.stdout:3/308: dread - d1/d2d/f5c zero size 2024-03-04T13:43:05.588 INFO:tasks.workunit.client.0.smithi107.stdout:8/346: dwrite d6/d16/d59/d3e/f56 [0,4194304] 0 2024-03-04T13:43:05.590 INFO:tasks.workunit.client.1.smithi169.stdout:8/824: dread d7/f6a [0,4194304] 0 2024-03-04T13:43:05.590 INFO:tasks.workunit.client.0.smithi107.stdout:6/378: mknod df/d17/d6b/c75 0 2024-03-04T13:43:05.592 INFO:tasks.workunit.client.1.smithi169.stdout:0/698: truncate d5/d9/dc/d12/d15/d19/f20 5111202 0 2024-03-04T13:43:05.592 INFO:tasks.workunit.client.1.smithi169.stdout:9/871: read d7/dc/f80 [696437,92600] 0 2024-03-04T13:43:05.593 INFO:tasks.workunit.client.0.smithi107.stdout:0/377: dwrite f4 [0,4194304] 0 2024-03-04T13:43:05.595 INFO:tasks.workunit.client.1.smithi169.stdout:7/864: mkdir d2/d15/d17/d6f/d126 0 2024-03-04T13:43:05.599 INFO:tasks.workunit.client.0.smithi107.stdout:0/378: write da/dc/dd/d3e/f44 [173249,108281] 0 2024-03-04T13:43:05.599 INFO:tasks.workunit.client.1.smithi169.stdout:1/774: dread d4/d5/d36/d35/d5b/fcc [0,4194304] 0 2024-03-04T13:43:05.600 INFO:tasks.workunit.client.0.smithi107.stdout:0/379: write da/dc/dd/d3e/d26/f58 [464267,61434] 0 2024-03-04T13:43:05.603 INFO:tasks.workunit.client.1.smithi169.stdout:1/775: readlink d4/d5/d36/d75/l99 0 2024-03-04T13:43:05.605 INFO:tasks.workunit.client.1.smithi169.stdout:1/776: chown d4/d5/d36/d35/d8f/le2 28 1 2024-03-04T13:43:05.607 INFO:tasks.workunit.client.1.smithi169.stdout:8/825: symlink d7/d10/d17/d44/l10a 0 2024-03-04T13:43:05.608 INFO:tasks.workunit.client.1.smithi169.stdout:2/881: write d1/d5/d22/d3b/f4d [1120752,62935] 0 2024-03-04T13:43:05.610 INFO:tasks.workunit.client.0.smithi107.stdout:0/380: dread da/f59 [0,4194304] 0 2024-03-04T13:43:05.611 INFO:tasks.workunit.client.0.smithi107.stdout:0/381: readlink da/l18 0 2024-03-04T13:43:05.615 INFO:tasks.workunit.client.0.smithi107.stdout:1/302: rmdir d1/d3/df/d33/d5d/d24/d5c 0 2024-03-04T13:43:05.618 INFO:tasks.workunit.client.1.smithi169.stdout:0/699: mknod d5/d9/dc/d12/d15/d19/d61/d71/d8c/cf1 0 2024-03-04T13:43:05.619 INFO:tasks.workunit.client.0.smithi107.stdout:2/324: getdents da/d49 0 2024-03-04T13:43:05.620 INFO:tasks.workunit.client.1.smithi169.stdout:7/865: fsync d2/d15/d17/d6f/f9b 0 2024-03-04T13:43:05.621 INFO:tasks.workunit.client.0.smithi107.stdout:2/325: write da/f10 [8824460,130367] 0 2024-03-04T13:43:05.622 INFO:tasks.workunit.client.0.smithi107.stdout:2/326: fdatasync da/df/d27/f5e 0 2024-03-04T13:43:05.623 INFO:tasks.workunit.client.0.smithi107.stdout:9/292: creat d4/da/d11/d4c/d51/f5c x:0 0 0 2024-03-04T13:43:05.624 INFO:tasks.workunit.client.1.smithi169.stdout:4/799: rename d4/d11/c24 to d4/d11/d2e/dcd/dee/d5c/d86/d8f/dca/cf0 0 2024-03-04T13:43:05.624 INFO:tasks.workunit.client.1.smithi169.stdout:1/777: creat d4/d5/d2d/d4e/d71/ffb x:0 0 0 2024-03-04T13:43:05.628 INFO:tasks.workunit.client.1.smithi169.stdout:1/778: chown d4/d5/d36/d35/d16/d81/db4/dd7 6342050 1 2024-03-04T13:43:05.629 INFO:tasks.workunit.client.0.smithi107.stdout:4/291: creat d0/d9/d18/d29/f5c x:0 0 0 2024-03-04T13:43:05.630 INFO:tasks.workunit.client.1.smithi169.stdout:5/911: dwrite d1/d16/d17/d2d/ff2 [0,4194304] 0 2024-03-04T13:43:05.631 INFO:tasks.workunit.client.0.smithi107.stdout:0/382: dwrite da/dc/dd/d3e/f44 [0,4194304] 0 2024-03-04T13:43:05.636 INFO:tasks.workunit.client.0.smithi107.stdout:7/297: rename d6/d13/d16/d19/f26 to d6/d1d/d36/f61 0 2024-03-04T13:43:05.641 INFO:tasks.workunit.client.0.smithi107.stdout:8/347: readlink d6/d16/d59/l49 0 2024-03-04T13:43:05.642 INFO:tasks.workunit.client.1.smithi169.stdout:7/866: chown d2/de/d3e/d41/d5b/d7a/l84 189312 1 2024-03-04T13:43:05.647 INFO:tasks.workunit.client.1.smithi169.stdout:4/800: dread d4/d1c/f50 [0,4194304] 0 2024-03-04T13:43:05.648 INFO:tasks.workunit.client.1.smithi169.stdout:3/807: sync 2024-03-04T13:43:05.648 INFO:tasks.workunit.client.1.smithi169.stdout:1/779: symlink d4/d5/d2d/d4e/d71/d82/lfc 0 2024-03-04T13:43:05.649 INFO:tasks.workunit.client.0.smithi107.stdout:5/271: symlink d5/d8/d14/d5a/l64 0 2024-03-04T13:43:05.652 INFO:tasks.workunit.client.1.smithi169.stdout:5/912: mknod d1/d16/d2a/d46/d9e/d105/c146 0 2024-03-04T13:43:05.652 INFO:tasks.workunit.client.1.smithi169.stdout:4/801: write d4/d11/d2e/d39/f60 [5787445,18362] 0 2024-03-04T13:43:05.652 INFO:tasks.workunit.client.0.smithi107.stdout:8/348: dread d6/f3d [0,4194304] 0 2024-03-04T13:43:05.654 INFO:tasks.workunit.client.1.smithi169.stdout:5/913: stat d1/d16/d2f/d39/f5f 0 2024-03-04T13:43:05.656 INFO:tasks.workunit.client.1.smithi169.stdout:9/872: dwrite d7/d25/f3d [0,4194304] 0 2024-03-04T13:43:05.660 INFO:tasks.workunit.client.1.smithi169.stdout:9/873: chown d7/d11e/d120 851443 1 2024-03-04T13:43:05.660 INFO:tasks.workunit.client.1.smithi169.stdout:2/882: rename d1/d6d/dbc/d105/f125 to d1/d5/d22/d45/df3/f130 0 2024-03-04T13:43:05.662 INFO:tasks.workunit.client.0.smithi107.stdout:5/272: dwrite d5/dc/f41 [0,4194304] 0 2024-03-04T13:43:05.666 INFO:tasks.workunit.client.0.smithi107.stdout:9/293: unlink d4/d32/f52 0 2024-03-04T13:43:05.666 INFO:tasks.workunit.client.1.smithi169.stdout:7/867: fdatasync d2/de/fcb 0 2024-03-04T13:43:05.668 INFO:tasks.workunit.client.1.smithi169.stdout:8/826: dwrite d7/f6a [0,4194304] 0 2024-03-04T13:43:05.669 INFO:tasks.workunit.client.0.smithi107.stdout:4/292: creat d0/d9/de/d14/dc/d21/f5d x:0 0 0 2024-03-04T13:43:05.671 INFO:tasks.workunit.client.0.smithi107.stdout:0/383: rmdir da/dc/dd/d3e/d26/d69 39 2024-03-04T13:43:05.673 INFO:tasks.workunit.client.1.smithi169.stdout:8/827: chown d7/l63 124915176 1 2024-03-04T13:43:05.676 INFO:tasks.workunit.client.0.smithi107.stdout:3/309: symlink d1/d41/l6a 0 2024-03-04T13:43:05.678 INFO:tasks.workunit.client.1.smithi169.stdout:8/828: dread - d7/d10/d22/d75/d81/f100 zero size 2024-03-04T13:43:05.678 INFO:tasks.workunit.client.1.smithi169.stdout:4/802: unlink d4/d11/d2e/d39/c7a 0 2024-03-04T13:43:05.680 INFO:tasks.workunit.client.1.smithi169.stdout:0/700: dwrite d5/d9/dc/d12/d15/d19/d61/d71/fa8 [0,4194304] 0 2024-03-04T13:43:05.682 INFO:tasks.workunit.client.1.smithi169.stdout:8/829: write d7/d10/d24/f3c [419234,78437] 0 2024-03-04T13:43:05.691 INFO:tasks.workunit.client.1.smithi169.stdout:3/808: rename d7/de/d2b/d7d/d85/d13/d1a/d2d/d56/d5e/d67/dd7 to d7/de/d2b/d7d/d85/d13/d1a/d2d/dbe/d108 0 2024-03-04T13:43:05.692 INFO:tasks.workunit.client.0.smithi107.stdout:8/349: chown d6/f29 7 1 2024-03-04T13:43:05.694 INFO:tasks.workunit.client.0.smithi107.stdout:3/310: dwrite d1/d2d/f5c [0,4194304] 0 2024-03-04T13:43:05.694 INFO:tasks.workunit.client.0.smithi107.stdout:8/350: chown d6/d16/d1f/f43 1182 1 2024-03-04T13:43:05.699 INFO:tasks.workunit.client.0.smithi107.stdout:2/327: mkdir da/d6c 0 2024-03-04T13:43:05.703 INFO:tasks.workunit.client.1.smithi169.stdout:1/780: unlink d4/d5/d36/d35/c8d 0 2024-03-04T13:43:05.704 INFO:tasks.workunit.client.1.smithi169.stdout:6/813: sync 2024-03-04T13:43:05.705 INFO:tasks.workunit.client.0.smithi107.stdout:9/294: rename d4/d34/c46 to d4/da/d11/d20/d2f/c5d 0 2024-03-04T13:43:05.707 INFO:tasks.workunit.client.0.smithi107.stdout:9/295: fsync d4/da/d11/d20/f56 0 2024-03-04T13:43:05.707 INFO:tasks.workunit.client.0.smithi107.stdout:4/293: rmdir d0/d9/de/d14/d33 39 2024-03-04T13:43:05.709 INFO:tasks.workunit.client.1.smithi169.stdout:4/803: mkdir d4/d11/d2e/d39/d61/d64/df1 0 2024-03-04T13:43:05.710 INFO:tasks.workunit.client.0.smithi107.stdout:0/384: fdatasync da/dc/dd/f1f 0 2024-03-04T13:43:05.712 INFO:tasks.workunit.client.0.smithi107.stdout:7/298: mkdir d6/d13/d16/d19/d38/d62 0 2024-03-04T13:43:05.713 INFO:tasks.workunit.client.1.smithi169.stdout:8/830: mkdir d7/d10/d17/d44/d5d/d10b 0 2024-03-04T13:43:05.716 INFO:tasks.workunit.client.0.smithi107.stdout:1/303: sync 2024-03-04T13:43:05.720 INFO:tasks.workunit.client.0.smithi107.stdout:4/294: dwrite d0/d9/f5b [0,4194304] 0 2024-03-04T13:43:05.721 INFO:tasks.workunit.client.1.smithi169.stdout:9/874: rename d7/d27/d84/d12b to d7/d25/d3c/dd2/d139 0 2024-03-04T13:43:05.723 INFO:tasks.workunit.client.0.smithi107.stdout:6/379: truncate df/d17/f28 2443036 0 2024-03-04T13:43:05.726 INFO:tasks.workunit.client.0.smithi107.stdout:3/311: unlink d1/d2d/f39 0 2024-03-04T13:43:05.726 INFO:tasks.workunit.client.1.smithi169.stdout:2/883: write d1/d5/d19/d68/f88 [2549058,15130] 0 2024-03-04T13:43:05.726 INFO:tasks.workunit.client.1.smithi169.stdout:3/809: fsync d7/de/d2b/d7d/d85/d47/d63/d6c/fba 0 2024-03-04T13:43:05.727 INFO:tasks.workunit.client.0.smithi107.stdout:6/380: stat df/d2e/d36/d71/c60 0 2024-03-04T13:43:05.730 INFO:tasks.workunit.client.0.smithi107.stdout:5/273: mkdir d5/d65 0 2024-03-04T13:43:05.731 INFO:tasks.workunit.client.1.smithi169.stdout:7/868: symlink d2/de/d3e/d7e/dbb/l127 0 2024-03-04T13:43:05.731 INFO:tasks.workunit.client.0.smithi107.stdout:5/274: read - d5/d2b/f38 zero size 2024-03-04T13:43:05.733 INFO:tasks.workunit.client.0.smithi107.stdout:5/275: chown d5/dc/d22 1919532 1 2024-03-04T13:43:05.739 INFO:tasks.workunit.client.0.smithi107.stdout:8/351: dwrite d6/da/f1b [0,4194304] 0 2024-03-04T13:43:05.745 INFO:tasks.workunit.client.0.smithi107.stdout:3/312: dwrite d1/f6 [0,4194304] 0 2024-03-04T13:43:05.751 INFO:tasks.workunit.client.0.smithi107.stdout:1/304: creat d1/d3/df/d33/d43/d5b/f5f x:0 0 0 2024-03-04T13:43:05.755 INFO:tasks.workunit.client.1.smithi169.stdout:1/781: write d4/d5/d2d/f6c [2978469,68077] 0 2024-03-04T13:43:05.762 INFO:tasks.workunit.client.1.smithi169.stdout:4/804: write d4/d11/d2e/d6c/d78/da5/fbf [811103,42003] 0 2024-03-04T13:43:05.762 INFO:tasks.workunit.client.0.smithi107.stdout:6/381: rmdir df/d2e/d36/d5b 39 2024-03-04T13:43:05.764 INFO:tasks.workunit.client.0.smithi107.stdout:6/382: dread - df/d38/f5f zero size 2024-03-04T13:43:05.766 INFO:tasks.workunit.client.1.smithi169.stdout:0/701: truncate d5/d4c/d65/faf 990242 0 2024-03-04T13:43:05.769 INFO:tasks.workunit.client.0.smithi107.stdout:7/299: mknod d6/d13/d16/d19/d1e/d3d/c63 0 2024-03-04T13:43:05.769 INFO:tasks.workunit.client.0.smithi107.stdout:0/385: dwrite da/dc/d34/f42 [0,4194304] 0 2024-03-04T13:43:05.770 INFO:tasks.workunit.client.1.smithi169.stdout:5/914: getdents d1/d16/d17/d2d 0 2024-03-04T13:43:05.771 INFO:tasks.workunit.client.0.smithi107.stdout:8/352: mkdir d6/d16/d1f/d63 0 2024-03-04T13:43:05.773 INFO:tasks.workunit.client.0.smithi107.stdout:5/276: dread d5/f6 [0,4194304] 0 2024-03-04T13:43:05.774 INFO:tasks.workunit.client.1.smithi169.stdout:8/831: creat d7/d10/d22/d46/da5/dcb/f10c x:0 0 0 2024-03-04T13:43:05.774 INFO:tasks.workunit.client.0.smithi107.stdout:3/313: mkdir d1/de/d18/d33/d6b 0 2024-03-04T13:43:05.776 INFO:tasks.workunit.client.1.smithi169.stdout:8/832: read d7/d10/d17/d44/d4c/fa3 [107370,90102] 0 2024-03-04T13:43:05.777 INFO:tasks.workunit.client.1.smithi169.stdout:9/875: creat d7/dc/d3e/d125/f13a x:0 0 0 2024-03-04T13:43:05.777 INFO:tasks.workunit.client.1.smithi169.stdout:6/814: truncate d0/d3c/d40/d52/f8c 2996938 0 2024-03-04T13:43:05.779 INFO:tasks.workunit.client.1.smithi169.stdout:8/833: chown d7/d10/d22/d75/d81/c76 346430664 1 2024-03-04T13:43:05.779 INFO:tasks.workunit.client.1.smithi169.stdout:9/876: dread - d7/dc/d12/d20/d4e/fc8 zero size 2024-03-04T13:43:05.785 INFO:tasks.workunit.client.1.smithi169.stdout:2/884: mknod d1/d6d/dbc/c131 0 2024-03-04T13:43:05.785 INFO:tasks.workunit.client.0.smithi107.stdout:1/305: rename d1/d3/df/d33/d43/d4d to d1/d3/d4e/d60 0 2024-03-04T13:43:05.786 INFO:tasks.workunit.client.0.smithi107.stdout:6/383: rename df/d57 to df/d57/d76 22 2024-03-04T13:43:05.787 INFO:tasks.workunit.client.0.smithi107.stdout:6/384: fsync df/d38/f4e 0 2024-03-04T13:43:05.793 INFO:tasks.workunit.client.1.smithi169.stdout:0/702: read d5/d9/dc/d12/d15/f74 [111509,93083] 0 2024-03-04T13:43:05.794 INFO:tasks.workunit.client.1.smithi169.stdout:5/915: mknod d1/d16/d2a/c147 0 2024-03-04T13:43:05.798 INFO:tasks.workunit.client.1.smithi169.stdout:6/815: creat d0/d3c/d40/d52/f126 x:0 0 0 2024-03-04T13:43:05.800 INFO:tasks.workunit.client.0.smithi107.stdout:4/295: write d0/d9/d18/d29/f35 [1068052,111133] 0 2024-03-04T13:43:05.802 INFO:tasks.workunit.client.0.smithi107.stdout:4/296: write d0/d9/d10/f4a [984288,29864] 0 2024-03-04T13:43:05.802 INFO:tasks.workunit.client.1.smithi169.stdout:8/834: rmdir d7/d10/d22/d46/da5 39 2024-03-04T13:43:05.803 INFO:tasks.workunit.client.0.smithi107.stdout:2/328: getdents da/d34 0 2024-03-04T13:43:05.804 INFO:tasks.workunit.client.0.smithi107.stdout:4/297: chown d0/d9/d18/d29/f5c 47 1 2024-03-04T13:43:05.806 INFO:tasks.workunit.client.1.smithi169.stdout:2/885: rmdir d1/d5/d19/d3c/db8 39 2024-03-04T13:43:05.807 INFO:tasks.workunit.client.0.smithi107.stdout:1/306: truncate d1/d10/d16/d1b/d40/f57 1040742 0 2024-03-04T13:43:05.811 INFO:tasks.workunit.client.0.smithi107.stdout:9/296: getdents d4/da/d11/d4c 0 2024-03-04T13:43:05.811 INFO:tasks.workunit.client.1.smithi169.stdout:4/805: fsync d4/d11/f79 0 2024-03-04T13:43:05.814 INFO:tasks.workunit.client.0.smithi107.stdout:7/300: truncate f0 4093810 0 2024-03-04T13:43:05.817 INFO:tasks.workunit.client.0.smithi107.stdout:2/329: dwrite da/d34/f41 [0,4194304] 0 2024-03-04T13:43:05.817 INFO:tasks.workunit.client.0.smithi107.stdout:7/301: dread - d6/d13/d16/d19/d1e/f5c zero size 2024-03-04T13:43:05.818 INFO:tasks.workunit.client.0.smithi107.stdout:2/330: chown da/d29 347557744 1 2024-03-04T13:43:05.820 INFO:tasks.workunit.client.0.smithi107.stdout:3/314: creat d1/de/d18/d33/d6b/f6c x:0 0 0 2024-03-04T13:43:05.823 INFO:tasks.workunit.client.1.smithi169.stdout:9/877: symlink d7/dc/d3e/da3/l13b 0 2024-03-04T13:43:05.824 INFO:tasks.workunit.client.1.smithi169.stdout:5/916: write d1/d16/d17/d2d/d5a/d67/f6b [79297,10017] 0 2024-03-04T13:43:05.828 INFO:tasks.workunit.client.0.smithi107.stdout:8/353: truncate d6/f5b 170794 0 2024-03-04T13:43:05.830 INFO:tasks.workunit.client.1.smithi169.stdout:3/810: getdents d7/de/d2b/d7d 0 2024-03-04T13:43:05.832 INFO:tasks.workunit.client.0.smithi107.stdout:1/307: mkdir d1/d3/df/d33/d43/d5b/d61 0 2024-03-04T13:43:05.832 INFO:tasks.workunit.client.0.smithi107.stdout:7/302: dwrite d6/d13/d16/d1c/f48 [0,4194304] 0 2024-03-04T13:43:05.836 INFO:tasks.workunit.client.0.smithi107.stdout:6/385: mkdir df/d2e/d36/d77 0 2024-03-04T13:43:05.836 INFO:tasks.workunit.client.0.smithi107.stdout:5/277: truncate d5/d8/d2a/f39 659501 0 2024-03-04T13:43:05.838 INFO:tasks.workunit.client.0.smithi107.stdout:9/297: rename d4/da/d11/d4c/d3b to d4/da/d11/d4c/d3b/d5e 22 2024-03-04T13:43:05.838 INFO:tasks.workunit.client.1.smithi169.stdout:1/782: link d4/c47 d4/d5/d2d/d4e/d71/d26/dfa/cfd 0 2024-03-04T13:43:05.839 INFO:tasks.workunit.client.0.smithi107.stdout:8/354: dread d6/da/f2c [0,4194304] 0 2024-03-04T13:43:05.840 INFO:tasks.workunit.client.0.smithi107.stdout:2/331: chown da/f26 29439 1 2024-03-04T13:43:05.840 INFO:tasks.workunit.client.0.smithi107.stdout:8/355: fdatasync d6/da/d25/f3f 0 2024-03-04T13:43:05.841 INFO:tasks.workunit.client.1.smithi169.stdout:4/806: mknod d4/d11/d2e/d39/d48/cf2 0 2024-03-04T13:43:05.842 INFO:tasks.workunit.client.1.smithi169.stdout:4/807: chown d4/d1c/f59 2246043 1 2024-03-04T13:43:05.842 INFO:tasks.workunit.client.1.smithi169.stdout:0/703: rename d5/d9/dc/l55 to d5/d9/d3d/d5d/lf2 0 2024-03-04T13:43:05.845 INFO:tasks.workunit.client.1.smithi169.stdout:4/808: write d4/d11/d3b/d67/fa0 [1908067,75842] 0 2024-03-04T13:43:05.847 INFO:tasks.workunit.client.1.smithi169.stdout:9/878: creat d7/d25/d3c/dd2/d139/f13c x:0 0 0 2024-03-04T13:43:05.848 INFO:tasks.workunit.client.0.smithi107.stdout:7/303: mkdir d6/d1d/d64 0 2024-03-04T13:43:05.850 INFO:tasks.workunit.client.0.smithi107.stdout:1/308: creat d1/d3/df/d33/d5d/d24/f62 x:0 0 0 2024-03-04T13:43:05.851 INFO:tasks.workunit.client.1.smithi169.stdout:5/917: mkdir d1/d16/d2a/d46/d9e/d105/d148 0 2024-03-04T13:43:05.852 INFO:tasks.workunit.client.0.smithi107.stdout:1/309: dread - d1/d3/df/d33/d5d/d24/f62 zero size 2024-03-04T13:43:05.853 INFO:tasks.workunit.client.1.smithi169.stdout:5/918: stat d1/d16/d17/d2d/d5a/d8a/fe8 0 2024-03-04T13:43:05.853 INFO:tasks.workunit.client.1.smithi169.stdout:8/835: creat d7/d10/d22/d46/da5/da6/f10d x:0 0 0 2024-03-04T13:43:05.855 INFO:tasks.workunit.client.0.smithi107.stdout:6/386: creat df/d17/d24/d30/d3b/f78 x:0 0 0 2024-03-04T13:43:05.855 INFO:tasks.workunit.client.1.smithi169.stdout:8/836: read d7/d10/fcc [2606641,63861] 0 2024-03-04T13:43:05.856 INFO:tasks.workunit.client.1.smithi169.stdout:5/919: chown d1/d81/dc1/fca 1561 1 2024-03-04T13:43:05.857 INFO:tasks.workunit.client.1.smithi169.stdout:3/811: mkdir d7/de/d2b/d7d/d85/d13/d1a/d98/d9a/d109 0 2024-03-04T13:43:05.858 INFO:tasks.workunit.client.1.smithi169.stdout:8/837: fdatasync d7/d12/fed 0 2024-03-04T13:43:05.858 INFO:tasks.workunit.client.1.smithi169.stdout:5/920: chown d1/d16/d2a/d62 2694 1 2024-03-04T13:43:05.860 INFO:tasks.workunit.client.1.smithi169.stdout:7/869: link d2/d6b/l117 d2/d15/l128 0 2024-03-04T13:43:05.864 INFO:tasks.workunit.client.1.smithi169.stdout:4/809: dread d4/d11/d3b/d67/dbb/d6e/fc5 [0,4194304] 0 2024-03-04T13:43:05.864 INFO:tasks.workunit.client.1.smithi169.stdout:7/870: chown d2/d28/d10a/f118 3784 1 2024-03-04T13:43:05.864 INFO:tasks.workunit.client.0.smithi107.stdout:3/315: mknod d1/de/d18/d33/d4d/d4f/d69/c6d 0 2024-03-04T13:43:05.866 INFO:tasks.workunit.client.1.smithi169.stdout:2/886: dwrite d1/d5/d19/f6a [0,4194304] 0 2024-03-04T13:43:05.867 INFO:tasks.workunit.client.0.smithi107.stdout:6/387: dwrite df/d38/f4e [0,4194304] 0 2024-03-04T13:43:05.875 INFO:tasks.workunit.client.0.smithi107.stdout:7/304: creat d6/d1d/d36/d59/f65 x:0 0 0 2024-03-04T13:43:05.876 INFO:tasks.workunit.client.1.smithi169.stdout:9/879: truncate d7/d27/f76 1454060 0 2024-03-04T13:43:05.878 INFO:tasks.workunit.client.0.smithi107.stdout:5/278: mkdir d5/d66 0 2024-03-04T13:43:05.881 INFO:tasks.workunit.client.0.smithi107.stdout:1/310: creat d1/d3/d4e/d60/f63 x:0 0 0 2024-03-04T13:43:05.886 INFO:tasks.workunit.client.1.smithi169.stdout:3/812: creat d7/d53/f10a x:0 0 0 2024-03-04T13:43:05.889 INFO:tasks.workunit.client.1.smithi169.stdout:8/838: readlink d7/d10/d22/d46/d49/lfb 0 2024-03-04T13:43:05.889 INFO:tasks.workunit.client.0.smithi107.stdout:3/316: write d1/f3 [2439542,10082] 0 2024-03-04T13:43:05.891 INFO:tasks.workunit.client.1.smithi169.stdout:6/816: truncate d0/d3c/d40/d52/f8c 2316455 0 2024-03-04T13:43:05.892 INFO:tasks.workunit.client.0.smithi107.stdout:2/332: symlink da/df/d33/d68/l6d 0 2024-03-04T13:43:05.894 INFO:tasks.workunit.client.1.smithi169.stdout:4/810: chown d4/d11/d2e/d21/la7 62 1 2024-03-04T13:43:05.894 INFO:tasks.workunit.client.1.smithi169.stdout:7/871: creat d2/d28/d70/f129 x:0 0 0 2024-03-04T13:43:05.895 INFO:tasks.workunit.client.0.smithi107.stdout:8/356: dwrite f2 [0,4194304] 0 2024-03-04T13:43:05.897 INFO:tasks.workunit.client.1.smithi169.stdout:4/811: write d4/d11/d2e/d39/d48/f3c [4444705,111650] 0 2024-03-04T13:43:05.903 INFO:tasks.workunit.client.1.smithi169.stdout:9/880: creat d7/dc/d3e/f13d x:0 0 0 2024-03-04T13:43:05.906 INFO:tasks.workunit.client.1.smithi169.stdout:5/921: write d1/d16/d17/d2d/d5a/fd1 [38209,36459] 0 2024-03-04T13:43:05.910 INFO:tasks.workunit.client.0.smithi107.stdout:6/388: mknod df/d1b/d2c/c79 0 2024-03-04T13:43:05.910 INFO:tasks.workunit.client.1.smithi169.stdout:5/922: dread - d1/d16/d17/d28/f131 zero size 2024-03-04T13:43:05.911 INFO:tasks.workunit.client.0.smithi107.stdout:6/389: dread - df/d1b/d2c/d46/f68 zero size 2024-03-04T13:43:05.912 INFO:tasks.workunit.client.0.smithi107.stdout:3/317: creat d1/de/d18/d33/d4d/d4f/d69/f6e x:0 0 0 2024-03-04T13:43:05.913 INFO:tasks.workunit.client.0.smithi107.stdout:6/390: stat df/d17/d24/d40/d63 0 2024-03-04T13:43:05.914 INFO:tasks.workunit.client.1.smithi169.stdout:8/839: truncate d7/d10/d24/d3e/f40 545720 0 2024-03-04T13:43:05.914 INFO:tasks.workunit.client.1.smithi169.stdout:1/783: truncate d4/d5/d2d/d4e/fa3 1247242 0 2024-03-04T13:43:05.916 INFO:tasks.workunit.client.0.smithi107.stdout:5/279: write d5/d8/d14/f4c [927536,21376] 0 2024-03-04T13:43:05.917 INFO:tasks.workunit.client.1.smithi169.stdout:0/704: dread d5/d9/d3d/d5d/de4/fb4 [0,4194304] 0 2024-03-04T13:43:05.917 INFO:tasks.workunit.client.0.smithi107.stdout:5/280: stat d5/f20 0 2024-03-04T13:43:05.918 INFO:tasks.workunit.client.1.smithi169.stdout:6/817: creat d0/d32/d3a/d43/d74/da6/f127 x:0 0 0 2024-03-04T13:43:05.920 INFO:tasks.workunit.client.0.smithi107.stdout:1/311: mkdir d1/d3/df/d33/d43/d5b/d61/d64 0 2024-03-04T13:43:05.923 INFO:tasks.workunit.client.0.smithi107.stdout:9/298: rename d4/da/c27 to d4/da/d11/d4c/d3b/c5f 0 2024-03-04T13:43:05.925 INFO:tasks.workunit.client.1.smithi169.stdout:3/813: mknod d7/de/d2b/d7d/d85/d13/d1a/d2d/d56/d5e/d67/c10b 0 2024-03-04T13:43:05.927 INFO:tasks.workunit.client.1.smithi169.stdout:9/881: rename d7/dc/d12/d32/d42/d4f/f8d to d7/dc/d12/d20/da6/dd1/f13e 0 2024-03-04T13:43:05.928 INFO:tasks.workunit.client.0.smithi107.stdout:9/299: dread - d4/d34/f44 zero size 2024-03-04T13:43:05.928 INFO:tasks.workunit.client.1.smithi169.stdout:3/814: chown d7/de/d2b/d7d/d85/d13/d1a/d2d/d56/d5e/d67/d5c/d84 168105 1 2024-03-04T13:43:05.929 INFO:tasks.workunit.client.0.smithi107.stdout:5/281: mknod d5/d8/d14/c67 0 2024-03-04T13:43:05.929 INFO:tasks.workunit.client.1.smithi169.stdout:7/872: mkdir d2/de/dfb/d12a 0 2024-03-04T13:43:05.930 INFO:tasks.workunit.client.1.smithi169.stdout:9/882: fdatasync d7/dc/d12/d32/d42/f46 0 2024-03-04T13:43:05.931 INFO:tasks.workunit.client.0.smithi107.stdout:1/312: readlink d1/d10/d16/d1b/l50 0 2024-03-04T13:43:05.932 INFO:tasks.workunit.client.0.smithi107.stdout:5/282: truncate d5/d2b/f38 926506 0 2024-03-04T13:43:05.933 INFO:tasks.workunit.client.0.smithi107.stdout:3/318: dwrite d1/de/d18/f59 [0,4194304] 0 2024-03-04T13:43:05.935 INFO:tasks.workunit.client.0.smithi107.stdout:8/357: dwrite d6/d16/d1f/f34 [4194304,4194304] 0 2024-03-04T13:43:05.936 INFO:tasks.workunit.client.0.smithi107.stdout:3/319: write d1/de/d18/d33/f50 [457765,95998] 0 2024-03-04T13:43:05.940 INFO:tasks.workunit.client.1.smithi169.stdout:4/812: dread d4/d11/d3b/d67/dbb/f76 [0,4194304] 0 2024-03-04T13:43:05.944 INFO:tasks.workunit.client.1.smithi169.stdout:0/705: creat d5/d9/ff3 x:0 0 0 2024-03-04T13:43:05.946 INFO:tasks.workunit.client.1.smithi169.stdout:0/706: dread - d5/d9/d3d/d5d/fec zero size 2024-03-04T13:43:05.950 INFO:tasks.workunit.client.1.smithi169.stdout:0/707: fdatasync d5/d9/d3d/d59/d64/d7f/fcc 0 2024-03-04T13:43:05.953 INFO:tasks.workunit.client.1.smithi169.stdout:5/923: rename d1/d16/d17/d1b/d74/ddc/de4 to d1/d16/d2a/d46/d9e/d105/d148/d149 0 2024-03-04T13:43:05.954 INFO:tasks.workunit.client.1.smithi169.stdout:9/883: rmdir d7 39 2024-03-04T13:43:05.959 INFO:tasks.workunit.client.1.smithi169.stdout:3/815: mknod d7/de/d2b/d7d/d85/d18/d21/d3f/d68/c10c 0 2024-03-04T13:43:05.962 INFO:tasks.workunit.client.1.smithi169.stdout:6/818: truncate d0/f2f 850812 0 2024-03-04T13:43:05.962 INFO:tasks.workunit.client.1.smithi169.stdout:3/816: dread - d7/de/d2b/d7d/d85/d18/d21/d66/fd9 zero size 2024-03-04T13:43:05.965 INFO:tasks.workunit.client.1.smithi169.stdout:4/813: stat d4/d11/d2e/d21/c34 0 2024-03-04T13:43:05.965 INFO:tasks.workunit.client.1.smithi169.stdout:2/887: truncate d1/d5/d19/d3c/db8/fca 1809256 0 2024-03-04T13:43:05.967 INFO:tasks.workunit.client.1.smithi169.stdout:4/814: dread - d4/d11/d2e/d21/da9/fcb zero size 2024-03-04T13:43:05.967 INFO:tasks.workunit.client.1.smithi169.stdout:0/708: mknod d5/d4c/d65/d95/db0/cf4 0 2024-03-04T13:43:05.971 INFO:tasks.workunit.client.1.smithi169.stdout:3/817: chown d7/de/d2b/d7d/d85/d13/d5a/d65/f9b 41 1 2024-03-04T13:43:05.971 INFO:tasks.workunit.client.1.smithi169.stdout:4/815: dread - d4/d1c/d58/d9d/fd0 zero size 2024-03-04T13:43:05.972 INFO:tasks.workunit.client.1.smithi169.stdout:5/924: creat d1/d81/db1/f14a x:0 0 0 2024-03-04T13:43:05.974 INFO:tasks.workunit.client.1.smithi169.stdout:5/925: dread - d1/d23/f11d zero size 2024-03-04T13:43:05.974 INFO:tasks.workunit.client.1.smithi169.stdout:1/784: link d4/d5/d2d/d4e/da5/cca d4/d5/d36/d35/df/cfe 0 2024-03-04T13:43:05.976 INFO:tasks.workunit.client.1.smithi169.stdout:1/785: read d4/f20 [902360,5593] 0 2024-03-04T13:43:05.977 INFO:tasks.workunit.client.1.smithi169.stdout:0/709: fdatasync d5/d9/dc/d12/d15/d19/fcb 0 2024-03-04T13:43:05.982 INFO:tasks.workunit.client.0.smithi107.stdout:0/386: sync 2024-03-04T13:43:05.982 INFO:tasks.workunit.client.0.smithi107.stdout:4/298: sync 2024-03-04T13:43:05.983 INFO:tasks.workunit.client.0.smithi107.stdout:5/283: chown d5/d8/d2a/f53 891 1 2024-03-04T13:43:05.984 INFO:tasks.workunit.client.1.smithi169.stdout:2/888: dwrite d1/d5/d19/d1e/ffb [0,4194304] 0 2024-03-04T13:43:05.984 INFO:tasks.workunit.client.0.smithi107.stdout:7/305: truncate d6/d13/d16/d1c/f48 3712951 0 2024-03-04T13:43:05.986 INFO:tasks.workunit.client.0.smithi107.stdout:4/299: chown d0/d30 15510619 1 2024-03-04T13:43:05.988 INFO:tasks.workunit.client.1.smithi169.stdout:4/816: mknod d4/d11/d2e/d21/cf3 0 2024-03-04T13:43:05.988 INFO:tasks.workunit.client.1.smithi169.stdout:0/710: creat d5/ff5 x:0 0 0 2024-03-04T13:43:05.990 INFO:tasks.workunit.client.0.smithi107.stdout:3/320: write d1/de/d18/d1f/d42/f5d [392974,14678] 0 2024-03-04T13:43:05.990 INFO:tasks.workunit.client.1.smithi169.stdout:4/817: write d4/f69 [518797,42120] 0 2024-03-04T13:43:05.991 INFO:tasks.workunit.client.0.smithi107.stdout:6/391: dread df/d17/f23 [0,4194304] 0 2024-03-04T13:43:05.991 INFO:tasks.workunit.client.0.smithi107.stdout:0/387: write da/dc/dd/d3e/f77 [73192,76143] 0 2024-03-04T13:43:05.994 INFO:tasks.workunit.client.0.smithi107.stdout:5/284: readlink d5/d8/l27 0 2024-03-04T13:43:05.998 INFO:tasks.workunit.client.0.smithi107.stdout:8/358: mknod d6/d16/c64 0 2024-03-04T13:43:05.998 INFO:tasks.workunit.client.1.smithi169.stdout:7/873: link d2/de/d3e/d41/f5a d2/de/d3e/d7e/dbb/f12b 0 2024-03-04T13:43:06.000 INFO:tasks.workunit.client.0.smithi107.stdout:1/313: dwrite d1/d10/d16/d1b/f3e [0,4194304] 0 2024-03-04T13:43:06.000 INFO:tasks.workunit.client.0.smithi107.stdout:5/285: readlink d5/dc/d22/l36 0 2024-03-04T13:43:06.001 INFO:tasks.workunit.client.1.smithi169.stdout:7/874: write d2/d15/dad/f100 [99535,109146] 0 2024-03-04T13:43:06.002 INFO:tasks.workunit.client.1.smithi169.stdout:1/786: read d4/d5/d36/d35/df/f28 [3625480,50849] 0 2024-03-04T13:43:06.004 INFO:tasks.workunit.client.1.smithi169.stdout:6/819: rename d0/f8 to d0/d32/d3a/d43/f128 0 2024-03-04T13:43:06.004 INFO:tasks.workunit.client.0.smithi107.stdout:7/306: dread d6/f45 [0,4194304] 0 2024-03-04T13:43:06.008 INFO:tasks.workunit.client.1.smithi169.stdout:3/818: truncate d7/de/d2b/d7d/d85/d13/d5a/da1/fdd 963155 0 2024-03-04T13:43:06.009 INFO:tasks.workunit.client.0.smithi107.stdout:6/392: mknod df/d17/d24/c7a 0 2024-03-04T13:43:06.012 INFO:tasks.workunit.client.1.smithi169.stdout:2/889: creat d1/d5/d19/d68/dd9/d115/f132 x:0 0 0 2024-03-04T13:43:06.014 INFO:tasks.workunit.client.0.smithi107.stdout:8/359: creat d6/d16/d1f/f65 x:0 0 0 2024-03-04T13:43:06.016 INFO:tasks.workunit.client.0.smithi107.stdout:7/307: dwrite d6/fa [0,4194304] 0 2024-03-04T13:43:06.019 INFO:tasks.workunit.client.0.smithi107.stdout:1/314: mkdir d1/d3/d65 0 2024-03-04T13:43:06.020 INFO:tasks.workunit.client.0.smithi107.stdout:4/300: symlink d0/l5e 0 2024-03-04T13:43:06.020 INFO:tasks.workunit.client.0.smithi107.stdout:5/286: mkdir d5/d8/d14/d5a/d68 0 2024-03-04T13:43:06.022 INFO:tasks.workunit.client.1.smithi169.stdout:5/926: dwrite d1/d60/fae [0,4194304] 0 2024-03-04T13:43:06.022 INFO:tasks.workunit.client.0.smithi107.stdout:1/315: write d1/f31 [16400,64979] 0 2024-03-04T13:43:06.023 INFO:tasks.workunit.client.0.smithi107.stdout:5/287: readlink d5/d8/l27 0 2024-03-04T13:43:06.033 INFO:tasks.workunit.client.0.smithi107.stdout:0/388: rename da/f31 to da/f81 0 2024-03-04T13:43:06.036 INFO:tasks.workunit.client.0.smithi107.stdout:7/308: mkdir d6/d13/d16/d19/d1e/d66 0 2024-03-04T13:43:06.040 INFO:tasks.workunit.client.0.smithi107.stdout:0/389: write f4 [4362983,118971] 0 2024-03-04T13:43:06.045 INFO:tasks.workunit.client.0.smithi107.stdout:8/360: dwrite d6/da/f1b [0,4194304] 0 2024-03-04T13:43:06.047 INFO:tasks.workunit.client.0.smithi107.stdout:6/393: getdents df/d17 0 2024-03-04T13:43:06.049 INFO:tasks.workunit.client.0.smithi107.stdout:2/333: sync 2024-03-04T13:43:06.053 INFO:tasks.workunit.client.0.smithi107.stdout:0/390: write da/dc/f4e [3759936,84708] 0 2024-03-04T13:43:06.057 INFO:tasks.workunit.client.0.smithi107.stdout:4/301: dwrite d0/d9/f11 [0,4194304] 0 2024-03-04T13:43:06.057 INFO:tasks.workunit.client.0.smithi107.stdout:5/288: dwrite d5/dc/f55 [0,4194304] 0 2024-03-04T13:43:06.060 INFO:tasks.workunit.client.0.smithi107.stdout:3/321: truncate d1/d2d/f5c 2823373 0 2024-03-04T13:43:06.063 INFO:tasks.workunit.client.0.smithi107.stdout:8/361: mknod d6/da/d25/d2a/c66 0 2024-03-04T13:43:06.064 INFO:tasks.workunit.client.0.smithi107.stdout:1/316: dwrite d1/d3/df/d33/d5d/d24/f62 [0,4194304] 0 2024-03-04T13:43:06.065 INFO:tasks.workunit.client.0.smithi107.stdout:5/289: chown d5/d65 203132 1 2024-03-04T13:43:06.065 INFO:tasks.workunit.client.0.smithi107.stdout:2/334: dread - da/f20 zero size 2024-03-04T13:43:06.066 INFO:tasks.workunit.client.0.smithi107.stdout:4/302: mkdir d0/d2f/d5f 0 2024-03-04T13:43:06.068 INFO:tasks.workunit.client.0.smithi107.stdout:3/322: fdatasync d1/de/d18/d1f/d42/f5d 0 2024-03-04T13:43:06.071 INFO:tasks.workunit.client.0.smithi107.stdout:6/394: rename df/d2e/d36/d71/c60 to df/d57/c7b 0 2024-03-04T13:43:06.074 INFO:tasks.workunit.client.0.smithi107.stdout:8/362: mkdir d6/d16/d59/d3e/d67 0 2024-03-04T13:43:06.080 INFO:tasks.workunit.client.0.smithi107.stdout:2/335: symlink da/d44/l6e 0 2024-03-04T13:43:06.082 INFO:tasks.workunit.client.0.smithi107.stdout:2/336: write da/df/d3e/f43 [567757,24961] 0 2024-03-04T13:43:06.082 INFO:tasks.workunit.client.0.smithi107.stdout:9/300: sync 2024-03-04T13:43:06.082 INFO:tasks.workunit.client.0.smithi107.stdout:4/303: unlink d0/d9/de/f3b 0 2024-03-04T13:43:06.084 INFO:tasks.workunit.client.0.smithi107.stdout:7/309: write f4 [458557,122335] 0 2024-03-04T13:43:06.086 INFO:tasks.workunit.client.0.smithi107.stdout:9/301: write d4/da/d11/d4c/d51/f5a [617886,68065] 0 2024-03-04T13:43:06.086 INFO:tasks.workunit.client.1.smithi169.stdout:4/818: creat d4/d1c/d75/ff4 x:0 0 0 2024-03-04T13:43:06.088 INFO:tasks.workunit.client.0.smithi107.stdout:7/310: chown d6/d13/d16/d19/d38/d2c 1559 1 2024-03-04T13:43:06.088 INFO:tasks.workunit.client.0.smithi107.stdout:6/395: read - df/d2e/d36/d5b/f6c zero size 2024-03-04T13:43:06.091 INFO:tasks.workunit.client.0.smithi107.stdout:2/337: dread da/d38/d3a/f60 [0,4194304] 0 2024-03-04T13:43:06.092 INFO:tasks.workunit.client.1.smithi169.stdout:7/875: read d2/f7 [2952520,52038] 0 2024-03-04T13:43:06.093 INFO:tasks.workunit.client.1.smithi169.stdout:9/884: link d7/d2d/f36 d7/dc/d3e/d125/f13f 0 2024-03-04T13:43:06.095 INFO:tasks.workunit.client.0.smithi107.stdout:1/317: creat d1/d3/df/d33/d43/d5b/d61/d64/f66 x:0 0 0 2024-03-04T13:43:06.096 INFO:tasks.workunit.client.1.smithi169.stdout:3/819: mknod d7/de/d2b/d7d/d85/d13/d5a/da1/c10d 0 2024-03-04T13:43:06.096 INFO:tasks.workunit.client.1.smithi169.stdout:6/820: dread d0/d32/d3a/d43/d74/f62 [0,4194304] 0 2024-03-04T13:43:06.099 INFO:tasks.workunit.client.0.smithi107.stdout:6/396: dread df/d17/f52 [0,4194304] 0 2024-03-04T13:43:06.103 INFO:tasks.workunit.client.1.smithi169.stdout:5/927: creat d1/d16/d2f/d72/db8/f14b x:0 0 0 2024-03-04T13:43:06.106 INFO:tasks.workunit.client.0.smithi107.stdout:7/311: dwrite d6/fa [0,4194304] 0 2024-03-04T13:43:06.107 INFO:tasks.workunit.client.1.smithi169.stdout:1/787: mkdir d4/d5/d36/d35/dff 0 2024-03-04T13:43:06.109 INFO:tasks.workunit.client.0.smithi107.stdout:2/338: dwrite da/d39/f46 [4194304,4194304] 0 2024-03-04T13:43:06.109 INFO:tasks.workunit.client.1.smithi169.stdout:1/788: readlink d4/d5/d2d/d4e/d71/d26/d7e/lf8 0 2024-03-04T13:43:06.109 INFO:tasks.workunit.client.0.smithi107.stdout:0/391: link da/dc/dd/d3e/d26/c41 da/dc/dd/d3e/d6e/d73/d78/c82 0 2024-03-04T13:43:06.112 INFO:tasks.workunit.client.0.smithi107.stdout:8/363: rename d6/l9 to d6/da/d25/d5f/l68 0 2024-03-04T13:43:06.113 INFO:tasks.workunit.client.0.smithi107.stdout:2/339: read da/f2e [3270403,52145] 0 2024-03-04T13:43:06.113 INFO:tasks.workunit.client.0.smithi107.stdout:1/318: mkdir d1/d3/df/d67 0 2024-03-04T13:43:06.114 INFO:tasks.workunit.client.1.smithi169.stdout:2/890: mkdir d1/d5/d19/d1e/d133 0 2024-03-04T13:43:06.118 INFO:tasks.workunit.client.1.smithi169.stdout:3/820: rmdir d7/de/d2b/d7d/d85/d13/d1a/d2d/d56/d5e 39 2024-03-04T13:43:06.119 INFO:tasks.workunit.client.0.smithi107.stdout:4/304: symlink d0/d2f/d5f/l60 0 2024-03-04T13:43:06.122 INFO:tasks.workunit.client.0.smithi107.stdout:5/290: write d5/d8/f11 [1853162,77331] 0 2024-03-04T13:43:06.124 INFO:tasks.workunit.client.0.smithi107.stdout:7/312: creat d6/d13/d16/f67 x:0 0 0 2024-03-04T13:43:06.124 INFO:tasks.workunit.client.0.smithi107.stdout:4/305: truncate d0/d9/de/f41 446984 0 2024-03-04T13:43:06.129 INFO:tasks.workunit.client.0.smithi107.stdout:4/306: chown d0/d9/de/d14/f4b 51 1 2024-03-04T13:43:06.130 INFO:tasks.workunit.client.1.smithi169.stdout:9/885: mkdir d7/d140 0 2024-03-04T13:43:06.130 INFO:tasks.workunit.client.1.smithi169.stdout:1/789: mknod d4/d5/d36/d35/d70/c100 0 2024-03-04T13:43:06.131 INFO:tasks.workunit.client.0.smithi107.stdout:6/397: rename df/d17/d24/d27/f26 to df/d57/f7c 0 2024-03-04T13:43:06.132 INFO:tasks.workunit.client.1.smithi169.stdout:2/891: truncate d1/d5/d19/d68/fe8 1843519 0 2024-03-04T13:43:06.133 INFO:tasks.workunit.client.0.smithi107.stdout:8/364: rmdir d6/da/d25/d5f 39 2024-03-04T13:43:06.134 INFO:tasks.workunit.client.1.smithi169.stdout:5/928: fsync d1/d76/f100 0 2024-03-04T13:43:06.136 INFO:tasks.workunit.client.1.smithi169.stdout:2/892: truncate d1/d5/d19/d3c/f104 4299875 0 2024-03-04T13:43:06.136 INFO:tasks.workunit.client.1.smithi169.stdout:0/711: getdents d5/d9/dc/d12/d15/d19/d61 0 2024-03-04T13:43:06.137 INFO:tasks.workunit.client.0.smithi107.stdout:2/340: symlink da/df/d27/d58/l6f 0 2024-03-04T13:43:06.137 INFO:tasks.workunit.client.1.smithi169.stdout:5/929: dread - d1/d16/d2f/d72/dab/f114 zero size 2024-03-04T13:43:06.141 INFO:tasks.workunit.client.0.smithi107.stdout:1/319: unlink d1/d3/df/d33/d5d/d24/l42 0 2024-03-04T13:43:06.141 INFO:tasks.workunit.client.0.smithi107.stdout:9/302: dread d4/da/d11/d4c/f3c [0,4194304] 0 2024-03-04T13:43:06.141 INFO:tasks.workunit.client.0.smithi107.stdout:5/291: dread d5/f9 [0,4194304] 0 2024-03-04T13:43:06.142 INFO:tasks.workunit.client.1.smithi169.stdout:0/712: fdatasync d5/d9/dc/d12/d15/d19/d61/d71/d8c/d90/fd2 0 2024-03-04T13:43:06.142 INFO:tasks.workunit.client.1.smithi169.stdout:5/930: truncate d1/d16/dbd/dbb/f121 956558 0 2024-03-04T13:43:06.142 INFO:tasks.workunit.client.0.smithi107.stdout:3/323: link d1/de/d11/d12/d4c/f57 d1/d1e/d26/f6f 0 2024-03-04T13:43:06.145 INFO:tasks.workunit.client.0.smithi107.stdout:7/313: fsync d6/d13/d16/d1c/f2d 0 2024-03-04T13:43:06.147 INFO:tasks.workunit.client.0.smithi107.stdout:7/314: readlink d6/d13/d16/d19/d38/d2c/l41 0 2024-03-04T13:43:06.148 INFO:tasks.workunit.client.1.smithi169.stdout:6/821: getdents d0/d3c/d40/df6/dca 0 2024-03-04T13:43:06.153 INFO:tasks.workunit.client.1.smithi169.stdout:0/713: rmdir d5/d9/d3d/d5d/de4/d98/dc1 39 2024-03-04T13:43:06.159 INFO:tasks.workunit.client.0.smithi107.stdout:6/398: dwrite df/d57/f7c [0,4194304] 0 2024-03-04T13:43:06.160 INFO:tasks.workunit.client.1.smithi169.stdout:0/714: chown d5/d9/d3d/d59/d64/d7f/fcc 1 1 2024-03-04T13:43:06.160 INFO:tasks.workunit.client.0.smithi107.stdout:1/320: dread d1/d3/df/f3a [0,4194304] 0 2024-03-04T13:43:06.161 INFO:tasks.workunit.client.0.smithi107.stdout:9/303: unlink d4/d32/l48 0 2024-03-04T13:43:06.162 INFO:tasks.workunit.client.0.smithi107.stdout:6/399: write df/d17/d24/d40/f44 [261625,45796] 0 2024-03-04T13:43:06.163 INFO:tasks.workunit.client.0.smithi107.stdout:1/321: write d1/d10/d16/f37 [4577248,15954] 0 2024-03-04T13:43:06.163 INFO:tasks.workunit.client.1.smithi169.stdout:2/893: dwrite d1/f30 [4194304,4194304] 0 2024-03-04T13:43:06.164 INFO:tasks.workunit.client.0.smithi107.stdout:5/292: mkdir d5/d8/d14/d5a/d69 0 2024-03-04T13:43:06.165 INFO:tasks.workunit.client.0.smithi107.stdout:6/400: write df/f29 [1187850,114659] 0 2024-03-04T13:43:06.167 INFO:tasks.workunit.client.1.smithi169.stdout:3/821: dread d7/de/d2b/d7d/d85/d13/d5a/da1/fa7 [0,4194304] 0 2024-03-04T13:43:06.167 INFO:tasks.workunit.client.0.smithi107.stdout:5/293: write d5/dc/f41 [5217514,17561] 0 2024-03-04T13:43:06.168 INFO:tasks.workunit.client.1.smithi169.stdout:2/894: chown d1/d5/d19/d1e/d29/f39 462 1 2024-03-04T13:43:06.168 INFO:tasks.workunit.client.0.smithi107.stdout:1/322: fdatasync d1/d3/df/d33/d43/d5b/f5f 0 2024-03-04T13:43:06.169 INFO:tasks.workunit.client.0.smithi107.stdout:3/324: creat d1/de/d18/d1f/d42/f70 x:0 0 0 2024-03-04T13:43:06.170 INFO:tasks.workunit.client.1.smithi169.stdout:3/822: readlink d7/de/d2b/d7d/l9e 0 2024-03-04T13:43:06.172 INFO:tasks.workunit.client.0.smithi107.stdout:1/323: write d1/d10/d16/d1b/d40/f57 [428742,86172] 0 2024-03-04T13:43:06.172 INFO:tasks.workunit.client.0.smithi107.stdout:6/401: dread df/d1b/f4b [0,4194304] 0 2024-03-04T13:43:06.173 INFO:tasks.workunit.client.1.smithi169.stdout:3/823: chown d7/de/d2b/d7d/d85/f6b 28936 1 2024-03-04T13:43:06.180 INFO:tasks.workunit.client.0.smithi107.stdout:0/392: rename da/l23 to da/dc/dd/d3e/d26/d69/d6b/l83 0 2024-03-04T13:43:06.180 INFO:tasks.workunit.client.0.smithi107.stdout:6/402: write df/d57/f64 [50643,115222] 0 2024-03-04T13:43:06.188 INFO:tasks.workunit.client.0.smithi107.stdout:8/365: mkdir d6/d16/d69 0 2024-03-04T13:43:06.189 INFO:tasks.workunit.client.1.smithi169.stdout:3/824: stat d7/de/d2b/d7d/d85/d13/d5a/d65/da4/cdf 0 2024-03-04T13:43:06.190 INFO:tasks.workunit.client.1.smithi169.stdout:7/876: dwrite d2/de/d3e/d41/d69/f92 [0,4194304] 0 2024-03-04T13:43:06.192 INFO:tasks.workunit.client.0.smithi107.stdout:5/294: dwrite d5/dc/df/f59 [0,4194304] 0 2024-03-04T13:43:06.193 INFO:tasks.workunit.client.0.smithi107.stdout:9/304: rmdir d4/da 39 2024-03-04T13:43:06.194 INFO:tasks.workunit.client.0.smithi107.stdout:2/341: dread da/f2e [0,4194304] 0 2024-03-04T13:43:06.196 INFO:tasks.workunit.client.1.smithi169.stdout:0/715: getdents d5/d9/d3d/d59/d64 0 2024-03-04T13:43:06.198 INFO:tasks.workunit.client.0.smithi107.stdout:2/342: write da/f10 [9195882,10329] 0 2024-03-04T13:43:06.199 INFO:tasks.workunit.client.1.smithi169.stdout:9/886: dwrite d7/fb [4194304,4194304] 0 2024-03-04T13:43:06.200 INFO:tasks.workunit.client.1.smithi169.stdout:6/822: write d0/d3c/fda [867640,85590] 0 2024-03-04T13:43:06.201 INFO:tasks.workunit.client.0.smithi107.stdout:7/315: dwrite d6/d13/d16/d1c/f35 [0,4194304] 0 2024-03-04T13:43:06.204 INFO:tasks.workunit.client.0.smithi107.stdout:7/316: stat d6/d13/d14 0 2024-03-04T13:43:06.206 INFO:tasks.workunit.client.0.smithi107.stdout:2/343: chown da/df/c11 6982104 1 2024-03-04T13:43:06.206 INFO:tasks.workunit.client.0.smithi107.stdout:7/317: chown d6/d13/d16/d19/d1e/d3d/f43 5345589 1 2024-03-04T13:43:06.208 INFO:tasks.workunit.client.1.smithi169.stdout:6/823: stat d0/d3c/d95/d9e/l10d 0 2024-03-04T13:43:06.208 INFO:tasks.workunit.client.1.smithi169.stdout:9/887: dread - d7/dc/d12/d32/d42/fe4 zero size 2024-03-04T13:43:06.209 INFO:tasks.workunit.client.1.smithi169.stdout:1/790: dwrite d4/d5/d2d/d4e/d71/d26/f6d [0,4194304] 0 2024-03-04T13:43:06.215 INFO:tasks.workunit.client.1.smithi169.stdout:1/791: fdatasync d4/d5/d2d/f74 0 2024-03-04T13:43:06.217 INFO:tasks.workunit.client.0.smithi107.stdout:3/325: write d1/d29/f2a [787849,22553] 0 2024-03-04T13:43:06.221 INFO:tasks.workunit.client.0.smithi107.stdout:3/326: truncate d1/de/d18/d1f/f43 57698 0 2024-03-04T13:43:06.221 INFO:tasks.workunit.client.0.smithi107.stdout:4/307: rename d0/d9/d18/d29/c51 to d0/d9/de/d14/dc/d21/c61 0 2024-03-04T13:43:06.222 INFO:tasks.workunit.client.1.smithi169.stdout:2/895: write d1/d5/d19/d3c/db8/de1/d64/fb4 [220402,13085] 0 2024-03-04T13:43:06.222 INFO:tasks.workunit.client.1.smithi169.stdout:1/792: chown d4/d5/d36/d35/d70/le6 14494734 1 2024-03-04T13:43:06.223 INFO:tasks.workunit.client.0.smithi107.stdout:3/327: chown d1/d1e/d26/c55 121330 1 2024-03-04T13:43:06.223 INFO:tasks.workunit.client.1.smithi169.stdout:5/931: dwrite d1/d16/d107/f122 [0,4194304] 0 2024-03-04T13:43:06.225 INFO:tasks.workunit.client.0.smithi107.stdout:3/328: dread - d1/de/d18/d33/d4d/f5b zero size 2024-03-04T13:43:06.228 INFO:tasks.workunit.client.0.smithi107.stdout:8/366: symlink d6/d16/d1f/d4c/l6a 0 2024-03-04T13:43:06.228 INFO:tasks.workunit.client.1.smithi169.stdout:1/793: chown d4/d5/d36/df1 81646 1 2024-03-04T13:43:06.230 INFO:tasks.workunit.client.1.smithi169.stdout:7/877: write d2/de/d3e/d7e/fe2 [5204715,33103] 0 2024-03-04T13:43:06.231 INFO:tasks.workunit.client.1.smithi169.stdout:5/932: stat d1/d16/d17/d1b 0 2024-03-04T13:43:06.237 INFO:tasks.workunit.client.0.smithi107.stdout:1/324: truncate d1/d3/df/d33/d5d/d24/f44 977012 0 2024-03-04T13:43:06.241 INFO:tasks.workunit.client.1.smithi169.stdout:2/896: unlink d1/d5/d22/d3b/d99/f93 0 2024-03-04T13:43:06.242 INFO:tasks.workunit.client.0.smithi107.stdout:9/305: truncate d4/da/d11/f2b 1974623 0 2024-03-04T13:43:06.244 INFO:tasks.workunit.client.0.smithi107.stdout:9/306: rename d4 to d4/da/d11/d4c/d4d/d60 22 2024-03-04T13:43:06.249 INFO:tasks.workunit.client.1.smithi169.stdout:5/933: read d1/d60/f127 [3057261,28630] 0 2024-03-04T13:43:06.250 INFO:tasks.workunit.client.1.smithi169.stdout:1/794: dread - d4/d5/d2d/d4e/d71/d26/d4f/fb3 zero size 2024-03-04T13:43:06.253 INFO:tasks.workunit.client.0.smithi107.stdout:2/344: mknod da/d44/c70 0 2024-03-04T13:43:06.253 INFO:tasks.workunit.client.1.smithi169.stdout:6/824: dread d0/f1 [0,4194304] 0 2024-03-04T13:43:06.254 INFO:tasks.workunit.client.0.smithi107.stdout:0/393: write da/dc/dd/f1f [656060,13601] 0 2024-03-04T13:43:06.255 INFO:tasks.workunit.client.0.smithi107.stdout:7/318: mknod d6/d13/d16/d19/d39/c68 0 2024-03-04T13:43:06.258 INFO:tasks.workunit.client.1.smithi169.stdout:9/888: dwrite d7/dc/f80 [4194304,4194304] 0 2024-03-04T13:43:06.259 INFO:tasks.workunit.client.0.smithi107.stdout:9/307: dwrite d4/f19 [0,4194304] 0 2024-03-04T13:43:06.261 INFO:tasks.workunit.client.1.smithi169.stdout:0/716: write d5/d9/dc/d12/d15/d19/d61/f54 [3062398,49123] 0 2024-03-04T13:43:06.262 INFO:tasks.workunit.client.1.smithi169.stdout:7/878: symlink d2/d15/d17/d6f/d126/l12c 0 2024-03-04T13:43:06.263 INFO:tasks.workunit.client.0.smithi107.stdout:4/308: unlink d0/d9/de/d14/dc/f52 0 2024-03-04T13:43:06.264 INFO:tasks.workunit.client.0.smithi107.stdout:4/309: chown d0/d2f/d5f 129728819 1 2024-03-04T13:43:06.265 INFO:tasks.workunit.client.1.smithi169.stdout:6/825: mkdir d0/d32/d3a/d43/d74/d93/dbe/d129 0 2024-03-04T13:43:06.265 INFO:tasks.workunit.client.1.smithi169.stdout:0/717: readlink d5/d9/d3d/d59/la7 0 2024-03-04T13:43:06.266 INFO:tasks.workunit.client.0.smithi107.stdout:4/310: write d0/d30/f50 [1660676,115330] 0 2024-03-04T13:43:06.266 INFO:tasks.workunit.client.0.smithi107.stdout:8/367: chown d6/f5b 44651140 1 2024-03-04T13:43:06.268 INFO:tasks.workunit.client.1.smithi169.stdout:2/897: fsync d1/d5/d22/f94 0 2024-03-04T13:43:06.269 INFO:tasks.workunit.client.1.smithi169.stdout:7/879: write d2/de/d3e/d41/f4c [1223734,93140] 0 2024-03-04T13:43:06.271 INFO:tasks.workunit.client.0.smithi107.stdout:2/345: mknod da/d34/c71 0 2024-03-04T13:43:06.271 INFO:tasks.workunit.client.1.smithi169.stdout:1/795: dread d4/d5/d36/f88 [0,4194304] 0 2024-03-04T13:43:06.272 INFO:tasks.workunit.client.0.smithi107.stdout:0/394: mknod da/dc/dd/d3e/d26/d69/c84 0 2024-03-04T13:43:06.273 INFO:tasks.workunit.client.1.smithi169.stdout:9/889: mknod d7/dc/d95/c141 0 2024-03-04T13:43:06.273 INFO:tasks.workunit.client.1.smithi169.stdout:5/934: creat d1/d16/d107/d137/f14c x:0 0 0 2024-03-04T13:43:06.274 INFO:tasks.workunit.client.0.smithi107.stdout:0/395: fdatasync da/dc/dd/f1f 0 2024-03-04T13:43:06.274 INFO:tasks.workunit.client.1.smithi169.stdout:0/718: stat d5/d9/dc/d12/d66/lc5 0 2024-03-04T13:43:06.275 INFO:tasks.workunit.client.1.smithi169.stdout:7/880: truncate d2/d28/d70/ded/f107 841038 0 2024-03-04T13:43:06.275 INFO:tasks.workunit.client.0.smithi107.stdout:7/319: truncate d6/d1d/f3b 3296751 0 2024-03-04T13:43:06.279 INFO:tasks.workunit.client.1.smithi169.stdout:0/719: truncate d5/ff5 207297 0 2024-03-04T13:43:06.279 INFO:tasks.workunit.client.1.smithi169.stdout:2/898: mkdir d1/d5/d22/d3b/d41/d134 0 2024-03-04T13:43:06.280 INFO:tasks.workunit.client.1.smithi169.stdout:1/796: creat d4/d5/d36/d75/dba/f101 x:0 0 0 2024-03-04T13:43:06.281 INFO:tasks.workunit.client.1.smithi169.stdout:0/720: write d5/d9/dc/d12/d15/d19/d77/f9b [2830711,69180] 0 2024-03-04T13:43:06.286 INFO:tasks.workunit.client.0.smithi107.stdout:8/368: fdatasync f5 0 2024-03-04T13:43:06.287 INFO:tasks.workunit.client.0.smithi107.stdout:8/369: stat d6/da/f2c 0 2024-03-04T13:43:06.288 INFO:tasks.workunit.client.0.smithi107.stdout:2/346: creat da/d34/f72 x:0 0 0 2024-03-04T13:43:06.289 INFO:tasks.workunit.client.0.smithi107.stdout:2/347: chown da/f26 23257042 1 2024-03-04T13:43:06.291 INFO:tasks.workunit.client.1.smithi169.stdout:4/819: sync 2024-03-04T13:43:06.291 INFO:tasks.workunit.client.1.smithi169.stdout:8/840: sync 2024-03-04T13:43:06.291 INFO:tasks.workunit.client.1.smithi169.stdout:0/721: mknod d5/d4c/cf6 0 2024-03-04T13:43:06.292 INFO:tasks.workunit.client.0.smithi107.stdout:0/396: rename da/dc/dd/d1e/f5f to da/dc/dd/d1e/f85 0 2024-03-04T13:43:06.292 INFO:tasks.workunit.client.0.smithi107.stdout:2/348: dread da/f30 [0,4194304] 0 2024-03-04T13:43:06.294 INFO:tasks.workunit.client.0.smithi107.stdout:0/397: chown da/dc/dd/d3e/d26/f64 0 1 2024-03-04T13:43:06.295 INFO:tasks.workunit.client.0.smithi107.stdout:0/398: readlink da/dc/dd/d1e/l60 0 2024-03-04T13:43:06.295 INFO:tasks.workunit.client.1.smithi169.stdout:3/825: dread d7/f70 [0,4194304] 0 2024-03-04T13:43:06.296 INFO:tasks.workunit.client.0.smithi107.stdout:3/329: rmdir d1/de/d11/d12/d4c/d54/d66 0 2024-03-04T13:43:06.296 INFO:tasks.workunit.client.0.smithi107.stdout:0/399: write da/dc/f4d [55806,102864] 0 2024-03-04T13:43:06.297 INFO:tasks.workunit.client.0.smithi107.stdout:0/400: readlink da/dc/l30 0 2024-03-04T13:43:06.297 INFO:tasks.workunit.client.1.smithi169.stdout:4/820: chown d4/d11/d3b/d67/dbb/f76 2 1 2024-03-04T13:43:06.298 INFO:tasks.workunit.client.0.smithi107.stdout:7/320: truncate d6/d13/d16/d19/d38/f47 2075191 0 2024-03-04T13:43:06.299 INFO:tasks.workunit.client.0.smithi107.stdout:5/295: getdents d5/d8/d14/d2d 0 2024-03-04T13:43:06.302 INFO:tasks.workunit.client.0.smithi107.stdout:8/370: readlink d6/da/l42 0 2024-03-04T13:43:06.302 INFO:tasks.workunit.client.0.smithi107.stdout:8/371: write d6/f39 [3396674,66391] 0 2024-03-04T13:43:06.306 INFO:tasks.workunit.client.1.smithi169.stdout:6/826: dwrite d0/d32/d3e/d122/ffc [0,4194304] 0 2024-03-04T13:43:06.310 INFO:tasks.workunit.client.1.smithi169.stdout:8/841: rename d7/d10/d17/d44/d36/d62/dc1/fe2 to d7/d10/d22/d75/de9/f10e 0 2024-03-04T13:43:06.311 INFO:tasks.workunit.client.0.smithi107.stdout:2/349: mkdir da/df/d73 0 2024-03-04T13:43:06.312 INFO:tasks.workunit.client.1.smithi169.stdout:0/722: dread - d5/d9/d3d/d5d/de4/d98/dc1/dc8/fee zero size 2024-03-04T13:43:06.313 INFO:tasks.workunit.client.0.smithi107.stdout:5/296: dwrite d5/d8/d2a/f40 [0,4194304] 0 2024-03-04T13:43:06.319 INFO:tasks.workunit.client.0.smithi107.stdout:5/297: read d5/d8/d2a/f53 [3286415,112481] 0 2024-03-04T13:43:06.322 INFO:tasks.workunit.client.0.smithi107.stdout:4/311: creat d0/d9/de/f62 x:0 0 0 2024-03-04T13:43:06.324 INFO:tasks.workunit.client.1.smithi169.stdout:1/797: dread d4/d5/fc4 [0,4194304] 0 2024-03-04T13:43:06.325 INFO:tasks.workunit.client.1.smithi169.stdout:4/821: mkdir d4/d11/d2e/d6c/d78/da5/d77/dc0/df5 0 2024-03-04T13:43:06.325 INFO:tasks.workunit.client.0.smithi107.stdout:4/312: write d0/d9/d18/d29/f35 [2015870,74528] 0 2024-03-04T13:43:06.330 INFO:tasks.workunit.client.0.smithi107.stdout:0/401: dwrite da/dc/f7c [0,4194304] 0 2024-03-04T13:43:06.338 INFO:tasks.workunit.client.1.smithi169.stdout:8/842: dwrite d7/d10/d22/f30 [0,4194304] 0 2024-03-04T13:43:06.342 INFO:tasks.workunit.client.1.smithi169.stdout:2/899: creat d1/d5/d19/d3c/db8/de1/d64/dce/dd5/df5/f135 x:0 0 0 2024-03-04T13:43:06.342 INFO:tasks.workunit.client.1.smithi169.stdout:3/826: dwrite d7/de/d2b/d7d/d85/d13/d1a/d98/d9a/ff4 [0,4194304] 0 2024-03-04T13:43:06.343 INFO:tasks.workunit.client.1.smithi169.stdout:9/890: write d7/d2d/f85 [42855,106734] 0 2024-03-04T13:43:06.345 INFO:tasks.workunit.client.0.smithi107.stdout:0/402: dwrite da/dc/d34/f5a [0,4194304] 0 2024-03-04T13:43:06.347 INFO:tasks.workunit.client.0.smithi107.stdout:5/298: mkdir d5/d8/d14/d5a/d6a 0 2024-03-04T13:43:06.348 INFO:tasks.workunit.client.1.smithi169.stdout:7/881: getdents d2/d15/d17/d6f/d126 0 2024-03-04T13:43:06.348 INFO:tasks.workunit.client.1.smithi169.stdout:2/900: fdatasync d1/d5/d22/d3b/ffa 0 2024-03-04T13:43:06.355 INFO:tasks.workunit.client.1.smithi169.stdout:7/882: dread - d2/de/d3e/d41/d5b/d7a/f121 zero size 2024-03-04T13:43:06.357 INFO:tasks.workunit.client.1.smithi169.stdout:5/935: truncate d1/d16/d17/d28/f4c 4258384 0 2024-03-04T13:43:06.360 INFO:tasks.workunit.client.0.smithi107.stdout:3/330: rename d1/de/d11/d12 to d1/de/d18/d33/d48/d71 0 2024-03-04T13:43:06.361 INFO:tasks.workunit.client.1.smithi169.stdout:9/891: rmdir d7/d25/d64 39 2024-03-04T13:43:06.369 INFO:tasks.workunit.client.1.smithi169.stdout:4/822: dwrite d4/fa [8388608,4194304] 0 2024-03-04T13:43:06.369 INFO:tasks.workunit.client.1.smithi169.stdout:8/843: dread d7/d10/d22/d75/d81/da2/fbf [0,4194304] 0 2024-03-04T13:43:06.370 INFO:tasks.workunit.client.0.smithi107.stdout:6/403: sync 2024-03-04T13:43:06.370 INFO:tasks.workunit.client.0.smithi107.stdout:1/325: sync 2024-03-04T13:43:06.370 INFO:tasks.workunit.client.0.smithi107.stdout:9/308: dwrite d4/ff [0,4194304] 0 2024-03-04T13:43:06.372 INFO:tasks.workunit.client.1.smithi169.stdout:8/844: fsync d7/d10/d24/d3e/f68 0 2024-03-04T13:43:06.373 INFO:tasks.workunit.client.0.smithi107.stdout:1/326: truncate d1/d3/df/f32 4571213 0 2024-03-04T13:43:06.373 INFO:tasks.workunit.client.0.smithi107.stdout:9/309: chown d4/da/d11/d4c/d51/f5c 36987 1 2024-03-04T13:43:06.374 INFO:tasks.workunit.client.1.smithi169.stdout:4/823: truncate d4/d11/d2e/d39/f60 13446897 0 2024-03-04T13:43:06.375 INFO:tasks.workunit.client.0.smithi107.stdout:0/403: fsync da/f2a 0 2024-03-04T13:43:06.375 INFO:tasks.workunit.client.0.smithi107.stdout:3/331: dread d1/de/d18/f59 [0,4194304] 0 2024-03-04T13:43:06.375 INFO:tasks.workunit.client.1.smithi169.stdout:6/827: write d0/d32/f89 [695300,10313] 0 2024-03-04T13:43:06.376 INFO:tasks.workunit.client.0.smithi107.stdout:1/327: chown d1/d3/d4e/c54 3425 1 2024-03-04T13:43:06.376 INFO:tasks.workunit.client.0.smithi107.stdout:9/310: read - d4/f45 zero size 2024-03-04T13:43:06.376 INFO:tasks.workunit.client.0.smithi107.stdout:7/321: write d6/d13/d16/d19/d38/d2c/f49 [561814,127323] 0 2024-03-04T13:43:06.377 INFO:tasks.workunit.client.0.smithi107.stdout:0/404: chown da/dc/dd/d3e/d26/f67 360209 1 2024-03-04T13:43:06.380 INFO:tasks.workunit.client.0.smithi107.stdout:1/328: dread - d1/d3/d4e/d60/f63 zero size 2024-03-04T13:43:06.380 INFO:tasks.workunit.client.0.smithi107.stdout:2/350: write da/df/f24 [828893,16238] 0 2024-03-04T13:43:06.380 INFO:tasks.workunit.client.0.smithi107.stdout:9/311: fsync d4/da/d11/d20/f56 0 2024-03-04T13:43:06.384 INFO:tasks.workunit.client.0.smithi107.stdout:8/372: rename d6/d16/l3c to d6/d16/d69/l6b 0 2024-03-04T13:43:06.386 INFO:tasks.workunit.client.0.smithi107.stdout:7/322: write d6/d13/d16/d19/d39/f54 [891187,13406] 0 2024-03-04T13:43:06.386 INFO:tasks.workunit.client.0.smithi107.stdout:4/313: rename d0 to d0/d2f/d5f/d63 22 2024-03-04T13:43:06.387 INFO:tasks.workunit.client.0.smithi107.stdout:8/373: fdatasync d6/d16/f22 0 2024-03-04T13:43:06.389 INFO:tasks.workunit.client.1.smithi169.stdout:0/723: creat d5/d9/dc/d12/d15/d19/d77/ff7 x:0 0 0 2024-03-04T13:43:06.391 INFO:tasks.workunit.client.1.smithi169.stdout:5/936: dwrite d1/d16/d2a/d46/f6c [0,4194304] 0 2024-03-04T13:43:06.397 INFO:tasks.workunit.client.0.smithi107.stdout:6/404: mkdir df/d2e/d36/d5b/d7d 0 2024-03-04T13:43:06.397 INFO:tasks.workunit.client.1.smithi169.stdout:9/892: creat d7/d27/d84/f142 x:0 0 0 2024-03-04T13:43:06.402 INFO:tasks.workunit.client.0.smithi107.stdout:9/312: dwrite d4/ff [0,4194304] 0 2024-03-04T13:43:06.405 INFO:tasks.workunit.client.1.smithi169.stdout:3/827: mknod d7/de/d2b/d7d/d85/d13/d1a/d2d/dbe/d108/dde/c10e 0 2024-03-04T13:43:06.407 INFO:tasks.workunit.client.0.smithi107.stdout:8/374: dwrite f3 [4194304,4194304] 0 2024-03-04T13:43:06.413 INFO:tasks.workunit.client.1.smithi169.stdout:6/828: fdatasync d0/d3c/d40/f6f 0 2024-03-04T13:43:06.415 INFO:tasks.workunit.client.1.smithi169.stdout:6/829: chown d0/d32/d3e/l6c 178031353 1 2024-03-04T13:43:06.420 INFO:tasks.workunit.client.0.smithi107.stdout:1/329: symlink d1/d10/d16/d1b/d40/l68 0 2024-03-04T13:43:06.422 INFO:tasks.workunit.client.0.smithi107.stdout:1/330: fdatasync d1/d3/df/f32 0 2024-03-04T13:43:06.424 INFO:tasks.workunit.client.0.smithi107.stdout:2/351: dread da/d29/f2d [0,4194304] 0 2024-03-04T13:43:06.424 INFO:tasks.workunit.client.1.smithi169.stdout:1/798: rename d4/d5/d2d/d4e/d71/l18 to d4/l102 0 2024-03-04T13:43:06.424 INFO:tasks.workunit.client.1.smithi169.stdout:4/824: truncate d4/d11/d2e/d39/d48/f65 1230108 0 2024-03-04T13:43:06.427 INFO:tasks.workunit.client.1.smithi169.stdout:5/937: creat d1/d81/f14d x:0 0 0 2024-03-04T13:43:06.428 INFO:tasks.workunit.client.0.smithi107.stdout:1/331: read d1/d3/df/f32 [651526,104907] 0 2024-03-04T13:43:06.430 INFO:tasks.workunit.client.1.smithi169.stdout:9/893: dread - d7/dc/d95/fb6 zero size 2024-03-04T13:43:06.434 INFO:tasks.workunit.client.0.smithi107.stdout:7/323: symlink d6/d13/d16/d19/d38/d37/l69 0 2024-03-04T13:43:06.436 INFO:tasks.workunit.client.0.smithi107.stdout:4/314: rename d0/d2f/f5a to d0/d30/f64 0 2024-03-04T13:43:06.439 INFO:tasks.workunit.client.0.smithi107.stdout:9/313: symlink d4/d32/l61 0 2024-03-04T13:43:06.441 INFO:tasks.workunit.client.0.smithi107.stdout:1/332: dwrite d1/d3/d17/f5a [0,4194304] 0 2024-03-04T13:43:06.443 INFO:tasks.workunit.client.0.smithi107.stdout:5/299: truncate d5/dc/f41 3796067 0 2024-03-04T13:43:06.444 INFO:tasks.workunit.client.1.smithi169.stdout:1/799: mkdir d4/d5/d2d/d4e/da5/d103 0 2024-03-04T13:43:06.445 INFO:tasks.workunit.client.0.smithi107.stdout:5/300: chown d5/dc/df/f35 1478040 1 2024-03-04T13:43:06.447 INFO:tasks.workunit.client.0.smithi107.stdout:0/405: mkdir da/dc/dd/d3e/d6e/d73/d78/d86 0 2024-03-04T13:43:06.450 INFO:tasks.workunit.client.0.smithi107.stdout:2/352: symlink da/d34/l74 0 2024-03-04T13:43:06.451 INFO:tasks.workunit.client.1.smithi169.stdout:2/901: write d1/d5/d19/d3c/db8/de1/f4b [2446423,95068] 0 2024-03-04T13:43:06.451 INFO:tasks.workunit.client.0.smithi107.stdout:2/353: chown da/d49/l4f 1 1 2024-03-04T13:43:06.452 INFO:tasks.workunit.client.1.smithi169.stdout:8/845: creat d7/d10/f10f x:0 0 0 2024-03-04T13:43:06.456 INFO:tasks.workunit.client.1.smithi169.stdout:7/883: dwrite d2/d6b/f76 [0,4194304] 0 2024-03-04T13:43:06.464 INFO:tasks.workunit.client.1.smithi169.stdout:4/825: mknod d4/d11/d3b/cf6 0 2024-03-04T13:43:06.470 INFO:tasks.workunit.client.0.smithi107.stdout:9/314: readlink d4/da/l2e 0 2024-03-04T13:43:06.473 INFO:tasks.workunit.client.1.smithi169.stdout:3/828: dwrite d7/f33 [0,4194304] 0 2024-03-04T13:43:06.474 INFO:tasks.workunit.client.1.smithi169.stdout:1/800: mkdir d4/d5/d36/d35/d16/d81/db4/d104 0 2024-03-04T13:43:06.477 INFO:tasks.workunit.client.1.smithi169.stdout:1/801: stat d4/d5/d36/d35/d16/d81/db4/dd7/cdb 0 2024-03-04T13:43:06.477 INFO:tasks.workunit.client.0.smithi107.stdout:8/375: mknod d6/da/d25/d5f/c6c 0 2024-03-04T13:43:06.486 INFO:tasks.workunit.client.0.smithi107.stdout:5/301: creat d5/d8/d14/d2d/f6b x:0 0 0 2024-03-04T13:43:06.488 INFO:tasks.workunit.client.1.smithi169.stdout:9/894: write d7/d2d/d48/d66/f75 [724224,34427] 0 2024-03-04T13:43:06.489 INFO:tasks.workunit.client.1.smithi169.stdout:7/884: dwrite d2/de/d3e/d7e/fdd [0,4194304] 0 2024-03-04T13:43:06.493 INFO:tasks.workunit.client.0.smithi107.stdout:2/354: rename da/df/c1a to da/d38/d3a/c75 0 2024-03-04T13:43:06.495 INFO:tasks.workunit.client.0.smithi107.stdout:7/324: write d6/d1d/d36/f61 [1241454,23552] 0 2024-03-04T13:43:06.499 INFO:tasks.workunit.client.1.smithi169.stdout:4/826: rename d4/d1c/d75/d81/f85 to d4/d11/d2e/d21/da9/ff7 0 2024-03-04T13:43:06.499 INFO:tasks.workunit.client.1.smithi169.stdout:8/846: dread d7/d10/d17/d44/d4c/f7d [0,4194304] 0 2024-03-04T13:43:06.501 INFO:tasks.workunit.client.0.smithi107.stdout:7/325: chown d6/d13/d16/d19/d1e/d3d 2414966 1 2024-03-04T13:43:06.503 INFO:tasks.workunit.client.0.smithi107.stdout:5/302: dwrite d5/dc/d22/f61 [0,4194304] 0 2024-03-04T13:43:06.504 INFO:tasks.workunit.client.1.smithi169.stdout:5/938: write d1/f8c [3589344,45315] 0 2024-03-04T13:43:06.504 INFO:tasks.workunit.client.0.smithi107.stdout:6/405: write df/d17/d24/d40/f62 [555844,95626] 0 2024-03-04T13:43:06.507 INFO:tasks.workunit.client.0.smithi107.stdout:7/326: write d6/d13/d16/d19/d39/f4e [983493,69793] 0 2024-03-04T13:43:06.509 INFO:tasks.workunit.client.0.smithi107.stdout:5/303: read d5/dc/f55 [1115775,106541] 0 2024-03-04T13:43:06.509 INFO:tasks.workunit.client.0.smithi107.stdout:6/406: fdatasync df/d2e/f5a 0 2024-03-04T13:43:06.510 INFO:tasks.workunit.client.1.smithi169.stdout:6/830: getdents d0/d3c/d40 0 2024-03-04T13:43:06.510 INFO:tasks.workunit.client.0.smithi107.stdout:7/327: fdatasync f0 0 2024-03-04T13:43:06.512 INFO:tasks.workunit.client.0.smithi107.stdout:1/333: creat d1/d3/df/d67/f69 x:0 0 0 2024-03-04T13:43:06.515 INFO:tasks.workunit.client.0.smithi107.stdout:5/304: write d5/f42 [363795,89644] 0 2024-03-04T13:43:06.515 INFO:tasks.workunit.client.0.smithi107.stdout:6/407: write df/d17/d24/d40/f62 [596276,47788] 0 2024-03-04T13:43:06.518 INFO:tasks.workunit.client.1.smithi169.stdout:9/895: write d7/dc/d12/d59/f6d [9092924,60400] 0 2024-03-04T13:43:06.519 INFO:tasks.workunit.client.0.smithi107.stdout:8/376: dread f3 [0,4194304] 0 2024-03-04T13:43:06.520 INFO:tasks.workunit.client.1.smithi169.stdout:9/896: readlink d7/lf6 0 2024-03-04T13:43:06.520 INFO:tasks.workunit.client.0.smithi107.stdout:5/305: write d5/dc/df/f12 [3741765,46226] 0 2024-03-04T13:43:06.522 INFO:tasks.workunit.client.1.smithi169.stdout:7/885: creat d2/d15/d17/f12d x:0 0 0 2024-03-04T13:43:06.526 INFO:tasks.workunit.client.1.smithi169.stdout:2/902: write d1/d5/d19/d3c/db8/de1/fe6 [842632,55273] 0 2024-03-04T13:43:06.530 INFO:tasks.workunit.client.1.smithi169.stdout:8/847: mknod d7/dd4/c110 0 2024-03-04T13:43:06.532 INFO:tasks.workunit.client.1.smithi169.stdout:1/802: creat d4/d5/d2d/d4e/d71/d82/df7/f105 x:0 0 0 2024-03-04T13:43:06.536 INFO:tasks.workunit.client.1.smithi169.stdout:5/939: creat d1/d81/db1/f14e x:0 0 0 2024-03-04T13:43:06.541 INFO:tasks.workunit.client.1.smithi169.stdout:6/831: creat d0/d32/d3a/d43/d74/d93/dbe/f12a x:0 0 0 2024-03-04T13:43:06.550 INFO:tasks.workunit.client.0.smithi107.stdout:3/332: sync 2024-03-04T13:43:06.551 INFO:tasks.workunit.client.0.smithi107.stdout:1/334: rename d1/d3 to d1/d46/d6a 0 2024-03-04T13:43:06.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:06.555+0000 7fb81a600700 1 -- 172.21.15.107:0/698306734 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8140fcef0 msgr2=0x7fb8140fd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:43:06.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:06.555+0000 7fb81a600700 1 --2- 172.21.15.107:0/698306734 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8140fcef0 0x7fb8140fd320 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb804004970 tx=0x7fb804009e30 comp rx=0 tx=0).stop 2024-03-04T13:43:06.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:06.555+0000 7fb81a600700 1 -- 172.21.15.107:0/698306734 shutdown_connections 2024-03-04T13:43:06.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:06.555+0000 7fb81a600700 1 --2- 172.21.15.107:0/698306734 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8140fe0e0 0x7fb8140fe550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:43:06.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:06.555+0000 7fb81a600700 1 --2- 172.21.15.107:0/698306734 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8140fcef0 0x7fb8140fd320 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:43:06.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:06.555+0000 7fb81a600700 1 -- 172.21.15.107:0/698306734 >> 172.21.15.107:0/698306734 conn(0x7fb8140f8620 msgr2=0x7fb8140faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:43:06.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:06.555+0000 7fb81a600700 1 -- 172.21.15.107:0/698306734 shutdown_connections 2024-03-04T13:43:06.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:06.555+0000 7fb81a600700 1 -- 172.21.15.107:0/698306734 wait complete. 2024-03-04T13:43:06.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:06.556+0000 7fb81a600700 1 Processor -- start 2024-03-04T13:43:06.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:06.556+0000 7fb81a600700 1 -- start start 2024-03-04T13:43:06.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:06.557+0000 7fb81a600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8140fcef0 0x7fb814113f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:43:06.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:06.557+0000 7fb81a600700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8140fe0e0 0x7fb814114450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:43:06.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:06.557+0000 7fb81a600700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb814114a50 con 0x7fb8140fcef0 2024-03-04T13:43:06.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:06.557+0000 7fb81a600700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8141152f0 con 0x7fb8140fe0e0 2024-03-04T13:43:06.556 INFO:tasks.workunit.client.1.smithi169.stdout:8/848: mknod d7/d10/d22/d75/d81/da4/c111 0 2024-03-04T13:43:06.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:06.557+0000 7fb813400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8140fcef0 0x7fb814113f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:43:06.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:06.557+0000 7fb812a00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8140fe0e0 0x7fb814114450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:43:06.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:06.557+0000 7fb812a00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8140fe0e0 0x7fb814114450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.107:50042/0 (socket says 172.21.15.107:50042) 2024-03-04T13:43:06.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:06.557+0000 7fb813400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8140fcef0 0x7fb814113f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51986/0 (socket says 172.21.15.107:51986) 2024-03-04T13:43:06.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:06.557+0000 7fb812a00700 1 -- 172.21.15.107:0/736352979 learned_addr learned my addr 172.21.15.107:0/736352979 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:43:06.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:06.558+0000 7fb813400700 1 -- 172.21.15.107:0/736352979 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8140fe0e0 msgr2=0x7fb814114450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:43:06.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:06.558+0000 7fb813400700 1 --2- 172.21.15.107:0/736352979 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8140fe0e0 0x7fb814114450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:43:06.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:06.558+0000 7fb813400700 1 -- 172.21.15.107:0/736352979 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb804005040 con 0x7fb8140fcef0 2024-03-04T13:43:06.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:06.558+0000 7fb813400700 1 --2- 172.21.15.107:0/736352979 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8140fcef0 0x7fb814113f10 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fb80400b370 tx=0x7fb804009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:43:06.557 INFO:tasks.workunit.client.0.smithi107.stdout:3/333: dread d1/de/d18/d33/f50 [0,4194304] 0 2024-03-04T13:43:06.557 INFO:tasks.workunit.client.0.smithi107.stdout:9/315: write d4/f8 [6093636,16956] 0 2024-03-04T13:43:06.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:06.558+0000 7fb80b400700 1 -- 172.21.15.107:0/736352979 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb80400e040 con 0x7fb8140fcef0 2024-03-04T13:43:06.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:06.558+0000 7fb80b400700 1 -- 172.21.15.107:0/736352979 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb804013420 con 0x7fb8140fcef0 2024-03-04T13:43:06.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:06.558+0000 7fb80b400700 1 -- 172.21.15.107:0/736352979 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb80401c630 con 0x7fb8140fcef0 2024-03-04T13:43:06.558 INFO:tasks.workunit.client.0.smithi107.stdout:6/408: mkdir df/d1b/d2c/d46/d7e 0 2024-03-04T13:43:06.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:06.559+0000 7fb81a600700 1 -- 172.21.15.107:0/736352979 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb814115570 con 0x7fb8140fcef0 2024-03-04T13:43:06.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:06.559+0000 7fb81a600700 1 -- 172.21.15.107:0/736352979 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb814115ab0 con 0x7fb8140fcef0 2024-03-04T13:43:06.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:06.561+0000 7fb81a600700 1 -- 172.21.15.107:0/736352979 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb81404bed0 con 0x7fb8140fcef0 2024-03-04T13:43:06.559 INFO:tasks.workunit.client.1.smithi169.stdout:9/897: dwrite d7/dc/d12/d20/d4e/f109 [0,4194304] 0 2024-03-04T13:43:06.560 INFO:tasks.workunit.client.0.smithi107.stdout:3/334: chown d1/d1e/d26/c55 56168 1 2024-03-04T13:43:06.560 INFO:tasks.workunit.client.0.smithi107.stdout:4/315: getdents d0/d30 0 2024-03-04T13:43:06.560 INFO:tasks.workunit.client.0.smithi107.stdout:6/409: write df/d17/d5c/f67 [329691,102888] 0 2024-03-04T13:43:06.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:06.563+0000 7fb80b400700 1 -- 172.21.15.107:0/736352979 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) v1 ==== 88911+0+0 (secure 0 0 0) 0x7fb804013a70 con 0x7fb8140fcef0 2024-03-04T13:43:06.564 INFO:tasks.workunit.client.0.smithi107.stdout:8/377: creat d6/d16/d59/d3e/f6d x:0 0 0 2024-03-04T13:43:06.566 INFO:tasks.workunit.client.1.smithi169.stdout:6/832: truncate d0/d32/fe8 509923 0 2024-03-04T13:43:06.567 INFO:tasks.workunit.client.1.smithi169.stdout:7/886: symlink d2/de/dfb/d12a/l12e 0 2024-03-04T13:43:06.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:06.570+0000 7fb80b400700 1 --2- 172.21.15.107:0/736352979 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7fb7f806b1d0 0x7fb7f806d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:43:06.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:06.570+0000 7fb80b400700 1 -- 172.21.15.107:0/736352979 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5808+0+0 (secure 0 0 0) 0x7fb80408a7b0 con 0x7fb8140fcef0 2024-03-04T13:43:06.571 INFO:tasks.workunit.client.1.smithi169.stdout:3/829: rename d7/de/d2b/d7d/d85/d13/d1a/d78/dc1/le9 to d7/l10f 0 2024-03-04T13:43:06.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:06.574+0000 7fb812a00700 1 --2- 172.21.15.107:0/736352979 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7fb7f806b1d0 0x7fb7f806d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:43:06.573 INFO:tasks.workunit.client.1.smithi169.stdout:2/903: symlink d1/d6d/dbc/l136 0 2024-03-04T13:43:06.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:06.574+0000 7fb80b400700 1 -- 172.21.15.107:0/736352979 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7fb804058510 con 0x7fb8140fcef0 2024-03-04T13:43:06.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:06.576+0000 7fb812a00700 1 --2- 172.21.15.107:0/736352979 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7fb7f806b1d0 0x7fb7f806d680 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb8000041d0 tx=0x7fb80000b040 comp rx=0 tx=0).ready entity=mgr.24331 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:43:06.574 INFO:tasks.workunit.client.1.smithi169.stdout:2/904: fsync d1/d6d/f101 0 2024-03-04T13:43:06.578 INFO:tasks.workunit.client.1.smithi169.stdout:4/827: truncate d4/d11/d2e/d39/d61/fc1 2868828 0 2024-03-04T13:43:06.579 INFO:tasks.workunit.client.0.smithi107.stdout:6/410: dread df/d17/d24/f2b [0,4194304] 0 2024-03-04T13:43:06.582 INFO:tasks.workunit.client.0.smithi107.stdout:0/406: getdents da/dc/dd/d3e 0 2024-03-04T13:43:06.586 INFO:tasks.workunit.client.0.smithi107.stdout:9/316: symlink d4/da/d11/d20/d2f/l62 0 2024-03-04T13:43:06.586 INFO:tasks.workunit.client.0.smithi107.stdout:4/316: dwrite d0/d9/d18/d29/f5c [0,4194304] 0 2024-03-04T13:43:06.586 INFO:tasks.workunit.client.1.smithi169.stdout:1/803: creat d4/d5/d2d/d4e/d71/d26/dfa/f106 x:0 0 0 2024-03-04T13:43:06.587 INFO:tasks.workunit.client.1.smithi169.stdout:7/887: mkdir d2/d15/d17/d6f/d126/d12f 0 2024-03-04T13:43:06.589 INFO:tasks.workunit.client.0.smithi107.stdout:4/317: dread - d0/d9/de/d14/dc/d21/f5d zero size 2024-03-04T13:43:06.591 INFO:tasks.workunit.client.0.smithi107.stdout:4/318: fsync d0/d9/d10/f34 0 2024-03-04T13:43:06.591 INFO:tasks.workunit.client.1.smithi169.stdout:3/830: symlink d7/de/d2b/d7d/d85/d13/d5a/da1/l110 0 2024-03-04T13:43:06.594 INFO:tasks.workunit.client.0.smithi107.stdout:8/378: dwrite d6/d16/d1f/f5c [0,4194304] 0 2024-03-04T13:43:06.596 INFO:tasks.workunit.client.0.smithi107.stdout:4/319: write d0/d2f/f39 [30010,94608] 0 2024-03-04T13:43:06.596 INFO:tasks.workunit.client.1.smithi169.stdout:5/940: dwrite d1/d16/dbd/dbb/dcc/fe5 [0,4194304] 0 2024-03-04T13:43:06.598 INFO:tasks.workunit.client.1.smithi169.stdout:2/905: creat d1/d5/d22/d45/d4a/de7/f137 x:0 0 0 2024-03-04T13:43:06.599 INFO:tasks.workunit.client.0.smithi107.stdout:8/379: dread - d6/d16/d1f/d4c/f51 zero size 2024-03-04T13:43:06.600 INFO:tasks.workunit.client.0.smithi107.stdout:2/355: link da/d49/d5c/d6a/c5a da/d44/c76 0 2024-03-04T13:43:06.601 INFO:tasks.workunit.client.1.smithi169.stdout:8/849: unlink d7/d10/d24/d3e/f57 0 2024-03-04T13:43:06.603 INFO:tasks.workunit.client.0.smithi107.stdout:7/328: creat d6/d13/d16/d19/d1e/f6a x:0 0 0 2024-03-04T13:43:06.603 INFO:tasks.workunit.client.1.smithi169.stdout:5/941: write d1/d16/f132 [1040006,52887] 0 2024-03-04T13:43:06.609 INFO:tasks.workunit.client.1.smithi169.stdout:1/804: fsync d4/d5/fc6 0 2024-03-04T13:43:06.611 INFO:tasks.workunit.client.0.smithi107.stdout:6/411: mknod df/d17/d5c/c7f 0 2024-03-04T13:43:06.611 INFO:tasks.workunit.client.1.smithi169.stdout:8/850: write d7/d10/d17/d44/d4c/ff7 [2905880,2008] 0 2024-03-04T13:43:06.615 INFO:tasks.workunit.client.1.smithi169.stdout:6/833: dwrite d0/f2f [0,4194304] 0 2024-03-04T13:43:06.616 INFO:tasks.workunit.client.0.smithi107.stdout:9/317: unlink d4/da/d11/d20/f56 0 2024-03-04T13:43:06.616 INFO:tasks.workunit.client.0.smithi107.stdout:8/380: dread d6/da/f1b [0,4194304] 0 2024-03-04T13:43:06.620 INFO:tasks.workunit.client.0.smithi107.stdout:8/381: write d6/d16/d1f/d4c/f51 [631815,23956] 0 2024-03-04T13:43:06.620 INFO:tasks.workunit.client.0.smithi107.stdout:9/318: read d4/da/d11/d20/f28 [4354355,29646] 0 2024-03-04T13:43:06.620 INFO:tasks.workunit.client.0.smithi107.stdout:5/306: rmdir d5/d65 0 2024-03-04T13:43:06.621 INFO:tasks.workunit.client.0.smithi107.stdout:5/307: chown d5/d2b 7663 1 2024-03-04T13:43:06.625 INFO:tasks.workunit.client.0.smithi107.stdout:8/382: write d6/f45 [1029909,8609] 0 2024-03-04T13:43:06.635 INFO:tasks.workunit.client.1.smithi169.stdout:5/942: dwrite d1/d16/d17/d2d/d5a/d67/f6b [0,4194304] 0 2024-03-04T13:43:06.636 INFO:tasks.workunit.client.0.smithi107.stdout:1/335: write d1/d46/d6a/d17/f34 [1365073,25976] 0 2024-03-04T13:43:06.641 INFO:tasks.workunit.client.0.smithi107.stdout:5/308: dwrite d5/d8/d14/d2d/f6b [0,4194304] 0 2024-03-04T13:43:06.643 INFO:tasks.workunit.client.1.smithi169.stdout:3/831: creat d7/de/d2b/d7d/d85/d13/d1a/d2d/dbe/d108/df7/f111 x:0 0 0 2024-03-04T13:43:06.650 INFO:tasks.workunit.client.0.smithi107.stdout:3/335: truncate d1/f56 3893381 0 2024-03-04T13:43:06.651 INFO:tasks.workunit.client.1.smithi169.stdout:7/888: dwrite d2/dc0/fc7 [0,4194304] 0 2024-03-04T13:43:06.653 INFO:tasks.workunit.client.0.smithi107.stdout:3/336: read d1/de/d11/f3e [580383,1769] 0 2024-03-04T13:43:06.654 INFO:tasks.workunit.client.1.smithi169.stdout:8/851: symlink d7/d10/d22/d75/l112 0 2024-03-04T13:43:06.655 INFO:tasks.workunit.client.1.smithi169.stdout:4/828: link d4/d1c/lc8 d4/d11/d2e/dcd/dee/d5c/lf8 0 2024-03-04T13:43:06.658 INFO:tasks.workunit.client.0.smithi107.stdout:1/336: dread d1/d10/d16/d1b/d40/f57 [0,4194304] 0 2024-03-04T13:43:06.658 INFO:tasks.workunit.client.0.smithi107.stdout:0/407: truncate da/dc/dd/d3e/f44 2703771 0 2024-03-04T13:43:06.660 INFO:tasks.workunit.client.0.smithi107.stdout:8/383: creat d6/da/d25/d5f/f6e x:0 0 0 2024-03-04T13:43:06.662 INFO:tasks.workunit.client.0.smithi107.stdout:2/356: mknod da/d6c/c77 0 2024-03-04T13:43:06.662 INFO:tasks.workunit.client.0.smithi107.stdout:8/384: dread - d6/d16/d1f/f65 zero size 2024-03-04T13:43:06.663 INFO:tasks.workunit.client.0.smithi107.stdout:2/357: fsync da/d44/f64 0 2024-03-04T13:43:06.668 INFO:tasks.workunit.client.0.smithi107.stdout:3/337: creat d1/de/d18/d33/d48/d71/f72 x:0 0 0 2024-03-04T13:43:06.669 INFO:tasks.workunit.client.0.smithi107.stdout:3/338: readlink d1/l10 0 2024-03-04T13:43:06.670 INFO:tasks.workunit.client.0.smithi107.stdout:4/320: creat d0/d9/de/d14/f65 x:0 0 0 2024-03-04T13:43:06.670 INFO:tasks.workunit.client.1.smithi169.stdout:3/832: readlink d7/de/d2b/d7d/d85/d13/d1a/d2d/d56/d5e/d67/d5c/d84/le8 0 2024-03-04T13:43:06.671 INFO:tasks.workunit.client.1.smithi169.stdout:4/829: mkdir d4/d11/d2e/dcd/dee/d5c/d86/db8/df9 0 2024-03-04T13:43:06.671 INFO:tasks.workunit.client.1.smithi169.stdout:2/906: link d1/d5/d19/d1e/d29/f77 d1/d5/d19/d1e/f138 0 2024-03-04T13:43:06.672 INFO:tasks.workunit.client.0.smithi107.stdout:0/408: creat da/dc/dd/d3e/d6e/f87 x:0 0 0 2024-03-04T13:43:06.672 INFO:tasks.workunit.client.0.smithi107.stdout:4/321: truncate d0/d9/de/f41 725295 0 2024-03-04T13:43:06.674 INFO:tasks.workunit.client.1.smithi169.stdout:8/852: rename d7/d10/d24/l84 to d7/d10/d17/d44/d36/dd3/l113 0 2024-03-04T13:43:06.674 INFO:tasks.workunit.client.0.smithi107.stdout:1/337: mkdir d1/d46/d6a/df/d67/d6b 0 2024-03-04T13:43:06.674 INFO:tasks.workunit.client.0.smithi107.stdout:4/322: chown d0/d9/d18/d29/f35 0 1 2024-03-04T13:43:06.674 INFO:tasks.workunit.client.0.smithi107.stdout:8/385: fdatasync d6/f29 0 2024-03-04T13:43:06.676 INFO:tasks.workunit.client.0.smithi107.stdout:2/358: truncate f7 2253147 0 2024-03-04T13:43:06.678 INFO:tasks.workunit.client.0.smithi107.stdout:2/359: chown da/df/d27/l51 2640198 1 2024-03-04T13:43:06.678 INFO:tasks.workunit.client.1.smithi169.stdout:7/889: dread d2/de/d37/fd4 [0,4194304] 0 2024-03-04T13:43:06.678 INFO:tasks.workunit.client.1.smithi169.stdout:0/724: sync 2024-03-04T13:43:06.678 INFO:tasks.workunit.client.0.smithi107.stdout:9/319: creat d4/da/f63 x:0 0 0 2024-03-04T13:43:06.680 INFO:tasks.workunit.client.0.smithi107.stdout:3/339: symlink d1/de/d18/l73 0 2024-03-04T13:43:06.680 INFO:tasks.workunit.client.0.smithi107.stdout:1/338: write d1/d46/d6a/d4e/d60/f4f [421141,67664] 0 2024-03-04T13:43:06.680 INFO:tasks.workunit.client.1.smithi169.stdout:6/834: dwrite d0/d3c/d40/d52/d98/fb6 [4194304,4194304] 0 2024-03-04T13:43:06.681 INFO:tasks.workunit.client.0.smithi107.stdout:9/320: chown d4/da/d11/d20/c47 15 1 2024-03-04T13:43:06.682 INFO:tasks.workunit.client.1.smithi169.stdout:1/805: dwrite d4/d5/d36/f67 [4194304,4194304] 0 2024-03-04T13:43:06.690 INFO:tasks.workunit.client.1.smithi169.stdout:8/853: read d7/d10/f1e [1297153,67074] 0 2024-03-04T13:43:06.692 INFO:tasks.workunit.client.1.smithi169.stdout:1/806: write d4/d5/d36/d35/d5b/fbe [3023294,6710] 0 2024-03-04T13:43:06.694 INFO:tasks.workunit.client.0.smithi107.stdout:4/323: dwrite d0/d9/f11 [0,4194304] 0 2024-03-04T13:43:06.699 INFO:tasks.workunit.client.0.smithi107.stdout:2/360: dwrite da/d38/f56 [0,4194304] 0 2024-03-04T13:43:06.703 INFO:tasks.workunit.client.1.smithi169.stdout:3/833: read d7/de/d2b/d7d/d85/d13/d1a/d2d/d56/d5e/d67/f3a [484711,127023] 0 2024-03-04T13:43:06.703 INFO:tasks.workunit.client.0.smithi107.stdout:9/321: dread d4/f19 [0,4194304] 0 2024-03-04T13:43:06.704 INFO:tasks.workunit.client.0.smithi107.stdout:2/361: chown da/df/d3e/f43 843 1 2024-03-04T13:43:06.705 INFO:tasks.workunit.client.0.smithi107.stdout:7/329: truncate d6/d13/d16/d19/d39/f4e 780779 0 2024-03-04T13:43:06.706 INFO:tasks.workunit.client.1.smithi169.stdout:7/890: creat d2/de/d3e/d41/d5b/d7a/dd3/f130 x:0 0 0 2024-03-04T13:43:06.708 INFO:tasks.workunit.client.0.smithi107.stdout:6/412: truncate df/f43 1948197 0 2024-03-04T13:43:06.708 INFO:tasks.workunit.client.0.smithi107.stdout:1/339: dwrite d1/d46/d6a/df/d33/d43/f55 [0,4194304] 0 2024-03-04T13:43:06.708 INFO:tasks.workunit.client.0.smithi107.stdout:9/322: stat d4/da/d11/d4c/d51/f5c 0 2024-03-04T13:43:06.709 INFO:tasks.workunit.client.1.smithi169.stdout:7/891: chown d2/de/d3e/d41/d5b/d7a 1617584 1 2024-03-04T13:43:06.711 INFO:tasks.workunit.client.0.smithi107.stdout:9/323: chown d4/d32/l61 616 1 2024-03-04T13:43:06.712 INFO:tasks.workunit.client.1.smithi169.stdout:4/830: rename d4/d1c/lb4 to d4/d11/d2e/d6c/d78/da5/d77/dc0/lfa 0 2024-03-04T13:43:06.713 INFO:tasks.workunit.client.0.smithi107.stdout:0/409: mkdir da/dc/dd/d3e/d26/d88 0 2024-03-04T13:43:06.714 INFO:tasks.workunit.client.0.smithi107.stdout:5/309: link d5/d8/d2a/c47 d5/d8/c6c 0 2024-03-04T13:43:06.717 INFO:tasks.workunit.client.1.smithi169.stdout:1/807: creat d4/d5/d36/d75/f107 x:0 0 0 2024-03-04T13:43:06.717 INFO:tasks.workunit.client.0.smithi107.stdout:0/410: write da/dc/dd/f1f [821626,64420] 0 2024-03-04T13:43:06.717 INFO:tasks.workunit.client.1.smithi169.stdout:6/835: mknod d0/d32/c12b 0 2024-03-04T13:43:06.722 INFO:tasks.workunit.client.0.smithi107.stdout:7/330: creat d6/d13/d14/f6b x:0 0 0 2024-03-04T13:43:06.723 INFO:tasks.workunit.client.0.smithi107.stdout:7/331: chown d6/l5e 42117 1 2024-03-04T13:43:06.724 INFO:tasks.workunit.client.1.smithi169.stdout:8/854: rename d7/f9b to d7/d10/d17/d44/d5d/ddd/f114 0 2024-03-04T13:43:06.728 INFO:tasks.workunit.client.0.smithi107.stdout:1/340: write d1/d10/d16/d1b/d40/f57 [1238481,124187] 0 2024-03-04T13:43:06.730 INFO:tasks.workunit.client.1.smithi169.stdout:7/892: read d2/de/d37/f10b [256494,98275] 0 2024-03-04T13:43:06.730 INFO:tasks.workunit.client.1.smithi169.stdout:4/831: stat d4/d11/d2e/d39/d61/d6b/c7d 0 2024-03-04T13:43:06.731 INFO:tasks.workunit.client.0.smithi107.stdout:4/324: mkdir d0/d9/de/d14/d66 0 2024-03-04T13:43:06.731 INFO:tasks.workunit.client.0.smithi107.stdout:1/341: write d1/d10/d16/f37 [672250,59931] 0 2024-03-04T13:43:06.733 INFO:tasks.workunit.client.0.smithi107.stdout:4/325: read d0/d9/f5b [755975,40655] 0 2024-03-04T13:43:06.735 INFO:tasks.workunit.client.0.smithi107.stdout:4/326: chown d0/d9/de/d14/dc 1383543284 1 2024-03-04T13:43:06.736 INFO:tasks.workunit.client.0.smithi107.stdout:7/332: dread d6/d1d/d36/f61 [0,4194304] 0 2024-03-04T13:43:06.740 INFO:tasks.workunit.client.1.smithi169.stdout:6/836: mkdir d0/d32/d70/d12c 0 2024-03-04T13:43:06.741 INFO:tasks.workunit.client.0.smithi107.stdout:7/333: chown d6/d13/d16/d19/d38/d2c/f49 1 1 2024-03-04T13:43:06.742 INFO:tasks.workunit.client.0.smithi107.stdout:7/334: dread - d6/d13/d14/f6b zero size 2024-03-04T13:43:06.742 INFO:tasks.workunit.client.0.smithi107.stdout:2/362: mkdir da/d38/d3a/d5d/d78 0 2024-03-04T13:43:06.745 INFO:tasks.workunit.client.0.smithi107.stdout:6/413: mknod df/d2e/d36/d77/c80 0 2024-03-04T13:43:06.747 INFO:tasks.workunit.client.1.smithi169.stdout:8/855: rename d7/d12/c94 to d7/d10/d17/d44/d5d/d10b/c115 0 2024-03-04T13:43:06.749 INFO:tasks.workunit.client.1.smithi169.stdout:3/834: link d7/de/d2b/d7d/d85/d18/d21/ce3 d7/de/d2b/d7d/d85/d13/d1a/d2d/d56/d5e/d67/d5c/d84/d95/c112 0 2024-03-04T13:43:06.751 INFO:tasks.workunit.client.1.smithi169.stdout:4/832: creat d4/d11/d2e/dcd/dd1/ffb x:0 0 0 2024-03-04T13:43:06.755 INFO:tasks.workunit.client.0.smithi107.stdout:9/324: mkdir d4/da/d11/d20/d64 0 2024-03-04T13:43:06.756 INFO:tasks.workunit.client.0.smithi107.stdout:8/386: write d6/d16/d1f/f43 [1173129,20590] 0 2024-03-04T13:43:06.756 INFO:tasks.workunit.client.1.smithi169.stdout:6/837: fdatasync d0/d3c/d40/fde 0 2024-03-04T13:43:06.761 INFO:tasks.workunit.client.1.smithi169.stdout:8/856: chown d7/d10/d22/d75/d81/da2/d96/lfc 926442 1 2024-03-04T13:43:06.762 INFO:tasks.workunit.client.0.smithi107.stdout:1/342: creat d1/d46/d6a/d4e/f6c x:0 0 0 2024-03-04T13:43:06.762 INFO:tasks.workunit.client.0.smithi107.stdout:8/387: write d6/d16/d1f/f5c [3788536,31633] 0 2024-03-04T13:43:06.762 INFO:tasks.workunit.client.0.smithi107.stdout:2/363: dwrite da/d49/d5c/d6a/f62 [0,4194304] 0 2024-03-04T13:43:06.764 INFO:tasks.workunit.client.1.smithi169.stdout:0/725: write d5/d9/d3d/d59/d64/d7f/f9f [438890,128318] 0 2024-03-04T13:43:06.766 INFO:tasks.workunit.client.0.smithi107.stdout:8/388: dread - d6/d16/d1f/d4c/f61 zero size 2024-03-04T13:43:06.766 INFO:tasks.workunit.client.1.smithi169.stdout:9/898: sync 2024-03-04T13:43:06.767 INFO:tasks.workunit.client.1.smithi169.stdout:0/726: write d5/d9/fb8 [1935745,4159] 0 2024-03-04T13:43:06.767 INFO:tasks.workunit.client.0.smithi107.stdout:1/343: write d1/d46/d6a/d4e/d60/f63 [429004,39680] 0 2024-03-04T13:43:06.770 INFO:tasks.workunit.client.0.smithi107.stdout:4/327: dread d0/d9/d1e/f2b [4194304,4194304] 0 2024-03-04T13:43:06.771 INFO:tasks.workunit.client.1.smithi169.stdout:0/727: dread - d5/d9/dc/d12/d66/f78 zero size 2024-03-04T13:43:06.771 INFO:tasks.workunit.client.1.smithi169.stdout:4/833: rename d4/d11/d2e/dcd/dee/f87 to d4/d11/d2e/dcd/ffc 0 2024-03-04T13:43:06.775 INFO:tasks.workunit.client.0.smithi107.stdout:1/344: truncate d1/d46/d6a/d4e/d60/f4f 1540481 0 2024-03-04T13:43:06.776 INFO:tasks.workunit.client.0.smithi107.stdout:6/414: rmdir df/d38 39 2024-03-04T13:43:06.777 INFO:tasks.workunit.client.1.smithi169.stdout:0/728: truncate d5/d9/d3d/d5d/fec 98247 0 2024-03-04T13:43:06.779 INFO:tasks.workunit.client.1.smithi169.stdout:2/907: dwrite d1/d5/d19/d1e/d29/f77 [0,4194304] 0 2024-03-04T13:43:06.782 INFO:tasks.workunit.client.0.smithi107.stdout:0/411: unlink da/dc/dd/d3e/d26/c41 0 2024-03-04T13:43:06.783 INFO:tasks.workunit.client.0.smithi107.stdout:0/412: truncate da/f3c 5012386 0 2024-03-04T13:43:06.792 INFO:tasks.workunit.client.1.smithi169.stdout:9/899: dwrite d7/d2d/d48/fef [0,4194304] 0 2024-03-04T13:43:06.796 INFO:tasks.workunit.client.0.smithi107.stdout:3/340: rename d1/d1e/f5e to d1/d1e/f74 0 2024-03-04T13:43:06.797 INFO:tasks.workunit.client.0.smithi107.stdout:3/341: dread - d1/de/d11/f46 zero size 2024-03-04T13:43:06.800 INFO:tasks.workunit.client.1.smithi169.stdout:1/808: dwrite d4/d5/d2d/d4e/fa6 [0,4194304] 0 2024-03-04T13:43:06.801 INFO:tasks.workunit.client.1.smithi169.stdout:6/838: getdents d0/d3c/d10b 0 2024-03-04T13:43:06.804 INFO:tasks.workunit.client.1.smithi169.stdout:0/729: creat d5/d9/ff8 x:0 0 0 2024-03-04T13:43:06.804 INFO:tasks.workunit.client.1.smithi169.stdout:6/839: dread - d0/d3c/d40/df6/dca/f117 zero size 2024-03-04T13:43:06.808 INFO:tasks.workunit.client.1.smithi169.stdout:3/835: link d7/de/d2b/d7d/d85/d18/d21/f9f d7/de/d2b/d7d/d85/d13/d5a/f113 0 2024-03-04T13:43:06.808 INFO:tasks.workunit.client.1.smithi169.stdout:0/730: stat d5/d9/dc/d12/d15/d19/d61/d71/d8c/d90 0 2024-03-04T13:43:06.811 INFO:tasks.workunit.client.0.smithi107.stdout:9/325: rename d4/da/d11/d4c/d3b to d4/da/d11/d65 0 2024-03-04T13:43:06.812 INFO:tasks.workunit.client.1.smithi169.stdout:7/893: write d2/d15/d17/d6f/f72 [7864787,2662] 0 2024-03-04T13:43:06.816 INFO:tasks.workunit.client.0.smithi107.stdout:2/364: creat da/d44/f79 x:0 0 0 2024-03-04T13:43:06.826 INFO:tasks.workunit.client.0.smithi107.stdout:4/328: creat d0/d9/de/d14/dc/f67 x:0 0 0 2024-03-04T13:43:06.828 INFO:tasks.workunit.client.1.smithi169.stdout:6/840: chown d0/d3c/fbc 62889 1 2024-03-04T13:43:06.829 INFO:tasks.workunit.client.0.smithi107.stdout:1/345: mknod d1/d46/d6a/df/d33/d43/d5b/d61/d64/c6d 0 2024-03-04T13:43:06.834 INFO:tasks.workunit.client.1.smithi169.stdout:8/857: dwrite d7/d10/d24/fff [4194304,4194304] 0 2024-03-04T13:43:06.834 INFO:tasks.workunit.client.0.smithi107.stdout:0/413: rmdir da/dc/d34 39 2024-03-04T13:43:06.835 INFO:tasks.workunit.client.0.smithi107.stdout:0/414: write f0 [1001648,45326] 0 2024-03-04T13:43:06.837 INFO:tasks.workunit.client.0.smithi107.stdout:0/415: write da/dc/f4e [4806247,95781] 0 2024-03-04T13:43:06.840 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:06 smithi107 ceph-mon[28739]: pgmap v55: 65 pgs: 65 active+clean; 2.1 GiB data, 7.7 GiB used, 529 GiB / 536 GiB avail; 49 MiB/s rd, 178 MiB/s wr, 331 op/s 2024-03-04T13:43:06.843 INFO:tasks.workunit.client.1.smithi169.stdout:1/809: truncate d4/d5/fc6 7230175 0 2024-03-04T13:43:06.845 INFO:tasks.workunit.client.0.smithi107.stdout:3/342: symlink d1/de/d18/d33/l75 0 2024-03-04T13:43:06.848 INFO:tasks.workunit.client.1.smithi169.stdout:9/900: write d7/dc/d12/d8a/f93 [2598383,104731] 0 2024-03-04T13:43:06.849 INFO:tasks.workunit.client.1.smithi169.stdout:6/841: creat d0/d3c/d40/f12d x:0 0 0 2024-03-04T13:43:06.852 INFO:tasks.workunit.client.1.smithi169.stdout:0/731: symlink d5/d9/dc/d12/d15/lf9 0 2024-03-04T13:43:06.853 INFO:tasks.workunit.client.0.smithi107.stdout:6/415: write f3 [890263,120144] 0 2024-03-04T13:43:06.854 INFO:tasks.workunit.client.0.smithi107.stdout:5/310: getdents d5/d8/da 0 2024-03-04T13:43:06.854 INFO:tasks.workunit.client.1.smithi169.stdout:7/894: write d2/de/d3e/d41/d69/d78/fb4 [5210338,56609] 0 2024-03-04T13:43:06.856 INFO:tasks.workunit.client.1.smithi169.stdout:3/836: creat d7/de/d2b/d7d/d85/d13/d5a/dd8/f114 x:0 0 0 2024-03-04T13:43:06.857 INFO:tasks.workunit.client.1.smithi169.stdout:4/834: dwrite d4/f15 [0,4194304] 0 2024-03-04T13:43:06.862 INFO:tasks.workunit.client.1.smithi169.stdout:4/835: dread - d4/d1c/d58/d9d/fd0 zero size 2024-03-04T13:43:06.863 INFO:tasks.workunit.client.0.smithi107.stdout:8/389: creat d6/d16/d59/d3e/d67/f6f x:0 0 0 2024-03-04T13:43:06.864 INFO:tasks.workunit.client.0.smithi107.stdout:4/329: creat d0/d30/f68 x:0 0 0 2024-03-04T13:43:06.865 INFO:tasks.workunit.client.1.smithi169.stdout:1/810: chown d4/d5/d2d/d4e/d71/d26/d54/c85 4225 1 2024-03-04T13:43:06.868 INFO:tasks.workunit.client.1.smithi169.stdout:6/842: mkdir d0/d32/d3e/d12e 0 2024-03-04T13:43:06.870 INFO:tasks.workunit.client.0.smithi107.stdout:9/326: dwrite d4/da/d11/d4c/f3c [0,4194304] 0 2024-03-04T13:43:06.875 INFO:tasks.workunit.client.0.smithi107.stdout:7/335: chown d6/d13/d16/d19/d39/f4e 771286280 1 2024-03-04T13:43:06.879 INFO:tasks.workunit.client.0.smithi107.stdout:3/343: rename d1/d2d/c40 to d1/de/d4b/c76 0 2024-03-04T13:43:06.882 INFO:tasks.workunit.client.0.smithi107.stdout:2/365: mkdir da/df/d73/d7a 0 2024-03-04T13:43:06.883 INFO:tasks.workunit.client.1.smithi169.stdout:8/858: dwrite d7/d10/d17/d44/d36/fa8 [0,4194304] 0 2024-03-04T13:43:06.885 INFO:tasks.workunit.client.0.smithi107.stdout:6/416: creat df/d17/d24/f81 x:0 0 0 2024-03-04T13:43:06.886 INFO:tasks.workunit.client.1.smithi169.stdout:4/836: creat d4/d11/d3b/d67/ffd x:0 0 0 2024-03-04T13:43:06.886 INFO:tasks.workunit.client.1.smithi169.stdout:1/811: creat d4/d5/d36/d35/d16/d81/db4/f108 x:0 0 0 2024-03-04T13:43:06.888 INFO:tasks.workunit.client.1.smithi169.stdout:4/837: write d4/d11/d2e/d21/da9/fcb [152605,493] 0 2024-03-04T13:43:06.889 INFO:tasks.workunit.client.1.smithi169.stdout:6/843: chown d0/d3c/d40/d52/dac/f11f 0 1 2024-03-04T13:43:06.890 INFO:tasks.workunit.client.0.smithi107.stdout:1/346: creat d1/d46/d6a/df/d67/d6b/f6e x:0 0 0 2024-03-04T13:43:06.892 INFO:tasks.workunit.client.1.smithi169.stdout:9/901: link d7/dc/d3e/fab d7/dc/d12/d32/d42/d4f/f143 0 2024-03-04T13:43:06.893 INFO:tasks.workunit.client.1.smithi169.stdout:1/812: read - d4/d5/d2d/d4e/d71/d82/df7/f105 zero size 2024-03-04T13:43:06.893 INFO:tasks.workunit.client.1.smithi169.stdout:6/844: write d0/f2b [5789034,78413] 0 2024-03-04T13:43:06.895 INFO:tasks.workunit.client.1.smithi169.stdout:7/895: mkdir d2/d131 0 2024-03-04T13:43:06.896 INFO:tasks.workunit.client.0.smithi107.stdout:7/336: symlink d6/d13/d16/d19/d38/d2c/l6c 0 2024-03-04T13:43:06.899 INFO:tasks.workunit.client.0.smithi107.stdout:5/311: rename d5/d8/f5b to d5/d8/d14/d5a/d6a/f6d 0 2024-03-04T13:43:06.899 INFO:tasks.workunit.client.0.smithi107.stdout:7/337: fsync d6/d13/d16/d19/d38/d2c/f4a 0 2024-03-04T13:43:06.901 INFO:tasks.workunit.client.0.smithi107.stdout:5/312: stat d5/d8/da/f16 0 2024-03-04T13:43:06.902 INFO:tasks.workunit.client.0.smithi107.stdout:2/366: symlink da/df/d27/d58/d5f/l7b 0 2024-03-04T13:43:06.905 INFO:tasks.workunit.client.0.smithi107.stdout:7/338: dread d6/d13/d16/d19/d39/f4e [0,4194304] 0 2024-03-04T13:43:06.907 INFO:tasks.workunit.client.0.smithi107.stdout:1/347: mkdir d1/d46/d6a/d4e/d6f 0 2024-03-04T13:43:06.910 INFO:tasks.workunit.client.1.smithi169.stdout:9/902: unlink d7/dc/d12/d32/d42/d4f/fb5 0 2024-03-04T13:43:06.911 INFO:tasks.workunit.client.0.smithi107.stdout:9/327: mknod d4/da/d11/d65/c66 0 2024-03-04T13:43:06.912 INFO:tasks.workunit.client.1.smithi169.stdout:6/845: symlink d0/d32/d3a/d43/d74/d93/dbe/l12f 0 2024-03-04T13:43:06.915 INFO:tasks.workunit.client.1.smithi169.stdout:5/943: sync 2024-03-04T13:43:06.917 INFO:tasks.workunit.client.1.smithi169.stdout:1/813: rename d4/d5/d36/d35/d16/d81/ff3 to d4/d5/d2d/d4e/d71/d26/daf/dd0/f109 0 2024-03-04T13:43:06.919 INFO:tasks.workunit.client.1.smithi169.stdout:7/896: dwrite d2/d28/d70/ded/f125 [0,4194304] 0 2024-03-04T13:43:06.920 INFO:tasks.workunit.client.1.smithi169.stdout:8/859: dwrite d7/d10/d22/d75/de9/f10e [0,4194304] 0 2024-03-04T13:43:06.923 INFO:tasks.workunit.client.0.smithi107.stdout:4/330: rename d0/d30/f68 to d0/d9/de/d14/dc/f69 0 2024-03-04T13:43:06.924 INFO:tasks.workunit.client.1.smithi169.stdout:8/860: dread - d7/d10/f10f zero size 2024-03-04T13:43:06.928 INFO:tasks.workunit.client.1.smithi169.stdout:3/837: write d7/de/d2b/d7d/d85/d13/d1a/d2d/d56/f81 [333540,43261] 0 2024-03-04T13:43:06.930 INFO:tasks.workunit.client.1.smithi169.stdout:9/903: readlink d7/d25/d3c/l131 0 2024-03-04T13:43:06.930 INFO:tasks.workunit.client.1.smithi169.stdout:0/732: dwrite d5/d9/dc/fa6 [0,4194304] 0 2024-03-04T13:43:06.932 INFO:tasks.workunit.client.1.smithi169.stdout:8/861: dread - d7/d10/d22/d75/d81/f100 zero size 2024-03-04T13:43:06.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:06.934+0000 7fb81a600700 1 -- 172.21.15.107:0/736352979 --> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb814115fc0 con 0x7fb7f806b1d0 2024-03-04T13:43:06.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:06.938+0000 7fb80b400700 1 -- 172.21.15.107:0/736352979 <== mgr.24331 v2:172.21.15.169:6828/3524931243 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7fb814115fc0 con 0x7fb7f806b1d0 2024-03-04T13:43:06.937 INFO:tasks.workunit.client.0.smithi107.stdout:5/313: unlink d5/d8/d14/c25 0 2024-03-04T13:43:06.938 INFO:tasks.workunit.client.1.smithi169.stdout:3/838: readlink d7/de/d2b/d7d/d85/d47/d63/d6c/l72 0 2024-03-04T13:43:06.938 INFO:tasks.workunit.client.1.smithi169.stdout:8/862: read d7/d10/d22/d75/d81/da2/fbf [314572,79458] 0 2024-03-04T13:43:06.940 INFO:tasks.workunit.client.0.smithi107.stdout:6/417: symlink df/d38/l82 0 2024-03-04T13:43:06.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:06.943+0000 7fb81a600700 1 -- 172.21.15.107:0/736352979 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7fb7f806b1d0 msgr2=0x7fb7f806d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:43:06.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:06.943+0000 7fb81a600700 1 --2- 172.21.15.107:0/736352979 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7fb7f806b1d0 0x7fb7f806d680 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb8000041d0 tx=0x7fb80000b040 comp rx=0 tx=0).stop 2024-03-04T13:43:06.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:06.943+0000 7fb81a600700 1 -- 172.21.15.107:0/736352979 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8140fcef0 msgr2=0x7fb814113f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:43:06.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:06.943+0000 7fb81a600700 1 --2- 172.21.15.107:0/736352979 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8140fcef0 0x7fb814113f10 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fb80400b370 tx=0x7fb804009e30 comp rx=0 tx=0).stop 2024-03-04T13:43:06.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:06.943+0000 7fb81a600700 1 -- 172.21.15.107:0/736352979 shutdown_connections 2024-03-04T13:43:06.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:06.943+0000 7fb81a600700 1 --2- 172.21.15.107:0/736352979 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7fb7f806b1d0 0x7fb7f806d680 secure :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb8000041d0 tx=0x7fb80000b040 comp rx=0 tx=0).stop 2024-03-04T13:43:06.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:06.943+0000 7fb81a600700 1 --2- 172.21.15.107:0/736352979 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8140fcef0 0x7fb814113f10 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:43:06.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:06.943+0000 7fb81a600700 1 --2- 172.21.15.107:0/736352979 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8140fe0e0 0x7fb814114450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:43:06.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:06.943+0000 7fb81a600700 1 -- 172.21.15.107:0/736352979 >> 172.21.15.107:0/736352979 conn(0x7fb8140f8620 msgr2=0x7fb8140fa8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:43:06.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:06.944+0000 7fb81a600700 1 -- 172.21.15.107:0/736352979 shutdown_connections 2024-03-04T13:43:06.942 INFO:tasks.workunit.client.1.smithi169.stdout:2/908: sync 2024-03-04T13:43:06.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:06.945+0000 7fb81a600700 1 -- 172.21.15.107:0/736352979 wait complete. 2024-03-04T13:43:06.945 INFO:tasks.workunit.client.1.smithi169.stdout:2/909: dread - d1/d5/d22/d3b/f113 zero size 2024-03-04T13:43:06.946 INFO:tasks.workunit.client.0.smithi107.stdout:8/390: creat d6/da/f70 x:0 0 0 2024-03-04T13:43:06.947 INFO:tasks.workunit.client.1.smithi169.stdout:6/846: fdatasync d0/d3c/d40/f6f 0 2024-03-04T13:43:06.951 INFO:tasks.workunit.client.1.smithi169.stdout:5/944: fsync d1/d16/d2f/f33 0 2024-03-04T13:43:06.952 INFO:tasks.workunit.client.0.smithi107.stdout:9/328: mknod d4/da/d11/d4c/d4d/c67 0 2024-03-04T13:43:06.952 INFO:tasks.workunit.client.1.smithi169.stdout:6/847: truncate d0/d32/d3a/d43/d74/f103 541540 0 2024-03-04T13:43:06.953 INFO:tasks.workunit.client.0.smithi107.stdout:0/416: getdents da/d7e 0 2024-03-04T13:43:06.954 INFO:tasks.workunit.client.1.smithi169.stdout:7/897: truncate d2/d6b/fd7 4315996 0 2024-03-04T13:43:06.955 INFO:tasks.workunit.client.1.smithi169.stdout:1/814: mknod d4/d5/d2d/d4e/d71/d26/d54/c10a 0 2024-03-04T13:43:06.957 INFO:tasks.workunit.client.0.smithi107.stdout:3/344: rename d1/de/d18/l73 to d1/de/d18/d33/d4d/d4f/d69/l77 0 2024-03-04T13:43:06.958 INFO:tasks.workunit.client.0.smithi107.stdout:1/348: rename d1/d38 to d1/d38/d70 22 2024-03-04T13:43:06.960 INFO:tasks.workunit.client.0.smithi107.stdout:4/331: fdatasync d0/d9/de/d14/dc/f26 0 2024-03-04T13:43:06.960 INFO:tasks.workunit.client.1.smithi169.stdout:0/733: creat d5/d9/d3d/d59/d64/d7f/ffa x:0 0 0 2024-03-04T13:43:06.961 INFO:teuthology.orchestra.run.smithi107.stdout:true 2024-03-04T13:43:06.962 INFO:tasks.workunit.client.0.smithi107.stdout:0/417: dread da/dc/dd/d1e/f85 [0,4194304] 0 2024-03-04T13:43:06.962 INFO:tasks.workunit.client.0.smithi107.stdout:4/332: chown d0/d9/d10/f34 53239968 1 2024-03-04T13:43:06.963 INFO:tasks.workunit.client.0.smithi107.stdout:3/345: truncate d1/de/d18/d33/d48/d71/f72 572774 0 2024-03-04T13:43:06.963 INFO:tasks.workunit.client.1.smithi169.stdout:9/904: mknod d7/dc/d12/d20/da6/d111/c144 0 2024-03-04T13:43:06.964 INFO:tasks.workunit.client.1.smithi169.stdout:4/838: dwrite d4/d11/d2e/d39/f68 [0,4194304] 0 2024-03-04T13:43:06.965 INFO:tasks.workunit.client.0.smithi107.stdout:3/346: dread - d1/de/d11/f46 zero size 2024-03-04T13:43:06.978 INFO:tasks.workunit.client.1.smithi169.stdout:4/839: dread d4/d11/d2e/dcd/dee/d5c/d86/f99 [0,4194304] 0 2024-03-04T13:43:06.979 INFO:tasks.workunit.client.0.smithi107.stdout:4/333: dwrite d0/d9/d18/d29/f5c [0,4194304] 0 2024-03-04T13:43:06.980 INFO:tasks.workunit.client.1.smithi169.stdout:7/898: dwrite d2/de/d3e/d41/d5b/f11f [0,4194304] 0 2024-03-04T13:43:06.980 INFO:tasks.workunit.client.0.smithi107.stdout:4/334: fdatasync d0/d9/d10/f1c 0 2024-03-04T13:43:06.986 INFO:tasks.workunit.client.1.smithi169.stdout:3/839: write f4 [4776913,98904] 0 2024-03-04T13:43:06.988 INFO:tasks.workunit.client.1.smithi169.stdout:8/863: write d7/d10/d22/fa7 [295528,81660] 0 2024-03-04T13:43:06.990 INFO:tasks.workunit.client.1.smithi169.stdout:3/840: chown d7/de/d2b/d7d/d85/d18/f30 5 1 2024-03-04T13:43:06.990 INFO:tasks.workunit.client.1.smithi169.stdout:1/815: symlink d4/d5/d2d/d4e/d71/d82/df7/l10b 0 2024-03-04T13:43:06.993 INFO:tasks.workunit.client.1.smithi169.stdout:0/734: write d5/d9/dc/f1d [1957622,126249] 0 2024-03-04T13:43:06.997 INFO:tasks.workunit.client.0.smithi107.stdout:5/314: symlink d5/d8/d14/d2d/l6e 0 2024-03-04T13:43:06.997 INFO:tasks.workunit.client.1.smithi169.stdout:9/905: creat d7/dc/d12/d8a/df9/f145 x:0 0 0 2024-03-04T13:43:06.998 INFO:tasks.workunit.client.0.smithi107.stdout:6/418: fdatasync df/f15 0 2024-03-04T13:43:07.003 INFO:tasks.workunit.client.0.smithi107.stdout:9/329: mkdir d4/da/d11/d68 0 2024-03-04T13:43:07.005 INFO:tasks.workunit.client.0.smithi107.stdout:1/349: mkdir d1/d10/d16/d47/d71 0 2024-03-04T13:43:07.006 INFO:tasks.workunit.client.1.smithi169.stdout:5/945: write d1/d16/d2f/d72/f116 [265444,105974] 0 2024-03-04T13:43:07.007 INFO:tasks.workunit.client.0.smithi107.stdout:0/418: write da/dc/d34/f45 [4963868,60830] 0 2024-03-04T13:43:07.009 INFO:tasks.workunit.client.0.smithi107.stdout:3/347: creat d1/de/d3f/f78 x:0 0 0 2024-03-04T13:43:07.013 INFO:tasks.workunit.client.1.smithi169.stdout:7/899: read - d2/de/d3e/d41/d69/fb0 zero size 2024-03-04T13:43:07.015 INFO:tasks.workunit.client.1.smithi169.stdout:3/841: fdatasync d7/de/d2b/d7d/d85/d13/d1a/d2d/d56/f90 0 2024-03-04T13:43:07.020 INFO:tasks.workunit.client.0.smithi107.stdout:1/350: dwrite d1/d46/d6a/d4e/d60/f4f [0,4194304] 0 2024-03-04T13:43:07.025 INFO:tasks.workunit.client.1.smithi169.stdout:6/848: dwrite d0/d32/d3a/d43/d74/d93/dbe/fec [0,4194304] 0 2024-03-04T13:43:07.025 INFO:tasks.workunit.client.1.smithi169.stdout:3/842: fsync d7/de/d2b/d7d/d85/d13/d1a/d98/d9a/ff4 0 2024-03-04T13:43:07.025 INFO:tasks.workunit.client.1.smithi169.stdout:8/864: dwrite d7/d10/d17/f98 [0,4194304] 0 2024-03-04T13:43:07.027 INFO:tasks.workunit.client.0.smithi107.stdout:6/419: mkdir df/d17/d24/d83 0 2024-03-04T13:43:07.031 INFO:tasks.workunit.client.1.smithi169.stdout:0/735: dwrite d5/d9/dc/d12/d15/d19/d77/d94/fa2 [0,4194304] 0 2024-03-04T13:43:07.032 INFO:tasks.workunit.client.0.smithi107.stdout:2/367: dwrite f7 [0,4194304] 0 2024-03-04T13:43:07.032 INFO:tasks.workunit.client.0.smithi107.stdout:6/420: chown df/d1b/d2c/d46/f70 118 1 2024-03-04T13:43:07.034 INFO:tasks.workunit.client.1.smithi169.stdout:0/736: chown d5/cae 327516382 1 2024-03-04T13:43:07.036 INFO:tasks.workunit.client.0.smithi107.stdout:6/421: fsync df/d17/d24/d40/f62 0 2024-03-04T13:43:07.037 INFO:tasks.workunit.client.0.smithi107.stdout:9/330: symlink d4/da/d11/d4c/d4d/l69 0 2024-03-04T13:43:07.042 INFO:tasks.workunit.client.0.smithi107.stdout:4/335: creat d0/d9/de/d14/d66/f6a x:0 0 0 2024-03-04T13:43:07.043 INFO:tasks.workunit.client.1.smithi169.stdout:8/865: dread d7/d10/d22/fa7 [0,4194304] 0 2024-03-04T13:43:07.047 INFO:tasks.workunit.client.0.smithi107.stdout:2/368: mkdir da/d29/d7c 0 2024-03-04T13:43:07.052 INFO:tasks.workunit.client.0.smithi107.stdout:7/339: getdents d6/d13/d16/d19/d38 0 2024-03-04T13:43:07.052 INFO:tasks.workunit.client.0.smithi107.stdout:6/422: fdatasync df/d17/f23 0 2024-03-04T13:43:07.057 INFO:tasks.workunit.client.0.smithi107.stdout:0/419: link da/f81 da/d7e/f89 0 2024-03-04T13:43:07.057 INFO:tasks.workunit.client.0.smithi107.stdout:5/315: link d5/l5e d5/dc/df/l6f 0 2024-03-04T13:43:07.057 INFO:tasks.workunit.client.0.smithi107.stdout:2/369: write da/df/d3e/f4c [920229,129481] 0 2024-03-04T13:43:07.057 INFO:tasks.workunit.client.0.smithi107.stdout:4/336: stat d0/l16 0 2024-03-04T13:43:07.062 INFO:tasks.workunit.client.0.smithi107.stdout:0/420: chown da/dc/dd/l19 1773 1 2024-03-04T13:43:07.063 INFO:tasks.workunit.client.0.smithi107.stdout:7/340: creat d6/d17/f6d x:0 0 0 2024-03-04T13:43:07.063 INFO:tasks.workunit.client.0.smithi107.stdout:8/391: getdents d6/da/d25 0 2024-03-04T13:43:07.064 INFO:tasks.workunit.client.0.smithi107.stdout:1/351: creat d1/f72 x:0 0 0 2024-03-04T13:43:07.066 INFO:tasks.workunit.client.1.smithi169.stdout:2/910: mkdir d1/d5/d19/d3c/db8/de1/d64/dce/dd5/df5/d120/d11c/d139 0 2024-03-04T13:43:07.068 INFO:tasks.workunit.client.0.smithi107.stdout:8/392: stat d6/d16/d1f/d4c/l6a 0 2024-03-04T13:43:07.068 INFO:tasks.workunit.client.1.smithi169.stdout:2/911: fsync d1/d5/d19/d3c/db8/de1/fe6 0 2024-03-04T13:43:07.070 INFO:tasks.workunit.client.1.smithi169.stdout:4/840: mknod d4/d11/d2e/cfe 0 2024-03-04T13:43:07.070 INFO:tasks.workunit.client.0.smithi107.stdout:5/316: creat d5/d8/d2a/f70 x:0 0 0 2024-03-04T13:43:07.072 INFO:tasks.workunit.client.0.smithi107.stdout:9/331: dwrite d4/da/d11/d20/f4e [0,4194304] 0 2024-03-04T13:43:07.072 INFO:tasks.workunit.client.1.smithi169.stdout:9/906: symlink d7/d49/df8/l146 0 2024-03-04T13:43:07.075 INFO:tasks.workunit.client.0.smithi107.stdout:2/370: mknod da/df/d33/c7d 0 2024-03-04T13:43:07.078 INFO:tasks.workunit.client.1.smithi169.stdout:5/946: rename d1/d16/d17/f109 to d1/d76/f14f 0 2024-03-04T13:43:07.079 INFO:tasks.workunit.client.0.smithi107.stdout:8/393: fdatasync f4 0 2024-03-04T13:43:07.080 INFO:tasks.workunit.client.1.smithi169.stdout:7/900: mknod d2/d28/db8/dd5/dea/c132 0 2024-03-04T13:43:07.082 INFO:tasks.workunit.client.1.smithi169.stdout:7/901: chown d2/f120 442150 1 2024-03-04T13:43:07.084 INFO:tasks.workunit.client.1.smithi169.stdout:6/849: symlink d0/d32/d3e/d122/l130 0 2024-03-04T13:43:07.084 INFO:tasks.workunit.client.0.smithi107.stdout:2/371: write da/d44/f52 [249441,99421] 0 2024-03-04T13:43:07.085 INFO:tasks.workunit.client.0.smithi107.stdout:0/421: mknod da/dc/dd/d3e/d26/d88/c8a 0 2024-03-04T13:43:07.085 INFO:tasks.workunit.client.0.smithi107.stdout:6/423: dwrite df/d17/d24/d40/f44 [0,4194304] 0 2024-03-04T13:43:07.087 INFO:tasks.workunit.client.0.smithi107.stdout:9/332: unlink d4/da/d11/d20/f4e 0 2024-03-04T13:43:07.091 INFO:tasks.workunit.client.1.smithi169.stdout:4/841: dwrite d4/d11/d2e/d39/f60 [4194304,4194304] 0 2024-03-04T13:43:07.093 INFO:tasks.workunit.client.1.smithi169.stdout:3/843: symlink d7/de/d2b/d7d/d85/d13/d1a/d2d/d56/d5e/d67/d5c/d84/l115 0 2024-03-04T13:43:07.094 INFO:tasks.workunit.client.1.smithi169.stdout:4/842: write d4/f41 [505740,10033] 0 2024-03-04T13:43:07.096 INFO:tasks.workunit.client.0.smithi107.stdout:7/341: dwrite d6/d13/d16/d19/d39/f40 [0,4194304] 0 2024-03-04T13:43:07.096 INFO:tasks.workunit.client.0.smithi107.stdout:1/352: dwrite d1/d46/d6a/df/d33/d43/d5b/f5f [0,4194304] 0 2024-03-04T13:43:07.097 INFO:tasks.workunit.client.0.smithi107.stdout:5/317: symlink d5/d66/l71 0 2024-03-04T13:43:07.100 INFO:tasks.workunit.client.1.smithi169.stdout:0/737: fdatasync d5/d9/d3d/d5d/de4/d98/dc1/fe6 0 2024-03-04T13:43:07.103 INFO:tasks.workunit.client.0.smithi107.stdout:4/337: rename d0/d9/de/d14/dc/d21/f28 to d0/d9/f6b 0 2024-03-04T13:43:07.108 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:06 smithi169 ceph-mon[36313]: pgmap v55: 65 pgs: 65 active+clean; 2.1 GiB data, 7.7 GiB used, 529 GiB / 536 GiB avail; 49 MiB/s rd, 178 MiB/s wr, 331 op/s 2024-03-04T13:43:07.108 INFO:tasks.workunit.client.0.smithi107.stdout:7/342: dread d6/d13/d16/d19/d1e/d3d/f43 [0,4194304] 0 2024-03-04T13:43:07.110 INFO:tasks.workunit.client.0.smithi107.stdout:6/424: creat df/d2e/d36/d77/f84 x:0 0 0 2024-03-04T13:43:07.112 INFO:tasks.workunit.client.0.smithi107.stdout:7/343: truncate d6/d13/d14/f5d 402307 0 2024-03-04T13:43:07.113 INFO:tasks.workunit.client.0.smithi107.stdout:7/344: read - d6/d17/f6d zero size 2024-03-04T13:43:07.119 INFO:tasks.workunit.client.0.smithi107.stdout:1/353: symlink d1/d46/d6a/df/d33/d43/d5b/d61/d64/l73 0 2024-03-04T13:43:07.120 INFO:tasks.workunit.client.1.smithi169.stdout:5/947: truncate d1/d16/dbd/dbb/fe3 7136 0 2024-03-04T13:43:07.124 INFO:tasks.workunit.client.0.smithi107.stdout:5/318: dread d5/f49 [0,4194304] 0 2024-03-04T13:43:07.127 INFO:tasks.workunit.client.1.smithi169.stdout:0/738: dwrite d5/d9/dc/d12/d15/d19/d77/f9b [0,4194304] 0 2024-03-04T13:43:07.128 INFO:tasks.workunit.client.0.smithi107.stdout:2/372: rename da/d49/d5c/d6a/f62 to da/d49/f7e 0 2024-03-04T13:43:07.130 INFO:tasks.workunit.client.1.smithi169.stdout:8/866: dread d7/d10/d24/d5c/f90 [0,4194304] 0 2024-03-04T13:43:07.134 INFO:tasks.workunit.client.0.smithi107.stdout:4/338: creat d0/d9/de/d14/dc/d21/f6c x:0 0 0 2024-03-04T13:43:07.134 INFO:tasks.workunit.client.1.smithi169.stdout:8/867: chown d7/d10/d17/d44/d4c/cae 250 1 2024-03-04T13:43:07.140 INFO:tasks.workunit.client.1.smithi169.stdout:1/816: write d4/d5/d2d/d4e/d71/d26/f7b [4910937,8224] 0 2024-03-04T13:43:07.141 INFO:tasks.workunit.client.1.smithi169.stdout:4/843: symlink d4/d11/d2e/dcd/dee/lff 0 2024-03-04T13:43:07.142 INFO:tasks.workunit.client.0.smithi107.stdout:7/345: mkdir d6/d13/d16/d19/d38/d6e 0 2024-03-04T13:43:07.143 INFO:tasks.workunit.client.0.smithi107.stdout:7/346: dread - d6/d17/f55 zero size 2024-03-04T13:43:07.146 INFO:tasks.workunit.client.1.smithi169.stdout:7/902: dwrite d2/de/d3e/d41/d5b/d7a/f121 [0,4194304] 0 2024-03-04T13:43:07.149 INFO:tasks.workunit.client.1.smithi169.stdout:9/907: symlink d7/l147 0 2024-03-04T13:43:07.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:07.154+0000 7f939a600700 1 -- 172.21.15.107:0/821322322 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9394070860 msgr2=0x7f9394070cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:43:07.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:07.154+0000 7f939a600700 1 --2- 172.21.15.107:0/821322322 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9394070860 0x7f9394070cd0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f9380004940 tx=0x7f9380009e30 comp rx=0 tx=0).stop 2024-03-04T13:43:07.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:07.154+0000 7f939a600700 1 -- 172.21.15.107:0/821322322 shutdown_connections 2024-03-04T13:43:07.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:07.154+0000 7f939a600700 1 --2- 172.21.15.107:0/821322322 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9394070860 0x7f9394070cd0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:43:07.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:07.154+0000 7f939a600700 1 --2- 172.21.15.107:0/821322322 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9394071eb0 0x7f9394070320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:43:07.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:07.154+0000 7f939a600700 1 -- 172.21.15.107:0/821322322 >> 172.21.15.107:0/821322322 conn(0x7f93940f8620 msgr2=0x7f93940faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:43:07.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:07.155+0000 7f939a600700 1 -- 172.21.15.107:0/821322322 shutdown_connections 2024-03-04T13:43:07.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:07.155+0000 7f939a600700 1 -- 172.21.15.107:0/821322322 wait complete. 2024-03-04T13:43:07.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:07.155+0000 7f939a600700 1 Processor -- start 2024-03-04T13:43:07.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:07.155+0000 7f939a600700 1 -- start start 2024-03-04T13:43:07.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:07.156+0000 7f939a600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9394070860 0x7f939410fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:43:07.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:07.156+0000 7f939a600700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9394071eb0 0x7f93941100c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:43:07.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:07.156+0000 7f939a600700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93941106c0 con 0x7f9394070860 2024-03-04T13:43:07.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:07.156+0000 7f939a600700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9394110f60 con 0x7f9394071eb0 2024-03-04T13:43:07.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:07.156+0000 7f9393400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9394070860 0x7f939410fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:43:07.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:07.156+0000 7f9393400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9394070860 0x7f939410fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52016/0 (socket says 172.21.15.107:52016) 2024-03-04T13:43:07.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:07.156+0000 7f9393400700 1 -- 172.21.15.107:0/4214986892 learned_addr learned my addr 172.21.15.107:0/4214986892 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:43:07.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:07.156+0000 7f9392a00700 1 --2- 172.21.15.107:0/4214986892 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9394071eb0 0x7f93941100c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:43:07.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:07.156+0000 7f9393400700 1 -- 172.21.15.107:0/4214986892 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9394071eb0 msgr2=0x7f93941100c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:43:07.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:07.156+0000 7f9393400700 1 --2- 172.21.15.107:0/4214986892 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9394071eb0 0x7f93941100c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:43:07.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:07.156+0000 7f9393400700 1 -- 172.21.15.107:0/4214986892 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9380005040 con 0x7f9394070860 2024-03-04T13:43:07.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:07.157+0000 7f9393400700 1 --2- 172.21.15.107:0/4214986892 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9394070860 0x7f939410fb80 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f9384002a50 tx=0x7f93840046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:43:07.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:07.157+0000 7f938b400700 1 -- 172.21.15.107:0/4214986892 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9384007cc0 con 0x7f9394070860 2024-03-04T13:43:07.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:07.157+0000 7f938b400700 1 -- 172.21.15.107:0/4214986892 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f938400f040 con 0x7f9394070860 2024-03-04T13:43:07.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:07.157+0000 7f939a600700 1 -- 172.21.15.107:0/4214986892 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9394111240 con 0x7f9394070860 2024-03-04T13:43:07.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:07.157+0000 7f938b400700 1 -- 172.21.15.107:0/4214986892 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f938400db70 con 0x7f9394070860 2024-03-04T13:43:07.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:07.157+0000 7f939a600700 1 -- 172.21.15.107:0/4214986892 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9394111780 con 0x7f9394070860 2024-03-04T13:43:07.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:07.159+0000 7f939a600700 1 -- 172.21.15.107:0/4214986892 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f939404bed0 con 0x7f9394070860 2024-03-04T13:43:07.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:07.160+0000 7f938b400700 1 -- 172.21.15.107:0/4214986892 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) v1 ==== 88911+0+0 (secure 0 0 0) 0x7f938400dd10 con 0x7f9394070860 2024-03-04T13:43:07.157 INFO:tasks.workunit.client.1.smithi169.stdout:5/948: unlink d1/d16/d17/d2d/f6a 0 2024-03-04T13:43:07.159 INFO:tasks.workunit.client.1.smithi169.stdout:0/739: dwrite d5/d9/ff8 [0,4194304] 0 2024-03-04T13:43:07.159 INFO:tasks.workunit.client.1.smithi169.stdout:2/912: truncate d1/d5/d19/d3c/db8/de1/fe6 80109 0 2024-03-04T13:43:07.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:07.163+0000 7f938b400700 1 --2- 172.21.15.107:0/4214986892 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7f937806b1d0 0x7f937806d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:43:07.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:07.163+0000 7f938b400700 1 -- 172.21.15.107:0/4214986892 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f9384089260 con 0x7f9394070860 2024-03-04T13:43:07.162 INFO:tasks.workunit.client.1.smithi169.stdout:5/949: fsync d1/d16/d17/d2d/d5a/d67/f6b 0 2024-03-04T13:43:07.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:07.164+0000 7f9392a00700 1 --2- 172.21.15.107:0/4214986892 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7f937806b1d0 0x7f937806d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:43:07.164 INFO:tasks.workunit.client.1.smithi169.stdout:2/913: write d1/d5/d22/f10b [850986,87431] 0 2024-03-04T13:43:07.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:07.167+0000 7f9392a00700 1 --2- 172.21.15.107:0/4214986892 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7f937806b1d0 0x7f937806d680 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f938000b020 tx=0x7f9380009e30 comp rx=0 tx=0).ready entity=mgr.24331 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:43:07.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:07.169+0000 7f938b400700 1 -- 172.21.15.107:0/4214986892 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f9384056f40 con 0x7f9394070860 2024-03-04T13:43:07.167 INFO:tasks.workunit.client.0.smithi107.stdout:2/373: creat da/d49/d5c/d6a/f7f x:0 0 0 2024-03-04T13:43:07.168 INFO:tasks.workunit.client.0.smithi107.stdout:5/319: dread d5/d8/f17 [0,4194304] 0 2024-03-04T13:43:07.169 INFO:tasks.workunit.client.1.smithi169.stdout:1/817: dwrite d4/d5/d36/d75/dba/fc9 [0,4194304] 0 2024-03-04T13:43:07.170 INFO:tasks.workunit.client.0.smithi107.stdout:5/320: stat d5/d8/da 0 2024-03-04T13:43:07.172 INFO:tasks.workunit.client.0.smithi107.stdout:6/425: symlink df/d1b/d2c/d46/d7e/l85 0 2024-03-04T13:43:07.172 INFO:tasks.workunit.client.0.smithi107.stdout:9/333: write d4/da/d11/f13 [2534604,83146] 0 2024-03-04T13:43:07.174 INFO:tasks.workunit.client.0.smithi107.stdout:4/339: creat d0/d9/de/d14/dc/d21/f6d x:0 0 0 2024-03-04T13:43:07.176 INFO:tasks.workunit.client.0.smithi107.stdout:7/347: symlink d6/d1d/l6f 0 2024-03-04T13:43:07.179 INFO:tasks.workunit.client.0.smithi107.stdout:0/422: creat da/dc/dd/d3e/d6e/d73/d78/f8b x:0 0 0 2024-03-04T13:43:07.180 INFO:tasks.workunit.client.1.smithi169.stdout:6/850: write d0/d3c/d40/df6/dca/ff1 [1554662,28495] 0 2024-03-04T13:43:07.182 INFO:tasks.workunit.client.0.smithi107.stdout:0/423: fsync da/dc/dd/d3e/d26/f64 0 2024-03-04T13:43:07.183 INFO:tasks.workunit.client.1.smithi169.stdout:7/903: creat d2/d6b/f133 x:0 0 0 2024-03-04T13:43:07.183 INFO:tasks.workunit.client.1.smithi169.stdout:3/844: write d7/de/d2b/d7d/d85/d13/d1a/d98/d9a/f8e [1782829,113343] 0 2024-03-04T13:43:07.184 INFO:tasks.workunit.client.0.smithi107.stdout:0/424: write da/dc/dd/f1f [636143,27741] 0 2024-03-04T13:43:07.188 INFO:tasks.workunit.client.0.smithi107.stdout:5/321: fsync d5/f9 0 2024-03-04T13:43:07.192 INFO:tasks.workunit.client.0.smithi107.stdout:4/340: mknod d0/d9/d1e/c6e 0 2024-03-04T13:43:07.192 INFO:tasks.workunit.client.1.smithi169.stdout:0/740: symlink d5/d9/dc/d12/d66/lfb 0 2024-03-04T13:43:07.193 INFO:tasks.workunit.client.0.smithi107.stdout:5/322: chown d5/d66 9376 1 2024-03-04T13:43:07.194 INFO:tasks.workunit.client.0.smithi107.stdout:6/426: mkdir df/d1b/d2c/d46/d7e/d86 0 2024-03-04T13:43:07.194 INFO:tasks.workunit.client.1.smithi169.stdout:5/950: fsync d1/d16/d17/d1b/f1e 0 2024-03-04T13:43:07.197 INFO:tasks.workunit.client.0.smithi107.stdout:9/334: unlink d4/da/d11/d20/f30 0 2024-03-04T13:43:07.197 INFO:tasks.workunit.client.0.smithi107.stdout:5/323: write d5/dc/d22/f61 [2190524,100185] 0 2024-03-04T13:43:07.200 INFO:tasks.workunit.client.0.smithi107.stdout:9/335: chown d4/da/f15 213042578 1 2024-03-04T13:43:07.201 INFO:tasks.workunit.client.0.smithi107.stdout:1/354: link d1/d10/d16/d1b/l50 d1/d46/l74 0 2024-03-04T13:43:07.201 INFO:tasks.workunit.client.1.smithi169.stdout:9/908: dread d7/dc/d12/d32/d42/d4f/f143 [0,4194304] 0 2024-03-04T13:43:07.201 INFO:tasks.workunit.client.1.smithi169.stdout:1/818: rename d4/d5/d36/c3c to d4/d5/d36/d35/d16/d81/db4/d104/c10c 0 2024-03-04T13:43:07.202 INFO:tasks.workunit.client.1.smithi169.stdout:8/868: symlink d7/d10/d22/d75/db9/df3/l116 0 2024-03-04T13:43:07.203 INFO:tasks.workunit.client.0.smithi107.stdout:1/355: write d1/d46/d6a/d4e/d60/f63 [1405138,2849] 0 2024-03-04T13:43:07.204 INFO:tasks.workunit.client.1.smithi169.stdout:7/904: readlink d2/l3 0 2024-03-04T13:43:07.205 INFO:tasks.workunit.client.0.smithi107.stdout:0/425: symlink da/dc/dd/d3e/d26/d69/l8c 0 2024-03-04T13:43:07.207 INFO:tasks.workunit.client.1.smithi169.stdout:3/845: fsync d7/de/d2b/d7d/d85/d18/d21/d3f/fc7 0 2024-03-04T13:43:07.211 INFO:tasks.workunit.client.0.smithi107.stdout:6/427: creat df/d1b/d2c/d31/f87 x:0 0 0 2024-03-04T13:43:07.216 INFO:tasks.workunit.client.1.smithi169.stdout:0/741: unlink d5/d9/d3d/d59/d64/f81 0 2024-03-04T13:43:07.223 INFO:tasks.workunit.client.0.smithi107.stdout:1/356: readlink d1/d46/d6a/df/d33/d5d/l2e 0 2024-03-04T13:43:07.226 INFO:tasks.workunit.client.1.smithi169.stdout:5/951: rename d1/d16/d17/d2d/d5a to d1/d16/d2f/d72/dab/d150 0 2024-03-04T13:43:07.226 INFO:tasks.workunit.client.1.smithi169.stdout:4/844: write d4/d1c/d58/f97 [1037968,105622] 0 2024-03-04T13:43:07.229 INFO:tasks.workunit.client.0.smithi107.stdout:6/428: dread f0 [0,4194304] 0 2024-03-04T13:43:07.229 INFO:tasks.workunit.client.1.smithi169.stdout:5/952: fsync d1/d16/d17/d28/f41 0 2024-03-04T13:43:07.231 INFO:tasks.workunit.client.0.smithi107.stdout:2/374: write f0 [1444549,18501] 0 2024-03-04T13:43:07.233 INFO:tasks.workunit.client.1.smithi169.stdout:8/869: dread d7/d10/d17/f27 [0,4194304] 0 2024-03-04T13:43:07.233 INFO:tasks.workunit.client.1.smithi169.stdout:6/851: creat d0/d32/d3e/d12e/f131 x:0 0 0 2024-03-04T13:43:07.233 INFO:tasks.workunit.client.0.smithi107.stdout:7/348: getdents d6/d13/d16/d19/d38/d6e 0 2024-03-04T13:43:07.234 INFO:tasks.workunit.client.1.smithi169.stdout:6/852: chown d0/d3c 465368 1 2024-03-04T13:43:07.236 INFO:tasks.workunit.client.0.smithi107.stdout:7/349: write d6/d17/f6d [718675,103583] 0 2024-03-04T13:43:07.237 INFO:tasks.workunit.client.0.smithi107.stdout:1/357: rename d1/d46/d6a/df/d33/d43/d5b/d61/d64 to d1/d46/d6a/d75 0 2024-03-04T13:43:07.239 INFO:tasks.workunit.client.1.smithi169.stdout:3/846: creat d7/de/d2b/d7d/d85/d13/d1a/d2d/dbe/d108/dde/f116 x:0 0 0 2024-03-04T13:43:07.242 INFO:tasks.workunit.client.0.smithi107.stdout:7/350: readlink d6/d13/d16/d19/d1e/l2f 0 2024-03-04T13:43:07.242 INFO:tasks.workunit.client.0.smithi107.stdout:6/429: symlink df/d17/l88 0 2024-03-04T13:43:07.244 INFO:tasks.workunit.client.1.smithi169.stdout:3/847: chown d7/de/d2b/d7d/d85/d13/d1a/d2d/c3e 398 1 2024-03-04T13:43:07.246 INFO:tasks.workunit.client.0.smithi107.stdout:4/341: getdents d0/d9 0 2024-03-04T13:43:07.246 INFO:tasks.workunit.client.1.smithi169.stdout:2/914: dwrite d1/d5/d22/d45/d4a/f5d [0,4194304] 0 2024-03-04T13:43:07.249 INFO:tasks.workunit.client.1.smithi169.stdout:2/915: chown d1/d5/d19/d1e/ffb 13001632 1 2024-03-04T13:43:07.249 INFO:tasks.workunit.client.0.smithi107.stdout:4/342: write d0/d9/d18/d29/f35 [2888454,59050] 0 2024-03-04T13:43:07.251 INFO:tasks.workunit.client.1.smithi169.stdout:1/819: mknod d4/d5/d2d/c10d 0 2024-03-04T13:43:07.252 INFO:tasks.workunit.client.0.smithi107.stdout:4/343: fdatasync d0/d9/de/d14/f65 0 2024-03-04T13:43:07.255 INFO:tasks.workunit.client.0.smithi107.stdout:0/426: symlink da/dc/dd/d3e/d6e/d73/d78/d86/l8d 0 2024-03-04T13:43:07.256 INFO:tasks.workunit.client.1.smithi169.stdout:9/909: truncate d7/dc/d12/d32/d42/f46 5573950 0 2024-03-04T13:43:07.256 INFO:tasks.workunit.client.0.smithi107.stdout:4/344: chown d0/d9/d18/d29/f5c 10 1 2024-03-04T13:43:07.259 INFO:tasks.workunit.client.0.smithi107.stdout:5/324: dread d5/f29 [0,4194304] 0 2024-03-04T13:43:07.261 INFO:tasks.workunit.client.1.smithi169.stdout:0/742: mkdir d5/d9/d3d/d5d/de4/d98/dfc 0 2024-03-04T13:43:07.261 INFO:tasks.workunit.client.1.smithi169.stdout:3/848: dread - d7/de/d2b/d7d/d85/d13/d5a/d65/f9b zero size 2024-03-04T13:43:07.262 INFO:tasks.workunit.client.0.smithi107.stdout:1/358: dread d1/d10/d16/d1b/f3e [0,4194304] 0 2024-03-04T13:43:07.262 INFO:tasks.workunit.client.0.smithi107.stdout:5/325: write d5/d8/d2a/f48 [89929,70652] 0 2024-03-04T13:43:07.262 INFO:tasks.workunit.client.1.smithi169.stdout:8/870: unlink d7/d10/d22/d75/l112 0 2024-03-04T13:43:07.263 INFO:tasks.workunit.client.0.smithi107.stdout:1/359: fsync d1/f31 0 2024-03-04T13:43:07.264 INFO:tasks.workunit.client.1.smithi169.stdout:9/910: symlink d7/dc/d12/d20/da6/d111/l148 0 2024-03-04T13:43:07.265 INFO:tasks.workunit.client.1.smithi169.stdout:6/853: mkdir d0/d32/d70/d12c/d132 0 2024-03-04T13:43:07.266 INFO:tasks.workunit.client.0.smithi107.stdout:1/360: write d1/d46/d6a/d4e/f6c [381604,111673] 0 2024-03-04T13:43:07.268 INFO:tasks.workunit.client.0.smithi107.stdout:5/326: rename d5/f7 to d5/d8/d14/d5a/d69/f72 0 2024-03-04T13:43:07.274 INFO:tasks.workunit.client.0.smithi107.stdout:1/361: rename d1/d46/d6a/df/d33/d5d/d24/l39 to d1/d46/d6a/df/d33/d43/d5b/l76 0 2024-03-04T13:43:07.276 INFO:tasks.workunit.client.0.smithi107.stdout:4/345: dread d0/d9/f37 [0,4194304] 0 2024-03-04T13:43:07.278 INFO:tasks.workunit.client.0.smithi107.stdout:4/346: dread - d0/d9/de/d14/dc/d21/f3f zero size 2024-03-04T13:43:07.280 INFO:tasks.workunit.client.0.smithi107.stdout:1/362: creat d1/d10/d16/d47/f77 x:0 0 0 2024-03-04T13:43:07.282 INFO:tasks.workunit.client.1.smithi169.stdout:4/845: creat d4/d11/d3b/d67/dbb/f100 x:0 0 0 2024-03-04T13:43:07.283 INFO:tasks.workunit.client.0.smithi107.stdout:4/347: mkdir d0/d9/d10/d6f 0 2024-03-04T13:43:07.284 INFO:tasks.workunit.client.1.smithi169.stdout:6/854: dread d0/d32/d3a/d43/d74/d93/dbe/ff8 [0,4194304] 0 2024-03-04T13:43:07.285 INFO:tasks.workunit.client.1.smithi169.stdout:9/911: mkdir d7/dc/d12/d149 0 2024-03-04T13:43:07.286 INFO:tasks.workunit.client.1.smithi169.stdout:0/743: dread d5/d9/dc/d12/d15/d19/d61/f23 [0,4194304] 0 2024-03-04T13:43:07.288 INFO:tasks.workunit.client.1.smithi169.stdout:9/912: stat d7/d25 0 2024-03-04T13:43:07.288 INFO:tasks.workunit.client.1.smithi169.stdout:5/953: dread d1/d16/dbd/f112 [0,4194304] 0 2024-03-04T13:43:07.292 INFO:tasks.workunit.client.1.smithi169.stdout:0/744: chown d5/d9/d3d/led 21346954 1 2024-03-04T13:43:07.292 INFO:tasks.workunit.client.1.smithi169.stdout:5/954: chown d1/d81/db1/cfd 4326 1 2024-03-04T13:43:07.294 INFO:tasks.workunit.client.1.smithi169.stdout:8/871: dwrite d7/d10/d17/d44/d4c/fa3 [4194304,4194304] 0 2024-03-04T13:43:07.299 INFO:tasks.workunit.client.1.smithi169.stdout:1/820: write d4/d5/d2d/d4e/d71/f37 [3716732,16671] 0 2024-03-04T13:43:07.301 INFO:tasks.workunit.client.1.smithi169.stdout:9/913: dread d7/f33 [0,4194304] 0 2024-03-04T13:43:07.302 INFO:tasks.workunit.client.1.smithi169.stdout:3/849: write d7/de/d2b/d7d/d85/d13/d5a/d65/da4/fd3 [546088,105207] 0 2024-03-04T13:43:07.303 INFO:tasks.workunit.client.1.smithi169.stdout:5/955: truncate d1/d16/d107/d137/f14c 910503 0 2024-03-04T13:43:07.305 INFO:tasks.workunit.client.0.smithi107.stdout:6/430: dread df/d17/f23 [0,4194304] 0 2024-03-04T13:43:07.308 INFO:tasks.workunit.client.0.smithi107.stdout:4/348: truncate d0/d9/de/d14/dc/f2d 1911364 0 2024-03-04T13:43:07.309 INFO:tasks.workunit.client.0.smithi107.stdout:5/327: truncate d5/dc/df/f37 3794037 0 2024-03-04T13:43:07.311 INFO:tasks.workunit.client.1.smithi169.stdout:6/855: write d0/d3c/d40/d52/dac/f11f [1191814,63076] 0 2024-03-04T13:43:07.311 INFO:tasks.workunit.client.0.smithi107.stdout:5/328: dread - d5/dc/f3d zero size 2024-03-04T13:43:07.313 INFO:tasks.workunit.client.0.smithi107.stdout:4/349: mkdir d0/d9/d1e/d70 0 2024-03-04T13:43:07.313 INFO:tasks.workunit.client.1.smithi169.stdout:8/872: creat d7/d10/d22/d75/d81/f117 x:0 0 0 2024-03-04T13:43:07.314 INFO:tasks.workunit.client.0.smithi107.stdout:4/350: chown d0/d9/d10/c4c 88016 1 2024-03-04T13:43:07.315 INFO:tasks.workunit.client.0.smithi107.stdout:1/363: getdents d1/d46/d6a/df/d33/d43 0 2024-03-04T13:43:07.316 INFO:tasks.workunit.client.1.smithi169.stdout:9/914: rename d7/d27/dc3 to d7/d49/d14a 0 2024-03-04T13:43:07.317 INFO:tasks.workunit.client.0.smithi107.stdout:4/351: chown d0/d9/d18/d29/f35 272 1 2024-03-04T13:43:07.318 INFO:tasks.workunit.client.1.smithi169.stdout:3/850: symlink d7/de/d2b/d7d/d85/d13/d1a/d2d/d56/d5e/d5f/l117 0 2024-03-04T13:43:07.320 INFO:tasks.workunit.client.0.smithi107.stdout:1/364: creat d1/d46/d6a/d4e/d60/f78 x:0 0 0 2024-03-04T13:43:07.321 INFO:tasks.workunit.client.1.smithi169.stdout:4/846: dread d4/f41 [0,4194304] 0 2024-03-04T13:43:07.321 INFO:tasks.workunit.client.1.smithi169.stdout:6/856: dread d0/d32/f81 [0,4194304] 0 2024-03-04T13:43:07.321 INFO:tasks.workunit.client.1.smithi169.stdout:5/956: truncate d1/d76/f14f 975754 0 2024-03-04T13:43:07.328 INFO:tasks.workunit.client.0.smithi107.stdout:5/329: dwrite d5/d8/d14/d2d/f56 [0,4194304] 0 2024-03-04T13:43:07.330 INFO:tasks.workunit.client.0.smithi107.stdout:5/330: dread - d5/d8/d2a/d43/f57 zero size 2024-03-04T13:43:07.336 INFO:tasks.workunit.client.1.smithi169.stdout:9/915: getdents d7/d2d/d48 0 2024-03-04T13:43:07.341 INFO:tasks.workunit.client.0.smithi107.stdout:1/365: dwrite d1/d46/d6a/d17/f34 [0,4194304] 0 2024-03-04T13:43:07.345 INFO:tasks.workunit.client.0.smithi107.stdout:1/366: dread - d1/d46/d6a/df/d67/f69 zero size 2024-03-04T13:43:07.346 INFO:tasks.workunit.client.1.smithi169.stdout:9/916: rmdir d7/d25 39 2024-03-04T13:43:07.349 INFO:tasks.workunit.client.1.smithi169.stdout:4/847: dread d4/f15 [0,4194304] 0 2024-03-04T13:43:07.349 INFO:tasks.workunit.client.0.smithi107.stdout:1/367: write d1/d46/d6a/d4e/f6c [857746,10934] 0 2024-03-04T13:43:07.357 INFO:tasks.workunit.client.0.smithi107.stdout:5/331: dwrite d5/d8/d14/d5a/d69/f72 [0,4194304] 0 2024-03-04T13:43:07.360 INFO:tasks.workunit.client.1.smithi169.stdout:5/957: dwrite d1/d16/d2a/d46/f130 [0,4194304] 0 2024-03-04T13:43:07.361 INFO:tasks.workunit.client.1.smithi169.stdout:0/745: dread d5/d9/dc/d12/d15/d19/f67 [0,4194304] 0 2024-03-04T13:43:07.366 INFO:tasks.workunit.client.1.smithi169.stdout:8/873: dwrite d7/d10/d17/d44/d5d/ddd/f114 [0,4194304] 0 2024-03-04T13:43:07.366 INFO:tasks.workunit.client.1.smithi169.stdout:9/917: fsync d7/d2d/d48/d66/f114 0 2024-03-04T13:43:07.369 INFO:tasks.workunit.client.0.smithi107.stdout:1/368: link d1/d46/d6a/df/d33/d5d/d24/l48 d1/d46/d6a/d75/l79 0 2024-03-04T13:43:07.369 INFO:tasks.workunit.client.1.smithi169.stdout:9/918: chown d7/cc1 50 1 2024-03-04T13:43:07.369 INFO:tasks.workunit.client.1.smithi169.stdout:3/851: dwrite d7/de/d2b/d7d/d85/d13/d1a/d2d/d56/d5e/d67/f51 [0,4194304] 0 2024-03-04T13:43:07.370 INFO:tasks.workunit.client.1.smithi169.stdout:6/857: dwrite d0/f4e [0,4194304] 0 2024-03-04T13:43:07.378 INFO:tasks.workunit.client.0.smithi107.stdout:1/369: unlink d1/d10/l3b 0 2024-03-04T13:43:07.381 INFO:tasks.workunit.client.1.smithi169.stdout:9/919: fsync d7/d25/d3c/dd2/d139/f13c 0 2024-03-04T13:43:07.382 INFO:tasks.workunit.client.0.smithi107.stdout:1/370: write d1/d46/d6a/df/d67/d6b/f6e [774375,76392] 0 2024-03-04T13:43:07.384 INFO:tasks.workunit.client.1.smithi169.stdout:5/958: truncate d1/d16/dbd/fa9 42326 0 2024-03-04T13:43:07.385 INFO:tasks.workunit.client.1.smithi169.stdout:3/852: mkdir d7/de/d2b/d7d/d85/d18/d21/d3f/d68/d118 0 2024-03-04T13:43:07.386 INFO:tasks.workunit.client.1.smithi169.stdout:6/858: mknod d0/d3c/d95/d9e/c133 0 2024-03-04T13:43:07.387 INFO:tasks.workunit.client.0.smithi107.stdout:1/371: mkdir d1/d10/d16/d47/d7a 0 2024-03-04T13:43:07.393 INFO:tasks.workunit.client.0.smithi107.stdout:1/372: mknod d1/d46/d6a/df/d33/d5d/d24/c7b 0 2024-03-04T13:43:07.394 INFO:tasks.workunit.client.1.smithi169.stdout:0/746: link d5/d9/dc/d12/d15/d19/d61/f54 d5/d9/ffd 0 2024-03-04T13:43:07.398 INFO:tasks.workunit.client.1.smithi169.stdout:5/959: mknod d1/d16/d2a/c151 0 2024-03-04T13:43:07.399 INFO:tasks.workunit.client.1.smithi169.stdout:8/874: rename d7/d10/d17/d44/d4c/f7d to d7/d10/d22/d75/f118 0 2024-03-04T13:43:07.401 INFO:tasks.workunit.client.1.smithi169.stdout:9/920: dread d7/dc/d12/d20/da6/fc5 [0,4194304] 0 2024-03-04T13:43:07.401 INFO:tasks.workunit.client.0.smithi107.stdout:1/373: dread d1/d10/d16/d1b/d40/f57 [0,4194304] 0 2024-03-04T13:43:07.402 INFO:tasks.workunit.client.1.smithi169.stdout:9/921: fsync d7/f127 0 2024-03-04T13:43:07.405 INFO:tasks.workunit.client.1.smithi169.stdout:4/848: dwrite d4/d11/d3b/d67/d7e/fe9 [0,4194304] 0 2024-03-04T13:43:07.405 INFO:tasks.workunit.client.1.smithi169.stdout:0/747: mknod d5/cfe 0 2024-03-04T13:43:07.406 INFO:tasks.workunit.client.0.smithi107.stdout:1/374: chown d1/d46/d6a/d17/f5a 1505010 1 2024-03-04T13:43:07.408 INFO:tasks.workunit.client.1.smithi169.stdout:8/875: dread - d7/d10/d17/d44/d4c/d93/ffd zero size 2024-03-04T13:43:07.409 INFO:tasks.workunit.client.1.smithi169.stdout:0/748: write d5/d9/dc/d12/d15/f40 [2564613,110767] 0 2024-03-04T13:43:07.410 INFO:tasks.workunit.client.0.smithi107.stdout:1/375: truncate d1/d46/d6a/df/f3a 660650 0 2024-03-04T13:43:07.415 INFO:tasks.workunit.client.1.smithi169.stdout:0/749: readlink d5/d9/dc/d12/d66/l6a 0 2024-03-04T13:43:07.416 INFO:tasks.workunit.client.1.smithi169.stdout:6/859: truncate d0/d32/d3a/d43/d74/f62 215835 0 2024-03-04T13:43:07.424 INFO:tasks.workunit.client.1.smithi169.stdout:3/853: creat d7/de/d2b/d7d/d85/d13/d1a/d98/d9a/d109/f119 x:0 0 0 2024-03-04T13:43:07.431 INFO:tasks.workunit.client.1.smithi169.stdout:0/750: creat d5/d9/dc/d12/d15/d19/d61/d71/fff x:0 0 0 2024-03-04T13:43:07.431 INFO:tasks.workunit.client.1.smithi169.stdout:5/960: creat d1/d16/f152 x:0 0 0 2024-03-04T13:43:07.436 INFO:tasks.workunit.client.1.smithi169.stdout:8/876: link d7/d10/d17/d44/d5d/ddd/f114 d7/d10/d17/d44/f119 0 2024-03-04T13:43:07.440 INFO:tasks.workunit.client.1.smithi169.stdout:0/751: creat d5/f100 x:0 0 0 2024-03-04T13:43:07.441 INFO:tasks.workunit.client.1.smithi169.stdout:9/922: write d7/d25/d64/fa7 [3101477,66918] 0 2024-03-04T13:43:07.443 INFO:tasks.workunit.client.1.smithi169.stdout:5/961: creat d1/d16/d2a/d46/d9e/d105/f153 x:0 0 0 2024-03-04T13:43:07.445 INFO:tasks.workunit.client.1.smithi169.stdout:3/854: fsync d7/de/d2b/d7d/d85/d13/d5a/da1/fdd 0 2024-03-04T13:43:07.447 INFO:tasks.workunit.client.1.smithi169.stdout:4/849: getdents d4/d11/d2e/d21 0 2024-03-04T13:43:07.447 INFO:tasks.workunit.client.1.smithi169.stdout:8/877: truncate d7/d10/d22/d46/da5/da6/fb4 755236 0 2024-03-04T13:43:07.449 INFO:tasks.workunit.client.1.smithi169.stdout:3/855: readlink d7/de/d2b/d7d/d85/d18/d21/d66/lcf 0 2024-03-04T13:43:07.450 INFO:tasks.workunit.client.1.smithi169.stdout:9/923: unlink d7/d25/ld7 0 2024-03-04T13:43:07.455 INFO:tasks.workunit.client.1.smithi169.stdout:0/752: mknod d5/d9/dc/d12/d15/d19/d61/d71/d8c/d90/c101 0 2024-03-04T13:43:07.457 INFO:tasks.workunit.client.1.smithi169.stdout:4/850: mkdir d4/d11/d2e/dcd/dee/d73/d101 0 2024-03-04T13:43:07.457 INFO:tasks.workunit.client.1.smithi169.stdout:6/860: dwrite d0/d32/d3a/def/fff [0,4194304] 0 2024-03-04T13:43:07.459 INFO:tasks.workunit.client.1.smithi169.stdout:8/878: symlink d7/d10/d22/d75/db9/dbe/l11a 0 2024-03-04T13:43:07.460 INFO:tasks.workunit.client.1.smithi169.stdout:9/924: rmdir d7/d2d/d48/d66 39 2024-03-04T13:43:07.464 INFO:tasks.workunit.client.1.smithi169.stdout:0/753: mknod d5/d9/dc/d12/d15/d19/d61/d71/c102 0 2024-03-04T13:43:07.468 INFO:tasks.workunit.client.1.smithi169.stdout:5/962: dwrite d1/d16/d2f/fde [0,4194304] 0 2024-03-04T13:43:07.469 INFO:tasks.workunit.client.1.smithi169.stdout:3/856: mknod d7/de/d2b/d7d/d85/d13/d5a/d65/da4/d104/c11a 0 2024-03-04T13:43:07.472 INFO:tasks.workunit.client.1.smithi169.stdout:0/754: rmdir d5/d9/dc 39 2024-03-04T13:43:07.477 INFO:tasks.workunit.client.1.smithi169.stdout:8/879: creat d7/d10/d24/f11b x:0 0 0 2024-03-04T13:43:07.479 INFO:tasks.workunit.client.1.smithi169.stdout:9/925: unlink d7/d2d/d48/d66/lfe 0 2024-03-04T13:43:07.481 INFO:tasks.workunit.client.1.smithi169.stdout:4/851: dwrite d4/d11/d2e/d6c/d78/da5/fbf [0,4194304] 0 2024-03-04T13:43:07.484 INFO:tasks.workunit.client.1.smithi169.stdout:5/963: truncate d1/d16/d17/d1b/f1e 627007 0 2024-03-04T13:43:07.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:07.491+0000 7f939a600700 1 -- 172.21.15.107:0/4214986892 --> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9394111be0 con 0x7f937806b1d0 2024-03-04T13:43:07.489 INFO:tasks.workunit.client.1.smithi169.stdout:6/861: getdents d0/d32/d3a/d43/d74 0 2024-03-04T13:43:07.489 INFO:tasks.workunit.client.0.smithi107.stdout:3/348: sync 2024-03-04T13:43:07.489 INFO:tasks.workunit.client.0.smithi107.stdout:8/394: sync 2024-03-04T13:43:07.491 INFO:tasks.workunit.client.0.smithi107.stdout:1/376: truncate d1/d46/d6a/df/d33/d43/d5b/f5f 1881775 0 2024-03-04T13:43:07.491 INFO:tasks.workunit.client.1.smithi169.stdout:9/926: rmdir d7/dc/d12/d32/d42/d10b 39 2024-03-04T13:43:07.491 INFO:tasks.workunit.client.1.smithi169.stdout:6/862: dread - d0/d3c/d40/df6/dca/f117 zero size 2024-03-04T13:43:07.491 INFO:tasks.workunit.client.1.smithi169.stdout:4/852: symlink d4/d11/d2e/dcd/dee/d73/l102 0 2024-03-04T13:43:07.493 INFO:tasks.workunit.client.1.smithi169.stdout:3/857: link d7/de/d2b/d7d/d85/fc4 d7/db8/ded/f11b 0 2024-03-04T13:43:07.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:07.495+0000 7f938b400700 1 -- 172.21.15.107:0/4214986892 <== mgr.24331 v2:172.21.15.169:6828/3524931243 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7f9394111be0 con 0x7f937806b1d0 2024-03-04T13:43:07.494 INFO:tasks.workunit.client.0.smithi107.stdout:8/395: fdatasync d6/d16/d1f/f43 0 2024-03-04T13:43:07.496 INFO:tasks.workunit.client.1.smithi169.stdout:4/853: dread - d4/d11/d2e/dcd/dee/d73/fea zero size 2024-03-04T13:43:07.497 INFO:tasks.workunit.client.1.smithi169.stdout:9/927: symlink d7/d11e/l14b 0 2024-03-04T13:43:07.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:07.500+0000 7f939a600700 1 -- 172.21.15.107:0/4214986892 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7f937806b1d0 msgr2=0x7f937806d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:43:07.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:07.500+0000 7f939a600700 1 --2- 172.21.15.107:0/4214986892 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7f937806b1d0 0x7f937806d680 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f938000b020 tx=0x7f9380009e30 comp rx=0 tx=0).stop 2024-03-04T13:43:07.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:07.500+0000 7f939a600700 1 -- 172.21.15.107:0/4214986892 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9394070860 msgr2=0x7f939410fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:43:07.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:07.500+0000 7f939a600700 1 --2- 172.21.15.107:0/4214986892 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9394070860 0x7f939410fb80 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f9384002a50 tx=0x7f93840046b0 comp rx=0 tx=0).stop 2024-03-04T13:43:07.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:07.500+0000 7f939a600700 1 -- 172.21.15.107:0/4214986892 shutdown_connections 2024-03-04T13:43:07.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:07.500+0000 7f939a600700 1 --2- 172.21.15.107:0/4214986892 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7f937806b1d0 0x7f937806d680 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:43:07.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:07.500+0000 7f939a600700 1 --2- 172.21.15.107:0/4214986892 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9394070860 0x7f939410fb80 secure :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f9384002a50 tx=0x7f93840046b0 comp rx=0 tx=0).stop 2024-03-04T13:43:07.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:07.500+0000 7f939a600700 1 --2- 172.21.15.107:0/4214986892 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9394071eb0 0x7f93941100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:43:07.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:07.500+0000 7f939a600700 1 -- 172.21.15.107:0/4214986892 >> 172.21.15.107:0/4214986892 conn(0x7f93940f8620 msgr2=0x7f9394102e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:43:07.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:07.500+0000 7f939a600700 1 -- 172.21.15.107:0/4214986892 shutdown_connections 2024-03-04T13:43:07.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:07.500+0000 7f939a600700 1 -- 172.21.15.107:0/4214986892 wait complete. 2024-03-04T13:43:07.499 INFO:tasks.workunit.client.0.smithi107.stdout:8/396: fdatasync d6/d16/d1f/f5c 0 2024-03-04T13:43:07.499 INFO:tasks.workunit.client.0.smithi107.stdout:5/332: dwrite d5/d8/d2a/f39 [0,4194304] 0 2024-03-04T13:43:07.499 INFO:tasks.workunit.client.1.smithi169.stdout:8/880: getdents d7/d10/d24/d3e 0 2024-03-04T13:43:07.500 INFO:tasks.workunit.client.0.smithi107.stdout:3/349: fdatasync d1/de/d11/f46 0 2024-03-04T13:43:07.502 INFO:tasks.workunit.client.0.smithi107.stdout:8/397: truncate d6/f54 808844 0 2024-03-04T13:43:07.503 INFO:tasks.workunit.client.0.smithi107.stdout:8/398: dread - d6/f57 zero size 2024-03-04T13:43:07.504 INFO:tasks.workunit.client.1.smithi169.stdout:8/881: dread - d7/d10/d17/d44/d4c/fa0 zero size 2024-03-04T13:43:07.509 INFO:tasks.workunit.client.1.smithi169.stdout:0/755: dwrite d5/d9/d3d/d5d/de4/d98/dc1/dc8/fee [0,4194304] 0 2024-03-04T13:43:07.516 INFO:tasks.workunit.client.1.smithi169.stdout:5/964: write d1/d16/d17/d2d/f9a [1042101,14982] 0 2024-03-04T13:43:07.517 INFO:tasks.workunit.client.0.smithi107.stdout:5/333: write d5/f6 [952363,75997] 0 2024-03-04T13:43:07.518 INFO:tasks.workunit.client.1.smithi169.stdout:6/863: write d0/d32/d3e/fb7 [457427,39763] 0 2024-03-04T13:43:07.520 INFO:tasks.workunit.client.1.smithi169.stdout:3/858: dwrite d7/de/d2b/d7d/d85/d13/d1a/d98/d9a/f8e [0,4194304] 0 2024-03-04T13:43:07.522 INFO:tasks.workunit.client.1.smithi169.stdout:8/882: readlink d7/d10/d22/d46/d49/l7c 0 2024-03-04T13:43:07.526 INFO:tasks.workunit.client.0.smithi107.stdout:5/334: read d5/f42 [2257167,71171] 0 2024-03-04T13:43:07.529 INFO:tasks.workunit.client.1.smithi169.stdout:3/859: truncate d7/db8/ded/ff6 4990475 0 2024-03-04T13:43:07.530 INFO:tasks.workunit.client.1.smithi169.stdout:5/965: mkdir d1/d16/d17/d2d/d51/d154 0 2024-03-04T13:43:07.531 INFO:tasks.workunit.client.0.smithi107.stdout:3/350: truncate d1/de/f68 1629684 0 2024-03-04T13:43:07.534 INFO:tasks.workunit.client.1.smithi169.stdout:6/864: mkdir d0/d32/d3a/d43/d74/da6/d134 0 2024-03-04T13:43:07.543 INFO:tasks.workunit.client.1.smithi169.stdout:8/883: rename d7/d10/d24/d3e/l78 to d7/d10/d24/d5c/l11c 0 2024-03-04T13:43:07.544 INFO:tasks.workunit.client.0.smithi107.stdout:8/399: link d6/da/d25/c4e d6/d16/d1f/d63/c71 0 2024-03-04T13:43:07.547 INFO:tasks.workunit.client.1.smithi169.stdout:8/884: chown d7/d10/d22/d46/d49/c109 0 1 2024-03-04T13:43:07.547 INFO:tasks.workunit.client.1.smithi169.stdout:9/928: dread d7/fb [0,4194304] 0 2024-03-04T13:43:07.548 INFO:tasks.workunit.client.1.smithi169.stdout:4/854: getdents d4/d11/d3b/d67/d7e 0 2024-03-04T13:43:07.559 INFO:tasks.workunit.client.0.smithi107.stdout:3/351: rename d1/d2d/f5c to d1/de/d18/d33/d48/d71/d4c/d54/f79 0 2024-03-04T13:43:07.560 INFO:tasks.workunit.client.0.smithi107.stdout:1/377: write d1/d46/d6a/df/d33/d43/d5b/f5f [1476472,82746] 0 2024-03-04T13:43:07.560 INFO:tasks.workunit.client.1.smithi169.stdout:8/885: rmdir d7/dd4 39 2024-03-04T13:43:07.564 INFO:tasks.workunit.client.1.smithi169.stdout:4/855: dwrite d4/fa [12582912,4194304] 0 2024-03-04T13:43:07.568 INFO:tasks.workunit.client.1.smithi169.stdout:4/856: truncate d4/d1c/d58/f97 1247019 0 2024-03-04T13:43:07.569 INFO:tasks.workunit.client.0.smithi107.stdout:5/335: creat d5/f73 x:0 0 0 2024-03-04T13:43:07.570 INFO:tasks.workunit.client.0.smithi107.stdout:3/352: rmdir d1/de/d18/d1f 39 2024-03-04T13:43:07.572 INFO:tasks.workunit.client.1.smithi169.stdout:9/929: rename d7/dc/d12/d59/f106 to d7/dc/d3e/da3/dbc/f14c 0 2024-03-04T13:43:07.573 INFO:tasks.workunit.client.0.smithi107.stdout:1/378: dread d1/d46/d6a/d17/f5a [0,4194304] 0 2024-03-04T13:43:07.574 INFO:tasks.workunit.client.1.smithi169.stdout:6/865: write d0/d32/d3e/d122/fed [444305,46979] 0 2024-03-04T13:43:07.576 INFO:tasks.workunit.client.0.smithi107.stdout:3/353: read - d1/de/d18/d1f/d42/f52 zero size 2024-03-04T13:43:07.577 INFO:tasks.workunit.client.0.smithi107.stdout:8/400: dread d6/d16/d1f/f3a [0,4194304] 0 2024-03-04T13:43:07.578 INFO:tasks.workunit.client.1.smithi169.stdout:8/886: creat d7/d10/d22/d75/db9/df3/f11d x:0 0 0 2024-03-04T13:43:07.581 INFO:tasks.workunit.client.1.smithi169.stdout:0/756: dwrite d5/d9/dc/d12/d15/d19/d61/f23 [0,4194304] 0 2024-03-04T13:43:07.582 INFO:tasks.workunit.client.0.smithi107.stdout:3/354: creat d1/de/d4b/f7a x:0 0 0 2024-03-04T13:43:07.585 INFO:tasks.workunit.client.1.smithi169.stdout:8/887: creat d7/d12/f11e x:0 0 0 2024-03-04T13:43:07.586 INFO:tasks.workunit.client.0.smithi107.stdout:3/355: creat d1/de/d18/d33/d4d/d4f/d69/f7b x:0 0 0 2024-03-04T13:43:07.590 INFO:tasks.workunit.client.1.smithi169.stdout:9/930: rename d7/dc/d12/d20/da6/faf to d7/d25/d3c/dd2/d139/f14d 0 2024-03-04T13:43:07.592 INFO:tasks.workunit.client.0.smithi107.stdout:8/401: rename d6/d16/d59/d3e/f40 to d6/da/f72 0 2024-03-04T13:43:07.594 INFO:tasks.workunit.client.1.smithi169.stdout:3/860: dwrite d7/de/d2b/d7d/d85/d13/d5a/f83 [0,4194304] 0 2024-03-04T13:43:07.595 INFO:tasks.workunit.client.1.smithi169.stdout:6/866: mkdir d0/d3c/d40/df6/dca/d121/d135 0 2024-03-04T13:43:07.596 INFO:tasks.workunit.client.0.smithi107.stdout:1/379: dread d1/d46/d6a/d4e/d60/f53 [0,4194304] 0 2024-03-04T13:43:07.597 INFO:tasks.workunit.client.0.smithi107.stdout:8/402: mkdir d6/d16/d1f/d63/d73 0 2024-03-04T13:43:07.599 INFO:tasks.workunit.client.1.smithi169.stdout:4/857: dwrite d4/fa [0,4194304] 0 2024-03-04T13:43:07.602 INFO:tasks.workunit.client.0.smithi107.stdout:1/380: mknod d1/d46/d6a/df/d33/d43/c7c 0 2024-03-04T13:43:07.602 INFO:tasks.workunit.client.0.smithi107.stdout:3/356: dwrite d1/de/d18/d33/d48/d71/f49 [0,4194304] 0 2024-03-04T13:43:07.604 INFO:tasks.workunit.client.0.smithi107.stdout:1/381: readlink d1/d46/d6a/df/d33/d43/d5b/l5e 0 2024-03-04T13:43:07.605 INFO:tasks.workunit.client.0.smithi107.stdout:8/403: creat d6/d16/d1f/d63/f74 x:0 0 0 2024-03-04T13:43:07.606 INFO:tasks.workunit.client.0.smithi107.stdout:8/404: chown d6/d16/d59/d3e/d67 1020 1 2024-03-04T13:43:07.608 INFO:tasks.workunit.client.0.smithi107.stdout:3/357: creat d1/d29/f7c x:0 0 0 2024-03-04T13:43:07.608 INFO:tasks.workunit.client.0.smithi107.stdout:9/336: sync 2024-03-04T13:43:07.609 INFO:tasks.workunit.client.1.smithi169.stdout:3/861: creat d7/de/d2b/d7d/d85/d13/d1a/d2d/d56/d5e/d5f/f11c x:0 0 0 2024-03-04T13:43:07.610 INFO:tasks.workunit.client.0.smithi107.stdout:8/405: creat d6/da/d25/d2a/f75 x:0 0 0 2024-03-04T13:43:07.614 INFO:tasks.workunit.client.0.smithi107.stdout:3/358: rename d1/de/d3f/l4a to d1/d41/l7d 0 2024-03-04T13:43:07.614 INFO:tasks.workunit.client.0.smithi107.stdout:8/406: readlink d6/lc 0 2024-03-04T13:43:07.621 INFO:tasks.workunit.client.0.smithi107.stdout:9/337: symlink d4/da/d11/d20/d23/l6a 0 2024-03-04T13:43:07.622 INFO:tasks.workunit.client.1.smithi169.stdout:8/888: dwrite d7/d10/d22/d46/ff5 [0,4194304] 0 2024-03-04T13:43:07.624 INFO:tasks.workunit.client.1.smithi169.stdout:6/867: dwrite d0/d3c/d40/f12d [0,4194304] 0 2024-03-04T13:43:07.624 INFO:tasks.workunit.client.0.smithi107.stdout:3/359: mkdir d1/de/d18/d7e 0 2024-03-04T13:43:07.624 INFO:tasks.workunit.client.0.smithi107.stdout:8/407: mknod d6/d16/d1f/d63/d73/c76 0 2024-03-04T13:43:07.626 INFO:tasks.workunit.client.0.smithi107.stdout:8/408: readlink d6/d16/d59/l49 0 2024-03-04T13:43:07.628 INFO:tasks.workunit.client.1.smithi169.stdout:8/889: rmdir d7/d10/d17/d44 39 2024-03-04T13:43:07.632 INFO:tasks.workunit.client.0.smithi107.stdout:8/409: mknod d6/d16/d1f/d63/c77 0 2024-03-04T13:43:07.635 INFO:tasks.workunit.client.0.smithi107.stdout:9/338: getdents d4/d34 0 2024-03-04T13:43:07.636 INFO:tasks.workunit.client.1.smithi169.stdout:8/890: truncate d7/d10/d17/d44/d4c/d93/ffd 999503 0 2024-03-04T13:43:07.636 INFO:tasks.workunit.client.1.smithi169.stdout:6/868: link d0/d7a/c10e d0/d3c/d40/df6/dca/d121/d135/c136 0 2024-03-04T13:43:07.639 INFO:tasks.workunit.client.1.smithi169.stdout:8/891: creat d7/d10/d22/d75/de9/f11f x:0 0 0 2024-03-04T13:43:07.640 INFO:tasks.workunit.client.1.smithi169.stdout:6/869: symlink d0/d3c/d40/l137 0 2024-03-04T13:43:07.642 INFO:tasks.workunit.client.1.smithi169.stdout:8/892: mkdir d7/d120 0 2024-03-04T13:43:07.643 INFO:tasks.workunit.client.1.smithi169.stdout:6/870: read d0/d32/d3a/f21 [1744937,77882] 0 2024-03-04T13:43:07.644 INFO:tasks.workunit.client.1.smithi169.stdout:8/893: chown d7/d10/d22/d75/db9/df3/f11d 363 1 2024-03-04T13:43:07.650 INFO:tasks.workunit.client.0.smithi107.stdout:8/410: dwrite d6/f29 [0,4194304] 0 2024-03-04T13:43:07.674 INFO:tasks.workunit.client.1.smithi169.stdout:7/905: sync 2024-03-04T13:43:07.676 INFO:tasks.workunit.client.1.smithi169.stdout:7/906: creat d2/de/dfb/f134 x:0 0 0 2024-03-04T13:43:07.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:07.687+0000 7f029fe00700 1 -- 172.21.15.107:0/4232541707 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f02980fe0e0 msgr2=0x7f02980fe550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:43:07.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:07.687+0000 7f029fe00700 1 --2- 172.21.15.107:0/4232541707 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f02980fe0e0 0x7f02980fe550 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f0284004940 tx=0x7f0284009e30 comp rx=0 tx=0).stop 2024-03-04T13:43:07.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:07.687+0000 7f029fe00700 1 -- 172.21.15.107:0/4232541707 shutdown_connections 2024-03-04T13:43:07.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:07.687+0000 7f029fe00700 1 --2- 172.21.15.107:0/4232541707 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f02980fe0e0 0x7f02980fe550 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:43:07.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:07.687+0000 7f029fe00700 1 --2- 172.21.15.107:0/4232541707 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02980fcef0 0x7f02980fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:43:07.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:07.687+0000 7f029fe00700 1 -- 172.21.15.107:0/4232541707 >> 172.21.15.107:0/4232541707 conn(0x7f02980f8620 msgr2=0x7f02980faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:43:07.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:07.687+0000 7f029fe00700 1 -- 172.21.15.107:0/4232541707 shutdown_connections 2024-03-04T13:43:07.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:07.687+0000 7f029fe00700 1 -- 172.21.15.107:0/4232541707 wait complete. 2024-03-04T13:43:07.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:07.688+0000 7f029fe00700 1 Processor -- start 2024-03-04T13:43:07.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:07.688+0000 7f029fe00700 1 -- start start 2024-03-04T13:43:07.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:07.688+0000 7f029fe00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02980fcef0 0x7f029810fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:43:07.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:07.688+0000 7f029fe00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f02980fe0e0 0x7f02981100c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:43:07.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:07.688+0000 7f029fe00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0298110750 con 0x7f02980fe0e0 2024-03-04T13:43:07.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:07.688+0000 7f029fe00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0298110f90 con 0x7f02980fcef0 2024-03-04T13:43:07.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:07.689+0000 7f029ca00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f02980fe0e0 0x7f02981100c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:43:07.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:07.689+0000 7f029ca00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f02980fe0e0 0x7f02981100c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52030/0 (socket says 172.21.15.107:52030) 2024-03-04T13:43:07.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:07.689+0000 7f029ca00700 1 -- 172.21.15.107:0/4149826752 learned_addr learned my addr 172.21.15.107:0/4149826752 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:43:07.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:07.689+0000 7f029d400700 1 --2- 172.21.15.107:0/4149826752 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02980fcef0 0x7f029810fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:43:07.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:07.689+0000 7f029ca00700 1 -- 172.21.15.107:0/4149826752 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02980fcef0 msgr2=0x7f029810fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:43:07.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:07.689+0000 7f029ca00700 1 --2- 172.21.15.107:0/4149826752 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02980fcef0 0x7f029810fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:43:07.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:07.689+0000 7f029ca00700 1 -- 172.21.15.107:0/4149826752 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0284005040 con 0x7f02980fe0e0 2024-03-04T13:43:07.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:07.689+0000 7f029d400700 1 --2- 172.21.15.107:0/4149826752 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02980fcef0 0x7f029810fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T13:43:07.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:07.689+0000 7f029ca00700 1 --2- 172.21.15.107:0/4149826752 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f02980fe0e0 0x7f02981100c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f0284009fd0 tx=0x7f0284009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:43:07.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:07.690+0000 7f0295600700 1 -- 172.21.15.107:0/4149826752 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f028400e040 con 0x7f02980fe0e0 2024-03-04T13:43:07.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:07.690+0000 7f0295600700 1 -- 172.21.15.107:0/4149826752 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f028400bcc0 con 0x7f02980fe0e0 2024-03-04T13:43:07.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:07.690+0000 7f029fe00700 1 -- 172.21.15.107:0/4149826752 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0298111210 con 0x7f02980fe0e0 2024-03-04T13:43:07.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:07.690+0000 7f0295600700 1 -- 172.21.15.107:0/4149826752 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f028401c630 con 0x7f02980fe0e0 2024-03-04T13:43:07.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:07.690+0000 7f029fe00700 1 -- 172.21.15.107:0/4149826752 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0298111750 con 0x7f02980fe0e0 2024-03-04T13:43:07.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:07.691+0000 7f028aa00700 1 -- 172.21.15.107:0/4149826752 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02740052b0 con 0x7f02980fe0e0 2024-03-04T13:43:07.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:07.692+0000 7f0295600700 1 -- 172.21.15.107:0/4149826752 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) v1 ==== 88911+0+0 (secure 0 0 0) 0x7f0284013980 con 0x7f02980fe0e0 2024-03-04T13:43:07.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:07.693+0000 7f0295600700 1 --2- 172.21.15.107:0/4149826752 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7f028006b1d0 0x7f028006d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:43:07.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:07.693+0000 7f0295600700 1 -- 172.21.15.107:0/4149826752 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f028408aff0 con 0x7f02980fe0e0 2024-03-04T13:43:07.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:07.693+0000 7f029d400700 1 --2- 172.21.15.107:0/4149826752 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7f028006b1d0 0x7f028006d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:43:07.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:07.694+0000 7f029d400700 1 --2- 172.21.15.107:0/4149826752 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7f028006b1d0 0x7f028006d680 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f028c006fd0 tx=0x7f028c008040 comp rx=0 tx=0).ready entity=mgr.24331 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:43:07.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:07.697+0000 7f0295600700 1 -- 172.21.15.107:0/4149826752 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f02840be620 con 0x7f02980fe0e0 2024-03-04T13:43:07.713 INFO:tasks.workunit.client.1.smithi169.stdout:9/931: write d7/f11 [1987113,79534] 0 2024-03-04T13:43:07.713 INFO:tasks.workunit.client.1.smithi169.stdout:0/757: write d5/d9/d3d/d5d/de4/f8a [320922,63451] 0 2024-03-04T13:43:07.715 INFO:tasks.workunit.client.1.smithi169.stdout:0/758: chown d5/d9/d3d 6623 1 2024-03-04T13:43:07.717 INFO:tasks.workunit.client.1.smithi169.stdout:0/759: creat d5/f103 x:0 0 0 2024-03-04T13:43:07.719 INFO:tasks.workunit.client.0.smithi107.stdout:9/339: getdents d4/da/d11/d20/d23 0 2024-03-04T13:43:07.720 INFO:tasks.workunit.client.1.smithi169.stdout:9/932: symlink d7/dc/d12/d32/d12c/l14e 0 2024-03-04T13:43:07.723 INFO:tasks.workunit.client.0.smithi107.stdout:9/340: mknod d4/da/d11/d20/d23/d2c/c6b 0 2024-03-04T13:43:07.724 INFO:tasks.workunit.client.0.smithi107.stdout:3/360: write d1/de/d18/f59 [2491534,128269] 0 2024-03-04T13:43:07.727 INFO:tasks.workunit.client.1.smithi169.stdout:4/858: dwrite d4/d11/d2e/d39/d48/f65 [0,4194304] 0 2024-03-04T13:43:07.728 INFO:tasks.workunit.client.0.smithi107.stdout:3/361: write d1/de/d18/d33/d48/d71/f72 [246694,24398] 0 2024-03-04T13:43:07.729 INFO:tasks.workunit.client.0.smithi107.stdout:3/362: chown d1/f2b 84836569 1 2024-03-04T13:43:07.731 INFO:tasks.workunit.client.0.smithi107.stdout:9/341: creat d4/da/d11/d20/d23/f6c x:0 0 0 2024-03-04T13:43:07.732 INFO:tasks.workunit.client.1.smithi169.stdout:4/859: chown d4/d11/d2e/d39/d48/cf2 7254820 1 2024-03-04T13:43:07.733 INFO:tasks.workunit.client.0.smithi107.stdout:3/363: creat d1/d2d/f7f x:0 0 0 2024-03-04T13:43:07.733 INFO:tasks.workunit.client.1.smithi169.stdout:0/760: dread d5/d9/f6e [0,4194304] 0 2024-03-04T13:43:07.737 INFO:tasks.workunit.client.0.smithi107.stdout:3/364: chown d1/de/d18/d33/d6b/f6c 54893687 1 2024-03-04T13:43:07.739 INFO:tasks.workunit.client.1.smithi169.stdout:0/761: symlink d5/d4c/d65/d95/db0/l104 0 2024-03-04T13:43:07.739 INFO:tasks.workunit.client.1.smithi169.stdout:8/894: dwrite d7/d10/d17/d44/d5d/f88 [0,4194304] 0 2024-03-04T13:43:07.740 INFO:tasks.workunit.client.1.smithi169.stdout:4/860: creat d4/d11/d2e/d6c/d78/da5/d77/dc0/df5/f103 x:0 0 0 2024-03-04T13:43:07.741 INFO:tasks.workunit.client.0.smithi107.stdout:3/365: creat d1/de/d3f/f80 x:0 0 0 2024-03-04T13:43:07.745 INFO:tasks.workunit.client.1.smithi169.stdout:0/762: creat d5/d9/dc/d12/d15/d19/d61/d71/dbb/f105 x:0 0 0 2024-03-04T13:43:07.748 INFO:tasks.workunit.client.1.smithi169.stdout:9/933: write d7/d27/f76 [366059,126458] 0 2024-03-04T13:43:07.751 INFO:tasks.workunit.client.1.smithi169.stdout:4/861: read d4/d11/d2e/dcd/dee/d5c/f8c [2973837,104989] 0 2024-03-04T13:43:07.754 INFO:tasks.workunit.client.1.smithi169.stdout:9/934: creat d7/dc/d3e/da3/dbc/f14f x:0 0 0 2024-03-04T13:43:07.755 INFO:tasks.workunit.client.1.smithi169.stdout:4/862: rmdir d4/d11/d3b/d67 39 2024-03-04T13:43:07.755 INFO:tasks.workunit.client.1.smithi169.stdout:0/763: read d5/d9/dc/d12/d15/d19/d61/d71/d8c/d90/f97 [458304,14481] 0 2024-03-04T13:43:07.757 INFO:tasks.workunit.client.1.smithi169.stdout:6/871: dread d0/d32/d3a/f27 [0,4194304] 0 2024-03-04T13:43:07.759 INFO:tasks.workunit.client.1.smithi169.stdout:6/872: fdatasync d0/d3c/d95/d9e/ffb 0 2024-03-04T13:43:07.759 INFO:tasks.workunit.client.1.smithi169.stdout:0/764: fdatasync d5/d9/dc/d12/d15/d19/d61/d71/d8c/d90/fc0 0 2024-03-04T13:43:07.760 INFO:tasks.workunit.client.1.smithi169.stdout:4/863: unlink d4/d1c/d75/c94 0 2024-03-04T13:43:07.766 INFO:tasks.workunit.client.0.smithi107.stdout:9/342: dread d4/da/f1b [0,4194304] 0 2024-03-04T13:43:07.767 INFO:tasks.workunit.client.1.smithi169.stdout:0/765: dread d5/d9/dc/fdf [0,4194304] 0 2024-03-04T13:43:07.767 INFO:tasks.workunit.client.1.smithi169.stdout:6/873: unlink d0/d3c/d95/dbd/fc5 0 2024-03-04T13:43:07.771 INFO:tasks.workunit.client.0.smithi107.stdout:0/427: sync 2024-03-04T13:43:07.771 INFO:tasks.workunit.client.0.smithi107.stdout:2/375: sync 2024-03-04T13:43:07.772 INFO:tasks.workunit.client.0.smithi107.stdout:7/351: sync 2024-03-04T13:43:07.773 INFO:tasks.workunit.client.1.smithi169.stdout:4/864: creat d4/d11/d2e/d6c/f104 x:0 0 0 2024-03-04T13:43:07.775 INFO:tasks.workunit.client.0.smithi107.stdout:0/428: dread da/f62 [0,4194304] 0 2024-03-04T13:43:07.779 INFO:tasks.workunit.client.1.smithi169.stdout:6/874: rename d0/d32/d3e/l90 to d0/d7a/l138 0 2024-03-04T13:43:07.780 INFO:tasks.workunit.client.0.smithi107.stdout:2/376: mknod da/d6c/c80 0 2024-03-04T13:43:07.780 INFO:tasks.workunit.client.1.smithi169.stdout:6/875: stat d0/d3c/d40/l137 0 2024-03-04T13:43:07.782 INFO:tasks.workunit.client.0.smithi107.stdout:9/343: symlink d4/da/d11/d20/d64/l6d 0 2024-03-04T13:43:07.782 INFO:tasks.workunit.client.0.smithi107.stdout:3/366: write d1/d2d/f51 [3871529,7549] 0 2024-03-04T13:43:07.786 INFO:tasks.workunit.client.0.smithi107.stdout:0/429: symlink da/dc/dd/d3e/d26/d88/l8e 0 2024-03-04T13:43:07.787 INFO:tasks.workunit.client.0.smithi107.stdout:2/377: unlink da/d49/d5c/d6a/c5a 0 2024-03-04T13:43:07.788 INFO:tasks.workunit.client.0.smithi107.stdout:2/378: readlink da/df/l2c 0 2024-03-04T13:43:07.792 INFO:tasks.workunit.client.1.smithi169.stdout:9/935: dwrite d7/dc/d3e/da3/fad [0,4194304] 0 2024-03-04T13:43:07.794 INFO:tasks.workunit.client.1.smithi169.stdout:6/876: dwrite d0/d32/d3e/fb7 [0,4194304] 0 2024-03-04T13:43:07.796 INFO:tasks.workunit.client.1.smithi169.stdout:6/877: read d0/d32/f89 [568100,48253] 0 2024-03-04T13:43:07.798 INFO:tasks.workunit.client.1.smithi169.stdout:6/878: chown d0/d3c/d40/d52/d98/fd5 476 1 2024-03-04T13:43:07.825 INFO:tasks.workunit.client.1.smithi169.stdout:6/879: dread d0/d3c/d40/f9d [0,4194304] 0 2024-03-04T13:43:07.840 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:07 smithi107 ceph-mon[28739]: from='client.14676 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:43:07.840 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:07 smithi107 ceph-mon[28739]: from='mgr.24331 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:43:07.851 INFO:tasks.workunit.client.1.smithi169.stdout:2/916: sync 2024-03-04T13:43:07.851 INFO:tasks.workunit.client.1.smithi169.stdout:1/821: sync 2024-03-04T13:43:07.862 INFO:tasks.workunit.client.0.smithi107.stdout:6/431: sync 2024-03-04T13:43:07.862 INFO:tasks.workunit.client.0.smithi107.stdout:4/352: sync 2024-03-04T13:43:07.864 INFO:tasks.workunit.client.1.smithi169.stdout:0/766: rename d5/d9/dc/l45 to d5/d9/dc/d12/d15/d19/d61/d71/dbb/l106 0 2024-03-04T13:43:07.869 INFO:tasks.workunit.client.1.smithi169.stdout:9/936: creat d7/dc/d12/d20/d4e/f150 x:0 0 0 2024-03-04T13:43:07.872 INFO:tasks.workunit.client.0.smithi107.stdout:3/367: mkdir d1/d41/d81 0 2024-03-04T13:43:07.874 INFO:tasks.workunit.client.0.smithi107.stdout:3/368: chown d1/d1e/f22 2034 1 2024-03-04T13:43:07.877 INFO:tasks.workunit.client.0.smithi107.stdout:0/430: mknod da/d7e/c8f 0 2024-03-04T13:43:07.882 INFO:tasks.workunit.client.0.smithi107.stdout:6/432: dread df/f29 [0,4194304] 0 2024-03-04T13:43:07.884 INFO:tasks.workunit.client.0.smithi107.stdout:2/379: rename da/d38/f56 to da/d44/f81 0 2024-03-04T13:43:07.885 INFO:tasks.workunit.client.0.smithi107.stdout:6/433: dread - df/d1b/d2c/d31/f87 zero size 2024-03-04T13:43:07.895 INFO:tasks.workunit.client.1.smithi169.stdout:1/822: rmdir d4/d5/d2d/d4e/da5 39 2024-03-04T13:43:07.898 INFO:tasks.workunit.client.0.smithi107.stdout:7/352: truncate d6/d13/f18 2271190 0 2024-03-04T13:43:07.900 INFO:tasks.workunit.client.1.smithi169.stdout:0/767: fsync d5/d9/dc/d12/d15/d19/f28 0 2024-03-04T13:43:07.905 INFO:tasks.workunit.client.0.smithi107.stdout:3/369: symlink d1/d1e/d26/l82 0 2024-03-04T13:43:07.907 INFO:tasks.workunit.client.0.smithi107.stdout:3/370: fdatasync d1/de/d11/f45 0 2024-03-04T13:43:07.909 INFO:tasks.workunit.client.0.smithi107.stdout:0/431: unlink da/dc/dd/d3e/d26/d88/c8a 0 2024-03-04T13:43:07.913 INFO:tasks.workunit.client.1.smithi169.stdout:1/823: dwrite d4/d5/d2d/d4e/d71/d26/fdf [0,4194304] 0 2024-03-04T13:43:07.915 INFO:tasks.workunit.client.1.smithi169.stdout:2/917: write d1/d5/d22/d45/f62 [6081711,38544] 0 2024-03-04T13:43:07.915 INFO:tasks.workunit.client.1.smithi169.stdout:6/880: mknod d0/d32/d3a/d43/d74/d93/dee/d118/c139 0 2024-03-04T13:43:07.919 INFO:tasks.workunit.client.1.smithi169.stdout:0/768: symlink d5/d9/d3d/l107 0 2024-03-04T13:43:07.923 INFO:tasks.workunit.client.0.smithi107.stdout:5/336: sync 2024-03-04T13:43:07.923 INFO:tasks.workunit.client.0.smithi107.stdout:8/411: sync 2024-03-04T13:43:07.923 INFO:tasks.workunit.client.0.smithi107.stdout:1/382: sync 2024-03-04T13:43:07.923 INFO:tasks.workunit.client.0.smithi107.stdout:9/344: creat d4/da/d11/d65/f6e x:0 0 0 2024-03-04T13:43:07.924 INFO:tasks.workunit.client.0.smithi107.stdout:5/337: stat d5/d8/f63 0 2024-03-04T13:43:07.930 INFO:tasks.workunit.client.1.smithi169.stdout:2/918: write d1/d5/d22/d45/df3/f130 [25786,126428] 0 2024-03-04T13:43:07.934 INFO:tasks.workunit.client.0.smithi107.stdout:7/353: rename d6/d1d/d36/f61 to d6/d13/d16/d19/d39/f70 0 2024-03-04T13:43:07.935 INFO:tasks.workunit.client.0.smithi107.stdout:1/383: fsync d1/d46/d6a/df/d33/d5d/d24/f62 0 2024-03-04T13:43:07.935 INFO:tasks.workunit.client.1.smithi169.stdout:0/769: stat d5/d9/dc/d12/d15/d19/d77/d94/fab 0 2024-03-04T13:43:07.937 INFO:tasks.workunit.client.0.smithi107.stdout:1/384: chown d1/d46/d6a/df/d33/d5d/l58 0 1 2024-03-04T13:43:07.938 INFO:tasks.workunit.client.0.smithi107.stdout:0/432: creat da/dc/dd/d3e/d6e/d73/d78/d86/f90 x:0 0 0 2024-03-04T13:43:07.940 INFO:tasks.workunit.client.0.smithi107.stdout:0/433: write da/f3c [2105972,103175] 0 2024-03-04T13:43:07.940 INFO:tasks.workunit.client.1.smithi169.stdout:2/919: write d1/d5/d22/d45/f72 [707475,70212] 0 2024-03-04T13:43:07.940 INFO:tasks.workunit.client.1.smithi169.stdout:9/937: dwrite d7/dc/d12/d59/f12a [4194304,4194304] 0 2024-03-04T13:43:07.943 INFO:tasks.workunit.client.0.smithi107.stdout:6/434: unlink df/d17/d24/f2b 0 2024-03-04T13:43:07.946 INFO:tasks.workunit.client.0.smithi107.stdout:6/435: dread df/d1b/d2c/d31/f48 [0,4194304] 0 2024-03-04T13:43:07.946 INFO:tasks.workunit.client.0.smithi107.stdout:3/371: dread d1/fc [0,4194304] 0 2024-03-04T13:43:07.953 INFO:tasks.workunit.client.0.smithi107.stdout:5/338: write d5/f42 [1197625,85457] 0 2024-03-04T13:43:07.959 INFO:tasks.workunit.client.1.smithi169.stdout:1/824: dread d4/d5/f5a [0,4194304] 0 2024-03-04T13:43:07.959 INFO:tasks.workunit.client.1.smithi169.stdout:0/770: rename d5/d9/dc/d12/c1e to d5/d9/dc/d12/d15/d19/d77/d94/c108 0 2024-03-04T13:43:07.960 INFO:tasks.workunit.client.1.smithi169.stdout:9/938: mknod d7/d140/c151 0 2024-03-04T13:43:07.962 INFO:tasks.workunit.client.0.smithi107.stdout:6/436: dwrite df/d2e/d36/d5b/f6c [0,4194304] 0 2024-03-04T13:43:07.963 INFO:tasks.workunit.client.1.smithi169.stdout:6/881: dread d0/d32/d3a/f4f [0,4194304] 0 2024-03-04T13:43:07.966 INFO:tasks.workunit.client.0.smithi107.stdout:6/437: chown df/d1b/d2c/c56 12 1 2024-03-04T13:43:07.969 INFO:tasks.workunit.client.1.smithi169.stdout:6/882: mkdir d0/d3c/d40/d13a 0 2024-03-04T13:43:07.976 INFO:tasks.workunit.client.0.smithi107.stdout:1/385: unlink d1/d46/d6a/d17/c1d 0 2024-03-04T13:43:07.977 INFO:tasks.workunit.client.1.smithi169.stdout:0/771: dread d5/d9/f6e [0,4194304] 0 2024-03-04T13:43:07.977 INFO:tasks.workunit.client.0.smithi107.stdout:1/386: dread - d1/f72 zero size 2024-03-04T13:43:07.980 INFO:tasks.workunit.client.0.smithi107.stdout:0/434: rename da/d7e/c8f to da/dc/dd/d3e/d26/d88/c91 0 2024-03-04T13:43:07.981 INFO:tasks.workunit.client.0.smithi107.stdout:2/380: link da/d34/l42 da/d44/l82 0 2024-03-04T13:43:07.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:07.985+0000 7f028aa00700 1 -- 172.21.15.107:0/4149826752 --> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f0274000bc0 con 0x7f028006b1d0 2024-03-04T13:43:07.983 INFO:tasks.workunit.client.0.smithi107.stdout:0/435: read da/dc/dd/d3e/d26/f58 [34800,28117] 0 2024-03-04T13:43:07.983 INFO:tasks.workunit.client.0.smithi107.stdout:2/381: chown da/d38/d3a/l45 1 1 2024-03-04T13:43:07.985 INFO:tasks.workunit.client.1.smithi169.stdout:3/862: sync 2024-03-04T13:43:07.985 INFO:tasks.workunit.client.1.smithi169.stdout:5/966: sync 2024-03-04T13:43:07.985 INFO:tasks.workunit.client.0.smithi107.stdout:3/372: rmdir d1/d1e/d26 39 2024-03-04T13:43:07.990 INFO:tasks.workunit.client.0.smithi107.stdout:3/373: chown d1/de/d18/d33/d48/d71/f72 270371979 1 2024-03-04T13:43:07.992 INFO:tasks.workunit.client.0.smithi107.stdout:5/339: symlink d5/d8/d2a/d43/l74 0 2024-03-04T13:43:07.993 INFO:tasks.workunit.client.1.smithi169.stdout:1/825: dread d4/d5/d2d/d4e/d71/d26/daf/fee [0,4194304] 0 2024-03-04T13:43:07.994 INFO:tasks.workunit.client.0.smithi107.stdout:5/340: chown d5/d8/d2a/d43/c52 7 1 2024-03-04T13:43:07.994 INFO:tasks.workunit.client.1.smithi169.stdout:5/967: unlink d1/d16/d17/f4b 0 2024-03-04T13:43:07.995 INFO:tasks.workunit.client.0.smithi107.stdout:5/341: readlink d5/dc/df/l3f 0 2024-03-04T13:43:07.997 INFO:tasks.workunit.client.0.smithi107.stdout:5/342: chown d5/dc/c1e 119366 1 2024-03-04T13:43:07.997 INFO:tasks.workunit.client.1.smithi169.stdout:9/939: truncate d7/d25/f3d 1207102 0 2024-03-04T13:43:07.997 INFO:tasks.workunit.client.1.smithi169.stdout:1/826: dread - d4/d5/d36/d75/f107 zero size 2024-03-04T13:43:07.998 INFO:tasks.workunit.client.0.smithi107.stdout:0/436: dwrite da/dc/dd/d3e/d26/f64 [0,4194304] 0 2024-03-04T13:43:07.999 INFO:tasks.workunit.client.1.smithi169.stdout:2/920: dwrite d1/d5/d19/d3c/db8/de1/fe6 [0,4194304] 0 2024-03-04T13:43:08.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:08.003+0000 7f0295600700 1 -- 172.21.15.107:0/4149826752 <== mgr.24331 v2:172.21.15.169:6828/3524931243 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3410 (secure 0 0 0) 0x7f0274000bc0 con 0x7f028006b1d0 2024-03-04T13:43:08.001 INFO:tasks.workunit.client.0.smithi107.stdout:0/437: fsync da/dc/dd/d3e/d6e/d73/d78/f8b 0 2024-03-04T13:43:08.002 INFO:tasks.workunit.client.1.smithi169.stdout:9/940: chown d7/dc/d12/d32/ca5 513445 1 2024-03-04T13:43:08.002 INFO:tasks.workunit.client.1.smithi169.stdout:5/968: truncate d1/d16/d17/d28/dfe/f110 666138 0 2024-03-04T13:43:08.002 INFO:tasks.workunit.client.1.smithi169.stdout:1/827: write d4/d5/d36/d35/df/fd6 [772939,32717] 0 2024-03-04T13:43:08.005 INFO:teuthology.orchestra.run.smithi107.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-04T13:43:08.005 INFO:teuthology.orchestra.run.smithi107.stdout:alertmanager.smithi107 smithi107 *:9093,9094 starting - - - - 2024-03-04T13:43:08.005 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi107 smithi107 running (11m) 91s ago 11m 6665k - 17.2.7-545-g42608bb1 9eec3f22859d 03e1570c2eff 2024-03-04T13:43:08.005 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi169 smithi169 running (10m) 90s ago 10m 6656k - 17.2.7-545-g42608bb1 9eec3f22859d 9c8881a5338e 2024-03-04T13:43:08.005 INFO:teuthology.orchestra.run.smithi107.stdout:grafana.smithi107 smithi107 *:3000 starting - - - - 2024-03-04T13:43:08.005 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.riexmm smithi107 running (4m) 91s ago 4m 11.4M - 17.2.7-545-g42608bb1 9eec3f22859d fc921f7d4328 2024-03-04T13:43:08.005 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.thycho smithi107 running (4m) 91s ago 4m 85.9M - 17.2.7-545-g42608bb1 9eec3f22859d 4069c6e0e063 2024-03-04T13:43:08.006 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi169.bxitzk smithi169 running (4m) 90s ago 4m 11.5M - 17.2.7-545-g42608bb1 9eec3f22859d 7d4115228c7c 2024-03-04T13:43:08.006 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi169.jusudl smithi169 running (4m) 90s ago 4m 101M - 17.2.7-545-g42608bb1 9eec3f22859d 2d2f2cc452e8 2024-03-04T13:43:08.006 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi107.wpzgwl smithi107 *:9283 running (13m) 91s ago 13m 367M - 17.2.7-545-g42608bb1 9eec3f22859d 4831dce06263 2024-03-04T13:43:08.006 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi169.eihczr smithi169 *:8443,9283 running (118s) 90s ago 10m 489M - 19.0.0-1845-g717ce3cc d71b839abdf8 18b4dfa896b5 2024-03-04T13:43:08.006 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi107 smithi107 running (13m) 91s ago 13m 50.0M 2048M 17.2.7-545-g42608bb1 9eec3f22859d 49f5da990d35 2024-03-04T13:43:08.006 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi169 smithi169 running (10m) 90s ago 10m 38.2M 2048M 17.2.7-545-g42608bb1 9eec3f22859d ba4ee916297f 2024-03-04T13:43:08.006 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi107 smithi107 *:9100 starting - - - - 2024-03-04T13:43:08.006 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi169 smithi169 *:9100 starting - - - - 2024-03-04T13:43:08.006 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0 smithi107 running (9m) 91s ago 9m 64.8M 1558M 17.2.7-545-g42608bb1 9eec3f22859d c0605b68f696 2024-03-04T13:43:08.006 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1 smithi107 running (8m) 91s ago 8m 64.4M 1558M 17.2.7-545-g42608bb1 9eec3f22859d 5fb2c0dd5769 2024-03-04T13:43:08.006 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2 smithi107 running (7m) 91s ago 7m 59.0M 1558M 17.2.7-545-g42608bb1 9eec3f22859d 61f30f643549 2024-03-04T13:43:08.006 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3 smithi169 running (7m) 90s ago 7m 69.7M 2582M 17.2.7-545-g42608bb1 9eec3f22859d cd91d8694764 2024-03-04T13:43:08.006 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4 smithi169 running (6m) 90s ago 6m 64.6M 2582M 17.2.7-545-g42608bb1 9eec3f22859d 08545e3d0659 2024-03-04T13:43:08.006 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5 smithi169 running (6m) 90s ago 6m 62.3M 2582M 17.2.7-545-g42608bb1 9eec3f22859d dcffe6cb3eac 2024-03-04T13:43:08.007 INFO:teuthology.orchestra.run.smithi107.stdout:prometheus.smithi107 smithi107 *:9095 starting - - - - 2024-03-04T13:43:08.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:08.005+0000 7f028aa00700 1 -- 172.21.15.107:0/4149826752 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7f028006b1d0 msgr2=0x7f028006d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:43:08.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:08.005+0000 7f028aa00700 1 --2- 172.21.15.107:0/4149826752 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7f028006b1d0 0x7f028006d680 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f028c006fd0 tx=0x7f028c008040 comp rx=0 tx=0).stop 2024-03-04T13:43:08.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:08.005+0000 7f028aa00700 1 -- 172.21.15.107:0/4149826752 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f02980fe0e0 msgr2=0x7f02981100c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:43:08.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:08.005+0000 7f028aa00700 1 --2- 172.21.15.107:0/4149826752 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f02980fe0e0 0x7f02981100c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f0284009fd0 tx=0x7f0284009e30 comp rx=0 tx=0).stop 2024-03-04T13:43:08.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:08.006+0000 7f028aa00700 1 -- 172.21.15.107:0/4149826752 shutdown_connections 2024-03-04T13:43:08.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:08.006+0000 7f028aa00700 1 --2- 172.21.15.107:0/4149826752 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7f028006b1d0 0x7f028006d680 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:43:08.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:08.006+0000 7f028aa00700 1 --2- 172.21.15.107:0/4149826752 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02980fcef0 0x7f029810fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:43:08.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:08.006+0000 7f028aa00700 1 --2- 172.21.15.107:0/4149826752 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f02980fe0e0 0x7f02981100c0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:43:08.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:08.006+0000 7f028aa00700 1 -- 172.21.15.107:0/4149826752 >> 172.21.15.107:0/4149826752 conn(0x7f02980f8620 msgr2=0x7f02980ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:43:08.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:08.006+0000 7f028aa00700 1 -- 172.21.15.107:0/4149826752 shutdown_connections 2024-03-04T13:43:08.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:08.006+0000 7f028aa00700 1 -- 172.21.15.107:0/4149826752 wait complete. 2024-03-04T13:43:08.008 INFO:tasks.workunit.client.0.smithi107.stdout:9/345: unlink d4/da/d11/d20/d2f/c5d 0 2024-03-04T13:43:08.008 INFO:tasks.workunit.client.1.smithi169.stdout:3/863: symlink d7/de/d2b/d7d/d85/d13/d1a/d2d/d56/d5e/d5f/l11d 0 2024-03-04T13:43:08.009 INFO:tasks.workunit.client.0.smithi107.stdout:7/354: mkdir d6/d13/d16/d19/d1e/d66/d71 0 2024-03-04T13:43:08.011 INFO:tasks.workunit.client.0.smithi107.stdout:7/355: fdatasync d6/d13/d16/d19/d1e/f6a 0 2024-03-04T13:43:08.012 INFO:tasks.workunit.client.1.smithi169.stdout:0/772: getdents d5/d4c/d65/d95/db0 0 2024-03-04T13:43:08.013 INFO:tasks.workunit.client.0.smithi107.stdout:7/356: stat d6/d1d/d36 0 2024-03-04T13:43:08.016 INFO:tasks.workunit.client.1.smithi169.stdout:6/883: dread d0/d32/d3e/f66 [0,4194304] 0 2024-03-04T13:43:08.018 INFO:tasks.workunit.client.1.smithi169.stdout:2/921: rmdir d1/d5/d19/d1e 39 2024-03-04T13:43:08.019 INFO:tasks.workunit.client.1.smithi169.stdout:7/907: sync 2024-03-04T13:43:08.021 INFO:tasks.workunit.client.0.smithi107.stdout:2/382: creat da/d34/f83 x:0 0 0 2024-03-04T13:43:08.022 INFO:tasks.workunit.client.1.smithi169.stdout:9/941: symlink d7/d25/d3c/l152 0 2024-03-04T13:43:08.023 INFO:tasks.workunit.client.0.smithi107.stdout:2/383: write da/d34/f36 [5004194,123944] 0 2024-03-04T13:43:08.027 INFO:tasks.workunit.client.0.smithi107.stdout:3/374: creat d1/de/d4b/f83 x:0 0 0 2024-03-04T13:43:08.027 INFO:tasks.workunit.client.1.smithi169.stdout:0/773: dread d5/f8 [0,4194304] 0 2024-03-04T13:43:08.029 INFO:tasks.workunit.client.0.smithi107.stdout:7/357: dwrite d6/d13/d16/d19/d1e/f42 [0,4194304] 0 2024-03-04T13:43:08.030 INFO:tasks.workunit.client.0.smithi107.stdout:9/346: symlink d4/da/d11/d20/d23/d2c/l6f 0 2024-03-04T13:43:08.033 INFO:tasks.workunit.client.0.smithi107.stdout:4/353: getdents d0/d9 0 2024-03-04T13:43:08.035 INFO:tasks.workunit.client.0.smithi107.stdout:9/347: dread d4/da/f4a [0,4194304] 0 2024-03-04T13:43:08.041 INFO:tasks.workunit.client.0.smithi107.stdout:8/412: link d6/da/f72 d6/da/d25/d5f/f78 0 2024-03-04T13:43:08.043 INFO:tasks.workunit.client.1.smithi169.stdout:1/828: write d4/f20 [3152364,71379] 0 2024-03-04T13:43:08.044 INFO:tasks.workunit.client.0.smithi107.stdout:2/384: mknod da/d38/d3a/d5d/c84 0 2024-03-04T13:43:08.045 INFO:tasks.workunit.client.0.smithi107.stdout:2/385: fdatasync da/df/d3e/f40 0 2024-03-04T13:43:08.046 INFO:tasks.workunit.client.0.smithi107.stdout:5/343: creat d5/d8/d2a/d43/d5f/f75 x:0 0 0 2024-03-04T13:43:08.051 INFO:tasks.workunit.client.1.smithi169.stdout:1/829: dread d4/d5/d36/d35/df/fd6 [0,4194304] 0 2024-03-04T13:43:08.055 INFO:tasks.workunit.client.0.smithi107.stdout:9/348: symlink d4/da/d11/d4c/d51/l70 0 2024-03-04T13:43:08.057 INFO:tasks.workunit.client.0.smithi107.stdout:8/413: creat d6/d16/d1f/d63/d73/f79 x:0 0 0 2024-03-04T13:43:08.058 INFO:tasks.workunit.client.0.smithi107.stdout:2/386: unlink da/d38/d3a/d5d/c84 0 2024-03-04T13:43:08.060 INFO:tasks.workunit.client.0.smithi107.stdout:0/438: rename da/d7e/l50 to da/dc/dd/l92 0 2024-03-04T13:43:08.063 INFO:tasks.workunit.client.0.smithi107.stdout:6/438: link df/d17/c58 df/d2e/d36/d71/c89 0 2024-03-04T13:43:08.064 INFO:tasks.workunit.client.0.smithi107.stdout:5/344: dwrite d5/d8/f63 [0,4194304] 0 2024-03-04T13:43:08.066 INFO:tasks.workunit.client.0.smithi107.stdout:6/439: truncate df/d57/f69 427995 0 2024-03-04T13:43:08.067 INFO:tasks.workunit.client.1.smithi169.stdout:8/895: sync 2024-03-04T13:43:08.067 INFO:tasks.workunit.client.1.smithi169.stdout:4/865: sync 2024-03-04T13:43:08.069 INFO:tasks.workunit.client.0.smithi107.stdout:8/414: symlink d6/da/d25/l7a 0 2024-03-04T13:43:08.072 INFO:tasks.workunit.client.0.smithi107.stdout:2/387: mkdir da/df/d85 0 2024-03-04T13:43:08.076 INFO:tasks.workunit.client.0.smithi107.stdout:7/358: rename d6/d13/d16/d19/d1e/d3d/c63 to d6/d1d/c72 0 2024-03-04T13:43:08.079 INFO:tasks.workunit.client.1.smithi169.stdout:8/896: dread d7/d10/d22/d75/d81/da2/dac/fb8 [0,4194304] 0 2024-03-04T13:43:08.080 INFO:tasks.workunit.client.0.smithi107.stdout:7/359: write d6/d17/f6d [580304,2889] 0 2024-03-04T13:43:08.081 INFO:tasks.workunit.client.0.smithi107.stdout:0/439: dwrite da/dc/dd/d3e/f35 [0,4194304] 0 2024-03-04T13:43:08.082 INFO:tasks.workunit.client.0.smithi107.stdout:9/349: dread d4/f8 [0,4194304] 0 2024-03-04T13:43:08.087 INFO:tasks.workunit.client.0.smithi107.stdout:2/388: write da/d29/f2d [1559295,11999] 0 2024-03-04T13:43:08.092 INFO:tasks.workunit.client.0.smithi107.stdout:6/440: rename df/d57/f69 to df/d1b/d2c/d61/f8a 0 2024-03-04T13:43:08.092 INFO:tasks.workunit.client.0.smithi107.stdout:3/375: truncate d1/fc 169590 0 2024-03-04T13:43:08.093 INFO:tasks.workunit.client.0.smithi107.stdout:6/441: truncate f6 5890585 0 2024-03-04T13:43:08.094 INFO:tasks.workunit.client.0.smithi107.stdout:6/442: write f8 [578365,127387] 0 2024-03-04T13:43:08.095 INFO:tasks.workunit.client.0.smithi107.stdout:7/360: mknod d6/d1d/c73 0 2024-03-04T13:43:08.096 INFO:tasks.workunit.client.0.smithi107.stdout:4/354: truncate d0/d30/f3d 2504167 0 2024-03-04T13:43:08.099 INFO:tasks.workunit.client.0.smithi107.stdout:4/355: read - d0/d9/de/d14/dc/d21/f6d zero size 2024-03-04T13:43:08.101 INFO:tasks.workunit.client.0.smithi107.stdout:2/389: symlink da/df/d33/d68/l86 0 2024-03-04T13:43:08.107 INFO:tasks.workunit.client.0.smithi107.stdout:3/376: mknod d1/de/d18/d1f/d42/c84 0 2024-03-04T13:43:08.107 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:07 smithi169 ceph-mon[36313]: from='client.14676 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:43:08.107 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:07 smithi169 ceph-mon[36313]: from='mgr.24331 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:43:08.108 INFO:tasks.workunit.client.0.smithi107.stdout:0/440: symlink da/dc/dd/d3e/d6e/d73/d78/l93 0 2024-03-04T13:43:08.109 INFO:tasks.workunit.client.0.smithi107.stdout:8/415: write d6/d16/d1f/f3a [2990304,77756] 0 2024-03-04T13:43:08.112 INFO:tasks.workunit.client.0.smithi107.stdout:7/361: chown d6/d13/d16/d19/d38/d2c/f49 4781 1 2024-03-04T13:43:08.115 INFO:tasks.workunit.client.0.smithi107.stdout:1/387: sync 2024-03-04T13:43:08.116 INFO:tasks.workunit.client.1.smithi169.stdout:5/969: creat d1/d16/d17/d2d/d51/f155 x:0 0 0 2024-03-04T13:43:08.117 INFO:tasks.workunit.client.0.smithi107.stdout:5/345: dwrite d5/f24 [0,4194304] 0 2024-03-04T13:43:08.119 INFO:tasks.workunit.client.0.smithi107.stdout:2/390: fdatasync da/df/f1e 0 2024-03-04T13:43:08.122 INFO:tasks.workunit.client.1.smithi169.stdout:6/884: symlink d0/d3c/d40/l13b 0 2024-03-04T13:43:08.125 INFO:tasks.workunit.client.0.smithi107.stdout:8/416: creat d6/d16/d1f/d63/d73/f7b x:0 0 0 2024-03-04T13:43:08.126 INFO:tasks.workunit.client.1.smithi169.stdout:2/922: symlink d1/d5/d22/d45/d4a/de7/l13a 0 2024-03-04T13:43:08.128 INFO:tasks.workunit.client.0.smithi107.stdout:9/350: link d4/d32/l37 d4/d34/l71 0 2024-03-04T13:43:08.136 INFO:tasks.workunit.client.0.smithi107.stdout:3/377: dread d1/de/d18/d1f/f20 [0,4194304] 0 2024-03-04T13:43:08.136 INFO:tasks.workunit.client.0.smithi107.stdout:4/356: symlink d0/d9/de/d14/l71 0 2024-03-04T13:43:08.138 INFO:tasks.workunit.client.0.smithi107.stdout:1/388: readlink d1/d46/d6a/df/d33/d5d/d24/l48 0 2024-03-04T13:43:08.138 INFO:tasks.workunit.client.0.smithi107.stdout:4/357: chown d0/d9/d10/c4c 166 1 2024-03-04T13:43:08.139 INFO:tasks.workunit.client.0.smithi107.stdout:3/378: fsync d1/de/d18/d1f/d42/f52 0 2024-03-04T13:43:08.144 INFO:tasks.workunit.client.0.smithi107.stdout:2/391: mknod da/d39/c87 0 2024-03-04T13:43:08.146 INFO:tasks.workunit.client.1.smithi169.stdout:8/897: creat d7/d10/d17/d44/d4c/f121 x:0 0 0 2024-03-04T13:43:08.146 INFO:tasks.workunit.client.1.smithi169.stdout:3/864: write d7/de/d2b/d7d/d85/d13/d1a/d2d/d56/d5e/d67/d5c/d84/fac [590569,27690] 0 2024-03-04T13:43:08.149 INFO:tasks.workunit.client.1.smithi169.stdout:5/970: truncate d1/d81/d91/f92 924947 0 2024-03-04T13:43:08.151 INFO:tasks.workunit.client.0.smithi107.stdout:9/351: stat d4/da/cc 0 2024-03-04T13:43:08.151 INFO:tasks.workunit.client.0.smithi107.stdout:5/346: dread d5/d8/da/f16 [0,4194304] 0 2024-03-04T13:43:08.152 INFO:tasks.workunit.client.0.smithi107.stdout:7/362: rename d6/d13/d14/c44 to d6/d1d/d64/c74 0 2024-03-04T13:43:08.154 INFO:tasks.workunit.client.1.smithi169.stdout:6/885: creat d0/d32/d3a/d43/d74/f13c x:0 0 0 2024-03-04T13:43:08.155 INFO:tasks.workunit.client.1.smithi169.stdout:9/942: unlink d7/d2d/d48/cf4 0 2024-03-04T13:43:08.156 INFO:tasks.workunit.client.1.smithi169.stdout:9/943: chown d7/d11e/d120 22680747 1 2024-03-04T13:43:08.156 INFO:tasks.workunit.client.0.smithi107.stdout:6/443: truncate df/d1b/d2c/d31/f39 563948 0 2024-03-04T13:43:08.156 INFO:tasks.workunit.client.0.smithi107.stdout:2/392: dread da/d34/f36 [4194304,4194304] 0 2024-03-04T13:43:08.157 INFO:tasks.workunit.client.1.smithi169.stdout:2/923: truncate d1/d5/d22/d45/d4a/f92 1237656 0 2024-03-04T13:43:08.159 INFO:tasks.workunit.client.0.smithi107.stdout:1/389: dwrite d1/d46/d6a/d4e/d60/f78 [0,4194304] 0 2024-03-04T13:43:08.160 INFO:tasks.workunit.client.0.smithi107.stdout:3/379: dwrite d1/de/d18/f5f [0,4194304] 0 2024-03-04T13:43:08.160 INFO:tasks.workunit.client.1.smithi169.stdout:7/908: dread d2/de/d3e/d41/f8b [0,4194304] 0 2024-03-04T13:43:08.161 INFO:tasks.workunit.client.0.smithi107.stdout:1/390: fdatasync d1/d46/d6a/df/f32 0 2024-03-04T13:43:08.161 INFO:tasks.workunit.client.0.smithi107.stdout:1/391: chown d1/d10 27091 1 2024-03-04T13:43:08.162 INFO:tasks.workunit.client.1.smithi169.stdout:7/909: readlink d2/de/d3e/d41/l64 0 2024-03-04T13:43:08.165 INFO:tasks.workunit.client.0.smithi107.stdout:8/417: dread - d6/da/d25/d2a/f75 zero size 2024-03-04T13:43:08.168 INFO:tasks.workunit.client.0.smithi107.stdout:9/352: stat d4/c1e 0 2024-03-04T13:43:08.171 INFO:tasks.workunit.client.1.smithi169.stdout:6/886: dread d0/d32/d3e/f66 [0,4194304] 0 2024-03-04T13:43:08.172 INFO:tasks.workunit.client.0.smithi107.stdout:2/393: dwrite da/d29/f69 [0,4194304] 0 2024-03-04T13:43:08.175 INFO:tasks.workunit.client.0.smithi107.stdout:4/358: rmdir d0/d9/de/d14/d33 39 2024-03-04T13:43:08.175 INFO:tasks.workunit.client.1.smithi169.stdout:1/830: mknod d4/d5/d2d/d4e/c10e 0 2024-03-04T13:43:08.177 INFO:tasks.workunit.client.1.smithi169.stdout:5/971: fsync d1/d16/d17/fa4 0 2024-03-04T13:43:08.179 INFO:tasks.workunit.client.1.smithi169.stdout:1/831: chown d4/f20 823373 1 2024-03-04T13:43:08.179 INFO:tasks.workunit.client.1.smithi169.stdout:0/774: rename d5/d9/d3d/l50 to d5/d9/dc/d12/d15/d19/d61/d71/d8c/l109 0 2024-03-04T13:43:08.179 INFO:tasks.workunit.client.1.smithi169.stdout:2/924: readlink d1/d5/d19/d1e/d29/l2f 0 2024-03-04T13:43:08.185 INFO:tasks.workunit.client.1.smithi169.stdout:0/775: write d5/d9/dc/d12/d15/d19/d77/fcf [885610,82755] 0 2024-03-04T13:43:08.187 INFO:tasks.workunit.client.1.smithi169.stdout:7/910: mknod d2/de/d3e/d41/d5b/d7a/dd3/c135 0 2024-03-04T13:43:08.193 INFO:tasks.workunit.client.1.smithi169.stdout:9/944: getdents d7/d11e/d120 0 2024-03-04T13:43:08.198 INFO:tasks.workunit.client.1.smithi169.stdout:6/887: dwrite d0/d32/d3e/d122/ffc [0,4194304] 0 2024-03-04T13:43:08.202 INFO:tasks.workunit.client.1.smithi169.stdout:6/888: chown d0/d32/d3a/d43/cd2 14 1 2024-03-04T13:43:08.202 INFO:tasks.workunit.client.0.smithi107.stdout:0/441: getdents da/dc 0 2024-03-04T13:43:08.203 INFO:tasks.workunit.client.0.smithi107.stdout:9/353: mkdir d4/da/d11/d4c/d4d/d72 0 2024-03-04T13:43:08.203 INFO:tasks.workunit.client.0.smithi107.stdout:2/394: symlink da/df/d27/l88 0 2024-03-04T13:43:08.205 INFO:tasks.workunit.client.1.smithi169.stdout:4/866: link d4/d1c/f59 d4/d11/d2e/dcd/dee/d5c/f105 0 2024-03-04T13:43:08.208 INFO:tasks.workunit.client.1.smithi169.stdout:8/898: write d7/d10/d17/d44/fd5 [1068448,66430] 0 2024-03-04T13:43:08.212 INFO:tasks.workunit.client.1.smithi169.stdout:9/945: readlink d7/d2d/l2e 0 2024-03-04T13:43:08.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:08.216+0000 7f0672200700 1 -- 172.21.15.107:0/710030942 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f066c104f90 msgr2=0x7f066c107370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:43:08.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:08.216+0000 7f0672200700 1 --2- 172.21.15.107:0/710030942 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f066c104f90 0x7f066c107370 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f0660004940 tx=0x7f0660009e30 comp rx=0 tx=0).stop 2024-03-04T13:43:08.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:08.216+0000 7f0672200700 1 -- 172.21.15.107:0/710030942 shutdown_connections 2024-03-04T13:43:08.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:08.216+0000 7f0672200700 1 --2- 172.21.15.107:0/710030942 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f066c1078b0 0x7f066c109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:43:08.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:08.217+0000 7f0672200700 1 --2- 172.21.15.107:0/710030942 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f066c104f90 0x7f066c107370 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:43:08.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:08.217+0000 7f0672200700 1 -- 172.21.15.107:0/710030942 >> 172.21.15.107:0/710030942 conn(0x7f066c069ec0 msgr2=0x7f066c06c2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:43:08.217 INFO:tasks.workunit.client.0.smithi107.stdout:4/359: dwrite d0/d9/d10/f34 [0,4194304] 0 2024-03-04T13:43:08.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:08.220+0000 7f0672200700 1 -- 172.21.15.107:0/710030942 shutdown_connections 2024-03-04T13:43:08.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:08.220+0000 7f0672200700 1 -- 172.21.15.107:0/710030942 wait complete. 2024-03-04T13:43:08.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:08.221+0000 7f0672200700 1 Processor -- start 2024-03-04T13:43:08.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:08.221+0000 7f0672200700 1 -- start start 2024-03-04T13:43:08.219 INFO:tasks.workunit.client.1.smithi169.stdout:1/832: creat d4/d5/d36/d75/dba/df6/f10f x:0 0 0 2024-03-04T13:43:08.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:08.222+0000 7f0672200700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f066c104f90 0x7f066c116020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:43:08.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:08.222+0000 7f0672200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f066c1078b0 0x7f066c116560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:43:08.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:08.222+0000 7f0672200700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f066c116bf0 con 0x7f066c1078b0 2024-03-04T13:43:08.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:08.222+0000 7f0672200700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f066c117430 con 0x7f066c104f90 2024-03-04T13:43:08.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:08.223+0000 7f066be00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f066c1078b0 0x7f066c116560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:43:08.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:08.223+0000 7f066be00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f066c1078b0 0x7f066c116560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56590/0 (socket says 172.21.15.107:56590) 2024-03-04T13:43:08.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:08.223+0000 7f066be00700 1 -- 172.21.15.107:0/1880087072 learned_addr learned my addr 172.21.15.107:0/1880087072 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:43:08.221 INFO:tasks.workunit.client.0.smithi107.stdout:3/380: symlink d1/d1e/l85 0 2024-03-04T13:43:08.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:08.224+0000 7f0670e00700 1 --2- 172.21.15.107:0/1880087072 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f066c104f90 0x7f066c116020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:43:08.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:08.225+0000 7f066be00700 1 -- 172.21.15.107:0/1880087072 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f066c104f90 msgr2=0x7f066c116020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:43:08.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:08.225+0000 7f066be00700 1 --2- 172.21.15.107:0/1880087072 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f066c104f90 0x7f066c116020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:43:08.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:08.225+0000 7f066be00700 1 -- 172.21.15.107:0/1880087072 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0660005040 con 0x7f066c1078b0 2024-03-04T13:43:08.224 INFO:tasks.workunit.client.1.smithi169.stdout:0/776: mknod d5/d9/dc/d12/d15/c10a 0 2024-03-04T13:43:08.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:08.226+0000 7f066be00700 1 --2- 172.21.15.107:0/1880087072 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f066c1078b0 0x7f066c116560 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f065800b580 tx=0x7f06580091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:43:08.224 INFO:tasks.workunit.client.0.smithi107.stdout:8/418: write d6/da/f11 [4281159,86570] 0 2024-03-04T13:43:08.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:08.227+0000 7f0668c00700 1 -- 172.21.15.107:0/1880087072 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f065800f6f0 con 0x7f066c1078b0 2024-03-04T13:43:08.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:08.227+0000 7f0668c00700 1 -- 172.21.15.107:0/1880087072 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f065800fd20 con 0x7f066c1078b0 2024-03-04T13:43:08.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:08.227+0000 7f0668c00700 1 -- 172.21.15.107:0/1880087072 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f06580103a0 con 0x7f066c1078b0 2024-03-04T13:43:08.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:08.227+0000 7f0672200700 1 -- 172.21.15.107:0/1880087072 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f066c117710 con 0x7f066c1078b0 2024-03-04T13:43:08.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:08.227+0000 7f0672200700 1 -- 172.21.15.107:0/1880087072 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f066c117cd0 con 0x7f066c1078b0 2024-03-04T13:43:08.227 INFO:tasks.workunit.client.1.smithi169.stdout:4/867: dwrite d4/d1c/fb2 [0,4194304] 0 2024-03-04T13:43:08.227 INFO:tasks.workunit.client.1.smithi169.stdout:1/833: write d4/d5/d36/d35/d16/d81/db4/f108 [157156,38167] 0 2024-03-04T13:43:08.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:08.230+0000 7f0672200700 1 -- 172.21.15.107:0/1880087072 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f066c110150 con 0x7f066c1078b0 2024-03-04T13:43:08.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:08.231+0000 7f0668c00700 1 -- 172.21.15.107:0/1880087072 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) v1 ==== 88911+0+0 (secure 0 0 0) 0x7f065800f850 con 0x7f066c1078b0 2024-03-04T13:43:08.230 INFO:tasks.workunit.client.1.smithi169.stdout:3/865: dread d7/f36 [0,4194304] 0 2024-03-04T13:43:08.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:08.233+0000 7f0668c00700 1 --2- 172.21.15.107:0/1880087072 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7f065406b2a0 0x7f065406d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:43:08.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:08.233+0000 7f0668c00700 1 -- 172.21.15.107:0/1880087072 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f0658089740 con 0x7f066c1078b0 2024-03-04T13:43:08.231 INFO:tasks.workunit.client.0.smithi107.stdout:8/419: dread - d6/d16/d1f/d63/f74 zero size 2024-03-04T13:43:08.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:08.234+0000 7f0670e00700 1 --2- 172.21.15.107:0/1880087072 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7f065406b2a0 0x7f065406d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:43:08.232 INFO:tasks.workunit.client.1.smithi169.stdout:7/911: creat d2/d15/d17/d6f/d126/d12f/f136 x:0 0 0 2024-03-04T13:43:08.233 INFO:tasks.workunit.client.1.smithi169.stdout:8/899: creat d7/d10/d22/d75/d81/da4/f122 x:0 0 0 2024-03-04T13:43:08.235 INFO:tasks.workunit.client.0.smithi107.stdout:9/354: symlink d4/da/d11/d4c/l73 0 2024-03-04T13:43:08.235 INFO:tasks.workunit.client.0.smithi107.stdout:5/347: truncate d5/d8/d14/d5a/d69/f72 3416062 0 2024-03-04T13:43:08.235 INFO:tasks.workunit.client.0.smithi107.stdout:0/442: dwrite da/dc/d34/f5a [4194304,4194304] 0 2024-03-04T13:43:08.237 INFO:tasks.workunit.client.0.smithi107.stdout:5/348: fdatasync d5/dc/f3d 0 2024-03-04T13:43:08.238 INFO:tasks.workunit.client.0.smithi107.stdout:0/443: stat da/dc/c65 0 2024-03-04T13:43:08.239 INFO:tasks.workunit.client.1.smithi169.stdout:6/889: read d0/d32/f6b [4314400,41853] 0 2024-03-04T13:43:08.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:08.241+0000 7f0670e00700 1 --2- 172.21.15.107:0/1880087072 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7f065406b2a0 0x7f065406d750 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f066000b840 tx=0x7f0660009e30 comp rx=0 tx=0).ready entity=mgr.24331 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:43:08.240 INFO:tasks.workunit.client.0.smithi107.stdout:4/360: creat d0/d2f/f72 x:0 0 0 2024-03-04T13:43:08.242 INFO:tasks.workunit.client.1.smithi169.stdout:0/777: creat d5/d9/d3d/d5d/de4/f10b x:0 0 0 2024-03-04T13:43:08.243 INFO:tasks.workunit.client.1.smithi169.stdout:8/900: write d7/d10/d17/d44/d4c/ff7 [4496406,87591] 0 2024-03-04T13:43:08.246 INFO:tasks.workunit.client.1.smithi169.stdout:1/834: chown d4/c77 10463245 1 2024-03-04T13:43:08.246 INFO:tasks.workunit.client.1.smithi169.stdout:3/866: fsync d7/f1d 0 2024-03-04T13:43:08.247 INFO:tasks.workunit.client.1.smithi169.stdout:6/890: write d0/d32/d3e/fb7 [1895379,8940] 0 2024-03-04T13:43:08.248 INFO:tasks.workunit.client.1.smithi169.stdout:8/901: chown d7/d10/d17 5314 1 2024-03-04T13:43:08.248 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:08.250+0000 7f0668c00700 1 -- 172.21.15.107:0/1880087072 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f06580574a0 con 0x7f066c1078b0 2024-03-04T13:43:08.249 INFO:tasks.workunit.client.0.smithi107.stdout:6/444: getdents df/d2e 0 2024-03-04T13:43:08.253 INFO:tasks.workunit.client.0.smithi107.stdout:8/420: mkdir d6/d16/d59/d7c 0 2024-03-04T13:43:08.253 INFO:tasks.workunit.client.0.smithi107.stdout:7/363: dwrite d6/d13/d16/d19/d38/f47 [0,4194304] 0 2024-03-04T13:43:08.256 INFO:tasks.workunit.client.1.smithi169.stdout:2/925: write d1/d5/fe [862428,94332] 0 2024-03-04T13:43:08.258 INFO:tasks.workunit.client.1.smithi169.stdout:2/926: readlink d1/d5/d22/d3b/lb2 0 2024-03-04T13:43:08.262 INFO:tasks.workunit.client.0.smithi107.stdout:9/355: rename d4/da/d11/d20/d23/d2c/l2d to d4/da/d11/d20/d39/l74 0 2024-03-04T13:43:08.263 INFO:tasks.workunit.client.0.smithi107.stdout:3/381: symlink d1/d1e/d26/l86 0 2024-03-04T13:43:08.263 INFO:tasks.workunit.client.0.smithi107.stdout:0/444: creat da/dc/dd/d3e/d26/d69/d6b/f94 x:0 0 0 2024-03-04T13:43:08.264 INFO:tasks.workunit.client.1.smithi169.stdout:0/778: mkdir d5/d4c/d65/d95/db0/d10c 0 2024-03-04T13:43:08.265 INFO:tasks.workunit.client.0.smithi107.stdout:3/382: chown d1/de/d11/l3b 368938 1 2024-03-04T13:43:08.265 INFO:tasks.workunit.client.0.smithi107.stdout:2/395: dread da/d29/f2d [0,4194304] 0 2024-03-04T13:43:08.265 INFO:tasks.workunit.client.1.smithi169.stdout:2/927: chown d1/d5/d19/d1e/f5c 7185756 1 2024-03-04T13:43:08.267 INFO:tasks.workunit.client.0.smithi107.stdout:0/445: truncate da/f81 335842 0 2024-03-04T13:43:08.268 INFO:tasks.workunit.client.1.smithi169.stdout:5/972: dread d1/f4a [0,4194304] 0 2024-03-04T13:43:08.271 INFO:tasks.workunit.client.1.smithi169.stdout:1/835: creat d4/d5/d36/d35/d8f/d93/f110 x:0 0 0 2024-03-04T13:43:08.273 INFO:tasks.workunit.client.0.smithi107.stdout:0/446: write da/dc/d34/f42 [3501041,77690] 0 2024-03-04T13:43:08.274 INFO:tasks.workunit.client.1.smithi169.stdout:7/912: creat d2/de/d3e/d41/d89/dd6/f137 x:0 0 0 2024-03-04T13:43:08.278 INFO:tasks.workunit.client.1.smithi169.stdout:4/868: write d4/f5a [695949,130788] 0 2024-03-04T13:43:08.279 INFO:tasks.workunit.client.1.smithi169.stdout:6/891: dwrite d0/f2f [0,4194304] 0 2024-03-04T13:43:08.279 INFO:tasks.workunit.client.0.smithi107.stdout:0/447: readlink da/dc/dd/d3e/d26/d69/l8c 0 2024-03-04T13:43:08.280 INFO:tasks.workunit.client.1.smithi169.stdout:0/779: stat d5/d9/l73 0 2024-03-04T13:43:08.285 INFO:tasks.workunit.client.1.smithi169.stdout:2/928: creat d1/d6d/f13b x:0 0 0 2024-03-04T13:43:08.285 INFO:tasks.workunit.client.0.smithi107.stdout:8/421: dwrite d6/f57 [0,4194304] 0 2024-03-04T13:43:08.287 INFO:tasks.workunit.client.0.smithi107.stdout:5/349: link d5/d8/l2c d5/d8/d14/d5a/d69/l76 0 2024-03-04T13:43:08.289 INFO:tasks.workunit.client.1.smithi169.stdout:0/780: write d5/d9/dc/d12/d15/d19/d61/d71/fff [416000,68905] 0 2024-03-04T13:43:08.294 INFO:tasks.workunit.client.1.smithi169.stdout:3/867: mkdir d7/de/d2b/d7d/d85/d18/d21/d103/d11e 0 2024-03-04T13:43:08.295 INFO:tasks.workunit.client.1.smithi169.stdout:1/836: dread - d4/fd1 zero size 2024-03-04T13:43:08.295 INFO:tasks.workunit.client.1.smithi169.stdout:7/913: creat d2/de/d3e/d7e/f138 x:0 0 0 2024-03-04T13:43:08.296 INFO:tasks.workunit.client.1.smithi169.stdout:8/902: truncate d7/d10/d22/d46/da5/da6/fb4 1294775 0 2024-03-04T13:43:08.296 INFO:tasks.workunit.client.1.smithi169.stdout:5/973: chown d1/d16/d17/d1b/l64 226 1 2024-03-04T13:43:08.297 INFO:tasks.workunit.client.0.smithi107.stdout:5/350: dread d5/dc/df/f2e [0,4194304] 0 2024-03-04T13:43:08.297 INFO:tasks.workunit.client.0.smithi107.stdout:3/383: fdatasync d1/de/f32 0 2024-03-04T13:43:08.300 INFO:tasks.workunit.client.1.smithi169.stdout:2/929: chown d1/d5/d22/d45/d66/c8b 3173 1 2024-03-04T13:43:08.301 INFO:tasks.workunit.client.0.smithi107.stdout:2/396: creat da/df/d85/f89 x:0 0 0 2024-03-04T13:43:08.303 INFO:tasks.workunit.client.1.smithi169.stdout:0/781: creat d5/d4c/f10d x:0 0 0 2024-03-04T13:43:08.304 INFO:tasks.workunit.client.1.smithi169.stdout:9/946: dread d7/dc/d12/d20/da6/dd1/f13e [0,4194304] 0 2024-03-04T13:43:08.305 INFO:tasks.workunit.client.0.smithi107.stdout:4/361: rename d0/d9/d10/f34 to d0/f73 0 2024-03-04T13:43:08.306 INFO:tasks.workunit.client.0.smithi107.stdout:8/422: mknod d6/d16/d59/d3e/d67/c7d 0 2024-03-04T13:43:08.307 INFO:tasks.workunit.client.1.smithi169.stdout:4/869: dread d4/d11/d2e/d39/d48/f3c [4194304,4194304] 0 2024-03-04T13:43:08.307 INFO:tasks.workunit.client.1.smithi169.stdout:7/914: creat d2/d28/f139 x:0 0 0 2024-03-04T13:43:08.309 INFO:tasks.workunit.client.1.smithi169.stdout:9/947: chown d7/dc/d12/d20/da6/dd1/f13e 182968 1 2024-03-04T13:43:08.310 INFO:tasks.workunit.client.0.smithi107.stdout:5/351: symlink d5/d8/da/l77 0 2024-03-04T13:43:08.310 INFO:tasks.workunit.client.0.smithi107.stdout:7/364: dread d6/d13/d16/d1c/f48 [0,4194304] 0 2024-03-04T13:43:08.311 INFO:tasks.workunit.client.0.smithi107.stdout:3/384: dread d1/de/d11/f45 [0,4194304] 0 2024-03-04T13:43:08.315 INFO:tasks.workunit.client.0.smithi107.stdout:6/445: rename df/d1b/d2c/d31 to df/d17/d24/d30/d8b 0 2024-03-04T13:43:08.317 INFO:tasks.workunit.client.0.smithi107.stdout:2/397: symlink da/d38/l8a 0 2024-03-04T13:43:08.318 INFO:tasks.workunit.client.1.smithi169.stdout:0/782: truncate d5/d9/d3d/d59/d64/d7f/fcc 549640 0 2024-03-04T13:43:08.319 INFO:tasks.workunit.client.0.smithi107.stdout:3/385: dread - d1/de/d18/d33/d4d/d4f/d69/f6e zero size 2024-03-04T13:43:08.324 INFO:tasks.workunit.client.0.smithi107.stdout:9/356: link d4/l1a d4/da/d11/d4c/d4d/d72/l75 0 2024-03-04T13:43:08.325 INFO:tasks.workunit.client.1.smithi169.stdout:5/974: rename d1/d16/d2a/l58 to d1/d81/dc1/l156 0 2024-03-04T13:43:08.326 INFO:tasks.workunit.client.1.smithi169.stdout:7/915: unlink d2/de/d3e/d41/d69/c102 0 2024-03-04T13:43:08.328 INFO:tasks.workunit.client.0.smithi107.stdout:3/386: truncate d1/d2d/f7f 843621 0 2024-03-04T13:43:08.329 INFO:tasks.workunit.client.0.smithi107.stdout:5/352: rmdir d5/d8/d14 39 2024-03-04T13:43:08.330 INFO:tasks.workunit.client.0.smithi107.stdout:8/423: write d6/d16/f5e [424973,78865] 0 2024-03-04T13:43:08.331 INFO:tasks.workunit.client.0.smithi107.stdout:9/357: fsync d4/da/d11/d20/d23/f6c 0 2024-03-04T13:43:08.335 INFO:tasks.workunit.client.0.smithi107.stdout:2/398: unlink da/df/d27/d58/c61 0 2024-03-04T13:43:08.335 INFO:tasks.workunit.client.1.smithi169.stdout:8/903: creat d7/d10/d22/d46/f123 x:0 0 0 2024-03-04T13:43:08.337 INFO:tasks.workunit.client.0.smithi107.stdout:6/446: getdents df/d17/d24/d83 0 2024-03-04T13:43:08.340 INFO:tasks.workunit.client.0.smithi107.stdout:4/362: dwrite d0/d9/d10/f19 [0,4194304] 0 2024-03-04T13:43:08.340 INFO:tasks.workunit.client.0.smithi107.stdout:0/448: rename da/c7d to da/dc/dd/d1e/c95 0 2024-03-04T13:43:08.342 INFO:tasks.workunit.client.1.smithi169.stdout:0/783: creat d5/d9/dc/d12/d15/d19/d61/f10e x:0 0 0 2024-03-04T13:43:08.342 INFO:tasks.workunit.client.1.smithi169.stdout:4/870: dread d4/d11/d2e/dcd/dee/d5c/d86/f99 [0,4194304] 0 2024-03-04T13:43:08.343 INFO:tasks.workunit.client.1.smithi169.stdout:6/892: dread d0/f30 [0,4194304] 0 2024-03-04T13:43:08.343 INFO:tasks.workunit.client.1.smithi169.stdout:7/916: fdatasync d2/d15/d17/f5d 0 2024-03-04T13:43:08.344 INFO:tasks.workunit.client.0.smithi107.stdout:5/353: write d5/d8/d2a/d43/f4a [2401107,38961] 0 2024-03-04T13:43:08.350 INFO:tasks.workunit.client.0.smithi107.stdout:3/387: mknod d1/de/d18/d33/d48/d71/d4c/c87 0 2024-03-04T13:43:08.350 INFO:tasks.workunit.client.1.smithi169.stdout:9/948: creat d7/dc/df1/f153 x:0 0 0 2024-03-04T13:43:08.353 INFO:tasks.workunit.client.1.smithi169.stdout:8/904: symlink d7/d12/l124 0 2024-03-04T13:43:08.356 INFO:tasks.workunit.client.0.smithi107.stdout:3/388: truncate d1/de/d18/d33/d4d/f5b 728499 0 2024-03-04T13:43:08.357 INFO:tasks.workunit.client.1.smithi169.stdout:8/905: fdatasync d7/d10/d22/d75/d81/f100 0 2024-03-04T13:43:08.357 INFO:tasks.workunit.client.0.smithi107.stdout:7/365: dwrite d6/fa [4194304,4194304] 0 2024-03-04T13:43:08.358 INFO:tasks.workunit.client.1.smithi169.stdout:1/837: getdents d4/d5/d36/d75/dba/df6 0 2024-03-04T13:43:08.361 INFO:tasks.workunit.client.0.smithi107.stdout:5/354: truncate d5/dc/f58 1071636 0 2024-03-04T13:43:08.364 INFO:tasks.workunit.client.0.smithi107.stdout:9/358: rename d4/f53 to d4/da/d11/d20/d64/f76 0 2024-03-04T13:43:08.364 INFO:tasks.workunit.client.0.smithi107.stdout:8/424: mkdir d6/da/d7e 0 2024-03-04T13:43:08.364 INFO:tasks.workunit.client.1.smithi169.stdout:3/868: dwrite f6 [0,4194304] 0 2024-03-04T13:43:08.366 INFO:tasks.workunit.client.0.smithi107.stdout:6/447: symlink df/d1b/d2c/d46/l8c 0 2024-03-04T13:43:08.366 INFO:tasks.workunit.client.1.smithi169.stdout:4/871: dwrite d4/d1c/d75/ff4 [0,4194304] 0 2024-03-04T13:43:08.371 INFO:tasks.workunit.client.0.smithi107.stdout:4/363: mkdir d0/d9/d74 0 2024-03-04T13:43:08.372 INFO:tasks.workunit.client.1.smithi169.stdout:4/872: chown d4/f41 25639075 1 2024-03-04T13:43:08.372 INFO:tasks.workunit.client.1.smithi169.stdout:8/906: truncate d7/d10/d22/d75/db9/df3/f11d 545797 0 2024-03-04T13:43:08.373 INFO:tasks.workunit.client.1.smithi169.stdout:6/893: creat d0/d32/d70/f13d x:0 0 0 2024-03-04T13:43:08.374 INFO:tasks.workunit.client.0.smithi107.stdout:4/364: write d0/d9/d10/f19 [5276375,67] 0 2024-03-04T13:43:08.378 INFO:tasks.workunit.client.0.smithi107.stdout:3/389: rmdir d1/de/d18/d33/d4d 39 2024-03-04T13:43:08.379 INFO:tasks.workunit.client.1.smithi169.stdout:4/873: write d4/d11/d2e/dcd/dee/d73/fea [553365,86753] 0 2024-03-04T13:43:08.379 INFO:tasks.workunit.client.1.smithi169.stdout:8/907: fsync d7/d12/f11e 0 2024-03-04T13:43:08.380 INFO:tasks.workunit.client.1.smithi169.stdout:2/930: dwrite d1/d5/d22/d45/f8f [0,4194304] 0 2024-03-04T13:43:08.382 INFO:tasks.workunit.client.0.smithi107.stdout:3/390: write d1/de/d4b/f7a [764761,104011] 0 2024-03-04T13:43:08.387 INFO:tasks.workunit.client.1.smithi169.stdout:5/975: creat d1/d16/d17/d1b/d74/d12c/f157 x:0 0 0 2024-03-04T13:43:08.389 INFO:tasks.workunit.client.1.smithi169.stdout:9/949: creat d7/d25/d3c/dd2/f154 x:0 0 0 2024-03-04T13:43:08.392 INFO:tasks.workunit.client.0.smithi107.stdout:0/449: rename da/f3a to da/dc/dd/d3e/d26/d69/f96 0 2024-03-04T13:43:08.394 INFO:tasks.workunit.client.0.smithi107.stdout:8/425: symlink d6/d16/d59/l7f 0 2024-03-04T13:43:08.396 INFO:tasks.workunit.client.0.smithi107.stdout:6/448: mkdir df/d17/d24/d8d 0 2024-03-04T13:43:08.397 INFO:tasks.workunit.client.0.smithi107.stdout:2/399: mkdir da/d44/d8b 0 2024-03-04T13:43:08.397 INFO:tasks.workunit.client.1.smithi169.stdout:8/908: chown d7/d10/l39 3700 1 2024-03-04T13:43:08.399 INFO:tasks.workunit.client.0.smithi107.stdout:1/392: sync 2024-03-04T13:43:08.400 INFO:tasks.workunit.client.1.smithi169.stdout:2/931: fsync d1/d5/d22/d3b/d41/fba 0 2024-03-04T13:43:08.403 INFO:tasks.workunit.client.0.smithi107.stdout:2/400: chown da/f26 191 1 2024-03-04T13:43:08.404 INFO:tasks.workunit.client.0.smithi107.stdout:1/393: write d1/d46/d6a/df/d33/d43/d5b/f5f [1758026,95264] 0 2024-03-04T13:43:08.405 INFO:tasks.workunit.client.1.smithi169.stdout:4/874: creat d4/d11/d2e/d39/d61/d64/df1/f106 x:0 0 0 2024-03-04T13:43:08.407 INFO:tasks.workunit.client.1.smithi169.stdout:9/950: mkdir d7/dc/d12/d149/d155 0 2024-03-04T13:43:08.409 INFO:tasks.workunit.client.0.smithi107.stdout:1/394: read d1/d46/d6a/d4e/d60/f53 [687964,21851] 0 2024-03-04T13:43:08.413 INFO:tasks.workunit.client.0.smithi107.stdout:0/450: creat da/dc/dd/d3e/d26/d69/d6b/f97 x:0 0 0 2024-03-04T13:43:08.413 INFO:tasks.workunit.client.0.smithi107.stdout:8/426: symlink d6/da/d25/d2a/l80 0 2024-03-04T13:43:08.414 INFO:tasks.workunit.client.1.smithi169.stdout:6/894: mknod d0/d32/d3a/d43/d74/d93/d10c/c13e 0 2024-03-04T13:43:08.414 INFO:tasks.workunit.client.1.smithi169.stdout:9/951: chown d7/d49/l92 4 1 2024-03-04T13:43:08.414 INFO:tasks.workunit.client.1.smithi169.stdout:4/875: read d4/d11/d3b/f83 [1615958,27689] 0 2024-03-04T13:43:08.414 INFO:tasks.workunit.client.0.smithi107.stdout:8/427: chown d6/da 41 1 2024-03-04T13:43:08.415 INFO:tasks.workunit.client.1.smithi169.stdout:6/895: fdatasync d0/d3c/d95/dbd/fcd 0 2024-03-04T13:43:08.417 INFO:tasks.workunit.client.0.smithi107.stdout:6/449: mknod df/d17/d5c/c8e 0 2024-03-04T13:43:08.417 INFO:tasks.workunit.client.0.smithi107.stdout:9/359: getdents d4/d34 0 2024-03-04T13:43:08.418 INFO:tasks.workunit.client.0.smithi107.stdout:7/366: rename d6/d17/f55 to d6/d13/d16/d19/d1e/f75 0 2024-03-04T13:43:08.419 INFO:tasks.workunit.client.1.smithi169.stdout:9/952: write d7/dc/d12/d8a/df9/f145 [689688,108237] 0 2024-03-04T13:43:08.420 INFO:tasks.workunit.client.0.smithi107.stdout:9/360: chown d4/da/d11/d20 5715200 1 2024-03-04T13:43:08.420 INFO:tasks.workunit.client.0.smithi107.stdout:1/395: mknod d1/d10/d16/d47/d71/c7d 0 2024-03-04T13:43:08.422 INFO:tasks.workunit.client.1.smithi169.stdout:8/909: dwrite d7/d10/ff9 [0,4194304] 0 2024-03-04T13:43:08.423 INFO:tasks.workunit.client.0.smithi107.stdout:8/428: fdatasync d6/d16/f27 0 2024-03-04T13:43:08.423 INFO:tasks.workunit.client.1.smithi169.stdout:6/896: fdatasync d0/f16 0 2024-03-04T13:43:08.426 INFO:tasks.workunit.client.0.smithi107.stdout:6/450: unlink df/d2e/d36/d5b/l73 0 2024-03-04T13:43:08.427 INFO:tasks.workunit.client.0.smithi107.stdout:9/361: write d4/da/d11/d4c/d51/f5c [201410,14831] 0 2024-03-04T13:43:08.428 INFO:tasks.workunit.client.1.smithi169.stdout:4/876: mkdir d4/d11/d2e/dcd/dee/d5c/d86/db8/d107 0 2024-03-04T13:43:08.430 INFO:tasks.workunit.client.0.smithi107.stdout:5/355: dwrite d5/d8/d14/f4c [0,4194304] 0 2024-03-04T13:43:08.431 INFO:tasks.workunit.client.1.smithi169.stdout:9/953: mknod d7/d49/df8/c156 0 2024-03-04T13:43:08.434 INFO:tasks.workunit.client.1.smithi169.stdout:7/917: write d2/d28/d70/fc5 [126667,65520] 0 2024-03-04T13:43:08.434 INFO:tasks.workunit.client.0.smithi107.stdout:1/396: symlink d1/d46/d6a/d65/l7e 0 2024-03-04T13:43:08.434 INFO:tasks.workunit.client.0.smithi107.stdout:0/451: symlink da/dc/dd/d3e/d26/l98 0 2024-03-04T13:43:08.439 INFO:tasks.workunit.client.0.smithi107.stdout:8/429: rmdir d6/d16/d1f/d4c 39 2024-03-04T13:43:08.441 INFO:tasks.workunit.client.0.smithi107.stdout:7/367: rename d6/d13/d16/l46 to d6/d13/d16/l76 0 2024-03-04T13:43:08.446 INFO:tasks.workunit.client.1.smithi169.stdout:6/897: rename d0/l99 to d0/d32/d3a/d43/d74/d93/dbe/l13f 0 2024-03-04T13:43:08.446 INFO:tasks.workunit.client.1.smithi169.stdout:1/838: write d4/d5/d36/f88 [4731502,1870] 0 2024-03-04T13:43:08.450 INFO:tasks.workunit.client.0.smithi107.stdout:5/356: creat d5/d66/f78 x:0 0 0 2024-03-04T13:43:08.450 INFO:tasks.workunit.client.0.smithi107.stdout:4/365: dwrite d0/d9/f37 [0,4194304] 0 2024-03-04T13:43:08.451 INFO:tasks.workunit.client.0.smithi107.stdout:9/362: mkdir d4/da/d11/d20/d64/d77 0 2024-03-04T13:43:08.453 INFO:tasks.workunit.client.1.smithi169.stdout:8/910: fsync d7/d10/d22/d75/f118 0 2024-03-04T13:43:08.454 INFO:tasks.workunit.client.0.smithi107.stdout:0/452: creat da/dc/f99 x:0 0 0 2024-03-04T13:43:08.455 INFO:tasks.workunit.client.1.smithi169.stdout:5/976: dread d1/d16/d17/d1b/f69 [0,4194304] 0 2024-03-04T13:43:08.456 INFO:tasks.workunit.client.0.smithi107.stdout:1/397: truncate d1/d46/d6a/df/d33/d5d/d24/f52 4981921 0 2024-03-04T13:43:08.456 INFO:tasks.workunit.client.1.smithi169.stdout:4/877: mknod d4/c108 0 2024-03-04T13:43:08.457 INFO:tasks.workunit.client.0.smithi107.stdout:4/366: write d0/d9/d18/f53 [776139,125181] 0 2024-03-04T13:43:08.459 INFO:tasks.workunit.client.0.smithi107.stdout:6/451: dwrite df/d17/d5c/f67 [0,4194304] 0 2024-03-04T13:43:08.459 INFO:tasks.workunit.client.1.smithi169.stdout:9/954: mkdir d7/dc/d3e/da3/d157 0 2024-03-04T13:43:08.460 INFO:tasks.workunit.client.1.smithi169.stdout:4/878: fsync d4/d11/d2e/dcd/dee/d73/fea 0 2024-03-04T13:43:08.462 INFO:tasks.workunit.client.0.smithi107.stdout:3/391: truncate d1/f6 1489931 0 2024-03-04T13:43:08.466 INFO:tasks.workunit.client.1.smithi169.stdout:8/911: dread d7/d10/d22/d75/d81/da2/dac/fb8 [0,4194304] 0 2024-03-04T13:43:08.468 INFO:tasks.workunit.client.0.smithi107.stdout:7/368: rename d6/d13/d16/d19/d39/c68 to d6/d13/d16/d19/d38/d2c/c77 0 2024-03-04T13:43:08.469 INFO:tasks.workunit.client.1.smithi169.stdout:6/898: dread d0/d32/d3a/f4f [4194304,4194304] 0 2024-03-04T13:43:08.469 INFO:tasks.workunit.client.1.smithi169.stdout:3/869: dread d7/de/d2b/d7d/d85/d13/d5a/f80 [0,4194304] 0 2024-03-04T13:43:08.469 INFO:tasks.workunit.client.0.smithi107.stdout:2/401: write da/d38/d3a/f60 [636788,40614] 0 2024-03-04T13:43:08.473 INFO:tasks.workunit.client.1.smithi169.stdout:8/912: chown d7/d10/d22/d75/db9/dbe/f104 0 1 2024-03-04T13:43:08.477 INFO:tasks.workunit.client.0.smithi107.stdout:9/363: creat d4/da/d11/d20/d64/f78 x:0 0 0 2024-03-04T13:43:08.478 INFO:tasks.workunit.client.1.smithi169.stdout:1/839: dwrite d4/d5/d2d/d4e/fa6 [0,4194304] 0 2024-03-04T13:43:08.479 INFO:tasks.workunit.client.1.smithi169.stdout:8/913: stat d7/d10/d22/d75/db9/df3/l116 0 2024-03-04T13:43:08.481 INFO:tasks.workunit.client.0.smithi107.stdout:4/367: creat d0/d9/d1e/f75 x:0 0 0 2024-03-04T13:43:08.482 INFO:tasks.workunit.client.0.smithi107.stdout:2/402: dread da/d49/f7e [0,4194304] 0 2024-03-04T13:43:08.482 INFO:tasks.workunit.client.1.smithi169.stdout:2/932: dwrite d1/d5/d22/d3b/f9c [0,4194304] 0 2024-03-04T13:43:08.482 INFO:tasks.workunit.client.0.smithi107.stdout:4/368: dread - d0/d9/de/d14/dc/f3c zero size 2024-03-04T13:43:08.487 INFO:tasks.workunit.client.1.smithi169.stdout:9/955: symlink d7/d25/d3c/l158 0 2024-03-04T13:43:08.494 INFO:tasks.workunit.client.1.smithi169.stdout:4/879: truncate d4/d1c/f50 837776 0 2024-03-04T13:43:08.496 INFO:tasks.workunit.client.0.smithi107.stdout:1/398: creat d1/d46/d6a/d65/f7f x:0 0 0 2024-03-04T13:43:08.496 INFO:tasks.workunit.client.1.smithi169.stdout:1/840: dread d4/d5/d2d/d4e/d71/d26/daf/fee [0,4194304] 0 2024-03-04T13:43:08.497 INFO:tasks.workunit.client.0.smithi107.stdout:1/399: chown d1/f72 72 1 2024-03-04T13:43:08.498 INFO:tasks.workunit.client.0.smithi107.stdout:5/357: mkdir d5/d79 0 2024-03-04T13:43:08.499 INFO:tasks.workunit.client.0.smithi107.stdout:3/392: mknod d1/de/d4b/c88 0 2024-03-04T13:43:08.499 INFO:tasks.workunit.client.0.smithi107.stdout:8/430: rmdir d6/da/d7e 0 2024-03-04T13:43:08.500 INFO:tasks.workunit.client.1.smithi169.stdout:7/918: write d2/d15/d17/f1d [16870,81828] 0 2024-03-04T13:43:08.502 INFO:tasks.workunit.client.0.smithi107.stdout:5/358: fdatasync d5/d8/d2a/f39 0 2024-03-04T13:43:08.502 INFO:tasks.workunit.client.0.smithi107.stdout:3/393: chown d1/de/d11/l2e 3404 1 2024-03-04T13:43:08.502 INFO:tasks.workunit.client.0.smithi107.stdout:5/359: read - d5/d8/d2a/f44 zero size 2024-03-04T13:43:08.504 INFO:tasks.workunit.client.0.smithi107.stdout:7/369: rename d6/d13/d16/d19/d39/f4e to d6/d13/d16/d19/d1e/d3d/f78 0 2024-03-04T13:43:08.504 INFO:tasks.workunit.client.0.smithi107.stdout:5/360: write d5/dc/d22/f61 [1090817,29247] 0 2024-03-04T13:43:08.512 INFO:tasks.workunit.client.1.smithi169.stdout:6/899: rename d0/f9 to d0/d3c/d40/df6/dca/f140 0 2024-03-04T13:43:08.513 INFO:tasks.workunit.client.0.smithi107.stdout:2/403: stat da/d29/c4d 0 2024-03-04T13:43:08.516 INFO:tasks.workunit.client.1.smithi169.stdout:8/914: mkdir d7/d10/d17/d44/d5d/ddd/d125 0 2024-03-04T13:43:08.517 INFO:tasks.workunit.client.0.smithi107.stdout:7/370: dread d6/d13/d16/d19/d39/f40 [0,4194304] 0 2024-03-04T13:43:08.518 INFO:tasks.workunit.client.1.smithi169.stdout:8/915: write d7/d10/d17/d44/fd5 [1325469,26160] 0 2024-03-04T13:43:08.521 INFO:tasks.workunit.client.1.smithi169.stdout:8/916: stat d7/d10/d17/d44/d5d/fbb 0 2024-03-04T13:43:08.525 INFO:tasks.workunit.client.1.smithi169.stdout:4/880: fsync d4/f41 0 2024-03-04T13:43:08.526 INFO:tasks.workunit.client.0.smithi107.stdout:3/394: write d1/de/d18/d33/d4d/d4f/d69/f6e [323892,91887] 0 2024-03-04T13:43:08.532 INFO:tasks.workunit.client.1.smithi169.stdout:7/919: symlink d2/d15/dad/l13a 0 2024-03-04T13:43:08.532 INFO:tasks.workunit.client.0.smithi107.stdout:5/361: chown d5/dc/df/f37 0 1 2024-03-04T13:43:08.534 INFO:tasks.workunit.client.1.smithi169.stdout:5/977: write d1/d16/d17/d1b/f29 [3780257,56127] 0 2024-03-04T13:43:08.538 INFO:tasks.workunit.client.1.smithi169.stdout:6/900: mkdir d0/d32/d3a/d43/d74/d93/dee/d141 0 2024-03-04T13:43:08.540 INFO:tasks.workunit.client.0.smithi107.stdout:1/400: mkdir d1/d10/d16/d47/d7a/d80 0 2024-03-04T13:43:08.541 INFO:tasks.workunit.client.0.smithi107.stdout:7/371: creat d6/d13/d16/d1c/f79 x:0 0 0 2024-03-04T13:43:08.542 INFO:tasks.workunit.client.1.smithi169.stdout:2/933: mknod d1/d5/d19/d3c/db8/de1/d64/dce/dd5/df5/c13c 0 2024-03-04T13:43:08.544 INFO:tasks.workunit.client.1.smithi169.stdout:9/956: symlink d7/dc/df1/l159 0 2024-03-04T13:43:08.546 INFO:tasks.workunit.client.0.smithi107.stdout:3/395: rename d1/de/d18/d1f/d42/f5a to d1/de/d3f/f89 0 2024-03-04T13:43:08.547 INFO:tasks.workunit.client.1.smithi169.stdout:7/920: dwrite d2/de/d3e/d41/f4c [0,4194304] 0 2024-03-04T13:43:08.547 INFO:tasks.workunit.client.0.smithi107.stdout:3/396: write d1/de/d3f/f80 [868703,33078] 0 2024-03-04T13:43:08.548 INFO:tasks.workunit.client.0.smithi107.stdout:0/453: creat da/dc/dd/d3e/d6e/d73/f9a x:0 0 0 2024-03-04T13:43:08.550 INFO:tasks.workunit.client.0.smithi107.stdout:3/397: write d1/de/d18/d33/d4d/d4f/d69/f6e [249841,46800] 0 2024-03-04T13:43:08.556 INFO:tasks.workunit.client.1.smithi169.stdout:6/901: mkdir d0/d3c/d95/d142 0 2024-03-04T13:43:08.560 INFO:tasks.workunit.client.1.smithi169.stdout:6/902: write d0/d32/d3a/d43/d74/d93/dbe/fec [3554307,53439] 0 2024-03-04T13:43:08.561 INFO:tasks.workunit.client.1.smithi169.stdout:9/957: symlink d7/d25/d3c/dd2/l15a 0 2024-03-04T13:43:08.565 INFO:tasks.workunit.client.1.smithi169.stdout:8/917: rename d7/d10/d22/d75/d81/da2/deb/l105 to d7/d120/l126 0 2024-03-04T13:43:08.567 INFO:tasks.workunit.client.0.smithi107.stdout:5/362: creat d5/d79/f7a x:0 0 0 2024-03-04T13:43:08.573 INFO:tasks.workunit.client.1.smithi169.stdout:3/870: link d7/de/d2b/d7d/d85/d18/d21/d3f/d68/f4b d7/de/d2b/d7d/d85/d13/d1a/d2d/d56/d5e/d67/f11f 0 2024-03-04T13:43:08.574 INFO:tasks.workunit.client.0.smithi107.stdout:9/364: getdents d4/da/d11/d20/d2f 0 2024-03-04T13:43:08.578 INFO:tasks.workunit.client.0.smithi107.stdout:3/398: mknod d1/de/c8a 0 2024-03-04T13:43:08.580 INFO:tasks.workunit.client.1.smithi169.stdout:9/958: mkdir d7/dc/d3e/da3/dbc/d15b 0 2024-03-04T13:43:08.583 INFO:tasks.workunit.client.0.smithi107.stdout:1/401: symlink d1/l81 0 2024-03-04T13:43:08.583 INFO:tasks.workunit.client.1.smithi169.stdout:8/918: dwrite d7/d10/d22/d75/d81/f107 [0,4194304] 0 2024-03-04T13:43:08.584 INFO:tasks.workunit.client.1.smithi169.stdout:8/919: stat d7 0 2024-03-04T13:43:08.584 INFO:tasks.workunit.client.1.smithi169.stdout:5/978: dread d1/d16/d17/d1b/f106 [0,4194304] 0 2024-03-04T13:43:08.586 INFO:tasks.workunit.client.1.smithi169.stdout:8/920: chown d7/d10/d22/f25 1 1 2024-03-04T13:43:08.587 INFO:tasks.workunit.client.0.smithi107.stdout:7/372: unlink d6/d13/d16/d19/d39/f70 0 2024-03-04T13:43:08.587 INFO:tasks.workunit.client.1.smithi169.stdout:8/921: readlink d7/d12/l89 0 2024-03-04T13:43:08.589 INFO:tasks.workunit.client.0.smithi107.stdout:8/431: link d6/d16/d59/l49 d6/l81 0 2024-03-04T13:43:08.592 INFO:tasks.workunit.client.1.smithi169.stdout:1/841: link d4/d5/d36/d35/d5b/c9d d4/d5/d36/d35/d16/d81/db4/c111 0 2024-03-04T13:43:08.592 INFO:tasks.workunit.client.1.smithi169.stdout:7/921: symlink d2/de/d3e/d7e/dbb/l13b 0 2024-03-04T13:43:08.593 INFO:tasks.workunit.client.0.smithi107.stdout:5/363: readlink d5/dc/df/l6f 0 2024-03-04T13:43:08.595 INFO:tasks.workunit.client.1.smithi169.stdout:3/871: readlink d7/de/d2b/d7d/d85/d18/d21/la8 0 2024-03-04T13:43:08.595 INFO:tasks.workunit.client.1.smithi169.stdout:1/842: write d4/d1d/feb [2416983,111663] 0 2024-03-04T13:43:08.598 INFO:tasks.workunit.client.0.smithi107.stdout:5/364: truncate d5/d8/f45 954323 0 2024-03-04T13:43:08.598 INFO:tasks.workunit.client.0.smithi107.stdout:2/404: getdents da/d49 0 2024-03-04T13:43:08.599 INFO:tasks.workunit.client.1.smithi169.stdout:4/881: creat d4/d11/d3b/d67/f109 x:0 0 0 2024-03-04T13:43:08.600 INFO:tasks.workunit.client.0.smithi107.stdout:1/402: mkdir d1/d10/d16/d47/d71/d82 0 2024-03-04T13:43:08.601 INFO:tasks.workunit.client.1.smithi169.stdout:5/979: mkdir d1/d16/d107/d137/d158 0 2024-03-04T13:43:08.602 INFO:tasks.workunit.client.0.smithi107.stdout:9/365: dread d4/da/f15 [0,4194304] 0 2024-03-04T13:43:08.603 INFO:tasks.workunit.client.0.smithi107.stdout:7/373: mkdir d6/d13/d16/d19/d39/d7a 0 2024-03-04T13:43:08.604 INFO:tasks.workunit.client.0.smithi107.stdout:9/366: dread - d4/da/d11/d65/f6e zero size 2024-03-04T13:43:08.606 INFO:tasks.workunit.client.0.smithi107.stdout:8/432: mknod d6/d16/d1f/d63/d73/c82 0 2024-03-04T13:43:08.606 INFO:tasks.workunit.client.0.smithi107.stdout:9/367: write d4/f45 [31443,12193] 0 2024-03-04T13:43:08.609 INFO:tasks.workunit.client.0.smithi107.stdout:9/368: write d4/ff [19414,126689] 0 2024-03-04T13:43:08.613 INFO:tasks.workunit.client.0.smithi107.stdout:1/403: fdatasync d1/d46/d6a/d17/f5a 0 2024-03-04T13:43:08.614 INFO:tasks.workunit.client.1.smithi169.stdout:7/922: truncate d2/d28/d70/f87 93382 0 2024-03-04T13:43:08.614 INFO:tasks.workunit.client.1.smithi169.stdout:8/922: write d7/d10/d17/d44/d36/f8a [1755297,47596] 0 2024-03-04T13:43:08.615 INFO:tasks.workunit.client.0.smithi107.stdout:7/374: rmdir d6/d1d/d64 39 2024-03-04T13:43:08.616 INFO:tasks.workunit.client.0.smithi107.stdout:1/404: write d1/d46/d6a/d17/f5a [3228612,100514] 0 2024-03-04T13:43:08.616 INFO:tasks.workunit.client.0.smithi107.stdout:5/365: dwrite d5/d8/d2a/f70 [0,4194304] 0 2024-03-04T13:43:08.618 INFO:tasks.workunit.client.1.smithi169.stdout:3/872: write d7/d53/f8c [1769495,111133] 0 2024-03-04T13:43:08.621 INFO:tasks.workunit.client.1.smithi169.stdout:5/980: mknod d1/d16/d2f/d72/dab/c159 0 2024-03-04T13:43:08.623 INFO:tasks.workunit.client.1.smithi169.stdout:9/959: link d7/f10 d7/dc/df1/d56/f15c 0 2024-03-04T13:43:08.624 INFO:tasks.workunit.client.0.smithi107.stdout:0/454: getdents da/dc/dd/d3e/d26 0 2024-03-04T13:43:08.625 INFO:tasks.workunit.client.1.smithi169.stdout:3/873: chown d7/de/d2b/d7d/d85/d13/d5a/da1/ca5 3828880 1 2024-03-04T13:43:08.625 INFO:tasks.workunit.client.0.smithi107.stdout:3/399: creat d1/de/d18/f8b x:0 0 0 2024-03-04T13:43:08.627 INFO:tasks.workunit.client.0.smithi107.stdout:3/400: write d1/de/d4b/f7a [1812047,45828] 0 2024-03-04T13:43:08.633 INFO:tasks.workunit.client.1.smithi169.stdout:4/882: mkdir d4/d11/d2e/dcd/dee/d5c/d10a 0 2024-03-04T13:43:08.636 INFO:tasks.workunit.client.0.smithi107.stdout:8/433: dwrite d6/da/f72 [0,4194304] 0 2024-03-04T13:43:08.636 INFO:tasks.workunit.client.1.smithi169.stdout:7/923: creat d2/d28/d68/d124/f13c x:0 0 0 2024-03-04T13:43:08.637 INFO:tasks.workunit.client.1.smithi169.stdout:7/924: chown d2/d28/d68/d124 66 1 2024-03-04T13:43:08.638 INFO:tasks.workunit.client.1.smithi169.stdout:3/874: creat d7/de/d2b/d7d/d85/d18/d21/d103/f120 x:0 0 0 2024-03-04T13:43:08.639 INFO:tasks.workunit.client.0.smithi107.stdout:7/375: truncate d6/d13/f18 1645165 0 2024-03-04T13:43:08.642 INFO:tasks.workunit.client.0.smithi107.stdout:5/366: symlink d5/l7b 0 2024-03-04T13:43:08.643 INFO:tasks.workunit.client.1.smithi169.stdout:1/843: link d4/d5/d36/d35/d8f/d93/le5 d4/d5/d2d/d4e/l112 0 2024-03-04T13:43:08.644 INFO:tasks.workunit.client.0.smithi107.stdout:5/367: chown d5/d8/d14/d5a/d6a 5 1 2024-03-04T13:43:08.645 INFO:tasks.workunit.client.0.smithi107.stdout:9/369: creat d4/f79 x:0 0 0 2024-03-04T13:43:08.646 INFO:tasks.workunit.client.1.smithi169.stdout:3/875: readlink d7/de/d2b/d7d/d85/d13/d1a/d2d/d56/d5e/d67/d5c/d84/l115 0 2024-03-04T13:43:08.646 INFO:tasks.workunit.client.1.smithi169.stdout:4/883: mknod d4/d11/d2e/c10b 0 2024-03-04T13:43:08.646 INFO:tasks.workunit.client.1.smithi169.stdout:8/923: dwrite d7/d10/d17/f27 [0,4194304] 0 2024-03-04T13:43:08.647 INFO:tasks.workunit.client.0.smithi107.stdout:8/434: readlink d6/da/l30 0 2024-03-04T13:43:08.649 INFO:tasks.workunit.client.0.smithi107.stdout:7/376: creat d6/d13/d16/f7b x:0 0 0 2024-03-04T13:43:08.649 INFO:tasks.workunit.client.0.smithi107.stdout:3/401: getdents d1/d41/d81 0 2024-03-04T13:43:08.651 INFO:tasks.workunit.client.1.smithi169.stdout:5/981: getdents d1/d81/db1 0 2024-03-04T13:43:08.652 INFO:tasks.workunit.client.1.smithi169.stdout:1/844: write d4/d5/d2d/d4e/d71/d82/df7/f105 [540049,48168] 0 2024-03-04T13:43:08.653 INFO:tasks.workunit.client.0.smithi107.stdout:1/405: rmdir d1/d10/d16/d47/d7a/d80 0 2024-03-04T13:43:08.654 INFO:tasks.workunit.client.0.smithi107.stdout:1/406: chown d1/d46/d6a/df/d33/d43/d5b/d61 0 1 2024-03-04T13:43:08.658 INFO:tasks.workunit.client.0.smithi107.stdout:5/368: unlink d5/dc/l1c 0 2024-03-04T13:43:08.659 INFO:tasks.workunit.client.1.smithi169.stdout:3/876: creat d7/de/d2b/d7d/d85/f121 x:0 0 0 2024-03-04T13:43:08.662 INFO:tasks.workunit.client.0.smithi107.stdout:5/369: write d5/f32 [4901615,13905] 0 2024-03-04T13:43:08.663 INFO:tasks.workunit.client.1.smithi169.stdout:3/877: write d7/de/d2b/d7d/d85/d13/d5a/da1/fb7 [1670901,102961] 0 2024-03-04T13:43:08.663 INFO:tasks.workunit.client.1.smithi169.stdout:4/884: fdatasync d4/d11/d3b/d67/d7e/fe9 0 2024-03-04T13:43:08.664 INFO:tasks.workunit.client.0.smithi107.stdout:5/370: chown d5/d8/f45 154 1 2024-03-04T13:43:08.667 INFO:tasks.workunit.client.1.smithi169.stdout:3/878: fdatasync d7/db8/ded/ff1 0 2024-03-04T13:43:08.669 INFO:tasks.workunit.client.0.smithi107.stdout:9/370: dwrite d4/da/d11/f17 [0,4194304] 0 2024-03-04T13:43:08.671 INFO:tasks.workunit.client.0.smithi107.stdout:3/402: symlink d1/de/d18/d1f/d42/l8c 0 2024-03-04T13:43:08.673 INFO:tasks.workunit.client.1.smithi169.stdout:5/982: creat d1/d16/d17/d1b/d74/f15a x:0 0 0 2024-03-04T13:43:08.674 INFO:tasks.workunit.client.0.smithi107.stdout:7/377: symlink d6/d13/d16/d19/d1e/d66/l7c 0 2024-03-04T13:43:08.676 INFO:tasks.workunit.client.0.smithi107.stdout:7/378: fsync d6/fa 0 2024-03-04T13:43:08.677 INFO:tasks.workunit.client.0.smithi107.stdout:0/455: truncate f0 81421 0 2024-03-04T13:43:08.677 INFO:tasks.workunit.client.1.smithi169.stdout:1/845: read d4/d1d/f40 [4135820,25691] 0 2024-03-04T13:43:08.679 INFO:tasks.workunit.client.0.smithi107.stdout:1/407: rename d1/d46/d6a/d4e/d6f to d1/d46/d6a/d4e/d60/d83 0 2024-03-04T13:43:08.679 INFO:tasks.workunit.client.1.smithi169.stdout:9/960: dread d7/d2d/f40 [4194304,4194304] 0 2024-03-04T13:43:08.682 INFO:tasks.workunit.client.1.smithi169.stdout:8/924: dread d7/f4d [0,4194304] 0 2024-03-04T13:43:08.684 INFO:tasks.workunit.client.1.smithi169.stdout:8/925: read - d7/d10/d22/d75/d81/da2/f101 zero size 2024-03-04T13:43:08.689 INFO:tasks.workunit.client.1.smithi169.stdout:8/926: write d7/d10/d22/d75/d81/f100 [497178,15879] 0 2024-03-04T13:43:08.691 INFO:tasks.workunit.client.0.smithi107.stdout:3/403: dwrite d1/fa [0,4194304] 0 2024-03-04T13:43:08.694 INFO:tasks.workunit.client.1.smithi169.stdout:8/927: write d7/d10/d22/d75/d81/da4/f122 [60512,71827] 0 2024-03-04T13:43:08.694 INFO:tasks.workunit.client.0.smithi107.stdout:9/371: dwrite d4/da/d11/d4c/d51/f5c [0,4194304] 0 2024-03-04T13:43:08.695 INFO:tasks.workunit.client.0.smithi107.stdout:3/404: read d1/de/d18/d33/d4d/f5b [374649,28281] 0 2024-03-04T13:43:08.696 INFO:tasks.workunit.client.1.smithi169.stdout:8/928: stat d7/d10/d17/d44/d4c/f121 0 2024-03-04T13:43:08.699 INFO:tasks.workunit.client.1.smithi169.stdout:5/983: dread d1/d81/d91/f92 [0,4194304] 0 2024-03-04T13:43:08.700 INFO:tasks.workunit.client.1.smithi169.stdout:4/885: symlink d4/d11/d2e/d39/l10c 0 2024-03-04T13:43:08.701 INFO:tasks.workunit.client.1.smithi169.stdout:9/961: dread d7/d2d/fb0 [0,4194304] 0 2024-03-04T13:43:08.702 INFO:tasks.workunit.client.0.smithi107.stdout:7/379: creat d6/d13/d16/d1c/f7d x:0 0 0 2024-03-04T13:43:08.703 INFO:tasks.workunit.client.1.smithi169.stdout:8/929: truncate d7/d10/d22/d75/d81/da2/f101 895178 0 2024-03-04T13:43:08.704 INFO:tasks.workunit.client.0.smithi107.stdout:8/435: dread d6/da/f1b [0,4194304] 0 2024-03-04T13:43:08.706 INFO:tasks.workunit.client.0.smithi107.stdout:0/456: mknod da/dc/dd/d3e/d26/d88/c9b 0 2024-03-04T13:43:08.707 INFO:tasks.workunit.client.1.smithi169.stdout:3/879: write d7/de/d2b/d7d/d85/d13/d1a/d2d/fce [999721,56969] 0 2024-03-04T13:43:08.708 INFO:tasks.workunit.client.0.smithi107.stdout:1/408: creat d1/d46/d6a/df/d67/d6b/f84 x:0 0 0 2024-03-04T13:43:08.711 INFO:tasks.workunit.client.1.smithi169.stdout:4/886: fdatasync d4/d11/d3b/d67/dbb/d6e/fd4 0 2024-03-04T13:43:08.713 INFO:tasks.workunit.client.1.smithi169.stdout:9/962: mkdir d7/d25/d64/d10a/d15d 0 2024-03-04T13:43:08.716 INFO:tasks.workunit.client.1.smithi169.stdout:3/880: rmdir d7/de/d2b/d7d/d85/d13/d1a/d2d/dbe 39 2024-03-04T13:43:08.717 INFO:tasks.workunit.client.0.smithi107.stdout:7/380: dread d6/d13/d16/d19/d38/d2c/f49 [0,4194304] 0 2024-03-04T13:43:08.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:08.721+0000 7f0672200700 1 -- 172.21.15.107:0/1880087072 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f066c04bed0 con 0x7f066c1078b0 2024-03-04T13:43:08.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:08.721+0000 7f0668c00700 1 -- 172.21.15.107:0/1880087072 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+846 (secure 0 0 0) 0x7f0658057030 con 0x7f066c1078b0 2024-03-04T13:43:08.719 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-03-04T13:43:08.719 INFO:teuthology.orchestra.run.smithi107.stdout: "mon": { 2024-03-04T13:43:08.719 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-545-g42608bb1 (42608bb191117be10dba720e6bf8a02f3171cd34) quincy (stable)": 2 2024-03-04T13:43:08.719 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:43:08.720 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr": { 2024-03-04T13:43:08.720 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-545-g42608bb1 (42608bb191117be10dba720e6bf8a02f3171cd34) quincy (stable)": 1, 2024-03-04T13:43:08.720 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-1845-g717ce3cc (717ce3cce8d0166e4e577faa8004e6f9cb4128c0) squid (dev)": 1 2024-03-04T13:43:08.720 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:43:08.720 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": { 2024-03-04T13:43:08.720 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-545-g42608bb1 (42608bb191117be10dba720e6bf8a02f3171cd34) quincy (stable)": 6 2024-03-04T13:43:08.720 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:43:08.720 INFO:teuthology.orchestra.run.smithi107.stdout: "mds": { 2024-03-04T13:43:08.720 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-545-g42608bb1 (42608bb191117be10dba720e6bf8a02f3171cd34) quincy (stable)": 4 2024-03-04T13:43:08.720 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:43:08.720 INFO:teuthology.orchestra.run.smithi107.stdout: "overall": { 2024-03-04T13:43:08.720 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-545-g42608bb1 (42608bb191117be10dba720e6bf8a02f3171cd34) quincy (stable)": 13, 2024-03-04T13:43:08.720 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-1845-g717ce3cc (717ce3cce8d0166e4e577faa8004e6f9cb4128c0) squid (dev)": 1 2024-03-04T13:43:08.720 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-04T13:43:08.721 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-03-04T13:43:08.721 INFO:tasks.workunit.client.1.smithi169.stdout:5/984: rename d1/d23/f85 to d1/d81/d91/f15b 0 2024-03-04T13:43:08.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:08.724+0000 7f0672200700 1 -- 172.21.15.107:0/1880087072 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7f065406b2a0 msgr2=0x7f065406d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:43:08.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:08.724+0000 7f0672200700 1 --2- 172.21.15.107:0/1880087072 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7f065406b2a0 0x7f065406d750 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f066000b840 tx=0x7f0660009e30 comp rx=0 tx=0).stop 2024-03-04T13:43:08.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:08.724+0000 7f0672200700 1 -- 172.21.15.107:0/1880087072 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f066c1078b0 msgr2=0x7f066c116560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:43:08.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:08.724+0000 7f0672200700 1 --2- 172.21.15.107:0/1880087072 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f066c1078b0 0x7f066c116560 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f065800b580 tx=0x7f06580091a0 comp rx=0 tx=0).stop 2024-03-04T13:43:08.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:08.725+0000 7f0672200700 1 -- 172.21.15.107:0/1880087072 shutdown_connections 2024-03-04T13:43:08.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:08.725+0000 7f0672200700 1 --2- 172.21.15.107:0/1880087072 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7f065406b2a0 0x7f065406d750 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:43:08.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:08.725+0000 7f0672200700 1 --2- 172.21.15.107:0/1880087072 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f066c104f90 0x7f066c116020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:43:08.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:08.725+0000 7f0672200700 1 --2- 172.21.15.107:0/1880087072 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f066c1078b0 0x7f066c116560 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:43:08.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:08.725+0000 7f0672200700 1 -- 172.21.15.107:0/1880087072 >> 172.21.15.107:0/1880087072 conn(0x7f066c069ec0 msgr2=0x7f066c1093a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:43:08.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:08.725+0000 7f0672200700 1 -- 172.21.15.107:0/1880087072 shutdown_connections 2024-03-04T13:43:08.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:08.725+0000 7f0672200700 1 -- 172.21.15.107:0/1880087072 wait complete. 2024-03-04T13:43:08.724 INFO:tasks.workunit.client.1.smithi169.stdout:3/881: dread - d7/de/d2b/d7d/d85/d13/d1a/d2d/d56/fc3 zero size 2024-03-04T13:43:08.726 INFO:tasks.workunit.client.0.smithi107.stdout:0/457: rename da/dc/dd/d3e/d6e/f7b to da/dc/dd/d1e/f9c 0 2024-03-04T13:43:08.728 INFO:tasks.workunit.client.1.smithi169.stdout:9/963: rename d7/l17 to d7/d27/l15e 0 2024-03-04T13:43:08.729 INFO:tasks.workunit.client.1.smithi169.stdout:1/846: dread d4/d5/fc6 [0,4194304] 0 2024-03-04T13:43:08.729 INFO:tasks.workunit.client.0.smithi107.stdout:1/409: symlink d1/d46/d6a/d4e/d60/l85 0 2024-03-04T13:43:08.730 INFO:tasks.workunit.client.1.smithi169.stdout:8/930: getdents d7/d10/d22/d75 0 2024-03-04T13:43:08.731 INFO:tasks.workunit.client.0.smithi107.stdout:1/410: chown d1/d46/d6a/df/d33/d5d/d24/l48 152388 1 2024-03-04T13:43:08.731 INFO:tasks.workunit.client.0.smithi107.stdout:5/371: creat d5/d8/f7c x:0 0 0 2024-03-04T13:43:08.733 INFO:tasks.workunit.client.0.smithi107.stdout:5/372: chown d5/c4e 6239 1 2024-03-04T13:43:08.740 INFO:tasks.workunit.client.1.smithi169.stdout:8/931: dread d7/d10/d22/d75/d81/f100 [0,4194304] 0 2024-03-04T13:43:08.742 INFO:tasks.workunit.client.0.smithi107.stdout:0/458: dwrite da/dc/dd/d3e/f7a [0,4194304] 0 2024-03-04T13:43:08.743 INFO:tasks.workunit.client.1.smithi169.stdout:1/847: rename d4/d5/cb1 to d4/d5/d2d/d4e/d71/d26/daf/c113 0 2024-03-04T13:43:08.744 INFO:tasks.workunit.client.0.smithi107.stdout:1/411: dread d1/d46/d6a/d17/f5a [0,4194304] 0 2024-03-04T13:43:08.747 INFO:tasks.workunit.client.1.smithi169.stdout:5/985: write d1/d16/dbd/dbb/f104 [25301,78150] 0 2024-03-04T13:43:08.747 INFO:tasks.workunit.client.0.smithi107.stdout:1/412: fsync d1/d46/d6a/df/d33/d43/d5b/f5f 0 2024-03-04T13:43:08.750 INFO:tasks.workunit.client.0.smithi107.stdout:7/381: rename d6/d17/l30 to d6/d13/d16/d1c/l7e 0 2024-03-04T13:43:08.753 INFO:tasks.workunit.client.0.smithi107.stdout:5/373: dwrite d5/d79/f7a [0,4194304] 0 2024-03-04T13:43:08.756 INFO:tasks.workunit.client.1.smithi169.stdout:1/848: chown d4/d5/d36/d35/c58 1731417 1 2024-03-04T13:43:08.757 INFO:tasks.workunit.client.0.smithi107.stdout:0/459: creat da/dc/dd/d3e/d6e/f9d x:0 0 0 2024-03-04T13:43:08.759 INFO:tasks.workunit.client.0.smithi107.stdout:8/436: link d6/lc d6/d16/d1f/d63/l83 0 2024-03-04T13:43:08.764 INFO:tasks.workunit.client.1.smithi169.stdout:0/784: sync 2024-03-04T13:43:08.765 INFO:tasks.workunit.client.1.smithi169.stdout:3/882: write d7/de/d2b/d7d/d85/d18/d21/f31 [764573,4469] 0 2024-03-04T13:43:08.768 INFO:tasks.workunit.client.0.smithi107.stdout:3/405: getdents d1/de/d18/d33/d6b 0 2024-03-04T13:43:08.768 INFO:tasks.workunit.client.1.smithi169.stdout:3/883: stat d7/de/d2b/d7d/d85/d13/d1a/d78/dc1 0 2024-03-04T13:43:08.769 INFO:tasks.workunit.client.1.smithi169.stdout:9/964: write d7/d25/d3c/dd2/d139/f14d [5019,20253] 0 2024-03-04T13:43:08.769 INFO:tasks.workunit.client.0.smithi107.stdout:0/460: dread da/dc/dd/d3e/f1c [0,4194304] 0 2024-03-04T13:43:08.773 INFO:tasks.workunit.client.0.smithi107.stdout:0/461: chown da/dc/dd/d3e/d6e/d73/f9a 7706 1 2024-03-04T13:43:08.773 INFO:tasks.workunit.client.1.smithi169.stdout:0/785: symlink d5/d9/dc/d12/d66/l10f 0 2024-03-04T13:43:08.774 INFO:tasks.workunit.client.0.smithi107.stdout:8/437: mknod d6/d16/d1f/d63/d73/c84 0 2024-03-04T13:43:08.779 INFO:tasks.workunit.client.1.smithi169.stdout:8/932: getdents d7/d10/d24 0 2024-03-04T13:43:08.779 INFO:tasks.workunit.client.0.smithi107.stdout:3/406: symlink d1/de/d18/d33/d4d/d4f/d69/l8d 0 2024-03-04T13:43:08.780 INFO:tasks.workunit.client.0.smithi107.stdout:3/407: readlink d1/de/d11/l2e 0 2024-03-04T13:43:08.782 INFO:tasks.workunit.client.1.smithi169.stdout:3/884: unlink d7/de/d2b/d7d/d85/d13/d5a/da1/l110 0 2024-03-04T13:43:08.782 INFO:tasks.workunit.client.0.smithi107.stdout:3/408: fsync d1/de/d18/d33/d4d/d4f/d69/f6e 0 2024-03-04T13:43:08.784 INFO:tasks.workunit.client.0.smithi107.stdout:5/374: link d5/dc/df/f12 d5/d8/d2a/d43/f7d 0 2024-03-04T13:43:08.786 INFO:tasks.workunit.client.1.smithi169.stdout:5/986: getdents d1/d81 0 2024-03-04T13:43:08.788 INFO:tasks.workunit.client.1.smithi169.stdout:9/965: dwrite d7/dc/d12/d59/f10c [0,4194304] 0 2024-03-04T13:43:08.791 INFO:tasks.workunit.client.1.smithi169.stdout:9/966: read - d7/dc/d12/d32/d42/fe4 zero size 2024-03-04T13:43:08.794 INFO:tasks.workunit.client.1.smithi169.stdout:3/885: dread d7/de/d2b/d7d/d85/d13/d1a/d2d/fce [0,4194304] 0 2024-03-04T13:43:08.832 INFO:tasks.workunit.client.1.smithi169.stdout:8/933: chown d7/d120/l126 51965358 1 2024-03-04T13:43:08.839 INFO:tasks.workunit.client.1.smithi169.stdout:1/849: dwrite d4/d5/d2d/d4e/da5/fc5 [0,4194304] 0 2024-03-04T13:43:08.883 INFO:tasks.workunit.client.0.smithi107.stdout:1/413: truncate d1/d46/d6a/d4e/d60/f63 157802 0 2024-03-04T13:43:08.886 INFO:tasks.workunit.client.1.smithi169.stdout:0/786: rename d5/d9/dc/d12/d15/d19/d61/d71/d8c/d90 to d5/d9/dc/d12/d15/d19/d69/d110 0 2024-03-04T13:43:08.889 INFO:tasks.workunit.client.0.smithi107.stdout:6/452: sync 2024-03-04T13:43:08.890 INFO:tasks.workunit.client.0.smithi107.stdout:3/409: fsync d1/de/f32 0 2024-03-04T13:43:08.893 INFO:tasks.workunit.client.0.smithi107.stdout:5/375: mknod d5/dc/df/c7e 0 2024-03-04T13:43:08.894 INFO:tasks.workunit.client.1.smithi169.stdout:5/987: creat d1/d16/d2a/d46/d9e/d105/f15c x:0 0 0 2024-03-04T13:43:08.898 INFO:tasks.workunit.client.1.smithi169.stdout:5/988: read d1/d16/d17/d1b/f29 [2082799,45192] 0 2024-03-04T13:43:08.898 INFO:tasks.workunit.client.1.smithi169.stdout:9/967: creat d7/d2d/d48/d66/dec/f15f x:0 0 0 2024-03-04T13:43:08.902 INFO:tasks.workunit.client.0.smithi107.stdout:8/438: rename d6/da/l42 to d6/d16/d1f/d63/d73/l85 0 2024-03-04T13:43:08.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:08.905+0000 7f2891c00700 1 -- 172.21.15.107:0/3517219950 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f288c0fdf50 msgr2=0x7f288c0fe3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:43:08.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:08.905+0000 7f2891c00700 1 --2- 172.21.15.107:0/3517219950 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f288c0fdf50 0x7f288c0fe3a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f2878004940 tx=0x7f2878009e30 comp rx=0 tx=0).stop 2024-03-04T13:43:08.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:08.906+0000 7f2891c00700 1 -- 172.21.15.107:0/3517219950 shutdown_connections 2024-03-04T13:43:08.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:08.906+0000 7f2891c00700 1 --2- 172.21.15.107:0/3517219950 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f288c0fdf50 0x7f288c0fe3a0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:43:08.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:08.906+0000 7f2891c00700 1 --2- 172.21.15.107:0/3517219950 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f288c0fcd40 0x7f288c0fd150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:43:08.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:08.906+0000 7f2891c00700 1 -- 172.21.15.107:0/3517219950 >> 172.21.15.107:0/3517219950 conn(0x7f288c0f83e0 msgr2=0x7f288c0fa830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:43:08.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:08.906+0000 7f2891c00700 1 -- 172.21.15.107:0/3517219950 shutdown_connections 2024-03-04T13:43:08.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:08.906+0000 7f2891c00700 1 -- 172.21.15.107:0/3517219950 wait complete. 2024-03-04T13:43:08.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:08.907+0000 7f2891c00700 1 Processor -- start 2024-03-04T13:43:08.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:08.907+0000 7f2891c00700 1 -- start start 2024-03-04T13:43:08.905 INFO:tasks.workunit.client.0.smithi107.stdout:1/414: mknod d1/d46/d6a/df/d67/c86 0 2024-03-04T13:43:08.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:08.908+0000 7f2891c00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f288c0fcd40 0x7f288c10b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:43:08.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:08.908+0000 7f2891c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f288c0fdf50 0x7f288c10b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:43:08.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:08.908+0000 7f2891c00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f288c10bfc0 con 0x7f288c0fdf50 2024-03-04T13:43:08.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:08.908+0000 7f2891c00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f288c10c860 con 0x7f288c0fcd40 2024-03-04T13:43:08.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:08.908+0000 7f288b400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f288c0fdf50 0x7f288c10b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:43:08.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:08.908+0000 7f288b400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f288c0fdf50 0x7f288c10b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56604/0 (socket says 172.21.15.107:56604) 2024-03-04T13:43:08.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:08.908+0000 7f288b400700 1 -- 172.21.15.107:0/4159611883 learned_addr learned my addr 172.21.15.107:0/4159611883 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:43:08.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:08.909+0000 7f288be00700 1 --2- 172.21.15.107:0/4159611883 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f288c0fcd40 0x7f288c10b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:43:08.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:08.909+0000 7f288b400700 1 -- 172.21.15.107:0/4159611883 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f288c0fcd40 msgr2=0x7f288c10b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:43:08.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:08.909+0000 7f288b400700 1 --2- 172.21.15.107:0/4159611883 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f288c0fcd40 0x7f288c10b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:43:08.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:08.909+0000 7f288b400700 1 -- 172.21.15.107:0/4159611883 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2878005040 con 0x7f288c0fdf50 2024-03-04T13:43:08.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:08.909+0000 7f288be00700 1 --2- 172.21.15.107:0/4159611883 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f288c0fcd40 0x7f288c10b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T13:43:08.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:08.909+0000 7f288b400700 1 --2- 172.21.15.107:0/4159611883 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f288c0fdf50 0x7f288c10b9c0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f2878000c00 tx=0x7f2878009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:43:08.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:08.910+0000 7f2883e00700 1 -- 172.21.15.107:0/4159611883 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2878021070 con 0x7f288c0fdf50 2024-03-04T13:43:08.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:08.910+0000 7f2883e00700 1 -- 172.21.15.107:0/4159611883 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f287801c400 con 0x7f288c0fdf50 2024-03-04T13:43:08.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:08.910+0000 7f2883e00700 1 -- 172.21.15.107:0/4159611883 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f287800e070 con 0x7f288c0fdf50 2024-03-04T13:43:08.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:08.910+0000 7f2891c00700 1 -- 172.21.15.107:0/4159611883 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f288c10cae0 con 0x7f288c0fdf50 2024-03-04T13:43:08.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:08.910+0000 7f2891c00700 1 -- 172.21.15.107:0/4159611883 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f288c10cfc0 con 0x7f288c0fdf50 2024-03-04T13:43:08.909 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:08 smithi107 ceph-mon[28739]: from='client.14680 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:43:08.909 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:08 smithi107 ceph-mon[28739]: pgmap v56: 65 pgs: 65 active+clean; 2.4 GiB data, 8.4 GiB used, 528 GiB / 536 GiB avail; 66 MiB/s rd, 246 MiB/s wr, 431 op/s 2024-03-04T13:43:08.909 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:08 smithi107 ceph-mon[28739]: from='client.14684 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:43:08.909 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:08 smithi107 ceph-mon[28739]: from='client.? 172.21.15.107:0/1880087072' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:43:08.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:08.911+0000 7f2891c00700 1 -- 172.21.15.107:0/4159611883 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f288c04bed0 con 0x7f288c0fdf50 2024-03-04T13:43:08.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:08.913+0000 7f2883e00700 1 -- 172.21.15.107:0/4159611883 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) v1 ==== 88911+0+0 (secure 0 0 0) 0x7f287801ca20 con 0x7f288c0fdf50 2024-03-04T13:43:08.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:08.914+0000 7f2883e00700 1 --2- 172.21.15.107:0/4159611883 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7f287006f5e0 0x7f2870071a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:43:08.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:08.914+0000 7f2883e00700 1 -- 172.21.15.107:0/4159611883 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f287808adb0 con 0x7f288c0fdf50 2024-03-04T13:43:08.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:08.915+0000 7f288be00700 1 --2- 172.21.15.107:0/4159611883 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7f287006f5e0 0x7f2870071a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:43:08.913 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:08.915+0000 7f288be00700 1 --2- 172.21.15.107:0/4159611883 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7f287006f5e0 0x7f2870071a90 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f287c004600 tx=0x7f287c00b040 comp rx=0 tx=0).ready entity=mgr.24331 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:43:08.914 INFO:tasks.workunit.client.0.smithi107.stdout:6/453: creat df/d17/d24/d30/d8b/f8f x:0 0 0 2024-03-04T13:43:08.915 INFO:tasks.workunit.client.1.smithi169.stdout:5/989: dwrite d1/d16/d17/d2d/ff2 [0,4194304] 0 2024-03-04T13:43:08.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:08.919+0000 7f2883e00700 1 -- 172.21.15.107:0/4159611883 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f2878058bd0 con 0x7f288c0fdf50 2024-03-04T13:43:08.918 INFO:tasks.workunit.client.0.smithi107.stdout:3/410: mknod d1/de/d18/d33/d4d/c8e 0 2024-03-04T13:43:08.927 INFO:tasks.workunit.client.0.smithi107.stdout:6/454: dread f6 [4194304,4194304] 0 2024-03-04T13:43:08.936 INFO:tasks.workunit.client.0.smithi107.stdout:1/415: readlink d1/d46/l74 0 2024-03-04T13:43:08.937 INFO:tasks.workunit.client.0.smithi107.stdout:7/382: sync 2024-03-04T13:43:08.937 INFO:tasks.workunit.client.0.smithi107.stdout:4/369: sync 2024-03-04T13:43:08.937 INFO:tasks.workunit.client.0.smithi107.stdout:2/405: sync 2024-03-04T13:43:08.937 INFO:tasks.workunit.client.0.smithi107.stdout:9/372: sync 2024-03-04T13:43:08.947 INFO:tasks.workunit.client.0.smithi107.stdout:6/455: creat df/d2e/d36/d77/f90 x:0 0 0 2024-03-04T13:43:08.948 INFO:tasks.workunit.client.0.smithi107.stdout:8/439: symlink d6/d16/d59/d7c/l86 0 2024-03-04T13:43:08.950 INFO:tasks.workunit.client.0.smithi107.stdout:8/440: fdatasync d6/d16/d1f/f34 0 2024-03-04T13:43:08.951 INFO:tasks.workunit.client.0.smithi107.stdout:8/441: readlink d6/l81 0 2024-03-04T13:43:08.955 INFO:tasks.workunit.client.0.smithi107.stdout:9/373: dwrite d4/da/f63 [0,4194304] 0 2024-03-04T13:43:08.965 INFO:tasks.workunit.client.0.smithi107.stdout:2/406: creat da/df/d27/d58/d5f/f8c x:0 0 0 2024-03-04T13:43:08.968 INFO:tasks.workunit.client.0.smithi107.stdout:0/462: link da/c55 da/dc/d34/c9e 0 2024-03-04T13:43:08.968 INFO:tasks.workunit.client.0.smithi107.stdout:2/407: fsync da/df/f13 0 2024-03-04T13:43:08.968 INFO:tasks.workunit.client.0.smithi107.stdout:6/456: symlink df/d57/l91 0 2024-03-04T13:43:08.970 INFO:tasks.workunit.client.0.smithi107.stdout:8/442: fdatasync d6/f5b 0 2024-03-04T13:43:08.972 INFO:tasks.workunit.client.0.smithi107.stdout:7/383: symlink d6/d13/d16/d19/d38/d37/l7f 0 2024-03-04T13:43:08.972 INFO:tasks.workunit.client.0.smithi107.stdout:0/463: write da/dc/dd/d3e/d6e/f87 [160151,55634] 0 2024-03-04T13:43:08.974 INFO:tasks.workunit.client.0.smithi107.stdout:7/384: write d6/f3f [3194665,125398] 0 2024-03-04T13:43:08.976 INFO:tasks.workunit.client.0.smithi107.stdout:6/457: creat df/d17/d24/d83/f92 x:0 0 0 2024-03-04T13:43:08.978 INFO:tasks.workunit.client.0.smithi107.stdout:8/443: truncate d6/da/d25/f3f 139104 0 2024-03-04T13:43:08.978 INFO:tasks.workunit.client.0.smithi107.stdout:7/385: readlink d6/d13/d16/d19/d1e/d66/l7c 0 2024-03-04T13:43:08.984 INFO:tasks.workunit.client.0.smithi107.stdout:7/386: dread - d6/d1d/d36/d59/f65 zero size 2024-03-04T13:43:08.984 INFO:tasks.workunit.client.0.smithi107.stdout:6/458: truncate df/d2e/d36/d5b/f6c 4345313 0 2024-03-04T13:43:08.991 INFO:tasks.workunit.client.0.smithi107.stdout:7/387: write d6/d17/f6d [1613613,85368] 0 2024-03-04T13:43:08.992 INFO:tasks.workunit.client.0.smithi107.stdout:6/459: creat df/d2e/d36/d5b/f93 x:0 0 0 2024-03-04T13:43:08.994 INFO:tasks.workunit.client.1.smithi169.stdout:2/934: sync 2024-03-04T13:43:08.999 INFO:tasks.workunit.client.0.smithi107.stdout:0/464: dwrite da/dc/dd/d3e/f77 [0,4194304] 0 2024-03-04T13:43:09.001 INFO:tasks.workunit.client.0.smithi107.stdout:6/460: creat df/d17/d24/d30/d5d/d6a/f94 x:0 0 0 2024-03-04T13:43:09.004 INFO:tasks.workunit.client.0.smithi107.stdout:9/374: dread d4/da/d11/d20/d64/f76 [0,4194304] 0 2024-03-04T13:43:09.011 INFO:tasks.workunit.client.0.smithi107.stdout:9/375: creat d4/da/d11/d20/d39/f7a x:0 0 0 2024-03-04T13:43:09.012 INFO:tasks.workunit.client.0.smithi107.stdout:0/465: link da/c55 da/d7e/c9f 0 2024-03-04T13:43:09.018 INFO:tasks.workunit.client.0.smithi107.stdout:2/408: dread da/d44/f81 [0,4194304] 0 2024-03-04T13:43:09.018 INFO:tasks.workunit.client.0.smithi107.stdout:6/461: dread f8 [0,4194304] 0 2024-03-04T13:43:09.020 INFO:tasks.workunit.client.0.smithi107.stdout:2/409: readlink da/df/d33/d68/l6d 0 2024-03-04T13:43:09.025 INFO:tasks.workunit.client.0.smithi107.stdout:2/410: read da/df/d27/f5e [1879518,31657] 0 2024-03-04T13:43:09.026 INFO:tasks.workunit.client.0.smithi107.stdout:3/411: write d1/d2d/f51 [915949,81137] 0 2024-03-04T13:43:09.027 INFO:tasks.workunit.client.0.smithi107.stdout:9/376: mkdir d4/da/d11/d4c/d4d/d72/d7b 0 2024-03-04T13:43:09.029 INFO:tasks.workunit.client.0.smithi107.stdout:5/376: rename d5/d8/d2a/f44 to d5/dc/d22/f7f 0 2024-03-04T13:43:09.030 INFO:tasks.workunit.client.0.smithi107.stdout:3/412: fdatasync d1/de/d18/d1f/d42/f62 0 2024-03-04T13:43:09.030 INFO:tasks.workunit.client.0.smithi107.stdout:0/466: symlink da/dc/dd/d3e/d6e/d73/d78/la0 0 2024-03-04T13:43:09.031 INFO:tasks.workunit.client.0.smithi107.stdout:1/416: rename d1/d46 to d1/d46/d6a/d4e/d60/d87 22 2024-03-04T13:43:09.033 INFO:tasks.workunit.client.0.smithi107.stdout:2/411: unlink da/c5b 0 2024-03-04T13:43:09.034 INFO:tasks.workunit.client.0.smithi107.stdout:5/377: read d5/d8/f17 [3859664,10269] 0 2024-03-04T13:43:09.037 INFO:tasks.workunit.client.0.smithi107.stdout:7/388: rename d6/d13/d16/d19/d38/d2c/f49 to d6/d13/d16/d1c/f80 0 2024-03-04T13:43:09.041 INFO:tasks.workunit.client.0.smithi107.stdout:6/462: dwrite df/d38/f4e [0,4194304] 0 2024-03-04T13:43:09.042 INFO:tasks.workunit.client.1.smithi169.stdout:2/935: dread d1/d5/d22/d45/f89 [0,4194304] 0 2024-03-04T13:43:09.045 INFO:tasks.workunit.client.1.smithi169.stdout:2/936: stat d1/d5/d19/d3c/db8/de1/daf 0 2024-03-04T13:43:09.045 INFO:tasks.workunit.client.0.smithi107.stdout:2/412: unlink da/df/d3e/f4c 0 2024-03-04T13:43:09.050 INFO:tasks.workunit.client.0.smithi107.stdout:9/377: dwrite d4/f45 [0,4194304] 0 2024-03-04T13:43:09.052 INFO:tasks.workunit.client.1.smithi169.stdout:4/887: sync 2024-03-04T13:43:09.052 INFO:tasks.workunit.client.1.smithi169.stdout:6/903: sync 2024-03-04T13:43:09.053 INFO:tasks.workunit.client.1.smithi169.stdout:7/925: sync 2024-03-04T13:43:09.053 INFO:tasks.workunit.client.0.smithi107.stdout:5/378: mknod d5/dc/c80 0 2024-03-04T13:43:09.056 INFO:tasks.workunit.client.0.smithi107.stdout:1/417: dwrite d1/d46/d6a/d65/f7f [0,4194304] 0 2024-03-04T13:43:09.057 INFO:tasks.workunit.client.0.smithi107.stdout:1/418: fsync d1/d10/d16/d47/f77 0 2024-03-04T13:43:09.060 INFO:tasks.workunit.client.1.smithi169.stdout:0/787: write d5/d9/dc/d12/d15/d19/d61/f54 [501645,68632] 0 2024-03-04T13:43:09.062 INFO:tasks.workunit.client.0.smithi107.stdout:0/467: rename da/dc/dd/d3e/d26/d69/f96 to da/dc/dd/d1e/fa1 0 2024-03-04T13:43:09.065 INFO:tasks.workunit.client.0.smithi107.stdout:7/389: link d6/d13/d14/f6b d6/d1d/d36/f81 0 2024-03-04T13:43:09.066 INFO:tasks.workunit.client.0.smithi107.stdout:7/390: readlink d6/d17/l3a 0 2024-03-04T13:43:09.068 INFO:tasks.workunit.client.0.smithi107.stdout:2/413: creat da/df/d85/f8d x:0 0 0 2024-03-04T13:43:09.074 INFO:tasks.workunit.client.0.smithi107.stdout:6/463: dwrite df/d17/d24/d40/f62 [0,4194304] 0 2024-03-04T13:43:09.076 INFO:tasks.workunit.client.0.smithi107.stdout:9/378: readlink d4/da/l16 0 2024-03-04T13:43:09.077 INFO:tasks.workunit.client.0.smithi107.stdout:5/379: creat d5/d66/f81 x:0 0 0 2024-03-04T13:43:09.078 INFO:tasks.workunit.client.0.smithi107.stdout:6/464: chown df/d2e/d36/d5b/f93 103119 1 2024-03-04T13:43:09.080 INFO:tasks.workunit.client.0.smithi107.stdout:1/419: symlink d1/d46/d6a/d75/l88 0 2024-03-04T13:43:09.082 INFO:tasks.workunit.client.1.smithi169.stdout:8/934: write d7/d10/d17/d44/d36/d62/fca [1062560,9768] 0 2024-03-04T13:43:09.083 INFO:tasks.workunit.client.0.smithi107.stdout:7/391: creat d6/d13/d14/f82 x:0 0 0 2024-03-04T13:43:09.086 INFO:tasks.workunit.client.1.smithi169.stdout:3/886: dwrite d7/de/d2b/d7d/d85/d13/d1a/d2d/dbe/fcb [0,4194304] 0 2024-03-04T13:43:09.089 INFO:tasks.workunit.client.1.smithi169.stdout:3/887: chown d7/de/d2b/d7d/d85/d13/d5a/da1/fdd 0 1 2024-03-04T13:43:09.090 INFO:tasks.workunit.client.0.smithi107.stdout:6/465: readlink df/d1b/d2c/d46/d7e/l85 0 2024-03-04T13:43:09.093 INFO:tasks.workunit.client.1.smithi169.stdout:2/937: unlink d1/d5/d22/d45/fc4 0 2024-03-04T13:43:09.094 INFO:tasks.workunit.client.0.smithi107.stdout:0/468: mknod da/dc/dd/d3e/d6e/d73/d78/d86/ca2 0 2024-03-04T13:43:09.094 INFO:tasks.workunit.client.1.smithi169.stdout:3/888: write d7/de/d2b/d7d/d85/d13/d1a/d2d/d56/d5e/d5f/fff [733432,97982] 0 2024-03-04T13:43:09.098 INFO:tasks.workunit.client.1.smithi169.stdout:4/888: creat d4/d11/d2e/d39/d61/d6b/f10d x:0 0 0 2024-03-04T13:43:09.098 INFO:tasks.workunit.client.1.smithi169.stdout:6/904: dread d0/d32/d3a/d43/d74/f103 [0,4194304] 0 2024-03-04T13:43:09.099 INFO:tasks.workunit.client.0.smithi107.stdout:7/392: symlink d6/d1d/d36/l83 0 2024-03-04T13:43:09.099 INFO:tasks.workunit.client.1.smithi169.stdout:8/935: symlink d7/d10/d22/d75/db9/df3/l127 0 2024-03-04T13:43:09.100 INFO:tasks.workunit.client.1.smithi169.stdout:4/889: chown d4/d11/d2e/dcd/dd1 120 1 2024-03-04T13:43:09.105 INFO:tasks.workunit.client.1.smithi169.stdout:2/938: fsync d1/d5/d19/d69/f7e 0 2024-03-04T13:43:09.107 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:08 smithi169 ceph-mon[36313]: from='client.14680 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:43:09.107 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:08 smithi169 ceph-mon[36313]: pgmap v56: 65 pgs: 65 active+clean; 2.4 GiB data, 8.4 GiB used, 528 GiB / 536 GiB avail; 66 MiB/s rd, 246 MiB/s wr, 431 op/s 2024-03-04T13:43:09.107 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:08 smithi169 ceph-mon[36313]: from='client.14684 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:43:09.107 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:08 smithi169 ceph-mon[36313]: from='client.? 172.21.15.107:0/1880087072' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:43:09.109 INFO:tasks.workunit.client.1.smithi169.stdout:3/889: unlink d7/d53/c74 0 2024-03-04T13:43:09.111 INFO:tasks.workunit.client.0.smithi107.stdout:9/379: dwrite d4/f79 [0,4194304] 0 2024-03-04T13:43:09.112 INFO:tasks.workunit.client.1.smithi169.stdout:7/926: creat d2/de/d3e/f13d x:0 0 0 2024-03-04T13:43:09.117 INFO:tasks.workunit.client.0.smithi107.stdout:2/414: dwrite da/f20 [0,4194304] 0 2024-03-04T13:43:09.118 INFO:tasks.workunit.client.0.smithi107.stdout:0/469: unlink da/dc/c65 0 2024-03-04T13:43:09.120 INFO:tasks.workunit.client.1.smithi169.stdout:4/890: mknod d4/d11/d2e/dcd/dee/d5c/d86/d8f/c10e 0 2024-03-04T13:43:09.122 INFO:tasks.workunit.client.0.smithi107.stdout:4/370: write d0/d9/f6b [1571584,39883] 0 2024-03-04T13:43:09.122 INFO:tasks.workunit.client.0.smithi107.stdout:8/444: truncate d6/d16/d1f/f34 677652 0 2024-03-04T13:43:09.126 INFO:tasks.workunit.client.1.smithi169.stdout:2/939: creat d1/d5/d22/d45/d66/f13d x:0 0 0 2024-03-04T13:43:09.127 INFO:tasks.workunit.client.0.smithi107.stdout:2/415: read da/d29/f2d [1581021,67663] 0 2024-03-04T13:43:09.129 INFO:tasks.workunit.client.0.smithi107.stdout:1/420: dwrite d1/f72 [0,4194304] 0 2024-03-04T13:43:09.129 INFO:tasks.workunit.client.0.smithi107.stdout:5/380: dread d5/d8/da/f16 [0,4194304] 0 2024-03-04T13:43:09.129 INFO:tasks.workunit.client.1.smithi169.stdout:6/905: symlink d0/d32/d3a/d43/d74/l143 0 2024-03-04T13:43:09.133 INFO:tasks.workunit.client.1.smithi169.stdout:1/850: mkdir d4/d5/d2d/d4e/d114 0 2024-03-04T13:43:09.133 INFO:tasks.workunit.client.1.smithi169.stdout:6/906: truncate d0/d7a/d112/f123 427711 0 2024-03-04T13:43:09.133 INFO:tasks.workunit.client.0.smithi107.stdout:9/380: creat d4/da/d11/d20/d23/d2c/f7c x:0 0 0 2024-03-04T13:43:09.134 INFO:tasks.workunit.client.1.smithi169.stdout:3/890: rmdir d7/de/d2b/d7d/d85/d13/d1a/d2d/d56/d5e/d67 39 2024-03-04T13:43:09.136 INFO:tasks.workunit.client.0.smithi107.stdout:6/466: dwrite f8 [0,4194304] 0 2024-03-04T13:43:09.140 INFO:tasks.workunit.client.1.smithi169.stdout:7/927: mkdir d2/d28/d10a/d13e 0 2024-03-04T13:43:09.147 INFO:tasks.workunit.client.1.smithi169.stdout:9/968: rename d7/dc/d3e to d7/dc/df1/d160 0 2024-03-04T13:43:09.148 INFO:tasks.workunit.client.0.smithi107.stdout:5/381: symlink d5/d8/d14/d2d/l82 0 2024-03-04T13:43:09.149 INFO:tasks.workunit.client.0.smithi107.stdout:7/393: getdents d6/d13/d16/d19/d38 0 2024-03-04T13:43:09.150 INFO:tasks.workunit.client.0.smithi107.stdout:0/470: symlink da/dc/dd/d3e/d26/d69/la3 0 2024-03-04T13:43:09.152 INFO:tasks.workunit.client.1.smithi169.stdout:3/891: creat d7/de/d2b/d7d/d85/d18/d21/d3f/d86/f122 x:0 0 0 2024-03-04T13:43:09.155 INFO:tasks.workunit.client.0.smithi107.stdout:2/416: dread da/d44/f52 [0,4194304] 0 2024-03-04T13:43:09.155 INFO:tasks.workunit.client.0.smithi107.stdout:2/417: stat da/d39 0 2024-03-04T13:43:09.158 INFO:tasks.workunit.client.0.smithi107.stdout:2/418: truncate da/df/d27/d58/d5f/f8c 262500 0 2024-03-04T13:43:09.161 INFO:tasks.workunit.client.0.smithi107.stdout:8/445: creat d6/da/d25/d5f/f87 x:0 0 0 2024-03-04T13:43:09.162 INFO:tasks.workunit.client.1.smithi169.stdout:0/788: mkdir d5/d9/d3d/d5d/de4/d111 0 2024-03-04T13:43:09.163 INFO:tasks.workunit.client.1.smithi169.stdout:4/891: dread d4/d11/d2e/d39/fb9 [0,4194304] 0 2024-03-04T13:43:09.164 INFO:tasks.workunit.client.1.smithi169.stdout:1/851: mkdir d4/d5/d36/d35/d16/d115 0 2024-03-04T13:43:09.166 INFO:tasks.workunit.client.0.smithi107.stdout:4/371: mknod d0/d9/de/c76 0 2024-03-04T13:43:09.168 INFO:tasks.workunit.client.1.smithi169.stdout:6/907: symlink d0/d32/d3a/d43/d74/l144 0 2024-03-04T13:43:09.171 INFO:tasks.workunit.client.1.smithi169.stdout:5/990: rename d1/d16/d17/d1b/f1e to d1/d81/db1/d141/f15d 0 2024-03-04T13:43:09.175 INFO:tasks.workunit.client.0.smithi107.stdout:0/471: stat da/dc/d34/f38 0 2024-03-04T13:43:09.176 INFO:tasks.workunit.client.1.smithi169.stdout:9/969: symlink d7/d25/d64/d10a/d15d/l161 0 2024-03-04T13:43:09.177 INFO:tasks.workunit.client.0.smithi107.stdout:0/472: chown da/dc/dd/d3e/d6e/d73/f9a 64944 1 2024-03-04T13:43:09.177 INFO:tasks.workunit.client.1.smithi169.stdout:9/970: readlink d7/d25/d3c/l121 0 2024-03-04T13:43:09.178 INFO:tasks.workunit.client.0.smithi107.stdout:6/467: fdatasync df/d2e/d36/f49 0 2024-03-04T13:43:09.182 INFO:tasks.workunit.client.0.smithi107.stdout:3/413: creat d1/de/d18/d33/f8f x:0 0 0 2024-03-04T13:43:09.183 INFO:tasks.workunit.client.0.smithi107.stdout:8/446: creat d6/d16/d59/d7c/f88 x:0 0 0 2024-03-04T13:43:09.190 INFO:tasks.workunit.client.1.smithi169.stdout:3/892: rename d7/de/d2b/d7d/d85/d13/f4a to d7/de/d2b/d7d/d85/d13/d1a/d98/d9a/d109/f123 0 2024-03-04T13:43:09.191 INFO:tasks.workunit.client.1.smithi169.stdout:1/852: dread d4/d5/d2d/d4e/fc1 [0,4194304] 0 2024-03-04T13:43:09.191 INFO:tasks.workunit.client.0.smithi107.stdout:5/382: getdents d5/d8/d2a/d62 0 2024-03-04T13:43:09.193 INFO:tasks.workunit.client.1.smithi169.stdout:4/892: creat d4/d11/d2e/dcd/dde/f10f x:0 0 0 2024-03-04T13:43:09.194 INFO:tasks.workunit.client.0.smithi107.stdout:7/394: read d6/d13/f50 [361033,104364] 0 2024-03-04T13:43:09.194 INFO:tasks.workunit.client.1.smithi169.stdout:4/893: fdatasync d4/d11/d2e/d39/f68 0 2024-03-04T13:43:09.197 INFO:tasks.workunit.client.1.smithi169.stdout:5/991: mkdir d1/d16/d17/d2d/d51/d154/d15e 0 2024-03-04T13:43:09.197 INFO:tasks.workunit.client.1.smithi169.stdout:4/894: write d4/d11/d2e/d6c/d78/da5/fbf [1906754,111258] 0 2024-03-04T13:43:09.198 INFO:tasks.workunit.client.0.smithi107.stdout:2/419: dwrite da/d44/f81 [0,4194304] 0 2024-03-04T13:43:09.198 INFO:tasks.workunit.client.0.smithi107.stdout:6/468: mkdir df/d2e/d36/d77/d95 0 2024-03-04T13:43:09.201 INFO:tasks.workunit.client.1.smithi169.stdout:9/971: rename d7/dc/d12/d32/d42/d4f to d7/dc/df1/d56/d162 0 2024-03-04T13:43:09.204 INFO:tasks.workunit.client.0.smithi107.stdout:3/414: creat d1/de/d18/d33/d4d/f90 x:0 0 0 2024-03-04T13:43:09.205 INFO:tasks.workunit.client.0.smithi107.stdout:9/381: link d4/da/d11/d20/d39/l40 d4/da/d11/d20/d39/l7d 0 2024-03-04T13:43:09.208 INFO:tasks.workunit.client.1.smithi169.stdout:5/992: dread - d1/d16/dbd/dbb/dcc/f10d zero size 2024-03-04T13:43:09.208 INFO:tasks.workunit.client.0.smithi107.stdout:3/415: truncate d1/de/d18/d33/f8f 98278 0 2024-03-04T13:43:09.209 INFO:tasks.workunit.client.0.smithi107.stdout:5/383: symlink d5/d8/d14/d54/l83 0 2024-03-04T13:43:09.210 INFO:tasks.workunit.client.0.smithi107.stdout:3/416: chown d1/de/d18/d1f/d42 650847 1 2024-03-04T13:43:09.212 INFO:tasks.workunit.client.1.smithi169.stdout:1/853: rename d4/d5/d36/d35/dff to d4/d5/d36/d35/df/d116 0 2024-03-04T13:43:09.212 INFO:tasks.workunit.client.0.smithi107.stdout:3/417: fdatasync d1/d1e/f22 0 2024-03-04T13:43:09.214 INFO:tasks.workunit.client.0.smithi107.stdout:7/395: fdatasync d6/d13/d14/f6b 0 2024-03-04T13:43:09.215 INFO:tasks.workunit.client.1.smithi169.stdout:6/908: getdents d0/d3c/d40/d52/d98 0 2024-03-04T13:43:09.216 INFO:tasks.workunit.client.0.smithi107.stdout:6/469: mknod df/d38/c96 0 2024-03-04T13:43:09.216 INFO:tasks.workunit.client.1.smithi169.stdout:6/909: fdatasync d0/d32/d3a/d43/d74/f13c 0 2024-03-04T13:43:09.217 INFO:tasks.workunit.client.0.smithi107.stdout:5/384: write d5/d8/f11 [1945094,105878] 0 2024-03-04T13:43:09.220 INFO:tasks.workunit.client.0.smithi107.stdout:2/420: creat da/df/d3e/f8e x:0 0 0 2024-03-04T13:43:09.221 INFO:tasks.workunit.client.0.smithi107.stdout:8/447: symlink d6/da/l89 0 2024-03-04T13:43:09.222 INFO:tasks.workunit.client.0.smithi107.stdout:2/421: readlink da/d49/l66 0 2024-03-04T13:43:09.223 INFO:tasks.workunit.client.0.smithi107.stdout:9/382: mkdir d4/da/d11/d20/d23/d7e 0 2024-03-04T13:43:09.226 INFO:tasks.workunit.client.0.smithi107.stdout:8/448: write d6/d16/d59/d3e/f56 [2359706,22405] 0 2024-03-04T13:43:09.229 INFO:tasks.workunit.client.1.smithi169.stdout:6/910: mkdir d0/d3c/d40/df6/d145 0 2024-03-04T13:43:09.229 INFO:tasks.workunit.client.0.smithi107.stdout:5/385: truncate d5/d66/f78 327816 0 2024-03-04T13:43:09.232 INFO:tasks.workunit.client.0.smithi107.stdout:2/422: truncate da/d29/f4e 602330 0 2024-03-04T13:43:09.234 INFO:tasks.workunit.client.0.smithi107.stdout:8/449: creat d6/d16/d59/d7c/f8a x:0 0 0 2024-03-04T13:43:09.234 INFO:tasks.workunit.client.0.smithi107.stdout:6/470: creat df/d1b/d2c/d46/d7e/d86/f97 x:0 0 0 2024-03-04T13:43:09.235 INFO:tasks.workunit.client.1.smithi169.stdout:6/911: truncate d0/d3c/d40/d52/d98/fd5 172753 0 2024-03-04T13:43:09.236 INFO:tasks.workunit.client.0.smithi107.stdout:3/418: dwrite d1/de/d3f/f80 [0,4194304] 0 2024-03-04T13:43:09.236 INFO:tasks.workunit.client.0.smithi107.stdout:5/386: mknod d5/d8/d2a/d43/d5f/c84 0 2024-03-04T13:43:09.239 INFO:tasks.workunit.client.0.smithi107.stdout:7/396: truncate d6/d13/f18 129594 0 2024-03-04T13:43:09.240 INFO:tasks.workunit.client.0.smithi107.stdout:2/423: symlink da/d34/l8f 0 2024-03-04T13:43:09.240 INFO:tasks.workunit.client.0.smithi107.stdout:8/450: write d6/f26 [2598658,114831] 0 2024-03-04T13:43:09.243 INFO:tasks.workunit.client.0.smithi107.stdout:8/451: dread - d6/da/d25/d5f/f87 zero size 2024-03-04T13:43:09.245 INFO:tasks.workunit.client.0.smithi107.stdout:6/471: unlink df/d17/d24/d30/d3b/f78 0 2024-03-04T13:43:09.247 INFO:tasks.workunit.client.0.smithi107.stdout:5/387: chown d5/d8/da/l60 1 1 2024-03-04T13:43:09.248 INFO:tasks.workunit.client.0.smithi107.stdout:8/452: read - d6/d16/d59/d3e/f5a zero size 2024-03-04T13:43:09.249 INFO:tasks.workunit.client.0.smithi107.stdout:5/388: chown d5/dc/df/c7e 422094274 1 2024-03-04T13:43:09.250 INFO:tasks.workunit.client.1.smithi169.stdout:9/972: dread d7/d25/d64/fa7 [0,4194304] 0 2024-03-04T13:43:09.252 INFO:tasks.workunit.client.0.smithi107.stdout:2/424: mkdir da/d6c/d90 0 2024-03-04T13:43:09.252 INFO:tasks.workunit.client.1.smithi169.stdout:6/912: dread d0/d3c/d40/d52/dac/fc7 [0,4194304] 0 2024-03-04T13:43:09.253 INFO:tasks.workunit.client.0.smithi107.stdout:9/383: link d4/da/d11/d65/c5f d4/da/d11/d4c/d4d/d72/d7b/c7f 0 2024-03-04T13:43:09.254 INFO:tasks.workunit.client.0.smithi107.stdout:9/384: fdatasync d4/da/f63 0 2024-03-04T13:43:09.255 INFO:tasks.workunit.client.1.smithi169.stdout:9/973: rmdir d7/dc/d12/d20/da6/d111 39 2024-03-04T13:43:09.257 INFO:tasks.workunit.client.0.smithi107.stdout:2/425: truncate da/df/d3e/f40 1020142 0 2024-03-04T13:43:09.257 INFO:tasks.workunit.client.0.smithi107.stdout:8/453: creat d6/d16/d1f/d63/f8b x:0 0 0 2024-03-04T13:43:09.258 INFO:tasks.workunit.client.1.smithi169.stdout:6/913: symlink d0/d3c/d40/d52/dac/l146 0 2024-03-04T13:43:09.259 INFO:tasks.workunit.client.0.smithi107.stdout:8/454: chown d6/da/d25/d5f/c6c 8 1 2024-03-04T13:43:09.260 INFO:tasks.workunit.client.0.smithi107.stdout:7/397: creat d6/d13/d16/d19/d60/f84 x:0 0 0 2024-03-04T13:43:09.263 INFO:tasks.workunit.client.1.smithi169.stdout:6/914: creat d0/d3c/d40/df6/dca/f147 x:0 0 0 2024-03-04T13:43:09.264 INFO:tasks.workunit.client.0.smithi107.stdout:8/455: write d6/d16/d1f/f43 [2729566,76253] 0 2024-03-04T13:43:09.266 INFO:tasks.workunit.client.0.smithi107.stdout:5/389: rename d5/d79/f7a to d5/d66/f85 0 2024-03-04T13:43:09.267 INFO:tasks.workunit.client.1.smithi169.stdout:9/974: mknod d7/dc/d12/d149/d155/c163 0 2024-03-04T13:43:09.271 INFO:tasks.workunit.client.0.smithi107.stdout:6/472: creat df/d38/d4f/d74/f98 x:0 0 0 2024-03-04T13:43:09.273 INFO:tasks.workunit.client.0.smithi107.stdout:1/421: mkdir d1/d46/d6a/df/d33/d89 0 2024-03-04T13:43:09.274 INFO:tasks.workunit.client.1.smithi169.stdout:9/975: rename d7/dc/d12/d32/d12c/l14e to d7/dc/df1/d160/da3/dbc/l164 0 2024-03-04T13:43:09.278 INFO:tasks.workunit.client.0.smithi107.stdout:7/398: mknod d6/d13/d16/d19/d60/c85 0 2024-03-04T13:43:09.281 INFO:tasks.workunit.client.0.smithi107.stdout:5/390: creat d5/d66/f86 x:0 0 0 2024-03-04T13:43:09.283 INFO:tasks.workunit.client.0.smithi107.stdout:6/473: mkdir df/d1b/d2c/d61/d99 0 2024-03-04T13:43:09.285 INFO:tasks.workunit.client.1.smithi169.stdout:9/976: creat d7/d2d/d48/f165 x:0 0 0 2024-03-04T13:43:09.289 INFO:tasks.workunit.client.0.smithi107.stdout:8/456: write d6/da/f19 [5570822,23338] 0 2024-03-04T13:43:09.289 INFO:tasks.workunit.client.1.smithi169.stdout:8/936: dwrite d7/d10/d17/d44/fd2 [0,4194304] 0 2024-03-04T13:43:09.292 INFO:tasks.workunit.client.0.smithi107.stdout:7/399: read d6/d13/d16/d1c/f1f [636183,74980] 0 2024-03-04T13:43:09.293 INFO:tasks.workunit.client.0.smithi107.stdout:1/422: dwrite d1/d10/d16/f37 [0,4194304] 0 2024-03-04T13:43:09.296 INFO:tasks.workunit.client.0.smithi107.stdout:2/426: rmdir da/d38/d3a/d5d/d78 0 2024-03-04T13:43:09.297 INFO:tasks.workunit.client.1.smithi169.stdout:6/915: dread d0/d32/d3a/d43/f128 [0,4194304] 0 2024-03-04T13:43:09.297 INFO:tasks.workunit.client.0.smithi107.stdout:5/391: write d5/dc/f55 [2203868,105469] 0 2024-03-04T13:43:09.299 INFO:tasks.workunit.client.0.smithi107.stdout:2/427: truncate da/df/d85/f89 101238 0 2024-03-04T13:43:09.303 INFO:tasks.workunit.client.0.smithi107.stdout:1/423: rmdir d1/d10/d16/d47/d71 39 2024-03-04T13:43:09.309 INFO:tasks.workunit.client.1.smithi169.stdout:7/928: dwrite d2/de/d3e/d41/d69/fc8 [0,4194304] 0 2024-03-04T13:43:09.312 INFO:tasks.workunit.client.0.smithi107.stdout:8/457: dwrite d6/d16/d1f/d63/d73/f79 [0,4194304] 0 2024-03-04T13:43:09.313 INFO:tasks.workunit.client.1.smithi169.stdout:0/789: write d5/d9/dc/d12/d15/d19/f2e [4667789,7880] 0 2024-03-04T13:43:09.314 INFO:tasks.workunit.client.0.smithi107.stdout:8/458: readlink d6/da/d25/d2a/l32 0 2024-03-04T13:43:09.317 INFO:tasks.workunit.client.1.smithi169.stdout:7/929: chown d2/de/d3e/d41/f59 60642184 1 2024-03-04T13:43:09.317 INFO:tasks.workunit.client.1.smithi169.stdout:6/916: mknod d0/d32/d3a/d43/d74/da6/d134/c148 0 2024-03-04T13:43:09.317 INFO:tasks.workunit.client.0.smithi107.stdout:8/459: chown d6/d16/d59/d3e/f5a 93 1 2024-03-04T13:43:09.318 INFO:tasks.workunit.client.0.smithi107.stdout:0/473: sync 2024-03-04T13:43:09.322 INFO:tasks.workunit.client.0.smithi107.stdout:7/400: link d6/d17/f6d d6/d13/d14/f86 0 2024-03-04T13:43:09.323 INFO:tasks.workunit.client.0.smithi107.stdout:8/460: write d6/fe [1856673,107870] 0 2024-03-04T13:43:09.324 INFO:tasks.workunit.client.1.smithi169.stdout:8/937: dwrite d7/d10/fcc [4194304,4194304] 0 2024-03-04T13:43:09.324 INFO:tasks.workunit.client.1.smithi169.stdout:0/790: truncate d5/d9/dc/d12/d15/d19/d61/d71/dbb/f105 987596 0 2024-03-04T13:43:09.330 INFO:tasks.workunit.client.1.smithi169.stdout:8/938: dread d7/d10/d22/d46/f83 [0,4194304] 0 2024-03-04T13:43:09.330 INFO:tasks.workunit.client.1.smithi169.stdout:0/791: truncate d5/d9/dc/d12/d15/d19/d61/f68 4401113 0 2024-03-04T13:43:09.331 INFO:tasks.workunit.client.0.smithi107.stdout:2/428: link da/d49/d5c/f67 da/d49/d5c/f91 0 2024-03-04T13:43:09.331 INFO:tasks.workunit.client.0.smithi107.stdout:4/372: dwrite d0/d9/de/d14/dc/d21/f2c [0,4194304] 0 2024-03-04T13:43:09.332 INFO:tasks.workunit.client.1.smithi169.stdout:8/939: chown d7/d10/f1e 81935508 1 2024-03-04T13:43:09.342 INFO:tasks.workunit.client.1.smithi169.stdout:6/917: rename d0/d32/d70/fe0 to d0/d3c/d95/dbd/f149 0 2024-03-04T13:43:09.345 INFO:tasks.workunit.client.1.smithi169.stdout:0/792: unlink d5/d9/dc/d12/d66/l9a 0 2024-03-04T13:43:09.346 INFO:tasks.workunit.client.0.smithi107.stdout:8/461: dwrite d6/d16/d1f/f5c [0,4194304] 0 2024-03-04T13:43:09.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:09.351+0000 7f2891c00700 1 -- 172.21.15.107:0/4159611883 --> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f288c1028a0 con 0x7f287006f5e0 2024-03-04T13:43:09.351 INFO:tasks.workunit.client.1.smithi169.stdout:5/993: write d1/d76/fb2 [4114955,7192] 0 2024-03-04T13:43:09.352 INFO:tasks.workunit.client.1.smithi169.stdout:8/940: symlink d7/d12/l128 0 2024-03-04T13:43:09.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:09.355+0000 7f2883e00700 1 -- 172.21.15.107:0/4159611883 <== mgr.24331 v2:172.21.15.169:6828/3524931243 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7f288c1028a0 con 0x7f287006f5e0 2024-03-04T13:43:09.354 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-03-04T13:43:09.354 INFO:teuthology.orchestra.run.smithi107.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377", 2024-03-04T13:43:09.354 INFO:teuthology.orchestra.run.smithi107.stdout: "in_progress": true, 2024-03-04T13:43:09.354 INFO:teuthology.orchestra.run.smithi107.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-03-04T13:43:09.354 INFO:teuthology.orchestra.run.smithi107.stdout: "services_complete": [], 2024-03-04T13:43:09.354 INFO:teuthology.orchestra.run.smithi107.stdout: "progress": "1/2 daemons upgraded", 2024-03-04T13:43:09.354 INFO:teuthology.orchestra.run.smithi107.stdout: "message": "Currently upgrading mgr daemons", 2024-03-04T13:43:09.354 INFO:teuthology.orchestra.run.smithi107.stdout: "is_paused": false 2024-03-04T13:43:09.354 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-03-04T13:43:09.355 INFO:tasks.workunit.client.0.smithi107.stdout:7/401: creat d6/d13/d16/d19/d38/d2c/f87 x:0 0 0 2024-03-04T13:43:09.355 INFO:tasks.workunit.client.0.smithi107.stdout:7/402: chown d6/d13/d16/d19/d39 41 1 2024-03-04T13:43:09.355 INFO:tasks.workunit.client.1.smithi169.stdout:4/895: write d4/d11/d2e/dcd/dee/d5c/f105 [2202102,128586] 0 2024-03-04T13:43:09.355 INFO:tasks.workunit.client.0.smithi107.stdout:2/429: mknod da/d39/c92 0 2024-03-04T13:43:09.358 INFO:tasks.workunit.client.1.smithi169.stdout:6/918: fdatasync d0/d32/f81 0 2024-03-04T13:43:09.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:09.362+0000 7f2891c00700 1 -- 172.21.15.107:0/4159611883 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7f287006f5e0 msgr2=0x7f2870071a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:43:09.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:09.362+0000 7f2891c00700 1 --2- 172.21.15.107:0/4159611883 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7f287006f5e0 0x7f2870071a90 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f287c004600 tx=0x7f287c00b040 comp rx=0 tx=0).stop 2024-03-04T13:43:09.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:09.362+0000 7f2891c00700 1 -- 172.21.15.107:0/4159611883 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f288c0fdf50 msgr2=0x7f288c10b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:43:09.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:09.362+0000 7f2891c00700 1 --2- 172.21.15.107:0/4159611883 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f288c0fdf50 0x7f288c10b9c0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f2878000c00 tx=0x7f2878009e30 comp rx=0 tx=0).stop 2024-03-04T13:43:09.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:09.363+0000 7f2891c00700 1 -- 172.21.15.107:0/4159611883 shutdown_connections 2024-03-04T13:43:09.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:09.363+0000 7f2891c00700 1 --2- 172.21.15.107:0/4159611883 >> [v2:172.21.15.169:6828/3524931243,v1:172.21.15.169:6829/3524931243] conn(0x7f287006f5e0 0x7f2870071a90 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:43:09.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:09.363+0000 7f2891c00700 1 --2- 172.21.15.107:0/4159611883 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f288c0fcd40 0x7f288c10b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:43:09.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:09.363+0000 7f2891c00700 1 --2- 172.21.15.107:0/4159611883 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f288c0fdf50 0x7f288c10b9c0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:43:09.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:09.363+0000 7f2891c00700 1 -- 172.21.15.107:0/4159611883 >> 172.21.15.107:0/4159611883 conn(0x7f288c0f83e0 msgr2=0x7f288c101180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:43:09.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:09.363+0000 7f2891c00700 1 -- 172.21.15.107:0/4159611883 shutdown_connections 2024-03-04T13:43:09.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:09.363+0000 7f2891c00700 1 -- 172.21.15.107:0/4159611883 wait complete. 2024-03-04T13:43:09.367 INFO:tasks.workunit.client.1.smithi169.stdout:0/793: creat d5/d4c/d65/d95/f112 x:0 0 0 2024-03-04T13:43:09.369 INFO:tasks.workunit.client.0.smithi107.stdout:8/462: chown d6/d16/d1f/d4c/f51 52786 1 2024-03-04T13:43:09.373 INFO:tasks.workunit.client.0.smithi107.stdout:3/419: write d1/de/d18/d33/d4d/f5b [1206267,39379] 0 2024-03-04T13:43:09.376 INFO:tasks.workunit.client.1.smithi169.stdout:8/941: mkdir d7/d10/d22/d75/d79/d129 0 2024-03-04T13:43:09.383 INFO:tasks.workunit.client.1.smithi169.stdout:4/896: truncate d4/d11/d2e/d6c/d78/da5/fce 539083 0 2024-03-04T13:43:09.384 INFO:tasks.workunit.client.0.smithi107.stdout:3/420: read d1/de/d18/d33/d4d/d4f/d69/f6e [44011,98294] 0 2024-03-04T13:43:09.386 INFO:tasks.workunit.client.1.smithi169.stdout:0/794: symlink d5/d9/dc/d12/d15/d19/d61/l113 0 2024-03-04T13:43:09.387 INFO:tasks.workunit.client.0.smithi107.stdout:7/403: symlink d6/d13/d16/d19/d39/d7a/l88 0 2024-03-04T13:43:09.388 INFO:tasks.workunit.client.0.smithi107.stdout:0/474: dread f4 [0,4194304] 0 2024-03-04T13:43:09.390 INFO:tasks.workunit.client.0.smithi107.stdout:4/373: link d0/d9/de/d14/l71 d0/d9/d10/l77 0 2024-03-04T13:43:09.393 INFO:tasks.workunit.client.0.smithi107.stdout:2/430: link da/d38/d3a/f60 da/df/d27/d58/d5f/f93 0 2024-03-04T13:43:09.394 INFO:tasks.workunit.client.1.smithi169.stdout:4/897: mkdir d4/d11/d3b/d67/d7e/d110 0 2024-03-04T13:43:09.394 INFO:tasks.workunit.client.0.smithi107.stdout:9/385: sync 2024-03-04T13:43:09.396 INFO:tasks.workunit.client.0.smithi107.stdout:8/463: symlink d6/da/l8c 0 2024-03-04T13:43:09.398 INFO:tasks.workunit.client.1.smithi169.stdout:0/795: mkdir d5/d9/d3d/d114 0 2024-03-04T13:43:09.399 INFO:tasks.workunit.client.1.smithi169.stdout:4/898: mknod d4/d1c/d58/d9d/c111 0 2024-03-04T13:43:09.402 INFO:tasks.workunit.client.0.smithi107.stdout:6/474: write df/f15 [63026,32685] 0 2024-03-04T13:43:09.402 INFO:tasks.workunit.client.1.smithi169.stdout:9/977: truncate d7/d25/d64/fa7 2084667 0 2024-03-04T13:43:09.403 INFO:tasks.workunit.client.1.smithi169.stdout:6/919: dread d0/f29 [0,4194304] 0 2024-03-04T13:43:09.403 INFO:tasks.workunit.client.0.smithi107.stdout:3/421: dwrite d1/d1e/f74 [0,4194304] 0 2024-03-04T13:43:09.404 INFO:tasks.workunit.client.0.smithi107.stdout:4/374: mkdir d0/d9/d78 0 2024-03-04T13:43:09.405 INFO:tasks.workunit.client.0.smithi107.stdout:4/375: chown d0/d2f 3 1 2024-03-04T13:43:09.406 INFO:tasks.workunit.client.1.smithi169.stdout:9/978: write d7/dc/d12/d8a/f93 [2082297,49152] 0 2024-03-04T13:43:09.407 INFO:tasks.workunit.client.0.smithi107.stdout:6/475: fdatasync df/d2e/d36/d5b/f93 0 2024-03-04T13:43:09.407 INFO:tasks.workunit.client.0.smithi107.stdout:2/431: mknod da/d38/d3a/c94 0 2024-03-04T13:43:09.408 INFO:tasks.workunit.client.0.smithi107.stdout:1/424: write d1/d46/d6a/f51 [651343,43831] 0 2024-03-04T13:43:09.410 INFO:tasks.workunit.client.0.smithi107.stdout:5/392: write d5/dc/f41 [2251827,62144] 0 2024-03-04T13:43:09.411 INFO:tasks.workunit.client.0.smithi107.stdout:8/464: readlink d6/lc 0 2024-03-04T13:43:09.414 INFO:tasks.workunit.client.0.smithi107.stdout:7/404: mkdir d6/d1d/d64/d89 0 2024-03-04T13:43:09.414 INFO:tasks.workunit.client.1.smithi169.stdout:7/930: write d2/f2d [2587409,120834] 0 2024-03-04T13:43:09.415 INFO:tasks.workunit.client.1.smithi169.stdout:0/796: mkdir d5/d9/dc/d12/da5/d115 0 2024-03-04T13:43:09.417 INFO:tasks.workunit.client.0.smithi107.stdout:1/425: fdatasync d1/d46/d6a/d4e/d60/f4f 0 2024-03-04T13:43:09.419 INFO:tasks.workunit.client.0.smithi107.stdout:1/426: chown d1/d10/d16 7013837 1 2024-03-04T13:43:09.419 INFO:tasks.workunit.client.0.smithi107.stdout:4/376: rmdir d0/d9 39 2024-03-04T13:43:09.422 INFO:tasks.workunit.client.0.smithi107.stdout:8/465: chown d6/d16/d1f/d4c/f52 416112 1 2024-03-04T13:43:09.422 INFO:tasks.workunit.client.0.smithi107.stdout:0/475: dread da/dc/dd/d3e/d26/f71 [0,4194304] 0 2024-03-04T13:43:09.424 INFO:tasks.workunit.client.0.smithi107.stdout:9/386: dwrite d4/da/d11/d20/d64/f78 [0,4194304] 0 2024-03-04T13:43:09.428 INFO:tasks.workunit.client.1.smithi169.stdout:0/797: creat d5/d9/dc/d12/d15/d19/d69/d110/f116 x:0 0 0 2024-03-04T13:43:09.430 INFO:tasks.workunit.client.0.smithi107.stdout:9/387: write d4/da/d11/d4c/d4d/f54 [502451,14753] 0 2024-03-04T13:43:09.430 INFO:tasks.workunit.client.0.smithi107.stdout:0/476: fsync da/dc/dd/d3e/d6e/d73/d78/d86/f90 0 2024-03-04T13:43:09.433 INFO:tasks.workunit.client.0.smithi107.stdout:3/422: rename d1/de/d18/d33/f50 to d1/de/d18/d33/d48/d71/d4c/f91 0 2024-03-04T13:43:09.435 INFO:tasks.workunit.client.0.smithi107.stdout:5/393: symlink d5/dc/d22/l87 0 2024-03-04T13:43:09.441 INFO:tasks.workunit.client.0.smithi107.stdout:2/432: dwrite da/d44/f81 [0,4194304] 0 2024-03-04T13:43:09.446 INFO:tasks.workunit.client.1.smithi169.stdout:5/994: dwrite d1/d23/f40 [0,4194304] 0 2024-03-04T13:43:09.448 INFO:tasks.workunit.client.0.smithi107.stdout:1/427: symlink d1/d46/d6a/d17/l8a 0 2024-03-04T13:43:09.451 INFO:tasks.workunit.client.1.smithi169.stdout:7/931: link d2/de/d3e/f62 d2/de/dcf/f13f 0 2024-03-04T13:43:09.454 INFO:tasks.workunit.client.0.smithi107.stdout:6/476: unlink df/d17/d24/d30/d8b/f39 0 2024-03-04T13:43:09.454 INFO:tasks.workunit.client.1.smithi169.stdout:0/798: dwrite d5/d9/d3d/d5d/de4/d98/dc1/fe6 [0,4194304] 0 2024-03-04T13:43:09.458 INFO:tasks.workunit.client.1.smithi169.stdout:6/920: dread d0/d3c/d40/f6f [0,4194304] 0 2024-03-04T13:43:09.461 INFO:tasks.workunit.client.1.smithi169.stdout:8/942: dwrite d7/d10/d22/d75/d79/fb1 [0,4194304] 0 2024-03-04T13:43:09.462 INFO:tasks.workunit.client.0.smithi107.stdout:0/477: symlink da/dc/dd/d3e/d26/la4 0 2024-03-04T13:43:09.463 INFO:tasks.workunit.client.1.smithi169.stdout:2/940: sync 2024-03-04T13:43:09.466 INFO:tasks.workunit.client.0.smithi107.stdout:4/377: dwrite d0/d9/d18/d29/f5c [0,4194304] 0 2024-03-04T13:43:09.475 INFO:tasks.workunit.client.1.smithi169.stdout:0/799: chown d5/c14 0 1 2024-03-04T13:43:09.480 INFO:tasks.workunit.client.1.smithi169.stdout:7/932: dread d2/de/d3e/d7e/fa7 [0,4194304] 0 2024-03-04T13:43:09.481 INFO:tasks.workunit.client.0.smithi107.stdout:2/433: creat da/df/d85/f95 x:0 0 0 2024-03-04T13:43:09.482 INFO:tasks.workunit.client.1.smithi169.stdout:8/943: mknod d7/d10/d22/d75/de9/c12a 0 2024-03-04T13:43:09.483 INFO:tasks.workunit.client.1.smithi169.stdout:4/899: truncate d4/d11/d3b/d67/f84 2784673 0 2024-03-04T13:43:09.486 INFO:tasks.workunit.client.1.smithi169.stdout:7/933: write d2/d28/d70/ded/f108 [5095753,56101] 0 2024-03-04T13:43:09.487 INFO:tasks.workunit.client.0.smithi107.stdout:8/466: dread f2 [0,4194304] 0 2024-03-04T13:43:09.487 INFO:tasks.workunit.client.0.smithi107.stdout:5/394: rmdir d5/dc 39 2024-03-04T13:43:09.490 INFO:tasks.workunit.client.0.smithi107.stdout:6/477: symlink df/d2e/d36/d71/l9a 0 2024-03-04T13:43:09.492 INFO:tasks.workunit.client.0.smithi107.stdout:9/388: creat d4/da/d11/d20/d23/d7e/f80 x:0 0 0 2024-03-04T13:43:09.492 INFO:tasks.workunit.client.0.smithi107.stdout:1/428: dread d1/d46/d6a/d4e/d60/f63 [0,4194304] 0 2024-03-04T13:43:09.493 INFO:tasks.workunit.client.1.smithi169.stdout:9/979: dwrite d7/d49/fe9 [0,4194304] 0 2024-03-04T13:43:09.496 INFO:tasks.workunit.client.0.smithi107.stdout:5/395: write d5/f6 [3137326,81395] 0 2024-03-04T13:43:09.497 INFO:tasks.workunit.client.0.smithi107.stdout:2/434: mknod da/d38/c96 0 2024-03-04T13:43:09.498 INFO:tasks.workunit.client.0.smithi107.stdout:1/429: write d1/d46/d6a/df/d67/f69 [416790,64145] 0 2024-03-04T13:43:09.498 INFO:tasks.workunit.client.1.smithi169.stdout:2/941: dwrite d1/d5/d19/d63/f127 [0,4194304] 0 2024-03-04T13:43:09.502 INFO:tasks.workunit.client.0.smithi107.stdout:2/435: chown da/d38/d3a/d5d 910637403 1 2024-03-04T13:43:09.503 INFO:tasks.workunit.client.1.smithi169.stdout:0/800: mkdir d5/d9/d117 0 2024-03-04T13:43:09.503 INFO:tasks.workunit.client.1.smithi169.stdout:7/934: fdatasync d2/d28/d70/f79 0 2024-03-04T13:43:09.504 INFO:tasks.workunit.client.0.smithi107.stdout:4/378: dwrite d0/d9/de/d14/dc/d21/f5d [0,4194304] 0 2024-03-04T13:43:09.506 INFO:tasks.workunit.client.1.smithi169.stdout:0/801: readlink d5/d9/d3d/d59/la7 0 2024-03-04T13:43:09.507 INFO:tasks.workunit.client.1.smithi169.stdout:5/995: dwrite d1/d16/d2f/f33 [0,4194304] 0 2024-03-04T13:43:09.508 INFO:tasks.workunit.client.1.smithi169.stdout:0/802: chown d5/d9/ff8 7112814 1 2024-03-04T13:43:09.509 INFO:tasks.workunit.client.0.smithi107.stdout:7/405: getdents d6/d13/d16/d19 0 2024-03-04T13:43:09.511 INFO:tasks.workunit.client.0.smithi107.stdout:4/379: fdatasync d0/d9/de/d14/dc/d21/f2c 0 2024-03-04T13:43:09.513 INFO:tasks.workunit.client.0.smithi107.stdout:5/396: creat d5/d2b/f88 x:0 0 0 2024-03-04T13:43:09.514 INFO:tasks.workunit.client.1.smithi169.stdout:9/980: dread - d7/d25/fbf zero size 2024-03-04T13:43:09.514 INFO:tasks.workunit.client.0.smithi107.stdout:8/467: symlink d6/d16/d1f/l8d 0 2024-03-04T13:43:09.518 INFO:tasks.workunit.client.0.smithi107.stdout:1/430: creat d1/d10/d16/f8b x:0 0 0 2024-03-04T13:43:09.520 INFO:tasks.workunit.client.0.smithi107.stdout:2/436: creat da/df/d3e/f97 x:0 0 0 2024-03-04T13:43:09.522 INFO:tasks.workunit.client.0.smithi107.stdout:9/389: creat d4/da/d11/d4c/d4d/d72/d7b/f81 x:0 0 0 2024-03-04T13:43:09.526 INFO:tasks.workunit.client.0.smithi107.stdout:7/406: creat d6/d1d/d36/d59/f8a x:0 0 0 2024-03-04T13:43:09.526 INFO:tasks.workunit.client.0.smithi107.stdout:0/478: dwrite da/dc/dd/d1e/f9c [0,4194304] 0 2024-03-04T13:43:09.530 INFO:tasks.workunit.client.1.smithi169.stdout:9/981: rename d7/dc/d12/d20/da6/fc5 to d7/d27/d84/f166 0 2024-03-04T13:43:09.530 INFO:tasks.workunit.client.0.smithi107.stdout:7/407: readlink d6/d13/d16/d19/d38/d2c/l6c 0 2024-03-04T13:43:09.532 INFO:tasks.workunit.client.0.smithi107.stdout:7/408: read - d6/d13/d16/d19/d38/d2c/f87 zero size 2024-03-04T13:43:09.535 INFO:tasks.workunit.client.1.smithi169.stdout:6/921: dwrite d0/d32/d3e/f110 [0,4194304] 0 2024-03-04T13:43:09.535 INFO:tasks.workunit.client.1.smithi169.stdout:6/922: stat d0/d3c/d40/d52/dac/lb8 0 2024-03-04T13:43:09.539 INFO:tasks.workunit.client.0.smithi107.stdout:5/397: mkdir d5/d79/d89 0 2024-03-04T13:43:09.544 INFO:tasks.workunit.client.1.smithi169.stdout:8/944: write d7/d10/d17/d44/d36/d62/fa1 [845267,32384] 0 2024-03-04T13:43:09.545 INFO:tasks.workunit.client.0.smithi107.stdout:8/468: rmdir d6/da/d25 39 2024-03-04T13:43:09.552 INFO:tasks.workunit.client.1.smithi169.stdout:0/803: creat d5/d9/dc/f118 x:0 0 0 2024-03-04T13:43:09.557 INFO:tasks.workunit.client.1.smithi169.stdout:9/982: rename d7/d2d/d48/d66/f10e to d7/dc/d12/d32/f167 0 2024-03-04T13:43:09.557 INFO:tasks.workunit.client.0.smithi107.stdout:0/479: unlink da/dc/dd/f1f 0 2024-03-04T13:43:09.560 INFO:tasks.workunit.client.1.smithi169.stdout:6/923: chown d0/d32/d3a/d43/d74/d93/dee 6708 1 2024-03-04T13:43:09.561 INFO:tasks.workunit.client.1.smithi169.stdout:7/935: link d2/d15/fce d2/de/d3e/d41/d69/f140 0 2024-03-04T13:43:09.571 INFO:tasks.workunit.client.0.smithi107.stdout:8/469: dread - d6/da/d25/d5f/f6e zero size 2024-03-04T13:43:09.572 INFO:tasks.workunit.client.1.smithi169.stdout:9/983: chown d7/dc/d95/la9 148 1 2024-03-04T13:43:09.574 INFO:tasks.workunit.client.1.smithi169.stdout:7/936: creat d2/d15/d54/f141 x:0 0 0 2024-03-04T13:43:09.575 INFO:tasks.workunit.client.0.smithi107.stdout:7/409: link d6/d13/d14/f5d d6/d13/d16/f8b 0 2024-03-04T13:43:09.581 INFO:tasks.workunit.client.1.smithi169.stdout:9/984: mknod d7/d25/d3c/c168 0 2024-03-04T13:43:09.582 INFO:tasks.workunit.client.0.smithi107.stdout:8/470: chown d6/d16/d1f/f34 1268951 1 2024-03-04T13:43:09.582 INFO:tasks.workunit.client.1.smithi169.stdout:6/924: dwrite d0/d3c/d40/fde [0,4194304] 0 2024-03-04T13:43:09.584 INFO:tasks.workunit.client.1.smithi169.stdout:7/937: creat d2/d6b/f142 x:0 0 0 2024-03-04T13:43:09.586 INFO:tasks.workunit.client.1.smithi169.stdout:0/804: getdents d5/d9/d3d/d59 0 2024-03-04T13:43:09.586 INFO:tasks.workunit.client.1.smithi169.stdout:4/900: dwrite d4/d1c/f50 [0,4194304] 0 2024-03-04T13:43:09.586 INFO:tasks.workunit.client.0.smithi107.stdout:7/410: symlink d6/d13/d16/d19/d1e/d66/l8c 0 2024-03-04T13:43:09.592 INFO:tasks.workunit.client.1.smithi169.stdout:9/985: creat d7/dc/d12/d149/f169 x:0 0 0 2024-03-04T13:43:09.593 INFO:tasks.workunit.client.1.smithi169.stdout:6/925: rmdir d0/d32/d3a/d43/d74 39 2024-03-04T13:43:09.595 INFO:tasks.workunit.client.1.smithi169.stdout:6/926: chown d0/d32/d3e/d122/l130 42398875 1 2024-03-04T13:43:09.596 INFO:tasks.workunit.client.0.smithi107.stdout:3/423: sync 2024-03-04T13:43:09.596 INFO:tasks.workunit.client.0.smithi107.stdout:7/411: mkdir d6/d13/d16/d19/d38/d2c/d8d 0 2024-03-04T13:43:09.597 INFO:tasks.workunit.client.0.smithi107.stdout:5/398: dread d5/f42 [0,4194304] 0 2024-03-04T13:43:09.600 INFO:tasks.workunit.client.1.smithi169.stdout:5/996: truncate d1/d16/d107/f122 1499918 0 2024-03-04T13:43:09.603 INFO:tasks.workunit.client.0.smithi107.stdout:3/424: chown d1/de/d18/d33/d4d/d4f/d69/f7b 1872687 1 2024-03-04T13:43:09.603 INFO:tasks.workunit.client.0.smithi107.stdout:7/412: readlink d6/d13/d16/d19/d1e/d66/l7c 0 2024-03-04T13:43:09.603 INFO:tasks.workunit.client.0.smithi107.stdout:8/471: rename d6/d16/d1f/c4f to d6/da/c8e 0 2024-03-04T13:43:09.608 INFO:tasks.workunit.client.0.smithi107.stdout:3/425: chown d1/de/d18/d33/d48/d71/d4c 3567311 1 2024-03-04T13:43:09.609 INFO:tasks.workunit.client.0.smithi107.stdout:5/399: unlink d5/d8/d2a/f70 0 2024-03-04T13:43:09.610 INFO:tasks.workunit.client.0.smithi107.stdout:5/400: dread - d5/d66/f86 zero size 2024-03-04T13:43:09.615 INFO:tasks.workunit.client.1.smithi169.stdout:8/945: write d7/d10/d22/f25 [1523359,110809] 0 2024-03-04T13:43:09.615 INFO:tasks.workunit.client.0.smithi107.stdout:4/380: write d0/d9/d18/f57 [2440648,101300] 0 2024-03-04T13:43:09.616 INFO:tasks.workunit.client.1.smithi169.stdout:4/901: creat d4/d11/d2e/f112 x:0 0 0 2024-03-04T13:43:09.621 INFO:tasks.workunit.client.1.smithi169.stdout:0/805: link d5/d9/dc/d12/d66/lc2 d5/d9/d3d/d5d/de4/l119 0 2024-03-04T13:43:09.622 INFO:tasks.workunit.client.1.smithi169.stdout:9/986: creat d7/d25/f16a x:0 0 0 2024-03-04T13:43:09.625 INFO:tasks.workunit.client.1.smithi169.stdout:4/902: chown d4/d11/d3b/d67/d98/la8 714 1 2024-03-04T13:43:09.626 INFO:tasks.workunit.client.1.smithi169.stdout:7/938: link d2/de/d3e/d41/d5b/l82 d2/de/dfb/l143 0 2024-03-04T13:43:09.627 INFO:tasks.workunit.client.0.smithi107.stdout:7/413: rename d6/d13/d16/d19/d1e to d6/d1d/d64/d8e 0 2024-03-04T13:43:09.631 INFO:tasks.workunit.client.0.smithi107.stdout:3/426: creat d1/de/d18/f92 x:0 0 0 2024-03-04T13:43:09.632 INFO:tasks.workunit.client.1.smithi169.stdout:5/997: link d1/d16/d2f/d72/dab/d150/d67/lfb d1/d16/d2f/d39/l15f 0 2024-03-04T13:43:09.633 INFO:tasks.workunit.client.1.smithi169.stdout:4/903: mknod d4/d11/d2e/dcd/dee/d5c/d86/db8/c113 0 2024-03-04T13:43:09.635 INFO:tasks.workunit.client.0.smithi107.stdout:2/437: dwrite da/d44/f52 [0,4194304] 0 2024-03-04T13:43:09.638 INFO:tasks.workunit.client.1.smithi169.stdout:8/946: link d7/d10/d22/d75/d81/cb7 d7/d10/d17/d44/d36/dd3/c12b 0 2024-03-04T13:43:09.643 INFO:tasks.workunit.client.1.smithi169.stdout:8/947: write d7/d10/d17/d44/d5d/fbb [3518278,82658] 0 2024-03-04T13:43:09.644 INFO:tasks.workunit.client.1.smithi169.stdout:5/998: truncate d1/d16/d17/d28/f2c 1353466 0 2024-03-04T13:43:09.645 INFO:tasks.workunit.client.1.smithi169.stdout:1/854: sync 2024-03-04T13:43:09.645 INFO:tasks.workunit.client.1.smithi169.stdout:3/893: sync 2024-03-04T13:43:09.646 INFO:tasks.workunit.client.1.smithi169.stdout:7/939: getdents d2/de/d3e/d41/d89/dd6 0 2024-03-04T13:43:09.648 INFO:tasks.workunit.client.0.smithi107.stdout:8/472: dwrite d6/fe [0,4194304] 0 2024-03-04T13:43:09.649 INFO:tasks.workunit.client.0.smithi107.stdout:7/414: write d6/d1d/d36/d59/f8a [514435,102451] 0 2024-03-04T13:43:09.654 INFO:tasks.workunit.client.1.smithi169.stdout:0/806: dwrite d5/d9/f6e [0,4194304] 0 2024-03-04T13:43:09.655 INFO:tasks.workunit.client.0.smithi107.stdout:3/427: fdatasync d1/de/f68 0 2024-03-04T13:43:09.656 INFO:tasks.workunit.client.0.smithi107.stdout:1/431: dwrite d1/d46/d6a/df/d33/d5d/d24/f44 [0,4194304] 0 2024-03-04T13:43:09.661 INFO:tasks.workunit.client.0.smithi107.stdout:7/415: write d6/d13/d16/f7b [122177,22958] 0 2024-03-04T13:43:09.664 INFO:tasks.workunit.client.1.smithi169.stdout:5/999: unlink d1/d16/d17/d1b/f106 0 2024-03-04T13:43:09.665 INFO:tasks.workunit.client.0.smithi107.stdout:7/416: chown f0 1 1 2024-03-04T13:43:09.665 INFO:tasks.workunit.client.0.smithi107.stdout:0/480: dwrite da/dc/dd/d3e/d26/f58 [0,4194304] 0 2024-03-04T13:43:09.667 INFO:tasks.workunit.client.1.smithi169.stdout:1/855: readlink d4/d5/d36/d35/d70/lac 0 2024-03-04T13:43:09.670 INFO:tasks.workunit.client.1.smithi169.stdout:7/940: write d2/d15/d54/f141 [498750,6808] 0 2024-03-04T13:43:09.671 INFO:tasks.workunit.client.0.smithi107.stdout:3/428: write d1/de/d18/d33/d4d/d4f/d69/f7b [116488,37323] 0 2024-03-04T13:43:09.673 INFO:tasks.workunit.client.0.smithi107.stdout:5/401: dwrite d5/d8/f11 [0,4194304] 0 2024-03-04T13:43:09.681 INFO:tasks.workunit.client.1.smithi169.stdout:4/904: dwrite d4/d1c/f59 [0,4194304] 0 2024-03-04T13:43:09.681 INFO:tasks.workunit.client.1.smithi169.stdout:6/927: dwrite d0/d3c/d40/d52/dac/fd9 [0,4194304] 0 2024-03-04T13:43:09.685 INFO:tasks.workunit.client.0.smithi107.stdout:6/478: sync 2024-03-04T13:43:09.685 INFO:tasks.workunit.client.0.smithi107.stdout:4/381: getdents d0/d9/d18/d29 0 2024-03-04T13:43:09.687 INFO:tasks.workunit.client.1.smithi169.stdout:8/948: dwrite d7/d10/d22/d75/d81/da4/f122 [0,4194304] 0 2024-03-04T13:43:09.688 INFO:tasks.workunit.client.0.smithi107.stdout:1/432: mknod d1/d46/c8c 0 2024-03-04T13:43:09.688 INFO:tasks.workunit.client.0.smithi107.stdout:4/382: write d0/d9/f11 [904938,72811] 0 2024-03-04T13:43:09.697 INFO:tasks.workunit.client.0.smithi107.stdout:4/383: chown d0/d9/de/d14/dc/d21/f3f 3343 1 2024-03-04T13:43:09.697 INFO:tasks.workunit.client.1.smithi169.stdout:2/942: sync 2024-03-04T13:43:09.697 INFO:tasks.workunit.client.1.smithi169.stdout:7/941: mknod d2/d28/d68/c144 0 2024-03-04T13:43:09.697 INFO:tasks.workunit.client.1.smithi169.stdout:2/943: readlink d1/d5/d19/d68/dd9/d115/l11d 0 2024-03-04T13:43:09.698 INFO:tasks.workunit.client.0.smithi107.stdout:2/438: mkdir da/d44/d8b/d98 0 2024-03-04T13:43:09.699 INFO:tasks.workunit.client.1.smithi169.stdout:6/928: symlink d0/d3c/d40/d52/d98/l14a 0 2024-03-04T13:43:09.705 INFO:tasks.workunit.client.1.smithi169.stdout:2/944: chown d1/d5/d22/d45/d4a/c59 65166 1 2024-03-04T13:43:09.706 INFO:tasks.workunit.client.1.smithi169.stdout:8/949: mkdir d7/d120/d12c 0 2024-03-04T13:43:09.710 INFO:tasks.workunit.client.0.smithi107.stdout:5/402: rename d5/d2b/d4b/c51 to d5/d2b/d4b/c8a 0 2024-03-04T13:43:09.711 INFO:tasks.workunit.client.0.smithi107.stdout:8/473: write f5 [3443612,117596] 0 2024-03-04T13:43:09.712 INFO:tasks.workunit.client.0.smithi107.stdout:6/479: stat df/f43 0 2024-03-04T13:43:09.714 INFO:tasks.workunit.client.1.smithi169.stdout:7/942: fdatasync d2/f22 0 2024-03-04T13:43:09.716 INFO:tasks.workunit.client.0.smithi107.stdout:5/403: fsync d5/d8/d2a/d43/f57 0 2024-03-04T13:43:09.717 INFO:tasks.workunit.client.0.smithi107.stdout:8/474: chown d6/d16/d59/d3e/f5a 33 1 2024-03-04T13:43:09.717 INFO:tasks.workunit.client.0.smithi107.stdout:2/439: symlink da/d6c/l99 0 2024-03-04T13:43:09.719 INFO:tasks.workunit.client.1.smithi169.stdout:7/943: chown d2/de/d3e/f13d 119578 1 2024-03-04T13:43:09.721 INFO:tasks.workunit.client.0.smithi107.stdout:9/390: sync 2024-03-04T13:43:09.721 INFO:tasks.workunit.client.0.smithi107.stdout:2/440: chown da/df/c17 12061 1 2024-03-04T13:43:09.722 INFO:tasks.workunit.client.0.smithi107.stdout:6/480: write df/d17/d24/d30/d8b/f87 [913821,100936] 0 2024-03-04T13:43:09.724 INFO:tasks.workunit.client.0.smithi107.stdout:9/391: chown d4/f45 12867 1 2024-03-04T13:43:09.726 INFO:tasks.workunit.client.1.smithi169.stdout:4/905: dwrite d4/d11/d2e/dcd/dde/f10f [0,4194304] 0 2024-03-04T13:43:09.726 INFO:tasks.workunit.client.0.smithi107.stdout:4/384: creat d0/d9/d78/f79 x:0 0 0 2024-03-04T13:43:09.728 INFO:tasks.workunit.client.0.smithi107.stdout:1/433: rename d1/d46/d6a/df/d33/d5d/d24/c45 to d1/d46/d6a/d4e/d60/c8d 0 2024-03-04T13:43:09.730 INFO:tasks.workunit.client.0.smithi107.stdout:4/385: write d0/d9/d78/f79 [882965,70962] 0 2024-03-04T13:43:09.736 INFO:tasks.workunit.client.0.smithi107.stdout:9/392: read d4/da/f4a [183788,18347] 0 2024-03-04T13:43:09.737 INFO:tasks.workunit.client.1.smithi169.stdout:2/945: dwrite d1/d5/d22/d3b/f9c [0,4194304] 0 2024-03-04T13:43:09.739 INFO:tasks.workunit.client.1.smithi169.stdout:0/807: write d5/d9/dc/d12/d15/d19/d69/d110/f97 [1677802,67824] 0 2024-03-04T13:43:09.740 INFO:tasks.workunit.client.1.smithi169.stdout:9/987: dread d7/dc/d12/d59/f6d [4194304,4194304] 0 2024-03-04T13:43:09.741 INFO:tasks.workunit.client.0.smithi107.stdout:3/429: dread d1/de/d11/f3e [0,4194304] 0 2024-03-04T13:43:09.741 INFO:tasks.workunit.client.0.smithi107.stdout:5/404: getdents d5/d8/d14/d5a/d68 0 2024-03-04T13:43:09.742 INFO:tasks.workunit.client.0.smithi107.stdout:3/430: chown d1 0 1 2024-03-04T13:43:09.745 INFO:tasks.workunit.client.1.smithi169.stdout:4/906: creat d4/d11/d2e/d6c/d78/da5/d77/f114 x:0 0 0 2024-03-04T13:43:09.746 INFO:tasks.workunit.client.0.smithi107.stdout:2/441: dwrite da/d34/f41 [0,4194304] 0 2024-03-04T13:43:09.746 INFO:tasks.workunit.client.1.smithi169.stdout:0/808: stat d5/d9/dc/d12/d15/l87 0 2024-03-04T13:43:09.747 INFO:tasks.workunit.client.1.smithi169.stdout:2/946: dread - d1/d5/d19/d68/dd9/d115/ff8 zero size 2024-03-04T13:43:09.747 INFO:tasks.workunit.client.0.smithi107.stdout:2/442: readlink da/d49/l66 0 2024-03-04T13:43:09.749 INFO:tasks.workunit.client.0.smithi107.stdout:1/434: symlink d1/d46/d6a/df/d67/l8e 0 2024-03-04T13:43:09.750 INFO:tasks.workunit.client.1.smithi169.stdout:1/856: dwrite d4/d5/f57 [0,4194304] 0 2024-03-04T13:43:09.752 INFO:tasks.workunit.client.0.smithi107.stdout:4/386: creat d0/d9/d10/f7a x:0 0 0 2024-03-04T13:43:09.755 INFO:tasks.workunit.client.0.smithi107.stdout:4/387: readlink d0/d9/d1e/l4f 0 2024-03-04T13:43:09.766 INFO:tasks.workunit.client.1.smithi169.stdout:1/857: truncate d4/d5/d36/d75/dba/f101 464140 0 2024-03-04T13:43:09.770 INFO:tasks.workunit.client.1.smithi169.stdout:4/907: creat d4/d11/d2e/dcd/dee/d5c/d86/db8/f115 x:0 0 0 2024-03-04T13:43:09.772 INFO:tasks.workunit.client.0.smithi107.stdout:0/481: rename da/dc/dd/d3e/d6e/d73/d78/c82 to da/dc/dd/d3e/d26/d69/d6b/ca5 0 2024-03-04T13:43:09.773 INFO:tasks.workunit.client.1.smithi169.stdout:3/894: dread f5 [4194304,4194304] 0 2024-03-04T13:43:09.775 INFO:tasks.workunit.client.1.smithi169.stdout:6/929: dread d0/d3c/d40/f51 [0,4194304] 0 2024-03-04T13:43:09.775 INFO:tasks.workunit.client.1.smithi169.stdout:0/809: rmdir d5/d9/d3d/d59/d64/d7f 39 2024-03-04T13:43:09.777 INFO:tasks.workunit.client.1.smithi169.stdout:0/810: stat d5/d9/d117 0 2024-03-04T13:43:09.779 INFO:tasks.workunit.client.1.smithi169.stdout:3/895: read d7/de/d2b/d7d/d85/f4f [2068633,67375] 0 2024-03-04T13:43:09.782 INFO:tasks.workunit.client.1.smithi169.stdout:9/988: getdents d7/d27/d84/de6 0 2024-03-04T13:43:09.783 INFO:tasks.workunit.client.0.smithi107.stdout:0/482: dread da/f81 [0,4194304] 0 2024-03-04T13:43:09.783 INFO:tasks.workunit.client.0.smithi107.stdout:1/435: creat d1/d46/d6a/d65/f8f x:0 0 0 2024-03-04T13:43:09.783 INFO:tasks.workunit.client.1.smithi169.stdout:0/811: write d5/d9/dc/d12/d15/d19/d69/d110/f116 [822327,80478] 0 2024-03-04T13:43:09.784 INFO:tasks.workunit.client.0.smithi107.stdout:8/475: dread d6/d16/f24 [0,4194304] 0 2024-03-04T13:43:09.784 INFO:tasks.workunit.client.0.smithi107.stdout:4/388: mkdir d0/d9/de/d14/d66/d7b 0 2024-03-04T13:43:09.786 INFO:tasks.workunit.client.0.smithi107.stdout:7/417: write d6/d1d/d64/d8e/d3d/f78 [180470,87740] 0 2024-03-04T13:43:09.789 INFO:tasks.workunit.client.0.smithi107.stdout:9/393: creat d4/da/d11/d20/d64/d77/f82 x:0 0 0 2024-03-04T13:43:09.790 INFO:tasks.workunit.client.1.smithi169.stdout:7/944: dwrite d2/f22 [4194304,4194304] 0 2024-03-04T13:43:09.790 INFO:tasks.workunit.client.0.smithi107.stdout:7/418: fdatasync d6/d1d/d64/d8e/d3d/f78 0 2024-03-04T13:43:09.792 INFO:tasks.workunit.client.1.smithi169.stdout:7/945: write d2/de/d3e/d7e/fdd [311032,31781] 0 2024-03-04T13:43:09.793 INFO:tasks.workunit.client.0.smithi107.stdout:7/419: chown d6/d1d/d64/d8e/f5c 1784 1 2024-03-04T13:43:09.794 INFO:tasks.workunit.client.1.smithi169.stdout:7/946: readlink d2/d15/dad/lba 0 2024-03-04T13:43:09.796 INFO:tasks.workunit.client.0.smithi107.stdout:7/420: chown d6/d1d/d64/d8e/d3d/f43 775 1 2024-03-04T13:43:09.796 INFO:tasks.workunit.client.1.smithi169.stdout:7/947: chown d2/de/d3e/d7e/dbb/cfe 695056 1 2024-03-04T13:43:09.798 INFO:tasks.workunit.client.0.smithi107.stdout:6/481: rename df/d1b/d2c to df/d17/d6b/d9b 0 2024-03-04T13:43:09.798 INFO:tasks.workunit.client.0.smithi107.stdout:5/405: write d5/d8/d2a/f53 [4219745,123077] 0 2024-03-04T13:43:09.801 INFO:tasks.workunit.client.1.smithi169.stdout:0/812: creat d5/d9/d3d/f11a x:0 0 0 2024-03-04T13:43:09.804 INFO:tasks.workunit.client.0.smithi107.stdout:3/431: write d1/d1e/d26/f60 [19912,128787] 0 2024-03-04T13:43:09.808 INFO:tasks.workunit.client.0.smithi107.stdout:4/389: mkdir d0/d9/d10/d7c 0 2024-03-04T13:43:09.809 INFO:tasks.workunit.client.0.smithi107.stdout:8/476: dread d6/d16/f27 [4194304,4194304] 0 2024-03-04T13:43:09.811 INFO:tasks.workunit.client.1.smithi169.stdout:6/930: rmdir d0/d32/d3a/d43/d74/da6 39 2024-03-04T13:43:09.814 INFO:tasks.workunit.client.1.smithi169.stdout:3/896: mknod d7/de/d2b/d7d/d85/d13/df5/c124 0 2024-03-04T13:43:09.815 INFO:tasks.workunit.client.0.smithi107.stdout:9/394: creat d4/da/d11/d4c/d4d/d72/d7b/f83 x:0 0 0 2024-03-04T13:43:09.817 INFO:tasks.workunit.client.1.smithi169.stdout:3/897: chown d7/de/l57 1061986345 1 2024-03-04T13:43:09.817 INFO:tasks.workunit.client.0.smithi107.stdout:0/483: write da/dc/dd/f46 [832656,52730] 0 2024-03-04T13:43:09.818 INFO:tasks.workunit.client.0.smithi107.stdout:7/421: creat d6/d17/f8f x:0 0 0 2024-03-04T13:43:09.819 INFO:tasks.workunit.client.0.smithi107.stdout:5/406: dwrite d5/f73 [0,4194304] 0 2024-03-04T13:43:09.820 INFO:tasks.workunit.client.1.smithi169.stdout:0/813: creat d5/d9/dc/d12/d15/d19/d61/f11b x:0 0 0 2024-03-04T13:43:09.821 INFO:tasks.workunit.client.0.smithi107.stdout:7/422: readlink d6/d13/d14/l4f 0 2024-03-04T13:43:09.821 INFO:tasks.workunit.client.0.smithi107.stdout:2/443: rename da/df/d33 to da/df/d73/d7a/d9a 0 2024-03-04T13:43:09.822 INFO:tasks.workunit.client.0.smithi107.stdout:7/423: read d6/d13/d16/d1c/f2d [421376,79467] 0 2024-03-04T13:43:09.823 INFO:tasks.workunit.client.0.smithi107.stdout:7/424: dread - d6/d13/d16/f67 zero size 2024-03-04T13:43:09.824 INFO:tasks.workunit.client.1.smithi169.stdout:9/989: creat d7/d2d/f16b x:0 0 0 2024-03-04T13:43:09.825 INFO:tasks.workunit.client.1.smithi169.stdout:4/908: dread d4/d11/d3b/d67/dbb/f76 [0,4194304] 0 2024-03-04T13:43:09.826 INFO:tasks.workunit.client.0.smithi107.stdout:6/482: mknod df/d17/d24/d30/d5d/c9c 0 2024-03-04T13:43:09.826 INFO:tasks.workunit.client.0.smithi107.stdout:3/432: unlink d1/l2 0 2024-03-04T13:43:09.827 INFO:tasks.workunit.client.0.smithi107.stdout:3/433: write d1/d29/f7c [614637,38976] 0 2024-03-04T13:43:09.828 INFO:tasks.workunit.client.1.smithi169.stdout:4/909: stat d4/d11/d2e/d6c/cc4 0 2024-03-04T13:43:09.829 INFO:tasks.workunit.client.0.smithi107.stdout:4/390: mkdir d0/d2f/d7d 0 2024-03-04T13:43:09.830 INFO:tasks.workunit.client.1.smithi169.stdout:6/931: creat d0/d32/d3a/def/f14b x:0 0 0 2024-03-04T13:43:09.832 INFO:tasks.workunit.client.1.smithi169.stdout:3/898: symlink d7/de/d2b/d7d/d85/d13/d5a/l125 0 2024-03-04T13:43:09.832 INFO:tasks.workunit.client.0.smithi107.stdout:1/436: symlink d1/d46/d6a/df/d33/d89/l90 0 2024-03-04T13:43:09.834 INFO:tasks.workunit.client.0.smithi107.stdout:8/477: creat d6/d16/d69/f8f x:0 0 0 2024-03-04T13:43:09.836 INFO:tasks.workunit.client.1.smithi169.stdout:7/948: dread d2/d15/d17/d6f/f72 [0,4194304] 0 2024-03-04T13:43:09.837 INFO:tasks.workunit.client.0.smithi107.stdout:9/395: mkdir d4/da/d11/d20/d23/d7e/d84 0 2024-03-04T13:43:09.838 INFO:tasks.workunit.client.1.smithi169.stdout:1/858: getdents d4/dd8 0 2024-03-04T13:43:09.845 INFO:tasks.workunit.client.0.smithi107.stdout:2/444: creat da/d49/f9b x:0 0 0 2024-03-04T13:43:09.845 INFO:tasks.workunit.client.0.smithi107.stdout:7/425: rename d6/d17/l25 to d6/d13/d16/d19/d38/d62/l90 0 2024-03-04T13:43:09.848 INFO:tasks.workunit.client.0.smithi107.stdout:6/483: mknod df/d17/d24/d30/c9d 0 2024-03-04T13:43:09.849 INFO:tasks.workunit.client.1.smithi169.stdout:6/932: dread d0/d3c/d40/f6f [0,4194304] 0 2024-03-04T13:43:09.850 INFO:tasks.workunit.client.1.smithi169.stdout:7/949: symlink d2/de/d3e/d41/d69/d78/l145 0 2024-03-04T13:43:09.851 INFO:tasks.workunit.client.0.smithi107.stdout:4/391: unlink d0/d9/d1e/f2b 0 2024-03-04T13:43:09.854 INFO:tasks.workunit.client.1.smithi169.stdout:9/990: dread d7/dc/d12/d32/d42/f58 [0,4194304] 0 2024-03-04T13:43:09.854 INFO:tasks.workunit.client.1.smithi169.stdout:1/859: mknod d4/d5/d36/df1/c117 0 2024-03-04T13:43:09.854 INFO:tasks.workunit.client.0.smithi107.stdout:4/392: write d0/d9/d18/d29/f35 [2338052,106968] 0 2024-03-04T13:43:09.854 INFO:tasks.workunit.client.0.smithi107.stdout:5/407: dwrite d5/dc/f55 [0,4194304] 0 2024-03-04T13:43:09.855 INFO:tasks.workunit.client.1.smithi169.stdout:4/910: dwrite d4/d11/d3b/d67/dbb/d6e/fd4 [4194304,4194304] 0 2024-03-04T13:43:09.857 INFO:tasks.workunit.client.0.smithi107.stdout:9/396: readlink d4/d34/l71 0 2024-03-04T13:43:09.860 INFO:tasks.workunit.client.1.smithi169.stdout:6/933: creat d0/d3c/d40/f14c x:0 0 0 2024-03-04T13:43:09.861 INFO:tasks.workunit.client.1.smithi169.stdout:4/911: dread - d4/d11/d2e/d6c/d78/da5/d77/dc0/df5/f103 zero size 2024-03-04T13:43:09.864 INFO:tasks.workunit.client.1.smithi169.stdout:9/991: creat d7/d140/f16c x:0 0 0 2024-03-04T13:43:09.865 INFO:tasks.workunit.client.0.smithi107.stdout:0/484: mknod da/dc/dd/d3e/d26/ca6 0 2024-03-04T13:43:09.868 INFO:tasks.workunit.client.1.smithi169.stdout:7/950: symlink d2/d15/l146 0 2024-03-04T13:43:09.869 INFO:tasks.workunit.client.1.smithi169.stdout:0/814: write d5/d9/d3d/d5d/de4/f57 [1495812,129551] 0 2024-03-04T13:43:09.872 INFO:tasks.workunit.client.1.smithi169.stdout:3/899: write d7/de/d2b/d7d/ffa [141148,37989] 0 2024-03-04T13:43:09.875 INFO:tasks.workunit.client.1.smithi169.stdout:1/860: rename d4/d5/d36/d35/d5b/f7f to d4/d5/d36/d35/d16/d81/db4/f118 0 2024-03-04T13:43:09.877 INFO:tasks.workunit.client.1.smithi169.stdout:4/912: creat d4/d11/f116 x:0 0 0 2024-03-04T13:43:09.878 INFO:tasks.workunit.client.0.smithi107.stdout:6/484: mknod df/d17/d24/c9e 0 2024-03-04T13:43:09.878 INFO:tasks.workunit.client.1.smithi169.stdout:0/815: symlink d5/d4c/d65/d95/db0/d10c/l11c 0 2024-03-04T13:43:09.879 INFO:tasks.workunit.client.0.smithi107.stdout:1/437: symlink d1/d46/d6a/df/d33/d5d/d4c/l91 0 2024-03-04T13:43:09.880 INFO:tasks.workunit.client.1.smithi169.stdout:6/934: rename d0/f15 to d0/d32/d3a/d43/d74/da6/f14d 0 2024-03-04T13:43:09.881 INFO:tasks.workunit.client.0.smithi107.stdout:6/485: write df/d17/d6b/d9b/d46/f68 [458028,71401] 0 2024-03-04T13:43:09.882 INFO:tasks.workunit.client.1.smithi169.stdout:6/935: fdatasync d0/d32/d3e/d12e/f131 0 2024-03-04T13:43:09.886 INFO:tasks.workunit.client.1.smithi169.stdout:3/900: mknod d7/de/d2b/d7d/d85/d13/d1a/d98/c126 0 2024-03-04T13:43:09.887 INFO:tasks.workunit.client.0.smithi107.stdout:8/478: mknod d6/da/d25/c90 0 2024-03-04T13:43:09.894 INFO:tasks.workunit.client.0.smithi107.stdout:9/397: rmdir d4/da/d11/d65 39 2024-03-04T13:43:09.894 INFO:tasks.workunit.client.0.smithi107.stdout:3/434: truncate d1/d2d/f7f 480897 0 2024-03-04T13:43:09.895 INFO:tasks.workunit.client.1.smithi169.stdout:4/913: truncate d4/f41 7561058 0 2024-03-04T13:43:09.898 INFO:tasks.workunit.client.0.smithi107.stdout:5/408: dwrite d5/d8/f17 [0,4194304] 0 2024-03-04T13:43:09.899 INFO:tasks.workunit.client.0.smithi107.stdout:6/486: symlink df/d17/d24/d27/l9f 0 2024-03-04T13:43:09.900 INFO:tasks.workunit.client.1.smithi169.stdout:3/901: getdents d7/de/d2b/d7d 0 2024-03-04T13:43:09.902 INFO:tasks.workunit.client.0.smithi107.stdout:7/426: link d6/d13/d16/d19/d38/d37/c4c d6/d13/d16/d19/d38/d62/c91 0 2024-03-04T13:43:09.902 INFO:tasks.workunit.client.1.smithi169.stdout:1/861: write d4/d5/f5a [1348090,74000] 0 2024-03-04T13:43:09.904 INFO:tasks.workunit.client.0.smithi107.stdout:6/487: readlink df/d17/d6b/d9b/d46/l8c 0 2024-03-04T13:43:09.904 INFO:tasks.workunit.client.0.smithi107.stdout:0/485: symlink da/dc/dd/d3e/d26/d88/la7 0 2024-03-04T13:43:09.906 INFO:tasks.workunit.client.1.smithi169.stdout:3/902: truncate f5 3578011 0 2024-03-04T13:43:09.909 INFO:tasks.workunit.client.1.smithi169.stdout:4/914: symlink d4/d11/d3b/d67/l117 0 2024-03-04T13:43:09.914 INFO:tasks.workunit.client.1.smithi169.stdout:4/915: truncate d4/d11/d2e/d39/d61/d6b/f10d 179302 0 2024-03-04T13:43:09.915 INFO:tasks.workunit.client.0.smithi107.stdout:2/445: dwrite da/d49/d5c/f91 [0,4194304] 0 2024-03-04T13:43:09.918 INFO:tasks.workunit.client.1.smithi169.stdout:3/903: symlink d7/de/d2b/d7d/d85/d13/d1a/d2d/d56/d5e/d67/l127 0 2024-03-04T13:43:09.922 INFO:tasks.workunit.client.1.smithi169.stdout:3/904: stat d7/de/d2b/d7d/d85/d13/d1a/d2d/d56/d5e/d67/d5c/l102 0 2024-03-04T13:43:09.922 INFO:tasks.workunit.client.1.smithi169.stdout:4/916: unlink d4/d11/d2e/d6c/d78/da5/d77/ld9 0 2024-03-04T13:43:09.925 INFO:tasks.workunit.client.0.smithi107.stdout:6/488: mknod df/d57/ca0 0 2024-03-04T13:43:09.925 INFO:tasks.workunit.client.0.smithi107.stdout:5/409: mknod d5/c8b 0 2024-03-04T13:43:09.929 INFO:tasks.workunit.client.0.smithi107.stdout:8/479: dwrite d6/d16/d1f/d63/d73/f7b [0,4194304] 0 2024-03-04T13:43:09.931 INFO:tasks.workunit.client.1.smithi169.stdout:4/917: mknod d4/d11/d2e/d6c/d78/da5/d77/dc0/c118 0 2024-03-04T13:43:09.933 INFO:tasks.workunit.client.1.smithi169.stdout:4/918: fdatasync d4/d11/d2e/d21/da9/fcb 0 2024-03-04T13:43:09.934 INFO:tasks.workunit.client.0.smithi107.stdout:0/486: creat da/dc/dd/d3e/d6e/fa8 x:0 0 0 2024-03-04T13:43:09.934 INFO:tasks.workunit.client.0.smithi107.stdout:4/393: dwrite d0/d9/d10/f25 [0,4194304] 0 2024-03-04T13:43:09.934 INFO:tasks.workunit.client.0.smithi107.stdout:1/438: write d1/d46/d6a/df/f3a [1174582,106622] 0 2024-03-04T13:43:09.934 INFO:tasks.workunit.client.0.smithi107.stdout:9/398: write d4/f45 [5186224,123837] 0 2024-03-04T13:43:09.935 INFO:tasks.workunit.client.0.smithi107.stdout:3/435: dwrite d1/de/d18/d33/f8f [0,4194304] 0 2024-03-04T13:43:09.936 INFO:tasks.workunit.client.1.smithi169.stdout:4/919: chown d4/d1c/lc8 4242163 1 2024-03-04T13:43:09.948 INFO:tasks.workunit.client.1.smithi169.stdout:3/905: dwrite d7/de/d2b/d7d/d85/d13/d1a/d2d/d56/d5e/d5f/f100 [0,4194304] 0 2024-03-04T13:43:09.961 INFO:tasks.workunit.client.0.smithi107.stdout:6/489: truncate df/d2e/d36/d77/f84 897962 0 2024-03-04T13:43:09.961 INFO:tasks.workunit.client.1.smithi169.stdout:4/920: dread d4/d11/d2e/dcd/dee/d5c/f105 [0,4194304] 0 2024-03-04T13:43:09.962 INFO:tasks.workunit.client.0.smithi107.stdout:1/439: truncate d1/d46/d6a/d75/f66 373787 0 2024-03-04T13:43:09.962 INFO:tasks.workunit.client.0.smithi107.stdout:2/446: mkdir da/d44/d9c 0 2024-03-04T13:43:09.963 INFO:tasks.workunit.client.0.smithi107.stdout:7/427: dwrite f0 [0,4194304] 0 2024-03-04T13:43:09.964 INFO:tasks.workunit.client.0.smithi107.stdout:4/394: read d0/d9/f5b [493519,76135] 0 2024-03-04T13:43:09.965 INFO:tasks.workunit.client.0.smithi107.stdout:1/440: chown d1/d46/c8c 1556 1 2024-03-04T13:43:09.965 INFO:tasks.workunit.client.0.smithi107.stdout:0/487: creat da/dc/dd/d3e/d26/d88/fa9 x:0 0 0 2024-03-04T13:43:09.966 INFO:tasks.workunit.client.1.smithi169.stdout:4/921: chown d4/d11/d2e/d6c/d78/da5/d77 3 1 2024-03-04T13:43:09.972 INFO:tasks.workunit.client.0.smithi107.stdout:7/428: chown d6/d13/d16/d19/d38/d2c/l41 10 1 2024-03-04T13:43:09.972 INFO:tasks.workunit.client.0.smithi107.stdout:9/399: creat d4/da/d11/d20/d2f/f85 x:0 0 0 2024-03-04T13:43:09.972 INFO:tasks.workunit.client.0.smithi107.stdout:0/488: chown da/dc/dd/d3e/d6e/d73/d78/d86/l8d 1362208177 1 2024-03-04T13:43:09.972 INFO:tasks.workunit.client.0.smithi107.stdout:6/490: creat df/d2e/d36/d5b/fa1 x:0 0 0 2024-03-04T13:43:09.975 INFO:tasks.workunit.client.0.smithi107.stdout:1/441: symlink d1/d46/l92 0 2024-03-04T13:43:09.976 INFO:tasks.workunit.client.0.smithi107.stdout:4/395: symlink d0/d9/de/l7e 0 2024-03-04T13:43:09.976 INFO:tasks.workunit.client.0.smithi107.stdout:9/400: readlink d4/d32/l37 0 2024-03-04T13:43:09.977 INFO:tasks.workunit.client.0.smithi107.stdout:5/410: dwrite d5/f73 [0,4194304] 0 2024-03-04T13:43:09.982 INFO:tasks.workunit.client.0.smithi107.stdout:9/401: mknod d4/d32/c86 0 2024-03-04T13:43:09.983 INFO:tasks.workunit.client.0.smithi107.stdout:6/491: creat df/d2e/d36/fa2 x:0 0 0 2024-03-04T13:43:09.984 INFO:tasks.workunit.client.1.smithi169.stdout:2/947: sync 2024-03-04T13:43:09.984 INFO:tasks.workunit.client.1.smithi169.stdout:8/950: sync 2024-03-04T13:43:09.987 INFO:tasks.workunit.client.1.smithi169.stdout:8/951: dread - d7/d10/d22/d75/d81/f117 zero size 2024-03-04T13:43:09.988 INFO:tasks.workunit.client.0.smithi107.stdout:4/396: symlink d0/d9/d10/l7f 0 2024-03-04T13:43:09.989 INFO:tasks.workunit.client.1.smithi169.stdout:8/952: chown d7/d10/d22 209 1 2024-03-04T13:43:09.993 INFO:tasks.workunit.client.0.smithi107.stdout:0/489: rename da/dc/d34/f42 to da/dc/dd/d3e/d26/d88/faa 0 2024-03-04T13:43:09.999 INFO:tasks.workunit.client.1.smithi169.stdout:8/953: mkdir d7/d10/d22/d75/d81/da2/dac/d12d 0 2024-03-04T13:43:10.000 INFO:tasks.workunit.client.0.smithi107.stdout:2/447: dread da/d49/d5c/f91 [0,4194304] 0 2024-03-04T13:43:10.002 INFO:tasks.workunit.client.1.smithi169.stdout:2/948: creat d1/d5/d19/d3c/db8/de1/d64/dce/dd5/df5/d120/d11c/d139/f13e x:0 0 0 2024-03-04T13:43:10.004 INFO:tasks.workunit.client.1.smithi169.stdout:8/954: mknod d7/d10/d22/d75/c12e 0 2024-03-04T13:43:10.006 INFO:tasks.workunit.client.0.smithi107.stdout:9/402: rename d4/da/d11/d20/d39/f3a to d4/d34/f87 0 2024-03-04T13:43:10.006 INFO:tasks.workunit.client.0.smithi107.stdout:4/397: dwrite d0/d9/de/f41 [0,4194304] 0 2024-03-04T13:43:10.010 INFO:tasks.workunit.client.0.smithi107.stdout:9/403: fsync d4/da/d11/d4c/d4d/d72/d7b/f81 0 2024-03-04T13:43:10.012 INFO:tasks.workunit.client.0.smithi107.stdout:9/404: fdatasync d4/da/f63 0 2024-03-04T13:43:10.013 INFO:tasks.workunit.client.0.smithi107.stdout:4/398: dread d0/d9/de/d14/f4e [0,4194304] 0 2024-03-04T13:43:10.015 INFO:tasks.workunit.client.0.smithi107.stdout:0/490: getdents da/dc/dd/d3e/d26/d69/d6b 0 2024-03-04T13:43:10.017 INFO:tasks.workunit.client.0.smithi107.stdout:1/442: truncate d1/d46/d6a/d4e/d60/f78 2104883 0 2024-03-04T13:43:10.018 INFO:tasks.workunit.client.0.smithi107.stdout:5/411: dwrite d5/f49 [0,4194304] 0 2024-03-04T13:43:10.027 INFO:tasks.workunit.client.0.smithi107.stdout:4/399: read d0/d9/d10/f1c [1048102,119484] 0 2024-03-04T13:43:10.028 INFO:tasks.workunit.client.0.smithi107.stdout:5/412: rmdir d5/d8/d14/d5a/d69 39 2024-03-04T13:43:10.030 INFO:tasks.workunit.client.0.smithi107.stdout:1/443: rename d1/f72 to d1/d46/d6a/df/d33/d5d/d4c/f93 0 2024-03-04T13:43:10.033 INFO:tasks.workunit.client.0.smithi107.stdout:4/400: stat d0/d9/de/d14 0 2024-03-04T13:43:10.034 INFO:tasks.workunit.client.0.smithi107.stdout:7/429: dwrite d6/d13/d16/d1c/f80 [0,4194304] 0 2024-03-04T13:43:10.037 INFO:tasks.workunit.client.0.smithi107.stdout:2/448: dwrite da/f10 [4194304,4194304] 0 2024-03-04T13:43:10.041 INFO:tasks.workunit.client.0.smithi107.stdout:7/430: symlink d6/l92 0 2024-03-04T13:43:10.045 INFO:tasks.workunit.client.0.smithi107.stdout:7/431: chown d6/d13/d16/d19/d39/f54 247648 1 2024-03-04T13:43:10.047 INFO:tasks.workunit.client.0.smithi107.stdout:4/401: rename d0/d9/d10/d7c to d0/d9/d10/d80 0 2024-03-04T13:43:10.052 INFO:tasks.workunit.client.0.smithi107.stdout:4/402: write d0/d9/d10/f25 [463902,45786] 0 2024-03-04T13:43:10.055 INFO:tasks.workunit.client.0.smithi107.stdout:5/413: link d5/d8/d14/d5a/d69/l76 d5/d8/d14/d54/l8c 0 2024-03-04T13:43:10.056 INFO:tasks.workunit.client.0.smithi107.stdout:2/449: dread da/d44/f81 [0,4194304] 0 2024-03-04T13:43:10.056 INFO:tasks.workunit.client.0.smithi107.stdout:0/491: dread da/dc/dd/d3e/d26/f71 [0,4194304] 0 2024-03-04T13:43:10.058 INFO:tasks.workunit.client.0.smithi107.stdout:4/403: chown d0/d9/f11 218452 1 2024-03-04T13:43:10.062 INFO:tasks.workunit.client.0.smithi107.stdout:5/414: symlink d5/dc/l8d 0 2024-03-04T13:43:10.062 INFO:tasks.workunit.client.0.smithi107.stdout:2/450: chown da/c2a 420 1 2024-03-04T13:43:10.062 INFO:tasks.workunit.client.0.smithi107.stdout:9/405: dread d4/da/f15 [0,4194304] 0 2024-03-04T13:43:10.064 INFO:tasks.workunit.client.0.smithi107.stdout:7/432: dread d6/d13/d16/d19/d39/f54 [0,4194304] 0 2024-03-04T13:43:10.064 INFO:tasks.workunit.client.0.smithi107.stdout:0/492: dread da/f62 [0,4194304] 0 2024-03-04T13:43:10.068 INFO:tasks.workunit.client.0.smithi107.stdout:7/433: chown d6/d17/f6d 39 1 2024-03-04T13:43:10.068 INFO:tasks.workunit.client.0.smithi107.stdout:0/493: readlink da/dc/dd/d3e/d6e/d73/d78/d86/l8d 0 2024-03-04T13:43:10.069 INFO:tasks.workunit.client.0.smithi107.stdout:9/406: rename d4/d34/l71 to d4/d32/l88 0 2024-03-04T13:43:10.070 INFO:tasks.workunit.client.0.smithi107.stdout:7/434: stat d6/d13/d16/d19/d38/d6e 0 2024-03-04T13:43:10.072 INFO:tasks.workunit.client.0.smithi107.stdout:5/415: getdents d5/d2b 0 2024-03-04T13:43:10.073 INFO:tasks.workunit.client.0.smithi107.stdout:1/444: truncate d1/d46/d6a/d17/f34 1855500 0 2024-03-04T13:43:10.073 INFO:tasks.workunit.client.1.smithi169.stdout:2/949: symlink d1/d5/d22/d45/d4a/de7/dcb/l13f 0 2024-03-04T13:43:10.077 INFO:tasks.workunit.client.0.smithi107.stdout:1/445: dread d1/d46/d6a/d75/f66 [0,4194304] 0 2024-03-04T13:43:10.078 INFO:tasks.workunit.client.1.smithi169.stdout:3/906: dwrite d7/de/d2b/d7d/d85/d13/d1a/d2d/d56/d5e/d67/ff2 [0,4194304] 0 2024-03-04T13:43:10.078 INFO:tasks.workunit.client.0.smithi107.stdout:4/404: dwrite d0/d9/de/f41 [0,4194304] 0 2024-03-04T13:43:10.081 INFO:tasks.workunit.client.0.smithi107.stdout:2/451: truncate da/df/f24 399853 0 2024-03-04T13:43:10.082 INFO:tasks.workunit.client.0.smithi107.stdout:9/407: mkdir d4/da/d11/d4c/d89 0 2024-03-04T13:43:10.084 INFO:tasks.workunit.client.0.smithi107.stdout:7/435: unlink d6/d1d/c72 0 2024-03-04T13:43:10.086 INFO:tasks.workunit.client.0.smithi107.stdout:4/405: mkdir d0/d9/d10/d80/d81 0 2024-03-04T13:43:10.089 INFO:tasks.workunit.client.0.smithi107.stdout:4/406: write d0/d9/d1e/f75 [352210,102632] 0 2024-03-04T13:43:10.093 INFO:tasks.workunit.client.0.smithi107.stdout:4/407: read - d0/d9/de/f62 zero size 2024-03-04T13:43:10.094 INFO:tasks.workunit.client.0.smithi107.stdout:2/452: write da/d49/f7e [1335263,71150] 0 2024-03-04T13:43:10.095 INFO:tasks.workunit.client.0.smithi107.stdout:4/408: chown d0/l5e 843539314 1 2024-03-04T13:43:10.099 INFO:tasks.workunit.client.0.smithi107.stdout:0/494: creat da/dc/dd/d3e/d6e/d73/d78/fab x:0 0 0 2024-03-04T13:43:10.101 INFO:tasks.workunit.client.0.smithi107.stdout:7/436: rmdir d6/d1d/d36/d59 39 2024-03-04T13:43:10.102 INFO:tasks.workunit.client.0.smithi107.stdout:1/446: getdents d1/d46/d6a/df/d33/d43/d59 0 2024-03-04T13:43:10.104 INFO:tasks.workunit.client.0.smithi107.stdout:9/408: symlink d4/da/d11/d20/d23/d7e/d84/l8a 0 2024-03-04T13:43:10.108 INFO:tasks.workunit.client.0.smithi107.stdout:2/453: mkdir da/d44/d8b/d9d 0 2024-03-04T13:43:10.110 INFO:tasks.workunit.client.0.smithi107.stdout:0/495: creat da/dc/dd/d3e/fac x:0 0 0 2024-03-04T13:43:10.111 INFO:tasks.workunit.client.0.smithi107.stdout:5/416: dwrite d5/dc/f3b [0,4194304] 0 2024-03-04T13:43:10.112 INFO:tasks.workunit.client.0.smithi107.stdout:7/437: rename c1 to d6/d13/d16/d19/d38/c93 0 2024-03-04T13:43:10.113 INFO:tasks.workunit.client.0.smithi107.stdout:0/496: dread - da/dc/dd/d3e/d26/d69/d6b/f94 zero size 2024-03-04T13:43:10.115 INFO:tasks.workunit.client.0.smithi107.stdout:7/438: truncate d6/d13/d14/f86 2617802 0 2024-03-04T13:43:10.117 INFO:tasks.workunit.client.0.smithi107.stdout:4/409: mknod d0/d9/de/d14/d66/d7b/c82 0 2024-03-04T13:43:10.123 INFO:tasks.workunit.client.0.smithi107.stdout:5/417: creat d5/d8/d14/d5a/d6a/f8e x:0 0 0 2024-03-04T13:43:10.128 INFO:tasks.workunit.client.0.smithi107.stdout:0/497: write da/dc/f7c [1435937,32441] 0 2024-03-04T13:43:10.130 INFO:tasks.workunit.client.0.smithi107.stdout:4/410: dread d0/d9/f6b [0,4194304] 0 2024-03-04T13:43:10.130 INFO:tasks.workunit.client.0.smithi107.stdout:9/409: link d4/d34/f3f d4/da/d11/d20/d23/d7e/f8b 0 2024-03-04T13:43:10.131 INFO:tasks.workunit.client.0.smithi107.stdout:0/498: chown da/dc/dd/d3e/d6e/f9d 87511 1 2024-03-04T13:43:10.131 INFO:tasks.workunit.client.0.smithi107.stdout:7/439: dwrite d6/d13/d14/f6b [0,4194304] 0 2024-03-04T13:43:10.136 INFO:tasks.workunit.client.1.smithi169.stdout:3/907: symlink d7/de/d2b/d7d/d85/d18/d21/d3f/d86/l128 0 2024-03-04T13:43:10.137 INFO:tasks.workunit.client.0.smithi107.stdout:5/418: mknod d5/d8/d2a/d62/c8f 0 2024-03-04T13:43:10.139 INFO:tasks.workunit.client.0.smithi107.stdout:9/410: dread - d4/da/d11/d65/f6e zero size 2024-03-04T13:43:10.139 INFO:tasks.workunit.client.0.smithi107.stdout:5/419: fsync d5/d8/d14/d2d/f56 0 2024-03-04T13:43:10.140 INFO:tasks.workunit.client.0.smithi107.stdout:4/411: symlink d0/d9/de/d14/d66/d7b/l83 0 2024-03-04T13:43:10.141 INFO:tasks.workunit.client.1.smithi169.stdout:9/992: sync 2024-03-04T13:43:10.141 INFO:tasks.workunit.client.0.smithi107.stdout:0/499: mkdir da/dad 0 2024-03-04T13:43:10.144 INFO:tasks.workunit.client.1.smithi169.stdout:9/993: chown d7/dc/d12/d20/da6/dd1/l12d 29095 1 2024-03-04T13:43:10.146 INFO:tasks.workunit.client.0.smithi107.stdout:9/411: chown d4/da/d11/d20/d39/l74 1 1 2024-03-04T13:43:10.147 INFO:tasks.workunit.client.1.smithi169.stdout:8/955: rmdir d7/d10/d17/d44/d5d/ddd/d125 0 2024-03-04T13:43:10.147 INFO:tasks.workunit.client.0.smithi107.stdout:3/436: sync 2024-03-04T13:43:10.147 INFO:tasks.workunit.client.0.smithi107.stdout:8/480: sync 2024-03-04T13:43:10.152 INFO:tasks.workunit.client.1.smithi169.stdout:8/956: chown d7/d10/d24/d3e/l99 921237016 1 2024-03-04T13:43:10.153 INFO:tasks.workunit.client.0.smithi107.stdout:5/420: symlink d5/d79/d89/l90 0 2024-03-04T13:43:10.159 INFO:tasks.workunit.client.1.smithi169.stdout:3/908: getdents d7/de/d2b/d7d/d85/d13/d5a/dd8 0 2024-03-04T13:43:10.163 INFO:tasks.workunit.client.0.smithi107.stdout:0/500: truncate da/dc/dd/d3e/d26/f71 89842 0 2024-03-04T13:43:10.164 INFO:tasks.workunit.client.1.smithi169.stdout:8/957: creat d7/dd4/f12f x:0 0 0 2024-03-04T13:43:10.165 INFO:tasks.workunit.client.1.smithi169.stdout:9/994: getdents d7/d25/d64/d10a/d15d 0 2024-03-04T13:43:10.165 INFO:tasks.workunit.client.1.smithi169.stdout:2/950: write d1/d5/d22/f32 [2253276,3372] 0 2024-03-04T13:43:10.167 INFO:tasks.workunit.client.0.smithi107.stdout:3/437: dwrite d1/de/d18/f5f [0,4194304] 0 2024-03-04T13:43:10.167 INFO:tasks.workunit.client.1.smithi169.stdout:3/909: truncate d7/de/d2b/d7d/d85/d18/f30 160074 0 2024-03-04T13:43:10.169 INFO:tasks.workunit.client.0.smithi107.stdout:7/440: dread d6/d13/d16/d1c/f48 [0,4194304] 0 2024-03-04T13:43:10.172 INFO:tasks.workunit.client.1.smithi169.stdout:9/995: mkdir d7/dc/d12/d59/d16d 0 2024-03-04T13:43:10.174 INFO:tasks.workunit.client.0.smithi107.stdout:5/421: dwrite d5/d8/d14/d5a/d6a/f8e [0,4194304] 0 2024-03-04T13:43:10.175 INFO:tasks.workunit.client.1.smithi169.stdout:2/951: dread - d1/d5/d19/d3c/db8/de1/d64/fb1 zero size 2024-03-04T13:43:10.183 INFO:tasks.workunit.client.1.smithi169.stdout:9/996: unlink d7/dc/df1/d160/d125/f13a 0 2024-03-04T13:43:10.184 INFO:tasks.workunit.client.0.smithi107.stdout:4/412: getdents d0/d9/d1e 0 2024-03-04T13:43:10.186 INFO:tasks.workunit.client.1.smithi169.stdout:2/952: mkdir d1/d5/d19/d3c/db8/d140 0 2024-03-04T13:43:10.187 INFO:tasks.workunit.client.0.smithi107.stdout:7/441: dread d6/d1d/d36/f81 [0,4194304] 0 2024-03-04T13:43:10.187 INFO:tasks.workunit.client.0.smithi107.stdout:0/501: unlink da/dc/dd/d1e/c95 0 2024-03-04T13:43:10.187 INFO:tasks.workunit.client.0.smithi107.stdout:7/442: readlink d6/l5e 0 2024-03-04T13:43:10.189 INFO:tasks.workunit.client.0.smithi107.stdout:9/412: getdents d4/da/d11/d20/d23 0 2024-03-04T13:43:10.190 INFO:tasks.workunit.client.1.smithi169.stdout:3/910: truncate d7/de/d2b/d7d/d85/d18/d21/fca 481074 0 2024-03-04T13:43:10.190 INFO:tasks.workunit.client.0.smithi107.stdout:8/481: getdents d6/da/d25 0 2024-03-04T13:43:10.192 INFO:tasks.workunit.client.0.smithi107.stdout:5/422: readlink d5/d8/d14/l1a 0 2024-03-04T13:43:10.192 INFO:tasks.workunit.client.0.smithi107.stdout:9/413: truncate d4/d34/f44 917973 0 2024-03-04T13:43:10.194 INFO:tasks.workunit.client.1.smithi169.stdout:3/911: rmdir d7/de/d2b/d7d/d85 39 2024-03-04T13:43:10.195 INFO:tasks.workunit.client.1.smithi169.stdout:9/997: creat d7/dc/df1/d160/da3/f16e x:0 0 0 2024-03-04T13:43:10.197 INFO:tasks.workunit.client.0.smithi107.stdout:4/413: unlink d0/d9/f37 0 2024-03-04T13:43:10.199 INFO:tasks.workunit.client.0.smithi107.stdout:0/502: unlink da/dc/dd/d3e/d26/d88/fa9 0 2024-03-04T13:43:10.200 INFO:tasks.workunit.client.0.smithi107.stdout:7/443: mknod d6/d1d/d64/d8e/d3d/c94 0 2024-03-04T13:43:10.200 INFO:tasks.workunit.client.1.smithi169.stdout:8/958: dread d7/d10/d24/f7a [0,4194304] 0 2024-03-04T13:43:10.202 INFO:tasks.workunit.client.0.smithi107.stdout:7/444: chown d6/d1d/d64/d8e/l2f 220271 1 2024-03-04T13:43:10.203 INFO:tasks.workunit.client.0.smithi107.stdout:5/423: rmdir d5/d8/da 39 2024-03-04T13:43:10.204 INFO:tasks.workunit.client.0.smithi107.stdout:9/414: dread d4/da/f15 [0,4194304] 0 2024-03-04T13:43:10.206 INFO:tasks.workunit.client.0.smithi107.stdout:3/438: write d1/d1e/d26/f35 [681101,103771] 0 2024-03-04T13:43:10.207 INFO:tasks.workunit.client.0.smithi107.stdout:5/424: fdatasync d5/dc/df/f2e 0 2024-03-04T13:43:10.208 INFO:tasks.workunit.client.0.smithi107.stdout:8/482: dwrite d6/da/d25/d2a/f75 [0,4194304] 0 2024-03-04T13:43:10.208 INFO:tasks.workunit.client.1.smithi169.stdout:3/912: symlink d7/de/d2b/d7d/d85/d13/d1a/d2d/dbe/d108/df7/l129 0 2024-03-04T13:43:10.210 INFO:tasks.workunit.client.0.smithi107.stdout:4/414: write d0/d9/de/d14/dc/f27 [2183177,91898] 0 2024-03-04T13:43:10.211 INFO:tasks.workunit.client.1.smithi169.stdout:2/953: dread d1/d5/d22/d3b/d99/fa6 [0,4194304] 0 2024-03-04T13:43:10.211 INFO:tasks.workunit.client.1.smithi169.stdout:8/959: symlink d7/d10/d24/d3e/l130 0 2024-03-04T13:43:10.212 INFO:tasks.workunit.client.1.smithi169.stdout:3/913: readlink d7/de/d2b/d7d/d85/d13/d5a/l125 0 2024-03-04T13:43:10.217 INFO:tasks.workunit.client.0.smithi107.stdout:7/445: mknod d6/d13/d16/d19/d38/c95 0 2024-03-04T13:43:10.225 INFO:tasks.workunit.client.1.smithi169.stdout:2/954: truncate d1/d5/d22/d45/fb0 216173 0 2024-03-04T13:43:10.226 INFO:tasks.workunit.client.0.smithi107.stdout:0/503: creat da/dad/fae x:0 0 0 2024-03-04T13:43:10.230 INFO:tasks.workunit.client.1.smithi169.stdout:8/960: link d7/d10/d22/f30 d7/d10/d17/d44/d5d/f131 0 2024-03-04T13:43:10.233 INFO:tasks.workunit.client.1.smithi169.stdout:9/998: dwrite d7/dc/d12/d20/d4e/fca [0,4194304] 0 2024-03-04T13:43:10.234 INFO:tasks.workunit.client.0.smithi107.stdout:5/425: dread d5/d8/da/f16 [0,4194304] 0 2024-03-04T13:43:10.235 INFO:tasks.workunit.client.1.smithi169.stdout:7/951: sync 2024-03-04T13:43:10.236 INFO:tasks.workunit.client.0.smithi107.stdout:0/504: symlink da/dc/dd/d3e/d26/laf 0 2024-03-04T13:43:10.238 INFO:tasks.workunit.client.0.smithi107.stdout:9/415: dread d4/da/d11/d20/f28 [0,4194304] 0 2024-03-04T13:43:10.239 INFO:tasks.workunit.client.0.smithi107.stdout:5/426: readlink d5/d8/d14/l1a 0 2024-03-04T13:43:10.239 INFO:tasks.workunit.client.0.smithi107.stdout:8/483: write d6/f5b [1206128,129091] 0 2024-03-04T13:43:10.241 INFO:tasks.workunit.client.0.smithi107.stdout:0/505: write da/dc/dd/d3e/fac [1032452,94262] 0 2024-03-04T13:43:10.242 INFO:tasks.workunit.client.0.smithi107.stdout:3/439: getdents d1/de/d18/d33/d48/d71/d4c 0 2024-03-04T13:43:10.244 INFO:tasks.workunit.client.0.smithi107.stdout:0/506: chown da/dc/dd/f2c 61365 1 2024-03-04T13:43:10.245 INFO:tasks.workunit.client.1.smithi169.stdout:3/914: rename d7/db8/ded/f11b to d7/de/d2b/d7d/d85/d13/f12a 0 2024-03-04T13:43:10.245 INFO:tasks.workunit.client.0.smithi107.stdout:3/440: chown d1/de/d3f/l64 3969 1 2024-03-04T13:43:10.247 INFO:tasks.workunit.client.0.smithi107.stdout:5/427: creat d5/d2b/d4b/f91 x:0 0 0 2024-03-04T13:43:10.247 INFO:tasks.workunit.client.1.smithi169.stdout:9/999: mknod d7/dc/d12/d32/d42/d10b/c16f 0 2024-03-04T13:43:10.247 INFO:tasks.workunit.client.1.smithi169.stdout:7/952: read d2/d15/fce [592112,13766] 0 2024-03-04T13:43:10.250 INFO:tasks.workunit.client.0.smithi107.stdout:3/441: truncate d1/de/d18/d33/d48/d71/f67 900321 0 2024-03-04T13:43:10.253 INFO:tasks.workunit.client.0.smithi107.stdout:0/507: write da/dc/f28 [3650855,26539] 0 2024-03-04T13:43:10.253 INFO:tasks.workunit.client.0.smithi107.stdout:7/446: dread d6/d13/d16/d19/d38/d2c/f4a [0,4194304] 0 2024-03-04T13:43:10.254 INFO:tasks.workunit.client.0.smithi107.stdout:5/428: mkdir d5/d8/d14/d2d/d92 0 2024-03-04T13:43:10.255 INFO:tasks.workunit.client.0.smithi107.stdout:7/447: fdatasync d6/d13/d16/d1c/f35 0 2024-03-04T13:43:10.255 INFO:tasks.workunit.client.1.smithi169.stdout:3/915: mknod d7/de/d2b/d7d/d85/d13/d1a/d78/c12b 0 2024-03-04T13:43:10.256 INFO:tasks.workunit.client.1.smithi169.stdout:8/961: write d7/d10/d17/d44/f119 [1930885,41300] 0 2024-03-04T13:43:10.258 INFO:tasks.workunit.client.1.smithi169.stdout:8/962: write d7/f6a [5239816,2557] 0 2024-03-04T13:43:10.261 INFO:tasks.workunit.client.0.smithi107.stdout:9/416: write d4/da/f4a [934571,24247] 0 2024-03-04T13:43:10.262 INFO:tasks.workunit.client.0.smithi107.stdout:3/442: rmdir d1/d1e/d26 39 2024-03-04T13:43:10.262 INFO:tasks.workunit.client.0.smithi107.stdout:9/417: fsync d4/da/f63 0 2024-03-04T13:43:10.263 INFO:tasks.workunit.client.1.smithi169.stdout:2/955: dwrite d1/d5/d19/d69/fc8 [0,4194304] 0 2024-03-04T13:43:10.267 INFO:tasks.workunit.client.0.smithi107.stdout:5/429: creat d5/d8/d14/d54/f93 x:0 0 0 2024-03-04T13:43:10.268 INFO:tasks.workunit.client.1.smithi169.stdout:3/916: creat d7/de/d2b/d7d/d85/d13/d1a/d2d/d56/d5e/d5f/f12c x:0 0 0 2024-03-04T13:43:10.268 INFO:tasks.workunit.client.0.smithi107.stdout:8/484: link d6/da/l30 d6/d16/d59/d3e/d67/l91 0 2024-03-04T13:43:10.269 INFO:tasks.workunit.client.0.smithi107.stdout:7/448: read d6/d1d/f3b [849504,50634] 0 2024-03-04T13:43:10.272 INFO:tasks.workunit.client.0.smithi107.stdout:8/485: stat d6/d16/d59/d3e/f56 0 2024-03-04T13:43:10.272 INFO:tasks.workunit.client.0.smithi107.stdout:3/443: dread d1/de/d18/d33/d48/d71/d4c/f91 [0,4194304] 0 2024-03-04T13:43:10.276 INFO:tasks.workunit.client.0.smithi107.stdout:3/444: chown d1/de/d4b/f83 0 1 2024-03-04T13:43:10.277 INFO:tasks.workunit.client.1.smithi169.stdout:7/953: creat d2/de/d3e/d41/d5b/d7a/f147 x:0 0 0 2024-03-04T13:43:10.279 INFO:tasks.workunit.client.1.smithi169.stdout:7/954: readlink d2/d15/dad/l13a 0 2024-03-04T13:43:10.280 INFO:tasks.workunit.client.0.smithi107.stdout:3/445: write d1/de/d18/d1f/f43 [664628,51088] 0 2024-03-04T13:43:10.280 INFO:tasks.workunit.client.1.smithi169.stdout:8/963: mknod d7/d10/d22/d75/d81/c132 0 2024-03-04T13:43:10.282 INFO:tasks.workunit.client.1.smithi169.stdout:7/955: readlink d2/de/d3e/d41/d69/d78/l8e 0 2024-03-04T13:43:10.284 INFO:tasks.workunit.client.1.smithi169.stdout:4/922: sync 2024-03-04T13:43:10.284 INFO:tasks.workunit.client.1.smithi169.stdout:6/936: sync 2024-03-04T13:43:10.284 INFO:tasks.workunit.client.1.smithi169.stdout:0/816: sync 2024-03-04T13:43:10.284 INFO:tasks.workunit.client.1.smithi169.stdout:1/862: sync 2024-03-04T13:43:10.285 INFO:tasks.workunit.client.0.smithi107.stdout:9/418: dwrite d4/da/d11/d20/d39/f7a [0,4194304] 0 2024-03-04T13:43:10.285 INFO:tasks.workunit.client.0.smithi107.stdout:7/449: mknod d6/d1d/d36/c96 0 2024-03-04T13:43:10.286 INFO:tasks.workunit.client.1.smithi169.stdout:2/956: getdents d1/d5/d22/d3b/d99 0 2024-03-04T13:43:10.288 INFO:tasks.workunit.client.0.smithi107.stdout:9/419: chown d4/f21 18719 1 2024-03-04T13:43:10.288 INFO:tasks.workunit.client.1.smithi169.stdout:7/956: fsync d2/d15/d17/fbf 0 2024-03-04T13:43:10.289 INFO:tasks.workunit.client.0.smithi107.stdout:0/508: link da/d7e/l5e da/dc/dd/lb0 0 2024-03-04T13:43:10.290 INFO:tasks.workunit.client.0.smithi107.stdout:5/430: dwrite d5/d8/f45 [0,4194304] 0 2024-03-04T13:43:10.293 INFO:tasks.workunit.client.0.smithi107.stdout:9/420: truncate d4/da/d11/d20/d23/d2c/f7c 83236 0 2024-03-04T13:43:10.294 INFO:tasks.workunit.client.0.smithi107.stdout:3/446: mkdir d1/de/d18/d33/d48/d93 0 2024-03-04T13:43:10.307 INFO:tasks.workunit.client.0.smithi107.stdout:8/486: truncate f2 261892 0 2024-03-04T13:43:10.309 INFO:tasks.workunit.client.1.smithi169.stdout:3/917: dwrite d7/de/d2b/d7d/d85/d47/ff3 [0,4194304] 0 2024-03-04T13:43:10.310 INFO:tasks.workunit.client.1.smithi169.stdout:2/957: creat d1/d5/d22/d3b/d99/f141 x:0 0 0 2024-03-04T13:43:10.311 INFO:tasks.workunit.client.0.smithi107.stdout:3/447: creat d1/de/d18/d1f/f94 x:0 0 0 2024-03-04T13:43:10.312 INFO:tasks.workunit.client.0.smithi107.stdout:5/431: write d5/dc/df/f35 [992854,45660] 0 2024-03-04T13:43:10.314 INFO:tasks.workunit.client.1.smithi169.stdout:0/817: dwrite d5/d9/dc/d12/d15/d19/d77/f88 [0,4194304] 0 2024-03-04T13:43:10.317 INFO:tasks.workunit.client.0.smithi107.stdout:9/421: unlink d4/da/d11/d20/d23/d7e/f80 0 2024-03-04T13:43:10.321 INFO:tasks.workunit.client.1.smithi169.stdout:6/937: dwrite d0/d3c/d40/df6/dca/f140 [0,4194304] 0 2024-03-04T13:43:10.321 INFO:tasks.workunit.client.1.smithi169.stdout:2/958: read d1/d5/d19/d3c/db8/f114 [3566576,54408] 0 2024-03-04T13:43:10.330 INFO:tasks.workunit.client.0.smithi107.stdout:7/450: dread d6/d1d/d36/d59/f8a [0,4194304] 0 2024-03-04T13:43:10.331 INFO:tasks.workunit.client.1.smithi169.stdout:3/918: fdatasync d7/de/d2b/d7d/d85/d13/d1a/d2d/fc0 0 2024-03-04T13:43:10.332 INFO:tasks.workunit.client.1.smithi169.stdout:6/938: readlink d0/d3c/ld7 0 2024-03-04T13:43:10.333 INFO:tasks.workunit.client.1.smithi169.stdout:8/964: rename d7/d10/d22/d75/d81/da2/d96/cc7 to d7/d10/d22/d46/c133 0 2024-03-04T13:43:10.334 INFO:tasks.workunit.client.1.smithi169.stdout:7/957: symlink d2/de/d3e/d7e/dbb/l148 0 2024-03-04T13:43:10.335 INFO:tasks.workunit.client.0.smithi107.stdout:5/432: dwrite d5/f49 [0,4194304] 0 2024-03-04T13:43:10.338 INFO:tasks.workunit.client.1.smithi169.stdout:3/919: chown d7/de/d2b/d7d/d85/d18/d21/d3f/d68/fb2 39935 1 2024-03-04T13:43:10.340 INFO:tasks.workunit.client.1.smithi169.stdout:4/923: dread d4/d11/f79 [0,4194304] 0 2024-03-04T13:43:10.340 INFO:tasks.workunit.client.0.smithi107.stdout:7/451: creat d6/d17/f97 x:0 0 0 2024-03-04T13:43:10.341 INFO:tasks.workunit.client.0.smithi107.stdout:0/509: link da/d7e/l5e da/lb1 0 2024-03-04T13:43:10.342 INFO:tasks.workunit.client.1.smithi169.stdout:8/965: rmdir d7/d10/d17/d44/d36/d62 39 2024-03-04T13:43:10.343 INFO:tasks.workunit.client.0.smithi107.stdout:9/422: dwrite d4/ff [0,4194304] 0 2024-03-04T13:43:10.345 INFO:tasks.workunit.client.1.smithi169.stdout:6/939: fdatasync d0/d32/d3a/f23 0 2024-03-04T13:43:10.345 INFO:tasks.workunit.client.1.smithi169.stdout:2/959: creat d1/d5/d22/d45/f142 x:0 0 0 2024-03-04T13:43:10.347 INFO:tasks.workunit.client.0.smithi107.stdout:0/510: fdatasync da/dc/dd/d3e/d26/d69/d6b/f97 0 2024-03-04T13:43:10.348 INFO:tasks.workunit.client.1.smithi169.stdout:8/966: truncate d7/d10/d22/f30 4833485 0 2024-03-04T13:43:10.350 INFO:tasks.workunit.client.0.smithi107.stdout:5/433: mknod d5/d8/d14/d5a/d68/c94 0 2024-03-04T13:43:10.352 INFO:tasks.workunit.client.1.smithi169.stdout:4/924: mknod d4/d11/d2e/dcd/dee/d5c/d86/db8/c119 0 2024-03-04T13:43:10.353 INFO:tasks.workunit.client.0.smithi107.stdout:5/434: write d5/dc/d22/f61 [3220777,85204] 0 2024-03-04T13:43:10.355 INFO:tasks.workunit.client.1.smithi169.stdout:2/960: truncate d1/d5/f9 5146873 0 2024-03-04T13:43:10.357 INFO:tasks.workunit.client.1.smithi169.stdout:6/940: mknod d0/d32/d3a/d43/d74/c14e 0 2024-03-04T13:43:10.357 INFO:tasks.workunit.client.1.smithi169.stdout:8/967: dread d7/d10/d22/d46/f83 [0,4194304] 0 2024-03-04T13:43:10.360 INFO:tasks.workunit.client.0.smithi107.stdout:6/492: sync 2024-03-04T13:43:10.361 INFO:tasks.workunit.client.0.smithi107.stdout:8/487: write d6/d16/f27 [4292330,114570] 0 2024-03-04T13:43:10.361 INFO:tasks.workunit.client.0.smithi107.stdout:5/435: mknod d5/d8/d2a/d43/d5f/c95 0 2024-03-04T13:43:10.363 INFO:tasks.workunit.client.1.smithi169.stdout:0/818: write d5/d9/dc/d12/d66/f78 [736058,75111] 0 2024-03-04T13:43:10.364 INFO:tasks.workunit.client.0.smithi107.stdout:3/448: write d1/de/d18/d33/d48/d71/d4c/d54/f79 [473962,98840] 0 2024-03-04T13:43:10.367 INFO:tasks.workunit.client.0.smithi107.stdout:0/511: rename da/dc/dd/d3e/d26/ca6 to da/dc/dd/cb2 0 2024-03-04T13:43:10.369 INFO:tasks.workunit.client.0.smithi107.stdout:5/436: truncate d5/d2b/f88 948960 0 2024-03-04T13:43:10.370 INFO:tasks.workunit.client.0.smithi107.stdout:8/488: write d6/d16/d1f/d63/d73/f7b [2787085,55161] 0 2024-03-04T13:43:10.371 INFO:tasks.workunit.client.1.smithi169.stdout:1/863: dread d4/d5/d2d/d4e/d71/d26/f2e [0,4194304] 0 2024-03-04T13:43:10.371 INFO:tasks.workunit.client.1.smithi169.stdout:6/941: creat d0/d32/d3e/d122/f14f x:0 0 0 2024-03-04T13:43:10.373 INFO:tasks.workunit.client.1.smithi169.stdout:6/942: readlink d0/lae 0 2024-03-04T13:43:10.373 INFO:tasks.workunit.client.1.smithi169.stdout:1/864: read - d4/d5/d36/f9f zero size 2024-03-04T13:43:10.374 INFO:tasks.workunit.client.0.smithi107.stdout:7/452: dwrite d6/d13/d16/d19/d38/d2c/f4a [4194304,4194304] 0 2024-03-04T13:43:10.375 INFO:tasks.workunit.client.1.smithi169.stdout:0/819: chown d5/d9/d3d/d5d/lf2 26 1 2024-03-04T13:43:10.375 INFO:tasks.workunit.client.0.smithi107.stdout:8/489: write d6/d16/d59/f60 [111971,23749] 0 2024-03-04T13:43:10.379 INFO:tasks.workunit.client.0.smithi107.stdout:7/453: truncate d6/d1d/d36/f5b 1033966 0 2024-03-04T13:43:10.379 INFO:tasks.workunit.client.0.smithi107.stdout:5/437: mkdir d5/d66/d96 0 2024-03-04T13:43:10.381 INFO:tasks.workunit.client.0.smithi107.stdout:6/493: unlink df/d17/d24/d30/d8b/l66 0 2024-03-04T13:43:10.383 INFO:tasks.workunit.client.1.smithi169.stdout:6/943: mknod d0/d32/d3a/d43/d74/d93/d10c/c150 0 2024-03-04T13:43:10.383 INFO:tasks.workunit.client.0.smithi107.stdout:7/454: truncate d6/d1d/d64/d8e/d3d/f78 918185 0 2024-03-04T13:43:10.385 INFO:tasks.workunit.client.1.smithi169.stdout:6/944: fdatasync d0/d3c/d40/f14c 0 2024-03-04T13:43:10.389 INFO:tasks.workunit.client.1.smithi169.stdout:3/920: dread d7/de/d2b/d7d/d85/d13/d5a/fe0 [0,4194304] 0 2024-03-04T13:43:10.389 INFO:tasks.workunit.client.1.smithi169.stdout:4/925: link d4/d1c/d58/d9d/dbc/fd6 d4/d11/d2e/dcd/dee/d5c/d86/d8f/dca/f11a 0 2024-03-04T13:43:10.390 INFO:tasks.workunit.client.0.smithi107.stdout:0/512: mkdir da/dc/dd/d3e/d26/d69/d6b/db3 0 2024-03-04T13:43:10.391 INFO:tasks.workunit.client.0.smithi107.stdout:8/490: creat d6/d16/d59/d7c/f92 x:0 0 0 2024-03-04T13:43:10.391 INFO:tasks.workunit.client.1.smithi169.stdout:1/865: creat d4/d5/d36/d35/d16/d3a/f119 x:0 0 0 2024-03-04T13:43:10.392 INFO:tasks.workunit.client.1.smithi169.stdout:6/945: creat d0/d32/d3a/d43/d74/d93/f151 x:0 0 0 2024-03-04T13:43:10.394 INFO:tasks.workunit.client.0.smithi107.stdout:5/438: creat d5/d8/d14/d5a/d68/f97 x:0 0 0 2024-03-04T13:43:10.398 INFO:tasks.workunit.client.1.smithi169.stdout:3/921: creat d7/de/d2b/d7d/d85/d13/d1a/d2d/dbe/d108/df7/f12d x:0 0 0 2024-03-04T13:43:10.398 INFO:tasks.workunit.client.1.smithi169.stdout:6/946: chown d0/d3c/d40/cdd 1321 1 2024-03-04T13:43:10.399 INFO:tasks.workunit.client.0.smithi107.stdout:0/513: symlink da/dc/dd/d3e/lb4 0 2024-03-04T13:43:10.399 INFO:tasks.workunit.client.1.smithi169.stdout:6/947: stat d0/d3c/d95/dbd 0 2024-03-04T13:43:10.402 INFO:tasks.workunit.client.1.smithi169.stdout:1/866: mkdir d4/d5/d2d/d4e/da5/d11a 0 2024-03-04T13:43:10.404 INFO:tasks.workunit.client.0.smithi107.stdout:8/491: mkdir d6/d16/d1f/d63/d93 0 2024-03-04T13:43:10.405 INFO:tasks.workunit.client.0.smithi107.stdout:8/492: fdatasync d6/d16/d59/d3e/f56 0 2024-03-04T13:43:10.408 INFO:tasks.workunit.client.1.smithi169.stdout:3/922: fdatasync d7/de/d24/f89 0 2024-03-04T13:43:10.410 INFO:tasks.workunit.client.0.smithi107.stdout:5/439: readlink d5/d8/d14/d54/l8c 0 2024-03-04T13:43:10.411 INFO:tasks.workunit.client.1.smithi169.stdout:6/948: symlink d0/d3c/d10b/l152 0 2024-03-04T13:43:10.412 INFO:tasks.workunit.client.0.smithi107.stdout:0/514: readlink da/dc/dd/d3e/d26/l2b 0 2024-03-04T13:43:10.412 INFO:tasks.workunit.client.0.smithi107.stdout:0/515: stat da/c4c 0 2024-03-04T13:43:10.415 INFO:tasks.workunit.client.0.smithi107.stdout:1/447: sync 2024-03-04T13:43:10.416 INFO:tasks.workunit.client.1.smithi169.stdout:4/926: creat d4/d11/d2e/dcd/dee/d73/d101/f11b x:0 0 0 2024-03-04T13:43:10.419 INFO:tasks.workunit.client.0.smithi107.stdout:1/448: read d1/d46/d6a/df/d33/d5d/d24/f62 [763106,63388] 0 2024-03-04T13:43:10.420 INFO:tasks.workunit.client.1.smithi169.stdout:7/958: write d2/de/f2a [4413979,100288] 0 2024-03-04T13:43:10.422 INFO:tasks.workunit.client.0.smithi107.stdout:0/516: creat da/dc/dd/d1e/fb5 x:0 0 0 2024-03-04T13:43:10.424 INFO:tasks.workunit.client.1.smithi169.stdout:6/949: creat d0/d3c/d95/f153 x:0 0 0 2024-03-04T13:43:10.427 INFO:tasks.workunit.client.0.smithi107.stdout:9/423: dwrite d4/da/d11/d65/f3e [0,4194304] 0 2024-03-04T13:43:10.430 INFO:tasks.workunit.client.1.smithi169.stdout:2/961: truncate d1/d5/d22/d45/f8f 2811193 0 2024-03-04T13:43:10.440 INFO:tasks.workunit.client.1.smithi169.stdout:1/867: rmdir d4/d5/d36/d35/df/da8 0 2024-03-04T13:43:10.442 INFO:tasks.workunit.client.0.smithi107.stdout:5/440: dread d5/d8/d14/f4c [0,4194304] 0 2024-03-04T13:43:10.443 INFO:tasks.workunit.client.0.smithi107.stdout:8/493: link d6/d16/d59/l49 d6/d16/d59/d3e/l94 0 2024-03-04T13:43:10.444 INFO:tasks.workunit.client.0.smithi107.stdout:5/441: write d5/d8/d14/d2d/f6b [2621498,63839] 0 2024-03-04T13:43:10.445 INFO:tasks.workunit.client.0.smithi107.stdout:8/494: stat d6/d16/d1f/d4c 0 2024-03-04T13:43:10.446 INFO:tasks.workunit.client.1.smithi169.stdout:6/950: dwrite d0/d32/d3a/def/f14b [0,4194304] 0 2024-03-04T13:43:10.446 INFO:tasks.workunit.client.0.smithi107.stdout:9/424: mknod d4/da/d11/d4c/d4d/c8c 0 2024-03-04T13:43:10.447 INFO:tasks.workunit.client.0.smithi107.stdout:7/455: write d6/d1d/d64/d8e/f53 [880548,118345] 0 2024-03-04T13:43:10.448 INFO:tasks.workunit.client.0.smithi107.stdout:9/425: chown d4/f8 1908 1 2024-03-04T13:43:10.454 INFO:tasks.workunit.client.1.smithi169.stdout:0/820: dwrite d5/d9/d3d/d5d/de4/fb4 [0,4194304] 0 2024-03-04T13:43:10.456 INFO:tasks.workunit.client.1.smithi169.stdout:1/868: dread - d4/d5/d2d/d4e/d71/d82/fb2 zero size 2024-03-04T13:43:10.457 INFO:tasks.workunit.client.0.smithi107.stdout:8/495: creat d6/da/d25/d2a/f95 x:0 0 0 2024-03-04T13:43:10.458 INFO:tasks.workunit.client.0.smithi107.stdout:6/494: dwrite df/d17/f28 [0,4194304] 0 2024-03-04T13:43:10.460 INFO:tasks.workunit.client.1.smithi169.stdout:2/962: link d1/d5/d19/d68/l9f d1/d5/d22/d45/d4a/d11f/l143 0 2024-03-04T13:43:10.464 INFO:tasks.workunit.client.0.smithi107.stdout:9/426: dwrite d4/da/d11/f13 [0,4194304] 0 2024-03-04T13:43:10.465 INFO:tasks.workunit.client.1.smithi169.stdout:1/869: readlink d4/d5/d2d/d4e/d71/l62 0 2024-03-04T13:43:10.467 INFO:tasks.workunit.client.1.smithi169.stdout:6/951: mknod d0/d32/d3a/d43/d74/d93/d10c/c154 0 2024-03-04T13:43:10.467 INFO:tasks.workunit.client.1.smithi169.stdout:4/927: dread d4/f7 [0,4194304] 0 2024-03-04T13:43:10.469 INFO:tasks.workunit.client.0.smithi107.stdout:1/449: creat d1/f94 x:0 0 0 2024-03-04T13:43:10.473 INFO:tasks.workunit.client.1.smithi169.stdout:7/959: write d2/de/d37/fd0 [3174198,26543] 0 2024-03-04T13:43:10.480 INFO:tasks.workunit.client.1.smithi169.stdout:6/952: rename d0/cc0 to d0/d3c/d95/d142/c155 0 2024-03-04T13:43:10.480 INFO:tasks.workunit.client.0.smithi107.stdout:6/495: rename df/d17/d6b/d9b/d46 to df/d17/d24/d40/d63/da3 0 2024-03-04T13:43:10.481 INFO:tasks.workunit.client.0.smithi107.stdout:7/456: getdents d6/d1d/d64/d89 0 2024-03-04T13:43:10.484 INFO:tasks.workunit.client.0.smithi107.stdout:0/517: getdents da/dc/dd 0 2024-03-04T13:43:10.485 INFO:tasks.workunit.client.0.smithi107.stdout:7/457: dread d6/d1d/d36/f5b [0,4194304] 0 2024-03-04T13:43:10.488 INFO:tasks.workunit.client.1.smithi169.stdout:1/870: rename d4/dd8/fe1 to d4/d5/d36/d35/d16/d81/db4/f11b 0 2024-03-04T13:43:10.488 INFO:tasks.workunit.client.0.smithi107.stdout:9/427: creat d4/da/d11/d4c/d89/f8d x:0 0 0 2024-03-04T13:43:10.490 INFO:tasks.workunit.client.0.smithi107.stdout:6/496: unlink df/d17/d24/d30/d8b/f48 0 2024-03-04T13:43:10.491 INFO:tasks.workunit.client.1.smithi169.stdout:3/923: dwrite d7/f1d [4194304,4194304] 0 2024-03-04T13:43:10.498 INFO:tasks.workunit.client.0.smithi107.stdout:0/518: dwrite da/f3c [4194304,4194304] 0 2024-03-04T13:43:10.499 INFO:tasks.workunit.client.1.smithi169.stdout:0/821: dwrite d5/d9/dc/d12/d15/d19/f2e [0,4194304] 0 2024-03-04T13:43:10.500 INFO:tasks.workunit.client.0.smithi107.stdout:4/415: sync 2024-03-04T13:43:10.501 INFO:tasks.workunit.client.0.smithi107.stdout:2/454: sync 2024-03-04T13:43:10.503 INFO:tasks.workunit.client.0.smithi107.stdout:9/428: dread d4/da/d11/d20/d64/f78 [0,4194304] 0 2024-03-04T13:43:10.505 INFO:tasks.workunit.client.1.smithi169.stdout:4/928: getdents d4/d11/d2e/d39/d61 0 2024-03-04T13:43:10.510 INFO:tasks.workunit.client.0.smithi107.stdout:0/519: dread da/f62 [0,4194304] 0 2024-03-04T13:43:10.511 INFO:tasks.workunit.client.0.smithi107.stdout:6/497: rename df/d17/l88 to df/d38/la4 0 2024-03-04T13:43:10.516 INFO:tasks.workunit.client.1.smithi169.stdout:7/960: link d2/d28/d68/lfd d2/l149 0 2024-03-04T13:43:10.516 INFO:tasks.workunit.client.1.smithi169.stdout:1/871: dread d4/d5/d36/d35/d16/d81/db4/f108 [0,4194304] 0 2024-03-04T13:43:10.519 INFO:tasks.workunit.client.0.smithi107.stdout:5/442: dread d5/d8/d14/d2d/f6b [0,4194304] 0 2024-03-04T13:43:10.519 INFO:tasks.workunit.client.0.smithi107.stdout:4/416: dwrite d0/d9/de/d14/d66/f6a [0,4194304] 0 2024-03-04T13:43:10.520 INFO:tasks.workunit.client.0.smithi107.stdout:5/443: fsync d5/dc/f3d 0 2024-03-04T13:43:10.520 INFO:tasks.workunit.client.0.smithi107.stdout:9/429: unlink d4/da/d11/f17 0 2024-03-04T13:43:10.521 INFO:tasks.workunit.client.1.smithi169.stdout:1/872: write d4/d5/d2d/f6c [265181,13627] 0 2024-03-04T13:43:10.521 INFO:tasks.workunit.client.1.smithi169.stdout:7/961: chown d2/de/fcb 132 1 2024-03-04T13:43:10.522 INFO:tasks.workunit.client.1.smithi169.stdout:0/822: rename d5/d4c/d65/d95/db0/ce2 to d5/d9/dc/d12/d15/d19/d61/d71/d8c/c11d 0 2024-03-04T13:43:10.523 INFO:tasks.workunit.client.0.smithi107.stdout:8/496: write d6/d16/d1f/f23 [324813,70478] 0 2024-03-04T13:43:10.525 INFO:tasks.workunit.client.0.smithi107.stdout:0/520: mknod da/dc/dd/d3e/d6e/cb6 0 2024-03-04T13:43:10.527 INFO:tasks.workunit.client.0.smithi107.stdout:9/430: read d4/da/d11/f13 [1699386,4516] 0 2024-03-04T13:43:10.528 INFO:tasks.workunit.client.1.smithi169.stdout:3/924: mkdir d7/de/d2b/d7d/d85/d18/d21/d12e 0 2024-03-04T13:43:10.530 INFO:tasks.workunit.client.0.smithi107.stdout:5/444: chown d5/f49 3 1 2024-03-04T13:43:10.532 INFO:tasks.workunit.client.1.smithi169.stdout:2/963: dwrite d1/d5/d22/d45/f90 [0,4194304] 0 2024-03-04T13:43:10.535 INFO:tasks.workunit.client.0.smithi107.stdout:1/450: truncate d1/d46/d6a/df/d33/d5d/d4c/f93 361544 0 2024-03-04T13:43:10.535 INFO:tasks.workunit.client.0.smithi107.stdout:8/497: read d6/d16/d59/f60 [64554,57900] 0 2024-03-04T13:43:10.538 INFO:tasks.workunit.client.0.smithi107.stdout:4/417: fdatasync d0/d30/f64 0 2024-03-04T13:43:10.539 INFO:tasks.workunit.client.0.smithi107.stdout:1/451: fsync d1/d46/d6a/df/d67/d6b/f6e 0 2024-03-04T13:43:10.541 INFO:tasks.workunit.client.0.smithi107.stdout:6/498: dwrite df/d2e/d36/fa2 [0,4194304] 0 2024-03-04T13:43:10.542 INFO:tasks.workunit.client.1.smithi169.stdout:0/823: unlink d5/d9/dc/d12/d15/d19/d61/d71/cc7 0 2024-03-04T13:43:10.545 INFO:tasks.workunit.client.1.smithi169.stdout:3/925: fsync d7/de/d2b/d7d/d85/d13/d1a/d2d/dbe/f101 0 2024-03-04T13:43:10.545 INFO:tasks.workunit.client.0.smithi107.stdout:1/452: stat d1/d46/d6a/df/d33 0 2024-03-04T13:43:10.551 INFO:tasks.workunit.client.0.smithi107.stdout:1/453: write d1/d46/d6a/df/d67/d6b/f6e [1658979,20731] 0 2024-03-04T13:43:10.552 INFO:tasks.workunit.client.0.smithi107.stdout:5/445: dwrite d5/dc/f3b [0,4194304] 0 2024-03-04T13:43:10.554 INFO:tasks.workunit.client.0.smithi107.stdout:7/458: link d6/d13/d16/d19/d38/d2c/l6c d6/d1d/l98 0 2024-03-04T13:43:10.555 INFO:tasks.workunit.client.1.smithi169.stdout:2/964: rmdir d1/d5/d22/d45/d4a 39 2024-03-04T13:43:10.559 INFO:tasks.workunit.client.0.smithi107.stdout:0/521: symlink da/dc/dd/d3e/d6e/d73/d78/d86/lb7 0 2024-03-04T13:43:10.560 INFO:tasks.workunit.client.1.smithi169.stdout:7/962: truncate d2/de/d3e/d41/f71 1930995 0 2024-03-04T13:43:10.561 INFO:tasks.workunit.client.0.smithi107.stdout:4/418: dwrite d0/d9/d10/f1c [0,4194304] 0 2024-03-04T13:43:10.562 INFO:tasks.workunit.client.0.smithi107.stdout:9/431: symlink d4/da/d11/d20/d64/d77/l8e 0 2024-03-04T13:43:10.569 INFO:tasks.workunit.client.0.smithi107.stdout:8/498: mkdir d6/d16/d59/d3e/d96 0 2024-03-04T13:43:10.569 INFO:tasks.workunit.client.1.smithi169.stdout:0/824: chown d5/d9/dc/d12/l84 19745 1 2024-03-04T13:43:10.574 INFO:tasks.workunit.client.0.smithi107.stdout:2/455: dwrite da/f26 [0,4194304] 0 2024-03-04T13:43:10.575 INFO:tasks.workunit.client.1.smithi169.stdout:0/825: write d5/d9/dc/d12/d15/d19/f2e [9364970,9560] 0 2024-03-04T13:43:10.576 INFO:tasks.workunit.client.1.smithi169.stdout:4/929: dread d4/d11/d3b/d67/dbb/d6e/fc5 [0,4194304] 0 2024-03-04T13:43:10.579 INFO:tasks.workunit.client.0.smithi107.stdout:2/456: write da/df/d27/f5e [3463979,3119] 0 2024-03-04T13:43:10.580 INFO:tasks.workunit.client.1.smithi169.stdout:7/963: truncate d2/de/d3e/d41/f49 505578 0 2024-03-04T13:43:10.581 INFO:tasks.workunit.client.1.smithi169.stdout:3/926: getdents d7/de/d2b/d7d/d85/d13/d5a/dd8 0 2024-03-04T13:43:10.582 INFO:tasks.workunit.client.1.smithi169.stdout:3/927: fdatasync d7/d53/f10a 0 2024-03-04T13:43:10.582 INFO:tasks.workunit.client.0.smithi107.stdout:3/449: sync 2024-03-04T13:43:10.590 INFO:tasks.workunit.client.1.smithi169.stdout:0/826: creat d5/d9/d3d/d5d/de4/f11e x:0 0 0 2024-03-04T13:43:10.591 INFO:tasks.workunit.client.0.smithi107.stdout:5/446: mknod d5/d2b/c98 0 2024-03-04T13:43:10.591 INFO:tasks.workunit.client.1.smithi169.stdout:4/930: mkdir d4/d1c/d58/d9d/dbc/d11c 0 2024-03-04T13:43:10.593 INFO:tasks.workunit.client.0.smithi107.stdout:7/459: creat d6/d13/d16/d19/d38/f99 x:0 0 0 2024-03-04T13:43:10.594 INFO:tasks.workunit.client.1.smithi169.stdout:1/873: write d4/d5/d36/d35/df/f28 [4960797,47921] 0 2024-03-04T13:43:10.596 INFO:tasks.workunit.client.0.smithi107.stdout:0/522: fsync da/f81 0 2024-03-04T13:43:10.597 INFO:tasks.workunit.client.0.smithi107.stdout:5/447: read d5/d8/d14/d2d/f56 [1782095,128970] 0 2024-03-04T13:43:10.597 INFO:tasks.workunit.client.1.smithi169.stdout:3/928: unlink f6 0 2024-03-04T13:43:10.599 INFO:tasks.workunit.client.0.smithi107.stdout:4/419: creat d0/d9/de/d14/dc/f84 x:0 0 0 2024-03-04T13:43:10.601 INFO:tasks.workunit.client.0.smithi107.stdout:9/432: mknod d4/da/d11/d20/d2f/c8f 0 2024-03-04T13:43:10.602 INFO:tasks.workunit.client.1.smithi169.stdout:0/827: dread d5/d9/dc/d12/d15/d19/d77/fcf [0,4194304] 0 2024-03-04T13:43:10.604 INFO:tasks.workunit.client.0.smithi107.stdout:8/499: mknod d6/d16/d59/d7c/c97 0 2024-03-04T13:43:10.605 INFO:tasks.workunit.client.0.smithi107.stdout:2/457: creat da/d49/d5c/f9e x:0 0 0 2024-03-04T13:43:10.605 INFO:tasks.workunit.client.1.smithi169.stdout:1/874: mkdir d4/d5/d2d/d4e/d71/d26/d7e/d11c 0 2024-03-04T13:43:10.607 INFO:tasks.workunit.client.0.smithi107.stdout:6/499: write df/d17/f23 [3094278,110586] 0 2024-03-04T13:43:10.608 INFO:tasks.workunit.client.0.smithi107.stdout:3/450: creat d1/de/d11/f95 x:0 0 0 2024-03-04T13:43:10.613 INFO:tasks.workunit.client.0.smithi107.stdout:7/460: rmdir d6/d13/d16/d19/d38/d62 39 2024-03-04T13:43:10.619 INFO:tasks.workunit.client.0.smithi107.stdout:8/500: dwrite d6/fe [0,4194304] 0 2024-03-04T13:43:10.619 INFO:tasks.workunit.client.0.smithi107.stdout:0/523: fdatasync da/d7e/f89 0 2024-03-04T13:43:10.621 INFO:tasks.workunit.client.1.smithi169.stdout:1/875: dwrite d4/d5/d2d/d4e/d71/ffb [0,4194304] 0 2024-03-04T13:43:10.621 INFO:tasks.workunit.client.0.smithi107.stdout:0/524: chown da/d7e/f7f 1 1 2024-03-04T13:43:10.621 INFO:tasks.workunit.client.1.smithi169.stdout:0/828: unlink d5/d9/dc/d12/c35 0 2024-03-04T13:43:10.622 INFO:tasks.workunit.client.1.smithi169.stdout:2/965: write d1/d5/d19/d69/f111 [150179,129030] 0 2024-03-04T13:43:10.622 INFO:tasks.workunit.client.0.smithi107.stdout:5/448: creat d5/d8/d14/d5a/d6a/f99 x:0 0 0 2024-03-04T13:43:10.627 INFO:tasks.workunit.client.1.smithi169.stdout:8/968: sync 2024-03-04T13:43:10.627 INFO:tasks.workunit.client.0.smithi107.stdout:4/420: creat d0/d9/d78/f85 x:0 0 0 2024-03-04T13:43:10.629 INFO:tasks.workunit.client.1.smithi169.stdout:0/829: truncate d5/d9/dc/d12/d15/d19/d61/f11b 281533 0 2024-03-04T13:43:10.630 INFO:tasks.workunit.client.0.smithi107.stdout:9/433: creat d4/da/d11/d20/d39/f90 x:0 0 0 2024-03-04T13:43:10.632 INFO:tasks.workunit.client.1.smithi169.stdout:1/876: fsync d4/d5/d36/d35/d5b/ff0 0 2024-03-04T13:43:10.636 INFO:tasks.workunit.client.1.smithi169.stdout:7/964: write d2/de/d3e/d41/dec/ff4 [1047293,75768] 0 2024-03-04T13:43:10.637 INFO:tasks.workunit.client.1.smithi169.stdout:8/969: mknod d7/d10/d22/d75/d81/da2/dac/c134 0 2024-03-04T13:43:10.639 INFO:tasks.workunit.client.1.smithi169.stdout:1/877: chown d4/d5/d2d/d4e/d71/d26/dfa/cfd 92208488 1 2024-03-04T13:43:10.643 INFO:tasks.workunit.client.1.smithi169.stdout:0/830: mknod d5/d9/dc/d12/d15/d19/d61/d71/dbb/c11f 0 2024-03-04T13:43:10.649 INFO:tasks.workunit.client.1.smithi169.stdout:0/831: truncate d5/d9/d3d/d5d/de4/d98/dc1/fe6 4894467 0 2024-03-04T13:43:10.651 INFO:tasks.workunit.client.0.smithi107.stdout:4/421: fsync d0/d9/de/f43 0 2024-03-04T13:43:10.654 INFO:tasks.workunit.client.1.smithi169.stdout:1/878: mknod d4/d5/d2d/d4e/c11d 0 2024-03-04T13:43:10.654 INFO:tasks.workunit.client.0.smithi107.stdout:4/422: write d0/d30/f50 [1512818,78208] 0 2024-03-04T13:43:10.656 INFO:tasks.workunit.client.1.smithi169.stdout:3/929: write d7/de/d2b/d7d/d85/f4f [819572,96651] 0 2024-03-04T13:43:10.657 INFO:tasks.workunit.client.0.smithi107.stdout:9/434: mknod d4/da/d11/d4c/d4d/d72/d7b/c91 0 2024-03-04T13:43:10.661 INFO:tasks.workunit.client.0.smithi107.stdout:9/435: chown d4/da/d11/d4c/d4d/c67 23586342 1 2024-03-04T13:43:10.662 INFO:tasks.workunit.client.0.smithi107.stdout:1/454: getdents d1 0 2024-03-04T13:43:10.663 INFO:tasks.workunit.client.1.smithi169.stdout:4/931: dwrite d4/d11/d2e/d39/d48/f3c [0,4194304] 0 2024-03-04T13:43:10.664 INFO:tasks.workunit.client.1.smithi169.stdout:4/932: fsync d4/f5a 0 2024-03-04T13:43:10.671 INFO:tasks.workunit.client.0.smithi107.stdout:6/500: write f6 [1188046,116349] 0 2024-03-04T13:43:10.672 INFO:tasks.workunit.client.1.smithi169.stdout:0/832: mknod d5/d9/dc/d12/da5/d115/c120 0 2024-03-04T13:43:10.672 INFO:tasks.workunit.client.0.smithi107.stdout:5/449: dwrite d5/dc/d22/f7f [0,4194304] 0 2024-03-04T13:43:10.674 INFO:tasks.workunit.client.1.smithi169.stdout:0/833: dread - d5/d9/dc/f118 zero size 2024-03-04T13:43:10.676 INFO:tasks.workunit.client.1.smithi169.stdout:6/953: sync 2024-03-04T13:43:10.681 INFO:tasks.workunit.client.1.smithi169.stdout:4/933: mkdir d4/d11/d2e/d6c/d78/da5/d11d 0 2024-03-04T13:43:10.681 INFO:tasks.workunit.client.1.smithi169.stdout:0/834: read d5/d9/dc/d12/d15/d19/d77/fcf [449415,15663] 0 2024-03-04T13:43:10.681 INFO:tasks.workunit.client.0.smithi107.stdout:5/450: fdatasync d5/f24 0 2024-03-04T13:43:10.681 INFO:tasks.workunit.client.1.smithi169.stdout:7/965: dread d2/d28/f33 [0,4194304] 0 2024-03-04T13:43:10.683 INFO:tasks.workunit.client.0.smithi107.stdout:2/458: dread da/d34/f36 [0,4194304] 0 2024-03-04T13:43:10.683 INFO:tasks.workunit.client.1.smithi169.stdout:3/930: dwrite d7/de/d2b/d7d/d85/d18/d21/d103/f120 [0,4194304] 0 2024-03-04T13:43:10.684 INFO:tasks.workunit.client.0.smithi107.stdout:7/461: truncate d6/d13/d14/f86 1335227 0 2024-03-04T13:43:10.684 INFO:tasks.workunit.client.1.smithi169.stdout:1/879: write d4/d1d/f40 [4872842,90763] 0 2024-03-04T13:43:10.686 INFO:tasks.workunit.client.0.smithi107.stdout:4/423: mknod d0/d9/de/d14/d66/c86 0 2024-03-04T13:43:10.687 INFO:tasks.workunit.client.0.smithi107.stdout:7/462: dread - d6/d13/d16/d19/d60/f84 zero size 2024-03-04T13:43:10.689 INFO:tasks.workunit.client.0.smithi107.stdout:7/463: chown d6/d13/d16/d19/d39/d7a 200 1 2024-03-04T13:43:10.690 INFO:tasks.workunit.client.0.smithi107.stdout:6/501: dwrite df/d17/f23 [0,4194304] 0 2024-03-04T13:43:10.691 INFO:tasks.workunit.client.0.smithi107.stdout:4/424: read d0/d9/d10/f4a [464948,39170] 0 2024-03-04T13:43:10.692 INFO:tasks.workunit.client.0.smithi107.stdout:7/464: chown d6/d1d/d64/d8e/c51 1 1 2024-03-04T13:43:10.692 INFO:tasks.workunit.client.1.smithi169.stdout:2/966: dread d1/d5/d19/d1e/d29/f3e [0,4194304] 0 2024-03-04T13:43:10.692 INFO:tasks.workunit.client.1.smithi169.stdout:0/835: write d5/d9/dc/d12/d15/d19/d61/f23 [1560842,278] 0 2024-03-04T13:43:10.693 INFO:tasks.workunit.client.0.smithi107.stdout:3/451: rename d1/de/d4b/f83 to d1/de/d18/d33/f96 0 2024-03-04T13:43:10.696 INFO:tasks.workunit.client.0.smithi107.stdout:1/455: rmdir d1/d46/d6a/df/d33/d5d/d4c 39 2024-03-04T13:43:10.698 INFO:tasks.workunit.client.1.smithi169.stdout:7/966: dread d2/d28/d70/ded/f107 [0,4194304] 0 2024-03-04T13:43:10.699 INFO:tasks.workunit.client.1.smithi169.stdout:8/970: dwrite d7/d10/d22/d46/fc0 [4194304,4194304] 0 2024-03-04T13:43:10.702 INFO:tasks.workunit.client.0.smithi107.stdout:6/502: dread f3 [0,4194304] 0 2024-03-04T13:43:10.702 INFO:tasks.workunit.client.1.smithi169.stdout:0/836: read d5/d4c/d65/f8e [413306,130822] 0 2024-03-04T13:43:10.702 INFO:tasks.workunit.client.1.smithi169.stdout:8/971: stat d7/d10 0 2024-03-04T13:43:10.704 INFO:tasks.workunit.client.1.smithi169.stdout:2/967: creat d1/f144 x:0 0 0 2024-03-04T13:43:10.705 INFO:tasks.workunit.client.0.smithi107.stdout:5/451: symlink d5/d8/d14/d5a/d6a/l9a 0 2024-03-04T13:43:10.707 INFO:tasks.workunit.client.1.smithi169.stdout:4/934: symlink d4/d11/d3b/d67/dbb/l11e 0 2024-03-04T13:43:10.708 INFO:tasks.workunit.client.0.smithi107.stdout:2/459: creat da/d38/f9f x:0 0 0 2024-03-04T13:43:10.708 INFO:tasks.workunit.client.0.smithi107.stdout:4/425: dread d0/d9/de/d14/d66/f6a [0,4194304] 0 2024-03-04T13:43:10.709 INFO:tasks.workunit.client.0.smithi107.stdout:4/426: fsync d0/d9/d10/f25 0 2024-03-04T13:43:10.712 INFO:tasks.workunit.client.1.smithi169.stdout:4/935: write d4/d11/d2e/d39/d61/d6b/f10d [496243,70017] 0 2024-03-04T13:43:10.714 INFO:tasks.workunit.client.0.smithi107.stdout:9/436: mkdir d4/da/d11/d68/d92 0 2024-03-04T13:43:10.715 INFO:tasks.workunit.client.1.smithi169.stdout:3/931: mknod d7/de/d2b/d7d/d85/d13/d1a/d98/d9a/d109/c12f 0 2024-03-04T13:43:10.718 INFO:tasks.workunit.client.1.smithi169.stdout:1/880: creat d4/d5/d2d/d4e/d114/f11e x:0 0 0 2024-03-04T13:43:10.722 INFO:tasks.workunit.client.1.smithi169.stdout:1/881: fsync d4/d5/d36/d35/d5b/fbe 0 2024-03-04T13:43:10.722 INFO:tasks.workunit.client.0.smithi107.stdout:7/465: mkdir d6/d1d/d64/d9a 0 2024-03-04T13:43:10.723 INFO:tasks.workunit.client.0.smithi107.stdout:5/452: dwrite d5/dc/f3b [0,4194304] 0 2024-03-04T13:43:10.727 INFO:tasks.workunit.client.0.smithi107.stdout:2/460: dwrite da/d49/d5c/d6a/f7f [0,4194304] 0 2024-03-04T13:43:10.728 INFO:tasks.workunit.client.1.smithi169.stdout:6/954: dread d0/d32/d3e/fb7 [0,4194304] 0 2024-03-04T13:43:10.730 INFO:tasks.workunit.client.1.smithi169.stdout:4/936: dwrite d4/fa [0,4194304] 0 2024-03-04T13:43:10.734 INFO:tasks.workunit.client.1.smithi169.stdout:8/972: unlink d7/d10/d24/f3c 0 2024-03-04T13:43:10.736 INFO:tasks.workunit.client.0.smithi107.stdout:8/501: rename d6/f57 to d6/da/d25/d5f/f98 0 2024-03-04T13:43:10.741 INFO:tasks.workunit.client.0.smithi107.stdout:3/452: dread d1/de/d18/d33/d48/d71/f49 [0,4194304] 0 2024-03-04T13:43:10.745 INFO:tasks.workunit.client.0.smithi107.stdout:0/525: link da/dc/d34/c9e da/dc/d34/cb8 0 2024-03-04T13:43:10.752 INFO:tasks.workunit.client.1.smithi169.stdout:3/932: creat d7/de/d2b/d7d/d85/f130 x:0 0 0 2024-03-04T13:43:10.754 INFO:tasks.workunit.client.1.smithi169.stdout:0/837: mknod d5/d9/dc/d12/c121 0 2024-03-04T13:43:10.755 INFO:tasks.workunit.client.0.smithi107.stdout:4/427: symlink d0/d2f/l87 0 2024-03-04T13:43:10.760 INFO:tasks.workunit.client.1.smithi169.stdout:6/955: unlink d0/d3c/d40/f5c 0 2024-03-04T13:43:10.763 INFO:tasks.workunit.client.1.smithi169.stdout:8/973: unlink d7/d120/l126 0 2024-03-04T13:43:10.764 INFO:tasks.workunit.client.1.smithi169.stdout:4/937: dread d4/d11/d2e/d6c/d78/da5/fce [0,4194304] 0 2024-03-04T13:43:10.764 INFO:tasks.workunit.client.0.smithi107.stdout:2/461: fdatasync f7 0 2024-03-04T13:43:10.770 INFO:tasks.workunit.client.1.smithi169.stdout:2/968: getdents d1/d5/d12e 0 2024-03-04T13:43:10.772 INFO:tasks.workunit.client.1.smithi169.stdout:3/933: mkdir d7/de/d2b/d7d/d85/d18/d21/d66/d131 0 2024-03-04T13:43:10.773 INFO:tasks.workunit.client.0.smithi107.stdout:9/437: write d4/d34/f3f [1299667,34305] 0 2024-03-04T13:43:10.773 INFO:tasks.workunit.client.1.smithi169.stdout:0/838: symlink d5/d9/dc/d12/d15/d19/d77/db2/l122 0 2024-03-04T13:43:10.778 INFO:tasks.workunit.client.0.smithi107.stdout:3/453: symlink d1/d29/l97 0 2024-03-04T13:43:10.784 INFO:tasks.workunit.client.1.smithi169.stdout:1/882: dwrite d4/d5/d2d/d4e/d71/d26/f79 [0,4194304] 0 2024-03-04T13:43:10.784 INFO:tasks.workunit.client.1.smithi169.stdout:4/938: fdatasync d4/d1c/d75/fb3 0 2024-03-04T13:43:10.785 INFO:tasks.workunit.client.1.smithi169.stdout:6/956: link d0/f125 d0/d32/d3e/d122/f156 0 2024-03-04T13:43:10.786 INFO:tasks.workunit.client.0.smithi107.stdout:6/503: creat df/d17/d24/d40/fa5 x:0 0 0 2024-03-04T13:43:10.791 INFO:tasks.workunit.client.0.smithi107.stdout:7/466: creat d6/d13/d16/d19/d38/d62/f9b x:0 0 0 2024-03-04T13:43:10.792 INFO:tasks.workunit.client.1.smithi169.stdout:1/883: chown d4/d5/d2d/d4e/d71/c8a 0 1 2024-03-04T13:43:10.792 INFO:tasks.workunit.client.1.smithi169.stdout:8/974: dwrite d7/d10/ff9 [0,4194304] 0 2024-03-04T13:43:10.794 INFO:tasks.workunit.client.0.smithi107.stdout:3/454: dwrite d1/d2d/f44 [0,4194304] 0 2024-03-04T13:43:10.797 INFO:tasks.workunit.client.0.smithi107.stdout:2/462: rename da/d34/l74 to da/d34/la0 0 2024-03-04T13:43:10.799 INFO:tasks.workunit.client.0.smithi107.stdout:8/502: symlink d6/l99 0 2024-03-04T13:43:10.799 INFO:tasks.workunit.client.0.smithi107.stdout:2/463: dread - da/d49/f9b zero size 2024-03-04T13:43:10.800 INFO:tasks.workunit.client.1.smithi169.stdout:2/969: dread d1/d5/d19/d3c/db8/de1/d64/ff4 [0,4194304] 0 2024-03-04T13:43:10.802 INFO:tasks.workunit.client.0.smithi107.stdout:8/503: chown d6/da/d25/d5f/f98 738186 1 2024-03-04T13:43:10.803 INFO:tasks.workunit.client.1.smithi169.stdout:7/967: dwrite d2/d15/d17/f6e [0,4194304] 0 2024-03-04T13:43:10.804 INFO:tasks.workunit.client.0.smithi107.stdout:9/438: unlink d4/d34/l49 0 2024-03-04T13:43:10.804 INFO:tasks.workunit.client.1.smithi169.stdout:7/968: chown d2/d15/dad 100595 1 2024-03-04T13:43:10.808 INFO:tasks.workunit.client.0.smithi107.stdout:9/439: dread d4/d34/f44 [0,4194304] 0 2024-03-04T13:43:10.808 INFO:tasks.workunit.client.0.smithi107.stdout:3/455: dread d1/de/d18/d33/f8f [0,4194304] 0 2024-03-04T13:43:10.816 INFO:tasks.workunit.client.1.smithi169.stdout:3/934: rename d7/de/d2b/d7d/d85/d13/d1a/d2d/d56/d5e/d5f/lbb to d7/de/d2b/d7d/d85/d13/d1a/d2d/dbe/d108/df7/l132 0 2024-03-04T13:43:10.818 INFO:tasks.workunit.client.1.smithi169.stdout:6/957: mknod d0/d3c/d40/df6/c157 0 2024-03-04T13:43:10.821 INFO:tasks.workunit.client.1.smithi169.stdout:7/969: mknod d2/de/d3e/d41/d5b/d7a/dd3/c14a 0 2024-03-04T13:43:10.824 INFO:tasks.workunit.client.1.smithi169.stdout:6/958: mkdir d0/d32/d70/d158 0 2024-03-04T13:43:10.826 INFO:tasks.workunit.client.1.smithi169.stdout:3/935: mknod d7/de/d2b/d7d/d85/d18/d21/d103/d11e/c133 0 2024-03-04T13:43:10.829 INFO:tasks.workunit.client.1.smithi169.stdout:6/959: symlink d0/d32/d3a/d43/l159 0 2024-03-04T13:43:10.834 INFO:tasks.workunit.client.1.smithi169.stdout:6/960: rename d0/d32/d3a/l25 to d0/d3c/d95/d9e/l15a 0 2024-03-04T13:43:10.834 INFO:tasks.workunit.client.0.smithi107.stdout:0/526: unlink da/dc/f4d 0 2024-03-04T13:43:10.836 INFO:tasks.workunit.client.0.smithi107.stdout:2/464: read da/d29/f4e [233834,5430] 0 2024-03-04T13:43:10.838 INFO:tasks.workunit.client.0.smithi107.stdout:5/453: rename d5/d8/d14/d5a/d69/l76 to d5/d8/da/l9b 0 2024-03-04T13:43:10.840 INFO:tasks.workunit.client.0.smithi107.stdout:0/527: write da/f59 [1043613,55379] 0 2024-03-04T13:43:10.841 INFO:tasks.workunit.client.1.smithi169.stdout:6/961: creat d0/d32/d3a/def/f15b x:0 0 0 2024-03-04T13:43:10.841 INFO:tasks.workunit.client.1.smithi169.stdout:1/884: dread d4/d5/d36/d35/d5b/fe9 [0,4194304] 0 2024-03-04T13:43:10.843 INFO:tasks.workunit.client.0.smithi107.stdout:0/528: write da/dc/dd/f46 [1601160,19898] 0 2024-03-04T13:43:10.844 INFO:tasks.workunit.client.1.smithi169.stdout:0/839: write d5/d9/dc/f1d [2500722,111069] 0 2024-03-04T13:43:10.847 INFO:tasks.workunit.client.1.smithi169.stdout:6/962: fsync d0/d32/d70/fd1 0 2024-03-04T13:43:10.855 INFO:tasks.workunit.client.1.smithi169.stdout:6/963: unlink d0/d3c/d40/d52/dac/fd9 0 2024-03-04T13:43:10.859 INFO:tasks.workunit.client.1.smithi169.stdout:6/964: write d0/d32/d3a/d43/d74/d93/dbe/f12a [205487,29844] 0 2024-03-04T13:43:10.862 INFO:tasks.workunit.client.1.smithi169.stdout:2/970: write d1/d5/d19/d3c/db8/de1/d64/dce/f123 [870889,60030] 0 2024-03-04T13:43:10.864 INFO:tasks.workunit.client.1.smithi169.stdout:6/965: creat d0/d3c/d40/d52/f15c x:0 0 0 2024-03-04T13:43:10.865 INFO:tasks.workunit.client.1.smithi169.stdout:4/939: dread d4/d11/d3b/d67/f84 [0,4194304] 0 2024-03-04T13:43:10.866 INFO:tasks.workunit.client.1.smithi169.stdout:6/966: chown d0/d7a/d112/l11c 20 1 2024-03-04T13:43:10.866 INFO:tasks.workunit.client.1.smithi169.stdout:4/940: stat d4/d11/d3b/d67/f109 0 2024-03-04T13:43:10.868 INFO:tasks.workunit.client.1.smithi169.stdout:6/967: chown d0/f30 343 1 2024-03-04T13:43:10.872 INFO:tasks.workunit.client.1.smithi169.stdout:4/941: mknod d4/d11/d2e/d39/c11f 0 2024-03-04T13:43:10.877 INFO:tasks.workunit.client.1.smithi169.stdout:8/975: dread d7/d10/d24/d3e/f33 [0,4194304] 0 2024-03-04T13:43:10.881 INFO:tasks.workunit.client.1.smithi169.stdout:3/936: dwrite d7/de/d2b/d7d/d85/d13/d5a/da1/fdd [0,4194304] 0 2024-03-04T13:43:10.884 INFO:tasks.workunit.client.1.smithi169.stdout:7/970: dwrite d2/d28/d70/f87 [0,4194304] 0 2024-03-04T13:43:10.886 INFO:tasks.workunit.client.1.smithi169.stdout:0/840: truncate d5/d9/d3d/d5d/de4/f57 3645487 0 2024-03-04T13:43:10.886 INFO:tasks.workunit.client.1.smithi169.stdout:6/968: mknod d0/d3c/d40/df6/dca/d121/d135/c15d 0 2024-03-04T13:43:10.888 INFO:tasks.workunit.client.1.smithi169.stdout:1/885: dread d4/d5/d2d/d4e/d71/d26/f6d [0,4194304] 0 2024-03-04T13:43:10.889 INFO:tasks.workunit.client.0.smithi107.stdout:5/454: mknod d5/d8/d14/d5a/c9c 0 2024-03-04T13:43:10.889 INFO:tasks.workunit.client.1.smithi169.stdout:3/937: chown d7/de/d2b/d7d/d85/d13/d5a/dd8 125 1 2024-03-04T13:43:10.891 INFO:tasks.workunit.client.0.smithi107.stdout:1/456: write d1/d46/d6a/df/d33/d5d/d4c/f93 [259631,127031] 0 2024-03-04T13:43:10.895 INFO:tasks.workunit.client.0.smithi107.stdout:4/428: rename d0/d9/d10/l12 to d0/d9/de/d14/l88 0 2024-03-04T13:43:10.902 INFO:tasks.workunit.client.1.smithi169.stdout:0/841: symlink d5/d9/dc/d12/d15/d19/d69/d110/l123 0 2024-03-04T13:43:10.904 INFO:tasks.workunit.client.0.smithi107.stdout:0/529: write da/dc/dd/d3e/d6e/d73/d78/d86/f90 [76195,55056] 0 2024-03-04T13:43:10.908 INFO:tasks.workunit.client.1.smithi169.stdout:1/886: symlink d4/d5/d2d/d4e/d71/d82/l11f 0 2024-03-04T13:43:10.908 INFO:tasks.workunit.client.0.smithi107.stdout:1/457: fdatasync d1/d46/d6a/d75/f66 0 2024-03-04T13:43:10.911 INFO:tasks.workunit.client.1.smithi169.stdout:1/887: fsync d4/d5/f57 0 2024-03-04T13:43:10.912 INFO:tasks.workunit.client.0.smithi107.stdout:8/504: dwrite d6/d16/d1f/d4c/f52 [0,4194304] 0 2024-03-04T13:43:10.913 INFO:tasks.workunit.client.0.smithi107.stdout:9/440: dwrite d4/d34/f87 [0,4194304] 0 2024-03-04T13:43:10.914 INFO:tasks.workunit.client.1.smithi169.stdout:4/942: dwrite d4/d11/d3b/d67/dbb/f76 [0,4194304] 0 2024-03-04T13:43:10.915 INFO:tasks.workunit.client.0.smithi107.stdout:3/456: dwrite d1/fa [0,4194304] 0 2024-03-04T13:43:10.923 INFO:tasks.workunit.client.0.smithi107.stdout:5/455: dwrite d5/d2b/f88 [0,4194304] 0 2024-03-04T13:43:10.927 INFO:tasks.workunit.client.1.smithi169.stdout:7/971: dwrite d2/d15/d17/d6f/d126/d12f/f136 [0,4194304] 0 2024-03-04T13:43:10.927 INFO:tasks.workunit.client.1.smithi169.stdout:4/943: write d4/d11/d2e/dcd/dee/d73/d101/f11b [330904,67196] 0 2024-03-04T13:43:10.929 INFO:tasks.workunit.client.1.smithi169.stdout:0/842: write d5/d9/d3d/d59/d64/d7f/f9f [280144,109603] 0 2024-03-04T13:43:10.930 INFO:tasks.workunit.client.0.smithi107.stdout:5/456: read d5/dc/d22/f26 [3788544,85223] 0 2024-03-04T13:43:10.930 INFO:tasks.workunit.client.0.smithi107.stdout:3/457: read d1/f56 [2907975,37024] 0 2024-03-04T13:43:10.933 INFO:tasks.workunit.client.1.smithi169.stdout:7/972: chown d2/de/d3e/d7e/fef 30 1 2024-03-04T13:43:10.935 INFO:tasks.workunit.client.0.smithi107.stdout:1/458: unlink d1/d46/d6a/cd 0 2024-03-04T13:43:10.938 INFO:tasks.workunit.client.0.smithi107.stdout:0/530: creat da/dc/fb9 x:0 0 0 2024-03-04T13:43:10.938 INFO:tasks.workunit.client.0.smithi107.stdout:6/504: dread df/d1b/f1d [0,4194304] 0 2024-03-04T13:43:10.939 INFO:tasks.workunit.client.1.smithi169.stdout:8/976: dwrite d7/d10/d24/d3e/f45 [0,4194304] 0 2024-03-04T13:43:10.944 INFO:tasks.workunit.client.1.smithi169.stdout:6/969: dwrite d0/d32/d3e/d122/f156 [0,4194304] 0 2024-03-04T13:43:10.952 INFO:tasks.workunit.client.0.smithi107.stdout:9/441: symlink d4/da/l93 0 2024-03-04T13:43:10.952 INFO:tasks.workunit.client.0.smithi107.stdout:5/457: rmdir d5/d8/d2a/d62 39 2024-03-04T13:43:10.954 INFO:tasks.workunit.client.0.smithi107.stdout:9/442: chown d4/d34/f3f 11581 1 2024-03-04T13:43:10.959 INFO:tasks.workunit.client.1.smithi169.stdout:4/944: mkdir d4/d11/d2e/d39/d61/d120 0 2024-03-04T13:43:10.960 INFO:tasks.workunit.client.0.smithi107.stdout:0/531: chown da/dc/dd/lb0 161 1 2024-03-04T13:43:10.962 INFO:tasks.workunit.client.0.smithi107.stdout:6/505: symlink df/d17/d24/d30/d3b/la6 0 2024-03-04T13:43:10.963 INFO:tasks.workunit.client.1.smithi169.stdout:4/945: fsync d4/d11/d2e/d21/da9/fcb 0 2024-03-04T13:43:10.966 INFO:tasks.workunit.client.1.smithi169.stdout:7/973: rename d2/de/dfb/f134 to d2/de/d3e/d41/d69/deb/f14b 0 2024-03-04T13:43:10.968 INFO:tasks.workunit.client.0.smithi107.stdout:0/532: dread da/dc/dd/d3e/d6e/f87 [0,4194304] 0 2024-03-04T13:43:10.970 INFO:tasks.workunit.client.1.smithi169.stdout:0/843: dwrite d5/d9/dc/d12/d15/d19/d69/d110/f116 [0,4194304] 0 2024-03-04T13:43:10.972 INFO:tasks.workunit.client.0.smithi107.stdout:5/458: dwrite d5/d8/d14/d5a/d68/f97 [0,4194304] 0 2024-03-04T13:43:10.974 INFO:tasks.workunit.client.0.smithi107.stdout:5/459: chown d5/d8/d14 60 1 2024-03-04T13:43:10.977 INFO:tasks.workunit.client.1.smithi169.stdout:6/970: fdatasync d0/d3c/d95/dbd/fc6 0 2024-03-04T13:43:10.978 INFO:tasks.workunit.client.1.smithi169.stdout:1/888: getdents d4/d5/d36/d35/d16/d3a 0 2024-03-04T13:43:10.980 INFO:tasks.workunit.client.1.smithi169.stdout:3/938: dwrite d7/de/d2b/d7d/d85/d18/f30 [0,4194304] 0 2024-03-04T13:43:10.989 INFO:tasks.workunit.client.0.smithi107.stdout:3/458: getdents d1/d41/d81 0 2024-03-04T13:43:10.991 INFO:tasks.workunit.client.1.smithi169.stdout:0/844: rmdir d5/d4c/d65/d95/db0/d10c 39 2024-03-04T13:43:10.995 INFO:tasks.workunit.client.0.smithi107.stdout:9/443: creat d4/da/d11/d4c/d89/f94 x:0 0 0 2024-03-04T13:43:10.997 INFO:tasks.workunit.client.1.smithi169.stdout:8/977: rename d7/d10/d17/d44/c38 to d7/d10/d22/d46/c135 0 2024-03-04T13:43:10.999 INFO:tasks.workunit.client.1.smithi169.stdout:0/845: write d5/d9/dc/d12/d15/d19/d69/d110/f97 [619209,70624] 0 2024-03-04T13:43:11.000 INFO:tasks.workunit.client.1.smithi169.stdout:1/889: mknod d4/d5/d36/d35/c120 0 2024-03-04T13:43:11.002 INFO:tasks.workunit.client.1.smithi169.stdout:1/890: chown d4/d5/d36/d35 279212562 1 2024-03-04T13:43:11.003 INFO:tasks.workunit.client.0.smithi107.stdout:3/459: dwrite d1/de/d11/f45 [0,4194304] 0 2024-03-04T13:43:11.004 INFO:tasks.workunit.client.0.smithi107.stdout:5/460: rmdir d5/d8/d14/d54 39 2024-03-04T13:43:11.005 INFO:tasks.workunit.client.0.smithi107.stdout:9/444: readlink d4/da/d11/d20/d39/l7d 0 2024-03-04T13:43:11.006 INFO:tasks.workunit.client.0.smithi107.stdout:5/461: write d5/d8/d14/d2d/f56 [3003466,4438] 0 2024-03-04T13:43:11.010 INFO:tasks.workunit.client.0.smithi107.stdout:1/459: dread d1/d46/d6a/d65/f7f [0,4194304] 0 2024-03-04T13:43:11.013 INFO:tasks.workunit.client.1.smithi169.stdout:7/974: getdents d2/de/dcf 0 2024-03-04T13:43:11.014 INFO:tasks.workunit.client.0.smithi107.stdout:6/506: mkdir df/d2e/d36/d77/d95/da7 0 2024-03-04T13:43:11.016 INFO:tasks.workunit.client.1.smithi169.stdout:1/891: rmdir d4/d5/d36/d35/d16/d81 39 2024-03-04T13:43:11.022 INFO:tasks.workunit.client.0.smithi107.stdout:3/460: dwrite d1/de/d18/d1f/d42/f52 [0,4194304] 0 2024-03-04T13:43:11.023 INFO:tasks.workunit.client.1.smithi169.stdout:0/846: symlink d5/d9/dc/l124 0 2024-03-04T13:43:11.027 INFO:tasks.workunit.client.0.smithi107.stdout:1/460: creat d1/d46/d6a/d75/f95 x:0 0 0 2024-03-04T13:43:11.027 INFO:tasks.workunit.client.0.smithi107.stdout:8/505: dwrite d6/da/f35 [4194304,4194304] 0 2024-03-04T13:43:11.033 INFO:tasks.workunit.client.1.smithi169.stdout:0/847: rmdir d5/d9/d3d/d5d/de4 39 2024-03-04T13:43:11.038 INFO:tasks.workunit.client.1.smithi169.stdout:1/892: creat d4/d5/d36/d35/d16/d115/f121 x:0 0 0 2024-03-04T13:43:11.040 INFO:tasks.workunit.client.0.smithi107.stdout:3/461: write d1/de/d18/d33/d6b/f6c [1038817,7217] 0 2024-03-04T13:43:11.041 INFO:tasks.workunit.client.0.smithi107.stdout:1/461: fsync d1/d10/d16/d1b/f3e 0 2024-03-04T13:43:11.042 INFO:tasks.workunit.client.0.smithi107.stdout:1/462: chown d1/d10/d16 867 1 2024-03-04T13:43:11.042 INFO:tasks.workunit.client.0.smithi107.stdout:8/506: unlink d6/d16/d59/f60 0 2024-03-04T13:43:11.044 INFO:tasks.workunit.client.0.smithi107.stdout:6/507: mkdir df/d17/d6b/da8 0 2024-03-04T13:43:11.044 INFO:tasks.workunit.client.0.smithi107.stdout:8/507: stat d6/da/d25/d5f/f6e 0 2024-03-04T13:43:11.045 INFO:tasks.workunit.client.1.smithi169.stdout:0/848: symlink d5/d4c/d65/d95/dba/l125 0 2024-03-04T13:43:11.045 INFO:tasks.workunit.client.0.smithi107.stdout:8/508: dread - d6/d16/d1f/d4c/f61 zero size 2024-03-04T13:43:11.046 INFO:tasks.workunit.client.0.smithi107.stdout:0/533: getdents da/dc/dd/d1e 0 2024-03-04T13:43:11.047 INFO:tasks.workunit.client.1.smithi169.stdout:1/893: creat d4/d5/d36/d35/d16/d3a/f122 x:0 0 0 2024-03-04T13:43:11.050 INFO:tasks.workunit.client.0.smithi107.stdout:9/445: link d4/da/d11/d4c/d51/l70 d4/da/d11/d4c/d89/l95 0 2024-03-04T13:43:11.050 INFO:tasks.workunit.client.0.smithi107.stdout:5/462: creat d5/d8/d2a/f9d x:0 0 0 2024-03-04T13:43:11.054 INFO:tasks.workunit.client.0.smithi107.stdout:3/462: symlink d1/de/d11/l98 0 2024-03-04T13:43:11.059 INFO:tasks.workunit.client.1.smithi169.stdout:8/978: dread d7/d10/d22/d46/fc0 [0,4194304] 0 2024-03-04T13:43:11.060 INFO:tasks.workunit.client.1.smithi169.stdout:6/971: dread d0/d32/d3e/d122/f6d [0,4194304] 0 2024-03-04T13:43:11.060 INFO:tasks.workunit.client.1.smithi169.stdout:8/979: dread - d7/d12/fed zero size 2024-03-04T13:43:11.060 INFO:tasks.workunit.client.0.smithi107.stdout:1/463: dwrite d1/f94 [0,4194304] 0 2024-03-04T13:43:11.061 INFO:tasks.workunit.client.1.smithi169.stdout:1/894: creat d4/d5/d36/d35/d70/f123 x:0 0 0 2024-03-04T13:43:11.065 INFO:tasks.workunit.client.1.smithi169.stdout:8/980: write d7/d10/d17/d44/fd5 [25935,108734] 0 2024-03-04T13:43:11.070 INFO:tasks.workunit.client.0.smithi107.stdout:3/463: read d1/f3 [11637748,104498] 0 2024-03-04T13:43:11.071 INFO:tasks.workunit.client.0.smithi107.stdout:1/464: dread d1/d46/d6a/df/f3a [0,4194304] 0 2024-03-04T13:43:11.072 INFO:tasks.workunit.client.1.smithi169.stdout:0/849: dwrite d5/d9/dc/d12/d15/d19/d61/f23 [0,4194304] 0 2024-03-04T13:43:11.079 INFO:tasks.workunit.client.0.smithi107.stdout:0/534: unlink da/dc/dd/d3e/d26/d69/d6b/ca5 0 2024-03-04T13:43:11.080 INFO:tasks.workunit.client.1.smithi169.stdout:6/972: truncate d0/d32/d3a/f65 5772816 0 2024-03-04T13:43:11.080 INFO:tasks.workunit.client.1.smithi169.stdout:3/939: write d7/de/d2b/d7d/d85/d13/d1a/d2d/fc0 [475359,8666] 0 2024-03-04T13:43:11.082 INFO:tasks.workunit.client.1.smithi169.stdout:7/975: dread d2/de/d3e/d41/d5b/f11f [0,4194304] 0 2024-03-04T13:43:11.082 INFO:tasks.workunit.client.1.smithi169.stdout:8/981: unlink d7/d10/d22/d75/d81/da4/f122 0 2024-03-04T13:43:11.087 INFO:tasks.workunit.client.1.smithi169.stdout:6/973: read - d0/d3c/d95/d9e/ffb zero size 2024-03-04T13:43:11.095 INFO:tasks.workunit.client.0.smithi107.stdout:1/465: mknod d1/d46/d6a/df/d33/d43/d5b/d61/c96 0 2024-03-04T13:43:11.098 INFO:tasks.workunit.client.0.smithi107.stdout:0/535: mknod da/dad/cba 0 2024-03-04T13:43:11.100 INFO:tasks.workunit.client.0.smithi107.stdout:9/446: creat d4/da/f96 x:0 0 0 2024-03-04T13:43:11.101 INFO:tasks.workunit.client.0.smithi107.stdout:9/447: chown d4/da/d11/d65 122951 1 2024-03-04T13:43:11.103 INFO:tasks.workunit.client.0.smithi107.stdout:3/464: symlink d1/d1e/l99 0 2024-03-04T13:43:11.103 INFO:tasks.workunit.client.0.smithi107.stdout:8/509: write d6/d16/f24 [2608842,69815] 0 2024-03-04T13:43:11.104 INFO:tasks.workunit.client.0.smithi107.stdout:6/508: dwrite df/d38/f5f [0,4194304] 0 2024-03-04T13:43:11.109 INFO:tasks.workunit.client.0.smithi107.stdout:9/448: rmdir d4/da/d11/d20/d2f 39 2024-03-04T13:43:11.110 INFO:tasks.workunit.client.0.smithi107.stdout:1/466: dread d1/f94 [0,4194304] 0 2024-03-04T13:43:11.111 INFO:tasks.workunit.client.0.smithi107.stdout:9/449: readlink d4/da/d11/d20/d64/l6d 0 2024-03-04T13:43:11.112 INFO:tasks.workunit.client.0.smithi107.stdout:5/463: dwrite d5/d2b/f38 [0,4194304] 0 2024-03-04T13:43:11.112 INFO:tasks.workunit.client.1.smithi169.stdout:4/946: sync 2024-03-04T13:43:11.112 INFO:tasks.workunit.client.1.smithi169.stdout:2/971: sync 2024-03-04T13:43:11.113 INFO:tasks.workunit.client.0.smithi107.stdout:1/467: read d1/f94 [1580455,62930] 0 2024-03-04T13:43:11.114 INFO:tasks.workunit.client.0.smithi107.stdout:9/450: write d4/da/d11/d4c/d89/f8d [30092,26633] 0 2024-03-04T13:43:11.115 INFO:tasks.workunit.client.0.smithi107.stdout:1/468: readlink d1/d10/d16/d1b/d40/l68 0 2024-03-04T13:43:11.117 INFO:tasks.workunit.client.0.smithi107.stdout:1/469: stat d1/d10/d16/d1b 0 2024-03-04T13:43:11.117 INFO:tasks.workunit.client.0.smithi107.stdout:3/465: mknod d1/de/c9a 0 2024-03-04T13:43:11.119 INFO:tasks.workunit.client.0.smithi107.stdout:3/466: stat d1/de/d18/d33/d4d/d4f/d69 0 2024-03-04T13:43:11.120 INFO:tasks.workunit.client.0.smithi107.stdout:3/467: read - d1/de/d18/d1f/d42/f70 zero size 2024-03-04T13:43:11.121 INFO:tasks.workunit.client.1.smithi169.stdout:1/895: dread d4/d5/d36/d35/d16/d81/db4/f11b [0,4194304] 0 2024-03-04T13:43:11.123 INFO:tasks.workunit.client.0.smithi107.stdout:6/509: truncate df/d1b/f4b 980935 0 2024-03-04T13:43:11.154 INFO:tasks.workunit.client.1.smithi169.stdout:3/940: truncate d7/de/d2b/d7d/d85/d18/f7a 108367 0 2024-03-04T13:43:11.163 INFO:tasks.workunit.client.1.smithi169.stdout:2/972: symlink d1/d6d/dbc/l145 0 2024-03-04T13:43:11.164 INFO:tasks.workunit.client.1.smithi169.stdout:1/896: mknod d4/d5/d2d/d4e/da5/d11a/c124 0 2024-03-04T13:43:11.167 INFO:tasks.workunit.client.1.smithi169.stdout:7/976: link d2/d15/d17/f5d d2/de/d123/f14c 0 2024-03-04T13:43:11.168 INFO:tasks.workunit.client.1.smithi169.stdout:1/897: read d4/d5/fc4 [506883,119404] 0 2024-03-04T13:43:11.168 INFO:tasks.workunit.client.1.smithi169.stdout:8/982: mkdir d7/d10/d17/d44/d36/d62/d136 0 2024-03-04T13:43:11.170 INFO:tasks.workunit.client.1.smithi169.stdout:3/941: mknod d7/de/d2b/d7d/d85/d13/d1a/d2d/d56/d5e/d67/c134 0 2024-03-04T13:43:11.172 INFO:tasks.workunit.client.1.smithi169.stdout:1/898: stat d4/d5/d2d/d4e/d71/d26/dfa 0 2024-03-04T13:43:11.174 INFO:tasks.workunit.client.1.smithi169.stdout:0/850: dread d5/d9/dc/d12/d15/d19/d61/d71/fa8 [0,4194304] 0 2024-03-04T13:43:11.176 INFO:tasks.workunit.client.1.smithi169.stdout:3/942: readlink d7/de/d2b/ld2 0 2024-03-04T13:43:11.177 INFO:tasks.workunit.client.1.smithi169.stdout:0/851: read d5/d9/dc/d12/d15/d19/d77/fcf [827626,103972] 0 2024-03-04T13:43:11.179 INFO:tasks.workunit.client.1.smithi169.stdout:7/977: mkdir d2/d15/d17/dff/d14d 0 2024-03-04T13:43:11.184 INFO:tasks.workunit.client.1.smithi169.stdout:2/973: link d1/f75 d1/d5/d12e/f146 0 2024-03-04T13:43:11.185 INFO:tasks.workunit.client.1.smithi169.stdout:4/947: link d4/d11/d2e/dcd/dee/f57 d4/d11/d2e/f121 0 2024-03-04T13:43:11.190 INFO:tasks.workunit.client.1.smithi169.stdout:8/983: dwrite d7/d10/d22/d75/de9/f11f [0,4194304] 0 2024-03-04T13:43:11.196 INFO:tasks.workunit.client.1.smithi169.stdout:6/974: truncate d0/d3c/d40/df6/dca/ff1 1049717 0 2024-03-04T13:43:11.197 INFO:tasks.workunit.client.0.smithi107.stdout:6/510: truncate df/d57/f64 909426 0 2024-03-04T13:43:11.199 INFO:tasks.workunit.client.0.smithi107.stdout:0/536: creat da/dc/fbb x:0 0 0 2024-03-04T13:43:11.199 INFO:tasks.workunit.client.1.smithi169.stdout:2/974: rmdir d1/d5/d19/d3c/db8/de1/d64/dce/dd5/df5/d120/d11c/d139 39 2024-03-04T13:43:11.202 INFO:tasks.workunit.client.1.smithi169.stdout:0/852: creat d5/d9/dc/d12/d15/d19/f126 x:0 0 0 2024-03-04T13:43:11.206 INFO:tasks.workunit.client.1.smithi169.stdout:2/975: mknod d1/c147 0 2024-03-04T13:43:11.207 INFO:tasks.workunit.client.0.smithi107.stdout:9/451: dwrite d4/da/d11/d4c/d4d/f54 [0,4194304] 0 2024-03-04T13:43:11.210 INFO:tasks.workunit.client.1.smithi169.stdout:8/984: truncate d7/d10/d22/d46/da5/da6/f10d 710941 0 2024-03-04T13:43:11.213 INFO:tasks.workunit.client.0.smithi107.stdout:6/511: creat df/d17/d24/d40/d63/da3/d7e/fa9 x:0 0 0 2024-03-04T13:43:11.213 INFO:tasks.workunit.client.1.smithi169.stdout:6/975: truncate d0/d3c/d40/d52/dac/fc1 595584 0 2024-03-04T13:43:11.215 INFO:tasks.workunit.client.0.smithi107.stdout:3/468: dwrite d1/de/d11/f46 [0,4194304] 0 2024-03-04T13:43:11.218 INFO:tasks.workunit.client.0.smithi107.stdout:3/469: stat d1/de/d18/f92 0 2024-03-04T13:43:11.219 INFO:tasks.workunit.client.1.smithi169.stdout:3/943: truncate d7/de/d2b/d7d/d85/f4f 231206 0 2024-03-04T13:43:11.223 INFO:tasks.workunit.client.0.smithi107.stdout:1/470: dwrite d1/d46/d6a/df/f3a [0,4194304] 0 2024-03-04T13:43:11.224 INFO:tasks.workunit.client.1.smithi169.stdout:4/948: dread d4/d11/d2e/dcd/dde/f10f [0,4194304] 0 2024-03-04T13:43:11.226 INFO:tasks.workunit.client.1.smithi169.stdout:1/899: dwrite d4/f1b [0,4194304] 0 2024-03-04T13:43:11.228 INFO:tasks.workunit.client.1.smithi169.stdout:8/985: rename d7/d10/d22/d75/de9 to d7/d10/d22/d75/d81/da4/d137 0 2024-03-04T13:43:11.229 INFO:tasks.workunit.client.1.smithi169.stdout:1/900: readlink d4/d5/l43 0 2024-03-04T13:43:11.230 INFO:tasks.workunit.client.0.smithi107.stdout:3/470: unlink d1/l10 0 2024-03-04T13:43:11.234 INFO:tasks.workunit.client.0.smithi107.stdout:6/512: dwrite df/d17/d24/d40/f44 [0,4194304] 0 2024-03-04T13:43:11.235 INFO:tasks.workunit.client.1.smithi169.stdout:0/853: symlink d5/d9/dc/l127 0 2024-03-04T13:43:11.238 INFO:tasks.workunit.client.0.smithi107.stdout:3/471: unlink d1/d29/f7c 0 2024-03-04T13:43:11.239 INFO:tasks.workunit.client.1.smithi169.stdout:3/944: mkdir d7/de/d2b/d7d/d85/d18/d21/d103/d135 0 2024-03-04T13:43:11.242 INFO:tasks.workunit.client.1.smithi169.stdout:3/945: fdatasync d7/db8/ded/ff1 0 2024-03-04T13:43:11.245 INFO:tasks.workunit.client.1.smithi169.stdout:4/949: rmdir d4/d11/d2e/dcd 39 2024-03-04T13:43:11.245 INFO:tasks.workunit.client.0.smithi107.stdout:1/471: link d1/d46/d6a/df/d33/d5d/d24/f44 d1/d46/d6a/df/d33/d5d/f97 0 2024-03-04T13:43:11.247 INFO:tasks.workunit.client.0.smithi107.stdout:6/513: mknod df/d17/d24/d40/d63/da3/d7e/caa 0 2024-03-04T13:43:11.248 INFO:tasks.workunit.client.0.smithi107.stdout:7/467: sync 2024-03-04T13:43:11.249 INFO:tasks.workunit.client.0.smithi107.stdout:5/464: dread d5/dc/d22/f61 [0,4194304] 0 2024-03-04T13:43:11.249 INFO:tasks.workunit.client.1.smithi169.stdout:1/901: mknod d4/d5/d36/d35/d70/d78/c125 0 2024-03-04T13:43:11.251 INFO:tasks.workunit.client.1.smithi169.stdout:0/854: truncate d5/d4c/d65/d95/fcd 348359 0 2024-03-04T13:43:11.251 INFO:tasks.workunit.client.1.smithi169.stdout:1/902: chown d4/dd8 5 1 2024-03-04T13:43:11.252 INFO:tasks.workunit.client.1.smithi169.stdout:4/950: read d4/d11/d3b/d67/dbb/f76 [1328033,58719] 0 2024-03-04T13:43:11.253 INFO:tasks.workunit.client.0.smithi107.stdout:6/514: write df/d57/f7c [3720287,95855] 0 2024-03-04T13:43:11.260 INFO:tasks.workunit.client.0.smithi107.stdout:5/465: fsync d5/d8/f1f 0 2024-03-04T13:43:11.263 INFO:tasks.workunit.client.1.smithi169.stdout:0/855: truncate d5/d9/d3d/d59/d64/d7f/f9f 987223 0 2024-03-04T13:43:11.265 INFO:tasks.workunit.client.1.smithi169.stdout:0/856: read - d5/d9/d3d/f11a zero size 2024-03-04T13:43:11.274 INFO:tasks.workunit.client.0.smithi107.stdout:5/466: symlink d5/d8/d2a/d43/d5f/l9e 0 2024-03-04T13:43:11.275 INFO:tasks.workunit.client.1.smithi169.stdout:0/857: rmdir d5/d4c/d65/d95/db0 39 2024-03-04T13:43:11.276 INFO:tasks.workunit.client.1.smithi169.stdout:2/976: dwrite d1/d5/d19/fa1 [0,4194304] 0 2024-03-04T13:43:11.279 INFO:tasks.workunit.client.0.smithi107.stdout:9/452: dread d4/da/f4b [0,4194304] 0 2024-03-04T13:43:11.282 INFO:tasks.workunit.client.1.smithi169.stdout:1/903: symlink d4/d5/d2d/d4e/d71/d26/l126 0 2024-03-04T13:43:11.284 INFO:tasks.workunit.client.0.smithi107.stdout:5/467: symlink d5/d8/d14/d5a/d69/l9f 0 2024-03-04T13:43:11.286 INFO:tasks.workunit.client.1.smithi169.stdout:2/977: mknod d1/d5/d19/d3c/db8/de1/d64/dce/dd5/df5/d120/c148 0 2024-03-04T13:43:11.286 INFO:tasks.workunit.client.1.smithi169.stdout:6/976: write d0/d32/d3a/d43/d74/fa4 [799967,49643] 0 2024-03-04T13:43:11.286 INFO:tasks.workunit.client.1.smithi169.stdout:4/951: getdents d4/d11/d2e/d6c/d78/da5/d77 0 2024-03-04T13:43:11.288 INFO:tasks.workunit.client.0.smithi107.stdout:5/468: mknod d5/d79/d89/ca0 0 2024-03-04T13:43:11.289 INFO:tasks.workunit.client.1.smithi169.stdout:4/952: write d4/d1c/d75/ff4 [3740489,63827] 0 2024-03-04T13:43:11.289 INFO:tasks.workunit.client.1.smithi169.stdout:2/978: fdatasync d1/d5/d19/d1e/d29/f77 0 2024-03-04T13:43:11.289 INFO:tasks.workunit.client.0.smithi107.stdout:5/469: chown d5/d8/d14/d5a/d69 41 1 2024-03-04T13:43:11.291 INFO:tasks.workunit.client.1.smithi169.stdout:4/953: readlink d4/d11/le5 0 2024-03-04T13:43:11.292 INFO:tasks.workunit.client.1.smithi169.stdout:0/858: rename d5/d9/d3d/d5d/de4/d98/dfc to d5/d9/d3d/d59/d64/d7f/d128 0 2024-03-04T13:43:11.298 INFO:tasks.workunit.client.0.smithi107.stdout:5/470: truncate d5/dc/d22/f26 4986102 0 2024-03-04T13:43:11.298 INFO:tasks.workunit.client.1.smithi169.stdout:6/977: symlink d0/d32/l15e 0 2024-03-04T13:43:11.299 INFO:tasks.workunit.client.0.smithi107.stdout:1/472: write d1/d46/d6a/d17/f34 [518791,18047] 0 2024-03-04T13:43:11.299 INFO:tasks.workunit.client.0.smithi107.stdout:3/472: dwrite d1/f56 [0,4194304] 0 2024-03-04T13:43:11.304 INFO:tasks.workunit.client.0.smithi107.stdout:7/468: write d6/d13/d16/d1c/f1f [2196046,60505] 0 2024-03-04T13:43:11.306 INFO:tasks.workunit.client.1.smithi169.stdout:0/859: truncate d5/d9/dc/d12/d15/f9e 4924638 0 2024-03-04T13:43:11.306 INFO:tasks.workunit.client.1.smithi169.stdout:8/986: dwrite d7/d10/d22/d75/f118 [0,4194304] 0 2024-03-04T13:43:11.308 INFO:tasks.workunit.client.0.smithi107.stdout:1/473: chown d1/d46/d6a/d4e/d60/f78 33064463 1 2024-03-04T13:43:11.308 INFO:tasks.workunit.client.1.smithi169.stdout:2/979: read d1/d5/f9 [2710338,67545] 0 2024-03-04T13:43:11.308 INFO:tasks.workunit.client.1.smithi169.stdout:8/987: chown d7/d10/d22/d46/f123 1047447785 1 2024-03-04T13:43:11.309 INFO:tasks.workunit.client.0.smithi107.stdout:6/515: truncate df/f15 1744028 0 2024-03-04T13:43:11.310 INFO:tasks.workunit.client.0.smithi107.stdout:1/474: fdatasync d1/d10/d16/d47/f77 0 2024-03-04T13:43:11.311 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:11 smithi107 ceph-mon[28739]: from='client.14692 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:43:11.311 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:11 smithi107 ceph-mon[28739]: pgmap v57: 65 pgs: 65 active+clean; 2.5 GiB data, 8.8 GiB used, 528 GiB / 536 GiB avail; 62 MiB/s rd, 208 MiB/s wr, 401 op/s 2024-03-04T13:43:11.311 INFO:tasks.workunit.client.0.smithi107.stdout:3/473: unlink d1/de/d18/d33/d4d/c8e 0 2024-03-04T13:43:11.312 INFO:tasks.workunit.client.0.smithi107.stdout:5/471: creat d5/d8/da/fa1 x:0 0 0 2024-03-04T13:43:11.322 INFO:tasks.workunit.client.0.smithi107.stdout:1/475: readlink d1/d10/d16/l1c 0 2024-03-04T13:43:11.323 INFO:tasks.workunit.client.1.smithi169.stdout:8/988: dread - d7/d10/d24/f11b zero size 2024-03-04T13:43:11.324 INFO:tasks.workunit.client.1.smithi169.stdout:6/978: truncate d0/d32/d3a/d43/d74/f62 1252763 0 2024-03-04T13:43:11.325 INFO:tasks.workunit.client.0.smithi107.stdout:1/476: readlink d1/d46/d6a/df/d33/d43/d5b/l5e 0 2024-03-04T13:43:11.327 INFO:tasks.workunit.client.1.smithi169.stdout:1/904: rename d4/d5/d2d/d4e/d71/d26/d54/f72 to d4/d5/f127 0 2024-03-04T13:43:11.329 INFO:tasks.workunit.client.1.smithi169.stdout:1/905: chown d4/d5/d36/d75/dba 550375 1 2024-03-04T13:43:11.334 INFO:tasks.workunit.client.0.smithi107.stdout:3/474: dread d1/de/d18/d33/f8f [0,4194304] 0 2024-03-04T13:43:11.335 INFO:tasks.workunit.client.1.smithi169.stdout:6/979: creat d0/d32/d3a/d43/d74/da6/d134/f15f x:0 0 0 2024-03-04T13:43:11.335 INFO:tasks.workunit.client.1.smithi169.stdout:4/954: write d4/d11/d3b/d67/fa0 [5496550,103237] 0 2024-03-04T13:43:11.339 INFO:tasks.workunit.client.1.smithi169.stdout:2/980: rename d1/d5/d19/d1e/d29 to d1/d5/d19/d68/dd9/d115/d149 0 2024-03-04T13:43:11.339 INFO:tasks.workunit.client.1.smithi169.stdout:6/980: write d0/d32/d3a/d43/d74/d93/f151 [92369,96309] 0 2024-03-04T13:43:11.341 INFO:tasks.workunit.client.0.smithi107.stdout:1/477: dwrite d1/d46/d6a/df/f3a [0,4194304] 0 2024-03-04T13:43:11.345 INFO:tasks.workunit.client.0.smithi107.stdout:6/516: write df/d1b/f1d [7058151,124736] 0 2024-03-04T13:43:11.348 INFO:tasks.workunit.client.0.smithi107.stdout:7/469: dwrite d6/d1d/d36/f5b [0,4194304] 0 2024-03-04T13:43:11.349 INFO:tasks.workunit.client.1.smithi169.stdout:8/989: fsync d7/d10/d22/d75/db9/dbe/f104 0 2024-03-04T13:43:11.350 INFO:tasks.workunit.client.1.smithi169.stdout:0/860: write d5/d4c/d65/faf [1881698,47840] 0 2024-03-04T13:43:11.351 INFO:tasks.workunit.client.0.smithi107.stdout:1/478: mknod d1/d46/c98 0 2024-03-04T13:43:11.353 INFO:tasks.workunit.client.0.smithi107.stdout:1/479: chown d1/d10/d16/d1b 12205372 1 2024-03-04T13:43:11.353 INFO:tasks.workunit.client.0.smithi107.stdout:7/470: truncate d6/d13/d16/d19/d38/d62/f9b 351924 0 2024-03-04T13:43:11.354 INFO:tasks.workunit.client.0.smithi107.stdout:1/480: stat d1/d46/d6a/d75/f95 0 2024-03-04T13:43:11.355 INFO:tasks.workunit.client.0.smithi107.stdout:7/471: write d6/fa [1342267,57190] 0 2024-03-04T13:43:11.356 INFO:tasks.workunit.client.1.smithi169.stdout:6/981: unlink d0/d3c/d40/c9f 0 2024-03-04T13:43:11.357 INFO:tasks.workunit.client.0.smithi107.stdout:5/472: write d5/dc/d22/f26 [3484361,53559] 0 2024-03-04T13:43:11.358 INFO:tasks.workunit.client.0.smithi107.stdout:3/475: mkdir d1/d1e/d9b 0 2024-03-04T13:43:11.359 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:11 smithi169 ceph-mon[36313]: from='client.14692 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:43:11.359 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:11 smithi169 ceph-mon[36313]: pgmap v57: 65 pgs: 65 active+clean; 2.5 GiB data, 8.8 GiB used, 528 GiB / 536 GiB avail; 62 MiB/s rd, 208 MiB/s wr, 401 op/s 2024-03-04T13:43:11.360 INFO:tasks.workunit.client.0.smithi107.stdout:6/517: mkdir df/d17/d24/d40/d63/da3/dab 0 2024-03-04T13:43:11.360 INFO:tasks.workunit.client.1.smithi169.stdout:4/955: dwrite d4/d11/d3b/d67/dbb/d6e/fd4 [4194304,4194304] 0 2024-03-04T13:43:11.362 INFO:tasks.workunit.client.1.smithi169.stdout:1/906: symlink d4/l128 0 2024-03-04T13:43:11.367 INFO:tasks.workunit.client.1.smithi169.stdout:8/990: dread - d7/f9f zero size 2024-03-04T13:43:11.368 INFO:tasks.workunit.client.1.smithi169.stdout:0/861: creat d5/d9/dc/d12/d15/d19/d69/d110/f129 x:0 0 0 2024-03-04T13:43:11.371 INFO:tasks.workunit.client.1.smithi169.stdout:2/981: symlink d1/d5/d19/d3c/db8/de1/d64/dce/dd5/df5/l14a 0 2024-03-04T13:43:11.372 INFO:tasks.workunit.client.1.smithi169.stdout:0/862: write d5/d9/dc/d12/d15/f40 [3340032,111235] 0 2024-03-04T13:43:11.374 INFO:tasks.workunit.client.1.smithi169.stdout:1/907: unlink d4/d5/d36/d35/df/f2b 0 2024-03-04T13:43:11.375 INFO:tasks.workunit.client.1.smithi169.stdout:8/991: creat d7/d10/d17/d44/d5d/f138 x:0 0 0 2024-03-04T13:43:11.376 INFO:tasks.workunit.client.1.smithi169.stdout:6/982: mknod d0/d32/d70/d158/c160 0 2024-03-04T13:43:11.378 INFO:tasks.workunit.client.0.smithi107.stdout:6/518: creat df/fac x:0 0 0 2024-03-04T13:43:11.378 INFO:tasks.workunit.client.0.smithi107.stdout:7/472: dwrite d6/d1d/d64/d8e/f75 [0,4194304] 0 2024-03-04T13:43:11.379 INFO:tasks.workunit.client.1.smithi169.stdout:7/978: sync 2024-03-04T13:43:11.380 INFO:tasks.workunit.client.1.smithi169.stdout:2/982: unlink d1/d5/d22/d3b/d41/c103 0 2024-03-04T13:43:11.382 INFO:tasks.workunit.client.1.smithi169.stdout:0/863: unlink d5/cae 0 2024-03-04T13:43:11.383 INFO:tasks.workunit.client.0.smithi107.stdout:2/465: sync 2024-03-04T13:43:11.383 INFO:tasks.workunit.client.0.smithi107.stdout:4/429: sync 2024-03-04T13:43:11.384 INFO:tasks.workunit.client.1.smithi169.stdout:1/908: creat d4/d5/d2d/d4e/d71/d26/daf/f129 x:0 0 0 2024-03-04T13:43:11.384 INFO:tasks.workunit.client.1.smithi169.stdout:0/864: dread - d5/d9/dc/d12/d15/d19/d77/ff7 zero size 2024-03-04T13:43:11.385 INFO:tasks.workunit.client.0.smithi107.stdout:7/473: mknod d6/d13/d16/d19/d39/d7a/c9c 0 2024-03-04T13:43:11.386 INFO:tasks.workunit.client.0.smithi107.stdout:7/474: dread - d6/d13/d16/f67 zero size 2024-03-04T13:43:11.386 INFO:tasks.workunit.client.1.smithi169.stdout:8/992: unlink d7/d10/d17/d44/d36/d62/fab 0 2024-03-04T13:43:11.387 INFO:tasks.workunit.client.0.smithi107.stdout:6/519: symlink df/d17/d24/d40/d63/lad 0 2024-03-04T13:43:11.389 INFO:tasks.workunit.client.1.smithi169.stdout:6/983: truncate d0/d3c/d95/fa7 2869919 0 2024-03-04T13:43:11.393 INFO:tasks.workunit.client.0.smithi107.stdout:3/476: dread d1/de/d18/d33/f34 [0,4194304] 0 2024-03-04T13:43:11.394 INFO:tasks.workunit.client.1.smithi169.stdout:7/979: rename d2/d15/d17/d6f/f72 to d2/d28/f14e 0 2024-03-04T13:43:11.396 INFO:tasks.workunit.client.0.smithi107.stdout:7/475: creat d6/d13/d14/f9d x:0 0 0 2024-03-04T13:43:11.397 INFO:tasks.workunit.client.0.smithi107.stdout:4/430: getdents d0/d2f/d7d 0 2024-03-04T13:43:11.402 INFO:tasks.workunit.client.1.smithi169.stdout:0/865: creat d5/d9/d3d/d59/f12a x:0 0 0 2024-03-04T13:43:11.402 INFO:tasks.workunit.client.1.smithi169.stdout:8/993: dwrite d7/d10/d17/d44/d4c/f121 [0,4194304] 0 2024-03-04T13:43:11.406 INFO:tasks.workunit.client.1.smithi169.stdout:6/984: mkdir d0/d32/d3a/d43/d74/d93/dee/d118/d161 0 2024-03-04T13:43:11.407 INFO:tasks.workunit.client.1.smithi169.stdout:6/985: stat d0/d32/c97 0 2024-03-04T13:43:11.409 INFO:tasks.workunit.client.0.smithi107.stdout:5/473: truncate d5/dc/f58 684514 0 2024-03-04T13:43:11.409 INFO:tasks.workunit.client.0.smithi107.stdout:1/481: dwrite d1/d46/d6a/df/d33/d5d/d24/f44 [0,4194304] 0 2024-03-04T13:43:11.415 INFO:tasks.workunit.client.0.smithi107.stdout:6/520: mknod df/d2e/d36/cae 0 2024-03-04T13:43:11.418 INFO:tasks.workunit.client.1.smithi169.stdout:4/956: getdents d4/d11/d2e/dcd/dd1 0 2024-03-04T13:43:11.421 INFO:tasks.workunit.client.1.smithi169.stdout:0/866: truncate d5/f16 4809882 0 2024-03-04T13:43:11.422 INFO:tasks.workunit.client.0.smithi107.stdout:5/474: symlink d5/d8/d2a/d43/d5f/la2 0 2024-03-04T13:43:11.423 INFO:tasks.workunit.client.0.smithi107.stdout:9/453: sync 2024-03-04T13:43:11.423 INFO:tasks.workunit.client.0.smithi107.stdout:8/510: sync 2024-03-04T13:43:11.423 INFO:tasks.workunit.client.0.smithi107.stdout:0/537: sync 2024-03-04T13:43:11.425 INFO:tasks.workunit.client.0.smithi107.stdout:0/538: write da/f59 [1804239,89344] 0 2024-03-04T13:43:11.426 INFO:tasks.workunit.client.0.smithi107.stdout:1/482: rmdir d1/d46/d6a/d17 39 2024-03-04T13:43:11.431 INFO:tasks.workunit.client.1.smithi169.stdout:1/909: write d4/d1d/f8e [30948,89002] 0 2024-03-04T13:43:11.432 INFO:tasks.workunit.client.1.smithi169.stdout:2/983: creat d1/d5/d19/d3c/f14b x:0 0 0 2024-03-04T13:43:11.435 INFO:tasks.workunit.client.0.smithi107.stdout:7/476: link d6/d13/d16/d1c/f35 d6/d13/d16/d19/d38/d37/f9e 0 2024-03-04T13:43:11.435 INFO:tasks.workunit.client.0.smithi107.stdout:2/466: truncate da/df/d27/f5e 141121 0 2024-03-04T13:43:11.438 INFO:tasks.workunit.client.1.smithi169.stdout:3/946: sync 2024-03-04T13:43:11.439 INFO:tasks.workunit.client.0.smithi107.stdout:6/521: mkdir df/d38/d4f/daf 0 2024-03-04T13:43:11.440 INFO:tasks.workunit.client.0.smithi107.stdout:3/477: write d1/de/d18/d33/d48/d71/d4c/f91 [564817,14096] 0 2024-03-04T13:43:11.441 INFO:tasks.workunit.client.0.smithi107.stdout:8/511: dwrite d6/d16/d59/d7c/f8a [0,4194304] 0 2024-03-04T13:43:11.444 INFO:tasks.workunit.client.0.smithi107.stdout:5/475: rename d5/d8/d14/d5a to d5/d79/d89/da3 0 2024-03-04T13:43:11.447 INFO:tasks.workunit.client.1.smithi169.stdout:7/980: dwrite d2/ff3 [0,4194304] 0 2024-03-04T13:43:11.449 INFO:tasks.workunit.client.0.smithi107.stdout:1/483: dwrite d1/d46/d6a/df/d33/d43/f55 [0,4194304] 0 2024-03-04T13:43:11.450 INFO:tasks.workunit.client.1.smithi169.stdout:0/867: unlink d5/d9/dc/d12/d15/d19/f32 0 2024-03-04T13:43:11.451 INFO:tasks.workunit.client.0.smithi107.stdout:0/539: unlink da/dc/dd/d3e/d26/d69/c84 0 2024-03-04T13:43:11.459 INFO:tasks.workunit.client.0.smithi107.stdout:4/431: rmdir d0/d9/d74 0 2024-03-04T13:43:11.459 INFO:tasks.workunit.client.0.smithi107.stdout:2/467: symlink da/d6c/la1 0 2024-03-04T13:43:11.461 INFO:tasks.workunit.client.1.smithi169.stdout:4/957: creat d4/d11/d2e/dcd/dee/d5c/d86/db8/df9/f122 x:0 0 0 2024-03-04T13:43:11.463 INFO:tasks.workunit.client.0.smithi107.stdout:7/477: truncate d6/d13/d16/d1c/f48 1180606 0 2024-03-04T13:43:11.466 INFO:tasks.workunit.client.0.smithi107.stdout:6/522: fsync df/d17/d24/d40/d63/da3/f70 0 2024-03-04T13:43:11.466 INFO:tasks.workunit.client.1.smithi169.stdout:7/981: symlink d2/d28/db8/l14f 0 2024-03-04T13:43:11.468 INFO:tasks.workunit.client.1.smithi169.stdout:1/910: dread d4/d5/d36/d75/dba/fc9 [0,4194304] 0 2024-03-04T13:43:11.469 INFO:tasks.workunit.client.1.smithi169.stdout:8/994: creat d7/d10/d22/d46/d49/f139 x:0 0 0 2024-03-04T13:43:11.470 INFO:tasks.workunit.client.0.smithi107.stdout:6/523: write df/d17/d24/d40/d63/da3/f68 [49451,104901] 0 2024-03-04T13:43:11.474 INFO:tasks.workunit.client.1.smithi169.stdout:8/995: readlink d7/d10/d22/d75/db9/dbe/l11a 0 2024-03-04T13:43:11.482 INFO:tasks.workunit.client.0.smithi107.stdout:1/484: dread d1/d46/d6a/df/d33/d5d/d24/f62 [0,4194304] 0 2024-03-04T13:43:11.482 INFO:tasks.workunit.client.1.smithi169.stdout:3/947: rename d7/de/d2b/d7d/d85/d13/d1a/d2d/dbe/d108/df7/l132 to d7/de/d2b/d7d/d85/d18/d21/d66/l136 0 2024-03-04T13:43:11.485 INFO:tasks.workunit.client.1.smithi169.stdout:4/958: rmdir d4/d1c/d75/d81 39 2024-03-04T13:43:11.486 INFO:tasks.workunit.client.1.smithi169.stdout:6/986: dwrite d0/d3c/d40/f7d [0,4194304] 0 2024-03-04T13:43:11.486 INFO:tasks.workunit.client.0.smithi107.stdout:5/476: dwrite d5/dc/d22/f61 [0,4194304] 0 2024-03-04T13:43:11.487 INFO:tasks.workunit.client.0.smithi107.stdout:0/540: read da/dc/dd/d3e/f44 [2616292,74952] 0 2024-03-04T13:43:11.488 INFO:tasks.workunit.client.0.smithi107.stdout:4/432: write d0/d9/de/d14/dc/f69 [515309,37164] 0 2024-03-04T13:43:11.490 INFO:tasks.workunit.client.0.smithi107.stdout:3/478: symlink d1/de/d18/d33/l9c 0 2024-03-04T13:43:11.494 INFO:tasks.workunit.client.1.smithi169.stdout:2/984: creat d1/d5/d19/f14c x:0 0 0 2024-03-04T13:43:11.494 INFO:tasks.workunit.client.0.smithi107.stdout:1/485: dread d1/d46/d6a/d4e/f6c [0,4194304] 0 2024-03-04T13:43:11.494 INFO:tasks.workunit.client.0.smithi107.stdout:2/468: truncate da/df/f1e 360404 0 2024-03-04T13:43:11.496 INFO:tasks.workunit.client.0.smithi107.stdout:1/486: fsync d1/d46/d6a/d4e/f6c 0 2024-03-04T13:43:11.497 INFO:tasks.workunit.client.0.smithi107.stdout:7/478: creat d6/d13/d16/d19/d38/d2c/f9f x:0 0 0 2024-03-04T13:43:11.499 INFO:tasks.workunit.client.1.smithi169.stdout:7/982: read d2/de/d3e/d41/f5a [2022819,3568] 0 2024-03-04T13:43:11.500 INFO:tasks.workunit.client.0.smithi107.stdout:7/479: dread - d6/d13/d16/d1c/f7d zero size 2024-03-04T13:43:11.500 INFO:tasks.workunit.client.0.smithi107.stdout:1/487: write d1/d46/d6a/df/d33/d5d/d24/f44 [1486588,112130] 0 2024-03-04T13:43:11.502 INFO:tasks.workunit.client.1.smithi169.stdout:0/868: rename d5/d9/dc/d12/d15/d19/d61/f33 to d5/d9/dc/d12/d15/d19/d61/d71/f12b 0 2024-03-04T13:43:11.508 INFO:tasks.workunit.client.0.smithi107.stdout:8/512: mknod d6/d16/c9a 0 2024-03-04T13:43:11.509 INFO:tasks.workunit.client.1.smithi169.stdout:4/959: mknod d4/d11/d2e/d6c/d78/da5/c123 0 2024-03-04T13:43:11.511 INFO:tasks.workunit.client.0.smithi107.stdout:7/480: read d6/d13/d16/f58 [2615163,41985] 0 2024-03-04T13:43:11.517 INFO:tasks.workunit.client.1.smithi169.stdout:0/869: dread d5/d9/dc/d12/d15/d19/d61/f2c [0,4194304] 0 2024-03-04T13:43:11.519 INFO:tasks.workunit.client.1.smithi169.stdout:0/870: readlink d5/d9/dc/d12/d66/lfb 0 2024-03-04T13:43:11.521 INFO:tasks.workunit.client.1.smithi169.stdout:3/948: symlink d7/de/d2b/d7d/d85/d13/d1a/l137 0 2024-03-04T13:43:11.522 INFO:tasks.workunit.client.1.smithi169.stdout:0/871: write d5/d4c/d65/faf [1999582,111247] 0 2024-03-04T13:43:11.523 INFO:tasks.workunit.client.0.smithi107.stdout:2/469: dread da/d44/f81 [0,4194304] 0 2024-03-04T13:43:11.526 INFO:tasks.workunit.client.1.smithi169.stdout:7/983: symlink d2/de/l150 0 2024-03-04T13:43:11.527 INFO:tasks.workunit.client.0.smithi107.stdout:0/541: mknod da/dc/dd/d3e/d6e/d73/cbc 0 2024-03-04T13:43:11.528 INFO:tasks.workunit.client.0.smithi107.stdout:0/542: fdatasync da/dad/fae 0 2024-03-04T13:43:11.529 INFO:tasks.workunit.client.0.smithi107.stdout:0/543: chown da/dc/dd/d3e/d26/d88/c9b 11 1 2024-03-04T13:43:11.530 INFO:tasks.workunit.client.0.smithi107.stdout:4/433: creat d0/d9/d18/d29/f89 x:0 0 0 2024-03-04T13:43:11.548 INFO:tasks.workunit.client.1.smithi169.stdout:8/996: dread d7/f4d [0,4194304] 0 2024-03-04T13:43:11.548 INFO:tasks.workunit.client.1.smithi169.stdout:0/872: fsync d5/d9/dc/d12/d15/d19/d61/d71/dbb/fd4 0 2024-03-04T13:43:11.550 INFO:tasks.workunit.client.1.smithi169.stdout:1/911: dwrite d4/d5/d36/d35/d16/d81/db4/f11b [0,4194304] 0 2024-03-04T13:43:11.552 INFO:tasks.workunit.client.0.smithi107.stdout:7/481: mknod d6/d13/d16/d1c/ca0 0 2024-03-04T13:43:11.554 INFO:tasks.workunit.client.0.smithi107.stdout:6/524: rename df/f19 to df/d17/d5c/fb0 0 2024-03-04T13:43:11.556 INFO:tasks.workunit.client.1.smithi169.stdout:4/960: rename d4/d11/d2e/d6c/d78 to d4/d11/d3b/d67/d7e/d124 0 2024-03-04T13:43:11.556 INFO:tasks.workunit.client.1.smithi169.stdout:1/912: write d4/d5/d2d/d4e/fa6 [263693,77368] 0 2024-03-04T13:43:11.557 INFO:tasks.workunit.client.0.smithi107.stdout:9/454: getdents d4/da/d11/d20/d64/d77 0 2024-03-04T13:43:11.557 INFO:tasks.workunit.client.1.smithi169.stdout:6/987: dwrite d0/d3c/f86 [0,4194304] 0 2024-03-04T13:43:11.561 INFO:tasks.workunit.client.1.smithi169.stdout:7/984: symlink d2/d131/l151 0 2024-03-04T13:43:11.566 INFO:tasks.workunit.client.1.smithi169.stdout:7/985: write d2/d28/f139 [143036,36757] 0 2024-03-04T13:43:11.566 INFO:tasks.workunit.client.0.smithi107.stdout:5/477: creat d5/d8/d14/d54/fa4 x:0 0 0 2024-03-04T13:43:11.569 INFO:tasks.workunit.client.0.smithi107.stdout:0/544: write da/d7e/f89 [789881,76799] 0 2024-03-04T13:43:11.570 INFO:tasks.workunit.client.1.smithi169.stdout:8/997: dread - d7/dd4/fdc zero size 2024-03-04T13:43:11.570 INFO:tasks.workunit.client.1.smithi169.stdout:7/986: dread - d2/de/d3e/d41/d89/dd6/f137 zero size 2024-03-04T13:43:11.573 INFO:tasks.workunit.client.0.smithi107.stdout:4/434: chown d0/d9/de/d14/d33/l4d 919 1 2024-03-04T13:43:11.574 INFO:tasks.workunit.client.1.smithi169.stdout:1/913: chown d4/d5/d36/d35/c21 3642398 1 2024-03-04T13:43:11.577 INFO:tasks.workunit.client.1.smithi169.stdout:4/961: dwrite d4/d11/d3b/d67/d7e/fe9 [0,4194304] 0 2024-03-04T13:43:11.580 INFO:tasks.workunit.client.0.smithi107.stdout:7/482: rmdir d6/d13/d14 39 2024-03-04T13:43:11.583 INFO:tasks.workunit.client.0.smithi107.stdout:6/525: fsync f3 0 2024-03-04T13:43:11.587 INFO:tasks.workunit.client.0.smithi107.stdout:2/470: rename da/df/d3e/f43 to da/df/fa2 0 2024-03-04T13:43:11.588 INFO:tasks.workunit.client.0.smithi107.stdout:1/488: write d1/d46/d6a/d4e/d60/f78 [346031,62738] 0 2024-03-04T13:43:11.589 INFO:tasks.workunit.client.1.smithi169.stdout:3/949: write d7/de/d2b/d7d/d85/f7f [54674,89250] 0 2024-03-04T13:43:11.590 INFO:tasks.workunit.client.0.smithi107.stdout:9/455: creat d4/d32/f97 x:0 0 0 2024-03-04T13:43:11.595 INFO:tasks.workunit.client.0.smithi107.stdout:4/435: symlink d0/d9/de/d14/dc/l8a 0 2024-03-04T13:43:11.597 INFO:tasks.workunit.client.1.smithi169.stdout:6/988: getdents d0/d3c/d40/df6/dca/d121 0 2024-03-04T13:43:11.598 INFO:tasks.workunit.client.1.smithi169.stdout:3/950: creat d7/de/d2b/d7d/d85/d18/f138 x:0 0 0 2024-03-04T13:43:11.600 INFO:tasks.workunit.client.1.smithi169.stdout:8/998: creat d7/d10/d24/f13a x:0 0 0 2024-03-04T13:43:11.601 INFO:tasks.workunit.client.1.smithi169.stdout:3/951: chown d7/de/d2b/d7d/d85/d13/d5a/d65 2033336894 1 2024-03-04T13:43:11.601 INFO:tasks.workunit.client.1.smithi169.stdout:1/914: link d4/d5/d36/d35/d5b/ca7 d4/d5/d36/d35/d16/d115/c12a 0 2024-03-04T13:43:11.605 INFO:tasks.workunit.client.0.smithi107.stdout:1/489: dwrite d1/d46/d6a/df/f3a [0,4194304] 0 2024-03-04T13:43:11.605 INFO:tasks.workunit.client.0.smithi107.stdout:3/479: rename d1/d1e/d26/f35 to d1/de/d18/d33/d48/d71/d4c/d54/f9d 0 2024-03-04T13:43:11.606 INFO:tasks.workunit.client.1.smithi169.stdout:8/999: mknod d7/d10/d22/d75/d81/da2/dac/c13b 0 2024-03-04T13:43:11.611 INFO:tasks.workunit.client.1.smithi169.stdout:3/952: mkdir d7/de/d2b/d7d/d85/d18/d21/d3f/d86/d139 0 2024-03-04T13:43:11.613 INFO:tasks.workunit.client.0.smithi107.stdout:8/513: getdents d6/da/d25/d5f 0 2024-03-04T13:43:11.614 INFO:tasks.workunit.client.0.smithi107.stdout:6/526: getdents df/d17/d6b/da8 0 2024-03-04T13:43:11.616 INFO:tasks.workunit.client.0.smithi107.stdout:3/480: mkdir d1/d1e/d26/d9e 0 2024-03-04T13:43:11.617 INFO:tasks.workunit.client.0.smithi107.stdout:5/478: rename d5/d8/d14/d2d/l82 to d5/d8/d14/d54/la5 0 2024-03-04T13:43:11.619 INFO:tasks.workunit.client.0.smithi107.stdout:2/471: dwrite f0 [0,4194304] 0 2024-03-04T13:43:11.620 INFO:tasks.workunit.client.0.smithi107.stdout:7/483: dwrite d6/d13/d16/d19/d39/f54 [0,4194304] 0 2024-03-04T13:43:11.622 INFO:tasks.workunit.client.0.smithi107.stdout:7/484: stat d6/d1d/d64/d8e 0 2024-03-04T13:43:11.623 INFO:tasks.workunit.client.0.smithi107.stdout:1/490: mknod d1/d46/d6a/df/d67/d6b/c99 0 2024-03-04T13:43:11.623 INFO:tasks.workunit.client.0.smithi107.stdout:3/481: write d1/de/d18/d1f/d42/f70 [187685,112984] 0 2024-03-04T13:43:11.624 INFO:tasks.workunit.client.0.smithi107.stdout:7/485: chown f0 54818 1 2024-03-04T13:43:11.628 INFO:tasks.workunit.client.0.smithi107.stdout:7/486: chown d6/d13/d16/d1c/f2d 5 1 2024-03-04T13:43:11.628 INFO:tasks.workunit.client.1.smithi169.stdout:0/873: write d5/f16 [412393,43988] 0 2024-03-04T13:43:11.630 INFO:tasks.workunit.client.1.smithi169.stdout:7/987: write d2/de/d3e/d41/d5b/f9f [4351653,54939] 0 2024-03-04T13:43:11.631 INFO:tasks.workunit.client.0.smithi107.stdout:5/479: mkdir d5/d2b/d4b/da6 0 2024-03-04T13:43:11.632 INFO:tasks.workunit.client.0.smithi107.stdout:0/545: dread da/dc/dd/d1e/f9c [0,4194304] 0 2024-03-04T13:43:11.633 INFO:tasks.workunit.client.1.smithi169.stdout:1/915: dwrite d4/d5/d2d/d4e/fc1 [0,4194304] 0 2024-03-04T13:43:11.636 INFO:tasks.workunit.client.1.smithi169.stdout:0/874: readlink d5/d9/d3d/l107 0 2024-03-04T13:43:11.637 INFO:tasks.workunit.client.1.smithi169.stdout:4/962: dread d4/d11/d2e/dcd/dee/d5c/f105 [0,4194304] 0 2024-03-04T13:43:11.638 INFO:tasks.workunit.client.0.smithi107.stdout:7/487: creat d6/d13/d16/d19/fa1 x:0 0 0 2024-03-04T13:43:11.639 INFO:tasks.workunit.client.1.smithi169.stdout:1/916: read - d4/d5/d36/d75/f107 zero size 2024-03-04T13:43:11.640 INFO:tasks.workunit.client.0.smithi107.stdout:3/482: mknod d1/de/d18/d33/d4d/c9f 0 2024-03-04T13:43:11.640 INFO:tasks.workunit.client.1.smithi169.stdout:0/875: write d5/d9/ffd [1112489,22032] 0 2024-03-04T13:43:11.644 INFO:tasks.workunit.client.1.smithi169.stdout:0/876: chown d5/d9/dc/d12/d15/d19/d77/db2 746 1 2024-03-04T13:43:11.645 INFO:tasks.workunit.client.1.smithi169.stdout:0/877: readlink d5/d9/l73 0 2024-03-04T13:43:11.647 INFO:tasks.workunit.client.0.smithi107.stdout:5/480: creat d5/d66/fa7 x:0 0 0 2024-03-04T13:43:11.648 INFO:tasks.workunit.client.1.smithi169.stdout:2/985: sync 2024-03-04T13:43:11.649 INFO:tasks.workunit.client.0.smithi107.stdout:0/546: creat da/dc/dd/d3e/d6e/fbd x:0 0 0 2024-03-04T13:43:11.652 INFO:tasks.workunit.client.1.smithi169.stdout:4/963: fsync d4/d1c/d75/fb3 0 2024-03-04T13:43:11.654 INFO:tasks.workunit.client.0.smithi107.stdout:6/527: dwrite f6 [4194304,4194304] 0 2024-03-04T13:43:11.654 INFO:tasks.workunit.client.1.smithi169.stdout:6/989: write d0/d32/fe8 [433909,97218] 0 2024-03-04T13:43:11.655 INFO:tasks.workunit.client.0.smithi107.stdout:8/514: dwrite d6/d16/d1f/d63/d73/f79 [0,4194304] 0 2024-03-04T13:43:11.656 INFO:tasks.workunit.client.0.smithi107.stdout:9/456: truncate d4/da/d11/f2b 1563605 0 2024-03-04T13:43:11.657 INFO:tasks.workunit.client.1.smithi169.stdout:2/986: mkdir d1/d5/d19/d1e/d14d 0 2024-03-04T13:43:11.658 INFO:tasks.workunit.client.0.smithi107.stdout:0/547: write f4 [3001824,21336] 0 2024-03-04T13:43:11.662 INFO:tasks.workunit.client.0.smithi107.stdout:3/483: fdatasync d1/de/d18/d33/d4d/f5b 0 2024-03-04T13:43:11.664 INFO:tasks.workunit.client.0.smithi107.stdout:5/481: rename d5/d66/f85 to d5/dc/d22/fa8 0 2024-03-04T13:43:11.664 INFO:tasks.workunit.client.0.smithi107.stdout:9/457: write d4/da/d11/d4c/d4d/f54 [2975053,6740] 0 2024-03-04T13:43:11.664 INFO:tasks.workunit.client.1.smithi169.stdout:0/878: rename d5/d9/dc/d12/d15/d19/d61/f23 to d5/d9/dc/d12/f12c 0 2024-03-04T13:43:11.666 INFO:tasks.workunit.client.0.smithi107.stdout:0/548: fdatasync da/dc/dd/d3e/d6e/fa8 0 2024-03-04T13:43:11.667 INFO:tasks.workunit.client.1.smithi169.stdout:1/917: dwrite d4/d5/d36/d75/f107 [0,4194304] 0 2024-03-04T13:43:11.668 INFO:tasks.workunit.client.0.smithi107.stdout:3/484: fsync d1/d1e/f22 0 2024-03-04T13:43:11.668 INFO:tasks.workunit.client.0.smithi107.stdout:6/528: dread - df/d17/d24/d40/fa5 zero size 2024-03-04T13:43:11.669 INFO:tasks.workunit.client.0.smithi107.stdout:1/491: dwrite d1/d46/d6a/df/d33/d5d/d24/f44 [0,4194304] 0 2024-03-04T13:43:11.670 INFO:tasks.workunit.client.1.smithi169.stdout:1/918: readlink d4/d5/l14 0 2024-03-04T13:43:11.672 INFO:tasks.workunit.client.0.smithi107.stdout:7/488: link d6/d13/d16/f8b d6/d1d/d36/d59/fa2 0 2024-03-04T13:43:11.675 INFO:tasks.workunit.client.1.smithi169.stdout:1/919: write d4/d5/d2d/d4e/d71/ffb [4533471,66053] 0 2024-03-04T13:43:11.678 INFO:tasks.workunit.client.1.smithi169.stdout:6/990: mknod d0/d3c/d40/df6/c162 0 2024-03-04T13:43:11.678 INFO:tasks.workunit.client.0.smithi107.stdout:5/482: unlink d5/d8/d2a/l2f 0 2024-03-04T13:43:11.679 INFO:tasks.workunit.client.0.smithi107.stdout:8/515: dread d6/d16/d59/d7c/f8a [0,4194304] 0 2024-03-04T13:43:11.680 INFO:tasks.workunit.client.0.smithi107.stdout:5/483: chown d5/d79/d89/ca0 3006 1 2024-03-04T13:43:11.682 INFO:tasks.workunit.client.0.smithi107.stdout:8/516: truncate d6/da/d25/d5f/f98 4701728 0 2024-03-04T13:43:11.683 INFO:tasks.workunit.client.0.smithi107.stdout:8/517: chown d6/d16 222 1 2024-03-04T13:43:11.684 INFO:tasks.workunit.client.0.smithi107.stdout:8/518: stat d6/d16/d59/d3e/f5a 0 2024-03-04T13:43:11.687 INFO:tasks.workunit.client.1.smithi169.stdout:0/879: dread d5/d9/dc/d12/d15/d19/d69/d110/f97 [0,4194304] 0 2024-03-04T13:43:11.688 INFO:tasks.workunit.client.0.smithi107.stdout:9/458: mknod d4/da/d11/d4c/d4d/c98 0 2024-03-04T13:43:11.689 INFO:tasks.workunit.client.0.smithi107.stdout:1/492: creat d1/f9a x:0 0 0 2024-03-04T13:43:11.690 INFO:tasks.workunit.client.0.smithi107.stdout:3/485: mknod d1/de/d18/d33/d48/ca0 0 2024-03-04T13:43:11.696 INFO:tasks.workunit.client.0.smithi107.stdout:0/549: rename da/dc/dd/d3e/d26/d69/d6b/db3 to da/dc/dd/d3e/d6e/dbe 0 2024-03-04T13:43:11.702 INFO:tasks.workunit.client.0.smithi107.stdout:3/486: dread d1/de/d18/d33/f8f [0,4194304] 0 2024-03-04T13:43:11.705 INFO:tasks.workunit.client.0.smithi107.stdout:3/487: truncate d1/de/d3f/f78 402627 0 2024-03-04T13:43:11.706 INFO:tasks.workunit.client.1.smithi169.stdout:3/953: dwrite d7/de/d2b/d7d/d85/d18/d21/d3f/f82 [0,4194304] 0 2024-03-04T13:43:11.707 INFO:tasks.workunit.client.0.smithi107.stdout:3/488: chown d1/de/d18/d33/d4d/f5b 1054 1 2024-03-04T13:43:11.709 INFO:tasks.workunit.client.0.smithi107.stdout:8/519: chown d6/da/l12 27 1 2024-03-04T13:43:11.709 INFO:tasks.workunit.client.0.smithi107.stdout:3/489: stat d1/de/d18/d33/d4d/d4f/d69/c6d 0 2024-03-04T13:43:11.710 INFO:tasks.workunit.client.1.smithi169.stdout:1/920: unlink d4/d5/d36/d35/d16/d3a/ce8 0 2024-03-04T13:43:11.710 INFO:tasks.workunit.client.1.smithi169.stdout:6/991: dwrite d0/d3c/d40/df6/dca/f140 [4194304,4194304] 0 2024-03-04T13:43:11.712 INFO:tasks.workunit.client.0.smithi107.stdout:7/489: fdatasync d6/d13/d14/f6b 0 2024-03-04T13:43:11.713 INFO:tasks.workunit.client.0.smithi107.stdout:0/550: mkdir da/dc/dd/d1e/dbf 0 2024-03-04T13:43:11.713 INFO:tasks.workunit.client.0.smithi107.stdout:8/520: write d6/da/d25/d5f/f87 [991793,110735] 0 2024-03-04T13:43:11.714 INFO:tasks.workunit.client.1.smithi169.stdout:1/921: dread - d4/d5/d36/d35/d8f/d93/fed zero size 2024-03-04T13:43:11.714 INFO:tasks.workunit.client.0.smithi107.stdout:5/484: creat d5/d8/d2a/d62/fa9 x:0 0 0 2024-03-04T13:43:11.716 INFO:tasks.workunit.client.1.smithi169.stdout:3/954: mknod d7/de/d2b/d7d/d85/d18/d21/d3f/c13a 0 2024-03-04T13:43:11.718 INFO:tasks.workunit.client.1.smithi169.stdout:1/922: fsync d4/d5/d2d/f6c 0 2024-03-04T13:43:11.718 INFO:tasks.workunit.client.0.smithi107.stdout:9/459: symlink d4/da/d11/d20/l99 0 2024-03-04T13:43:11.718 INFO:tasks.workunit.client.0.smithi107.stdout:0/551: fsync da/dc/dd/d3e/d6e/d73/f9a 0 2024-03-04T13:43:11.720 INFO:tasks.workunit.client.0.smithi107.stdout:7/490: dread d6/d13/d16/d19/d38/d62/f9b [0,4194304] 0 2024-03-04T13:43:11.721 INFO:tasks.workunit.client.1.smithi169.stdout:7/988: dwrite d2/f95 [0,4194304] 0 2024-03-04T13:43:11.727 INFO:tasks.workunit.client.0.smithi107.stdout:3/490: unlink d1/de/d18/d33/f96 0 2024-03-04T13:43:11.729 INFO:tasks.workunit.client.0.smithi107.stdout:8/521: symlink d6/d16/d1f/d63/d73/l9b 0 2024-03-04T13:43:11.732 INFO:tasks.workunit.client.0.smithi107.stdout:9/460: dread d4/da/d11/d4c/d51/f5a [0,4194304] 0 2024-03-04T13:43:11.734 INFO:tasks.workunit.client.1.smithi169.stdout:3/955: fdatasync d7/de/d2b/fb4 0 2024-03-04T13:43:11.737 INFO:tasks.workunit.client.1.smithi169.stdout:0/880: dread d5/d9/ff8 [0,4194304] 0 2024-03-04T13:43:11.738 INFO:tasks.workunit.client.1.smithi169.stdout:6/992: dwrite d0/d32/d3a/f21 [0,4194304] 0 2024-03-04T13:43:11.740 INFO:tasks.workunit.client.1.smithi169.stdout:7/989: unlink d2/de/d3e/d41/d69/ccd 0 2024-03-04T13:43:11.742 INFO:tasks.workunit.client.0.smithi107.stdout:6/529: getdents df/d17/d5c 0 2024-03-04T13:43:11.742 INFO:tasks.workunit.client.1.smithi169.stdout:0/881: readlink d5/d9/dc/d12/l84 0 2024-03-04T13:43:11.743 INFO:tasks.workunit.client.0.smithi107.stdout:6/530: chown df/d17/d24/d27 119 1 2024-03-04T13:43:11.743 INFO:tasks.workunit.client.1.smithi169.stdout:6/993: chown d0/d32/d3a/def/fff 7570 1 2024-03-04T13:43:11.745 INFO:tasks.workunit.client.0.smithi107.stdout:0/552: mknod da/dc/dd/d3e/d26/d69/d6b/cc0 0 2024-03-04T13:43:11.753 INFO:tasks.workunit.client.0.smithi107.stdout:3/491: write d1/de/d11/f3e [1541740,45040] 0 2024-03-04T13:43:11.754 INFO:tasks.workunit.client.1.smithi169.stdout:7/990: mkdir d2/de/d152 0 2024-03-04T13:43:11.755 INFO:tasks.workunit.client.0.smithi107.stdout:9/461: dread - d4/da/d11/d65/f58 zero size 2024-03-04T13:43:11.756 INFO:tasks.workunit.client.1.smithi169.stdout:6/994: mkdir d0/d32/d3a/d43/d74/d163 0 2024-03-04T13:43:11.759 INFO:tasks.workunit.client.1.smithi169.stdout:3/956: dwrite d7/d53/f8c [0,4194304] 0 2024-03-04T13:43:11.761 INFO:tasks.workunit.client.0.smithi107.stdout:9/462: write d4/da/d11/d4c/d4d/d72/d7b/f81 [134747,67649] 0 2024-03-04T13:43:11.764 INFO:tasks.workunit.client.1.smithi169.stdout:7/991: unlink d2/d28/d70/ded/f125 0 2024-03-04T13:43:11.765 INFO:tasks.workunit.client.0.smithi107.stdout:7/491: link d6/d13/d16/d1c/f80 d6/d13/d16/d19/fa3 0 2024-03-04T13:43:11.767 INFO:tasks.workunit.client.0.smithi107.stdout:0/553: dwrite da/dc/fbb [0,4194304] 0 2024-03-04T13:43:11.773 INFO:tasks.workunit.client.0.smithi107.stdout:4/436: sync 2024-03-04T13:43:11.773 INFO:tasks.workunit.client.0.smithi107.stdout:0/554: chown da/dc/dd/d3e/d26/d69/la3 133 1 2024-03-04T13:43:11.774 INFO:tasks.workunit.client.0.smithi107.stdout:5/485: getdents d5/d8/d2a/d43 0 2024-03-04T13:43:11.780 INFO:tasks.workunit.client.0.smithi107.stdout:9/463: rename d4/da/d11/d20/d23/d7e/f8b to d4/da/d11/d20/d39/f9a 0 2024-03-04T13:43:11.782 INFO:tasks.workunit.client.1.smithi169.stdout:6/995: rename d0/d3c/d40/d52/ldb to d0/d3c/d40/df6/l164 0 2024-03-04T13:43:11.786 INFO:tasks.workunit.client.1.smithi169.stdout:4/964: write d4/d11/d2e/d39/d61/fc1 [740239,130849] 0 2024-03-04T13:43:11.788 INFO:tasks.workunit.client.1.smithi169.stdout:6/996: read d0/d3c/d40/d52/d98/fd5 [88149,114140] 0 2024-03-04T13:43:11.789 INFO:tasks.workunit.client.1.smithi169.stdout:7/992: link d2/d28/d70/f129 d2/d114/f153 0 2024-03-04T13:43:11.789 INFO:tasks.workunit.client.0.smithi107.stdout:0/555: creat da/dc/dd/d3e/d6e/d73/d78/d86/fc1 x:0 0 0 2024-03-04T13:43:11.791 INFO:tasks.workunit.client.1.smithi169.stdout:7/993: readlink d2/d28/l50 0 2024-03-04T13:43:11.791 INFO:tasks.workunit.client.0.smithi107.stdout:4/437: dwrite d0/d9/de/d14/dc/d21/f6d [0,4194304] 0 2024-03-04T13:43:11.797 INFO:tasks.workunit.client.1.smithi169.stdout:7/994: creat d2/d15/d54/f154 x:0 0 0 2024-03-04T13:43:11.799 INFO:tasks.workunit.client.0.smithi107.stdout:4/438: truncate d0/d9/de/d14/f65 355213 0 2024-03-04T13:43:11.799 INFO:tasks.workunit.client.0.smithi107.stdout:0/556: symlink da/dc/dd/d1e/lc2 0 2024-03-04T13:43:11.799 INFO:tasks.workunit.client.0.smithi107.stdout:6/531: getdents df/d2e/d36/d77 0 2024-03-04T13:43:11.803 INFO:tasks.workunit.client.1.smithi169.stdout:7/995: rmdir d2/de/d3e/d7e 39 2024-03-04T13:43:11.803 INFO:tasks.workunit.client.0.smithi107.stdout:6/532: chown df/d2e/d36/d77/d95 6569470 1 2024-03-04T13:43:11.804 INFO:tasks.workunit.client.0.smithi107.stdout:3/492: link d1/de/d18/d33/c63 d1/de/d3f/ca1 0 2024-03-04T13:43:11.804 INFO:tasks.workunit.client.0.smithi107.stdout:4/439: dread - d0/d9/de/d14/dc/d21/f6c zero size 2024-03-04T13:43:11.805 INFO:tasks.workunit.client.1.smithi169.stdout:4/965: link d4/d11/d3b/la1 d4/d11/d2e/dcd/dd1/de8/l125 0 2024-03-04T13:43:11.808 INFO:tasks.workunit.client.0.smithi107.stdout:7/492: getdents d6/d13/d16/d19/d39/d7a 0 2024-03-04T13:43:11.809 INFO:tasks.workunit.client.1.smithi169.stdout:7/996: fsync d2/d15/d17/f10e 0 2024-03-04T13:43:11.812 INFO:tasks.workunit.client.1.smithi169.stdout:4/966: write d4/d11/d2e/dcd/dee/d5c/f105 [480514,67535] 0 2024-03-04T13:43:11.813 INFO:tasks.workunit.client.0.smithi107.stdout:5/486: dwrite d5/d79/d89/da3/d6a/f6d [0,4194304] 0 2024-03-04T13:43:11.816 INFO:tasks.workunit.client.0.smithi107.stdout:3/493: mknod d1/de/d18/d33/d4d/ca2 0 2024-03-04T13:43:11.817 INFO:tasks.workunit.client.0.smithi107.stdout:9/464: getdents d4/da/d11/d20/d23/d7e/d84 0 2024-03-04T13:43:11.819 INFO:tasks.workunit.client.1.smithi169.stdout:2/987: dwrite d1/d6d/dbc/fd1 [0,4194304] 0 2024-03-04T13:43:11.819 INFO:tasks.workunit.client.0.smithi107.stdout:3/494: write d1/de/d18/d1f/d42/f70 [1295973,78427] 0 2024-03-04T13:43:11.820 INFO:tasks.workunit.client.0.smithi107.stdout:9/465: chown d4/da/f15 34 1 2024-03-04T13:43:11.822 INFO:tasks.workunit.client.0.smithi107.stdout:7/493: creat d6/d13/d14/fa4 x:0 0 0 2024-03-04T13:43:11.823 INFO:tasks.workunit.client.0.smithi107.stdout:9/466: write d4/da/f15 [2545961,18728] 0 2024-03-04T13:43:11.826 INFO:tasks.workunit.client.1.smithi169.stdout:4/967: symlink d4/d1c/d58/d9d/dbc/l126 0 2024-03-04T13:43:11.828 INFO:tasks.workunit.client.0.smithi107.stdout:0/557: rename da/dc/c22 to da/dc/dd/cc3 0 2024-03-04T13:43:11.833 INFO:tasks.workunit.client.0.smithi107.stdout:4/440: dread d0/d9/d10/f4a [0,4194304] 0 2024-03-04T13:43:11.834 INFO:tasks.workunit.client.0.smithi107.stdout:8/522: rmdir d6/d16/d1f/d63/d73 39 2024-03-04T13:43:11.834 INFO:tasks.workunit.client.0.smithi107.stdout:1/493: readlink d1/d46/d6a/d65/l7e 0 2024-03-04T13:43:11.835 INFO:tasks.workunit.client.0.smithi107.stdout:4/441: dread - d0/d9/de/f43 zero size 2024-03-04T13:43:11.841 INFO:tasks.workunit.client.1.smithi169.stdout:1/923: write d4/d5/d2d/d4e/d71/d26/daf/dd0/f109 [228038,69202] 0 2024-03-04T13:43:11.842 INFO:tasks.workunit.client.1.smithi169.stdout:4/968: creat d4/d11/d2e/dcd/dd1/de8/f127 x:0 0 0 2024-03-04T13:43:11.843 INFO:tasks.workunit.client.0.smithi107.stdout:9/467: truncate d4/da/d11/d20/d23/f41 727502 0 2024-03-04T13:43:11.844 INFO:tasks.workunit.client.1.smithi169.stdout:2/988: truncate d1/d5/d22/d45/d4a/f7d 1537752 0 2024-03-04T13:43:11.846 INFO:tasks.workunit.client.1.smithi169.stdout:4/969: write d4/d11/d2e/d39/d48/f65 [3360614,107563] 0 2024-03-04T13:43:11.847 INFO:tasks.workunit.client.0.smithi107.stdout:6/533: rename df/d17/d24/d27 to df/d17/d5c/db1 0 2024-03-04T13:43:11.847 INFO:tasks.workunit.client.0.smithi107.stdout:9/468: rename d4/da/d11 to d4/da/d11/d4c/d89/d9b 22 2024-03-04T13:43:11.849 INFO:tasks.workunit.client.1.smithi169.stdout:4/970: dread - d4/d11/d2e/d6c/f104 zero size 2024-03-04T13:43:11.850 INFO:tasks.workunit.client.1.smithi169.stdout:0/882: truncate d5/d9/d3d/d59/d64/d7f/f9f 590901 0 2024-03-04T13:43:11.851 INFO:tasks.workunit.client.0.smithi107.stdout:2/472: sync 2024-03-04T13:43:11.853 INFO:tasks.workunit.client.1.smithi169.stdout:2/989: mkdir d1/d5/d19/d63/d14e 0 2024-03-04T13:43:11.855 INFO:tasks.workunit.client.0.smithi107.stdout:1/494: readlink d1/l81 0 2024-03-04T13:43:11.856 INFO:tasks.workunit.client.1.smithi169.stdout:4/971: mknod d4/d11/d3b/d67/d7e/d124/da5/d77/dc0/c128 0 2024-03-04T13:43:11.857 INFO:tasks.workunit.client.0.smithi107.stdout:5/487: symlink d5/d8/laa 0 2024-03-04T13:43:11.858 INFO:tasks.workunit.client.1.smithi169.stdout:0/883: creat d5/d9/dc/d12/d15/d19/d69/d110/f12d x:0 0 0 2024-03-04T13:43:11.859 INFO:tasks.workunit.client.1.smithi169.stdout:3/957: write d7/de/d2b/d7d/d85/d13/f12a [996246,14283] 0 2024-03-04T13:43:11.862 INFO:tasks.workunit.client.1.smithi169.stdout:2/990: fsync d1/d5/d19/f47 0 2024-03-04T13:43:11.864 INFO:tasks.workunit.client.0.smithi107.stdout:4/442: write d0/d9/f6b [1823536,67384] 0 2024-03-04T13:43:11.867 INFO:tasks.workunit.client.0.smithi107.stdout:3/495: mknod d1/de/d18/ca3 0 2024-03-04T13:43:11.867 INFO:tasks.workunit.client.1.smithi169.stdout:0/884: rmdir d5/d9/dc/d12/d15/d19/d77/d94 39 2024-03-04T13:43:11.869 INFO:tasks.workunit.client.0.smithi107.stdout:7/494: mkdir d6/d1d/d64/d8e/d66/d71/da5 0 2024-03-04T13:43:11.871 INFO:tasks.workunit.client.0.smithi107.stdout:7/495: chown d6/d13/d16/d19 37831611 1 2024-03-04T13:43:11.872 INFO:tasks.workunit.client.0.smithi107.stdout:6/534: creat df/d17/d5c/db1/fb2 x:0 0 0 2024-03-04T13:43:11.873 INFO:tasks.workunit.client.1.smithi169.stdout:3/958: dread - d7/de/d2b/d7d/d85/d13/d1a/d2d/d56/d5e/ffe zero size 2024-03-04T13:43:11.877 INFO:tasks.workunit.client.1.smithi169.stdout:6/997: dwrite d0/d32/d3e/d122/fd4 [0,4194304] 0 2024-03-04T13:43:11.879 INFO:tasks.workunit.client.0.smithi107.stdout:6/535: truncate df/d17/d24/d40/d63/da3/d7e/fa9 928820 0 2024-03-04T13:43:11.880 INFO:tasks.workunit.client.0.smithi107.stdout:3/496: dread d1/de/d18/d33/f34 [0,4194304] 0 2024-03-04T13:43:11.880 INFO:tasks.workunit.client.1.smithi169.stdout:2/991: rename d1/d5/d19/d3c/db8/de1/d64/fb1 to d1/d5/d22/d45/d4a/d11f/f14f 0 2024-03-04T13:43:11.881 INFO:tasks.workunit.client.1.smithi169.stdout:6/998: read d0/d32/f81 [95020,8059] 0 2024-03-04T13:43:11.886 INFO:tasks.workunit.client.0.smithi107.stdout:8/523: mknod d6/d16/d1f/c9c 0 2024-03-04T13:43:11.888 INFO:tasks.workunit.client.1.smithi169.stdout:7/997: truncate d2/de/d37/fd0 2595645 0 2024-03-04T13:43:11.889 INFO:tasks.workunit.client.1.smithi169.stdout:4/972: dread d4/f7 [0,4194304] 0 2024-03-04T13:43:11.892 INFO:tasks.workunit.client.1.smithi169.stdout:4/973: chown d4/d11/d2e/l55 4 1 2024-03-04T13:43:11.893 INFO:tasks.workunit.client.0.smithi107.stdout:5/488: dread d5/d8/d2a/f39 [0,4194304] 0 2024-03-04T13:43:11.894 INFO:tasks.workunit.client.1.smithi169.stdout:4/974: stat d4/d1c/d58/d9d/fd0 0 2024-03-04T13:43:11.894 INFO:tasks.workunit.client.1.smithi169.stdout:1/924: write d4/d5/d36/d35/d8f/d93/f9e [894375,117381] 0 2024-03-04T13:43:11.895 INFO:tasks.workunit.client.1.smithi169.stdout:3/959: rename d7/de/d2b/d7d/d85/d18/d21/d3f/l42 to d7/db8/l13b 0 2024-03-04T13:43:11.896 INFO:tasks.workunit.client.0.smithi107.stdout:6/536: dwrite df/d17/d5c/f67 [0,4194304] 0 2024-03-04T13:43:11.901 INFO:tasks.workunit.client.1.smithi169.stdout:7/998: dread d2/d15/d17/d6f/f9b [0,4194304] 0 2024-03-04T13:43:11.901 INFO:tasks.workunit.client.0.smithi107.stdout:7/496: symlink d6/d1d/d36/d59/la6 0 2024-03-04T13:43:11.902 INFO:tasks.workunit.client.0.smithi107.stdout:9/469: getdents d4/da/d11/d20/d64/d77 0 2024-03-04T13:43:11.902 INFO:tasks.workunit.client.1.smithi169.stdout:3/960: write d7/de/d2b/d7d/d85/d13/d1a/d2d/d56/d5e/d67/ff2 [673543,54572] 0 2024-03-04T13:43:11.909 INFO:tasks.workunit.client.0.smithi107.stdout:3/497: rename d1/de/d18/d33/d4d/ca2 to d1/de/ca4 0 2024-03-04T13:43:11.914 INFO:tasks.workunit.client.1.smithi169.stdout:4/975: read d4/d11/d2e/dcd/dee/f45 [4070262,71485] 0 2024-03-04T13:43:11.916 INFO:tasks.workunit.client.1.smithi169.stdout:0/885: creat d5/d9/d3d/d5d/de4/d98/f12e x:0 0 0 2024-03-04T13:43:11.917 INFO:tasks.workunit.client.1.smithi169.stdout:0/886: read - d5/d4c/f10d zero size 2024-03-04T13:43:11.918 INFO:tasks.workunit.client.0.smithi107.stdout:8/524: unlink d6/da/d25/d5f/l68 0 2024-03-04T13:43:11.918 INFO:tasks.workunit.client.0.smithi107.stdout:4/443: dread d0/d9/de/f41 [0,4194304] 0 2024-03-04T13:43:11.920 INFO:tasks.workunit.client.1.smithi169.stdout:0/887: dread - d5/d9/dc/d12/d15/d19/d61/f10e zero size 2024-03-04T13:43:11.921 INFO:tasks.workunit.client.0.smithi107.stdout:4/444: fsync d0/d9/f11 0 2024-03-04T13:43:11.924 INFO:tasks.workunit.client.0.smithi107.stdout:9/470: dwrite d4/da/d11/d4c/d4d/d72/d7b/f83 [0,4194304] 0 2024-03-04T13:43:11.924 INFO:tasks.workunit.client.0.smithi107.stdout:0/558: dwrite da/dc/dd/d3e/d26/f71 [0,4194304] 0 2024-03-04T13:43:11.925 INFO:tasks.workunit.client.1.smithi169.stdout:3/961: dwrite d7/de/d2b/d7d/d85/d18/d21/d3f/d86/f122 [0,4194304] 0 2024-03-04T13:43:11.925 INFO:tasks.workunit.client.0.smithi107.stdout:4/445: chown d0/d9/d10/d6f 56 1 2024-03-04T13:43:11.926 INFO:tasks.workunit.client.0.smithi107.stdout:4/446: read - d0/d9/d10/f7a zero size 2024-03-04T13:43:11.930 INFO:tasks.workunit.client.0.smithi107.stdout:0/559: write da/dc/dd/d3e/d6e/fa8 [712978,113411] 0 2024-03-04T13:43:11.930 INFO:tasks.workunit.client.1.smithi169.stdout:2/992: fsync d1/d5/d22/d45/d4a/f92 0 2024-03-04T13:43:11.931 INFO:tasks.workunit.client.0.smithi107.stdout:2/473: write da/df/f24 [16514,50212] 0 2024-03-04T13:43:11.933 INFO:tasks.workunit.client.1.smithi169.stdout:6/999: creat d0/d32/d3a/d43/d74/d163/f165 x:0 0 0 2024-03-04T13:43:11.936 INFO:tasks.workunit.client.0.smithi107.stdout:3/498: mknod d1/de/d18/d33/d4d/d4f/ca5 0 2024-03-04T13:43:11.937 INFO:tasks.workunit.client.0.smithi107.stdout:3/499: fsync d1/d2d/f51 0 2024-03-04T13:43:11.938 INFO:tasks.workunit.client.1.smithi169.stdout:4/976: dwrite d4/d11/d3b/d67/d7e/f9c [0,4194304] 0 2024-03-04T13:43:11.941 INFO:tasks.workunit.client.1.smithi169.stdout:7/999: dread d2/d28/f4b [0,4194304] 0 2024-03-04T13:43:11.949 INFO:tasks.workunit.client.1.smithi169.stdout:1/925: creat d4/d5/d36/d35/d16/f12b x:0 0 0 2024-03-04T13:43:11.950 INFO:tasks.workunit.client.0.smithi107.stdout:7/497: mknod d6/d13/ca7 0 2024-03-04T13:43:11.957 INFO:tasks.workunit.client.1.smithi169.stdout:2/993: creat d1/d5/d12e/f150 x:0 0 0 2024-03-04T13:43:11.959 INFO:tasks.workunit.client.1.smithi169.stdout:2/994: chown d1/d5/d19/d3c 4032 1 2024-03-04T13:43:11.960 INFO:tasks.workunit.client.0.smithi107.stdout:1/495: symlink d1/d10/d16/d47/d71/d82/l9b 0 2024-03-04T13:43:11.960 INFO:tasks.workunit.client.1.smithi169.stdout:1/926: chown d4/d5/d36/d35/c1f 3620 1 2024-03-04T13:43:11.961 INFO:tasks.workunit.client.1.smithi169.stdout:4/977: mknod d4/d1c/d75/d81/c129 0 2024-03-04T13:43:11.962 INFO:tasks.workunit.client.0.smithi107.stdout:1/496: stat d1/d10/c49 0 2024-03-04T13:43:11.962 INFO:tasks.workunit.client.0.smithi107.stdout:4/447: dread d0/d9/de/d14/dc/d21/f6d [0,4194304] 0 2024-03-04T13:43:11.963 INFO:tasks.workunit.client.1.smithi169.stdout:1/927: fsync d4/d5/d2d/d4e/d71/d82/df7/f105 0 2024-03-04T13:43:11.963 INFO:tasks.workunit.client.0.smithi107.stdout:5/489: link d5/d8/d14/d2d/f6b d5/dc/fab 0 2024-03-04T13:43:11.964 INFO:tasks.workunit.client.1.smithi169.stdout:0/888: rmdir d5/d9/d3d/d59/d64/d7f/d128 0 2024-03-04T13:43:11.966 INFO:tasks.workunit.client.0.smithi107.stdout:9/471: rename d4/da/d11/d68 to d4/da/d11/d20/d2f/d9c 0 2024-03-04T13:43:11.966 INFO:tasks.workunit.client.0.smithi107.stdout:5/490: read d5/dc/f41 [2666362,25390] 0 2024-03-04T13:43:11.968 INFO:tasks.workunit.client.1.smithi169.stdout:0/889: dread d5/d9/d3d/d59/d64/d7f/fcc [0,4194304] 0 2024-03-04T13:43:11.971 INFO:tasks.workunit.client.1.smithi169.stdout:2/995: mknod d1/d5/d19/d1e/d14d/c151 0 2024-03-04T13:43:11.973 INFO:tasks.workunit.client.1.smithi169.stdout:2/996: fdatasync d1/d5/d22/d45/d4a/de7/f137 0 2024-03-04T13:43:11.981 INFO:tasks.workunit.client.1.smithi169.stdout:0/890: mknod d5/d9/dc/c12f 0 2024-03-04T13:43:11.982 INFO:tasks.workunit.client.1.smithi169.stdout:1/928: link d4/d5/d36/d35/d16/d3a/c8c d4/d5/d36/d75/c12c 0 2024-03-04T13:43:11.983 INFO:tasks.workunit.client.1.smithi169.stdout:1/929: rmdir d4/d5/d2d 39 2024-03-04T13:43:11.989 INFO:tasks.workunit.client.1.smithi169.stdout:1/930: chown d4/d5/d2d/d4e/d71/d82/l11f 196 1 2024-03-04T13:43:11.990 INFO:tasks.workunit.client.0.smithi107.stdout:4/448: creat d0/d9/de/d14/d66/f8b x:0 0 0 2024-03-04T13:43:11.992 INFO:tasks.workunit.client.0.smithi107.stdout:4/449: fdatasync d0/d9/de/d14/dc/f2d 0 2024-03-04T13:43:11.993 INFO:tasks.workunit.client.0.smithi107.stdout:0/560: creat da/dc/fc4 x:0 0 0 2024-03-04T13:43:11.994 INFO:tasks.workunit.client.1.smithi169.stdout:1/931: truncate d4/d5/d36/d35/d16/d3a/f122 726468 0 2024-03-04T13:43:11.995 INFO:tasks.workunit.client.0.smithi107.stdout:5/491: unlink d5/d79/d89/da3/d68/c94 0 2024-03-04T13:43:11.996 INFO:tasks.workunit.client.0.smithi107.stdout:2/474: link da/d44/l82 da/d38/la3 0 2024-03-04T13:43:11.997 INFO:tasks.workunit.client.1.smithi169.stdout:1/932: readlink d4/d5/d2d/d4e/l112 0 2024-03-04T13:43:11.998 INFO:tasks.workunit.client.0.smithi107.stdout:0/561: read da/dc/dd/d3e/d6e/d73/d78/d86/f90 [18613,107004] 0 2024-03-04T13:43:11.998 INFO:tasks.workunit.client.0.smithi107.stdout:1/497: symlink d1/l9c 0 2024-03-04T13:43:12.003 INFO:tasks.workunit.client.1.smithi169.stdout:3/962: dwrite d7/de/d2b/d7d/d85/d18/d21/d3f/d68/fb2 [0,4194304] 0 2024-03-04T13:43:12.003 INFO:tasks.workunit.client.0.smithi107.stdout:4/450: creat d0/d9/d10/f8c x:0 0 0 2024-03-04T13:43:12.004 INFO:tasks.workunit.client.0.smithi107.stdout:4/451: fsync d0/d2f/f72 0 2024-03-04T13:43:12.005 INFO:tasks.workunit.client.1.smithi169.stdout:1/933: chown d4/d5/d36/d35/d70/d78/fd4 1231 1 2024-03-04T13:43:12.006 INFO:tasks.workunit.client.0.smithi107.stdout:2/475: creat da/d49/fa4 x:0 0 0 2024-03-04T13:43:12.006 INFO:tasks.workunit.client.0.smithi107.stdout:4/452: chown d0/d9/de/d14/dc/d21/c56 820 1 2024-03-04T13:43:12.006 INFO:tasks.workunit.client.0.smithi107.stdout:5/492: mkdir d5/d8/d2a/d43/dac 0 2024-03-04T13:43:12.008 INFO:tasks.workunit.client.0.smithi107.stdout:0/562: symlink da/dad/lc5 0 2024-03-04T13:43:12.011 INFO:tasks.workunit.client.0.smithi107.stdout:1/498: fdatasync d1/d46/d6a/df/d33/d5d/d24/f44 0 2024-03-04T13:43:12.013 INFO:tasks.workunit.client.1.smithi169.stdout:1/934: unlink d4/d5/d2d/d4e/c10e 0 2024-03-04T13:43:12.014 INFO:tasks.workunit.client.0.smithi107.stdout:4/453: dread d0/d9/de/d14/dc/f69 [0,4194304] 0 2024-03-04T13:43:12.015 INFO:tasks.workunit.client.1.smithi169.stdout:3/963: rename d7/de/d2b/d7d/d85/d13/d1a/d78 to d7/de/d2b/d7d/d85/d18/d21/d103/d13c 0 2024-03-04T13:43:12.018 INFO:tasks.workunit.client.0.smithi107.stdout:4/454: readlink d0/l44 0 2024-03-04T13:43:12.018 INFO:tasks.workunit.client.0.smithi107.stdout:8/525: truncate d6/f5b 1161518 0 2024-03-04T13:43:12.019 INFO:tasks.workunit.client.1.smithi169.stdout:1/935: unlink d4/d5/d36/f59 0 2024-03-04T13:43:12.021 INFO:tasks.workunit.client.0.smithi107.stdout:2/476: getdents da/df/d27 0 2024-03-04T13:43:12.022 INFO:tasks.workunit.client.1.smithi169.stdout:1/936: stat d4/d1d/l55 0 2024-03-04T13:43:12.023 INFO:tasks.workunit.client.1.smithi169.stdout:1/937: stat d4/d5/d36/f67 0 2024-03-04T13:43:12.026 INFO:tasks.workunit.client.0.smithi107.stdout:4/455: write d0/d9/de/d14/dc/f69 [366450,6487] 0 2024-03-04T13:43:12.026 INFO:tasks.workunit.client.0.smithi107.stdout:3/500: dwrite d1/de/f32 [4194304,4194304] 0 2024-03-04T13:43:12.027 INFO:tasks.workunit.client.1.smithi169.stdout:3/964: creat d7/de/d2b/d7d/d85/d18/d21/f13d x:0 0 0 2024-03-04T13:43:12.030 INFO:tasks.workunit.client.0.smithi107.stdout:1/499: dread d1/d46/d6a/df/d33/d5d/d24/f62 [0,4194304] 0 2024-03-04T13:43:12.032 INFO:tasks.workunit.client.0.smithi107.stdout:3/501: stat d1/d1e 0 2024-03-04T13:43:12.035 INFO:tasks.workunit.client.1.smithi169.stdout:4/978: truncate d4/d11/d3b/d67/d7e/f9c 3206217 0 2024-03-04T13:43:12.035 INFO:tasks.workunit.client.0.smithi107.stdout:2/477: symlink da/d44/d8b/d98/la5 0 2024-03-04T13:43:12.035 INFO:tasks.workunit.client.0.smithi107.stdout:4/456: symlink d0/d9/d18/d29/l8d 0 2024-03-04T13:43:12.036 INFO:tasks.workunit.client.0.smithi107.stdout:3/502: truncate d1/de/d18/d33/d4d/f90 226659 0 2024-03-04T13:43:12.039 INFO:tasks.workunit.client.1.smithi169.stdout:2/997: write d1/f50 [431623,46734] 0 2024-03-04T13:43:12.040 INFO:tasks.workunit.client.0.smithi107.stdout:1/500: symlink d1/d10/d16/d47/d71/l9d 0 2024-03-04T13:43:12.041 INFO:tasks.workunit.client.0.smithi107.stdout:4/457: mknod d0/d9/d10/c8e 0 2024-03-04T13:43:12.042 INFO:tasks.workunit.client.1.smithi169.stdout:2/998: chown d1/d5/d22/d45/d4a/fe3 1915305 1 2024-03-04T13:43:12.045 INFO:tasks.workunit.client.0.smithi107.stdout:2/478: getdents da/d6c 0 2024-03-04T13:43:12.045 INFO:tasks.workunit.client.1.smithi169.stdout:2/999: fsync d1/d5/d19/d3c/db8/de1/d64/f95 0 2024-03-04T13:43:12.046 INFO:tasks.workunit.client.0.smithi107.stdout:5/493: dread d5/dc/d22/fa8 [0,4194304] 0 2024-03-04T13:43:12.047 INFO:tasks.workunit.client.0.smithi107.stdout:4/458: creat d0/d9/de/d14/d66/d7b/f8f x:0 0 0 2024-03-04T13:43:12.050 INFO:tasks.workunit.client.0.smithi107.stdout:7/498: write d6/d13/d16/d19/d38/f47 [4514935,55377] 0 2024-03-04T13:43:12.050 INFO:tasks.workunit.client.0.smithi107.stdout:4/459: creat d0/d30/f90 x:0 0 0 2024-03-04T13:43:12.054 INFO:tasks.workunit.client.0.smithi107.stdout:0/563: dread da/dc/dd/d3e/f11 [0,4194304] 0 2024-03-04T13:43:12.055 INFO:tasks.workunit.client.1.smithi169.stdout:0/891: dwrite d5/d9/dc/d12/d15/f74 [0,4194304] 0 2024-03-04T13:43:12.055 INFO:tasks.workunit.client.0.smithi107.stdout:7/499: dread - d6/d13/d16/d1c/f79 zero size 2024-03-04T13:43:12.058 INFO:tasks.workunit.client.0.smithi107.stdout:2/479: link da/df/d73/d7a/d9a/d68/l6d da/d49/d5c/la6 0 2024-03-04T13:43:12.058 INFO:tasks.workunit.client.1.smithi169.stdout:4/979: dread d4/d11/d2e/dcd/dee/d5c/d86/f99 [0,4194304] 0 2024-03-04T13:43:12.063 INFO:tasks.workunit.client.0.smithi107.stdout:0/564: chown da/dc/dd/d3e/d26/d69/d6b/l83 202674 1 2024-03-04T13:43:12.066 INFO:tasks.workunit.client.0.smithi107.stdout:9/472: dwrite d4/f21 [0,4194304] 0 2024-03-04T13:43:12.068 INFO:tasks.workunit.client.0.smithi107.stdout:1/501: dwrite d1/d46/d6a/df/d67/f69 [0,4194304] 0 2024-03-04T13:43:12.069 INFO:tasks.workunit.client.1.smithi169.stdout:0/892: link d5/d9/dc/d12/l4b d5/d9/d3d/d59/l130 0 2024-03-04T13:43:12.072 INFO:tasks.workunit.client.0.smithi107.stdout:9/473: chown d4/da/d11/d4c/d51/f5a 0 1 2024-03-04T13:43:12.072 INFO:tasks.workunit.client.0.smithi107.stdout:7/500: mknod d6/d1d/d64/d8e/ca8 0 2024-03-04T13:43:12.073 INFO:tasks.workunit.client.1.smithi169.stdout:1/938: truncate d4/d5/d36/d75/dba/f101 80584 0 2024-03-04T13:43:12.075 INFO:tasks.workunit.client.0.smithi107.stdout:0/565: mknod da/dc/dd/d3e/d26/d88/cc6 0 2024-03-04T13:43:12.076 INFO:tasks.workunit.client.1.smithi169.stdout:1/939: stat d4/d5/d36/d35/d16/d81/db4/dd7 0 2024-03-04T13:43:12.079 INFO:tasks.workunit.client.1.smithi169.stdout:4/980: dwrite d4/f7 [0,4194304] 0 2024-03-04T13:43:12.080 INFO:tasks.workunit.client.0.smithi107.stdout:9/474: fsync d4/da/d11/d20/d2f/f85 0 2024-03-04T13:43:12.081 INFO:tasks.workunit.client.0.smithi107.stdout:4/460: dwrite d0/d9/d1e/f20 [0,4194304] 0 2024-03-04T13:43:12.085 INFO:tasks.workunit.client.0.smithi107.stdout:5/494: write d5/d8/d14/d2d/f6b [2021387,81184] 0 2024-03-04T13:43:12.088 INFO:tasks.workunit.client.0.smithi107.stdout:7/501: rename d6/d1d/d64/d8e/d66 to d6/d1d/d64/d9a/da9 0 2024-03-04T13:43:12.088 INFO:tasks.workunit.client.0.smithi107.stdout:2/480: dwrite da/df/f24 [0,4194304] 0 2024-03-04T13:43:12.090 INFO:tasks.workunit.client.0.smithi107.stdout:2/481: chown da/df/d73 18 1 2024-03-04T13:43:12.092 INFO:tasks.workunit.client.1.smithi169.stdout:1/940: creat d4/d5/d36/d35/d16/d115/f12d x:0 0 0 2024-03-04T13:43:12.095 INFO:tasks.workunit.client.0.smithi107.stdout:9/475: unlink d4/da/d11/d4c/d4d/d72/d7b/c91 0 2024-03-04T13:43:12.096 INFO:tasks.workunit.client.0.smithi107.stdout:1/502: dwrite d1/d46/d6a/df/d67/d6b/f84 [0,4194304] 0 2024-03-04T13:43:12.105 INFO:tasks.workunit.client.0.smithi107.stdout:0/566: symlink da/dc/dd/d3e/d6e/d73/d78/lc7 0 2024-03-04T13:43:12.111 INFO:tasks.workunit.client.0.smithi107.stdout:9/476: creat d4/da/d11/d4c/d51/f9d x:0 0 0 2024-03-04T13:43:12.112 INFO:tasks.workunit.client.0.smithi107.stdout:1/503: creat d1/d46/d6a/df/d67/d6b/f9e x:0 0 0 2024-03-04T13:43:12.113 INFO:tasks.workunit.client.0.smithi107.stdout:7/502: dwrite d6/d13/d16/d1c/f79 [0,4194304] 0 2024-03-04T13:43:12.117 INFO:tasks.workunit.client.0.smithi107.stdout:7/503: write d6/d1d/d36/d59/f65 [308657,52825] 0 2024-03-04T13:43:12.118 INFO:tasks.workunit.client.0.smithi107.stdout:5/495: creat d5/fad x:0 0 0 2024-03-04T13:43:12.120 INFO:tasks.workunit.client.0.smithi107.stdout:9/477: mknod d4/da/d11/d4c/d4d/c9e 0 2024-03-04T13:43:12.122 INFO:tasks.workunit.client.0.smithi107.stdout:5/496: stat d5/d79/d89/da3/c9c 0 2024-03-04T13:43:12.122 INFO:tasks.workunit.client.0.smithi107.stdout:9/478: write d4/da/d11/d20/d23/f6c [264902,9833] 0 2024-03-04T13:43:12.123 INFO:tasks.workunit.client.1.smithi169.stdout:0/893: dwrite d5/d9/d3d/d59/d64/d7f/fcc [0,4194304] 0 2024-03-04T13:43:12.128 INFO:tasks.workunit.client.0.smithi107.stdout:5/497: write d5/dc/df/f2e [2011953,62908] 0 2024-03-04T13:43:12.132 INFO:tasks.workunit.client.0.smithi107.stdout:5/498: fdatasync d5/f73 0 2024-03-04T13:43:12.135 INFO:tasks.workunit.client.0.smithi107.stdout:1/504: dwrite d1/d10/d16/f8b [0,4194304] 0 2024-03-04T13:43:12.148 INFO:tasks.workunit.client.0.smithi107.stdout:1/505: mkdir d1/d46/d9f 0 2024-03-04T13:43:12.150 INFO:tasks.workunit.client.1.smithi169.stdout:4/981: dread d4/d11/d2e/d39/f68 [0,4194304] 0 2024-03-04T13:43:12.152 INFO:tasks.workunit.client.0.smithi107.stdout:5/499: rename d5/d8/d14/d54/la5 to d5/d79/d89/da3/d68/lae 0 2024-03-04T13:43:12.167 INFO:tasks.workunit.client.0.smithi107.stdout:0/567: dread da/dc/dd/d3e/d26/d88/faa [0,4194304] 0 2024-03-04T13:43:12.168 INFO:tasks.workunit.client.0.smithi107.stdout:0/568: stat da/dc/d34/c4b 0 2024-03-04T13:43:12.169 INFO:tasks.workunit.client.0.smithi107.stdout:5/500: rename d5/d2b/d4b/c8a to d5/d66/d96/caf 0 2024-03-04T13:43:12.175 INFO:tasks.workunit.client.0.smithi107.stdout:7/504: dread d6/d13/d16/f58 [0,4194304] 0 2024-03-04T13:43:12.176 INFO:tasks.workunit.client.0.smithi107.stdout:5/501: mkdir d5/d8/da/db0 0 2024-03-04T13:43:12.179 INFO:tasks.workunit.client.0.smithi107.stdout:7/505: dread - d6/d13/d16/d19/d38/d2c/f87 zero size 2024-03-04T13:43:12.180 INFO:tasks.workunit.client.0.smithi107.stdout:9/479: truncate d4/da/f4a 306964 0 2024-03-04T13:43:12.180 INFO:tasks.workunit.client.1.smithi169.stdout:1/941: mkdir d4/d5/d36/d35/d8f/d93/d12e 0 2024-03-04T13:43:12.183 INFO:tasks.workunit.client.0.smithi107.stdout:7/506: dread - d6/d13/d16/d1c/f7d zero size 2024-03-04T13:43:12.183 INFO:tasks.workunit.client.1.smithi169.stdout:1/942: chown d4/d5/d36/d35/d8f/d93/f110 1 1 2024-03-04T13:43:12.185 INFO:tasks.workunit.client.1.smithi169.stdout:0/894: rename d5/d9/d3d/d5d/de4/d98/dc1/dc8/le1 to d5/d4c/d65/d95/dba/l131 0 2024-03-04T13:43:12.186 INFO:tasks.workunit.client.0.smithi107.stdout:0/569: dwrite da/dc/fb9 [0,4194304] 0 2024-03-04T13:43:12.189 INFO:tasks.workunit.client.0.smithi107.stdout:7/507: creat d6/d13/d16/d19/faa x:0 0 0 2024-03-04T13:43:12.189 INFO:tasks.workunit.client.1.smithi169.stdout:1/943: mknod d4/d5/d36/d35/d16/d81/db4/c12f 0 2024-03-04T13:43:12.190 INFO:tasks.workunit.client.1.smithi169.stdout:4/982: mkdir d4/d1c/d75/d12a 0 2024-03-04T13:43:12.191 INFO:tasks.workunit.client.0.smithi107.stdout:9/480: creat d4/da/d11/d4c/d89/f9f x:0 0 0 2024-03-04T13:43:12.194 INFO:tasks.workunit.client.1.smithi169.stdout:4/983: creat d4/d11/d2e/dcd/dee/d5c/d86/f12b x:0 0 0 2024-03-04T13:43:12.195 INFO:tasks.workunit.client.0.smithi107.stdout:0/570: dread da/f2a [0,4194304] 0 2024-03-04T13:43:12.199 INFO:tasks.workunit.client.1.smithi169.stdout:4/984: read d4/d11/d2e/dcd/dee/d5c/f8c [3005241,59069] 0 2024-03-04T13:43:12.201 INFO:tasks.workunit.client.1.smithi169.stdout:1/944: rename d4/d5/d2d/d4e/d71/d82/fad to d4/d5/d2d/f130 0 2024-03-04T13:43:12.202 INFO:tasks.workunit.client.1.smithi169.stdout:4/985: chown d4/d11/d2e/d6c/cc4 1043432769 1 2024-03-04T13:43:12.205 INFO:tasks.workunit.client.0.smithi107.stdout:1/506: dwrite d1/d46/d6a/d4e/d60/f53 [0,4194304] 0 2024-03-04T13:43:12.212 INFO:tasks.workunit.client.0.smithi107.stdout:9/481: symlink d4/da/d11/d20/d2f/d9c/d92/la0 0 2024-03-04T13:43:12.212 INFO:tasks.workunit.client.1.smithi169.stdout:4/986: mkdir d4/d11/d3b/d67/d98/d12c 0 2024-03-04T13:43:12.217 INFO:tasks.workunit.client.0.smithi107.stdout:0/571: getdents da/dc/dd/d1e/dbf 0 2024-03-04T13:43:12.217 INFO:tasks.workunit.client.0.smithi107.stdout:5/502: dread d5/f6 [0,4194304] 0 2024-03-04T13:43:12.217 INFO:tasks.workunit.client.1.smithi169.stdout:1/945: getdents d4/d5/d36/d35 0 2024-03-04T13:43:12.219 INFO:tasks.workunit.client.0.smithi107.stdout:9/482: symlink d4/da/d11/d4c/d51/la1 0 2024-03-04T13:43:12.220 INFO:tasks.workunit.client.1.smithi169.stdout:4/987: creat d4/d11/d2e/dcd/dee/d5c/d86/db8/df9/f12d x:0 0 0 2024-03-04T13:43:12.224 INFO:tasks.workunit.client.1.smithi169.stdout:1/946: creat d4/d5/d36/d35/d16/d3a/f131 x:0 0 0 2024-03-04T13:43:12.225 INFO:tasks.workunit.client.0.smithi107.stdout:5/503: mkdir d5/d66/d96/db1 0 2024-03-04T13:43:12.230 INFO:tasks.workunit.client.0.smithi107.stdout:9/483: dread d4/da/d11/d4c/d4d/f54 [0,4194304] 0 2024-03-04T13:43:12.233 INFO:tasks.workunit.client.0.smithi107.stdout:5/504: unlink d5/d8/d14/d54/fa4 0 2024-03-04T13:43:12.234 INFO:tasks.workunit.client.1.smithi169.stdout:1/947: mkdir d4/d5/d36/d35/d16/d81/db4/d104/d132 0 2024-03-04T13:43:12.234 INFO:tasks.workunit.client.0.smithi107.stdout:7/508: truncate d6/d1d/d36/f5b 1101009 0 2024-03-04T13:43:12.235 INFO:tasks.workunit.client.0.smithi107.stdout:1/507: link d1/c29 d1/d10/d16/ca0 0 2024-03-04T13:43:12.238 INFO:tasks.workunit.client.1.smithi169.stdout:1/948: symlink d4/d5/d36/d35/d16/d81/db4/dd7/l133 0 2024-03-04T13:43:12.239 INFO:tasks.workunit.client.0.smithi107.stdout:9/484: creat d4/da/d11/d4c/d89/fa2 x:0 0 0 2024-03-04T13:43:12.239 INFO:tasks.workunit.client.0.smithi107.stdout:5/505: symlink d5/dc/d22/lb2 0 2024-03-04T13:43:12.241 INFO:tasks.workunit.client.1.smithi169.stdout:1/949: truncate d4/d5/d36/d35/d16/d3a/f131 976477 0 2024-03-04T13:43:12.242 INFO:tasks.workunit.client.0.smithi107.stdout:5/506: truncate d5/d8/f7c 875655 0 2024-03-04T13:43:12.242 INFO:tasks.workunit.client.0.smithi107.stdout:7/509: rmdir d6/d1d/d64/d8e 39 2024-03-04T13:43:12.242 INFO:tasks.workunit.client.0.smithi107.stdout:0/572: rename da/dc/l30 to da/dc/lc8 0 2024-03-04T13:43:12.244 INFO:tasks.workunit.client.0.smithi107.stdout:7/510: chown d6/d13/d16/d1c/f80 1 1 2024-03-04T13:43:12.245 INFO:tasks.workunit.client.0.smithi107.stdout:1/508: link d1/d10/d16/d47/d71/l9d d1/d46/d6a/d65/la1 0 2024-03-04T13:43:12.246 INFO:tasks.workunit.client.0.smithi107.stdout:5/507: fdatasync d5/dc/df/f12 0 2024-03-04T13:43:12.249 INFO:tasks.workunit.client.0.smithi107.stdout:0/573: getdents da/dc/d34 0 2024-03-04T13:43:12.250 INFO:tasks.workunit.client.0.smithi107.stdout:5/508: creat d5/d79/d89/fb3 x:0 0 0 2024-03-04T13:43:12.252 INFO:tasks.workunit.client.0.smithi107.stdout:0/574: mkdir da/dc/dd/dc9 0 2024-03-04T13:43:12.253 INFO:tasks.workunit.client.0.smithi107.stdout:1/509: creat d1/d10/d16/d1b/fa2 x:0 0 0 2024-03-04T13:43:12.256 INFO:tasks.workunit.client.0.smithi107.stdout:1/510: dread - d1/d46/d6a/d75/f95 zero size 2024-03-04T13:43:12.259 INFO:tasks.workunit.client.0.smithi107.stdout:0/575: rmdir da/dc/dd/d3e/d6e/dbe 0 2024-03-04T13:43:12.263 INFO:tasks.workunit.client.1.smithi169.stdout:1/950: write d4/d5/d2d/d4e/d71/d26/d4f/f51 [1152980,98097] 0 2024-03-04T13:43:12.270 INFO:tasks.workunit.client.1.smithi169.stdout:1/951: mkdir d4/d5/d2d/d4e/d71/d26/daf/d134 0 2024-03-04T13:43:12.270 INFO:tasks.workunit.client.0.smithi107.stdout:6/537: sync 2024-03-04T13:43:12.271 INFO:tasks.workunit.client.1.smithi169.stdout:1/952: write d4/d5/d2d/f6c [2546528,126122] 0 2024-03-04T13:43:12.271 INFO:tasks.workunit.client.0.smithi107.stdout:7/511: dwrite d6/d17/f97 [0,4194304] 0 2024-03-04T13:43:12.273 INFO:tasks.workunit.client.0.smithi107.stdout:5/509: dwrite d5/fad [0,4194304] 0 2024-03-04T13:43:12.274 INFO:tasks.workunit.client.0.smithi107.stdout:0/576: rename da/c55 to da/dc/dd/d3e/d6e/d73/cca 0 2024-03-04T13:43:12.274 INFO:tasks.workunit.client.1.smithi169.stdout:1/953: creat d4/d5/d2d/d4e/d71/d26/d4f/f135 x:0 0 0 2024-03-04T13:43:12.275 INFO:tasks.workunit.client.0.smithi107.stdout:6/538: readlink df/d17/d24/d30/d3b/la6 0 2024-03-04T13:43:12.275 INFO:tasks.workunit.client.0.smithi107.stdout:1/511: truncate d1/d46/d6a/d4e/d60/f53 1740216 0 2024-03-04T13:43:12.279 INFO:tasks.workunit.client.0.smithi107.stdout:9/485: dwrite d4/d32/f59 [0,4194304] 0 2024-03-04T13:43:12.281 INFO:tasks.workunit.client.0.smithi107.stdout:0/577: read da/dc/fbb [1470518,77976] 0 2024-03-04T13:43:12.284 INFO:tasks.workunit.client.0.smithi107.stdout:1/512: read d1/d46/d6a/d4e/f6c [163765,71036] 0 2024-03-04T13:43:12.285 INFO:tasks.workunit.client.0.smithi107.stdout:5/510: symlink d5/dc/lb4 0 2024-03-04T13:43:12.289 INFO:tasks.workunit.client.0.smithi107.stdout:0/578: symlink da/dc/dd/d1e/dbf/lcb 0 2024-03-04T13:43:12.289 INFO:tasks.workunit.client.0.smithi107.stdout:1/513: read d1/d46/d6a/df/d33/d5d/d4c/f93 [298369,26037] 0 2024-03-04T13:43:12.290 INFO:tasks.workunit.client.0.smithi107.stdout:9/486: write d4/da/d11/d4c/d4d/f54 [3059369,87751] 0 2024-03-04T13:43:12.290 INFO:tasks.workunit.client.0.smithi107.stdout:7/512: rename d6/d1d/f3b to d6/d1d/d64/d8e/fab 0 2024-03-04T13:43:12.292 INFO:tasks.workunit.client.0.smithi107.stdout:5/511: symlink d5/d66/d96/lb5 0 2024-03-04T13:43:12.296 INFO:tasks.workunit.client.0.smithi107.stdout:7/513: creat d6/d13/d16/d19/d38/d2c/fac x:0 0 0 2024-03-04T13:43:12.296 INFO:tasks.workunit.client.0.smithi107.stdout:5/512: write d5/d8/d14/f4c [483437,9935] 0 2024-03-04T13:43:12.297 INFO:tasks.workunit.client.0.smithi107.stdout:0/579: rmdir da/dc/dd/d3e/d6e 39 2024-03-04T13:43:12.298 INFO:tasks.workunit.client.0.smithi107.stdout:5/513: dread - d5/d66/f81 zero size 2024-03-04T13:43:12.300 INFO:tasks.workunit.client.0.smithi107.stdout:1/514: mkdir d1/d46/d6a/df/d33/d5d/da3 0 2024-03-04T13:43:12.302 INFO:tasks.workunit.client.0.smithi107.stdout:9/487: mknod d4/da/d11/d20/d2f/ca3 0 2024-03-04T13:43:12.304 INFO:tasks.workunit.client.0.smithi107.stdout:7/514: stat d6/d13/d16/d19/d38/d62/c91 0 2024-03-04T13:43:12.305 INFO:tasks.workunit.client.0.smithi107.stdout:0/580: symlink da/dc/lcc 0 2024-03-04T13:43:12.307 INFO:tasks.workunit.client.0.smithi107.stdout:5/514: read d5/dc/d22/f61 [3553991,131044] 0 2024-03-04T13:43:12.308 INFO:tasks.workunit.client.0.smithi107.stdout:6/539: dwrite df/d17/d5c/db1/fb2 [0,4194304] 0 2024-03-04T13:43:12.309 INFO:tasks.workunit.client.0.smithi107.stdout:9/488: rmdir d4/da/d11/d20/d2f 39 2024-03-04T13:43:12.316 INFO:tasks.workunit.client.0.smithi107.stdout:1/515: truncate d1/d46/d6a/d17/f5a 3907865 0 2024-03-04T13:43:12.319 INFO:tasks.workunit.client.0.smithi107.stdout:7/515: link d6/d1d/d64/d8e/f53 d6/d13/d16/d19/d38/d62/fad 0 2024-03-04T13:43:12.319 INFO:tasks.workunit.client.0.smithi107.stdout:0/581: link da/dc/dd/l5c da/dc/dd/d3e/d26/d88/lcd 0 2024-03-04T13:43:12.322 INFO:tasks.workunit.client.0.smithi107.stdout:0/582: truncate f8 1067219 0 2024-03-04T13:43:12.326 INFO:tasks.workunit.client.0.smithi107.stdout:5/515: link d5/d8/da/l77 d5/d79/d89/lb6 0 2024-03-04T13:43:12.328 INFO:tasks.workunit.client.0.smithi107.stdout:5/516: write d5/d66/f78 [1170349,1110] 0 2024-03-04T13:43:12.328 INFO:tasks.workunit.client.0.smithi107.stdout:7/516: rename d6/d13/d16/f67 to d6/d1d/d36/fae 0 2024-03-04T13:43:12.330 INFO:tasks.workunit.client.0.smithi107.stdout:8/526: sync 2024-03-04T13:43:12.330 INFO:tasks.workunit.client.0.smithi107.stdout:4/461: sync 2024-03-04T13:43:12.330 INFO:tasks.workunit.client.0.smithi107.stdout:2/482: sync 2024-03-04T13:43:12.330 INFO:tasks.workunit.client.0.smithi107.stdout:3/503: sync 2024-03-04T13:43:12.331 INFO:tasks.workunit.client.0.smithi107.stdout:0/583: symlink da/dc/dd/d3e/d6e/d73/d78/d86/lce 0 2024-03-04T13:43:12.333 INFO:tasks.workunit.client.0.smithi107.stdout:7/517: creat d6/d13/d16/d1c/faf x:0 0 0 2024-03-04T13:43:12.335 INFO:tasks.workunit.client.0.smithi107.stdout:0/584: write da/dc/dd/d3e/f77 [1943663,110225] 0 2024-03-04T13:43:12.337 INFO:tasks.workunit.client.0.smithi107.stdout:5/517: creat d5/d8/d2a/fb7 x:0 0 0 2024-03-04T13:43:12.339 INFO:tasks.workunit.client.0.smithi107.stdout:5/518: fsync d5/d8/d14/f4c 0 2024-03-04T13:43:12.341 INFO:tasks.workunit.client.0.smithi107.stdout:4/462: creat d0/d9/de/d14/d33/f91 x:0 0 0 2024-03-04T13:43:12.341 INFO:tasks.workunit.client.0.smithi107.stdout:5/519: fsync d5/dc/f55 0 2024-03-04T13:43:12.346 INFO:tasks.workunit.client.0.smithi107.stdout:4/463: chown d0/d9/de/d14/dc 449 1 2024-03-04T13:43:12.348 INFO:tasks.workunit.client.1.smithi169.stdout:1/954: truncate d4/d5/d36/d35/d5b/fcc 3038190 0 2024-03-04T13:43:12.351 INFO:tasks.workunit.client.0.smithi107.stdout:5/520: symlink d5/d8/d14/d2d/lb8 0 2024-03-04T13:43:12.352 INFO:tasks.workunit.client.0.smithi107.stdout:3/504: getdents d1/de/d11 0 2024-03-04T13:43:12.352 INFO:tasks.workunit.client.0.smithi107.stdout:0/585: creat da/dc/dd/d3e/d6e/d73/d78/fcf x:0 0 0 2024-03-04T13:43:12.353 INFO:tasks.workunit.client.1.smithi169.stdout:1/955: chown d4/d5/d36/d35/d16/d81/db4/d104/c10c 28912 1 2024-03-04T13:43:12.359 INFO:tasks.workunit.client.0.smithi107.stdout:0/586: creat da/dc/dd/d3e/d26/fd0 x:0 0 0 2024-03-04T13:43:12.362 INFO:tasks.workunit.client.0.smithi107.stdout:9/489: write d4/da/d11/f13 [4911498,3267] 0 2024-03-04T13:43:12.362 INFO:tasks.workunit.client.0.smithi107.stdout:6/540: write df/f15 [1552819,74053] 0 2024-03-04T13:43:12.365 INFO:tasks.workunit.client.0.smithi107.stdout:1/516: write d1/d46/d6a/d4e/f6c [1514196,102387] 0 2024-03-04T13:43:12.369 INFO:tasks.workunit.client.0.smithi107.stdout:8/527: dread d6/d16/f24 [0,4194304] 0 2024-03-04T13:43:12.372 INFO:tasks.workunit.client.0.smithi107.stdout:0/587: chown da/dc/dd/d3e/d26/l61 79 1 2024-03-04T13:43:12.373 INFO:tasks.workunit.client.0.smithi107.stdout:2/483: write da/df/f1e [225958,113975] 0 2024-03-04T13:43:12.373 INFO:tasks.workunit.client.1.smithi169.stdout:1/956: dread d4/d5/f127 [0,4194304] 0 2024-03-04T13:43:12.374 INFO:tasks.workunit.client.0.smithi107.stdout:4/464: dwrite d0/d9/de/d14/dc/f2d [0,4194304] 0 2024-03-04T13:43:12.376 INFO:tasks.workunit.client.0.smithi107.stdout:6/541: getdents df/d17/d24/d40/d63/da3/dab 0 2024-03-04T13:43:12.377 INFO:tasks.workunit.client.0.smithi107.stdout:5/521: dwrite d5/dc/f3d [0,4194304] 0 2024-03-04T13:43:12.384 INFO:tasks.workunit.client.0.smithi107.stdout:0/588: mknod da/dc/dd/d1e/cd1 0 2024-03-04T13:43:12.388 INFO:tasks.workunit.client.0.smithi107.stdout:8/528: rename d6/d16/d1f/l8d to d6/da/d25/d2a/l9d 0 2024-03-04T13:43:12.395 INFO:tasks.workunit.client.0.smithi107.stdout:7/518: dwrite d6/d13/d14/f5d [0,4194304] 0 2024-03-04T13:43:12.396 INFO:tasks.workunit.client.0.smithi107.stdout:1/517: link d1/d46/d6a/d75/c6d d1/d10/d16/d1b/d40/ca4 0 2024-03-04T13:43:12.398 INFO:tasks.workunit.client.0.smithi107.stdout:2/484: dread da/df/d27/f5e [0,4194304] 0 2024-03-04T13:43:12.405 INFO:tasks.workunit.client.0.smithi107.stdout:1/518: chown d1/d46/d6a/d65/f7f 1153 1 2024-03-04T13:43:12.407 INFO:tasks.workunit.client.0.smithi107.stdout:9/490: dwrite d4/d32/f59 [4194304,4194304] 0 2024-03-04T13:43:12.408 INFO:tasks.workunit.client.0.smithi107.stdout:0/589: rename da/dc/dd/d3e/d6e to da/dc/dd/d3e/d26/d88/dd2 0 2024-03-04T13:43:12.416 INFO:tasks.workunit.client.0.smithi107.stdout:1/519: read d1/d46/d6a/df/d33/d43/f55 [968292,124838] 0 2024-03-04T13:43:12.416 INFO:tasks.workunit.client.0.smithi107.stdout:5/522: getdents d5/d79/d89/da3/d68 0 2024-03-04T13:43:12.417 INFO:tasks.workunit.client.0.smithi107.stdout:8/529: write d6/d16/d1f/d63/d73/f7b [3039083,22040] 0 2024-03-04T13:43:12.419 INFO:tasks.workunit.client.0.smithi107.stdout:4/465: dread d0/d9/d18/f53 [0,4194304] 0 2024-03-04T13:43:12.419 INFO:tasks.workunit.client.0.smithi107.stdout:0/590: creat da/dc/dd/d3e/d26/d69/d6b/fd3 x:0 0 0 2024-03-04T13:43:12.419 INFO:tasks.workunit.client.0.smithi107.stdout:9/491: dread - d4/da/d11/d20/d23/d2c/f38 zero size 2024-03-04T13:43:12.420 INFO:tasks.workunit.client.1.smithi169.stdout:1/957: dread d4/d5/d36/f67 [0,4194304] 0 2024-03-04T13:43:12.427 INFO:tasks.workunit.client.0.smithi107.stdout:8/530: fsync d6/d16/d69/f8f 0 2024-03-04T13:43:12.427 INFO:tasks.workunit.client.0.smithi107.stdout:0/591: readlink da/dc/dd/d3e/d26/d69/l8c 0 2024-03-04T13:43:12.428 INFO:tasks.workunit.client.1.smithi169.stdout:1/958: write d4/d5/d2d/d4e/d71/d26/daf/f129 [272308,11947] 0 2024-03-04T13:43:12.428 INFO:tasks.workunit.client.0.smithi107.stdout:9/492: readlink d4/da/d11/d20/l29 0 2024-03-04T13:43:12.429 INFO:tasks.workunit.client.1.smithi169.stdout:3/965: sync 2024-03-04T13:43:12.431 INFO:tasks.workunit.client.1.smithi169.stdout:1/959: chown d4/d5/d36/d35/lb0 3828 1 2024-03-04T13:43:12.432 INFO:tasks.workunit.client.0.smithi107.stdout:5/523: rmdir d5/d8 39 2024-03-04T13:43:12.433 INFO:tasks.workunit.client.0.smithi107.stdout:4/466: mkdir d0/d9/d18/d29/d92 0 2024-03-04T13:43:12.435 INFO:tasks.workunit.client.0.smithi107.stdout:0/592: rmdir da/d7e 39 2024-03-04T13:43:12.436 INFO:tasks.workunit.client.0.smithi107.stdout:4/467: dread - d0/d9/d10/f8c zero size 2024-03-04T13:43:12.439 INFO:tasks.workunit.client.0.smithi107.stdout:8/531: write d6/d16/f24 [2207314,78002] 0 2024-03-04T13:43:12.447 INFO:tasks.workunit.client.0.smithi107.stdout:9/493: read d4/da/d11/d65/f3e [2351116,51083] 0 2024-03-04T13:43:12.448 INFO:tasks.workunit.client.1.smithi169.stdout:1/960: dwrite d4/d5/fc4 [0,4194304] 0 2024-03-04T13:43:12.450 INFO:tasks.workunit.client.0.smithi107.stdout:2/485: dwrite da/f20 [0,4194304] 0 2024-03-04T13:43:12.451 INFO:tasks.workunit.client.1.smithi169.stdout:3/966: dwrite d7/de/d2b/d7d/d85/d18/d21/d3f/f82 [0,4194304] 0 2024-03-04T13:43:12.452 INFO:tasks.workunit.client.1.smithi169.stdout:1/961: stat d4/d5/d2d/d4e/da5/d11a/c124 0 2024-03-04T13:43:12.452 INFO:tasks.workunit.client.0.smithi107.stdout:4/468: dread d0/d9/d18/f53 [0,4194304] 0 2024-03-04T13:43:12.456 INFO:tasks.workunit.client.0.smithi107.stdout:6/542: dread df/d17/d24/d40/f62 [0,4194304] 0 2024-03-04T13:43:12.457 INFO:tasks.workunit.client.0.smithi107.stdout:1/520: dwrite d1/d46/d6a/df/d67/d6b/f84 [0,4194304] 0 2024-03-04T13:43:12.458 INFO:tasks.workunit.client.0.smithi107.stdout:2/486: stat da/cc 0 2024-03-04T13:43:12.459 INFO:tasks.workunit.client.1.smithi169.stdout:1/962: truncate d4/d5/d36/d35/d70/d78/fd4 634584 0 2024-03-04T13:43:12.462 INFO:tasks.workunit.client.1.smithi169.stdout:1/963: readlink d4/d5/d36/d35/d5b/l73 0 2024-03-04T13:43:12.463 INFO:tasks.workunit.client.0.smithi107.stdout:2/487: chown da/df/d27 76 1 2024-03-04T13:43:12.467 INFO:tasks.workunit.client.0.smithi107.stdout:4/469: write d0/d9/de/d14/dc/d21/f6c [429073,45241] 0 2024-03-04T13:43:12.473 INFO:tasks.workunit.client.1.smithi169.stdout:1/964: fdatasync d4/d5/d2d/d4e/d71/d26/d4f/fb3 0 2024-03-04T13:43:12.474 INFO:tasks.workunit.client.0.smithi107.stdout:8/532: dread f2 [0,4194304] 0 2024-03-04T13:43:12.474 INFO:tasks.workunit.client.1.smithi169.stdout:3/967: dwrite d7/de/d2b/d7d/d85/d13/d1a/d2d/d56/d5e/d5f/f12c [0,4194304] 0 2024-03-04T13:43:12.475 INFO:tasks.workunit.client.0.smithi107.stdout:6/543: mknod df/d57/cb3 0 2024-03-04T13:43:12.477 INFO:tasks.workunit.client.0.smithi107.stdout:4/470: dread d0/d9/d10/f4a [0,4194304] 0 2024-03-04T13:43:12.478 INFO:tasks.workunit.client.1.smithi169.stdout:4/988: sync 2024-03-04T13:43:12.478 INFO:tasks.workunit.client.0.smithi107.stdout:1/521: chown d1/d46/d6a/d17/f5a 1376511 1 2024-03-04T13:43:12.479 INFO:tasks.workunit.client.1.smithi169.stdout:0/895: sync 2024-03-04T13:43:12.481 INFO:tasks.workunit.client.0.smithi107.stdout:9/494: mknod d4/da/d11/d20/d2f/d9c/ca4 0 2024-03-04T13:43:12.482 INFO:tasks.workunit.client.0.smithi107.stdout:0/593: link da/dc/dd/d3e/d26/d69/d6b/f94 da/dc/dd/d3e/fd4 0 2024-03-04T13:43:12.483 INFO:tasks.workunit.client.0.smithi107.stdout:7/519: dread d6/d13/d16/d19/d38/d2c/f4a [4194304,4194304] 0 2024-03-04T13:43:12.485 INFO:tasks.workunit.client.0.smithi107.stdout:8/533: creat d6/d16/d59/d3e/d67/f9e x:0 0 0 2024-03-04T13:43:12.485 INFO:tasks.workunit.client.0.smithi107.stdout:4/471: write d0/d9/d10/f7a [728152,87981] 0 2024-03-04T13:43:12.491 INFO:tasks.workunit.client.0.smithi107.stdout:6/544: rename df/d17/d5c/db1/c25 to df/d17/d6b/cb4 0 2024-03-04T13:43:12.493 INFO:tasks.workunit.client.0.smithi107.stdout:0/594: fdatasync da/dc/dd/d3e/d26/d88/dd2/f87 0 2024-03-04T13:43:12.493 INFO:tasks.workunit.client.1.smithi169.stdout:1/965: creat d4/d5/d36/d35/d16/d81/db4/d104/f136 x:0 0 0 2024-03-04T13:43:12.494 INFO:tasks.workunit.client.0.smithi107.stdout:2/488: dwrite da/df/f24 [0,4194304] 0 2024-03-04T13:43:12.496 INFO:tasks.workunit.client.1.smithi169.stdout:3/968: mkdir d7/de/d2b/d7d/d85/d47/d13e 0 2024-03-04T13:43:12.499 INFO:tasks.workunit.client.1.smithi169.stdout:3/969: readlink d7/de/d2b/d7d/d85/l23 0 2024-03-04T13:43:12.499 INFO:tasks.workunit.client.0.smithi107.stdout:5/524: truncate d5/d8/f45 3807006 0 2024-03-04T13:43:12.500 INFO:tasks.workunit.client.0.smithi107.stdout:6/545: write df/d17/d24/d40/d63/da3/f68 [212839,7361] 0 2024-03-04T13:43:12.502 INFO:tasks.workunit.client.1.smithi169.stdout:3/970: chown d7/db8/fc6 41 1 2024-03-04T13:43:12.503 INFO:tasks.workunit.client.0.smithi107.stdout:4/472: creat d0/d9/d1e/d70/f93 x:0 0 0 2024-03-04T13:43:12.508 INFO:tasks.workunit.client.1.smithi169.stdout:0/896: truncate d5/d9/dc/d12/f24 2487162 0 2024-03-04T13:43:12.511 INFO:tasks.workunit.client.0.smithi107.stdout:2/489: creat da/df/d27/fa7 x:0 0 0 2024-03-04T13:43:12.512 INFO:tasks.workunit.client.0.smithi107.stdout:1/522: link d1/d10/d16/d1b/l50 d1/d46/la5 0 2024-03-04T13:43:12.512 INFO:tasks.workunit.client.1.smithi169.stdout:4/989: dwrite d4/d11/d2e/dcd/dee/d5c/d86/db8/f115 [0,4194304] 0 2024-03-04T13:43:12.514 INFO:tasks.workunit.client.0.smithi107.stdout:6/546: mkdir df/d38/d4f/d74/db5 0 2024-03-04T13:43:12.519 INFO:tasks.workunit.client.1.smithi169.stdout:0/897: mknod d5/d9/d3d/d59/d64/d7f/c132 0 2024-03-04T13:43:12.521 INFO:tasks.workunit.client.1.smithi169.stdout:1/966: dread d4/d5/d36/d75/f107 [0,4194304] 0 2024-03-04T13:43:12.521 INFO:tasks.workunit.client.0.smithi107.stdout:0/595: unlink da/d7e/l5e 0 2024-03-04T13:43:12.522 INFO:tasks.workunit.client.1.smithi169.stdout:4/990: write d4/d11/d2e/dcd/dee/d5c/d86/db8/df9/f122 [255104,79835] 0 2024-03-04T13:43:12.525 INFO:tasks.workunit.client.0.smithi107.stdout:9/495: link d4/da/l2e d4/da/d11/d4c/la5 0 2024-03-04T13:43:12.529 INFO:tasks.workunit.client.0.smithi107.stdout:1/523: mknod d1/d10/d16/d47/d71/ca6 0 2024-03-04T13:43:12.529 INFO:tasks.workunit.client.0.smithi107.stdout:9/496: write d4/d32/f59 [6399708,125483] 0 2024-03-04T13:43:12.530 INFO:tasks.workunit.client.0.smithi107.stdout:5/525: getdents d5/d8/d14/d2d/d92 0 2024-03-04T13:43:12.531 INFO:tasks.workunit.client.0.smithi107.stdout:6/547: dwrite df/d17/d24/d40/fa5 [0,4194304] 0 2024-03-04T13:43:12.533 INFO:tasks.workunit.client.1.smithi169.stdout:4/991: fdatasync d4/d11/d2e/dcd/dee/d73/d101/f11b 0 2024-03-04T13:43:12.533 INFO:tasks.workunit.client.1.smithi169.stdout:4/992: chown d4/laa 427630 1 2024-03-04T13:43:12.534 INFO:tasks.workunit.client.1.smithi169.stdout:4/993: chown d4/d1c/fb2 11866229 1 2024-03-04T13:43:12.535 INFO:tasks.workunit.client.0.smithi107.stdout:6/548: stat df/d17/d24/d40/d63/da3/d7e/d86 0 2024-03-04T13:43:12.538 INFO:tasks.workunit.client.0.smithi107.stdout:8/534: rename d6/da/d25/d5f/f6e to d6/f9f 0 2024-03-04T13:43:12.544 INFO:tasks.workunit.client.1.smithi169.stdout:3/971: link d7/de/d2b/d7d/d85/d18/d21/d103/d11e/c133 d7/de/d2b/d7d/d85/d13/d1a/d2d/d56/d5e/d67/d5c/d84/d95/c13f 0 2024-03-04T13:43:12.550 INFO:tasks.workunit.client.0.smithi107.stdout:5/526: fsync d5/dc/d22/fa8 0 2024-03-04T13:43:12.553 INFO:tasks.workunit.client.0.smithi107.stdout:5/527: read d5/d79/d89/da3/d6a/f6d [4117764,37350] 0 2024-03-04T13:43:12.555 INFO:tasks.workunit.client.0.smithi107.stdout:1/524: mknod d1/d10/d16/d47/d7a/ca7 0 2024-03-04T13:43:12.557 INFO:tasks.workunit.client.1.smithi169.stdout:4/994: unlink d4/d11/d2e/d39/d61/d64/c7c 0 2024-03-04T13:43:12.560 INFO:tasks.workunit.client.1.smithi169.stdout:3/972: mkdir d7/de/d2b/d7d/d85/d13/d1a/d2d/d56/d5e/d140 0 2024-03-04T13:43:12.562 INFO:tasks.workunit.client.0.smithi107.stdout:6/549: symlink df/d17/d24/d40/lb6 0 2024-03-04T13:43:12.563 INFO:tasks.workunit.client.0.smithi107.stdout:6/550: chown df/f43 27028528 1 2024-03-04T13:43:12.563 INFO:tasks.workunit.client.0.smithi107.stdout:3/505: sync 2024-03-04T13:43:12.572 INFO:tasks.workunit.client.0.smithi107.stdout:2/490: write f0 [4696201,53839] 0 2024-03-04T13:43:12.579 INFO:tasks.workunit.client.0.smithi107.stdout:8/535: rename d6/l81 to d6/da/d25/la0 0 2024-03-04T13:43:12.581 INFO:tasks.workunit.client.1.smithi169.stdout:3/973: creat d7/de/d2b/d7d/d85/d18/d21/d103/d13c/f141 x:0 0 0 2024-03-04T13:43:12.582 INFO:tasks.workunit.client.0.smithi107.stdout:3/506: rmdir d1/de/d18/d33/d48 39 2024-03-04T13:43:12.582 INFO:tasks.workunit.client.1.smithi169.stdout:0/898: dwrite d5/d9/dc/fa6 [0,4194304] 0 2024-03-04T13:43:12.583 INFO:tasks.workunit.client.0.smithi107.stdout:6/551: truncate df/d2e/f5a 994308 0 2024-03-04T13:43:12.583 INFO:tasks.workunit.client.0.smithi107.stdout:3/507: write d1/fa [4689463,64707] 0 2024-03-04T13:43:12.586 INFO:tasks.workunit.client.0.smithi107.stdout:5/528: dread d5/dc/f3b [0,4194304] 0 2024-03-04T13:43:12.592 INFO:tasks.workunit.client.1.smithi169.stdout:0/899: symlink d5/d9/dc/d12/d15/d19/d61/d71/dbb/l133 0 2024-03-04T13:43:12.595 INFO:tasks.workunit.client.0.smithi107.stdout:1/525: unlink d1/lc 0 2024-03-04T13:43:12.599 INFO:tasks.workunit.client.0.smithi107.stdout:6/552: creat df/d2e/d36/d5b/fb7 x:0 0 0 2024-03-04T13:43:12.601 INFO:tasks.workunit.client.0.smithi107.stdout:6/553: readlink df/d17/d24/d40/d63/da3/d7e/l85 0 2024-03-04T13:43:12.601 INFO:tasks.workunit.client.0.smithi107.stdout:9/497: getdents d4/da/d11/d4c/d89 0 2024-03-04T13:43:12.602 INFO:tasks.workunit.client.0.smithi107.stdout:3/508: truncate d1/de/d18/d33/f8f 4344994 0 2024-03-04T13:43:12.604 INFO:tasks.workunit.client.1.smithi169.stdout:3/974: dwrite d7/de/d2b/d7d/d85/d18/f30 [0,4194304] 0 2024-03-04T13:43:12.607 INFO:tasks.workunit.client.0.smithi107.stdout:5/529: rmdir d5/d8/da 39 2024-03-04T13:43:12.607 INFO:tasks.workunit.client.0.smithi107.stdout:8/536: mknod d6/d16/d59/d3e/d96/ca1 0 2024-03-04T13:43:12.610 INFO:tasks.workunit.client.0.smithi107.stdout:8/537: chown d6/d16/d59/d3e 41317879 1 2024-03-04T13:43:12.610 INFO:tasks.workunit.client.0.smithi107.stdout:1/526: mknod d1/d46/d6a/df/d33/d43/d5b/ca8 0 2024-03-04T13:43:12.613 INFO:tasks.workunit.client.1.smithi169.stdout:4/995: truncate d4/f41 2622015 0 2024-03-04T13:43:12.615 INFO:tasks.workunit.client.1.smithi169.stdout:4/996: fdatasync d4/d11/d2e/dcd/dee/d5c/d86/db8/f115 0 2024-03-04T13:43:12.617 INFO:tasks.workunit.client.0.smithi107.stdout:1/527: write d1/d46/d6a/df/d67/d6b/f84 [220231,45243] 0 2024-03-04T13:43:12.617 INFO:tasks.workunit.client.0.smithi107.stdout:6/554: creat df/d17/d24/d40/d63/da3/d7e/fb8 x:0 0 0 2024-03-04T13:43:12.621 INFO:tasks.workunit.client.0.smithi107.stdout:1/528: stat d1/d46/d6a/d17 0 2024-03-04T13:43:12.622 INFO:tasks.workunit.client.0.smithi107.stdout:5/530: mkdir d5/d79/d89/da3/d69/db9 0 2024-03-04T13:43:12.623 INFO:tasks.workunit.client.0.smithi107.stdout:3/509: dwrite d1/d2d/f44 [4194304,4194304] 0 2024-03-04T13:43:12.623 INFO:tasks.workunit.client.0.smithi107.stdout:9/498: dwrite d4/da/f15 [0,4194304] 0 2024-03-04T13:43:12.624 INFO:tasks.workunit.client.0.smithi107.stdout:1/529: chown d1/d46/d6a/df/f32 30 1 2024-03-04T13:43:12.627 INFO:tasks.workunit.client.1.smithi169.stdout:3/975: creat d7/de/d2b/d7d/d85/d13/d1a/d2d/f142 x:0 0 0 2024-03-04T13:43:12.628 INFO:tasks.workunit.client.0.smithi107.stdout:2/491: write da/df/f13 [287434,18736] 0 2024-03-04T13:43:12.631 INFO:tasks.workunit.client.1.smithi169.stdout:4/997: mkdir d4/d11/d3b/d67/d98/d12e 0 2024-03-04T13:43:12.631 INFO:tasks.workunit.client.0.smithi107.stdout:9/499: dread - d4/da/d11/d4c/d89/f9f zero size 2024-03-04T13:43:12.638 INFO:tasks.workunit.client.1.smithi169.stdout:3/976: mkdir d7/de/d2b/d7d/d85/d18/d21/d103/d143 0 2024-03-04T13:43:12.640 INFO:tasks.workunit.client.1.smithi169.stdout:0/900: dread d5/d9/dc/d12/d15/d19/d77/d94/fa2 [0,4194304] 0 2024-03-04T13:43:12.643 INFO:tasks.workunit.client.1.smithi169.stdout:4/998: creat d4/d11/d3b/d67/d7e/f12f x:0 0 0 2024-03-04T13:43:12.647 INFO:tasks.workunit.client.0.smithi107.stdout:8/538: unlink d6/d16/d69/l6b 0 2024-03-04T13:43:12.650 INFO:tasks.workunit.client.1.smithi169.stdout:3/977: creat d7/de/d2b/d7d/d85/d13/d5a/d65/da4/d104/f144 x:0 0 0 2024-03-04T13:43:12.653 INFO:tasks.workunit.client.0.smithi107.stdout:6/555: getdents df/d17/d24/d8d 0 2024-03-04T13:43:12.654 INFO:tasks.workunit.client.1.smithi169.stdout:4/999: symlink d4/d11/d3b/d67/d7e/d124/da5/d77/dc0/l130 0 2024-03-04T13:43:12.657 INFO:tasks.workunit.client.1.smithi169.stdout:0/901: rename d5/d9/dc/d12/d15/d19/d61/d71/d8c/c9c to d5/d9/d3d/c134 0 2024-03-04T13:43:12.657 INFO:tasks.workunit.client.0.smithi107.stdout:1/530: write d1/d46/d6a/d4e/d60/f63 [696596,46956] 0 2024-03-04T13:43:12.658 INFO:tasks.workunit.client.0.smithi107.stdout:9/500: dread d4/da/d11/f13 [0,4194304] 0 2024-03-04T13:43:12.658 INFO:tasks.workunit.client.0.smithi107.stdout:7/520: sync 2024-03-04T13:43:12.659 INFO:tasks.workunit.client.1.smithi169.stdout:3/978: truncate d7/de/d2b/d7d/d85/d13/d5a/fcc 2725927 0 2024-03-04T13:43:12.660 INFO:tasks.workunit.client.0.smithi107.stdout:3/510: creat d1/de/d18/d33/d48/fa6 x:0 0 0 2024-03-04T13:43:12.662 INFO:tasks.workunit.client.0.smithi107.stdout:2/492: mknod da/d44/d9c/ca8 0 2024-03-04T13:43:12.664 INFO:tasks.workunit.client.0.smithi107.stdout:8/539: rename d6/d16/f22 to d6/d16/d1f/d63/d73/fa2 0 2024-03-04T13:43:12.665 INFO:tasks.workunit.client.1.smithi169.stdout:3/979: read d7/de/d2b/d7d/d85/d13/d5a/d65/da4/fd3 [159069,75580] 0 2024-03-04T13:43:12.665 INFO:tasks.workunit.client.0.smithi107.stdout:8/540: chown d6/da/f35 264070204 1 2024-03-04T13:43:12.666 INFO:tasks.workunit.client.0.smithi107.stdout:5/531: symlink d5/d8/da/lba 0 2024-03-04T13:43:12.666 INFO:tasks.workunit.client.0.smithi107.stdout:8/541: write f5 [214634,78182] 0 2024-03-04T13:43:12.668 INFO:tasks.workunit.client.1.smithi169.stdout:0/902: mknod d5/d4c/d65/d95/db0/d10c/c135 0 2024-03-04T13:43:12.671 INFO:tasks.workunit.client.0.smithi107.stdout:6/556: rmdir df/d17/d24/d40/d63/da3/d7e 39 2024-03-04T13:43:12.672 INFO:tasks.workunit.client.1.smithi169.stdout:3/980: mknod d7/de/d2b/d7d/d85/d13/d5a/d65/da4/d104/c145 0 2024-03-04T13:43:12.672 INFO:tasks.workunit.client.0.smithi107.stdout:1/531: chown d1/d46/la5 14442 1 2024-03-04T13:43:12.675 INFO:tasks.workunit.client.1.smithi169.stdout:0/903: creat d5/d9/dc/d12/d15/d19/d77/db2/f136 x:0 0 0 2024-03-04T13:43:12.678 INFO:tasks.workunit.client.1.smithi169.stdout:3/981: creat d7/de/d2b/d7d/d85/d13/d1a/d2d/d56/d5e/d5f/f146 x:0 0 0 2024-03-04T13:43:12.683 INFO:tasks.workunit.client.1.smithi169.stdout:1/967: sync 2024-03-04T13:43:12.684 INFO:tasks.workunit.client.0.smithi107.stdout:2/493: creat da/d44/d8b/fa9 x:0 0 0 2024-03-04T13:43:12.684 INFO:tasks.workunit.client.0.smithi107.stdout:5/532: dwrite d5/d66/f81 [0,4194304] 0 2024-03-04T13:43:12.685 INFO:tasks.workunit.client.1.smithi169.stdout:3/982: creat d7/db8/f147 x:0 0 0 2024-03-04T13:43:12.688 INFO:tasks.workunit.client.0.smithi107.stdout:2/494: read da/d49/f7e [855224,49682] 0 2024-03-04T13:43:12.688 INFO:tasks.workunit.client.0.smithi107.stdout:8/542: unlink d6/d16/d1f/d63/d73/c84 0 2024-03-04T13:43:12.690 INFO:tasks.workunit.client.1.smithi169.stdout:1/968: dread - d4/d5/d36/d35/d16/d81/db4/f118 zero size 2024-03-04T13:43:12.691 INFO:tasks.workunit.client.0.smithi107.stdout:2/495: write da/d44/f52 [4105980,124783] 0 2024-03-04T13:43:12.691 INFO:tasks.workunit.client.0.smithi107.stdout:8/543: chown d6/d16/d1f/c2f 1778833 1 2024-03-04T13:43:12.694 INFO:tasks.workunit.client.1.smithi169.stdout:0/904: creat d5/d4c/d65/d95/db0/f137 x:0 0 0 2024-03-04T13:43:12.695 INFO:tasks.workunit.client.0.smithi107.stdout:9/501: dwrite d4/f8 [4194304,4194304] 0 2024-03-04T13:43:12.695 INFO:tasks.workunit.client.0.smithi107.stdout:8/544: chown d6/d16/d59/d7c/l86 1 1 2024-03-04T13:43:12.695 INFO:tasks.workunit.client.0.smithi107.stdout:6/557: truncate df/d17/d24/d40/d63/da3/f70 399252 0 2024-03-04T13:43:12.698 INFO:tasks.workunit.client.0.smithi107.stdout:2/496: read da/d34/f41 [2109414,69235] 0 2024-03-04T13:43:12.699 INFO:tasks.workunit.client.1.smithi169.stdout:3/983: fdatasync d7/de/d2b/d7d/d85/d18/d21/fe4 0 2024-03-04T13:43:12.699 INFO:tasks.workunit.client.1.smithi169.stdout:1/969: symlink d4/d5/d2d/d4e/d114/l137 0 2024-03-04T13:43:12.702 INFO:tasks.workunit.client.1.smithi169.stdout:1/970: chown d4/d5/d36/d35/df/f28 2919 1 2024-03-04T13:43:12.702 INFO:tasks.workunit.client.0.smithi107.stdout:1/532: dread d1/d46/d6a/d4e/d60/f53 [0,4194304] 0 2024-03-04T13:43:12.703 INFO:tasks.workunit.client.0.smithi107.stdout:3/511: write d1/de/d18/d33/d48/d71/f15 [5092542,104444] 0 2024-03-04T13:43:12.704 INFO:tasks.workunit.client.1.smithi169.stdout:1/971: fdatasync d4/d5/d2d/d4e/fa6 0 2024-03-04T13:43:12.704 INFO:tasks.workunit.client.0.smithi107.stdout:5/533: truncate d5/dc/df/f12 1291006 0 2024-03-04T13:43:12.707 INFO:tasks.workunit.client.1.smithi169.stdout:0/905: symlink d5/d9/d3d/d59/d64/d7f/dd1/l138 0 2024-03-04T13:43:12.708 INFO:tasks.workunit.client.0.smithi107.stdout:3/512: fsync d1/de/d18/d33/d48/d71/d4c/f91 0 2024-03-04T13:43:12.714 INFO:tasks.workunit.client.1.smithi169.stdout:1/972: dread - d4/d5/d36/d35/d16/d81/db4/fde zero size 2024-03-04T13:43:12.716 INFO:tasks.workunit.client.0.smithi107.stdout:9/502: symlink d4/da/d11/d20/d23/d7e/d84/la6 0 2024-03-04T13:43:12.718 INFO:tasks.workunit.client.0.smithi107.stdout:2/497: dread - da/d44/f64 zero size 2024-03-04T13:43:12.719 INFO:tasks.workunit.client.0.smithi107.stdout:6/558: dread df/d17/d24/d30/d8b/f87 [0,4194304] 0 2024-03-04T13:43:12.721 INFO:tasks.workunit.client.1.smithi169.stdout:1/973: creat d4/d5/d2d/d4e/d71/d26/d7e/d11c/f138 x:0 0 0 2024-03-04T13:43:12.722 INFO:tasks.workunit.client.0.smithi107.stdout:7/521: dwrite d6/f7 [0,4194304] 0 2024-03-04T13:43:12.724 INFO:tasks.workunit.client.0.smithi107.stdout:2/498: truncate da/d49/fa4 193582 0 2024-03-04T13:43:12.724 INFO:tasks.workunit.client.1.smithi169.stdout:1/974: mknod d4/d5/d36/d75/dba/df6/c139 0 2024-03-04T13:43:12.726 INFO:tasks.workunit.client.1.smithi169.stdout:1/975: fsync d4/d5/d2d/d4e/d71/d26/fdf 0 2024-03-04T13:43:12.728 INFO:tasks.workunit.client.0.smithi107.stdout:0/596: sync 2024-03-04T13:43:12.728 INFO:tasks.workunit.client.0.smithi107.stdout:4/473: sync 2024-03-04T13:43:12.730 INFO:tasks.workunit.client.0.smithi107.stdout:3/513: creat d1/de/d18/d33/d48/d71/d4c/d54/fa7 x:0 0 0 2024-03-04T13:43:12.733 INFO:tasks.workunit.client.0.smithi107.stdout:3/514: fsync d1/fa 0 2024-03-04T13:43:12.737 INFO:tasks.workunit.client.0.smithi107.stdout:2/499: mkdir da/df/d73/daa 0 2024-03-04T13:43:12.742 INFO:tasks.workunit.client.1.smithi169.stdout:3/984: write f5 [1573460,128644] 0 2024-03-04T13:43:12.745 INFO:tasks.workunit.client.0.smithi107.stdout:4/474: dread - d0/d9/de/d14/dc/f67 zero size 2024-03-04T13:43:12.749 INFO:tasks.workunit.client.0.smithi107.stdout:2/500: creat da/df/d3e/fab x:0 0 0 2024-03-04T13:43:12.749 INFO:tasks.workunit.client.0.smithi107.stdout:7/522: mknod d6/d13/d16/d19/d38/d6e/cb0 0 2024-03-04T13:43:12.751 INFO:tasks.workunit.client.1.smithi169.stdout:3/985: mknod d7/de/d2b/d7d/d85/d13/d1a/d2d/d56/d5e/d67/d5c/d84/c148 0 2024-03-04T13:43:12.751 INFO:tasks.workunit.client.0.smithi107.stdout:2/501: read - da/df/d85/f95 zero size 2024-03-04T13:43:12.753 INFO:tasks.workunit.client.0.smithi107.stdout:8/545: link d6/f5b d6/d16/fa3 0 2024-03-04T13:43:12.754 INFO:tasks.workunit.client.1.smithi169.stdout:0/906: dwrite d5/d9/dc/d12/fa4 [4194304,4194304] 0 2024-03-04T13:43:12.754 INFO:tasks.workunit.client.1.smithi169.stdout:3/986: readlink d7/de/d2b/d7d/d85/d13/d1a/d2d/d56/d5e/d5f/l11d 0 2024-03-04T13:43:12.757 INFO:tasks.workunit.client.0.smithi107.stdout:1/533: dread d1/d46/d6a/d17/f5a [0,4194304] 0 2024-03-04T13:43:12.758 INFO:tasks.workunit.client.0.smithi107.stdout:0/597: mknod da/dc/dd/d1e/cd5 0 2024-03-04T13:43:12.759 INFO:tasks.workunit.client.0.smithi107.stdout:6/559: dwrite df/d17/d24/d40/d63/da3/f68 [0,4194304] 0 2024-03-04T13:43:12.767 INFO:tasks.workunit.client.0.smithi107.stdout:7/523: rmdir d6/d1d/d64/d8e/d3d 39 2024-03-04T13:43:12.767 INFO:tasks.workunit.client.0.smithi107.stdout:2/502: fdatasync da/df/d27/d58/d5f/f93 0 2024-03-04T13:43:12.768 INFO:tasks.workunit.client.0.smithi107.stdout:5/534: dwrite d5/d8/f1f [0,4194304] 0 2024-03-04T13:43:12.769 INFO:tasks.workunit.client.0.smithi107.stdout:2/503: write f0 [2348100,42666] 0 2024-03-04T13:43:12.769 INFO:tasks.workunit.client.0.smithi107.stdout:8/546: mkdir d6/d16/d1f/d4c/da4 0 2024-03-04T13:43:12.771 INFO:tasks.workunit.client.0.smithi107.stdout:4/475: fdatasync d0/d9/de/d14/dc/f26 0 2024-03-04T13:43:12.771 INFO:tasks.workunit.client.0.smithi107.stdout:1/534: mkdir d1/d10/d16/da9 0 2024-03-04T13:43:12.774 INFO:tasks.workunit.client.0.smithi107.stdout:3/515: write d1/d2d/f7f [560810,89588] 0 2024-03-04T13:43:12.776 INFO:tasks.workunit.client.0.smithi107.stdout:7/524: fsync d6/d13/d16/d19/faa 0 2024-03-04T13:43:12.782 INFO:tasks.workunit.client.0.smithi107.stdout:9/503: dwrite d4/da/d11/d20/d39/f9a [0,4194304] 0 2024-03-04T13:43:12.782 INFO:tasks.workunit.client.1.smithi169.stdout:3/987: readlink d7/de/d2b/d7d/d85/d18/d21/d3f/d86/l128 0 2024-03-04T13:43:12.786 INFO:tasks.workunit.client.1.smithi169.stdout:3/988: dread - d7/de/d2b/d7d/d85/d47/f106 zero size 2024-03-04T13:43:12.787 INFO:tasks.workunit.client.0.smithi107.stdout:9/504: dread - d4/d32/f97 zero size 2024-03-04T13:43:12.788 INFO:tasks.workunit.client.0.smithi107.stdout:8/547: mkdir d6/d16/d1f/d4c/da5 0 2024-03-04T13:43:12.789 INFO:tasks.workunit.client.1.smithi169.stdout:3/989: dread - d7/de/d2b/d7d/d85/d13/d1a/d2d/d56/d5e/ffe zero size 2024-03-04T13:43:12.791 INFO:tasks.workunit.client.0.smithi107.stdout:3/516: creat d1/de/fa8 x:0 0 0 2024-03-04T13:43:12.793 INFO:tasks.workunit.client.0.smithi107.stdout:4/476: rename d0/d2f/d5f/l60 to d0/d9/d10/d6f/l94 0 2024-03-04T13:43:12.797 INFO:tasks.workunit.client.1.smithi169.stdout:3/990: mkdir d7/de/d2b/d7d/d85/d18/d21/d103/d11e/d149 0 2024-03-04T13:43:12.803 INFO:tasks.workunit.client.0.smithi107.stdout:7/525: rename d6/d1d/f32 to d6/d1d/d64/d8e/d3d/fb1 0 2024-03-04T13:43:12.804 INFO:tasks.workunit.client.0.smithi107.stdout:8/548: creat d6/d16/d1f/d4c/da4/fa6 x:0 0 0 2024-03-04T13:43:12.804 INFO:tasks.workunit.client.0.smithi107.stdout:4/477: readlink d0/d9/d1e/l4f 0 2024-03-04T13:43:12.806 INFO:tasks.workunit.client.0.smithi107.stdout:3/517: getdents d1/de/d18/d33/d48/d93 0 2024-03-04T13:43:12.807 INFO:tasks.workunit.client.0.smithi107.stdout:1/535: write d1/d46/d6a/d17/f5a [1869509,74125] 0 2024-03-04T13:43:12.810 INFO:tasks.workunit.client.0.smithi107.stdout:9/505: getdents d4/da/d11/d20/d23/d7e 0 2024-03-04T13:43:12.813 INFO:tasks.workunit.client.0.smithi107.stdout:4/478: unlink d0/d9/d78/f79 0 2024-03-04T13:43:12.814 INFO:tasks.workunit.client.0.smithi107.stdout:7/526: chown d6/d13/d14/c57 1164162497 1 2024-03-04T13:43:12.820 INFO:tasks.workunit.client.0.smithi107.stdout:8/549: stat d6/d16/d1f/d63/d73 0 2024-03-04T13:43:12.823 INFO:tasks.workunit.client.0.smithi107.stdout:4/479: stat d0/d9/de/d14/d33/l4d 0 2024-03-04T13:43:12.824 INFO:tasks.workunit.client.0.smithi107.stdout:7/527: unlink f4 0 2024-03-04T13:43:12.830 INFO:tasks.workunit.client.0.smithi107.stdout:5/535: dwrite d5/d79/d89/fb3 [0,4194304] 0 2024-03-04T13:43:12.831 INFO:tasks.workunit.client.0.smithi107.stdout:9/506: creat d4/da/d11/d20/d39/fa7 x:0 0 0 2024-03-04T13:43:12.834 INFO:tasks.workunit.client.1.smithi169.stdout:0/907: dread d5/d9/d3d/d5d/de4/d98/dc1/dc8/fee [0,4194304] 0 2024-03-04T13:43:12.834 INFO:tasks.workunit.client.0.smithi107.stdout:3/518: rename d1/de/d18/d1f/d42/f70 to d1/de/d18/d33/d48/d71/fa9 0 2024-03-04T13:43:12.834 INFO:tasks.workunit.client.0.smithi107.stdout:6/560: dwrite df/f29 [0,4194304] 0 2024-03-04T13:43:12.840 INFO:tasks.workunit.client.0.smithi107.stdout:0/598: dwrite da/dc/dd/d3e/d26/d69/d6b/f94 [0,4194304] 0 2024-03-04T13:43:12.840 INFO:tasks.workunit.client.0.smithi107.stdout:2/504: dwrite da/d49/d5c/f67 [4194304,4194304] 0 2024-03-04T13:43:12.842 INFO:tasks.workunit.client.1.smithi169.stdout:0/908: fdatasync d5/d9/dc/d12/d15/d19/d61/d71/f12b 0 2024-03-04T13:43:12.846 INFO:tasks.workunit.client.0.smithi107.stdout:4/480: write d0/f73 [2063182,65458] 0 2024-03-04T13:43:12.847 INFO:tasks.workunit.client.1.smithi169.stdout:0/909: write d5/d9/dc/d12/d66/f78 [539817,101763] 0 2024-03-04T13:43:12.852 INFO:tasks.workunit.client.0.smithi107.stdout:1/536: dread d1/d46/d6a/d4e/f6c [0,4194304] 0 2024-03-04T13:43:12.855 INFO:tasks.workunit.client.0.smithi107.stdout:6/561: mknod df/d38/d4f/daf/cb9 0 2024-03-04T13:43:12.856 INFO:tasks.workunit.client.0.smithi107.stdout:3/519: mkdir d1/de/d18/d33/d48/d71/d4c/daa 0 2024-03-04T13:43:12.857 INFO:tasks.workunit.client.0.smithi107.stdout:9/507: rename d4/da/d11/d4c/d89/f94 to d4/d32/fa8 0 2024-03-04T13:43:12.857 INFO:tasks.workunit.client.0.smithi107.stdout:5/536: creat d5/d8/d2a/d43/fbb x:0 0 0 2024-03-04T13:43:12.857 INFO:tasks.workunit.client.0.smithi107.stdout:2/505: chown da/d49/d5c/la6 4860794 1 2024-03-04T13:43:12.862 INFO:tasks.workunit.client.0.smithi107.stdout:8/550: symlink d6/d16/la7 0 2024-03-04T13:43:12.862 INFO:tasks.workunit.client.0.smithi107.stdout:1/537: mkdir d1/d46/d6a/d4e/daa 0 2024-03-04T13:43:12.863 INFO:tasks.workunit.client.0.smithi107.stdout:1/538: readlink d1/d10/d16/d1b/l50 0 2024-03-04T13:43:12.864 INFO:tasks.workunit.client.0.smithi107.stdout:5/537: fdatasync d5/d79/d89/da3/d6a/f6d 0 2024-03-04T13:43:12.868 INFO:tasks.workunit.client.0.smithi107.stdout:5/538: write d5/f73 [2883087,44380] 0 2024-03-04T13:43:12.870 INFO:tasks.workunit.client.0.smithi107.stdout:8/551: mkdir d6/da/da8 0 2024-03-04T13:43:12.871 INFO:tasks.workunit.client.0.smithi107.stdout:9/508: creat d4/da/d11/d4c/d89/fa9 x:0 0 0 2024-03-04T13:43:12.885 INFO:tasks.workunit.client.0.smithi107.stdout:7/528: write d6/d1d/d64/d8e/d3d/f78 [972208,112221] 0 2024-03-04T13:43:12.885 INFO:tasks.workunit.client.0.smithi107.stdout:5/539: creat d5/d8/da/fbc x:0 0 0 2024-03-04T13:43:12.886 INFO:tasks.workunit.client.0.smithi107.stdout:9/509: mkdir d4/da/d11/d20/d2f/d9c/daa 0 2024-03-04T13:43:12.886 INFO:tasks.workunit.client.0.smithi107.stdout:3/520: dread d1/de/d18/d33/d48/d71/fa9 [0,4194304] 0 2024-03-04T13:43:12.887 INFO:tasks.workunit.client.0.smithi107.stdout:5/540: stat d5/d8/l3e 0 2024-03-04T13:43:12.891 INFO:tasks.workunit.client.0.smithi107.stdout:1/539: link d1/d46/d6a/df/d67/d6b/c99 d1/d46/d6a/df/d33/d43/d59/cab 0 2024-03-04T13:43:12.892 INFO:tasks.workunit.client.0.smithi107.stdout:7/529: stat d6/d17/f97 0 2024-03-04T13:43:12.896 INFO:tasks.workunit.client.0.smithi107.stdout:6/562: dwrite df/d2e/d36/d5b/f93 [0,4194304] 0 2024-03-04T13:43:12.905 INFO:tasks.workunit.client.0.smithi107.stdout:4/481: dwrite d0/d9/d18/f53 [0,4194304] 0 2024-03-04T13:43:12.907 INFO:tasks.workunit.client.0.smithi107.stdout:0/599: dwrite da/dc/dd/d3e/d26/d69/d6b/fd3 [0,4194304] 0 2024-03-04T13:43:12.910 INFO:tasks.workunit.client.0.smithi107.stdout:7/530: write d6/d13/d16/d19/d38/f47 [5028056,65874] 0 2024-03-04T13:43:12.914 INFO:tasks.workunit.client.0.smithi107.stdout:5/541: symlink d5/d8/d2a/d43/d5f/lbd 0 2024-03-04T13:43:12.916 INFO:tasks.workunit.client.1.smithi169.stdout:0/910: dwrite d5/d9/d3d/d59/d64/d7f/fa9 [0,4194304] 0 2024-03-04T13:43:12.917 INFO:tasks.workunit.client.0.smithi107.stdout:7/531: stat d6/c8 0 2024-03-04T13:43:12.918 INFO:tasks.workunit.client.0.smithi107.stdout:4/482: chown d0/d9/d18/d29/f55 0 1 2024-03-04T13:43:12.918 INFO:tasks.workunit.client.0.smithi107.stdout:1/540: mknod d1/d46/d6a/df/d33/d5d/d4c/cac 0 2024-03-04T13:43:12.918 INFO:tasks.workunit.client.0.smithi107.stdout:0/600: dread - da/dad/fae zero size 2024-03-04T13:43:12.919 INFO:tasks.workunit.client.0.smithi107.stdout:2/506: write da/d49/f65 [294816,98637] 0 2024-03-04T13:43:12.919 INFO:tasks.workunit.client.0.smithi107.stdout:9/510: dread d4/d34/f44 [0,4194304] 0 2024-03-04T13:43:12.934 INFO:tasks.workunit.client.0.smithi107.stdout:2/507: symlink da/df/d73/lac 0 2024-03-04T13:43:12.935 INFO:tasks.workunit.client.0.smithi107.stdout:0/601: symlink da/ld6 0 2024-03-04T13:43:12.936 INFO:tasks.workunit.client.0.smithi107.stdout:9/511: mknod d4/da/d11/d4c/d4d/d72/d7b/cab 0 2024-03-04T13:43:12.937 INFO:tasks.workunit.client.1.smithi169.stdout:0/911: dread d5/d9/dc/d12/d15/d19/f52 [0,4194304] 0 2024-03-04T13:43:12.940 INFO:tasks.workunit.client.0.smithi107.stdout:5/542: symlink d5/d8/da/db0/lbe 0 2024-03-04T13:43:12.941 INFO:tasks.workunit.client.0.smithi107.stdout:8/552: dwrite d6/da/d25/f4b [0,4194304] 0 2024-03-04T13:43:12.942 INFO:tasks.workunit.client.0.smithi107.stdout:6/563: link df/d2e/d36/f49 df/d17/d5c/db1/fba 0 2024-03-04T13:43:12.942 INFO:tasks.workunit.client.0.smithi107.stdout:8/553: fsync d6/f54 0 2024-03-04T13:43:12.943 INFO:tasks.workunit.client.1.smithi169.stdout:0/912: unlink d5/d4c/d65/c6f 0 2024-03-04T13:43:12.943 INFO:tasks.workunit.client.0.smithi107.stdout:7/532: creat d6/d1d/fb2 x:0 0 0 2024-03-04T13:43:12.946 INFO:tasks.workunit.client.0.smithi107.stdout:4/483: dread d0/d9/d10/f7a [0,4194304] 0 2024-03-04T13:43:12.946 INFO:tasks.workunit.client.0.smithi107.stdout:3/521: dread d1/d1e/d26/f3d [0,4194304] 0 2024-03-04T13:43:12.949 INFO:tasks.workunit.client.0.smithi107.stdout:5/543: symlink d5/dc/df/lbf 0 2024-03-04T13:43:12.950 INFO:tasks.workunit.client.0.smithi107.stdout:8/554: symlink d6/d16/d59/la9 0 2024-03-04T13:43:12.950 INFO:tasks.workunit.client.0.smithi107.stdout:2/508: dread - da/d34/f72 zero size 2024-03-04T13:43:12.950 INFO:tasks.workunit.client.1.smithi169.stdout:0/913: truncate d5/d9/dc/d12/d15/d19/d61/f2c 6111114 0 2024-03-04T13:43:12.951 INFO:tasks.workunit.client.0.smithi107.stdout:3/522: readlink d1/d1e/d26/l86 0 2024-03-04T13:43:12.952 INFO:tasks.workunit.client.0.smithi107.stdout:5/544: write d5/f73 [2223283,109363] 0 2024-03-04T13:43:12.953 INFO:tasks.workunit.client.0.smithi107.stdout:6/564: unlink df/d17/d5c/db1/fb2 0 2024-03-04T13:43:12.955 INFO:tasks.workunit.client.1.smithi169.stdout:0/914: chown d5/d9/dc/d12/d15/d19/d69/d110/ceb 988978 1 2024-03-04T13:43:12.957 INFO:tasks.workunit.client.0.smithi107.stdout:6/565: chown df/d38/c96 1321 1 2024-03-04T13:43:12.957 INFO:tasks.workunit.client.1.smithi169.stdout:1/976: sync 2024-03-04T13:43:12.959 INFO:tasks.workunit.client.0.smithi107.stdout:5/545: chown d5/d8/d14/d54 127 1 2024-03-04T13:43:12.959 INFO:tasks.workunit.client.0.smithi107.stdout:6/566: chown df/d38 12524 1 2024-03-04T13:43:12.959 INFO:tasks.workunit.client.0.smithi107.stdout:2/509: rmdir da/d44 39 2024-03-04T13:43:12.959 INFO:tasks.workunit.client.0.smithi107.stdout:4/484: rmdir d0/d9/d10 39 2024-03-04T13:43:12.961 INFO:tasks.workunit.client.1.smithi169.stdout:3/991: sync 2024-03-04T13:43:12.961 INFO:tasks.workunit.client.1.smithi169.stdout:1/977: write d4/d5/d36/d35/d16/d115/f12d [9652,8808] 0 2024-03-04T13:43:12.967 INFO:tasks.workunit.client.0.smithi107.stdout:7/533: creat d6/d1d/d64/d8e/d22/fb3 x:0 0 0 2024-03-04T13:43:12.967 INFO:tasks.workunit.client.0.smithi107.stdout:6/567: chown df/d17/d6b 90 1 2024-03-04T13:43:12.967 INFO:tasks.workunit.client.1.smithi169.stdout:1/978: symlink d4/d5/d2d/d4e/d71/d82/l13a 0 2024-03-04T13:43:12.969 INFO:tasks.workunit.client.1.smithi169.stdout:3/992: creat d7/de/d2b/d7d/d85/d13/d1a/d2d/d56/d5e/d67/d5c/d84/d95/f14a x:0 0 0 2024-03-04T13:43:12.970 INFO:tasks.workunit.client.1.smithi169.stdout:1/979: mknod d4/d5/d36/d35/c13b 0 2024-03-04T13:43:12.973 INFO:tasks.workunit.client.0.smithi107.stdout:1/541: truncate d1/d46/d6a/d17/f5a 1201657 0 2024-03-04T13:43:12.976 INFO:tasks.workunit.client.0.smithi107.stdout:6/568: fdatasync df/d38/f4e 0 2024-03-04T13:43:12.976 INFO:tasks.workunit.client.0.smithi107.stdout:8/555: mkdir d6/d16/d1f/d4c/da4/daa 0 2024-03-04T13:43:12.977 INFO:tasks.workunit.client.0.smithi107.stdout:4/485: rmdir d0/d9/d10/d6f 39 2024-03-04T13:43:12.978 INFO:tasks.workunit.client.1.smithi169.stdout:3/993: link d7/de/d2b/d7d/d85/d47/d63/d6c/f77 d7/de/d2b/d7d/d85/d13/d1a/d2d/d56/d5e/d67/d5c/d84/d105/f14b 0 2024-03-04T13:43:12.980 INFO:tasks.workunit.client.0.smithi107.stdout:7/534: write d6/d13/d16/d1c/faf [377748,89780] 0 2024-03-04T13:43:12.980 INFO:tasks.workunit.client.0.smithi107.stdout:1/542: creat d1/d10/d16/fad x:0 0 0 2024-03-04T13:43:12.981 INFO:tasks.workunit.client.0.smithi107.stdout:0/602: dread da/dc/fbb [0,4194304] 0 2024-03-04T13:43:12.984 INFO:tasks.workunit.client.1.smithi169.stdout:3/994: creat d7/de/d2b/d7d/d85/d13/d1a/d2d/dbe/d108/df7/f14c x:0 0 0 2024-03-04T13:43:12.985 INFO:tasks.workunit.client.0.smithi107.stdout:9/512: write d4/da/f4a [1055205,108188] 0 2024-03-04T13:43:12.986 INFO:tasks.workunit.client.0.smithi107.stdout:2/510: symlink da/d44/d8b/d98/lad 0 2024-03-04T13:43:12.987 INFO:tasks.workunit.client.0.smithi107.stdout:6/569: symlink df/d2e/d36/d77/d95/lbb 0 2024-03-04T13:43:12.988 INFO:tasks.workunit.client.1.smithi169.stdout:0/915: write d5/d9/dc/d12/d15/f9e [121551,82664] 0 2024-03-04T13:43:12.992 INFO:tasks.workunit.client.1.smithi169.stdout:3/995: read d7/de/d2b/d7d/d85/d13/d1a/d2d/d56/d5e/d67/d5c/d84/fac [131495,29574] 0 2024-03-04T13:43:12.994 INFO:tasks.workunit.client.0.smithi107.stdout:0/603: unlink da/dc/fb9 0 2024-03-04T13:43:12.994 INFO:tasks.workunit.client.0.smithi107.stdout:9/513: chown d4/da/d11/d20/d23/d7e/d84/l8a 4 1 2024-03-04T13:43:12.995 INFO:tasks.workunit.client.1.smithi169.stdout:0/916: symlink d5/d4c/d65/d95/db0/d10c/l139 0 2024-03-04T13:43:12.997 INFO:tasks.workunit.client.0.smithi107.stdout:5/546: getdents d5/d8/d14/d2d 0 2024-03-04T13:43:13.001 INFO:tasks.workunit.client.0.smithi107.stdout:5/547: chown d5/dc/f3d 222372433 1 2024-03-04T13:43:13.002 INFO:tasks.workunit.client.0.smithi107.stdout:2/511: creat da/d38/d3a/fae x:0 0 0 2024-03-04T13:43:13.004 INFO:tasks.workunit.client.0.smithi107.stdout:0/604: write da/dc/dd/d3e/d26/d69/d6b/fd3 [974314,126000] 0 2024-03-04T13:43:13.005 INFO:tasks.workunit.client.0.smithi107.stdout:5/548: readlink d5/dc/d22/l46 0 2024-03-04T13:43:13.005 INFO:tasks.workunit.client.0.smithi107.stdout:4/486: creat d0/d9/d10/d6f/f95 x:0 0 0 2024-03-04T13:43:13.007 INFO:tasks.workunit.client.1.smithi169.stdout:0/917: dread d5/d9/dc/fa6 [0,4194304] 0 2024-03-04T13:43:13.011 INFO:tasks.workunit.client.1.smithi169.stdout:1/980: dread d4/d5/d36/d35/d16/d3a/f122 [0,4194304] 0 2024-03-04T13:43:13.012 INFO:tasks.workunit.client.0.smithi107.stdout:4/487: write d0/d9/d10/d6f/f95 [896236,58774] 0 2024-03-04T13:43:13.012 INFO:tasks.workunit.client.1.smithi169.stdout:3/996: mknod d7/de/d2b/c14d 0 2024-03-04T13:43:13.012 INFO:tasks.workunit.client.0.smithi107.stdout:0/605: dread - da/dc/dd/d3e/d26/d88/dd2/d73/d78/fab zero size 2024-03-04T13:43:13.014 INFO:tasks.workunit.client.0.smithi107.stdout:1/543: dwrite d1/d46/d6a/df/d33/d5d/d24/f44 [4194304,4194304] 0 2024-03-04T13:43:13.015 INFO:tasks.workunit.client.1.smithi169.stdout:1/981: write d4/d5/d36/d35/d16/d3a/f119 [771260,119010] 0 2024-03-04T13:43:13.018 INFO:tasks.workunit.client.0.smithi107.stdout:9/514: mknod d4/da/d11/d20/d2f/d9c/d92/cac 0 2024-03-04T13:43:13.019 INFO:tasks.workunit.client.1.smithi169.stdout:1/982: chown d4/d5/d36/d35/d16/d81/db4/d104 30857 1 2024-03-04T13:43:13.019 INFO:tasks.workunit.client.0.smithi107.stdout:8/556: write d6/f45 [2054002,19870] 0 2024-03-04T13:43:13.021 INFO:tasks.workunit.client.1.smithi169.stdout:0/918: mkdir d5/d4c/d65/d95/dba/d13a 0 2024-03-04T13:43:13.022 INFO:tasks.workunit.client.0.smithi107.stdout:0/606: read da/dc/f7c [1483733,60727] 0 2024-03-04T13:43:13.024 INFO:tasks.workunit.client.0.smithi107.stdout:2/512: mkdir da/df/d27/d58/d5f/daf 0 2024-03-04T13:43:13.026 INFO:tasks.workunit.client.0.smithi107.stdout:0/607: fsync da/dc/dd/d1e/fb5 0 2024-03-04T13:43:13.032 INFO:tasks.workunit.client.1.smithi169.stdout:3/997: symlink d7/de/d2b/d7d/d85/d13/d5a/d65/da4/d104/l14e 0 2024-03-04T13:43:13.035 INFO:tasks.workunit.client.1.smithi169.stdout:3/998: chown d7/de/d2b/d7d/d85/d13/d1a/d2d/d56/d5e/d5f/l11d 2997 1 2024-03-04T13:43:13.037 INFO:tasks.workunit.client.0.smithi107.stdout:6/570: rename f0 to df/d2e/d36/fbc 0 2024-03-04T13:43:13.039 INFO:tasks.workunit.client.1.smithi169.stdout:0/919: creat d5/d9/d3d/d59/d64/d7f/f13b x:0 0 0 2024-03-04T13:43:13.041 INFO:tasks.workunit.client.1.smithi169.stdout:0/920: chown d5/d9/d3d/d59/c7a 10 1 2024-03-04T13:43:13.042 INFO:tasks.workunit.client.0.smithi107.stdout:4/488: rmdir d0/d9/d78 39 2024-03-04T13:43:13.042 INFO:tasks.workunit.client.1.smithi169.stdout:1/983: mknod d4/d5/d2d/d4e/d71/d26/d7e/df5/c13c 0 2024-03-04T13:43:13.045 INFO:tasks.workunit.client.0.smithi107.stdout:7/535: dwrite d6/d1d/d64/d8e/d3d/f43 [0,4194304] 0 2024-03-04T13:43:13.048 INFO:tasks.workunit.client.0.smithi107.stdout:1/544: unlink d1/d10/d16/d47/d71/c7d 0 2024-03-04T13:43:13.049 INFO:tasks.workunit.client.0.smithi107.stdout:4/489: write d0/d9/d18/f53 [4955012,93600] 0 2024-03-04T13:43:13.051 INFO:tasks.workunit.client.1.smithi169.stdout:1/984: write d4/d5/d36/d35/d8f/d93/f110 [148583,130195] 0 2024-03-04T13:43:13.052 INFO:tasks.workunit.client.1.smithi169.stdout:0/921: symlink d5/d9/d117/l13c 0 2024-03-04T13:43:13.053 INFO:tasks.workunit.client.0.smithi107.stdout:2/513: dwrite da/d49/d5c/f9e [0,4194304] 0 2024-03-04T13:43:13.055 INFO:tasks.workunit.client.1.smithi169.stdout:0/922: stat d5/d9/dc/d12/d15/d19/d61/c26 0 2024-03-04T13:43:13.059 INFO:tasks.workunit.client.0.smithi107.stdout:7/536: write d6/d13/d16/d1c/f1f [346605,99017] 0 2024-03-04T13:43:13.059 INFO:tasks.workunit.client.0.smithi107.stdout:0/608: rmdir da/dc/dd/d3e/d26/d69 39 2024-03-04T13:43:13.060 INFO:tasks.workunit.client.0.smithi107.stdout:5/549: creat d5/d8/d14/d2d/d92/fc0 x:0 0 0 2024-03-04T13:43:13.062 INFO:tasks.workunit.client.0.smithi107.stdout:5/550: dread - d5/d8/d2a/d43/f57 zero size 2024-03-04T13:43:13.063 INFO:tasks.workunit.client.0.smithi107.stdout:0/609: chown da/dc/dd/d3e/d26/d88/cc6 295 1 2024-03-04T13:43:13.064 INFO:tasks.workunit.client.0.smithi107.stdout:8/557: dread d6/d16/d1f/f23 [0,4194304] 0 2024-03-04T13:43:13.067 INFO:tasks.workunit.client.1.smithi169.stdout:0/923: read d5/d4c/d65/d95/fb9 [247310,117928] 0 2024-03-04T13:43:13.067 INFO:tasks.workunit.client.0.smithi107.stdout:8/558: chown d6/da/f11 0 1 2024-03-04T13:43:13.068 INFO:tasks.workunit.client.1.smithi169.stdout:1/985: dread d4/d5/d36/f67 [4194304,4194304] 0 2024-03-04T13:43:13.077 INFO:tasks.workunit.client.0.smithi107.stdout:9/515: link d4/da/d11/d4c/d4d/f54 d4/da/d11/d20/d2f/d9c/daa/fad 0 2024-03-04T13:43:13.082 INFO:tasks.workunit.client.1.smithi169.stdout:3/999: dwrite d7/de/d2b/d7d/d85/d18/d21/fe4 [0,4194304] 0 2024-03-04T13:43:13.083 INFO:tasks.workunit.client.1.smithi169.stdout:1/986: creat d4/d5/d36/da9/f13d x:0 0 0 2024-03-04T13:43:13.088 INFO:tasks.workunit.client.0.smithi107.stdout:1/545: unlink d1/d46/d6a/d75/f95 0 2024-03-04T13:43:13.090 INFO:tasks.workunit.client.1.smithi169.stdout:1/987: rename d4/d5/d2d to d4/d5/d2d/d4e/d71/d26/d54/d13e 22 2024-03-04T13:43:13.091 INFO:tasks.workunit.client.0.smithi107.stdout:1/546: write d1/d46/d6a/df/d33/d5d/d24/f52 [3546310,14003] 0 2024-03-04T13:43:13.095 INFO:tasks.workunit.client.0.smithi107.stdout:2/514: rename da/d38/f9f to da/df/fb0 0 2024-03-04T13:43:13.099 INFO:tasks.workunit.client.1.smithi169.stdout:0/924: dwrite d5/d9/dc/d12/d15/d19/d61/d71/dbb/f105 [0,4194304] 0 2024-03-04T13:43:13.103 INFO:tasks.workunit.client.0.smithi107.stdout:5/551: mkdir d5/d8/d14/d2d/d92/dc1 0 2024-03-04T13:43:13.106 INFO:tasks.workunit.client.0.smithi107.stdout:6/571: mkdir df/d2e/d36/d5b/d7d/dbd 0 2024-03-04T13:43:13.108 INFO:tasks.workunit.client.0.smithi107.stdout:6/572: dread - df/d2e/d36/d5b/fa1 zero size 2024-03-04T13:43:13.108 INFO:tasks.workunit.client.0.smithi107.stdout:8/559: dread - d6/d16/d59/d3e/d67/f6f zero size 2024-03-04T13:43:13.109 INFO:tasks.workunit.client.0.smithi107.stdout:9/516: symlink d4/da/d11/d20/d23/d7e/lae 0 2024-03-04T13:43:13.112 INFO:tasks.workunit.client.0.smithi107.stdout:9/517: fsync d4/da/d11/d20/d39/f90 0 2024-03-04T13:43:13.113 INFO:tasks.workunit.client.0.smithi107.stdout:4/490: dwrite d0/d9/de/d14/f4e [0,4194304] 0 2024-03-04T13:43:13.114 INFO:tasks.workunit.client.0.smithi107.stdout:9/518: dread - d4/da/d11/d20/d39/fa7 zero size 2024-03-04T13:43:13.121 INFO:tasks.workunit.client.0.smithi107.stdout:2/515: fsync da/df/fa2 0 2024-03-04T13:43:13.122 INFO:tasks.workunit.client.0.smithi107.stdout:1/547: creat d1/d46/d6a/df/d33/d5d/d24/fae x:0 0 0 2024-03-04T13:43:13.123 INFO:tasks.workunit.client.0.smithi107.stdout:7/537: symlink d6/d13/d16/d19/d38/lb4 0 2024-03-04T13:43:13.124 INFO:tasks.workunit.client.0.smithi107.stdout:2/516: write da/df/d85/f8d [1045995,92488] 0 2024-03-04T13:43:13.129 INFO:tasks.workunit.client.0.smithi107.stdout:5/552: unlink d5/d8/d14/c28 0 2024-03-04T13:43:13.131 INFO:tasks.workunit.client.0.smithi107.stdout:7/538: dread d6/d13/d16/d19/d38/d62/fad [0,4194304] 0 2024-03-04T13:43:13.136 INFO:tasks.workunit.client.0.smithi107.stdout:9/519: chown d4/da/l2e 94312476 1 2024-03-04T13:43:13.136 INFO:tasks.workunit.client.0.smithi107.stdout:3/523: sync 2024-03-04T13:43:13.137 INFO:tasks.workunit.client.0.smithi107.stdout:0/610: write da/f2a [249732,84556] 0 2024-03-04T13:43:13.139 INFO:tasks.workunit.client.0.smithi107.stdout:1/548: mkdir d1/d46/d6a/d65/daf 0 2024-03-04T13:43:13.139 INFO:tasks.workunit.client.0.smithi107.stdout:3/524: read d1/d2d/f44 [7782756,70188] 0 2024-03-04T13:43:13.140 INFO:tasks.workunit.client.0.smithi107.stdout:3/525: chown d1/de/c9a 3305502 1 2024-03-04T13:43:13.141 INFO:tasks.workunit.client.0.smithi107.stdout:3/526: stat d1/de/d11/f46 0 2024-03-04T13:43:13.143 INFO:tasks.workunit.client.0.smithi107.stdout:3/527: fsync d1/de/d18/f8b 0 2024-03-04T13:43:13.146 INFO:tasks.workunit.client.0.smithi107.stdout:8/560: symlink d6/da/da8/lab 0 2024-03-04T13:43:13.148 INFO:tasks.workunit.client.0.smithi107.stdout:7/539: mknod d6/d13/d16/d1c/cb5 0 2024-03-04T13:43:13.150 INFO:tasks.workunit.client.0.smithi107.stdout:7/540: fsync d6/d13/d16/d19/d38/d2c/f9f 0 2024-03-04T13:43:13.152 INFO:tasks.workunit.client.0.smithi107.stdout:0/611: stat da/d7e/f7f 0 2024-03-04T13:43:13.153 INFO:tasks.workunit.client.0.smithi107.stdout:9/520: unlink d4/da/d11/d20/d39/f9a 0 2024-03-04T13:43:13.155 INFO:tasks.workunit.client.0.smithi107.stdout:9/521: fdatasync d4/da/d11/d4c/d89/fa2 0 2024-03-04T13:43:13.156 INFO:tasks.workunit.client.0.smithi107.stdout:1/549: creat d1/d10/fb0 x:0 0 0 2024-03-04T13:43:13.156 INFO:tasks.workunit.client.0.smithi107.stdout:2/517: mkdir da/df/d27/d58/d5f/daf/db1 0 2024-03-04T13:43:13.161 INFO:tasks.workunit.client.0.smithi107.stdout:8/561: creat d6/d16/d1f/d63/d73/fac x:0 0 0 2024-03-04T13:43:13.169 INFO:tasks.workunit.client.0.smithi107.stdout:0/612: creat da/dc/dd/d3e/d26/d88/fd7 x:0 0 0 2024-03-04T13:43:13.175 INFO:tasks.workunit.client.0.smithi107.stdout:4/491: getdents d0/d9/d18 0 2024-03-04T13:43:13.177 INFO:tasks.workunit.client.0.smithi107.stdout:8/562: readlink d6/d16/d59/d3e/l94 0 2024-03-04T13:43:13.178 INFO:tasks.workunit.client.0.smithi107.stdout:0/613: mknod da/dc/dd/d3e/d26/d88/dd2/d73/cd8 0 2024-03-04T13:43:13.180 INFO:tasks.workunit.client.0.smithi107.stdout:5/553: getdents d5/dc 0 2024-03-04T13:43:13.182 INFO:tasks.workunit.client.0.smithi107.stdout:4/492: readlink d0/d9/d10/d6f/l94 0 2024-03-04T13:43:13.183 INFO:tasks.workunit.client.0.smithi107.stdout:7/541: creat d6/d13/d16/d19/d38/fb6 x:0 0 0 2024-03-04T13:43:13.186 INFO:tasks.workunit.client.0.smithi107.stdout:3/528: getdents d1/de/d18/d33/d48 0 2024-03-04T13:43:13.187 INFO:tasks.workunit.client.0.smithi107.stdout:2/518: dwrite da/df/fb0 [0,4194304] 0 2024-03-04T13:43:13.200 INFO:tasks.workunit.client.1.smithi169.stdout:1/988: write d4/d5/d36/d35/d16/f4d [4192288,70036] 0 2024-03-04T13:43:13.203 INFO:tasks.workunit.client.1.smithi169.stdout:1/989: creat d4/d5/d2d/d4e/d71/d82/f13f x:0 0 0 2024-03-04T13:43:13.207 INFO:tasks.workunit.client.1.smithi169.stdout:1/990: creat d4/d5/d2d/d4e/d71/d26/d7e/df5/f140 x:0 0 0 2024-03-04T13:43:13.209 INFO:tasks.workunit.client.1.smithi169.stdout:1/991: readlink d4/d5/d2d/d4e/d71/d82/l13a 0 2024-03-04T13:43:13.226 INFO:tasks.workunit.client.1.smithi169.stdout:1/992: dwrite d4/d5/d36/d75/dba/fbb [0,4194304] 0 2024-03-04T13:43:13.227 INFO:tasks.workunit.client.1.smithi169.stdout:1/993: stat d4/d5/d36/d75/dba/lbf 0 2024-03-04T13:43:13.231 INFO:tasks.workunit.client.1.smithi169.stdout:1/994: creat d4/d5/d2d/d4e/d71/d26/d7e/d11c/f141 x:0 0 0 2024-03-04T13:43:13.235 INFO:tasks.workunit.client.1.smithi169.stdout:1/995: fdatasync d4/d1d/f34 0 2024-03-04T13:43:13.238 INFO:tasks.workunit.client.1.smithi169.stdout:1/996: rename d4/d5/d36/c69 to d4/d5/d36/df1/c142 0 2024-03-04T13:43:13.253 INFO:tasks.workunit.client.1.smithi169.stdout:1/997: dwrite d4/d5/d2d/d4e/d71/d26/daf/fee [0,4194304] 0 2024-03-04T13:43:13.258 INFO:tasks.workunit.client.1.smithi169.stdout:1/998: read d4/d5/d2d/f6c [1799826,128513] 0 2024-03-04T13:43:13.263 INFO:tasks.workunit.client.1.smithi169.stdout:1/999: mkdir d4/d5/d2d/d4e/d114/d143 0 2024-03-04T13:43:13.275 INFO:tasks.workunit.client.1.smithi169.stdout:0/925: sync 2024-03-04T13:43:13.276 INFO:tasks.workunit.client.0.smithi107.stdout:2/519: rename da/df/d3e/f8e to da/df/d73/daa/fb2 0 2024-03-04T13:43:13.276 INFO:tasks.workunit.client.0.smithi107.stdout:3/529: read d1/d29/f2a [375925,12320] 0 2024-03-04T13:43:13.276 INFO:tasks.workunit.client.0.smithi107.stdout:8/563: creat d6/da/fad x:0 0 0 2024-03-04T13:43:13.280 INFO:tasks.workunit.client.0.smithi107.stdout:3/530: chown d1/de/f32 23807656 1 2024-03-04T13:43:13.281 INFO:tasks.workunit.client.0.smithi107.stdout:7/542: getdents d6/d17 0 2024-03-04T13:43:13.285 INFO:tasks.workunit.client.0.smithi107.stdout:8/564: dread - d6/d16/d1f/d63/d73/fac zero size 2024-03-04T13:43:13.288 INFO:tasks.workunit.client.0.smithi107.stdout:6/573: sync 2024-03-04T13:43:13.289 INFO:tasks.workunit.client.0.smithi107.stdout:2/520: rename da/df/d27/d58/d5f/daf to da/df/d73/db3 0 2024-03-04T13:43:13.290 INFO:tasks.workunit.client.0.smithi107.stdout:5/554: dwrite d5/d8/f1f [0,4194304] 0 2024-03-04T13:43:13.291 INFO:tasks.workunit.client.1.smithi169.stdout:0/926: dwrite d5/d9/dc/d12/d15/d19/d69/d110/fd2 [0,4194304] 0 2024-03-04T13:43:13.293 INFO:tasks.workunit.client.0.smithi107.stdout:5/555: fsync d5/d66/f78 0 2024-03-04T13:43:13.294 INFO:tasks.workunit.client.0.smithi107.stdout:7/543: rmdir d6/d1d/d64/d89 0 2024-03-04T13:43:13.296 INFO:tasks.workunit.client.0.smithi107.stdout:4/493: dwrite d0/d9/d10/f4a [0,4194304] 0 2024-03-04T13:43:13.297 INFO:tasks.workunit.client.0.smithi107.stdout:2/521: mknod da/df/d3e/cb4 0 2024-03-04T13:43:13.297 INFO:tasks.workunit.client.1.smithi169.stdout:0/927: creat d5/d9/d3d/d59/d64/d7f/f13d x:0 0 0 2024-03-04T13:43:13.298 INFO:tasks.workunit.client.0.smithi107.stdout:5/556: readlink d5/d8/d2a/l3c 0 2024-03-04T13:43:13.302 INFO:tasks.workunit.client.0.smithi107.stdout:8/565: write d6/d16/d1f/d63/d73/fa2 [4566543,127478] 0 2024-03-04T13:43:13.306 INFO:tasks.workunit.client.1.smithi169.stdout:0/928: rename d5/d9/dc/d12/d15/lf9 to d5/d4c/d65/d95/db0/l13e 0 2024-03-04T13:43:13.316 INFO:tasks.workunit.client.1.smithi169.stdout:0/929: rmdir d5/d9/d3d/d5d/de4/d111 0 2024-03-04T13:43:13.317 INFO:tasks.workunit.client.0.smithi107.stdout:2/522: chown da/d44/l82 0 1 2024-03-04T13:43:13.320 INFO:tasks.workunit.client.0.smithi107.stdout:6/574: write df/d17/d24/f81 [557088,82870] 0 2024-03-04T13:43:13.326 INFO:tasks.workunit.client.0.smithi107.stdout:8/566: getdents d6/d16/d59/d7c 0 2024-03-04T13:43:13.326 INFO:tasks.workunit.client.0.smithi107.stdout:5/557: truncate d5/d8/d2a/d43/fbb 929132 0 2024-03-04T13:43:13.326 INFO:tasks.workunit.client.0.smithi107.stdout:6/575: write df/d38/f5f [2362529,85565] 0 2024-03-04T13:43:13.329 INFO:tasks.workunit.client.0.smithi107.stdout:3/531: dwrite d1/de/d18/f59 [4194304,4194304] 0 2024-03-04T13:43:13.331 INFO:tasks.workunit.client.0.smithi107.stdout:2/523: rename da/d6c/l99 to da/d38/lb5 0 2024-03-04T13:43:13.336 INFO:tasks.workunit.client.0.smithi107.stdout:5/558: creat d5/dc/d22/fc2 x:0 0 0 2024-03-04T13:43:13.342 INFO:tasks.workunit.client.1.smithi169.stdout:0/930: dwrite d5/d9/dc/d12/d15/d19/d77/d94/fab [0,4194304] 0 2024-03-04T13:43:13.342 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:13 smithi107 ceph-mon[28739]: pgmap v58: 65 pgs: 65 active+clean; 2.5 GiB data, 8.8 GiB used, 528 GiB / 536 GiB avail; 56 MiB/s rd, 188 MiB/s wr, 374 op/s 2024-03-04T13:43:13.342 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:13 smithi107 ceph-mon[28739]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' 2024-03-04T13:43:13.342 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:13 smithi107 ceph-mon[28739]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' 2024-03-04T13:43:13.342 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:13 smithi107 ceph-mon[28739]: from='mgr.24331 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:43:13.348 INFO:tasks.workunit.client.0.smithi107.stdout:7/544: dwrite d6/d13/d16/d1c/f1f [4194304,4194304] 0 2024-03-04T13:43:13.348 INFO:tasks.workunit.client.0.smithi107.stdout:6/576: symlink df/d17/d5c/lbe 0 2024-03-04T13:43:13.348 INFO:tasks.workunit.client.1.smithi169.stdout:0/931: fdatasync d5/d9/dc/d12/d15/d19/fe8 0 2024-03-04T13:43:13.352 INFO:tasks.workunit.client.0.smithi107.stdout:6/577: symlink df/d2e/lbf 0 2024-03-04T13:43:13.353 INFO:tasks.workunit.client.1.smithi169.stdout:0/932: stat d5/d9/dc/d12/d15/d19/d61/d71/dbb/l106 0 2024-03-04T13:43:13.356 INFO:tasks.workunit.client.0.smithi107.stdout:3/532: getdents d1/de/d18/d33/d48/d71 0 2024-03-04T13:43:13.357 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:13 smithi169 ceph-mon[36313]: pgmap v58: 65 pgs: 65 active+clean; 2.5 GiB data, 8.8 GiB used, 528 GiB / 536 GiB avail; 56 MiB/s rd, 188 MiB/s wr, 374 op/s 2024-03-04T13:43:13.357 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:13 smithi169 ceph-mon[36313]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' 2024-03-04T13:43:13.357 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:13 smithi169 ceph-mon[36313]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' 2024-03-04T13:43:13.357 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:13 smithi169 ceph-mon[36313]: from='mgr.24331 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:43:13.359 INFO:tasks.workunit.client.1.smithi169.stdout:0/933: rmdir d5/d9/dc/d12 39 2024-03-04T13:43:13.360 INFO:tasks.workunit.client.0.smithi107.stdout:6/578: write df/d17/f23 [5232513,87591] 0 2024-03-04T13:43:13.361 INFO:tasks.workunit.client.0.smithi107.stdout:3/533: dread - d1/de/d3f/f89 zero size 2024-03-04T13:43:13.365 INFO:tasks.workunit.client.0.smithi107.stdout:7/545: rename d6/d13/d16/d19/d38/d37/c4c to d6/d1d/d64/d8e/d3d/cb7 0 2024-03-04T13:43:13.369 INFO:tasks.workunit.client.0.smithi107.stdout:6/579: fdatasync df/d17/d24/d40/d63/da3/d7e/d86/f97 0 2024-03-04T13:43:13.369 INFO:tasks.workunit.client.0.smithi107.stdout:3/534: mkdir d1/de/d18/d33/d48/d71/d4c/daa/dab 0 2024-03-04T13:43:13.371 INFO:tasks.workunit.client.0.smithi107.stdout:4/494: dread d0/d9/f5b [0,4194304] 0 2024-03-04T13:43:13.377 INFO:tasks.workunit.client.1.smithi169.stdout:0/934: dread d5/d9/dc/f1d [0,4194304] 0 2024-03-04T13:43:13.377 INFO:tasks.workunit.client.0.smithi107.stdout:4/495: symlink d0/d9/d1e/l96 0 2024-03-04T13:43:13.378 INFO:tasks.workunit.client.0.smithi107.stdout:3/535: symlink d1/d41/d81/lac 0 2024-03-04T13:43:13.383 INFO:tasks.workunit.client.0.smithi107.stdout:3/536: fdatasync d1/de/f68 0 2024-03-04T13:43:13.383 INFO:tasks.workunit.client.1.smithi169.stdout:0/935: dread d5/d9/dc/d12/d15/d19/f52 [0,4194304] 0 2024-03-04T13:43:13.388 INFO:tasks.workunit.client.0.smithi107.stdout:5/559: dread d5/d8/d2a/f40 [0,4194304] 0 2024-03-04T13:43:13.398 INFO:tasks.workunit.client.0.smithi107.stdout:6/580: dwrite f6 [4194304,4194304] 0 2024-03-04T13:43:13.402 INFO:tasks.workunit.client.0.smithi107.stdout:6/581: chown df/d2e/d36/d71 0 1 2024-03-04T13:43:13.403 INFO:tasks.workunit.client.0.smithi107.stdout:7/546: dwrite d6/d17/f8f [0,4194304] 0 2024-03-04T13:43:13.404 INFO:tasks.workunit.client.1.smithi169.stdout:0/936: dread d5/d9/dc/d12/f5c [0,4194304] 0 2024-03-04T13:43:13.407 INFO:tasks.workunit.client.0.smithi107.stdout:8/567: dread f3 [0,4194304] 0 2024-03-04T13:43:13.407 INFO:tasks.workunit.client.0.smithi107.stdout:3/537: truncate d1/de/d18/f59 3097940 0 2024-03-04T13:43:13.407 INFO:tasks.workunit.client.1.smithi169.stdout:0/937: creat d5/d9/dc/d12/f13f x:0 0 0 2024-03-04T13:43:13.411 INFO:tasks.workunit.client.1.smithi169.stdout:0/938: creat d5/d9/dc/d12/d15/d19/d77/d94/f140 x:0 0 0 2024-03-04T13:43:13.413 INFO:tasks.workunit.client.0.smithi107.stdout:4/496: dwrite d0/d9/de/f41 [0,4194304] 0 2024-03-04T13:43:13.414 INFO:tasks.workunit.client.0.smithi107.stdout:3/538: readlink d1/d1e/d26/l82 0 2024-03-04T13:43:13.415 INFO:tasks.workunit.client.0.smithi107.stdout:1/550: sync 2024-03-04T13:43:13.415 INFO:tasks.workunit.client.0.smithi107.stdout:9/522: sync 2024-03-04T13:43:13.429 INFO:tasks.workunit.client.0.smithi107.stdout:1/551: write d1/d46/d6a/df/d67/d6b/f9e [112401,53761] 0 2024-03-04T13:43:13.432 INFO:tasks.workunit.client.1.smithi169.stdout:0/939: dwrite d5/d9/d3d/d59/d64/d7f/f13d [0,4194304] 0 2024-03-04T13:43:13.433 INFO:tasks.workunit.client.0.smithi107.stdout:4/497: getdents d0/d2f/d7d 0 2024-03-04T13:43:13.433 INFO:tasks.workunit.client.0.smithi107.stdout:9/523: mknod d4/da/d11/d20/d23/d2c/caf 0 2024-03-04T13:43:13.444 INFO:tasks.workunit.client.0.smithi107.stdout:6/582: dwrite df/d17/d5c/f67 [0,4194304] 0 2024-03-04T13:43:13.448 INFO:tasks.workunit.client.1.smithi169.stdout:0/940: write d5/d9/d3d/d5d/fec [203249,42633] 0 2024-03-04T13:43:13.450 INFO:tasks.workunit.client.0.smithi107.stdout:9/524: creat d4/d34/fb0 x:0 0 0 2024-03-04T13:43:13.450 INFO:tasks.workunit.client.0.smithi107.stdout:3/539: dread d1/de/d18/d33/d48/d71/fa9 [0,4194304] 0 2024-03-04T13:43:13.451 INFO:tasks.workunit.client.0.smithi107.stdout:4/498: chown d0/d9/de/d14/dc/d21/f3f 56422392 1 2024-03-04T13:43:13.455 INFO:tasks.workunit.client.0.smithi107.stdout:4/499: stat d0/d9/de/d14/dc/d21/c56 0 2024-03-04T13:43:13.457 INFO:tasks.workunit.client.0.smithi107.stdout:5/560: dread d5/dc/df/f2e [0,4194304] 0 2024-03-04T13:43:13.457 INFO:tasks.workunit.client.0.smithi107.stdout:3/540: rmdir d1/de/d3f 39 2024-03-04T13:43:13.462 INFO:tasks.workunit.client.0.smithi107.stdout:4/500: mkdir d0/d9/de/d14/dc/d97 0 2024-03-04T13:43:13.464 INFO:tasks.workunit.client.0.smithi107.stdout:4/501: dread - d0/d30/f90 zero size 2024-03-04T13:43:13.465 INFO:tasks.workunit.client.0.smithi107.stdout:9/525: rename d4/da/d11/d4c/d4d/d72/d7b/c7f to d4/da/d11/d20/d23/d7e/cb1 0 2024-03-04T13:43:13.465 INFO:tasks.workunit.client.1.smithi169.stdout:0/941: dwrite d5/d9/ffd [0,4194304] 0 2024-03-04T13:43:13.465 INFO:tasks.workunit.client.0.smithi107.stdout:3/541: mkdir d1/de/dad 0 2024-03-04T13:43:13.466 INFO:tasks.workunit.client.0.smithi107.stdout:8/568: dread d6/d16/f27 [4194304,4194304] 0 2024-03-04T13:43:13.467 INFO:tasks.workunit.client.1.smithi169.stdout:0/942: write d5/d9/dc/d12/d66/f78 [180043,85143] 0 2024-03-04T13:43:13.471 INFO:tasks.workunit.client.1.smithi169.stdout:0/943: creat d5/d4c/d65/d95/dba/f141 x:0 0 0 2024-03-04T13:43:13.473 INFO:tasks.workunit.client.0.smithi107.stdout:9/526: mknod d4/cb2 0 2024-03-04T13:43:13.474 INFO:tasks.workunit.client.0.smithi107.stdout:4/502: mknod d0/d9/d18/d29/c98 0 2024-03-04T13:43:13.476 INFO:tasks.workunit.client.1.smithi169.stdout:0/944: write d5/d9/dc/d12/d15/f74 [5179240,39403] 0 2024-03-04T13:43:13.482 INFO:tasks.workunit.client.0.smithi107.stdout:1/552: dwrite d1/d46/d6a/d4e/f6c [0,4194304] 0 2024-03-04T13:43:13.484 INFO:tasks.workunit.client.0.smithi107.stdout:9/527: write d4/da/d11/d20/d23/d2c/f7c [647848,79124] 0 2024-03-04T13:43:13.486 INFO:tasks.workunit.client.0.smithi107.stdout:4/503: read d0/d9/de/d14/dc/f69 [145017,78480] 0 2024-03-04T13:43:13.486 INFO:tasks.workunit.client.1.smithi169.stdout:0/945: truncate d5/d9/dc/d12/d15/d19/f52 1022351 0 2024-03-04T13:43:13.486 INFO:tasks.workunit.client.0.smithi107.stdout:6/583: dread fe [0,4194304] 0 2024-03-04T13:43:13.490 INFO:tasks.workunit.client.0.smithi107.stdout:8/569: rename d6/da/l8c to d6/d16/d1f/lae 0 2024-03-04T13:43:13.491 INFO:tasks.workunit.client.1.smithi169.stdout:0/946: mknod d5/d4c/d65/d95/c142 0 2024-03-04T13:43:13.493 INFO:tasks.workunit.client.0.smithi107.stdout:9/528: write d4/da/d11/d4c/d89/f9f [927956,48904] 0 2024-03-04T13:43:13.495 INFO:tasks.workunit.client.1.smithi169.stdout:0/947: dread - d5/d9/dc/d12/d15/d19/d77/ff7 zero size 2024-03-04T13:43:13.496 INFO:tasks.workunit.client.0.smithi107.stdout:8/570: fsync d6/da/d25/d5f/f87 0 2024-03-04T13:43:13.496 INFO:tasks.workunit.client.0.smithi107.stdout:9/529: chown d4/da/d11/d4c/d4d/l69 3239 1 2024-03-04T13:43:13.499 INFO:tasks.workunit.client.1.smithi169.stdout:0/948: mkdir d5/d9/d3d/d5d/de4/def/d143 0 2024-03-04T13:43:13.500 INFO:tasks.workunit.client.1.smithi169.stdout:0/949: chown d5/d4c/cf6 14 1 2024-03-04T13:43:13.501 INFO:tasks.workunit.client.0.smithi107.stdout:3/542: write d1/de/d18/d33/d48/d71/d4c/f57 [929170,31457] 0 2024-03-04T13:43:13.502 INFO:tasks.workunit.client.0.smithi107.stdout:8/571: mknod d6/caf 0 2024-03-04T13:43:13.503 INFO:tasks.workunit.client.1.smithi169.stdout:0/950: creat d5/d9/d3d/d59/d64/d7f/f144 x:0 0 0 2024-03-04T13:43:13.506 INFO:tasks.workunit.client.1.smithi169.stdout:0/951: symlink d5/d9/d3d/d5d/de4/def/l145 0 2024-03-04T13:43:13.507 INFO:tasks.workunit.client.0.smithi107.stdout:3/543: write d1/de/d4b/f7a [1728346,91504] 0 2024-03-04T13:43:13.509 INFO:tasks.workunit.client.1.smithi169.stdout:0/952: rmdir d5/d9/d117 39 2024-03-04T13:43:13.509 INFO:tasks.workunit.client.0.smithi107.stdout:9/530: getdents d4/da/d11/d20/d23/d7e 0 2024-03-04T13:43:13.512 INFO:tasks.workunit.client.0.smithi107.stdout:6/584: write fe [1978736,83187] 0 2024-03-04T13:43:13.512 INFO:tasks.workunit.client.0.smithi107.stdout:3/544: dread - d1/de/d18/d1f/f94 zero size 2024-03-04T13:43:13.513 INFO:tasks.workunit.client.1.smithi169.stdout:0/953: write d5/d9/dc/d12/d15/d19/d77/fcf [641349,2891] 0 2024-03-04T13:43:13.518 INFO:tasks.workunit.client.0.smithi107.stdout:6/585: readlink df/d17/d24/d30/d3b/la6 0 2024-03-04T13:43:13.520 INFO:tasks.workunit.client.1.smithi169.stdout:0/954: link d5/d4c/d65/d95/db0/f137 d5/d9/d3d/d59/f146 0 2024-03-04T13:43:13.520 INFO:tasks.workunit.client.0.smithi107.stdout:4/504: dwrite d0/d9/d1e/f75 [0,4194304] 0 2024-03-04T13:43:13.521 INFO:tasks.workunit.client.0.smithi107.stdout:0/614: sync 2024-03-04T13:43:13.522 INFO:tasks.workunit.client.1.smithi169.stdout:0/955: mknod d5/d9/d3d/c147 0 2024-03-04T13:43:13.527 INFO:tasks.workunit.client.0.smithi107.stdout:6/586: mknod df/d17/d24/d30/d8b/cc0 0 2024-03-04T13:43:13.527 INFO:tasks.workunit.client.0.smithi107.stdout:3/545: truncate d1/de/d18/d33/d48/fa6 79300 0 2024-03-04T13:43:13.532 INFO:tasks.workunit.client.0.smithi107.stdout:6/587: fdatasync df/d38/f4e 0 2024-03-04T13:43:13.537 INFO:tasks.workunit.client.0.smithi107.stdout:1/553: dwrite d1/d46/d6a/d4e/d60/f53 [0,4194304] 0 2024-03-04T13:43:13.540 INFO:tasks.workunit.client.0.smithi107.stdout:9/531: write d4/da/f1b [319388,16334] 0 2024-03-04T13:43:13.548 INFO:tasks.workunit.client.0.smithi107.stdout:1/554: mkdir d1/d46/d6a/d65/db1 0 2024-03-04T13:43:13.550 INFO:tasks.workunit.client.0.smithi107.stdout:9/532: unlink d4/da/d11/f13 0 2024-03-04T13:43:13.551 INFO:tasks.workunit.client.0.smithi107.stdout:8/572: dread d6/d16/d1f/f5c [0,4194304] 0 2024-03-04T13:43:13.552 INFO:tasks.workunit.client.0.smithi107.stdout:4/505: rename d0/d9/de/d14/d33/f91 to d0/d9/d10/f99 0 2024-03-04T13:43:13.554 INFO:tasks.workunit.client.0.smithi107.stdout:0/615: dwrite da/dc/dd/d3e/fd4 [0,4194304] 0 2024-03-04T13:43:13.554 INFO:tasks.workunit.client.0.smithi107.stdout:9/533: chown d4/da/l2e 456321 1 2024-03-04T13:43:13.555 INFO:tasks.workunit.client.0.smithi107.stdout:1/555: chown d1/d46/d6a/df/d33/d5d/d24/f52 1964167 1 2024-03-04T13:43:13.557 INFO:tasks.workunit.client.0.smithi107.stdout:4/506: write d0/d9/d10/d6f/f95 [1970325,9421] 0 2024-03-04T13:43:13.558 INFO:tasks.workunit.client.0.smithi107.stdout:1/556: fdatasync d1/d10/d16/f8b 0 2024-03-04T13:43:13.560 INFO:tasks.workunit.client.0.smithi107.stdout:3/546: write d1/d29/f2a [2744298,3655] 0 2024-03-04T13:43:13.562 INFO:tasks.workunit.client.0.smithi107.stdout:8/573: symlink d6/d16/lb0 0 2024-03-04T13:43:13.567 INFO:tasks.workunit.client.0.smithi107.stdout:9/534: mkdir d4/da/d11/d20/d23/d7e/d84/db3 0 2024-03-04T13:43:13.571 INFO:tasks.workunit.client.0.smithi107.stdout:3/547: unlink d1/d2d/f7f 0 2024-03-04T13:43:13.571 INFO:tasks.workunit.client.0.smithi107.stdout:4/507: mkdir d0/d9/de/d14/dc/d9a 0 2024-03-04T13:43:13.574 INFO:tasks.workunit.client.0.smithi107.stdout:9/535: write d4/da/d11/d20/d39/f7a [350899,87915] 0 2024-03-04T13:43:13.575 INFO:tasks.workunit.client.0.smithi107.stdout:4/508: chown d0/d9/de/d14/f65 1 1 2024-03-04T13:43:13.578 INFO:tasks.workunit.client.0.smithi107.stdout:0/616: link da/dc/d34/l79 da/dc/dd/d3e/d26/d88/dd2/d73/d78/d86/ld9 0 2024-03-04T13:43:13.578 INFO:tasks.workunit.client.0.smithi107.stdout:9/536: readlink d4/da/d11/d20/d39/l40 0 2024-03-04T13:43:13.579 INFO:tasks.workunit.client.0.smithi107.stdout:3/548: mkdir d1/d1e/dae 0 2024-03-04T13:43:13.584 INFO:tasks.workunit.client.0.smithi107.stdout:9/537: fsync d4/f79 0 2024-03-04T13:43:13.591 INFO:tasks.workunit.client.0.smithi107.stdout:9/538: stat d4/d34/f3f 0 2024-03-04T13:43:13.603 INFO:tasks.workunit.client.0.smithi107.stdout:9/539: creat d4/da/d11/d20/d2f/d9c/fb4 x:0 0 0 2024-03-04T13:43:13.608 INFO:tasks.workunit.client.0.smithi107.stdout:8/574: dwrite d6/d16/d59/d3e/f56 [0,4194304] 0 2024-03-04T13:43:13.613 INFO:tasks.workunit.client.0.smithi107.stdout:9/540: write d4/da/f96 [471818,67623] 0 2024-03-04T13:43:13.619 INFO:tasks.workunit.client.0.smithi107.stdout:4/509: dwrite d0/d9/de/d14/dc/f2d [0,4194304] 0 2024-03-04T13:43:13.621 INFO:tasks.workunit.client.0.smithi107.stdout:1/557: dread d1/d46/d6a/d4e/d60/f4f [0,4194304] 0 2024-03-04T13:43:13.621 INFO:tasks.workunit.client.0.smithi107.stdout:3/549: dwrite d1/de/d18/d33/d48/d71/f72 [0,4194304] 0 2024-03-04T13:43:13.627 INFO:tasks.workunit.client.0.smithi107.stdout:0/617: dread da/dc/dd/d3e/f35 [0,4194304] 0 2024-03-04T13:43:13.628 INFO:tasks.workunit.client.0.smithi107.stdout:8/575: creat d6/d16/fb1 x:0 0 0 2024-03-04T13:43:13.628 INFO:tasks.workunit.client.0.smithi107.stdout:9/541: mknod d4/da/d11/cb5 0 2024-03-04T13:43:13.633 INFO:tasks.workunit.client.0.smithi107.stdout:8/576: chown d6/d16/d1f/d63/d73/fa2 2 1 2024-03-04T13:43:13.634 INFO:tasks.workunit.client.0.smithi107.stdout:9/542: dread - d4/da/d11/d20/d64/d77/f82 zero size 2024-03-04T13:43:13.635 INFO:tasks.workunit.client.0.smithi107.stdout:8/577: read - d6/d16/d1f/d63/f74 zero size 2024-03-04T13:43:13.636 INFO:tasks.workunit.client.0.smithi107.stdout:3/550: mkdir d1/d1e/daf 0 2024-03-04T13:43:13.637 INFO:tasks.workunit.client.0.smithi107.stdout:8/578: read d6/d16/f27 [5554141,51758] 0 2024-03-04T13:43:13.640 INFO:tasks.workunit.client.0.smithi107.stdout:3/551: write d1/de/d11/f3e [2290621,50566] 0 2024-03-04T13:43:13.641 INFO:tasks.workunit.client.0.smithi107.stdout:4/510: write d0/d9/de/d14/dc/f67 [171715,25616] 0 2024-03-04T13:43:13.642 INFO:tasks.workunit.client.0.smithi107.stdout:1/558: link d1/d10/d16/d1b/d40/ca4 d1/d46/d6a/d65/cb2 0 2024-03-04T13:43:13.644 INFO:tasks.workunit.client.0.smithi107.stdout:9/543: mkdir d4/da/d11/d20/d2f/d9c/daa/db6 0 2024-03-04T13:43:13.648 INFO:tasks.workunit.client.0.smithi107.stdout:8/579: creat d6/d16/d1f/d4c/da5/fb2 x:0 0 0 2024-03-04T13:43:13.649 INFO:tasks.workunit.client.0.smithi107.stdout:4/511: rename d0/d2f/l54 to d0/d9/de/d14/d66/d7b/l9b 0 2024-03-04T13:43:13.650 INFO:tasks.workunit.client.0.smithi107.stdout:5/561: sync 2024-03-04T13:43:13.650 INFO:tasks.workunit.client.0.smithi107.stdout:2/524: sync 2024-03-04T13:43:13.651 INFO:tasks.workunit.client.0.smithi107.stdout:0/618: dwrite da/dc/dd/d3e/fac [0,4194304] 0 2024-03-04T13:43:13.651 INFO:tasks.workunit.client.0.smithi107.stdout:7/547: sync 2024-03-04T13:43:13.654 INFO:tasks.workunit.client.0.smithi107.stdout:1/559: fsync d1/d46/d6a/d17/f5a 0 2024-03-04T13:43:13.654 INFO:tasks.workunit.client.0.smithi107.stdout:4/512: chown d0/d2f/f39 72 1 2024-03-04T13:43:13.663 INFO:tasks.workunit.client.0.smithi107.stdout:7/548: creat d6/d13/d14/fb8 x:0 0 0 2024-03-04T13:43:13.664 INFO:tasks.workunit.client.0.smithi107.stdout:5/562: creat d5/d79/d89/da3/d69/db9/fc3 x:0 0 0 2024-03-04T13:43:13.667 INFO:tasks.workunit.client.0.smithi107.stdout:4/513: write d0/d9/de/d14/dc/f26 [5335875,97290] 0 2024-03-04T13:43:13.668 INFO:tasks.workunit.client.0.smithi107.stdout:9/544: chown d4/da/d11/d20/d23/f41 900 1 2024-03-04T13:43:13.676 INFO:tasks.workunit.client.0.smithi107.stdout:1/560: rename d1/d46/d6a/df/d33/d5d/d24/f44 to d1/d46/d6a/df/d67/d6b/fb3 0 2024-03-04T13:43:13.682 INFO:tasks.workunit.client.0.smithi107.stdout:3/552: dwrite d1/de/d18/d1f/d42/f52 [0,4194304] 0 2024-03-04T13:43:13.682 INFO:tasks.workunit.client.0.smithi107.stdout:0/619: dread da/f59 [0,4194304] 0 2024-03-04T13:43:13.683 INFO:tasks.workunit.client.0.smithi107.stdout:1/561: stat d1/d46/d6a/df/d33/d5d/d24/f62 0 2024-03-04T13:43:13.685 INFO:tasks.workunit.client.0.smithi107.stdout:9/545: rename d4/da/c1f to d4/da/cb7 0 2024-03-04T13:43:13.687 INFO:tasks.workunit.client.0.smithi107.stdout:7/549: dwrite d6/f3f [0,4194304] 0 2024-03-04T13:43:13.688 INFO:tasks.workunit.client.0.smithi107.stdout:9/546: readlink d4/da/d11/d20/d39/l50 0 2024-03-04T13:43:13.691 INFO:tasks.workunit.client.0.smithi107.stdout:3/553: symlink d1/de/d18/d33/d6b/lb0 0 2024-03-04T13:43:13.691 INFO:tasks.workunit.client.0.smithi107.stdout:1/562: read d1/d46/d6a/d75/f66 [92221,47272] 0 2024-03-04T13:43:13.692 INFO:tasks.workunit.client.0.smithi107.stdout:0/620: getdents da/dc/dd/dc9 0 2024-03-04T13:43:13.693 INFO:tasks.workunit.client.0.smithi107.stdout:4/514: truncate d0/d9/d1e/f75 3724488 0 2024-03-04T13:43:13.695 INFO:tasks.workunit.client.0.smithi107.stdout:1/563: readlink d1/d46/d6a/d65/l7e 0 2024-03-04T13:43:13.696 INFO:tasks.workunit.client.0.smithi107.stdout:3/554: dread - d1/de/d18/d1f/d42/f62 zero size 2024-03-04T13:43:13.699 INFO:tasks.workunit.client.0.smithi107.stdout:9/547: link d4/d32/fa8 d4/da/d11/d4c/d51/fb8 0 2024-03-04T13:43:13.705 INFO:tasks.workunit.client.0.smithi107.stdout:3/555: creat d1/de/d18/d33/d6b/fb1 x:0 0 0 2024-03-04T13:43:13.705 INFO:tasks.workunit.client.0.smithi107.stdout:5/563: dread d5/d8/fb [0,4194304] 0 2024-03-04T13:43:13.708 INFO:tasks.workunit.client.0.smithi107.stdout:9/548: creat d4/da/d11/d20/d23/d7e/d84/fb9 x:0 0 0 2024-03-04T13:43:13.709 INFO:tasks.workunit.client.0.smithi107.stdout:5/564: creat d5/dc/fc4 x:0 0 0 2024-03-04T13:43:13.715 INFO:tasks.workunit.client.0.smithi107.stdout:3/556: symlink d1/de/d18/d33/d48/d93/lb2 0 2024-03-04T13:43:13.718 INFO:tasks.workunit.client.0.smithi107.stdout:5/565: mknod d5/d66/d96/cc5 0 2024-03-04T13:43:13.719 INFO:tasks.workunit.client.0.smithi107.stdout:7/550: write d6/d1d/d64/d8e/f5c [959022,1964] 0 2024-03-04T13:43:13.721 INFO:tasks.workunit.client.0.smithi107.stdout:0/621: write da/dc/dd/d3e/f11 [3456653,129634] 0 2024-03-04T13:43:13.725 INFO:tasks.workunit.client.1.smithi169.stdout:0/956: sync 2024-03-04T13:43:13.727 INFO:tasks.workunit.client.1.smithi169.stdout:0/957: readlink d5/d4c/d65/d95/db0/d10c/l139 0 2024-03-04T13:43:13.729 INFO:tasks.workunit.client.0.smithi107.stdout:4/515: dwrite d0/d2f/f72 [0,4194304] 0 2024-03-04T13:43:13.732 INFO:tasks.workunit.client.0.smithi107.stdout:7/551: mkdir d6/d1d/d64/d9a/da9/d71/db9 0 2024-03-04T13:43:13.732 INFO:tasks.workunit.client.0.smithi107.stdout:7/552: readlink d6/d13/d16/d19/d38/d2c/l34 0 2024-03-04T13:43:13.732 INFO:tasks.workunit.client.0.smithi107.stdout:1/564: dwrite d1/d10/d16/fad [0,4194304] 0 2024-03-04T13:43:13.741 INFO:tasks.workunit.client.0.smithi107.stdout:0/622: symlink da/dc/dd/d1e/lda 0 2024-03-04T13:43:13.743 INFO:tasks.workunit.client.0.smithi107.stdout:1/565: creat d1/d46/d6a/df/d67/d6b/fb4 x:0 0 0 2024-03-04T13:43:13.745 INFO:tasks.workunit.client.0.smithi107.stdout:4/516: truncate d0/d9/de/d14/dc/d21/f3f 742418 0 2024-03-04T13:43:13.748 INFO:tasks.workunit.client.1.smithi169.stdout:0/958: dwrite d5/d9/dc/d12/d15/d19/d61/f5a [0,4194304] 0 2024-03-04T13:43:13.751 INFO:tasks.workunit.client.0.smithi107.stdout:4/517: fsync d0/d9/d10/f8c 0 2024-03-04T13:43:13.751 INFO:tasks.workunit.client.0.smithi107.stdout:3/557: dwrite d1/de/fa8 [0,4194304] 0 2024-03-04T13:43:13.754 INFO:tasks.workunit.client.0.smithi107.stdout:7/553: rename d6/d1d/d64/d8e/d3d/f78 to d6/d1d/d64/d8e/fba 0 2024-03-04T13:43:13.757 INFO:tasks.workunit.client.0.smithi107.stdout:4/518: chown d0/d9/de/d14/dc/f67 799990 1 2024-03-04T13:43:13.757 INFO:tasks.workunit.client.0.smithi107.stdout:1/566: creat d1/d46/d6a/fb5 x:0 0 0 2024-03-04T13:43:13.759 INFO:tasks.workunit.client.0.smithi107.stdout:7/554: chown d6/d1d/d36/d59/f8a 4 1 2024-03-04T13:43:13.759 INFO:tasks.workunit.client.0.smithi107.stdout:4/519: fsync d0/d9/de/f41 0 2024-03-04T13:43:13.763 INFO:tasks.workunit.client.0.smithi107.stdout:0/623: dwrite da/dc/f99 [0,4194304] 0 2024-03-04T13:43:13.764 INFO:tasks.workunit.client.0.smithi107.stdout:6/588: sync 2024-03-04T13:43:13.765 INFO:tasks.workunit.client.1.smithi169.stdout:0/959: rename d5/ff5 to d5/d9/dc/d12/d15/d19/f148 0 2024-03-04T13:43:13.768 INFO:tasks.workunit.client.0.smithi107.stdout:7/555: chown d6/d1d/l98 86644323 1 2024-03-04T13:43:13.770 INFO:tasks.workunit.client.1.smithi169.stdout:0/960: chown d5/d4c/f10d 30 1 2024-03-04T13:43:13.772 INFO:tasks.workunit.client.0.smithi107.stdout:7/556: write d6/d1d/d36/d59/f65 [1072160,14865] 0 2024-03-04T13:43:13.772 INFO:tasks.workunit.client.0.smithi107.stdout:3/558: dread d1/de/d18/d33/d4d/d4f/d69/f7b [0,4194304] 0 2024-03-04T13:43:13.776 INFO:tasks.workunit.client.0.smithi107.stdout:5/566: rename d5/d8/d2a/d43/d5f to d5/dc/dc6 0 2024-03-04T13:43:13.785 INFO:tasks.workunit.client.0.smithi107.stdout:7/557: creat d6/d13/d16/fbb x:0 0 0 2024-03-04T13:43:13.788 INFO:tasks.workunit.client.0.smithi107.stdout:1/567: rename d1/d10/d16/f8b to d1/d38/fb6 0 2024-03-04T13:43:13.793 INFO:tasks.workunit.client.0.smithi107.stdout:7/558: fsync d6/d13/d16/d1c/faf 0 2024-03-04T13:43:13.799 INFO:tasks.workunit.client.0.smithi107.stdout:7/559: stat d6/d13/d14/fa4 0 2024-03-04T13:43:13.802 INFO:tasks.workunit.client.0.smithi107.stdout:3/559: dwrite d1/de/d18/d33/d48/d71/d4c/f57 [0,4194304] 0 2024-03-04T13:43:13.804 INFO:tasks.workunit.client.0.smithi107.stdout:5/567: dwrite d5/fad [0,4194304] 0 2024-03-04T13:43:13.827 INFO:tasks.workunit.client.0.smithi107.stdout:8/580: sync 2024-03-04T13:43:13.827 INFO:tasks.workunit.client.0.smithi107.stdout:8/581: readlink d6/da/da8/lab 0 2024-03-04T13:43:13.846 INFO:tasks.workunit.client.0.smithi107.stdout:8/582: dwrite d6/da/d25/d5f/f87 [0,4194304] 0 2024-03-04T13:43:13.855 INFO:tasks.workunit.client.0.smithi107.stdout:2/525: sync 2024-03-04T13:43:13.855 INFO:tasks.workunit.client.0.smithi107.stdout:9/549: sync 2024-03-04T13:43:13.877 INFO:tasks.workunit.client.0.smithi107.stdout:9/550: dwrite d4/da/d11/d20/d23/d7e/d84/fb9 [0,4194304] 0 2024-03-04T13:43:13.882 INFO:tasks.workunit.client.0.smithi107.stdout:9/551: chown d4/da/d11/d65/f58 105241 1 2024-03-04T13:43:13.909 INFO:tasks.workunit.client.0.smithi107.stdout:1/568: creat d1/d10/d16/d47/d71/d82/fb7 x:0 0 0 2024-03-04T13:43:13.915 INFO:tasks.workunit.client.1.smithi169.stdout:0/961: write d5/d9/f2d [1751902,24543] 0 2024-03-04T13:43:13.918 INFO:tasks.workunit.client.0.smithi107.stdout:4/520: truncate d0/d9/f6b 183869 0 2024-03-04T13:43:13.922 INFO:tasks.workunit.client.0.smithi107.stdout:2/526: stat da/d38/d3a/c75 0 2024-03-04T13:43:13.927 INFO:tasks.workunit.client.0.smithi107.stdout:2/527: write da/d44/f52 [1305427,7128] 0 2024-03-04T13:43:13.927 INFO:tasks.workunit.client.0.smithi107.stdout:0/624: getdents da/dc/dd/d3e/d26/d69/d6b 0 2024-03-04T13:43:13.931 INFO:tasks.workunit.client.0.smithi107.stdout:3/560: creat d1/de/d3f/fb3 x:0 0 0 2024-03-04T13:43:13.936 INFO:tasks.workunit.client.0.smithi107.stdout:3/561: write d1/fa [4276392,119314] 0 2024-03-04T13:43:13.939 INFO:tasks.workunit.client.0.smithi107.stdout:6/589: dwrite df/d2e/f5a [0,4194304] 0 2024-03-04T13:43:13.943 INFO:tasks.workunit.client.0.smithi107.stdout:0/625: mkdir da/dc/dd/ddb 0 2024-03-04T13:43:13.944 INFO:tasks.workunit.client.0.smithi107.stdout:3/562: symlink d1/de/d3f/lb4 0 2024-03-04T13:43:13.944 INFO:tasks.workunit.client.0.smithi107.stdout:4/521: dread - d0/d9/d78/f85 zero size 2024-03-04T13:43:13.952 INFO:tasks.workunit.client.0.smithi107.stdout:6/590: read df/d17/d24/d40/f44 [2003493,44022] 0 2024-03-04T13:43:13.953 INFO:tasks.workunit.client.0.smithi107.stdout:1/569: dwrite d1/d46/d6a/d4e/f6c [0,4194304] 0 2024-03-04T13:43:13.959 INFO:tasks.workunit.client.0.smithi107.stdout:3/563: write d1/de/d18/d33/d48/d71/d4c/f91 [326531,25156] 0 2024-03-04T13:43:13.966 INFO:tasks.workunit.client.1.smithi169.stdout:0/962: dread d5/f37 [4194304,4194304] 0 2024-03-04T13:43:13.972 INFO:tasks.workunit.client.0.smithi107.stdout:1/570: unlink d1/d10/d16/d47/d71/d82/fb7 0 2024-03-04T13:43:13.972 INFO:tasks.workunit.client.0.smithi107.stdout:0/626: rename da/dc/dd/d1e/lc2 to da/dc/dd/d3e/d26/d88/ldc 0 2024-03-04T13:43:13.981 INFO:tasks.workunit.client.0.smithi107.stdout:4/522: dwrite d0/d9/de/f62 [0,4194304] 0 2024-03-04T13:43:13.983 INFO:tasks.workunit.client.0.smithi107.stdout:4/523: readlink d0/d2f/l38 0 2024-03-04T13:43:13.990 INFO:tasks.workunit.client.1.smithi169.stdout:0/963: dwrite d5/d9/dc/d12/d15/d19/d69/d110/f12d [0,4194304] 0 2024-03-04T13:43:13.992 INFO:tasks.workunit.client.0.smithi107.stdout:2/528: dread da/d34/f41 [0,4194304] 0 2024-03-04T13:43:14.012 INFO:tasks.workunit.client.0.smithi107.stdout:2/529: mknod da/df/d73/db3/cb6 0 2024-03-04T13:43:14.026 INFO:tasks.workunit.client.0.smithi107.stdout:4/524: dread d0/d9/de/d14/dc/f27 [0,4194304] 0 2024-03-04T13:43:14.029 INFO:tasks.workunit.client.0.smithi107.stdout:4/525: fdatasync d0/d2f/f39 0 2024-03-04T13:43:14.054 INFO:tasks.workunit.client.0.smithi107.stdout:8/583: creat d6/d16/d59/d3e/fb3 x:0 0 0 2024-03-04T13:43:14.055 INFO:tasks.workunit.client.0.smithi107.stdout:8/584: chown d6/da/f19 237 1 2024-03-04T13:43:14.058 INFO:tasks.workunit.client.0.smithi107.stdout:9/552: unlink d4/da/d11/d4c/d4d/c98 0 2024-03-04T13:43:14.060 INFO:tasks.workunit.client.0.smithi107.stdout:9/553: chown d4/da/d11/d20/d2f/ca3 386 1 2024-03-04T13:43:14.074 INFO:tasks.workunit.client.0.smithi107.stdout:5/568: truncate d5/fad 3622604 0 2024-03-04T13:43:14.076 INFO:tasks.workunit.client.0.smithi107.stdout:8/585: dwrite d6/d16/d1f/d63/d73/fac [0,4194304] 0 2024-03-04T13:43:14.078 INFO:tasks.workunit.client.0.smithi107.stdout:5/569: fsync d5/dc/fc4 0 2024-03-04T13:43:14.081 INFO:tasks.workunit.client.0.smithi107.stdout:5/570: write d5/d66/fa7 [907197,23341] 0 2024-03-04T13:43:14.085 INFO:tasks.workunit.client.0.smithi107.stdout:5/571: truncate d5/dc/fc4 663102 0 2024-03-04T13:43:14.093 INFO:tasks.workunit.client.0.smithi107.stdout:5/572: mknod d5/d2b/d4b/da6/cc7 0 2024-03-04T13:43:14.110 INFO:tasks.workunit.client.0.smithi107.stdout:8/586: dread d6/da/f35 [4194304,4194304] 0 2024-03-04T13:43:14.122 INFO:tasks.workunit.client.1.smithi169.stdout:0/964: truncate d5/d9/dc/d12/d15/d19/d77/d94/fab 8964 0 2024-03-04T13:43:14.134 INFO:tasks.workunit.client.0.smithi107.stdout:0/627: dwrite da/f62 [0,4194304] 0 2024-03-04T13:43:14.138 INFO:tasks.workunit.client.1.smithi169.stdout:0/965: fdatasync d5/d9/ff3 0 2024-03-04T13:43:14.148 INFO:tasks.workunit.client.0.smithi107.stdout:4/526: dwrite d0/d9/de/d14/dc/d21/f3f [0,4194304] 0 2024-03-04T13:43:14.149 INFO:tasks.workunit.client.1.smithi169.stdout:0/966: link d5/d9/dc/d12/d15/d19/d77/d94/c108 d5/d9/dc/c149 0 2024-03-04T13:43:14.155 INFO:tasks.workunit.client.1.smithi169.stdout:0/967: creat d5/d9/dc/d12/d15/f14a x:0 0 0 2024-03-04T13:43:14.157 INFO:tasks.workunit.client.0.smithi107.stdout:7/560: mkdir d6/d13/dbc 0 2024-03-04T13:43:14.158 INFO:tasks.workunit.client.0.smithi107.stdout:1/571: symlink d1/lb8 0 2024-03-04T13:43:14.161 INFO:tasks.workunit.client.0.smithi107.stdout:7/561: getdents d6/d13/dbc 0 2024-03-04T13:43:14.164 INFO:tasks.workunit.client.0.smithi107.stdout:1/572: creat d1/d46/d6a/d4e/fb9 x:0 0 0 2024-03-04T13:43:14.166 INFO:tasks.workunit.client.0.smithi107.stdout:7/562: unlink d6/d13/d16/d19/d60/f84 0 2024-03-04T13:43:14.167 INFO:tasks.workunit.client.0.smithi107.stdout:1/573: write d1/d10/d16/fad [4645623,97810] 0 2024-03-04T13:43:14.169 INFO:tasks.workunit.client.0.smithi107.stdout:5/573: dread d5/fad [0,4194304] 0 2024-03-04T13:43:14.169 INFO:tasks.workunit.client.0.smithi107.stdout:7/563: mknod d6/d1d/d64/d8e/d22/cbd 0 2024-03-04T13:43:14.176 INFO:tasks.workunit.client.0.smithi107.stdout:8/587: write d6/d16/d1f/f65 [596006,59922] 0 2024-03-04T13:43:14.183 INFO:tasks.workunit.client.0.smithi107.stdout:8/588: truncate d6/da/d25/f3f 1037823 0 2024-03-04T13:43:14.187 INFO:tasks.workunit.client.0.smithi107.stdout:1/574: dwrite d1/d46/d6a/d4e/d60/f63 [0,4194304] 0 2024-03-04T13:43:14.193 INFO:tasks.workunit.client.0.smithi107.stdout:7/564: dwrite d6/d13/d14/fa4 [0,4194304] 0 2024-03-04T13:43:14.196 INFO:tasks.workunit.client.1.smithi169.stdout:0/968: dread d5/d9/f6e [0,4194304] 0 2024-03-04T13:43:14.197 INFO:tasks.workunit.client.0.smithi107.stdout:8/589: write d6/d16/d59/d3e/f6d [771576,109040] 0 2024-03-04T13:43:14.199 INFO:tasks.workunit.client.0.smithi107.stdout:1/575: mkdir d1/d46/d6a/df/d33/d43/d59/dba 0 2024-03-04T13:43:14.200 INFO:tasks.workunit.client.0.smithi107.stdout:7/565: mknod d6/d13/d16/d19/cbe 0 2024-03-04T13:43:14.201 INFO:tasks.workunit.client.0.smithi107.stdout:8/590: rmdir d6/da 39 2024-03-04T13:43:14.202 INFO:tasks.workunit.client.0.smithi107.stdout:4/527: dwrite d0/d9/de/f43 [0,4194304] 0 2024-03-04T13:43:14.204 INFO:tasks.workunit.client.1.smithi169.stdout:0/969: write d5/d9/dc/d12/d15/d19/d61/f2c [2459539,20631] 0 2024-03-04T13:43:14.207 INFO:tasks.workunit.client.0.smithi107.stdout:5/574: dwrite d5/d8/fb [0,4194304] 0 2024-03-04T13:43:14.207 INFO:tasks.workunit.client.0.smithi107.stdout:1/576: mknod d1/d46/d6a/d4e/cbb 0 2024-03-04T13:43:14.209 INFO:tasks.workunit.client.0.smithi107.stdout:4/528: chown d0/d9/de/d14/dc/f27 13729 1 2024-03-04T13:43:14.210 INFO:tasks.workunit.client.0.smithi107.stdout:1/577: chown d1/d10/d16/d47/d71 67 1 2024-03-04T13:43:14.210 INFO:tasks.workunit.client.0.smithi107.stdout:7/566: fdatasync d6/d13/d16/d19/d38/d2c/f4a 0 2024-03-04T13:43:14.211 INFO:tasks.workunit.client.1.smithi169.stdout:0/970: symlink d5/d9/dc/l14b 0 2024-03-04T13:43:14.212 INFO:tasks.workunit.client.0.smithi107.stdout:4/529: chown d0/d9/de/d14/dc/d21 13660925 1 2024-03-04T13:43:14.213 INFO:tasks.workunit.client.0.smithi107.stdout:7/567: chown d6/d13/d16/d19/d38 70564 1 2024-03-04T13:43:14.214 INFO:tasks.workunit.client.0.smithi107.stdout:5/575: chown d5/d79/d89/da3/d69/f72 14 1 2024-03-04T13:43:14.216 INFO:tasks.workunit.client.0.smithi107.stdout:4/530: dread - d0/d9/de/d14/d66/d7b/f8f zero size 2024-03-04T13:43:14.216 INFO:tasks.workunit.client.0.smithi107.stdout:5/576: stat d5/dc/dc6/f75 0 2024-03-04T13:43:14.219 INFO:tasks.workunit.client.1.smithi169.stdout:0/971: getdents d5/d9/dc/d12/d15 0 2024-03-04T13:43:14.222 INFO:tasks.workunit.client.1.smithi169.stdout:0/972: chown d5/d9/dc/d12/d15/d19/d77/db2/f136 2857 1 2024-03-04T13:43:14.226 INFO:tasks.workunit.client.0.smithi107.stdout:5/577: dread - d5/d8/d14/d2d/d92/fc0 zero size 2024-03-04T13:43:14.226 INFO:tasks.workunit.client.0.smithi107.stdout:7/568: mknod d6/d1d/cbf 0 2024-03-04T13:43:14.232 INFO:tasks.workunit.client.0.smithi107.stdout:5/578: creat d5/d8/d2a/d62/fc8 x:0 0 0 2024-03-04T13:43:14.233 INFO:tasks.workunit.client.0.smithi107.stdout:8/591: dwrite d6/d16/d1f/d63/d73/f7b [0,4194304] 0 2024-03-04T13:43:14.233 INFO:tasks.workunit.client.0.smithi107.stdout:1/578: dread d1/d46/d6a/df/d33/d5d/d4c/f93 [0,4194304] 0 2024-03-04T13:43:14.237 INFO:tasks.workunit.client.0.smithi107.stdout:5/579: write d5/d2b/d4b/f91 [1000458,62121] 0 2024-03-04T13:43:14.241 INFO:tasks.workunit.client.0.smithi107.stdout:1/579: dread d1/d46/d6a/df/d33/d5d/d4c/f93 [0,4194304] 0 2024-03-04T13:43:14.242 INFO:tasks.workunit.client.0.smithi107.stdout:4/531: dwrite d0/d9/d18/d29/f5c [4194304,4194304] 0 2024-03-04T13:43:14.242 INFO:tasks.workunit.client.1.smithi169.stdout:0/973: dwrite d5/d9/dc/d12/d15/d19/d61/f68 [0,4194304] 0 2024-03-04T13:43:14.248 INFO:tasks.workunit.client.0.smithi107.stdout:7/569: dread d6/d13/d16/d1c/f35 [0,4194304] 0 2024-03-04T13:43:14.256 INFO:tasks.workunit.client.1.smithi169.stdout:0/974: truncate d5/d9/dc/d12/d15/d19/d61/d71/fff 1115525 0 2024-03-04T13:43:14.259 INFO:tasks.workunit.client.0.smithi107.stdout:8/592: write d6/da/f72 [2662589,4271] 0 2024-03-04T13:43:14.269 INFO:tasks.workunit.client.0.smithi107.stdout:8/593: rename d6/d16/la7 to d6/d16/d1f/d4c/lb4 0 2024-03-04T13:43:14.273 INFO:tasks.workunit.client.0.smithi107.stdout:8/594: readlink d6/da/l30 0 2024-03-04T13:43:14.273 INFO:tasks.workunit.client.0.smithi107.stdout:1/580: dwrite d1/d46/d6a/df/d67/d6b/fb3 [0,4194304] 0 2024-03-04T13:43:14.276 INFO:tasks.workunit.client.1.smithi169.stdout:0/975: dwrite d5/f103 [0,4194304] 0 2024-03-04T13:43:14.278 INFO:tasks.workunit.client.0.smithi107.stdout:8/595: rmdir d6/d16/d59/d7c 39 2024-03-04T13:43:14.279 INFO:tasks.workunit.client.1.smithi169.stdout:0/976: chown d5/d9/dc/l127 1779176 1 2024-03-04T13:43:14.280 INFO:tasks.workunit.client.0.smithi107.stdout:8/596: write d6/d16/d59/d3e/d67/f9e [138747,112353] 0 2024-03-04T13:43:14.281 INFO:tasks.workunit.client.1.smithi169.stdout:0/977: mkdir d5/d9/d3d/d14c 0 2024-03-04T13:43:14.283 INFO:tasks.workunit.client.0.smithi107.stdout:3/564: sync 2024-03-04T13:43:14.284 INFO:tasks.workunit.client.0.smithi107.stdout:6/591: sync 2024-03-04T13:43:14.285 INFO:tasks.workunit.client.0.smithi107.stdout:4/532: dread d0/d9/de/d14/d66/f6a [0,4194304] 0 2024-03-04T13:43:14.285 INFO:tasks.workunit.client.0.smithi107.stdout:8/597: chown d6/da/f2c 1979913954 1 2024-03-04T13:43:14.285 INFO:tasks.workunit.client.0.smithi107.stdout:7/570: dread d6/d1d/d36/f5b [0,4194304] 0 2024-03-04T13:43:14.288 INFO:tasks.workunit.client.0.smithi107.stdout:6/592: chown df/d38/c96 3 1 2024-03-04T13:43:14.289 INFO:tasks.workunit.client.0.smithi107.stdout:4/533: creat d0/d30/f9c x:0 0 0 2024-03-04T13:43:14.289 INFO:tasks.workunit.client.0.smithi107.stdout:5/580: dread d5/d2b/f38 [0,4194304] 0 2024-03-04T13:43:14.290 INFO:tasks.workunit.client.0.smithi107.stdout:3/565: creat d1/d41/d81/fb5 x:0 0 0 2024-03-04T13:43:14.291 INFO:tasks.workunit.client.0.smithi107.stdout:7/571: write d6/d1d/d64/d8e/f75 [4256509,78839] 0 2024-03-04T13:43:14.292 INFO:tasks.workunit.client.0.smithi107.stdout:8/598: rmdir d6/d16/d1f/d4c/da5 39 2024-03-04T13:43:14.294 INFO:tasks.workunit.client.0.smithi107.stdout:3/566: chown d1/de/d18/d7e 58 1 2024-03-04T13:43:14.295 INFO:tasks.workunit.client.0.smithi107.stdout:7/572: fdatasync d6/d13/d14/f5d 0 2024-03-04T13:43:14.299 INFO:tasks.workunit.client.0.smithi107.stdout:4/534: symlink d0/d9/d78/l9d 0 2024-03-04T13:43:14.301 INFO:tasks.workunit.client.0.smithi107.stdout:8/599: mkdir d6/da/da8/db5 0 2024-03-04T13:43:14.302 INFO:tasks.workunit.client.0.smithi107.stdout:7/573: creat d6/d1d/d64/d9a/fc0 x:0 0 0 2024-03-04T13:43:14.302 INFO:tasks.workunit.client.0.smithi107.stdout:8/600: chown d6/l99 159 1 2024-03-04T13:43:14.305 INFO:tasks.workunit.client.0.smithi107.stdout:3/567: mknod d1/de/d18/d33/cb6 0 2024-03-04T13:43:14.307 INFO:tasks.workunit.client.0.smithi107.stdout:8/601: write d6/f54 [1189051,79960] 0 2024-03-04T13:43:14.307 INFO:tasks.workunit.client.0.smithi107.stdout:4/535: fdatasync d0/d9/de/d14/dc/f3c 0 2024-03-04T13:43:14.311 INFO:tasks.workunit.client.1.smithi169.stdout:0/978: write d5/d9/dc/d12/d15/d19/f52 [1869616,22205] 0 2024-03-04T13:43:14.313 INFO:tasks.workunit.client.0.smithi107.stdout:1/581: dwrite d1/d46/d6a/d75/f66 [0,4194304] 0 2024-03-04T13:43:14.313 INFO:tasks.workunit.client.0.smithi107.stdout:7/574: write d6/d1d/d36/d59/f8a [87193,45752] 0 2024-03-04T13:43:14.314 INFO:tasks.workunit.client.0.smithi107.stdout:4/536: chown d0/d9/de/d14/d33 1905 1 2024-03-04T13:43:14.317 INFO:tasks.workunit.client.1.smithi169.stdout:0/979: creat d5/d9/dc/d12/d15/d19/d61/d71/f14d x:0 0 0 2024-03-04T13:43:14.324 INFO:tasks.workunit.client.0.smithi107.stdout:6/593: dwrite fb [4194304,4194304] 0 2024-03-04T13:43:14.325 INFO:tasks.workunit.client.0.smithi107.stdout:8/602: symlink d6/d16/d59/d3e/d67/lb6 0 2024-03-04T13:43:14.325 INFO:tasks.workunit.client.0.smithi107.stdout:9/554: sync 2024-03-04T13:43:14.325 INFO:tasks.workunit.client.0.smithi107.stdout:0/628: sync 2024-03-04T13:43:14.325 INFO:tasks.workunit.client.0.smithi107.stdout:2/530: sync 2024-03-04T13:43:14.329 INFO:tasks.workunit.client.0.smithi107.stdout:1/582: creat d1/d46/d6a/fbc x:0 0 0 2024-03-04T13:43:14.330 INFO:tasks.workunit.client.1.smithi169.stdout:0/980: rename d5/d9/dc/d12/d15/d19/f52 to d5/d9/dc/f14e 0 2024-03-04T13:43:14.331 INFO:tasks.workunit.client.0.smithi107.stdout:5/581: dwrite d5/d8/da/fa1 [0,4194304] 0 2024-03-04T13:43:14.333 INFO:tasks.workunit.client.1.smithi169.stdout:0/981: chown d5/d9/d3d/d5d/de4/fb4 87794 1 2024-03-04T13:43:14.336 INFO:tasks.workunit.client.0.smithi107.stdout:7/575: rename d6/d13/d16/d1c/f80 to d6/d1d/d64/d9a/da9/fc1 0 2024-03-04T13:43:14.339 INFO:tasks.workunit.client.0.smithi107.stdout:8/603: mkdir d6/d16/d59/d3e/d67/db7 0 2024-03-04T13:43:14.340 INFO:tasks.workunit.client.0.smithi107.stdout:6/594: mkdir df/d17/d24/d30/d8b/dc1 0 2024-03-04T13:43:14.342 INFO:tasks.workunit.client.0.smithi107.stdout:9/555: write d4/da/d11/d20/d39/f90 [762810,73368] 0 2024-03-04T13:43:14.342 INFO:tasks.workunit.client.0.smithi107.stdout:1/583: fdatasync d1/d46/d6a/df/d33/d5d/d24/f52 0 2024-03-04T13:43:14.342 INFO:tasks.workunit.client.1.smithi169.stdout:0/982: link d5/d9/dc/d12/d15/d19/f20 d5/d4c/d65/d95/f14f 0 2024-03-04T13:43:14.344 INFO:tasks.workunit.client.0.smithi107.stdout:2/531: chown da/df/d27/d58/d5f/l7b 0 1 2024-03-04T13:43:14.344 INFO:tasks.workunit.client.0.smithi107.stdout:4/537: rename d0/d9/de/d14/dc/d21/f3f to d0/d9/de/d14/dc/f9e 0 2024-03-04T13:43:14.345 INFO:tasks.workunit.client.1.smithi169.stdout:0/983: rmdir d5/d9 39 2024-03-04T13:43:14.345 INFO:tasks.workunit.client.0.smithi107.stdout:3/568: dwrite d1/de/d18/d33/d48/d71/d4c/f57 [0,4194304] 0 2024-03-04T13:43:14.348 INFO:tasks.workunit.client.1.smithi169.stdout:0/984: write d5/d9/dc/d12/d15/f14a [40169,98312] 0 2024-03-04T13:43:14.349 INFO:tasks.workunit.client.0.smithi107.stdout:2/532: chown da/d44/f81 114 1 2024-03-04T13:43:14.351 INFO:tasks.workunit.client.1.smithi169.stdout:0/985: rmdir d5/d9/d3d/d5d 39 2024-03-04T13:43:14.353 INFO:tasks.workunit.client.1.smithi169.stdout:0/986: chown d5/d9/dc/d12/d66/lfb 4 1 2024-03-04T13:43:14.360 INFO:tasks.workunit.client.0.smithi107.stdout:3/569: mkdir d1/de/d18/d33/d48/d71/d4c/d54/db7 0 2024-03-04T13:43:14.362 INFO:tasks.workunit.client.0.smithi107.stdout:1/584: mkdir d1/d46/d6a/df/d33/dbd 0 2024-03-04T13:43:14.365 INFO:tasks.workunit.client.0.smithi107.stdout:4/538: mknod d0/d2f/d7d/c9f 0 2024-03-04T13:43:14.369 INFO:tasks.workunit.client.0.smithi107.stdout:3/570: truncate d1/d2d/f51 8711235 0 2024-03-04T13:43:14.371 INFO:tasks.workunit.client.0.smithi107.stdout:5/582: getdents d5/d8/d14/d2d/d92 0 2024-03-04T13:43:14.371 INFO:tasks.workunit.client.0.smithi107.stdout:2/533: rename da/d34/la0 to da/d49/d5c/d6a/lb7 0 2024-03-04T13:43:14.374 INFO:tasks.workunit.client.0.smithi107.stdout:9/556: truncate d4/da/d11/f2b 1035766 0 2024-03-04T13:43:14.374 INFO:tasks.workunit.client.1.smithi169.stdout:0/987: dwrite d5/d9/d3d/d59/d64/d7f/fa9 [0,4194304] 0 2024-03-04T13:43:14.375 INFO:tasks.workunit.client.0.smithi107.stdout:1/585: mknod d1/d10/d16/cbe 0 2024-03-04T13:43:14.380 INFO:tasks.workunit.client.0.smithi107.stdout:7/576: dwrite d6/d13/d14/fb8 [0,4194304] 0 2024-03-04T13:43:14.380 INFO:tasks.workunit.client.0.smithi107.stdout:2/534: dread da/df/d85/f89 [0,4194304] 0 2024-03-04T13:43:14.382 INFO:tasks.workunit.client.0.smithi107.stdout:1/586: write d1/d46/d6a/fbc [113657,42294] 0 2024-03-04T13:43:14.382 INFO:tasks.workunit.client.0.smithi107.stdout:0/629: dwrite da/dc/dd/d3e/d26/d69/d6b/f94 [0,4194304] 0 2024-03-04T13:43:14.383 INFO:tasks.workunit.client.0.smithi107.stdout:8/604: rename d6/d16/d1f/d63/d73/l85 to d6/da/d25/lb8 0 2024-03-04T13:43:14.389 INFO:tasks.workunit.client.0.smithi107.stdout:4/539: mknod d0/d9/d10/d80/d81/ca0 0 2024-03-04T13:43:14.395 INFO:tasks.workunit.client.0.smithi107.stdout:7/577: mkdir d6/d13/d16/d1c/dc2 0 2024-03-04T13:43:14.395 INFO:tasks.workunit.client.0.smithi107.stdout:4/540: readlink d0/d9/d18/d29/l8d 0 2024-03-04T13:43:14.396 INFO:tasks.workunit.client.0.smithi107.stdout:2/535: unlink da/d38/l8a 0 2024-03-04T13:43:14.401 INFO:tasks.workunit.client.1.smithi169.stdout:0/988: link d5/d4c/d65/d95/fcd d5/d4c/d65/f150 0 2024-03-04T13:43:14.405 INFO:tasks.workunit.client.0.smithi107.stdout:2/536: chown f7 1 1 2024-03-04T13:43:14.407 INFO:tasks.workunit.client.0.smithi107.stdout:3/571: write d1/de/d18/d33/f34 [4127169,94151] 0 2024-03-04T13:43:14.408 INFO:tasks.workunit.client.0.smithi107.stdout:4/541: write d0/d9/de/f62 [2400872,118050] 0 2024-03-04T13:43:14.414 INFO:tasks.workunit.client.0.smithi107.stdout:8/605: mknod d6/d16/d1f/d4c/da4/daa/cb9 0 2024-03-04T13:43:14.415 INFO:tasks.workunit.client.0.smithi107.stdout:9/557: dwrite d4/da/d11/d20/d39/fa7 [0,4194304] 0 2024-03-04T13:43:14.415 INFO:tasks.workunit.client.0.smithi107.stdout:6/595: dwrite df/d17/d24/d40/d63/da3/f70 [0,4194304] 0 2024-03-04T13:43:14.418 INFO:tasks.workunit.client.0.smithi107.stdout:5/583: link d5/d8/d14/d2d/d92/fc0 d5/dc/df/fc9 0 2024-03-04T13:43:14.418 INFO:tasks.workunit.client.0.smithi107.stdout:0/630: symlink da/dc/ldd 0 2024-03-04T13:43:14.418 INFO:tasks.workunit.client.0.smithi107.stdout:7/578: mknod d6/d13/d16/d19/d38/cc3 0 2024-03-04T13:43:14.420 INFO:tasks.workunit.client.0.smithi107.stdout:1/587: creat d1/d46/d6a/df/d33/d5d/fbf x:0 0 0 2024-03-04T13:43:14.420 INFO:tasks.workunit.client.0.smithi107.stdout:2/537: read da/f20 [1921654,56466] 0 2024-03-04T13:43:14.427 INFO:tasks.workunit.client.0.smithi107.stdout:5/584: creat d5/d8/d14/d2d/d92/fca x:0 0 0 2024-03-04T13:43:14.429 INFO:tasks.workunit.client.0.smithi107.stdout:2/538: chown da/d49/l66 16 1 2024-03-04T13:43:14.429 INFO:tasks.workunit.client.0.smithi107.stdout:8/606: symlink d6/da/d25/lba 0 2024-03-04T13:43:14.432 INFO:tasks.workunit.client.0.smithi107.stdout:7/579: read d6/d13/d16/d19/d38/d37/f9e [1083173,58304] 0 2024-03-04T13:43:14.432 INFO:tasks.workunit.client.0.smithi107.stdout:0/631: mknod da/dc/dd/d3e/d26/d88/dd2/d73/cde 0 2024-03-04T13:43:14.433 INFO:tasks.workunit.client.0.smithi107.stdout:5/585: readlink d5/d8/d14/d54/l83 0 2024-03-04T13:43:14.433 INFO:tasks.workunit.client.0.smithi107.stdout:5/586: chown d5/d79 18 1 2024-03-04T13:43:14.436 INFO:tasks.workunit.client.1.smithi169.stdout:0/989: truncate d5/d9/d3d/d5d/de4/d98/dc1/dc8/fee 3933376 0 2024-03-04T13:43:14.437 INFO:tasks.workunit.client.0.smithi107.stdout:3/572: getdents d1/de/d18/d33/d48/d71/d4c/daa 0 2024-03-04T13:43:14.440 INFO:tasks.workunit.client.1.smithi169.stdout:0/990: stat d5/d9/d117 0 2024-03-04T13:43:14.441 INFO:tasks.workunit.client.0.smithi107.stdout:5/587: creat d5/dc/dc6/fcb x:0 0 0 2024-03-04T13:43:14.441 INFO:tasks.workunit.client.0.smithi107.stdout:7/580: creat d6/d13/d16/d19/d39/d7a/fc4 x:0 0 0 2024-03-04T13:43:14.442 INFO:tasks.workunit.client.0.smithi107.stdout:3/573: creat d1/de/d4b/fb8 x:0 0 0 2024-03-04T13:43:14.443 INFO:tasks.workunit.client.0.smithi107.stdout:0/632: mknod da/dc/dd/dc9/cdf 0 2024-03-04T13:43:14.444 INFO:tasks.workunit.client.0.smithi107.stdout:1/588: link d1/d10/d16/d47/c56 d1/d46/d6a/d65/db1/cc0 0 2024-03-04T13:43:14.445 INFO:tasks.workunit.client.1.smithi169.stdout:0/991: truncate d5/d9/dc/d12/d15/d19/f28 1741341 0 2024-03-04T13:43:14.447 INFO:tasks.workunit.client.1.smithi169.stdout:0/992: readlink d5/d9/dc/d12/d15/l87 0 2024-03-04T13:43:14.449 INFO:tasks.workunit.client.0.smithi107.stdout:5/588: mkdir d5/d8/d14/d2d/d92/dc1/dcc 0 2024-03-04T13:43:14.449 INFO:tasks.workunit.client.0.smithi107.stdout:2/539: write f7 [4183855,105527] 0 2024-03-04T13:43:14.459 INFO:tasks.workunit.client.0.smithi107.stdout:9/558: dwrite d4/da/d11/d20/d64/f78 [4194304,4194304] 0 2024-03-04T13:43:14.461 INFO:tasks.workunit.client.0.smithi107.stdout:5/589: read d5/dc/f55 [3711847,4771] 0 2024-03-04T13:43:14.462 INFO:tasks.workunit.client.0.smithi107.stdout:1/589: rename d1/d46/d6a/df/d33/d5d/d24/l48 to d1/d10/d16/d1b/d40/lc1 0 2024-03-04T13:43:14.464 INFO:tasks.workunit.client.0.smithi107.stdout:8/607: dwrite d6/d16/d1f/d63/d73/f79 [0,4194304] 0 2024-03-04T13:43:14.465 INFO:tasks.workunit.client.0.smithi107.stdout:7/581: link d6/d13/d16/d19/faa d6/d1d/d64/d9a/da9/d71/db9/fc5 0 2024-03-04T13:43:14.467 INFO:tasks.workunit.client.0.smithi107.stdout:5/590: creat d5/d79/fcd x:0 0 0 2024-03-04T13:43:14.469 INFO:tasks.workunit.client.0.smithi107.stdout:1/590: stat d1/d10/d16/d47 0 2024-03-04T13:43:14.469 INFO:tasks.workunit.client.0.smithi107.stdout:9/559: mknod d4/da/d11/d20/d23/d7e/cba 0 2024-03-04T13:43:14.476 INFO:tasks.workunit.client.0.smithi107.stdout:5/591: fdatasync d5/dc/d22/f7f 0 2024-03-04T13:43:14.478 INFO:tasks.workunit.client.0.smithi107.stdout:7/582: truncate d6/d1d/d36/f5b 684418 0 2024-03-04T13:43:14.478 INFO:tasks.workunit.client.0.smithi107.stdout:9/560: rmdir d4/da/d11/d20/d2f/d9c/d92 39 2024-03-04T13:43:14.481 INFO:tasks.workunit.client.0.smithi107.stdout:8/608: creat d6/d16/d59/fbb x:0 0 0 2024-03-04T13:43:14.481 INFO:tasks.workunit.client.0.smithi107.stdout:6/596: dread df/d17/d24/f2d [0,4194304] 0 2024-03-04T13:43:14.484 INFO:tasks.workunit.client.0.smithi107.stdout:8/609: readlink d6/l99 0 2024-03-04T13:43:14.486 INFO:tasks.workunit.client.0.smithi107.stdout:9/561: chown d4/l18 3723 1 2024-03-04T13:43:14.487 INFO:tasks.workunit.client.0.smithi107.stdout:3/574: dwrite d1/de/d18/d1f/d42/f5d [0,4194304] 0 2024-03-04T13:43:14.487 INFO:tasks.workunit.client.0.smithi107.stdout:7/583: read d6/d13/d14/f86 [802519,42527] 0 2024-03-04T13:43:14.490 INFO:tasks.workunit.client.0.smithi107.stdout:7/584: fsync d6/d13/d14/f6b 0 2024-03-04T13:43:14.490 INFO:tasks.workunit.client.0.smithi107.stdout:1/591: link d1/d10/d16/d47/d71/ca6 d1/d46/d6a/d4e/d60/cc2 0 2024-03-04T13:43:14.492 INFO:tasks.workunit.client.0.smithi107.stdout:5/592: creat d5/d8/fce x:0 0 0 2024-03-04T13:43:14.498 INFO:tasks.workunit.client.0.smithi107.stdout:5/593: mkdir d5/d2b/d4b/da6/dcf 0 2024-03-04T13:43:14.498 INFO:tasks.workunit.client.0.smithi107.stdout:2/540: dread da/f10 [0,4194304] 0 2024-03-04T13:43:14.504 INFO:tasks.workunit.client.0.smithi107.stdout:1/592: truncate d1/f31 716180 0 2024-03-04T13:43:14.507 INFO:tasks.workunit.client.0.smithi107.stdout:2/541: stat da/d44/d8b/d98 0 2024-03-04T13:43:14.515 INFO:tasks.workunit.client.0.smithi107.stdout:5/594: truncate d5/d8/d2a/d62/fc8 964453 0 2024-03-04T13:43:14.521 INFO:tasks.workunit.client.0.smithi107.stdout:9/562: dwrite d4/d32/f97 [0,4194304] 0 2024-03-04T13:43:14.523 INFO:tasks.workunit.client.0.smithi107.stdout:7/585: dwrite d6/d13/d16/d1c/f7d [0,4194304] 0 2024-03-04T13:43:14.524 INFO:tasks.workunit.client.0.smithi107.stdout:3/575: dwrite d1/de/d18/d33/d48/d71/d4c/f91 [0,4194304] 0 2024-03-04T13:43:14.529 INFO:tasks.workunit.client.0.smithi107.stdout:5/595: truncate d5/d8/d2a/d62/fa9 242253 0 2024-03-04T13:43:14.537 INFO:tasks.workunit.client.0.smithi107.stdout:5/596: truncate d5/d8/d14/d54/f93 742551 0 2024-03-04T13:43:14.543 INFO:tasks.workunit.client.0.smithi107.stdout:7/586: mkdir d6/d1d/d64/d8e/d3d/dc6 0 2024-03-04T13:43:14.545 INFO:tasks.workunit.client.0.smithi107.stdout:9/563: rename d4/da/d11/d20/d39/l50 to d4/da/d11/d20/d2f/d9c/daa/lbb 0 2024-03-04T13:43:14.550 INFO:tasks.workunit.client.0.smithi107.stdout:6/597: dread df/d2e/d36/d5b/f6c [0,4194304] 0 2024-03-04T13:43:14.551 INFO:tasks.workunit.client.0.smithi107.stdout:2/542: dwrite da/f2e [0,4194304] 0 2024-03-04T13:43:14.558 INFO:tasks.workunit.client.0.smithi107.stdout:6/598: chown df/d2e/d36/d71/l9a 125292489 1 2024-03-04T13:43:14.560 INFO:tasks.workunit.client.0.smithi107.stdout:7/587: getdents d6/d13/d16/d19/d39/d7a 0 2024-03-04T13:43:14.561 INFO:tasks.workunit.client.0.smithi107.stdout:9/564: getdents d4/d32 0 2024-03-04T13:43:14.568 INFO:tasks.workunit.client.0.smithi107.stdout:9/565: write d4/da/d11/d20/d23/f6c [889332,2675] 0 2024-03-04T13:43:14.569 INFO:tasks.workunit.client.0.smithi107.stdout:2/543: rmdir da/d6c/d90 0 2024-03-04T13:43:14.573 INFO:tasks.workunit.client.0.smithi107.stdout:5/597: dwrite d5/d66/f78 [0,4194304] 0 2024-03-04T13:43:14.575 INFO:tasks.workunit.client.0.smithi107.stdout:6/599: creat df/d17/d24/d83/fc2 x:0 0 0 2024-03-04T13:43:14.592 INFO:tasks.workunit.client.0.smithi107.stdout:5/598: link d5/f32 d5/fd0 0 2024-03-04T13:43:14.594 INFO:tasks.workunit.client.0.smithi107.stdout:1/593: dread d1/d46/d6a/df/d67/f69 [0,4194304] 0 2024-03-04T13:43:14.594 INFO:tasks.workunit.client.0.smithi107.stdout:9/566: link d4/da/d11/d20/d23/d7e/d84/la6 d4/da/d11/d20/d2f/d9c/daa/lbc 0 2024-03-04T13:43:14.603 INFO:tasks.workunit.client.0.smithi107.stdout:3/576: dread d1/de/d18/f59 [0,4194304] 0 2024-03-04T13:43:14.603 INFO:tasks.workunit.client.0.smithi107.stdout:2/544: dwrite da/df/d27/fa7 [0,4194304] 0 2024-03-04T13:43:14.604 INFO:tasks.workunit.client.0.smithi107.stdout:5/599: symlink d5/d79/d89/da3/d69/ld1 0 2024-03-04T13:43:14.606 INFO:tasks.workunit.client.0.smithi107.stdout:9/567: truncate d4/d34/f3f 50433 0 2024-03-04T13:43:14.608 INFO:tasks.workunit.client.0.smithi107.stdout:7/588: dwrite d6/d13/f50 [4194304,4194304] 0 2024-03-04T13:43:14.610 INFO:tasks.workunit.client.0.smithi107.stdout:6/600: dwrite df/d17/d24/f2d [0,4194304] 0 2024-03-04T13:43:14.611 INFO:tasks.workunit.client.0.smithi107.stdout:2/545: readlink da/d44/d8b/d98/la5 0 2024-03-04T13:43:14.614 INFO:tasks.workunit.client.0.smithi107.stdout:5/600: write d5/d79/d89/da3/d6a/f6d [2067548,115248] 0 2024-03-04T13:43:14.614 INFO:tasks.workunit.client.0.smithi107.stdout:7/589: read - d6/d1d/d64/d9a/da9/d71/db9/fc5 zero size 2024-03-04T13:43:14.615 INFO:tasks.workunit.client.0.smithi107.stdout:5/601: write d5/d66/f78 [371844,11691] 0 2024-03-04T13:43:14.616 INFO:tasks.workunit.client.0.smithi107.stdout:1/594: rename d1/d46/d6a/df/d33/d43/d5b/ca8 to d1/d46/d6a/df/d33/cc3 0 2024-03-04T13:43:14.620 INFO:tasks.workunit.client.0.smithi107.stdout:7/590: read d6/d1d/d64/d8e/d3d/f43 [2141487,15264] 0 2024-03-04T13:43:14.626 INFO:tasks.workunit.client.0.smithi107.stdout:6/601: symlink df/d17/lc3 0 2024-03-04T13:43:14.628 INFO:tasks.workunit.client.0.smithi107.stdout:6/602: chown df/d2e/d36/d5b 8223508 1 2024-03-04T13:43:14.629 INFO:tasks.workunit.client.0.smithi107.stdout:1/595: creat d1/d46/d6a/d4e/d60/fc4 x:0 0 0 2024-03-04T13:43:14.629 INFO:tasks.workunit.client.0.smithi107.stdout:2/546: creat da/d29/d7c/fb8 x:0 0 0 2024-03-04T13:43:14.630 INFO:tasks.workunit.client.0.smithi107.stdout:7/591: rmdir d6/d13/d16/d19/d38/d2c 39 2024-03-04T13:43:14.632 INFO:tasks.workunit.client.0.smithi107.stdout:2/547: chown da/df/d3e/f97 1749 1 2024-03-04T13:43:14.634 INFO:tasks.workunit.client.0.smithi107.stdout:1/596: write d1/d46/d6a/d4e/d60/f63 [4005054,66040] 0 2024-03-04T13:43:14.638 INFO:tasks.workunit.client.0.smithi107.stdout:2/548: dread - da/d44/f79 zero size 2024-03-04T13:43:14.641 INFO:tasks.workunit.client.0.smithi107.stdout:5/602: link d5/d2b/f88 d5/d8/fd2 0 2024-03-04T13:43:14.643 INFO:tasks.workunit.client.0.smithi107.stdout:3/577: dread d1/de/d4b/f7a [0,4194304] 0 2024-03-04T13:43:14.643 INFO:tasks.workunit.client.0.smithi107.stdout:2/549: mknod da/d29/cb9 0 2024-03-04T13:43:14.645 INFO:tasks.workunit.client.0.smithi107.stdout:5/603: rmdir d5/d66 39 2024-03-04T13:43:14.646 INFO:tasks.workunit.client.0.smithi107.stdout:3/578: chown d1/de/d11/l2e 61 1 2024-03-04T13:43:14.648 INFO:tasks.workunit.client.0.smithi107.stdout:9/568: dread d4/ff [0,4194304] 0 2024-03-04T13:43:14.648 INFO:tasks.workunit.client.0.smithi107.stdout:6/603: dread df/d17/d24/d40/d63/da3/f70 [0,4194304] 0 2024-03-04T13:43:14.651 INFO:tasks.workunit.client.0.smithi107.stdout:7/592: dwrite d6/d1d/d36/f81 [0,4194304] 0 2024-03-04T13:43:14.655 INFO:tasks.workunit.client.0.smithi107.stdout:7/593: write d6/d1d/d64/d8e/f75 [2507379,37466] 0 2024-03-04T13:43:14.655 INFO:tasks.workunit.client.0.smithi107.stdout:5/604: creat d5/d79/d89/da3/d69/fd3 x:0 0 0 2024-03-04T13:43:14.658 INFO:tasks.workunit.client.0.smithi107.stdout:6/604: creat df/d17/d24/d30/fc4 x:0 0 0 2024-03-04T13:43:14.659 INFO:tasks.workunit.client.0.smithi107.stdout:4/542: sync 2024-03-04T13:43:14.665 INFO:tasks.workunit.client.0.smithi107.stdout:6/605: write df/d2e/f5a [3964530,32638] 0 2024-03-04T13:43:14.668 INFO:tasks.workunit.client.0.smithi107.stdout:6/606: readlink df/l22 0 2024-03-04T13:43:14.670 INFO:tasks.workunit.client.0.smithi107.stdout:4/543: creat d0/d9/de/d14/d33/fa1 x:0 0 0 2024-03-04T13:43:14.673 INFO:tasks.workunit.client.0.smithi107.stdout:5/605: rename d5/d8/fd2 to d5/d8/d2a/d43/fd4 0 2024-03-04T13:43:14.674 INFO:tasks.workunit.client.0.smithi107.stdout:7/594: read d6/d1d/d64/d8e/d3d/fb1 [387459,30425] 0 2024-03-04T13:43:14.674 INFO:tasks.workunit.client.0.smithi107.stdout:9/569: truncate d4/da/d11/d4c/f3c 1861878 0 2024-03-04T13:43:14.676 INFO:tasks.workunit.client.0.smithi107.stdout:4/544: creat d0/d9/de/d14/dc/d21/fa2 x:0 0 0 2024-03-04T13:43:14.678 INFO:tasks.workunit.client.0.smithi107.stdout:2/550: dwrite da/df/f15 [4194304,4194304] 0 2024-03-04T13:43:14.679 INFO:tasks.workunit.client.0.smithi107.stdout:7/595: write d6/d1d/fb2 [124940,43941] 0 2024-03-04T13:43:14.679 INFO:tasks.workunit.client.0.smithi107.stdout:4/545: readlink d0/d2f/l38 0 2024-03-04T13:43:14.682 INFO:tasks.workunit.client.0.smithi107.stdout:3/579: dwrite d1/de/f68 [0,4194304] 0 2024-03-04T13:43:14.684 INFO:tasks.workunit.client.0.smithi107.stdout:6/607: creat df/d17/d24/d40/d63/da3/d7e/d86/fc5 x:0 0 0 2024-03-04T13:43:14.689 INFO:tasks.workunit.client.0.smithi107.stdout:5/606: write d5/d8/f45 [3923904,35373] 0 2024-03-04T13:43:14.690 INFO:tasks.workunit.client.0.smithi107.stdout:9/570: symlink d4/da/d11/d20/d23/d7e/lbd 0 2024-03-04T13:43:14.691 INFO:tasks.workunit.client.0.smithi107.stdout:5/607: fdatasync d5/d8/d14/d54/f93 0 2024-03-04T13:43:14.695 INFO:tasks.workunit.client.0.smithi107.stdout:1/597: dread d1/d46/d6a/d4e/d60/f53 [0,4194304] 0 2024-03-04T13:43:14.699 INFO:tasks.workunit.client.0.smithi107.stdout:4/546: getdents d0/d9/de/d14/d66 0 2024-03-04T13:43:14.700 INFO:tasks.workunit.client.0.smithi107.stdout:3/580: getdents d1/de/d18/d33/d4d/d4f 0 2024-03-04T13:43:14.703 INFO:tasks.workunit.client.0.smithi107.stdout:3/581: readlink d1/d1e/l99 0 2024-03-04T13:43:14.705 INFO:tasks.workunit.client.0.smithi107.stdout:9/571: rename d4/da/d11/d20/d23/d2c/f38 to d4/da/d11/d4c/fbe 0 2024-03-04T13:43:14.706 INFO:tasks.workunit.client.0.smithi107.stdout:1/598: creat d1/d46/d6a/fc5 x:0 0 0 2024-03-04T13:43:14.706 INFO:tasks.workunit.client.0.smithi107.stdout:4/547: mknod d0/d30/ca3 0 2024-03-04T13:43:14.706 INFO:tasks.workunit.client.0.smithi107.stdout:3/582: readlink d1/d1e/d26/l86 0 2024-03-04T13:43:14.708 INFO:tasks.workunit.client.0.smithi107.stdout:5/608: write d5/f32 [2641998,99802] 0 2024-03-04T13:43:14.710 INFO:tasks.workunit.client.0.smithi107.stdout:7/596: dwrite d6/d13/d16/d19/d39/f54 [0,4194304] 0 2024-03-04T13:43:14.714 INFO:tasks.workunit.client.0.smithi107.stdout:4/548: fdatasync d0/d9/de/d14/dc/f9e 0 2024-03-04T13:43:14.715 INFO:tasks.workunit.client.0.smithi107.stdout:9/572: symlink d4/da/d11/d65/lbf 0 2024-03-04T13:43:14.715 INFO:tasks.workunit.client.0.smithi107.stdout:1/599: creat d1/d46/d6a/d4e/d60/d83/fc6 x:0 0 0 2024-03-04T13:43:14.715 INFO:tasks.workunit.client.0.smithi107.stdout:5/609: fdatasync d5/dc/df/f37 0 2024-03-04T13:43:14.717 INFO:tasks.workunit.client.0.smithi107.stdout:1/600: chown d1/d10/d16/d1b/d40/l68 10165 1 2024-03-04T13:43:14.719 INFO:tasks.workunit.client.0.smithi107.stdout:3/583: rename d1/de/d18/d33/d4d/d4f/d69/f6e to d1/de/d18/d33/d48/d71/d4c/fb9 0 2024-03-04T13:43:14.721 INFO:tasks.workunit.client.0.smithi107.stdout:9/573: stat d4/c14 0 2024-03-04T13:43:14.723 INFO:tasks.workunit.client.0.smithi107.stdout:1/601: write d1/d46/d6a/d4e/fb9 [885601,93486] 0 2024-03-04T13:43:14.724 INFO:tasks.workunit.client.0.smithi107.stdout:0/633: sync 2024-03-04T13:43:14.725 INFO:tasks.workunit.client.0.smithi107.stdout:8/610: sync 2024-03-04T13:43:14.730 INFO:tasks.workunit.client.0.smithi107.stdout:3/584: mkdir d1/d1e/dba 0 2024-03-04T13:43:14.732 INFO:tasks.workunit.client.0.smithi107.stdout:4/549: getdents d0/d9/d18 0 2024-03-04T13:43:14.734 INFO:tasks.workunit.client.0.smithi107.stdout:8/611: write d6/d16/d59/d3e/f5a [548738,10592] 0 2024-03-04T13:43:14.734 INFO:tasks.workunit.client.0.smithi107.stdout:3/585: fdatasync d1/de/d18/f5f 0 2024-03-04T13:43:14.736 INFO:tasks.workunit.client.0.smithi107.stdout:1/602: chown d1/d46/d6a/df/d33/d43/d5b/f5f 37847 1 2024-03-04T13:43:14.736 INFO:tasks.workunit.client.0.smithi107.stdout:9/574: creat d4/da/d11/d4c/d4d/d72/d7b/fc0 x:0 0 0 2024-03-04T13:43:14.736 INFO:tasks.workunit.client.0.smithi107.stdout:0/634: chown da/dc/dd/d3e/d26/d88/dd2/d73/d78 628771793 1 2024-03-04T13:43:14.738 INFO:tasks.workunit.client.0.smithi107.stdout:5/610: link d5/dc/dc6/la2 d5/d8/d2a/ld5 0 2024-03-04T13:43:14.738 INFO:tasks.workunit.client.0.smithi107.stdout:0/635: truncate f4 5182211 0 2024-03-04T13:43:14.740 INFO:tasks.workunit.client.0.smithi107.stdout:8/612: chown d6/da/da8/db5 100390 1 2024-03-04T13:43:14.741 INFO:tasks.workunit.client.0.smithi107.stdout:2/551: dread da/df/fb0 [0,4194304] 0 2024-03-04T13:43:14.742 INFO:tasks.workunit.client.0.smithi107.stdout:4/550: truncate d0/d30/f9c 20885 0 2024-03-04T13:43:14.743 INFO:tasks.workunit.client.0.smithi107.stdout:1/603: write d1/d10/d16/fad [5539437,344] 0 2024-03-04T13:43:14.747 INFO:tasks.workunit.client.0.smithi107.stdout:9/575: unlink d4/c22 0 2024-03-04T13:43:14.747 INFO:tasks.workunit.client.0.smithi107.stdout:0/636: fdatasync da/dc/dd/d3e/f77 0 2024-03-04T13:43:14.748 INFO:tasks.workunit.client.0.smithi107.stdout:5/611: fdatasync d5/dc/df/f37 0 2024-03-04T13:43:14.752 INFO:tasks.workunit.client.0.smithi107.stdout:6/608: dread df/d57/f7c [0,4194304] 0 2024-03-04T13:43:14.752 INFO:tasks.workunit.client.0.smithi107.stdout:8/613: write f5 [1396132,39069] 0 2024-03-04T13:43:14.753 INFO:tasks.workunit.client.0.smithi107.stdout:1/604: mknod d1/d46/d6a/d65/cc7 0 2024-03-04T13:43:14.754 INFO:tasks.workunit.client.0.smithi107.stdout:9/576: read d4/da/d11/d20/d2f/f3d [80819,87448] 0 2024-03-04T13:43:14.758 INFO:tasks.workunit.client.0.smithi107.stdout:0/637: mknod da/dc/dd/dc9/ce0 0 2024-03-04T13:43:14.759 INFO:tasks.workunit.client.0.smithi107.stdout:7/597: dwrite d6/d13/d16/d19/faa [0,4194304] 0 2024-03-04T13:43:14.761 INFO:tasks.workunit.client.0.smithi107.stdout:8/614: mknod d6/d16/d59/d7c/cbc 0 2024-03-04T13:43:14.765 INFO:tasks.workunit.client.0.smithi107.stdout:2/552: creat da/df/d73/d7a/d9a/fba x:0 0 0 2024-03-04T13:43:14.765 INFO:tasks.workunit.client.0.smithi107.stdout:8/615: fsync d6/d16/f27 0 2024-03-04T13:43:14.769 INFO:tasks.workunit.client.0.smithi107.stdout:6/609: mknod df/d17/d24/d40/cc6 0 2024-03-04T13:43:14.772 INFO:tasks.workunit.client.0.smithi107.stdout:4/551: rename d0/d9/de/d14/dc/f26 to d0/d9/de/d14/fa4 0 2024-03-04T13:43:14.773 INFO:tasks.workunit.client.0.smithi107.stdout:9/577: creat d4/da/d11/d20/d23/fc1 x:0 0 0 2024-03-04T13:43:14.774 INFO:tasks.workunit.client.0.smithi107.stdout:2/553: fdatasync da/d29/f4e 0 2024-03-04T13:43:14.775 INFO:tasks.workunit.client.0.smithi107.stdout:4/552: read - d0/d9/d10/f8c zero size 2024-03-04T13:43:14.781 INFO:tasks.workunit.client.0.smithi107.stdout:2/554: readlink da/d38/d3a/l45 0 2024-03-04T13:43:14.781 INFO:tasks.workunit.client.0.smithi107.stdout:1/605: rmdir d1/d46/d6a/df/d33/d5d/da3 0 2024-03-04T13:43:14.782 INFO:tasks.workunit.client.0.smithi107.stdout:3/586: dread d1/de/d18/d33/d48/d71/d4c/d54/f79 [0,4194304] 0 2024-03-04T13:43:14.782 INFO:tasks.workunit.client.0.smithi107.stdout:8/616: mkdir d6/d16/d1f/dbd 0 2024-03-04T13:43:14.783 INFO:tasks.workunit.client.0.smithi107.stdout:2/555: write da/d49/d5c/d6a/f7f [760859,82299] 0 2024-03-04T13:43:14.786 INFO:tasks.workunit.client.0.smithi107.stdout:7/598: dread d6/d1d/d64/d9a/da9/fc1 [4194304,4194304] 0 2024-03-04T13:43:14.791 INFO:tasks.workunit.client.0.smithi107.stdout:8/617: fdatasync d6/da/d25/d2a/f75 0 2024-03-04T13:43:14.792 INFO:tasks.workunit.client.0.smithi107.stdout:7/599: read - d6/d13/d16/d19/fa1 zero size 2024-03-04T13:43:14.792 INFO:tasks.workunit.client.0.smithi107.stdout:1/606: rmdir d1/d46/d6a/df/d67/d6b 39 2024-03-04T13:43:14.792 INFO:tasks.workunit.client.0.smithi107.stdout:9/578: rename d4/da/d11/f2b to d4/da/d11/d20/d2f/d9c/daa/fc2 0 2024-03-04T13:43:14.794 INFO:tasks.workunit.client.0.smithi107.stdout:2/556: symlink da/d39/lbb 0 2024-03-04T13:43:14.796 INFO:tasks.workunit.client.0.smithi107.stdout:9/579: chown d4/da/d11/d4c/d4d 15880266 1 2024-03-04T13:43:14.796 INFO:tasks.workunit.client.0.smithi107.stdout:4/553: creat d0/d9/de/d14/d66/fa5 x:0 0 0 2024-03-04T13:43:14.799 INFO:tasks.workunit.client.0.smithi107.stdout:4/554: stat d0/d9/d78/l9d 0 2024-03-04T13:43:14.800 INFO:tasks.workunit.client.0.smithi107.stdout:7/600: read d6/d1d/d36/d59/f8a [220933,36228] 0 2024-03-04T13:43:14.801 INFO:tasks.workunit.client.0.smithi107.stdout:5/612: dwrite d5/d8/d2a/d43/fd4 [0,4194304] 0 2024-03-04T13:43:14.802 INFO:tasks.workunit.client.0.smithi107.stdout:2/557: mknod da/d49/cbc 0 2024-03-04T13:43:14.805 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:14 smithi107 ceph-mon[28739]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' 2024-03-04T13:43:14.805 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:14 smithi107 ceph-mon[28739]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' 2024-03-04T13:43:14.805 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:14 smithi107 ceph-mon[28739]: pgmap v59: 65 pgs: 65 active+clean; 3.0 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 93 MiB/s rd, 278 MiB/s wr, 570 op/s 2024-03-04T13:43:14.808 INFO:tasks.workunit.client.0.smithi107.stdout:6/610: truncate df/d57/f7c 655224 0 2024-03-04T13:43:14.809 INFO:tasks.workunit.client.0.smithi107.stdout:8/618: write d6/f9f [569957,24634] 0 2024-03-04T13:43:14.809 INFO:tasks.workunit.client.0.smithi107.stdout:9/580: stat d4/da/d11/d20/d23/d7e/d84/fb9 0 2024-03-04T13:43:14.812 INFO:tasks.workunit.client.0.smithi107.stdout:6/611: chown df/d38/c96 3635852 1 2024-03-04T13:43:14.813 INFO:tasks.workunit.client.0.smithi107.stdout:2/558: rmdir da/df/d73/d7a 39 2024-03-04T13:43:14.814 INFO:tasks.workunit.client.0.smithi107.stdout:8/619: write d6/fe [4321327,73944] 0 2024-03-04T13:43:14.815 INFO:tasks.workunit.client.0.smithi107.stdout:3/587: write d1/de/d18/f59 [1743110,49797] 0 2024-03-04T13:43:14.819 INFO:tasks.workunit.client.0.smithi107.stdout:0/638: dread da/dc/dd/d3e/d26/f64 [0,4194304] 0 2024-03-04T13:43:14.819 INFO:tasks.workunit.client.0.smithi107.stdout:9/581: unlink d4/da/f4b 0 2024-03-04T13:43:14.821 INFO:tasks.workunit.client.0.smithi107.stdout:7/601: creat d6/d13/d16/d1c/dc2/fc7 x:0 0 0 2024-03-04T13:43:14.823 INFO:tasks.workunit.client.0.smithi107.stdout:5/613: mkdir d5/d8/d2a/dd6 0 2024-03-04T13:43:14.824 INFO:tasks.workunit.client.0.smithi107.stdout:1/607: truncate d1/d46/d6a/df/d67/d6b/fb4 625037 0 2024-03-04T13:43:14.825 INFO:tasks.workunit.client.0.smithi107.stdout:6/612: creat df/d38/d4f/daf/fc7 x:0 0 0 2024-03-04T13:43:14.826 INFO:tasks.workunit.client.0.smithi107.stdout:2/559: mkdir da/d29/d7c/dbd 0 2024-03-04T13:43:14.830 INFO:tasks.workunit.client.0.smithi107.stdout:0/639: symlink da/d7e/le1 0 2024-03-04T13:43:14.830 INFO:tasks.workunit.client.0.smithi107.stdout:5/614: mknod d5/dc/d22/cd7 0 2024-03-04T13:43:14.832 INFO:tasks.workunit.client.0.smithi107.stdout:9/582: creat d4/da/d11/d20/d2f/d9c/daa/fc3 x:0 0 0 2024-03-04T13:43:14.835 INFO:tasks.workunit.client.1.smithi169.stdout:0/993: sync 2024-03-04T13:43:14.837 INFO:tasks.workunit.client.1.smithi169.stdout:0/994: chown d5/d4c/d65/d95/c142 0 1 2024-03-04T13:43:14.837 INFO:tasks.workunit.client.0.smithi107.stdout:2/560: creat da/df/fbe x:0 0 0 2024-03-04T13:43:14.837 INFO:tasks.workunit.client.0.smithi107.stdout:8/620: rename d6/da/d25/lba to d6/da/lbe 0 2024-03-04T13:43:14.838 INFO:tasks.workunit.client.0.smithi107.stdout:5/615: fsync d5/dc/d22/f26 0 2024-03-04T13:43:14.839 INFO:tasks.workunit.client.0.smithi107.stdout:9/583: chown d4/d32/fa8 197823 1 2024-03-04T13:43:14.840 INFO:tasks.workunit.client.0.smithi107.stdout:7/602: creat d6/d1d/fc8 x:0 0 0 2024-03-04T13:43:14.840 INFO:tasks.workunit.client.1.smithi169.stdout:0/995: getdents d5/d9/dc/d12/da5/d115 0 2024-03-04T13:43:14.842 INFO:tasks.workunit.client.0.smithi107.stdout:9/584: truncate d4/ff 4349902 0 2024-03-04T13:43:14.845 INFO:tasks.workunit.client.0.smithi107.stdout:3/588: dread d1/de/d18/f59 [0,4194304] 0 2024-03-04T13:43:14.852 INFO:tasks.workunit.client.0.smithi107.stdout:7/603: symlink d6/d1d/d64/d9a/da9/d71/lc9 0 2024-03-04T13:43:14.853 INFO:tasks.workunit.client.0.smithi107.stdout:8/621: symlink d6/d16/d1f/d63/d73/lbf 0 2024-03-04T13:43:14.853 INFO:tasks.workunit.client.0.smithi107.stdout:5/616: write d5/d8/da/f16 [3738690,5572] 0 2024-03-04T13:43:14.854 INFO:tasks.workunit.client.0.smithi107.stdout:4/555: dread d0/d9/de/d14/dc/d21/f5d [0,4194304] 0 2024-03-04T13:43:14.856 INFO:tasks.workunit.client.0.smithi107.stdout:1/608: dread d1/d46/d6a/df/d67/d6b/fb3 [0,4194304] 0 2024-03-04T13:43:14.857 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:14 smithi169 ceph-mon[36313]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' 2024-03-04T13:43:14.857 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:14 smithi169 ceph-mon[36313]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' 2024-03-04T13:43:14.857 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:14 smithi169 ceph-mon[36313]: pgmap v59: 65 pgs: 65 active+clean; 3.0 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 93 MiB/s rd, 278 MiB/s wr, 570 op/s 2024-03-04T13:43:14.860 INFO:tasks.workunit.client.0.smithi107.stdout:9/585: write d4/d34/f44 [311718,11841] 0 2024-03-04T13:43:14.860 INFO:tasks.workunit.client.0.smithi107.stdout:4/556: rmdir d0 39 2024-03-04T13:43:14.860 INFO:tasks.workunit.client.0.smithi107.stdout:7/604: fdatasync d6/d13/d16/f7b 0 2024-03-04T13:43:14.863 INFO:tasks.workunit.client.0.smithi107.stdout:3/589: getdents d1/de/d18/d33/d48/d71/d4c/daa/dab 0 2024-03-04T13:43:14.863 INFO:tasks.workunit.client.0.smithi107.stdout:1/609: write d1/d46/d6a/fbc [587576,120720] 0 2024-03-04T13:43:14.863 INFO:tasks.workunit.client.0.smithi107.stdout:8/622: rename d6/da/d25/l7a to d6/d16/d69/lc0 0 2024-03-04T13:43:14.866 INFO:tasks.workunit.client.0.smithi107.stdout:5/617: mkdir d5/d8/d14/d2d/d92/dd8 0 2024-03-04T13:43:14.866 INFO:tasks.workunit.client.0.smithi107.stdout:9/586: chown l1 3 1 2024-03-04T13:43:14.867 INFO:tasks.workunit.client.0.smithi107.stdout:6/613: dwrite df/d2e/d36/d77/f90 [0,4194304] 0 2024-03-04T13:43:14.867 INFO:tasks.workunit.client.0.smithi107.stdout:7/605: chown d6/d13/d16/d19/d38/d62 3688790 1 2024-03-04T13:43:14.873 INFO:tasks.workunit.client.0.smithi107.stdout:0/640: dread da/dc/dd/d3e/f77 [0,4194304] 0 2024-03-04T13:43:14.874 INFO:tasks.workunit.client.0.smithi107.stdout:5/618: read d5/d8/d2a/d43/fbb [139250,9303] 0 2024-03-04T13:43:14.876 INFO:tasks.workunit.client.0.smithi107.stdout:8/623: stat d6/da/d25/d5f/f98 0 2024-03-04T13:43:14.877 INFO:tasks.workunit.client.0.smithi107.stdout:9/587: creat d4/da/d11/d20/d64/d77/fc4 x:0 0 0 2024-03-04T13:43:14.879 INFO:tasks.workunit.client.0.smithi107.stdout:5/619: fsync d5/d66/f78 0 2024-03-04T13:43:14.879 INFO:tasks.workunit.client.0.smithi107.stdout:0/641: fsync da/f81 0 2024-03-04T13:43:14.884 INFO:tasks.workunit.client.0.smithi107.stdout:1/610: link d1/d46/d6a/df/d33/d5d/l58 d1/d46/d6a/d65/db1/lc8 0 2024-03-04T13:43:14.885 INFO:tasks.workunit.client.0.smithi107.stdout:7/606: dread d6/d1d/d64/d8e/f5c [0,4194304] 0 2024-03-04T13:43:14.886 INFO:tasks.workunit.client.0.smithi107.stdout:0/642: creat da/dc/dd/d1e/fe2 x:0 0 0 2024-03-04T13:43:14.889 INFO:tasks.workunit.client.0.smithi107.stdout:3/590: write d1/f6 [1317538,26695] 0 2024-03-04T13:43:14.891 INFO:tasks.workunit.client.0.smithi107.stdout:4/557: write d0/d9/d18/d29/f55 [370144,6847] 0 2024-03-04T13:43:14.895 INFO:tasks.workunit.client.0.smithi107.stdout:3/591: read d1/de/d18/d1f/d42/f5d [1305886,96319] 0 2024-03-04T13:43:14.895 INFO:tasks.workunit.client.0.smithi107.stdout:4/558: dread - d0/d9/de/d14/dc/f84 zero size 2024-03-04T13:43:14.897 INFO:tasks.workunit.client.0.smithi107.stdout:0/643: unlink da/dc/dd/d3e/l5d 0 2024-03-04T13:43:14.899 INFO:tasks.workunit.client.0.smithi107.stdout:6/614: write df/d2e/d36/f49 [2477347,80050] 0 2024-03-04T13:43:14.899 INFO:tasks.workunit.client.0.smithi107.stdout:9/588: link d4/da/d11/d20/d23/d2c/caf d4/da/d11/d4c/d4d/d72/d7b/cc5 0 2024-03-04T13:43:14.900 INFO:tasks.workunit.client.0.smithi107.stdout:5/620: getdents d5/dc/dc6 0 2024-03-04T13:43:14.900 INFO:tasks.workunit.client.0.smithi107.stdout:8/624: dwrite d6/da/f35 [4194304,4194304] 0 2024-03-04T13:43:14.901 INFO:tasks.workunit.client.0.smithi107.stdout:4/559: dread d0/d2f/f39 [0,4194304] 0 2024-03-04T13:43:14.905 INFO:tasks.workunit.client.0.smithi107.stdout:8/625: truncate d6/da/fad 1007123 0 2024-03-04T13:43:14.905 INFO:tasks.workunit.client.0.smithi107.stdout:1/611: rename d1/d46/d6a/df to d1/d46/d6a/d4e/daa/dc9 0 2024-03-04T13:43:14.907 INFO:tasks.workunit.client.0.smithi107.stdout:5/621: rmdir d5/dc 39 2024-03-04T13:43:14.908 INFO:tasks.workunit.client.0.smithi107.stdout:4/560: unlink d0/d2f/f39 0 2024-03-04T13:43:14.908 INFO:tasks.workunit.client.0.smithi107.stdout:6/615: dread df/d2e/d36/d77/f84 [0,4194304] 0 2024-03-04T13:43:14.910 INFO:tasks.workunit.client.0.smithi107.stdout:0/644: write da/dc/dd/d1e/f9c [3577844,122183] 0 2024-03-04T13:43:14.911 INFO:tasks.workunit.client.0.smithi107.stdout:3/592: mkdir d1/de/dad/dbb 0 2024-03-04T13:43:14.915 INFO:tasks.workunit.client.0.smithi107.stdout:7/607: rename d6/d1d/d64/d8e to d6/d1d/d64/d9a/da9/d71/db9/dca 0 2024-03-04T13:43:14.915 INFO:tasks.workunit.client.0.smithi107.stdout:8/626: write d6/d16/d1f/d63/d73/fa2 [2322214,5831] 0 2024-03-04T13:43:14.917 INFO:tasks.workunit.client.0.smithi107.stdout:1/612: mknod d1/d46/d6a/d4e/cca 0 2024-03-04T13:43:14.918 INFO:tasks.workunit.client.0.smithi107.stdout:9/589: mknod d4/da/d11/d20/d2f/d9c/cc6 0 2024-03-04T13:43:14.921 INFO:tasks.workunit.client.0.smithi107.stdout:4/561: fsync d0/d9/de/d14/dc/d21/fa2 0 2024-03-04T13:43:14.921 INFO:tasks.workunit.client.0.smithi107.stdout:7/608: stat d6/d13/d14/f86 0 2024-03-04T13:43:14.925 INFO:tasks.workunit.client.0.smithi107.stdout:8/627: dread - d6/d16/d59/d7c/f92 zero size 2024-03-04T13:43:14.925 INFO:tasks.workunit.client.0.smithi107.stdout:5/622: write d5/dc/d22/f26 [1157459,43148] 0 2024-03-04T13:43:14.925 INFO:tasks.workunit.client.0.smithi107.stdout:6/616: link df/d57/c6f df/cc8 0 2024-03-04T13:43:14.929 INFO:tasks.workunit.client.0.smithi107.stdout:3/593: rename d1/de/d18/d33/d48/d71/d4c/c87 to d1/d1e/d26/d9e/cbc 0 2024-03-04T13:43:14.929 INFO:tasks.workunit.client.0.smithi107.stdout:0/645: creat da/dc/dd/d3e/d26/fe3 x:0 0 0 2024-03-04T13:43:14.930 INFO:tasks.workunit.client.0.smithi107.stdout:8/628: truncate d6/da/fad 1322610 0 2024-03-04T13:43:14.930 INFO:tasks.workunit.client.0.smithi107.stdout:9/590: mknod d4/da/d11/d4c/d51/cc7 0 2024-03-04T13:43:14.931 INFO:tasks.workunit.client.0.smithi107.stdout:6/617: symlink df/d2e/d36/d5b/lc9 0 2024-03-04T13:43:14.932 INFO:tasks.workunit.client.0.smithi107.stdout:4/562: symlink d0/d9/d18/d29/d92/la6 0 2024-03-04T13:43:14.933 INFO:tasks.workunit.client.0.smithi107.stdout:5/623: symlink d5/d79/d89/da3/ld9 0 2024-03-04T13:43:14.934 INFO:tasks.workunit.client.0.smithi107.stdout:7/609: mkdir d6/d1d/d64/d9a/da9/d71/db9/dca/d3d/dc6/dcb 0 2024-03-04T13:43:14.938 INFO:tasks.workunit.client.0.smithi107.stdout:5/624: dread - d5/d8/da/fbc zero size 2024-03-04T13:43:14.939 INFO:tasks.workunit.client.0.smithi107.stdout:3/594: write d1/de/d18/d33/d48/d71/f72 [631932,57517] 0 2024-03-04T13:43:14.940 INFO:tasks.workunit.client.0.smithi107.stdout:9/591: chown d4/da/d11/d4c/l73 8 1 2024-03-04T13:43:14.943 INFO:tasks.workunit.client.0.smithi107.stdout:1/613: dwrite d1/d46/d6a/fbc [0,4194304] 0 2024-03-04T13:43:14.948 INFO:tasks.workunit.client.0.smithi107.stdout:5/625: creat d5/d8/d14/d2d/fda x:0 0 0 2024-03-04T13:43:14.951 INFO:tasks.workunit.client.0.smithi107.stdout:7/610: chown d6/d13/d16/d19/d38/d2c/l6c 9039624 1 2024-03-04T13:43:14.955 INFO:tasks.workunit.client.0.smithi107.stdout:3/595: truncate d1/de/d11/f3e 3000067 0 2024-03-04T13:43:14.955 INFO:tasks.workunit.client.0.smithi107.stdout:9/592: fsync d4/da/d11/d4c/d51/f5c 0 2024-03-04T13:43:14.957 INFO:tasks.workunit.client.0.smithi107.stdout:8/629: truncate d6/fd 338489 0 2024-03-04T13:43:14.959 INFO:tasks.workunit.client.0.smithi107.stdout:5/626: creat d5/d8/d2a/d43/fdb x:0 0 0 2024-03-04T13:43:14.960 INFO:tasks.workunit.client.0.smithi107.stdout:6/618: dwrite f6 [8388608,4194304] 0 2024-03-04T13:43:14.961 INFO:tasks.workunit.client.0.smithi107.stdout:1/614: creat d1/d46/d6a/d4e/daa/dc9/d33/d89/fcb x:0 0 0 2024-03-04T13:43:14.966 INFO:tasks.workunit.client.0.smithi107.stdout:7/611: link d6/d1d/d64/d9a/da9/d71/db9/dca/f75 d6/d1d/d64/d9a/da9/d71/fcc 0 2024-03-04T13:43:14.967 INFO:tasks.workunit.client.0.smithi107.stdout:8/630: creat d6/d16/d59/fc1 x:0 0 0 2024-03-04T13:43:14.969 INFO:tasks.workunit.client.0.smithi107.stdout:5/627: link d5/f73 d5/d8/d2a/d43/fdc 0 2024-03-04T13:43:14.970 INFO:tasks.workunit.client.0.smithi107.stdout:9/593: rename d4/da/d11/d20/d39/l7d to d4/da/d11/d20/d23/d7e/d84/lc8 0 2024-03-04T13:43:14.979 INFO:tasks.workunit.client.0.smithi107.stdout:9/594: dread - d4/d32/fa8 zero size 2024-03-04T13:43:14.981 INFO:tasks.workunit.client.0.smithi107.stdout:0/646: dwrite da/dc/dd/d3e/d26/d88/dd2/d73/d78/d86/f90 [0,4194304] 0 2024-03-04T13:43:14.986 INFO:tasks.workunit.client.0.smithi107.stdout:0/647: write f4 [2412664,38051] 0 2024-03-04T13:43:14.996 INFO:tasks.workunit.client.0.smithi107.stdout:6/619: truncate df/d17/d24/d30/d8b/f8f 232167 0 2024-03-04T13:43:14.997 INFO:tasks.workunit.client.0.smithi107.stdout:1/615: symlink d1/d46/d6a/d4e/daa/dc9/d33/d43/d59/lcc 0 2024-03-04T13:43:14.998 INFO:tasks.workunit.client.0.smithi107.stdout:9/595: dwrite d4/da/d11/d20/d2f/d9c/fb4 [0,4194304] 0 2024-03-04T13:43:15.007 INFO:tasks.workunit.client.0.smithi107.stdout:6/620: read fe [607612,59825] 0 2024-03-04T13:43:15.007 INFO:tasks.workunit.client.0.smithi107.stdout:9/596: symlink d4/da/d11/d4c/d4d/lc9 0 2024-03-04T13:43:15.013 INFO:tasks.workunit.client.0.smithi107.stdout:1/616: creat d1/d46/d6a/d4e/daa/dc9/d67/fcd x:0 0 0 2024-03-04T13:43:15.013 INFO:tasks.workunit.client.0.smithi107.stdout:9/597: creat d4/da/d11/d20/d23/fca x:0 0 0 2024-03-04T13:43:15.016 INFO:tasks.workunit.client.0.smithi107.stdout:6/621: creat df/d2e/d36/d71/fca x:0 0 0 2024-03-04T13:43:15.018 INFO:tasks.workunit.client.0.smithi107.stdout:1/617: chown d1/d46/d6a/d4e/daa/dc9/d33/d43/f55 15740626 1 2024-03-04T13:43:15.020 INFO:tasks.workunit.client.0.smithi107.stdout:9/598: unlink d4/da/d11/d4c/d4d/d72/d7b/f83 0 2024-03-04T13:43:15.020 INFO:tasks.workunit.client.0.smithi107.stdout:1/618: fdatasync d1/d46/d6a/d17/f34 0 2024-03-04T13:43:15.020 INFO:tasks.workunit.client.0.smithi107.stdout:8/631: dread d6/d16/d1f/d63/d73/fa2 [0,4194304] 0 2024-03-04T13:43:15.021 INFO:tasks.workunit.client.0.smithi107.stdout:6/622: fsync df/d17/f52 0 2024-03-04T13:43:15.025 INFO:tasks.workunit.client.0.smithi107.stdout:8/632: write d6/d16/d1f/d63/d73/f79 [3658790,67805] 0 2024-03-04T13:43:15.029 INFO:tasks.workunit.client.0.smithi107.stdout:9/599: mknod d4/da/d11/d20/d2f/d9c/d92/ccb 0 2024-03-04T13:43:15.030 INFO:tasks.workunit.client.0.smithi107.stdout:5/628: write d5/d8/d2a/d43/fdc [1877798,125656] 0 2024-03-04T13:43:15.031 INFO:tasks.workunit.client.0.smithi107.stdout:1/619: creat d1/d46/d6a/d4e/daa/dc9/d33/d43/d5b/d61/fce x:0 0 0 2024-03-04T13:43:15.039 INFO:tasks.workunit.client.0.smithi107.stdout:5/629: symlink d5/d79/d89/ldd 0 2024-03-04T13:43:15.041 INFO:tasks.workunit.client.0.smithi107.stdout:8/633: write d6/da/f50 [426583,22845] 0 2024-03-04T13:43:15.046 INFO:tasks.workunit.client.0.smithi107.stdout:7/612: dwrite d6/d1d/d64/d9a/da9/d71/fcc [0,4194304] 0 2024-03-04T13:43:15.046 INFO:tasks.workunit.client.0.smithi107.stdout:1/620: write d1/d38/fb6 [3118923,17205] 0 2024-03-04T13:43:15.049 INFO:tasks.workunit.client.0.smithi107.stdout:7/613: stat d6/d1d/d64/d9a/fc0 0 2024-03-04T13:43:15.052 INFO:tasks.workunit.client.0.smithi107.stdout:0/648: dwrite da/dc/dd/d1e/fa1 [0,4194304] 0 2024-03-04T13:43:15.053 INFO:tasks.workunit.client.0.smithi107.stdout:1/621: readlink d1/d46/d6a/d4e/daa/dc9/d67/l8e 0 2024-03-04T13:43:15.054 INFO:tasks.workunit.client.0.smithi107.stdout:8/634: creat d6/d16/d1f/d4c/da4/fc2 x:0 0 0 2024-03-04T13:43:15.054 INFO:tasks.workunit.client.0.smithi107.stdout:2/561: sync 2024-03-04T13:43:15.055 INFO:tasks.workunit.client.0.smithi107.stdout:5/630: unlink d5/d8/d14/d54/l83 0 2024-03-04T13:43:15.061 INFO:tasks.workunit.client.0.smithi107.stdout:6/623: dwrite df/d17/d5c/f67 [4194304,4194304] 0 2024-03-04T13:43:15.063 INFO:tasks.workunit.client.0.smithi107.stdout:7/614: creat d6/d13/dbc/fcd x:0 0 0 2024-03-04T13:43:15.065 INFO:tasks.workunit.client.0.smithi107.stdout:8/635: chown d6/d16/d59/d7c/f8a 607743 1 2024-03-04T13:43:15.077 INFO:tasks.workunit.client.0.smithi107.stdout:9/600: dwrite d4/d34/f87 [0,4194304] 0 2024-03-04T13:43:15.078 INFO:tasks.workunit.client.0.smithi107.stdout:1/622: creat d1/d46/d6a/d4e/fcf x:0 0 0 2024-03-04T13:43:15.078 INFO:tasks.workunit.client.0.smithi107.stdout:6/624: rmdir df/d17/d24/d40/d63 39 2024-03-04T13:43:15.082 INFO:tasks.workunit.client.0.smithi107.stdout:2/562: mknod da/d44/cbf 0 2024-03-04T13:43:15.083 INFO:tasks.workunit.client.0.smithi107.stdout:0/649: fsync da/dc/dd/d3e/d26/f67 0 2024-03-04T13:43:15.085 INFO:tasks.workunit.client.0.smithi107.stdout:8/636: write d6/da/d25/d5f/f98 [534171,50627] 0 2024-03-04T13:43:15.089 INFO:tasks.workunit.client.0.smithi107.stdout:6/625: stat df/d57/l91 0 2024-03-04T13:43:15.091 INFO:tasks.workunit.client.0.smithi107.stdout:1/623: creat d1/d46/fd0 x:0 0 0 2024-03-04T13:43:15.092 INFO:tasks.workunit.client.1.smithi169.stdout:0/996: sync 2024-03-04T13:43:15.093 INFO:tasks.workunit.client.0.smithi107.stdout:0/650: readlink da/dc/dd/d3e/d26/d88/dd2/d73/d78/la0 0 2024-03-04T13:43:15.093 INFO:tasks.workunit.client.0.smithi107.stdout:2/563: dread da/d29/f4e [0,4194304] 0 2024-03-04T13:43:15.095 INFO:tasks.workunit.client.1.smithi169.stdout:0/997: chown d5/d9/d3d/d59/d64/d7f/fcc 18 1 2024-03-04T13:43:15.097 INFO:tasks.workunit.client.1.smithi169.stdout:0/998: symlink d5/d4c/l151 0 2024-03-04T13:43:15.098 INFO:tasks.workunit.client.0.smithi107.stdout:8/637: unlink d6/d16/d69/f8f 0 2024-03-04T13:43:15.102 INFO:tasks.workunit.client.0.smithi107.stdout:8/638: chown d6/d16/d59/d3e 541208 1 2024-03-04T13:43:15.102 INFO:tasks.workunit.client.0.smithi107.stdout:1/624: symlink d1/d46/d6a/d4e/daa/ld1 0 2024-03-04T13:43:15.103 INFO:tasks.workunit.client.1.smithi169.stdout:0/999: read d5/d9/dc/d12/d15/d19/d61/f54 [1629241,94379] 0 2024-03-04T13:43:15.105 INFO:tasks.workunit.client.0.smithi107.stdout:6/626: creat df/d2e/d36/d77/d95/da7/fcb x:0 0 0 2024-03-04T13:43:15.107 INFO:tasks.workunit.client.0.smithi107.stdout:1/625: fdatasync d1/d46/fd0 0 2024-03-04T13:43:15.111 INFO:tasks.workunit.client.1.smithi169.stderr:+ rm -rf -- ./tmp.a3Z4RkHcIM 2024-03-04T13:43:15.111 INFO:tasks.workunit.client.0.smithi107.stdout:6/627: write df/fac [519511,110336] 0 2024-03-04T13:43:15.113 INFO:tasks.workunit.client.0.smithi107.stdout:5/631: dwrite d5/d8/f17 [0,4194304] 0 2024-03-04T13:43:15.115 INFO:tasks.workunit.client.0.smithi107.stdout:8/639: creat d6/d16/d1f/d4c/da4/daa/fc3 x:0 0 0 2024-03-04T13:43:15.117 INFO:tasks.workunit.client.0.smithi107.stdout:7/615: dwrite d6/d1d/d64/d9a/da9/d71/db9/dca/fba [0,4194304] 0 2024-03-04T13:43:15.120 INFO:tasks.workunit.client.0.smithi107.stdout:9/601: dwrite d4/da/d11/d20/d64/f78 [0,4194304] 0 2024-03-04T13:43:15.121 INFO:tasks.workunit.client.0.smithi107.stdout:0/651: truncate da/dc/dd/d3e/d26/d88/dd2/fbd 243530 0 2024-03-04T13:43:15.125 INFO:tasks.workunit.client.0.smithi107.stdout:6/628: stat df/d57/ca0 0 2024-03-04T13:43:15.129 INFO:tasks.workunit.client.0.smithi107.stdout:0/652: chown da/dc/dd/d3e/f77 23872 1 2024-03-04T13:43:15.137 INFO:tasks.workunit.client.0.smithi107.stdout:6/629: rmdir df/d17/d24/d30 39 2024-03-04T13:43:15.138 INFO:tasks.workunit.client.0.smithi107.stdout:2/564: dwrite da/f18 [4194304,4194304] 0 2024-03-04T13:43:15.142 INFO:tasks.workunit.client.0.smithi107.stdout:9/602: mkdir d4/da/d11/d20/d23/dcc 0 2024-03-04T13:43:15.150 INFO:tasks.workunit.client.0.smithi107.stdout:2/565: creat da/d49/fc0 x:0 0 0 2024-03-04T13:43:15.164 INFO:tasks.workunit.client.0.smithi107.stdout:1/626: dread d1/d38/fb6 [0,4194304] 0 2024-03-04T13:43:15.168 INFO:tasks.workunit.client.0.smithi107.stdout:7/616: rename d6/d1d/d64/d9a/da9/d71/db9/dca/d22/cbd to d6/d13/cce 0 2024-03-04T13:43:15.169 INFO:tasks.workunit.client.0.smithi107.stdout:3/596: sync 2024-03-04T13:43:15.169 INFO:tasks.workunit.client.0.smithi107.stdout:4/563: sync 2024-03-04T13:43:15.170 INFO:tasks.workunit.client.0.smithi107.stdout:2/566: creat da/df/d73/db3/db1/fc1 x:0 0 0 2024-03-04T13:43:15.172 INFO:tasks.workunit.client.0.smithi107.stdout:8/640: link d6/d16/d59/d3e/d67/l91 d6/d16/d1f/lc4 0 2024-03-04T13:43:15.173 INFO:tasks.workunit.client.0.smithi107.stdout:5/632: dread d5/f42 [0,4194304] 0 2024-03-04T13:43:15.174 INFO:tasks.workunit.client.0.smithi107.stdout:0/653: getdents da/dc/dd/d3e/d26/d88/dd2/d73/d78 0 2024-03-04T13:43:15.175 INFO:tasks.workunit.client.0.smithi107.stdout:4/564: chown d0/d9/d1e 69551390 1 2024-03-04T13:43:15.179 INFO:tasks.workunit.client.0.smithi107.stdout:6/630: dwrite df/d17/d24/d40/d63/da3/d7e/d86/f97 [0,4194304] 0 2024-03-04T13:43:15.183 INFO:tasks.workunit.client.0.smithi107.stdout:9/603: write d4/da/d11/d20/d64/f76 [798669,44972] 0 2024-03-04T13:43:15.183 INFO:tasks.workunit.client.0.smithi107.stdout:0/654: fdatasync da/dc/dd/d3e/f35 0 2024-03-04T13:43:15.186 INFO:tasks.workunit.client.0.smithi107.stdout:5/633: creat d5/dc/df/fde x:0 0 0 2024-03-04T13:43:15.187 INFO:tasks.workunit.client.0.smithi107.stdout:4/565: readlink d0/d9/de/d14/d66/d7b/l9b 0 2024-03-04T13:43:15.190 INFO:tasks.workunit.client.0.smithi107.stdout:3/597: symlink d1/d1e/daf/lbd 0 2024-03-04T13:43:15.191 INFO:tasks.workunit.client.0.smithi107.stdout:9/604: write d4/da/d11/d65/f6e [1029982,128306] 0 2024-03-04T13:43:15.192 INFO:tasks.workunit.client.0.smithi107.stdout:0/655: read - da/dc/dd/d3e/d26/d88/dd2/d73/d78/fcf zero size 2024-03-04T13:43:15.193 INFO:tasks.workunit.client.0.smithi107.stdout:2/567: dwrite f0 [4194304,4194304] 0 2024-03-04T13:43:15.194 INFO:tasks.workunit.client.0.smithi107.stdout:1/627: creat d1/d10/d16/d1b/fd2 x:0 0 0 2024-03-04T13:43:15.200 INFO:tasks.workunit.client.0.smithi107.stdout:2/568: stat da/df/f1e 0 2024-03-04T13:43:15.202 INFO:tasks.workunit.client.0.smithi107.stdout:8/641: creat d6/d16/fc5 x:0 0 0 2024-03-04T13:43:15.204 INFO:tasks.workunit.client.0.smithi107.stdout:6/631: symlink df/d2e/d36/d5b/d7d/dbd/lcc 0 2024-03-04T13:43:15.205 INFO:tasks.workunit.client.0.smithi107.stdout:4/566: creat d0/d9/d78/fa7 x:0 0 0 2024-03-04T13:43:15.206 INFO:tasks.workunit.client.0.smithi107.stdout:8/642: fsync d6/da/d25/d5f/f87 0 2024-03-04T13:43:15.208 INFO:tasks.workunit.client.0.smithi107.stdout:7/617: dwrite d6/d13/d16/d19/d38/d2c/f4a [4194304,4194304] 0 2024-03-04T13:43:15.208 INFO:tasks.workunit.client.0.smithi107.stdout:2/569: rename da/df/d73/lac to da/df/d73/daa/lc2 0 2024-03-04T13:43:15.214 INFO:tasks.workunit.client.0.smithi107.stdout:2/570: readlink da/d44/d8b/d98/lad 0 2024-03-04T13:43:15.214 INFO:tasks.workunit.client.0.smithi107.stdout:0/656: mknod da/dc/dd/d3e/ce4 0 2024-03-04T13:43:15.216 INFO:tasks.workunit.client.0.smithi107.stdout:5/634: getdents d5/dc/d22 0 2024-03-04T13:43:15.216 INFO:tasks.workunit.client.0.smithi107.stdout:8/643: symlink d6/da/d25/lc6 0 2024-03-04T13:43:15.219 INFO:tasks.workunit.client.0.smithi107.stdout:4/567: chown d0/d9/de/d14/f4b 73 1 2024-03-04T13:43:15.220 INFO:tasks.workunit.client.0.smithi107.stdout:6/632: rename df/d38/f4e to df/d2e/fcd 0 2024-03-04T13:43:15.221 INFO:tasks.workunit.client.0.smithi107.stdout:0/657: chown da/dc/dd/d1e/fa1 42 1 2024-03-04T13:43:15.222 INFO:tasks.workunit.client.0.smithi107.stdout:6/633: chown c9 339 1 2024-03-04T13:43:15.223 INFO:tasks.workunit.client.0.smithi107.stdout:0/658: write da/f2a [1276651,123402] 0 2024-03-04T13:43:15.225 INFO:tasks.workunit.client.0.smithi107.stdout:2/571: unlink da/df/d27/l51 0 2024-03-04T13:43:15.226 INFO:tasks.workunit.client.0.smithi107.stdout:4/568: symlink d0/d30/la8 0 2024-03-04T13:43:15.228 INFO:tasks.workunit.client.0.smithi107.stdout:0/659: read - da/dc/dd/d3e/d26/d88/fd7 zero size 2024-03-04T13:43:15.234 INFO:tasks.workunit.client.0.smithi107.stdout:6/634: readlink df/d17/d24/d30/d3b/l4c 0 2024-03-04T13:43:15.235 INFO:tasks.workunit.client.0.smithi107.stdout:9/605: write d4/da/d11/d4c/d4d/f54 [2262834,54204] 0 2024-03-04T13:43:15.241 INFO:tasks.workunit.client.0.smithi107.stdout:6/635: unlink df/d17/d5c/f67 0 2024-03-04T13:43:15.242 INFO:tasks.workunit.client.0.smithi107.stdout:9/606: rmdir d4/da/d11/d20/d2f 39 2024-03-04T13:43:15.249 INFO:tasks.workunit.client.0.smithi107.stdout:9/607: creat d4/da/d11/d20/d23/d7e/d84/fcd x:0 0 0 2024-03-04T13:43:15.253 INFO:tasks.workunit.client.0.smithi107.stdout:5/635: truncate d5/dc/f3b 3638826 0 2024-03-04T13:43:15.261 INFO:tasks.workunit.client.0.smithi107.stdout:5/636: chown d5/d79/d89/da3 779656 1 2024-03-04T13:43:15.263 INFO:tasks.workunit.client.0.smithi107.stdout:4/569: write d0/d9/de/d14/dc/f3c [230087,19457] 0 2024-03-04T13:43:15.263 INFO:tasks.workunit.client.0.smithi107.stdout:0/660: dwrite da/f59 [0,4194304] 0 2024-03-04T13:43:15.263 INFO:tasks.workunit.client.0.smithi107.stdout:9/608: chown d4/da/d11/d4c/d51/l70 97783127 1 2024-03-04T13:43:15.268 INFO:tasks.workunit.client.0.smithi107.stdout:7/618: dwrite d6/d13/d16/d1c/f2d [0,4194304] 0 2024-03-04T13:43:15.272 INFO:tasks.workunit.client.0.smithi107.stdout:9/609: fsync d4/d34/f44 0 2024-03-04T13:43:15.282 INFO:tasks.workunit.client.0.smithi107.stdout:0/661: read - da/dc/dd/d1e/fb5 zero size 2024-03-04T13:43:15.283 INFO:tasks.workunit.client.0.smithi107.stdout:5/637: write d5/dc/d22/f61 [4097046,35678] 0 2024-03-04T13:43:15.285 INFO:tasks.workunit.client.0.smithi107.stdout:2/572: dwrite da/d34/f41 [4194304,4194304] 0 2024-03-04T13:43:15.286 INFO:tasks.workunit.client.0.smithi107.stdout:0/662: fsync da/f62 0 2024-03-04T13:43:15.289 INFO:tasks.workunit.client.0.smithi107.stdout:5/638: stat d5/d2b/c98 0 2024-03-04T13:43:15.290 INFO:tasks.workunit.client.0.smithi107.stdout:6/636: dwrite df/d2e/d36/d77/f84 [0,4194304] 0 2024-03-04T13:43:15.294 INFO:tasks.workunit.client.0.smithi107.stdout:8/644: dwrite d6/d16/fa3 [0,4194304] 0 2024-03-04T13:43:15.294 INFO:tasks.workunit.client.0.smithi107.stdout:3/598: dread d1/de/d18/d33/d48/d71/f72 [0,4194304] 0 2024-03-04T13:43:15.301 INFO:tasks.workunit.client.0.smithi107.stdout:6/637: chown df/d17/d5c/c8e 487007 1 2024-03-04T13:43:15.306 INFO:tasks.workunit.client.0.smithi107.stdout:5/639: rmdir d5/dc 39 2024-03-04T13:43:15.307 INFO:tasks.workunit.client.0.smithi107.stdout:5/640: stat d5/f29 0 2024-03-04T13:43:15.309 INFO:tasks.workunit.client.0.smithi107.stdout:4/570: getdents d0/d9/d1e/d70 0 2024-03-04T13:43:15.317 INFO:tasks.workunit.client.0.smithi107.stdout:0/663: rename da/dc/dd/d3e/d26/d88/dd2/d73/d78/d86 to da/dc/dd/dc9/de5 0 2024-03-04T13:43:15.319 INFO:tasks.workunit.client.0.smithi107.stdout:7/619: dwrite d6/d13/d16/d1c/f79 [0,4194304] 0 2024-03-04T13:43:15.319 INFO:tasks.workunit.client.0.smithi107.stdout:4/571: creat d0/d9/d18/fa9 x:0 0 0 2024-03-04T13:43:15.322 INFO:tasks.workunit.client.0.smithi107.stdout:8/645: dread d6/da/f35 [4194304,4194304] 0 2024-03-04T13:43:15.322 INFO:tasks.workunit.client.0.smithi107.stdout:9/610: rename d4/da/d11/d20/d23/d2c to d4/da/d11/d20/d2f/d9c/d92/dce 0 2024-03-04T13:43:15.324 INFO:tasks.workunit.client.0.smithi107.stdout:5/641: creat d5/dc/fdf x:0 0 0 2024-03-04T13:43:15.324 INFO:tasks.workunit.client.0.smithi107.stdout:0/664: truncate f0 489764 0 2024-03-04T13:43:15.326 INFO:tasks.workunit.client.0.smithi107.stdout:4/572: creat d0/d9/d18/faa x:0 0 0 2024-03-04T13:43:15.327 INFO:tasks.workunit.client.0.smithi107.stdout:7/620: mknod d6/d1d/d64/d9a/da9/d71/ccf 0 2024-03-04T13:43:15.328 INFO:tasks.workunit.client.0.smithi107.stdout:4/573: chown d0/d2f/d7d 956661270 1 2024-03-04T13:43:15.329 INFO:tasks.workunit.client.0.smithi107.stdout:9/611: write d4/da/d11/d20/d23/f6c [1925687,120543] 0 2024-03-04T13:43:15.330 INFO:tasks.workunit.client.0.smithi107.stdout:0/665: rmdir da/dc/dd/dc9 39 2024-03-04T13:43:15.332 INFO:tasks.workunit.client.0.smithi107.stdout:6/638: dread df/d2e/d36/d77/f84 [0,4194304] 0 2024-03-04T13:43:15.334 INFO:tasks.workunit.client.0.smithi107.stdout:3/599: write d1/d1e/d26/f3d [1444353,649] 0 2024-03-04T13:43:15.335 INFO:tasks.workunit.client.0.smithi107.stdout:7/621: rmdir d6/d13/d16/d19/d38 39 2024-03-04T13:43:15.339 INFO:tasks.workunit.client.0.smithi107.stdout:6/639: write df/d2e/d36/d77/f90 [2725773,87407] 0 2024-03-04T13:43:15.342 INFO:tasks.workunit.client.0.smithi107.stdout:5/642: creat d5/d66/d96/db1/fe0 x:0 0 0 2024-03-04T13:43:15.342 INFO:tasks.workunit.client.0.smithi107.stdout:4/574: truncate d0/d9/d1e/d70/f93 549805 0 2024-03-04T13:43:15.343 INFO:tasks.workunit.client.0.smithi107.stdout:9/612: mknod d4/da/d11/d20/d2f/d9c/d92/ccf 0 2024-03-04T13:43:15.345 INFO:tasks.workunit.client.0.smithi107.stdout:0/666: mknod da/dc/dd/d1e/ce6 0 2024-03-04T13:43:15.346 INFO:tasks.workunit.client.0.smithi107.stdout:2/573: dwrite da/d29/f4e [0,4194304] 0 2024-03-04T13:43:15.347 INFO:tasks.workunit.client.0.smithi107.stdout:4/575: dread d0/d30/f9c [0,4194304] 0 2024-03-04T13:43:15.353 INFO:tasks.workunit.client.0.smithi107.stdout:5/643: fdatasync d5/dc/f3d 0 2024-03-04T13:43:15.358 INFO:tasks.workunit.client.0.smithi107.stdout:8/646: dread f4 [4194304,4194304] 0 2024-03-04T13:43:15.358 INFO:tasks.workunit.client.0.smithi107.stdout:7/622: mkdir d6/d1d/d64/d9a/da9/d71/db9/dca/dd0 0 2024-03-04T13:43:15.360 INFO:tasks.workunit.client.0.smithi107.stdout:7/623: chown d6/d13/d16/d1c 447 1 2024-03-04T13:43:15.360 INFO:tasks.workunit.client.0.smithi107.stdout:6/640: dread f3 [0,4194304] 0 2024-03-04T13:43:15.360 INFO:tasks.workunit.client.0.smithi107.stdout:9/613: fsync d4/da/d11/d65/f3e 0 2024-03-04T13:43:15.363 INFO:tasks.workunit.client.0.smithi107.stdout:5/644: write d5/d8/d2a/f9d [2449,9171] 0 2024-03-04T13:43:15.367 INFO:tasks.workunit.client.0.smithi107.stdout:2/574: mkdir da/d44/d8b/d98/dc3 0 2024-03-04T13:43:15.368 INFO:tasks.workunit.client.0.smithi107.stdout:3/600: mkdir d1/de/d18/d33/d48/d71/d4c/d54/db7/dbe 0 2024-03-04T13:43:15.373 INFO:tasks.workunit.client.0.smithi107.stdout:8/647: symlink d6/d16/d1f/d63/d73/lc7 0 2024-03-04T13:43:15.374 INFO:tasks.workunit.client.0.smithi107.stdout:7/624: rmdir d6/d1d/d64/d9a/da9/d71/db9/dca/d22 39 2024-03-04T13:43:15.377 INFO:tasks.workunit.client.0.smithi107.stdout:3/601: dread d1/d1e/d26/f3d [0,4194304] 0 2024-03-04T13:43:15.378 INFO:tasks.workunit.client.0.smithi107.stdout:6/641: creat df/d17/d24/d40/d63/da3/d7e/d86/fce x:0 0 0 2024-03-04T13:43:15.379 INFO:tasks.workunit.client.0.smithi107.stdout:9/614: symlink d4/da/d11/d20/d64/d77/ld0 0 2024-03-04T13:43:15.383 INFO:tasks.workunit.client.0.smithi107.stdout:9/615: truncate d4/da/d11/d20/d39/f7a 5086690 0 2024-03-04T13:43:15.386 INFO:tasks.workunit.client.0.smithi107.stdout:5/645: creat d5/d79/d89/fe1 x:0 0 0 2024-03-04T13:43:15.388 INFO:tasks.workunit.client.0.smithi107.stdout:3/602: dread d1/de/f68 [0,4194304] 0 2024-03-04T13:43:15.389 INFO:tasks.workunit.client.0.smithi107.stdout:2/575: unlink da/df/f1e 0 2024-03-04T13:43:15.392 INFO:tasks.workunit.client.0.smithi107.stdout:0/667: creat da/dc/dd/fe7 x:0 0 0 2024-03-04T13:43:15.396 INFO:tasks.workunit.client.0.smithi107.stdout:1/628: sync 2024-03-04T13:43:15.398 INFO:tasks.workunit.client.0.smithi107.stdout:6/642: creat df/d38/d4f/fcf x:0 0 0 2024-03-04T13:43:15.401 INFO:tasks.workunit.client.0.smithi107.stdout:5/646: truncate d5/f9 1511552 0 2024-03-04T13:43:15.401 INFO:tasks.workunit.client.0.smithi107.stdout:6/643: write df/fac [247692,24711] 0 2024-03-04T13:43:15.401 INFO:tasks.workunit.client.0.smithi107.stdout:2/576: unlink da/d34/f83 0 2024-03-04T13:43:15.405 INFO:tasks.workunit.client.0.smithi107.stdout:5/647: truncate d5/dc/fdf 601517 0 2024-03-04T13:43:15.405 INFO:tasks.workunit.client.0.smithi107.stdout:2/577: chown da/df/d27/fa7 92949098 1 2024-03-04T13:43:15.406 INFO:tasks.workunit.client.0.smithi107.stdout:3/603: rename d1/de/d18/d1f/f43 to d1/de/d18/d33/d48/d71/d4c/d54/db7/fbf 0 2024-03-04T13:43:15.407 INFO:tasks.workunit.client.0.smithi107.stdout:4/576: dwrite d0/d9/d1e/f75 [0,4194304] 0 2024-03-04T13:43:15.407 INFO:tasks.workunit.client.0.smithi107.stdout:8/648: mkdir d6/d16/d1f/d63/dc8 0 2024-03-04T13:43:15.410 INFO:tasks.workunit.client.0.smithi107.stdout:1/629: symlink d1/d46/d6a/d4e/daa/dc9/d33/d43/ld3 0 2024-03-04T13:43:15.411 INFO:tasks.workunit.client.0.smithi107.stdout:8/649: chown d6/f29 2 1 2024-03-04T13:43:15.414 INFO:tasks.workunit.client.0.smithi107.stdout:2/578: readlink da/d49/d5c/d6a/lb7 0 2024-03-04T13:43:15.424 INFO:tasks.workunit.client.0.smithi107.stdout:9/616: dwrite d4/da/d11/d65/f58 [0,4194304] 0 2024-03-04T13:43:15.424 INFO:tasks.workunit.client.0.smithi107.stdout:3/604: dread - d1/de/d18/d33/d6b/fb1 zero size 2024-03-04T13:43:15.427 INFO:tasks.workunit.client.0.smithi107.stdout:7/625: dwrite d6/d1d/d64/d9a/da9/d71/db9/dca/d3d/f43 [0,4194304] 0 2024-03-04T13:43:15.430 INFO:tasks.workunit.client.0.smithi107.stdout:8/650: truncate d6/d16/d1f/d4c/da4/daa/fc3 36190 0 2024-03-04T13:43:15.430 INFO:tasks.workunit.client.0.smithi107.stdout:5/648: rename d5/dc/fab to d5/fe2 0 2024-03-04T13:43:15.436 INFO:tasks.workunit.client.0.smithi107.stdout:3/605: stat d1/de/d18/d33/d48/d71/d4c/d54/f79 0 2024-03-04T13:43:15.436 INFO:tasks.workunit.client.0.smithi107.stdout:2/579: write da/df/fb0 [3637856,87536] 0 2024-03-04T13:43:15.441 INFO:tasks.workunit.client.0.smithi107.stdout:6/644: dwrite df/d17/d24/d40/d63/da3/d7e/fa9 [0,4194304] 0 2024-03-04T13:43:15.441 INFO:tasks.workunit.client.0.smithi107.stdout:9/617: creat d4/da/d11/d20/d2f/fd1 x:0 0 0 2024-03-04T13:43:15.444 INFO:tasks.workunit.client.0.smithi107.stdout:3/606: write d1/de/d3f/fb3 [8923,128916] 0 2024-03-04T13:43:15.446 INFO:tasks.workunit.client.0.smithi107.stdout:1/630: truncate d1/d46/d6a/d4e/daa/dc9/d67/d6b/fb4 1670336 0 2024-03-04T13:43:15.453 INFO:tasks.workunit.client.0.smithi107.stdout:0/668: dwrite da/dc/f7c [0,4194304] 0 2024-03-04T13:43:15.453 INFO:tasks.workunit.client.0.smithi107.stdout:8/651: unlink d6/d16/l47 0 2024-03-04T13:43:15.453 INFO:tasks.workunit.client.0.smithi107.stdout:7/626: write d6/d13/d16/d19/d38/d2c/f4a [1433510,77645] 0 2024-03-04T13:43:15.454 INFO:tasks.workunit.client.0.smithi107.stdout:2/580: creat da/d39/fc4 x:0 0 0 2024-03-04T13:43:15.456 INFO:tasks.workunit.client.0.smithi107.stdout:9/618: unlink d4/f45 0 2024-03-04T13:43:15.457 INFO:tasks.workunit.client.0.smithi107.stdout:7/627: chown d6/d1d/d64/d9a/da9/d71 621082 1 2024-03-04T13:43:15.457 INFO:tasks.workunit.client.0.smithi107.stdout:8/652: write d6/da/f50 [128653,41244] 0 2024-03-04T13:43:15.457 INFO:tasks.workunit.client.0.smithi107.stdout:1/631: write d1/d46/d6a/d4e/daa/dc9/d33/d5d/f97 [3461350,35352] 0 2024-03-04T13:43:15.460 INFO:tasks.workunit.client.0.smithi107.stdout:2/581: dread - da/d38/d3a/fae zero size 2024-03-04T13:43:15.463 INFO:tasks.workunit.client.0.smithi107.stdout:5/649: symlink d5/d2b/d4b/da6/dcf/le3 0 2024-03-04T13:43:15.463 INFO:tasks.workunit.client.0.smithi107.stdout:8/653: chown d6/da/d25/d2a/l80 970107421 1 2024-03-04T13:43:15.463 INFO:tasks.workunit.client.0.smithi107.stdout:6/645: symlink df/d38/d4f/daf/ld0 0 2024-03-04T13:43:15.464 INFO:tasks.workunit.client.0.smithi107.stdout:4/577: rename d0/d9/d18/d29/d92 to d0/d9/d10/dab 0 2024-03-04T13:43:15.467 INFO:tasks.workunit.client.0.smithi107.stdout:2/582: mkdir da/d49/d5c/dc5 0 2024-03-04T13:43:15.471 INFO:tasks.workunit.client.0.smithi107.stdout:4/578: symlink d0/d30/lac 0 2024-03-04T13:43:15.473 INFO:tasks.workunit.client.0.smithi107.stdout:8/654: creat d6/d16/d1f/d4c/da5/fc9 x:0 0 0 2024-03-04T13:43:15.486 INFO:tasks.workunit.client.0.smithi107.stdout:7/628: dwrite d6/d13/d16/fbb [0,4194304] 0 2024-03-04T13:43:15.493 INFO:tasks.workunit.client.0.smithi107.stdout:8/655: creat d6/d16/d1f/d4c/da4/daa/fca x:0 0 0 2024-03-04T13:43:15.497 INFO:tasks.workunit.client.0.smithi107.stdout:1/632: link d1/d46/d6a/d65/db1/cc0 d1/d46/d6a/d65/cd4 0 2024-03-04T13:43:15.497 INFO:tasks.workunit.client.0.smithi107.stdout:8/656: truncate d6/d16/d1f/d4c/da5/fb2 659790 0 2024-03-04T13:43:15.500 INFO:tasks.workunit.client.0.smithi107.stdout:0/669: dread da/f2a [0,4194304] 0 2024-03-04T13:43:15.504 INFO:tasks.workunit.client.0.smithi107.stdout:4/579: write d0/d30/f3d [771684,79447] 0 2024-03-04T13:43:15.508 INFO:tasks.workunit.client.0.smithi107.stdout:7/629: dread d6/d1d/d64/d9a/da9/d71/fcc [0,4194304] 0 2024-03-04T13:43:15.509 INFO:tasks.workunit.client.0.smithi107.stdout:2/583: dwrite da/df/d3e/f50 [4194304,4194304] 0 2024-03-04T13:43:15.511 INFO:tasks.workunit.client.0.smithi107.stdout:6/646: dread df/d17/d24/f2d [0,4194304] 0 2024-03-04T13:43:15.511 INFO:tasks.workunit.client.0.smithi107.stdout:5/650: dread d5/d79/d89/da3/d6a/f8e [0,4194304] 0 2024-03-04T13:43:15.516 INFO:tasks.workunit.client.0.smithi107.stdout:4/580: creat d0/d9/d10/d6f/fad x:0 0 0 2024-03-04T13:43:15.516 INFO:tasks.workunit.client.0.smithi107.stdout:7/630: chown d6/d13/ca7 1155 1 2024-03-04T13:43:15.517 INFO:tasks.workunit.client.0.smithi107.stdout:6/647: read df/d17/d24/d40/fa5 [1693658,98447] 0 2024-03-04T13:43:15.518 INFO:tasks.workunit.client.0.smithi107.stdout:0/670: getdents da 0 2024-03-04T13:43:15.520 INFO:tasks.workunit.client.0.smithi107.stdout:2/584: dread da/df/d3e/f40 [0,4194304] 0 2024-03-04T13:43:15.523 INFO:tasks.workunit.client.0.smithi107.stdout:6/648: readlink df/d2e/d36/d5b/lc9 0 2024-03-04T13:43:15.523 INFO:tasks.workunit.client.0.smithi107.stdout:2/585: stat da/d49/d5c/dc5 0 2024-03-04T13:43:15.524 INFO:tasks.workunit.client.0.smithi107.stdout:5/651: dread d5/dc/f58 [0,4194304] 0 2024-03-04T13:43:15.536 INFO:tasks.workunit.client.0.smithi107.stdout:8/657: dwrite d6/f3d [0,4194304] 0 2024-03-04T13:43:15.536 INFO:tasks.workunit.client.0.smithi107.stdout:1/633: dwrite d1/d10/d16/d1b/d40/f57 [0,4194304] 0 2024-03-04T13:43:15.538 INFO:tasks.workunit.client.0.smithi107.stdout:2/586: unlink c9 0 2024-03-04T13:43:15.543 INFO:tasks.workunit.client.0.smithi107.stdout:1/634: write d1/d46/d6a/d4e/daa/dc9/d33/d43/d5b/d61/fce [866218,124222] 0 2024-03-04T13:43:15.543 INFO:tasks.workunit.client.0.smithi107.stdout:0/671: write da/dc/dd/f24 [1313378,16860] 0 2024-03-04T13:43:15.545 INFO:tasks.workunit.client.0.smithi107.stdout:7/631: truncate d6/d13/d16/d19/d38/d62/f9b 359834 0 2024-03-04T13:43:15.548 INFO:tasks.workunit.client.0.smithi107.stdout:4/581: creat d0/fae x:0 0 0 2024-03-04T13:43:15.551 INFO:tasks.workunit.client.0.smithi107.stdout:8/658: mkdir d6/da/d25/dcb 0 2024-03-04T13:43:15.553 INFO:tasks.workunit.client.0.smithi107.stdout:2/587: mknod da/df/d73/db3/db1/cc6 0 2024-03-04T13:43:15.559 INFO:tasks.workunit.client.0.smithi107.stdout:6/649: dread df/d1b/f4b [0,4194304] 0 2024-03-04T13:43:15.565 INFO:tasks.workunit.client.0.smithi107.stdout:1/635: symlink d1/ld5 0 2024-03-04T13:43:15.569 INFO:tasks.workunit.client.0.smithi107.stdout:5/652: rename d5/d79/d89/da3/d69/db9 to d5/dc/de4 0 2024-03-04T13:43:15.570 INFO:tasks.workunit.client.0.smithi107.stdout:0/672: symlink da/dc/dd/ddb/le8 0 2024-03-04T13:43:15.574 INFO:tasks.workunit.client.0.smithi107.stdout:7/632: mkdir d6/d13/d16/d19/d38/dd1 0 2024-03-04T13:43:15.574 INFO:tasks.workunit.client.0.smithi107.stdout:1/636: truncate d1/d46/d6a/d4e/daa/dc9/d33/d43/d5b/d61/fce 1065385 0 2024-03-04T13:43:15.575 INFO:tasks.workunit.client.0.smithi107.stdout:2/588: dread da/d34/f41 [4194304,4194304] 0 2024-03-04T13:43:15.577 INFO:tasks.workunit.client.0.smithi107.stdout:2/589: dread - da/d44/f64 zero size 2024-03-04T13:43:15.577 INFO:tasks.workunit.client.0.smithi107.stdout:7/633: stat d6/d1d/d64/d9a/da9/d71/db9/dca/d3d/dc6/dcb 0 2024-03-04T13:43:15.577 INFO:tasks.workunit.client.0.smithi107.stdout:1/637: read d1/d46/d6a/d4e/daa/dc9/d33/d5d/f97 [6513849,88278] 0 2024-03-04T13:43:15.578 INFO:tasks.workunit.client.0.smithi107.stdout:8/659: mkdir d6/dcc 0 2024-03-04T13:43:15.579 INFO:tasks.workunit.client.0.smithi107.stdout:8/660: stat d6/d16/d59/d3e/d96/ca1 0 2024-03-04T13:43:15.581 INFO:tasks.workunit.client.0.smithi107.stdout:6/650: write df/d17/d24/f2d [3406620,42214] 0 2024-03-04T13:43:15.586 INFO:tasks.workunit.client.0.smithi107.stdout:0/673: fdatasync da/dc/dd/d3e/f7a 0 2024-03-04T13:43:15.587 INFO:tasks.workunit.client.0.smithi107.stdout:4/582: dread d0/d9/de/d14/dc/f9e [0,4194304] 0 2024-03-04T13:43:15.593 INFO:tasks.workunit.client.0.smithi107.stdout:8/661: dwrite f4 [0,4194304] 0 2024-03-04T13:43:15.596 INFO:tasks.workunit.client.0.smithi107.stdout:2/590: mknod da/d38/cc7 0 2024-03-04T13:43:15.596 INFO:tasks.workunit.client.0.smithi107.stdout:8/662: truncate d6/d16/fc5 185676 0 2024-03-04T13:43:15.600 INFO:tasks.workunit.client.0.smithi107.stdout:8/663: stat d6/d16/d1f/d63/d73/lc7 0 2024-03-04T13:43:15.605 INFO:tasks.workunit.client.0.smithi107.stdout:7/634: symlink d6/d1d/d36/ld2 0 2024-03-04T13:43:15.605 INFO:tasks.workunit.client.0.smithi107.stdout:4/583: truncate d0/d9/de/d14/dc/d21/f2c 4384823 0 2024-03-04T13:43:15.607 INFO:tasks.workunit.client.0.smithi107.stdout:5/653: dwrite d5/d8/d14/d2d/d92/fc0 [0,4194304] 0 2024-03-04T13:43:15.614 INFO:tasks.workunit.client.0.smithi107.stdout:0/674: dread da/dc/f99 [0,4194304] 0 2024-03-04T13:43:15.622 INFO:tasks.workunit.client.0.smithi107.stdout:8/664: symlink d6/d16/d59/d7c/lcd 0 2024-03-04T13:43:15.625 INFO:tasks.workunit.client.0.smithi107.stdout:1/638: rename d1/d10/d16/d47/d71/d82 to d1/dd6 0 2024-03-04T13:43:15.625 INFO:tasks.workunit.client.0.smithi107.stdout:8/665: write d6/d16/d1f/d4c/da4/daa/fc3 [648818,22324] 0 2024-03-04T13:43:15.626 INFO:tasks.workunit.client.0.smithi107.stdout:2/591: unlink da/d39/c55 0 2024-03-04T13:43:15.627 INFO:tasks.workunit.client.0.smithi107.stdout:6/651: creat df/d17/fd1 x:0 0 0 2024-03-04T13:43:15.628 INFO:tasks.workunit.client.0.smithi107.stdout:2/592: read - da/d38/d3a/fae zero size 2024-03-04T13:43:15.629 INFO:tasks.workunit.client.0.smithi107.stdout:0/675: truncate da/dc/dd/f2c 1235949 0 2024-03-04T13:43:15.630 INFO:tasks.workunit.client.0.smithi107.stdout:2/593: write da/d49/d5c/f67 [2336813,12148] 0 2024-03-04T13:43:15.634 INFO:tasks.workunit.client.0.smithi107.stdout:2/594: write da/df/d73/db3/db1/fc1 [890991,31785] 0 2024-03-04T13:43:15.634 INFO:tasks.workunit.client.0.smithi107.stdout:6/652: creat df/d38/d4f/daf/fd2 x:0 0 0 2024-03-04T13:43:15.639 INFO:tasks.workunit.client.0.smithi107.stdout:9/619: sync 2024-03-04T13:43:15.639 INFO:tasks.workunit.client.0.smithi107.stdout:3/607: sync 2024-03-04T13:43:15.640 INFO:tasks.workunit.client.0.smithi107.stdout:3/608: stat d1/de/d3f/lb4 0 2024-03-04T13:43:15.642 INFO:tasks.workunit.client.0.smithi107.stdout:4/584: link d0/d9/d18/d29/l8d d0/d9/de/d14/dc/d97/laf 0 2024-03-04T13:43:15.645 INFO:tasks.workunit.client.0.smithi107.stdout:8/666: creat d6/d16/d1f/d63/d93/fce x:0 0 0 2024-03-04T13:43:15.646 INFO:tasks.workunit.client.0.smithi107.stdout:2/595: readlink da/d44/l6e 0 2024-03-04T13:43:15.649 INFO:tasks.workunit.client.0.smithi107.stdout:6/653: unlink df/d17/d24/d40/d63/da3/d7e/caa 0 2024-03-04T13:43:15.652 INFO:tasks.workunit.client.0.smithi107.stdout:1/639: dwrite d1/d46/d6a/d4e/daa/dc9/d33/d5d/d24/f2d [0,4194304] 0 2024-03-04T13:43:15.656 INFO:tasks.workunit.client.0.smithi107.stdout:6/654: chown df/d2e/d36/d77/d95/da7/fcb 61520013 1 2024-03-04T13:43:15.657 INFO:tasks.workunit.client.0.smithi107.stdout:9/620: dwrite d4/da/d11/d20/d23/d7e/d84/fb9 [0,4194304] 0 2024-03-04T13:43:15.658 INFO:tasks.workunit.client.0.smithi107.stdout:7/635: rename d6/d13/d16/d19/d38/f99 to d6/d1d/d64/d9a/da9/d71/db9/dca/fd3 0 2024-03-04T13:43:15.662 INFO:tasks.workunit.client.0.smithi107.stdout:1/640: write d1/d46/d6a/d4e/d60/f4f [3294483,86391] 0 2024-03-04T13:43:15.664 INFO:tasks.workunit.client.0.smithi107.stdout:1/641: chown d1/d46/d9f 3267768 1 2024-03-04T13:43:15.667 INFO:tasks.workunit.client.0.smithi107.stdout:4/585: dwrite d0/d9/d18/d29/f5c [0,4194304] 0 2024-03-04T13:43:15.668 INFO:tasks.workunit.client.0.smithi107.stdout:9/621: write d4/da/d11/d20/d23/d7e/d84/fb9 [3230123,24221] 0 2024-03-04T13:43:15.669 INFO:tasks.workunit.client.0.smithi107.stdout:3/609: truncate d1/de/d18/d33/f8f 3827144 0 2024-03-04T13:43:15.672 INFO:tasks.workunit.client.0.smithi107.stdout:6/655: rmdir df/d2e/d36/d77/d95/da7 39 2024-03-04T13:43:15.672 INFO:tasks.workunit.client.0.smithi107.stdout:7/636: rmdir d6/d1d/d64/d9a/da9/d71/db9 39 2024-03-04T13:43:15.676 INFO:tasks.workunit.client.0.smithi107.stdout:1/642: symlink d1/d46/ld7 0 2024-03-04T13:43:15.676 INFO:tasks.workunit.client.0.smithi107.stdout:3/610: rmdir d1/de/d4b 39 2024-03-04T13:43:15.683 INFO:tasks.workunit.client.0.smithi107.stdout:9/622: rmdir d4/da/d11/d20/d23/d7e 39 2024-03-04T13:43:15.684 INFO:tasks.workunit.client.0.smithi107.stdout:0/676: dwrite da/dc/dd/f2c [0,4194304] 0 2024-03-04T13:43:15.686 INFO:tasks.workunit.client.0.smithi107.stdout:6/656: symlink df/d2e/d36/ld3 0 2024-03-04T13:43:15.686 INFO:tasks.workunit.client.0.smithi107.stdout:5/654: rename d5/d8/d14/d2d/l6e to d5/d8/le5 0 2024-03-04T13:43:15.687 INFO:tasks.workunit.client.0.smithi107.stdout:9/623: stat d4/da/d11/d4c/d51/fb8 0 2024-03-04T13:43:15.691 INFO:tasks.workunit.client.0.smithi107.stdout:0/677: chown da/c40 52221905 1 2024-03-04T13:43:15.691 INFO:tasks.workunit.client.0.smithi107.stdout:3/611: creat d1/de/d18/d1f/d42/fc0 x:0 0 0 2024-03-04T13:43:15.692 INFO:tasks.workunit.client.0.smithi107.stdout:4/586: rename d0/d30/ca3 to d0/d9/d10/d80/cb0 0 2024-03-04T13:43:15.695 INFO:tasks.workunit.client.0.smithi107.stdout:6/657: truncate df/d17/f23 5568196 0 2024-03-04T13:43:15.696 INFO:tasks.workunit.client.0.smithi107.stdout:5/655: mkdir d5/d8/da/db0/de6 0 2024-03-04T13:43:15.697 INFO:tasks.workunit.client.0.smithi107.stdout:9/624: symlink d4/da/d11/d20/d2f/d9c/d92/ld2 0 2024-03-04T13:43:15.698 INFO:tasks.workunit.client.0.smithi107.stdout:0/678: mkdir da/dc/dd/d3e/d26/d69/de9 0 2024-03-04T13:43:15.699 INFO:tasks.workunit.client.0.smithi107.stdout:3/612: symlink d1/de/d3f/lc1 0 2024-03-04T13:43:15.699 INFO:tasks.workunit.client.0.smithi107.stdout:1/643: truncate d1/d46/d6a/d4e/daa/dc9/d33/d5d/f97 9249709 0 2024-03-04T13:43:15.704 INFO:tasks.workunit.client.0.smithi107.stdout:6/658: chown df/d17/d24/f2d 25063 1 2024-03-04T13:43:15.706 INFO:tasks.workunit.client.0.smithi107.stdout:4/587: fdatasync d0/d9/d10/f7a 0 2024-03-04T13:43:15.707 INFO:tasks.workunit.client.0.smithi107.stdout:9/625: chown d4/da/d11/d4c/d4d/d72/d7b/f81 1170 1 2024-03-04T13:43:15.710 INFO:tasks.workunit.client.0.smithi107.stdout:8/667: dwrite d6/da/f11 [0,4194304] 0 2024-03-04T13:43:15.712 INFO:tasks.workunit.client.0.smithi107.stdout:1/644: chown d1/d46/d6a/d4e/daa/dc9/d33/d5d/d4c 106870315 1 2024-03-04T13:43:15.712 INFO:tasks.workunit.client.0.smithi107.stdout:4/588: dread - d0/d9/d78/fa7 zero size 2024-03-04T13:43:15.713 INFO:tasks.workunit.client.0.smithi107.stdout:3/613: mknod d1/d1e/daf/cc2 0 2024-03-04T13:43:15.717 INFO:tasks.workunit.client.0.smithi107.stdout:3/614: stat d1/d41/d81/lac 0 2024-03-04T13:43:15.721 INFO:tasks.workunit.client.0.smithi107.stdout:8/668: fdatasync d6/d16/d59/fc1 0 2024-03-04T13:43:15.722 INFO:tasks.workunit.client.0.smithi107.stdout:6/659: fdatasync df/d17/d24/d30/d8b/f8f 0 2024-03-04T13:43:15.730 INFO:tasks.workunit.client.0.smithi107.stdout:5/656: dwrite d5/d8/f7c [0,4194304] 0 2024-03-04T13:43:15.733 INFO:tasks.workunit.client.0.smithi107.stdout:2/596: truncate da/df/d27/fa7 1930807 0 2024-03-04T13:43:15.734 INFO:tasks.workunit.client.0.smithi107.stdout:3/615: mkdir d1/de/d18/d33/d4d/d4f/dc3 0 2024-03-04T13:43:15.737 INFO:tasks.workunit.client.0.smithi107.stdout:6/660: creat df/d2e/fd4 x:0 0 0 2024-03-04T13:43:15.740 INFO:tasks.workunit.client.0.smithi107.stdout:5/657: write d5/fe2 [5065433,20369] 0 2024-03-04T13:43:15.742 INFO:tasks.workunit.client.0.smithi107.stdout:6/661: dread - df/d38/d4f/fcf zero size 2024-03-04T13:43:15.745 INFO:tasks.workunit.client.0.smithi107.stdout:3/616: read d1/de/d18/f5f [2100368,25376] 0 2024-03-04T13:43:15.747 INFO:tasks.workunit.client.0.smithi107.stdout:0/679: dwrite da/dc/d34/f5a [0,4194304] 0 2024-03-04T13:43:15.751 INFO:tasks.workunit.client.0.smithi107.stdout:4/589: link d0/d9/de/d14/dc/f27 d0/d9/de/d14/dc/d97/fb1 0 2024-03-04T13:43:15.753 INFO:tasks.workunit.client.0.smithi107.stdout:2/597: link da/d49/fa4 da/d34/fc8 0 2024-03-04T13:43:15.754 INFO:tasks.workunit.client.0.smithi107.stdout:7/637: dread d6/d1d/d64/d9a/da9/d71/db9/fc5 [0,4194304] 0 2024-03-04T13:43:15.755 INFO:tasks.workunit.client.0.smithi107.stdout:2/598: write da/df/f15 [667758,48119] 0 2024-03-04T13:43:15.758 INFO:tasks.workunit.client.0.smithi107.stdout:7/638: dread - d6/d1d/d64/d9a/fc0 zero size 2024-03-04T13:43:15.760 INFO:tasks.workunit.client.0.smithi107.stdout:9/626: link d4/da/d11/d4c/d4d/c8c d4/da/d11/d20/d2f/d9c/cd3 0 2024-03-04T13:43:15.762 INFO:tasks.workunit.client.0.smithi107.stdout:7/639: fdatasync d6/d13/d16/d1c/f1f 0 2024-03-04T13:43:15.763 INFO:tasks.workunit.client.0.smithi107.stdout:6/662: mknod df/d17/cd5 0 2024-03-04T13:43:15.766 INFO:tasks.workunit.client.0.smithi107.stdout:0/680: chown da/dc/dd/d3e/d26/d88/dd2/d73/cca 0 1 2024-03-04T13:43:15.769 INFO:tasks.workunit.client.0.smithi107.stdout:4/590: read d0/d9/de/d14/d33/f45 [4750,126546] 0 2024-03-04T13:43:15.770 INFO:tasks.workunit.client.0.smithi107.stdout:6/663: write df/d2e/fd4 [309240,106326] 0 2024-03-04T13:43:15.770 INFO:tasks.workunit.client.0.smithi107.stdout:1/645: link d1/d46/la5 d1/d46/d6a/d4e/daa/dc9/d33/d5d/ld8 0 2024-03-04T13:43:15.771 INFO:tasks.workunit.client.0.smithi107.stdout:3/617: symlink d1/de/d18/d33/lc4 0 2024-03-04T13:43:15.772 INFO:tasks.workunit.client.0.smithi107.stdout:7/640: write d6/d13/d16/d19/d39/f54 [232429,9023] 0 2024-03-04T13:43:15.773 INFO:tasks.workunit.client.0.smithi107.stdout:9/627: unlink d4/da/d11/d20/d2f/d9c/d92/cac 0 2024-03-04T13:43:15.903 INFO:tasks.workunit.client.0.smithi107.stdout:5/658: rename d5/dc/df/f59 to d5/d8/fe7 0 2024-03-04T13:43:15.903 INFO:tasks.workunit.client.0.smithi107.stdout:8/669: dwrite d6/fd [0,4194304] 0 2024-03-04T13:43:15.903 INFO:tasks.workunit.client.0.smithi107.stdout:9/628: write d4/da/f4a [95565,111384] 0 2024-03-04T13:43:15.903 INFO:tasks.workunit.client.0.smithi107.stdout:6/664: chown df/d2e/d36/d77/c80 404 1 2024-03-04T13:43:15.903 INFO:tasks.workunit.client.0.smithi107.stdout:2/599: creat da/df/d73/d7a/d9a/fc9 x:0 0 0 2024-03-04T13:43:15.903 INFO:tasks.workunit.client.0.smithi107.stdout:3/618: fsync d1/de/d18/d33/d48/d71/d4c/f91 0 2024-03-04T13:43:15.904 INFO:tasks.workunit.client.0.smithi107.stdout:7/641: chown d6/d1d/d64/d9a/da9/d71/db9/dca/d3d/dc6 212 1 2024-03-04T13:43:15.904 INFO:tasks.workunit.client.0.smithi107.stdout:8/670: dread - d6/da/d25/d2a/f95 zero size 2024-03-04T13:43:15.904 INFO:tasks.workunit.client.0.smithi107.stdout:6/665: fsync df/d2e/d36/d5b/fa1 0 2024-03-04T13:43:15.904 INFO:tasks.workunit.client.0.smithi107.stdout:5/659: write d5/d2b/d4b/f91 [1185702,95384] 0 2024-03-04T13:43:15.904 INFO:tasks.workunit.client.0.smithi107.stdout:9/629: chown d4/da/d11/d20/d2f/d9c/d92/la0 975 1 2024-03-04T13:43:15.904 INFO:tasks.workunit.client.0.smithi107.stdout:3/619: chown d1/d1e/daf/cc2 1 1 2024-03-04T13:43:15.904 INFO:tasks.workunit.client.0.smithi107.stdout:0/681: rename da/dc/d34/c4b to da/dc/dd/d3e/d26/d69/cea 0 2024-03-04T13:43:15.904 INFO:tasks.workunit.client.0.smithi107.stdout:9/630: stat d4/da/d11/d20/d23/fc1 0 2024-03-04T13:43:15.904 INFO:tasks.workunit.client.0.smithi107.stdout:1/646: truncate d1/d10/d16/f37 4699542 0 2024-03-04T13:43:15.904 INFO:tasks.workunit.client.0.smithi107.stdout:4/591: creat d0/d9/fb2 x:0 0 0 2024-03-04T13:43:15.904 INFO:tasks.workunit.client.0.smithi107.stdout:5/660: creat d5/d79/d89/da3/d68/fe8 x:0 0 0 2024-03-04T13:43:15.904 INFO:tasks.workunit.client.0.smithi107.stdout:9/631: symlink d4/da/d11/d20/d2f/d9c/d92/ld4 0 2024-03-04T13:43:15.904 INFO:tasks.workunit.client.0.smithi107.stdout:0/682: mkdir da/dc/dd/d1e/dbf/deb 0 2024-03-04T13:43:15.905 INFO:tasks.workunit.client.0.smithi107.stdout:3/620: mkdir d1/de/d18/d33/d48/d71/d4c/daa/dab/dc5 0 2024-03-04T13:43:15.905 INFO:tasks.workunit.client.0.smithi107.stdout:2/600: write da/df/d27/fa7 [403857,33072] 0 2024-03-04T13:43:15.905 INFO:tasks.workunit.client.0.smithi107.stdout:1/647: unlink d1/d46/d6a/d4e/d60/cc2 0 2024-03-04T13:43:15.905 INFO:tasks.workunit.client.0.smithi107.stdout:1/648: fdatasync d1/d10/d16/d1b/f3e 0 2024-03-04T13:43:15.905 INFO:tasks.workunit.client.0.smithi107.stdout:8/671: rename d6/da/f35 to d6/d16/fcf 0 2024-03-04T13:43:15.905 INFO:tasks.workunit.client.0.smithi107.stdout:7/642: rename d6/d1d/d64 to d6/d1d/d64/d9a/da9/d71/db9/dca/d3d/dc6/dcb/dd4 22 2024-03-04T13:43:15.905 INFO:tasks.workunit.client.0.smithi107.stdout:4/592: mkdir d0/d9/de/d14/dc/d9a/db3 0 2024-03-04T13:43:15.905 INFO:tasks.workunit.client.0.smithi107.stdout:2/601: symlink da/d39/lca 0 2024-03-04T13:43:15.905 INFO:tasks.workunit.client.0.smithi107.stdout:6/666: getdents df/d17/d24 0 2024-03-04T13:43:15.905 INFO:tasks.workunit.client.0.smithi107.stdout:8/672: mknod d6/d16/d1f/d63/d73/cd0 0 2024-03-04T13:43:15.905 INFO:tasks.workunit.client.0.smithi107.stdout:2/602: chown da/d44/f52 2461918 1 2024-03-04T13:43:15.905 INFO:tasks.workunit.client.0.smithi107.stdout:1/649: chown d1/d46/d6a/d4e/daa/dc9/d33/d43/f55 27265 1 2024-03-04T13:43:15.905 INFO:tasks.workunit.client.0.smithi107.stdout:0/683: dwrite da/dc/dd/d3e/d26/d69/d6b/fd3 [0,4194304] 0 2024-03-04T13:43:15.905 INFO:tasks.workunit.client.0.smithi107.stdout:3/621: dread d1/de/d11/f3e [0,4194304] 0 2024-03-04T13:43:15.906 INFO:tasks.workunit.client.0.smithi107.stdout:5/661: truncate d5/d66/fa7 54153 0 2024-03-04T13:43:15.906 INFO:tasks.workunit.client.0.smithi107.stdout:6/667: rmdir df/d17/d6b/d9b/d61 39 2024-03-04T13:43:15.906 INFO:tasks.workunit.client.0.smithi107.stdout:9/632: dread d4/da/f1b [0,4194304] 0 2024-03-04T13:43:15.906 INFO:tasks.workunit.client.0.smithi107.stdout:0/684: stat da/dc/dd/d3e/d26 0 2024-03-04T13:43:15.906 INFO:tasks.workunit.client.0.smithi107.stdout:7/643: creat d6/d1d/d64/d9a/da9/d71/da5/fd5 x:0 0 0 2024-03-04T13:43:15.906 INFO:tasks.workunit.client.0.smithi107.stdout:5/662: truncate d5/dc/de4/fc3 419669 0 2024-03-04T13:43:15.906 INFO:tasks.workunit.client.0.smithi107.stdout:8/673: mkdir d6/da/d25/dd1 0 2024-03-04T13:43:15.906 INFO:tasks.workunit.client.0.smithi107.stdout:2/603: mknod da/d38/ccb 0 2024-03-04T13:43:15.906 INFO:tasks.workunit.client.0.smithi107.stdout:6/668: symlink df/d2e/ld6 0 2024-03-04T13:43:15.906 INFO:tasks.workunit.client.0.smithi107.stdout:1/650: rename d1/d46/d6a/d4e/d60/f63 to d1/d10/d16/fd9 0 2024-03-04T13:43:15.906 INFO:tasks.workunit.client.0.smithi107.stdout:5/663: creat d5/d79/fe9 x:0 0 0 2024-03-04T13:43:15.906 INFO:tasks.workunit.client.0.smithi107.stdout:4/593: link d0/d9/f6b d0/d9/de/d14/dc/fb4 0 2024-03-04T13:43:15.906 INFO:tasks.workunit.client.0.smithi107.stdout:8/674: write d6/d16/d1f/f5c [2984102,125531] 0 2024-03-04T13:43:15.906 INFO:tasks.workunit.client.0.smithi107.stdout:4/594: read d0/d9/de/d14/dc/d21/f5d [2408271,6587] 0 2024-03-04T13:43:15.906 INFO:tasks.workunit.client.0.smithi107.stdout:0/685: creat da/dc/dd/dc9/de5/fec x:0 0 0 2024-03-04T13:43:15.907 INFO:tasks.workunit.client.0.smithi107.stdout:0/686: fsync da/f62 0 2024-03-04T13:43:15.907 INFO:tasks.workunit.client.0.smithi107.stdout:4/595: chown d0/d9/d1e/c6e 3447062 1 2024-03-04T13:43:15.907 INFO:tasks.workunit.client.0.smithi107.stdout:4/596: fsync d0/d30/f50 0 2024-03-04T13:43:15.907 INFO:tasks.workunit.client.0.smithi107.stdout:4/597: write d0/d9/d10/f4a [4984359,122831] 0 2024-03-04T13:43:15.907 INFO:tasks.workunit.client.0.smithi107.stdout:4/598: chown d0/d9/d1e/f75 23 1 2024-03-04T13:43:15.907 INFO:tasks.workunit.client.0.smithi107.stdout:8/675: rmdir d6/d16/d1f/d4c 39 2024-03-04T13:43:15.907 INFO:tasks.workunit.client.0.smithi107.stdout:9/633: creat d4/da/d11/d20/fd5 x:0 0 0 2024-03-04T13:43:15.907 INFO:tasks.workunit.client.0.smithi107.stdout:9/634: chown d4/da/d11/d4c/d51/c5b 487125272 1 2024-03-04T13:43:15.907 INFO:tasks.workunit.client.0.smithi107.stdout:9/635: dread - d4/da/d11/d20/d23/fca zero size 2024-03-04T13:43:15.907 INFO:tasks.workunit.client.0.smithi107.stdout:9/636: fdatasync d4/da/f15 0 2024-03-04T13:43:15.907 INFO:tasks.workunit.client.0.smithi107.stdout:3/622: dread d1/d1e/f74 [0,4194304] 0 2024-03-04T13:43:15.907 INFO:tasks.workunit.client.0.smithi107.stdout:5/664: mkdir d5/d8/d2a/dd6/dea 0 2024-03-04T13:43:15.908 INFO:tasks.workunit.client.0.smithi107.stdout:2/604: link da/d49/d5c/d6a/lb7 da/df/d73/db3/db1/lcc 0 2024-03-04T13:43:15.913 INFO:tasks.workunit.client.0.smithi107.stdout:4/599: link d0/d9/d18/fa9 d0/d9/de/d14/d33/fb5 0 2024-03-04T13:43:15.913 INFO:tasks.workunit.client.0.smithi107.stdout:5/665: creat d5/d8/d14/d2d/d92/dc1/feb x:0 0 0 2024-03-04T13:43:15.916 INFO:tasks.workunit.client.0.smithi107.stdout:1/651: dread d1/d46/d6a/d75/f66 [0,4194304] 0 2024-03-04T13:43:15.917 INFO:tasks.workunit.client.0.smithi107.stdout:5/666: symlink d5/dc/d22/lec 0 2024-03-04T13:43:15.925 INFO:tasks.workunit.client.0.smithi107.stdout:6/669: dread df/d2e/d36/fa2 [0,4194304] 0 2024-03-04T13:43:15.926 INFO:tasks.workunit.client.0.smithi107.stdout:1/652: mkdir d1/d10/d16/d47/d7a/dda 0 2024-03-04T13:43:15.930 INFO:tasks.workunit.client.0.smithi107.stdout:0/687: dwrite da/dc/dd/d3e/d26/d88/dd2/f9d [0,4194304] 0 2024-03-04T13:43:15.932 INFO:tasks.workunit.client.0.smithi107.stdout:6/670: rmdir df/d57 39 2024-03-04T13:43:15.939 INFO:tasks.workunit.client.0.smithi107.stdout:9/637: dwrite d4/da/d11/d20/d23/f6c [0,4194304] 0 2024-03-04T13:43:15.942 INFO:tasks.workunit.client.0.smithi107.stdout:1/653: mkdir d1/d46/d6a/d4e/daa/dc9/d33/d43/d5b/d61/ddb 0 2024-03-04T13:43:15.946 INFO:tasks.workunit.client.0.smithi107.stdout:7/644: dwrite d6/d13/d16/d1c/f48 [0,4194304] 0 2024-03-04T13:43:15.951 INFO:tasks.workunit.client.0.smithi107.stdout:6/671: unlink df/d17/d5c/db1/c33 0 2024-03-04T13:43:15.954 INFO:tasks.workunit.client.0.smithi107.stdout:5/667: truncate d5/dc/f3d 1309754 0 2024-03-04T13:43:15.954 INFO:tasks.workunit.client.0.smithi107.stdout:2/605: dwrite da/df/d73/d7a/d9a/fba [0,4194304] 0 2024-03-04T13:43:15.955 INFO:tasks.workunit.client.0.smithi107.stdout:3/623: dwrite d1/de/d18/d33/d48/d71/f72 [0,4194304] 0 2024-03-04T13:43:15.959 INFO:tasks.workunit.client.0.smithi107.stdout:3/624: chown d1/f2b 2225081 1 2024-03-04T13:43:15.963 INFO:tasks.workunit.client.0.smithi107.stdout:3/625: readlink d1/d41/l6a 0 2024-03-04T13:43:15.963 INFO:tasks.workunit.client.0.smithi107.stdout:2/606: rmdir da/d49/d5c 39 2024-03-04T13:43:15.969 INFO:tasks.workunit.client.0.smithi107.stdout:7/645: fsync d6/d1d/d64/d9a/da9/d71/db9/dca/f42 0 2024-03-04T13:43:15.974 INFO:tasks.workunit.client.0.smithi107.stdout:6/672: rename fb to df/d17/d6b/d9b/fd7 0 2024-03-04T13:43:15.988 INFO:tasks.workunit.client.0.smithi107.stdout:7/646: chown d6/d13/d16/d19/d38/f47 431503870 1 2024-03-04T13:43:15.989 INFO:tasks.workunit.client.0.smithi107.stdout:9/638: link d4/da/d11/d4c/d89/l95 d4/da/d11/d20/d64/ld6 0 2024-03-04T13:43:15.991 INFO:tasks.workunit.client.0.smithi107.stdout:1/654: write d1/d46/d6a/d17/f5a [31864,20299] 0 2024-03-04T13:43:15.993 INFO:tasks.workunit.client.0.smithi107.stdout:7/647: dread - d6/d13/d16/d19/d38/d2c/fac zero size 2024-03-04T13:43:15.994 INFO:tasks.workunit.client.0.smithi107.stdout:0/688: dread da/dc/d34/f5a [4194304,4194304] 0 2024-03-04T13:43:15.995 INFO:tasks.workunit.client.0.smithi107.stdout:2/607: dread da/d39/f46 [4194304,4194304] 0 2024-03-04T13:43:15.996 INFO:tasks.workunit.client.0.smithi107.stdout:0/689: chown da/dc/dd/d3e/d26 96605273 1 2024-03-04T13:43:15.997 INFO:tasks.workunit.client.0.smithi107.stdout:6/673: dread df/fac [0,4194304] 0 2024-03-04T13:43:16.045 INFO:tasks.workunit.client.0.smithi107.stdout:6/674: fsync df/d2e/d36/d77/f84 0 2024-03-04T13:43:16.047 INFO:tasks.workunit.client.0.smithi107.stdout:5/668: rename d5/d8/d14/f4c to d5/fed 0 2024-03-04T13:43:16.047 INFO:tasks.workunit.client.0.smithi107.stdout:3/626: dwrite d1/de/d11/f3e [0,4194304] 0 2024-03-04T13:43:16.048 INFO:tasks.workunit.client.0.smithi107.stdout:7/648: mkdir d6/d1d/d64/d9a/da9/d71/db9/dca/d3d/dc6/dcb/dd6 0 2024-03-04T13:43:16.049 INFO:tasks.workunit.client.0.smithi107.stdout:1/655: mknod d1/d46/d6a/cdc 0 2024-03-04T13:43:16.054 INFO:tasks.workunit.client.0.smithi107.stdout:3/627: unlink d1/de/d18/d33/l9c 0 2024-03-04T13:43:16.055 INFO:tasks.workunit.client.0.smithi107.stdout:6/675: mkdir df/d17/d24/d30/d3b/dd8 0 2024-03-04T13:43:16.055 INFO:tasks.workunit.client.0.smithi107.stdout:1/656: unlink d1/d46/d6a/d75/f66 0 2024-03-04T13:43:16.058 INFO:tasks.workunit.client.0.smithi107.stdout:0/690: dread da/dc/d34/f45 [4194304,4194304] 0 2024-03-04T13:43:16.061 INFO:tasks.workunit.client.0.smithi107.stdout:1/657: unlink d1/d46/d6a/d65/l7e 0 2024-03-04T13:43:16.065 INFO:tasks.workunit.client.0.smithi107.stdout:1/658: stat d1/d46/d6a/d4e/daa/ld1 0 2024-03-04T13:43:16.066 INFO:tasks.workunit.client.0.smithi107.stdout:0/691: unlink da/dc/dd/d3e/d26/d88/dd2/d73/f9a 0 2024-03-04T13:43:16.067 INFO:tasks.workunit.client.0.smithi107.stdout:5/669: dread d5/f20 [0,4194304] 0 2024-03-04T13:43:16.067 INFO:tasks.workunit.client.0.smithi107.stdout:0/692: fdatasync da/dad/fae 0 2024-03-04T13:43:16.067 INFO:tasks.workunit.client.0.smithi107.stdout:7/649: dwrite d6/d13/d16/d1c/f48 [0,4194304] 0 2024-03-04T13:43:16.070 INFO:tasks.workunit.client.0.smithi107.stdout:5/670: chown d5/dc/d22/f26 33030737 1 2024-03-04T13:43:16.070 INFO:tasks.workunit.client.0.smithi107.stdout:7/650: dread - d6/d13/dbc/fcd zero size 2024-03-04T13:43:16.083 INFO:tasks.workunit.client.0.smithi107.stdout:1/659: dread d1/d46/d6a/d4e/daa/dc9/d33/d5d/f97 [4194304,4194304] 0 2024-03-04T13:43:16.086 INFO:tasks.workunit.client.0.smithi107.stdout:5/671: truncate d5/f20 229768 0 2024-03-04T13:43:16.091 INFO:tasks.workunit.client.0.smithi107.stdout:0/693: creat da/dc/dd/d3e/d26/d69/de9/fed x:0 0 0 2024-03-04T13:43:16.098 INFO:tasks.workunit.client.0.smithi107.stdout:3/628: dwrite d1/de/d18/d33/d48/d71/f49 [0,4194304] 0 2024-03-04T13:43:16.101 INFO:tasks.workunit.client.0.smithi107.stdout:3/629: read d1/de/d18/d33/d48/d71/f72 [3781220,12939] 0 2024-03-04T13:43:16.101 INFO:tasks.workunit.client.0.smithi107.stdout:5/672: mknod d5/d2b/d4b/da6/cee 0 2024-03-04T13:43:16.105 INFO:tasks.workunit.client.0.smithi107.stdout:5/673: read - d5/dc/d22/fc2 zero size 2024-03-04T13:43:16.109 INFO:tasks.workunit.client.0.smithi107.stdout:7/651: getdents d6 0 2024-03-04T13:43:16.114 INFO:tasks.workunit.client.0.smithi107.stdout:1/660: dwrite d1/d38/fb6 [0,4194304] 0 2024-03-04T13:43:16.121 INFO:tasks.workunit.client.0.smithi107.stdout:1/661: fsync d1/d46/d6a/d4e/daa/dc9/d33/d5d/f97 0 2024-03-04T13:43:16.123 INFO:tasks.workunit.client.0.smithi107.stdout:3/630: link d1/f14 d1/de/d18/d33/d48/d71/d4c/daa/dab/dc5/fc6 0 2024-03-04T13:43:16.128 INFO:tasks.workunit.client.0.smithi107.stdout:1/662: rmdir d1/dd6 39 2024-03-04T13:43:16.129 INFO:tasks.workunit.client.0.smithi107.stdout:5/674: dwrite d5/dc/dc6/fcb [0,4194304] 0 2024-03-04T13:43:16.130 INFO:tasks.workunit.client.0.smithi107.stdout:7/652: link d6/d13/d16/d19/d38/d37/f9e d6/d1d/d64/d9a/da9/d71/db9/dca/d22/fd7 0 2024-03-04T13:43:16.131 INFO:tasks.workunit.client.0.smithi107.stdout:3/631: truncate d1/de/d18/d33/d48/d71/d4c/d54/f79 3705669 0 2024-03-04T13:43:16.137 INFO:tasks.workunit.client.0.smithi107.stdout:3/632: mknod d1/d2d/cc7 0 2024-03-04T13:43:16.139 INFO:tasks.workunit.client.0.smithi107.stdout:1/663: rename d1/d46/d6a/d4e/daa/dc9/d33/d5d/d24/c7b to d1/d46/d6a/d4e/cdd 0 2024-03-04T13:43:16.186 INFO:tasks.workunit.client.0.smithi107.stdout:3/633: mkdir d1/de/d18/d33/d48/dc8 0 2024-03-04T13:43:16.187 INFO:tasks.workunit.client.0.smithi107.stdout:1/664: write d1/d10/d16/d47/f77 [989414,31721] 0 2024-03-04T13:43:16.189 INFO:tasks.workunit.client.0.smithi107.stdout:0/694: dread da/dc/dd/f24 [0,4194304] 0 2024-03-04T13:43:16.189 INFO:tasks.workunit.client.0.smithi107.stdout:7/653: rename d6/d13/d14/f82 to d6/d1d/d64/d9a/da9/d71/db9/fd8 0 2024-03-04T13:43:16.192 INFO:tasks.workunit.client.0.smithi107.stdout:0/695: dread - da/dc/dd/fe7 zero size 2024-03-04T13:43:16.192 INFO:tasks.workunit.client.0.smithi107.stdout:1/665: chown d1/d46/d6a/d4e/cca 412840 1 2024-03-04T13:43:16.193 INFO:tasks.workunit.client.0.smithi107.stdout:3/634: unlink d1/de/d18/ca3 0 2024-03-04T13:43:16.195 INFO:tasks.workunit.client.0.smithi107.stdout:5/675: dwrite d5/dc/f58 [0,4194304] 0 2024-03-04T13:43:16.198 INFO:tasks.workunit.client.0.smithi107.stdout:5/676: write d5/dc/f55 [4722829,11827] 0 2024-03-04T13:43:16.198 INFO:tasks.workunit.client.0.smithi107.stdout:0/696: dread da/dc/dd/d3e/d26/d88/dd2/fbd [0,4194304] 0 2024-03-04T13:43:16.200 INFO:tasks.workunit.client.0.smithi107.stdout:3/635: chown d1/de/d18/d33/f8f 19421 1 2024-03-04T13:43:16.200 INFO:tasks.workunit.client.0.smithi107.stdout:0/697: fsync da/dc/dd/d3e/d26/d69/d6b/fd3 0 2024-03-04T13:43:16.203 INFO:tasks.workunit.client.0.smithi107.stdout:7/654: mkdir d6/d1d/dd9 0 2024-03-04T13:43:16.203 INFO:tasks.workunit.client.0.smithi107.stdout:3/636: truncate d1/de/d3f/f89 989750 0 2024-03-04T13:43:16.204 INFO:tasks.workunit.client.0.smithi107.stdout:1/666: fsync d1/d46/d6a/d4e/daa/dc9/d33/d5d/d4c/f93 0 2024-03-04T13:43:16.206 INFO:tasks.workunit.client.0.smithi107.stdout:5/677: rename d5/d66/f86 to d5/d79/fef 0 2024-03-04T13:43:16.207 INFO:tasks.workunit.client.0.smithi107.stdout:3/637: write d1/de/d11/f46 [4337694,94759] 0 2024-03-04T13:43:16.208 INFO:tasks.workunit.client.0.smithi107.stdout:0/698: dread - da/dc/dd/d3e/d26/d69/d6b/f97 zero size 2024-03-04T13:43:16.215 INFO:tasks.workunit.client.0.smithi107.stdout:7/655: creat d6/d1d/d64/d9a/da9/fda x:0 0 0 2024-03-04T13:43:16.216 INFO:tasks.workunit.client.0.smithi107.stdout:5/678: readlink d5/d8/d2a/ld5 0 2024-03-04T13:43:16.219 INFO:tasks.workunit.client.0.smithi107.stdout:7/656: write d6/d13/d16/d19/d38/d2c/f87 [287974,29181] 0 2024-03-04T13:43:16.219 INFO:tasks.workunit.client.0.smithi107.stdout:1/667: fdatasync d1/f31 0 2024-03-04T13:43:16.220 INFO:tasks.workunit.client.0.smithi107.stdout:3/638: mknod d1/de/d18/d7e/cc9 0 2024-03-04T13:43:16.221 INFO:tasks.workunit.client.0.smithi107.stdout:5/679: symlink d5/dc/d22/lf0 0 2024-03-04T13:43:16.224 INFO:tasks.workunit.client.0.smithi107.stdout:0/699: creat da/dc/dd/d3e/fee x:0 0 0 2024-03-04T13:43:16.225 INFO:tasks.workunit.client.0.smithi107.stdout:1/668: read d1/d46/d6a/d4e/daa/dc9/d33/d43/f55 [3419239,125906] 0 2024-03-04T13:43:16.227 INFO:tasks.workunit.client.0.smithi107.stdout:3/639: rename d1/de/d18/d33/d48/d71/f67 to d1/d2d/fca 0 2024-03-04T13:43:16.235 INFO:tasks.workunit.client.0.smithi107.stdout:5/680: dwrite d5/f32 [4194304,4194304] 0 2024-03-04T13:43:16.241 INFO:tasks.workunit.client.0.smithi107.stdout:3/640: truncate d1/d2d/fca 41953 0 2024-03-04T13:43:16.243 INFO:tasks.workunit.client.0.smithi107.stdout:0/700: write f0 [350387,117461] 0 2024-03-04T13:43:16.253 INFO:tasks.workunit.client.0.smithi107.stdout:3/641: rename d1/de/d18/d33/d48/d71/d4c/d54/f9d to d1/d1e/daf/fcb 0 2024-03-04T13:43:16.254 INFO:tasks.workunit.client.0.smithi107.stdout:5/681: dwrite d5/dc/d22/f7f [0,4194304] 0 2024-03-04T13:43:16.258 INFO:tasks.workunit.client.0.smithi107.stdout:5/682: chown d5/dc/l18 26577 1 2024-03-04T13:43:16.259 INFO:tasks.workunit.client.0.smithi107.stdout:1/669: dread d1/d10/d16/fad [0,4194304] 0 2024-03-04T13:43:16.261 INFO:tasks.workunit.client.0.smithi107.stdout:5/683: write d5/dc/fc4 [69316,30265] 0 2024-03-04T13:43:16.268 INFO:tasks.workunit.client.0.smithi107.stdout:0/701: dwrite da/d7e/f7f [0,4194304] 0 2024-03-04T13:43:16.271 INFO:tasks.workunit.client.0.smithi107.stdout:1/670: mkdir d1/d46/d6a/d4e/d60/d83/dde 0 2024-03-04T13:43:16.273 INFO:tasks.workunit.client.0.smithi107.stdout:5/684: rename d5/d8/d14/d54 to d5/d8/d14/d2d/df1 0 2024-03-04T13:43:16.277 INFO:tasks.workunit.client.0.smithi107.stdout:1/671: fdatasync d1/d10/d16/d1b/fd2 0 2024-03-04T13:43:16.279 INFO:tasks.workunit.client.0.smithi107.stdout:5/685: creat d5/d8/da/db0/de6/ff2 x:0 0 0 2024-03-04T13:43:16.282 INFO:tasks.workunit.client.0.smithi107.stdout:5/686: read - d5/d8/d14/d2d/d92/fca zero size 2024-03-04T13:43:16.285 INFO:tasks.workunit.client.0.smithi107.stdout:1/672: rename d1/d46/d6a/d17/l4a to d1/d46/d6a/d17/ldf 0 2024-03-04T13:43:16.290 INFO:tasks.workunit.client.0.smithi107.stdout:3/642: dread d1/de/d18/d33/d48/d71/d4c/d54/f79 [0,4194304] 0 2024-03-04T13:43:16.292 INFO:tasks.workunit.client.0.smithi107.stdout:3/643: readlink d1/de/l53 0 2024-03-04T13:43:16.294 INFO:tasks.workunit.client.0.smithi107.stdout:0/702: dwrite da/dc/dd/dc9/de5/fc1 [0,4194304] 0 2024-03-04T13:43:16.308 INFO:tasks.workunit.client.0.smithi107.stdout:5/687: dwrite d5/d8/f11 [4194304,4194304] 0 2024-03-04T13:43:16.311 INFO:tasks.workunit.client.0.smithi107.stdout:3/644: dwrite d1/de/d11/f46 [0,4194304] 0 2024-03-04T13:43:16.316 INFO:tasks.workunit.client.0.smithi107.stdout:5/688: mknod d5/d2b/d4b/cf3 0 2024-03-04T13:43:16.321 INFO:tasks.workunit.client.0.smithi107.stdout:4/600: sync 2024-03-04T13:43:16.322 INFO:tasks.workunit.client.0.smithi107.stdout:8/676: sync 2024-03-04T13:43:16.325 INFO:tasks.workunit.client.0.smithi107.stdout:3/645: chown d1/de/d18/d33/d4d/d4f/d69/l77 32573 1 2024-03-04T13:43:16.328 INFO:tasks.workunit.client.0.smithi107.stdout:4/601: mknod d0/d2f/d5f/cb6 0 2024-03-04T13:43:16.331 INFO:tasks.workunit.client.0.smithi107.stdout:8/677: creat d6/da/d25/dd1/fd2 x:0 0 0 2024-03-04T13:43:16.336 INFO:tasks.workunit.client.0.smithi107.stdout:4/602: link d0/d9/d18/d29/f55 d0/d9/de/d14/dc/d9a/db3/fb7 0 2024-03-04T13:43:16.339 INFO:tasks.workunit.client.0.smithi107.stdout:4/603: write d0/d9/de/d14/dc/d21/f6c [1383837,95644] 0 2024-03-04T13:43:16.350 INFO:tasks.workunit.client.0.smithi107.stdout:8/678: dwrite d6/da/d25/d2a/f95 [0,4194304] 0 2024-03-04T13:43:16.351 INFO:tasks.workunit.client.0.smithi107.stdout:5/689: dread d5/dc/f3d [0,4194304] 0 2024-03-04T13:43:16.353 INFO:tasks.workunit.client.0.smithi107.stdout:4/604: rename d0/d9/de/d14/dc to d0/d9/d10/d80/db8 0 2024-03-04T13:43:16.355 INFO:tasks.workunit.client.0.smithi107.stdout:5/690: creat d5/dc/dc6/ff4 x:0 0 0 2024-03-04T13:43:16.356 INFO:tasks.workunit.client.0.smithi107.stdout:8/679: readlink d6/l7 0 2024-03-04T13:43:16.357 INFO:tasks.workunit.client.0.smithi107.stdout:4/605: mknod d0/d9/d1e/d70/cb9 0 2024-03-04T13:43:16.357 INFO:tasks.workunit.client.0.smithi107.stdout:5/691: fsync d5/f6 0 2024-03-04T13:43:16.359 INFO:tasks.workunit.client.0.smithi107.stdout:8/680: creat d6/d16/d1f/dbd/fd3 x:0 0 0 2024-03-04T13:43:16.361 INFO:tasks.workunit.client.0.smithi107.stdout:5/692: mknod d5/d8/d14/d2d/d92/cf5 0 2024-03-04T13:43:16.364 INFO:tasks.workunit.client.0.smithi107.stdout:5/693: chown d5/dc/df/f12 1580594 1 2024-03-04T13:43:16.367 INFO:tasks.workunit.client.0.smithi107.stdout:4/606: rename d0/d9/d10/d80/db8/d97/laf to d0/d9/d10/lba 0 2024-03-04T13:43:16.369 INFO:tasks.workunit.client.0.smithi107.stdout:5/694: creat d5/d79/d89/da3/d68/ff6 x:0 0 0 2024-03-04T13:43:16.374 INFO:tasks.workunit.client.0.smithi107.stdout:5/695: read d5/d79/d89/da3/d6a/f6d [2255201,109855] 0 2024-03-04T13:43:16.376 INFO:tasks.workunit.client.0.smithi107.stdout:5/696: chown d5/d8/d14/d2d/df1/f93 562 1 2024-03-04T13:43:16.378 INFO:tasks.workunit.client.0.smithi107.stdout:5/697: fdatasync d5/d8/d14/d2d/d92/fca 0 2024-03-04T13:43:16.383 INFO:tasks.workunit.client.0.smithi107.stdout:5/698: link d5/d79/d89/da3/d69/fd3 d5/d2b/ff7 0 2024-03-04T13:43:16.384 INFO:tasks.workunit.client.0.smithi107.stdout:8/681: dread d6/d16/d1f/f3a [0,4194304] 0 2024-03-04T13:43:16.387 INFO:tasks.workunit.client.0.smithi107.stdout:5/699: rename d5/d79/d89/ca0 to d5/d79/d89/da3/d68/cf8 0 2024-03-04T13:43:16.399 INFO:tasks.workunit.client.0.smithi107.stdout:4/607: dread d0/d9/d10/f4a [0,4194304] 0 2024-03-04T13:43:16.402 INFO:tasks.workunit.client.0.smithi107.stdout:5/700: write d5/dc/f41 [3047769,125957] 0 2024-03-04T13:43:16.405 INFO:tasks.workunit.client.0.smithi107.stdout:5/701: write d5/d8/d2a/d43/f57 [522024,95376] 0 2024-03-04T13:43:16.414 INFO:tasks.workunit.client.0.smithi107.stdout:4/608: dwrite d0/d9/d10/d80/db8/f3c [0,4194304] 0 2024-03-04T13:43:16.421 INFO:tasks.workunit.client.0.smithi107.stdout:4/609: rename d0/d9/de/d14/f65 to d0/d9/d78/fbb 0 2024-03-04T13:43:16.423 INFO:tasks.workunit.client.0.smithi107.stdout:5/702: dwrite d5/d79/d89/da3/d6a/f6d [0,4194304] 0 2024-03-04T13:43:16.435 INFO:tasks.workunit.client.0.smithi107.stdout:4/610: dread d0/d30/f50 [0,4194304] 0 2024-03-04T13:43:16.438 INFO:tasks.workunit.client.0.smithi107.stdout:4/611: read d0/d9/de/d14/f4e [3678995,39966] 0 2024-03-04T13:43:16.442 INFO:tasks.workunit.client.0.smithi107.stdout:4/612: write d0/d9/d10/d80/db8/d21/f6c [2439425,113123] 0 2024-03-04T13:43:16.444 INFO:tasks.workunit.client.0.smithi107.stdout:4/613: chown d0/d9/de/d14/f4b 1319247 1 2024-03-04T13:43:16.445 INFO:tasks.workunit.client.0.smithi107.stdout:4/614: rename d0 to d0/d9/d1e/dbc 22 2024-03-04T13:43:16.452 INFO:tasks.workunit.client.0.smithi107.stdout:5/703: dwrite d5/dc/f3b [0,4194304] 0 2024-03-04T13:43:16.458 INFO:tasks.workunit.client.0.smithi107.stdout:5/704: mknod d5/d79/d89/da3/d69/cf9 0 2024-03-04T13:43:16.461 INFO:tasks.workunit.client.0.smithi107.stdout:5/705: readlink d5/d8/d14/l1a 0 2024-03-04T13:43:16.465 INFO:tasks.workunit.client.0.smithi107.stdout:5/706: mkdir d5/d8/d14/d2d/dfa 0 2024-03-04T13:43:16.467 INFO:tasks.workunit.client.0.smithi107.stdout:4/615: dwrite d0/d9/de/d14/d66/f6a [4194304,4194304] 0 2024-03-04T13:43:16.468 INFO:tasks.workunit.client.0.smithi107.stdout:4/616: stat d0/l44 0 2024-03-04T13:43:16.471 INFO:tasks.workunit.client.0.smithi107.stdout:4/617: write d0/d9/de/f62 [2627732,126796] 0 2024-03-04T13:43:16.482 INFO:tasks.workunit.client.0.smithi107.stdout:2/608: sync 2024-03-04T13:43:16.482 INFO:tasks.workunit.client.0.smithi107.stdout:9/639: sync 2024-03-04T13:43:16.483 INFO:tasks.workunit.client.0.smithi107.stdout:5/707: dwrite d5/d8/d2a/f39 [0,4194304] 0 2024-03-04T13:43:16.487 INFO:tasks.workunit.client.0.smithi107.stdout:5/708: fsync d5/dc/df/f2e 0 2024-03-04T13:43:16.491 INFO:tasks.workunit.client.0.smithi107.stdout:2/609: link da/d49/l66 da/df/d3e/lcd 0 2024-03-04T13:43:16.498 INFO:tasks.workunit.client.0.smithi107.stdout:9/640: dwrite d4/f8 [4194304,4194304] 0 2024-03-04T13:43:16.498 INFO:tasks.workunit.client.0.smithi107.stdout:2/610: symlink da/df/d27/d58/d5f/lce 0 2024-03-04T13:43:16.501 INFO:tasks.workunit.client.0.smithi107.stdout:5/709: link d5/d66/fa7 d5/d8/d14/ffb 0 2024-03-04T13:43:16.503 INFO:tasks.workunit.client.0.smithi107.stdout:2/611: chown da/d49/d5c/f67 3918 1 2024-03-04T13:43:16.555 INFO:tasks.workunit.client.0.smithi107.stdout:5/710: creat d5/d8/d2a/dd6/ffc x:0 0 0 2024-03-04T13:43:16.557 INFO:tasks.workunit.client.0.smithi107.stdout:9/641: getdents d4/da/d11/d20/d23/d7e 0 2024-03-04T13:43:16.561 INFO:tasks.workunit.client.0.smithi107.stdout:9/642: stat d4/da/d11/d4c/f3c 0 2024-03-04T13:43:16.563 INFO:tasks.workunit.client.0.smithi107.stdout:2/612: dread da/d38/d3a/f60 [0,4194304] 0 2024-03-04T13:43:16.563 INFO:tasks.workunit.client.0.smithi107.stdout:5/711: symlink d5/d2b/d4b/da6/lfd 0 2024-03-04T13:43:16.565 INFO:tasks.workunit.client.0.smithi107.stdout:9/643: stat d4/da/d11/d4c/d4d/c9e 0 2024-03-04T13:43:16.565 INFO:tasks.workunit.client.0.smithi107.stdout:2/613: chown da 6645045 1 2024-03-04T13:43:16.569 INFO:tasks.workunit.client.0.smithi107.stdout:5/712: symlink d5/dc/de4/lfe 0 2024-03-04T13:43:16.569 INFO:tasks.workunit.client.0.smithi107.stdout:9/644: dread - d4/da/d11/d20/d23/d7e/d84/fcd zero size 2024-03-04T13:43:16.572 INFO:tasks.workunit.client.0.smithi107.stdout:2/614: dread da/d34/fc8 [0,4194304] 0 2024-03-04T13:43:16.575 INFO:tasks.workunit.client.0.smithi107.stdout:5/713: mknod d5/d8/d2a/d43/cff 0 2024-03-04T13:43:16.580 INFO:tasks.workunit.client.0.smithi107.stdout:9/645: rmdir d4/da/d11/d4c/d51 39 2024-03-04T13:43:16.587 INFO:tasks.workunit.client.0.smithi107.stdout:9/646: symlink d4/da/d11/d20/ld7 0 2024-03-04T13:43:16.592 INFO:tasks.workunit.client.0.smithi107.stdout:9/647: dread - d4/da/d11/d20/d64/d77/f82 zero size 2024-03-04T13:43:16.596 INFO:tasks.workunit.client.0.smithi107.stdout:5/714: dwrite d5/d2b/f88 [0,4194304] 0 2024-03-04T13:43:16.605 INFO:tasks.workunit.client.0.smithi107.stdout:6/676: sync 2024-03-04T13:43:16.608 INFO:tasks.workunit.client.0.smithi107.stdout:6/677: write df/d17/d24/d30/fc4 [732719,25420] 0 2024-03-04T13:43:16.615 INFO:tasks.workunit.client.0.smithi107.stdout:9/648: link d4/da/d11/d4c/d51/la1 d4/da/d11/d4c/d51/ld8 0 2024-03-04T13:43:16.616 INFO:tasks.workunit.client.0.smithi107.stdout:6/678: chown df/d2e/d36/d77/d95/da7/fcb 1137367 1 2024-03-04T13:43:16.618 INFO:tasks.workunit.client.0.smithi107.stdout:6/679: chown df/d17/d6b/c75 5 1 2024-03-04T13:43:16.623 INFO:tasks.workunit.client.0.smithi107.stdout:6/680: unlink df/d2e/d36/c59 0 2024-03-04T13:43:16.624 INFO:tasks.workunit.client.0.smithi107.stdout:6/681: stat df/d17/d24/d83/fc2 0 2024-03-04T13:43:16.626 INFO:tasks.workunit.client.0.smithi107.stdout:6/682: fsync f3 0 2024-03-04T13:43:16.637 INFO:tasks.workunit.client.0.smithi107.stdout:6/683: link df/d38/l82 df/d17/d24/d40/d63/da3/d7e/d86/ld9 0 2024-03-04T13:43:16.637 INFO:tasks.workunit.client.0.smithi107.stdout:6/684: readlink df/l22 0 2024-03-04T13:43:16.639 INFO:tasks.workunit.client.0.smithi107.stdout:5/715: dread d5/dc/df/fc9 [0,4194304] 0 2024-03-04T13:43:16.641 INFO:tasks.workunit.client.0.smithi107.stdout:9/649: dread d4/da/d11/d65/f58 [0,4194304] 0 2024-03-04T13:43:16.643 INFO:tasks.workunit.client.0.smithi107.stdout:5/716: symlink d5/dc/df/l100 0 2024-03-04T13:43:16.652 INFO:tasks.workunit.client.0.smithi107.stdout:2/615: dwrite da/f30 [0,4194304] 0 2024-03-04T13:43:16.680 INFO:tasks.workunit.client.0.smithi107.stdout:2/616: fsync da/df/d85/f89 0 2024-03-04T13:43:16.683 INFO:tasks.workunit.client.0.smithi107.stdout:6/685: dread df/d17/d24/f2d [0,4194304] 0 2024-03-04T13:43:16.683 INFO:tasks.workunit.client.0.smithi107.stdout:2/617: creat da/df/d3e/fcf x:0 0 0 2024-03-04T13:43:16.686 INFO:tasks.workunit.client.0.smithi107.stdout:6/686: read df/d2e/f5a [3522925,13210] 0 2024-03-04T13:43:16.689 INFO:tasks.workunit.client.0.smithi107.stdout:2/618: write da/f1c [478222,10893] 0 2024-03-04T13:43:16.690 INFO:tasks.workunit.client.0.smithi107.stdout:5/717: dread d5/d8/d2a/d43/f4a [0,4194304] 0 2024-03-04T13:43:16.690 INFO:tasks.workunit.client.0.smithi107.stdout:2/619: readlink da/d39/lca 0 2024-03-04T13:43:16.690 INFO:tasks.workunit.client.0.smithi107.stdout:6/687: mknod df/d2e/d36/d5b/d7d/cda 0 2024-03-04T13:43:16.694 INFO:tasks.workunit.client.0.smithi107.stdout:6/688: creat df/d17/d24/fdb x:0 0 0 2024-03-04T13:43:16.698 INFO:tasks.workunit.client.0.smithi107.stdout:6/689: fsync df/d57/f7c 0 2024-03-04T13:43:16.703 INFO:tasks.workunit.client.0.smithi107.stdout:6/690: symlink df/ldc 0 2024-03-04T13:43:16.706 INFO:tasks.workunit.client.0.smithi107.stdout:6/691: read df/d1b/f1d [3676424,86404] 0 2024-03-04T13:43:16.706 INFO:tasks.workunit.client.0.smithi107.stdout:2/620: dwrite da/df/f15 [0,4194304] 0 2024-03-04T13:43:16.709 INFO:tasks.workunit.client.0.smithi107.stdout:2/621: chown da/d38/d3a/fae 89548912 1 2024-03-04T13:43:16.711 INFO:tasks.workunit.client.0.smithi107.stdout:5/718: dread d5/d79/d89/da3/d6a/f8e [0,4194304] 0 2024-03-04T13:43:16.713 INFO:tasks.workunit.client.0.smithi107.stdout:2/622: rename da/d6c to da/d44/d8b/d9d/dd0 0 2024-03-04T13:43:16.715 INFO:tasks.workunit.client.0.smithi107.stdout:5/719: chown d5/d79/d89/da3/d68/ff6 688 1 2024-03-04T13:43:16.716 INFO:tasks.workunit.client.0.smithi107.stdout:5/720: dread - d5/d79/d89/da3/d68/ff6 zero size 2024-03-04T13:43:16.718 INFO:tasks.workunit.client.0.smithi107.stdout:2/623: mkdir da/df/d73/dd1 0 2024-03-04T13:43:16.719 INFO:tasks.workunit.client.0.smithi107.stdout:5/721: creat d5/d8/d2a/d43/f101 x:0 0 0 2024-03-04T13:43:16.732 INFO:tasks.workunit.client.0.smithi107.stdout:5/722: link d5/d66/d96/lb5 d5/d8/d14/d2d/d92/dd8/l102 0 2024-03-04T13:43:16.735 INFO:tasks.workunit.client.0.smithi107.stdout:5/723: read - d5/dc/dc6/ff4 zero size 2024-03-04T13:43:16.736 INFO:tasks.workunit.client.0.smithi107.stdout:6/692: dwrite df/d17/d24/d83/f92 [0,4194304] 0 2024-03-04T13:43:16.737 INFO:tasks.workunit.client.0.smithi107.stdout:2/624: dwrite da/df/fa2 [4194304,4194304] 0 2024-03-04T13:43:16.738 INFO:tasks.workunit.client.0.smithi107.stdout:2/625: stat da/f30 0 2024-03-04T13:43:16.739 INFO:tasks.workunit.client.0.smithi107.stdout:5/724: chown d5/d8/d2a/f48 50 1 2024-03-04T13:43:16.744 INFO:tasks.workunit.client.0.smithi107.stdout:2/626: symlink da/d38/d3a/d5d/ld2 0 2024-03-04T13:43:16.746 INFO:tasks.workunit.client.0.smithi107.stdout:6/693: fsync df/d2e/d36/d71/fca 0 2024-03-04T13:43:16.750 INFO:tasks.workunit.client.0.smithi107.stdout:6/694: write df/d1b/f1d [220730,13006] 0 2024-03-04T13:43:16.751 INFO:tasks.workunit.client.0.smithi107.stdout:5/725: getdents d5/d8/d14/d2d/df1 0 2024-03-04T13:43:16.754 INFO:tasks.workunit.client.0.smithi107.stdout:4/618: sync 2024-03-04T13:43:16.758 INFO:tasks.workunit.client.0.smithi107.stdout:7/657: sync 2024-03-04T13:43:16.758 INFO:tasks.workunit.client.0.smithi107.stdout:8/682: sync 2024-03-04T13:43:16.758 INFO:tasks.workunit.client.0.smithi107.stdout:3/646: sync 2024-03-04T13:43:16.758 INFO:tasks.workunit.client.0.smithi107.stdout:0/703: sync 2024-03-04T13:43:16.758 INFO:tasks.workunit.client.0.smithi107.stdout:1/673: sync 2024-03-04T13:43:16.758 INFO:tasks.workunit.client.0.smithi107.stdout:4/619: unlink d0/d9/d10/d80/db8/f84 0 2024-03-04T13:43:16.760 INFO:tasks.workunit.client.0.smithi107.stdout:3/647: write d1/d1e/f22 [278563,112425] 0 2024-03-04T13:43:16.761 INFO:tasks.workunit.client.0.smithi107.stdout:1/674: mknod d1/d10/ce0 0 2024-03-04T13:43:16.764 INFO:tasks.workunit.client.0.smithi107.stdout:7/658: mkdir d6/d13/d16/d19/d38/d62/ddb 0 2024-03-04T13:43:16.766 INFO:tasks.workunit.client.0.smithi107.stdout:8/683: unlink d6/d16/d1f/d63/d73/fac 0 2024-03-04T13:43:16.768 INFO:tasks.workunit.client.0.smithi107.stdout:4/620: creat d0/d9/d10/d80/d81/fbd x:0 0 0 2024-03-04T13:43:16.771 INFO:tasks.workunit.client.0.smithi107.stdout:1/675: symlink d1/d46/d6a/d65/le1 0 2024-03-04T13:43:16.771 INFO:tasks.workunit.client.0.smithi107.stdout:3/648: truncate d1/de/d11/f95 735076 0 2024-03-04T13:43:16.772 INFO:tasks.workunit.client.0.smithi107.stdout:5/726: dwrite d5/dc/de4/fc3 [0,4194304] 0 2024-03-04T13:43:16.774 INFO:tasks.workunit.client.0.smithi107.stdout:1/676: chown d1/d10/d16/l21 2349 1 2024-03-04T13:43:16.777 INFO:tasks.workunit.client.0.smithi107.stdout:3/649: dread - d1/d41/d81/fb5 zero size 2024-03-04T13:43:16.779 INFO:tasks.workunit.client.0.smithi107.stdout:2/627: dwrite da/f26 [0,4194304] 0 2024-03-04T13:43:16.782 INFO:tasks.workunit.client.0.smithi107.stdout:1/677: rmdir d1/d46/d6a/d4e/daa/dc9/d33/d89 39 2024-03-04T13:43:16.783 INFO:tasks.workunit.client.0.smithi107.stdout:2/628: dread - da/df/d73/daa/fb2 zero size 2024-03-04T13:43:16.785 INFO:tasks.workunit.client.0.smithi107.stdout:1/678: chown d1/d46/d6a/d4e/daa/dc9/d33/d43 6442850 1 2024-03-04T13:43:16.787 INFO:tasks.workunit.client.0.smithi107.stdout:8/684: unlink d6/d16/c55 0 2024-03-04T13:43:16.789 INFO:tasks.workunit.client.0.smithi107.stdout:5/727: creat d5/d8/d14/d2d/dfa/f103 x:0 0 0 2024-03-04T13:43:16.791 INFO:tasks.workunit.client.0.smithi107.stdout:0/704: getdents da/dc/dd/d3e/d26/d88/dd2/d73 0 2024-03-04T13:43:16.791 INFO:tasks.workunit.client.0.smithi107.stdout:2/629: mknod da/d29/d7c/cd3 0 2024-03-04T13:43:16.798 INFO:tasks.workunit.client.0.smithi107.stdout:7/659: getdents d6 0 2024-03-04T13:43:16.800 INFO:tasks.workunit.client.0.smithi107.stdout:2/630: symlink da/d44/d8b/d9d/dd0/ld4 0 2024-03-04T13:43:16.800 INFO:tasks.workunit.client.0.smithi107.stdout:8/685: truncate d6/d16/d1f/d63/d93/fce 1043373 0 2024-03-04T13:43:16.801 INFO:tasks.workunit.client.0.smithi107.stdout:4/621: dwrite d0/d9/d10/d80/db8/f9e [0,4194304] 0 2024-03-04T13:43:16.803 INFO:tasks.workunit.client.0.smithi107.stdout:0/705: write da/dc/dd/d3e/d26/d88/dd2/fa8 [904415,91567] 0 2024-03-04T13:43:16.804 INFO:tasks.workunit.client.0.smithi107.stdout:5/728: truncate d5/dc/df/f35 1527431 0 2024-03-04T13:43:16.805 INFO:tasks.workunit.client.0.smithi107.stdout:4/622: chown d0/d2f/l87 11634 1 2024-03-04T13:43:16.806 INFO:tasks.workunit.client.0.smithi107.stdout:8/686: symlink d6/da/d25/d2a/ld4 0 2024-03-04T13:43:16.809 INFO:tasks.workunit.client.0.smithi107.stdout:4/623: chown d0/d9/d10/dab 65779 1 2024-03-04T13:43:16.811 INFO:tasks.workunit.client.0.smithi107.stdout:7/660: truncate d6/d13/d16/f8b 2861967 0 2024-03-04T13:43:16.814 INFO:tasks.workunit.client.0.smithi107.stdout:1/679: dwrite d1/d46/d6a/fb5 [0,4194304] 0 2024-03-04T13:43:16.814 INFO:tasks.workunit.client.0.smithi107.stdout:5/729: rename d5/dc/dc6/l9e to d5/d66/d96/db1/l104 0 2024-03-04T13:43:16.822 INFO:tasks.workunit.client.0.smithi107.stdout:4/624: link d0/d9/de/d14/d66/f6a d0/d9/de/fbe 0 2024-03-04T13:43:16.826 INFO:tasks.workunit.client.0.smithi107.stdout:8/687: read d6/da/d25/d5f/f87 [3280481,51143] 0 2024-03-04T13:43:16.826 INFO:tasks.workunit.client.0.smithi107.stdout:0/706: truncate da/dc/dd/d3e/d26/d88/dd2/fbd 239576 0 2024-03-04T13:43:16.826 INFO:tasks.workunit.client.0.smithi107.stdout:1/680: read d1/d46/d6a/d17/f34 [114544,71110] 0 2024-03-04T13:43:16.826 INFO:tasks.workunit.client.0.smithi107.stdout:0/707: stat da/dc 0 2024-03-04T13:43:16.828 INFO:tasks.workunit.client.0.smithi107.stdout:4/625: symlink d0/d2f/lbf 0 2024-03-04T13:43:16.830 INFO:tasks.workunit.client.0.smithi107.stdout:5/730: link d5/dc/f55 d5/d8/d14/f105 0 2024-03-04T13:43:16.832 INFO:tasks.workunit.client.0.smithi107.stdout:1/681: fsync d1/d46/d6a/d65/f7f 0 2024-03-04T13:43:16.835 INFO:tasks.workunit.client.0.smithi107.stdout:1/682: write d1/f9a [13118,23365] 0 2024-03-04T13:43:16.835 INFO:tasks.workunit.client.0.smithi107.stdout:7/661: dwrite d6/d13/d16/d19/d38/d2c/f87 [0,4194304] 0 2024-03-04T13:43:16.840 INFO:tasks.workunit.client.0.smithi107.stdout:4/626: rename d0/d9/de/l7e to d0/d9/de/d14/lc0 0 2024-03-04T13:43:16.843 INFO:tasks.workunit.client.0.smithi107.stdout:9/650: sync 2024-03-04T13:43:16.844 INFO:tasks.workunit.client.0.smithi107.stdout:9/651: chown d4/da/cc 152 1 2024-03-04T13:43:16.848 INFO:tasks.workunit.client.0.smithi107.stdout:5/731: dwrite d5/d8/f17 [0,4194304] 0 2024-03-04T13:43:16.853 INFO:tasks.workunit.client.0.smithi107.stdout:4/627: mkdir d0/d30/dc1 0 2024-03-04T13:43:16.855 INFO:tasks.workunit.client.0.smithi107.stdout:8/688: truncate d6/d16/d59/d7c/f8a 1721173 0 2024-03-04T13:43:16.857 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:16 smithi169 ceph-mon[36313]: pgmap v60: 65 pgs: 65 active+clean; 3.0 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 63 MiB/s rd, 179 MiB/s wr, 381 op/s 2024-03-04T13:43:16.859 INFO:tasks.workunit.client.0.smithi107.stdout:5/732: rmdir d5/d66/d96/db1 39 2024-03-04T13:43:16.861 INFO:tasks.workunit.client.0.smithi107.stdout:0/708: rename da/dc/dd/d3e/d26/d69/cea to da/dc/dd/d3e/d26/d88/dd2/d73/cef 0 2024-03-04T13:43:16.862 INFO:tasks.workunit.client.0.smithi107.stdout:4/628: symlink d0/d9/d10/d80/db8/lc2 0 2024-03-04T13:43:16.865 INFO:tasks.workunit.client.0.smithi107.stdout:0/709: write f4 [6198121,66724] 0 2024-03-04T13:43:16.865 INFO:tasks.workunit.client.0.smithi107.stdout:5/733: symlink d5/d79/d89/da3/d6a/l106 0 2024-03-04T13:43:16.866 INFO:tasks.workunit.client.0.smithi107.stdout:1/683: write d1/d46/d6a/f51 [3078364,118582] 0 2024-03-04T13:43:16.914 INFO:tasks.workunit.client.0.smithi107.stdout:9/652: rename d4/da/d11/d4c/d89 to d4/da/d11/d20/d23/dd9 0 2024-03-04T13:43:16.914 INFO:tasks.workunit.client.0.smithi107.stdout:5/734: symlink d5/d79/d89/da3/l107 0 2024-03-04T13:43:16.915 INFO:tasks.workunit.client.0.smithi107.stdout:0/710: mkdir da/dc/dd/d3e/d26/d69/d6b/df0 0 2024-03-04T13:43:16.917 INFO:tasks.workunit.client.0.smithi107.stdout:1/684: symlink d1/d46/d6a/d4e/daa/dc9/le2 0 2024-03-04T13:43:16.919 INFO:tasks.workunit.client.0.smithi107.stdout:4/629: rename d0/d9/d10/d80/db8/d9a/db3 to d0/d9/d78/dc3 0 2024-03-04T13:43:16.923 INFO:tasks.workunit.client.0.smithi107.stdout:9/653: fdatasync d4/da/d11/d4c/d51/f5a 0 2024-03-04T13:43:16.923 INFO:tasks.workunit.client.0.smithi107.stdout:0/711: creat da/dc/dd/d3e/d26/d69/d6b/ff1 x:0 0 0 2024-03-04T13:43:16.927 INFO:tasks.workunit.client.0.smithi107.stdout:9/654: write d4/da/d11/d20/d2f/d9c/d92/dce/f7c [20761,121358] 0 2024-03-04T13:43:16.929 INFO:tasks.workunit.client.0.smithi107.stdout:0/712: rename da/dc/lcc to da/d7e/lf2 0 2024-03-04T13:43:16.929 INFO:tasks.workunit.client.0.smithi107.stdout:1/685: creat d1/d10/d16/d1b/fe3 x:0 0 0 2024-03-04T13:43:16.932 INFO:tasks.workunit.client.0.smithi107.stdout:9/655: readlink d4/da/d11/d20/d2f/d9c/d92/la0 0 2024-03-04T13:43:16.932 INFO:tasks.workunit.client.0.smithi107.stdout:4/630: link d0/d2f/d7d/c9f d0/d9/de/d14/d66/cc4 0 2024-03-04T13:43:16.934 INFO:tasks.workunit.client.0.smithi107.stdout:1/686: creat d1/d38/fe4 x:0 0 0 2024-03-04T13:43:16.940 INFO:tasks.workunit.client.0.smithi107.stdout:9/656: fdatasync d4/da/d11/d20/d39/f90 0 2024-03-04T13:43:16.941 INFO:tasks.workunit.client.0.smithi107.stdout:8/689: dread d6/d16/d1f/d4c/f52 [0,4194304] 0 2024-03-04T13:43:16.943 INFO:tasks.workunit.client.0.smithi107.stdout:5/735: dread d5/fad [0,4194304] 0 2024-03-04T13:43:16.944 INFO:tasks.workunit.client.0.smithi107.stdout:1/687: write d1/d46/d6a/d4e/daa/dc9/d67/fcd [122252,127931] 0 2024-03-04T13:43:16.945 INFO:tasks.workunit.client.0.smithi107.stdout:8/690: truncate d6/d16/d59/fbb 508146 0 2024-03-04T13:43:16.945 INFO:tasks.workunit.client.0.smithi107.stdout:4/631: link d0/f73 d0/d9/d10/d80/db8/fc5 0 2024-03-04T13:43:16.945 INFO:tasks.workunit.client.0.smithi107.stdout:5/736: readlink d5/d8/l3e 0 2024-03-04T13:43:16.947 INFO:tasks.workunit.client.0.smithi107.stdout:0/713: link da/dc/dd/d3e/d26/d88/dd2/fa8 da/dc/dd/dc9/de5/ff3 0 2024-03-04T13:43:16.948 INFO:tasks.workunit.client.0.smithi107.stdout:0/714: readlink da/l36 0 2024-03-04T13:43:16.948 INFO:tasks.workunit.client.0.smithi107.stdout:4/632: dread - d0/d9/d78/f85 zero size 2024-03-04T13:43:16.948 INFO:tasks.workunit.client.0.smithi107.stdout:9/657: creat d4/da/d11/d20/d64/fda x:0 0 0 2024-03-04T13:43:16.949 INFO:tasks.workunit.client.0.smithi107.stdout:4/633: readlink d0/l5e 0 2024-03-04T13:43:16.951 INFO:tasks.workunit.client.0.smithi107.stdout:9/658: write d4/d34/f44 [1503008,74236] 0 2024-03-04T13:43:16.951 INFO:tasks.workunit.client.0.smithi107.stdout:0/715: write da/dc/dd/d3e/d26/fe3 [707592,114174] 0 2024-03-04T13:43:16.955 INFO:tasks.workunit.client.0.smithi107.stdout:1/688: symlink d1/d46/d6a/d4e/daa/dc9/d33/d89/le5 0 2024-03-04T13:43:16.956 INFO:tasks.workunit.client.0.smithi107.stdout:4/634: truncate d0/d9/de/d14/d66/d7b/f8f 621707 0 2024-03-04T13:43:16.956 INFO:tasks.workunit.client.0.smithi107.stdout:9/659: dread d4/da/d11/d20/d2f/f3d [0,4194304] 0 2024-03-04T13:43:16.957 INFO:tasks.workunit.client.0.smithi107.stdout:6/695: sync 2024-03-04T13:43:16.957 INFO:tasks.workunit.client.0.smithi107.stdout:0/716: symlink da/dc/dd/d3e/d26/d88/dd2/lf4 0 2024-03-04T13:43:16.958 INFO:tasks.workunit.client.0.smithi107.stdout:4/635: chown d0/d9/de/d14/d66 490655 1 2024-03-04T13:43:16.960 INFO:tasks.workunit.client.0.smithi107.stdout:4/636: readlink d0/d2f/lbf 0 2024-03-04T13:43:16.960 INFO:tasks.workunit.client.0.smithi107.stdout:5/737: rename d5/dc/lb4 to d5/l108 0 2024-03-04T13:43:16.964 INFO:tasks.workunit.client.0.smithi107.stdout:1/689: mkdir d1/d46/d6a/d4e/daa/dc9/d33/d43/d5b/d61/ddb/de6 0 2024-03-04T13:43:16.965 INFO:tasks.workunit.client.0.smithi107.stdout:0/717: symlink da/dc/dd/d3e/d26/d69/d6b/df0/lf5 0 2024-03-04T13:43:16.966 INFO:tasks.workunit.client.0.smithi107.stdout:6/696: rmdir df/d17/d24/d40/d63/da3/d7e/d86 39 2024-03-04T13:43:16.969 INFO:tasks.workunit.client.0.smithi107.stdout:4/637: creat d0/d9/d10/d80/db8/d97/fc6 x:0 0 0 2024-03-04T13:43:16.969 INFO:tasks.workunit.client.0.smithi107.stdout:6/697: chown df/d1b/f4b 378 1 2024-03-04T13:43:16.969 INFO:tasks.workunit.client.0.smithi107.stdout:8/691: dwrite d6/d16/fcf [0,4194304] 0 2024-03-04T13:43:16.971 INFO:tasks.workunit.client.0.smithi107.stdout:5/738: mkdir d5/d8/d14/d2d/dfa/d109 0 2024-03-04T13:43:16.972 INFO:tasks.workunit.client.0.smithi107.stdout:8/692: chown d6/fd 165016529 1 2024-03-04T13:43:16.972 INFO:tasks.workunit.client.0.smithi107.stdout:1/690: dread d1/d46/d6a/d4e/daa/dc9/d33/d5d/d4c/f93 [0,4194304] 0 2024-03-04T13:43:16.973 INFO:tasks.workunit.client.0.smithi107.stdout:0/718: creat da/dc/d34/ff6 x:0 0 0 2024-03-04T13:43:16.975 INFO:tasks.workunit.client.0.smithi107.stdout:0/719: fsync da/dc/dd/d3e/f44 0 2024-03-04T13:43:16.981 INFO:tasks.workunit.client.0.smithi107.stdout:5/739: fsync d5/f24 0 2024-03-04T13:43:16.981 INFO:tasks.workunit.client.0.smithi107.stdout:6/698: chown df/d17/d24/d30/d3b/dd8 4731 1 2024-03-04T13:43:16.984 INFO:tasks.workunit.client.0.smithi107.stdout:1/691: symlink d1/d46/d6a/d4e/daa/dc9/d33/d43/d5b/d61/ddb/le7 0 2024-03-04T13:43:16.990 INFO:tasks.workunit.client.0.smithi107.stdout:8/693: mkdir d6/da/d25/dcb/dd5 0 2024-03-04T13:43:16.993 INFO:tasks.workunit.client.0.smithi107.stdout:9/660: truncate d4/da/d11/d20/d23/d7e/d84/fb9 2970380 0 2024-03-04T13:43:16.995 INFO:tasks.workunit.client.0.smithi107.stdout:9/661: stat d4/da/d11/d4c/d4d/d72 0 2024-03-04T13:43:16.997 INFO:tasks.workunit.client.0.smithi107.stdout:1/692: chown d1/l81 5 1 2024-03-04T13:43:17.000 INFO:tasks.workunit.client.0.smithi107.stdout:1/693: chown d1/d46/d6a/d65/cc7 3835 1 2024-03-04T13:43:17.002 INFO:tasks.workunit.client.0.smithi107.stdout:4/638: dwrite d0/d30/f64 [0,4194304] 0 2024-03-04T13:43:17.010 INFO:tasks.workunit.client.0.smithi107.stdout:0/720: truncate da/dc/dd/d3e/d26/f71 1470068 0 2024-03-04T13:43:17.014 INFO:tasks.workunit.client.0.smithi107.stdout:9/662: dwrite d4/da/d11/d20/d2f/d9c/daa/fc3 [0,4194304] 0 2024-03-04T13:43:17.021 INFO:tasks.workunit.client.0.smithi107.stdout:3/650: sync 2024-03-04T13:43:17.023 INFO:tasks.workunit.client.0.smithi107.stdout:6/699: rename df/d17/d24/d40/c4d to df/d17/d6b/d9b/d61/cdd 0 2024-03-04T13:43:17.025 INFO:tasks.workunit.client.0.smithi107.stdout:5/740: mknod d5/d8/d14/d2d/d92/dc1/dcc/c10a 0 2024-03-04T13:43:17.025 INFO:tasks.workunit.client.0.smithi107.stdout:1/694: dwrite d1/d10/fb0 [0,4194304] 0 2024-03-04T13:43:17.032 INFO:tasks.workunit.client.0.smithi107.stdout:8/694: link d6/d16/d1f/f65 d6/da/da8/db5/fd6 0 2024-03-04T13:43:17.034 INFO:tasks.workunit.client.0.smithi107.stdout:6/700: truncate df/d17/d6b/d9b/d61/f8a 1025111 0 2024-03-04T13:43:17.034 INFO:tasks.workunit.client.0.smithi107.stdout:3/651: symlink d1/lcc 0 2024-03-04T13:43:17.039 INFO:tasks.workunit.client.0.smithi107.stdout:1/695: stat d1/d46/d6a/d65/db1/lc8 0 2024-03-04T13:43:17.039 INFO:tasks.workunit.client.0.smithi107.stdout:0/721: mknod da/dc/cf7 0 2024-03-04T13:43:17.041 INFO:tasks.workunit.client.0.smithi107.stdout:3/652: write d1/de/d18/f5f [4699127,46365] 0 2024-03-04T13:43:17.041 INFO:tasks.workunit.client.0.smithi107.stdout:5/741: getdents d5/d8/d2a 0 2024-03-04T13:43:17.043 INFO:tasks.workunit.client.0.smithi107.stdout:6/701: write df/d2e/fd4 [755361,103592] 0 2024-03-04T13:43:17.045 INFO:tasks.workunit.client.0.smithi107.stdout:1/696: fsync d1/d10/d16/d1b/f3e 0 2024-03-04T13:43:17.047 INFO:tasks.workunit.client.0.smithi107.stdout:9/663: dwrite d4/d34/f44 [0,4194304] 0 2024-03-04T13:43:17.050 INFO:tasks.workunit.client.0.smithi107.stdout:0/722: read - da/dc/dd/d1e/fe2 zero size 2024-03-04T13:43:17.050 INFO:tasks.workunit.client.0.smithi107.stdout:5/742: write d5/d8/d2a/d43/f57 [216638,120296] 0 2024-03-04T13:43:17.056 INFO:tasks.workunit.client.0.smithi107.stdout:4/639: write d0/d9/de/d14/f4b [3100308,13158] 0 2024-03-04T13:43:17.060 INFO:tasks.workunit.client.0.smithi107.stdout:1/697: write d1/d46/d6a/d4e/daa/dc9/d67/fcd [322913,33178] 0 2024-03-04T13:43:17.060 INFO:tasks.workunit.client.0.smithi107.stdout:9/664: dread d4/da/d11/d20/d2f/f3d [0,4194304] 0 2024-03-04T13:43:17.060 INFO:tasks.workunit.client.0.smithi107.stdout:0/723: write da/dc/dd/d1e/fb5 [311559,39715] 0 2024-03-04T13:43:17.063 INFO:tasks.workunit.client.0.smithi107.stdout:4/640: truncate d0/d9/d78/fa7 661355 0 2024-03-04T13:43:17.065 INFO:tasks.workunit.client.0.smithi107.stdout:3/653: creat d1/de/d18/d33/d48/d71/d4c/fcd x:0 0 0 2024-03-04T13:43:17.066 INFO:tasks.workunit.client.0.smithi107.stdout:8/695: dwrite d6/f9f [0,4194304] 0 2024-03-04T13:43:17.070 INFO:tasks.workunit.client.0.smithi107.stdout:0/724: mknod da/dc/cf8 0 2024-03-04T13:43:17.071 INFO:tasks.workunit.client.0.smithi107.stdout:3/654: fsync d1/d1e/f22 0 2024-03-04T13:43:17.072 INFO:tasks.workunit.client.0.smithi107.stdout:0/725: chown da/d7e 111728 1 2024-03-04T13:43:17.076 INFO:tasks.workunit.client.0.smithi107.stdout:4/641: write d0/d9/de/d14/f4b [2264448,58306] 0 2024-03-04T13:43:17.076 INFO:tasks.workunit.client.0.smithi107.stdout:8/696: chown d6/da/d25/d2a/c66 97 1 2024-03-04T13:43:17.079 INFO:tasks.workunit.client.0.smithi107.stdout:1/698: truncate d1/d46/d6a/d17/f34 2266183 0 2024-03-04T13:43:17.081 INFO:tasks.workunit.client.0.smithi107.stdout:8/697: chown d6/d16/d1f/d63/d73/fa2 3216929 1 2024-03-04T13:43:17.084 INFO:tasks.workunit.client.0.smithi107.stdout:4/642: dread d0/d9/f6b [0,4194304] 0 2024-03-04T13:43:17.084 INFO:tasks.workunit.client.0.smithi107.stdout:0/726: fsync da/dc/dd/d3e/d26/d88/dd2/d73/d78/fab 0 2024-03-04T13:43:17.085 INFO:tasks.workunit.client.0.smithi107.stdout:1/699: write d1/d10/d16/d1b/f3e [825277,80650] 0 2024-03-04T13:43:17.086 INFO:tasks.workunit.client.0.smithi107.stdout:0/727: stat da/f62 0 2024-03-04T13:43:17.086 INFO:tasks.workunit.client.0.smithi107.stdout:6/702: dwrite df/d2e/fcd [0,4194304] 0 2024-03-04T13:43:17.090 INFO:tasks.workunit.client.0.smithi107.stdout:8/698: mkdir d6/d16/d59/d3e/d67/dd7 0 2024-03-04T13:43:17.090 INFO:tasks.workunit.client.0.smithi107.stdout:5/743: write d5/d8/fe7 [3887609,45363] 0 2024-03-04T13:43:17.090 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:16 smithi107 ceph-mon[28739]: pgmap v60: 65 pgs: 65 active+clean; 3.0 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 63 MiB/s rd, 179 MiB/s wr, 381 op/s 2024-03-04T13:43:17.093 INFO:tasks.workunit.client.0.smithi107.stdout:5/744: write d5/d8/d14/d2d/dfa/f103 [637576,125528] 0 2024-03-04T13:43:17.095 INFO:tasks.workunit.client.0.smithi107.stdout:1/700: unlink d1/d46/d6a/d65/f8f 0 2024-03-04T13:43:17.095 INFO:tasks.workunit.client.0.smithi107.stdout:0/728: mkdir da/dc/dd/d3e/d26/d69/df9 0 2024-03-04T13:43:17.097 INFO:tasks.workunit.client.0.smithi107.stdout:0/729: readlink da/dc/ldd 0 2024-03-04T13:43:17.098 INFO:tasks.workunit.client.0.smithi107.stdout:6/703: rename df/d17/d24/d40/f44 to df/d17/d24/d30/d5d/d6a/fde 0 2024-03-04T13:43:17.099 INFO:tasks.workunit.client.0.smithi107.stdout:8/699: mkdir d6/d16/d59/d3e/d96/dd8 0 2024-03-04T13:43:17.101 INFO:tasks.workunit.client.0.smithi107.stdout:3/655: getdents d1/de/d18/d33/d48/d71/d4c/daa/dab/dc5 0 2024-03-04T13:43:17.102 INFO:tasks.workunit.client.0.smithi107.stdout:4/643: write d0/d9/d18/d29/f35 [3393682,16528] 0 2024-03-04T13:43:17.106 INFO:tasks.workunit.client.0.smithi107.stdout:9/665: dwrite d4/da/d11/d20/d23/dd9/f8d [0,4194304] 0 2024-03-04T13:43:17.107 INFO:tasks.workunit.client.0.smithi107.stdout:4/644: write d0/d30/f90 [572325,24694] 0 2024-03-04T13:43:17.109 INFO:tasks.workunit.client.0.smithi107.stdout:0/730: creat da/dc/dd/ffa x:0 0 0 2024-03-04T13:43:17.114 INFO:tasks.workunit.client.0.smithi107.stdout:0/731: readlink da/dc/dd/d1e/lda 0 2024-03-04T13:43:17.115 INFO:tasks.workunit.client.0.smithi107.stdout:5/745: getdents d5/dc/de4 0 2024-03-04T13:43:17.116 INFO:tasks.workunit.client.0.smithi107.stdout:2/631: sync 2024-03-04T13:43:17.117 INFO:tasks.workunit.client.0.smithi107.stdout:7/662: sync 2024-03-04T13:43:17.123 INFO:tasks.workunit.client.0.smithi107.stdout:8/700: rename d6/f39 to d6/d16/d1f/d4c/da4/daa/fd9 0 2024-03-04T13:43:17.123 INFO:tasks.workunit.client.0.smithi107.stdout:1/701: getdents d1/d46/d6a/d4e/d60 0 2024-03-04T13:43:17.124 INFO:tasks.workunit.client.0.smithi107.stdout:6/704: dwrite df/d17/d24/d40/d63/da3/f68 [0,4194304] 0 2024-03-04T13:43:17.127 INFO:tasks.workunit.client.0.smithi107.stdout:9/666: creat d4/da/d11/d20/d39/fdb x:0 0 0 2024-03-04T13:43:17.127 INFO:tasks.workunit.client.0.smithi107.stdout:1/702: read d1/d46/d6a/d17/f34 [78941,35148] 0 2024-03-04T13:43:17.132 INFO:tasks.workunit.client.0.smithi107.stdout:0/732: unlink da/dc/dd/c3b 0 2024-03-04T13:43:17.150 INFO:tasks.workunit.client.0.smithi107.stdout:8/701: rename d6/d16/d1f/d4c/da5/fb2 to d6/d16/d1f/d4c/da4/fda 0 2024-03-04T13:43:17.151 INFO:tasks.workunit.client.0.smithi107.stdout:2/632: dwrite da/d49/d5c/f9e [0,4194304] 0 2024-03-04T13:43:17.151 INFO:tasks.workunit.client.0.smithi107.stdout:6/705: chown df/d38/la4 4 1 2024-03-04T13:43:17.153 INFO:tasks.workunit.client.0.smithi107.stdout:9/667: mkdir d4/da/d11/d20/d64/d77/ddc 0 2024-03-04T13:43:17.153 INFO:tasks.workunit.client.0.smithi107.stdout:3/656: truncate d1/de/f32 8697622 0 2024-03-04T13:43:17.154 INFO:tasks.workunit.client.0.smithi107.stdout:3/657: stat d1/d41/d81 0 2024-03-04T13:43:17.155 INFO:tasks.workunit.client.0.smithi107.stdout:9/668: write d4/d34/fb0 [329451,78286] 0 2024-03-04T13:43:17.160 INFO:tasks.workunit.client.0.smithi107.stdout:4/645: link d0/d9/d10/d80/db8/fc5 d0/d2f/fc7 0 2024-03-04T13:43:17.162 INFO:tasks.workunit.client.0.smithi107.stdout:2/633: symlink da/d38/ld5 0 2024-03-04T13:43:17.163 INFO:tasks.workunit.client.0.smithi107.stdout:5/746: creat d5/d79/d89/da3/f10b x:0 0 0 2024-03-04T13:43:17.166 INFO:tasks.workunit.client.0.smithi107.stdout:4/646: read d0/d9/d10/f1c [446882,106918] 0 2024-03-04T13:43:17.167 INFO:tasks.workunit.client.0.smithi107.stdout:3/658: write d1/d1e/daf/fcb [915030,79613] 0 2024-03-04T13:43:17.170 INFO:tasks.workunit.client.0.smithi107.stdout:9/669: dread d4/da/d11/d65/f58 [0,4194304] 0 2024-03-04T13:43:17.176 INFO:tasks.workunit.client.0.smithi107.stdout:2/634: dwrite da/d44/f52 [0,4194304] 0 2024-03-04T13:43:17.180 INFO:tasks.workunit.client.0.smithi107.stdout:6/706: mknod df/d38/d4f/d74/db5/cdf 0 2024-03-04T13:43:17.181 INFO:tasks.workunit.client.0.smithi107.stdout:5/747: mkdir d5/d8/d2a/dd6/d10c 0 2024-03-04T13:43:17.183 INFO:tasks.workunit.client.0.smithi107.stdout:5/748: write d5/fed [163451,22713] 0 2024-03-04T13:43:17.185 INFO:tasks.workunit.client.0.smithi107.stdout:9/670: dwrite d4/d34/f44 [0,4194304] 0 2024-03-04T13:43:17.190 INFO:tasks.workunit.client.0.smithi107.stdout:1/703: write d1/f94 [4422982,20662] 0 2024-03-04T13:43:17.193 INFO:tasks.workunit.client.0.smithi107.stdout:1/704: chown d1/d46/d6a/d4e/daa/dc9/d33/d43/d59 2054525 1 2024-03-04T13:43:17.205 INFO:tasks.workunit.client.0.smithi107.stdout:0/733: truncate da/f2a 573525 0 2024-03-04T13:43:17.208 INFO:tasks.workunit.client.0.smithi107.stdout:8/702: link d6/da/d25/lb8 d6/d16/d1f/d4c/da4/ldb 0 2024-03-04T13:43:17.211 INFO:tasks.workunit.client.0.smithi107.stdout:2/635: link da/d49/fc0 da/d44/d8b/d9d/fd6 0 2024-03-04T13:43:17.219 INFO:tasks.workunit.client.0.smithi107.stdout:5/749: symlink d5/d8/d14/l10d 0 2024-03-04T13:43:17.221 INFO:tasks.workunit.client.0.smithi107.stdout:9/671: symlink d4/da/d11/d4c/ldd 0 2024-03-04T13:43:17.221 INFO:tasks.workunit.client.0.smithi107.stdout:5/750: write d5/d8/d2a/d43/f57 [1429309,66393] 0 2024-03-04T13:43:17.224 INFO:tasks.workunit.client.0.smithi107.stdout:8/703: rename f2 to d6/d16/d59/d3e/d67/fdc 0 2024-03-04T13:43:17.228 INFO:tasks.workunit.client.0.smithi107.stdout:0/734: dread da/dc/dd/d3e/d26/f64 [0,4194304] 0 2024-03-04T13:43:17.235 INFO:tasks.workunit.client.0.smithi107.stdout:4/647: getdents d0/d9/de/d14/d66/d7b 0 2024-03-04T13:43:17.237 INFO:tasks.workunit.client.0.smithi107.stdout:5/751: creat d5/d8/d14/d2d/d92/dc1/dcc/f10e x:0 0 0 2024-03-04T13:43:17.239 INFO:tasks.workunit.client.0.smithi107.stdout:8/704: symlink d6/da/d25/ldd 0 2024-03-04T13:43:17.240 INFO:tasks.workunit.client.0.smithi107.stdout:9/672: truncate d4/da/d11/d4c/d51/f5c 3445140 0 2024-03-04T13:43:17.241 INFO:tasks.workunit.client.0.smithi107.stdout:1/705: link d1/d46/d6a/d4e/daa/dc9/d33/d5d/d24/l36 d1/d10/d16/d47/d7a/le8 0 2024-03-04T13:43:17.242 INFO:tasks.workunit.client.0.smithi107.stdout:9/673: readlink d4/da/d11/d20/d23/d7e/d84/l8a 0 2024-03-04T13:43:17.246 INFO:tasks.workunit.client.0.smithi107.stdout:6/707: getdents df/d57 0 2024-03-04T13:43:17.248 INFO:tasks.workunit.client.0.smithi107.stdout:6/708: stat df/d2e/d36/d77/d95/da7 0 2024-03-04T13:43:17.249 INFO:tasks.workunit.client.0.smithi107.stdout:5/752: creat d5/d8/da/db0/f10f x:0 0 0 2024-03-04T13:43:17.251 INFO:tasks.workunit.client.0.smithi107.stdout:5/753: stat d5/d66 0 2024-03-04T13:43:17.251 INFO:tasks.workunit.client.0.smithi107.stdout:1/706: unlink d1/l81 0 2024-03-04T13:43:17.251 INFO:tasks.workunit.client.0.smithi107.stdout:6/709: write df/d2e/d36/d77/f90 [3809362,104216] 0 2024-03-04T13:43:17.252 INFO:tasks.workunit.client.0.smithi107.stdout:0/735: mknod da/dc/dd/d3e/d26/d88/dd2/d73/d78/cfb 0 2024-03-04T13:43:17.253 INFO:tasks.workunit.client.0.smithi107.stdout:1/707: read - d1/d46/fd0 zero size 2024-03-04T13:43:17.253 INFO:tasks.workunit.client.0.smithi107.stdout:9/674: creat d4/d34/fde x:0 0 0 2024-03-04T13:43:17.253 INFO:tasks.workunit.client.0.smithi107.stdout:2/636: dread da/d29/f69 [0,4194304] 0 2024-03-04T13:43:17.254 INFO:tasks.workunit.client.0.smithi107.stdout:8/705: dread d6/f45 [0,4194304] 0 2024-03-04T13:43:17.254 INFO:tasks.workunit.client.0.smithi107.stdout:5/754: dread - d5/d8/da/db0/f10f zero size 2024-03-04T13:43:17.260 INFO:tasks.workunit.client.0.smithi107.stdout:6/710: creat df/d17/d24/d30/d5d/d6a/fe0 x:0 0 0 2024-03-04T13:43:17.261 INFO:tasks.workunit.client.0.smithi107.stdout:0/736: write da/dc/dd/d3e/d26/d88/dd2/d73/d78/fcf [609003,60015] 0 2024-03-04T13:43:17.263 INFO:tasks.workunit.client.0.smithi107.stdout:9/675: rmdir d4/da/d11/d20/d64 39 2024-03-04T13:43:17.265 INFO:tasks.workunit.client.0.smithi107.stdout:9/676: stat d4/da/d11/d20/d23/dd9/f9f 0 2024-03-04T13:43:17.265 INFO:tasks.workunit.client.0.smithi107.stdout:2/637: rename da/d38/ccb to da/d29/d7c/dbd/cd7 0 2024-03-04T13:43:17.268 INFO:tasks.workunit.client.0.smithi107.stdout:8/706: symlink d6/d16/d59/d7c/lde 0 2024-03-04T13:43:17.269 INFO:tasks.workunit.client.0.smithi107.stdout:2/638: write da/df/d3e/fab [264226,68475] 0 2024-03-04T13:43:17.273 INFO:tasks.workunit.client.0.smithi107.stdout:0/737: rmdir da/d7e 39 2024-03-04T13:43:17.274 INFO:tasks.workunit.client.0.smithi107.stdout:2/639: dread da/df/f13 [0,4194304] 0 2024-03-04T13:43:17.276 INFO:tasks.workunit.client.0.smithi107.stdout:4/648: truncate d0/d9/d18/f57 1823551 0 2024-03-04T13:43:17.277 INFO:tasks.workunit.client.0.smithi107.stdout:3/659: write d1/d1e/f74 [5794912,104276] 0 2024-03-04T13:43:17.278 INFO:tasks.workunit.client.0.smithi107.stdout:7/663: sync 2024-03-04T13:43:17.279 INFO:tasks.workunit.client.0.smithi107.stdout:9/677: rmdir d4/da/d11/d20/d23/d7e/d84 39 2024-03-04T13:43:17.282 INFO:tasks.workunit.client.0.smithi107.stdout:1/708: write d1/d46/d6a/d17/f34 [1833163,55523] 0 2024-03-04T13:43:17.286 INFO:tasks.workunit.client.0.smithi107.stdout:8/707: mkdir d6/d16/d59/d7c/ddf 0 2024-03-04T13:43:17.289 INFO:tasks.workunit.client.0.smithi107.stdout:6/711: mkdir df/d17/d24/d30/d8b/dc1/de1 0 2024-03-04T13:43:17.292 INFO:tasks.workunit.client.0.smithi107.stdout:2/640: creat da/d38/d3a/fd8 x:0 0 0 2024-03-04T13:43:17.293 INFO:tasks.workunit.client.0.smithi107.stdout:4/649: stat d0/d9/d1e/l24 0 2024-03-04T13:43:17.297 INFO:tasks.workunit.client.0.smithi107.stdout:3/660: rmdir d1/de/d18/d33/d48/d71/d4c/daa/dab/dc5 39 2024-03-04T13:43:17.298 INFO:tasks.workunit.client.0.smithi107.stdout:5/755: getdents d5/d8/d2a/d62 0 2024-03-04T13:43:17.299 INFO:tasks.workunit.client.0.smithi107.stdout:5/756: fdatasync d5/d79/d89/fb3 0 2024-03-04T13:43:17.301 INFO:tasks.workunit.client.0.smithi107.stdout:1/709: truncate d1/d46/d6a/d4e/daa/dc9/f32 5494817 0 2024-03-04T13:43:17.302 INFO:tasks.workunit.client.0.smithi107.stdout:5/757: truncate d5/d8/da/db0/f10f 472462 0 2024-03-04T13:43:17.306 INFO:tasks.workunit.client.0.smithi107.stdout:6/712: unlink df/d17/d5c/lbe 0 2024-03-04T13:43:17.306 INFO:tasks.workunit.client.0.smithi107.stdout:2/641: fdatasync da/f20 0 2024-03-04T13:43:17.306 INFO:tasks.workunit.client.0.smithi107.stdout:4/650: creat d0/d9/d1e/fc8 x:0 0 0 2024-03-04T13:43:17.312 INFO:tasks.workunit.client.0.smithi107.stdout:1/710: read d1/f9a [11521,89831] 0 2024-03-04T13:43:17.312 INFO:tasks.workunit.client.0.smithi107.stdout:3/661: write d1/de/d18/d33/d48/d71/fa9 [257756,18740] 0 2024-03-04T13:43:17.312 INFO:tasks.workunit.client.0.smithi107.stdout:8/708: creat d6/d16/d59/d3e/d96/dd8/fe0 x:0 0 0 2024-03-04T13:43:17.313 INFO:tasks.workunit.client.0.smithi107.stdout:5/758: unlink d5/d8/f1f 0 2024-03-04T13:43:17.317 INFO:tasks.workunit.client.0.smithi107.stdout:2/642: creat da/df/d3e/fd9 x:0 0 0 2024-03-04T13:43:17.317 INFO:tasks.workunit.client.0.smithi107.stdout:7/664: dwrite d6/d13/d14/fa4 [4194304,4194304] 0 2024-03-04T13:43:17.319 INFO:tasks.workunit.client.0.smithi107.stdout:1/711: write d1/d46/d6a/d4e/d60/f4f [4430717,33395] 0 2024-03-04T13:43:17.321 INFO:tasks.workunit.client.0.smithi107.stdout:2/643: chown da/d38/d3a/d5d 15585 1 2024-03-04T13:43:17.325 INFO:tasks.workunit.client.0.smithi107.stdout:3/662: creat d1/de/d18/d33/d48/fce x:0 0 0 2024-03-04T13:43:17.339 INFO:tasks.workunit.client.0.smithi107.stdout:8/709: creat d6/da/da8/db5/fe1 x:0 0 0 2024-03-04T13:43:17.339 INFO:tasks.workunit.client.0.smithi107.stdout:0/738: dwrite da/dc/dd/d3e/f77 [0,4194304] 0 2024-03-04T13:43:17.341 INFO:tasks.workunit.client.0.smithi107.stdout:1/712: rmdir d1/d46/d6a/d4e/daa/dc9/d67 39 2024-03-04T13:43:17.342 INFO:tasks.workunit.client.0.smithi107.stdout:2/644: rmdir da/d44/d8b/d9d 39 2024-03-04T13:43:17.342 INFO:tasks.workunit.client.0.smithi107.stdout:3/663: fsync d1/de/d18/d33/d48/fce 0 2024-03-04T13:43:17.350 INFO:tasks.workunit.client.0.smithi107.stdout:0/739: write da/dc/dd/d3e/f11 [3776828,72854] 0 2024-03-04T13:43:17.351 INFO:tasks.workunit.client.0.smithi107.stdout:4/651: truncate d0/d2f/f72 1291234 0 2024-03-04T13:43:17.354 INFO:tasks.workunit.client.0.smithi107.stdout:1/713: truncate d1/d10/d16/d1b/fe3 678066 0 2024-03-04T13:43:17.354 INFO:tasks.workunit.client.0.smithi107.stdout:2/645: creat da/d29/d7c/dbd/fda x:0 0 0 2024-03-04T13:43:17.366 INFO:tasks.workunit.client.0.smithi107.stdout:3/664: mkdir d1/de/d18/d33/d4d/d4f/dc3/dcf 0 2024-03-04T13:43:17.367 INFO:tasks.workunit.client.0.smithi107.stdout:0/740: dread - da/dc/dd/d3e/fee zero size 2024-03-04T13:43:17.370 INFO:tasks.workunit.client.0.smithi107.stdout:3/665: stat d1/de/l53 0 2024-03-04T13:43:17.373 INFO:tasks.workunit.client.0.smithi107.stdout:1/714: creat d1/d46/d6a/d4e/daa/dc9/d33/d5d/d4c/fe9 x:0 0 0 2024-03-04T13:43:17.374 INFO:tasks.workunit.client.0.smithi107.stdout:2/646: readlink da/d44/d8b/d9d/dd0/la1 0 2024-03-04T13:43:17.377 INFO:tasks.workunit.client.0.smithi107.stdout:9/678: dread d4/da/d11/d4c/d51/f5c [0,4194304] 0 2024-03-04T13:43:17.381 INFO:tasks.workunit.client.0.smithi107.stdout:8/710: dwrite d6/d16/d59/fbb [0,4194304] 0 2024-03-04T13:43:17.383 INFO:tasks.workunit.client.0.smithi107.stdout:7/665: dwrite d6/d1d/d64/d9a/da9/d71/fcc [0,4194304] 0 2024-03-04T13:43:17.388 INFO:tasks.workunit.client.0.smithi107.stdout:1/715: write d1/d46/d6a/fb5 [1526557,59813] 0 2024-03-04T13:43:17.390 INFO:tasks.workunit.client.0.smithi107.stdout:3/666: symlink d1/de/d11/ld0 0 2024-03-04T13:43:17.390 INFO:tasks.workunit.client.0.smithi107.stdout:5/759: dwrite d5/d66/d96/db1/fe0 [0,4194304] 0 2024-03-04T13:43:17.394 INFO:tasks.workunit.client.0.smithi107.stdout:1/716: dread - d1/d46/d6a/d4e/d60/fc4 zero size 2024-03-04T13:43:17.394 INFO:tasks.workunit.client.0.smithi107.stdout:0/741: mknod da/dc/dd/d1e/dbf/cfc 0 2024-03-04T13:43:17.401 INFO:tasks.workunit.client.0.smithi107.stdout:3/667: unlink d1/d29/f2a 0 2024-03-04T13:43:17.401 INFO:tasks.workunit.client.0.smithi107.stdout:4/652: dwrite d0/d9/d10/d6f/f95 [0,4194304] 0 2024-03-04T13:43:17.404 INFO:tasks.workunit.client.0.smithi107.stdout:3/668: chown d1/de/l53 20840782 1 2024-03-04T13:43:17.404 INFO:tasks.workunit.client.0.smithi107.stdout:5/760: rename d5/d8/f45 to d5/d2b/d4b/f110 0 2024-03-04T13:43:17.404 INFO:tasks.workunit.client.0.smithi107.stdout:3/669: chown d1/d2d 1 1 2024-03-04T13:43:17.409 INFO:tasks.workunit.client.0.smithi107.stdout:2/647: mkdir da/df/d73/dd1/ddb 0 2024-03-04T13:43:17.410 INFO:tasks.workunit.client.0.smithi107.stdout:6/713: sync 2024-03-04T13:43:17.411 INFO:tasks.workunit.client.0.smithi107.stdout:5/761: read - d5/d79/d89/da3/d68/fe8 zero size 2024-03-04T13:43:17.417 INFO:tasks.workunit.client.0.smithi107.stdout:4/653: unlink d0/d9/de/d14/d66/f6a 0 2024-03-04T13:43:17.418 INFO:tasks.workunit.client.0.smithi107.stdout:2/648: mkdir da/df/d73/daa/ddc 0 2024-03-04T13:43:17.424 INFO:tasks.workunit.client.0.smithi107.stdout:1/717: chown d1/d46/d6a/d4e/daa/dc9/d67/fcd 264 1 2024-03-04T13:43:17.425 INFO:tasks.workunit.client.0.smithi107.stdout:5/762: write d5/d79/d89/da3/d6a/f8e [2618461,29089] 0 2024-03-04T13:43:17.428 INFO:tasks.workunit.client.0.smithi107.stdout:3/670: symlink d1/de/d18/d33/ld1 0 2024-03-04T13:43:17.428 INFO:tasks.workunit.client.0.smithi107.stdout:0/742: write da/dc/dd/d3e/d26/d88/dd2/f87 [460481,28677] 0 2024-03-04T13:43:17.432 INFO:tasks.workunit.client.0.smithi107.stdout:6/714: rename fe to df/d17/d24/d40/d63/da3/d7e/d86/fe2 0 2024-03-04T13:43:17.433 INFO:tasks.workunit.client.0.smithi107.stdout:5/763: mknod d5/dc/dc6/c111 0 2024-03-04T13:43:17.433 INFO:tasks.workunit.client.0.smithi107.stdout:9/679: link d4/da/d11/d20/d23/d7e/d84/la6 d4/da/d11/d20/d2f/ldf 0 2024-03-04T13:43:17.433 INFO:tasks.workunit.client.0.smithi107.stdout:0/743: chown da/dc/dd/d3e/fac 177814 1 2024-03-04T13:43:17.434 INFO:tasks.workunit.client.0.smithi107.stdout:6/715: chown df/d1b/f1d 230193 1 2024-03-04T13:43:17.435 INFO:tasks.workunit.client.0.smithi107.stdout:1/718: rmdir d1/d46/d6a/d4e/daa/dc9 39 2024-03-04T13:43:17.437 INFO:tasks.workunit.client.0.smithi107.stdout:3/671: mkdir d1/de/d18/d33/d6b/dd2 0 2024-03-04T13:43:17.441 INFO:tasks.workunit.client.0.smithi107.stdout:4/654: creat d0/fc9 x:0 0 0 2024-03-04T13:43:17.442 INFO:tasks.workunit.client.0.smithi107.stdout:6/716: chown df/d17/d6b/d9b/l32 13861 1 2024-03-04T13:43:17.446 INFO:tasks.workunit.client.0.smithi107.stdout:3/672: creat d1/de/d18/d1f/fd3 x:0 0 0 2024-03-04T13:43:17.452 INFO:tasks.workunit.client.0.smithi107.stdout:7/666: truncate d6/d13/d16/d1c/f7d 837843 0 2024-03-04T13:43:17.457 INFO:tasks.workunit.client.0.smithi107.stdout:2/649: dwrite da/d49/d5c/f67 [0,4194304] 0 2024-03-04T13:43:17.460 INFO:tasks.workunit.client.0.smithi107.stdout:2/650: chown da/df/d3e/f50 158 1 2024-03-04T13:43:17.460 INFO:tasks.workunit.client.0.smithi107.stdout:3/673: creat d1/de/d11/fd4 x:0 0 0 2024-03-04T13:43:17.463 INFO:tasks.workunit.client.0.smithi107.stdout:7/667: read d6/d17/f97 [2533128,113790] 0 2024-03-04T13:43:17.463 INFO:tasks.workunit.client.0.smithi107.stdout:9/680: dread d4/da/d11/d65/f58 [0,4194304] 0 2024-03-04T13:43:17.464 INFO:tasks.workunit.client.0.smithi107.stdout:6/717: rename df/d17/fd1 to df/d17/d24/d30/d5d/d6a/fe3 0 2024-03-04T13:43:17.470 INFO:tasks.workunit.client.0.smithi107.stdout:2/651: rmdir da/df/d27/d58 39 2024-03-04T13:43:17.473 INFO:tasks.workunit.client.0.smithi107.stdout:1/719: rename d1/d46/d6a/d4e/daa/dc9/d33/d43 to d1/d46/d6a/d4e/daa/dc9/d33/d5d/d4c/dea 0 2024-03-04T13:43:17.476 INFO:tasks.workunit.client.0.smithi107.stdout:9/681: link d4/da/d11/d20/d39/f90 d4/d34/fe0 0 2024-03-04T13:43:17.476 INFO:tasks.workunit.client.0.smithi107.stdout:0/744: link da/dc/dd/d3e/d26/d88/dd2/d73/cbc da/dc/dd/d3e/d26/cfd 0 2024-03-04T13:43:17.478 INFO:tasks.workunit.client.0.smithi107.stdout:1/720: fdatasync d1/d46/d6a/d4e/daa/dc9/d67/d6b/f84 0 2024-03-04T13:43:17.479 INFO:tasks.workunit.client.0.smithi107.stdout:8/711: dread d6/da/f19 [0,4194304] 0 2024-03-04T13:43:17.482 INFO:tasks.workunit.client.0.smithi107.stdout:3/674: rename d1/de/d18/d33/d48/d71/d4c/daa to d1/de/d11/dd5 0 2024-03-04T13:43:17.485 INFO:tasks.workunit.client.0.smithi107.stdout:2/652: mknod da/df/d27/d58/d5f/cdd 0 2024-03-04T13:43:17.486 INFO:tasks.workunit.client.0.smithi107.stdout:1/721: write d1/d46/d6a/d4e/daa/dc9/d33/d5d/d24/f62 [2207329,33244] 0 2024-03-04T13:43:17.486 INFO:tasks.workunit.client.0.smithi107.stdout:8/712: readlink d6/da/da8/lab 0 2024-03-04T13:43:17.490 INFO:tasks.workunit.client.0.smithi107.stdout:3/675: symlink d1/de/d18/d1f/ld6 0 2024-03-04T13:43:17.496 INFO:tasks.workunit.client.0.smithi107.stdout:8/713: creat d6/d16/d59/d3e/fe2 x:0 0 0 2024-03-04T13:43:17.496 INFO:tasks.workunit.client.0.smithi107.stdout:2/653: truncate da/d44/f79 495582 0 2024-03-04T13:43:17.502 INFO:tasks.workunit.client.0.smithi107.stdout:2/654: fsync da/d29/d7c/fb8 0 2024-03-04T13:43:17.508 INFO:tasks.workunit.client.0.smithi107.stdout:7/668: dwrite d6/d13/d16/d19/d38/d2c/fac [0,4194304] 0 2024-03-04T13:43:17.509 INFO:tasks.workunit.client.0.smithi107.stdout:3/676: rmdir d1/de/d18/d33/d6b/dd2 0 2024-03-04T13:43:17.512 INFO:tasks.workunit.client.0.smithi107.stdout:8/714: truncate d6/d16/d1f/f34 1651353 0 2024-03-04T13:43:17.513 INFO:tasks.workunit.client.0.smithi107.stdout:5/764: truncate d5/d8/d2a/d43/f57 730348 0 2024-03-04T13:43:17.514 INFO:tasks.workunit.client.0.smithi107.stdout:7/669: chown d6/d17/f8f 39 1 2024-03-04T13:43:17.516 INFO:tasks.workunit.client.0.smithi107.stdout:2/655: write da/d38/d3a/fae [319554,76553] 0 2024-03-04T13:43:17.518 INFO:tasks.workunit.client.0.smithi107.stdout:1/722: rmdir d1 39 2024-03-04T13:43:17.520 INFO:tasks.workunit.client.0.smithi107.stdout:8/715: mkdir d6/da/d25/dd1/de3 0 2024-03-04T13:43:17.530 INFO:tasks.workunit.client.0.smithi107.stdout:2/656: symlink da/df/d27/d58/lde 0 2024-03-04T13:43:17.530 INFO:tasks.workunit.client.0.smithi107.stdout:1/723: dread - d1/d46/d6a/d4e/daa/dc9/d33/d5d/fbf zero size 2024-03-04T13:43:17.532 INFO:tasks.workunit.client.0.smithi107.stdout:9/682: truncate d4/da/f96 388598 0 2024-03-04T13:43:17.535 INFO:tasks.workunit.client.0.smithi107.stdout:8/716: unlink d6/da/d25/lb8 0 2024-03-04T13:43:17.538 INFO:tasks.workunit.client.0.smithi107.stdout:4/655: dwrite d0/f73 [0,4194304] 0 2024-03-04T13:43:17.541 INFO:tasks.workunit.client.0.smithi107.stdout:2/657: write da/d49/d5c/d6a/f7f [3888582,12288] 0 2024-03-04T13:43:17.543 INFO:tasks.workunit.client.0.smithi107.stdout:0/745: dread da/dc/dd/d3e/fd4 [0,4194304] 0 2024-03-04T13:43:17.546 INFO:tasks.workunit.client.0.smithi107.stdout:9/683: mkdir d4/da/d11/de1 0 2024-03-04T13:43:17.551 INFO:tasks.workunit.client.0.smithi107.stdout:1/724: fsync d1/d46/d6a/d4e/d60/f53 0 2024-03-04T13:43:17.552 INFO:tasks.workunit.client.0.smithi107.stdout:3/677: dwrite d1/de/d11/f46 [0,4194304] 0 2024-03-04T13:43:17.556 INFO:tasks.workunit.client.0.smithi107.stdout:4/656: creat d0/d9/d10/d6f/fca x:0 0 0 2024-03-04T13:43:17.559 INFO:tasks.workunit.client.0.smithi107.stdout:1/725: dread - d1/d10/d16/d1b/fd2 zero size 2024-03-04T13:43:17.561 INFO:tasks.workunit.client.0.smithi107.stdout:9/684: creat d4/da/d11/d4c/d51/fe2 x:0 0 0 2024-03-04T13:43:17.561 INFO:tasks.workunit.client.0.smithi107.stdout:3/678: rename d1 to d1/de/d18/d33/d48/dc8/dd7 22 2024-03-04T13:43:17.564 INFO:tasks.workunit.client.0.smithi107.stdout:2/658: symlink da/d44/ldf 0 2024-03-04T13:43:17.566 INFO:tasks.workunit.client.0.smithi107.stdout:5/765: dread d5/dc/f3b [0,4194304] 0 2024-03-04T13:43:17.570 INFO:tasks.workunit.client.0.smithi107.stdout:6/718: sync 2024-03-04T13:43:17.570 INFO:tasks.workunit.client.0.smithi107.stdout:0/746: chown da/dc/dd/d3e/d26/d88/lcd 14170242 1 2024-03-04T13:43:17.571 INFO:tasks.workunit.client.0.smithi107.stdout:1/726: write d1/d46/fd0 [743593,109582] 0 2024-03-04T13:43:17.579 INFO:tasks.workunit.client.0.smithi107.stdout:1/727: write d1/d46/d6a/d4e/d60/d83/fc6 [872639,12865] 0 2024-03-04T13:43:17.579 INFO:tasks.workunit.client.0.smithi107.stdout:4/657: dread d0/d9/d10/f7a [0,4194304] 0 2024-03-04T13:43:17.583 INFO:tasks.workunit.client.0.smithi107.stdout:2/659: rename da/d38/d3a/c75 to da/d34/ce0 0 2024-03-04T13:43:17.586 INFO:tasks.workunit.client.0.smithi107.stdout:0/747: mknod da/d7e/cfe 0 2024-03-04T13:43:17.586 INFO:tasks.workunit.client.0.smithi107.stdout:9/685: creat d4/da/d11/d4c/fe3 x:0 0 0 2024-03-04T13:43:17.588 INFO:tasks.workunit.client.0.smithi107.stdout:5/766: symlink d5/d66/d96/l112 0 2024-03-04T13:43:17.588 INFO:tasks.workunit.client.0.smithi107.stdout:1/728: chown d1/d46/d6a/d4e/daa/dc9/d33/d5d/ld8 6185 1 2024-03-04T13:43:17.594 INFO:tasks.workunit.client.0.smithi107.stdout:4/658: symlink d0/d30/lcb 0 2024-03-04T13:43:17.595 INFO:tasks.workunit.client.0.smithi107.stdout:6/719: mkdir df/d17/d24/d40/d63/da3/de4 0 2024-03-04T13:43:17.601 INFO:tasks.workunit.client.0.smithi107.stdout:9/686: mkdir d4/da/d11/d20/d23/dcc/de4 0 2024-03-04T13:43:17.602 INFO:tasks.workunit.client.0.smithi107.stdout:1/729: symlink d1/d10/d16/d1b/leb 0 2024-03-04T13:43:17.603 INFO:tasks.workunit.client.0.smithi107.stdout:6/720: mknod df/d2e/ce5 0 2024-03-04T13:43:17.603 INFO:tasks.workunit.client.0.smithi107.stdout:2/660: link da/df/d3e/f97 da/d29/d7c/fe1 0 2024-03-04T13:43:17.607 INFO:tasks.workunit.client.0.smithi107.stdout:0/748: creat da/dc/fff x:0 0 0 2024-03-04T13:43:17.609 INFO:tasks.workunit.client.0.smithi107.stdout:4/659: getdents d0/d2f 0 2024-03-04T13:43:17.615 INFO:tasks.workunit.client.0.smithi107.stdout:6/721: chown df/d17/d5c/db1/l9f 433 1 2024-03-04T13:43:17.616 INFO:tasks.workunit.client.0.smithi107.stdout:7/670: dread d6/f45 [0,4194304] 0 2024-03-04T13:43:17.616 INFO:tasks.workunit.client.0.smithi107.stdout:0/749: chown da/dc/dd/d3e/d26/d88/dd2 2518 1 2024-03-04T13:43:17.617 INFO:tasks.workunit.client.0.smithi107.stdout:4/660: rmdir d0/d9/d10/d6f 39 2024-03-04T13:43:17.621 INFO:tasks.workunit.client.0.smithi107.stdout:1/730: mkdir d1/d10/d16/d47/d7a/dda/dec 0 2024-03-04T13:43:17.623 INFO:tasks.workunit.client.0.smithi107.stdout:2/661: read da/df/d3e/f50 [5206289,45106] 0 2024-03-04T13:43:17.630 INFO:tasks.workunit.client.0.smithi107.stdout:7/671: mknod d6/d13/d16/d1c/cdc 0 2024-03-04T13:43:17.631 INFO:tasks.workunit.client.0.smithi107.stdout:3/679: write d1/de/d18/d33/d4d/f5b [874732,17141] 0 2024-03-04T13:43:17.635 INFO:tasks.workunit.client.0.smithi107.stdout:1/731: rename d1/d10/d16/d1b/fd2 to d1/d46/d6a/d4e/d60/fed 0 2024-03-04T13:43:17.636 INFO:tasks.workunit.client.0.smithi107.stdout:9/687: link d4/da/d11/d4c/d4d/lc9 d4/da/d11/d20/d23/dd9/le5 0 2024-03-04T13:43:17.637 INFO:tasks.workunit.client.0.smithi107.stdout:7/672: write d6/d1d/d64/d9a/da9/d71/fcc [1065668,25202] 0 2024-03-04T13:43:17.640 INFO:tasks.workunit.client.0.smithi107.stdout:1/732: chown d1/d46/d6a/d4e/daa 233 1 2024-03-04T13:43:17.647 INFO:tasks.workunit.client.0.smithi107.stdout:8/717: dread d6/da/d25/d5f/f87 [0,4194304] 0 2024-03-04T13:43:17.651 INFO:tasks.workunit.client.0.smithi107.stdout:6/722: dwrite df/d2e/d36/f49 [0,4194304] 0 2024-03-04T13:43:17.652 INFO:tasks.workunit.client.0.smithi107.stdout:4/661: mknod d0/d9/de/d14/d66/d7b/ccc 0 2024-03-04T13:43:17.654 INFO:tasks.workunit.client.0.smithi107.stdout:6/723: write df/d17/f23 [1883043,112429] 0 2024-03-04T13:43:17.656 INFO:tasks.workunit.client.0.smithi107.stdout:6/724: chown df/d57/l91 29 1 2024-03-04T13:43:17.661 INFO:tasks.workunit.client.0.smithi107.stdout:7/673: rename d6/d13/d14/fb8 to d6/d1d/d64/d9a/da9/d71/db9/dca/d3d/fdd 0 2024-03-04T13:43:17.661 INFO:tasks.workunit.client.0.smithi107.stdout:8/718: creat d6/d16/d59/d3e/fe4 x:0 0 0 2024-03-04T13:43:17.665 INFO:tasks.workunit.client.0.smithi107.stdout:0/750: getdents da/dc/dd/d3e/d26/d88/dd2/d73/d78 0 2024-03-04T13:43:17.675 INFO:tasks.workunit.client.0.smithi107.stdout:0/751: dread f0 [0,4194304] 0 2024-03-04T13:43:17.675 INFO:tasks.workunit.client.0.smithi107.stdout:5/767: dwrite d5/d2b/ff7 [0,4194304] 0 2024-03-04T13:43:17.677 INFO:tasks.workunit.client.0.smithi107.stdout:6/725: dwrite df/d38/d4f/fcf [0,4194304] 0 2024-03-04T13:43:17.678 INFO:tasks.workunit.client.0.smithi107.stdout:0/752: chown da/dc/dd/d3e/d26/d88/lcd 63 1 2024-03-04T13:43:17.679 INFO:tasks.workunit.client.0.smithi107.stdout:2/662: write da/df/d27/d58/d5f/f8c [426215,19887] 0 2024-03-04T13:43:17.688 INFO:tasks.workunit.client.0.smithi107.stdout:9/688: getdents d4/d34 0 2024-03-04T13:43:17.690 INFO:tasks.workunit.client.0.smithi107.stdout:6/726: truncate df/d17/d24/d40/d63/da3/d7e/fb8 370847 0 2024-03-04T13:43:17.691 INFO:tasks.workunit.client.0.smithi107.stdout:7/674: unlink d6/d1d/d36/d59/fa2 0 2024-03-04T13:43:17.697 INFO:tasks.workunit.client.0.smithi107.stdout:3/680: dwrite d1/de/d3f/f78 [0,4194304] 0 2024-03-04T13:43:17.702 INFO:tasks.workunit.client.0.smithi107.stdout:2/663: creat da/d29/d7c/dbd/fe2 x:0 0 0 2024-03-04T13:43:17.703 INFO:tasks.workunit.client.0.smithi107.stdout:7/675: stat d6/d13/d16/d19/d60 0 2024-03-04T13:43:17.704 INFO:tasks.workunit.client.0.smithi107.stdout:0/753: creat da/dc/dd/dc9/de5/f100 x:0 0 0 2024-03-04T13:43:17.705 INFO:tasks.workunit.client.0.smithi107.stdout:5/768: dread d5/d8/d2a/d43/fbb [0,4194304] 0 2024-03-04T13:43:17.707 INFO:tasks.workunit.client.0.smithi107.stdout:1/733: dwrite d1/f31 [0,4194304] 0 2024-03-04T13:43:17.707 INFO:tasks.workunit.client.0.smithi107.stdout:4/662: dread d0/d9/d10/d80/db8/d21/f2c [0,4194304] 0 2024-03-04T13:43:17.708 INFO:tasks.workunit.client.0.smithi107.stdout:6/727: mkdir df/d17/d6b/da8/de6 0 2024-03-04T13:43:17.709 INFO:tasks.workunit.client.0.smithi107.stdout:8/719: write d6/d16/d59/d7c/f8a [2722038,31583] 0 2024-03-04T13:43:17.717 INFO:tasks.workunit.client.0.smithi107.stdout:8/720: stat d6/da/d25/d2a/f75 0 2024-03-04T13:43:17.721 INFO:tasks.workunit.client.0.smithi107.stdout:2/664: mkdir da/df/d3e/de3 0 2024-03-04T13:43:17.724 INFO:tasks.workunit.client.0.smithi107.stdout:3/681: mknod d1/de/d18/d33/d4d/d4f/d69/cd8 0 2024-03-04T13:43:17.724 INFO:tasks.workunit.client.0.smithi107.stdout:4/663: rmdir d0/d9/d78/dc3 39 2024-03-04T13:43:17.725 INFO:tasks.workunit.client.0.smithi107.stdout:6/728: write df/d17/d24/d30/d8b/f87 [615407,80580] 0 2024-03-04T13:43:17.728 INFO:tasks.workunit.client.0.smithi107.stdout:7/676: creat d6/d1d/d64/fde x:0 0 0 2024-03-04T13:43:17.739 INFO:tasks.workunit.client.0.smithi107.stdout:6/729: write df/d2e/f5a [3484839,118517] 0 2024-03-04T13:43:17.740 INFO:tasks.workunit.client.0.smithi107.stdout:5/769: dwrite d5/d8/d2a/d62/fc8 [0,4194304] 0 2024-03-04T13:43:17.743 INFO:tasks.workunit.client.0.smithi107.stdout:1/734: link d1/d10/d16/d47/c56 d1/d46/d9f/cee 0 2024-03-04T13:43:17.746 INFO:tasks.workunit.client.0.smithi107.stdout:8/721: creat d6/d16/d1f/fe5 x:0 0 0 2024-03-04T13:43:17.747 INFO:tasks.workunit.client.0.smithi107.stdout:9/689: dread d4/da/d11/d20/d2f/d9c/fb4 [0,4194304] 0 2024-03-04T13:43:17.748 INFO:tasks.workunit.client.0.smithi107.stdout:8/722: fsync d6/d16/fcf 0 2024-03-04T13:43:17.749 INFO:tasks.workunit.client.0.smithi107.stdout:9/690: fdatasync d4/da/d11/d20/fd5 0 2024-03-04T13:43:17.754 INFO:tasks.workunit.client.0.smithi107.stdout:7/677: rename d6/d13/d16/d19/l3c to d6/d13/d16/d19/d38/d2c/d8d/ldf 0 2024-03-04T13:43:17.756 INFO:tasks.workunit.client.0.smithi107.stdout:7/678: write d6/d13/d16/fbb [2590571,94495] 0 2024-03-04T13:43:17.761 INFO:tasks.workunit.client.0.smithi107.stdout:0/754: truncate da/dc/dd/d1e/fb5 326495 0 2024-03-04T13:43:17.766 INFO:tasks.workunit.client.0.smithi107.stdout:4/664: dwrite d0/d9/f6b [0,4194304] 0 2024-03-04T13:43:17.766 INFO:tasks.workunit.client.0.smithi107.stdout:0/755: read da/dc/fbb [935392,45458] 0 2024-03-04T13:43:17.768 INFO:tasks.workunit.client.0.smithi107.stdout:8/723: creat d6/da/d25/dcb/fe6 x:0 0 0 2024-03-04T13:43:17.775 INFO:tasks.workunit.client.0.smithi107.stdout:2/665: dwrite da/df/f13 [0,4194304] 0 2024-03-04T13:43:17.779 INFO:tasks.workunit.client.0.smithi107.stdout:4/665: mkdir d0/d9/d10/d80/db8/d97/dcd 0 2024-03-04T13:43:17.782 INFO:tasks.workunit.client.0.smithi107.stdout:1/735: dread d1/d10/d16/fd9 [0,4194304] 0 2024-03-04T13:43:17.783 INFO:tasks.workunit.client.0.smithi107.stdout:8/724: mknod d6/da/da8/db5/ce7 0 2024-03-04T13:43:17.787 INFO:tasks.workunit.client.0.smithi107.stdout:9/691: dwrite d4/f8 [0,4194304] 0 2024-03-04T13:43:17.789 INFO:tasks.workunit.client.0.smithi107.stdout:0/756: write da/f81 [14198,7358] 0 2024-03-04T13:43:17.790 INFO:tasks.workunit.client.0.smithi107.stdout:8/725: truncate d6/d16/d59/d3e/fe2 880118 0 2024-03-04T13:43:17.792 INFO:tasks.workunit.client.0.smithi107.stdout:9/692: chown d4/da/d11/d20/d39/f7a 340118 1 2024-03-04T13:43:17.795 INFO:tasks.workunit.client.0.smithi107.stdout:2/666: dread da/d29/f69 [0,4194304] 0 2024-03-04T13:43:17.796 INFO:tasks.workunit.client.0.smithi107.stdout:7/679: mknod d6/d1d/d64/d9a/da9/d71/ce0 0 2024-03-04T13:43:17.799 INFO:tasks.workunit.client.0.smithi107.stdout:5/770: truncate d5/d66/f81 2532781 0 2024-03-04T13:43:17.801 INFO:tasks.workunit.client.0.smithi107.stdout:0/757: truncate f0 221065 0 2024-03-04T13:43:17.803 INFO:tasks.workunit.client.0.smithi107.stdout:6/730: link df/d17/d24/d30/d3b/l4c df/d17/d24/d8d/le7 0 2024-03-04T13:43:17.804 INFO:tasks.workunit.client.0.smithi107.stdout:0/758: chown da/dc/d34/c6d 1792 1 2024-03-04T13:43:17.806 INFO:tasks.workunit.client.0.smithi107.stdout:0/759: chown da/dc/fc4 873688 1 2024-03-04T13:43:17.808 INFO:tasks.workunit.client.0.smithi107.stdout:4/666: write d0/d9/d18/d29/f55 [1246299,94667] 0 2024-03-04T13:43:17.808 INFO:tasks.workunit.client.0.smithi107.stdout:9/693: creat d4/da/d11/d20/d2f/fe6 x:0 0 0 2024-03-04T13:43:17.810 INFO:tasks.workunit.client.0.smithi107.stdout:5/771: mknod d5/d79/d89/da3/d69/c113 0 2024-03-04T13:43:17.813 INFO:tasks.workunit.client.0.smithi107.stdout:0/760: mknod da/dc/dd/d3e/d26/d69/c101 0 2024-03-04T13:43:17.815 INFO:tasks.workunit.client.0.smithi107.stdout:0/761: chown da/dc/dd/dc9/de5/lb7 3016120 1 2024-03-04T13:43:17.815 INFO:tasks.workunit.client.0.smithi107.stdout:4/667: symlink d0/d9/d10/d80/db8/d97/lce 0 2024-03-04T13:43:17.818 INFO:tasks.workunit.client.0.smithi107.stdout:2/667: creat da/d44/fe4 x:0 0 0 2024-03-04T13:43:17.821 INFO:tasks.workunit.client.0.smithi107.stdout:6/731: symlink df/d17/d24/d40/d63/da3/dab/le8 0 2024-03-04T13:43:17.823 INFO:tasks.workunit.client.0.smithi107.stdout:8/726: truncate d6/f3d 661843 0 2024-03-04T13:43:17.824 INFO:tasks.workunit.client.0.smithi107.stdout:9/694: mkdir d4/da/d11/d20/d64/d77/de7 0 2024-03-04T13:43:17.828 INFO:tasks.workunit.client.0.smithi107.stdout:1/736: dwrite d1/d46/d6a/d4e/daa/dc9/f32 [0,4194304] 0 2024-03-04T13:43:17.830 INFO:tasks.workunit.client.0.smithi107.stdout:6/732: mknod df/d17/d24/d40/d63/da3/dab/ce9 0 2024-03-04T13:43:17.838 INFO:tasks.workunit.client.0.smithi107.stdout:7/680: truncate d6/d1d/d36/f81 103113 0 2024-03-04T13:43:17.838 INFO:tasks.workunit.client.0.smithi107.stdout:4/668: mkdir d0/d9/d78/dcf 0 2024-03-04T13:43:17.842 INFO:tasks.workunit.client.0.smithi107.stdout:4/669: chown d0/d9/d1e/d70/cb9 51 1 2024-03-04T13:43:17.845 INFO:tasks.workunit.client.0.smithi107.stdout:2/668: write da/d44/f64 [951612,76335] 0 2024-03-04T13:43:17.847 INFO:tasks.workunit.client.0.smithi107.stdout:6/733: readlink df/d17/d24/d40/d63/da3/l50 0 2024-03-04T13:43:17.848 INFO:tasks.workunit.client.0.smithi107.stdout:4/670: fsync d0/d9/d10/d80/db8/d21/f5d 0 2024-03-04T13:43:17.851 INFO:tasks.workunit.client.0.smithi107.stdout:5/772: dread d5/d8/d14/d2d/f56 [0,4194304] 0 2024-03-04T13:43:17.854 INFO:tasks.workunit.client.0.smithi107.stdout:1/737: creat d1/d46/d6a/d4e/daa/dc9/fef x:0 0 0 2024-03-04T13:43:17.858 INFO:tasks.workunit.client.0.smithi107.stdout:8/727: getdents d6/d16/d1f/d4c/da4/daa 0 2024-03-04T13:43:17.861 INFO:tasks.workunit.client.0.smithi107.stdout:7/681: write d6/d1d/d36/d59/f8a [850011,105215] 0 2024-03-04T13:43:17.862 INFO:tasks.workunit.client.0.smithi107.stdout:9/695: dwrite d4/da/d11/d65/f58 [0,4194304] 0 2024-03-04T13:43:17.862 INFO:tasks.workunit.client.0.smithi107.stdout:8/728: write d6/d16/d59/d3e/fb3 [985170,12897] 0 2024-03-04T13:43:17.864 INFO:tasks.workunit.client.0.smithi107.stdout:0/762: dwrite da/dc/f4e [0,4194304] 0 2024-03-04T13:43:17.868 INFO:tasks.workunit.client.0.smithi107.stdout:9/696: chown d4/da/d11/d4c/d51/c5b 808 1 2024-03-04T13:43:17.874 INFO:tasks.workunit.client.0.smithi107.stdout:2/669: creat da/df/d73/dd1/ddb/fe5 x:0 0 0 2024-03-04T13:43:17.879 INFO:tasks.workunit.client.0.smithi107.stdout:9/697: unlink d4/da/d11/d65/f58 0 2024-03-04T13:43:17.881 INFO:tasks.workunit.client.0.smithi107.stdout:5/773: creat d5/d8/d14/d2d/d92/dd8/f114 x:0 0 0 2024-03-04T13:43:17.882 INFO:tasks.workunit.client.0.smithi107.stdout:8/729: mknod d6/d16/d1f/ce8 0 2024-03-04T13:43:17.885 INFO:tasks.workunit.client.0.smithi107.stdout:4/671: write d0/d9/d18/fa9 [46317,100832] 0 2024-03-04T13:43:17.885 INFO:tasks.workunit.client.0.smithi107.stdout:6/734: dwrite df/d17/d6b/d9b/d61/f8a [0,4194304] 0 2024-03-04T13:43:17.885 INFO:tasks.workunit.client.0.smithi107.stdout:0/763: creat da/f102 x:0 0 0 2024-03-04T13:43:17.892 INFO:tasks.workunit.client.0.smithi107.stdout:5/774: write d5/d79/d89/da3/d6a/f8e [2583521,123646] 0 2024-03-04T13:43:17.893 INFO:tasks.workunit.client.0.smithi107.stdout:1/738: dwrite d1/d46/d6a/d4e/daa/dc9/d33/d5d/d24/f62 [4194304,4194304] 0 2024-03-04T13:43:17.900 INFO:tasks.workunit.client.0.smithi107.stdout:8/730: unlink d6/da/c8e 0 2024-03-04T13:43:17.908 INFO:tasks.workunit.client.0.smithi107.stdout:5/775: symlink d5/d79/d89/da3/d68/l115 0 2024-03-04T13:43:17.911 INFO:tasks.workunit.client.0.smithi107.stdout:6/735: symlink df/d17/lea 0 2024-03-04T13:43:17.911 INFO:tasks.workunit.client.0.smithi107.stdout:1/739: chown d1/d46/d6a/d65/db1/cc0 18 1 2024-03-04T13:43:17.912 INFO:tasks.workunit.client.0.smithi107.stdout:0/764: creat da/dc/dd/d3e/d26/d69/df9/f103 x:0 0 0 2024-03-04T13:43:17.914 INFO:tasks.workunit.client.0.smithi107.stdout:7/682: link d6/d13/d16/d1c/l7e d6/d1d/d36/d59/le1 0 2024-03-04T13:43:17.915 INFO:tasks.workunit.client.0.smithi107.stdout:6/736: dread - df/d2e/d36/d77/d95/da7/fcb zero size 2024-03-04T13:43:17.918 INFO:tasks.workunit.client.0.smithi107.stdout:5/776: rmdir d5/d8/d14/d2d 39 2024-03-04T13:43:17.919 INFO:tasks.workunit.client.0.smithi107.stdout:6/737: chown df/d17/d6b/d9b/l32 1444 1 2024-03-04T13:43:17.920 INFO:tasks.workunit.client.0.smithi107.stdout:2/670: dwrite da/df/d27/d58/d5f/f93 [0,4194304] 0 2024-03-04T13:43:17.920 INFO:tasks.workunit.client.0.smithi107.stdout:9/698: getdents d4/da/d11 0 2024-03-04T13:43:17.921 INFO:tasks.workunit.client.0.smithi107.stdout:5/777: fdatasync d5/d8/d2a/dd6/ffc 0 2024-03-04T13:43:17.922 INFO:tasks.workunit.client.0.smithi107.stdout:6/738: chown df/d17/d24/d30/d8b/f8f 94582688 1 2024-03-04T13:43:17.923 INFO:tasks.workunit.client.0.smithi107.stdout:5/778: chown d5/d79/d89/da3/d68/f97 3 1 2024-03-04T13:43:17.924 INFO:tasks.workunit.client.0.smithi107.stdout:6/739: write df/d38/d4f/fcf [4152743,113444] 0 2024-03-04T13:43:17.924 INFO:tasks.workunit.client.0.smithi107.stdout:5/779: chown d5/d8/d2a 35365035 1 2024-03-04T13:43:17.926 INFO:tasks.workunit.client.0.smithi107.stdout:5/780: readlink d5/dc/l18 0 2024-03-04T13:43:17.926 INFO:tasks.workunit.client.0.smithi107.stdout:6/740: stat df/d38/d4f/daf/ld0 0 2024-03-04T13:43:17.926 INFO:tasks.workunit.client.0.smithi107.stdout:7/683: creat d6/d13/d16/fe2 x:0 0 0 2024-03-04T13:43:17.928 INFO:tasks.workunit.client.0.smithi107.stdout:0/765: creat da/dc/dd/d3e/d26/d88/dd2/d73/d78/f104 x:0 0 0 2024-03-04T13:43:17.929 INFO:tasks.workunit.client.0.smithi107.stdout:5/781: write d5/d8/d2a/d62/fc8 [777257,58656] 0 2024-03-04T13:43:17.929 INFO:tasks.workunit.client.0.smithi107.stdout:6/741: write df/d2e/d36/d77/f90 [4411865,104111] 0 2024-03-04T13:43:17.931 INFO:tasks.workunit.client.0.smithi107.stdout:7/684: write d6/d1d/d64/d9a/da9/d71/fcc [5121314,8511] 0 2024-03-04T13:43:17.935 INFO:tasks.workunit.client.0.smithi107.stdout:8/731: rmdir d6/d16/d59/d3e/d67/dd7 0 2024-03-04T13:43:17.941 INFO:tasks.workunit.client.0.smithi107.stdout:8/732: read d6/da/d25/d2a/f95 [3323763,69822] 0 2024-03-04T13:43:17.943 INFO:tasks.workunit.client.0.smithi107.stdout:2/671: dread da/df/d27/f5e [0,4194304] 0 2024-03-04T13:43:17.943 INFO:tasks.workunit.client.0.smithi107.stdout:6/742: mkdir df/d17/d24/d30/d3b/deb 0 2024-03-04T13:43:17.945 INFO:tasks.workunit.client.0.smithi107.stdout:9/699: mkdir d4/da/d11/d20/d23/de8 0 2024-03-04T13:43:17.954 INFO:tasks.workunit.client.0.smithi107.stdout:0/766: getdents da/dc/dd/d3e/d26/d69 0 2024-03-04T13:43:17.954 INFO:tasks.workunit.client.0.smithi107.stdout:6/743: mknod df/d17/d6b/d9b/d61/d99/cec 0 2024-03-04T13:43:17.958 INFO:tasks.workunit.client.0.smithi107.stdout:9/700: rename d4/d34/fde to d4/da/d11/d20/d64/d77/fe9 0 2024-03-04T13:43:17.958 INFO:tasks.workunit.client.0.smithi107.stdout:1/740: truncate d1/d46/d6a/d4e/d60/f4f 2067904 0 2024-03-04T13:43:17.959 INFO:tasks.workunit.client.0.smithi107.stdout:5/782: link d5/d79/d89/ldd d5/dc/l116 0 2024-03-04T13:43:17.962 INFO:tasks.workunit.client.0.smithi107.stdout:8/733: dwrite d6/d16/f27 [0,4194304] 0 2024-03-04T13:43:17.964 INFO:tasks.workunit.client.0.smithi107.stdout:6/744: rmdir df/d2e 39 2024-03-04T13:43:17.968 INFO:tasks.workunit.client.0.smithi107.stdout:2/672: dwrite da/df/d73/db3/db1/fc1 [0,4194304] 0 2024-03-04T13:43:17.969 INFO:tasks.workunit.client.0.smithi107.stdout:7/685: dread d6/d13/d16/d1c/f79 [0,4194304] 0 2024-03-04T13:43:17.971 INFO:tasks.workunit.client.0.smithi107.stdout:0/767: symlink da/dc/dd/d1e/dbf/deb/l105 0 2024-03-04T13:43:17.974 INFO:tasks.workunit.client.0.smithi107.stdout:6/745: write df/d17/d24/d30/fc4 [1262817,98874] 0 2024-03-04T13:43:17.976 INFO:tasks.workunit.client.0.smithi107.stdout:5/783: dread - d5/d8/d14/d2d/d92/dc1/feb zero size 2024-03-04T13:43:17.977 INFO:tasks.workunit.client.0.smithi107.stdout:6/746: readlink df/d57/l91 0 2024-03-04T13:43:17.978 INFO:tasks.workunit.client.0.smithi107.stdout:9/701: rename d4/da/d11/d20/d23/d7e/cba to d4/da/d11/d20/d23/d7e/d84/cea 0 2024-03-04T13:43:17.980 INFO:tasks.workunit.client.0.smithi107.stdout:2/673: mknod da/df/d73/d7a/d9a/d68/ce6 0 2024-03-04T13:43:17.983 INFO:tasks.workunit.client.0.smithi107.stdout:9/702: fdatasync d4/da/d11/d20/d23/dd9/fa9 0 2024-03-04T13:43:17.983 INFO:tasks.workunit.client.0.smithi107.stdout:6/747: stat df/d17/d24/d30/d5d/d6a/f94 0 2024-03-04T13:43:17.988 INFO:tasks.workunit.client.0.smithi107.stdout:8/734: getdents d6/d16/d1f/d4c/da5 0 2024-03-04T13:43:17.990 INFO:tasks.workunit.client.0.smithi107.stdout:0/768: rename da/dc/dd/d3e/l4f to da/dc/dd/l106 0 2024-03-04T13:43:17.998 INFO:tasks.workunit.client.0.smithi107.stdout:9/703: creat d4/da/d11/d20/d64/d77/de7/feb x:0 0 0 2024-03-04T13:43:17.999 INFO:tasks.workunit.client.0.smithi107.stdout:1/741: truncate d1/d10/d16/d1b/d40/f57 1496601 0 2024-03-04T13:43:18.000 INFO:tasks.workunit.client.0.smithi107.stdout:3/682: sync 2024-03-04T13:43:18.000 INFO:tasks.workunit.client.0.smithi107.stdout:5/784: write d5/dc/f3d [875774,77066] 0 2024-03-04T13:43:18.002 INFO:tasks.workunit.client.0.smithi107.stdout:1/742: stat d1/d46/d6a/d17/c2f 0 2024-03-04T13:43:18.002 INFO:tasks.workunit.client.0.smithi107.stdout:3/683: stat d1/d2d/f44 0 2024-03-04T13:43:18.004 INFO:tasks.workunit.client.0.smithi107.stdout:7/686: dread d6/d13/d14/fa4 [4194304,4194304] 0 2024-03-04T13:43:18.005 INFO:tasks.workunit.client.0.smithi107.stdout:1/743: write d1/f94 [2425105,106794] 0 2024-03-04T13:43:18.007 INFO:tasks.workunit.client.0.smithi107.stdout:7/687: dread - d6/d1d/fc8 zero size 2024-03-04T13:43:18.013 INFO:tasks.workunit.client.0.smithi107.stdout:1/744: dread d1/d46/d6a/d4e/daa/dc9/d33/d5d/d4c/f93 [0,4194304] 0 2024-03-04T13:43:18.014 INFO:tasks.workunit.client.0.smithi107.stdout:2/674: dwrite da/d44/fe4 [0,4194304] 0 2024-03-04T13:43:18.022 INFO:tasks.workunit.client.0.smithi107.stdout:6/748: symlink df/d2e/d36/d77/d95/da7/led 0 2024-03-04T13:43:18.024 INFO:tasks.workunit.client.0.smithi107.stdout:0/769: mkdir da/dc/dd/dc9/de5/d107 0 2024-03-04T13:43:18.026 INFO:tasks.workunit.client.0.smithi107.stdout:0/770: read - da/dc/dd/dc9/de5/fec zero size 2024-03-04T13:43:18.027 INFO:tasks.workunit.client.0.smithi107.stdout:9/704: unlink d4/da/d11/d20/d2f/l62 0 2024-03-04T13:43:18.033 INFO:tasks.workunit.client.0.smithi107.stdout:7/688: dread - d6/d1d/d64/d9a/da9/d71/db9/dca/d22/fb3 zero size 2024-03-04T13:43:18.037 INFO:tasks.workunit.client.0.smithi107.stdout:6/749: write df/d2e/d36/fa2 [3232759,19590] 0 2024-03-04T13:43:18.040 INFO:tasks.workunit.client.0.smithi107.stdout:0/771: symlink da/dc/dd/d3e/d26/d88/dd2/l108 0 2024-03-04T13:43:18.041 INFO:tasks.workunit.client.0.smithi107.stdout:9/705: mkdir d4/da/d11/d20/d2f/dec 0 2024-03-04T13:43:18.043 INFO:tasks.workunit.client.0.smithi107.stdout:9/706: fdatasync d4/da/d11/d4c/f3c 0 2024-03-04T13:43:18.045 INFO:tasks.workunit.client.0.smithi107.stdout:5/785: rename d5/d8/d14/d2d/df1/l8c to d5/d2b/d4b/da6/dcf/l117 0 2024-03-04T13:43:18.045 INFO:tasks.workunit.client.0.smithi107.stdout:9/707: chown d4/da/d11/d20/d64/f76 0 1 2024-03-04T13:43:18.050 INFO:tasks.workunit.client.0.smithi107.stdout:5/786: read d5/d79/d89/da3/d6a/f8e [2922155,114930] 0 2024-03-04T13:43:18.051 INFO:tasks.workunit.client.0.smithi107.stdout:7/689: unlink d6/d1d/d64/d9a/da9/d71/db9/dca/d3d/f43 0 2024-03-04T13:43:18.052 INFO:tasks.workunit.client.0.smithi107.stdout:6/750: creat df/d17/d5c/db1/fee x:0 0 0 2024-03-04T13:43:18.053 INFO:tasks.workunit.client.0.smithi107.stdout:5/787: write d5/d8/d14/d2d/d92/fca [146133,46713] 0 2024-03-04T13:43:18.054 INFO:tasks.workunit.client.0.smithi107.stdout:5/788: stat d5/d8/l27 0 2024-03-04T13:43:18.060 INFO:tasks.workunit.client.0.smithi107.stdout:6/751: unlink df/d2e/ce5 0 2024-03-04T13:43:18.062 INFO:tasks.workunit.client.0.smithi107.stdout:0/772: creat da/dad/f109 x:0 0 0 2024-03-04T13:43:18.062 INFO:tasks.workunit.client.0.smithi107.stdout:3/684: link d1/de/d18/c28 d1/de/d18/d33/d48/d71/d4c/d54/db7/cd9 0 2024-03-04T13:43:18.067 INFO:tasks.workunit.client.0.smithi107.stdout:5/789: mknod d5/d8/da/db0/c118 0 2024-03-04T13:43:18.069 INFO:tasks.workunit.client.0.smithi107.stdout:5/790: write d5/d8/da/f16 [2699417,37424] 0 2024-03-04T13:43:18.070 INFO:tasks.workunit.client.0.smithi107.stdout:2/675: truncate da/df/fa2 2688488 0 2024-03-04T13:43:18.071 INFO:tasks.workunit.client.0.smithi107.stdout:6/752: mknod df/d17/d24/d30/d8b/cef 0 2024-03-04T13:43:18.072 INFO:tasks.workunit.client.0.smithi107.stdout:1/745: truncate d1/d10/d16/fad 279366 0 2024-03-04T13:43:18.074 INFO:tasks.workunit.client.0.smithi107.stdout:3/685: mkdir d1/d29/dda 0 2024-03-04T13:43:18.075 INFO:tasks.workunit.client.0.smithi107.stdout:6/753: truncate df/d17/d24/d30/d5d/d6a/fe3 324534 0 2024-03-04T13:43:18.078 INFO:tasks.workunit.client.0.smithi107.stdout:7/690: creat d6/d13/d16/d19/fe3 x:0 0 0 2024-03-04T13:43:18.082 INFO:tasks.workunit.client.0.smithi107.stdout:5/791: truncate d5/d8/d2a/d43/fbb 1025673 0 2024-03-04T13:43:18.083 INFO:tasks.workunit.client.0.smithi107.stdout:9/708: write d4/da/d11/d4c/fbe [536920,93086] 0 2024-03-04T13:43:18.083 INFO:tasks.workunit.client.0.smithi107.stdout:1/746: fsync d1/d46/d6a/d4e/daa/dc9/f3a 0 2024-03-04T13:43:18.087 INFO:tasks.workunit.client.0.smithi107.stdout:6/754: symlink df/d17/d5c/lf0 0 2024-03-04T13:43:18.088 INFO:tasks.workunit.client.0.smithi107.stdout:1/747: dread - d1/d46/d6a/d4e/daa/dc9/d33/d89/fcb zero size 2024-03-04T13:43:18.090 INFO:tasks.workunit.client.0.smithi107.stdout:1/748: readlink d1/d46/d6a/d4e/daa/dc9/d67/l8e 0 2024-03-04T13:43:18.093 INFO:tasks.workunit.client.0.smithi107.stdout:9/709: unlink d4/da/d11/d20/d23/f41 0 2024-03-04T13:43:18.097 INFO:tasks.workunit.client.0.smithi107.stdout:2/676: rename da/d29/f4e to da/d49/fe7 0 2024-03-04T13:43:18.101 INFO:tasks.workunit.client.0.smithi107.stdout:6/755: read df/f29 [2136418,14314] 0 2024-03-04T13:43:18.102 INFO:tasks.workunit.client.0.smithi107.stdout:5/792: link d5/dc/f58 d5/d2b/d4b/da6/f119 0 2024-03-04T13:43:18.105 INFO:tasks.workunit.client.0.smithi107.stdout:8/735: sync 2024-03-04T13:43:18.106 INFO:tasks.workunit.client.0.smithi107.stdout:4/672: sync 2024-03-04T13:43:18.106 INFO:tasks.workunit.client.0.smithi107.stdout:6/756: write df/d2e/d36/d71/fca [150785,118912] 0 2024-03-04T13:43:18.106 INFO:tasks.workunit.client.0.smithi107.stdout:3/686: rename d1/cb to d1/d41/cdb 0 2024-03-04T13:43:18.106 INFO:tasks.workunit.client.0.smithi107.stdout:2/677: mknod da/df/d73/db3/ce8 0 2024-03-04T13:43:18.110 INFO:tasks.workunit.client.0.smithi107.stdout:0/773: truncate da/dc/f7c 2511747 0 2024-03-04T13:43:18.111 INFO:tasks.workunit.client.0.smithi107.stdout:6/757: read df/d2e/d36/d77/f84 [229308,95268] 0 2024-03-04T13:43:18.112 INFO:tasks.workunit.client.0.smithi107.stdout:1/749: dwrite d1/d46/d6a/d4e/daa/dc9/d67/d6b/f84 [0,4194304] 0 2024-03-04T13:43:18.112 INFO:tasks.workunit.client.0.smithi107.stdout:5/793: truncate d5/d8/d14/d2d/d92/dc1/dcc/f10e 639810 0 2024-03-04T13:43:18.113 INFO:tasks.workunit.client.0.smithi107.stdout:8/736: truncate d6/da/d25/dd1/fd2 967334 0 2024-03-04T13:43:18.113 INFO:tasks.workunit.client.0.smithi107.stdout:7/691: getdents d6/d1d/d64/d9a/da9/d71/db9/dca 0 2024-03-04T13:43:18.114 INFO:tasks.workunit.client.0.smithi107.stdout:5/794: chown d5/d2b/c98 3339986 1 2024-03-04T13:43:18.116 INFO:tasks.workunit.client.0.smithi107.stdout:2/678: write da/d39/f46 [4697663,89669] 0 2024-03-04T13:43:18.118 INFO:tasks.workunit.client.0.smithi107.stdout:8/737: write d6/d16/d59/d7c/f92 [372046,4688] 0 2024-03-04T13:43:18.121 INFO:tasks.workunit.client.0.smithi107.stdout:8/738: fdatasync d6/da/d25/d5f/f87 0 2024-03-04T13:43:18.122 INFO:tasks.workunit.client.0.smithi107.stdout:4/673: mkdir d0/d9/d10/d80/db8/d97/dd0 0 2024-03-04T13:43:18.123 INFO:tasks.workunit.client.0.smithi107.stdout:8/739: chown d6/c18 228842 1 2024-03-04T13:43:18.127 INFO:tasks.workunit.client.0.smithi107.stdout:8/740: read d6/d16/d1f/f65 [124747,89432] 0 2024-03-04T13:43:18.131 INFO:tasks.workunit.client.0.smithi107.stdout:0/774: creat da/dc/dd/ddb/f10a x:0 0 0 2024-03-04T13:43:18.134 INFO:tasks.workunit.client.0.smithi107.stdout:0/775: write da/dad/f109 [836156,68567] 0 2024-03-04T13:43:18.134 INFO:tasks.workunit.client.0.smithi107.stdout:1/750: dwrite d1/d46/d6a/d4e/f6c [0,4194304] 0 2024-03-04T13:43:18.137 INFO:tasks.workunit.client.0.smithi107.stdout:9/710: truncate d4/da/d11/d20/d2f/d9c/fb4 2039996 0 2024-03-04T13:43:18.140 INFO:tasks.workunit.client.0.smithi107.stdout:3/687: mknod d1/de/d18/cdc 0 2024-03-04T13:43:18.143 INFO:tasks.workunit.client.0.smithi107.stdout:5/795: rename d5/d8/d2a/d43/cff to d5/dc/d22/c11a 0 2024-03-04T13:43:18.147 INFO:tasks.workunit.client.0.smithi107.stdout:9/711: truncate d4/da/d11/d4c/d51/fe2 376691 0 2024-03-04T13:43:18.149 INFO:tasks.workunit.client.0.smithi107.stdout:6/758: mkdir df/d17/d24/df1 0 2024-03-04T13:43:18.152 INFO:tasks.workunit.client.0.smithi107.stdout:8/741: fsync d6/d16/d1f/d63/f8b 0 2024-03-04T13:43:18.153 INFO:tasks.workunit.client.0.smithi107.stdout:1/751: dwrite d1/d46/fd0 [0,4194304] 0 2024-03-04T13:43:18.156 INFO:tasks.workunit.client.0.smithi107.stdout:2/679: dread da/f1c [0,4194304] 0 2024-03-04T13:43:18.158 INFO:tasks.workunit.client.0.smithi107.stdout:3/688: creat d1/de/d18/d33/d48/d93/fdd x:0 0 0 2024-03-04T13:43:18.159 INFO:tasks.workunit.client.0.smithi107.stdout:5/796: symlink d5/d79/l11b 0 2024-03-04T13:43:18.161 INFO:tasks.workunit.client.0.smithi107.stdout:2/680: readlink da/d38/ld5 0 2024-03-04T13:43:18.168 INFO:tasks.workunit.client.0.smithi107.stdout:7/692: dread d6/f3f [0,4194304] 0 2024-03-04T13:43:18.173 INFO:tasks.workunit.client.0.smithi107.stdout:8/742: dwrite d6/f54 [0,4194304] 0 2024-03-04T13:43:18.175 INFO:tasks.workunit.client.0.smithi107.stdout:6/759: creat df/d17/d6b/d9b/d61/d99/ff2 x:0 0 0 2024-03-04T13:43:18.179 INFO:tasks.workunit.client.0.smithi107.stdout:1/752: dwrite d1/d46/d6a/d4e/daa/dc9/d67/d6b/f84 [0,4194304] 0 2024-03-04T13:43:18.182 INFO:tasks.workunit.client.0.smithi107.stdout:4/674: creat d0/d9/fd1 x:0 0 0 2024-03-04T13:43:18.182 INFO:tasks.workunit.client.0.smithi107.stdout:2/681: mkdir da/df/d27/d58/d5f/de9 0 2024-03-04T13:43:18.187 INFO:tasks.workunit.client.0.smithi107.stdout:0/776: link da/dc/dd/d3e/d26/l61 da/dc/dd/d1e/dbf/deb/l10b 0 2024-03-04T13:43:18.187 INFO:tasks.workunit.client.0.smithi107.stdout:2/682: write da/d49/d5c/f9e [3355798,105216] 0 2024-03-04T13:43:18.189 INFO:tasks.workunit.client.0.smithi107.stdout:0/777: readlink da/dc/ldd 0 2024-03-04T13:43:18.193 INFO:tasks.workunit.client.0.smithi107.stdout:8/743: fdatasync d6/da/f70 0 2024-03-04T13:43:18.198 INFO:tasks.workunit.client.0.smithi107.stdout:3/689: dwrite d1/de/d4b/fb8 [0,4194304] 0 2024-03-04T13:43:18.206 INFO:tasks.workunit.client.0.smithi107.stdout:5/797: link d5/d8/d14/d2d/d92/dc1/dcc/f10e d5/d8/da/db0/de6/f11c 0 2024-03-04T13:43:18.207 INFO:tasks.workunit.client.0.smithi107.stdout:7/693: mkdir d6/d1d/d64/d9a/da9/d71/db9/dca/d3d/dc6/dcb/dd6/de4 0 2024-03-04T13:43:18.208 INFO:tasks.workunit.client.0.smithi107.stdout:5/798: chown d5/d2b/d4b/f110 393 1 2024-03-04T13:43:18.210 INFO:tasks.workunit.client.0.smithi107.stdout:0/778: dwrite da/dc/dd/d3e/d26/d69/de9/fed [0,4194304] 0 2024-03-04T13:43:18.218 INFO:tasks.workunit.client.0.smithi107.stdout:8/744: mkdir d6/d16/d1f/d4c/da5/de9 0 2024-03-04T13:43:18.220 INFO:tasks.workunit.client.0.smithi107.stdout:4/675: mkdir d0/d9/dd2 0 2024-03-04T13:43:18.221 INFO:tasks.workunit.client.0.smithi107.stdout:4/676: fsync d0/d9/d18/f53 0 2024-03-04T13:43:18.224 INFO:tasks.workunit.client.0.smithi107.stdout:1/753: symlink d1/d46/d6a/d4e/daa/dc9/d33/lf0 0 2024-03-04T13:43:18.227 INFO:tasks.workunit.client.0.smithi107.stdout:3/690: read d1/f3 [2797767,118032] 0 2024-03-04T13:43:18.229 INFO:tasks.workunit.client.0.smithi107.stdout:3/691: fsync d1/d1e/f22 0 2024-03-04T13:43:18.229 INFO:tasks.workunit.client.0.smithi107.stdout:5/799: mkdir d5/d8/d14/d2d/d11d 0 2024-03-04T13:43:18.230 INFO:tasks.workunit.client.0.smithi107.stdout:3/692: stat d1/d1e/d26/f6f 0 2024-03-04T13:43:18.230 INFO:tasks.workunit.client.0.smithi107.stdout:7/694: symlink d6/d1d/d64/d9a/da9/d71/db9/dca/d22/le5 0 2024-03-04T13:43:18.237 INFO:tasks.workunit.client.0.smithi107.stdout:6/760: creat df/d17/d6b/d9b/ff3 x:0 0 0 2024-03-04T13:43:18.250 INFO:tasks.workunit.client.0.smithi107.stdout:7/695: dwrite d6/d13/d16/d1c/f1f [8388608,4194304] 0 2024-03-04T13:43:18.252 INFO:tasks.workunit.client.0.smithi107.stdout:5/800: dwrite d5/d8/d14/d2d/d92/fca [0,4194304] 0 2024-03-04T13:43:18.253 INFO:tasks.workunit.client.0.smithi107.stdout:6/761: truncate df/d2e/d36/d5b/fb7 40660 0 2024-03-04T13:43:18.254 INFO:tasks.workunit.client.0.smithi107.stdout:4/677: dread d0/d9/de/d14/f4b [0,4194304] 0 2024-03-04T13:43:18.256 INFO:tasks.workunit.client.0.smithi107.stdout:3/693: mknod d1/d1e/dba/cde 0 2024-03-04T13:43:18.257 INFO:tasks.workunit.client.0.smithi107.stdout:4/678: chown d0/d9/de/d14/l23 25534 1 2024-03-04T13:43:18.260 INFO:tasks.workunit.client.0.smithi107.stdout:8/745: getdents d6/d16/d1f/d4c/da4 0 2024-03-04T13:43:18.263 INFO:tasks.workunit.client.0.smithi107.stdout:2/683: truncate da/d44/f64 287170 0 2024-03-04T13:43:18.265 INFO:tasks.workunit.client.0.smithi107.stdout:4/679: rename d0/l16 to d0/d30/ld3 0 2024-03-04T13:43:18.265 INFO:tasks.workunit.client.0.smithi107.stdout:8/746: chown d6/caf 99572 1 2024-03-04T13:43:18.269 INFO:tasks.workunit.client.0.smithi107.stdout:7/696: getdents d6/d13/d16/d1c/dc2 0 2024-03-04T13:43:18.270 INFO:tasks.workunit.client.0.smithi107.stdout:6/762: mknod df/d1b/cf4 0 2024-03-04T13:43:18.271 INFO:tasks.workunit.client.0.smithi107.stdout:3/694: creat d1/d1e/daf/fdf x:0 0 0 2024-03-04T13:43:18.272 INFO:tasks.workunit.client.0.smithi107.stdout:1/754: truncate d1/d46/d6a/d17/f5a 1174888 0 2024-03-04T13:43:18.273 INFO:tasks.workunit.client.0.smithi107.stdout:2/684: dread da/d49/fa4 [0,4194304] 0 2024-03-04T13:43:18.275 INFO:tasks.workunit.client.0.smithi107.stdout:8/747: rmdir d6/da/da8 39 2024-03-04T13:43:18.277 INFO:tasks.workunit.client.0.smithi107.stdout:3/695: stat d1/de/d18/d33/d48/d71 0 2024-03-04T13:43:18.278 INFO:tasks.workunit.client.0.smithi107.stdout:5/801: dread d5/d8/d14/d2d/d92/fca [0,4194304] 0 2024-03-04T13:43:18.279 INFO:tasks.workunit.client.0.smithi107.stdout:7/697: write d6/d1d/d36/d59/f8a [865828,5182] 0 2024-03-04T13:43:18.279 INFO:tasks.workunit.client.0.smithi107.stdout:6/763: chown df/d17/d6b/d9b/c4a 7404 1 2024-03-04T13:43:18.282 INFO:tasks.workunit.client.0.smithi107.stdout:1/755: chown d1/d46/d6a/d4e/daa/dc9/d33/d5d/ld8 171398 1 2024-03-04T13:43:18.283 INFO:tasks.workunit.client.0.smithi107.stdout:0/779: dwrite da/dc/d34/f45 [0,4194304] 0 2024-03-04T13:43:18.287 INFO:tasks.workunit.client.0.smithi107.stdout:0/780: chown da/dc/dd/d1e/c3d 17248470 1 2024-03-04T13:43:18.291 INFO:tasks.workunit.client.0.smithi107.stdout:9/712: sync 2024-03-04T13:43:18.295 INFO:tasks.workunit.client.0.smithi107.stdout:5/802: creat d5/dc/dc6/f11e x:0 0 0 2024-03-04T13:43:18.299 INFO:tasks.workunit.client.0.smithi107.stdout:3/696: write d1/de/d18/d33/d4d/f90 [403714,20515] 0 2024-03-04T13:43:18.300 INFO:tasks.workunit.client.0.smithi107.stdout:5/803: write d5/d8/d2a/d43/fdb [397758,54828] 0 2024-03-04T13:43:18.302 INFO:tasks.workunit.client.0.smithi107.stdout:5/804: chown d5/d79/d89 20441 1 2024-03-04T13:43:18.303 INFO:tasks.workunit.client.0.smithi107.stdout:2/685: write da/df/fa2 [2923824,74590] 0 2024-03-04T13:43:18.308 INFO:tasks.workunit.client.0.smithi107.stdout:4/680: dwrite d0/d9/de/d14/d66/d7b/f8f [0,4194304] 0 2024-03-04T13:43:18.310 INFO:tasks.workunit.client.0.smithi107.stdout:7/698: rmdir d6/d13/d16/d19/d38/dd1 0 2024-03-04T13:43:18.316 INFO:tasks.workunit.client.0.smithi107.stdout:3/697: unlink d1/de/d18/d33/d6b/f6c 0 2024-03-04T13:43:18.319 INFO:tasks.workunit.client.0.smithi107.stdout:9/713: mknod d4/da/d11/d65/ced 0 2024-03-04T13:43:18.319 INFO:tasks.workunit.client.0.smithi107.stdout:6/764: dwrite df/d17/d24/d30/d8b/f87 [0,4194304] 0 2024-03-04T13:43:18.322 INFO:tasks.workunit.client.0.smithi107.stdout:2/686: truncate da/d34/f72 1042328 0 2024-03-04T13:43:18.325 INFO:tasks.workunit.client.0.smithi107.stdout:1/756: dwrite d1/d46/d6a/d4e/daa/dc9/d33/d5d/d4c/f93 [0,4194304] 0 2024-03-04T13:43:18.325 INFO:tasks.workunit.client.0.smithi107.stdout:6/765: dread - df/d17/d6b/d9b/d61/d99/ff2 zero size 2024-03-04T13:43:18.327 INFO:tasks.workunit.client.0.smithi107.stdout:7/699: write d6/d1d/d64/d9a/da9/fc1 [9056347,74290] 0 2024-03-04T13:43:18.330 INFO:tasks.workunit.client.0.smithi107.stdout:9/714: creat d4/da/d11/d20/d23/dcc/fee x:0 0 0 2024-03-04T13:43:18.332 INFO:tasks.workunit.client.0.smithi107.stdout:5/805: getdents d5/d8/d14/d2d/d11d 0 2024-03-04T13:43:18.332 INFO:tasks.workunit.client.0.smithi107.stdout:1/757: rmdir d1/d46/d6a/d65 39 2024-03-04T13:43:18.337 INFO:tasks.workunit.client.0.smithi107.stdout:0/781: link da/dc/dd/d3e/d26/d88/dd2/d73/d78/fab da/dc/dd/d3e/d26/d69/df9/f10c 0 2024-03-04T13:43:18.340 INFO:tasks.workunit.client.0.smithi107.stdout:4/681: getdents d0/d9/d78 0 2024-03-04T13:43:18.341 INFO:tasks.workunit.client.0.smithi107.stdout:5/806: mknod d5/d66/d96/db1/c11f 0 2024-03-04T13:43:18.345 INFO:tasks.workunit.client.0.smithi107.stdout:1/758: mknod d1/d46/d6a/d4e/daa/dc9/d33/d5d/d4c/dea/d59/cf1 0 2024-03-04T13:43:18.346 INFO:tasks.workunit.client.0.smithi107.stdout:3/698: dwrite d1/d1e/d26/f6f [0,4194304] 0 2024-03-04T13:43:18.347 INFO:tasks.workunit.client.0.smithi107.stdout:0/782: truncate da/dc/d34/ff6 252211 0 2024-03-04T13:43:18.348 INFO:tasks.workunit.client.0.smithi107.stdout:4/682: mknod d0/d9/d10/cd4 0 2024-03-04T13:43:18.350 INFO:tasks.workunit.client.0.smithi107.stdout:3/699: write d1/de/d11/fd4 [1001807,18358] 0 2024-03-04T13:43:18.352 INFO:tasks.workunit.client.0.smithi107.stdout:1/759: mkdir d1/d46/d6a/d4e/df2 0 2024-03-04T13:43:18.352 INFO:tasks.workunit.client.0.smithi107.stdout:0/783: chown da/dc/dd/d1e/fa1 22844702 1 2024-03-04T13:43:18.354 INFO:tasks.workunit.client.0.smithi107.stdout:6/766: dwrite df/d38/d4f/fcf [4194304,4194304] 0 2024-03-04T13:43:18.358 INFO:tasks.workunit.client.0.smithi107.stdout:4/683: mknod d0/d9/d10/d80/db8/cd5 0 2024-03-04T13:43:18.358 INFO:tasks.workunit.client.0.smithi107.stdout:5/807: truncate d5/d8/d2a/d43/f57 359440 0 2024-03-04T13:43:18.359 INFO:tasks.workunit.client.0.smithi107.stdout:3/700: mknod d1/de/d18/d33/d48/d93/ce0 0 2024-03-04T13:43:18.361 INFO:tasks.workunit.client.0.smithi107.stdout:0/784: mkdir da/dc/d34/d10d 0 2024-03-04T13:43:18.363 INFO:tasks.workunit.client.0.smithi107.stdout:6/767: creat df/d17/d24/d30/d5d/ff5 x:0 0 0 2024-03-04T13:43:18.364 INFO:tasks.workunit.client.0.smithi107.stdout:5/808: fsync d5/d79/d89/da3/d6a/f6d 0 2024-03-04T13:43:18.366 INFO:tasks.workunit.client.0.smithi107.stdout:5/809: fsync d5/d66/f78 0 2024-03-04T13:43:18.368 INFO:tasks.workunit.client.0.smithi107.stdout:4/684: readlink d0/d9/de/d14/lc0 0 2024-03-04T13:43:18.370 INFO:tasks.workunit.client.0.smithi107.stdout:4/685: chown d0/d30 223857 1 2024-03-04T13:43:18.370 INFO:tasks.workunit.client.0.smithi107.stdout:1/760: getdents d1/d10/d16/d1b/d40 0 2024-03-04T13:43:18.372 INFO:tasks.workunit.client.0.smithi107.stdout:3/701: unlink d1/de/d18/d33/d4d/d4f/d69/l77 0 2024-03-04T13:43:18.373 INFO:tasks.workunit.client.0.smithi107.stdout:5/810: mknod d5/d2b/d4b/da6/c120 0 2024-03-04T13:43:18.375 INFO:tasks.workunit.client.0.smithi107.stdout:5/811: readlink d5/d8/l3e 0 2024-03-04T13:43:18.377 INFO:tasks.workunit.client.0.smithi107.stdout:1/761: symlink d1/d10/d16/d1b/d40/lf3 0 2024-03-04T13:43:18.379 INFO:tasks.workunit.client.0.smithi107.stdout:4/686: rename d0/f73 to d0/fd6 0 2024-03-04T13:43:18.380 INFO:tasks.workunit.client.0.smithi107.stdout:0/785: truncate da/dc/dd/d3e/d26/d88/dd2/d73/d78/fcf 1399503 0 2024-03-04T13:43:18.383 INFO:tasks.workunit.client.0.smithi107.stdout:7/700: dwrite d6/fa [4194304,4194304] 0 2024-03-04T13:43:18.387 INFO:tasks.workunit.client.0.smithi107.stdout:7/701: write d6/d1d/fc8 [474637,70860] 0 2024-03-04T13:43:18.388 INFO:tasks.workunit.client.0.smithi107.stdout:6/768: dread df/d17/d24/d30/d8b/f87 [0,4194304] 0 2024-03-04T13:43:18.390 INFO:tasks.workunit.client.0.smithi107.stdout:4/687: getdents d0/d9/d78/dcf 0 2024-03-04T13:43:18.392 INFO:tasks.workunit.client.0.smithi107.stdout:7/702: fsync d6/f45 0 2024-03-04T13:43:18.393 INFO:tasks.workunit.client.0.smithi107.stdout:7/703: readlink d6/d17/l3a 0 2024-03-04T13:43:18.397 INFO:tasks.workunit.client.0.smithi107.stdout:5/812: write d5/d8/d2a/fb7 [67533,46634] 0 2024-03-04T13:43:18.398 INFO:tasks.workunit.client.0.smithi107.stdout:4/688: unlink d0/d9/d10/d80/db8/l8a 0 2024-03-04T13:43:18.398 INFO:tasks.workunit.client.0.smithi107.stdout:6/769: creat df/d17/d24/df1/ff6 x:0 0 0 2024-03-04T13:43:18.400 INFO:tasks.workunit.client.0.smithi107.stdout:7/704: creat d6/d13/d16/d19/d60/fe6 x:0 0 0 2024-03-04T13:43:18.405 INFO:tasks.workunit.client.0.smithi107.stdout:4/689: symlink d0/d2f/d7d/ld7 0 2024-03-04T13:43:18.406 INFO:tasks.workunit.client.0.smithi107.stdout:5/813: mkdir d5/d8/d14/d121 0 2024-03-04T13:43:18.407 INFO:tasks.workunit.client.0.smithi107.stdout:0/786: dwrite da/f62 [0,4194304] 0 2024-03-04T13:43:18.408 INFO:tasks.workunit.client.0.smithi107.stdout:3/702: dwrite d1/de/d18/d33/d48/d71/f15 [0,4194304] 0 2024-03-04T13:43:18.410 INFO:tasks.workunit.client.0.smithi107.stdout:6/770: dread df/d17/d24/d40/d63/da3/d7e/fb8 [0,4194304] 0 2024-03-04T13:43:18.411 INFO:tasks.workunit.client.0.smithi107.stdout:0/787: chown da/f81 158810996 1 2024-03-04T13:43:18.412 INFO:tasks.workunit.client.0.smithi107.stdout:4/690: symlink d0/d2f/ld8 0 2024-03-04T13:43:18.413 INFO:tasks.workunit.client.0.smithi107.stdout:5/814: creat d5/d8/d2a/dd6/dea/f122 x:0 0 0 2024-03-04T13:43:18.416 INFO:tasks.workunit.client.0.smithi107.stdout:0/788: mkdir da/dc/dd/d3e/d26/d69/d6b/df0/d10e 0 2024-03-04T13:43:18.417 INFO:tasks.workunit.client.0.smithi107.stdout:3/703: mknod d1/de/d18/d33/ce1 0 2024-03-04T13:43:18.421 INFO:tasks.workunit.client.0.smithi107.stdout:6/771: rename df/d17/d6b/d9b/c56 to df/d2e/d36/d5b/d7d/cf7 0 2024-03-04T13:43:18.426 INFO:tasks.workunit.client.0.smithi107.stdout:4/691: link d0/d9/fd1 d0/d9/fd9 0 2024-03-04T13:43:18.427 INFO:tasks.workunit.client.0.smithi107.stdout:7/705: dwrite d6/d1d/d64/d9a/da9/d71/fcc [4194304,4194304] 0 2024-03-04T13:43:18.430 INFO:tasks.workunit.client.0.smithi107.stdout:3/704: creat d1/de/dad/dbb/fe2 x:0 0 0 2024-03-04T13:43:18.432 INFO:tasks.workunit.client.0.smithi107.stdout:7/706: fdatasync d6/d1d/d36/fae 0 2024-03-04T13:43:18.436 INFO:tasks.workunit.client.0.smithi107.stdout:3/705: unlink d1/fc 0 2024-03-04T13:43:18.441 INFO:tasks.workunit.client.0.smithi107.stdout:3/706: fsync d1/de/d18/d33/d48/fa6 0 2024-03-04T13:43:18.442 INFO:tasks.workunit.client.0.smithi107.stdout:7/707: mknod d6/d13/d16/d19/d38/d2c/d8d/ce7 0 2024-03-04T13:43:18.444 INFO:tasks.workunit.client.0.smithi107.stdout:5/815: dwrite d5/dc/df/f2e [0,4194304] 0 2024-03-04T13:43:18.449 INFO:tasks.workunit.client.0.smithi107.stdout:5/816: fdatasync d5/f42 0 2024-03-04T13:43:18.457 INFO:tasks.workunit.client.0.smithi107.stdout:7/708: creat d6/d13/d16/d19/fe8 x:0 0 0 2024-03-04T13:43:18.458 INFO:tasks.workunit.client.0.smithi107.stdout:3/707: rename d1/de/ca4 to d1/d1e/d26/ce3 0 2024-03-04T13:43:18.461 INFO:tasks.workunit.client.0.smithi107.stdout:0/789: dwrite da/dc/f99 [0,4194304] 0 2024-03-04T13:43:18.464 INFO:tasks.workunit.client.0.smithi107.stdout:4/692: dwrite d0/d9/d18/d29/f55 [0,4194304] 0 2024-03-04T13:43:18.467 INFO:tasks.workunit.client.0.smithi107.stdout:5/817: getdents d5/dc/dc6 0 2024-03-04T13:43:18.469 INFO:tasks.workunit.client.0.smithi107.stdout:3/708: dread - d1/de/d18/d33/d48/fce zero size 2024-03-04T13:43:18.474 INFO:tasks.workunit.client.0.smithi107.stdout:4/693: mknod d0/d9/d10/d80/cda 0 2024-03-04T13:43:18.477 INFO:tasks.workunit.client.0.smithi107.stdout:7/709: rename d6/d13/d16/d19/d38/d62/fad to d6/d1d/d36/d59/fe9 0 2024-03-04T13:43:18.480 INFO:tasks.workunit.client.0.smithi107.stdout:5/818: mknod d5/d8/d2a/d43/c123 0 2024-03-04T13:43:18.481 INFO:tasks.workunit.client.0.smithi107.stdout:0/790: stat da/dc/dd/d3e/d26/d88/dd2/d73/d78/l93 0 2024-03-04T13:43:18.484 INFO:tasks.workunit.client.0.smithi107.stdout:6/772: dread df/d17/d24/d40/fa5 [0,4194304] 0 2024-03-04T13:43:18.484 INFO:tasks.workunit.client.0.smithi107.stdout:4/694: fdatasync d0/d9/de/f62 0 2024-03-04T13:43:18.491 INFO:tasks.workunit.client.0.smithi107.stdout:6/773: dread df/d17/d24/d30/d5d/d6a/fe3 [0,4194304] 0 2024-03-04T13:43:18.493 INFO:tasks.workunit.client.0.smithi107.stdout:5/819: truncate d5/d8/d14/ffb 406757 0 2024-03-04T13:43:18.494 INFO:tasks.workunit.client.0.smithi107.stdout:4/695: dread d0/d9/d18/fa9 [0,4194304] 0 2024-03-04T13:43:18.496 INFO:tasks.workunit.client.0.smithi107.stdout:6/774: symlink df/d2e/lf8 0 2024-03-04T13:43:18.498 INFO:tasks.workunit.client.0.smithi107.stdout:6/775: write df/d38/f5f [1719691,121771] 0 2024-03-04T13:43:18.502 INFO:tasks.workunit.client.0.smithi107.stdout:4/696: rmdir d0 39 2024-03-04T13:43:18.504 INFO:tasks.workunit.client.0.smithi107.stdout:6/776: creat df/d2e/ff9 x:0 0 0 2024-03-04T13:43:18.507 INFO:tasks.workunit.client.0.smithi107.stdout:3/709: truncate d1/de/d11/f3e 3330267 0 2024-03-04T13:43:18.510 INFO:tasks.workunit.client.0.smithi107.stdout:6/777: symlink df/d2e/d36/d5b/d7d/dbd/lfa 0 2024-03-04T13:43:18.510 INFO:tasks.workunit.client.0.smithi107.stdout:5/820: link d5/d2b/l31 d5/d8/da/db0/l124 0 2024-03-04T13:43:18.513 INFO:tasks.workunit.client.0.smithi107.stdout:4/697: readlink d0/d9/d18/d29/l8d 0 2024-03-04T13:43:18.515 INFO:tasks.workunit.client.0.smithi107.stdout:6/778: write df/d17/d24/f81 [796349,52680] 0 2024-03-04T13:43:18.518 INFO:tasks.workunit.client.0.smithi107.stdout:5/821: symlink d5/d8/d14/d2d/d92/dc1/dcc/l125 0 2024-03-04T13:43:18.518 INFO:tasks.workunit.client.0.smithi107.stdout:7/710: dwrite d6/d13/d14/f86 [0,4194304] 0 2024-03-04T13:43:18.519 INFO:tasks.workunit.client.0.smithi107.stdout:4/698: mknod d0/d9/d1e/d70/cdb 0 2024-03-04T13:43:18.520 INFO:tasks.workunit.client.0.smithi107.stdout:7/711: stat d6/d13/d16/f7b 0 2024-03-04T13:43:18.522 INFO:tasks.workunit.client.0.smithi107.stdout:3/710: rename d1/de/d18/d33/d4d/d4f/d69/cd8 to d1/de/d18/ce4 0 2024-03-04T13:43:18.528 INFO:tasks.workunit.client.0.smithi107.stdout:6/779: mknod df/d17/d24/d30/d8b/cfb 0 2024-03-04T13:43:18.532 INFO:tasks.workunit.client.0.smithi107.stdout:8/748: sync 2024-03-04T13:43:18.532 INFO:tasks.workunit.client.0.smithi107.stdout:4/699: creat d0/d9/d1e/fdc x:0 0 0 2024-03-04T13:43:18.533 INFO:tasks.workunit.client.0.smithi107.stdout:7/712: mknod d6/d1d/d64/d9a/da9/d71/db9/cea 0 2024-03-04T13:43:18.533 INFO:tasks.workunit.client.0.smithi107.stdout:6/780: write df/d17/d5c/db1/fba [2455246,109593] 0 2024-03-04T13:43:18.536 INFO:tasks.workunit.client.0.smithi107.stdout:3/711: fsync d1/de/d18/d33/f8f 0 2024-03-04T13:43:18.537 INFO:tasks.workunit.client.0.smithi107.stdout:8/749: readlink d6/da/lbe 0 2024-03-04T13:43:18.540 INFO:tasks.workunit.client.0.smithi107.stdout:4/700: fdatasync d0/d9/d10/d6f/fad 0 2024-03-04T13:43:18.541 INFO:tasks.workunit.client.0.smithi107.stdout:8/750: stat d6/da/d25/lc6 0 2024-03-04T13:43:18.549 INFO:tasks.workunit.client.0.smithi107.stdout:6/781: rename df/d17/d5c/db1 to df/d2e/d36/d5b/d7d/dfc 0 2024-03-04T13:43:18.550 INFO:tasks.workunit.client.0.smithi107.stdout:5/822: dread d5/dc/d22/f7f [0,4194304] 0 2024-03-04T13:43:18.554 INFO:tasks.workunit.client.0.smithi107.stdout:8/751: symlink d6/d16/d1f/d63/dc8/lea 0 2024-03-04T13:43:18.556 INFO:tasks.workunit.client.0.smithi107.stdout:5/823: readlink d5/d79/d89/da3/d69/l9f 0 2024-03-04T13:43:18.557 INFO:tasks.workunit.client.0.smithi107.stdout:8/752: chown d6/d16/d59/d3e/d67/c7d 1207 1 2024-03-04T13:43:18.558 INFO:tasks.workunit.client.0.smithi107.stdout:3/712: truncate d1/de/d18/d33/d48/d71/d4c/d54/fa7 17492 0 2024-03-04T13:43:18.560 INFO:tasks.workunit.client.0.smithi107.stdout:6/782: read df/d17/d24/d40/d63/da3/d7e/d86/fe2 [197866,97425] 0 2024-03-04T13:43:18.562 INFO:tasks.workunit.client.0.smithi107.stdout:5/824: creat d5/d79/d89/da3/d69/f126 x:0 0 0 2024-03-04T13:43:18.564 INFO:tasks.workunit.client.0.smithi107.stdout:8/753: fdatasync d6/d16/f5e 0 2024-03-04T13:43:18.565 INFO:tasks.workunit.client.0.smithi107.stdout:4/701: dwrite d0/d9/d10/d80/db8/d21/f5d [0,4194304] 0 2024-03-04T13:43:18.567 INFO:tasks.workunit.client.0.smithi107.stdout:2/687: sync 2024-03-04T13:43:18.568 INFO:tasks.workunit.client.0.smithi107.stdout:9/715: sync 2024-03-04T13:43:18.569 INFO:tasks.workunit.client.0.smithi107.stdout:6/783: unlink df/d17/d24/d30/c9d 0 2024-03-04T13:43:18.569 INFO:tasks.workunit.client.0.smithi107.stdout:5/825: rmdir d5/d2b/d4b/da6/dcf 39 2024-03-04T13:43:18.573 INFO:tasks.workunit.client.0.smithi107.stdout:6/784: stat df/d2e/d36/d77/d95 0 2024-03-04T13:43:18.577 INFO:tasks.workunit.client.0.smithi107.stdout:7/713: dwrite d6/d1d/d36/f81 [0,4194304] 0 2024-03-04T13:43:18.579 INFO:tasks.workunit.client.0.smithi107.stdout:4/702: creat d0/d9/d10/d80/db8/fdd x:0 0 0 2024-03-04T13:43:18.580 INFO:tasks.workunit.client.0.smithi107.stdout:4/703: fdatasync d0/d9/d78/f85 0 2024-03-04T13:43:18.580 INFO:tasks.workunit.client.0.smithi107.stdout:7/714: chown d6/d13/d16/d19/d38/d2c/f87 34 1 2024-03-04T13:43:18.588 INFO:tasks.workunit.client.0.smithi107.stdout:2/688: creat da/df/d3e/de3/fea x:0 0 0 2024-03-04T13:43:18.666 INFO:tasks.workunit.client.0.smithi107.stdout:4/704: rename d0/d9/d18/d29/c98 to d0/d9/de/d14/d66/cde 0 2024-03-04T13:43:18.672 INFO:tasks.workunit.client.0.smithi107.stdout:8/754: write d6/f3d [501421,105447] 0 2024-03-04T13:43:18.672 INFO:tasks.workunit.client.0.smithi107.stdout:9/716: link d4/da/d11/d20/d2f/d9c/d92/dce/c6b d4/da/d11/d20/d23/d7e/d84/cef 0 2024-03-04T13:43:18.674 INFO:tasks.workunit.client.0.smithi107.stdout:7/715: truncate d6/d13/d14/f5d 3362539 0 2024-03-04T13:43:18.679 INFO:tasks.workunit.client.0.smithi107.stdout:4/705: write d0/d9/d10/d80/db8/d21/f5d [4356626,127560] 0 2024-03-04T13:43:18.679 INFO:tasks.workunit.client.0.smithi107.stdout:2/689: getdents da/df/d3e/de3 0 2024-03-04T13:43:18.683 INFO:tasks.workunit.client.0.smithi107.stdout:7/716: creat d6/d13/d16/d1c/feb x:0 0 0 2024-03-04T13:43:18.684 INFO:tasks.workunit.client.0.smithi107.stdout:9/717: mkdir d4/da/d11/d20/d23/d7e/d84/db3/df0 0 2024-03-04T13:43:18.686 INFO:tasks.workunit.client.0.smithi107.stdout:5/826: dread d5/dc/f3b [0,4194304] 0 2024-03-04T13:43:18.689 INFO:tasks.workunit.client.0.smithi107.stdout:8/755: truncate d6/d16/d1f/d63/d93/fce 1644248 0 2024-03-04T13:43:18.690 INFO:tasks.workunit.client.0.smithi107.stdout:4/706: creat d0/d9/d1e/d70/fdf x:0 0 0 2024-03-04T13:43:18.694 INFO:tasks.workunit.client.0.smithi107.stdout:3/713: dwrite d1/de/d18/d33/d48/d71/d4c/d54/fa7 [0,4194304] 0 2024-03-04T13:43:18.696 INFO:tasks.workunit.client.0.smithi107.stdout:5/827: mkdir d5/d8/d14/d2d/d11d/d127 0 2024-03-04T13:43:18.698 INFO:tasks.workunit.client.0.smithi107.stdout:4/707: chown d0/d9/d18/d29/l8d 1 1 2024-03-04T13:43:18.700 INFO:tasks.workunit.client.0.smithi107.stdout:4/708: readlink d0/d9/d10/lba 0 2024-03-04T13:43:18.700 INFO:tasks.workunit.client.0.smithi107.stdout:3/714: chown d1/de 152284 1 2024-03-04T13:43:18.704 INFO:tasks.workunit.client.0.smithi107.stdout:9/718: mknod d4/da/d11/d4c/cf1 0 2024-03-04T13:43:18.706 INFO:tasks.workunit.client.0.smithi107.stdout:8/756: rename d6/d16/d59 to d6/d16/d69/deb 0 2024-03-04T13:43:18.710 INFO:tasks.workunit.client.0.smithi107.stdout:2/690: dread da/f1c [0,4194304] 0 2024-03-04T13:43:18.710 INFO:tasks.workunit.client.0.smithi107.stdout:0/791: sync 2024-03-04T13:43:18.710 INFO:tasks.workunit.client.0.smithi107.stdout:1/762: sync 2024-03-04T13:43:18.715 INFO:tasks.workunit.client.0.smithi107.stdout:0/792: chown da/dc/dd/d3e/d26/laf 14017 1 2024-03-04T13:43:18.718 INFO:tasks.workunit.client.0.smithi107.stdout:1/763: stat d1/d10/d16/ca0 0 2024-03-04T13:43:18.723 INFO:tasks.workunit.client.0.smithi107.stdout:2/691: symlink da/d44/d8b/d9d/dd0/leb 0 2024-03-04T13:43:18.724 INFO:tasks.workunit.client.0.smithi107.stdout:4/709: link d0/d9/de/d14/d33/fa1 d0/fe0 0 2024-03-04T13:43:18.724 INFO:tasks.workunit.client.0.smithi107.stdout:5/828: truncate d5/dc/df/f35 2343782 0 2024-03-04T13:43:18.726 INFO:tasks.workunit.client.0.smithi107.stdout:3/715: getdents d1/de/d4b 0 2024-03-04T13:43:18.731 INFO:tasks.workunit.client.0.smithi107.stdout:1/764: stat d1/d46/d6a/d4e/daa/dc9/d33/d5d/d24/fae 0 2024-03-04T13:43:18.734 INFO:tasks.workunit.client.0.smithi107.stdout:5/829: rename d5/dc/dc6/c95 to d5/d2b/d4b/c128 0 2024-03-04T13:43:18.738 INFO:tasks.workunit.client.0.smithi107.stdout:4/710: unlink d0/d9/d18/c22 0 2024-03-04T13:43:18.738 INFO:tasks.workunit.client.0.smithi107.stdout:3/716: dread d1/de/d3f/fb3 [0,4194304] 0 2024-03-04T13:43:18.744 INFO:tasks.workunit.client.0.smithi107.stdout:9/719: write d4/f26 [4595367,97187] 0 2024-03-04T13:43:18.744 INFO:tasks.workunit.client.0.smithi107.stdout:5/830: creat d5/d79/d89/f129 x:0 0 0 2024-03-04T13:43:18.748 INFO:tasks.workunit.client.0.smithi107.stdout:3/717: rmdir d1/de/d11 39 2024-03-04T13:43:18.748 INFO:tasks.workunit.client.0.smithi107.stdout:4/711: fdatasync d0/fd6 0 2024-03-04T13:43:18.749 INFO:tasks.workunit.client.0.smithi107.stdout:1/765: unlink d1/d46/d6a/d4e/daa/dc9/d33/d5d/l3d 0 2024-03-04T13:43:18.757 INFO:tasks.workunit.client.0.smithi107.stdout:3/718: stat d1/de/d18/d33/d48/d71/f15 0 2024-03-04T13:43:18.757 INFO:tasks.workunit.client.0.smithi107.stdout:8/757: dwrite d6/d16/d1f/f43 [0,4194304] 0 2024-03-04T13:43:18.760 INFO:tasks.workunit.client.0.smithi107.stdout:7/717: dread d6/d17/f8f [0,4194304] 0 2024-03-04T13:43:18.760 INFO:tasks.workunit.client.0.smithi107.stdout:4/712: write d0/d9/d10/d80/db8/f9e [1054600,127440] 0 2024-03-04T13:43:18.766 INFO:tasks.workunit.client.0.smithi107.stdout:4/713: stat d0/d9/d10/d80/db8/fdd 0 2024-03-04T13:43:18.769 INFO:tasks.workunit.client.0.smithi107.stdout:8/758: read - d6/d16/d1f/d4c/da4/fc2 zero size 2024-03-04T13:43:18.770 INFO:tasks.workunit.client.0.smithi107.stdout:5/831: getdents d5/d79/d89/da3/d68 0 2024-03-04T13:43:18.773 INFO:tasks.workunit.client.0.smithi107.stdout:1/766: creat d1/d46/d6a/d65/db1/ff4 x:0 0 0 2024-03-04T13:43:18.774 INFO:tasks.workunit.client.0.smithi107.stdout:7/718: rename d6/d13/d16/d19/d38/d62/c91 to d6/d13/d16/d19/d38/d37/cec 0 2024-03-04T13:43:18.778 INFO:tasks.workunit.client.0.smithi107.stdout:9/720: dread d4/da/d11/d4c/f3c [0,4194304] 0 2024-03-04T13:43:18.781 INFO:tasks.workunit.client.0.smithi107.stdout:4/714: creat d0/d9/d10/d80/db8/d97/fe1 x:0 0 0 2024-03-04T13:43:18.781 INFO:tasks.workunit.client.0.smithi107.stdout:8/759: mkdir d6/d16/d1f/d4c/dec 0 2024-03-04T13:43:18.782 INFO:tasks.workunit.client.0.smithi107.stdout:5/832: truncate d5/d8/d2a/dd6/dea/f122 599547 0 2024-03-04T13:43:18.783 INFO:tasks.workunit.client.0.smithi107.stdout:7/719: symlink d6/d1d/d64/d9a/da9/d71/db9/dca/d22/led 0 2024-03-04T13:43:18.784 INFO:tasks.workunit.client.0.smithi107.stdout:0/793: dread da/dc/dd/d3e/f11 [0,4194304] 0 2024-03-04T13:43:18.785 INFO:tasks.workunit.client.0.smithi107.stdout:4/715: chown d0/d9/d10/d6f/fad 1964 1 2024-03-04T13:43:18.789 INFO:tasks.workunit.client.0.smithi107.stdout:8/760: mknod d6/d16/d1f/d4c/da4/daa/ced 0 2024-03-04T13:43:18.797 INFO:tasks.workunit.client.0.smithi107.stdout:3/719: dwrite d1/de/d18/d33/f8f [0,4194304] 0 2024-03-04T13:43:18.900 INFO:tasks.workunit.client.0.smithi107.stdout:6/785: sync 2024-03-04T13:43:18.909 INFO:tasks.workunit.client.0.smithi107.stdout:1/767: write d1/d10/d16/d1b/d40/f57 [2306991,125604] 0 2024-03-04T13:43:18.912 INFO:tasks.workunit.client.0.smithi107.stdout:7/720: mknod d6/d13/d16/d19/d38/d2c/cee 0 2024-03-04T13:43:18.915 INFO:tasks.workunit.client.0.smithi107.stdout:3/720: write d1/de/d3f/fb3 [79701,56997] 0 2024-03-04T13:43:18.919 INFO:tasks.workunit.client.0.smithi107.stdout:9/721: link d4/da/d11/cb5 d4/da/d11/d20/d64/d77/cf2 0 2024-03-04T13:43:18.924 INFO:tasks.workunit.client.0.smithi107.stdout:8/761: dwrite d6/d16/d1f/f5c [4194304,4194304] 0 2024-03-04T13:43:18.928 INFO:tasks.workunit.client.0.smithi107.stdout:3/721: mknod d1/de/d4b/ce5 0 2024-03-04T13:43:18.934 INFO:tasks.workunit.client.0.smithi107.stdout:5/833: link d5/d79/d89/da3/d69/l9f d5/d79/d89/da3/l12a 0 2024-03-04T13:43:18.935 INFO:tasks.workunit.client.0.smithi107.stdout:4/716: dwrite d0/d9/d18/fa9 [0,4194304] 0 2024-03-04T13:43:18.946 INFO:tasks.workunit.client.0.smithi107.stdout:7/721: dwrite d6/d13/d16/d19/d60/fe6 [0,4194304] 0 2024-03-04T13:43:18.947 INFO:tasks.workunit.client.0.smithi107.stdout:4/717: mknod d0/d9/d18/ce2 0 2024-03-04T13:43:18.950 INFO:tasks.workunit.client.0.smithi107.stdout:4/718: write d0/d9/d78/f85 [83974,98242] 0 2024-03-04T13:43:18.953 INFO:tasks.workunit.client.0.smithi107.stdout:9/722: dread d4/da/f4a [0,4194304] 0 2024-03-04T13:43:18.954 INFO:tasks.workunit.client.0.smithi107.stdout:9/723: chown d4/ff 761 1 2024-03-04T13:43:18.954 INFO:tasks.workunit.client.0.smithi107.stdout:7/722: symlink d6/d13/d14/lef 0 2024-03-04T13:43:18.955 INFO:tasks.workunit.client.0.smithi107.stdout:9/724: fsync d4/d34/f44 0 2024-03-04T13:43:18.957 INFO:tasks.workunit.client.0.smithi107.stdout:4/719: mkdir d0/d9/d10/d80/db8/d97/de3 0 2024-03-04T13:43:18.965 INFO:tasks.workunit.client.0.smithi107.stdout:9/725: dread d4/da/d11/d4c/fbe [0,4194304] 0 2024-03-04T13:43:18.972 INFO:tasks.workunit.client.0.smithi107.stdout:4/720: symlink d0/d9/d10/d80/db8/d97/dcd/le4 0 2024-03-04T13:43:18.981 INFO:tasks.workunit.client.0.smithi107.stdout:4/721: rename d0/d9/de/c76 to d0/d9/d10/d80/db8/d97/dd0/ce5 0 2024-03-04T13:43:18.987 INFO:tasks.workunit.client.0.smithi107.stdout:4/722: rmdir d0/d9/d1e/d70 39 2024-03-04T13:43:18.988 INFO:tasks.workunit.client.0.smithi107.stdout:4/723: chown d0/d9/d10/d80/d81 2798 1 2024-03-04T13:43:18.990 INFO:tasks.workunit.client.0.smithi107.stdout:8/762: dread d6/d16/fa3 [0,4194304] 0 2024-03-04T13:43:18.990 INFO:tasks.workunit.client.0.smithi107.stdout:8/763: readlink d6/l99 0 2024-03-04T13:43:18.991 INFO:tasks.workunit.client.0.smithi107.stdout:4/724: mknod d0/d2f/ce6 0 2024-03-04T13:43:18.995 INFO:tasks.workunit.client.0.smithi107.stdout:8/764: unlink d6/d16/d69/deb/l49 0 2024-03-04T13:43:19.011 INFO:tasks.workunit.client.0.smithi107.stdout:8/765: dwrite d6/d16/d1f/d4c/da5/fc9 [0,4194304] 0 2024-03-04T13:43:19.027 INFO:tasks.workunit.client.0.smithi107.stdout:4/725: getdents d0/d9/d10/d80/db8/d9a 0 2024-03-04T13:43:19.100 INFO:tasks.workunit.client.0.smithi107.stdout:1/768: mknod d1/d46/d6a/d4e/daa/dc9/d33/cf5 0 2024-03-04T13:43:19.113 INFO:tasks.workunit.client.0.smithi107.stdout:1/769: dread d1/d46/d6a/d4e/d60/d83/fc6 [0,4194304] 0 2024-03-04T13:43:19.126 INFO:tasks.workunit.client.0.smithi107.stdout:1/770: getdents d1/d46/d6a 0 2024-03-04T13:43:19.140 INFO:tasks.workunit.client.0.smithi107.stdout:8/766: write d6/d16/d1f/d4c/f52 [1082978,49871] 0 2024-03-04T13:43:19.140 INFO:tasks.workunit.client.0.smithi107.stdout:4/726: write d0/d9/d10/f25 [312498,45881] 0 2024-03-04T13:43:19.143 INFO:tasks.workunit.client.0.smithi107.stdout:3/722: mknod d1/de/d18/d33/ce6 0 2024-03-04T13:43:19.148 INFO:tasks.workunit.client.0.smithi107.stdout:4/727: truncate d0/d9/fb2 766186 0 2024-03-04T13:43:19.150 INFO:tasks.workunit.client.0.smithi107.stdout:3/723: mknod d1/de/d18/d33/d6b/ce7 0 2024-03-04T13:43:19.153 INFO:tasks.workunit.client.0.smithi107.stdout:8/767: truncate d6/da/da8/db5/fe1 459834 0 2024-03-04T13:43:19.154 INFO:tasks.workunit.client.0.smithi107.stdout:4/728: creat d0/d9/de/d14/d66/fe7 x:0 0 0 2024-03-04T13:43:19.154 INFO:tasks.workunit.client.0.smithi107.stdout:1/771: dwrite d1/d46/d6a/d4e/fcf [0,4194304] 0 2024-03-04T13:43:19.159 INFO:tasks.workunit.client.0.smithi107.stdout:3/724: symlink d1/d1e/daf/le8 0 2024-03-04T13:43:19.160 INFO:tasks.workunit.client.0.smithi107.stdout:4/729: fdatasync d0/d9/d10/f8c 0 2024-03-04T13:43:19.163 INFO:tasks.workunit.client.0.smithi107.stdout:8/768: creat d6/d16/d1f/d4c/da5/fee x:0 0 0 2024-03-04T13:43:19.178 INFO:tasks.workunit.client.0.smithi107.stdout:3/725: mknod d1/de/ce9 0 2024-03-04T13:43:19.183 INFO:tasks.workunit.client.0.smithi107.stdout:3/726: write d1/de/d18/d33/d48/d71/f15 [10787107,90124] 0 2024-03-04T13:43:19.184 INFO:tasks.workunit.client.0.smithi107.stdout:8/769: creat d6/da/d25/d2a/fef x:0 0 0 2024-03-04T13:43:19.187 INFO:tasks.workunit.client.0.smithi107.stdout:8/770: dread - d6/d16/d1f/fe5 zero size 2024-03-04T13:43:19.188 INFO:tasks.workunit.client.0.smithi107.stdout:3/727: symlink d1/de/d4b/lea 0 2024-03-04T13:43:19.191 INFO:tasks.workunit.client.0.smithi107.stdout:8/771: mknod d6/dcc/cf0 0 2024-03-04T13:43:19.192 INFO:tasks.workunit.client.0.smithi107.stdout:1/772: write d1/d10/d16/fad [1046773,2702] 0 2024-03-04T13:43:19.195 INFO:tasks.workunit.client.0.smithi107.stdout:8/772: chown d6/d16/d1f/d4c/da5 1 1 2024-03-04T13:43:19.196 INFO:tasks.workunit.client.0.smithi107.stdout:3/728: dread d1/de/d18/d33/d4d/f90 [0,4194304] 0 2024-03-04T13:43:19.197 INFO:tasks.workunit.client.0.smithi107.stdout:1/773: read d1/d10/d16/d1b/fe3 [623228,84670] 0 2024-03-04T13:43:19.200 INFO:tasks.workunit.client.0.smithi107.stdout:3/729: chown d1/de/d3f/f78 0 1 2024-03-04T13:43:19.203 INFO:tasks.workunit.client.0.smithi107.stdout:3/730: write d1/f6 [181808,95013] 0 2024-03-04T13:43:19.207 INFO:tasks.workunit.client.0.smithi107.stdout:4/730: dwrite d0/d9/d10/d80/db8/f69 [0,4194304] 0 2024-03-04T13:43:19.220 INFO:tasks.workunit.client.0.smithi107.stdout:1/774: write d1/d46/d6a/d4e/daa/dc9/d33/d5d/f97 [5755137,106073] 0 2024-03-04T13:43:19.222 INFO:tasks.workunit.client.0.smithi107.stdout:4/731: write d0/d9/d18/d29/f35 [4206384,93757] 0 2024-03-04T13:43:19.224 INFO:tasks.workunit.client.0.smithi107.stdout:4/732: stat d0/d9/d10/d6f/f95 0 2024-03-04T13:43:19.229 INFO:tasks.workunit.client.0.smithi107.stdout:4/733: mkdir d0/d9/d18/d29/de8 0 2024-03-04T13:43:19.232 INFO:tasks.workunit.client.0.smithi107.stdout:8/773: dread d6/f29 [0,4194304] 0 2024-03-04T13:43:19.235 INFO:tasks.workunit.client.0.smithi107.stdout:4/734: symlink d0/d9/d10/d80/le9 0 2024-03-04T13:43:19.236 INFO:tasks.workunit.client.0.smithi107.stdout:3/731: dread d1/de/d18/f59 [0,4194304] 0 2024-03-04T13:43:19.237 INFO:tasks.workunit.client.0.smithi107.stdout:4/735: creat d0/d9/d10/dab/fea x:0 0 0 2024-03-04T13:43:19.239 INFO:tasks.workunit.client.0.smithi107.stdout:2/692: sync 2024-03-04T13:43:19.239 INFO:tasks.workunit.client.0.smithi107.stdout:4/736: chown d0/d9/f6b 5570 1 2024-03-04T13:43:19.240 INFO:tasks.workunit.client.0.smithi107.stdout:1/775: dread d1/d10/d16/fd9 [0,4194304] 0 2024-03-04T13:43:19.243 INFO:tasks.workunit.client.0.smithi107.stdout:2/693: creat da/d29/d7c/dbd/fec x:0 0 0 2024-03-04T13:43:19.244 INFO:tasks.workunit.client.0.smithi107.stdout:2/694: stat da/d49/d5c/d6a 0 2024-03-04T13:43:19.245 INFO:tasks.workunit.client.0.smithi107.stdout:1/776: chown d1/d46/d6a/d4e/daa/dc9/d33/d5d/d4c/dea/d5b/d61/fce 19 1 2024-03-04T13:43:19.245 INFO:tasks.workunit.client.0.smithi107.stdout:4/737: getdents d0/d9/d10/d80/db8/d9a 0 2024-03-04T13:43:19.248 INFO:tasks.workunit.client.0.smithi107.stdout:4/738: chown d0/d9/d1e/fdc 4607 1 2024-03-04T13:43:19.250 INFO:tasks.workunit.client.0.smithi107.stdout:2/695: read da/df/d73/db3/db1/fc1 [3441709,99548] 0 2024-03-04T13:43:19.252 INFO:tasks.workunit.client.0.smithi107.stdout:4/739: creat d0/d2f/feb x:0 0 0 2024-03-04T13:43:19.252 INFO:tasks.workunit.client.0.smithi107.stdout:1/777: rmdir d1/d46/d6a/d4e/daa/dc9/d33/d5d/d4c/dea/d59 39 2024-03-04T13:43:19.252 INFO:tasks.workunit.client.0.smithi107.stdout:2/696: write da/d34/f36 [3356586,10541] 0 2024-03-04T13:43:19.254 INFO:tasks.workunit.client.0.smithi107.stdout:1/778: dread - d1/d46/d6a/d4e/d60/fed zero size 2024-03-04T13:43:19.257 INFO:tasks.workunit.client.0.smithi107.stdout:2/697: dread - da/d29/d7c/fe1 zero size 2024-03-04T13:43:19.258 INFO:tasks.workunit.client.0.smithi107.stdout:4/740: creat d0/d9/d18/d29/de8/fec x:0 0 0 2024-03-04T13:43:19.260 INFO:tasks.workunit.client.0.smithi107.stdout:3/732: dread d1/f56 [0,4194304] 0 2024-03-04T13:43:19.260 INFO:tasks.workunit.client.0.smithi107.stdout:8/774: dread d6/d16/d1f/d4c/f52 [0,4194304] 0 2024-03-04T13:43:19.262 INFO:tasks.workunit.client.0.smithi107.stdout:8/775: fdatasync d6/da/d25/d2a/fef 0 2024-03-04T13:43:19.264 INFO:tasks.workunit.client.0.smithi107.stdout:1/779: creat d1/d46/d6a/d4e/daa/dc9/d33/d5d/d4c/dea/ff6 x:0 0 0 2024-03-04T13:43:19.264 INFO:tasks.workunit.client.0.smithi107.stdout:3/733: chown d1/de/d11/l98 7449081 1 2024-03-04T13:43:19.266 INFO:tasks.workunit.client.0.smithi107.stdout:8/776: rmdir d6/d16/d1f 39 2024-03-04T13:43:19.266 INFO:tasks.workunit.client.0.smithi107.stdout:4/741: symlink d0/d9/d78/dcf/led 0 2024-03-04T13:43:19.271 INFO:tasks.workunit.client.0.smithi107.stdout:4/742: fdatasync d0/d9/fd1 0 2024-03-04T13:43:19.272 INFO:tasks.workunit.client.0.smithi107.stdout:4/743: readlink d0/d9/de/d14/l23 0 2024-03-04T13:43:19.275 INFO:tasks.workunit.client.0.smithi107.stdout:8/777: rename d6/d16/d1f/f65 to d6/d16/d1f/d4c/da4/daa/ff1 0 2024-03-04T13:43:19.278 INFO:tasks.workunit.client.0.smithi107.stdout:3/734: write d1/de/d18/d1f/d42/f62 [107541,87078] 0 2024-03-04T13:43:19.278 INFO:tasks.workunit.client.0.smithi107.stdout:8/778: write d6/da/f11 [3321796,45170] 0 2024-03-04T13:43:19.282 INFO:tasks.workunit.client.0.smithi107.stdout:2/698: dread da/d44/f81 [0,4194304] 0 2024-03-04T13:43:19.282 INFO:tasks.workunit.client.0.smithi107.stdout:1/780: dwrite d1/d46/d6a/d4e/daa/dc9/d67/d6b/f84 [0,4194304] 0 2024-03-04T13:43:19.287 INFO:tasks.workunit.client.0.smithi107.stdout:2/699: mknod da/df/d27/d58/ced 0 2024-03-04T13:43:19.293 INFO:tasks.workunit.client.0.smithi107.stdout:4/744: rename d0/d9/d10/d80/db8/f2d to d0/fee 0 2024-03-04T13:43:19.297 INFO:tasks.workunit.client.0.smithi107.stdout:2/700: mkdir da/df/d73/db3/db1/dee 0 2024-03-04T13:43:19.298 INFO:tasks.workunit.client.0.smithi107.stdout:1/781: dwrite d1/d10/fb0 [4194304,4194304] 0 2024-03-04T13:43:19.300 INFO:tasks.workunit.client.0.smithi107.stdout:2/701: chown da/d38/d3a/l45 0 1 2024-03-04T13:43:19.301 INFO:tasks.workunit.client.0.smithi107.stdout:2/702: chown da/d49/fa4 126 1 2024-03-04T13:43:19.303 INFO:tasks.workunit.client.0.smithi107.stdout:4/745: creat d0/fef x:0 0 0 2024-03-04T13:43:19.312 INFO:tasks.workunit.client.0.smithi107.stdout:1/782: link d1/d10/d16/d47/f77 d1/d10/d16/d47/d7a/dda/ff7 0 2024-03-04T13:43:19.313 INFO:tasks.workunit.client.0.smithi107.stdout:3/735: write d1/d1e/f74 [8853362,48535] 0 2024-03-04T13:43:19.313 INFO:tasks.workunit.client.0.smithi107.stdout:4/746: fdatasync d0/d9/d1e/d70/fdf 0 2024-03-04T13:43:19.318 INFO:tasks.workunit.client.0.smithi107.stdout:3/736: dread - d1/de/d18/d33/d48/d71/d4c/fcd zero size 2024-03-04T13:43:19.320 INFO:tasks.workunit.client.0.smithi107.stdout:3/737: fsync d1/de/d3f/fb3 0 2024-03-04T13:43:19.320 INFO:tasks.workunit.client.0.smithi107.stdout:4/747: creat d0/d9/d10/d80/db8/d97/dcd/ff0 x:0 0 0 2024-03-04T13:43:19.323 INFO:tasks.workunit.client.0.smithi107.stdout:3/738: write d1/d41/d81/fb5 [502624,51734] 0 2024-03-04T13:43:19.324 INFO:tasks.workunit.client.0.smithi107.stdout:2/703: dwrite da/d39/fc4 [0,4194304] 0 2024-03-04T13:43:19.324 INFO:tasks.workunit.client.0.smithi107.stdout:4/748: dread d0/d9/d78/fbb [0,4194304] 0 2024-03-04T13:43:19.326 INFO:tasks.workunit.client.0.smithi107.stdout:3/739: chown d1/de/d18/cdc 1627 1 2024-03-04T13:43:19.334 INFO:tasks.workunit.client.0.smithi107.stdout:1/783: dwrite d1/d46/d6a/d4e/daa/dc9/f32 [4194304,4194304] 0 2024-03-04T13:43:19.337 INFO:tasks.workunit.client.0.smithi107.stdout:3/740: dread d1/f3 [0,4194304] 0 2024-03-04T13:43:19.338 INFO:tasks.workunit.client.0.smithi107.stdout:3/741: stat d1/d1e/daf/le8 0 2024-03-04T13:43:19.340 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:19 smithi107 ceph-mon[28739]: pgmap v61: 65 pgs: 65 active+clean; 3.3 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 84 MiB/s rd, 232 MiB/s wr, 524 op/s 2024-03-04T13:43:19.340 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:19 smithi107 ceph-mon[28739]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' 2024-03-04T13:43:19.340 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:19 smithi107 ceph-mon[28739]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' 2024-03-04T13:43:19.357 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:19 smithi169 ceph-mon[36313]: pgmap v61: 65 pgs: 65 active+clean; 3.3 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 84 MiB/s rd, 232 MiB/s wr, 524 op/s 2024-03-04T13:43:19.357 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:19 smithi169 ceph-mon[36313]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' 2024-03-04T13:43:19.357 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:19 smithi169 ceph-mon[36313]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' 2024-03-04T13:43:19.390 INFO:tasks.workunit.client.0.smithi107.stdout:0/794: sync 2024-03-04T13:43:19.397 INFO:tasks.workunit.client.0.smithi107.stdout:3/742: creat d1/de/d18/d33/d4d/d4f/dc3/feb x:0 0 0 2024-03-04T13:43:19.401 INFO:tasks.workunit.client.0.smithi107.stdout:3/743: chown d1/de/d18/f92 1 1 2024-03-04T13:43:19.405 INFO:tasks.workunit.client.0.smithi107.stdout:0/795: truncate da/dc/f99 3504416 0 2024-03-04T13:43:19.416 INFO:tasks.workunit.client.0.smithi107.stdout:1/784: dread d1/d46/d6a/d4e/daa/dc9/d33/d5d/d4c/dea/f55 [0,4194304] 0 2024-03-04T13:43:19.418 INFO:tasks.workunit.client.0.smithi107.stdout:0/796: truncate da/dc/dd/d1e/fa1 3765685 0 2024-03-04T13:43:19.419 INFO:tasks.workunit.client.0.smithi107.stdout:3/744: dwrite d1/de/f32 [0,4194304] 0 2024-03-04T13:43:19.422 INFO:tasks.workunit.client.0.smithi107.stdout:0/797: fdatasync da/dc/dd/d3e/d26/d88/dd2/f87 0 2024-03-04T13:43:19.424 INFO:tasks.workunit.client.0.smithi107.stdout:3/745: dread - d1/de/d18/d33/d4d/d4f/dc3/feb zero size 2024-03-04T13:43:19.425 INFO:tasks.workunit.client.0.smithi107.stdout:1/785: symlink d1/d46/d6a/d4e/d60/d83/dde/lf8 0 2024-03-04T13:43:19.427 INFO:tasks.workunit.client.0.smithi107.stdout:1/786: fdatasync d1/d46/d6a/d4e/fb9 0 2024-03-04T13:43:19.434 INFO:tasks.workunit.client.0.smithi107.stdout:0/798: link da/dc/dd/dc9/de5/f100 da/dc/dd/d3e/f10f 0 2024-03-04T13:43:19.437 INFO:tasks.workunit.client.0.smithi107.stdout:1/787: creat d1/d46/d6a/d4e/daa/dc9/d33/d5d/d4c/dea/d5b/ff9 x:0 0 0 2024-03-04T13:43:19.441 INFO:tasks.workunit.client.0.smithi107.stdout:3/746: write d1/f56 [2280144,116024] 0 2024-03-04T13:43:19.441 INFO:tasks.workunit.client.0.smithi107.stdout:1/788: fdatasync d1/d46/d6a/d17/f34 0 2024-03-04T13:43:19.446 INFO:tasks.workunit.client.0.smithi107.stdout:3/747: dread - d1/de/d18/d1f/d42/fc0 zero size 2024-03-04T13:43:19.450 INFO:tasks.workunit.client.0.smithi107.stdout:3/748: creat d1/d1e/d26/d9e/fec x:0 0 0 2024-03-04T13:43:19.456 INFO:tasks.workunit.client.0.smithi107.stdout:3/749: rename d1/d1e/f74 to d1/de/d18/d33/d4d/d4f/fed 0 2024-03-04T13:43:19.461 INFO:tasks.workunit.client.0.smithi107.stdout:3/750: creat d1/d29/dda/fee x:0 0 0 2024-03-04T13:43:19.463 INFO:tasks.workunit.client.0.smithi107.stdout:1/789: dwrite d1/d46/d6a/d4e/daa/dc9/d33/d5d/d4c/dea/ff6 [0,4194304] 0 2024-03-04T13:43:19.464 INFO:tasks.workunit.client.0.smithi107.stdout:3/751: creat d1/d2d/fef x:0 0 0 2024-03-04T13:43:19.465 INFO:tasks.workunit.client.0.smithi107.stdout:1/790: write d1/d10/fb0 [1164611,68753] 0 2024-03-04T13:43:19.473 INFO:tasks.workunit.client.0.smithi107.stdout:1/791: dread d1/d46/d6a/d4e/d60/d83/fc6 [0,4194304] 0 2024-03-04T13:43:19.478 INFO:tasks.workunit.client.0.smithi107.stdout:3/752: dread d1/f3 [0,4194304] 0 2024-03-04T13:43:19.480 INFO:tasks.workunit.client.0.smithi107.stdout:0/799: dread da/dc/dd/f24 [0,4194304] 0 2024-03-04T13:43:19.485 INFO:tasks.workunit.client.0.smithi107.stdout:0/800: symlink da/dc/dd/d3e/d26/d69/de9/l110 0 2024-03-04T13:43:19.486 INFO:tasks.workunit.client.0.smithi107.stdout:5/834: sync 2024-03-04T13:43:19.486 INFO:tasks.workunit.client.0.smithi107.stdout:7/723: sync 2024-03-04T13:43:19.486 INFO:tasks.workunit.client.0.smithi107.stdout:8/779: sync 2024-03-04T13:43:19.486 INFO:tasks.workunit.client.0.smithi107.stdout:9/726: sync 2024-03-04T13:43:19.487 INFO:tasks.workunit.client.0.smithi107.stdout:6/786: sync 2024-03-04T13:43:19.490 INFO:tasks.workunit.client.0.smithi107.stdout:1/792: dwrite d1/d46/d6a/d65/db1/ff4 [0,4194304] 0 2024-03-04T13:43:19.491 INFO:tasks.workunit.client.0.smithi107.stdout:0/801: mknod da/d7e/c111 0 2024-03-04T13:43:19.493 INFO:tasks.workunit.client.0.smithi107.stdout:0/802: chown da/dc/dd/dc9/de5 9957 1 2024-03-04T13:43:19.494 INFO:tasks.workunit.client.0.smithi107.stdout:3/753: write d1/de/d18/d1f/d42/f52 [3076658,109408] 0 2024-03-04T13:43:19.498 INFO:tasks.workunit.client.0.smithi107.stdout:7/724: symlink d6/d13/d16/d19/d38/d62/ddb/lf0 0 2024-03-04T13:43:19.498 INFO:tasks.workunit.client.0.smithi107.stdout:1/793: stat d1/d46/d6a/d4e/daa/dc9/d67/d6b/f6e 0 2024-03-04T13:43:19.499 INFO:tasks.workunit.client.0.smithi107.stdout:5/835: truncate d5/d79/d89/da3/d69/f72 3696025 0 2024-03-04T13:43:19.506 INFO:tasks.workunit.client.0.smithi107.stdout:6/787: rename df/d17/d24/d30/d8b/cfb to df/d17/d6b/d9b/cfd 0 2024-03-04T13:43:19.509 INFO:tasks.workunit.client.0.smithi107.stdout:8/780: dwrite d6/d16/fc5 [0,4194304] 0 2024-03-04T13:43:19.509 INFO:tasks.workunit.client.0.smithi107.stdout:3/754: creat d1/de/d18/d33/d4d/d4f/ff0 x:0 0 0 2024-03-04T13:43:19.510 INFO:tasks.workunit.client.0.smithi107.stdout:1/794: chown d1/d46/d6a/d4e/d60/c8d 136460 1 2024-03-04T13:43:19.516 INFO:tasks.workunit.client.0.smithi107.stdout:0/803: write da/dc/dd/d3e/d26/f64 [4387882,106006] 0 2024-03-04T13:43:19.521 INFO:tasks.workunit.client.0.smithi107.stdout:9/727: dread d4/da/d11/d20/d23/dd9/f8d [0,4194304] 0 2024-03-04T13:43:19.523 INFO:tasks.workunit.client.0.smithi107.stdout:3/755: rename d1/d2d/l3c to d1/d1e/dba/lf1 0 2024-03-04T13:43:19.524 INFO:tasks.workunit.client.0.smithi107.stdout:7/725: link d6/d13/dbc/fcd d6/d1d/d64/d9a/da9/d71/db9/dca/d3d/dc6/ff1 0 2024-03-04T13:43:19.530 INFO:tasks.workunit.client.0.smithi107.stdout:1/795: link d1/ld5 d1/d46/d6a/d4e/df2/lfa 0 2024-03-04T13:43:19.535 INFO:tasks.workunit.client.0.smithi107.stdout:5/836: dread d5/d8/fb [0,4194304] 0 2024-03-04T13:43:19.537 INFO:tasks.workunit.client.0.smithi107.stdout:1/796: mknod d1/d46/d6a/d4e/daa/dc9/d33/d5d/d4c/dea/cfb 0 2024-03-04T13:43:19.542 INFO:tasks.workunit.client.0.smithi107.stdout:6/788: dwrite df/d17/d24/f2d [0,4194304] 0 2024-03-04T13:43:19.544 INFO:tasks.workunit.client.0.smithi107.stdout:5/837: mkdir d5/d2b/d4b/da6/dcf/d12b 0 2024-03-04T13:43:19.545 INFO:tasks.workunit.client.0.smithi107.stdout:8/781: write d6/d16/d1f/f34 [182768,115293] 0 2024-03-04T13:43:19.545 INFO:tasks.workunit.client.0.smithi107.stdout:3/756: dwrite d1/de/d4b/f7a [0,4194304] 0 2024-03-04T13:43:19.548 INFO:tasks.workunit.client.0.smithi107.stdout:5/838: readlink d5/dc/d22/l36 0 2024-03-04T13:43:19.556 INFO:tasks.workunit.client.0.smithi107.stdout:1/797: dread d1/d46/d6a/d65/db1/ff4 [0,4194304] 0 2024-03-04T13:43:19.558 INFO:tasks.workunit.client.0.smithi107.stdout:7/726: dread d6/d13/d16/d1c/f1f [0,4194304] 0 2024-03-04T13:43:19.559 INFO:tasks.workunit.client.0.smithi107.stdout:3/757: mkdir d1/d1e/dae/df2 0 2024-03-04T13:43:19.567 INFO:tasks.workunit.client.0.smithi107.stdout:8/782: dread d6/d16/d1f/d4c/da5/fc9 [0,4194304] 0 2024-03-04T13:43:19.567 INFO:tasks.workunit.client.0.smithi107.stdout:3/758: dread d1/de/d18/d33/d48/fa6 [0,4194304] 0 2024-03-04T13:43:19.568 INFO:tasks.workunit.client.0.smithi107.stdout:1/798: creat d1/d46/d9f/ffc x:0 0 0 2024-03-04T13:43:19.572 INFO:tasks.workunit.client.0.smithi107.stdout:3/759: fdatasync d1/de/d3f/f80 0 2024-03-04T13:43:19.578 INFO:tasks.workunit.client.0.smithi107.stdout:7/727: dwrite d6/d1d/d64/d9a/da9/d71/fcc [4194304,4194304] 0 2024-03-04T13:43:19.578 INFO:tasks.workunit.client.0.smithi107.stdout:1/799: rmdir d1/dd6 39 2024-03-04T13:43:19.581 INFO:tasks.workunit.client.0.smithi107.stdout:3/760: symlink d1/de/d18/d7e/lf3 0 2024-03-04T13:43:19.585 INFO:tasks.workunit.client.0.smithi107.stdout:1/800: mkdir d1/d46/d6a/d75/dfd 0 2024-03-04T13:43:19.585 INFO:tasks.workunit.client.0.smithi107.stdout:3/761: read - d1/de/d18/d33/d48/d71/d4c/fcd zero size 2024-03-04T13:43:19.585 INFO:tasks.workunit.client.0.smithi107.stdout:7/728: read d6/f7 [1092731,43034] 0 2024-03-04T13:43:19.588 INFO:tasks.workunit.client.0.smithi107.stdout:7/729: chown d6/d1d/d64/fde 348381120 1 2024-03-04T13:43:19.588 INFO:tasks.workunit.client.0.smithi107.stdout:3/762: dread - d1/de/d18/d1f/d42/fc0 zero size 2024-03-04T13:43:19.589 INFO:tasks.workunit.client.0.smithi107.stdout:5/839: dwrite d5/d8/d14/ffb [0,4194304] 0 2024-03-04T13:43:19.590 INFO:tasks.workunit.client.0.smithi107.stdout:5/840: stat d5/fe2 0 2024-03-04T13:43:19.593 INFO:tasks.workunit.client.0.smithi107.stdout:3/763: mknod d1/d2d/cf4 0 2024-03-04T13:43:19.594 INFO:tasks.workunit.client.0.smithi107.stdout:4/749: sync 2024-03-04T13:43:19.594 INFO:tasks.workunit.client.0.smithi107.stdout:2/704: sync 2024-03-04T13:43:19.599 INFO:tasks.workunit.client.0.smithi107.stdout:5/841: truncate d5/d79/d89/da3/f10b 564916 0 2024-03-04T13:43:19.599 INFO:tasks.workunit.client.0.smithi107.stdout:3/764: read d1/de/d18/d33/d48/d71/fa9 [738615,128804] 0 2024-03-04T13:43:19.604 INFO:tasks.workunit.client.0.smithi107.stdout:1/801: creat d1/d46/d6a/d4e/daa/dc9/d33/d5d/d4c/dea/ffe x:0 0 0 2024-03-04T13:43:19.607 INFO:tasks.workunit.client.0.smithi107.stdout:1/802: chown d1/d46/d6a/d4e/daa/dc9/d33/d89 1 1 2024-03-04T13:43:19.610 INFO:tasks.workunit.client.0.smithi107.stdout:1/803: stat d1/d46/d6a/d4e/daa/dc9/d33/d5d 0 2024-03-04T13:43:19.610 INFO:tasks.workunit.client.0.smithi107.stdout:2/705: symlink da/d38/d3a/d5d/lef 0 2024-03-04T13:43:19.611 INFO:tasks.workunit.client.0.smithi107.stdout:8/783: dread d6/da/d25/d5f/f98 [0,4194304] 0 2024-03-04T13:43:19.612 INFO:tasks.workunit.client.0.smithi107.stdout:3/765: symlink d1/de/d4b/lf5 0 2024-03-04T13:43:19.613 INFO:tasks.workunit.client.0.smithi107.stdout:8/784: chown d6/da/d25/d2a/c48 58185 1 2024-03-04T13:43:19.616 INFO:tasks.workunit.client.0.smithi107.stdout:3/766: truncate d1/de/d18/d33/d6b/fb1 186083 0 2024-03-04T13:43:19.619 INFO:tasks.workunit.client.0.smithi107.stdout:2/706: unlink da/d29/d7c/cd3 0 2024-03-04T13:43:19.620 INFO:tasks.workunit.client.0.smithi107.stdout:2/707: chown da/df/c3d 247216577 1 2024-03-04T13:43:19.622 INFO:tasks.workunit.client.0.smithi107.stdout:5/842: truncate d5/dc/f58 2335522 0 2024-03-04T13:43:19.625 INFO:tasks.workunit.client.0.smithi107.stdout:8/785: dread d6/fd [0,4194304] 0 2024-03-04T13:43:19.625 INFO:tasks.workunit.client.0.smithi107.stdout:7/730: dread d6/d13/d16/d19/d38/d37/f9e [0,4194304] 0 2024-03-04T13:43:19.626 INFO:tasks.workunit.client.0.smithi107.stdout:3/767: creat d1/de/d18/d33/d48/ff6 x:0 0 0 2024-03-04T13:43:19.629 INFO:tasks.workunit.client.0.smithi107.stdout:2/708: rename da/d49 to da/d34/df0 0 2024-03-04T13:43:19.633 INFO:tasks.workunit.client.0.smithi107.stdout:8/786: mknod d6/d16/d69/deb/d7c/cf2 0 2024-03-04T13:43:19.636 INFO:tasks.workunit.client.0.smithi107.stdout:7/731: creat d6/d13/d16/d1c/ff2 x:0 0 0 2024-03-04T13:43:19.636 INFO:tasks.workunit.client.0.smithi107.stdout:5/843: dwrite d5/d8/d2a/fb7 [0,4194304] 0 2024-03-04T13:43:19.638 INFO:tasks.workunit.client.0.smithi107.stdout:3/768: creat d1/de/d18/d33/d48/d71/d4c/d54/db7/ff7 x:0 0 0 2024-03-04T13:43:19.639 INFO:tasks.workunit.client.0.smithi107.stdout:8/787: symlink d6/dcc/lf3 0 2024-03-04T13:43:19.639 INFO:tasks.workunit.client.0.smithi107.stdout:7/732: truncate d6/d13/d16/d1c/f2d 4957070 0 2024-03-04T13:43:19.645 INFO:tasks.workunit.client.0.smithi107.stdout:5/844: mknod d5/d8/d14/d2d/d11d/d127/c12c 0 2024-03-04T13:43:19.645 INFO:tasks.workunit.client.0.smithi107.stdout:3/769: mknod d1/d1e/dba/cf8 0 2024-03-04T13:43:19.645 INFO:tasks.workunit.client.0.smithi107.stdout:8/788: symlink d6/d16/d1f/d4c/da5/lf4 0 2024-03-04T13:43:19.653 INFO:tasks.workunit.client.0.smithi107.stdout:7/733: dwrite d6/d17/f6d [4194304,4194304] 0 2024-03-04T13:43:19.654 INFO:tasks.workunit.client.0.smithi107.stdout:8/789: rename d6/d16/d1f/d4c/da5/lf4 to d6/d16/d1f/d4c/da4/daa/lf5 0 2024-03-04T13:43:19.661 INFO:tasks.workunit.client.0.smithi107.stdout:5/845: creat d5/dc/f12d x:0 0 0 2024-03-04T13:43:19.661 INFO:tasks.workunit.client.0.smithi107.stdout:3/770: getdents d1 0 2024-03-04T13:43:19.668 INFO:tasks.workunit.client.0.smithi107.stdout:8/790: link d6/d16/d69/deb/d3e/f6d d6/d16/d69/deb/d3e/d67/ff6 0 2024-03-04T13:43:19.672 INFO:tasks.workunit.client.0.smithi107.stdout:4/750: dwrite d0/d9/d10/d80/db8/d21/f6d [0,4194304] 0 2024-03-04T13:43:19.708 INFO:tasks.workunit.client.0.smithi107.stdout:2/709: write da/d34/f72 [1695603,82025] 0 2024-03-04T13:43:19.710 INFO:tasks.workunit.client.0.smithi107.stdout:1/804: dwrite d1/d10/d16/d47/f77 [0,4194304] 0 2024-03-04T13:43:19.715 INFO:tasks.workunit.client.0.smithi107.stdout:4/751: write d0/d9/de/d14/d33/f45 [458387,5679] 0 2024-03-04T13:43:19.715 INFO:tasks.workunit.client.0.smithi107.stdout:5/846: dwrite d5/d8/d2a/f9d [0,4194304] 0 2024-03-04T13:43:19.718 INFO:tasks.workunit.client.0.smithi107.stdout:1/805: write d1/d46/d6a/d4e/d60/fc4 [111126,4807] 0 2024-03-04T13:43:19.719 INFO:tasks.workunit.client.0.smithi107.stdout:2/710: read da/df/d27/f5e [910970,52387] 0 2024-03-04T13:43:19.723 INFO:tasks.workunit.client.0.smithi107.stdout:5/847: symlink d5/dc/d22/l12e 0 2024-03-04T13:43:19.726 INFO:tasks.workunit.client.0.smithi107.stdout:4/752: rename d0/d30 to d0/d9/d18/df1 0 2024-03-04T13:43:19.728 INFO:tasks.workunit.client.0.smithi107.stdout:1/806: mknod d1/d46/d6a/d65/cff 0 2024-03-04T13:43:19.728 INFO:tasks.workunit.client.0.smithi107.stdout:4/753: chown d0/d9/c58 14136 1 2024-03-04T13:43:19.730 INFO:tasks.workunit.client.0.smithi107.stdout:5/848: mkdir d5/d8/d14/d2d/d92/dd8/d12f 0 2024-03-04T13:43:19.732 INFO:tasks.workunit.client.0.smithi107.stdout:4/754: dread - d0/d9/d10/d80/db8/d97/fe1 zero size 2024-03-04T13:43:19.732 INFO:tasks.workunit.client.0.smithi107.stdout:7/734: dread d6/d13/d16/d19/d38/d2c/f87 [0,4194304] 0 2024-03-04T13:43:19.736 INFO:tasks.workunit.client.0.smithi107.stdout:2/711: getdents da/df/d73/dd1 0 2024-03-04T13:43:19.739 INFO:tasks.workunit.client.0.smithi107.stdout:4/755: link d0/d9/d18/d29/f35 d0/d9/d18/df1/dc1/ff2 0 2024-03-04T13:43:19.741 INFO:tasks.workunit.client.0.smithi107.stdout:7/735: creat d6/d1d/ff3 x:0 0 0 2024-03-04T13:43:19.743 INFO:tasks.workunit.client.0.smithi107.stdout:0/804: sync 2024-03-04T13:43:19.743 INFO:tasks.workunit.client.0.smithi107.stdout:2/712: creat da/df/d27/d58/d5f/ff1 x:0 0 0 2024-03-04T13:43:19.746 INFO:tasks.workunit.client.0.smithi107.stdout:2/713: dread - da/df/d27/d58/d5f/ff1 zero size 2024-03-04T13:43:19.747 INFO:tasks.workunit.client.0.smithi107.stdout:5/849: dwrite d5/dc/df/f2e [0,4194304] 0 2024-03-04T13:43:19.750 INFO:tasks.workunit.client.0.smithi107.stdout:7/736: creat d6/d13/d16/d19/d39/d7a/ff4 x:0 0 0 2024-03-04T13:43:19.751 INFO:tasks.workunit.client.0.smithi107.stdout:8/791: truncate d6/da/d25/d5f/f98 3385413 0 2024-03-04T13:43:19.752 INFO:tasks.workunit.client.0.smithi107.stdout:2/714: mkdir da/df/d73/db3/db1/df2 0 2024-03-04T13:43:19.757 INFO:tasks.workunit.client.0.smithi107.stdout:7/737: dread - d6/d13/d16/d19/d38/fb6 zero size 2024-03-04T13:43:19.758 INFO:tasks.workunit.client.0.smithi107.stdout:0/805: dread f4 [4194304,4194304] 0 2024-03-04T13:43:19.761 INFO:tasks.workunit.client.0.smithi107.stdout:2/715: mknod da/df/d3e/cf3 0 2024-03-04T13:43:19.763 INFO:tasks.workunit.client.0.smithi107.stdout:0/806: dread - da/dc/dd/d3e/d26/d88/dd2/d73/d78/f104 zero size 2024-03-04T13:43:19.765 INFO:tasks.workunit.client.0.smithi107.stdout:7/738: unlink d6/d13/d16/d1c/ff2 0 2024-03-04T13:43:19.773 INFO:tasks.workunit.client.0.smithi107.stdout:4/756: dwrite d0/d9/d10/f1c [0,4194304] 0 2024-03-04T13:43:19.773 INFO:tasks.workunit.client.0.smithi107.stdout:1/807: dwrite d1/d46/d6a/d4e/d60/f4f [0,4194304] 0 2024-03-04T13:43:19.774 INFO:tasks.workunit.client.0.smithi107.stdout:8/792: dwrite d6/f26 [0,4194304] 0 2024-03-04T13:43:19.777 INFO:tasks.workunit.client.0.smithi107.stdout:2/716: creat da/d34/df0/d5c/d6a/ff4 x:0 0 0 2024-03-04T13:43:19.779 INFO:tasks.workunit.client.0.smithi107.stdout:8/793: chown d6/da/d25/d2a/f95 1653493236 1 2024-03-04T13:43:19.779 INFO:tasks.workunit.client.0.smithi107.stdout:5/850: write d5/d79/d89/da3/d68/f97 [2071804,48508] 0 2024-03-04T13:43:19.782 INFO:tasks.workunit.client.0.smithi107.stdout:4/757: creat d0/d9/d18/df1/ff3 x:0 0 0 2024-03-04T13:43:19.794 INFO:tasks.workunit.client.0.smithi107.stdout:5/851: unlink d5/dc/df/l100 0 2024-03-04T13:43:19.795 INFO:tasks.workunit.client.0.smithi107.stdout:7/739: dwrite d6/d13/d16/d19/d38/d37/f9e [0,4194304] 0 2024-03-04T13:43:19.797 INFO:tasks.workunit.client.0.smithi107.stdout:0/807: truncate da/dc/dd/dc9/de5/f90 1464646 0 2024-03-04T13:43:19.799 INFO:tasks.workunit.client.0.smithi107.stdout:7/740: stat d6/d1d/d36/c96 0 2024-03-04T13:43:19.800 INFO:tasks.workunit.client.0.smithi107.stdout:5/852: write d5/d8/d14/d2d/d92/dd8/f114 [510054,34633] 0 2024-03-04T13:43:19.805 INFO:tasks.workunit.client.0.smithi107.stdout:4/758: rename d0/d9/de/d14/d66/cde to d0/d9/dd2/cf4 0 2024-03-04T13:43:19.805 INFO:tasks.workunit.client.0.smithi107.stdout:0/808: creat da/dc/dd/d1e/f112 x:0 0 0 2024-03-04T13:43:19.805 INFO:tasks.workunit.client.0.smithi107.stdout:5/853: chown d5/d79/d89/fe1 8756 1 2024-03-04T13:43:19.806 INFO:tasks.workunit.client.0.smithi107.stdout:7/741: truncate d6/d13/d16/fbb 5036278 0 2024-03-04T13:43:19.810 INFO:tasks.workunit.client.0.smithi107.stdout:9/728: sync 2024-03-04T13:43:19.810 INFO:tasks.workunit.client.0.smithi107.stdout:6/789: sync 2024-03-04T13:43:19.810 INFO:tasks.workunit.client.0.smithi107.stdout:4/759: unlink d0/d9/d1e/l4f 0 2024-03-04T13:43:19.815 INFO:tasks.workunit.client.0.smithi107.stdout:7/742: stat d6/d1d/d64/d9a/da9/d71/db9/dca 0 2024-03-04T13:43:19.819 INFO:tasks.workunit.client.0.smithi107.stdout:2/717: truncate da/df/d27/d58/d5f/f8c 144520 0 2024-03-04T13:43:19.819 INFO:tasks.workunit.client.0.smithi107.stdout:9/729: chown d4/da/d11/d20/d23/f6c 298785710 1 2024-03-04T13:43:19.823 INFO:tasks.workunit.client.0.smithi107.stdout:9/730: stat d4/l18 0 2024-03-04T13:43:19.824 INFO:tasks.workunit.client.0.smithi107.stdout:0/809: chown da/dc/dd/d3e/d26/d88/dd2/d73/cef 403 1 2024-03-04T13:43:19.824 INFO:tasks.workunit.client.0.smithi107.stdout:1/808: link d1/d46/d6a/d4e/daa/dc9/d33/d5d/d4c/dea/d59/lcc d1/d46/d6a/d4e/daa/dc9/d33/dbd/l100 0 2024-03-04T13:43:19.827 INFO:tasks.workunit.client.0.smithi107.stdout:8/794: dwrite d6/d16/d1f/dbd/fd3 [0,4194304] 0 2024-03-04T13:43:19.828 INFO:tasks.workunit.client.0.smithi107.stdout:4/760: creat d0/d9/d10/d80/db8/ff5 x:0 0 0 2024-03-04T13:43:19.831 INFO:tasks.workunit.client.0.smithi107.stdout:9/731: stat d4/da/d11/d4c/f3c 0 2024-03-04T13:43:19.835 INFO:tasks.workunit.client.0.smithi107.stdout:9/732: stat d4/da/d11/d65 0 2024-03-04T13:43:19.835 INFO:tasks.workunit.client.0.smithi107.stdout:4/761: dread - d0/d9/d10/d6f/fad zero size 2024-03-04T13:43:19.843 INFO:tasks.workunit.client.0.smithi107.stdout:8/795: read d6/d16/f5e [383155,86337] 0 2024-03-04T13:43:19.850 INFO:tasks.workunit.client.0.smithi107.stdout:7/743: dwrite d6/d13/d16/fe2 [0,4194304] 0 2024-03-04T13:43:19.851 INFO:tasks.workunit.client.0.smithi107.stdout:2/718: link da/df/d3e/c47 da/d44/d8b/d9d/dd0/cf5 0 2024-03-04T13:43:19.852 INFO:tasks.workunit.client.0.smithi107.stdout:8/796: write d6/da/d25/d5f/f87 [2343937,5035] 0 2024-03-04T13:43:19.861 INFO:tasks.workunit.client.0.smithi107.stdout:6/790: dwrite df/f15 [0,4194304] 0 2024-03-04T13:43:19.862 INFO:tasks.workunit.client.0.smithi107.stdout:8/797: truncate d6/d16/d1f/dbd/fd3 4701685 0 2024-03-04T13:43:19.862 INFO:tasks.workunit.client.0.smithi107.stdout:3/771: sync 2024-03-04T13:43:19.864 INFO:tasks.workunit.client.0.smithi107.stdout:9/733: link d4/da/d11/d20/d23/d7e/lbd d4/da/d11/d4c/d4d/d72/lf3 0 2024-03-04T13:43:19.866 INFO:tasks.workunit.client.0.smithi107.stdout:3/772: write d1/f6 [1722669,24197] 0 2024-03-04T13:43:19.868 INFO:tasks.workunit.client.0.smithi107.stdout:2/719: write da/f1c [986957,118910] 0 2024-03-04T13:43:19.868 INFO:tasks.workunit.client.0.smithi107.stdout:5/854: dread d5/d8/da/f16 [0,4194304] 0 2024-03-04T13:43:19.870 INFO:tasks.workunit.client.0.smithi107.stdout:4/762: write d0/fee [827803,448] 0 2024-03-04T13:43:19.871 INFO:tasks.workunit.client.0.smithi107.stdout:0/810: write da/dc/dd/dc9/de5/f100 [213640,124144] 0 2024-03-04T13:43:19.877 INFO:tasks.workunit.client.0.smithi107.stdout:4/763: write d0/d9/d10/f25 [2891422,116854] 0 2024-03-04T13:43:19.878 INFO:tasks.workunit.client.0.smithi107.stdout:8/798: rmdir d6/d16/d69 39 2024-03-04T13:43:19.885 INFO:tasks.workunit.client.0.smithi107.stdout:6/791: symlink df/d17/d24/d30/d8b/dc1/de1/lfe 0 2024-03-04T13:43:19.886 INFO:tasks.workunit.client.0.smithi107.stdout:7/744: dwrite d6/d13/d16/d19/d39/d7a/fc4 [0,4194304] 0 2024-03-04T13:43:19.903 INFO:tasks.workunit.client.0.smithi107.stdout:5/855: write d5/d8/d2a/f40 [3476889,35775] 0 2024-03-04T13:43:19.903 INFO:tasks.workunit.client.0.smithi107.stdout:1/809: dread d1/d46/d6a/d4e/f6c [0,4194304] 0 2024-03-04T13:43:19.908 INFO:tasks.workunit.client.0.smithi107.stdout:9/734: rename d4/da/d11/d20/d23/c33 to d4/da/d11/d20/d23/d7e/cf4 0 2024-03-04T13:43:19.913 INFO:tasks.workunit.client.0.smithi107.stdout:2/720: truncate f0 4165247 0 2024-03-04T13:43:19.913 INFO:tasks.workunit.client.0.smithi107.stdout:7/745: mknod d6/d1d/d64/cf5 0 2024-03-04T13:43:19.913 INFO:tasks.workunit.client.0.smithi107.stdout:1/810: write d1/d46/d6a/d4e/fcf [1704667,1463] 0 2024-03-04T13:43:19.920 INFO:tasks.workunit.client.0.smithi107.stdout:2/721: read da/df/d3e/f40 [606447,45520] 0 2024-03-04T13:43:19.926 INFO:tasks.workunit.client.0.smithi107.stdout:2/722: creat da/d44/d8b/d98/dc3/ff6 x:0 0 0 2024-03-04T13:43:19.933 INFO:tasks.workunit.client.0.smithi107.stdout:6/792: dwrite df/fac [0,4194304] 0 2024-03-04T13:43:19.934 INFO:tasks.workunit.client.0.smithi107.stdout:1/811: creat d1/d46/d6a/d4e/daa/dc9/d33/d5d/d4c/dea/d5b/f101 x:0 0 0 2024-03-04T13:43:19.935 INFO:tasks.workunit.client.0.smithi107.stdout:2/723: creat da/df/d27/d58/ff7 x:0 0 0 2024-03-04T13:43:19.946 INFO:tasks.workunit.client.0.smithi107.stdout:6/793: getdents df/d2e 0 2024-03-04T13:43:19.949 INFO:tasks.workunit.client.0.smithi107.stdout:2/724: rename da/d34/df0 to da/d44/d8b/d9d/df8 0 2024-03-04T13:43:19.951 INFO:tasks.workunit.client.0.smithi107.stdout:8/799: dread d6/d16/d69/deb/d3e/fb3 [0,4194304] 0 2024-03-04T13:43:19.961 INFO:tasks.workunit.client.0.smithi107.stdout:2/725: write da/d34/f36 [1228566,123282] 0 2024-03-04T13:43:19.962 INFO:tasks.workunit.client.0.smithi107.stdout:8/800: mknod d6/da/da8/cf7 0 2024-03-04T13:43:19.962 INFO:tasks.workunit.client.0.smithi107.stdout:3/773: dread d1/d1e/d26/f6f [0,4194304] 0 2024-03-04T13:43:19.963 INFO:tasks.workunit.client.0.smithi107.stdout:7/746: dwrite d6/d1d/d36/fae [0,4194304] 0 2024-03-04T13:43:19.965 INFO:tasks.workunit.client.0.smithi107.stdout:5/856: dwrite d5/d79/d89/da3/d6a/f6d [4194304,4194304] 0 2024-03-04T13:43:19.971 INFO:tasks.workunit.client.0.smithi107.stdout:8/801: creat d6/d16/d69/deb/d7c/ff8 x:0 0 0 2024-03-04T13:43:19.973 INFO:tasks.workunit.client.0.smithi107.stdout:2/726: creat da/d38/d3a/d5d/ff9 x:0 0 0 2024-03-04T13:43:19.973 INFO:tasks.workunit.client.0.smithi107.stdout:6/794: write df/d17/d24/d30/d5d/d6a/fde [4637178,67193] 0 2024-03-04T13:43:19.976 INFO:tasks.workunit.client.0.smithi107.stdout:1/812: dread d1/d46/d6a/d4e/daa/dc9/d67/d6b/f84 [0,4194304] 0 2024-03-04T13:43:19.979 INFO:tasks.workunit.client.0.smithi107.stdout:3/774: symlink d1/d1e/d26/d9e/lf9 0 2024-03-04T13:43:19.979 INFO:tasks.workunit.client.0.smithi107.stdout:0/811: dread da/dc/dd/d3e/fac [0,4194304] 0 2024-03-04T13:43:19.982 INFO:tasks.workunit.client.0.smithi107.stdout:6/795: dread - df/d38/d4f/d74/f98 zero size 2024-03-04T13:43:19.983 INFO:tasks.workunit.client.0.smithi107.stdout:4/764: dread d0/d9/d1e/f20 [0,4194304] 0 2024-03-04T13:43:19.986 INFO:tasks.workunit.client.0.smithi107.stdout:3/775: rmdir d1/de/d4b 39 2024-03-04T13:43:19.989 INFO:tasks.workunit.client.0.smithi107.stdout:7/747: creat d6/d13/d16/d19/d38/ff6 x:0 0 0 2024-03-04T13:43:19.990 INFO:tasks.workunit.client.0.smithi107.stdout:6/796: symlink df/d38/d4f/daf/lff 0 2024-03-04T13:43:19.994 INFO:tasks.workunit.client.0.smithi107.stdout:5/857: link d5/d2b/d4b/da6/dcf/le3 d5/d8/d2a/l130 0 2024-03-04T13:43:19.996 INFO:tasks.workunit.client.0.smithi107.stdout:7/748: dread - d6/d1d/d64/d9a/da9/d71/db9/dca/fd3 zero size 2024-03-04T13:43:19.997 INFO:tasks.workunit.client.0.smithi107.stdout:1/813: getdents d1/d10/d16/d47/d7a/dda/dec 0 2024-03-04T13:43:20.000 INFO:tasks.workunit.client.0.smithi107.stdout:0/812: creat da/dc/dd/d3e/d26/d69/d6b/df0/d10e/f113 x:0 0 0 2024-03-04T13:43:20.000 INFO:tasks.workunit.client.0.smithi107.stdout:8/802: rename d6/d16/d69/deb/d3e/c44 to d6/d16/d1f/cf9 0 2024-03-04T13:43:20.001 INFO:tasks.workunit.client.0.smithi107.stdout:6/797: creat df/d17/d24/d40/d63/da3/d7e/d86/f100 x:0 0 0 2024-03-04T13:43:20.004 INFO:tasks.workunit.client.0.smithi107.stdout:1/814: write d1/d46/d6a/fc5 [599992,107175] 0 2024-03-04T13:43:20.011 INFO:tasks.workunit.client.0.smithi107.stdout:8/803: mknod d6/d16/d1f/d4c/dec/cfa 0 2024-03-04T13:43:20.012 INFO:tasks.workunit.client.0.smithi107.stdout:6/798: fdatasync df/d2e/d36/d77/f90 0 2024-03-04T13:43:20.017 INFO:tasks.workunit.client.0.smithi107.stdout:2/727: dwrite da/d44/d8b/fa9 [0,4194304] 0 2024-03-04T13:43:20.018 INFO:tasks.workunit.client.0.smithi107.stdout:2/728: chown da/f26 5 1 2024-03-04T13:43:20.020 INFO:tasks.workunit.client.0.smithi107.stdout:2/729: stat da/df/d85 0 2024-03-04T13:43:20.021 INFO:tasks.workunit.client.0.smithi107.stdout:3/776: truncate d1/de/d18/d1f/d42/f5d 2486709 0 2024-03-04T13:43:20.021 INFO:tasks.workunit.client.0.smithi107.stdout:5/858: write d5/d8/d2a/d43/f4a [683303,20127] 0 2024-03-04T13:43:20.021 INFO:tasks.workunit.client.0.smithi107.stdout:4/765: dwrite d0/d9/d18/df1/f3d [0,4194304] 0 2024-03-04T13:43:20.022 INFO:tasks.workunit.client.0.smithi107.stdout:5/859: read - d5/dc/f12d zero size 2024-03-04T13:43:20.066 INFO:tasks.workunit.client.0.smithi107.stdout:7/749: creat d6/d1d/d64/d9a/da9/d71/db9/dca/d3d/dc6/ff7 x:0 0 0 2024-03-04T13:43:20.070 INFO:tasks.workunit.client.0.smithi107.stdout:8/804: mkdir d6/d16/d1f/d4c/da5/dfb 0 2024-03-04T13:43:20.074 INFO:tasks.workunit.client.0.smithi107.stdout:4/766: stat d0/l8 0 2024-03-04T13:43:20.077 INFO:tasks.workunit.client.0.smithi107.stdout:4/767: dread d0/d9/fb2 [0,4194304] 0 2024-03-04T13:43:20.078 INFO:tasks.workunit.client.0.smithi107.stdout:2/730: creat da/df/d73/d7a/d9a/d68/ffa x:0 0 0 2024-03-04T13:43:20.083 INFO:tasks.workunit.client.0.smithi107.stdout:8/805: mkdir d6/dcc/dfc 0 2024-03-04T13:43:20.084 INFO:tasks.workunit.client.0.smithi107.stdout:1/815: creat d1/d46/d6a/d4e/daa/dc9/d33/dbd/f102 x:0 0 0 2024-03-04T13:43:20.085 INFO:tasks.workunit.client.0.smithi107.stdout:7/750: dread d6/d1d/d64/d9a/da9/fc1 [0,4194304] 0 2024-03-04T13:43:20.086 INFO:tasks.workunit.client.0.smithi107.stdout:1/816: write d1/d10/fb0 [8485227,109458] 0 2024-03-04T13:43:20.086 INFO:tasks.workunit.client.0.smithi107.stdout:3/777: truncate d1/de/d11/f95 247501 0 2024-03-04T13:43:20.095 INFO:tasks.workunit.client.0.smithi107.stdout:5/860: mknod d5/d8/d14/d121/c131 0 2024-03-04T13:43:20.095 INFO:tasks.workunit.client.0.smithi107.stdout:4/768: symlink d0/d9/de/d14/d66/lf6 0 2024-03-04T13:43:20.097 INFO:tasks.workunit.client.0.smithi107.stdout:8/806: rmdir d6 39 2024-03-04T13:43:20.100 INFO:tasks.workunit.client.0.smithi107.stdout:7/751: symlink d6/d1d/d64/d9a/da9/d71/db9/dca/d3d/dc6/lf8 0 2024-03-04T13:43:20.104 INFO:tasks.workunit.client.0.smithi107.stdout:1/817: dread d1/d46/d6a/d4e/daa/dc9/d67/d6b/f9e [0,4194304] 0 2024-03-04T13:43:20.105 INFO:tasks.workunit.client.0.smithi107.stdout:4/769: unlink d0/d9/d10/d80/db8/d97/lce 0 2024-03-04T13:43:20.107 INFO:tasks.workunit.client.0.smithi107.stdout:8/807: read d6/da/f1b [1363141,4875] 0 2024-03-04T13:43:20.107 INFO:tasks.workunit.client.0.smithi107.stdout:3/778: symlink d1/de/d11/dd5/dab/dc5/lfa 0 2024-03-04T13:43:20.109 INFO:tasks.workunit.client.0.smithi107.stdout:6/799: write df/f29 [223139,13857] 0 2024-03-04T13:43:20.110 INFO:tasks.workunit.client.0.smithi107.stdout:4/770: creat d0/d9/d10/dab/ff7 x:0 0 0 2024-03-04T13:43:20.112 INFO:tasks.workunit.client.0.smithi107.stdout:1/818: dread d1/f9a [0,4194304] 0 2024-03-04T13:43:20.112 INFO:tasks.workunit.client.0.smithi107.stdout:3/779: dread d1/de/d18/d1f/d42/f62 [0,4194304] 0 2024-03-04T13:43:20.115 INFO:tasks.workunit.client.0.smithi107.stdout:2/731: rmdir da/df 39 2024-03-04T13:43:20.119 INFO:tasks.workunit.client.0.smithi107.stdout:8/808: link d6/f45 d6/d16/d69/deb/d7c/ffd 0 2024-03-04T13:43:20.121 INFO:tasks.workunit.client.0.smithi107.stdout:1/819: creat d1/d46/d6a/d4e/daa/dc9/d67/d6b/f103 x:0 0 0 2024-03-04T13:43:20.124 INFO:tasks.workunit.client.0.smithi107.stdout:1/820: dread - d1/d46/d6a/d4e/daa/dc9/d33/d5d/fbf zero size 2024-03-04T13:43:20.125 INFO:tasks.workunit.client.0.smithi107.stdout:3/780: symlink d1/de/d18/d33/d48/d71/d4c/lfb 0 2024-03-04T13:43:20.128 INFO:tasks.workunit.client.0.smithi107.stdout:4/771: unlink d0/d9/dd2/cf4 0 2024-03-04T13:43:20.129 INFO:tasks.workunit.client.0.smithi107.stdout:2/732: dwrite da/d44/f52 [0,4194304] 0 2024-03-04T13:43:20.129 INFO:tasks.workunit.client.0.smithi107.stdout:6/800: dwrite df/d17/d24/d40/d63/da3/d7e/d86/f97 [0,4194304] 0 2024-03-04T13:43:20.131 INFO:tasks.workunit.client.0.smithi107.stdout:2/733: fdatasync da/d29/d7c/dbd/fda 0 2024-03-04T13:43:20.132 INFO:tasks.workunit.client.0.smithi107.stdout:8/809: creat d6/d16/d69/deb/d3e/d96/ffe x:0 0 0 2024-03-04T13:43:20.137 INFO:tasks.workunit.client.0.smithi107.stdout:5/861: write d5/d8/da/fa1 [888764,118602] 0 2024-03-04T13:43:20.138 INFO:tasks.workunit.client.0.smithi107.stdout:4/772: symlink d0/d9/d18/df1/lf8 0 2024-03-04T13:43:20.142 INFO:tasks.workunit.client.0.smithi107.stdout:4/773: dread d0/d9/d78/fa7 [0,4194304] 0 2024-03-04T13:43:20.143 INFO:tasks.workunit.client.0.smithi107.stdout:8/810: creat d6/dcc/fff x:0 0 0 2024-03-04T13:43:20.144 INFO:tasks.workunit.client.0.smithi107.stdout:4/774: stat d0/d2f/ld8 0 2024-03-04T13:43:20.145 INFO:tasks.workunit.client.0.smithi107.stdout:7/752: truncate d6/d13/d16/d19/d60/fe6 2879435 0 2024-03-04T13:43:20.147 INFO:tasks.workunit.client.0.smithi107.stdout:2/734: creat da/d44/d8b/d98/dc3/ffb x:0 0 0 2024-03-04T13:43:20.155 INFO:tasks.workunit.client.0.smithi107.stdout:3/781: dwrite d1/de/d18/d33/d48/d71/d4c/fb9 [0,4194304] 0 2024-03-04T13:43:20.159 INFO:tasks.workunit.client.0.smithi107.stdout:1/821: creat d1/d10/d16/d47/f104 x:0 0 0 2024-03-04T13:43:20.159 INFO:tasks.workunit.client.0.smithi107.stdout:7/753: creat d6/d13/d16/d1c/dc2/ff9 x:0 0 0 2024-03-04T13:43:20.161 INFO:tasks.workunit.client.0.smithi107.stdout:6/801: mkdir df/d17/d24/d30/d3b/deb/d101 0 2024-03-04T13:43:20.162 INFO:tasks.workunit.client.0.smithi107.stdout:3/782: write d1/de/d18/d33/d48/d71/f49 [4341947,23637] 0 2024-03-04T13:43:20.164 INFO:tasks.workunit.client.0.smithi107.stdout:5/862: rename d5/d8/l2c to d5/dc/df/l132 0 2024-03-04T13:43:20.166 INFO:tasks.workunit.client.0.smithi107.stdout:5/863: fdatasync d5/f6 0 2024-03-04T13:43:20.168 INFO:tasks.workunit.client.0.smithi107.stdout:3/783: write d1/d41/d81/fb5 [1098363,113967] 0 2024-03-04T13:43:20.171 INFO:tasks.workunit.client.0.smithi107.stdout:3/784: chown d1/de/d18/d33/d48/d71/f15 13654 1 2024-03-04T13:43:20.176 INFO:tasks.workunit.client.0.smithi107.stdout:3/785: creat d1/de/dad/dbb/ffc x:0 0 0 2024-03-04T13:43:20.177 INFO:tasks.workunit.client.0.smithi107.stdout:7/754: link d6/d13/d16/d19/d38/d2c/l6c d6/d17/lfa 0 2024-03-04T13:43:20.178 INFO:tasks.workunit.client.0.smithi107.stdout:4/775: rename d0/d9/dd2 to d0/d9/d10/d80/db8/d9a/df9 0 2024-03-04T13:43:20.178 INFO:tasks.workunit.client.0.smithi107.stdout:7/755: readlink d6/d1d/l6f 0 2024-03-04T13:43:20.179 INFO:tasks.workunit.client.0.smithi107.stdout:4/776: readlink d0/l5e 0 2024-03-04T13:43:20.181 INFO:tasks.workunit.client.0.smithi107.stdout:3/786: mkdir d1/d1e/d26/dfd 0 2024-03-04T13:43:20.183 INFO:tasks.workunit.client.0.smithi107.stdout:4/777: symlink d0/d9/d10/d80/db8/d9a/df9/lfa 0 2024-03-04T13:43:20.184 INFO:tasks.workunit.client.0.smithi107.stdout:7/756: stat d6/d1d/l98 0 2024-03-04T13:43:20.188 INFO:tasks.workunit.client.0.smithi107.stdout:3/787: read d1/de/d18/d33/d48/d71/f72 [846980,129686] 0 2024-03-04T13:43:20.189 INFO:tasks.workunit.client.0.smithi107.stdout:3/788: dread - d1/d2d/fef zero size 2024-03-04T13:43:20.190 INFO:tasks.workunit.client.0.smithi107.stdout:7/757: write d6/d1d/d64/d9a/da9/d71/db9/dca/fd3 [576455,37231] 0 2024-03-04T13:43:20.193 INFO:tasks.workunit.client.0.smithi107.stdout:7/758: chown d6/d13/d14/f86 37 1 2024-03-04T13:43:20.194 INFO:tasks.workunit.client.0.smithi107.stdout:3/789: creat d1/de/d18/d33/d4d/ffe x:0 0 0 2024-03-04T13:43:20.196 INFO:tasks.workunit.client.0.smithi107.stdout:7/759: dread - d6/d1d/d64/d9a/da9/d71/db9/dca/d3d/dc6/ff1 zero size 2024-03-04T13:43:20.198 INFO:tasks.workunit.client.0.smithi107.stdout:3/790: creat d1/fff x:0 0 0 2024-03-04T13:43:20.202 INFO:tasks.workunit.client.0.smithi107.stdout:7/760: mkdir d6/d1d/d64/d9a/da9/d71/db9/dca/d3d/dc6/dfb 0 2024-03-04T13:43:20.204 INFO:tasks.workunit.client.0.smithi107.stdout:5/864: dread d5/dc/f55 [0,4194304] 0 2024-03-04T13:43:20.205 INFO:tasks.workunit.client.0.smithi107.stdout:6/802: truncate df/d17/d24/d30/d8b/f87 4084967 0 2024-03-04T13:43:20.209 INFO:tasks.workunit.client.0.smithi107.stdout:7/761: creat d6/d1d/d64/d9a/da9/d71/db9/ffc x:0 0 0 2024-03-04T13:43:20.209 INFO:tasks.workunit.client.0.smithi107.stdout:1/822: truncate d1/d46/d6a/d4e/daa/dc9/d33/d5d/d4c/dea/d5b/d61/fce 506704 0 2024-03-04T13:43:20.210 INFO:tasks.workunit.client.0.smithi107.stdout:8/811: dwrite d6/da/da8/db5/fd6 [0,4194304] 0 2024-03-04T13:43:20.213 INFO:tasks.workunit.client.0.smithi107.stdout:5/865: rename d5/d8/d14/d2d/dfa/d109 to d5/d8/d14/d2d/d92/dc1/d133 0 2024-03-04T13:43:20.214 INFO:tasks.workunit.client.0.smithi107.stdout:6/803: stat df/d17/d24/c7a 0 2024-03-04T13:43:20.215 INFO:tasks.workunit.client.0.smithi107.stdout:7/762: mkdir d6/d13/d16/dfd 0 2024-03-04T13:43:20.217 INFO:tasks.workunit.client.0.smithi107.stdout:1/823: mknod d1/d46/d6a/c105 0 2024-03-04T13:43:20.220 INFO:tasks.workunit.client.0.smithi107.stdout:8/812: creat d6/d16/d69/deb/d7c/f100 x:0 0 0 2024-03-04T13:43:20.224 INFO:tasks.workunit.client.0.smithi107.stdout:7/763: rmdir d6/d13/d16/d19/d38 39 2024-03-04T13:43:20.225 INFO:tasks.workunit.client.0.smithi107.stdout:4/778: dwrite d0/d9/d18/df1/f9c [0,4194304] 0 2024-03-04T13:43:20.228 INFO:tasks.workunit.client.0.smithi107.stdout:8/813: unlink d6/d16/d69/deb/d7c/cbc 0 2024-03-04T13:43:20.233 INFO:tasks.workunit.client.0.smithi107.stdout:6/804: creat df/d17/d6b/d9b/f102 x:0 0 0 2024-03-04T13:43:20.234 INFO:tasks.workunit.client.0.smithi107.stdout:6/805: fsync df/f15 0 2024-03-04T13:43:20.234 INFO:tasks.workunit.client.0.smithi107.stdout:5/866: dwrite d5/d2b/ff7 [0,4194304] 0 2024-03-04T13:43:20.236 INFO:tasks.workunit.client.0.smithi107.stdout:7/764: creat d6/d13/d16/d19/d38/d6e/ffe x:0 0 0 2024-03-04T13:43:20.237 INFO:tasks.workunit.client.0.smithi107.stdout:3/791: dread d1/de/d18/d33/d48/d71/f49 [0,4194304] 0 2024-03-04T13:43:20.241 INFO:tasks.workunit.client.0.smithi107.stdout:5/867: creat d5/d79/d89/da3/d69/f134 x:0 0 0 2024-03-04T13:43:20.241 INFO:tasks.workunit.client.0.smithi107.stdout:6/806: mknod df/d2e/d36/d77/c103 0 2024-03-04T13:43:20.242 INFO:tasks.workunit.client.0.smithi107.stdout:3/792: write d1/de/d18/d33/f8f [3799951,75390] 0 2024-03-04T13:43:20.244 INFO:tasks.workunit.client.0.smithi107.stdout:3/793: stat d1/de/d3f/f80 0 2024-03-04T13:43:20.248 INFO:tasks.workunit.client.0.smithi107.stdout:4/779: dwrite d0/d9/d10/d80/db8/d97/fb1 [0,4194304] 0 2024-03-04T13:43:20.249 INFO:tasks.workunit.client.0.smithi107.stdout:5/868: truncate d5/d79/d89/da3/d69/f126 104837 0 2024-03-04T13:43:20.251 INFO:tasks.workunit.client.0.smithi107.stdout:4/780: chown d0/d9/d18/df1/lac 1953 1 2024-03-04T13:43:20.253 INFO:tasks.workunit.client.0.smithi107.stdout:6/807: mkdir df/d2e/d36/d5b/d7d/dbd/d104 0 2024-03-04T13:43:20.254 INFO:tasks.workunit.client.0.smithi107.stdout:4/781: chown d0/d9/d10 15379989 1 2024-03-04T13:43:20.255 INFO:tasks.workunit.client.0.smithi107.stdout:6/808: fdatasync df/d38/d4f/fcf 0 2024-03-04T13:43:20.256 INFO:tasks.workunit.client.0.smithi107.stdout:5/869: chown d5/d2b/d4b/da6/f119 4043027 1 2024-03-04T13:43:20.257 INFO:tasks.workunit.client.0.smithi107.stdout:6/809: fsync df/d2e/ff9 0 2024-03-04T13:43:20.262 INFO:tasks.workunit.client.0.smithi107.stdout:5/870: dread d5/d8/d2a/d62/fa9 [0,4194304] 0 2024-03-04T13:43:20.264 INFO:tasks.workunit.client.0.smithi107.stdout:0/813: sync 2024-03-04T13:43:20.264 INFO:tasks.workunit.client.0.smithi107.stdout:9/735: sync 2024-03-04T13:43:20.266 INFO:tasks.workunit.client.0.smithi107.stdout:1/824: rename d1/d46/d6a/d4e/daa/dc9/d33/d5d/d4c/dea/d5b to d1/d46/d6a/d65/d106 0 2024-03-04T13:43:20.267 INFO:tasks.workunit.client.0.smithi107.stdout:9/736: dread - d4/da/d11/d4c/d51/fb8 zero size 2024-03-04T13:43:20.268 INFO:tasks.workunit.client.0.smithi107.stdout:6/810: mkdir df/d17/d6b/da8/d105 0 2024-03-04T13:43:20.269 INFO:tasks.workunit.client.0.smithi107.stdout:0/814: read da/dc/dd/d3e/f44 [3223082,75080] 0 2024-03-04T13:43:20.270 INFO:tasks.workunit.client.0.smithi107.stdout:3/794: link d1/d41/cdb d1/d2d/c100 0 2024-03-04T13:43:20.272 INFO:tasks.workunit.client.0.smithi107.stdout:7/765: dread d6/d1d/d64/d9a/da9/d71/db9/dca/fab [0,4194304] 0 2024-03-04T13:43:20.273 INFO:tasks.workunit.client.0.smithi107.stdout:0/815: fdatasync da/dc/dd/d3e/d26/d88/dd2/d73/d78/f104 0 2024-03-04T13:43:20.274 INFO:tasks.workunit.client.0.smithi107.stdout:7/766: stat d6/d1d/c73 0 2024-03-04T13:43:20.275 INFO:tasks.workunit.client.0.smithi107.stdout:1/825: creat d1/d10/d16/d47/d71/f107 x:0 0 0 2024-03-04T13:43:20.277 INFO:tasks.workunit.client.0.smithi107.stdout:3/795: mknod d1/de/c101 0 2024-03-04T13:43:20.278 INFO:tasks.workunit.client.0.smithi107.stdout:0/816: chown da/dc/dd/d3e/d26/d88/dd2/c6f 4454 1 2024-03-04T13:43:20.284 INFO:tasks.workunit.client.0.smithi107.stdout:9/737: link d4/c1e d4/da/cf5 0 2024-03-04T13:43:20.286 INFO:tasks.workunit.client.0.smithi107.stdout:1/826: rename d1/d46/d6a/d17/l8a to d1/d46/d6a/d4e/daa/dc9/d33/d5d/d24/l108 0 2024-03-04T13:43:20.288 INFO:tasks.workunit.client.0.smithi107.stdout:3/796: creat d1/de/d18/d33/f102 x:0 0 0 2024-03-04T13:43:20.288 INFO:tasks.workunit.client.0.smithi107.stdout:0/817: rename da/dc/dd/d3e/d26/d88 to da/dc/dd/d3e/d26/d88/dd2/d73/d114 22 2024-03-04T13:43:20.292 INFO:tasks.workunit.client.0.smithi107.stdout:4/782: dread d0/d9/d10/d80/db8/fb4 [0,4194304] 0 2024-03-04T13:43:20.294 INFO:tasks.workunit.client.0.smithi107.stdout:1/827: creat d1/d10/d16/d1b/d40/f109 x:0 0 0 2024-03-04T13:43:20.295 INFO:tasks.workunit.client.0.smithi107.stdout:3/797: fdatasync d1/de/d4b/f7a 0 2024-03-04T13:43:20.297 INFO:tasks.workunit.client.0.smithi107.stdout:6/811: dwrite df/d2e/d36/d5b/d7d/dfc/fba [0,4194304] 0 2024-03-04T13:43:20.298 INFO:tasks.workunit.client.0.smithi107.stdout:9/738: dread - d4/da/d11/d20/d23/dd9/fa2 zero size 2024-03-04T13:43:20.298 INFO:tasks.workunit.client.0.smithi107.stdout:0/818: mkdir da/dc/dd/d3e/d26/d69/d6b/d115 0 2024-03-04T13:43:20.301 INFO:tasks.workunit.client.0.smithi107.stdout:7/767: dwrite d6/d1d/d36/fae [0,4194304] 0 2024-03-04T13:43:20.301 INFO:tasks.workunit.client.0.smithi107.stdout:4/783: truncate d0/fc9 847480 0 2024-03-04T13:43:20.302 INFO:tasks.workunit.client.0.smithi107.stdout:3/798: mknod d1/de/d18/d1f/c103 0 2024-03-04T13:43:20.307 INFO:tasks.workunit.client.0.smithi107.stdout:5/871: dread d5/d79/d89/da3/d69/f72 [0,4194304] 0 2024-03-04T13:43:20.307 INFO:tasks.workunit.client.0.smithi107.stdout:9/739: mkdir d4/da/d11/d20/d2f/d9c/d92/df6 0 2024-03-04T13:43:20.309 INFO:tasks.workunit.client.0.smithi107.stdout:7/768: dread - d6/d13/d16/d19/fa1 zero size 2024-03-04T13:43:20.311 INFO:tasks.workunit.client.0.smithi107.stdout:5/872: read d5/d2b/d4b/da6/f119 [2334496,78864] 0 2024-03-04T13:43:20.312 INFO:tasks.workunit.client.0.smithi107.stdout:6/812: rename df/d17/d24/d40/d63/da3 to df/d2e/d36/d77/d106 0 2024-03-04T13:43:20.313 INFO:tasks.workunit.client.0.smithi107.stdout:9/740: read d4/da/d11/d65/f6e [553393,108275] 0 2024-03-04T13:43:20.317 INFO:tasks.workunit.client.0.smithi107.stdout:7/769: mkdir d6/d13/d16/d19/d38/d2c/dff 0 2024-03-04T13:43:20.319 INFO:tasks.workunit.client.0.smithi107.stdout:9/741: write d4/da/d11/d20/d23/dcc/fee [599047,41697] 0 2024-03-04T13:43:20.320 INFO:tasks.workunit.client.0.smithi107.stdout:4/784: read d0/d9/de/d14/f4e [2964186,30199] 0 2024-03-04T13:43:20.321 INFO:tasks.workunit.client.0.smithi107.stdout:3/799: rename d1/l2c to d1/d41/l104 0 2024-03-04T13:43:20.324 INFO:tasks.workunit.client.0.smithi107.stdout:4/785: chown d0/d9/d10/d80/le9 5328483 1 2024-03-04T13:43:20.325 INFO:tasks.workunit.client.0.smithi107.stdout:3/800: write d1/de/d3f/fb3 [932140,37666] 0 2024-03-04T13:43:20.326 INFO:tasks.workunit.client.0.smithi107.stdout:5/873: mknod d5/d2b/c135 0 2024-03-04T13:43:20.328 INFO:tasks.workunit.client.0.smithi107.stdout:0/819: creat da/dc/dd/d3e/d26/d88/dd2/d73/d78/f116 x:0 0 0 2024-03-04T13:43:20.332 INFO:tasks.workunit.client.0.smithi107.stdout:1/828: dwrite d1/d10/d16/d1b/d40/f57 [0,4194304] 0 2024-03-04T13:43:20.332 INFO:tasks.workunit.client.0.smithi107.stdout:4/786: write d0/d9/d10/d80/db8/f3c [1213660,123874] 0 2024-03-04T13:43:20.334 INFO:tasks.workunit.client.0.smithi107.stdout:4/787: chown d0/d9/fd1 0 1 2024-03-04T13:43:20.335 INFO:tasks.workunit.client.0.smithi107.stdout:6/813: write df/d2e/d36/d77/d106/f70 [3750825,74295] 0 2024-03-04T13:43:20.336 INFO:tasks.workunit.client.0.smithi107.stdout:5/874: mknod d5/d2b/d4b/c136 0 2024-03-04T13:43:20.340 INFO:tasks.workunit.client.0.smithi107.stdout:5/875: rmdir d5/d79/d89/da3 39 2024-03-04T13:43:20.343 INFO:tasks.workunit.client.0.smithi107.stdout:3/801: getdents d1/de/d11/dd5/dab 0 2024-03-04T13:43:20.350 INFO:tasks.workunit.client.0.smithi107.stdout:6/814: creat df/d17/d6b/da8/de6/f107 x:0 0 0 2024-03-04T13:43:20.354 INFO:tasks.workunit.client.0.smithi107.stdout:5/876: dread d5/d8/d14/d2d/df1/f93 [0,4194304] 0 2024-03-04T13:43:20.356 INFO:tasks.workunit.client.0.smithi107.stdout:4/788: truncate d0/d9/d10/d80/db8/f27 3598439 0 2024-03-04T13:43:20.360 INFO:tasks.workunit.client.0.smithi107.stdout:3/802: write d1/d2d/fca [604810,111755] 0 2024-03-04T13:43:20.363 INFO:tasks.workunit.client.0.smithi107.stdout:7/770: dread d6/d13/d16/fbb [0,4194304] 0 2024-03-04T13:43:20.364 INFO:tasks.workunit.client.0.smithi107.stdout:5/877: rename d5/dc/df/c7e to d5/dc/de4/c137 0 2024-03-04T13:43:20.365 INFO:tasks.workunit.client.0.smithi107.stdout:9/742: dread d4/d34/f44 [0,4194304] 0 2024-03-04T13:43:20.366 INFO:tasks.workunit.client.0.smithi107.stdout:3/803: chown d1/de/d18/d33/d48/fce 711431 1 2024-03-04T13:43:20.367 INFO:tasks.workunit.client.0.smithi107.stdout:4/789: creat d0/d9/de/d14/ffb x:0 0 0 2024-03-04T13:43:20.367 INFO:tasks.workunit.client.0.smithi107.stdout:0/820: dwrite da/dc/dd/d3e/d26/d69/d6b/f97 [0,4194304] 0 2024-03-04T13:43:20.369 INFO:tasks.workunit.client.0.smithi107.stdout:7/771: fdatasync d6/d1d/d64/d9a/da9/fda 0 2024-03-04T13:43:20.372 INFO:tasks.workunit.client.0.smithi107.stdout:5/878: symlink d5/d8/da/l138 0 2024-03-04T13:43:20.376 INFO:tasks.workunit.client.0.smithi107.stdout:9/743: mkdir d4/da/d11/d20/d2f/df7 0 2024-03-04T13:43:20.377 INFO:tasks.workunit.client.0.smithi107.stdout:1/829: dread d1/f94 [0,4194304] 0 2024-03-04T13:43:20.378 INFO:tasks.workunit.client.0.smithi107.stdout:5/879: write d5/d8/d14/d2d/f6b [820900,44828] 0 2024-03-04T13:43:20.380 INFO:tasks.workunit.client.0.smithi107.stdout:4/790: mkdir d0/d9/d18/d29/dfc 0 2024-03-04T13:43:20.380 INFO:tasks.workunit.client.0.smithi107.stdout:7/772: chown d6/d13/d16/d19/d38/d37/f9e 89 1 2024-03-04T13:43:20.381 INFO:tasks.workunit.client.0.smithi107.stdout:1/830: readlink d1/d46/l92 0 2024-03-04T13:43:20.384 INFO:tasks.workunit.client.0.smithi107.stdout:9/744: readlink d4/da/d11/d4c/d51/l70 0 2024-03-04T13:43:20.384 INFO:tasks.workunit.client.0.smithi107.stdout:2/735: sync 2024-03-04T13:43:20.386 INFO:tasks.workunit.client.0.smithi107.stdout:0/821: rename da/dc/dd/d3e/ce4 to da/dc/dd/d3e/d26/d69/de9/c117 0 2024-03-04T13:43:20.387 INFO:tasks.workunit.client.0.smithi107.stdout:4/791: write d0/d9/d18/faa [832740,122581] 0 2024-03-04T13:43:20.389 INFO:tasks.workunit.client.0.smithi107.stdout:7/773: mknod d6/d13/d16/d19/d38/d2c/c100 0 2024-03-04T13:43:20.390 INFO:tasks.workunit.client.0.smithi107.stdout:1/831: truncate d1/f9a 889544 0 2024-03-04T13:43:20.393 INFO:tasks.workunit.client.0.smithi107.stdout:4/792: fsync d0/d9/d10/f8c 0 2024-03-04T13:43:20.394 INFO:tasks.workunit.client.0.smithi107.stdout:2/736: truncate da/d34/fc8 847265 0 2024-03-04T13:43:20.397 INFO:tasks.workunit.client.0.smithi107.stdout:4/793: readlink d0/d9/d18/df1/lcb 0 2024-03-04T13:43:20.398 INFO:tasks.workunit.client.0.smithi107.stdout:9/745: creat d4/da/ff8 x:0 0 0 2024-03-04T13:43:20.399 INFO:tasks.workunit.client.0.smithi107.stdout:2/737: truncate da/d29/d7c/fb8 37063 0 2024-03-04T13:43:20.400 INFO:tasks.workunit.client.0.smithi107.stdout:4/794: readlink d0/d9/d10/lba 0 2024-03-04T13:43:20.401 INFO:tasks.workunit.client.0.smithi107.stdout:1/832: symlink d1/d46/d6a/d65/l10a 0 2024-03-04T13:43:20.402 INFO:tasks.workunit.client.0.smithi107.stdout:3/804: dwrite d1/d1e/d26/f3d [0,4194304] 0 2024-03-04T13:43:20.402 INFO:tasks.workunit.client.0.smithi107.stdout:6/815: dread df/d2e/d36/d77/d106/f70 [0,4194304] 0 2024-03-04T13:43:20.407 INFO:tasks.workunit.client.0.smithi107.stdout:0/822: rename da/dc/dd/d3e/d26/d69/c101 to da/dc/dd/d3e/d26/d69/d6b/df0/c118 0 2024-03-04T13:43:20.407 INFO:tasks.workunit.client.0.smithi107.stdout:4/795: dread d0/fc9 [0,4194304] 0 2024-03-04T13:43:20.407 INFO:tasks.workunit.client.0.smithi107.stdout:5/880: write d5/d8/d2a/d43/f7d [391322,115524] 0 2024-03-04T13:43:20.409 INFO:tasks.workunit.client.0.smithi107.stdout:0/823: dread - da/f102 zero size 2024-03-04T13:43:20.412 INFO:tasks.workunit.client.0.smithi107.stdout:5/881: dread d5/d8/da/db0/de6/f11c [0,4194304] 0 2024-03-04T13:43:20.415 INFO:tasks.workunit.client.0.smithi107.stdout:2/738: unlink da/d44/d8b/d9d/df8/f9b 0 2024-03-04T13:43:20.416 INFO:tasks.workunit.client.0.smithi107.stdout:9/746: chown d4/da/d11/d20/d2f/d9c/daa/lbc 13294811 1 2024-03-04T13:43:20.417 INFO:tasks.workunit.client.0.smithi107.stdout:3/805: symlink d1/d1e/dae/l105 0 2024-03-04T13:43:20.419 INFO:tasks.workunit.client.0.smithi107.stdout:7/774: write d6/d1d/fb2 [513444,71906] 0 2024-03-04T13:43:20.419 INFO:tasks.workunit.client.0.smithi107.stdout:6/816: fdatasync df/d17/d24/d30/d5d/d6a/fe3 0 2024-03-04T13:43:20.421 INFO:tasks.workunit.client.0.smithi107.stdout:8/814: sync 2024-03-04T13:43:20.422 INFO:tasks.workunit.client.0.smithi107.stdout:0/824: creat da/dc/dd/d1e/dbf/deb/f119 x:0 0 0 2024-03-04T13:43:20.423 INFO:tasks.workunit.client.0.smithi107.stdout:5/882: mknod d5/d8/d2a/c139 0 2024-03-04T13:43:20.426 INFO:tasks.workunit.client.0.smithi107.stdout:4/796: dwrite d0/d9/d10/d80/db8/d21/f5d [0,4194304] 0 2024-03-04T13:43:20.428 INFO:tasks.workunit.client.0.smithi107.stdout:2/739: write da/df/d27/d58/ff7 [633284,25714] 0 2024-03-04T13:43:20.429 INFO:tasks.workunit.client.0.smithi107.stdout:9/747: mkdir d4/da/d11/d20/d64/d77/df9 0 2024-03-04T13:43:20.430 INFO:tasks.workunit.client.0.smithi107.stdout:3/806: unlink d1/de/d18/d1f/d42/f52 0 2024-03-04T13:43:20.430 INFO:tasks.workunit.client.0.smithi107.stdout:6/817: rmdir df 39 2024-03-04T13:43:20.432 INFO:tasks.workunit.client.0.smithi107.stdout:0/825: creat da/dad/f11a x:0 0 0 2024-03-04T13:43:20.434 INFO:tasks.workunit.client.0.smithi107.stdout:5/883: rmdir d5/d8/d2a/dd6/dea 39 2024-03-04T13:43:20.436 INFO:tasks.workunit.client.0.smithi107.stdout:1/833: creat d1/d46/d6a/d4e/daa/dc9/d67/f10b x:0 0 0 2024-03-04T13:43:20.442 INFO:tasks.workunit.client.0.smithi107.stdout:9/748: fsync d4/d34/fe0 0 2024-03-04T13:43:20.445 INFO:tasks.workunit.client.0.smithi107.stdout:6/818: write df/d2e/d36/fa2 [71973,19603] 0 2024-03-04T13:43:20.476 INFO:tasks.workunit.client.0.smithi107.stdout:2/740: truncate da/d44/f64 280383 0 2024-03-04T13:43:20.480 INFO:tasks.workunit.client.0.smithi107.stdout:1/834: symlink d1/d10/d16/d47/d7a/dda/dec/l10c 0 2024-03-04T13:43:20.482 INFO:tasks.workunit.client.0.smithi107.stdout:7/775: dread d6/d1d/d64/d9a/da9/d71/db9/fc5 [0,4194304] 0 2024-03-04T13:43:20.483 INFO:tasks.workunit.client.0.smithi107.stdout:1/835: readlink d1/d46/l92 0 2024-03-04T13:43:20.484 INFO:tasks.workunit.client.0.smithi107.stdout:2/741: symlink da/d44/d8b/d9d/df8/d5c/lfc 0 2024-03-04T13:43:20.488 INFO:tasks.workunit.client.0.smithi107.stdout:6/819: link f8 df/d17/d5c/f108 0 2024-03-04T13:43:20.490 INFO:tasks.workunit.client.0.smithi107.stdout:8/815: dwrite d6/d16/d1f/d4c/f61 [0,4194304] 0 2024-03-04T13:43:20.492 INFO:tasks.workunit.client.0.smithi107.stdout:2/742: creat da/d39/ffd x:0 0 0 2024-03-04T13:43:20.495 INFO:tasks.workunit.client.0.smithi107.stdout:7/776: creat d6/d1d/d64/d9a/da9/d71/db9/dca/f101 x:0 0 0 2024-03-04T13:43:20.495 INFO:tasks.workunit.client.0.smithi107.stdout:2/743: chown da/df/d3e/fcf 13021039 1 2024-03-04T13:43:20.495 INFO:tasks.workunit.client.0.smithi107.stdout:4/797: dwrite d0/d9/d18/df1/dc1/ff2 [0,4194304] 0 2024-03-04T13:43:20.495 INFO:tasks.workunit.client.0.smithi107.stdout:6/820: creat df/d38/d4f/d74/f109 x:0 0 0 2024-03-04T13:43:20.497 INFO:tasks.workunit.client.0.smithi107.stdout:2/744: dread - da/df/d73/daa/fb2 zero size 2024-03-04T13:43:20.498 INFO:tasks.workunit.client.0.smithi107.stdout:1/836: creat d1/d46/d6a/d75/f10d x:0 0 0 2024-03-04T13:43:20.499 INFO:tasks.workunit.client.0.smithi107.stdout:7/777: dread - d6/d13/d16/d19/fe8 zero size 2024-03-04T13:43:20.504 INFO:tasks.workunit.client.0.smithi107.stdout:4/798: truncate d0/d9/d78/f85 1100696 0 2024-03-04T13:43:20.506 INFO:tasks.workunit.client.0.smithi107.stdout:2/745: mkdir da/df/d3e/dfe 0 2024-03-04T13:43:20.506 INFO:tasks.workunit.client.0.smithi107.stdout:4/799: chown d0/d9/d10/f8c 5276 1 2024-03-04T13:43:20.508 INFO:tasks.workunit.client.0.smithi107.stdout:5/884: dread d5/d8/d14/d2d/f56 [0,4194304] 0 2024-03-04T13:43:20.509 INFO:tasks.workunit.client.0.smithi107.stdout:8/816: fsync d6/da/d25/d5f/f98 0 2024-03-04T13:43:20.511 INFO:tasks.workunit.client.0.smithi107.stdout:0/826: dwrite da/dc/dd/d1e/fa1 [0,4194304] 0 2024-03-04T13:43:20.512 INFO:tasks.workunit.client.0.smithi107.stdout:2/746: mknod da/d44/d8b/d9d/dd0/cff 0 2024-03-04T13:43:20.516 INFO:tasks.workunit.client.0.smithi107.stdout:5/885: symlink d5/d8/d14/d2d/df1/l13a 0 2024-03-04T13:43:20.519 INFO:tasks.workunit.client.0.smithi107.stdout:4/800: dread d0/d9/d10/d80/db8/f67 [0,4194304] 0 2024-03-04T13:43:20.521 INFO:tasks.workunit.client.0.smithi107.stdout:1/837: creat d1/d46/d6a/d4e/daa/f10e x:0 0 0 2024-03-04T13:43:20.522 INFO:tasks.workunit.client.0.smithi107.stdout:8/817: mkdir d6/da/d25/dd1/d101 0 2024-03-04T13:43:20.523 INFO:tasks.workunit.client.0.smithi107.stdout:6/821: dwrite df/d2e/d36/d77/d106/d7e/d86/f100 [0,4194304] 0 2024-03-04T13:43:20.531 INFO:tasks.workunit.client.0.smithi107.stdout:2/747: mkdir da/df/d73/dd1/d100 0 2024-03-04T13:43:20.534 INFO:tasks.workunit.client.0.smithi107.stdout:5/886: dwrite d5/d2b/d4b/f91 [0,4194304] 0 2024-03-04T13:43:20.537 INFO:tasks.workunit.client.0.smithi107.stdout:1/838: dread d1/d10/d16/d47/f77 [0,4194304] 0 2024-03-04T13:43:20.540 INFO:tasks.workunit.client.0.smithi107.stdout:4/801: symlink d0/d9/d10/d80/db8/d21/lfd 0 2024-03-04T13:43:20.542 INFO:tasks.workunit.client.0.smithi107.stdout:6/822: creat df/d17/d5c/f10a x:0 0 0 2024-03-04T13:43:20.546 INFO:tasks.workunit.client.0.smithi107.stdout:8/818: rename d6/d16/d69/deb/d7c/f8a to d6/da/d25/dd1/de3/f102 0 2024-03-04T13:43:20.548 INFO:tasks.workunit.client.0.smithi107.stdout:2/748: readlink da/df/d3e/lcd 0 2024-03-04T13:43:20.551 INFO:tasks.workunit.client.0.smithi107.stdout:5/887: creat d5/d8/d2a/d43/f13b x:0 0 0 2024-03-04T13:43:20.562 INFO:tasks.workunit.client.0.smithi107.stdout:7/778: creat d6/d1d/d64/d9a/da9/d71/db9/dca/d3d/dc6/dcb/dd6/f102 x:0 0 0 2024-03-04T13:43:20.563 INFO:tasks.workunit.client.0.smithi107.stdout:8/819: fdatasync d6/d16/d1f/d4c/da4/fda 0 2024-03-04T13:43:20.563 INFO:tasks.workunit.client.0.smithi107.stdout:0/827: dwrite da/dc/dd/d3e/d26/d88/dd2/fa8 [0,4194304] 0 2024-03-04T13:43:20.570 INFO:tasks.workunit.client.0.smithi107.stdout:2/749: dwrite da/df/d3e/fcf [0,4194304] 0 2024-03-04T13:43:20.574 INFO:tasks.workunit.client.0.smithi107.stdout:2/750: stat da/df/d73/db3/db1/df2 0 2024-03-04T13:43:20.575 INFO:tasks.workunit.client.0.smithi107.stdout:0/828: readlink da/dc/dd/lb0 0 2024-03-04T13:43:20.575 INFO:tasks.workunit.client.0.smithi107.stdout:5/888: fdatasync d5/d79/d89/da3/d69/f72 0 2024-03-04T13:43:20.578 INFO:tasks.workunit.client.0.smithi107.stdout:4/802: rename d0/d9/d10/c4c to d0/d9/d1e/cfe 0 2024-03-04T13:43:20.580 INFO:tasks.workunit.client.0.smithi107.stdout:2/751: fsync da/df/d3e/f40 0 2024-03-04T13:43:20.583 INFO:tasks.workunit.client.0.smithi107.stdout:8/820: creat d6/d16/d69/deb/d3e/d67/db7/f103 x:0 0 0 2024-03-04T13:43:20.583 INFO:tasks.workunit.client.0.smithi107.stdout:0/829: creat da/dc/dd/d1e/dbf/deb/f11b x:0 0 0 2024-03-04T13:43:20.586 INFO:tasks.workunit.client.0.smithi107.stdout:2/752: rename da/df/d73/db3/db1 to da/df/d73/db3/db1/df2/d101 22 2024-03-04T13:43:20.591 INFO:tasks.workunit.client.0.smithi107.stdout:1/839: dread d1/d46/d6a/fb5 [0,4194304] 0 2024-03-04T13:43:20.592 INFO:tasks.workunit.client.0.smithi107.stdout:2/753: creat da/df/d73/dd1/f102 x:0 0 0 2024-03-04T13:43:20.595 INFO:tasks.workunit.client.0.smithi107.stdout:7/779: write d6/d1d/d64/d9a/da9/d71/db9/dca/d3d/dc6/ff1 [708071,22792] 0 2024-03-04T13:43:20.597 INFO:tasks.workunit.client.0.smithi107.stdout:5/889: getdents d5 0 2024-03-04T13:43:20.600 INFO:tasks.workunit.client.0.smithi107.stdout:8/821: link d6/d16/d1f/d63/d73/f7b d6/d16/f104 0 2024-03-04T13:43:20.601 INFO:tasks.workunit.client.0.smithi107.stdout:7/780: stat d6/d13/d16/d19/d60/fe6 0 2024-03-04T13:43:20.601 INFO:tasks.workunit.client.0.smithi107.stdout:5/890: write d5/d79/d89/da3/d69/fd3 [654661,45285] 0 2024-03-04T13:43:20.603 INFO:tasks.workunit.client.0.smithi107.stdout:7/781: fsync d6/fa 0 2024-03-04T13:43:20.604 INFO:tasks.workunit.client.0.smithi107.stdout:0/830: dwrite da/dc/dd/d3e/d26/d88/dd2/f9d [0,4194304] 0 2024-03-04T13:43:20.604 INFO:tasks.workunit.client.0.smithi107.stdout:1/840: rename d1/d10/ce0 to d1/d46/d6a/d4e/daa/dc9/d33/d5d/d4c/dea/d59/dba/c10f 0 2024-03-04T13:43:20.612 INFO:tasks.workunit.client.0.smithi107.stdout:8/822: dread d6/d16/d69/deb/d3e/fb3 [0,4194304] 0 2024-03-04T13:43:20.616 INFO:tasks.workunit.client.0.smithi107.stdout:2/754: dwrite da/d34/f72 [0,4194304] 0 2024-03-04T13:43:20.619 INFO:tasks.workunit.client.0.smithi107.stdout:7/782: creat d6/d1d/d64/d9a/da9/d71/f103 x:0 0 0 2024-03-04T13:43:20.620 INFO:tasks.workunit.client.0.smithi107.stdout:1/841: mkdir d1/d46/d6a/d65/d106/d61/d110 0 2024-03-04T13:43:20.621 INFO:tasks.workunit.client.0.smithi107.stdout:5/891: truncate d5/d8/d14/d2d/d92/dd8/f114 1379075 0 2024-03-04T13:43:20.628 INFO:tasks.workunit.client.0.smithi107.stdout:4/803: dwrite d0/d9/f5b [0,4194304] 0 2024-03-04T13:43:20.629 INFO:tasks.workunit.client.0.smithi107.stdout:0/831: rename da/dc/dd/dc9/de5/lb7 to da/dc/dd/d3e/d26/d69/l11c 0 2024-03-04T13:43:20.630 INFO:tasks.workunit.client.0.smithi107.stdout:2/755: rmdir da/d44/d9c 39 2024-03-04T13:43:20.630 INFO:tasks.workunit.client.0.smithi107.stdout:7/783: truncate d6/d13/d16/d19/d39/f40 13049055 0 2024-03-04T13:43:20.636 INFO:tasks.workunit.client.0.smithi107.stdout:5/892: mkdir d5/d66/d96/db1/d13c 0 2024-03-04T13:43:20.636 INFO:tasks.workunit.client.0.smithi107.stdout:7/784: truncate d6/d13/d14/f86 8717112 0 2024-03-04T13:43:20.638 INFO:tasks.workunit.client.0.smithi107.stdout:3/807: sync 2024-03-04T13:43:20.640 INFO:tasks.workunit.client.0.smithi107.stdout:2/756: rmdir da/d44/d8b/d98/dc3 39 2024-03-04T13:43:20.645 INFO:tasks.workunit.client.0.smithi107.stdout:5/893: creat d5/d2b/d4b/f13d x:0 0 0 2024-03-04T13:43:20.645 INFO:tasks.workunit.client.0.smithi107.stdout:3/808: read - d1/d29/dda/fee zero size 2024-03-04T13:43:20.646 INFO:tasks.workunit.client.0.smithi107.stdout:2/757: fsync da/d38/d3a/d5d/ff9 0 2024-03-04T13:43:20.651 INFO:tasks.workunit.client.0.smithi107.stdout:8/823: dwrite d6/f9f [0,4194304] 0 2024-03-04T13:43:20.653 INFO:tasks.workunit.client.0.smithi107.stdout:7/785: mknod d6/d1d/d64/c104 0 2024-03-04T13:43:20.654 INFO:tasks.workunit.client.0.smithi107.stdout:5/894: rmdir d5/d8/d14 39 2024-03-04T13:43:20.657 INFO:tasks.workunit.client.0.smithi107.stdout:4/804: link d0/d2f/ld8 d0/d9/d10/d80/lff 0 2024-03-04T13:43:20.658 INFO:tasks.workunit.client.0.smithi107.stdout:8/824: dread - d6/d16/fb1 zero size 2024-03-04T13:43:20.658 INFO:tasks.workunit.client.0.smithi107.stdout:7/786: readlink d6/d1d/d36/l83 0 2024-03-04T13:43:20.659 INFO:tasks.workunit.client.0.smithi107.stdout:0/832: dwrite da/dad/f109 [0,4194304] 0 2024-03-04T13:43:20.665 INFO:tasks.workunit.client.0.smithi107.stdout:3/809: fdatasync d1/de/d18/d33/d48/d71/d4c/f57 0 2024-03-04T13:43:20.666 INFO:tasks.workunit.client.0.smithi107.stdout:1/842: truncate d1/d46/d6a/fb5 3805319 0 2024-03-04T13:43:20.666 INFO:tasks.workunit.client.0.smithi107.stdout:2/758: truncate da/df/d85/f95 612047 0 2024-03-04T13:43:20.667 INFO:tasks.workunit.client.0.smithi107.stdout:5/895: mknod d5/d2b/c13e 0 2024-03-04T13:43:20.669 INFO:tasks.workunit.client.0.smithi107.stdout:7/787: write d6/d1d/d64/d9a/da9/d71/f103 [310066,2400] 0 2024-03-04T13:43:20.669 INFO:tasks.workunit.client.0.smithi107.stdout:4/805: symlink d0/d9/d10/d80/l100 0 2024-03-04T13:43:20.669 INFO:tasks.workunit.client.0.smithi107.stdout:8/825: mkdir d6/da/d25/d5f/d105 0 2024-03-04T13:43:20.673 INFO:tasks.workunit.client.0.smithi107.stdout:5/896: fsync d5/d8/d14/d2d/dfa/f103 0 2024-03-04T13:43:20.675 INFO:tasks.workunit.client.0.smithi107.stdout:0/833: read da/dc/dd/d3e/d26/d88/dd2/f87 [104722,73302] 0 2024-03-04T13:43:20.676 INFO:tasks.workunit.client.0.smithi107.stdout:2/759: mkdir da/d44/d103 0 2024-03-04T13:43:20.677 INFO:tasks.workunit.client.0.smithi107.stdout:8/826: mknod d6/c106 0 2024-03-04T13:43:20.680 INFO:tasks.workunit.client.0.smithi107.stdout:3/810: chown d1/de/d18/d33/d48/d71/d4c/d54/db7/cd9 1 1 2024-03-04T13:43:20.684 INFO:tasks.workunit.client.0.smithi107.stdout:5/897: symlink d5/dc/dc6/l13f 0 2024-03-04T13:43:20.689 INFO:tasks.workunit.client.0.smithi107.stdout:8/827: dread d6/d16/d69/deb/d3e/f6d [0,4194304] 0 2024-03-04T13:43:20.689 INFO:tasks.workunit.client.0.smithi107.stdout:3/811: fdatasync d1/de/fa8 0 2024-03-04T13:43:20.695 INFO:tasks.workunit.client.0.smithi107.stdout:3/812: write d1/d2d/fca [297249,21589] 0 2024-03-04T13:43:20.698 INFO:tasks.workunit.client.0.smithi107.stdout:0/834: write f8 [77137,50443] 0 2024-03-04T13:43:20.699 INFO:tasks.workunit.client.0.smithi107.stdout:8/828: rename d6/da/d25/d2a/c66 to d6/da/d25/dd1/d101/c107 0 2024-03-04T13:43:20.700 INFO:tasks.workunit.client.0.smithi107.stdout:4/806: dwrite d0/d9/de/d14/f4b [4194304,4194304] 0 2024-03-04T13:43:20.704 INFO:tasks.workunit.client.0.smithi107.stdout:0/835: read da/f62 [3354508,73183] 0 2024-03-04T13:43:20.706 INFO:tasks.workunit.client.0.smithi107.stdout:2/760: dwrite da/d38/d3a/d5d/ff9 [0,4194304] 0 2024-03-04T13:43:20.706 INFO:tasks.workunit.client.0.smithi107.stdout:4/807: symlink d0/d2f/l101 0 2024-03-04T13:43:20.712 INFO:tasks.workunit.client.0.smithi107.stdout:8/829: chown d6/d16/d1f/d4c/da4/daa/lf5 173 1 2024-03-04T13:43:20.713 INFO:tasks.workunit.client.0.smithi107.stdout:4/808: readlink d0/d9/d10/d80/l100 0 2024-03-04T13:43:20.717 INFO:tasks.workunit.client.0.smithi107.stdout:4/809: stat d0/fef 0 2024-03-04T13:43:20.717 INFO:tasks.workunit.client.0.smithi107.stdout:3/813: truncate d1/de/d18/d33/d4d/d4f/d69/f7b 123069 0 2024-03-04T13:43:20.718 INFO:tasks.workunit.client.0.smithi107.stdout:1/843: dread d1/d10/d16/f37 [0,4194304] 0 2024-03-04T13:43:20.719 INFO:tasks.workunit.client.0.smithi107.stdout:8/830: truncate d6/d16/d1f/d4c/da4/fda 1560489 0 2024-03-04T13:43:20.722 INFO:tasks.workunit.client.0.smithi107.stdout:0/836: truncate da/dc/dd/fe7 703799 0 2024-03-04T13:43:20.722 INFO:tasks.workunit.client.0.smithi107.stdout:5/898: dwrite d5/d79/d89/da3/d68/fe8 [0,4194304] 0 2024-03-04T13:43:20.723 INFO:tasks.workunit.client.0.smithi107.stdout:2/761: getdents da/d44/d8b/d9d/dd0 0 2024-03-04T13:43:20.724 INFO:tasks.workunit.client.0.smithi107.stdout:8/831: read d6/da/d25/dd1/fd2 [438072,35457] 0 2024-03-04T13:43:20.727 INFO:tasks.workunit.client.0.smithi107.stdout:2/762: write da/d29/d7c/dbd/fe2 [558824,5648] 0 2024-03-04T13:43:20.731 INFO:tasks.workunit.client.0.smithi107.stdout:0/837: truncate da/dc/dd/d3e/d26/d88/dd2/f87 506351 0 2024-03-04T13:43:20.733 INFO:tasks.workunit.client.0.smithi107.stdout:5/899: mknod d5/d8/da/db0/c140 0 2024-03-04T13:43:20.734 INFO:tasks.workunit.client.0.smithi107.stdout:3/814: dread d1/d2d/fca [0,4194304] 0 2024-03-04T13:43:20.734 INFO:tasks.workunit.client.0.smithi107.stdout:2/763: mknod da/d44/d8b/d9d/c104 0 2024-03-04T13:43:20.735 INFO:tasks.workunit.client.0.smithi107.stdout:0/838: write da/dc/dd/dc9/de5/f100 [1212164,68442] 0 2024-03-04T13:43:20.736 INFO:tasks.workunit.client.0.smithi107.stdout:8/832: mkdir d6/d16/d1f/d4c/da4/daa/d108 0 2024-03-04T13:43:20.737 INFO:tasks.workunit.client.0.smithi107.stdout:5/900: write d5/d8/d14/d2d/f6b [2984461,63606] 0 2024-03-04T13:43:20.738 INFO:tasks.workunit.client.0.smithi107.stdout:4/810: rename d0/d9/de/d14/l71 to d0/d9/d1e/l102 0 2024-03-04T13:43:20.739 INFO:tasks.workunit.client.0.smithi107.stdout:2/764: write da/d34/f36 [615352,48976] 0 2024-03-04T13:43:20.741 INFO:tasks.workunit.client.0.smithi107.stdout:3/815: readlink d1/de/d18/d33/l75 0 2024-03-04T13:43:20.742 INFO:tasks.workunit.client.0.smithi107.stdout:4/811: readlink d0/d9/d18/df1/lac 0 2024-03-04T13:43:20.743 INFO:tasks.workunit.client.0.smithi107.stdout:0/839: write da/dc/dd/d1e/dbf/deb/f119 [48422,83586] 0 2024-03-04T13:43:20.745 INFO:tasks.workunit.client.0.smithi107.stdout:1/844: rename d1/d46/d6a/d65/d106/d61/c96 to d1/d10/d16/d47/d7a/dda/dec/c111 0 2024-03-04T13:43:20.752 INFO:tasks.workunit.client.0.smithi107.stdout:4/812: chown d0/d9/de/f41 1154 1 2024-03-04T13:43:20.754 INFO:tasks.workunit.client.0.smithi107.stdout:9/749: sync 2024-03-04T13:43:20.754 INFO:tasks.workunit.client.0.smithi107.stdout:6/823: sync 2024-03-04T13:43:20.755 INFO:tasks.workunit.client.0.smithi107.stdout:2/765: creat da/d44/d9c/f105 x:0 0 0 2024-03-04T13:43:20.755 INFO:tasks.workunit.client.0.smithi107.stdout:3/816: symlink d1/d1e/d26/dfd/l106 0 2024-03-04T13:43:20.757 INFO:tasks.workunit.client.0.smithi107.stdout:0/840: rename da/dc/dd/d3e/d26/d69/d6b/df0/d10e/f113 to da/dc/dd/d3e/d26/d88/f11d 0 2024-03-04T13:43:20.762 INFO:tasks.workunit.client.0.smithi107.stdout:1/845: truncate d1/d46/d6a/d65/f7f 4170146 0 2024-03-04T13:43:20.763 INFO:tasks.workunit.client.0.smithi107.stdout:8/833: dread d6/d16/f24 [0,4194304] 0 2024-03-04T13:43:20.769 INFO:tasks.workunit.client.0.smithi107.stdout:3/817: mknod d1/de/dad/dbb/c107 0 2024-03-04T13:43:20.770 INFO:tasks.workunit.client.0.smithi107.stdout:3/818: chown d1/de 25049 1 2024-03-04T13:43:20.773 INFO:tasks.workunit.client.0.smithi107.stdout:5/901: link d5/d8/d14/d2d/d92/cf5 d5/d8/d2a/d43/dac/c141 0 2024-03-04T13:43:20.775 INFO:tasks.workunit.client.0.smithi107.stdout:4/813: link d0/d9/d10/d80/db8/d21/f6c d0/d9/d10/dab/f103 0 2024-03-04T13:43:20.776 INFO:tasks.workunit.client.0.smithi107.stdout:1/846: mkdir d1/d10/d16/d47/d7a/dda/d112 0 2024-03-04T13:43:20.777 INFO:tasks.workunit.client.0.smithi107.stdout:6/824: mkdir df/d1b/d10b 0 2024-03-04T13:43:20.780 INFO:tasks.workunit.client.0.smithi107.stdout:0/841: mknod da/c11e 0 2024-03-04T13:43:20.780 INFO:tasks.workunit.client.0.smithi107.stdout:5/902: fdatasync d5/d8/f63 0 2024-03-04T13:43:20.781 INFO:tasks.workunit.client.0.smithi107.stdout:6/825: chown df/d17/d24/d40/d63/lad 2183873 1 2024-03-04T13:43:20.781 INFO:tasks.workunit.client.0.smithi107.stdout:2/766: dwrite da/df/d85/f8d [0,4194304] 0 2024-03-04T13:43:20.784 INFO:tasks.workunit.client.0.smithi107.stdout:2/767: write f7 [2753630,111533] 0 2024-03-04T13:43:20.784 INFO:tasks.workunit.client.0.smithi107.stdout:8/834: mknod d6/d16/d69/deb/d7c/ddf/c109 0 2024-03-04T13:43:20.787 INFO:tasks.workunit.client.0.smithi107.stdout:2/768: write da/d29/d7c/fb8 [15882,109699] 0 2024-03-04T13:43:20.790 INFO:tasks.workunit.client.0.smithi107.stdout:1/847: fdatasync d1/d46/d6a/d65/db1/ff4 0 2024-03-04T13:43:20.798 INFO:tasks.workunit.client.0.smithi107.stdout:8/835: unlink d6/da/da8/lab 0 2024-03-04T13:43:20.798 INFO:tasks.workunit.client.0.smithi107.stdout:6/826: unlink df/d17/d24/d40/lb6 0 2024-03-04T13:43:20.799 INFO:tasks.workunit.client.0.smithi107.stdout:3/819: dwrite d1/f2b [0,4194304] 0 2024-03-04T13:43:20.803 INFO:tasks.workunit.client.0.smithi107.stdout:5/903: write d5/f9 [1940599,30786] 0 2024-03-04T13:43:20.805 INFO:tasks.workunit.client.0.smithi107.stdout:6/827: symlink df/d17/d24/d30/d3b/l10c 0 2024-03-04T13:43:20.805 INFO:tasks.workunit.client.0.smithi107.stdout:8/836: mkdir d6/da/d25/d2a/d10a 0 2024-03-04T13:43:20.808 INFO:tasks.workunit.client.0.smithi107.stdout:1/848: unlink d1/d46/d6a/d65/f7f 0 2024-03-04T13:43:20.810 INFO:tasks.workunit.client.0.smithi107.stdout:3/820: creat d1/de/d18/d33/d48/d93/f108 x:0 0 0 2024-03-04T13:43:20.812 INFO:tasks.workunit.client.0.smithi107.stdout:5/904: symlink d5/d8/d2a/l142 0 2024-03-04T13:43:20.815 INFO:tasks.workunit.client.0.smithi107.stdout:8/837: stat d6/da/d25/d2a/l9d 0 2024-03-04T13:43:20.815 INFO:tasks.workunit.client.0.smithi107.stdout:6/828: creat df/d17/d24/d30/d5d/d6a/f10d x:0 0 0 2024-03-04T13:43:20.816 INFO:tasks.workunit.client.0.smithi107.stdout:1/849: symlink d1/d46/d6a/d4e/daa/dc9/d33/d5d/d4c/l113 0 2024-03-04T13:43:20.816 INFO:tasks.workunit.client.0.smithi107.stdout:0/842: dwrite da/dc/dd/d3e/fd4 [0,4194304] 0 2024-03-04T13:43:20.819 INFO:tasks.workunit.client.0.smithi107.stdout:6/829: readlink df/d38/l82 0 2024-03-04T13:43:20.826 INFO:tasks.workunit.client.0.smithi107.stdout:6/830: write df/d2e/d36/d5b/d7d/dfc/fee [494496,2752] 0 2024-03-04T13:43:20.828 INFO:tasks.workunit.client.0.smithi107.stdout:5/905: write d5/dc/dc6/f75 [207800,34386] 0 2024-03-04T13:43:20.830 INFO:tasks.workunit.client.0.smithi107.stdout:1/850: rename d1/d46/d6a/d4e/d60/f4f to d1/d46/d6a/d4e/daa/dc9/d33/d5d/d4c/dea/d59/f114 0 2024-03-04T13:43:20.835 INFO:tasks.workunit.client.0.smithi107.stdout:6/831: creat df/d2e/d36/d5b/d7d/f10e x:0 0 0 2024-03-04T13:43:20.836 INFO:tasks.workunit.client.0.smithi107.stdout:3/821: dwrite d1/de/d4b/fb8 [0,4194304] 0 2024-03-04T13:43:20.838 INFO:tasks.workunit.client.0.smithi107.stdout:3/822: fsync d1/de/d18/d33/d48/d93/fdd 0 2024-03-04T13:43:20.839 INFO:tasks.workunit.client.0.smithi107.stdout:2/769: dread da/df/f13 [0,4194304] 0 2024-03-04T13:43:20.840 INFO:tasks.workunit.client.0.smithi107.stdout:5/906: truncate d5/d79/d89/fb3 294621 0 2024-03-04T13:43:20.842 INFO:tasks.workunit.client.0.smithi107.stdout:5/907: chown d5/d8/d14/d121/c131 3 1 2024-03-04T13:43:20.842 INFO:tasks.workunit.client.0.smithi107.stdout:1/851: mkdir d1/d10/d16/da9/d115 0 2024-03-04T13:43:20.843 INFO:tasks.workunit.client.0.smithi107.stdout:0/843: truncate da/dc/dd/d3e/d26/d88/dd2/f9d 3891910 0 2024-03-04T13:43:20.844 INFO:tasks.workunit.client.0.smithi107.stdout:5/908: write d5/dc/f3d [110884,32655] 0 2024-03-04T13:43:20.846 INFO:tasks.workunit.client.0.smithi107.stdout:1/852: chown d1/d46/d6a/d4e/cca 1204775 1 2024-03-04T13:43:20.848 INFO:tasks.workunit.client.0.smithi107.stdout:5/909: chown d5/dc/de4/fc3 173 1 2024-03-04T13:43:20.850 INFO:tasks.workunit.client.0.smithi107.stdout:3/823: mknod d1/d1e/c109 0 2024-03-04T13:43:20.851 INFO:tasks.workunit.client.0.smithi107.stdout:0/844: chown da/dc/dd/d3e/d26/d69/df9/f103 401405 1 2024-03-04T13:43:20.854 INFO:tasks.workunit.client.0.smithi107.stdout:7/788: sync 2024-03-04T13:43:20.854 INFO:tasks.workunit.client.0.smithi107.stdout:5/910: rmdir d5/d8 39 2024-03-04T13:43:20.854 INFO:tasks.workunit.client.0.smithi107.stdout:0/845: readlink da/dc/dd/d3e/lb4 0 2024-03-04T13:43:20.855 INFO:tasks.workunit.client.0.smithi107.stdout:6/832: dwrite df/f29 [4194304,4194304] 0 2024-03-04T13:43:20.856 INFO:tasks.workunit.client.0.smithi107.stdout:1/853: creat d1/d10/d16/d47/d7a/dda/f116 x:0 0 0 2024-03-04T13:43:20.858 INFO:tasks.workunit.client.0.smithi107.stdout:0/846: dread - da/dc/dd/d1e/fe2 zero size 2024-03-04T13:43:20.864 INFO:tasks.workunit.client.0.smithi107.stdout:2/770: write da/d29/f69 [2877130,18337] 0 2024-03-04T13:43:20.864 INFO:tasks.workunit.client.0.smithi107.stdout:0/847: mknod da/dc/dd/d1e/dbf/c11f 0 2024-03-04T13:43:20.867 INFO:tasks.workunit.client.0.smithi107.stdout:5/911: stat d5/d8/d14/d2d/d11d/d127/c12c 0 2024-03-04T13:43:20.869 INFO:tasks.workunit.client.0.smithi107.stdout:7/789: creat d6/d1d/d64/d9a/da9/d71/db9/dca/dd0/f105 x:0 0 0 2024-03-04T13:43:20.870 INFO:tasks.workunit.client.0.smithi107.stdout:2/771: mknod da/d39/c106 0 2024-03-04T13:43:20.875 INFO:tasks.workunit.client.0.smithi107.stdout:3/824: write d1/de/d18/d1f/d42/f62 [616664,24918] 0 2024-03-04T13:43:20.878 INFO:tasks.workunit.client.0.smithi107.stdout:5/912: fdatasync d5/d79/d89/da3/d69/f134 0 2024-03-04T13:43:20.879 INFO:tasks.workunit.client.0.smithi107.stdout:1/854: dwrite d1/d46/d6a/d4e/daa/dc9/d33/d5d/fbf [0,4194304] 0 2024-03-04T13:43:20.880 INFO:tasks.workunit.client.0.smithi107.stdout:7/790: mkdir d6/d1d/dd9/d106 0 2024-03-04T13:43:20.881 INFO:tasks.workunit.client.0.smithi107.stdout:2/772: creat da/df/d73/dd1/d100/f107 x:0 0 0 2024-03-04T13:43:20.883 INFO:tasks.workunit.client.0.smithi107.stdout:5/913: chown d5/d8/d14/d2d/d92/dc1/dcc/f10e 0 1 2024-03-04T13:43:20.887 INFO:tasks.workunit.client.0.smithi107.stdout:1/855: write d1/d46/d6a/d4e/daa/dc9/f32 [7074673,69873] 0 2024-03-04T13:43:20.891 INFO:tasks.workunit.client.0.smithi107.stdout:2/773: creat da/df/d73/daa/ddc/f108 x:0 0 0 2024-03-04T13:43:20.892 INFO:tasks.workunit.client.0.smithi107.stdout:5/914: chown d5/dc/df/c5d 22712089 1 2024-03-04T13:43:20.893 INFO:tasks.workunit.client.0.smithi107.stdout:7/791: rename d6/d1d/d64/d9a/da9/d71/db9/cea to d6/d13/d16/d19/d39/c107 0 2024-03-04T13:43:20.896 INFO:tasks.workunit.client.0.smithi107.stdout:2/774: mknod da/d38/c109 0 2024-03-04T13:43:20.897 INFO:tasks.workunit.client.0.smithi107.stdout:5/915: write d5/d79/d89/da3/d68/f97 [3805961,41616] 0 2024-03-04T13:43:20.899 INFO:tasks.workunit.client.0.smithi107.stdout:1/856: write d1/d46/d6a/d4e/daa/dc9/d33/dbd/f102 [336083,40699] 0 2024-03-04T13:43:20.901 INFO:tasks.workunit.client.0.smithi107.stdout:0/848: dwrite da/dc/dd/d1e/fb5 [0,4194304] 0 2024-03-04T13:43:20.907 INFO:tasks.workunit.client.0.smithi107.stdout:2/775: fsync da/d44/d8b/d9d/df8/fa4 0 2024-03-04T13:43:20.907 INFO:tasks.workunit.client.0.smithi107.stdout:5/916: mkdir d5/d8/d14/d2d/d92/dc1/d133/d143 0 2024-03-04T13:43:20.909 INFO:tasks.workunit.client.0.smithi107.stdout:3/825: dread d1/d1e/d26/f6f [0,4194304] 0 2024-03-04T13:43:20.910 INFO:tasks.workunit.client.0.smithi107.stdout:0/849: symlink da/dc/dd/d3e/d26/d69/l120 0 2024-03-04T13:43:20.919 INFO:tasks.workunit.client.0.smithi107.stdout:3/826: rename d1/de/d18/d33/d48/d71/f15 to d1/de/d18/d33/d48/d71/f10a 0 2024-03-04T13:43:20.921 INFO:tasks.workunit.client.0.smithi107.stdout:2/776: creat da/d44/d103/f10a x:0 0 0 2024-03-04T13:43:20.923 INFO:tasks.workunit.client.0.smithi107.stdout:0/850: creat da/dc/dd/d3e/d26/d88/dd2/d73/d78/f121 x:0 0 0 2024-03-04T13:43:20.928 INFO:tasks.workunit.client.0.smithi107.stdout:3/827: creat d1/de/d18/d33/d48/d71/d4c/d54/db7/dbe/f10b x:0 0 0 2024-03-04T13:43:20.930 INFO:tasks.workunit.client.0.smithi107.stdout:0/851: creat da/dc/dd/dc9/f122 x:0 0 0 2024-03-04T13:43:20.930 INFO:tasks.workunit.client.0.smithi107.stdout:1/857: dwrite d1/d46/d6a/d4e/daa/dc9/d67/d6b/f84 [0,4194304] 0 2024-03-04T13:43:20.936 INFO:tasks.workunit.client.0.smithi107.stdout:3/828: read d1/de/d18/d33/d48/d71/d4c/d54/f79 [790751,61155] 0 2024-03-04T13:43:20.936 INFO:tasks.workunit.client.0.smithi107.stdout:5/917: dwrite d5/d8/f63 [0,4194304] 0 2024-03-04T13:43:20.940 INFO:tasks.workunit.client.0.smithi107.stdout:5/918: write d5/d8/da/db0/f10f [412621,65864] 0 2024-03-04T13:43:20.941 INFO:tasks.workunit.client.0.smithi107.stdout:1/858: rename d1/d46/d6a/d4e/daa/dc9/d33/dbd/f102 to d1/d46/d6a/d65/d106/f117 0 2024-03-04T13:43:20.945 INFO:tasks.workunit.client.0.smithi107.stdout:0/852: link da/d7e/f7f da/dc/dd/d3e/d26/f123 0 2024-03-04T13:43:20.946 INFO:tasks.workunit.client.0.smithi107.stdout:2/777: dread da/df/fb0 [0,4194304] 0 2024-03-04T13:43:20.948 INFO:tasks.workunit.client.0.smithi107.stdout:5/919: chown d5/f32 653401 1 2024-03-04T13:43:20.949 INFO:tasks.workunit.client.0.smithi107.stdout:2/778: chown da/df/d85/f89 38884 1 2024-03-04T13:43:20.949 INFO:tasks.workunit.client.0.smithi107.stdout:1/859: write d1/f94 [2852748,26374] 0 2024-03-04T13:43:20.951 INFO:tasks.workunit.client.0.smithi107.stdout:3/829: dread d1/de/d18/d33/d48/d71/f49 [0,4194304] 0 2024-03-04T13:43:20.952 INFO:tasks.workunit.client.0.smithi107.stdout:2/779: dread - da/d44/d8b/d9d/df8/d5c/d6a/ff4 zero size 2024-03-04T13:43:20.953 INFO:tasks.workunit.client.0.smithi107.stdout:9/750: sync 2024-03-04T13:43:20.955 INFO:tasks.workunit.client.0.smithi107.stdout:1/860: truncate d1/d46/d6a/d65/d106/ff9 223686 0 2024-03-04T13:43:20.955 INFO:tasks.workunit.client.0.smithi107.stdout:2/780: fdatasync da/df/d27/d58/d5f/ff1 0 2024-03-04T13:43:20.959 INFO:tasks.workunit.client.0.smithi107.stdout:1/861: chown d1/d46/d6a/d4e/daa/dc9/d67/d6b 7684608 1 2024-03-04T13:43:20.960 INFO:tasks.workunit.client.0.smithi107.stdout:3/830: read d1/de/d11/f45 [2504171,67003] 0 2024-03-04T13:43:20.962 INFO:tasks.workunit.client.0.smithi107.stdout:5/920: getdents d5/d2b/d4b/da6/dcf/d12b 0 2024-03-04T13:43:20.964 INFO:tasks.workunit.client.0.smithi107.stdout:3/831: creat d1/d41/f10c x:0 0 0 2024-03-04T13:43:20.968 INFO:tasks.workunit.client.0.smithi107.stdout:5/921: mkdir d5/d79/d89/da3/d144 0 2024-03-04T13:43:20.970 INFO:tasks.workunit.client.0.smithi107.stdout:1/862: getdents d1/d10/d16/d1b/d40 0 2024-03-04T13:43:20.974 INFO:tasks.workunit.client.0.smithi107.stdout:3/832: read - d1/de/d18/d33/d48/d71/d4c/d54/db7/dbe/f10b zero size 2024-03-04T13:43:20.976 INFO:tasks.workunit.client.0.smithi107.stdout:5/922: symlink d5/d8/d2a/d62/l145 0 2024-03-04T13:43:20.978 INFO:tasks.workunit.client.0.smithi107.stdout:9/751: dwrite d4/da/d11/d20/d23/f6c [4194304,4194304] 0 2024-03-04T13:43:20.978 INFO:tasks.workunit.client.0.smithi107.stdout:3/833: chown d1/d1e/d26/d9e/fec 63635 1 2024-03-04T13:43:20.978 INFO:tasks.workunit.client.0.smithi107.stdout:2/781: dwrite da/d29/d7c/dbd/fe2 [0,4194304] 0 2024-03-04T13:43:20.980 INFO:tasks.workunit.client.0.smithi107.stdout:1/863: mkdir d1/d46/d6a/d4e/df2/d118 0 2024-03-04T13:43:20.983 INFO:tasks.workunit.client.0.smithi107.stdout:9/752: readlink d4/l1a 0 2024-03-04T13:43:20.988 INFO:tasks.workunit.client.0.smithi107.stdout:3/834: unlink d1/de/d18/d1f/d42/l8c 0 2024-03-04T13:43:20.990 INFO:tasks.workunit.client.0.smithi107.stdout:1/864: mkdir d1/d46/d6a/d119 0 2024-03-04T13:43:20.993 INFO:tasks.workunit.client.0.smithi107.stdout:5/923: truncate d5/d8/d14/d2d/d92/dc1/feb 400781 0 2024-03-04T13:43:20.995 INFO:tasks.workunit.client.0.smithi107.stdout:9/753: symlink d4/da/d11/d20/d64/d77/de7/lfa 0 2024-03-04T13:43:20.996 INFO:tasks.workunit.client.0.smithi107.stdout:1/865: symlink d1/d10/d16/da9/l11a 0 2024-03-04T13:43:20.999 INFO:tasks.workunit.client.0.smithi107.stdout:2/782: write da/d44/f64 [959741,123261] 0 2024-03-04T13:43:21.000 INFO:tasks.workunit.client.0.smithi107.stdout:5/924: unlink d5/d8/d2a/d43/fdb 0 2024-03-04T13:43:21.005 INFO:tasks.workunit.client.0.smithi107.stdout:5/925: creat d5/d8/d14/d2d/df1/f146 x:0 0 0 2024-03-04T13:43:21.006 INFO:tasks.workunit.client.0.smithi107.stdout:1/866: chown d1/d46/d6a/d65/d106/l5e 0 1 2024-03-04T13:43:21.006 INFO:tasks.workunit.client.0.smithi107.stdout:0/853: dwrite da/dc/dd/d3e/d26/f71 [0,4194304] 0 2024-03-04T13:43:21.020 INFO:tasks.workunit.client.0.smithi107.stdout:0/854: rmdir da 39 2024-03-04T13:43:21.031 INFO:tasks.workunit.client.0.smithi107.stdout:5/926: dread d5/d79/d89/da3/d69/fd3 [0,4194304] 0 2024-03-04T13:43:21.035 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:21 smithi107 ceph-mon[28739]: pgmap v62: 65 pgs: 65 active+clean; 3.4 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 73 MiB/s rd, 180 MiB/s wr, 445 op/s 2024-03-04T13:43:21.042 INFO:tasks.workunit.client.0.smithi107.stdout:3/835: dread d1/d2d/f44 [0,4194304] 0 2024-03-04T13:43:21.056 INFO:tasks.workunit.client.0.smithi107.stdout:2/783: rename da/df/d27/d58 to da/df/d27/d10b 0 2024-03-04T13:43:21.057 INFO:tasks.workunit.client.0.smithi107.stdout:2/784: readlink da/d38/ld5 0 2024-03-04T13:43:21.059 INFO:tasks.workunit.client.0.smithi107.stdout:2/785: write da/f20 [754808,63142] 0 2024-03-04T13:43:21.059 INFO:tasks.workunit.client.0.smithi107.stdout:5/927: chown d5/f29 250236751 1 2024-03-04T13:43:21.059 INFO:tasks.workunit.client.0.smithi107.stdout:3/836: rename d1/d1e/daf/le8 to d1/d29/dda/l10d 0 2024-03-04T13:43:21.062 INFO:tasks.workunit.client.0.smithi107.stdout:0/855: stat da/dc/dd/d3e/d26/d88/c91 0 2024-03-04T13:43:21.063 INFO:tasks.workunit.client.0.smithi107.stdout:1/867: unlink d1/d46/d6a/d4e/daa/dc9/d33/dbd/l100 0 2024-03-04T13:43:21.066 INFO:tasks.workunit.client.0.smithi107.stdout:5/928: chown d5/d66/d96/db1/d13c 5 1 2024-03-04T13:43:21.067 INFO:tasks.workunit.client.0.smithi107.stdout:0/856: chown da/l36 353 1 2024-03-04T13:43:21.068 INFO:tasks.workunit.client.0.smithi107.stdout:2/786: creat da/d44/d8b/f10c x:0 0 0 2024-03-04T13:43:21.073 INFO:tasks.workunit.client.0.smithi107.stdout:5/929: unlink d5/d8/da/lba 0 2024-03-04T13:43:21.081 INFO:tasks.workunit.client.0.smithi107.stdout:0/857: creat da/dc/dd/d3e/d26/d69/d6b/f124 x:0 0 0 2024-03-04T13:43:21.087 INFO:tasks.workunit.client.0.smithi107.stdout:1/868: rename d1/d46/d6a/d4e/daa/dc9/l14 to d1/d46/d6a/d4e/daa/dc9/d33/d5d/d24/l11b 0 2024-03-04T13:43:21.090 INFO:tasks.workunit.client.0.smithi107.stdout:2/787: dwrite da/d34/f41 [4194304,4194304] 0 2024-03-04T13:43:21.092 INFO:tasks.workunit.client.0.smithi107.stdout:2/788: chown da/f10 497331 1 2024-03-04T13:43:21.092 INFO:tasks.workunit.client.0.smithi107.stdout:0/858: mknod da/dc/dd/ddb/c125 0 2024-03-04T13:43:21.094 INFO:tasks.workunit.client.0.smithi107.stdout:5/930: dread d5/d8/d14/ffb [0,4194304] 0 2024-03-04T13:43:21.094 INFO:tasks.workunit.client.0.smithi107.stdout:0/859: write da/dc/dd/d1e/f112 [576213,121060] 0 2024-03-04T13:43:21.095 INFO:tasks.workunit.client.0.smithi107.stdout:1/869: read d1/d46/d6a/d4e/daa/dc9/d67/f69 [63552,107533] 0 2024-03-04T13:43:21.096 INFO:tasks.workunit.client.0.smithi107.stdout:5/931: stat d5/d8/da/fa1 0 2024-03-04T13:43:21.099 INFO:tasks.workunit.client.0.smithi107.stdout:0/860: fdatasync da/dc/dd/d3e/d26/d88/fd7 0 2024-03-04T13:43:21.103 INFO:tasks.workunit.client.0.smithi107.stdout:5/932: chown d5/d79/d89/da3/l12a 1766951293 1 2024-03-04T13:43:21.105 INFO:tasks.workunit.client.0.smithi107.stdout:5/933: chown d5/d66/l71 1 1 2024-03-04T13:43:21.106 INFO:tasks.workunit.client.0.smithi107.stdout:1/870: write d1/d46/d6a/d4e/f6c [5105205,19102] 0 2024-03-04T13:43:21.107 INFO:tasks.workunit.client.0.smithi107.stdout:5/934: chown d5/dc/d22/fc2 128 1 2024-03-04T13:43:21.114 INFO:tasks.workunit.client.0.smithi107.stdout:0/861: rmdir da/dc/dd/d3e/d26/d88/dd2 39 2024-03-04T13:43:21.121 INFO:tasks.workunit.client.0.smithi107.stdout:1/871: readlink d1/d10/d16/d47/d7a/le8 0 2024-03-04T13:43:21.123 INFO:tasks.workunit.client.0.smithi107.stdout:5/935: dwrite d5/d79/d89/da3/d69/f134 [0,4194304] 0 2024-03-04T13:43:21.126 INFO:tasks.workunit.client.0.smithi107.stdout:1/872: mkdir d1/d46/d6a/d65/d11c 0 2024-03-04T13:43:21.126 INFO:tasks.workunit.client.0.smithi107.stdout:0/862: mknod da/dc/dd/d3e/d26/d88/dd2/d73/d78/c126 0 2024-03-04T13:43:21.129 INFO:tasks.workunit.client.0.smithi107.stdout:0/863: write da/dc/dd/d3e/d26/f64 [4651212,63] 0 2024-03-04T13:43:21.133 INFO:tasks.workunit.client.0.smithi107.stdout:4/814: sync 2024-03-04T13:43:21.133 INFO:tasks.workunit.client.0.smithi107.stdout:5/936: mkdir d5/d8/da/d147 0 2024-03-04T13:43:21.135 INFO:tasks.workunit.client.0.smithi107.stdout:0/864: mknod da/dc/dd/d3e/d26/d69/df9/c127 0 2024-03-04T13:43:21.135 INFO:tasks.workunit.client.0.smithi107.stdout:4/815: write d0/d9/d18/df1/f9c [5018337,36654] 0 2024-03-04T13:43:21.139 INFO:tasks.workunit.client.0.smithi107.stdout:0/865: creat da/dc/dd/d1e/dbf/f128 x:0 0 0 2024-03-04T13:43:21.145 INFO:tasks.workunit.client.0.smithi107.stdout:5/937: mknod d5/d2b/d4b/da6/c148 0 2024-03-04T13:43:21.149 INFO:tasks.workunit.client.0.smithi107.stdout:0/866: creat da/dc/dd/d3e/d26/d69/d6b/df0/f129 x:0 0 0 2024-03-04T13:43:21.149 INFO:tasks.workunit.client.0.smithi107.stdout:4/816: link d0/d9/d18/d29/f5c d0/d9/d78/dcf/f104 0 2024-03-04T13:43:21.150 INFO:tasks.workunit.client.0.smithi107.stdout:3/837: write d1/de/fa8 [262382,57236] 0 2024-03-04T13:43:21.157 INFO:tasks.workunit.client.0.smithi107.stdout:4/817: write d0/d9/de/d14/d33/f45 [1008018,77236] 0 2024-03-04T13:43:21.158 INFO:tasks.workunit.client.0.smithi107.stdout:3/838: creat d1/de/d18/d33/d48/d71/d4c/d54/db7/dbe/f10e x:0 0 0 2024-03-04T13:43:21.158 INFO:tasks.workunit.client.0.smithi107.stdout:4/818: stat d0/d9/d10/d6f/f95 0 2024-03-04T13:43:21.160 INFO:tasks.workunit.client.0.smithi107.stdout:3/839: stat d1/d1e/d26/dfd/l106 0 2024-03-04T13:43:21.162 INFO:tasks.workunit.client.0.smithi107.stdout:4/819: read d0/d9/de/f41 [813355,1341] 0 2024-03-04T13:43:21.164 INFO:tasks.workunit.client.0.smithi107.stdout:3/840: fdatasync d1/de/d18/d33/d48/d93/f108 0 2024-03-04T13:43:21.169 INFO:tasks.workunit.client.0.smithi107.stdout:2/789: dwrite da/df/d85/f95 [0,4194304] 0 2024-03-04T13:43:21.174 INFO:tasks.workunit.client.0.smithi107.stdout:5/938: truncate d5/d66/fa7 2398090 0 2024-03-04T13:43:21.174 INFO:tasks.workunit.client.0.smithi107.stdout:2/790: chown da/l23 150995679 1 2024-03-04T13:43:21.175 INFO:tasks.workunit.client.0.smithi107.stdout:0/867: dwrite da/dc/dd/d3e/d26/d69/d6b/f94 [0,4194304] 0 2024-03-04T13:43:21.177 INFO:tasks.workunit.client.0.smithi107.stdout:2/791: fdatasync da/df/d73/d7a/d9a/d68/ffa 0 2024-03-04T13:43:21.182 INFO:tasks.workunit.client.0.smithi107.stdout:4/820: link d0/d9/d78/l9d d0/d9/d18/d29/dfc/l105 0 2024-03-04T13:43:21.183 INFO:tasks.workunit.client.0.smithi107.stdout:2/792: chown da/df/d27/d10b/d5f/lce 183 1 2024-03-04T13:43:21.188 INFO:tasks.workunit.client.0.smithi107.stdout:0/868: getdents da/dc/dd/d3e/d26/d69/df9 0 2024-03-04T13:43:21.188 INFO:tasks.workunit.client.0.smithi107.stdout:5/939: link d5/d8/d2a/l3c d5/d8/l149 0 2024-03-04T13:43:21.189 INFO:tasks.workunit.client.0.smithi107.stdout:4/821: rename d0/d9/d18/df1/f3d to d0/d2f/f106 0 2024-03-04T13:43:21.192 INFO:tasks.workunit.client.0.smithi107.stdout:0/869: chown da/dc/dd/d1e/dbf/deb/l105 48 1 2024-03-04T13:43:21.194 INFO:tasks.workunit.client.0.smithi107.stdout:4/822: creat d0/d9/de/d14/d66/d7b/f107 x:0 0 0 2024-03-04T13:43:21.196 INFO:tasks.workunit.client.0.smithi107.stdout:5/940: mkdir d5/d8/d14/d2d/d92/dc1/d133/d143/d14a 0 2024-03-04T13:43:21.199 INFO:tasks.workunit.client.0.smithi107.stdout:4/823: readlink d0/d9/d10/d80/db8/d21/lfd 0 2024-03-04T13:43:21.200 INFO:tasks.workunit.client.0.smithi107.stdout:3/841: dread d1/d1e/f22 [0,4194304] 0 2024-03-04T13:43:21.200 INFO:tasks.workunit.client.0.smithi107.stdout:4/824: mkdir d0/d9/d10/d80/db8/d97/de3/d108 0 2024-03-04T13:43:21.203 INFO:tasks.workunit.client.0.smithi107.stdout:3/842: readlink d1/d1e/dba/lf1 0 2024-03-04T13:43:21.205 INFO:tasks.workunit.client.0.smithi107.stdout:5/941: dread d5/f20 [0,4194304] 0 2024-03-04T13:43:21.207 INFO:tasks.workunit.client.0.smithi107.stdout:3/843: mkdir d1/d1e/d26/d9e/d10f 0 2024-03-04T13:43:21.210 INFO:tasks.workunit.client.0.smithi107.stdout:5/942: dread d5/d79/d89/da3/f10b [0,4194304] 0 2024-03-04T13:43:21.214 INFO:tasks.workunit.client.0.smithi107.stdout:3/844: mkdir d1/de/d18/d33/d4d/d4f/d69/d110 0 2024-03-04T13:43:21.217 INFO:tasks.workunit.client.0.smithi107.stdout:3/845: mkdir d1/d29/d111 0 2024-03-04T13:43:21.217 INFO:tasks.workunit.client.0.smithi107.stdout:4/825: dwrite d0/d9/d10/d6f/f95 [0,4194304] 0 2024-03-04T13:43:21.219 INFO:tasks.workunit.client.0.smithi107.stdout:5/943: read d5/d8/d2a/d43/f57 [196256,81210] 0 2024-03-04T13:43:21.219 INFO:tasks.workunit.client.0.smithi107.stdout:4/826: stat d0/d9/d1e/d70/f93 0 2024-03-04T13:43:21.219 INFO:tasks.workunit.client.0.smithi107.stdout:2/793: dread da/d39/fc4 [0,4194304] 0 2024-03-04T13:43:21.221 INFO:tasks.workunit.client.0.smithi107.stdout:4/827: stat d0/d9/de/d14/d66/f8b 0 2024-03-04T13:43:21.223 INFO:tasks.workunit.client.0.smithi107.stdout:4/828: stat d0/d9/d10/d80/cda 0 2024-03-04T13:43:21.224 INFO:tasks.workunit.client.0.smithi107.stdout:5/944: creat d5/d8/da/db0/de6/f14b x:0 0 0 2024-03-04T13:43:21.226 INFO:tasks.workunit.client.0.smithi107.stdout:4/829: write d0/d9/de/d14/d66/d7b/f107 [813516,63969] 0 2024-03-04T13:43:21.229 INFO:tasks.workunit.client.0.smithi107.stdout:4/830: stat d0/d9/d10/d80/db8/fdd 0 2024-03-04T13:43:21.232 INFO:tasks.workunit.client.0.smithi107.stdout:5/945: mknod d5/d8/d14/c14c 0 2024-03-04T13:43:21.234 INFO:tasks.workunit.client.0.smithi107.stdout:4/831: truncate d0/d9/de/d14/d33/fa1 213754 0 2024-03-04T13:43:21.236 INFO:tasks.workunit.client.0.smithi107.stdout:3/846: dwrite d1/de/d18/d33/d48/d93/fdd [0,4194304] 0 2024-03-04T13:43:21.236 INFO:tasks.workunit.client.0.smithi107.stdout:5/946: truncate d5/d8/f7c 3642293 0 2024-03-04T13:43:21.239 INFO:tasks.workunit.client.0.smithi107.stdout:4/832: read d0/d9/d10/f25 [4003072,63727] 0 2024-03-04T13:43:21.240 INFO:tasks.workunit.client.0.smithi107.stdout:5/947: write d5/f6 [4736643,27196] 0 2024-03-04T13:43:21.240 INFO:tasks.workunit.client.0.smithi107.stdout:3/847: mkdir d1/de/d18/d33/d48/dc8/d112 0 2024-03-04T13:43:21.242 INFO:tasks.workunit.client.0.smithi107.stdout:3/848: chown d1/fa 615354 1 2024-03-04T13:43:21.249 INFO:tasks.workunit.client.0.smithi107.stdout:4/833: rename d0/d9/d18/d29/f5c to d0/d2f/d5f/f109 0 2024-03-04T13:43:21.251 INFO:tasks.workunit.client.0.smithi107.stdout:5/948: creat d5/d8/d2a/dd6/d10c/f14d x:0 0 0 2024-03-04T13:43:21.258 INFO:tasks.workunit.client.0.smithi107.stdout:3/849: dwrite d1/de/d18/d33/d6b/fb1 [0,4194304] 0 2024-03-04T13:43:21.265 INFO:tasks.workunit.client.0.smithi107.stdout:2/794: dread da/f20 [0,4194304] 0 2024-03-04T13:43:21.266 INFO:tasks.workunit.client.0.smithi107.stdout:2/795: write da/d44/f64 [1131057,69008] 0 2024-03-04T13:43:21.277 INFO:tasks.workunit.client.0.smithi107.stdout:3/850: dwrite d1/f3 [0,4194304] 0 2024-03-04T13:43:21.278 INFO:tasks.workunit.client.0.smithi107.stdout:5/949: truncate d5/d8/d2a/d62/fc8 2642611 0 2024-03-04T13:43:21.278 INFO:tasks.workunit.client.0.smithi107.stdout:4/834: dread d0/d9/d1e/f20 [0,4194304] 0 2024-03-04T13:43:21.282 INFO:tasks.workunit.client.0.smithi107.stdout:5/950: fsync d5/d2b/d4b/da6/f119 0 2024-03-04T13:43:21.283 INFO:tasks.workunit.client.0.smithi107.stdout:3/851: mkdir d1/de/d18/d33/d4d/d4f/d69/d113 0 2024-03-04T13:43:21.285 INFO:tasks.workunit.client.0.smithi107.stdout:5/951: dread - d5/d8/d2a/dd6/d10c/f14d zero size 2024-03-04T13:43:21.290 INFO:tasks.workunit.client.0.smithi107.stdout:6/833: sync 2024-03-04T13:43:21.290 INFO:tasks.workunit.client.0.smithi107.stdout:8/838: sync 2024-03-04T13:43:21.291 INFO:tasks.workunit.client.0.smithi107.stdout:5/952: write d5/d8/d2a/d43/f13b [312892,127132] 0 2024-03-04T13:43:21.294 INFO:tasks.workunit.client.0.smithi107.stdout:6/834: fdatasync f3 0 2024-03-04T13:43:21.296 INFO:tasks.workunit.client.0.smithi107.stdout:3/852: creat d1/de/d18/d33/d4d/d4f/dc3/dcf/f114 x:0 0 0 2024-03-04T13:43:21.301 INFO:tasks.workunit.client.0.smithi107.stdout:5/953: rename d5/dc/dc6/f11e to d5/d8/d14/d2d/d92/dd8/f14e 0 2024-03-04T13:43:21.306 INFO:tasks.workunit.client.0.smithi107.stdout:6/835: getdents df/d2e/d36/d77/d106/de4 0 2024-03-04T13:43:21.307 INFO:tasks.workunit.client.0.smithi107.stdout:2/796: dread da/d44/d8b/d9d/df8/d5c/f67 [4194304,4194304] 0 2024-03-04T13:43:21.307 INFO:tasks.workunit.client.0.smithi107.stdout:4/835: dread d0/d9/d18/f53 [0,4194304] 0 2024-03-04T13:43:21.308 INFO:tasks.workunit.client.0.smithi107.stdout:3/853: mkdir d1/de/d18/d33/d4d/d4f/d69/d110/d115 0 2024-03-04T13:43:21.308 INFO:tasks.workunit.client.0.smithi107.stdout:5/954: symlink d5/d2b/d4b/da6/dcf/d12b/l14f 0 2024-03-04T13:43:21.309 INFO:tasks.workunit.client.0.smithi107.stdout:6/836: fsync df/d2e/d36/d5b/d7d/dfc/fba 0 2024-03-04T13:43:21.311 INFO:tasks.workunit.client.0.smithi107.stdout:2/797: write da/d38/d3a/d5d/ff9 [4926653,32068] 0 2024-03-04T13:43:21.312 INFO:tasks.workunit.client.0.smithi107.stdout:6/837: chown df/d2e/d36/d5b/d7d/cda 4088 1 2024-03-04T13:43:21.313 INFO:tasks.workunit.client.0.smithi107.stdout:4/836: chown d0/d9/d18 0 1 2024-03-04T13:43:21.315 INFO:tasks.workunit.client.0.smithi107.stdout:5/955: creat d5/d8/f150 x:0 0 0 2024-03-04T13:43:21.322 INFO:tasks.workunit.client.0.smithi107.stdout:5/956: creat d5/dc/d22/f151 x:0 0 0 2024-03-04T13:43:21.324 INFO:tasks.workunit.client.0.smithi107.stdout:3/854: dread d1/de/d4b/fb8 [0,4194304] 0 2024-03-04T13:43:21.327 INFO:tasks.workunit.client.0.smithi107.stdout:3/855: chown d1/de/d18/d33/lc4 2061 1 2024-03-04T13:43:21.327 INFO:tasks.workunit.client.0.smithi107.stdout:8/839: dread d6/da/d25/d5f/f87 [0,4194304] 0 2024-03-04T13:43:21.328 INFO:tasks.workunit.client.0.smithi107.stdout:5/957: unlink d5/d79/d89/da3/d68/cf8 0 2024-03-04T13:43:21.329 INFO:tasks.workunit.client.0.smithi107.stdout:8/840: stat d6/d16/f24 0 2024-03-04T13:43:21.330 INFO:tasks.workunit.client.0.smithi107.stdout:4/837: write d0/d9/fd9 [821719,85213] 0 2024-03-04T13:43:21.333 INFO:tasks.workunit.client.0.smithi107.stdout:6/838: dwrite df/d1b/f1d [4194304,4194304] 0 2024-03-04T13:43:21.335 INFO:tasks.workunit.client.0.smithi107.stdout:4/838: mknod d0/d9/d10/d80/db8/c10a 0 2024-03-04T13:43:21.351 INFO:tasks.workunit.client.0.smithi107.stdout:2/798: dread da/d39/f46 [4194304,4194304] 0 2024-03-04T13:43:21.351 INFO:tasks.workunit.client.0.smithi107.stdout:3/856: dwrite d1/de/d18/d33/d48/d71/d4c/d54/f79 [0,4194304] 0 2024-03-04T13:43:21.355 INFO:tasks.workunit.client.0.smithi107.stdout:4/839: write d0/d9/de/f41 [1218883,21726] 0 2024-03-04T13:43:21.355 INFO:tasks.workunit.client.0.smithi107.stdout:3/857: write d1/de/d18/d33/d4d/d4f/dc3/dcf/f114 [472183,87080] 0 2024-03-04T13:43:21.358 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:21 smithi169 ceph-mon[36313]: pgmap v62: 65 pgs: 65 active+clean; 3.4 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 73 MiB/s rd, 180 MiB/s wr, 445 op/s 2024-03-04T13:43:21.360 INFO:tasks.workunit.client.0.smithi107.stdout:2/799: rename da/d38/c109 to da/df/d3e/dfe/c10d 0 2024-03-04T13:43:21.363 INFO:tasks.workunit.client.0.smithi107.stdout:8/841: dread d6/d16/d1f/d4c/da5/fc9 [0,4194304] 0 2024-03-04T13:43:21.366 INFO:tasks.workunit.client.0.smithi107.stdout:3/858: dread d1/d2d/f44 [4194304,4194304] 0 2024-03-04T13:43:21.367 INFO:tasks.workunit.client.0.smithi107.stdout:4/840: creat d0/d9/d78/dcf/f10b x:0 0 0 2024-03-04T13:43:21.369 INFO:tasks.workunit.client.0.smithi107.stdout:4/841: write d0/d9/d10/d80/d81/fbd [952951,71925] 0 2024-03-04T13:43:21.372 INFO:tasks.workunit.client.0.smithi107.stdout:2/800: rmdir da/d44/d8b/d9d 39 2024-03-04T13:43:21.374 INFO:tasks.workunit.client.0.smithi107.stdout:3/859: fdatasync d1/de/d3f/f80 0 2024-03-04T13:43:21.378 INFO:tasks.workunit.client.0.smithi107.stdout:4/842: rename d0/d9/de/d14/d66/lf6 to d0/d9/d10/d80/db8/d97/de3/l10c 0 2024-03-04T13:43:21.423 INFO:tasks.workunit.client.0.smithi107.stdout:2/801: mknod da/d44/d8b/d9d/dd0/c10e 0 2024-03-04T13:43:21.424 INFO:tasks.workunit.client.0.smithi107.stdout:9/754: sync 2024-03-04T13:43:21.424 INFO:tasks.workunit.client.0.smithi107.stdout:7/792: sync 2024-03-04T13:43:21.425 INFO:tasks.workunit.client.0.smithi107.stdout:4/843: mknod d0/d9/d10/d80/db8/d97/dcd/c10d 0 2024-03-04T13:43:21.427 INFO:tasks.workunit.client.0.smithi107.stdout:3/860: dread d1/de/d18/d33/d4d/d4f/d69/f7b [0,4194304] 0 2024-03-04T13:43:21.430 INFO:tasks.workunit.client.0.smithi107.stdout:3/861: write d1/f56 [2830448,73261] 0 2024-03-04T13:43:21.430 INFO:tasks.workunit.client.0.smithi107.stdout:7/793: fsync d6/d1d/d64/fde 0 2024-03-04T13:43:21.431 INFO:tasks.workunit.client.0.smithi107.stdout:9/755: mkdir d4/da/d11/d20/d39/dfb 0 2024-03-04T13:43:21.437 INFO:tasks.workunit.client.0.smithi107.stdout:7/794: write d6/d13/d16/d1c/f2d [5293829,67397] 0 2024-03-04T13:43:21.443 INFO:tasks.workunit.client.0.smithi107.stdout:7/795: symlink d6/d1d/d64/d9a/da9/d71/db9/dca/d3d/dc6/l108 0 2024-03-04T13:43:21.446 INFO:tasks.workunit.client.0.smithi107.stdout:2/802: dwrite da/d29/d7c/dbd/fe2 [0,4194304] 0 2024-03-04T13:43:21.446 INFO:tasks.workunit.client.0.smithi107.stdout:9/756: rename d4/da/f96 to d4/da/d11/d20/d2f/d9c/ffc 0 2024-03-04T13:43:21.448 INFO:tasks.workunit.client.0.smithi107.stdout:7/796: creat d6/d17/f109 x:0 0 0 2024-03-04T13:43:21.451 INFO:tasks.workunit.client.0.smithi107.stdout:2/803: write da/d39/fc4 [1271309,58185] 0 2024-03-04T13:43:21.456 INFO:tasks.workunit.client.0.smithi107.stdout:3/862: dwrite d1/de/d18/f8b [0,4194304] 0 2024-03-04T13:43:21.456 INFO:tasks.workunit.client.0.smithi107.stdout:7/797: truncate d6/d1d/d64/d9a/da9/fda 884140 0 2024-03-04T13:43:21.462 INFO:tasks.workunit.client.0.smithi107.stdout:9/757: getdents d4/da/d11/d20/d2f/d9c/daa 0 2024-03-04T13:43:21.462 INFO:tasks.workunit.client.0.smithi107.stdout:4/844: dread d0/d9/de/d14/d66/d7b/f8f [0,4194304] 0 2024-03-04T13:43:21.466 INFO:tasks.workunit.client.0.smithi107.stdout:7/798: readlink d6/d13/d16/d19/d38/d62/ddb/lf0 0 2024-03-04T13:43:21.472 INFO:tasks.workunit.client.0.smithi107.stdout:9/758: rename d4/da/d11/d20/d2f/d9c/d92/dce to d4/da/d11/d4c/d51/dfd 0 2024-03-04T13:43:21.479 INFO:tasks.workunit.client.0.smithi107.stdout:2/804: dwrite da/df/d85/f8d [0,4194304] 0 2024-03-04T13:43:21.482 INFO:tasks.workunit.client.0.smithi107.stdout:9/759: truncate d4/da/d11/d65/f6e 1276559 0 2024-03-04T13:43:21.485 INFO:tasks.workunit.client.0.smithi107.stdout:3/863: dwrite d1/d1e/d26/f60 [4194304,4194304] 0 2024-03-04T13:43:21.485 INFO:tasks.workunit.client.0.smithi107.stdout:4/845: dread d0/d9/de/d14/f4b [4194304,4194304] 0 2024-03-04T13:43:21.490 INFO:tasks.workunit.client.0.smithi107.stdout:4/846: chown d0/d9/d18/ce2 3045 1 2024-03-04T13:43:21.492 INFO:tasks.workunit.client.0.smithi107.stdout:3/864: fsync d1/de/d18/d1f/f94 0 2024-03-04T13:43:21.494 INFO:tasks.workunit.client.0.smithi107.stdout:2/805: write da/d44/d8b/d98/dc3/ffb [359486,73211] 0 2024-03-04T13:43:21.494 INFO:tasks.workunit.client.0.smithi107.stdout:7/799: dwrite d6/d17/f109 [0,4194304] 0 2024-03-04T13:43:21.495 INFO:tasks.workunit.client.0.smithi107.stdout:9/760: mknod d4/da/d11/d4c/cfe 0 2024-03-04T13:43:21.496 INFO:tasks.workunit.client.0.smithi107.stdout:9/761: readlink d4/l18 0 2024-03-04T13:43:21.498 INFO:tasks.workunit.client.0.smithi107.stdout:2/806: fdatasync da/df/d27/d10b/d5f/f93 0 2024-03-04T13:43:21.501 INFO:tasks.workunit.client.0.smithi107.stdout:4/847: mknod d0/d9/d18/d29/dfc/c10e 0 2024-03-04T13:43:21.506 INFO:tasks.workunit.client.0.smithi107.stdout:4/848: chown d0/d9/d10/dab/f103 50667325 1 2024-03-04T13:43:21.506 INFO:tasks.workunit.client.0.smithi107.stdout:9/762: fsync d4/da/d11/d20/d2f/d9c/ffc 0 2024-03-04T13:43:21.509 INFO:tasks.workunit.client.0.smithi107.stdout:3/865: rmdir d1/d29/d111 0 2024-03-04T13:43:21.510 INFO:tasks.workunit.client.0.smithi107.stdout:9/763: mknod d4/da/d11/d4c/d51/cff 0 2024-03-04T13:43:21.512 INFO:tasks.workunit.client.0.smithi107.stdout:7/800: link d6/d1d/d64/d9a/da9/l7c d6/d13/d16/d19/d38/d6e/l10a 0 2024-03-04T13:43:21.514 INFO:tasks.workunit.client.0.smithi107.stdout:4/849: creat d0/d9/d1e/d70/f10f x:0 0 0 2024-03-04T13:43:21.517 INFO:tasks.workunit.client.0.smithi107.stdout:4/850: fsync d0/d9/d18/df1/f50 0 2024-03-04T13:43:21.518 INFO:tasks.workunit.client.0.smithi107.stdout:2/807: dread da/df/d85/f8d [0,4194304] 0 2024-03-04T13:43:21.519 INFO:tasks.workunit.client.0.smithi107.stdout:9/764: mknod d4/da/d11/d20/d2f/d9c/c100 0 2024-03-04T13:43:21.521 INFO:tasks.workunit.client.0.smithi107.stdout:7/801: creat d6/d1d/d64/d9a/da9/d71/db9/dca/d3d/dc6/dcb/f10b x:0 0 0 2024-03-04T13:43:21.527 INFO:tasks.workunit.client.0.smithi107.stdout:2/808: rename da/d44/f64 to da/d44/d8b/d98/dc3/f10f 0 2024-03-04T13:43:21.528 INFO:tasks.workunit.client.0.smithi107.stdout:4/851: mknod d0/d9/d10/d80/db8/d97/de3/d108/c110 0 2024-03-04T13:43:21.531 INFO:tasks.workunit.client.0.smithi107.stdout:9/765: truncate d4/da/d11/d20/d2f/d9c/ffc 153855 0 2024-03-04T13:43:21.534 INFO:tasks.workunit.client.0.smithi107.stdout:3/866: truncate d1/de/d18/d33/d48/d93/fdd 3171903 0 2024-03-04T13:43:21.537 INFO:tasks.workunit.client.0.smithi107.stdout:4/852: fsync d0/d2f/f106 0 2024-03-04T13:43:21.537 INFO:tasks.workunit.client.0.smithi107.stdout:9/766: truncate d4/d34/f87 1729726 0 2024-03-04T13:43:21.542 INFO:tasks.workunit.client.0.smithi107.stdout:4/853: mkdir d0/d9/d10/d111 0 2024-03-04T13:43:21.543 INFO:tasks.workunit.client.0.smithi107.stdout:1/873: sync 2024-03-04T13:43:21.543 INFO:tasks.workunit.client.0.smithi107.stdout:0/870: sync 2024-03-04T13:43:21.548 INFO:tasks.workunit.client.0.smithi107.stdout:3/867: rename d1/de/d18/d33/d48/d71/d4c/fb9 to d1/de/f116 0 2024-03-04T13:43:21.549 INFO:tasks.workunit.client.0.smithi107.stdout:9/767: readlink d4/da/d11/d4c/d51/ld8 0 2024-03-04T13:43:21.550 INFO:tasks.workunit.client.0.smithi107.stdout:4/854: write d0/d9/d10/f8c [61720,80550] 0 2024-03-04T13:43:21.552 INFO:tasks.workunit.client.0.smithi107.stdout:7/802: dwrite d6/d1d/d64/d9a/da9/d71/db9/dca/fd3 [0,4194304] 0 2024-03-04T13:43:21.553 INFO:tasks.workunit.client.0.smithi107.stdout:1/874: creat d1/d46/d6a/d65/d106/f11d x:0 0 0 2024-03-04T13:43:21.557 INFO:tasks.workunit.client.0.smithi107.stdout:3/868: readlink d1/de/d11/l3b 0 2024-03-04T13:43:21.559 INFO:tasks.workunit.client.0.smithi107.stdout:4/855: truncate d0/d9/d18/f57 2504479 0 2024-03-04T13:43:21.559 INFO:tasks.workunit.client.0.smithi107.stdout:2/809: dread da/d29/f69 [0,4194304] 0 2024-03-04T13:43:21.561 INFO:tasks.workunit.client.0.smithi107.stdout:3/869: chown d1/de/d18/d33/d4d 2018506 1 2024-03-04T13:43:21.564 INFO:tasks.workunit.client.0.smithi107.stdout:1/875: creat d1/d46/d6a/d65/d106/f11e x:0 0 0 2024-03-04T13:43:21.565 INFO:tasks.workunit.client.0.smithi107.stdout:4/856: fdatasync d0/d9/de/d14/d33/fb5 0 2024-03-04T13:43:21.571 INFO:tasks.workunit.client.0.smithi107.stdout:3/870: creat d1/d2d/f117 x:0 0 0 2024-03-04T13:43:21.583 INFO:tasks.workunit.client.0.smithi107.stdout:0/871: getdents da/dc/dd/d1e 0 2024-03-04T13:43:21.587 INFO:tasks.workunit.client.0.smithi107.stdout:9/768: write d4/da/d11/d4c/d4d/d72/d7b/f81 [763152,121457] 0 2024-03-04T13:43:21.589 INFO:tasks.workunit.client.0.smithi107.stdout:3/871: unlink d1/de/d18/d33/lc4 0 2024-03-04T13:43:21.591 INFO:tasks.workunit.client.0.smithi107.stdout:3/872: stat d1/de/d18/d33/d4d/d4f/dc3/dcf 0 2024-03-04T13:43:21.595 INFO:tasks.workunit.client.0.smithi107.stdout:0/872: symlink da/dad/l12a 0 2024-03-04T13:43:21.595 INFO:tasks.workunit.client.0.smithi107.stdout:7/803: write d6/d13/d16/d1c/faf [1430884,103694] 0 2024-03-04T13:43:21.599 INFO:tasks.workunit.client.0.smithi107.stdout:9/769: read d4/da/d11/d65/f6e [883673,20708] 0 2024-03-04T13:43:21.599 INFO:tasks.workunit.client.0.smithi107.stdout:4/857: creat d0/f112 x:0 0 0 2024-03-04T13:43:21.600 INFO:tasks.workunit.client.0.smithi107.stdout:6/839: sync 2024-03-04T13:43:21.600 INFO:tasks.workunit.client.0.smithi107.stdout:5/958: sync 2024-03-04T13:43:21.600 INFO:tasks.workunit.client.0.smithi107.stdout:8/842: sync 2024-03-04T13:43:21.600 INFO:tasks.workunit.client.0.smithi107.stdout:3/873: creat d1/de/d18/d7e/f118 x:0 0 0 2024-03-04T13:43:21.601 INFO:tasks.workunit.client.0.smithi107.stdout:2/810: getdents da/df/d73/d7a 0 2024-03-04T13:43:21.602 INFO:tasks.workunit.client.0.smithi107.stdout:1/876: write d1/d46/d6a/d4e/daa/dc9/d33/d5d/d24/fae [146548,88949] 0 2024-03-04T13:43:21.608 INFO:tasks.workunit.client.0.smithi107.stdout:2/811: write da/df/d73/daa/ddc/f108 [479408,34024] 0 2024-03-04T13:43:21.608 INFO:tasks.workunit.client.0.smithi107.stdout:2/812: chown da/d38/c96 0 1 2024-03-04T13:43:21.611 INFO:tasks.workunit.client.0.smithi107.stdout:3/874: rmdir d1/d1e 39 2024-03-04T13:43:21.612 INFO:tasks.workunit.client.0.smithi107.stdout:6/840: symlink df/d17/d6b/d9b/d61/l10f 0 2024-03-04T13:43:21.615 INFO:tasks.workunit.client.0.smithi107.stdout:1/877: unlink d1/d10/d16/fd9 0 2024-03-04T13:43:21.616 INFO:tasks.workunit.client.0.smithi107.stdout:8/843: mkdir d6/da/d25/d10b 0 2024-03-04T13:43:21.620 INFO:tasks.workunit.client.0.smithi107.stdout:9/770: creat d4/da/d11/d20/d64/d77/ddc/f101 x:0 0 0 2024-03-04T13:43:21.622 INFO:tasks.workunit.client.0.smithi107.stdout:5/959: dread d5/d79/d89/da3/d68/fe8 [0,4194304] 0 2024-03-04T13:43:21.624 INFO:tasks.workunit.client.0.smithi107.stdout:7/804: rmdir d6/d13/d16/dfd 0 2024-03-04T13:43:21.626 INFO:tasks.workunit.client.0.smithi107.stdout:8/844: unlink f3 0 2024-03-04T13:43:21.626 INFO:tasks.workunit.client.0.smithi107.stdout:3/875: dread d1/de/d18/d33/d4d/f5b [0,4194304] 0 2024-03-04T13:43:21.627 INFO:tasks.workunit.client.0.smithi107.stdout:2/813: symlink da/d44/d8b/d98/l110 0 2024-03-04T13:43:21.628 INFO:tasks.workunit.client.0.smithi107.stdout:4/858: dwrite d0/d9/d10/d80/db8/fb4 [0,4194304] 0 2024-03-04T13:43:21.629 INFO:tasks.workunit.client.0.smithi107.stdout:9/771: symlink d4/da/d11/d20/d64/d77/ddc/l102 0 2024-03-04T13:43:21.636 INFO:tasks.workunit.client.0.smithi107.stdout:5/960: mknod d5/dc/c152 0 2024-03-04T13:43:21.640 INFO:tasks.workunit.client.0.smithi107.stdout:1/878: rename d1/d46/d6a/d4e/f6c to d1/d46/d6a/f11f 0 2024-03-04T13:43:21.641 INFO:tasks.workunit.client.0.smithi107.stdout:9/772: dread - d4/da/d11/d20/d64/d77/f82 zero size 2024-03-04T13:43:21.645 INFO:tasks.workunit.client.0.smithi107.stdout:7/805: truncate d6/d13/d16/d19/d38/d2c/f9f 661673 0 2024-03-04T13:43:21.647 INFO:tasks.workunit.client.0.smithi107.stdout:0/873: dwrite da/dc/dd/d3e/f7a [0,4194304] 0 2024-03-04T13:43:21.649 INFO:tasks.workunit.client.0.smithi107.stdout:6/841: dwrite df/d17/d6b/d9b/d61/f8a [0,4194304] 0 2024-03-04T13:43:21.652 INFO:tasks.workunit.client.0.smithi107.stdout:8/845: mkdir d6/d16/d1f/d4c/da4/daa/d108/d10c 0 2024-03-04T13:43:21.656 INFO:tasks.workunit.client.0.smithi107.stdout:5/961: unlink d5/d79/d89/da3/d68/fe8 0 2024-03-04T13:43:21.665 INFO:tasks.workunit.client.0.smithi107.stdout:4/859: creat d0/f113 x:0 0 0 2024-03-04T13:43:21.666 INFO:tasks.workunit.client.0.smithi107.stdout:3/876: dread d1/f6 [0,4194304] 0 2024-03-04T13:43:21.668 INFO:tasks.workunit.client.0.smithi107.stdout:7/806: mknod d6/d1d/d64/d9a/da9/d71/db9/dca/d3d/dc6/dcb/c10c 0 2024-03-04T13:43:21.674 INFO:tasks.workunit.client.0.smithi107.stdout:6/842: creat df/d17/d6b/da8/d105/f110 x:0 0 0 2024-03-04T13:43:21.674 INFO:tasks.workunit.client.0.smithi107.stdout:2/814: dwrite da/d29/d7c/dbd/fe2 [0,4194304] 0 2024-03-04T13:43:21.675 INFO:tasks.workunit.client.0.smithi107.stdout:3/877: dread - d1/de/d18/d33/d48/d71/d4c/d54/db7/dbe/f10b zero size 2024-03-04T13:43:21.679 INFO:tasks.workunit.client.0.smithi107.stdout:0/874: symlink da/dc/dd/d3e/d26/d69/df9/l12b 0 2024-03-04T13:43:21.679 INFO:tasks.workunit.client.0.smithi107.stdout:8/846: rmdir d6/da/d25/dd1 39 2024-03-04T13:43:21.687 INFO:tasks.workunit.client.0.smithi107.stdout:5/962: unlink d5/d8/d14/d2d/d11d/d127/c12c 0 2024-03-04T13:43:21.687 INFO:tasks.workunit.client.0.smithi107.stdout:3/878: stat d1/de/d18/d33/d48/fce 0 2024-03-04T13:43:21.688 INFO:tasks.workunit.client.0.smithi107.stdout:0/875: dread - da/dc/dd/d3e/d26/d69/df9/f103 zero size 2024-03-04T13:43:21.690 INFO:tasks.workunit.client.0.smithi107.stdout:4/860: creat d0/d9/d1e/d70/f114 x:0 0 0 2024-03-04T13:43:21.693 INFO:tasks.workunit.client.0.smithi107.stdout:2/815: creat da/df/d73/db3/db1/f111 x:0 0 0 2024-03-04T13:43:21.694 INFO:tasks.workunit.client.0.smithi107.stdout:7/807: creat d6/d13/d16/d19/d39/d7a/f10d x:0 0 0 2024-03-04T13:43:21.696 INFO:tasks.workunit.client.0.smithi107.stdout:8/847: creat d6/d16/d69/deb/d7c/ddf/f10d x:0 0 0 2024-03-04T13:43:21.701 INFO:tasks.workunit.client.0.smithi107.stdout:5/963: dread d5/d8/d14/d2d/d92/dc1/feb [0,4194304] 0 2024-03-04T13:43:21.707 INFO:tasks.workunit.client.0.smithi107.stdout:8/848: chown d6/da/d25/d2a/l9d 245305369 1 2024-03-04T13:43:21.708 INFO:tasks.workunit.client.0.smithi107.stdout:4/861: creat d0/d9/d10/d80/db8/f115 x:0 0 0 2024-03-04T13:43:21.712 INFO:tasks.workunit.client.0.smithi107.stdout:4/862: stat d0/d9/d10/d80/d81/fbd 0 2024-03-04T13:43:21.720 INFO:tasks.workunit.client.0.smithi107.stdout:2/816: mkdir da/d44/d112 0 2024-03-04T13:43:21.720 INFO:tasks.workunit.client.0.smithi107.stdout:3/879: creat d1/d1e/dae/df2/f119 x:0 0 0 2024-03-04T13:43:21.723 INFO:tasks.workunit.client.0.smithi107.stdout:8/849: read d6/da/f72 [603239,56276] 0 2024-03-04T13:43:21.727 INFO:tasks.workunit.client.0.smithi107.stdout:2/817: rmdir da/d44/d8b/d98/dc3 39 2024-03-04T13:43:21.729 INFO:tasks.workunit.client.0.smithi107.stdout:8/850: creat d6/d16/d69/deb/d7c/ddf/f10e x:0 0 0 2024-03-04T13:43:21.729 INFO:tasks.workunit.client.0.smithi107.stdout:9/773: write d4/da/d11/d65/f6e [1961226,41432] 0 2024-03-04T13:43:21.730 INFO:tasks.workunit.client.0.smithi107.stdout:3/880: stat d1/de/d18/d33/d48/d71/d4c/d54/db7/ff7 0 2024-03-04T13:43:21.736 INFO:tasks.workunit.client.0.smithi107.stdout:2/818: mkdir da/d38/d113 0 2024-03-04T13:43:21.737 INFO:tasks.workunit.client.0.smithi107.stdout:9/774: chown d4/d34/c4f 6624 1 2024-03-04T13:43:21.737 INFO:tasks.workunit.client.0.smithi107.stdout:4/863: dwrite d0/d9/de/d14/ffb [0,4194304] 0 2024-03-04T13:43:21.743 INFO:tasks.workunit.client.0.smithi107.stdout:4/864: chown d0/d9/d18/d29/l8d 25 1 2024-03-04T13:43:21.743 INFO:tasks.workunit.client.0.smithi107.stdout:8/851: chown d6/d16/d69/deb/d3e/d67/fdc 53114 1 2024-03-04T13:43:21.749 INFO:tasks.workunit.client.0.smithi107.stdout:6/843: write df/d57/f64 [1259910,95506] 0 2024-03-04T13:43:21.751 INFO:tasks.workunit.client.0.smithi107.stdout:1/879: sync 2024-03-04T13:43:21.753 INFO:tasks.workunit.client.0.smithi107.stdout:1/880: chown d1/d10/d16/d47/d71 28272 1 2024-03-04T13:43:21.756 INFO:tasks.workunit.client.0.smithi107.stdout:8/852: creat d6/d16/d1f/d4c/da5/f10f x:0 0 0 2024-03-04T13:43:21.758 INFO:tasks.workunit.client.0.smithi107.stdout:9/775: truncate d4/d34/f87 1012702 0 2024-03-04T13:43:21.761 INFO:tasks.workunit.client.0.smithi107.stdout:0/876: write da/dc/dd/d3e/f1c [270168,80113] 0 2024-03-04T13:43:21.762 INFO:tasks.workunit.client.0.smithi107.stdout:8/853: write d6/da/d25/d2a/fef [771881,115595] 0 2024-03-04T13:43:21.763 INFO:tasks.workunit.client.0.smithi107.stdout:3/881: link d1/de/d18/d33/d4d/ffe d1/de/d18/d33/d48/d71/f11a 0 2024-03-04T13:43:21.765 INFO:tasks.workunit.client.0.smithi107.stdout:6/844: mkdir df/d2e/d36/d71/d111 0 2024-03-04T13:43:21.766 INFO:tasks.workunit.client.0.smithi107.stdout:7/808: write d6/d1d/d64/d9a/da9/d71/db9/dca/d22/fb3 [754576,55685] 0 2024-03-04T13:43:21.766 INFO:tasks.workunit.client.0.smithi107.stdout:3/882: readlink d1/de/d4b/lea 0 2024-03-04T13:43:21.769 INFO:tasks.workunit.client.0.smithi107.stdout:1/881: rename d1/d10/d16/l1c to d1/d10/d16/d47/d7a/dda/l120 0 2024-03-04T13:43:21.772 INFO:tasks.workunit.client.0.smithi107.stdout:9/776: creat d4/da/d11/d4c/d51/f103 x:0 0 0 2024-03-04T13:43:21.774 INFO:tasks.workunit.client.0.smithi107.stdout:8/854: readlink d6/d16/d1f/d63/l83 0 2024-03-04T13:43:21.775 INFO:tasks.workunit.client.0.smithi107.stdout:2/819: write da/d44/d8b/d9d/fd6 [754421,83222] 0 2024-03-04T13:43:21.779 INFO:tasks.workunit.client.0.smithi107.stdout:0/877: creat da/dc/dd/d3e/d26/d69/de9/f12c x:0 0 0 2024-03-04T13:43:21.779 INFO:tasks.workunit.client.0.smithi107.stdout:6/845: mkdir df/d2e/d36/d5b/d7d/d112 0 2024-03-04T13:43:21.783 INFO:tasks.workunit.client.0.smithi107.stdout:5/964: dwrite d5/d8/d2a/d62/fc8 [0,4194304] 0 2024-03-04T13:43:21.789 INFO:tasks.workunit.client.0.smithi107.stdout:4/865: dwrite d0/d9/d78/fbb [0,4194304] 0 2024-03-04T13:43:21.797 INFO:tasks.workunit.client.0.smithi107.stdout:8/855: truncate d6/d16/d69/deb/d3e/fe2 925348 0 2024-03-04T13:43:21.798 INFO:tasks.workunit.client.0.smithi107.stdout:6/846: unlink df/ldc 0 2024-03-04T13:43:21.799 INFO:tasks.workunit.client.0.smithi107.stdout:4/866: fdatasync d0/d9/d10/d80/db8/f69 0 2024-03-04T13:43:21.805 INFO:tasks.workunit.client.0.smithi107.stdout:3/883: dwrite d1/fff [0,4194304] 0 2024-03-04T13:43:21.814 INFO:tasks.workunit.client.0.smithi107.stdout:2/820: rename da/df/d73/daa/ddc/f108 to da/df/d27/d10b/f114 0 2024-03-04T13:43:21.819 INFO:tasks.workunit.client.0.smithi107.stdout:2/821: read - da/d44/d8b/f10c zero size 2024-03-04T13:43:21.821 INFO:tasks.workunit.client.0.smithi107.stdout:5/965: write d5/d8/d2a/f53 [662936,65488] 0 2024-03-04T13:43:21.829 INFO:tasks.workunit.client.0.smithi107.stdout:1/882: dread d1/d46/d6a/d4e/d60/f53 [0,4194304] 0 2024-03-04T13:43:21.832 INFO:tasks.workunit.client.0.smithi107.stdout:0/878: dwrite da/dc/dd/d3e/fee [0,4194304] 0 2024-03-04T13:43:21.833 INFO:tasks.workunit.client.0.smithi107.stdout:6/847: dread - df/d17/d24/d30/d5d/d6a/f94 zero size 2024-03-04T13:43:21.833 INFO:tasks.workunit.client.0.smithi107.stdout:7/809: dwrite d6/d1d/d64/d9a/da9/d71/db9/dca/d3d/fdd [0,4194304] 0 2024-03-04T13:43:21.837 INFO:tasks.workunit.client.0.smithi107.stdout:8/856: read d6/da/d25/dd1/fd2 [669480,45888] 0 2024-03-04T13:43:21.842 INFO:tasks.workunit.client.0.smithi107.stdout:2/822: rename da/df/d73/d7a/d9a/d68/ffa to da/d38/d3a/f115 0 2024-03-04T13:43:21.843 INFO:tasks.workunit.client.0.smithi107.stdout:4/867: dread d0/fee [0,4194304] 0 2024-03-04T13:43:21.845 INFO:tasks.workunit.client.0.smithi107.stdout:9/777: link d4/da/d11/d20/d39/f90 d4/da/d11/d20/d23/de8/f104 0 2024-03-04T13:43:21.847 INFO:tasks.workunit.client.0.smithi107.stdout:3/884: mkdir d1/de/d18/d33/d4d/d4f/d69/d113/d11b 0 2024-03-04T13:43:21.848 INFO:tasks.workunit.client.0.smithi107.stdout:1/883: rmdir d1/d46/d6a/d4e/daa/dc9/d33 39 2024-03-04T13:43:21.851 INFO:tasks.workunit.client.0.smithi107.stdout:7/810: read d6/d13/d16/d1c/faf [1144232,119596] 0 2024-03-04T13:43:21.854 INFO:tasks.workunit.client.0.smithi107.stdout:8/857: mknod d6/d16/d69/deb/d3e/d96/dd8/c110 0 2024-03-04T13:43:21.855 INFO:tasks.workunit.client.0.smithi107.stdout:7/811: readlink d6/d1d/d64/d9a/da9/d71/lc9 0 2024-03-04T13:43:21.859 INFO:tasks.workunit.client.0.smithi107.stdout:7/812: write d6/d13/d16/d19/d38/d37/f9e [3798529,122313] 0 2024-03-04T13:43:21.861 INFO:tasks.workunit.client.0.smithi107.stdout:3/885: write d1/de/f68 [1480207,66982] 0 2024-03-04T13:43:21.863 INFO:tasks.workunit.client.0.smithi107.stdout:0/879: mkdir da/dc/d34/d10d/d12d 0 2024-03-04T13:43:21.863 INFO:tasks.workunit.client.0.smithi107.stdout:5/966: getdents d5/d79/d89/da3/d68 0 2024-03-04T13:43:21.871 INFO:tasks.workunit.client.0.smithi107.stdout:7/813: write d6/d1d/d64/d9a/da9/d71/db9/dca/d3d/dc6/ff1 [151620,108150] 0 2024-03-04T13:43:21.872 INFO:tasks.workunit.client.0.smithi107.stdout:7/814: readlink d6/d13/d16/d19/d38/d2c/l41 0 2024-03-04T13:43:21.872 INFO:tasks.workunit.client.0.smithi107.stdout:3/886: symlink d1/de/d18/d33/d48/dc8/l11c 0 2024-03-04T13:43:21.873 INFO:tasks.workunit.client.0.smithi107.stdout:2/823: write da/df/d27/fa7 [1153424,94830] 0 2024-03-04T13:43:21.881 INFO:tasks.workunit.client.0.smithi107.stdout:8/858: write d6/d16/d69/deb/d3e/d67/fdc [1061186,47697] 0 2024-03-04T13:43:21.881 INFO:tasks.workunit.client.0.smithi107.stdout:1/884: symlink d1/d46/d6a/d75/dfd/l121 0 2024-03-04T13:43:21.882 INFO:tasks.workunit.client.0.smithi107.stdout:9/778: creat d4/da/d11/d20/f105 x:0 0 0 2024-03-04T13:43:21.884 INFO:tasks.workunit.client.0.smithi107.stdout:2/824: creat da/d44/d103/f116 x:0 0 0 2024-03-04T13:43:21.886 INFO:tasks.workunit.client.0.smithi107.stdout:3/887: rmdir d1/de/d18/d33/d48/d71/d4c/d54 39 2024-03-04T13:43:21.891 INFO:tasks.workunit.client.0.smithi107.stdout:1/885: write d1/d46/d6a/d65/d106/f11e [878004,14694] 0 2024-03-04T13:43:21.894 INFO:tasks.workunit.client.0.smithi107.stdout:8/859: mkdir d6/d16/d69/deb/d3e/d111 0 2024-03-04T13:43:21.895 INFO:tasks.workunit.client.0.smithi107.stdout:6/848: dread df/d2e/d36/d77/d106/d7e/d86/f100 [0,4194304] 0 2024-03-04T13:43:21.895 INFO:tasks.workunit.client.0.smithi107.stdout:0/880: dwrite da/dc/dd/d3e/d26/d69/de9/f12c [0,4194304] 0 2024-03-04T13:43:21.895 INFO:tasks.workunit.client.0.smithi107.stdout:9/779: mknod d4/da/d11/d20/d2f/df7/c106 0 2024-03-04T13:43:21.896 INFO:tasks.workunit.client.0.smithi107.stdout:3/888: dread d1/de/d11/f95 [0,4194304] 0 2024-03-04T13:43:21.897 INFO:tasks.workunit.client.0.smithi107.stdout:2/825: mkdir da/d29/d7c/dbd/d117 0 2024-03-04T13:43:21.899 INFO:tasks.workunit.client.0.smithi107.stdout:7/815: truncate d6/d13/d14/fa4 2690046 0 2024-03-04T13:43:21.904 INFO:tasks.workunit.client.0.smithi107.stdout:4/868: dread d0/d9/d10/d80/db8/d21/f6d [0,4194304] 0 2024-03-04T13:43:21.906 INFO:tasks.workunit.client.0.smithi107.stdout:5/967: dwrite d5/d2b/ff7 [0,4194304] 0 2024-03-04T13:43:21.906 INFO:tasks.workunit.client.0.smithi107.stdout:0/881: dread da/dc/dd/d3e/d26/d88/dd2/fbd [0,4194304] 0 2024-03-04T13:43:21.906 INFO:tasks.workunit.client.0.smithi107.stdout:8/860: symlink d6/d16/d69/l112 0 2024-03-04T13:43:21.917 INFO:tasks.workunit.client.0.smithi107.stdout:3/889: rename d1/l8 to d1/de/d18/d33/d4d/d4f/d69/d110/l11d 0 2024-03-04T13:43:21.924 INFO:tasks.workunit.client.0.smithi107.stdout:0/882: symlink da/dc/dd/d1e/dbf/l12e 0 2024-03-04T13:43:21.927 INFO:tasks.workunit.client.0.smithi107.stdout:3/890: read - d1/d1e/dae/df2/f119 zero size 2024-03-04T13:43:21.927 INFO:tasks.workunit.client.0.smithi107.stdout:2/826: rename da/df/d73/d7a/d9a/d68/l86 to da/d44/d8b/d9d/l118 0 2024-03-04T13:43:21.929 INFO:tasks.workunit.client.0.smithi107.stdout:9/780: dwrite d4/da/d11/d20/d64/f76 [0,4194304] 0 2024-03-04T13:43:21.930 INFO:tasks.workunit.client.0.smithi107.stdout:8/861: mkdir d6/da/d25/d113 0 2024-03-04T13:43:21.932 INFO:tasks.workunit.client.0.smithi107.stdout:4/869: write d0/d9/d10/d80/db8/d21/fa2 [1019241,3534] 0 2024-03-04T13:43:21.934 INFO:tasks.workunit.client.0.smithi107.stdout:6/849: dwrite df/d2e/d36/d77/f84 [0,4194304] 0 2024-03-04T13:43:21.934 INFO:tasks.workunit.client.0.smithi107.stdout:3/891: rmdir d1/d1e/dae 39 2024-03-04T13:43:21.937 INFO:tasks.workunit.client.0.smithi107.stdout:9/781: write d4/f26 [4490778,79606] 0 2024-03-04T13:43:21.937 INFO:tasks.workunit.client.0.smithi107.stdout:8/862: dread - d6/da/f70 zero size 2024-03-04T13:43:21.939 INFO:tasks.workunit.client.0.smithi107.stdout:1/886: dwrite d1/d46/d6a/d4e/d60/f78 [0,4194304] 0 2024-03-04T13:43:21.939 INFO:tasks.workunit.client.0.smithi107.stdout:0/883: mknod da/dc/dd/d1e/c12f 0 2024-03-04T13:43:21.944 INFO:tasks.workunit.client.0.smithi107.stdout:3/892: creat d1/de/d4b/f11e x:0 0 0 2024-03-04T13:43:21.956 INFO:tasks.workunit.client.0.smithi107.stdout:6/850: dread - df/d2e/d36/d77/d106/d7e/d86/fc5 zero size 2024-03-04T13:43:21.959 INFO:tasks.workunit.client.0.smithi107.stdout:7/816: dwrite d6/d13/d16/d19/d39/f40 [8388608,4194304] 0 2024-03-04T13:43:21.961 INFO:tasks.workunit.client.0.smithi107.stdout:0/884: chown da/dc/dd/d1e/dbf/l12e 220816 1 2024-03-04T13:43:21.963 INFO:tasks.workunit.client.0.smithi107.stdout:5/968: dwrite d5/d2b/d4b/da6/f119 [0,4194304] 0 2024-03-04T13:43:21.972 INFO:tasks.workunit.client.0.smithi107.stdout:9/782: mkdir d4/da/d11/d20/d23/d7e/d84/d107 0 2024-03-04T13:43:21.973 INFO:tasks.workunit.client.0.smithi107.stdout:6/851: chown df/d2e/d36/d5b/d7d/d112 11 1 2024-03-04T13:43:21.974 INFO:tasks.workunit.client.0.smithi107.stdout:2/827: creat da/d44/d8b/d98/f119 x:0 0 0 2024-03-04T13:43:21.976 INFO:tasks.workunit.client.0.smithi107.stdout:5/969: dread - d5/dc/d22/f151 zero size 2024-03-04T13:43:21.979 INFO:tasks.workunit.client.0.smithi107.stdout:8/863: rmdir d6/da/d25/d10b 0 2024-03-04T13:43:21.981 INFO:tasks.workunit.client.0.smithi107.stdout:2/828: readlink da/df/d27/d10b/d5f/lce 0 2024-03-04T13:43:21.981 INFO:tasks.workunit.client.0.smithi107.stdout:1/887: creat d1/d46/d6a/d4e/daa/dc9/d33/dbd/f122 x:0 0 0 2024-03-04T13:43:21.985 INFO:tasks.workunit.client.0.smithi107.stdout:7/817: symlink d6/d1d/d36/l10e 0 2024-03-04T13:43:21.986 INFO:tasks.workunit.client.0.smithi107.stdout:2/829: mkdir da/df/d85/d11a 0 2024-03-04T13:43:21.988 INFO:tasks.workunit.client.0.smithi107.stdout:9/783: readlink d4/da/d11/d4c/d4d/d72/lf3 0 2024-03-04T13:43:21.989 INFO:tasks.workunit.client.0.smithi107.stdout:2/830: fsync da/d44/d103/f10a 0 2024-03-04T13:43:21.992 INFO:tasks.workunit.client.0.smithi107.stdout:6/852: fsync df/d2e/d36/d77/d106/f70 0 2024-03-04T13:43:21.992 INFO:tasks.workunit.client.0.smithi107.stdout:3/893: dwrite d1/de/d18/d33/d48/fa6 [0,4194304] 0 2024-03-04T13:43:21.997 INFO:tasks.workunit.client.0.smithi107.stdout:1/888: mkdir d1/d46/d6a/d4e/df2/d123 0 2024-03-04T13:43:22.000 INFO:tasks.workunit.client.0.smithi107.stdout:2/831: fdatasync da/d44/d103/f10a 0 2024-03-04T13:43:22.001 INFO:tasks.workunit.client.0.smithi107.stdout:7/818: creat d6/d13/dbc/f10f x:0 0 0 2024-03-04T13:43:22.002 INFO:tasks.workunit.client.0.smithi107.stdout:4/870: dwrite d0/d9/de/fbe [4194304,4194304] 0 2024-03-04T13:43:22.002 INFO:tasks.workunit.client.0.smithi107.stdout:9/784: mknod d4/da/d11/d20/d2f/c108 0 2024-03-04T13:43:22.006 INFO:tasks.workunit.client.0.smithi107.stdout:3/894: readlink d1/d29/dda/l10d 0 2024-03-04T13:43:22.007 INFO:tasks.workunit.client.0.smithi107.stdout:5/970: truncate d5/d8/d2a/d43/fbb 943819 0 2024-03-04T13:43:22.008 INFO:tasks.workunit.client.0.smithi107.stdout:1/889: write d1/d46/d6a/d4e/fcf [4284709,52455] 0 2024-03-04T13:43:22.011 INFO:tasks.workunit.client.0.smithi107.stdout:6/853: unlink df/d17/d6b/d9b/l32 0 2024-03-04T13:43:22.016 INFO:tasks.workunit.client.0.smithi107.stdout:4/871: creat d0/d9/d18/f116 x:0 0 0 2024-03-04T13:43:22.017 INFO:tasks.workunit.client.0.smithi107.stdout:1/890: rmdir d1/d46/d6a/d4e/d60/d83 39 2024-03-04T13:43:22.030 INFO:tasks.workunit.client.0.smithi107.stdout:2/832: truncate da/df/d73/d7a/d9a/fc9 626012 0 2024-03-04T13:43:22.030 INFO:tasks.workunit.client.0.smithi107.stdout:4/872: write d0/d9/d1e/d70/f10f [282050,15446] 0 2024-03-04T13:43:22.037 INFO:tasks.workunit.client.0.smithi107.stdout:5/971: truncate d5/f32 7758880 0 2024-03-04T13:43:22.044 INFO:tasks.workunit.client.0.smithi107.stdout:0/885: dread da/f3c [0,4194304] 0 2024-03-04T13:43:22.050 INFO:tasks.workunit.client.0.smithi107.stdout:9/785: dwrite d4/da/f4a [0,4194304] 0 2024-03-04T13:43:22.052 INFO:tasks.workunit.client.0.smithi107.stdout:3/895: dwrite d1/d1e/d26/d9e/fec [0,4194304] 0 2024-03-04T13:43:22.052 INFO:tasks.workunit.client.0.smithi107.stdout:0/886: chown da/dc/dd/d1e/dbf 1157 1 2024-03-04T13:43:22.057 INFO:tasks.workunit.client.0.smithi107.stdout:8/864: dread d6/d16/d1f/f5c [4194304,4194304] 0 2024-03-04T13:43:22.065 INFO:tasks.workunit.client.0.smithi107.stdout:4/873: write d0/d9/d10/d80/db8/d97/fc6 [344643,51193] 0 2024-03-04T13:43:22.069 INFO:tasks.workunit.client.0.smithi107.stdout:1/891: truncate d1/d10/d16/d47/d7a/dda/ff7 4570620 0 2024-03-04T13:43:22.069 INFO:tasks.workunit.client.0.smithi107.stdout:4/874: write d0/d9/fd1 [1046813,87989] 0 2024-03-04T13:43:22.070 INFO:tasks.workunit.client.0.smithi107.stdout:5/972: creat d5/d8/d14/d2d/d92/dd8/f153 x:0 0 0 2024-03-04T13:43:22.070 INFO:tasks.workunit.client.0.smithi107.stdout:0/887: truncate da/dc/fc4 434914 0 2024-03-04T13:43:22.073 INFO:tasks.workunit.client.0.smithi107.stdout:1/892: chown d1/d46/d6a/d65/d106/d61/ddb/le7 1729 1 2024-03-04T13:43:22.078 INFO:tasks.workunit.client.0.smithi107.stdout:7/819: dread d6/f45 [0,4194304] 0 2024-03-04T13:43:22.078 INFO:tasks.workunit.client.0.smithi107.stdout:6/854: dread df/d2e/d36/d77/d106/d7e/d86/f97 [0,4194304] 0 2024-03-04T13:43:22.081 INFO:tasks.workunit.client.0.smithi107.stdout:7/820: write d6/d17/f6d [8087228,70354] 0 2024-03-04T13:43:22.082 INFO:tasks.workunit.client.0.smithi107.stdout:2/833: rename da/d34/l3b to da/d44/d8b/d98/dc3/l11b 0 2024-03-04T13:43:22.084 INFO:tasks.workunit.client.0.smithi107.stdout:7/821: chown d6/d13/d16/d19/d38/d37/f9e 109894 1 2024-03-04T13:43:22.084 INFO:tasks.workunit.client.0.smithi107.stdout:2/834: chown da/f30 665385 1 2024-03-04T13:43:22.088 INFO:tasks.workunit.client.0.smithi107.stdout:2/835: dread da/d34/fc8 [0,4194304] 0 2024-03-04T13:43:22.091 INFO:tasks.workunit.client.0.smithi107.stdout:7/822: read d6/d1d/d64/d9a/da9/d71/db9/dca/d3d/dc6/ff1 [691530,15646] 0 2024-03-04T13:43:22.091 INFO:tasks.workunit.client.0.smithi107.stdout:8/865: getdents d6/da/d25/d113 0 2024-03-04T13:43:22.099 INFO:tasks.workunit.client.0.smithi107.stdout:6/855: mkdir df/d17/d6b/da8/d113 0 2024-03-04T13:43:22.099 INFO:tasks.workunit.client.0.smithi107.stdout:1/893: rename d1/d46/d6a/d4e/daa/dc9/d33/d5d/d4c/dea/d59 to d1/d46/d6a/d65/d106/d61/ddb/de6/d124 0 2024-03-04T13:43:22.101 INFO:tasks.workunit.client.0.smithi107.stdout:4/875: symlink d0/l117 0 2024-03-04T13:43:22.102 INFO:tasks.workunit.client.0.smithi107.stdout:1/894: chown d1/d46/fd0 986 1 2024-03-04T13:43:22.106 INFO:tasks.workunit.client.0.smithi107.stdout:2/836: chown da/df/d73/d7a/d9a/fc9 15190493 1 2024-03-04T13:43:22.108 INFO:tasks.workunit.client.0.smithi107.stdout:9/786: write d4/da/d11/d20/d2f/d9c/daa/fc2 [1363485,72074] 0 2024-03-04T13:43:22.109 INFO:tasks.workunit.client.0.smithi107.stdout:5/973: dwrite d5/d8/d14/f105 [4194304,4194304] 0 2024-03-04T13:43:22.111 INFO:tasks.workunit.client.0.smithi107.stdout:8/866: truncate d6/d16/d69/deb/d3e/d67/ff6 1111083 0 2024-03-04T13:43:22.113 INFO:tasks.workunit.client.0.smithi107.stdout:7/823: dwrite d6/d13/d14/f6b [0,4194304] 0 2024-03-04T13:43:22.118 INFO:tasks.workunit.client.0.smithi107.stdout:2/837: write da/d44/d8b/d9d/fd6 [1495048,38226] 0 2024-03-04T13:43:22.131 INFO:tasks.workunit.client.0.smithi107.stdout:9/787: fsync d4/d32/fa8 0 2024-03-04T13:43:22.132 INFO:tasks.workunit.client.0.smithi107.stdout:6/856: fdatasync df/d17/f28 0 2024-03-04T13:43:22.136 INFO:tasks.workunit.client.0.smithi107.stdout:3/896: dwrite d1/de/d18/d33/d48/d71/f11a [0,4194304] 0 2024-03-04T13:43:22.136 INFO:tasks.workunit.client.0.smithi107.stdout:8/867: creat d6/d16/d69/deb/d3e/d96/dd8/f114 x:0 0 0 2024-03-04T13:43:22.138 INFO:tasks.workunit.client.0.smithi107.stdout:0/888: dwrite da/dc/dd/d3e/d26/f58 [0,4194304] 0 2024-03-04T13:43:22.142 INFO:tasks.workunit.client.0.smithi107.stdout:1/895: link d1/d46/d9f/ffc d1/d38/f125 0 2024-03-04T13:43:22.144 INFO:tasks.workunit.client.0.smithi107.stdout:4/876: dwrite d0/d9/d18/faa [0,4194304] 0 2024-03-04T13:43:22.146 INFO:tasks.workunit.client.0.smithi107.stdout:8/868: write d6/d16/d1f/d4c/da5/fee [584661,69018] 0 2024-03-04T13:43:22.152 INFO:tasks.workunit.client.0.smithi107.stdout:5/974: rename d5/d79/d89/da3/d6a/f8e to d5/d8/d14/d2d/d92/dd8/f154 0 2024-03-04T13:43:22.152 INFO:tasks.workunit.client.0.smithi107.stdout:9/788: truncate d4/da/d11/d4c/d4d/d72/d7b/f81 996372 0 2024-03-04T13:43:22.153 INFO:tasks.workunit.client.0.smithi107.stdout:7/824: truncate d6/d13/d16/d1c/dc2/fc7 794749 0 2024-03-04T13:43:22.154 INFO:tasks.workunit.client.0.smithi107.stdout:2/838: read da/f26 [1562713,65473] 0 2024-03-04T13:43:22.159 INFO:tasks.workunit.client.0.smithi107.stdout:6/857: chown df/d17/c55 11525906 1 2024-03-04T13:43:22.160 INFO:tasks.workunit.client.0.smithi107.stdout:2/839: fdatasync da/df/d27/d10b/ff7 0 2024-03-04T13:43:22.162 INFO:tasks.workunit.client.0.smithi107.stdout:8/869: stat d6/da/d25/dcb 0 2024-03-04T13:43:22.166 INFO:tasks.workunit.client.0.smithi107.stdout:6/858: write df/d38/d4f/fcf [8654460,14588] 0 2024-03-04T13:43:22.167 INFO:tasks.workunit.client.0.smithi107.stdout:2/840: write da/df/d73/dd1/f102 [529233,104753] 0 2024-03-04T13:43:22.178 INFO:tasks.workunit.client.0.smithi107.stdout:9/789: creat d4/da/d11/d20/d2f/d9c/d92/f109 x:0 0 0 2024-03-04T13:43:22.178 INFO:tasks.workunit.client.0.smithi107.stdout:2/841: truncate da/d29/d7c/dbd/fda 732285 0 2024-03-04T13:43:22.183 INFO:tasks.workunit.client.0.smithi107.stdout:8/870: mknod d6/da/d25/dd1/de3/c115 0 2024-03-04T13:43:22.184 INFO:tasks.workunit.client.0.smithi107.stdout:4/877: read d0/d9/d18/df1/dc1/ff2 [606075,101919] 0 2024-03-04T13:43:22.186 INFO:tasks.workunit.client.0.smithi107.stdout:6/859: read df/d38/f5f [1941250,57441] 0 2024-03-04T13:43:22.186 INFO:tasks.workunit.client.0.smithi107.stdout:7/825: truncate d6/d1d/d36/f5b 823365 0 2024-03-04T13:43:22.190 INFO:tasks.workunit.client.0.smithi107.stdout:1/896: dwrite d1/d46/d6a/d4e/daa/dc9/d33/d5d/f97 [8388608,4194304] 0 2024-03-04T13:43:22.196 INFO:tasks.workunit.client.0.smithi107.stdout:0/889: getdents da/dc/dd/d3e/d26 0 2024-03-04T13:43:22.197 INFO:tasks.workunit.client.0.smithi107.stdout:8/871: read d6/f5b [3081207,7454] 0 2024-03-04T13:43:22.202 INFO:tasks.workunit.client.0.smithi107.stdout:0/890: stat da/dc/dd/d3e/d26/d69/d6b/ff1 0 2024-03-04T13:43:22.203 INFO:tasks.workunit.client.0.smithi107.stdout:9/790: mknod d4/da/d11/d20/d23/d7e/d84/d107/c10a 0 2024-03-04T13:43:22.203 INFO:tasks.workunit.client.0.smithi107.stdout:5/975: dwrite d5/dc/f55 [4194304,4194304] 0 2024-03-04T13:43:22.208 INFO:tasks.workunit.client.0.smithi107.stdout:6/860: mkdir df/d2e/d36/d5b/d7d/d112/d114 0 2024-03-04T13:43:22.216 INFO:tasks.workunit.client.0.smithi107.stdout:3/897: dread d1/de/d18/d33/d6b/fb1 [0,4194304] 0 2024-03-04T13:43:22.220 INFO:tasks.workunit.client.0.smithi107.stdout:3/898: stat d1/d2d/f44 0 2024-03-04T13:43:22.223 INFO:tasks.workunit.client.0.smithi107.stdout:4/878: dwrite d0/d9/de/d14/ffb [4194304,4194304] 0 2024-03-04T13:43:22.225 INFO:tasks.workunit.client.0.smithi107.stdout:6/861: fdatasync df/d17/d5c/fb0 0 2024-03-04T13:43:22.227 INFO:tasks.workunit.client.0.smithi107.stdout:8/872: dread - d6/d16/d1f/d4c/da4/daa/fca zero size 2024-03-04T13:43:22.234 INFO:tasks.workunit.client.0.smithi107.stdout:0/891: read da/f59 [2919263,34421] 0 2024-03-04T13:43:22.237 INFO:tasks.workunit.client.0.smithi107.stdout:4/879: mknod d0/d9/d10/d80/db8/d21/c118 0 2024-03-04T13:43:22.238 INFO:tasks.workunit.client.0.smithi107.stdout:7/826: link d6/d13/d16/d19/d38/d6e/cb0 d6/d13/d16/d19/d39/c110 0 2024-03-04T13:43:22.238 INFO:tasks.workunit.client.0.smithi107.stdout:5/976: creat d5/d8/d2a/dd6/dea/f155 x:0 0 0 2024-03-04T13:43:22.238 INFO:tasks.workunit.client.0.smithi107.stdout:6/862: chown df/d2e/d36/fbc 14 1 2024-03-04T13:43:22.241 INFO:tasks.workunit.client.0.smithi107.stdout:8/873: rename d6/dcc/cf0 to d6/d16/d1f/d63/d93/c116 0 2024-03-04T13:43:22.247 INFO:tasks.workunit.client.0.smithi107.stdout:8/874: fsync d6/da/fad 0 2024-03-04T13:43:22.250 INFO:tasks.workunit.client.0.smithi107.stdout:7/827: symlink d6/d1d/dd9/l111 0 2024-03-04T13:43:22.259 INFO:tasks.workunit.client.0.smithi107.stdout:5/977: write d5/d8/d14/d2d/fda [108191,130384] 0 2024-03-04T13:43:22.260 INFO:tasks.workunit.client.0.smithi107.stdout:0/892: getdents da/dc/dd/d3e 0 2024-03-04T13:43:22.266 INFO:tasks.workunit.client.0.smithi107.stdout:6/863: getdents df/d1b 0 2024-03-04T13:43:22.266 INFO:tasks.workunit.client.0.smithi107.stdout:8/875: truncate d6/d16/d1f/d4c/f61 360794 0 2024-03-04T13:43:22.266 INFO:tasks.workunit.client.0.smithi107.stdout:7/828: truncate d6/d13/d16/d1c/dc2/ff9 407616 0 2024-03-04T13:43:22.268 INFO:tasks.workunit.client.0.smithi107.stdout:9/791: dwrite d4/da/d11/d20/d2f/d9c/fb4 [0,4194304] 0 2024-03-04T13:43:22.279 INFO:tasks.workunit.client.0.smithi107.stdout:5/978: creat d5/d79/d89/da3/d68/f156 x:0 0 0 2024-03-04T13:43:22.280 INFO:tasks.workunit.client.0.smithi107.stdout:4/880: dwrite d0/fef [0,4194304] 0 2024-03-04T13:43:22.288 INFO:tasks.workunit.client.0.smithi107.stdout:2/842: dread da/d44/d8b/d9d/df8/f7e [0,4194304] 0 2024-03-04T13:43:22.288 INFO:tasks.workunit.client.0.smithi107.stdout:3/899: dread d1/de/d18/d33/d48/d93/fdd [0,4194304] 0 2024-03-04T13:43:22.291 INFO:tasks.workunit.client.0.smithi107.stdout:6/864: mknod df/d17/d24/d40/d63/c115 0 2024-03-04T13:43:22.294 INFO:tasks.workunit.client.0.smithi107.stdout:8/876: mknod d6/da/d25/c117 0 2024-03-04T13:43:22.301 INFO:tasks.workunit.client.0.smithi107.stdout:2/843: mkdir da/df/d73/daa/ddc/d11c 0 2024-03-04T13:43:22.303 INFO:tasks.workunit.client.0.smithi107.stdout:8/877: mknod d6/dcc/c118 0 2024-03-04T13:43:22.306 INFO:tasks.workunit.client.0.smithi107.stdout:3/900: truncate d1/de/d11/f95 709615 0 2024-03-04T13:43:22.308 INFO:tasks.workunit.client.0.smithi107.stdout:9/792: dread d4/da/d11/d20/d23/dcc/fee [0,4194304] 0 2024-03-04T13:43:22.308 INFO:tasks.workunit.client.0.smithi107.stdout:2/844: unlink da/d29/cb9 0 2024-03-04T13:43:22.313 INFO:tasks.workunit.client.0.smithi107.stdout:5/979: link d5/d79/d89/da3/d68/l115 d5/d8/d14/d2d/d92/dc1/l157 0 2024-03-04T13:43:22.319 INFO:tasks.workunit.client.0.smithi107.stdout:0/893: dwrite da/dc/dd/d3e/f7a [0,4194304] 0 2024-03-04T13:43:22.319 INFO:tasks.workunit.client.0.smithi107.stdout:8/878: chown d6/da/d25/la0 8077105 1 2024-03-04T13:43:22.323 INFO:tasks.workunit.client.0.smithi107.stdout:3/901: fsync d1/de/d18/d33/d48/d71/fa9 0 2024-03-04T13:43:22.324 INFO:tasks.workunit.client.0.smithi107.stdout:5/980: chown d5/d8/d14/d121/c131 1077 1 2024-03-04T13:43:22.326 INFO:tasks.workunit.client.0.smithi107.stdout:4/881: dwrite d0/d9/de/d14/ffb [0,4194304] 0 2024-03-04T13:43:22.331 INFO:tasks.workunit.client.0.smithi107.stdout:7/829: dwrite d6/d13/d16/d19/fa1 [0,4194304] 0 2024-03-04T13:43:22.334 INFO:tasks.workunit.client.0.smithi107.stdout:5/981: chown d5/f32 1470575106 1 2024-03-04T13:43:22.335 INFO:tasks.workunit.client.0.smithi107.stdout:8/879: mkdir d6/d16/d1f/d119 0 2024-03-04T13:43:22.336 INFO:tasks.workunit.client.0.smithi107.stdout:3/902: rename d1/de/d18/c65 to d1/de/d18/d33/d4d/d4f/dc3/c11f 0 2024-03-04T13:43:22.336 INFO:tasks.workunit.client.0.smithi107.stdout:5/982: write d5/d2b/d4b/f91 [2176329,52631] 0 2024-03-04T13:43:22.336 INFO:tasks.workunit.client.0.smithi107.stdout:2/845: write da/df/d27/d10b/d5f/f8c [265475,43981] 0 2024-03-04T13:43:22.343 INFO:tasks.workunit.client.0.smithi107.stdout:0/894: getdents da/dc/dd/dc9 0 2024-03-04T13:43:22.345 INFO:tasks.workunit.client.0.smithi107.stdout:3/903: symlink d1/de/d3f/l120 0 2024-03-04T13:43:22.350 INFO:tasks.workunit.client.0.smithi107.stdout:2/846: mknod da/df/d3e/dfe/c11d 0 2024-03-04T13:43:22.353 INFO:tasks.workunit.client.0.smithi107.stdout:0/895: chown da/dc/dd/dc9/de5/l8d 205 1 2024-03-04T13:43:22.354 INFO:tasks.workunit.client.0.smithi107.stdout:3/904: creat d1/d1e/d26/dfd/f121 x:0 0 0 2024-03-04T13:43:22.355 INFO:tasks.workunit.client.0.smithi107.stdout:5/983: dread d5/d8/d14/d2d/df1/f93 [0,4194304] 0 2024-03-04T13:43:22.357 INFO:tasks.workunit.client.0.smithi107.stdout:3/905: readlink d1/de/d11/dd5/dab/dc5/lfa 0 2024-03-04T13:43:22.359 INFO:tasks.workunit.client.0.smithi107.stdout:8/880: write d6/da/d25/f3f [986069,27341] 0 2024-03-04T13:43:22.361 INFO:tasks.workunit.client.0.smithi107.stdout:2/847: symlink da/df/d3e/dfe/l11e 0 2024-03-04T13:43:22.363 INFO:tasks.workunit.client.0.smithi107.stdout:5/984: unlink d5/d8/d14/d2d/d92/dd8/f114 0 2024-03-04T13:43:22.363 INFO:tasks.workunit.client.0.smithi107.stdout:7/830: rename d6/d1d/d64/d9a/da9/d71/db9/dca/f42 to d6/d1d/d64/d9a/da9/d71/db9/dca/d3d/dc6/dfb/f112 0 2024-03-04T13:43:22.367 INFO:tasks.workunit.client.0.smithi107.stdout:8/881: stat d6/d16/d1f/d4c/da5 0 2024-03-04T13:43:22.367 INFO:tasks.workunit.client.0.smithi107.stdout:3/906: mknod d1/de/c122 0 2024-03-04T13:43:22.373 INFO:tasks.workunit.client.0.smithi107.stdout:7/831: chown d6/d13/cce 44 1 2024-03-04T13:43:22.375 INFO:tasks.workunit.client.0.smithi107.stdout:8/882: creat d6/dcc/f11a x:0 0 0 2024-03-04T13:43:22.381 INFO:tasks.workunit.client.0.smithi107.stdout:2/848: write da/d29/d7c/fe1 [518305,83038] 0 2024-03-04T13:43:22.382 INFO:tasks.workunit.client.0.smithi107.stdout:5/985: stat d5/d79/d89/da3/d68/l115 0 2024-03-04T13:43:22.384 INFO:tasks.workunit.client.0.smithi107.stdout:3/907: dread - d1/d1e/dae/df2/f119 zero size 2024-03-04T13:43:22.384 INFO:tasks.workunit.client.0.smithi107.stdout:0/896: dwrite da/dc/dd/f24 [0,4194304] 0 2024-03-04T13:43:22.389 INFO:tasks.workunit.client.0.smithi107.stdout:8/883: mkdir d6/dcc/dfc/d11b 0 2024-03-04T13:43:22.392 INFO:tasks.workunit.client.0.smithi107.stdout:3/908: symlink d1/d1e/l123 0 2024-03-04T13:43:22.397 INFO:tasks.workunit.client.0.smithi107.stdout:9/793: dread d4/ff [0,4194304] 0 2024-03-04T13:43:22.402 INFO:tasks.workunit.client.0.smithi107.stdout:8/884: creat d6/d16/d69/f11c x:0 0 0 2024-03-04T13:43:22.403 INFO:tasks.workunit.client.0.smithi107.stdout:7/832: dwrite d6/d13/d16/d19/fe3 [0,4194304] 0 2024-03-04T13:43:22.403 INFO:tasks.workunit.client.0.smithi107.stdout:9/794: write d4/da/d11/d20/d2f/d9c/daa/fc2 [904717,16814] 0 2024-03-04T13:43:22.405 INFO:tasks.workunit.client.0.smithi107.stdout:3/909: readlink d1/de/d18/d33/d48/d71/l58 0 2024-03-04T13:43:22.406 INFO:tasks.workunit.client.0.smithi107.stdout:7/833: chown d6/d13/d16/d1c/f79 1535722 1 2024-03-04T13:43:22.408 INFO:tasks.workunit.client.0.smithi107.stdout:2/849: link da/d44/d8b/d9d/df8/d5c/d6a/lb7 da/d29/d7c/dbd/d117/l11f 0 2024-03-04T13:43:22.411 INFO:tasks.workunit.client.0.smithi107.stdout:9/795: truncate d4/da/f1b 750008 0 2024-03-04T13:43:22.412 INFO:tasks.workunit.client.0.smithi107.stdout:0/897: rename da/dc/dd/fe7 to da/dc/dd/d3e/d26/d88/f130 0 2024-03-04T13:43:22.416 INFO:tasks.workunit.client.0.smithi107.stdout:7/834: write d6/d13/d16/d19/d38/d37/f9e [2345561,127670] 0 2024-03-04T13:43:22.421 INFO:tasks.workunit.client.0.smithi107.stdout:8/885: write d6/d16/fb1 [159544,19166] 0 2024-03-04T13:43:22.423 INFO:tasks.workunit.client.0.smithi107.stdout:9/796: read d4/f21 [287598,122974] 0 2024-03-04T13:43:22.424 INFO:tasks.workunit.client.0.smithi107.stdout:8/886: chown d6/d16/d69/deb/d3e/d96/dd8 1079 1 2024-03-04T13:43:22.426 INFO:tasks.workunit.client.0.smithi107.stdout:3/910: rmdir d1/de/d18/d33/d4d/d4f/d69/d110/d115 0 2024-03-04T13:43:22.428 INFO:tasks.workunit.client.0.smithi107.stdout:0/898: getdents da/dc/dd/d3e/d26/d69/d6b/d115 0 2024-03-04T13:43:22.429 INFO:tasks.workunit.client.0.smithi107.stdout:8/887: mknod d6/d16/d1f/dbd/c11d 0 2024-03-04T13:43:22.431 INFO:tasks.workunit.client.0.smithi107.stdout:0/899: readlink da/l36 0 2024-03-04T13:43:22.433 INFO:tasks.workunit.client.0.smithi107.stdout:5/986: dread d5/f42 [0,4194304] 0 2024-03-04T13:43:22.433 INFO:tasks.workunit.client.0.smithi107.stdout:3/911: write d1/de/f68 [2807137,47045] 0 2024-03-04T13:43:22.435 INFO:tasks.workunit.client.0.smithi107.stdout:7/835: mknod d6/d1d/d64/d9a/da9/d71/db9/dca/d3d/dc6/c113 0 2024-03-04T13:43:22.441 INFO:tasks.workunit.client.0.smithi107.stdout:8/888: mknod d6/d16/d69/deb/d3e/d96/c11e 0 2024-03-04T13:43:22.443 INFO:tasks.workunit.client.0.smithi107.stdout:2/850: dwrite da/d34/f41 [4194304,4194304] 0 2024-03-04T13:43:22.447 INFO:tasks.workunit.client.0.smithi107.stdout:7/836: chown d6/d1d/d64/d9a/da9/fda 99194690 1 2024-03-04T13:43:22.448 INFO:tasks.workunit.client.0.smithi107.stdout:0/900: write da/dc/dd/d3e/d26/fe3 [400384,29923] 0 2024-03-04T13:43:22.454 INFO:tasks.workunit.client.0.smithi107.stdout:7/837: mkdir d6/d1d/dd9/d114 0 2024-03-04T13:43:22.458 INFO:tasks.workunit.client.0.smithi107.stdout:0/901: chown da/dc/dd/d3e/d26/d69/d6b/cc0 20 1 2024-03-04T13:43:22.461 INFO:tasks.workunit.client.0.smithi107.stdout:7/838: rmdir d6/d13/dbc 39 2024-03-04T13:43:22.462 INFO:tasks.workunit.client.0.smithi107.stdout:2/851: dread da/df/d27/d10b/ff7 [0,4194304] 0 2024-03-04T13:43:22.464 INFO:tasks.workunit.client.0.smithi107.stdout:5/987: dwrite d5/d79/d89/f129 [0,4194304] 0 2024-03-04T13:43:22.470 INFO:tasks.workunit.client.0.smithi107.stdout:3/912: truncate d1/de/d18/d33/d48/d71/d4c/d54/fa7 2492860 0 2024-03-04T13:43:22.473 INFO:tasks.workunit.client.0.smithi107.stdout:5/988: unlink d5/d66/d96/caf 0 2024-03-04T13:43:22.477 INFO:tasks.workunit.client.0.smithi107.stdout:9/797: dread d4/da/d11/d4c/d51/f5c [0,4194304] 0 2024-03-04T13:43:22.477 INFO:tasks.workunit.client.0.smithi107.stdout:7/839: mkdir d6/d1d/d64/d9a/da9/d71/d115 0 2024-03-04T13:43:22.481 INFO:tasks.workunit.client.0.smithi107.stdout:8/889: dwrite d6/da/f70 [0,4194304] 0 2024-03-04T13:43:22.482 INFO:tasks.workunit.client.0.smithi107.stdout:2/852: truncate da/df/d3e/f40 981949 0 2024-03-04T13:43:22.491 INFO:tasks.workunit.client.0.smithi107.stdout:9/798: symlink d4/da/d11/d20/d39/l10b 0 2024-03-04T13:43:22.493 INFO:tasks.workunit.client.0.smithi107.stdout:0/902: dwrite da/dc/dd/d3e/d26/d88/dd2/d73/d78/fcf [0,4194304] 0 2024-03-04T13:43:22.494 INFO:tasks.workunit.client.0.smithi107.stdout:3/913: mkdir d1/d1e/d26/d9e/d10f/d124 0 2024-03-04T13:43:22.499 INFO:tasks.workunit.client.0.smithi107.stdout:8/890: truncate d6/da/d25/f4b 3826986 0 2024-03-04T13:43:22.500 INFO:tasks.workunit.client.0.smithi107.stdout:5/989: truncate d5/d79/d89/da3/d69/f72 3758297 0 2024-03-04T13:43:22.500 INFO:tasks.workunit.client.0.smithi107.stdout:9/799: rmdir d4/da/d11/d20/d23 39 2024-03-04T13:43:22.504 INFO:tasks.workunit.client.0.smithi107.stdout:3/914: fsync d1/de/d18/d33/d4d/d4f/d69/f7b 0 2024-03-04T13:43:22.507 INFO:tasks.workunit.client.0.smithi107.stdout:3/915: chown d1/de/c9a 6425020 1 2024-03-04T13:43:22.507 INFO:tasks.workunit.client.0.smithi107.stdout:8/891: unlink d6/f45 0 2024-03-04T13:43:22.508 INFO:tasks.workunit.client.0.smithi107.stdout:9/800: mknod d4/d32/c10c 0 2024-03-04T13:43:22.510 INFO:tasks.workunit.client.0.smithi107.stdout:7/840: write d6/d13/d14/f5d [2345492,103505] 0 2024-03-04T13:43:22.513 INFO:tasks.workunit.client.0.smithi107.stdout:0/903: dwrite da/dc/dd/d3e/d26/d69/de9/fed [0,4194304] 0 2024-03-04T13:43:22.513 INFO:tasks.workunit.client.0.smithi107.stdout:5/990: creat d5/dc/de4/f158 x:0 0 0 2024-03-04T13:43:22.521 INFO:tasks.workunit.client.0.smithi107.stdout:2/853: getdents da/df/d27/d10b/d5f 0 2024-03-04T13:43:22.527 INFO:tasks.workunit.client.0.smithi107.stdout:9/801: creat d4/da/d11/f10d x:0 0 0 2024-03-04T13:43:22.527 INFO:tasks.workunit.client.0.smithi107.stdout:7/841: mkdir d6/d116 0 2024-03-04T13:43:22.529 INFO:tasks.workunit.client.0.smithi107.stdout:5/991: creat d5/d2b/f159 x:0 0 0 2024-03-04T13:43:22.531 INFO:tasks.workunit.client.0.smithi107.stdout:1/897: sync 2024-03-04T13:43:22.532 INFO:tasks.workunit.client.0.smithi107.stdout:5/992: write d5/d66/f78 [2180204,52254] 0 2024-03-04T13:43:22.534 INFO:tasks.workunit.client.0.smithi107.stdout:7/842: write d6/d13/d16/d19/fe3 [4828822,110405] 0 2024-03-04T13:43:22.534 INFO:tasks.workunit.client.0.smithi107.stdout:2/854: mknod da/d44/d8b/d9d/dd0/c120 0 2024-03-04T13:43:22.539 INFO:tasks.workunit.client.0.smithi107.stdout:2/855: chown da/df/d73/db3 0 1 2024-03-04T13:43:22.542 INFO:tasks.workunit.client.0.smithi107.stdout:8/892: dwrite d6/d16/d69/deb/d7c/f92 [0,4194304] 0 2024-03-04T13:43:22.544 INFO:tasks.workunit.client.0.smithi107.stdout:2/856: read da/d44/d8b/d9d/df8/d5c/f91 [6337152,104897] 0 2024-03-04T13:43:22.552 INFO:tasks.workunit.client.0.smithi107.stdout:7/843: symlink d6/d13/d16/d1c/dc2/l117 0 2024-03-04T13:43:22.554 INFO:tasks.workunit.client.0.smithi107.stdout:7/844: fdatasync d6/d13/d14/f6b 0 2024-03-04T13:43:22.555 INFO:tasks.workunit.client.0.smithi107.stdout:0/904: creat da/f131 x:0 0 0 2024-03-04T13:43:22.557 INFO:tasks.workunit.client.0.smithi107.stdout:5/993: mkdir d5/d8/d14/d2d/d92/dc1/d133/d15a 0 2024-03-04T13:43:22.560 INFO:tasks.workunit.client.0.smithi107.stdout:1/898: write d1/d10/d16/d1b/fe3 [1218722,66963] 0 2024-03-04T13:43:22.561 INFO:tasks.workunit.client.0.smithi107.stdout:9/802: read d4/da/d11/d20/d2f/d9c/fb4 [2332222,101251] 0 2024-03-04T13:43:22.564 INFO:tasks.workunit.client.0.smithi107.stdout:2/857: mkdir da/df/d73/db3/d121 0 2024-03-04T13:43:22.564 INFO:tasks.workunit.client.0.smithi107.stdout:8/893: dread d6/d16/d1f/d4c/da5/fc9 [0,4194304] 0 2024-03-04T13:43:22.565 INFO:tasks.workunit.client.0.smithi107.stdout:9/803: chown d4/da/d11/d20/d64 380207 1 2024-03-04T13:43:22.570 INFO:tasks.workunit.client.0.smithi107.stdout:5/994: mkdir d5/d79/d89/da3/d69/d15b 0 2024-03-04T13:43:22.571 INFO:tasks.workunit.client.0.smithi107.stdout:7/845: mknod d6/d1d/d64/d9a/da9/d71/db9/dca/d3d/dc6/c118 0 2024-03-04T13:43:22.575 INFO:tasks.workunit.client.0.smithi107.stdout:1/899: mknod d1/d10/d16/d47/d7a/dda/dec/c126 0 2024-03-04T13:43:22.575 INFO:tasks.workunit.client.0.smithi107.stdout:2/858: mkdir da/d39/d122 0 2024-03-04T13:43:22.577 INFO:tasks.workunit.client.0.smithi107.stdout:3/916: link d1/de/d11/f3e d1/de/d18/d33/d48/d71/d4c/f125 0 2024-03-04T13:43:22.579 INFO:tasks.workunit.client.0.smithi107.stdout:3/917: chown d1/de/dad/dbb/ffc 15 1 2024-03-04T13:43:22.580 INFO:tasks.workunit.client.0.smithi107.stdout:7/846: fdatasync d6/d13/d16/f7b 0 2024-03-04T13:43:22.581 INFO:tasks.workunit.client.0.smithi107.stdout:8/894: dwrite d6/da/f11 [0,4194304] 0 2024-03-04T13:43:22.584 INFO:tasks.workunit.client.0.smithi107.stdout:2/859: fdatasync da/df/fa2 0 2024-03-04T13:43:22.587 INFO:tasks.workunit.client.0.smithi107.stdout:5/995: rename d5/d79/d89/lb6 to d5/d8/d14/d121/l15c 0 2024-03-04T13:43:22.590 INFO:tasks.workunit.client.0.smithi107.stdout:3/918: truncate d1/de/d3f/f89 412201 0 2024-03-04T13:43:22.591 INFO:tasks.workunit.client.0.smithi107.stdout:8/895: rmdir d6/dcc 39 2024-03-04T13:43:22.593 INFO:tasks.workunit.client.0.smithi107.stdout:8/896: chown d6/da/d25/d2a/f75 30634696 1 2024-03-04T13:43:22.598 INFO:tasks.workunit.client.0.smithi107.stdout:0/905: link f0 da/dc/dd/dc9/de5/d107/f132 0 2024-03-04T13:43:22.601 INFO:tasks.workunit.client.0.smithi107.stdout:2/860: mkdir da/d44/d8b/d98/dc3/d123 0 2024-03-04T13:43:22.604 INFO:tasks.workunit.client.0.smithi107.stdout:0/906: truncate da/dc/dd/d3e/d26/d88/dd2/d73/d78/f121 862627 0 2024-03-04T13:43:22.605 INFO:tasks.workunit.client.0.smithi107.stdout:3/919: write d1/d2d/fca [1372548,112222] 0 2024-03-04T13:43:22.608 INFO:tasks.workunit.client.0.smithi107.stdout:7/847: symlink d6/d13/d16/d19/d38/d2c/dff/l119 0 2024-03-04T13:43:22.609 INFO:tasks.workunit.client.0.smithi107.stdout:1/900: link d1/d38/fe4 d1/d46/d6a/d65/db1/f127 0 2024-03-04T13:43:22.612 INFO:tasks.workunit.client.0.smithi107.stdout:0/907: mknod da/dc/dd/ddb/c133 0 2024-03-04T13:43:22.617 INFO:tasks.workunit.client.0.smithi107.stdout:7/848: mknod d6/d13/d16/d19/d39/c11a 0 2024-03-04T13:43:22.618 INFO:tasks.workunit.client.0.smithi107.stdout:7/849: stat d6/d13/d16/f7b 0 2024-03-04T13:43:22.619 INFO:tasks.workunit.client.0.smithi107.stdout:1/901: mkdir d1/d10/d16/d47/d7a/d128 0 2024-03-04T13:43:22.624 INFO:tasks.workunit.client.0.smithi107.stdout:6/865: sync 2024-03-04T13:43:22.624 INFO:tasks.workunit.client.0.smithi107.stdout:4/882: sync 2024-03-04T13:43:22.625 INFO:tasks.workunit.client.0.smithi107.stdout:5/996: creat d5/d8/da/f15d x:0 0 0 2024-03-04T13:43:22.626 INFO:tasks.workunit.client.0.smithi107.stdout:5/997: readlink d5/d79/d89/da3/d6a/l106 0 2024-03-04T13:43:22.628 INFO:tasks.workunit.client.0.smithi107.stdout:3/920: mknod d1/de/d18/d33/c126 0 2024-03-04T13:43:22.632 INFO:tasks.workunit.client.0.smithi107.stdout:9/804: rename d4/da/d11/d20/d2f/d9c/cd3 to d4/da/d11/d4c/c10e 0 2024-03-04T13:43:22.634 INFO:tasks.workunit.client.0.smithi107.stdout:2/861: creat da/df/d73/f124 x:0 0 0 2024-03-04T13:43:22.636 INFO:tasks.workunit.client.0.smithi107.stdout:4/883: creat d0/d9/d10/d6f/f119 x:0 0 0 2024-03-04T13:43:22.636 INFO:tasks.workunit.client.0.smithi107.stdout:2/862: readlink da/d39/lca 0 2024-03-04T13:43:22.637 INFO:tasks.workunit.client.0.smithi107.stdout:5/998: mknod d5/d66/c15e 0 2024-03-04T13:43:22.638 INFO:tasks.workunit.client.0.smithi107.stdout:1/902: dread d1/f9a [0,4194304] 0 2024-03-04T13:43:22.640 INFO:tasks.workunit.client.0.smithi107.stdout:5/999: stat d5/d8/d14/d2d 0 2024-03-04T13:43:22.640 INFO:tasks.workunit.client.0.smithi107.stdout:4/884: chown d0/d9/d10/d80/db8/d97/fb1 76 1 2024-03-04T13:43:22.642 INFO:tasks.workunit.client.0.smithi107.stdout:6/866: mkdir df/d2e/d36/d5b/d7d/d112/d114/d116 0 2024-03-04T13:43:22.643 INFO:tasks.workunit.client.0.smithi107.stdout:0/908: link da/l36 da/dad/l134 0 2024-03-04T13:43:22.644 INFO:tasks.workunit.client.0.smithi107.stdout:2/863: mknod da/d34/c125 0 2024-03-04T13:43:22.646 INFO:tasks.workunit.client.0.smithi107.stdout:3/921: dwrite d1/d1e/d26/f3d [0,4194304] 0 2024-03-04T13:43:22.650 INFO:tasks.workunit.client.0.smithi107.stdout:4/885: mknod d0/d9/d10/d80/db8/d9a/c11a 0 2024-03-04T13:43:22.650 INFO:tasks.workunit.client.0.smithi107.stdout:3/922: write d1/de/d18/d33/d4d/d4f/ff0 [464619,16964] 0 2024-03-04T13:43:22.651 INFO:tasks.workunit.client.0.smithi107.stdout:9/805: chown d4/da/d11/d20/d23/d7e/cb1 47022058 1 2024-03-04T13:43:22.651 INFO:tasks.workunit.client.0.smithi107.stdout:8/897: dwrite d6/da/fad [0,4194304] 0 2024-03-04T13:43:22.654 INFO:tasks.workunit.client.0.smithi107.stdout:7/850: getdents d6/d13/d16/d1c 0 2024-03-04T13:43:22.654 INFO:tasks.workunit.client.0.smithi107.stdout:9/806: stat d4/da/d11/d4c/f3c 0 2024-03-04T13:43:22.655 INFO:tasks.workunit.client.0.smithi107.stdout:1/903: mkdir d1/d10/d16/d47/d7a/d128/d129 0 2024-03-04T13:43:22.655 INFO:tasks.workunit.client.0.smithi107.stdout:0/909: creat da/dc/dd/ddb/f135 x:0 0 0 2024-03-04T13:43:22.659 INFO:tasks.workunit.client.0.smithi107.stdout:3/923: mkdir d1/de/d18/d1f/d127 0 2024-03-04T13:43:22.660 INFO:tasks.workunit.client.0.smithi107.stdout:6/867: mknod df/c117 0 2024-03-04T13:43:22.666 INFO:tasks.workunit.client.0.smithi107.stdout:8/898: unlink d6/fd 0 2024-03-04T13:43:22.666 INFO:tasks.workunit.client.0.smithi107.stdout:0/910: mkdir da/dc/d34/d10d/d136 0 2024-03-04T13:43:22.667 INFO:tasks.workunit.client.0.smithi107.stdout:9/807: dread - d4/da/d11/d20/d23/fca zero size 2024-03-04T13:43:22.683 INFO:tasks.workunit.client.0.smithi107.stdout:0/911: rmdir da/dc/dd/d3e/d26/d69 39 2024-03-04T13:43:22.685 INFO:tasks.workunit.client.0.smithi107.stdout:4/886: dread d0/d9/d10/f7a [0,4194304] 0 2024-03-04T13:43:22.686 INFO:tasks.workunit.client.0.smithi107.stdout:7/851: link d6/d13/d16/d19/d38/d2c/l34 d6/d1d/d64/d9a/da9/d71/db9/dca/dd0/l11b 0 2024-03-04T13:43:22.689 INFO:tasks.workunit.client.0.smithi107.stdout:1/904: dwrite d1/d46/d6a/d4e/daa/dc9/d33/dbd/f122 [0,4194304] 0 2024-03-04T13:43:22.691 INFO:tasks.workunit.client.0.smithi107.stdout:3/924: dwrite d1/de/d18/d33/d4d/ffe [4194304,4194304] 0 2024-03-04T13:43:22.691 INFO:tasks.workunit.client.0.smithi107.stdout:8/899: symlink d6/d16/d69/deb/d3e/d67/db7/l11f 0 2024-03-04T13:43:22.699 INFO:tasks.workunit.client.0.smithi107.stdout:3/925: read d1/d2d/fca [1459942,69556] 0 2024-03-04T13:43:22.700 INFO:tasks.workunit.client.0.smithi107.stdout:9/808: rename d4/da/d11/d4c/d51/dfd to d4/da/d11/d20/d10f 0 2024-03-04T13:43:22.702 INFO:tasks.workunit.client.0.smithi107.stdout:7/852: unlink d6/d13/d16/d19/d38/d2c/cee 0 2024-03-04T13:43:22.704 INFO:tasks.workunit.client.0.smithi107.stdout:6/868: dwrite df/d2e/d36/d77/d106/f68 [0,4194304] 0 2024-03-04T13:43:22.704 INFO:tasks.workunit.client.0.smithi107.stdout:0/912: write da/dc/dd/d3e/d26/d69/df9/f103 [541881,114456] 0 2024-03-04T13:43:22.706 INFO:tasks.workunit.client.0.smithi107.stdout:1/905: unlink d1/d46/d6a/d4e/daa/dc9/d67/d6b/f9e 0 2024-03-04T13:43:22.707 INFO:tasks.workunit.client.0.smithi107.stdout:4/887: rename d0/d9/d10/lba to d0/d9/d1e/d70/l11b 0 2024-03-04T13:43:22.709 INFO:tasks.workunit.client.0.smithi107.stdout:8/900: truncate d6/dcc/fff 97839 0 2024-03-04T13:43:22.712 INFO:tasks.workunit.client.0.smithi107.stdout:4/888: read - d0/d9/d18/f116 zero size 2024-03-04T13:43:22.712 INFO:tasks.workunit.client.0.smithi107.stdout:3/926: fdatasync d1/de/d18/d33/d48/d71/d4c/d54/db7/fbf 0 2024-03-04T13:43:22.725 INFO:tasks.workunit.client.0.smithi107.stdout:2/864: write da/df/d73/d7a/d9a/fc9 [83356,110525] 0 2024-03-04T13:43:22.726 INFO:tasks.workunit.client.0.smithi107.stdout:9/809: rename d4/da/d11/d4c/d51/f9d to d4/da/d11/d20/d64/d77/de7/f110 0 2024-03-04T13:43:22.733 INFO:tasks.workunit.client.0.smithi107.stdout:3/927: creat d1/de/d11/dd5/dab/dc5/f128 x:0 0 0 2024-03-04T13:43:22.735 INFO:tasks.workunit.client.0.smithi107.stdout:9/810: symlink d4/da/d11/d20/d23/d7e/d84/l111 0 2024-03-04T13:43:22.736 INFO:tasks.workunit.client.0.smithi107.stdout:0/913: rename da/dc/dd/f2c to da/dc/dd/f137 0 2024-03-04T13:43:22.739 INFO:tasks.workunit.client.0.smithi107.stdout:3/928: write d1/d2d/f117 [356091,19797] 0 2024-03-04T13:43:22.749 INFO:tasks.workunit.client.0.smithi107.stdout:0/914: creat da/dad/f138 x:0 0 0 2024-03-04T13:43:22.750 INFO:tasks.workunit.client.0.smithi107.stdout:3/929: mknod d1/d41/d81/c129 0 2024-03-04T13:43:22.755 INFO:tasks.workunit.client.0.smithi107.stdout:2/865: creat da/d44/d8b/d9d/df8/d5c/dc5/f126 x:0 0 0 2024-03-04T13:43:22.756 INFO:tasks.workunit.client.0.smithi107.stdout:9/811: truncate d4/d34/f87 204936 0 2024-03-04T13:43:22.757 INFO:tasks.workunit.client.0.smithi107.stdout:8/901: dwrite d6/d16/d69/deb/d3e/d96/dd8/fe0 [0,4194304] 0 2024-03-04T13:43:22.762 INFO:tasks.workunit.client.0.smithi107.stdout:9/812: dread - d4/da/d11/d20/d64/d77/de7/feb zero size 2024-03-04T13:43:22.765 INFO:tasks.workunit.client.0.smithi107.stdout:3/930: mkdir d1/de/d18/d33/d48/d71/d4c/d54/d12a 0 2024-03-04T13:43:22.766 INFO:tasks.workunit.client.0.smithi107.stdout:0/915: symlink da/dc/dd/dc9/de5/d107/l139 0 2024-03-04T13:43:22.767 INFO:tasks.workunit.client.0.smithi107.stdout:6/869: dread df/d2e/d36/d5b/f93 [0,4194304] 0 2024-03-04T13:43:22.767 INFO:tasks.workunit.client.0.smithi107.stdout:7/853: write d6/d13/d14/fa4 [617827,11995] 0 2024-03-04T13:43:22.771 INFO:tasks.workunit.client.0.smithi107.stdout:1/906: dread d1/d10/d16/d1b/f3e [0,4194304] 0 2024-03-04T13:43:22.773 INFO:tasks.workunit.client.0.smithi107.stdout:2/866: truncate da/d44/d8b/d9d/df8/d5c/d6a/ff4 819564 0 2024-03-04T13:43:22.778 INFO:tasks.workunit.client.0.smithi107.stdout:7/854: chown d6/d1d/d36/f5b 1566 1 2024-03-04T13:43:22.778 INFO:tasks.workunit.client.0.smithi107.stdout:3/931: symlink d1/de/d18/d33/l12b 0 2024-03-04T13:43:22.779 INFO:tasks.workunit.client.0.smithi107.stdout:9/813: creat d4/f112 x:0 0 0 2024-03-04T13:43:22.781 INFO:tasks.workunit.client.0.smithi107.stdout:6/870: getdents df/d1b 0 2024-03-04T13:43:22.781 INFO:tasks.workunit.client.0.smithi107.stdout:0/916: dread da/dc/dd/d3e/d26/d69/df9/f103 [0,4194304] 0 2024-03-04T13:43:22.783 INFO:tasks.workunit.client.0.smithi107.stdout:2/867: symlink da/df/d73/l127 0 2024-03-04T13:43:22.783 INFO:tasks.workunit.client.0.smithi107.stdout:3/932: write d1/de/d4b/fb8 [2359824,95621] 0 2024-03-04T13:43:22.787 INFO:tasks.workunit.client.0.smithi107.stdout:3/933: dread - d1/de/d18/d1f/d42/fc0 zero size 2024-03-04T13:43:22.789 INFO:tasks.workunit.client.0.smithi107.stdout:1/907: rename d1/d46/d6a/d4e/daa/dc9/d33/d5d/d4c/dea/cfb to d1/d46/d6a/d119/c12a 0 2024-03-04T13:43:22.792 INFO:tasks.workunit.client.0.smithi107.stdout:1/908: write d1/d10/d16/d1b/fe3 [992348,59632] 0 2024-03-04T13:43:22.796 INFO:tasks.workunit.client.0.smithi107.stdout:7/855: rename d6/d13/d16/f7b to d6/d13/d16/d19/d39/d7a/f11c 0 2024-03-04T13:43:22.798 INFO:tasks.workunit.client.0.smithi107.stdout:2/868: symlink da/d29/d7c/dbd/d117/l128 0 2024-03-04T13:43:22.803 INFO:tasks.workunit.client.0.smithi107.stdout:2/869: chown da/d38/d3a/f115 511 1 2024-03-04T13:43:22.803 INFO:tasks.workunit.client.0.smithi107.stdout:8/902: link d6/d16/d69/deb/d7c/lde d6/d16/d1f/d63/l120 0 2024-03-04T13:43:22.805 INFO:tasks.workunit.client.0.smithi107.stdout:1/909: chown d1/d46/d6a/d17/f34 48673 1 2024-03-04T13:43:22.805 INFO:tasks.workunit.client.0.smithi107.stdout:6/871: rename df/d17/d24/d30/d3b to df/d2e/d36/d77/d118 0 2024-03-04T13:43:22.806 INFO:tasks.workunit.client.0.smithi107.stdout:0/917: link da/dad/l12a da/dc/dd/d3e/d26/d69/d6b/d115/l13a 0 2024-03-04T13:43:22.809 INFO:tasks.workunit.client.0.smithi107.stdout:1/910: readlink d1/d10/d16/d1b/l50 0 2024-03-04T13:43:22.809 INFO:tasks.workunit.client.0.smithi107.stdout:9/814: dwrite d4/f19 [4194304,4194304] 0 2024-03-04T13:43:22.809 INFO:tasks.workunit.client.0.smithi107.stdout:2/870: mknod da/d44/d112/c129 0 2024-03-04T13:43:22.811 INFO:tasks.workunit.client.0.smithi107.stdout:6/872: chown df/d17/d6b/d9b/d61/d99 182903 1 2024-03-04T13:43:22.811 INFO:tasks.workunit.client.0.smithi107.stdout:1/911: stat d1/d46/d6a/d65/d106/f11d 0 2024-03-04T13:43:22.811 INFO:tasks.workunit.client.0.smithi107.stdout:2/871: chown da/f26 97 1 2024-03-04T13:43:22.813 INFO:tasks.workunit.client.0.smithi107.stdout:0/918: dread da/dc/dd/d3e/d26/d88/dd2/fbd [0,4194304] 0 2024-03-04T13:43:22.818 INFO:tasks.workunit.client.0.smithi107.stdout:7/856: link d6/d1d/d64/d9a/da9/d71/db9/dca/d3d/fb1 d6/d13/d16/d19/f11d 0 2024-03-04T13:43:22.819 INFO:tasks.workunit.client.0.smithi107.stdout:8/903: mknod d6/d16/d1f/d4c/da5/de9/c121 0 2024-03-04T13:43:22.823 INFO:tasks.workunit.client.0.smithi107.stdout:7/857: read d6/d13/d16/d1c/f1f [1933588,28899] 0 2024-03-04T13:43:22.823 INFO:tasks.workunit.client.0.smithi107.stdout:0/919: read f0 [55125,31886] 0 2024-03-04T13:43:22.823 INFO:tasks.workunit.client.0.smithi107.stdout:8/904: truncate d6/d16/d1f/d4c/da5/fee 1237350 0 2024-03-04T13:43:22.827 INFO:tasks.workunit.client.0.smithi107.stdout:3/934: dread d1/de/d18/d33/d4d/d4f/fed [4194304,4194304] 0 2024-03-04T13:43:22.828 INFO:tasks.workunit.client.0.smithi107.stdout:2/872: symlink da/d44/d8b/l12a 0 2024-03-04T13:43:22.833 INFO:tasks.workunit.client.0.smithi107.stdout:9/815: write d4/ff [3237827,60502] 0 2024-03-04T13:43:22.836 INFO:tasks.workunit.client.0.smithi107.stdout:0/920: write da/dc/dd/d3e/f11 [1698476,28354] 0 2024-03-04T13:43:22.837 INFO:tasks.workunit.client.0.smithi107.stdout:9/816: chown d4/da/d11/d20/d2f/f3d 223652665 1 2024-03-04T13:43:22.838 INFO:tasks.workunit.client.0.smithi107.stdout:3/935: symlink d1/de/d18/d7e/l12c 0 2024-03-04T13:43:22.843 INFO:tasks.workunit.client.0.smithi107.stdout:6/873: getdents df/d17/d6b/d9b 0 2024-03-04T13:43:22.843 INFO:tasks.workunit.client.0.smithi107.stdout:0/921: creat da/dc/dd/dc9/f13b x:0 0 0 2024-03-04T13:43:22.844 INFO:tasks.workunit.client.0.smithi107.stdout:1/912: link d1/d46/d6a/d65/cff d1/d46/d6a/d4e/daa/dc9/c12b 0 2024-03-04T13:43:22.846 INFO:tasks.workunit.client.0.smithi107.stdout:3/936: creat d1/d41/f12d x:0 0 0 2024-03-04T13:43:22.846 INFO:tasks.workunit.client.0.smithi107.stdout:8/905: write d6/d16/f104 [4121894,101855] 0 2024-03-04T13:43:22.847 INFO:tasks.workunit.client.0.smithi107.stdout:7/858: write d6/d1d/d36/d59/fe9 [507233,76530] 0 2024-03-04T13:43:22.847 INFO:tasks.workunit.client.0.smithi107.stdout:6/874: stat df/d17/d24/d40/f62 0 2024-03-04T13:43:22.848 INFO:tasks.workunit.client.0.smithi107.stdout:0/922: chown da/dc/dd/d1e/fe2 3106 1 2024-03-04T13:43:22.849 INFO:tasks.workunit.client.0.smithi107.stdout:9/817: creat d4/da/d11/d4c/d4d/d72/d7b/f113 x:0 0 0 2024-03-04T13:43:22.850 INFO:tasks.workunit.client.0.smithi107.stdout:2/873: dwrite da/d44/d8b/d9d/df8/d5c/dc5/f126 [0,4194304] 0 2024-03-04T13:43:22.851 INFO:tasks.workunit.client.0.smithi107.stdout:1/913: mkdir d1/d12c 0 2024-03-04T13:43:22.854 INFO:tasks.workunit.client.0.smithi107.stdout:0/923: write da/dc/d34/ff6 [583220,61824] 0 2024-03-04T13:43:22.855 INFO:tasks.workunit.client.0.smithi107.stdout:6/875: write df/d17/d6b/d9b/fd7 [983634,61519] 0 2024-03-04T13:43:22.855 INFO:tasks.workunit.client.0.smithi107.stdout:9/818: write d4/f26 [3337825,110249] 0 2024-03-04T13:43:22.857 INFO:tasks.workunit.client.0.smithi107.stdout:8/906: chown d6/d16/d69/deb/d3e/d96/dd8/fe0 10353 1 2024-03-04T13:43:22.860 INFO:tasks.workunit.client.0.smithi107.stdout:3/937: mknod d1/d1e/dba/c12e 0 2024-03-04T13:43:22.861 INFO:tasks.workunit.client.0.smithi107.stdout:0/924: read da/f59 [2448821,59539] 0 2024-03-04T13:43:22.865 INFO:tasks.workunit.client.0.smithi107.stdout:6/876: creat df/d2e/d36/d5b/d7d/dfc/f119 x:0 0 0 2024-03-04T13:43:22.865 INFO:tasks.workunit.client.0.smithi107.stdout:3/938: write d1/de/d18/d33/d4d/ffe [9137461,87262] 0 2024-03-04T13:43:22.875 INFO:tasks.workunit.client.0.smithi107.stdout:2/874: rename da/df/fbe to da/f12b 0 2024-03-04T13:43:22.876 INFO:tasks.workunit.client.0.smithi107.stdout:8/907: write d6/da/f1b [2619993,70849] 0 2024-03-04T13:43:22.877 INFO:tasks.workunit.client.0.smithi107.stdout:3/939: dread d1/d1e/d26/d9e/fec [0,4194304] 0 2024-03-04T13:43:22.879 INFO:tasks.workunit.client.0.smithi107.stdout:7/859: link d6/d13/d16/d19/d38/d2c/d8d/ce7 d6/d116/c11e 0 2024-03-04T13:43:22.881 INFO:tasks.workunit.client.0.smithi107.stdout:1/914: creat d1/d10/d16/d1b/f12d x:0 0 0 2024-03-04T13:43:22.888 INFO:tasks.workunit.client.0.smithi107.stdout:0/925: rmdir da/dc/dd/d3e/d26/d69/d6b/df0/d10e 0 2024-03-04T13:43:22.890 INFO:tasks.workunit.client.0.smithi107.stdout:8/908: symlink d6/d16/d1f/d4c/da5/dfb/l122 0 2024-03-04T13:43:22.892 INFO:tasks.workunit.client.0.smithi107.stdout:1/915: rmdir d1/d46/d6a/d4e/daa/dc9/d33/d5d/d24 39 2024-03-04T13:43:22.894 INFO:tasks.workunit.client.0.smithi107.stdout:1/916: chown d1/d46/d6a/d65/d106/f117 63886 1 2024-03-04T13:43:22.897 INFO:tasks.workunit.client.0.smithi107.stdout:9/819: rename d4/da/d11/d20/d2f/d9c/d92/ld2 to d4/l114 0 2024-03-04T13:43:22.898 INFO:tasks.workunit.client.0.smithi107.stdout:0/926: creat da/dc/dd/ddb/f13c x:0 0 0 2024-03-04T13:43:22.899 INFO:tasks.workunit.client.0.smithi107.stdout:6/877: dread df/d2e/d36/d5b/f6c [0,4194304] 0 2024-03-04T13:43:22.900 INFO:tasks.workunit.client.0.smithi107.stdout:8/909: creat d6/d16/d1f/d63/d93/f123 x:0 0 0 2024-03-04T13:43:22.902 INFO:tasks.workunit.client.0.smithi107.stdout:8/910: write d6/d16/d69/deb/fbb [3325974,43037] 0 2024-03-04T13:43:22.905 INFO:tasks.workunit.client.0.smithi107.stdout:7/860: rename d6/d1d/d64/d9a/da9/d71/ce0 to d6/d1d/d64/d9a/c11f 0 2024-03-04T13:43:22.910 INFO:tasks.workunit.client.0.smithi107.stdout:6/878: creat df/d2e/f11a x:0 0 0 2024-03-04T13:43:22.911 INFO:tasks.workunit.client.0.smithi107.stdout:6/879: readlink df/d17/lc3 0 2024-03-04T13:43:22.911 INFO:tasks.workunit.client.0.smithi107.stdout:2/875: link da/d44/f79 da/df/d73/db3/db1/f12c 0 2024-03-04T13:43:22.916 INFO:tasks.workunit.client.0.smithi107.stdout:8/911: rename d6/l99 to d6/d16/d1f/d119/l124 0 2024-03-04T13:43:22.928 INFO:tasks.workunit.client.0.smithi107.stdout:6/880: symlink df/d2e/d36/d77/d106/d7e/d86/l11b 0 2024-03-04T13:43:22.928 INFO:tasks.workunit.client.0.smithi107.stdout:9/820: write d4/da/d11/d65/f3e [3991270,54705] 0 2024-03-04T13:43:22.933 INFO:tasks.workunit.client.0.smithi107.stdout:7/861: fdatasync d6/d1d/d64/d9a/da9/d71/db9/dca/f6a 0 2024-03-04T13:43:22.933 INFO:tasks.workunit.client.0.smithi107.stdout:1/917: dwrite d1/d46/d6a/d4e/daa/dc9/d33/d5d/d24/f2d [0,4194304] 0 2024-03-04T13:43:22.939 INFO:tasks.workunit.client.0.smithi107.stdout:6/881: read - df/d38/d4f/daf/fc7 zero size 2024-03-04T13:43:22.940 INFO:tasks.workunit.client.0.smithi107.stdout:2/876: dread da/d44/f52 [0,4194304] 0 2024-03-04T13:43:22.941 INFO:tasks.workunit.client.0.smithi107.stdout:0/927: link da/f2a da/dc/dd/d1e/f13d 0 2024-03-04T13:43:22.941 INFO:tasks.workunit.client.0.smithi107.stdout:8/912: link d6/d16/fc5 d6/da/f125 0 2024-03-04T13:43:22.942 INFO:tasks.workunit.client.0.smithi107.stdout:1/918: creat d1/d46/d6a/d4e/daa/dc9/d33/d5d/d4c/f12e x:0 0 0 2024-03-04T13:43:22.948 INFO:tasks.workunit.client.0.smithi107.stdout:6/882: write df/d2e/d36/d5b/d7d/f10e [1021864,23720] 0 2024-03-04T13:43:22.950 INFO:tasks.workunit.client.0.smithi107.stdout:1/919: mknod d1/d46/d6a/d65/d106/d61/c12f 0 2024-03-04T13:43:22.951 INFO:tasks.workunit.client.0.smithi107.stdout:0/928: mkdir da/dc/dd/dc9/d13e 0 2024-03-04T13:43:22.952 INFO:tasks.workunit.client.0.smithi107.stdout:8/913: chown d6/da/d25/d2a/c48 211 1 2024-03-04T13:43:22.955 INFO:tasks.workunit.client.0.smithi107.stdout:0/929: creat da/dc/dd/dc9/f13f x:0 0 0 2024-03-04T13:43:22.960 INFO:tasks.workunit.client.0.smithi107.stdout:6/883: chown df/d17/d24/d8d/le7 4 1 2024-03-04T13:43:22.964 INFO:tasks.workunit.client.0.smithi107.stdout:8/914: symlink d6/d16/d69/deb/d7c/l126 0 2024-03-04T13:43:22.971 INFO:tasks.workunit.client.0.smithi107.stdout:1/920: rename d1/d10/d16/d1b/d40/ca4 to d1/d46/d6a/d4e/daa/dc9/d67/d6b/c130 0 2024-03-04T13:43:22.974 INFO:tasks.workunit.client.0.smithi107.stdout:2/877: dwrite da/d34/f41 [0,4194304] 0 2024-03-04T13:43:22.976 INFO:tasks.workunit.client.0.smithi107.stdout:7/862: dread d6/d13/d16/f8b [0,4194304] 0 2024-03-04T13:43:22.976 INFO:tasks.workunit.client.0.smithi107.stdout:6/884: rename df/d2e/d36/d71 to df/d17/d24/d40/d63/d11c 0 2024-03-04T13:43:22.982 INFO:tasks.workunit.client.0.smithi107.stdout:9/821: dread d4/f26 [0,4194304] 0 2024-03-04T13:43:22.983 INFO:tasks.workunit.client.0.smithi107.stdout:1/921: mkdir d1/d10/d16/d1b/d40/d131 0 2024-03-04T13:43:22.985 INFO:tasks.workunit.client.0.smithi107.stdout:0/930: dread da/dc/dd/f24 [0,4194304] 0 2024-03-04T13:43:22.987 INFO:tasks.workunit.client.0.smithi107.stdout:9/822: stat d4/da/d11/cb5 0 2024-03-04T13:43:22.989 INFO:tasks.workunit.client.0.smithi107.stdout:1/922: write d1/d10/d16/d47/d7a/dda/f116 [181371,113797] 0 2024-03-04T13:43:22.989 INFO:tasks.workunit.client.0.smithi107.stdout:0/931: stat da/dc/dd/d3e/d26/d88 0 2024-03-04T13:43:22.991 INFO:tasks.workunit.client.0.smithi107.stdout:6/885: rename df/d38/c96 to df/c11d 0 2024-03-04T13:43:22.992 INFO:tasks.workunit.client.0.smithi107.stdout:8/915: dwrite d6/da/d25/f3f [0,4194304] 0 2024-03-04T13:43:22.996 INFO:tasks.workunit.client.0.smithi107.stdout:1/923: write d1/d46/d6a/d4e/daa/dc9/d67/fcd [363654,30379] 0 2024-03-04T13:43:23.005 INFO:tasks.workunit.client.0.smithi107.stdout:8/916: write d6/d16/d1f/d4c/f52 [125089,28686] 0 2024-03-04T13:43:23.007 INFO:tasks.workunit.client.0.smithi107.stdout:6/886: truncate df/d17/d6b/da8/d105/f110 984495 0 2024-03-04T13:43:23.007 INFO:tasks.workunit.client.0.smithi107.stdout:9/823: truncate d4/da/d11/d20/d64/fda 390706 0 2024-03-04T13:43:23.009 INFO:tasks.workunit.client.0.smithi107.stdout:9/824: read d4/ff [1911861,13333] 0 2024-03-04T13:43:23.010 INFO:tasks.workunit.client.0.smithi107.stdout:4/889: sync 2024-03-04T13:43:23.011 INFO:tasks.workunit.client.0.smithi107.stdout:2/878: write da/d34/fc8 [1360235,26660] 0 2024-03-04T13:43:23.015 INFO:tasks.workunit.client.0.smithi107.stdout:7/863: write d6/d13/d16/d19/d39/f54 [940991,31625] 0 2024-03-04T13:43:23.015 INFO:tasks.workunit.client.0.smithi107.stdout:8/917: write d6/da/f11 [4959992,38794] 0 2024-03-04T13:43:23.016 INFO:tasks.workunit.client.0.smithi107.stdout:4/890: creat d0/d2f/d7d/f11c x:0 0 0 2024-03-04T13:43:23.017 INFO:tasks.workunit.client.0.smithi107.stdout:6/887: stat df/d17/d24/d30/d5d/d6a/fe3 0 2024-03-04T13:43:23.022 INFO:tasks.workunit.client.0.smithi107.stdout:0/932: dwrite da/dc/dd/d3e/f11 [4194304,4194304] 0 2024-03-04T13:43:23.022 INFO:tasks.workunit.client.0.smithi107.stdout:6/888: fsync df/d17/d5c/f10a 0 2024-03-04T13:43:23.022 INFO:tasks.workunit.client.0.smithi107.stdout:9/825: read - d4/da/d11/d20/d23/d7e/d84/fcd zero size 2024-03-04T13:43:23.045 INFO:tasks.workunit.client.0.smithi107.stdout:8/918: dwrite d6/da/f70 [0,4194304] 0 2024-03-04T13:43:23.049 INFO:tasks.workunit.client.0.smithi107.stdout:7/864: dread d6/d1d/d36/d59/f8a [0,4194304] 0 2024-03-04T13:43:23.066 INFO:tasks.workunit.client.0.smithi107.stdout:7/865: dwrite d6/d1d/d64/d9a/da9/d71/fcc [8388608,4194304] 0 2024-03-04T13:43:23.069 INFO:tasks.workunit.client.0.smithi107.stdout:0/933: dread da/dc/dd/d1e/fb5 [0,4194304] 0 2024-03-04T13:43:23.088 INFO:tasks.workunit.client.0.smithi107.stdout:2/879: getdents da/d44/d8b/d9d/dd0 0 2024-03-04T13:43:23.090 INFO:tasks.workunit.client.0.smithi107.stdout:2/880: write da/d44/d8b/d9d/fd6 [584615,71506] 0 2024-03-04T13:43:23.090 INFO:tasks.workunit.client.0.smithi107.stdout:4/891: mkdir d0/d9/d18/d29/dfc/d11d 0 2024-03-04T13:43:23.092 INFO:tasks.workunit.client.0.smithi107.stdout:6/889: fsync df/d17/d24/d30/d5d/d6a/f94 0 2024-03-04T13:43:23.100 INFO:tasks.workunit.client.0.smithi107.stdout:7/866: write d6/d13/d16/d19/faa [5182534,105340] 0 2024-03-04T13:43:23.103 INFO:tasks.workunit.client.0.smithi107.stdout:0/934: rmdir da/dc/dd/dc9 39 2024-03-04T13:43:23.109 INFO:tasks.workunit.client.0.smithi107.stdout:2/881: rename da/df/d27/f5e to da/df/d27/d10b/d5f/de9/f12d 0 2024-03-04T13:43:23.110 INFO:tasks.workunit.client.0.smithi107.stdout:8/919: symlink d6/d16/d69/deb/l127 0 2024-03-04T13:43:23.113 INFO:tasks.workunit.client.0.smithi107.stdout:0/935: mkdir da/dc/dd/d3e/d26/d69/de9/d140 0 2024-03-04T13:43:23.116 INFO:tasks.workunit.client.0.smithi107.stdout:6/890: rename df/d2e/d36/d5b/d7d/dfc/c6d to df/d17/d24/d40/d63/c11e 0 2024-03-04T13:43:23.119 INFO:tasks.workunit.client.0.smithi107.stdout:8/920: read d6/fe [4298988,6088] 0 2024-03-04T13:43:23.119 INFO:tasks.workunit.client.0.smithi107.stdout:7/867: dwrite d6/f45 [4194304,4194304] 0 2024-03-04T13:43:23.121 INFO:tasks.workunit.client.0.smithi107.stdout:9/826: dwrite d4/da/d11/d20/d23/de8/f104 [0,4194304] 0 2024-03-04T13:43:23.123 INFO:tasks.workunit.client.0.smithi107.stdout:0/936: dread - da/dc/dd/ffa zero size 2024-03-04T13:43:23.127 INFO:tasks.workunit.client.0.smithi107.stdout:6/891: unlink df/d57/c6f 0 2024-03-04T13:43:23.129 INFO:tasks.workunit.client.0.smithi107.stdout:4/892: dwrite d0/d9/d1e/f20 [0,4194304] 0 2024-03-04T13:43:23.130 INFO:tasks.workunit.client.0.smithi107.stdout:4/893: readlink d0/d9/d18/l3a 0 2024-03-04T13:43:23.134 INFO:tasks.workunit.client.0.smithi107.stdout:7/868: fsync d6/d13/d16/d19/d38/d2c/f4a 0 2024-03-04T13:43:23.138 INFO:tasks.workunit.client.0.smithi107.stdout:7/869: write d6/d1d/d36/f81 [2035398,121821] 0 2024-03-04T13:43:23.145 INFO:tasks.workunit.client.0.smithi107.stdout:2/882: dwrite da/d39/f46 [4194304,4194304] 0 2024-03-04T13:43:23.145 INFO:tasks.workunit.client.0.smithi107.stdout:8/921: write d6/d16/d69/deb/fc1 [755158,82964] 0 2024-03-04T13:43:23.147 INFO:tasks.workunit.client.0.smithi107.stdout:6/892: creat df/d17/d6b/da8/de6/f11f x:0 0 0 2024-03-04T13:43:23.154 INFO:tasks.workunit.client.0.smithi107.stdout:9/827: write d4/da/d11/d4c/d51/f5a [1563099,79227] 0 2024-03-04T13:43:23.154 INFO:tasks.workunit.client.0.smithi107.stdout:2/883: stat da/l23 0 2024-03-04T13:43:23.154 INFO:tasks.workunit.client.0.smithi107.stdout:8/922: stat d6/d16/d1f/d4c/da5/de9/c121 0 2024-03-04T13:43:23.154 INFO:tasks.workunit.client.0.smithi107.stdout:7/870: symlink d6/l120 0 2024-03-04T13:43:23.155 INFO:tasks.workunit.client.0.smithi107.stdout:2/884: chown da/d44/d8b/d9d/df8/d5c/f9e 0 1 2024-03-04T13:43:23.156 INFO:tasks.workunit.client.0.smithi107.stdout:0/937: rmdir da/dc/d34/d10d/d136 0 2024-03-04T13:43:23.159 INFO:tasks.workunit.client.0.smithi107.stdout:2/885: write da/d44/d8b/d98/f119 [207452,96489] 0 2024-03-04T13:43:23.165 INFO:tasks.workunit.client.0.smithi107.stdout:9/828: read d4/da/d11/d20/d39/f7a [2090124,5832] 0 2024-03-04T13:43:23.165 INFO:tasks.workunit.client.0.smithi107.stdout:3/940: sync 2024-03-04T13:43:23.166 INFO:tasks.workunit.client.0.smithi107.stdout:9/829: chown d4/da/d11/d20/d23 1939897 1 2024-03-04T13:43:23.168 INFO:tasks.workunit.client.0.smithi107.stdout:2/886: rmdir da/df/d73/daa 39 2024-03-04T13:43:23.169 INFO:tasks.workunit.client.0.smithi107.stdout:6/893: write f6 [913659,98184] 0 2024-03-04T13:43:23.173 INFO:tasks.workunit.client.0.smithi107.stdout:9/830: fsync d4/f8 0 2024-03-04T13:43:23.173 INFO:tasks.workunit.client.0.smithi107.stdout:4/894: dread d0/d9/d18/df1/f50 [0,4194304] 0 2024-03-04T13:43:23.176 INFO:tasks.workunit.client.0.smithi107.stdout:2/887: read da/d44/d8b/d98/dc3/f10f [346810,11958] 0 2024-03-04T13:43:23.177 INFO:tasks.workunit.client.0.smithi107.stdout:9/831: write d4/da/d11/d4c/d51/fe2 [504305,28533] 0 2024-03-04T13:43:23.177 INFO:tasks.workunit.client.0.smithi107.stdout:7/871: truncate d6/d13/d16/d19/d38/f47 2336579 0 2024-03-04T13:43:23.178 INFO:tasks.workunit.client.0.smithi107.stdout:3/941: write d1/de/d11/f45 [2881210,83547] 0 2024-03-04T13:43:23.187 INFO:tasks.workunit.client.0.smithi107.stdout:4/895: mknod d0/d9/de/d14/d33/c11e 0 2024-03-04T13:43:23.188 INFO:tasks.workunit.client.0.smithi107.stdout:3/942: dread d1/de/d18/d1f/d42/f62 [0,4194304] 0 2024-03-04T13:43:23.198 INFO:tasks.workunit.client.0.smithi107.stdout:9/832: creat d4/da/d11/d20/d64/d77/df9/f115 x:0 0 0 2024-03-04T13:43:23.201 INFO:tasks.workunit.client.0.smithi107.stdout:7/872: dwrite d6/d13/d16/d19/fe8 [0,4194304] 0 2024-03-04T13:43:23.204 INFO:tasks.workunit.client.0.smithi107.stdout:3/943: dread d1/de/d18/d33/d48/d93/fdd [0,4194304] 0 2024-03-04T13:43:23.205 INFO:tasks.workunit.client.0.smithi107.stdout:6/894: write df/d2e/d36/d5b/fa1 [770754,40029] 0 2024-03-04T13:43:23.208 INFO:tasks.workunit.client.0.smithi107.stdout:4/896: dwrite d0/f112 [0,4194304] 0 2024-03-04T13:43:23.210 INFO:tasks.workunit.client.0.smithi107.stdout:3/944: chown d1/d1e/d26/c61 26971 1 2024-03-04T13:43:23.211 INFO:tasks.workunit.client.0.smithi107.stdout:9/833: creat d4/da/d11/d20/d23/dcc/de4/f116 x:0 0 0 2024-03-04T13:43:23.215 INFO:tasks.workunit.client.0.smithi107.stdout:2/888: dwrite da/d44/f81 [0,4194304] 0 2024-03-04T13:43:23.215 INFO:tasks.workunit.client.0.smithi107.stdout:3/945: chown d1/de/d18/d33/d48/d71/f72 3156 1 2024-03-04T13:43:23.228 INFO:tasks.workunit.client.0.smithi107.stdout:7/873: write d6/d1d/d64/d9a/da9/d71/db9/dca/f6a [227975,68353] 0 2024-03-04T13:43:23.233 INFO:tasks.workunit.client.0.smithi107.stdout:3/946: write d1/de/d18/d1f/f94 [918472,14668] 0 2024-03-04T13:43:23.239 INFO:tasks.workunit.client.0.smithi107.stdout:7/874: fdatasync d6/d13/d16/d19/d38/ff6 0 2024-03-04T13:43:23.242 INFO:tasks.workunit.client.0.smithi107.stdout:3/947: mkdir d1/d1e/dae/d12f 0 2024-03-04T13:43:23.246 INFO:tasks.workunit.client.0.smithi107.stdout:7/875: fsync d6/f7 0 2024-03-04T13:43:23.249 INFO:tasks.workunit.client.0.smithi107.stdout:2/889: dwrite da/df/d73/db3/db1/f111 [0,4194304] 0 2024-03-04T13:43:23.251 INFO:tasks.workunit.client.0.smithi107.stdout:3/948: symlink d1/d1e/dba/l130 0 2024-03-04T13:43:23.252 INFO:tasks.workunit.client.0.smithi107.stdout:6/895: dwrite df/d38/d4f/daf/fd2 [0,4194304] 0 2024-03-04T13:43:23.252 INFO:tasks.workunit.client.0.smithi107.stdout:4/897: dwrite d0/d9/d18/df1/f64 [0,4194304] 0 2024-03-04T13:43:23.261 INFO:tasks.workunit.client.0.smithi107.stdout:3/949: symlink d1/de/d3f/l131 0 2024-03-04T13:43:23.263 INFO:tasks.workunit.client.0.smithi107.stdout:7/876: mknod d6/d1d/d64/d9a/da9/d71/db9/dca/dd0/c121 0 2024-03-04T13:43:23.265 INFO:tasks.workunit.client.0.smithi107.stdout:3/950: chown d1/de/d18/d33/d48/ff6 50342 1 2024-03-04T13:43:23.267 INFO:tasks.workunit.client.0.smithi107.stdout:3/951: readlink d1/de/d18/d7e/lf3 0 2024-03-04T13:43:23.268 INFO:tasks.workunit.client.0.smithi107.stdout:6/896: truncate df/d38/d4f/daf/fc7 687234 0 2024-03-04T13:43:23.270 INFO:tasks.workunit.client.0.smithi107.stdout:2/890: getdents da 0 2024-03-04T13:43:23.270 INFO:tasks.workunit.client.0.smithi107.stdout:3/952: write d1/d1e/f22 [650940,126241] 0 2024-03-04T13:43:23.273 INFO:tasks.workunit.client.0.smithi107.stdout:7/877: chown d6/d17/lfa 1229183640 1 2024-03-04T13:43:23.273 INFO:tasks.workunit.client.0.smithi107.stdout:3/953: stat d1/de/d18/d33/d48/fa6 0 2024-03-04T13:43:23.278 INFO:tasks.workunit.client.0.smithi107.stdout:7/878: mknod d6/d13/d16/d19/d38/d6e/c122 0 2024-03-04T13:43:23.280 INFO:tasks.workunit.client.0.smithi107.stdout:7/879: chown d6/d17/f109 457602 1 2024-03-04T13:43:23.280 INFO:tasks.workunit.client.0.smithi107.stdout:6/897: getdents df/d1b 0 2024-03-04T13:43:23.282 INFO:tasks.workunit.client.0.smithi107.stdout:3/954: creat d1/de/d18/d33/d48/d71/d4c/d54/db7/f132 x:0 0 0 2024-03-04T13:43:23.282 INFO:tasks.workunit.client.0.smithi107.stdout:2/891: link da/df/d3e/lcd da/d44/d8b/d9d/df8/l12e 0 2024-03-04T13:43:23.285 INFO:tasks.workunit.client.0.smithi107.stdout:6/898: dread - df/d17/d6b/d9b/f102 zero size 2024-03-04T13:43:23.288 INFO:tasks.workunit.client.0.smithi107.stdout:3/955: symlink d1/d1e/d26/l133 0 2024-03-04T13:43:23.292 INFO:tasks.workunit.client.0.smithi107.stdout:3/956: symlink d1/de/d18/d33/d4d/d4f/dc3/l134 0 2024-03-04T13:43:23.302 INFO:tasks.workunit.client.0.smithi107.stdout:4/898: dread d0/d9/d18/df1/dc1/ff2 [0,4194304] 0 2024-03-04T13:43:23.303 INFO:tasks.workunit.client.0.smithi107.stdout:3/957: creat d1/f135 x:0 0 0 2024-03-04T13:43:23.306 INFO:tasks.workunit.client.0.smithi107.stdout:6/899: link df/d17/d24/d40/d63/c11e df/d2e/d36/d77/d106/de4/c120 0 2024-03-04T13:43:23.308 INFO:tasks.workunit.client.0.smithi107.stdout:2/892: dread da/d29/f2d [0,4194304] 0 2024-03-04T13:43:23.309 INFO:tasks.workunit.client.0.smithi107.stdout:3/958: creat d1/de/d18/d1f/d42/f136 x:0 0 0 2024-03-04T13:43:23.311 INFO:tasks.workunit.client.0.smithi107.stdout:4/899: rename d0/d9/d10/d80/db8/d97/fe1 to d0/d9/d10/d80/db8/d97/de3/d108/f11f 0 2024-03-04T13:43:23.312 INFO:tasks.workunit.client.0.smithi107.stdout:6/900: mknod df/d2e/d36/d5b/d7d/d112/c121 0 2024-03-04T13:43:23.313 INFO:tasks.workunit.client.0.smithi107.stdout:4/900: readlink d0/d9/de/d14/lc0 0 2024-03-04T13:43:23.313 INFO:tasks.workunit.client.0.smithi107.stdout:3/959: symlink d1/de/d18/d1f/d42/l137 0 2024-03-04T13:43:23.313 INFO:tasks.workunit.client.0.smithi107.stdout:1/924: sync 2024-03-04T13:43:23.316 INFO:tasks.workunit.client.0.smithi107.stdout:6/901: chown df/f29 5 1 2024-03-04T13:43:23.320 INFO:tasks.workunit.client.0.smithi107.stdout:3/960: mkdir d1/d138 0 2024-03-04T13:43:23.320 INFO:tasks.workunit.client.0.smithi107.stdout:7/880: dwrite d6/d13/d16/d19/d38/d62/f9b [0,4194304] 0 2024-03-04T13:43:23.322 INFO:tasks.workunit.client.0.smithi107.stdout:1/925: dread d1/d46/d6a/d65/d106/f117 [0,4194304] 0 2024-03-04T13:43:23.324 INFO:tasks.workunit.client.0.smithi107.stdout:3/961: dread - d1/de/d18/d33/d48/d71/d4c/fcd zero size 2024-03-04T13:43:23.329 INFO:tasks.workunit.client.0.smithi107.stdout:4/901: rename d0/d9/f11 to d0/d9/d18/d29/f120 0 2024-03-04T13:43:23.329 INFO:tasks.workunit.client.0.smithi107.stdout:4/902: fsync d0/d9/de/f62 0 2024-03-04T13:43:23.334 INFO:tasks.workunit.client.0.smithi107.stdout:6/902: dwrite df/d38/d4f/d74/f109 [0,4194304] 0 2024-03-04T13:43:23.336 INFO:tasks.workunit.client.0.smithi107.stdout:2/893: truncate da/df/d3e/fcf 3121774 0 2024-03-04T13:43:23.340 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:23 smithi107 ceph-mon[28739]: pgmap v63: 65 pgs: 65 active+clean; 3.4 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 64 MiB/s rd, 159 MiB/s wr, 359 op/s 2024-03-04T13:43:23.340 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:23 smithi107 ceph-mon[28739]: from='mgr.24331 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:43:23.350 INFO:tasks.workunit.client.0.smithi107.stdout:3/962: dwrite d1/de/d18/d33/d48/d71/d4c/d54/db7/dbe/f10e [0,4194304] 0 2024-03-04T13:43:23.353 INFO:tasks.workunit.client.0.smithi107.stdout:1/926: mkdir d1/d46/d6a/d65/d11c/d132 0 2024-03-04T13:43:23.356 INFO:tasks.workunit.client.0.smithi107.stdout:2/894: symlink da/df/d73/db3/l12f 0 2024-03-04T13:43:23.357 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:23 smithi169 ceph-mon[36313]: pgmap v63: 65 pgs: 65 active+clean; 3.4 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 64 MiB/s rd, 159 MiB/s wr, 359 op/s 2024-03-04T13:43:23.357 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:23 smithi169 ceph-mon[36313]: from='mgr.24331 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:43:23.362 INFO:tasks.workunit.client.0.smithi107.stdout:2/895: dread - da/df/d27/d10b/d5f/ff1 zero size 2024-03-04T13:43:23.362 INFO:tasks.workunit.client.0.smithi107.stdout:6/903: dwrite df/d2e/d36/d5b/d7d/dfc/fba [0,4194304] 0 2024-03-04T13:43:23.366 INFO:tasks.workunit.client.0.smithi107.stdout:1/927: rmdir d1/d10/d16/d1b 39 2024-03-04T13:43:23.367 INFO:tasks.workunit.client.0.smithi107.stdout:8/923: sync 2024-03-04T13:43:23.367 INFO:tasks.workunit.client.0.smithi107.stdout:7/881: dwrite d6/d1d/d64/d9a/da9/d71/db9/dca/fd3 [0,4194304] 0 2024-03-04T13:43:23.371 INFO:tasks.workunit.client.0.smithi107.stdout:2/896: write da/df/d27/d10b/ff7 [1522143,74784] 0 2024-03-04T13:43:23.372 INFO:tasks.workunit.client.0.smithi107.stdout:6/904: stat df/d2e/d36/d5b/d7d/cda 0 2024-03-04T13:43:23.372 INFO:tasks.workunit.client.0.smithi107.stdout:1/928: chown d1/d46/d6a/d4e/daa/dc9/d67/f10b 114 1 2024-03-04T13:43:23.378 INFO:tasks.workunit.client.0.smithi107.stdout:7/882: write d6/d17/f109 [3894136,51680] 0 2024-03-04T13:43:23.381 INFO:tasks.workunit.client.0.smithi107.stdout:1/929: stat d1/d46/d6a/d4e/daa/dc9/d67/d6b 0 2024-03-04T13:43:23.382 INFO:tasks.workunit.client.0.smithi107.stdout:6/905: creat df/d17/d6b/da8/f122 x:0 0 0 2024-03-04T13:43:23.388 INFO:tasks.workunit.client.0.smithi107.stdout:7/883: chown d6/d13/d16/d19/d38/d62/ddb/lf0 260001 1 2024-03-04T13:43:23.390 INFO:tasks.workunit.client.0.smithi107.stdout:6/906: mkdir df/d2e/d36/d77/d118/deb/d123 0 2024-03-04T13:43:23.391 INFO:tasks.workunit.client.0.smithi107.stdout:4/903: dwrite d0/d9/de/f43 [0,4194304] 0 2024-03-04T13:43:23.395 INFO:tasks.workunit.client.0.smithi107.stdout:1/930: read d1/d10/d16/f37 [2065854,115347] 0 2024-03-04T13:43:23.396 INFO:tasks.workunit.client.0.smithi107.stdout:3/963: dwrite d1/de/d11/f95 [0,4194304] 0 2024-03-04T13:43:23.401 INFO:tasks.workunit.client.0.smithi107.stdout:7/884: chown d6/d1d/d64/d9a/da9/d71/db9/fd8 0 1 2024-03-04T13:43:23.403 INFO:tasks.workunit.client.0.smithi107.stdout:1/931: rmdir d1/d46/d6a/d4e 39 2024-03-04T13:43:23.406 INFO:tasks.workunit.client.0.smithi107.stdout:6/907: truncate df/d2e/d36/d77/d106/f70 3341626 0 2024-03-04T13:43:23.407 INFO:tasks.workunit.client.0.smithi107.stdout:2/897: dwrite da/d39/fc4 [0,4194304] 0 2024-03-04T13:43:23.408 INFO:tasks.workunit.client.0.smithi107.stdout:3/964: creat d1/de/d18/d33/d48/d93/f139 x:0 0 0 2024-03-04T13:43:23.413 INFO:tasks.workunit.client.0.smithi107.stdout:6/908: rmdir df/d17/d6b 39 2024-03-04T13:43:23.415 INFO:tasks.workunit.client.0.smithi107.stdout:7/885: creat d6/d13/dbc/f123 x:0 0 0 2024-03-04T13:43:23.420 INFO:tasks.workunit.client.0.smithi107.stdout:6/909: chown df/d2e/d36/d77/c80 0 1 2024-03-04T13:43:23.428 INFO:tasks.workunit.client.0.smithi107.stdout:2/898: mknod da/d44/d8b/d98/c130 0 2024-03-04T13:43:23.428 INFO:tasks.workunit.client.0.smithi107.stdout:3/965: creat d1/de/d18/d33/d48/d71/d4c/f13a x:0 0 0 2024-03-04T13:43:23.431 INFO:tasks.workunit.client.0.smithi107.stdout:6/910: stat df/d17/d6b/d9b/c4a 0 2024-03-04T13:43:23.432 INFO:tasks.workunit.client.0.smithi107.stdout:2/899: write da/df/d27/d10b/ff7 [105212,79742] 0 2024-03-04T13:43:23.436 INFO:tasks.workunit.client.0.smithi107.stdout:3/966: write d1/de/d18/d33/d4d/f90 [158855,1556] 0 2024-03-04T13:43:23.438 INFO:tasks.workunit.client.0.smithi107.stdout:6/911: rmdir df/d2e/d36/d5b/d7d/dfc 39 2024-03-04T13:43:23.439 INFO:tasks.workunit.client.0.smithi107.stdout:2/900: mknod da/df/d73/dd1/ddb/c131 0 2024-03-04T13:43:23.445 INFO:tasks.workunit.client.0.smithi107.stdout:3/967: creat d1/de/d18/f13b x:0 0 0 2024-03-04T13:43:23.464 INFO:tasks.workunit.client.0.smithi107.stdout:4/904: dread d0/d2f/d5f/f109 [0,4194304] 0 2024-03-04T13:43:23.470 INFO:tasks.workunit.client.0.smithi107.stdout:2/901: dwrite da/df/d27/d10b/d5f/f93 [4194304,4194304] 0 2024-03-04T13:43:23.471 INFO:tasks.workunit.client.0.smithi107.stdout:3/968: dwrite d1/d1e/d26/f3d [0,4194304] 0 2024-03-04T13:43:23.480 INFO:tasks.workunit.client.0.smithi107.stdout:3/969: creat d1/d2d/f13c x:0 0 0 2024-03-04T13:43:23.482 INFO:tasks.workunit.client.0.smithi107.stdout:7/886: dread d6/d13/d16/fbb [0,4194304] 0 2024-03-04T13:43:23.483 INFO:tasks.workunit.client.0.smithi107.stdout:9/834: sync 2024-03-04T13:43:23.483 INFO:tasks.workunit.client.0.smithi107.stdout:0/938: sync 2024-03-04T13:43:23.484 INFO:tasks.workunit.client.0.smithi107.stdout:2/902: creat da/df/d85/f132 x:0 0 0 2024-03-04T13:43:23.488 INFO:tasks.workunit.client.0.smithi107.stdout:1/932: dread d1/d46/d6a/d4e/daa/dc9/d33/d5d/d4c/dea/ff6 [0,4194304] 0 2024-03-04T13:43:23.491 INFO:tasks.workunit.client.0.smithi107.stdout:4/905: dwrite d0/d9/d10/d80/db8/d21/fa2 [0,4194304] 0 2024-03-04T13:43:23.500 INFO:tasks.workunit.client.0.smithi107.stdout:9/835: symlink d4/da/d11/d20/d2f/dec/l117 0 2024-03-04T13:43:23.500 INFO:tasks.workunit.client.0.smithi107.stdout:3/970: write d1/de/d18/f92 [276844,66060] 0 2024-03-04T13:43:23.502 INFO:tasks.workunit.client.0.smithi107.stdout:4/906: mknod d0/d9/d10/d80/db8/d97/c121 0 2024-03-04T13:43:23.503 INFO:tasks.workunit.client.0.smithi107.stdout:1/933: mkdir d1/d38/d133 0 2024-03-04T13:43:23.506 INFO:tasks.workunit.client.0.smithi107.stdout:9/836: write d4/d34/fe0 [2578711,130176] 0 2024-03-04T13:43:23.509 INFO:tasks.workunit.client.0.smithi107.stdout:0/939: rename da/d7e/f7f to da/dc/dd/d3e/d26/d88/f141 0 2024-03-04T13:43:23.512 INFO:tasks.workunit.client.0.smithi107.stdout:2/903: getdents da/d38/d3a/d5d 0 2024-03-04T13:43:23.516 INFO:tasks.workunit.client.0.smithi107.stdout:3/971: read d1/de/d18/d33/d4d/d4f/ff0 [426105,18156] 0 2024-03-04T13:43:23.520 INFO:tasks.workunit.client.0.smithi107.stdout:7/887: dwrite d6/d1d/d36/f81 [0,4194304] 0 2024-03-04T13:43:23.520 INFO:tasks.workunit.client.0.smithi107.stdout:1/934: chown d1/d46/d6a/d65/d106/d61/ddb/de6/d124/dba/c10f 3292976 1 2024-03-04T13:43:23.524 INFO:tasks.workunit.client.0.smithi107.stdout:3/972: chown d1/de/d18/d1f/l27 63708 1 2024-03-04T13:43:23.525 INFO:tasks.workunit.client.0.smithi107.stdout:7/888: stat d6/d1d/c73 0 2024-03-04T13:43:23.527 INFO:tasks.workunit.client.0.smithi107.stdout:0/940: mknod da/dc/dd/d3e/d26/d69/d6b/d115/c142 0 2024-03-04T13:43:23.532 INFO:tasks.workunit.client.0.smithi107.stdout:7/889: fsync d6/d13/d16/d19/d39/f54 0 2024-03-04T13:43:23.534 INFO:tasks.workunit.client.0.smithi107.stdout:4/907: dwrite d0/d9/de/f43 [0,4194304] 0 2024-03-04T13:43:23.538 INFO:tasks.workunit.client.0.smithi107.stdout:0/941: chown da/dc/dd/d1e/fe2 41 1 2024-03-04T13:43:23.541 INFO:tasks.workunit.client.0.smithi107.stdout:0/942: readlink da/d7e/le1 0 2024-03-04T13:43:23.544 INFO:tasks.workunit.client.0.smithi107.stdout:3/973: rename d1/de/d18/d1f/f20 to d1/d1e/d26/d9e/d10f/f13d 0 2024-03-04T13:43:23.545 INFO:tasks.workunit.client.0.smithi107.stdout:0/943: chown da/dc/dd/d3e/d26/l2b 1 1 2024-03-04T13:43:23.546 INFO:tasks.workunit.client.0.smithi107.stdout:9/837: dwrite d4/da/d11/d4c/d4d/d72/d7b/f113 [0,4194304] 0 2024-03-04T13:43:23.551 INFO:tasks.workunit.client.0.smithi107.stdout:0/944: read - da/dc/dd/d1e/dbf/f128 zero size 2024-03-04T13:43:23.556 INFO:tasks.workunit.client.0.smithi107.stdout:2/904: truncate da/df/d3e/f50 3770119 0 2024-03-04T13:43:23.556 INFO:tasks.workunit.client.0.smithi107.stdout:1/935: write d1/d46/d6a/f51 [4731178,59869] 0 2024-03-04T13:43:23.556 INFO:tasks.workunit.client.0.smithi107.stdout:9/838: rename d4/da/d11/d20/d64/l6d to d4/d32/l118 0 2024-03-04T13:43:23.557 INFO:tasks.workunit.client.0.smithi107.stdout:7/890: link d6/d13/d16/d19/d39/c107 d6/d1d/d64/d9a/da9/d71/db9/dca/d3d/dc6/dcb/dd6/de4/c124 0 2024-03-04T13:43:23.561 INFO:tasks.workunit.client.0.smithi107.stdout:1/936: dread - d1/d46/d6a/d4e/daa/dc9/d67/f10b zero size 2024-03-04T13:43:23.562 INFO:tasks.workunit.client.0.smithi107.stdout:2/905: mknod da/df/d3e/dfe/c133 0 2024-03-04T13:43:23.563 INFO:tasks.workunit.client.0.smithi107.stdout:7/891: dread - d6/d13/d14/f9d zero size 2024-03-04T13:43:23.572 INFO:tasks.workunit.client.0.smithi107.stdout:3/974: dwrite d1/de/d18/d1f/f94 [0,4194304] 0 2024-03-04T13:43:23.573 INFO:tasks.workunit.client.0.smithi107.stdout:1/937: unlink d1/d46/d6a/d4e/daa/dc9/d67/l8e 0 2024-03-04T13:43:23.576 INFO:tasks.workunit.client.0.smithi107.stdout:1/938: chown d1/d46/d6a/f51 21303 1 2024-03-04T13:43:23.576 INFO:tasks.workunit.client.0.smithi107.stdout:3/975: readlink d1/de/d18/d33/d48/d93/lb2 0 2024-03-04T13:43:23.581 INFO:tasks.workunit.client.0.smithi107.stdout:9/839: rename d4/da/d11/d20/d10f/f7c to d4/da/d11/d20/d2f/d9c/daa/db6/f119 0 2024-03-04T13:43:23.586 INFO:tasks.workunit.client.0.smithi107.stdout:3/976: creat d1/de/d11/dd5/f13e x:0 0 0 2024-03-04T13:43:23.591 INFO:tasks.workunit.client.0.smithi107.stdout:7/892: write d6/d13/d16/d1c/f48 [4784266,107972] 0 2024-03-04T13:43:23.592 INFO:tasks.workunit.client.0.smithi107.stdout:1/939: symlink d1/d46/d6a/d65/d106/d61/ddb/de6/d124/dba/l134 0 2024-03-04T13:43:23.597 INFO:tasks.workunit.client.0.smithi107.stdout:2/906: truncate da/d39/fc4 1063693 0 2024-03-04T13:43:23.598 INFO:tasks.workunit.client.0.smithi107.stdout:9/840: dread d4/da/d11/d20/d2f/d9c/daa/fc2 [0,4194304] 0 2024-03-04T13:43:23.598 INFO:tasks.workunit.client.0.smithi107.stdout:2/907: fdatasync da/d29/d7c/dbd/fda 0 2024-03-04T13:43:23.600 INFO:tasks.workunit.client.0.smithi107.stdout:1/940: rename d1/lb8 to d1/d46/d6a/d65/d106/d61/ddb/de6/d124/dba/l135 0 2024-03-04T13:43:23.606 INFO:tasks.workunit.client.0.smithi107.stdout:1/941: read d1/d10/d16/d1b/f3e [1310606,1823] 0 2024-03-04T13:43:23.608 INFO:tasks.workunit.client.0.smithi107.stdout:1/942: fsync d1/d46/d6a/d4e/daa/dc9/d33/d5d/fbf 0 2024-03-04T13:43:23.610 INFO:tasks.workunit.client.0.smithi107.stdout:9/841: creat d4/da/f11a x:0 0 0 2024-03-04T13:43:23.611 INFO:tasks.workunit.client.0.smithi107.stdout:4/908: dread d0/d9/de/d14/f4e [0,4194304] 0 2024-03-04T13:43:23.613 INFO:tasks.workunit.client.0.smithi107.stdout:9/842: chown d4/da/d11/d20/d2f/d9c/daa/fc3 3715 1 2024-03-04T13:43:23.615 INFO:tasks.workunit.client.0.smithi107.stdout:1/943: symlink d1/d10/d16/d1b/l136 0 2024-03-04T13:43:23.616 INFO:tasks.workunit.client.0.smithi107.stdout:3/977: truncate d1/de/f116 695007 0 2024-03-04T13:43:23.617 INFO:tasks.workunit.client.0.smithi107.stdout:9/843: fsync d4/da/d11/d20/d2f/d9c/fb4 0 2024-03-04T13:43:23.618 INFO:tasks.workunit.client.0.smithi107.stdout:3/978: symlink d1/d41/l13f 0 2024-03-04T13:43:23.622 INFO:tasks.workunit.client.0.smithi107.stdout:9/844: fsync d4/da/d11/d20/d2f/d9c/daa/fc3 0 2024-03-04T13:43:23.625 INFO:tasks.workunit.client.0.smithi107.stdout:3/979: symlink d1/de/d18/d33/d48/d71/d4c/d54/d12a/l140 0 2024-03-04T13:43:23.631 INFO:tasks.workunit.client.0.smithi107.stdout:9/845: write d4/da/d11/d20/d2f/d9c/daa/fc2 [1988068,130646] 0 2024-03-04T13:43:23.634 INFO:tasks.workunit.client.0.smithi107.stdout:1/944: dwrite d1/d46/d6a/d65/d106/f11e [0,4194304] 0 2024-03-04T13:43:23.641 INFO:tasks.workunit.client.0.smithi107.stdout:2/908: dread da/d39/f46 [0,4194304] 0 2024-03-04T13:43:23.647 INFO:tasks.workunit.client.0.smithi107.stdout:3/980: dwrite d1/de/d4b/f7a [0,4194304] 0 2024-03-04T13:43:23.647 INFO:tasks.workunit.client.0.smithi107.stdout:4/909: dread d0/d9/d18/faa [0,4194304] 0 2024-03-04T13:43:23.648 INFO:tasks.workunit.client.0.smithi107.stdout:2/909: mkdir da/df/d73/db3/db1/dee/d134 0 2024-03-04T13:43:23.649 INFO:tasks.workunit.client.0.smithi107.stdout:3/981: dread - d1/d1e/dae/df2/f119 zero size 2024-03-04T13:43:23.652 INFO:tasks.workunit.client.0.smithi107.stdout:4/910: dread - d0/d9/de/d14/d66/f8b zero size 2024-03-04T13:43:23.652 INFO:tasks.workunit.client.0.smithi107.stdout:3/982: fdatasync d1/de/d18/d33/d48/d71/d4c/fcd 0 2024-03-04T13:43:23.660 INFO:tasks.workunit.client.0.smithi107.stdout:4/911: mkdir d0/d9/d18/d29/d122 0 2024-03-04T13:43:23.663 INFO:tasks.workunit.client.0.smithi107.stdout:9/846: write d4/da/d11/d20/d64/f78 [5355378,126989] 0 2024-03-04T13:43:23.667 INFO:tasks.workunit.client.0.smithi107.stdout:9/847: dread - d4/da/d11/d20/d64/d77/ddc/f101 zero size 2024-03-04T13:43:23.682 INFO:tasks.workunit.client.0.smithi107.stdout:9/848: creat d4/da/d11/d20/d2f/df7/f11b x:0 0 0 2024-03-04T13:43:23.691 INFO:tasks.workunit.client.0.smithi107.stdout:4/912: dwrite d0/d9/d10/d80/db8/f9e [4194304,4194304] 0 2024-03-04T13:43:23.692 INFO:tasks.workunit.client.0.smithi107.stdout:2/910: dread da/d44/d8b/d9d/df8/d5c/f9e [0,4194304] 0 2024-03-04T13:43:23.696 INFO:tasks.workunit.client.0.smithi107.stdout:1/945: dread d1/d46/d6a/d4e/daa/dc9/d33/d5d/d24/f62 [4194304,4194304] 0 2024-03-04T13:43:23.699 INFO:tasks.workunit.client.0.smithi107.stdout:4/913: read d0/d9/d10/d80/db8/d97/fc6 [4357,48957] 0 2024-03-04T13:43:23.700 INFO:tasks.workunit.client.0.smithi107.stdout:9/849: dread d4/f26 [0,4194304] 0 2024-03-04T13:43:23.703 INFO:tasks.workunit.client.0.smithi107.stdout:1/946: creat d1/d46/d6a/d75/dfd/f137 x:0 0 0 2024-03-04T13:43:23.703 INFO:tasks.workunit.client.0.smithi107.stdout:9/850: stat d4/da/d11/d20/d2f/fd1 0 2024-03-04T13:43:23.705 INFO:tasks.workunit.client.0.smithi107.stdout:4/914: mkdir d0/d9/d10/d123 0 2024-03-04T13:43:23.710 INFO:tasks.workunit.client.0.smithi107.stdout:8/924: sync 2024-03-04T13:43:23.712 INFO:tasks.workunit.client.0.smithi107.stdout:4/915: creat d0/d2f/f124 x:0 0 0 2024-03-04T13:43:23.713 INFO:tasks.workunit.client.0.smithi107.stdout:8/925: stat d6/da/d25/dd1/de3/f102 0 2024-03-04T13:43:23.713 INFO:tasks.workunit.client.0.smithi107.stdout:9/851: creat d4/da/d11/d4c/d4d/d72/d7b/f11c x:0 0 0 2024-03-04T13:43:23.715 INFO:tasks.workunit.client.0.smithi107.stdout:1/947: rename d1/d46/d6a/d17/f5a to d1/d10/d16/d47/d7a/f138 0 2024-03-04T13:43:23.721 INFO:tasks.workunit.client.0.smithi107.stdout:9/852: chown d4/da/d11/d20/d23/d7e/d84/d107 43036939 1 2024-03-04T13:43:23.722 INFO:tasks.workunit.client.0.smithi107.stdout:1/948: creat d1/d10/d16/da9/f139 x:0 0 0 2024-03-04T13:43:23.723 INFO:tasks.workunit.client.0.smithi107.stdout:4/916: truncate d0/d9/d10/d80/db8/fdd 756551 0 2024-03-04T13:43:23.727 INFO:tasks.workunit.client.0.smithi107.stdout:1/949: chown d1/d10/d16/d47/d7a/d128/d129 2942628 1 2024-03-04T13:43:23.732 INFO:tasks.workunit.client.0.smithi107.stdout:4/917: rmdir d0/d9/d10/d80/db8/d9a 39 2024-03-04T13:43:23.736 INFO:tasks.workunit.client.0.smithi107.stdout:1/950: creat d1/d10/d16/da9/f13a x:0 0 0 2024-03-04T13:43:23.736 INFO:tasks.workunit.client.0.smithi107.stdout:9/853: dread d4/da/d11/d20/d2f/d9c/ffc [0,4194304] 0 2024-03-04T13:43:23.738 INFO:tasks.workunit.client.0.smithi107.stdout:4/918: write d0/d9/d10/d80/db8/f115 [395537,6521] 0 2024-03-04T13:43:23.741 INFO:tasks.workunit.client.0.smithi107.stdout:4/919: fdatasync d0/d9/de/f62 0 2024-03-04T13:43:23.749 INFO:tasks.workunit.client.0.smithi107.stdout:8/926: dwrite d6/da/d25/d2a/f75 [0,4194304] 0 2024-03-04T13:43:23.749 INFO:tasks.workunit.client.0.smithi107.stdout:9/854: rename d4/da/f15 to d4/da/d11/d4c/d4d/d72/d7b/f11d 0 2024-03-04T13:43:23.753 INFO:tasks.workunit.client.0.smithi107.stdout:9/855: readlink d4/da/d11/d20/d64/d77/l8e 0 2024-03-04T13:43:23.761 INFO:tasks.workunit.client.0.smithi107.stdout:9/856: creat d4/da/d11/d20/d64/f11e x:0 0 0 2024-03-04T13:43:23.761 INFO:tasks.workunit.client.0.smithi107.stdout:2/911: dread da/f30 [0,4194304] 0 2024-03-04T13:43:23.761 INFO:tasks.workunit.client.0.smithi107.stdout:8/927: truncate d6/da/d25/d2a/f95 704146 0 2024-03-04T13:43:23.762 INFO:tasks.workunit.client.0.smithi107.stdout:1/951: write d1/d46/d6a/d65/db1/f127 [552212,78010] 0 2024-03-04T13:43:23.766 INFO:tasks.workunit.client.0.smithi107.stdout:2/912: symlink da/df/d3e/l135 0 2024-03-04T13:43:23.767 INFO:tasks.workunit.client.0.smithi107.stdout:4/920: dwrite d0/d9/d18/fa9 [0,4194304] 0 2024-03-04T13:43:23.771 INFO:tasks.workunit.client.0.smithi107.stdout:1/952: symlink d1/d38/d133/l13b 0 2024-03-04T13:43:23.775 INFO:tasks.workunit.client.0.smithi107.stdout:2/913: creat da/df/d73/db3/db1/df2/f136 x:0 0 0 2024-03-04T13:43:23.776 INFO:tasks.workunit.client.0.smithi107.stdout:4/921: mkdir d0/d9/d10/d80/db8/d125 0 2024-03-04T13:43:23.777 INFO:tasks.workunit.client.0.smithi107.stdout:1/953: truncate d1/d46/d6a/d4e/daa/f10e 561204 0 2024-03-04T13:43:23.781 INFO:tasks.workunit.client.0.smithi107.stdout:1/954: write d1/d46/d6a/d65/db1/f127 [1517440,51270] 0 2024-03-04T13:43:23.781 INFO:tasks.workunit.client.0.smithi107.stdout:9/857: link d4/da/d11/d4c/d51/cff d4/da/d11/d4c/d4d/d72/c11f 0 2024-03-04T13:43:23.781 INFO:tasks.workunit.client.0.smithi107.stdout:2/914: creat da/f137 x:0 0 0 2024-03-04T13:43:23.784 INFO:tasks.workunit.client.0.smithi107.stdout:4/922: fsync d0/d9/d10/d80/db8/f27 0 2024-03-04T13:43:23.784 INFO:tasks.workunit.client.0.smithi107.stdout:8/928: write d6/d16/d69/deb/d3e/f56 [2356715,43862] 0 2024-03-04T13:43:23.793 INFO:tasks.workunit.client.0.smithi107.stdout:9/858: mkdir d4/da/d11/d20/d23/dcc/d120 0 2024-03-04T13:43:23.802 INFO:tasks.workunit.client.0.smithi107.stdout:1/955: dwrite d1/d46/d6a/d65/d106/f117 [0,4194304] 0 2024-03-04T13:43:23.812 INFO:tasks.workunit.client.0.smithi107.stdout:4/923: dread d0/d9/d18/f57 [0,4194304] 0 2024-03-04T13:43:23.814 INFO:tasks.workunit.client.0.smithi107.stdout:6/912: sync 2024-03-04T13:43:23.816 INFO:tasks.workunit.client.0.smithi107.stdout:6/913: chown df/d2e/d36/c53 128610 1 2024-03-04T13:43:23.911 INFO:tasks.workunit.client.0.smithi107.stdout:6/914: dread - df/d2e/d36/d77/d95/da7/fcb zero size 2024-03-04T13:43:23.923 INFO:tasks.workunit.client.0.smithi107.stdout:7/893: sync 2024-03-04T13:43:23.923 INFO:tasks.workunit.client.0.smithi107.stdout:0/945: sync 2024-03-04T13:43:23.923 INFO:tasks.workunit.client.0.smithi107.stdout:3/983: sync 2024-03-04T13:43:23.928 INFO:tasks.workunit.client.0.smithi107.stdout:0/946: stat da/c40 0 2024-03-04T13:43:23.931 INFO:tasks.workunit.client.0.smithi107.stdout:0/947: dread - da/dc/dd/d3e/d26/d69/d6b/f124 zero size 2024-03-04T13:43:23.932 INFO:tasks.workunit.client.0.smithi107.stdout:3/984: creat d1/de/d18/d33/d48/d71/d4c/d54/db7/f141 x:0 0 0 2024-03-04T13:43:23.934 INFO:tasks.workunit.client.0.smithi107.stdout:7/894: creat d6/d1d/d64/d9a/da9/d71/db9/dca/d3d/dc6/dcb/dd6/f125 x:0 0 0 2024-03-04T13:43:23.934 INFO:tasks.workunit.client.0.smithi107.stdout:0/948: chown da/dc/dd/d3e/d26/d88 93 1 2024-03-04T13:43:23.956 INFO:tasks.workunit.client.0.smithi107.stdout:7/895: dwrite d6/d1d/d64/d9a/da9/d71/db9/dca/dd0/f105 [0,4194304] 0 2024-03-04T13:43:24.003 INFO:tasks.workunit.client.0.smithi107.stdout:1/956: sync 2024-03-04T13:43:24.011 INFO:tasks.workunit.client.0.smithi107.stdout:4/924: truncate d0/d9/d10/d80/db8/d97/fc6 20267 0 2024-03-04T13:43:24.014 INFO:tasks.workunit.client.0.smithi107.stdout:1/957: unlink d1/d46/d6a/d65/db1/cc0 0 2024-03-04T13:43:24.129 INFO:tasks.workunit.client.0.smithi107.stdout:4/925: read d0/d2f/f106 [1880137,88109] 0 2024-03-04T13:43:24.130 INFO:tasks.workunit.client.0.smithi107.stdout:7/896: unlink d6/l92 0 2024-03-04T13:43:24.132 INFO:tasks.workunit.client.0.smithi107.stdout:6/915: rmdir df/d2e/d36/d77/d106/d7e 39 2024-03-04T13:43:24.139 INFO:tasks.workunit.client.0.smithi107.stdout:8/929: dwrite d6/d16/d1f/d4c/da4/daa/fca [0,4194304] 0 2024-03-04T13:43:24.140 INFO:tasks.workunit.client.0.smithi107.stdout:4/926: dread - d0/d9/d1e/d70/f114 zero size 2024-03-04T13:43:24.143 INFO:tasks.workunit.client.0.smithi107.stdout:9/859: dwrite d4/da/d11/d4c/d4d/d72/d7b/fc0 [0,4194304] 0 2024-03-04T13:43:24.152 INFO:tasks.workunit.client.0.smithi107.stdout:6/916: creat df/d17/d24/d40/d63/d11c/f124 x:0 0 0 2024-03-04T13:43:24.158 INFO:tasks.workunit.client.0.smithi107.stdout:7/897: creat d6/d1d/d64/d9a/da9/f126 x:0 0 0 2024-03-04T13:43:24.159 INFO:tasks.workunit.client.0.smithi107.stdout:9/860: fsync d4/da/d11/d20/d23/fc1 0 2024-03-04T13:43:24.165 INFO:tasks.workunit.client.0.smithi107.stdout:2/915: dwrite da/f12b [0,4194304] 0 2024-03-04T13:43:24.177 INFO:tasks.workunit.client.0.smithi107.stdout:7/898: getdents d6/d1d 0 2024-03-04T13:43:24.182 INFO:tasks.workunit.client.0.smithi107.stdout:2/916: unlink da/df/d85/f132 0 2024-03-04T13:43:24.184 INFO:tasks.workunit.client.0.smithi107.stdout:6/917: dwrite f6 [0,4194304] 0 2024-03-04T13:43:24.190 INFO:tasks.workunit.client.0.smithi107.stdout:1/958: dwrite d1/d46/d6a/d65/d106/d61/ddb/de6/d124/f114 [4194304,4194304] 0 2024-03-04T13:43:24.193 INFO:tasks.workunit.client.0.smithi107.stdout:6/918: chown df/d17/d6b/d9b/d61/d99/ff2 1725940 1 2024-03-04T13:43:24.195 INFO:tasks.workunit.client.0.smithi107.stdout:2/917: mkdir da/d44/d8b/d9d/dd0/d138 0 2024-03-04T13:43:24.198 INFO:tasks.workunit.client.0.smithi107.stdout:6/919: symlink df/d17/d6b/d9b/d61/l125 0 2024-03-04T13:43:24.200 INFO:tasks.workunit.client.0.smithi107.stdout:2/918: creat da/df/d73/db3/db1/dee/f139 x:0 0 0 2024-03-04T13:43:24.201 INFO:tasks.workunit.client.0.smithi107.stdout:6/920: chown df/d17/d5c/fb0 129355 1 2024-03-04T13:43:24.201 INFO:tasks.workunit.client.0.smithi107.stdout:1/959: mkdir d1/d10/d16/d1b/d13c 0 2024-03-04T13:43:24.202 INFO:tasks.workunit.client.0.smithi107.stdout:8/930: write d6/d16/d1f/d63/f8b [305301,26065] 0 2024-03-04T13:43:24.205 INFO:tasks.workunit.client.0.smithi107.stdout:2/919: creat da/df/f13a x:0 0 0 2024-03-04T13:43:24.207 INFO:tasks.workunit.client.0.smithi107.stdout:6/921: truncate df/d2e/f11a 534492 0 2024-03-04T13:43:24.208 INFO:tasks.workunit.client.0.smithi107.stdout:8/931: fsync d6/d16/d1f/d63/d73/fa2 0 2024-03-04T13:43:24.211 INFO:tasks.workunit.client.0.smithi107.stdout:6/922: dread - df/d17/d24/d30/d5d/d6a/f94 zero size 2024-03-04T13:43:24.211 INFO:tasks.workunit.client.0.smithi107.stdout:8/932: chown d6/da/d25/dd1 9762161 1 2024-03-04T13:43:24.214 INFO:tasks.workunit.client.0.smithi107.stdout:1/960: symlink d1/d10/d16/d47/d7a/d128/l13d 0 2024-03-04T13:43:24.214 INFO:tasks.workunit.client.0.smithi107.stdout:9/861: dread d4/f8 [0,4194304] 0 2024-03-04T13:43:24.215 INFO:tasks.workunit.client.0.smithi107.stdout:6/923: readlink df/d17/d24/d30/d5d/l6e 0 2024-03-04T13:43:24.216 INFO:tasks.workunit.client.0.smithi107.stdout:2/920: chown da/d39/f46 8019 1 2024-03-04T13:43:24.216 INFO:tasks.workunit.client.0.smithi107.stdout:3/985: sync 2024-03-04T13:43:24.220 INFO:tasks.workunit.client.0.smithi107.stdout:8/933: mknod d6/da/da8/db5/c128 0 2024-03-04T13:43:24.222 INFO:tasks.workunit.client.0.smithi107.stdout:2/921: readlink da/d44/d8b/d98/l110 0 2024-03-04T13:43:24.223 INFO:tasks.workunit.client.0.smithi107.stdout:1/961: mkdir d1/d46/d6a/d4e/daa/dc9/d33/d5d/d4c/d13e 0 2024-03-04T13:43:24.225 INFO:tasks.workunit.client.0.smithi107.stdout:2/922: write da/df/f13a [707139,87458] 0 2024-03-04T13:43:24.227 INFO:tasks.workunit.client.0.smithi107.stdout:2/923: chown da/d39/d122 3075300 1 2024-03-04T13:43:24.231 INFO:tasks.workunit.client.0.smithi107.stdout:1/962: chown d1/dd6 9550 1 2024-03-04T13:43:24.231 INFO:tasks.workunit.client.0.smithi107.stdout:1/963: readlink d1/d46/ld7 0 2024-03-04T13:43:24.231 INFO:tasks.workunit.client.0.smithi107.stdout:1/964: readlink d1/d46/l74 0 2024-03-04T13:43:24.233 INFO:tasks.workunit.client.0.smithi107.stdout:8/934: rename d6/d16/fcf to d6/d16/d69/deb/d3e/d111/f129 0 2024-03-04T13:43:24.236 INFO:tasks.workunit.client.0.smithi107.stdout:9/862: mkdir d4/da/d11/d20/d2f/d9c/d92/df6/d121 0 2024-03-04T13:43:24.239 INFO:tasks.workunit.client.0.smithi107.stdout:1/965: mkdir d1/d46/d6a/d4e/daa/dc9/d33/dbd/d13f 0 2024-03-04T13:43:24.239 INFO:tasks.workunit.client.0.smithi107.stdout:9/863: read d4/da/d11/d20/d39/f7a [4044116,12227] 0 2024-03-04T13:43:24.240 INFO:tasks.workunit.client.0.smithi107.stdout:8/935: symlink d6/d16/d69/deb/d3e/d67/l12a 0 2024-03-04T13:43:24.243 INFO:tasks.workunit.client.0.smithi107.stdout:6/924: dwrite df/d2e/d36/d77/d106/f68 [0,4194304] 0 2024-03-04T13:43:24.247 INFO:tasks.workunit.client.0.smithi107.stdout:7/899: dread d6/d13/d16/d19/d38/d37/f9e [0,4194304] 0 2024-03-04T13:43:24.249 INFO:tasks.workunit.client.0.smithi107.stdout:9/864: stat d4/da/d11/de1 0 2024-03-04T13:43:24.255 INFO:tasks.workunit.client.0.smithi107.stdout:2/924: dwrite da/df/d73/dd1/d100/f107 [0,4194304] 0 2024-03-04T13:43:24.255 INFO:tasks.workunit.client.0.smithi107.stdout:3/986: dread d1/de/d11/f46 [4194304,4194304] 0 2024-03-04T13:43:24.259 INFO:tasks.workunit.client.0.smithi107.stdout:9/865: write d4/da/d11/d20/d64/d77/f82 [390897,98746] 0 2024-03-04T13:43:24.265 INFO:tasks.workunit.client.0.smithi107.stdout:1/966: mknod d1/d46/d6a/d4e/daa/dc9/d33/d5d/d4c/c140 0 2024-03-04T13:43:24.267 INFO:tasks.workunit.client.0.smithi107.stdout:8/936: symlink d6/d16/d1f/d4c/da4/l12b 0 2024-03-04T13:43:24.269 INFO:tasks.workunit.client.0.smithi107.stdout:7/900: truncate d6/d1d/d64/d9a/fc0 728118 0 2024-03-04T13:43:24.270 INFO:tasks.workunit.client.0.smithi107.stdout:3/987: readlink d1/d41/l7d 0 2024-03-04T13:43:24.272 INFO:tasks.workunit.client.0.smithi107.stdout:2/925: rmdir da/df/d27/d10b 39 2024-03-04T13:43:24.272 INFO:tasks.workunit.client.0.smithi107.stdout:6/925: dread - df/d2e/d36/d77/d106/d7e/d86/fce zero size 2024-03-04T13:43:24.275 INFO:tasks.workunit.client.0.smithi107.stdout:1/967: symlink d1/d46/d6a/d75/dfd/l141 0 2024-03-04T13:43:24.276 INFO:tasks.workunit.client.0.smithi107.stdout:8/937: write d6/d16/d69/deb/d3e/d67/db7/f103 [413751,19908] 0 2024-03-04T13:43:24.278 INFO:tasks.workunit.client.0.smithi107.stdout:7/901: symlink d6/d13/d16/d19/d38/d2c/d8d/l127 0 2024-03-04T13:43:24.279 INFO:tasks.workunit.client.0.smithi107.stdout:3/988: creat d1/de/d11/dd5/dab/dc5/f142 x:0 0 0 2024-03-04T13:43:24.280 INFO:tasks.workunit.client.0.smithi107.stdout:9/866: dwrite d4/da/d11/d20/d64/d77/ddc/f101 [0,4194304] 0 2024-03-04T13:43:24.281 INFO:tasks.workunit.client.0.smithi107.stdout:6/926: mknod df/d2e/d36/d77/d106/dab/c126 0 2024-03-04T13:43:24.283 INFO:tasks.workunit.client.0.smithi107.stdout:3/989: chown d1/de/d11/dd5/dab/dc5 6229 1 2024-03-04T13:43:24.286 INFO:tasks.workunit.client.0.smithi107.stdout:9/867: chown d4/da/l10 239 1 2024-03-04T13:43:24.289 INFO:tasks.workunit.client.0.smithi107.stdout:3/990: fdatasync d1/de/d18/d33/d48/d71/d4c/f13a 0 2024-03-04T13:43:24.292 INFO:tasks.workunit.client.0.smithi107.stdout:0/949: sync 2024-03-04T13:43:24.292 INFO:tasks.workunit.client.0.smithi107.stdout:4/927: sync 2024-03-04T13:43:24.293 INFO:tasks.workunit.client.0.smithi107.stdout:8/938: truncate d6/d16/d69/deb/d3e/d67/f6f 522040 0 2024-03-04T13:43:24.293 INFO:tasks.workunit.client.0.smithi107.stdout:2/926: link da/d39/c92 da/d38/d3a/d5d/c13b 0 2024-03-04T13:43:24.294 INFO:tasks.workunit.client.0.smithi107.stdout:6/927: creat df/d2e/d36/f127 x:0 0 0 2024-03-04T13:43:24.296 INFO:tasks.workunit.client.0.smithi107.stdout:8/939: stat d6/da/da8/db5/fd6 0 2024-03-04T13:43:24.296 INFO:tasks.workunit.client.0.smithi107.stdout:1/968: dread d1/d46/d6a/d4e/fb9 [0,4194304] 0 2024-03-04T13:43:24.297 INFO:tasks.workunit.client.0.smithi107.stdout:8/940: stat d6/f26 0 2024-03-04T13:43:24.299 INFO:tasks.workunit.client.0.smithi107.stdout:7/902: rename d6/d13/d16/d19/d38/d37/f9e to d6/d1d/d64/d9a/da9/d71/d115/f128 0 2024-03-04T13:43:24.299 INFO:tasks.workunit.client.0.smithi107.stdout:3/991: mkdir d1/de/d18/d33/d4d/d143 0 2024-03-04T13:43:24.300 INFO:tasks.workunit.client.0.smithi107.stdout:4/928: mknod d0/d9/d10/dab/c126 0 2024-03-04T13:43:24.300 INFO:tasks.workunit.client.0.smithi107.stdout:8/941: chown d6/d16/d69 0 1 2024-03-04T13:43:24.305 INFO:tasks.workunit.client.0.smithi107.stdout:6/928: symlink df/d1b/l128 0 2024-03-04T13:43:24.306 INFO:tasks.workunit.client.0.smithi107.stdout:3/992: chown d1/de/d18/f59 0 1 2024-03-04T13:43:24.309 INFO:tasks.workunit.client.0.smithi107.stdout:6/929: read - df/d2e/ff9 zero size 2024-03-04T13:43:24.310 INFO:tasks.workunit.client.0.smithi107.stdout:2/927: write da/d44/d8b/d9d/df8/fe7 [3729577,67178] 0 2024-03-04T13:43:24.311 INFO:tasks.workunit.client.0.smithi107.stdout:4/929: creat d0/d2f/f127 x:0 0 0 2024-03-04T13:43:24.315 INFO:tasks.workunit.client.0.smithi107.stdout:8/942: creat d6/da/d25/dd1/d101/f12c x:0 0 0 2024-03-04T13:43:24.315 INFO:tasks.workunit.client.0.smithi107.stdout:9/868: dwrite d4/da/d11/d20/f105 [0,4194304] 0 2024-03-04T13:43:24.316 INFO:tasks.workunit.client.0.smithi107.stdout:7/903: write d6/d13/d14/f5d [3809024,20405] 0 2024-03-04T13:43:24.317 INFO:tasks.workunit.client.0.smithi107.stdout:3/993: read d1/de/d18/d33/d6b/fb1 [2991044,24185] 0 2024-03-04T13:43:24.325 INFO:tasks.workunit.client.0.smithi107.stdout:8/943: mknod d6/d16/d1f/d4c/da5/de9/c12d 0 2024-03-04T13:43:24.327 INFO:tasks.workunit.client.0.smithi107.stdout:2/928: creat da/df/d73/db3/db1/df2/f13c x:0 0 0 2024-03-04T13:43:24.329 INFO:tasks.workunit.client.0.smithi107.stdout:4/930: rmdir d0/d9/d18/d29/dfc 39 2024-03-04T13:43:24.330 INFO:tasks.workunit.client.0.smithi107.stdout:7/904: creat d6/d13/d16/f129 x:0 0 0 2024-03-04T13:43:24.331 INFO:tasks.workunit.client.0.smithi107.stdout:6/930: dread df/d17/d24/d30/d5d/d6a/fe3 [0,4194304] 0 2024-03-04T13:43:24.331 INFO:tasks.workunit.client.0.smithi107.stdout:3/994: mkdir d1/de/d11/dd5/dab/d144 0 2024-03-04T13:43:24.334 INFO:tasks.workunit.client.0.smithi107.stdout:0/950: dread da/dc/dd/d3e/f7a [0,4194304] 0 2024-03-04T13:43:24.336 INFO:tasks.workunit.client.0.smithi107.stdout:3/995: readlink d1/d41/l104 0 2024-03-04T13:43:24.337 INFO:tasks.workunit.client.0.smithi107.stdout:7/905: rmdir d6/d13/d16/d19/d60 39 2024-03-04T13:43:24.339 INFO:tasks.workunit.client.0.smithi107.stdout:4/931: mkdir d0/d9/d10/d128 0 2024-03-04T13:43:24.340 INFO:tasks.workunit.client.0.smithi107.stdout:2/929: rename da/df/d3e/fcf to da/d39/f13d 0 2024-03-04T13:43:24.343 INFO:tasks.workunit.client.0.smithi107.stdout:6/931: mkdir df/d17/d24/d30/d8b/dc1/de1/d129 0 2024-03-04T13:43:24.346 INFO:tasks.workunit.client.0.smithi107.stdout:3/996: creat d1/d1e/daf/f145 x:0 0 0 2024-03-04T13:43:24.347 INFO:tasks.workunit.client.0.smithi107.stdout:1/969: dread d1/d46/d6a/d4e/d60/f78 [0,4194304] 0 2024-03-04T13:43:24.348 INFO:tasks.workunit.client.0.smithi107.stdout:9/869: truncate d4/da/f4a 3060696 0 2024-03-04T13:43:24.353 INFO:tasks.workunit.client.0.smithi107.stdout:0/951: rename da/dc/ldd to da/dc/dd/dc9/l143 0 2024-03-04T13:43:24.355 INFO:tasks.workunit.client.0.smithi107.stdout:6/932: readlink df/d2e/d36/d5b/d7d/dfc/l9f 0 2024-03-04T13:43:24.357 INFO:tasks.workunit.client.0.smithi107.stdout:0/952: truncate da/dc/dd/ddb/f13c 668338 0 2024-03-04T13:43:24.357 INFO:tasks.workunit.client.0.smithi107.stdout:7/906: mknod d6/d1d/dd9/d114/c12a 0 2024-03-04T13:43:24.358 INFO:tasks.workunit.client.0.smithi107.stdout:6/933: chown df/d2e/d36/d77/d118 158236 1 2024-03-04T13:43:24.362 INFO:tasks.workunit.client.0.smithi107.stdout:3/997: read d1/de/d18/d1f/d42/f5d [2160945,5115] 0 2024-03-04T13:43:24.364 INFO:tasks.workunit.client.0.smithi107.stdout:9/870: rename d4/da/d11/d20/d64/d77/de7 to d4/da/d11/d4c/d51/d122 0 2024-03-04T13:43:24.368 INFO:tasks.workunit.client.0.smithi107.stdout:8/944: dread d6/d16/d69/deb/d3e/d111/f129 [0,4194304] 0 2024-03-04T13:43:24.370 INFO:tasks.workunit.client.0.smithi107.stdout:0/953: dread da/dc/dd/d1e/f112 [0,4194304] 0 2024-03-04T13:43:24.371 INFO:tasks.workunit.client.0.smithi107.stdout:9/871: symlink d4/d32/l123 0 2024-03-04T13:43:24.375 INFO:tasks.workunit.client.0.smithi107.stdout:8/945: dread - d6/da/d25/dd1/d101/f12c zero size 2024-03-04T13:43:24.375 INFO:tasks.workunit.client.0.smithi107.stdout:3/998: symlink d1/de/l146 0 2024-03-04T13:43:24.377 INFO:tasks.workunit.client.0.smithi107.stdout:4/932: dwrite d0/d9/d78/fa7 [0,4194304] 0 2024-03-04T13:43:24.378 INFO:tasks.workunit.client.0.smithi107.stdout:0/954: unlink da/dc/dd/d3e/f10f 0 2024-03-04T13:43:24.380 INFO:tasks.workunit.client.0.smithi107.stdout:2/930: dwrite da/df/f15 [4194304,4194304] 0 2024-03-04T13:43:24.387 INFO:tasks.workunit.client.0.smithi107.stdout:6/934: dwrite df/d17/d24/d30/d5d/d6a/fe3 [0,4194304] 0 2024-03-04T13:43:24.387 INFO:tasks.workunit.client.0.smithi107.stdout:0/955: rmdir da/dc/d34/d10d 39 2024-03-04T13:43:24.388 INFO:tasks.workunit.client.0.smithi107.stdout:1/970: dread d1/d46/d6a/d65/db1/ff4 [0,4194304] 0 2024-03-04T13:43:24.390 INFO:tasks.workunit.client.0.smithi107.stdout:4/933: rename d0/d2f/fc7 to d0/d2f/d7d/f129 0 2024-03-04T13:43:24.393 INFO:tasks.workunit.client.0.smithi107.stdout:9/872: link d4/da/d11/cb5 d4/da/d11/d20/d64/d77/ddc/c124 0 2024-03-04T13:43:24.396 INFO:tasks.workunit.client.0.smithi107.stdout:3/999: truncate d1/de/d18/d33/d48/d71/d4c/d54/fa7 1807957 0 2024-03-04T13:43:24.397 INFO:tasks.workunit.client.0.smithi107.stdout:1/971: fdatasync d1/d46/d6a/d4e/daa/dc9/d33/d5d/fbf 0 2024-03-04T13:43:24.397 INFO:tasks.workunit.client.0.smithi107.stdout:4/934: creat d0/d9/d10/d123/f12a x:0 0 0 2024-03-04T13:43:24.398 INFO:tasks.workunit.client.0.smithi107.stdout:0/956: mknod da/dc/dd/c144 0 2024-03-04T13:43:24.401 INFO:tasks.workunit.client.0.smithi107.stdout:9/873: mknod d4/da/d11/d20/d64/d77/df9/c125 0 2024-03-04T13:43:24.406 INFO:tasks.workunit.client.0.smithi107.stdout:1/972: creat d1/d38/f142 x:0 0 0 2024-03-04T13:43:24.408 INFO:tasks.workunit.client.0.smithi107.stdout:4/935: chown d0/d9/d10/d80/db8/d9a/df9/lfa 12488 1 2024-03-04T13:43:24.410 INFO:tasks.workunit.client.0.smithi107.stdout:4/936: fsync d0/d9/fd9 0 2024-03-04T13:43:24.416 INFO:tasks.workunit.client.0.smithi107.stdout:6/935: link df/d17/d5c/c7f df/d17/d24/d40/d63/d11c/c12a 0 2024-03-04T13:43:24.418 INFO:tasks.workunit.client.0.smithi107.stdout:4/937: mkdir d0/d9/d10/dab/d12b 0 2024-03-04T13:43:24.421 INFO:tasks.workunit.client.0.smithi107.stdout:4/938: write d0/d9/d10/d6f/fad [63705,71185] 0 2024-03-04T13:43:24.424 INFO:tasks.workunit.client.0.smithi107.stdout:4/939: stat d0/d9/de/d14/d66/fa5 0 2024-03-04T13:43:24.424 INFO:tasks.workunit.client.0.smithi107.stdout:6/936: creat df/d17/d24/d40/d63/f12b x:0 0 0 2024-03-04T13:43:24.426 INFO:tasks.workunit.client.0.smithi107.stdout:2/931: dread da/d34/f36 [0,4194304] 0 2024-03-04T13:43:24.427 INFO:tasks.workunit.client.0.smithi107.stdout:9/874: getdents d4/da/d11/d20/d23/dcc/de4 0 2024-03-04T13:43:24.429 INFO:tasks.workunit.client.0.smithi107.stdout:9/875: dread - d4/f112 zero size 2024-03-04T13:43:24.429 INFO:tasks.workunit.client.0.smithi107.stdout:0/957: rename da/dc/dd/d3e/d26/d88/lcd to da/dc/dd/dc9/de5/d107/l145 0 2024-03-04T13:43:24.433 INFO:tasks.workunit.client.0.smithi107.stdout:8/946: dread d6/d16/f27 [4194304,4194304] 0 2024-03-04T13:43:24.436 INFO:tasks.workunit.client.0.smithi107.stdout:0/958: symlink da/dc/dd/dc9/de5/d107/l146 0 2024-03-04T13:43:24.439 INFO:tasks.workunit.client.0.smithi107.stdout:2/932: symlink da/d44/l13e 0 2024-03-04T13:43:24.442 INFO:tasks.workunit.client.0.smithi107.stdout:4/940: dwrite d0/d9/d78/f85 [0,4194304] 0 2024-03-04T13:43:24.445 INFO:tasks.workunit.client.0.smithi107.stdout:4/941: truncate d0/d9/fd1 1820955 0 2024-03-04T13:43:24.449 INFO:tasks.workunit.client.0.smithi107.stdout:8/947: dwrite d6/d16/d1f/f34 [0,4194304] 0 2024-03-04T13:43:24.451 INFO:tasks.workunit.client.0.smithi107.stdout:0/959: mknod da/dc/dd/d3e/d26/c147 0 2024-03-04T13:43:24.453 INFO:tasks.workunit.client.0.smithi107.stdout:8/948: chown d6/d16/f24 224095 1 2024-03-04T13:43:24.454 INFO:tasks.workunit.client.0.smithi107.stdout:8/949: write d6/f9f [496960,111387] 0 2024-03-04T13:43:24.456 INFO:tasks.workunit.client.0.smithi107.stdout:8/950: chown d6/da/d25/dd1/de3/c115 247739750 1 2024-03-04T13:43:24.458 INFO:tasks.workunit.client.0.smithi107.stdout:6/937: dread df/d2e/d36/f49 [0,4194304] 0 2024-03-04T13:43:24.461 INFO:tasks.workunit.client.0.smithi107.stdout:0/960: rename da/dc/f4e to da/dc/dd/d1e/dbf/deb/f148 0 2024-03-04T13:43:24.479 INFO:tasks.workunit.client.0.smithi107.stdout:2/933: write da/f10 [679249,77187] 0 2024-03-04T13:43:24.479 INFO:tasks.workunit.client.0.smithi107.stdout:4/942: truncate d0/d9/d78/f85 91174 0 2024-03-04T13:43:24.480 INFO:tasks.workunit.client.0.smithi107.stdout:2/934: chown da/df/d73/daa/fb2 1659 1 2024-03-04T13:43:24.481 INFO:tasks.workunit.client.0.smithi107.stdout:8/951: write d6/d16/d1f/d4c/da4/daa/fd9 [3681894,48084] 0 2024-03-04T13:43:24.481 INFO:tasks.workunit.client.0.smithi107.stdout:6/938: write df/d2e/d36/d5b/f93 [4399414,26666] 0 2024-03-04T13:43:24.481 INFO:tasks.workunit.client.0.smithi107.stdout:0/961: truncate da/dc/dd/d3e/d26/d88/dd2/d73/d78/fcf 2410749 0 2024-03-04T13:43:24.483 INFO:tasks.workunit.client.0.smithi107.stdout:4/943: fsync d0/d9/d10/d80/db8/d21/f2c 0 2024-03-04T13:43:24.484 INFO:tasks.workunit.client.0.smithi107.stdout:2/935: fsync da/d44/d8b/d9d/df8/fa4 0 2024-03-04T13:43:24.485 INFO:tasks.workunit.client.0.smithi107.stdout:6/939: dread - df/d2e/d36/f127 zero size 2024-03-04T13:43:24.486 INFO:tasks.workunit.client.0.smithi107.stdout:8/952: write d6/d16/d1f/d4c/da4/fda [2167777,51960] 0 2024-03-04T13:43:24.493 INFO:tasks.workunit.client.0.smithi107.stdout:2/936: write da/d44/f52 [3243333,7164] 0 2024-03-04T13:43:24.495 INFO:tasks.workunit.client.0.smithi107.stdout:4/944: creat d0/d9/d10/d6f/f12c x:0 0 0 2024-03-04T13:43:24.498 INFO:tasks.workunit.client.0.smithi107.stdout:0/962: mknod da/dc/dd/d3e/d26/d69/de9/d140/c149 0 2024-03-04T13:43:24.499 INFO:tasks.workunit.client.0.smithi107.stdout:2/937: fsync da/f18 0 2024-03-04T13:43:24.500 INFO:tasks.workunit.client.0.smithi107.stdout:2/938: fsync da/df/d3e/f97 0 2024-03-04T13:43:24.501 INFO:tasks.workunit.client.0.smithi107.stdout:6/940: mkdir df/d2e/d36/d77/d118/dd8/d12c 0 2024-03-04T13:43:24.503 INFO:tasks.workunit.client.0.smithi107.stdout:4/945: mknod d0/d9/d10/d80/db8/d97/de3/c12d 0 2024-03-04T13:43:24.503 INFO:tasks.workunit.client.0.smithi107.stdout:6/941: readlink df/d2e/d36/d5b/d7d/dbd/lfa 0 2024-03-04T13:43:24.505 INFO:tasks.workunit.client.0.smithi107.stdout:4/946: chown d0/d9/d10/d6f/f119 12532 1 2024-03-04T13:43:24.506 INFO:tasks.workunit.client.0.smithi107.stdout:6/942: symlink df/d2e/d36/d77/l12d 0 2024-03-04T13:43:24.507 INFO:tasks.workunit.client.0.smithi107.stdout:8/953: getdents d6/d16/d1f/d4c/da4 0 2024-03-04T13:43:24.508 INFO:tasks.workunit.client.0.smithi107.stdout:2/939: mkdir da/d29/d13f 0 2024-03-04T13:43:24.510 INFO:tasks.workunit.client.0.smithi107.stdout:2/940: read - da/d44/d8b/d98/dc3/ff6 zero size 2024-03-04T13:43:24.510 INFO:tasks.workunit.client.0.smithi107.stdout:6/943: mkdir df/d2e/d36/d77/d118/deb/d12e 0 2024-03-04T13:43:24.514 INFO:tasks.workunit.client.0.smithi107.stdout:4/947: truncate d0/d9/d10/dab/f103 2082583 0 2024-03-04T13:43:24.519 INFO:tasks.workunit.client.0.smithi107.stdout:0/963: dread da/dc/dd/f24 [0,4194304] 0 2024-03-04T13:43:24.520 INFO:tasks.workunit.client.0.smithi107.stdout:2/941: dread da/d39/f46 [0,4194304] 0 2024-03-04T13:43:24.521 INFO:tasks.workunit.client.0.smithi107.stdout:8/954: write d6/d16/d69/deb/d3e/f5a [383671,37658] 0 2024-03-04T13:43:24.524 INFO:tasks.workunit.client.0.smithi107.stdout:0/964: write da/dc/dd/d3e/f11 [6631103,27203] 0 2024-03-04T13:43:24.525 INFO:tasks.workunit.client.0.smithi107.stdout:2/942: mknod da/d44/d8b/d9d/df8/d5c/dc5/c140 0 2024-03-04T13:43:24.526 INFO:tasks.workunit.client.0.smithi107.stdout:6/944: write df/d38/d4f/d74/f98 [459611,102698] 0 2024-03-04T13:43:24.526 INFO:tasks.workunit.client.0.smithi107.stdout:2/943: stat da/f137 0 2024-03-04T13:43:24.531 INFO:tasks.workunit.client.0.smithi107.stdout:6/945: unlink df/d2e/d36/d77/d118/l4c 0 2024-03-04T13:43:24.541 INFO:tasks.workunit.client.0.smithi107.stdout:8/955: getdents d6/d16/d69/deb/d3e/d67/db7 0 2024-03-04T13:43:24.543 INFO:tasks.workunit.client.0.smithi107.stdout:6/946: symlink df/d1b/d10b/l12f 0 2024-03-04T13:43:24.544 INFO:tasks.workunit.client.0.smithi107.stdout:2/944: rmdir da/df/d73/daa/ddc/d11c 0 2024-03-04T13:43:24.544 INFO:tasks.workunit.client.0.smithi107.stdout:8/956: truncate d6/d16/d69/deb/fbb 4202482 0 2024-03-04T13:43:24.545 INFO:tasks.workunit.client.0.smithi107.stdout:2/945: stat da/d29 0 2024-03-04T13:43:24.546 INFO:tasks.workunit.client.0.smithi107.stdout:6/947: write df/d2e/d36/d5b/d7d/dfc/f119 [763245,106858] 0 2024-03-04T13:43:24.548 INFO:tasks.workunit.client.0.smithi107.stdout:2/946: creat da/d44/d9c/f141 x:0 0 0 2024-03-04T13:43:24.549 INFO:tasks.workunit.client.0.smithi107.stdout:8/957: chown d6/d16/d69/deb/d7c/ddf 505139 1 2024-03-04T13:43:24.549 INFO:tasks.workunit.client.0.smithi107.stdout:0/965: link da/dc/dd/d3e/d26/d88/dd2/d73/d78/fcf da/f14a 0 2024-03-04T13:43:24.551 INFO:tasks.workunit.client.0.smithi107.stdout:6/948: write df/d17/d24/d40/d63/d11c/f124 [1022554,61959] 0 2024-03-04T13:43:24.551 INFO:tasks.workunit.client.0.smithi107.stdout:0/966: chown da/dc/dd/ddb 1274 1 2024-03-04T13:43:24.554 INFO:tasks.workunit.client.0.smithi107.stdout:2/947: fdatasync da/d44/d8b/d98/dc3/f10f 0 2024-03-04T13:43:24.558 INFO:tasks.workunit.client.0.smithi107.stdout:0/967: creat da/dc/dd/d1e/f14b x:0 0 0 2024-03-04T13:43:24.563 INFO:tasks.workunit.client.0.smithi107.stdout:0/968: mknod da/dc/dd/d3e/d26/c14c 0 2024-03-04T13:43:24.564 INFO:tasks.workunit.client.0.smithi107.stdout:8/958: dwrite d6/d16/d69/deb/d3e/d67/fdc [0,4194304] 0 2024-03-04T13:43:24.566 INFO:tasks.workunit.client.0.smithi107.stdout:2/948: write da/df/d3e/de3/fea [801190,65515] 0 2024-03-04T13:43:24.576 INFO:tasks.workunit.client.0.smithi107.stdout:6/949: link df/d17/d24/d40/d63/d11c/c12a df/d17/d24/d40/d63/d11c/d111/c130 0 2024-03-04T13:43:24.576 INFO:tasks.workunit.client.0.smithi107.stdout:2/949: fdatasync da/d34/f36 0 2024-03-04T13:43:24.579 INFO:tasks.workunit.client.0.smithi107.stdout:0/969: getdents da/dc/dd/d1e 0 2024-03-04T13:43:24.581 INFO:tasks.workunit.client.0.smithi107.stdout:0/970: dread - da/dc/dd/ffa zero size 2024-03-04T13:43:24.585 INFO:tasks.workunit.client.0.smithi107.stdout:0/971: truncate da/dc/dd/d3e/d26/d69/d6b/f124 357959 0 2024-03-04T13:43:24.586 INFO:tasks.workunit.client.0.smithi107.stdout:6/950: write df/d2e/d36/d77/d106/d7e/fa9 [2978709,242] 0 2024-03-04T13:43:24.588 INFO:tasks.workunit.client.0.smithi107.stdout:0/972: mkdir da/dc/d14d 0 2024-03-04T13:43:24.591 INFO:tasks.workunit.client.0.smithi107.stdout:0/973: rmdir da/dc/dd/d3e/d26/d69/d6b 39 2024-03-04T13:43:24.594 INFO:tasks.workunit.client.0.smithi107.stdout:0/974: readlink da/dc/dd/d3e/d26/d88/dd2/lf4 0 2024-03-04T13:43:24.597 INFO:tasks.workunit.client.0.smithi107.stdout:0/975: creat da/dad/f14e x:0 0 0 2024-03-04T13:43:24.627 INFO:tasks.workunit.client.0.smithi107.stdout:6/951: dwrite df/d17/d24/f81 [0,4194304] 0 2024-03-04T13:43:24.637 INFO:tasks.workunit.client.0.smithi107.stdout:6/952: creat df/d2e/d36/d77/d118/deb/f131 x:0 0 0 2024-03-04T13:43:24.638 INFO:tasks.workunit.client.0.smithi107.stdout:0/976: write da/dc/dd/d3e/f77 [2261511,54103] 0 2024-03-04T13:43:24.661 INFO:tasks.workunit.client.0.smithi107.stdout:6/953: write df/d17/d24/d30/fc4 [2119081,64701] 0 2024-03-04T13:43:24.666 INFO:tasks.workunit.client.0.smithi107.stdout:0/977: write da/dc/fc4 [757552,41053] 0 2024-03-04T13:43:24.668 INFO:tasks.workunit.client.0.smithi107.stdout:6/954: mknod df/d17/d6b/d9b/c132 0 2024-03-04T13:43:24.670 INFO:tasks.workunit.client.0.smithi107.stdout:6/955: chown df/d1b/f1d 76089 1 2024-03-04T13:43:24.674 INFO:tasks.workunit.client.0.smithi107.stdout:6/956: unlink df/d2e/d36/d77/d106/l50 0 2024-03-04T13:43:24.677 INFO:tasks.workunit.client.0.smithi107.stdout:6/957: read - df/d2e/d36/d77/d95/da7/fcb zero size 2024-03-04T13:43:24.682 INFO:tasks.workunit.client.0.smithi107.stdout:6/958: creat df/d17/d24/d40/d63/d11c/d111/f133 x:0 0 0 2024-03-04T13:43:24.689 INFO:tasks.workunit.client.0.smithi107.stdout:6/959: creat df/d17/d6b/d9b/d61/d99/f134 x:0 0 0 2024-03-04T13:43:24.690 INFO:tasks.workunit.client.0.smithi107.stdout:0/978: dread da/dad/f109 [0,4194304] 0 2024-03-04T13:43:24.691 INFO:tasks.workunit.client.0.smithi107.stdout:6/960: rmdir df/d17 39 2024-03-04T13:43:24.704 INFO:tasks.workunit.client.0.smithi107.stdout:7/907: sync 2024-03-04T13:43:24.716 INFO:tasks.workunit.client.0.smithi107.stdout:6/961: dread df/d38/d4f/daf/fd2 [0,4194304] 0 2024-03-04T13:43:24.716 INFO:tasks.workunit.client.0.smithi107.stdout:7/908: write d6/d1d/d64/d9a/da9/d71/db9/dca/d3d/fb1 [1502381,48749] 0 2024-03-04T13:43:24.721 INFO:tasks.workunit.client.0.smithi107.stdout:6/962: symlink df/d17/d6b/l135 0 2024-03-04T13:43:24.724 INFO:tasks.workunit.client.0.smithi107.stdout:6/963: truncate df/d17/d24/df1/ff6 705295 0 2024-03-04T13:43:24.815 INFO:tasks.workunit.client.0.smithi107.stdout:7/909: dwrite d6/d1d/d64/d9a/da9/d71/db9/dca/f53 [0,4194304] 0 2024-03-04T13:43:24.829 INFO:tasks.workunit.client.0.smithi107.stdout:7/910: getdents d6/d1d/d64/d9a/da9/d71/db9 0 2024-03-04T13:43:24.831 INFO:tasks.workunit.client.0.smithi107.stdout:6/964: dwrite df/d2e/d36/d77/d118/deb/f131 [0,4194304] 0 2024-03-04T13:43:24.832 INFO:tasks.workunit.client.0.smithi107.stdout:7/911: chown d6/d13/d16/d19/fa3 836 1 2024-03-04T13:43:24.836 INFO:tasks.workunit.client.0.smithi107.stdout:7/912: mkdir d6/d17/d12b 0 2024-03-04T13:43:24.850 INFO:tasks.workunit.client.0.smithi107.stdout:6/965: dwrite df/d17/d24/f81 [0,4194304] 0 2024-03-04T13:43:24.856 INFO:tasks.workunit.client.0.smithi107.stdout:6/966: symlink df/d17/d6b/da8/d113/l136 0 2024-03-04T13:43:24.860 INFO:tasks.workunit.client.0.smithi107.stdout:1/973: sync 2024-03-04T13:43:24.861 INFO:tasks.workunit.client.0.smithi107.stdout:1/974: dread - d1/d38/f142 zero size 2024-03-04T13:43:24.861 INFO:tasks.workunit.client.0.smithi107.stdout:6/967: mknod df/d2e/d36/d77/d106/c137 0 2024-03-04T13:43:24.862 INFO:tasks.workunit.client.0.smithi107.stdout:7/913: dwrite d6/d1d/d64/d9a/da9/d71/db9/dca/fab [0,4194304] 0 2024-03-04T13:43:24.865 INFO:tasks.workunit.client.0.smithi107.stdout:7/914: write d6/d1d/d36/d59/fe9 [1106900,29030] 0 2024-03-04T13:43:24.867 INFO:tasks.workunit.client.0.smithi107.stdout:6/968: unlink df/d1b/f4b 0 2024-03-04T13:43:24.869 INFO:tasks.workunit.client.0.smithi107.stdout:1/975: readlink d1/d46/d6a/d4e/daa/dc9/d33/d5d/d24/l11b 0 2024-03-04T13:43:24.871 INFO:tasks.workunit.client.0.smithi107.stdout:1/976: readlink d1/d10/d16/d47/d7a/dda/dec/l10c 0 2024-03-04T13:43:24.872 INFO:tasks.workunit.client.0.smithi107.stdout:7/915: creat d6/d13/d16/d19/d38/f12c x:0 0 0 2024-03-04T13:43:24.878 INFO:tasks.workunit.client.0.smithi107.stdout:7/916: creat d6/d1d/d64/d9a/da9/d71/db9/dca/d3d/dc6/dfb/f12d x:0 0 0 2024-03-04T13:43:24.883 INFO:tasks.workunit.client.0.smithi107.stdout:7/917: truncate d6/d1d/d64/d9a/da9/d71/db9/fd8 291620 0 2024-03-04T13:43:24.890 INFO:tasks.workunit.client.0.smithi107.stdout:7/918: link d6/d13/d16/d19/d38/d2c/f87 d6/d1d/d64/d9a/da9/d71/db9/dca/dd0/f12e 0 2024-03-04T13:43:24.895 INFO:tasks.workunit.client.0.smithi107.stdout:7/919: rename d6/d1d/d64/c104 to d6/d1d/d64/d9a/da9/d71/db9/dca/d3d/dc6/c12f 0 2024-03-04T13:43:24.901 INFO:tasks.workunit.client.0.smithi107.stdout:7/920: unlink d6/d1d/d64/d9a/da9/d71/db9/dca/f53 0 2024-03-04T13:43:24.903 INFO:tasks.workunit.client.0.smithi107.stdout:7/921: stat d6/d13/d14/f86 0 2024-03-04T13:43:24.906 INFO:tasks.workunit.client.0.smithi107.stdout:1/977: dread d1/d46/d6a/d4e/fcf [0,4194304] 0 2024-03-04T13:43:24.908 INFO:tasks.workunit.client.0.smithi107.stdout:7/922: mkdir d6/d1d/d64/d9a/da9/d71/db9/dca/d3d/dc6/dcb/d130 0 2024-03-04T13:43:24.909 INFO:tasks.workunit.client.0.smithi107.stdout:7/923: read - d6/d13/d16/d19/d38/f12c zero size 2024-03-04T13:43:24.918 INFO:tasks.workunit.client.0.smithi107.stdout:7/924: creat d6/d13/f131 x:0 0 0 2024-03-04T13:43:24.924 INFO:tasks.workunit.client.0.smithi107.stdout:1/978: dwrite d1/d46/d6a/d4e/daa/dc9/d33/d5d/d24/f52 [0,4194304] 0 2024-03-04T13:43:24.933 INFO:tasks.workunit.client.0.smithi107.stdout:1/979: readlink d1/d46/d6a/d4e/d60/d83/dde/lf8 0 2024-03-04T13:43:24.933 INFO:tasks.workunit.client.0.smithi107.stdout:7/925: dwrite d6/d13/d14/f86 [4194304,4194304] 0 2024-03-04T13:43:24.943 INFO:tasks.workunit.client.0.smithi107.stdout:9/876: sync 2024-03-04T13:43:24.946 INFO:tasks.workunit.client.0.smithi107.stdout:9/877: stat d4/da/d11/d4c/d51/c5b 0 2024-03-04T13:43:24.949 INFO:tasks.workunit.client.0.smithi107.stdout:1/980: dwrite d1/d46/d6a/d4e/fcf [0,4194304] 0 2024-03-04T13:43:24.954 INFO:tasks.workunit.client.0.smithi107.stdout:9/878: symlink d4/da/d11/d20/d64/d77/l126 0 2024-03-04T13:43:24.956 INFO:tasks.workunit.client.0.smithi107.stdout:7/926: dwrite d6/d13/d16/d1c/dc2/ff9 [0,4194304] 0 2024-03-04T13:43:24.957 INFO:tasks.workunit.client.0.smithi107.stdout:1/981: creat d1/d46/d6a/d65/d106/d61/ddb/de6/f143 x:0 0 0 2024-03-04T13:43:24.958 INFO:tasks.workunit.client.0.smithi107.stdout:9/879: chown d4/da/d11/d20/d23/d7e/d84/l8a 385 1 2024-03-04T13:43:24.960 INFO:tasks.workunit.client.0.smithi107.stdout:1/982: write d1/d10/d16/d1b/fe3 [148747,79030] 0 2024-03-04T13:43:24.976 INFO:tasks.workunit.client.0.smithi107.stdout:1/983: creat d1/d10/d16/da9/d115/f144 x:0 0 0 2024-03-04T13:43:24.976 INFO:tasks.workunit.client.0.smithi107.stdout:9/880: link d4/da/d11/d20/d64/d77/fe9 d4/da/d11/d20/d2f/d9c/d92/f127 0 2024-03-04T13:43:24.978 INFO:tasks.workunit.client.0.smithi107.stdout:1/984: write d1/d38/f142 [28812,7218] 0 2024-03-04T13:43:24.978 INFO:tasks.workunit.client.0.smithi107.stdout:7/927: creat d6/d1d/d64/d9a/da9/d71/f132 x:0 0 0 2024-03-04T13:43:24.979 INFO:tasks.workunit.client.0.smithi107.stdout:9/881: chown d4/d32/l61 5609237 1 2024-03-04T13:43:24.981 INFO:tasks.workunit.client.0.smithi107.stdout:7/928: mknod d6/d13/d16/d1c/dc2/c133 0 2024-03-04T13:43:24.985 INFO:tasks.workunit.client.0.smithi107.stdout:7/929: mknod d6/d1d/d64/d9a/c134 0 2024-03-04T13:43:24.986 INFO:tasks.workunit.client.0.smithi107.stdout:1/985: dread d1/d38/fe4 [0,4194304] 0 2024-03-04T13:43:24.989 INFO:tasks.workunit.client.0.smithi107.stdout:7/930: mkdir d6/d1d/d64/d9a/da9/d71/db9/dca/dd0/d135 0 2024-03-04T13:43:24.991 INFO:tasks.workunit.client.0.smithi107.stdout:7/931: fsync d6/d13/d16/d19/faa 0 2024-03-04T13:43:24.994 INFO:tasks.workunit.client.0.smithi107.stdout:0/979: sync 2024-03-04T13:43:24.995 INFO:tasks.workunit.client.0.smithi107.stdout:2/950: sync 2024-03-04T13:43:24.995 INFO:tasks.workunit.client.0.smithi107.stdout:8/959: sync 2024-03-04T13:43:24.995 INFO:tasks.workunit.client.0.smithi107.stdout:4/948: sync 2024-03-04T13:43:24.995 INFO:tasks.workunit.client.0.smithi107.stdout:6/969: sync 2024-03-04T13:43:24.995 INFO:tasks.workunit.client.0.smithi107.stdout:9/882: dwrite d4/da/d11/d20/d64/f78 [4194304,4194304] 0 2024-03-04T13:43:24.996 INFO:tasks.workunit.client.0.smithi107.stdout:1/986: write d1/d46/d6a/d4e/daa/dc9/d33/d5d/d4c/dea/ff6 [3408987,46772] 0 2024-03-04T13:43:24.998 INFO:tasks.workunit.client.0.smithi107.stdout:2/951: fsync da/d44/f81 0 2024-03-04T13:43:25.000 INFO:tasks.workunit.client.0.smithi107.stdout:2/952: readlink da/d44/l6e 0 2024-03-04T13:43:25.002 INFO:tasks.workunit.client.0.smithi107.stdout:4/949: creat d0/d2f/d5f/f12e x:0 0 0 2024-03-04T13:43:25.003 INFO:tasks.workunit.client.0.smithi107.stdout:8/960: creat d6/d16/d1f/dbd/f12e x:0 0 0 2024-03-04T13:43:25.006 INFO:tasks.workunit.client.0.smithi107.stdout:0/980: truncate da/dc/d34/f45 2363201 0 2024-03-04T13:43:25.009 INFO:tasks.workunit.client.0.smithi107.stdout:6/970: dread df/d17/f52 [0,4194304] 0 2024-03-04T13:43:25.011 INFO:tasks.workunit.client.0.smithi107.stdout:9/883: creat d4/da/d11/d4c/f128 x:0 0 0 2024-03-04T13:43:25.015 INFO:tasks.workunit.client.0.smithi107.stdout:4/950: mknod d0/d9/d18/d29/c12f 0 2024-03-04T13:43:25.022 INFO:tasks.workunit.client.0.smithi107.stdout:6/971: creat df/d57/f138 x:0 0 0 2024-03-04T13:43:25.023 INFO:tasks.workunit.client.0.smithi107.stdout:2/953: link da/df/d73/db3/db1/cc6 da/d29/d13f/c142 0 2024-03-04T13:43:25.025 INFO:tasks.workunit.client.0.smithi107.stdout:7/932: truncate d6/d13/d16/f8b 2445854 0 2024-03-04T13:43:25.025 INFO:tasks.workunit.client.0.smithi107.stdout:9/884: mkdir d4/da/d11/d65/d129 0 2024-03-04T13:43:25.028 INFO:tasks.workunit.client.0.smithi107.stdout:1/987: truncate d1/d10/fb0 6594193 0 2024-03-04T13:43:25.030 INFO:tasks.workunit.client.0.smithi107.stdout:6/972: read df/d2e/d36/d5b/fb7 [11017,110961] 0 2024-03-04T13:43:25.033 INFO:tasks.workunit.client.0.smithi107.stdout:4/951: dwrite d0/d9/f5b [4194304,4194304] 0 2024-03-04T13:43:25.038 INFO:tasks.workunit.client.0.smithi107.stdout:0/981: write da/dc/dd/d3e/d26/d88/dd2/f9d [3339167,107161] 0 2024-03-04T13:43:25.041 INFO:tasks.workunit.client.0.smithi107.stdout:1/988: symlink d1/d46/d6a/d75/dfd/l145 0 2024-03-04T13:43:25.042 INFO:tasks.workunit.client.0.smithi107.stdout:9/885: rename d4/da/d11/d20/d23/dd9/fa2 to d4/da/d11/d20/d2f/d9c/d92/df6/f12a 0 2024-03-04T13:43:25.046 INFO:tasks.workunit.client.0.smithi107.stdout:9/886: dread - d4/da/f11a zero size 2024-03-04T13:43:25.046 INFO:tasks.workunit.client.0.smithi107.stdout:7/933: rmdir d6/d13/d16/d19/d60 39 2024-03-04T13:43:25.046 INFO:tasks.workunit.client.0.smithi107.stdout:4/952: unlink d0/d2f/d7d/ld7 0 2024-03-04T13:43:25.048 INFO:tasks.workunit.client.0.smithi107.stdout:0/982: symlink da/dc/d34/l14f 0 2024-03-04T13:43:25.049 INFO:tasks.workunit.client.0.smithi107.stdout:2/954: link da/d38/d3a/d5d/ld2 da/df/d73/db3/db1/dee/d134/l143 0 2024-03-04T13:43:25.050 INFO:tasks.workunit.client.0.smithi107.stdout:2/955: chown da/d44/cbf 109248 1 2024-03-04T13:43:25.057 INFO:tasks.workunit.client.0.smithi107.stdout:9/887: symlink d4/da/d11/d20/d64/d77/ddc/l12b 0 2024-03-04T13:43:25.058 INFO:tasks.workunit.client.0.smithi107.stdout:4/953: read - d0/d9/d18/d29/f89 zero size 2024-03-04T13:43:25.058 INFO:tasks.workunit.client.0.smithi107.stdout:2/956: write da/df/d73/db3/db1/f111 [4527769,14264] 0 2024-03-04T13:43:25.058 INFO:tasks.workunit.client.0.smithi107.stdout:2/957: readlink da/d39/lca 0 2024-03-04T13:43:25.058 INFO:tasks.workunit.client.0.smithi107.stdout:0/983: truncate da/dc/dd/d3e/d26/d88/dd2/d73/d78/f116 243420 0 2024-03-04T13:43:25.058 INFO:tasks.workunit.client.0.smithi107.stdout:1/989: rename d1/d46/d6a/d4e/daa/dc9/d33/cf5 to d1/d46/d6a/d4e/daa/dc9/d33/c146 0 2024-03-04T13:43:25.059 INFO:tasks.workunit.client.0.smithi107.stdout:9/888: mkdir d4/da/d11/d65/d129/d12c 0 2024-03-04T13:43:25.064 INFO:tasks.workunit.client.0.smithi107.stdout:0/984: fdatasync da/dc/dd/d3e/d26/fd0 0 2024-03-04T13:43:25.065 INFO:tasks.workunit.client.0.smithi107.stdout:7/934: dread d6/d13/d16/d19/f11d [0,4194304] 0 2024-03-04T13:43:25.066 INFO:tasks.workunit.client.0.smithi107.stdout:1/990: write d1/d46/d6a/d65/db1/ff4 [1047123,61463] 0 2024-03-04T13:43:25.076 INFO:tasks.workunit.client.0.smithi107.stdout:0/985: chown da/dc/dd/d3e/d26/l20 51892612 1 2024-03-04T13:43:25.078 INFO:tasks.workunit.client.0.smithi107.stdout:1/991: getdents d1/d46/d6a/d75/dfd 0 2024-03-04T13:43:25.078 INFO:tasks.workunit.client.0.smithi107.stdout:2/958: dread da/df/d73/d7a/d9a/fc9 [0,4194304] 0 2024-03-04T13:43:25.080 INFO:tasks.workunit.client.0.smithi107.stdout:6/973: dwrite df/d2e/d36/fbc [0,4194304] 0 2024-03-04T13:43:25.093 INFO:tasks.workunit.client.0.smithi107.stdout:1/992: getdents d1/d10/d16/d47/d7a/d128/d129 0 2024-03-04T13:43:25.095 INFO:tasks.workunit.client.0.smithi107.stdout:9/889: dwrite d4/da/d11/d20/d64/d77/ddc/f101 [0,4194304] 0 2024-03-04T13:43:25.102 INFO:tasks.workunit.client.0.smithi107.stdout:7/935: dwrite d6/d1d/d64/d9a/da9/d71/db9/dca/d3d/dc6/dcb/f10b [0,4194304] 0 2024-03-04T13:43:25.112 INFO:tasks.workunit.client.0.smithi107.stdout:2/959: dwrite da/df/d27/d10b/d5f/f93 [4194304,4194304] 0 2024-03-04T13:43:25.114 INFO:tasks.workunit.client.0.smithi107.stdout:7/936: readlink d6/d1d/d64/d9a/da9/d71/db9/dca/dd0/l11b 0 2024-03-04T13:43:25.128 INFO:tasks.workunit.client.0.smithi107.stdout:7/937: mknod d6/d1d/d64/c136 0 2024-03-04T13:43:25.131 INFO:tasks.workunit.client.0.smithi107.stdout:8/961: sync 2024-03-04T13:43:25.142 INFO:tasks.workunit.client.0.smithi107.stdout:0/986: dwrite da/dc/dd/d3e/d26/d69/d6b/f124 [0,4194304] 0 2024-03-04T13:43:25.142 INFO:tasks.workunit.client.0.smithi107.stdout:6/974: dwrite df/d2e/d36/f49 [0,4194304] 0 2024-03-04T13:43:25.152 INFO:tasks.workunit.client.0.smithi107.stdout:7/938: fsync d6/d1d/d64/d9a/da9/d71/db9/dca/f101 0 2024-03-04T13:43:25.152 INFO:tasks.workunit.client.0.smithi107.stdout:0/987: read - da/dc/dd/d1e/fe2 zero size 2024-03-04T13:43:25.153 INFO:tasks.workunit.client.0.smithi107.stdout:9/890: dwrite d4/da/d11/d4c/f128 [0,4194304] 0 2024-03-04T13:43:25.160 INFO:tasks.workunit.client.0.smithi107.stdout:2/960: dwrite da/d38/d3a/d5d/ff9 [0,4194304] 0 2024-03-04T13:43:25.171 INFO:tasks.workunit.client.0.smithi107.stdout:4/954: sync 2024-03-04T13:43:25.191 INFO:tasks.workunit.client.0.smithi107.stdout:8/962: creat d6/da/d25/dcb/f12f x:0 0 0 2024-03-04T13:43:25.192 INFO:tasks.workunit.client.0.smithi107.stdout:0/988: readlink da/ld6 0 2024-03-04T13:43:25.192 INFO:tasks.workunit.client.0.smithi107.stdout:2/961: rmdir da/d44/d8b/d9d 39 2024-03-04T13:43:25.194 INFO:tasks.workunit.client.0.smithi107.stdout:6/975: mknod df/d17/d24/c139 0 2024-03-04T13:43:25.196 INFO:tasks.workunit.client.0.smithi107.stdout:0/989: fdatasync da/dc/dd/d3e/d26/d69/de9/f12c 0 2024-03-04T13:43:25.196 INFO:tasks.workunit.client.0.smithi107.stdout:7/939: symlink d6/d1d/d64/d9a/da9/d71/db9/dca/l137 0 2024-03-04T13:43:25.197 INFO:tasks.workunit.client.0.smithi107.stdout:9/891: symlink d4/da/d11/d20/d23/d7e/l12d 0 2024-03-04T13:43:25.200 INFO:tasks.workunit.client.0.smithi107.stdout:8/963: creat d6/d16/d1f/d63/d73/f130 x:0 0 0 2024-03-04T13:43:25.207 INFO:tasks.workunit.client.0.smithi107.stdout:4/955: dread d0/d9/de/d14/ffb [0,4194304] 0 2024-03-04T13:43:25.211 INFO:tasks.workunit.client.0.smithi107.stdout:8/964: rename d6/d16/d1f/d4c/da4/daa/fc3 to d6/d16/d69/deb/d3e/f131 0 2024-03-04T13:43:25.215 INFO:tasks.workunit.client.0.smithi107.stdout:2/962: truncate da/d44/d8b/d9d/df8/d5c/d6a/f7f 2260196 0 2024-03-04T13:43:25.216 INFO:tasks.workunit.client.0.smithi107.stdout:1/993: write d1/d10/fb0 [3120076,111135] 0 2024-03-04T13:43:25.219 INFO:tasks.workunit.client.0.smithi107.stdout:8/965: read d6/f54 [3282188,54214] 0 2024-03-04T13:43:25.224 INFO:tasks.workunit.client.0.smithi107.stdout:4/956: dread d0/d9/fb2 [0,4194304] 0 2024-03-04T13:43:25.227 INFO:tasks.workunit.client.0.smithi107.stdout:9/892: dwrite d4/d34/fe0 [0,4194304] 0 2024-03-04T13:43:25.236 INFO:tasks.workunit.client.0.smithi107.stdout:1/994: rename d1/d46/d6a/d4e/daa/dc9/d33/d5d/d24/f52 to d1/d10/d16/da9/f147 0 2024-03-04T13:43:25.243 INFO:tasks.workunit.client.0.smithi107.stdout:9/893: fdatasync d4/da/d11/d20/d2f/d9c/daa/fc2 0 2024-03-04T13:43:25.244 INFO:tasks.workunit.client.0.smithi107.stdout:0/990: dwrite da/dc/dd/d3e/d26/d88/dd2/fbd [0,4194304] 0 2024-03-04T13:43:25.249 INFO:tasks.workunit.client.0.smithi107.stdout:4/957: rename d0/d9/d10/dab/d12b to d0/d9/d18/df1/dc1/d130 0 2024-03-04T13:43:25.252 INFO:tasks.workunit.client.0.smithi107.stdout:6/976: dwrite df/d2e/d36/d77/d106/d7e/d86/f97 [4194304,4194304] 0 2024-03-04T13:43:25.258 INFO:tasks.workunit.client.0.smithi107.stdout:7/940: dwrite d6/d13/d16/d19/d38/f47 [0,4194304] 0 2024-03-04T13:43:25.258 INFO:tasks.workunit.client.0.smithi107.stdout:9/894: readlink d4/da/d11/d4c/d51/d122/lfa 0 2024-03-04T13:43:25.258 INFO:tasks.workunit.client.0.smithi107.stdout:1/995: rmdir d1/d46/d6a/d65/d106/d61/ddb/de6/d124/dba 39 2024-03-04T13:43:25.259 INFO:tasks.workunit.client.0.smithi107.stdout:4/958: fdatasync d0/d2f/d5f/f12e 0 2024-03-04T13:43:25.261 INFO:tasks.workunit.client.0.smithi107.stdout:8/966: dwrite d6/dcc/f11a [0,4194304] 0 2024-03-04T13:43:25.275 INFO:tasks.workunit.client.0.smithi107.stdout:7/941: dwrite d6/d13/d16/d19/faa [0,4194304] 0 2024-03-04T13:43:25.278 INFO:tasks.workunit.client.0.smithi107.stdout:7/942: write d6/d13/d16/d19/fa1 [2146524,69433] 0 2024-03-04T13:43:25.285 INFO:tasks.workunit.client.0.smithi107.stdout:9/895: mknod d4/da/d11/d65/c12e 0 2024-03-04T13:43:25.286 INFO:tasks.workunit.client.0.smithi107.stdout:6/977: symlink df/d17/d24/d83/l13a 0 2024-03-04T13:43:25.287 INFO:tasks.workunit.client.0.smithi107.stdout:1/996: symlink d1/d46/d6a/d65/d106/d61/ddb/de6/d124/l148 0 2024-03-04T13:43:25.296 INFO:tasks.workunit.client.0.smithi107.stdout:2/963: dread da/d44/d8b/d9d/df8/d5c/f67 [4194304,4194304] 0 2024-03-04T13:43:25.296 INFO:tasks.workunit.client.0.smithi107.stdout:4/959: creat d0/d9/d10/d80/db8/f131 x:0 0 0 2024-03-04T13:43:25.296 INFO:tasks.workunit.client.0.smithi107.stdout:0/991: getdents da/dc/dd/d3e/d26/d69/de9 0 2024-03-04T13:43:25.296 INFO:tasks.workunit.client.0.smithi107.stdout:7/943: unlink d6/d1d/d64/d9a/da9/d71/db9/dca/d3d/dc6/lf8 0 2024-03-04T13:43:25.297 INFO:tasks.workunit.client.0.smithi107.stdout:6/978: rename f8 to df/d2e/d36/d77/d118/deb/d101/f13b 0 2024-03-04T13:43:25.308 INFO:tasks.workunit.client.0.smithi107.stdout:1/997: dread d1/d46/d6a/d65/db1/f127 [0,4194304] 0 2024-03-04T13:43:25.310 INFO:tasks.workunit.client.0.smithi107.stdout:8/967: link d6/d16/d1f/lc4 d6/d16/d1f/d63/d73/l132 0 2024-03-04T13:43:25.313 INFO:tasks.workunit.client.0.smithi107.stdout:9/896: write d4/da/d11/d20/d2f/fd1 [575980,23380] 0 2024-03-04T13:43:25.316 INFO:tasks.workunit.client.0.smithi107.stdout:0/992: write da/dc/dd/d1e/dbf/deb/f11b [964690,117125] 0 2024-03-04T13:43:25.322 INFO:tasks.workunit.client.0.smithi107.stdout:1/998: creat d1/d46/d6a/d4e/daa/dc9/d33/dbd/d13f/f149 x:0 0 0 2024-03-04T13:43:25.331 INFO:tasks.workunit.client.0.smithi107.stdout:1/999: symlink d1/d46/d6a/d65/d11c/d132/l14a 0 2024-03-04T13:43:25.333 INFO:tasks.workunit.client.0.smithi107.stdout:9/897: mknod d4/da/d11/d20/d64/d77/df9/c12f 0 2024-03-04T13:43:25.337 INFO:tasks.workunit.client.0.smithi107.stdout:2/964: dwrite da/df/d73/dd1/f102 [0,4194304] 0 2024-03-04T13:43:25.343 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:25 smithi107 ceph-mon[28739]: pgmap v64: 65 pgs: 65 active+clean; 3.4 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 95 MiB/s rd, 219 MiB/s wr, 530 op/s 2024-03-04T13:43:25.355 INFO:tasks.workunit.client.0.smithi107.stdout:6/979: dwrite df/d17/d24/d30/d8b/f8f [0,4194304] 0 2024-03-04T13:43:25.357 INFO:tasks.workunit.client.0.smithi107.stdout:7/944: dwrite d6/d1d/d64/d9a/da9/d71/db9/dca/d3d/fb1 [0,4194304] 0 2024-03-04T13:43:25.357 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:25 smithi169 ceph-mon[36313]: pgmap v64: 65 pgs: 65 active+clean; 3.4 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 95 MiB/s rd, 219 MiB/s wr, 530 op/s 2024-03-04T13:43:25.361 INFO:tasks.workunit.client.0.smithi107.stdout:2/965: mknod da/d39/d122/c144 0 2024-03-04T13:43:25.361 INFO:tasks.workunit.client.0.smithi107.stdout:9/898: truncate d4/da/d11/d20/d39/fdb 400840 0 2024-03-04T13:43:25.364 INFO:tasks.workunit.client.0.smithi107.stdout:0/993: dwrite da/dc/dd/d1e/dbf/deb/f119 [0,4194304] 0 2024-03-04T13:43:25.369 INFO:tasks.workunit.client.0.smithi107.stdout:4/960: dread d0/d9/d10/d80/db8/f3c [0,4194304] 0 2024-03-04T13:43:25.369 INFO:tasks.workunit.client.0.smithi107.stdout:7/945: write d6/d1d/d64/d9a/da9/d71/db9/dca/f101 [553007,98380] 0 2024-03-04T13:43:25.370 INFO:tasks.workunit.client.0.smithi107.stdout:2/966: creat da/df/d73/dd1/d100/f145 x:0 0 0 2024-03-04T13:43:25.370 INFO:tasks.workunit.client.0.smithi107.stdout:9/899: write d4/da/d11/d20/d23/dcc/de4/f116 [307660,92521] 0 2024-03-04T13:43:25.371 INFO:tasks.workunit.client.0.smithi107.stdout:7/946: dread - d6/d13/f131 zero size 2024-03-04T13:43:25.372 INFO:tasks.workunit.client.0.smithi107.stdout:4/961: chown d0/d9/d10/d6f/fad 537 1 2024-03-04T13:43:25.376 INFO:tasks.workunit.client.0.smithi107.stdout:9/900: readlink d4/da/d11/d20/d2f/d9c/daa/lbc 0 2024-03-04T13:43:25.385 INFO:tasks.workunit.client.0.smithi107.stdout:9/901: unlink d4/da/d11/d20/d2f/d9c/d92/ccf 0 2024-03-04T13:43:25.386 INFO:tasks.workunit.client.0.smithi107.stdout:6/980: getdents df/d2e/d36/d5b/d7d 0 2024-03-04T13:43:25.388 INFO:tasks.workunit.client.0.smithi107.stdout:4/962: symlink d0/d9/d10/l132 0 2024-03-04T13:43:25.388 INFO:tasks.workunit.client.0.smithi107.stdout:0/994: truncate da/dc/dd/d3e/f44 9013011 0 2024-03-04T13:43:25.390 INFO:tasks.workunit.client.0.smithi107.stdout:7/947: fsync d6/d13/d16/d1c/dc2/fc7 0 2024-03-04T13:43:25.391 INFO:tasks.workunit.client.0.smithi107.stdout:9/902: write d4/da/d11/d4c/f128 [3333643,59947] 0 2024-03-04T13:43:25.394 INFO:tasks.workunit.client.0.smithi107.stdout:2/967: dwrite da/d44/f81 [0,4194304] 0 2024-03-04T13:43:25.395 INFO:tasks.workunit.client.0.smithi107.stdout:9/903: write d4/d34/f44 [2687383,83633] 0 2024-03-04T13:43:25.398 INFO:tasks.workunit.client.0.smithi107.stdout:4/963: mknod d0/d9/d10/d6f/c133 0 2024-03-04T13:43:25.399 INFO:tasks.workunit.client.0.smithi107.stdout:2/968: write da/f137 [431054,42307] 0 2024-03-04T13:43:25.405 INFO:tasks.workunit.client.0.smithi107.stdout:4/964: rmdir d0/d9/de 39 2024-03-04T13:43:25.406 INFO:tasks.workunit.client.0.smithi107.stdout:2/969: rmdir da/d39/d122 39 2024-03-04T13:43:25.409 INFO:tasks.workunit.client.0.smithi107.stdout:7/948: creat d6/d1d/d64/d9a/da9/d71/db9/dca/d3d/dc6/dcb/f138 x:0 0 0 2024-03-04T13:43:25.410 INFO:tasks.workunit.client.0.smithi107.stdout:6/981: dread df/d17/d24/f81 [0,4194304] 0 2024-03-04T13:43:25.413 INFO:tasks.workunit.client.0.smithi107.stdout:7/949: truncate d6/d13/dbc/f123 485927 0 2024-03-04T13:43:25.414 INFO:tasks.workunit.client.0.smithi107.stdout:4/965: symlink d0/d9/d10/d80/db8/d9a/df9/l134 0 2024-03-04T13:43:25.422 INFO:tasks.workunit.client.0.smithi107.stdout:6/982: creat df/d17/d24/d40/f13c x:0 0 0 2024-03-04T13:43:25.424 INFO:tasks.workunit.client.0.smithi107.stdout:4/966: symlink d0/d9/d18/d29/dfc/l135 0 2024-03-04T13:43:25.429 INFO:tasks.workunit.client.0.smithi107.stdout:6/983: creat df/d2e/d36/d5b/d7d/f13d x:0 0 0 2024-03-04T13:43:25.430 INFO:tasks.workunit.client.0.smithi107.stdout:0/995: dread da/dc/dd/d1e/dbf/deb/f148 [0,4194304] 0 2024-03-04T13:43:25.430 INFO:tasks.workunit.client.0.smithi107.stdout:4/967: mkdir d0/d9/d18/df1/d136 0 2024-03-04T13:43:25.432 INFO:tasks.workunit.client.0.smithi107.stdout:6/984: creat df/d57/f13e x:0 0 0 2024-03-04T13:43:25.433 INFO:tasks.workunit.client.0.smithi107.stdout:7/950: dwrite d6/d13/d16/d1c/f35 [4194304,4194304] 0 2024-03-04T13:43:25.436 INFO:tasks.workunit.client.0.smithi107.stdout:4/968: symlink d0/d9/d1e/d70/l137 0 2024-03-04T13:43:25.440 INFO:tasks.workunit.client.0.smithi107.stdout:0/996: read da/dc/dd/dc9/de5/f90 [225867,78501] 0 2024-03-04T13:43:25.441 INFO:tasks.workunit.client.0.smithi107.stdout:7/951: creat d6/d1d/dd9/d106/f139 x:0 0 0 2024-03-04T13:43:25.444 INFO:tasks.workunit.client.0.smithi107.stdout:0/997: fsync da/dc/fc4 0 2024-03-04T13:43:25.444 INFO:tasks.workunit.client.0.smithi107.stdout:4/969: unlink d0/d9/d78/dcf/f104 0 2024-03-04T13:43:25.444 INFO:tasks.workunit.client.0.smithi107.stdout:9/904: dread d4/da/d11/d20/d23/f6c [0,4194304] 0 2024-03-04T13:43:25.447 INFO:tasks.workunit.client.0.smithi107.stdout:7/952: rmdir d6/d1d/d64/d9a/da9/d71/db9/dca/d3d 39 2024-03-04T13:43:25.452 INFO:tasks.workunit.client.0.smithi107.stdout:4/970: creat d0/d9/d10/d80/db8/d21/f138 x:0 0 0 2024-03-04T13:43:25.453 INFO:tasks.workunit.client.0.smithi107.stdout:4/971: write d0/f112 [338645,19215] 0 2024-03-04T13:43:25.466 INFO:tasks.workunit.client.0.smithi107.stdout:0/998: dwrite da/dc/dd/d3e/d26/d69/d6b/df0/f129 [0,4194304] 0 2024-03-04T13:43:25.473 INFO:tasks.workunit.client.0.smithi107.stdout:7/953: dwrite d6/d13/d16/f129 [0,4194304] 0 2024-03-04T13:43:25.475 INFO:tasks.workunit.client.0.smithi107.stdout:0/999: mkdir da/dc/d14d/d150 0 2024-03-04T13:43:25.478 INFO:tasks.workunit.client.0.smithi107.stdout:7/954: symlink d6/d1d/d64/d9a/da9/d71/db9/dca/dd0/l13a 0 2024-03-04T13:43:25.479 INFO:tasks.workunit.client.0.smithi107.stdout:7/955: chown d6/d13/dbc 11544 1 2024-03-04T13:43:25.483 INFO:tasks.workunit.client.0.smithi107.stdout:6/985: truncate df/d17/d6b/d9b/d61/f8a 289166 0 2024-03-04T13:43:25.493 INFO:tasks.workunit.client.0.smithi107.stdout:4/972: write d0/d9/d10/d80/db8/f67 [844980,35988] 0 2024-03-04T13:43:25.493 INFO:tasks.workunit.client.0.smithi107.stdout:8/968: sync 2024-03-04T13:43:25.494 INFO:tasks.workunit.client.0.smithi107.stdout:9/905: dwrite d4/da/d11/d20/d2f/f3d [0,4194304] 0 2024-03-04T13:43:25.496 INFO:tasks.workunit.client.0.smithi107.stdout:8/969: fdatasync d6/d16/d1f/d4c/da4/daa/fd9 0 2024-03-04T13:43:25.499 INFO:tasks.workunit.client.0.smithi107.stdout:9/906: rmdir d4/d34 39 2024-03-04T13:43:25.503 INFO:tasks.workunit.client.0.smithi107.stdout:7/956: write d6/d1d/d36/f5b [1041290,94813] 0 2024-03-04T13:43:25.505 INFO:tasks.workunit.client.0.smithi107.stdout:8/970: symlink d6/d16/d1f/d119/l133 0 2024-03-04T13:43:25.508 INFO:tasks.workunit.client.0.smithi107.stdout:8/971: write d6/da/d25/d2a/f75 [3357970,22269] 0 2024-03-04T13:43:25.509 INFO:tasks.workunit.client.0.smithi107.stdout:7/957: symlink d6/d13/d16/d19/d38/d2c/d8d/l13b 0 2024-03-04T13:43:25.511 INFO:tasks.workunit.client.0.smithi107.stdout:7/958: dread - d6/d13/f131 zero size 2024-03-04T13:43:25.513 INFO:tasks.workunit.client.0.smithi107.stdout:8/972: creat d6/dcc/dfc/d11b/f134 x:0 0 0 2024-03-04T13:43:25.516 INFO:tasks.workunit.client.0.smithi107.stdout:6/986: dread df/d17/d6b/d9b/d61/f8a [0,4194304] 0 2024-03-04T13:43:25.518 INFO:tasks.workunit.client.0.smithi107.stdout:8/973: truncate d6/d16/d69/deb/d7c/ddf/f10e 582551 0 2024-03-04T13:43:25.519 INFO:tasks.workunit.client.0.smithi107.stdout:4/973: write d0/d9/d1e/fdc [669660,79133] 0 2024-03-04T13:43:25.525 INFO:tasks.workunit.client.0.smithi107.stdout:8/974: creat d6/d16/d1f/d119/f135 x:0 0 0 2024-03-04T13:43:25.525 INFO:tasks.workunit.client.0.smithi107.stdout:4/974: chown d0/d2f/d7d/f129 1644866178 1 2024-03-04T13:43:25.527 INFO:tasks.workunit.client.0.smithi107.stdout:4/975: dread - d0/d9/d10/f99 zero size 2024-03-04T13:43:25.535 INFO:tasks.workunit.client.0.smithi107.stdout:8/975: rename d6/d16/d69/deb/d7c/ddf/f10d to d6/da/d25/f136 0 2024-03-04T13:43:25.535 INFO:tasks.workunit.client.0.smithi107.stdout:6/987: dwrite df/d2e/d36/d77/d118/deb/d101/f13b [4194304,4194304] 0 2024-03-04T13:43:25.538 INFO:tasks.workunit.client.0.smithi107.stdout:8/976: write d6/da/d25/d2a/f75 [2528234,47750] 0 2024-03-04T13:43:25.540 INFO:tasks.workunit.client.0.smithi107.stdout:7/959: dread d6/d1d/d36/f81 [0,4194304] 0 2024-03-04T13:43:25.542 INFO:tasks.workunit.client.0.smithi107.stdout:8/977: mknod d6/d16/d1f/d63/d73/c137 0 2024-03-04T13:43:25.544 INFO:tasks.workunit.client.0.smithi107.stdout:6/988: dread df/d57/f7c [0,4194304] 0 2024-03-04T13:43:25.545 INFO:tasks.workunit.client.0.smithi107.stdout:8/978: chown d6/da/d25/d2a/c48 0 1 2024-03-04T13:43:25.548 INFO:tasks.workunit.client.0.smithi107.stdout:6/989: write f6 [8198862,115051] 0 2024-03-04T13:43:25.553 INFO:tasks.workunit.client.0.smithi107.stdout:6/990: write df/d2e/d36/fa2 [2664034,87761] 0 2024-03-04T13:43:25.554 INFO:tasks.workunit.client.0.smithi107.stdout:4/976: dwrite d0/d9/d10/f99 [0,4194304] 0 2024-03-04T13:43:25.560 INFO:tasks.workunit.client.0.smithi107.stdout:8/979: dread d6/da/d25/d2a/fef [0,4194304] 0 2024-03-04T13:43:25.562 INFO:tasks.workunit.client.0.smithi107.stdout:4/977: creat d0/d9/d10/d80/d81/f139 x:0 0 0 2024-03-04T13:43:25.563 INFO:tasks.workunit.client.0.smithi107.stdout:6/991: mkdir df/d17/d24/d40/d13f 0 2024-03-04T13:43:25.565 INFO:tasks.workunit.client.0.smithi107.stdout:4/978: read - d0/d9/d10/d80/db8/d21/f138 zero size 2024-03-04T13:43:25.567 INFO:tasks.workunit.client.0.smithi107.stdout:6/992: mknod df/d17/d24/d30/d5d/c140 0 2024-03-04T13:43:25.568 INFO:tasks.workunit.client.0.smithi107.stdout:8/980: link d6/d16/fa3 d6/d16/d1f/d4c/da5/dfb/f138 0 2024-03-04T13:43:25.572 INFO:tasks.workunit.client.0.smithi107.stdout:4/979: read d0/d9/d10/d80/db8/fb4 [3877339,114645] 0 2024-03-04T13:43:25.573 INFO:tasks.workunit.client.0.smithi107.stdout:7/960: dread f0 [0,4194304] 0 2024-03-04T13:43:25.573 INFO:tasks.workunit.client.0.smithi107.stdout:6/993: mkdir df/d17/d24/d141 0 2024-03-04T13:43:25.575 INFO:tasks.workunit.client.0.smithi107.stdout:8/981: link d6/d16/f104 d6/da/d25/d113/f139 0 2024-03-04T13:43:25.584 INFO:tasks.workunit.client.0.smithi107.stdout:4/980: symlink d0/d9/de/d14/l13a 0 2024-03-04T13:43:25.605 INFO:tasks.workunit.client.0.smithi107.stdout:6/994: getdents df/d2e/d36/d5b/d7d/dbd/d104 0 2024-03-04T13:43:25.609 INFO:tasks.workunit.client.0.smithi107.stdout:6/995: mkdir df/d38/d142 0 2024-03-04T13:43:25.609 INFO:tasks.workunit.client.0.smithi107.stdout:8/982: dwrite d6/d16/d69/deb/d3e/f56 [4194304,4194304] 0 2024-03-04T13:43:25.615 INFO:tasks.workunit.client.0.smithi107.stdout:6/996: unlink df/d17/d24/d83/l13a 0 2024-03-04T13:43:25.615 INFO:tasks.workunit.client.0.smithi107.stdout:8/983: truncate d6/d16/d69/deb/d3e/d96/dd8/f114 861313 0 2024-03-04T13:43:25.622 INFO:tasks.workunit.client.0.smithi107.stdout:6/997: mkdir df/d17/d24/d40/d63/d11c/d111/d143 0 2024-03-04T13:43:25.624 INFO:tasks.workunit.client.0.smithi107.stdout:8/984: mkdir d6/d16/d69/deb/d13a 0 2024-03-04T13:43:25.627 INFO:tasks.workunit.client.0.smithi107.stdout:6/998: creat df/d2e/d36/d5b/f144 x:0 0 0 2024-03-04T13:43:25.629 INFO:tasks.workunit.client.0.smithi107.stdout:8/985: dread - d6/d16/d69/deb/d3e/fe4 zero size 2024-03-04T13:43:25.635 INFO:tasks.workunit.client.0.smithi107.stdout:6/999: symlink df/d2e/d36/d77/d118/dd8/d12c/l145 0 2024-03-04T13:43:25.651 INFO:tasks.workunit.client.0.smithi107.stdout:4/981: dread d0/d9/d10/d80/db8/d21/f5d [0,4194304] 0 2024-03-04T13:43:25.655 INFO:tasks.workunit.client.0.smithi107.stdout:2/970: sync 2024-03-04T13:43:25.655 INFO:tasks.workunit.client.0.smithi107.stdout:9/907: sync 2024-03-04T13:43:25.664 INFO:tasks.workunit.client.0.smithi107.stdout:8/986: write d6/d16/d1f/d63/d73/f79 [4634124,100186] 0 2024-03-04T13:43:25.665 INFO:tasks.workunit.client.0.smithi107.stdout:9/908: dread d4/da/d11/d20/d39/f90 [0,4194304] 0 2024-03-04T13:43:25.666 INFO:tasks.workunit.client.0.smithi107.stdout:4/982: write d0/d9/fb2 [1285652,122627] 0 2024-03-04T13:43:25.670 INFO:tasks.workunit.client.0.smithi107.stdout:9/909: truncate d4/da/d11/d20/d23/dcc/fee 726467 0 2024-03-04T13:43:25.672 INFO:tasks.workunit.client.0.smithi107.stdout:8/987: dread d6/d16/d69/deb/d3e/d67/f9e [0,4194304] 0 2024-03-04T13:43:25.676 INFO:tasks.workunit.client.0.smithi107.stdout:8/988: dread d6/d16/d69/deb/d3e/d96/dd8/f114 [0,4194304] 0 2024-03-04T13:43:25.677 INFO:tasks.workunit.client.0.smithi107.stdout:4/983: dread d0/d2f/f72 [0,4194304] 0 2024-03-04T13:43:25.678 INFO:tasks.workunit.client.0.smithi107.stdout:8/989: creat d6/d16/d1f/dbd/f13b x:0 0 0 2024-03-04T13:43:25.685 INFO:tasks.workunit.client.0.smithi107.stdout:9/910: dwrite d4/f26 [4194304,4194304] 0 2024-03-04T13:43:25.687 INFO:tasks.workunit.client.0.smithi107.stdout:9/911: stat d4/da/d11/d20/d39/f90 0 2024-03-04T13:43:25.687 INFO:tasks.workunit.client.0.smithi107.stdout:4/984: write d0/fe0 [1144166,72657] 0 2024-03-04T13:43:25.690 INFO:tasks.workunit.client.0.smithi107.stdout:8/990: write d6/d16/d69/deb/d7c/ff8 [341991,87479] 0 2024-03-04T13:43:25.697 INFO:tasks.workunit.client.0.smithi107.stdout:8/991: fdatasync d6/da/d25/dd1/fd2 0 2024-03-04T13:43:25.699 INFO:tasks.workunit.client.0.smithi107.stdout:4/985: creat d0/d9/d18/d29/f13b x:0 0 0 2024-03-04T13:43:25.711 INFO:tasks.workunit.client.0.smithi107.stdout:8/992: rename d6/d16/d1f/d63/f74 to d6/d16/d1f/f13c 0 2024-03-04T13:43:25.711 INFO:tasks.workunit.client.0.smithi107.stdout:9/912: dwrite d4/da/f4a [0,4194304] 0 2024-03-04T13:43:25.711 INFO:tasks.workunit.client.0.smithi107.stdout:4/986: write d0/d9/d18/d29/f89 [386963,74692] 0 2024-03-04T13:43:25.714 INFO:tasks.workunit.client.0.smithi107.stdout:4/987: write d0/d9/de/f43 [4417014,7619] 0 2024-03-04T13:43:25.714 INFO:tasks.workunit.client.0.smithi107.stdout:9/913: chown d4/da/d11/d65/ced 0 1 2024-03-04T13:43:25.718 INFO:tasks.workunit.client.0.smithi107.stdout:9/914: readlink d4/da/d11/d20/d23/l6a 0 2024-03-04T13:43:25.722 INFO:tasks.workunit.client.0.smithi107.stdout:9/915: chown d4/da/d11/d20/d23/d7e/lbd 9794608 1 2024-03-04T13:43:25.722 INFO:tasks.workunit.client.0.smithi107.stdout:4/988: creat d0/d9/d10/dab/f13c x:0 0 0 2024-03-04T13:43:25.722 INFO:tasks.workunit.client.0.smithi107.stdout:8/993: dread d6/d16/d1f/d63/d93/fce [0,4194304] 0 2024-03-04T13:43:25.725 INFO:tasks.workunit.client.0.smithi107.stdout:8/994: readlink d6/da/d25/d2a/ld4 0 2024-03-04T13:43:25.727 INFO:tasks.workunit.client.0.smithi107.stdout:9/916: symlink d4/da/d11/d20/d10f/l130 0 2024-03-04T13:43:25.729 INFO:tasks.workunit.client.0.smithi107.stdout:4/989: creat d0/d9/d18/d29/d122/f13d x:0 0 0 2024-03-04T13:43:25.730 INFO:tasks.workunit.client.0.smithi107.stdout:8/995: write d6/d16/d1f/d4c/da5/fc9 [5201363,17077] 0 2024-03-04T13:43:25.732 INFO:tasks.workunit.client.0.smithi107.stdout:9/917: mknod d4/da/d11/d20/d2f/d9c/d92/df6/c131 0 2024-03-04T13:43:25.735 INFO:tasks.workunit.client.0.smithi107.stdout:8/996: truncate d6/dcc/fff 635239 0 2024-03-04T13:43:25.737 INFO:tasks.workunit.client.0.smithi107.stdout:4/990: mknod d0/d9/d10/d80/d81/c13e 0 2024-03-04T13:43:25.740 INFO:tasks.workunit.client.0.smithi107.stdout:9/918: rename d4/da/d11/d65/d129 to d4/da/d11/d20/d23/d7e/d84/db3/d132 0 2024-03-04T13:43:25.745 INFO:tasks.workunit.client.0.smithi107.stdout:8/997: rename d6/da/da8/db5/fd6 to d6/da/d25/dd1/d101/f13d 0 2024-03-04T13:43:25.749 INFO:tasks.workunit.client.0.smithi107.stdout:9/919: unlink d4/da/d11/d4c/d4d/d72/d7b/f11c 0 2024-03-04T13:43:25.751 INFO:tasks.workunit.client.0.smithi107.stdout:7/961: sync 2024-03-04T13:43:25.757 INFO:tasks.workunit.client.0.smithi107.stdout:8/998: symlink d6/da/d25/d2a/d10a/l13e 0 2024-03-04T13:43:25.761 INFO:tasks.workunit.client.0.smithi107.stdout:4/991: link d0/d9/d18/df1/ld3 d0/d9/d10/d80/db8/d21/l13f 0 2024-03-04T13:43:25.762 INFO:tasks.workunit.client.0.smithi107.stdout:7/962: rmdir d6/d13/d16/d19/d60 39 2024-03-04T13:43:25.764 INFO:tasks.workunit.client.0.smithi107.stdout:7/963: chown d6/d13/d16/d19/d38/d2c/d8d/ce7 1226 1 2024-03-04T13:43:25.765 INFO:tasks.workunit.client.0.smithi107.stdout:8/999: rename d6/d16/d69/deb/d7c/ddf/c109 to d6/d16/d69/deb/d3e/c13f 0 2024-03-04T13:43:25.767 INFO:tasks.workunit.client.0.smithi107.stdout:7/964: chown d6/d13/d16/d19/d38/ff6 14125436 1 2024-03-04T13:43:25.769 INFO:tasks.workunit.client.0.smithi107.stdout:7/965: fdatasync d6/d1d/d64/d9a/da9/f126 0 2024-03-04T13:43:25.771 INFO:tasks.workunit.client.0.smithi107.stdout:4/992: rename d0/d9/d10/d80/db8/d21/f6c to d0/d9/d10/d80/db8/d97/de3/d108/f140 0 2024-03-04T13:43:25.773 INFO:tasks.workunit.client.0.smithi107.stdout:2/971: sync 2024-03-04T13:43:25.774 INFO:tasks.workunit.client.0.smithi107.stdout:7/966: creat d6/d116/f13c x:0 0 0 2024-03-04T13:43:25.776 INFO:tasks.workunit.client.0.smithi107.stdout:4/993: mkdir d0/d9/d10/d80/db8/d97/de3/d141 0 2024-03-04T13:43:25.780 INFO:tasks.workunit.client.0.smithi107.stdout:9/920: truncate d4/da/d11/d20/d2f/f3d 758321 0 2024-03-04T13:43:25.783 INFO:tasks.workunit.client.0.smithi107.stdout:9/921: stat d4/da/d11/d20/d2f/d9c/daa/db6 0 2024-03-04T13:43:25.786 INFO:tasks.workunit.client.0.smithi107.stdout:2/972: creat da/f146 x:0 0 0 2024-03-04T13:43:25.789 INFO:tasks.workunit.client.0.smithi107.stdout:4/994: rename d0/d9/d10/d80/db8/d97/de3/d108 to d0/d9/de/d14/d66/d7b/d142 0 2024-03-04T13:43:25.790 INFO:tasks.workunit.client.0.smithi107.stdout:4/995: fsync d0/fe0 0 2024-03-04T13:43:25.791 INFO:tasks.workunit.client.0.smithi107.stdout:4/996: stat d0/l5e 0 2024-03-04T13:43:25.791 INFO:tasks.workunit.client.0.smithi107.stdout:9/922: mknod d4/da/d11/d20/d23/d7e/c133 0 2024-03-04T13:43:25.797 INFO:tasks.workunit.client.0.smithi107.stdout:7/967: write d6/d13/d16/d19/d39/d7a/fc4 [865385,77975] 0 2024-03-04T13:43:25.799 INFO:tasks.workunit.client.0.smithi107.stdout:4/997: dread d0/d9/d10/d80/db8/f67 [0,4194304] 0 2024-03-04T13:43:25.800 INFO:tasks.workunit.client.0.smithi107.stdout:4/998: stat d0/d2f/d7d 0 2024-03-04T13:43:25.801 INFO:tasks.workunit.client.0.smithi107.stdout:9/923: creat d4/da/d11/d4c/d51/d122/f134 x:0 0 0 2024-03-04T13:43:25.804 INFO:tasks.workunit.client.0.smithi107.stdout:4/999: dread - d0/d2f/feb zero size 2024-03-04T13:43:25.804 INFO:tasks.workunit.client.0.smithi107.stdout:9/924: rmdir d4/d32 39 2024-03-04T13:43:25.805 INFO:tasks.workunit.client.0.smithi107.stdout:7/968: symlink d6/d1d/d64/l13d 0 2024-03-04T13:43:25.807 INFO:tasks.workunit.client.0.smithi107.stdout:7/969: readlink d6/d13/d16/d19/d38/d62/ddb/lf0 0 2024-03-04T13:43:25.809 INFO:tasks.workunit.client.0.smithi107.stdout:2/973: dread da/d44/d8b/d9d/df8/f7e [0,4194304] 0 2024-03-04T13:43:25.810 INFO:tasks.workunit.client.0.smithi107.stdout:7/970: mknod d6/d116/c13e 0 2024-03-04T13:43:25.811 INFO:tasks.workunit.client.0.smithi107.stdout:9/925: creat d4/da/d11/d65/f135 x:0 0 0 2024-03-04T13:43:25.813 INFO:tasks.workunit.client.0.smithi107.stdout:9/926: write d4/da/d11/d20/d64/d77/f82 [452485,6155] 0 2024-03-04T13:43:25.817 INFO:tasks.workunit.client.0.smithi107.stdout:9/927: symlink d4/da/d11/d20/d10f/l136 0 2024-03-04T13:43:25.821 INFO:tasks.workunit.client.0.smithi107.stdout:9/928: mkdir d4/da/d11/d20/d64/d77/ddc/d137 0 2024-03-04T13:43:25.823 INFO:tasks.workunit.client.0.smithi107.stdout:9/929: fsync d4/f79 0 2024-03-04T13:43:25.838 INFO:tasks.workunit.client.0.smithi107.stdout:7/971: dread d6/d1d/d64/d9a/da9/d71/db9/dca/dd0/f12e [0,4194304] 0 2024-03-04T13:43:25.838 INFO:tasks.workunit.client.0.smithi107.stdout:9/930: dwrite d4/da/d11/d20/d64/f76 [0,4194304] 0 2024-03-04T13:43:25.840 INFO:tasks.workunit.client.0.smithi107.stdout:7/972: chown d6/d13/d16/d19/faa 102923550 1 2024-03-04T13:43:25.842 INFO:tasks.workunit.client.0.smithi107.stdout:2/974: dread da/df/f15 [0,4194304] 0 2024-03-04T13:43:25.843 INFO:tasks.workunit.client.0.smithi107.stdout:9/931: mknod d4/da/d11/d20/d23/d7e/d84/d107/c138 0 2024-03-04T13:43:25.846 INFO:tasks.workunit.client.0.smithi107.stdout:9/932: readlink d4/da/d11/d20/d2f/d9c/daa/lbc 0 2024-03-04T13:43:25.853 INFO:tasks.workunit.client.0.smithi107.stdout:7/973: dread d6/f3f [0,4194304] 0 2024-03-04T13:43:25.856 INFO:tasks.workunit.client.0.smithi107.stdout:2/975: rmdir da/d39/d122 39 2024-03-04T13:43:25.859 INFO:tasks.workunit.client.0.smithi107.stdout:9/933: rmdir d4/da/d11/d20/d23/d7e/d84/db3/df0 0 2024-03-04T13:43:25.860 INFO:tasks.workunit.client.0.smithi107.stdout:7/974: mkdir d6/d1d/d64/d9a/da9/d71/db9/dca/d3d/dc6/d13f 0 2024-03-04T13:43:25.862 INFO:tasks.workunit.client.0.smithi107.stdout:2/976: fdatasync f0 0 2024-03-04T13:43:25.864 INFO:tasks.workunit.client.0.smithi107.stdout:7/975: mkdir d6/d13/d16/d1c/dc2/d140 0 2024-03-04T13:43:25.865 INFO:tasks.workunit.client.0.smithi107.stdout:2/977: rmdir da/df/d27/d10b/d5f 39 2024-03-04T13:43:25.873 INFO:tasks.workunit.client.0.smithi107.stdout:9/934: dwrite d4/da/d11/d65/f6e [0,4194304] 0 2024-03-04T13:43:25.875 INFO:tasks.workunit.client.0.smithi107.stdout:2/978: creat da/df/d85/d11a/f147 x:0 0 0 2024-03-04T13:43:25.882 INFO:tasks.workunit.client.0.smithi107.stdout:2/979: creat da/df/d73/d7a/f148 x:0 0 0 2024-03-04T13:43:25.882 INFO:tasks.workunit.client.0.smithi107.stdout:7/976: write d6/d1d/d64/d9a/da9/d71/db9/dca/d3d/dc6/ff7 [131091,78962] 0 2024-03-04T13:43:25.885 INFO:tasks.workunit.client.0.smithi107.stdout:7/977: truncate d6/d13/d16/d1c/faf 2218854 0 2024-03-04T13:43:25.887 INFO:tasks.workunit.client.0.smithi107.stdout:2/980: unlink da/df/c3d 0 2024-03-04T13:43:25.890 INFO:tasks.workunit.client.0.smithi107.stdout:7/978: truncate d6/d1d/d64/d9a/da9/d71/db9/ffc 871466 0 2024-03-04T13:43:25.892 INFO:tasks.workunit.client.0.smithi107.stdout:2/981: dread - da/d38/d3a/f115 zero size 2024-03-04T13:43:25.893 INFO:tasks.workunit.client.0.smithi107.stdout:7/979: symlink d6/d1d/d64/d9a/da9/d71/d115/l141 0 2024-03-04T13:43:25.894 INFO:tasks.workunit.client.0.smithi107.stdout:2/982: write da/df/d3e/de3/fea [905788,85397] 0 2024-03-04T13:43:25.896 INFO:tasks.workunit.client.0.smithi107.stdout:9/935: dread d4/f79 [0,4194304] 0 2024-03-04T13:43:25.899 INFO:tasks.workunit.client.0.smithi107.stdout:7/980: creat d6/d1d/d64/d9a/da9/d71/db9/dca/d3d/dc6/dcb/f142 x:0 0 0 2024-03-04T13:43:25.900 INFO:tasks.workunit.client.0.smithi107.stdout:2/983: rename da/d38/d3a/d5d/ff9 to da/df/d73/db3/db1/f149 0 2024-03-04T13:43:25.903 INFO:tasks.workunit.client.0.smithi107.stdout:7/981: mknod d6/d13/d16/d19/d38/c143 0 2024-03-04T13:43:25.903 INFO:tasks.workunit.client.0.smithi107.stdout:2/984: read da/f12b [783982,4884] 0 2024-03-04T13:43:25.905 INFO:tasks.workunit.client.0.smithi107.stdout:9/936: rename d4/da/d11/cb5 to d4/da/d11/d20/d2f/c139 0 2024-03-04T13:43:25.908 INFO:tasks.workunit.client.0.smithi107.stdout:2/985: mknod da/d44/d112/c14a 0 2024-03-04T13:43:25.929 INFO:tasks.workunit.client.0.smithi107.stdout:7/982: truncate d6/d1d/d64/d9a/da9/d71/db9/dca/d22/fd7 8363719 0 2024-03-04T13:43:25.932 INFO:tasks.workunit.client.0.smithi107.stdout:9/937: creat d4/da/d11/d20/d64/d77/df9/f13a x:0 0 0 2024-03-04T13:43:25.934 INFO:tasks.workunit.client.0.smithi107.stdout:7/983: getdents d6/d1d/d64/d9a 0 2024-03-04T13:43:25.936 INFO:tasks.workunit.client.0.smithi107.stdout:9/938: creat d4/da/d11/d20/d39/f13b x:0 0 0 2024-03-04T13:43:25.941 INFO:tasks.workunit.client.0.smithi107.stdout:2/986: truncate da/df/d27/d10b/d5f/f93 4109001 0 2024-03-04T13:43:25.944 INFO:tasks.workunit.client.0.smithi107.stdout:2/987: read da/d44/d8b/d9d/df8/f7e [113955,49697] 0 2024-03-04T13:43:25.951 INFO:tasks.workunit.client.0.smithi107.stdout:2/988: mknod da/df/d3e/dfe/c14b 0 2024-03-04T13:43:25.953 INFO:tasks.workunit.client.0.smithi107.stdout:9/939: dwrite d4/da/d11/d20/d2f/d9c/fb4 [0,4194304] 0 2024-03-04T13:43:25.955 INFO:tasks.workunit.client.0.smithi107.stdout:2/989: truncate da/d38/d3a/fd8 615488 0 2024-03-04T13:43:25.955 INFO:tasks.workunit.client.0.smithi107.stdout:9/940: truncate d4/da/f4a 5088738 0 2024-03-04T13:43:25.957 INFO:tasks.workunit.client.0.smithi107.stdout:2/990: stat da/d34/c4a 0 2024-03-04T13:43:25.960 INFO:tasks.workunit.client.0.smithi107.stdout:9/941: mknod d4/c13c 0 2024-03-04T13:43:25.963 INFO:tasks.workunit.client.0.smithi107.stdout:2/991: mkdir da/d44/d8b/d9d/df8/d14c 0 2024-03-04T13:43:25.968 INFO:tasks.workunit.client.0.smithi107.stdout:7/984: dread d6/d13/d16/d1c/f79 [0,4194304] 0 2024-03-04T13:43:25.969 INFO:tasks.workunit.client.0.smithi107.stdout:2/992: rename da/df/d27/d10b/d5f/c6b to da/d44/d8b/d9d/df8/d5c/d6a/c14d 0 2024-03-04T13:43:25.969 INFO:tasks.workunit.client.0.smithi107.stdout:7/985: write d6/d1d/d64/d9a/da9/d71/f103 [1148017,106839] 0 2024-03-04T13:43:25.970 INFO:tasks.workunit.client.0.smithi107.stdout:2/993: readlink da/d44/l13e 0 2024-03-04T13:43:25.972 INFO:tasks.workunit.client.0.smithi107.stdout:9/942: write d4/da/d11/d20/d39/fa7 [1036082,2873] 0 2024-03-04T13:43:25.974 INFO:tasks.workunit.client.0.smithi107.stdout:7/986: symlink d6/d13/d16/d19/d38/d62/ddb/l144 0 2024-03-04T13:43:25.978 INFO:tasks.workunit.client.0.smithi107.stdout:9/943: symlink d4/da/d11/d20/d23/dcc/de4/l13d 0 2024-03-04T13:43:25.981 INFO:tasks.workunit.client.0.smithi107.stdout:7/987: rmdir d6/d1d/d64/d9a/da9/d71/db9/dca/d3d/dc6/dcb/dd6 39 2024-03-04T13:43:25.985 INFO:tasks.workunit.client.0.smithi107.stdout:2/994: dwrite da/d44/d9c/f141 [0,4194304] 0 2024-03-04T13:43:25.990 INFO:tasks.workunit.client.0.smithi107.stdout:9/944: write d4/da/d11/d4c/d51/f5c [211623,61865] 0 2024-03-04T13:43:25.991 INFO:tasks.workunit.client.0.smithi107.stdout:7/988: truncate d6/d1d/d36/d59/fe9 2767779 0 2024-03-04T13:43:25.995 INFO:tasks.workunit.client.0.smithi107.stdout:2/995: dread da/d44/d8b/d9d/df8/fc0 [0,4194304] 0 2024-03-04T13:43:25.996 INFO:tasks.workunit.client.0.smithi107.stdout:9/945: creat d4/da/d11/d20/d2f/d9c/daa/db6/f13e x:0 0 0 2024-03-04T13:43:26.001 INFO:tasks.workunit.client.0.smithi107.stdout:2/996: mkdir da/d38/d113/d14e 0 2024-03-04T13:43:26.004 INFO:tasks.workunit.client.0.smithi107.stdout:2/997: chown da/d39/d122 97114901 1 2024-03-04T13:43:26.009 INFO:tasks.workunit.client.0.smithi107.stdout:2/998: chown da/d44/d9c 43 1 2024-03-04T13:43:26.012 INFO:tasks.workunit.client.0.smithi107.stdout:9/946: write d4/f21 [1466981,22314] 0 2024-03-04T13:43:26.014 INFO:tasks.workunit.client.0.smithi107.stdout:2/999: fsync da/d44/d8b/d9d/df8/d5c/f9e 0 2024-03-04T13:43:26.014 INFO:tasks.workunit.client.0.smithi107.stdout:7/989: dwrite d6/d1d/d64/d9a/da9/d71/f132 [0,4194304] 0 2024-03-04T13:43:26.019 INFO:tasks.workunit.client.0.smithi107.stdout:7/990: write d6/d13/d16/d19/d38/d62/f9b [155432,112806] 0 2024-03-04T13:43:26.023 INFO:tasks.workunit.client.0.smithi107.stdout:9/947: mkdir d4/da/d11/d20/d23/d13f 0 2024-03-04T13:43:26.028 INFO:tasks.workunit.client.0.smithi107.stdout:9/948: rmdir d4/da/d11/d20/d2f/dec 39 2024-03-04T13:43:26.042 INFO:tasks.workunit.client.0.smithi107.stdout:7/991: dwrite d6/d1d/d64/d9a/da9/d71/db9/dca/dd0/f12e [4194304,4194304] 0 2024-03-04T13:43:26.045 INFO:tasks.workunit.client.0.smithi107.stdout:9/949: dread d4/da/f4a [0,4194304] 0 2024-03-04T13:43:26.050 INFO:tasks.workunit.client.0.smithi107.stdout:7/992: rename d6/d13/d14/f5d to d6/d1d/d64/d9a/da9/d71/db9/dca/d22/f145 0 2024-03-04T13:43:26.073 INFO:tasks.workunit.client.0.smithi107.stdout:7/993: dread d6/d13/d14/f86 [4194304,4194304] 0 2024-03-04T13:43:26.079 INFO:tasks.workunit.client.0.smithi107.stdout:7/994: dread d6/d1d/d64/d9a/da9/d71/db9/dca/f6a [0,4194304] 0 2024-03-04T13:43:26.081 INFO:tasks.workunit.client.0.smithi107.stdout:7/995: creat d6/d1d/dd9/d106/f146 x:0 0 0 2024-03-04T13:43:26.085 INFO:tasks.workunit.client.0.smithi107.stdout:7/996: rename d6/d1d/d64/d9a/c11f to d6/d13/d16/d19/d38/d62/c147 0 2024-03-04T13:43:26.089 INFO:tasks.workunit.client.0.smithi107.stdout:7/997: truncate d6/d1d/d64/d9a/da9/d71/db9/dca/d3d/fb1 4353006 0 2024-03-04T13:43:26.101 INFO:tasks.workunit.client.0.smithi107.stdout:7/998: dwrite d6/d1d/d36/f5b [0,4194304] 0 2024-03-04T13:43:26.106 INFO:tasks.workunit.client.0.smithi107.stdout:7/999: truncate d6/d1d/d64/d9a/da9/d71/db9/dca/d3d/dc6/ff7 384097 0 2024-03-04T13:43:26.337 INFO:tasks.workunit.client.0.smithi107.stdout:9/950: sync 2024-03-04T13:43:26.348 INFO:tasks.workunit.client.0.smithi107.stdout:9/951: mknod d4/da/d11/d20/d64/d77/ddc/d137/c140 0 2024-03-04T13:43:26.357 INFO:tasks.workunit.client.0.smithi107.stdout:9/952: unlink d4/da/d11/d20/d23/f6c 0 2024-03-04T13:43:26.370 INFO:tasks.workunit.client.0.smithi107.stdout:9/953: dread d4/da/d11/d20/d23/dcc/de4/f116 [0,4194304] 0 2024-03-04T13:43:26.387 INFO:tasks.workunit.client.0.smithi107.stdout:9/954: creat d4/da/d11/d20/d23/d7e/d84/db3/d132/d12c/f141 x:0 0 0 2024-03-04T13:43:26.395 INFO:tasks.workunit.client.0.smithi107.stdout:9/955: symlink d4/da/d11/d4c/d4d/d72/d7b/l142 0 2024-03-04T13:43:26.429 INFO:tasks.workunit.client.0.smithi107.stdout:9/956: dwrite d4/da/d11/d4c/d51/fe2 [0,4194304] 0 2024-03-04T13:43:26.437 INFO:tasks.workunit.client.0.smithi107.stdout:9/957: chown d4/da/d11/d20/d2f/d9c/d92/df6/c131 588 1 2024-03-04T13:43:26.448 INFO:tasks.workunit.client.0.smithi107.stdout:9/958: fdatasync d4/da/d11/d20/d2f/fe6 0 2024-03-04T13:43:26.465 INFO:tasks.workunit.client.0.smithi107.stdout:9/959: write d4/da/d11/d20/d64/d77/fc4 [1000930,92293] 0 2024-03-04T13:43:26.472 INFO:tasks.workunit.client.0.smithi107.stdout:9/960: chown d4/da/d11/d20/d2f/d9c/d92/la0 45 1 2024-03-04T13:43:26.479 INFO:tasks.workunit.client.0.smithi107.stdout:9/961: unlink d4/da/ff8 0 2024-03-04T13:43:26.494 INFO:tasks.workunit.client.0.smithi107.stdout:9/962: rename d4/da/d11/d20/d2f/f85 to d4/da/d11/d20/d23/de8/f143 0 2024-03-04T13:43:26.508 INFO:tasks.workunit.client.0.smithi107.stdout:9/963: rename d4/da/d11/d20/d2f/d9c/d92/ccb to d4/da/d11/d20/d23/dcc/c144 0 2024-03-04T13:43:26.516 INFO:tasks.workunit.client.0.smithi107.stdout:9/964: creat d4/da/d11/d20/d23/dd9/f145 x:0 0 0 2024-03-04T13:43:26.610 INFO:tasks.workunit.client.0.smithi107.stdout:9/965: sync 2024-03-04T13:43:26.684 INFO:tasks.workunit.client.0.smithi107.stdout:9/966: dread d4/da/d11/d20/f28 [0,4194304] 0 2024-03-04T13:43:26.717 INFO:tasks.workunit.client.0.smithi107.stdout:9/967: write d4/d34/f87 [1040115,90538] 0 2024-03-04T13:43:26.721 INFO:tasks.workunit.client.0.smithi107.stdout:9/968: mknod d4/da/d11/d20/d23/d13f/c146 0 2024-03-04T13:43:26.723 INFO:tasks.workunit.client.0.smithi107.stdout:9/969: dread - d4/da/d11/d20/fd5 zero size 2024-03-04T13:43:26.726 INFO:tasks.workunit.client.0.smithi107.stdout:9/970: mknod d4/c147 0 2024-03-04T13:43:26.737 INFO:tasks.workunit.client.0.smithi107.stdout:9/971: dread d4/da/d11/d20/d64/f76 [0,4194304] 0 2024-03-04T13:43:26.741 INFO:tasks.workunit.client.0.smithi107.stdout:9/972: dread d4/da/d11/d20/d23/dcc/de4/f116 [0,4194304] 0 2024-03-04T13:43:26.745 INFO:tasks.workunit.client.0.smithi107.stdout:9/973: creat d4/da/d11/d20/d23/dd9/f148 x:0 0 0 2024-03-04T13:43:26.751 INFO:tasks.workunit.client.0.smithi107.stdout:9/974: rmdir d4/da/d11/d65 39 2024-03-04T13:43:26.756 INFO:tasks.workunit.client.0.smithi107.stdout:9/975: write d4/da/d11/d20/d39/fa7 [4100394,101608] 0 2024-03-04T13:43:26.761 INFO:tasks.workunit.client.0.smithi107.stdout:9/976: mkdir d4/da/d11/d20/d10f/d149 0 2024-03-04T13:43:26.764 INFO:tasks.workunit.client.0.smithi107.stdout:9/977: truncate d4/da/d11/d20/d64/f11e 617163 0 2024-03-04T13:43:26.765 INFO:tasks.workunit.client.0.smithi107.stdout:9/978: fsync d4/da/d11/f10d 0 2024-03-04T13:43:26.769 INFO:tasks.workunit.client.0.smithi107.stdout:9/979: symlink d4/da/d11/d20/d23/dcc/l14a 0 2024-03-04T13:43:26.774 INFO:tasks.workunit.client.0.smithi107.stdout:9/980: getdents d4/da/d11/d20/d23/de8 0 2024-03-04T13:43:26.778 INFO:tasks.workunit.client.0.smithi107.stdout:9/981: creat d4/da/d11/d20/d64/d77/df9/f14b x:0 0 0 2024-03-04T13:43:26.783 INFO:tasks.workunit.client.0.smithi107.stdout:9/982: creat d4/d32/f14c x:0 0 0 2024-03-04T13:43:26.793 INFO:tasks.workunit.client.0.smithi107.stdout:9/983: write d4/da/d11/d20/d23/dd9/fa9 [675828,99160] 0 2024-03-04T13:43:26.814 INFO:tasks.workunit.client.0.smithi107.stdout:9/984: dwrite d4/f19 [0,4194304] 0 2024-03-04T13:43:26.820 INFO:tasks.workunit.client.0.smithi107.stdout:9/985: write d4/da/d11/d20/d39/f90 [2107087,46433] 0 2024-03-04T13:43:26.827 INFO:tasks.workunit.client.0.smithi107.stdout:9/986: symlink d4/da/d11/d20/d2f/df7/l14d 0 2024-03-04T13:43:26.829 INFO:tasks.workunit.client.0.smithi107.stdout:9/987: read - d4/da/d11/d20/d23/dd9/f148 zero size 2024-03-04T13:43:26.831 INFO:tasks.workunit.client.0.smithi107.stdout:9/988: chown d4/c147 41366773 1 2024-03-04T13:43:26.832 INFO:tasks.workunit.client.0.smithi107.stdout:9/989: stat d4/da 0 2024-03-04T13:43:26.836 INFO:tasks.workunit.client.0.smithi107.stdout:9/990: mknod d4/da/d11/d20/d2f/d9c/c14e 0 2024-03-04T13:43:26.840 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:26 smithi107 ceph-mon[28739]: Standby manager daemon smithi107.wpzgwl restarted 2024-03-04T13:43:26.840 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:26 smithi107 ceph-mon[28739]: Standby manager daemon smithi107.wpzgwl started 2024-03-04T13:43:26.840 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:26 smithi107 ceph-mon[28739]: from='mgr.? 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi107.wpzgwl/crt"}]: dispatch 2024-03-04T13:43:26.840 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:26 smithi107 ceph-mon[28739]: from='mgr.? 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-04T13:43:26.840 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:26 smithi107 ceph-mon[28739]: from='mgr.? 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi107.wpzgwl/key"}]: dispatch 2024-03-04T13:43:26.840 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:26 smithi107 ceph-mon[28739]: from='mgr.? 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-04T13:43:26.840 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:26 smithi107 ceph-mon[28739]: pgmap v65: 65 pgs: 65 active+clean; 3.4 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 57 MiB/s rd, 129 MiB/s wr, 334 op/s 2024-03-04T13:43:27.098 INFO:tasks.workunit.client.0.smithi107.stdout:9/991: sync 2024-03-04T13:43:27.108 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:26 smithi169 ceph-mon[36313]: Standby manager daemon smithi107.wpzgwl restarted 2024-03-04T13:43:27.108 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:26 smithi169 ceph-mon[36313]: Standby manager daemon smithi107.wpzgwl started 2024-03-04T13:43:27.108 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:26 smithi169 ceph-mon[36313]: from='mgr.? 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi107.wpzgwl/crt"}]: dispatch 2024-03-04T13:43:27.108 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:26 smithi169 ceph-mon[36313]: from='mgr.? 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-04T13:43:27.108 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:26 smithi169 ceph-mon[36313]: from='mgr.? 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi107.wpzgwl/key"}]: dispatch 2024-03-04T13:43:27.108 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:26 smithi169 ceph-mon[36313]: from='mgr.? 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-04T13:43:27.108 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:26 smithi169 ceph-mon[36313]: pgmap v65: 65 pgs: 65 active+clean; 3.4 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 57 MiB/s rd, 129 MiB/s wr, 334 op/s 2024-03-04T13:43:27.114 INFO:tasks.workunit.client.0.smithi107.stdout:9/992: mkdir d4/da/d11/d20/d10f/d149/d14f 0 2024-03-04T13:43:27.122 INFO:tasks.workunit.client.0.smithi107.stdout:9/993: chown d4/da/d11/d4c/d51/l70 16026 1 2024-03-04T13:43:27.128 INFO:tasks.workunit.client.0.smithi107.stdout:9/994: write d4/da/d11/d4c/d51/f5c [2941022,109702] 0 2024-03-04T13:43:27.146 INFO:tasks.workunit.client.0.smithi107.stdout:9/995: creat d4/da/d11/d20/d2f/d9c/d92/df6/d121/f150 x:0 0 0 2024-03-04T13:43:27.366 INFO:tasks.workunit.client.0.smithi107.stdout:9/996: dwrite d4/da/d11/d20/d39/fdb [0,4194304] 0 2024-03-04T13:43:27.383 INFO:tasks.workunit.client.0.smithi107.stdout:9/997: dread d4/da/d11/d20/f28 [0,4194304] 0 2024-03-04T13:43:27.387 INFO:tasks.workunit.client.0.smithi107.stdout:9/998: truncate d4/da/d11/d20/d39/f13b 959438 0 2024-03-04T13:43:27.393 INFO:tasks.workunit.client.0.smithi107.stdout:9/999: mknod d4/da/d11/d20/d23/c151 0 2024-03-04T13:43:27.400 INFO:tasks.workunit.client.0.smithi107.stderr:+ rm -rf -- ./tmp.BZKaCoIoCT 2024-03-04T13:43:27.997 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:27 smithi107 ceph-mon[28739]: mgrmap e26: smithi169.eihczr(active, since 2m), standbys: smithi107.wpzgwl 2024-03-04T13:43:28.107 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:27 smithi169 ceph-mon[36313]: mgrmap e26: smithi169.eihczr(active, since 2m), standbys: smithi107.wpzgwl 2024-03-04T13:43:28.963 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:28 smithi107 ceph-mon[28739]: pgmap v66: 65 pgs: 65 active+clean; 3.2 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 73 MiB/s rd, 157 MiB/s wr, 442 op/s 2024-03-04T13:43:29.107 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:28 smithi169 ceph-mon[36313]: pgmap v66: 65 pgs: 65 active+clean; 3.2 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 73 MiB/s rd, 157 MiB/s wr, 442 op/s 2024-03-04T13:43:31.340 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:30 smithi107 ceph-mon[28739]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' 2024-03-04T13:43:31.340 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:30 smithi107 ceph-mon[28739]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' 2024-03-04T13:43:31.340 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:30 smithi107 ceph-mon[28739]: pgmap v67: 65 pgs: 65 active+clean; 3.2 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 55 MiB/s rd, 111 MiB/s wr, 311 op/s 2024-03-04T13:43:31.358 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:30 smithi169 ceph-mon[36313]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' 2024-03-04T13:43:31.358 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:30 smithi169 ceph-mon[36313]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' 2024-03-04T13:43:31.358 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:30 smithi169 ceph-mon[36313]: pgmap v67: 65 pgs: 65 active+clean; 3.2 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 55 MiB/s rd, 111 MiB/s wr, 311 op/s 2024-03-04T13:43:33.186 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:32 smithi107 ceph-mon[28739]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' 2024-03-04T13:43:33.187 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:32 smithi107 ceph-mon[28739]: pgmap v68: 65 pgs: 65 active+clean; 3.2 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 49 MiB/s rd, 95 MiB/s wr, 290 op/s 2024-03-04T13:43:33.187 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:32 smithi107 ceph-mon[28739]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' 2024-03-04T13:43:33.357 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:32 smithi169 ceph-mon[36313]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' 2024-03-04T13:43:33.358 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:32 smithi169 ceph-mon[36313]: pgmap v68: 65 pgs: 65 active+clean; 3.2 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 49 MiB/s rd, 95 MiB/s wr, 290 op/s 2024-03-04T13:43:33.358 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:32 smithi169 ceph-mon[36313]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' 2024-03-04T13:43:35.209 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:35 smithi107 ceph-mon[28739]: pgmap v69: 65 pgs: 65 active+clean; 2.6 GiB data, 9.4 GiB used, 527 GiB / 536 GiB avail; 49 MiB/s rd, 95 MiB/s wr, 347 op/s 2024-03-04T13:43:35.357 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:35 smithi169 ceph-mon[36313]: pgmap v69: 65 pgs: 65 active+clean; 2.6 GiB data, 9.4 GiB used, 527 GiB / 536 GiB avail; 49 MiB/s rd, 95 MiB/s wr, 347 op/s 2024-03-04T13:43:37.032 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:36 smithi107 ceph-mon[28739]: pgmap v70: 65 pgs: 65 active+clean; 2.6 GiB data, 9.4 GiB used, 527 GiB / 536 GiB avail; 19 MiB/s rd, 35 MiB/s wr, 176 op/s 2024-03-04T13:43:37.107 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:36 smithi169 ceph-mon[36313]: pgmap v70: 65 pgs: 65 active+clean; 2.6 GiB data, 9.4 GiB used, 527 GiB / 536 GiB avail; 19 MiB/s rd, 35 MiB/s wr, 176 op/s 2024-03-04T13:43:38.090 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:37 smithi107 ceph-mon[28739]: from='mgr.24331 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:43:38.107 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:37 smithi169 ceph-mon[36313]: from='mgr.24331 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:43:38.817 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:38 smithi107 ceph-mon[28739]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' 2024-03-04T13:43:38.817 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:38 smithi107 ceph-mon[28739]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' 2024-03-04T13:43:38.817 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:38 smithi107 ceph-mon[28739]: from='mgr.24331 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:43:38.817 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:38 smithi107 ceph-mon[28739]: from='mgr.24331 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:43:38.817 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:38 smithi107 ceph-mon[28739]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' 2024-03-04T13:43:38.817 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:38 smithi107 ceph-mon[28739]: from='mgr.24331 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:43:38.817 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:38 smithi107 ceph-mon[28739]: from='mgr.24331 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "mgr fail", "who": "smithi169.eihczr"}]: dispatch 2024-03-04T13:43:38.817 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:38 smithi107 ceph-mon[28739]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "mgr fail", "who": "smithi169.eihczr"}]: dispatch 2024-03-04T13:43:38.817 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:38 smithi107 ceph-mon[28739]: osdmap e49: 6 total, 6 up, 6 in 2024-03-04T13:43:38.817 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:38 smithi107 ceph-mon[28739]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' cmd='[{"prefix": "mgr fail", "who": "smithi169.eihczr"}]': finished 2024-03-04T13:43:38.817 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:38 smithi107 ceph-mon[28739]: mgrmap e27: smithi107.wpzgwl(active, starting, since 0.0153267s) 2024-03-04T13:43:38.818 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:38 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mon metadata", "id": "smithi107"}]: dispatch 2024-03-04T13:43:38.818 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:38 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mon metadata", "id": "smithi169"}]: dispatch 2024-03-04T13:43:38.818 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:38 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi107.thycho"}]: dispatch 2024-03-04T13:43:38.818 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:38 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi107.riexmm"}]: dispatch 2024-03-04T13:43:38.818 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:38 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi169.jusudl"}]: dispatch 2024-03-04T13:43:38.818 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:38 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi169.bxitzk"}]: dispatch 2024-03-04T13:43:38.818 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:38 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mgr metadata", "who": "smithi107.wpzgwl", "id": "smithi107.wpzgwl"}]: dispatch 2024-03-04T13:43:38.818 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:38 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-04T13:43:38.818 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:38 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-04T13:43:38.818 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:38 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-04T13:43:38.818 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:38 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-04T13:43:38.818 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:38 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-04T13:43:38.818 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:38 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-04T13:43:38.818 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:38 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-04T13:43:38.819 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:38 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-04T13:43:38.819 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:38 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-04T13:43:38.819 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:38 smithi107 ceph-mon[28739]: Manager daemon smithi107.wpzgwl is now available 2024-03-04T13:43:38.819 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:38 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:43:38.819 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:38 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:43:38.819 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:38 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:43:38.819 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:38 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi107.wpzgwl/mirror_snapshot_schedule"}]: dispatch 2024-03-04T13:43:38.819 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:38 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi107.wpzgwl/trash_purge_schedule"}]: dispatch 2024-03-04T13:43:38.819 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:38 smithi107 ceph-mon[28739]: [04/Mar/2024:13:43:38] ENGINE Bus STARTING 2024-03-04T13:43:39.109 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:38 smithi169 ceph-mon[36313]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' 2024-03-04T13:43:39.109 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:38 smithi169 ceph-mon[36313]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' 2024-03-04T13:43:39.109 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:38 smithi169 ceph-mon[36313]: from='mgr.24331 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:43:39.109 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:38 smithi169 ceph-mon[36313]: from='mgr.24331 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:43:39.109 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:38 smithi169 ceph-mon[36313]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' 2024-03-04T13:43:39.109 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:38 smithi169 ceph-mon[36313]: from='mgr.24331 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:43:39.109 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:38 smithi169 ceph-mon[36313]: from='mgr.24331 172.21.15.169:0/2514166165' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "mgr fail", "who": "smithi169.eihczr"}]: dispatch 2024-03-04T13:43:39.109 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:38 smithi169 ceph-mon[36313]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "mgr fail", "who": "smithi169.eihczr"}]: dispatch 2024-03-04T13:43:39.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:38 smithi169 ceph-mon[36313]: osdmap e49: 6 total, 6 up, 6 in 2024-03-04T13:43:39.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:38 smithi169 ceph-mon[36313]: from='mgr.24331 ' entity='mgr.smithi169.eihczr' cmd='[{"prefix": "mgr fail", "who": "smithi169.eihczr"}]': finished 2024-03-04T13:43:39.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:38 smithi169 ceph-mon[36313]: mgrmap e27: smithi107.wpzgwl(active, starting, since 0.0153267s) 2024-03-04T13:43:39.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:38 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mon metadata", "id": "smithi107"}]: dispatch 2024-03-04T13:43:39.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:38 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mon metadata", "id": "smithi169"}]: dispatch 2024-03-04T13:43:39.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:38 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi107.thycho"}]: dispatch 2024-03-04T13:43:39.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:38 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi107.riexmm"}]: dispatch 2024-03-04T13:43:39.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:38 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi169.jusudl"}]: dispatch 2024-03-04T13:43:39.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:38 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi169.bxitzk"}]: dispatch 2024-03-04T13:43:39.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:38 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mgr metadata", "who": "smithi107.wpzgwl", "id": "smithi107.wpzgwl"}]: dispatch 2024-03-04T13:43:39.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:38 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-04T13:43:39.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:38 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-04T13:43:39.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:38 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-04T13:43:39.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:38 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-04T13:43:39.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:38 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-04T13:43:39.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:38 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-04T13:43:39.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:38 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-04T13:43:39.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:38 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-04T13:43:39.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:38 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-04T13:43:39.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:38 smithi169 ceph-mon[36313]: Manager daemon smithi107.wpzgwl is now available 2024-03-04T13:43:39.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:38 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:43:39.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:38 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:43:39.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:38 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:43:39.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:38 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi107.wpzgwl/mirror_snapshot_schedule"}]: dispatch 2024-03-04T13:43:39.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:38 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi107.wpzgwl/trash_purge_schedule"}]: dispatch 2024-03-04T13:43:39.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:38 smithi169 ceph-mon[36313]: [04/Mar/2024:13:43:38] ENGINE Bus STARTING 2024-03-04T13:43:39.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:39.524+0000 7f4dcea00700 1 -- 172.21.15.107:0/3809306308 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4dc80fda80 msgr2=0x7f4dc80ffe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:43:39.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:39.524+0000 7f4dcea00700 1 --2- 172.21.15.107:0/3809306308 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4dc80fda80 0x7f4dc80ffe60 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f4db8004940 tx=0x7f4db8009e30 comp rx=0 tx=0).stop 2024-03-04T13:43:39.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:39.524+0000 7f4dcea00700 1 -- 172.21.15.107:0/3809306308 shutdown_connections 2024-03-04T13:43:39.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:39.524+0000 7f4dcea00700 1 --2- 172.21.15.107:0/3809306308 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4dc81003a0 0x7f4dc8102780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:43:39.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:39.524+0000 7f4dcea00700 1 --2- 172.21.15.107:0/3809306308 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4dc80fda80 0x7f4dc80ffe60 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:43:39.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:39.524+0000 7f4dcea00700 1 -- 172.21.15.107:0/3809306308 >> 172.21.15.107:0/3809306308 conn(0x7f4dc80f74e0 msgr2=0x7f4dc80f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:43:39.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:39.524+0000 7f4dcea00700 1 -- 172.21.15.107:0/3809306308 shutdown_connections 2024-03-04T13:43:39.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:39.524+0000 7f4dcea00700 1 -- 172.21.15.107:0/3809306308 wait complete. 2024-03-04T13:43:39.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:39.525+0000 7f4dcea00700 1 Processor -- start 2024-03-04T13:43:39.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:39.525+0000 7f4dcea00700 1 -- start start 2024-03-04T13:43:39.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:39.525+0000 7f4dcea00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4dc80fda80 0x7f4dc810fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:43:39.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:39.525+0000 7f4dcea00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4dc81003a0 0x7f4dc8110070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:43:39.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:39.526+0000 7f4dcea00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dc8110670 con 0x7f4dc80fda80 2024-03-04T13:43:39.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:39.526+0000 7f4dcea00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dc8110f10 con 0x7f4dc81003a0 2024-03-04T13:43:39.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:39.526+0000 7f4dc7400700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4dc81003a0 0x7f4dc8110070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:43:39.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:39.526+0000 7f4dc7400700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4dc81003a0 0x7f4dc8110070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.107:55544/0 (socket says 172.21.15.107:55544) 2024-03-04T13:43:39.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:39.526+0000 7f4dc7e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4dc80fda80 0x7f4dc810fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:43:39.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:39.526+0000 7f4dc7400700 1 -- 172.21.15.107:0/1947070230 learned_addr learned my addr 172.21.15.107:0/1947070230 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:43:39.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:39.527+0000 7f4dc7e00700 1 -- 172.21.15.107:0/1947070230 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4dc81003a0 msgr2=0x7f4dc8110070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:43:39.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:39.527+0000 7f4dc7e00700 1 --2- 172.21.15.107:0/1947070230 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4dc81003a0 0x7f4dc8110070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:43:39.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:39.527+0000 7f4dc7e00700 1 -- 172.21.15.107:0/1947070230 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4db8005040 con 0x7f4dc80fda80 2024-03-04T13:43:39.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:39.527+0000 7f4dc7e00700 1 --2- 172.21.15.107:0/1947070230 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4dc80fda80 0x7f4dc810fb30 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f4db8000c00 tx=0x7f4db8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:43:39.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:39.527+0000 7f4dbfe00700 1 -- 172.21.15.107:0/1947070230 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4db800e040 con 0x7f4dc80fda80 2024-03-04T13:43:39.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:39.527+0000 7f4dbfe00700 1 -- 172.21.15.107:0/1947070230 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4db8004290 con 0x7f4dc80fda80 2024-03-04T13:43:39.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:39.527+0000 7f4dcea00700 1 -- 172.21.15.107:0/1947070230 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4dc8111190 con 0x7f4dc80fda80 2024-03-04T13:43:39.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:39.527+0000 7f4dbfe00700 1 -- 172.21.15.107:0/1947070230 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4db801c3f0 con 0x7f4dc80fda80 2024-03-04T13:43:39.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:39.527+0000 7f4dcea00700 1 -- 172.21.15.107:0/1947070230 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dc81116d0 con 0x7f4dc80fda80 2024-03-04T13:43:39.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:39.528+0000 7f4dcea00700 1 -- 172.21.15.107:0/1947070230 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4dc804bed0 con 0x7f4dc80fda80 2024-03-04T13:43:39.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:39.529+0000 7f4dbfe00700 1 -- 172.21.15.107:0/1947070230 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 28) v1 ==== 45577+0+0 (secure 0 0 0) 0x7f4db800bbf0 con 0x7f4dc80fda80 2024-03-04T13:43:39.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:39.529+0000 7f4dbfe00700 1 --2- 172.21.15.107:0/1947070230 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f4dac038650 0x7f4dac03ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:43:39.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:39.529+0000 7f4dbfe00700 1 -- 172.21.15.107:0/1947070230 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f4db804d290 con 0x7f4dc80fda80 2024-03-04T13:43:39.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:39.530+0000 7f4dc7400700 1 --2- 172.21.15.107:0/1947070230 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f4dac038650 0x7f4dac03ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:43:39.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:39.530+0000 7f4dc7400700 1 --2- 172.21.15.107:0/1947070230 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f4dac038650 0x7f4dac03ab00 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f4db40041d0 tx=0x7f4db400b040 comp rx=0 tx=0).ready entity=mgr.14696 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:43:39.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:39.535+0000 7f4dbfe00700 1 -- 172.21.15.107:0/1947070230 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f4db8011550 con 0x7f4dc80fda80 2024-03-04T13:43:39.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:39.831+0000 7f4dcea00700 1 -- 172.21.15.107:0/1947070230 --> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4dc80fbf80 con 0x7f4dac038650 2024-03-04T13:43:39.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:39.834+0000 7f4dbfe00700 1 -- 172.21.15.107:0/1947070230 <== mgr.14696 v2:172.21.15.107:6800/1613321046 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7f4dc80fbf80 con 0x7f4dac038650 2024-03-04T13:43:39.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:39.837+0000 7f4dcea00700 1 -- 172.21.15.107:0/1947070230 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f4dac038650 msgr2=0x7f4dac03ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:43:39.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:39.837+0000 7f4dcea00700 1 --2- 172.21.15.107:0/1947070230 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f4dac038650 0x7f4dac03ab00 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f4db40041d0 tx=0x7f4db400b040 comp rx=0 tx=0).stop 2024-03-04T13:43:39.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:39.838+0000 7f4dcea00700 1 -- 172.21.15.107:0/1947070230 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4dc80fda80 msgr2=0x7f4dc810fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:43:39.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:39.838+0000 7f4dcea00700 1 --2- 172.21.15.107:0/1947070230 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4dc80fda80 0x7f4dc810fb30 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f4db8000c00 tx=0x7f4db8009e30 comp rx=0 tx=0).stop 2024-03-04T13:43:39.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:39.838+0000 7f4dcea00700 1 -- 172.21.15.107:0/1947070230 shutdown_connections 2024-03-04T13:43:39.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:39.838+0000 7f4dcea00700 1 --2- 172.21.15.107:0/1947070230 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f4dac038650 0x7f4dac03ab00 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:43:39.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:39.838+0000 7f4dcea00700 1 --2- 172.21.15.107:0/1947070230 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4dc80fda80 0x7f4dc810fb30 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:43:39.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:39.838+0000 7f4dcea00700 1 --2- 172.21.15.107:0/1947070230 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4dc81003a0 0x7f4dc8110070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:43:39.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:39.838+0000 7f4dcea00700 1 -- 172.21.15.107:0/1947070230 >> 172.21.15.107:0/1947070230 conn(0x7f4dc80f74e0 msgr2=0x7f4dc80fb870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:43:39.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:39.839+0000 7f4dcea00700 1 -- 172.21.15.107:0/1947070230 shutdown_connections 2024-03-04T13:43:39.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:39.839+0000 7f4dcea00700 1 -- 172.21.15.107:0/1947070230 wait complete. 2024-03-04T13:43:39.858 INFO:teuthology.orchestra.run.smithi107.stdout:true 2024-03-04T13:43:40.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.026+0000 7fcd04800700 1 -- 172.21.15.107:0/2380976096 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fccfc0fe0e0 msgr2=0x7fccfc0fe550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:43:40.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.026+0000 7fcd04800700 1 --2- 172.21.15.107:0/2380976096 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fccfc0fe0e0 0x7fccfc0fe550 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fccec004940 tx=0x7fccec009e30 comp rx=0 tx=0).stop 2024-03-04T13:43:40.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.026+0000 7fcd04800700 1 -- 172.21.15.107:0/2380976096 shutdown_connections 2024-03-04T13:43:40.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.026+0000 7fcd04800700 1 --2- 172.21.15.107:0/2380976096 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fccfc0fe0e0 0x7fccfc0fe550 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:43:40.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.026+0000 7fcd04800700 1 --2- 172.21.15.107:0/2380976096 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fccfc0fcef0 0x7fccfc0fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:43:40.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.026+0000 7fcd04800700 1 -- 172.21.15.107:0/2380976096 >> 172.21.15.107:0/2380976096 conn(0x7fccfc0f8620 msgr2=0x7fccfc0faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:43:40.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.027+0000 7fcd04800700 1 -- 172.21.15.107:0/2380976096 shutdown_connections 2024-03-04T13:43:40.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.027+0000 7fcd04800700 1 -- 172.21.15.107:0/2380976096 wait complete. 2024-03-04T13:43:40.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.028+0000 7fcd04800700 1 Processor -- start 2024-03-04T13:43:40.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.028+0000 7fcd04800700 1 -- start start 2024-03-04T13:43:40.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.028+0000 7fcd04800700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fccfc0fcef0 0x7fccfc10d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:43:40.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.028+0000 7fcd04800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fccfc0fe0e0 0x7fccfc10de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:43:40.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.029+0000 7fcd04800700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccfc10e410 con 0x7fccfc0fe0e0 2024-03-04T13:43:40.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.029+0000 7fcd04800700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccfc10ecb0 con 0x7fccfc0fcef0 2024-03-04T13:43:40.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.029+0000 7fcd01400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fccfc0fe0e0 0x7fccfc10de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:43:40.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.029+0000 7fcd01400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fccfc0fe0e0 0x7fccfc10de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59042/0 (socket says 172.21.15.107:59042) 2024-03-04T13:43:40.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.029+0000 7fcd01400700 1 -- 172.21.15.107:0/3914159465 learned_addr learned my addr 172.21.15.107:0/3914159465 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:43:40.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.029+0000 7fcd01e00700 1 --2- 172.21.15.107:0/3914159465 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fccfc0fcef0 0x7fccfc10d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:43:40.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.029+0000 7fcd01400700 1 -- 172.21.15.107:0/3914159465 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fccfc0fcef0 msgr2=0x7fccfc10d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:43:40.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.029+0000 7fcd01400700 1 --2- 172.21.15.107:0/3914159465 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fccfc0fcef0 0x7fccfc10d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:43:40.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.029+0000 7fcd01400700 1 -- 172.21.15.107:0/3914159465 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccec005040 con 0x7fccfc0fe0e0 2024-03-04T13:43:40.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.030+0000 7fcd01400700 1 --2- 172.21.15.107:0/3914159465 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fccfc0fe0e0 0x7fccfc10de10 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fccec00b3b0 tx=0x7fccec009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:43:40.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.030+0000 7fccf6000700 1 -- 172.21.15.107:0/3914159465 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fccec00e040 con 0x7fccfc0fe0e0 2024-03-04T13:43:40.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.030+0000 7fccf6000700 1 -- 172.21.15.107:0/3914159465 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fccec00bd90 con 0x7fccfc0fe0e0 2024-03-04T13:43:40.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.030+0000 7fcd04800700 1 -- 172.21.15.107:0/3914159465 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccfc10ef30 con 0x7fccfc0fe0e0 2024-03-04T13:43:40.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.030+0000 7fccf6000700 1 -- 172.21.15.107:0/3914159465 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fccec01c630 con 0x7fccfc0fe0e0 2024-03-04T13:43:40.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.030+0000 7fcd04800700 1 -- 172.21.15.107:0/3914159465 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccfc10f470 con 0x7fccfc0fe0e0 2024-03-04T13:43:40.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.032+0000 7fcd04800700 1 -- 172.21.15.107:0/3914159465 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccfc04bed0 con 0x7fccfc0fe0e0 2024-03-04T13:43:40.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.033+0000 7fccf6000700 1 -- 172.21.15.107:0/3914159465 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 28) v1 ==== 45577+0+0 (secure 0 0 0) 0x7fccec0139f0 con 0x7fccfc0fe0e0 2024-03-04T13:43:40.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.033+0000 7fccf6000700 1 --2- 172.21.15.107:0/3914159465 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7fcce40386a0 0x7fcce403ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:43:40.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.033+0000 7fccf6000700 1 -- 172.21.15.107:0/3914159465 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fccec04e030 con 0x7fccfc0fe0e0 2024-03-04T13:43:40.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.034+0000 7fcd01e00700 1 --2- 172.21.15.107:0/3914159465 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7fcce40386a0 0x7fcce403ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:43:40.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.034+0000 7fcd01e00700 1 --2- 172.21.15.107:0/3914159465 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7fcce40386a0 0x7fcce403ab50 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fccfc0fcc30 tx=0x7fccf0007ea0 comp rx=0 tx=0).ready entity=mgr.14696 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:43:40.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.043+0000 7fccf6000700 1 -- 172.21.15.107:0/3914159465 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7fccec01c790 con 0x7fccfc0fe0e0 2024-03-04T13:43:40.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.127+0000 7fccf6000700 1 -- 172.21.15.107:0/3914159465 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mgrmap(e 29) v1 ==== 45873+0+0 (secure 0 0 0) 0x7fccec04bea0 con 0x7fccfc0fe0e0 2024-03-04T13:43:40.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.323+0000 7fcd04800700 1 -- 172.21.15.107:0/3914159465 --> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fccfc102ac0 con 0x7fcce40386a0 2024-03-04T13:43:40.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.325+0000 7fccf6000700 1 -- 172.21.15.107:0/3914159465 <== mgr.14696 v2:172.21.15.107:6800/1613321046 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7fccfc102ac0 con 0x7fcce40386a0 2024-03-04T13:43:40.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.328+0000 7fcd04800700 1 -- 172.21.15.107:0/3914159465 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7fcce40386a0 msgr2=0x7fcce403ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:43:40.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.328+0000 7fcd04800700 1 --2- 172.21.15.107:0/3914159465 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7fcce40386a0 0x7fcce403ab50 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fccfc0fcc30 tx=0x7fccf0007ea0 comp rx=0 tx=0).stop 2024-03-04T13:43:40.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.328+0000 7fcd04800700 1 -- 172.21.15.107:0/3914159465 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fccfc0fe0e0 msgr2=0x7fccfc10de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:43:40.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.328+0000 7fcd04800700 1 --2- 172.21.15.107:0/3914159465 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fccfc0fe0e0 0x7fccfc10de10 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fccec00b3b0 tx=0x7fccec009e30 comp rx=0 tx=0).stop 2024-03-04T13:43:40.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.328+0000 7fcd04800700 1 -- 172.21.15.107:0/3914159465 shutdown_connections 2024-03-04T13:43:40.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.328+0000 7fcd04800700 1 --2- 172.21.15.107:0/3914159465 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fccfc0fcef0 0x7fccfc10d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:43:40.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.328+0000 7fcd04800700 1 --2- 172.21.15.107:0/3914159465 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7fcce40386a0 0x7fcce403ab50 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:43:40.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.328+0000 7fcd04800700 1 --2- 172.21.15.107:0/3914159465 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fccfc0fe0e0 0x7fccfc10de10 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:43:40.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.328+0000 7fcd04800700 1 -- 172.21.15.107:0/3914159465 >> 172.21.15.107:0/3914159465 conn(0x7fccfc0f8620 msgr2=0x7fccfc1013a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:43:40.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.328+0000 7fcd04800700 1 -- 172.21.15.107:0/3914159465 shutdown_connections 2024-03-04T13:43:40.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.329+0000 7fcd04800700 1 -- 172.21.15.107:0/3914159465 wait complete. 2024-03-04T13:43:40.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.480+0000 7ff307e00700 1 -- 172.21.15.107:0/3539361363 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff3080fef70 msgr2=0x7ff3080ff380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:43:40.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.480+0000 7ff307e00700 1 --2- 172.21.15.107:0/3539361363 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff3080fef70 0x7ff3080ff380 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7ff2f8004940 tx=0x7ff2f8009e30 comp rx=0 tx=0).stop 2024-03-04T13:43:40.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.480+0000 7ff307e00700 1 -- 172.21.15.107:0/3539361363 shutdown_connections 2024-03-04T13:43:40.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.480+0000 7ff307e00700 1 --2- 172.21.15.107:0/3539361363 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff308100180 0x7ff3081005d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:43:40.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.480+0000 7ff307e00700 1 --2- 172.21.15.107:0/3539361363 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff3080fef70 0x7ff3080ff380 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:43:40.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.480+0000 7ff307e00700 1 -- 172.21.15.107:0/3539361363 >> 172.21.15.107:0/3539361363 conn(0x7ff3080fa760 msgr2=0x7ff3080fcb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:43:40.482 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:40 smithi107 ceph-mon[28739]: [04/Mar/2024:13:43:38] ENGINE Serving on https://172.21.15.107:7150 2024-03-04T13:43:40.482 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:40 smithi107 ceph-mon[28739]: [04/Mar/2024:13:43:38] ENGINE Serving on http://172.21.15.107:8765 2024-03-04T13:43:40.482 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:40 smithi107 ceph-mon[28739]: [04/Mar/2024:13:43:38] ENGINE Bus STARTED 2024-03-04T13:43:40.482 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:40 smithi107 ceph-mon[28739]: mgrmap e28: smithi107.wpzgwl(active, since 1.01992s) 2024-03-04T13:43:40.482 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:40 smithi107 ceph-mon[28739]: pgmap v3: 65 pgs: 65 active+clean; 1.7 GiB data, 7.1 GiB used, 529 GiB / 536 GiB avail 2024-03-04T13:43:40.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.483+0000 7ff307e00700 1 -- 172.21.15.107:0/3539361363 shutdown_connections 2024-03-04T13:43:40.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.483+0000 7ff307e00700 1 -- 172.21.15.107:0/3539361363 wait complete. 2024-03-04T13:43:40.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.484+0000 7ff307e00700 1 Processor -- start 2024-03-04T13:43:40.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.484+0000 7ff307e00700 1 -- start start 2024-03-04T13:43:40.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.485+0000 7ff307e00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff3080fef70 0x7ff30810b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:43:40.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.485+0000 7ff307e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff308100180 0x7ff30810b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:43:40.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.485+0000 7ff307e00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff30810bfe0 con 0x7ff308100180 2024-03-04T13:43:40.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.485+0000 7ff307e00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff30810c880 con 0x7ff3080fef70 2024-03-04T13:43:40.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.485+0000 7ff306a00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff3080fef70 0x7ff30810b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:43:40.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.485+0000 7ff306a00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff3080fef70 0x7ff30810b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.107:55604/0 (socket says 172.21.15.107:55604) 2024-03-04T13:43:40.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.485+0000 7ff306a00700 1 -- 172.21.15.107:0/799457540 learned_addr learned my addr 172.21.15.107:0/799457540 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:43:40.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.485+0000 7ff306000700 1 --2- 172.21.15.107:0/799457540 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff308100180 0x7ff30810b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:43:40.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.486+0000 7ff306a00700 1 -- 172.21.15.107:0/799457540 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff308100180 msgr2=0x7ff30810b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:43:40.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.486+0000 7ff306a00700 1 --2- 172.21.15.107:0/799457540 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff308100180 0x7ff30810b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:43:40.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.486+0000 7ff306a00700 1 -- 172.21.15.107:0/799457540 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2f8005040 con 0x7ff3080fef70 2024-03-04T13:43:40.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.486+0000 7ff306a00700 1 --2- 172.21.15.107:0/799457540 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff3080fef70 0x7ff30810b450 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff2f8000c00 tx=0x7ff2f8009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:43:40.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.487+0000 7ff2fea00700 1 -- 172.21.15.107:0/799457540 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff2f8021070 con 0x7ff3080fef70 2024-03-04T13:43:40.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.487+0000 7ff307e00700 1 -- 172.21.15.107:0/799457540 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff30810cb00 con 0x7ff3080fef70 2024-03-04T13:43:40.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.487+0000 7ff2fea00700 1 -- 172.21.15.107:0/799457540 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff2f801c400 con 0x7ff3080fef70 2024-03-04T13:43:40.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.487+0000 7ff2fea00700 1 -- 172.21.15.107:0/799457540 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff2f800e070 con 0x7ff3080fef70 2024-03-04T13:43:40.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.488+0000 7ff307e00700 1 -- 172.21.15.107:0/799457540 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff30810cfe0 con 0x7ff3080fef70 2024-03-04T13:43:40.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.489+0000 7ff307e00700 1 -- 172.21.15.107:0/799457540 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff30804bed0 con 0x7ff3080fef70 2024-03-04T13:43:40.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.490+0000 7ff2fea00700 1 -- 172.21.15.107:0/799457540 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 29) v1 ==== 45873+0+0 (secure 0 0 0) 0x7ff2f801c570 con 0x7ff3080fef70 2024-03-04T13:43:40.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.490+0000 7ff2fea00700 1 --2- 172.21.15.107:0/799457540 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7ff2ec038a20 0x7ff2ec03aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:43:40.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.490+0000 7ff2fea00700 1 -- 172.21.15.107:0/799457540 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7ff2f804d210 con 0x7ff3080fef70 2024-03-04T13:43:40.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.490+0000 7ff306000700 1 --2- 172.21.15.107:0/799457540 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7ff2ec038a20 0x7ff2ec03aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:43:40.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.491+0000 7ff306000700 1 --2- 172.21.15.107:0/799457540 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7ff2ec038a20 0x7ff2ec03aed0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4009b80 tx=0x7ff2f400b040 comp rx=0 tx=0).ready entity=mgr.14696 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:43:40.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.500+0000 7ff2fea00700 1 -- 172.21.15.107:0/799457540 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7ff2f8024900 con 0x7ff3080fef70 2024-03-04T13:43:40.527 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:40 smithi169 ceph-mon[36313]: [04/Mar/2024:13:43:38] ENGINE Serving on https://172.21.15.107:7150 2024-03-04T13:43:40.528 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:40 smithi169 ceph-mon[36313]: [04/Mar/2024:13:43:38] ENGINE Serving on http://172.21.15.107:8765 2024-03-04T13:43:40.528 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:40 smithi169 ceph-mon[36313]: [04/Mar/2024:13:43:38] ENGINE Bus STARTED 2024-03-04T13:43:40.528 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:40 smithi169 ceph-mon[36313]: mgrmap e28: smithi107.wpzgwl(active, since 1.01992s) 2024-03-04T13:43:40.528 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:40 smithi169 ceph-mon[36313]: pgmap v3: 65 pgs: 65 active+clean; 1.7 GiB data, 7.1 GiB used, 529 GiB / 536 GiB avail 2024-03-04T13:43:40.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.775+0000 7ff307e00700 1 -- 172.21.15.107:0/799457540 --> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7ff308104ad0 con 0x7ff2ec038a20 2024-03-04T13:43:40.791 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.793+0000 7ff2fea00700 1 -- 172.21.15.107:0/799457540 <== mgr.14696 v2:172.21.15.107:6800/1613321046 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7ff308104ad0 con 0x7ff2ec038a20 2024-03-04T13:43:40.791 INFO:teuthology.orchestra.run.smithi107.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-04T13:43:40.791 INFO:teuthology.orchestra.run.smithi107.stdout:alertmanager.smithi107 smithi107 *:9093,9094 running (101s) 10s ago 12m 15.0M - 0.25.0 c8568f914cd2 ac0e4f11c97a 2024-03-04T13:43:40.791 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi107 smithi107 running (12m) 10s ago 12m 6665k - 17.2.7-545-g42608bb1 9eec3f22859d 03e1570c2eff 2024-03-04T13:43:40.791 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi169 smithi169 running (10m) 22s ago 10m 6656k - 17.2.7-545-g42608bb1 9eec3f22859d 9c8881a5338e 2024-03-04T13:43:40.791 INFO:teuthology.orchestra.run.smithi107.stdout:grafana.smithi107 smithi107 *:3000 running (74s) 10s ago 11m 73.9M - 9.4.7 954c08fa6188 bd720c203473 2024-03-04T13:43:40.792 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.riexmm smithi107 running (5m) 10s ago 5m 12.2M - 17.2.7-545-g42608bb1 9eec3f22859d fc921f7d4328 2024-03-04T13:43:40.792 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.thycho smithi107 running (4m) 10s ago 4m 265M - 17.2.7-545-g42608bb1 9eec3f22859d 4069c6e0e063 2024-03-04T13:43:40.792 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi169.bxitzk smithi169 running (4m) 22s ago 4m 12.3M - 17.2.7-545-g42608bb1 9eec3f22859d 7d4115228c7c 2024-03-04T13:43:40.792 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi169.jusudl smithi169 running (5m) 22s ago 5m 238M - 17.2.7-545-g42608bb1 9eec3f22859d 2d2f2cc452e8 2024-03-04T13:43:40.792 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi107.wpzgwl smithi107 *:8443,9283,8765 running (30s) 10s ago 13m 75.1M - 19.0.0-1845-g717ce3cc d71b839abdf8 1c5fdee45eac 2024-03-04T13:43:40.792 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi169.eihczr smithi169 *:8443,9283 running (2m) 22s ago 10m 499M - 19.0.0-1845-g717ce3cc d71b839abdf8 18b4dfa896b5 2024-03-04T13:43:40.792 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi107 smithi107 running (13m) 10s ago 13m 48.2M 2048M 17.2.7-545-g42608bb1 9eec3f22859d 49f5da990d35 2024-03-04T13:43:40.792 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi169 smithi169 running (10m) 22s ago 10m 39.0M 2048M 17.2.7-545-g42608bb1 9eec3f22859d ba4ee916297f 2024-03-04T13:43:40.792 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi107 smithi107 *:9100 running (109s) 10s ago 12m 7365k - 1.5.0 0da6a335fe13 b31d3c803078 2024-03-04T13:43:40.792 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi169 smithi169 *:9100 running (52s) 22s ago 10m 7390k - 1.5.0 0da6a335fe13 043154b90aa2 2024-03-04T13:43:40.792 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0 smithi107 running (9m) 10s ago 9m 180M 1558M 17.2.7-545-g42608bb1 9eec3f22859d c0605b68f696 2024-03-04T13:43:40.792 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1 smithi107 running (9m) 10s ago 9m 197M 1558M 17.2.7-545-g42608bb1 9eec3f22859d 5fb2c0dd5769 2024-03-04T13:43:40.793 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2 smithi107 running (8m) 10s ago 8m 165M 1558M 17.2.7-545-g42608bb1 9eec3f22859d 61f30f643549 2024-03-04T13:43:40.793 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3 smithi169 running (7m) 22s ago 7m 186M 2582M 17.2.7-545-g42608bb1 9eec3f22859d cd91d8694764 2024-03-04T13:43:40.793 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4 smithi169 running (7m) 22s ago 7m 174M 2582M 17.2.7-545-g42608bb1 9eec3f22859d 08545e3d0659 2024-03-04T13:43:40.793 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5 smithi169 running (6m) 22s ago 6m 190M 2582M 17.2.7-545-g42608bb1 9eec3f22859d dcffe6cb3eac 2024-03-04T13:43:40.793 INFO:teuthology.orchestra.run.smithi107.stdout:prometheus.smithi107 smithi107 *:9095 running (60s) 10s ago 11m 43.2M - 2.43.0 a07b618ecd1d 04a77b80edfd 2024-03-04T13:43:40.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.795+0000 7ff307e00700 1 -- 172.21.15.107:0/799457540 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7ff2ec038a20 msgr2=0x7ff2ec03aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:43:40.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.795+0000 7ff307e00700 1 --2- 172.21.15.107:0/799457540 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7ff2ec038a20 0x7ff2ec03aed0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4009b80 tx=0x7ff2f400b040 comp rx=0 tx=0).stop 2024-03-04T13:43:40.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.795+0000 7ff307e00700 1 -- 172.21.15.107:0/799457540 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff3080fef70 msgr2=0x7ff30810b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:43:40.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.795+0000 7ff307e00700 1 --2- 172.21.15.107:0/799457540 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff3080fef70 0x7ff30810b450 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff2f8000c00 tx=0x7ff2f8009e30 comp rx=0 tx=0).stop 2024-03-04T13:43:40.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.796+0000 7ff307e00700 1 -- 172.21.15.107:0/799457540 shutdown_connections 2024-03-04T13:43:40.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.796+0000 7ff307e00700 1 --2- 172.21.15.107:0/799457540 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff3080fef70 0x7ff30810b450 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:43:40.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.796+0000 7ff307e00700 1 --2- 172.21.15.107:0/799457540 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7ff2ec038a20 0x7ff2ec03aed0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:43:40.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.796+0000 7ff307e00700 1 --2- 172.21.15.107:0/799457540 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff308100180 0x7ff30810b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:43:40.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.796+0000 7ff307e00700 1 -- 172.21.15.107:0/799457540 >> 172.21.15.107:0/799457540 conn(0x7ff3080fa760 msgr2=0x7ff3081033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:43:40.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.796+0000 7ff307e00700 1 -- 172.21.15.107:0/799457540 shutdown_connections 2024-03-04T13:43:40.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.796+0000 7ff307e00700 1 -- 172.21.15.107:0/799457540 wait complete. 2024-03-04T13:43:40.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.979+0000 7f012bc00700 1 -- 172.21.15.107:0/965539330 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01240fcef0 msgr2=0x7f01240fd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:43:40.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.979+0000 7f012bc00700 1 --2- 172.21.15.107:0/965539330 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01240fcef0 0x7f01240fd320 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f0118004940 tx=0x7f0118009e30 comp rx=0 tx=0).stop 2024-03-04T13:43:40.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.979+0000 7f012bc00700 1 -- 172.21.15.107:0/965539330 shutdown_connections 2024-03-04T13:43:40.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.979+0000 7f012bc00700 1 --2- 172.21.15.107:0/965539330 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f01240fe0e0 0x7f01240fe550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:43:40.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.979+0000 7f012bc00700 1 --2- 172.21.15.107:0/965539330 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01240fcef0 0x7f01240fd320 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:43:40.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.979+0000 7f012bc00700 1 -- 172.21.15.107:0/965539330 >> 172.21.15.107:0/965539330 conn(0x7f01240f8620 msgr2=0x7f01240faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:43:40.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.979+0000 7f012bc00700 1 -- 172.21.15.107:0/965539330 shutdown_connections 2024-03-04T13:43:40.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.979+0000 7f012bc00700 1 -- 172.21.15.107:0/965539330 wait complete. 2024-03-04T13:43:40.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.980+0000 7f012bc00700 1 Processor -- start 2024-03-04T13:43:40.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.981+0000 7f012bc00700 1 -- start start 2024-03-04T13:43:40.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.981+0000 7f012bc00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f01240fcef0 0x7f012410b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:43:40.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.981+0000 7f012bc00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01240fe0e0 0x7f012410bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:43:40.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.981+0000 7f012bc00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f012410c2b0 con 0x7f01240fe0e0 2024-03-04T13:43:40.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.981+0000 7f012bc00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f012410cb50 con 0x7f01240fcef0 2024-03-04T13:43:40.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.982+0000 7f0129200700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f01240fcef0 0x7f012410b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:43:40.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.982+0000 7f0129200700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f01240fcef0 0x7f012410b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.107:55630/0 (socket says 172.21.15.107:55630) 2024-03-04T13:43:40.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.982+0000 7f0129200700 1 -- 172.21.15.107:0/579413735 learned_addr learned my addr 172.21.15.107:0/579413735 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:43:40.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.983+0000 7f0123e00700 1 --2- 172.21.15.107:0/579413735 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01240fe0e0 0x7f012410bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:43:40.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.983+0000 7f0129200700 1 -- 172.21.15.107:0/579413735 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01240fe0e0 msgr2=0x7f012410bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:43:40.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.983+0000 7f0129200700 1 --2- 172.21.15.107:0/579413735 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01240fe0e0 0x7f012410bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:43:40.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.983+0000 7f0129200700 1 -- 172.21.15.107:0/579413735 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0118005040 con 0x7f01240fcef0 2024-03-04T13:43:40.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.983+0000 7f0123e00700 1 --2- 172.21.15.107:0/579413735 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01240fe0e0 0x7f012410bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T13:43:40.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.984+0000 7f0129200700 1 --2- 172.21.15.107:0/579413735 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f01240fcef0 0x7f012410b720 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0118000c00 tx=0x7f0118009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:43:40.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.984+0000 7f0120c00700 1 -- 172.21.15.107:0/579413735 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f011800e040 con 0x7f01240fcef0 2024-03-04T13:43:40.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.984+0000 7f0120c00700 1 -- 172.21.15.107:0/579413735 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f011800bcc0 con 0x7f01240fcef0 2024-03-04T13:43:40.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.984+0000 7f012bc00700 1 -- 172.21.15.107:0/579413735 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f012410cdd0 con 0x7f01240fcef0 2024-03-04T13:43:40.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.984+0000 7f012bc00700 1 -- 172.21.15.107:0/579413735 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f012410d310 con 0x7f01240fcef0 2024-03-04T13:43:40.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.984+0000 7f0120c00700 1 -- 172.21.15.107:0/579413735 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0118013ea0 con 0x7f01240fcef0 2024-03-04T13:43:40.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.985+0000 7f012bc00700 1 -- 172.21.15.107:0/579413735 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f012404bed0 con 0x7f01240fcef0 2024-03-04T13:43:40.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.986+0000 7f0120c00700 1 -- 172.21.15.107:0/579413735 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 29) v1 ==== 45873+0+0 (secure 0 0 0) 0x7f011801c630 con 0x7f01240fcef0 2024-03-04T13:43:40.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.987+0000 7f0120c00700 1 --2- 172.21.15.107:0/579413735 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f010c038a20 0x7f010c03aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:43:40.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.987+0000 7f0120c00700 1 -- 172.21.15.107:0/579413735 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f011804daa0 con 0x7f01240fcef0 2024-03-04T13:43:40.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.987+0000 7f0123e00700 1 --2- 172.21.15.107:0/579413735 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f010c038a20 0x7f010c03aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:43:40.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.988+0000 7f0123e00700 1 --2- 172.21.15.107:0/579413735 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f010c038a20 0x7f010c03aed0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0110009fd0 tx=0x7f011000b040 comp rx=0 tx=0).ready entity=mgr.14696 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:43:40.989 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-04T13:43:40.989 DEBUG:teuthology.orchestra.run.smithi169:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1/tmp 2024-03-04T13:43:40.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:40.995+0000 7f0120c00700 1 -- 172.21.15.107:0/579413735 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f0118025400 con 0x7f01240fcef0 2024-03-04T13:43:41.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:41.341+0000 7f012bc00700 1 -- 172.21.15.107:0/579413735 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f012410d640 con 0x7f01240fcef0 2024-03-04T13:43:41.340 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:41 smithi107 ceph-mon[28739]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:43:41.340 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:41 smithi107 ceph-mon[28739]: mgrmap e29: smithi107.wpzgwl(active, since 2s) 2024-03-04T13:43:41.340 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:41 smithi107 ceph-mon[28739]: pgmap v4: 65 pgs: 65 active+clean; 1.7 GiB data, 7.1 GiB used, 529 GiB / 536 GiB avail 2024-03-04T13:43:41.340 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:41 smithi107 ceph-mon[28739]: from='client.14728 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:43:41.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:41.342+0000 7f0120c00700 1 -- 172.21.15.107:0/579413735 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7f011800bea0 con 0x7f01240fcef0 2024-03-04T13:43:41.341 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-03-04T13:43:41.341 INFO:teuthology.orchestra.run.smithi107.stdout: "mon": { 2024-03-04T13:43:41.341 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-545-g42608bb1 (42608bb191117be10dba720e6bf8a02f3171cd34) quincy (stable)": 2 2024-03-04T13:43:41.341 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:43:41.341 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr": { 2024-03-04T13:43:41.341 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-1845-g717ce3cc (717ce3cce8d0166e4e577faa8004e6f9cb4128c0) squid (dev)": 1 2024-03-04T13:43:41.341 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:43:41.341 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": { 2024-03-04T13:43:41.341 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-545-g42608bb1 (42608bb191117be10dba720e6bf8a02f3171cd34) quincy (stable)": 6 2024-03-04T13:43:41.341 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:43:41.341 INFO:teuthology.orchestra.run.smithi107.stdout: "mds": { 2024-03-04T13:43:41.342 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-545-g42608bb1 (42608bb191117be10dba720e6bf8a02f3171cd34) quincy (stable)": 4 2024-03-04T13:43:41.342 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:43:41.342 INFO:teuthology.orchestra.run.smithi107.stdout: "overall": { 2024-03-04T13:43:41.342 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-545-g42608bb1 (42608bb191117be10dba720e6bf8a02f3171cd34) quincy (stable)": 12, 2024-03-04T13:43:41.342 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-1845-g717ce3cc (717ce3cce8d0166e4e577faa8004e6f9cb4128c0) squid (dev)": 1 2024-03-04T13:43:41.342 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-04T13:43:41.342 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-03-04T13:43:41.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:41.345+0000 7f012bc00700 1 -- 172.21.15.107:0/579413735 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f010c038a20 msgr2=0x7f010c03aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:43:41.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:41.345+0000 7f012bc00700 1 --2- 172.21.15.107:0/579413735 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f010c038a20 0x7f010c03aed0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0110009fd0 tx=0x7f011000b040 comp rx=0 tx=0).stop 2024-03-04T13:43:41.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:41.345+0000 7f012bc00700 1 -- 172.21.15.107:0/579413735 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f01240fcef0 msgr2=0x7f012410b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:43:41.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:41.345+0000 7f012bc00700 1 --2- 172.21.15.107:0/579413735 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f01240fcef0 0x7f012410b720 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0118000c00 tx=0x7f0118009e30 comp rx=0 tx=0).stop 2024-03-04T13:43:41.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:41.345+0000 7f012bc00700 1 -- 172.21.15.107:0/579413735 shutdown_connections 2024-03-04T13:43:41.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:41.345+0000 7f012bc00700 1 --2- 172.21.15.107:0/579413735 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f01240fcef0 0x7f012410b720 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:43:41.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:41.345+0000 7f012bc00700 1 --2- 172.21.15.107:0/579413735 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f010c038a20 0x7f010c03aed0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:43:41.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:41.345+0000 7f012bc00700 1 --2- 172.21.15.107:0/579413735 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01240fe0e0 0x7f012410bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:43:41.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:41.345+0000 7f012bc00700 1 -- 172.21.15.107:0/579413735 >> 172.21.15.107:0/579413735 conn(0x7f01240f8620 msgr2=0x7f01240ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:43:41.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:41.346+0000 7f012bc00700 1 -- 172.21.15.107:0/579413735 shutdown_connections 2024-03-04T13:43:41.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:41.346+0000 7f012bc00700 1 -- 172.21.15.107:0/579413735 wait complete. 2024-03-04T13:43:41.357 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:41 smithi169 ceph-mon[36313]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:43:41.357 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:41 smithi169 ceph-mon[36313]: mgrmap e29: smithi107.wpzgwl(active, since 2s) 2024-03-04T13:43:41.358 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:41 smithi169 ceph-mon[36313]: pgmap v4: 65 pgs: 65 active+clean; 1.7 GiB data, 7.1 GiB used, 529 GiB / 536 GiB avail 2024-03-04T13:43:41.358 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:41 smithi169 ceph-mon[36313]: from='client.14728 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:43:41.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:41.519+0000 7efc29c00700 1 -- 172.21.15.107:0/1338240849 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc240fcef0 msgr2=0x7efc240fd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:43:41.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:41.519+0000 7efc29c00700 1 --2- 172.21.15.107:0/1338240849 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc240fcef0 0x7efc240fd320 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7efc14004940 tx=0x7efc14009e30 comp rx=0 tx=0).stop 2024-03-04T13:43:41.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:41.519+0000 7efc29c00700 1 -- 172.21.15.107:0/1338240849 shutdown_connections 2024-03-04T13:43:41.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:41.519+0000 7efc29c00700 1 --2- 172.21.15.107:0/1338240849 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efc240fe0e0 0x7efc240fe550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:43:41.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:41.519+0000 7efc29c00700 1 --2- 172.21.15.107:0/1338240849 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc240fcef0 0x7efc240fd320 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:43:41.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:41.519+0000 7efc29c00700 1 -- 172.21.15.107:0/1338240849 >> 172.21.15.107:0/1338240849 conn(0x7efc240f8620 msgr2=0x7efc240faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:43:41.518 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:41.520+0000 7efc29c00700 1 -- 172.21.15.107:0/1338240849 shutdown_connections 2024-03-04T13:43:41.518 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:41.520+0000 7efc29c00700 1 -- 172.21.15.107:0/1338240849 wait complete. 2024-03-04T13:43:41.518 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:41.521+0000 7efc29c00700 1 Processor -- start 2024-03-04T13:43:41.519 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:41.521+0000 7efc29c00700 1 -- start start 2024-03-04T13:43:41.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:41.522+0000 7efc29c00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efc240fcef0 0x7efc2410fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:43:41.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:41.522+0000 7efc29c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc240fe0e0 0x7efc241100c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:43:41.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:41.522+0000 7efc29c00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc241106c0 con 0x7efc240fe0e0 2024-03-04T13:43:41.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:41.522+0000 7efc29c00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc24110f60 con 0x7efc240fcef0 2024-03-04T13:43:41.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:41.522+0000 7efc22000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc240fe0e0 0x7efc241100c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:43:41.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:41.522+0000 7efc22000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc240fe0e0 0x7efc241100c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59086/0 (socket says 172.21.15.107:59086) 2024-03-04T13:43:41.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:41.522+0000 7efc22000700 1 -- 172.21.15.107:0/1227375527 learned_addr learned my addr 172.21.15.107:0/1227375527 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:43:41.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:41.523+0000 7efc22a00700 1 --2- 172.21.15.107:0/1227375527 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efc240fcef0 0x7efc2410fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:43:41.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:41.523+0000 7efc22000700 1 -- 172.21.15.107:0/1227375527 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efc240fcef0 msgr2=0x7efc2410fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:43:41.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:41.523+0000 7efc22000700 1 --2- 172.21.15.107:0/1227375527 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efc240fcef0 0x7efc2410fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:43:41.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:41.523+0000 7efc22000700 1 -- 172.21.15.107:0/1227375527 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc14005040 con 0x7efc240fe0e0 2024-03-04T13:43:41.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:41.523+0000 7efc22000700 1 --2- 172.21.15.107:0/1227375527 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc240fe0e0 0x7efc241100c0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7efc1000c8c0 tx=0x7efc100044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:43:41.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:41.524+0000 7efc1aa00700 1 -- 172.21.15.107:0/1227375527 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efc1000a760 con 0x7efc240fe0e0 2024-03-04T13:43:41.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:41.524+0000 7efc1aa00700 1 -- 172.21.15.107:0/1227375527 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efc1000ad90 con 0x7efc240fe0e0 2024-03-04T13:43:41.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:41.524+0000 7efc1aa00700 1 -- 172.21.15.107:0/1227375527 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efc100113a0 con 0x7efc240fe0e0 2024-03-04T13:43:41.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:41.524+0000 7efc29c00700 1 -- 172.21.15.107:0/1227375527 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc24111240 con 0x7efc240fe0e0 2024-03-04T13:43:41.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:41.524+0000 7efc29c00700 1 -- 172.21.15.107:0/1227375527 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc24111780 con 0x7efc240fe0e0 2024-03-04T13:43:41.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:41.526+0000 7efc29c00700 1 -- 172.21.15.107:0/1227375527 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc24102210 con 0x7efc240fe0e0 2024-03-04T13:43:41.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:41.526+0000 7efc1aa00700 1 -- 172.21.15.107:0/1227375527 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 29) v1 ==== 45873+0+0 (secure 0 0 0) 0x7efc1000a8c0 con 0x7efc240fe0e0 2024-03-04T13:43:41.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:41.526+0000 7efc1aa00700 1 --2- 172.21.15.107:0/1227375527 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7efc080389d0 0x7efc0803ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:43:41.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:41.526+0000 7efc1aa00700 1 -- 172.21.15.107:0/1227375527 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7efc1004c440 con 0x7efc240fe0e0 2024-03-04T13:43:41.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:41.527+0000 7efc22a00700 1 --2- 172.21.15.107:0/1227375527 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7efc080389d0 0x7efc0803ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:43:41.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:41.527+0000 7efc22a00700 1 --2- 172.21.15.107:0/1227375527 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7efc080389d0 0x7efc0803ae80 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7efc1400b840 tx=0x7efc14009e30 comp rx=0 tx=0).ready entity=mgr.14696 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:43:41.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:41.533+0000 7efc1aa00700 1 -- 172.21.15.107:0/1227375527 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7efc10012400 con 0x7efc240fe0e0 2024-03-04T13:43:41.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:41.809+0000 7efc29c00700 1 -- 172.21.15.107:0/1227375527 --> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7efc24111ac0 con 0x7efc080389d0 2024-03-04T13:43:41.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:41.811+0000 7efc1aa00700 1 -- 172.21.15.107:0/1227375527 <== mgr.14696 v2:172.21.15.107:6800/1613321046 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7efc24111ac0 con 0x7efc080389d0 2024-03-04T13:43:41.809 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-03-04T13:43:41.809 INFO:teuthology.orchestra.run.smithi107.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377", 2024-03-04T13:43:41.809 INFO:teuthology.orchestra.run.smithi107.stdout: "in_progress": true, 2024-03-04T13:43:41.809 INFO:teuthology.orchestra.run.smithi107.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-03-04T13:43:41.809 INFO:teuthology.orchestra.run.smithi107.stdout: "services_complete": [ 2024-03-04T13:43:41.809 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr" 2024-03-04T13:43:41.809 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-04T13:43:41.810 INFO:teuthology.orchestra.run.smithi107.stdout: "progress": "2/2 daemons upgraded", 2024-03-04T13:43:41.810 INFO:teuthology.orchestra.run.smithi107.stdout: "message": "", 2024-03-04T13:43:41.810 INFO:teuthology.orchestra.run.smithi107.stdout: "is_paused": false 2024-03-04T13:43:41.810 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-03-04T13:43:41.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:41.814+0000 7efc29c00700 1 -- 172.21.15.107:0/1227375527 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7efc080389d0 msgr2=0x7efc0803ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:43:41.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:41.814+0000 7efc29c00700 1 --2- 172.21.15.107:0/1227375527 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7efc080389d0 0x7efc0803ae80 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7efc1400b840 tx=0x7efc14009e30 comp rx=0 tx=0).stop 2024-03-04T13:43:41.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:41.814+0000 7efc29c00700 1 -- 172.21.15.107:0/1227375527 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc240fe0e0 msgr2=0x7efc241100c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:43:41.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:41.814+0000 7efc29c00700 1 --2- 172.21.15.107:0/1227375527 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc240fe0e0 0x7efc241100c0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7efc1000c8c0 tx=0x7efc100044c0 comp rx=0 tx=0).stop 2024-03-04T13:43:41.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:41.814+0000 7efc29c00700 1 -- 172.21.15.107:0/1227375527 shutdown_connections 2024-03-04T13:43:41.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:41.814+0000 7efc29c00700 1 --2- 172.21.15.107:0/1227375527 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efc240fcef0 0x7efc2410fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:43:41.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:41.814+0000 7efc29c00700 1 --2- 172.21.15.107:0/1227375527 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7efc080389d0 0x7efc0803ae80 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:43:41.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:41.814+0000 7efc29c00700 1 --2- 172.21.15.107:0/1227375527 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc240fe0e0 0x7efc241100c0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:43:41.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:41.814+0000 7efc29c00700 1 -- 172.21.15.107:0/1227375527 >> 172.21.15.107:0/1227375527 conn(0x7efc240f8620 msgr2=0x7efc240ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:43:41.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:41.814+0000 7efc29c00700 1 -- 172.21.15.107:0/1227375527 shutdown_connections 2024-03-04T13:43:41.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:43:41.814+0000 7efc29c00700 1 -- 172.21.15.107:0/1227375527 wait complete. 2024-03-04T13:43:42.577 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:42 smithi169 ceph-mon[36313]: from='client.24467 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:43:42.577 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:42 smithi169 ceph-mon[36313]: from='client.? 172.21.15.107:0/579413735' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:43:42.590 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:42 smithi107 ceph-mon[28739]: from='client.24467 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:43:42.590 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:42 smithi107 ceph-mon[28739]: from='client.? 172.21.15.107:0/579413735' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:43:43.407 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:43 smithi107 ceph-mon[28739]: from='client.14740 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:43:43.408 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:43 smithi107 ceph-mon[28739]: pgmap v5: 65 pgs: 65 active+clean; 1.7 GiB data, 7.1 GiB used, 529 GiB / 536 GiB avail 2024-03-04T13:43:43.408 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:43 smithi107 ceph-mon[28739]: mgrmap e30: smithi107.wpzgwl(active, since 4s) 2024-03-04T13:43:43.607 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:43 smithi169 ceph-mon[36313]: from='client.14740 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:43:43.607 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:43 smithi169 ceph-mon[36313]: pgmap v5: 65 pgs: 65 active+clean; 1.7 GiB data, 7.1 GiB used, 529 GiB / 536 GiB avail 2024-03-04T13:43:43.607 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:43 smithi169 ceph-mon[36313]: mgrmap e30: smithi107.wpzgwl(active, since 4s) 2024-03-04T13:43:45.349 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:45 smithi169 ceph-mon[36313]: pgmap v6: 65 pgs: 65 active+clean; 1.7 GiB data, 7.1 GiB used, 529 GiB / 536 GiB avail 2024-03-04T13:43:45.539 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:45 smithi107 ceph-mon[28739]: pgmap v6: 65 pgs: 65 active+clean; 1.7 GiB data, 7.1 GiB used, 529 GiB / 536 GiB avail 2024-03-04T13:43:47.766 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:47 smithi107 ceph-mon[28739]: pgmap v7: 65 pgs: 65 active+clean; 1.2 GiB data, 5.4 GiB used, 531 GiB / 536 GiB avail; 628 KiB/s rd, 589 KiB/s wr, 95 op/s 2024-03-04T13:43:47.766 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:47 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:43:47.766 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:47 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:43:47.857 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:47 smithi169 ceph-mon[36313]: pgmap v7: 65 pgs: 65 active+clean; 1.2 GiB data, 5.4 GiB used, 531 GiB / 536 GiB avail; 628 KiB/s rd, 589 KiB/s wr, 95 op/s 2024-03-04T13:43:47.858 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:47 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:43:47.858 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:47 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:43:49.585 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:49 smithi107 ceph-mon[28739]: pgmap v8: 65 pgs: 65 active+clean; 1.2 GiB data, 5.4 GiB used, 531 GiB / 536 GiB avail; 488 KiB/s rd, 458 KiB/s wr, 74 op/s 2024-03-04T13:43:49.585 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:49 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:43:49.585 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:49 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:43:49.586 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:49 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:43:49.586 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:49 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:43:49.591 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:49 smithi169 ceph-mon[36313]: pgmap v8: 65 pgs: 65 active+clean; 1.2 GiB data, 5.4 GiB used, 531 GiB / 536 GiB avail; 488 KiB/s rd, 458 KiB/s wr, 74 op/s 2024-03-04T13:43:49.591 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:49 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:43:49.591 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:49 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:43:49.592 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:49 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:43:49.592 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:49 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:43:51.306 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:51 smithi107 ceph-mon[28739]: pgmap v9: 65 pgs: 65 active+clean; 966 MiB data, 4.6 GiB used, 532 GiB / 536 GiB avail; 862 KiB/s rd, 922 KiB/s wr, 108 op/s 2024-03-04T13:43:51.306 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:51 smithi107 ceph-mon[28739]: Standby manager daemon smithi169.eihczr started 2024-03-04T13:43:51.306 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:51 smithi107 ceph-mon[28739]: from='mgr.? 172.21.15.169:0/683294296' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi169.eihczr/crt"}]: dispatch 2024-03-04T13:43:51.306 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:51 smithi107 ceph-mon[28739]: from='mgr.? 172.21.15.169:0/683294296' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-04T13:43:51.306 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:51 smithi107 ceph-mon[28739]: from='mgr.? 172.21.15.169:0/683294296' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi169.eihczr/key"}]: dispatch 2024-03-04T13:43:51.306 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:51 smithi107 ceph-mon[28739]: from='mgr.? 172.21.15.169:0/683294296' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-04T13:43:51.306 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:51 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:43:51.306 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:51 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:43:51.607 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:51 smithi169 ceph-mon[36313]: pgmap v9: 65 pgs: 65 active+clean; 966 MiB data, 4.6 GiB used, 532 GiB / 536 GiB avail; 862 KiB/s rd, 922 KiB/s wr, 108 op/s 2024-03-04T13:43:51.608 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:51 smithi169 ceph-mon[36313]: Standby manager daemon smithi169.eihczr started 2024-03-04T13:43:51.608 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:51 smithi169 ceph-mon[36313]: from='mgr.? 172.21.15.169:0/683294296' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi169.eihczr/crt"}]: dispatch 2024-03-04T13:43:51.608 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:51 smithi169 ceph-mon[36313]: from='mgr.? 172.21.15.169:0/683294296' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-04T13:43:51.608 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:51 smithi169 ceph-mon[36313]: from='mgr.? 172.21.15.169:0/683294296' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi169.eihczr/key"}]: dispatch 2024-03-04T13:43:51.608 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:51 smithi169 ceph-mon[36313]: from='mgr.? 172.21.15.169:0/683294296' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-04T13:43:51.608 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:51 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:43:51.608 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:51 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:43:52.590 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:52 smithi107 ceph-mon[28739]: mgrmap e31: smithi107.wpzgwl(active, since 13s), standbys: smithi169.eihczr 2024-03-04T13:43:52.591 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:52 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mgr metadata", "who": "smithi169.eihczr", "id": "smithi169.eihczr"}]: dispatch 2024-03-04T13:43:52.607 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:52 smithi169 ceph-mon[36313]: mgrmap e31: smithi107.wpzgwl(active, since 13s), standbys: smithi169.eihczr 2024-03-04T13:43:52.607 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:52 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mgr metadata", "who": "smithi169.eihczr", "id": "smithi169.eihczr"}]: dispatch 2024-03-04T13:43:53.340 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:53 smithi107 ceph-mon[28739]: pgmap v10: 65 pgs: 65 active+clean; 966 MiB data, 4.6 GiB used, 532 GiB / 536 GiB avail; 789 KiB/s rd, 844 KiB/s wr, 99 op/s 2024-03-04T13:43:53.608 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:53 smithi169 ceph-mon[36313]: pgmap v10: 65 pgs: 65 active+clean; 966 MiB data, 4.6 GiB used, 532 GiB / 536 GiB avail; 789 KiB/s rd, 844 KiB/s wr, 99 op/s 2024-03-04T13:43:54.341 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:54 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:43:54.607 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:54 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:43:55.472 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-04T13:43:55.472 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-04T13:43:55.840 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:55 smithi107 ceph-mon[28739]: pgmap v11: 65 pgs: 65 active+clean; 966 MiB data, 4.6 GiB used, 532 GiB / 536 GiB avail; 789 KiB/s rd, 844 KiB/s wr, 99 op/s 2024-03-04T13:43:55.840 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:55 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:43:55.841 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:55 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:43:55.841 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:55 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:43:55.841 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:55 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:43:55.841 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:55 smithi107 ceph-mon[28739]: Updating smithi107:/etc/ceph/ceph.conf 2024-03-04T13:43:55.841 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:55 smithi107 ceph-mon[28739]: Updating smithi169:/etc/ceph/ceph.conf 2024-03-04T13:43:55.857 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:55 smithi169 ceph-mon[36313]: pgmap v11: 65 pgs: 65 active+clean; 966 MiB data, 4.6 GiB used, 532 GiB / 536 GiB avail; 789 KiB/s rd, 844 KiB/s wr, 99 op/s 2024-03-04T13:43:55.858 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:55 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:43:55.858 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:55 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:43:55.858 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:55 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:43:55.858 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:55 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:43:55.858 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:55 smithi169 ceph-mon[36313]: Updating smithi107:/etc/ceph/ceph.conf 2024-03-04T13:43:55.858 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:55 smithi169 ceph-mon[36313]: Updating smithi169:/etc/ceph/ceph.conf 2024-03-04T13:43:56.840 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:56 smithi107 ceph-mon[28739]: Updating smithi169:/var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-04T13:43:56.840 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:56 smithi107 ceph-mon[28739]: Updating smithi107:/var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-04T13:43:56.857 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:56 smithi169 ceph-mon[36313]: Updating smithi169:/var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-04T13:43:56.857 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:56 smithi169 ceph-mon[36313]: Updating smithi107:/var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-04T13:43:57.841 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:57 smithi107 ceph-mon[28739]: Updating smithi169:/etc/ceph/ceph.client.admin.keyring 2024-03-04T13:43:57.841 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:57 smithi107 ceph-mon[28739]: Updating smithi107:/etc/ceph/ceph.client.admin.keyring 2024-03-04T13:43:57.841 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:57 smithi107 ceph-mon[28739]: pgmap v12: 65 pgs: 65 active+clean; 626 MiB data, 3.6 GiB used, 533 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 141 op/s 2024-03-04T13:43:57.841 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:57 smithi107 ceph-mon[28739]: Updating smithi169:/var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/config/ceph.client.admin.keyring 2024-03-04T13:43:57.841 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:57 smithi107 ceph-mon[28739]: Updating smithi107:/var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/config/ceph.client.admin.keyring 2024-03-04T13:43:57.842 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:57 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:43:57.842 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:57 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:43:57.842 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:57 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:43:57.842 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:57 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:43:57.842 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:57 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:43:57.857 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:57 smithi169 ceph-mon[36313]: Updating smithi169:/etc/ceph/ceph.client.admin.keyring 2024-03-04T13:43:57.858 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:57 smithi169 ceph-mon[36313]: Updating smithi107:/etc/ceph/ceph.client.admin.keyring 2024-03-04T13:43:57.858 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:57 smithi169 ceph-mon[36313]: pgmap v12: 65 pgs: 65 active+clean; 626 MiB data, 3.6 GiB used, 533 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 141 op/s 2024-03-04T13:43:57.858 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:57 smithi169 ceph-mon[36313]: Updating smithi169:/var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/config/ceph.client.admin.keyring 2024-03-04T13:43:57.858 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:57 smithi169 ceph-mon[36313]: Updating smithi107:/var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/config/ceph.client.admin.keyring 2024-03-04T13:43:57.858 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:57 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:43:57.858 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:57 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:43:57.858 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:57 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:43:57.858 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:57 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:43:57.858 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:57 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:43:59.778 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:59 smithi107 ceph-mon[28739]: Reconfiguring prometheus.smithi107 (dependencies changed)... 2024-03-04T13:43:59.778 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:59 smithi107 ceph-mon[28739]: Reconfiguring daemon prometheus.smithi107 on smithi107 2024-03-04T13:43:59.779 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:43:59 smithi107 ceph-mon[28739]: pgmap v13: 65 pgs: 65 active+clean; 626 MiB data, 3.6 GiB used, 533 GiB / 536 GiB avail; 939 KiB/s rd, 981 KiB/s wr, 85 op/s 2024-03-04T13:43:59.858 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:59 smithi169 ceph-mon[36313]: Reconfiguring prometheus.smithi107 (dependencies changed)... 2024-03-04T13:43:59.858 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:59 smithi169 ceph-mon[36313]: Reconfiguring daemon prometheus.smithi107 on smithi107 2024-03-04T13:43:59.858 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:43:59 smithi169 ceph-mon[36313]: pgmap v13: 65 pgs: 65 active+clean; 626 MiB data, 3.6 GiB used, 533 GiB / 536 GiB avail; 939 KiB/s rd, 981 KiB/s wr, 85 op/s 2024-03-04T13:44:01.764 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:01 smithi107 ceph-mon[28739]: pgmap v14: 65 pgs: 65 active+clean; 288 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 128 op/s 2024-03-04T13:44:01.858 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:01 smithi169 ceph-mon[36313]: pgmap v14: 65 pgs: 65 active+clean; 288 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 128 op/s 2024-03-04T13:44:03.799 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:03 smithi107 ceph-mon[28739]: pgmap v15: 65 pgs: 65 active+clean; 288 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.0 MiB/s rd, 993 KiB/s wr, 84 op/s 2024-03-04T13:44:03.857 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:03 smithi169 ceph-mon[36313]: pgmap v15: 65 pgs: 65 active+clean; 288 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.0 MiB/s rd, 993 KiB/s wr, 84 op/s 2024-03-04T13:44:05.565 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:05 smithi107 ceph-mon[28739]: pgmap v16: 65 pgs: 65 active+clean; 288 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.0 MiB/s rd, 993 KiB/s wr, 86 op/s 2024-03-04T13:44:05.565 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:05 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:44:05.565 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:05 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:44:05.566 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:05 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-04T13:44:05.566 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:05 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:44:05.858 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:05 smithi169 ceph-mon[36313]: pgmap v16: 65 pgs: 65 active+clean; 288 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.0 MiB/s rd, 993 KiB/s wr, 86 op/s 2024-03-04T13:44:05.858 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:05 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:44:05.858 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:05 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:44:05.858 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:05 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-04T13:44:05.858 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:05 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:44:06.841 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:06 smithi107 ceph-mon[28739]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-04T13:44:06.854 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:06 smithi169 ceph-mon[36313]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-04T13:44:07.841 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:07 smithi107 ceph-mon[28739]: pgmap v17: 65 pgs: 65 active+clean; 292 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 124 op/s 2024-03-04T13:44:07.858 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:07 smithi169 ceph-mon[36313]: pgmap v17: 65 pgs: 65 active+clean; 292 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 124 op/s 2024-03-04T13:44:08.841 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:08 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi169.eihczr", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-04T13:44:08.841 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:08 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-04T13:44:08.841 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:08 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:44:08.841 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:08 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:44:08.858 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:08 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi169.eihczr", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-04T13:44:08.858 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:08 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-04T13:44:08.858 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:08 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:44:08.858 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:08 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:44:09.841 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:09 smithi107 ceph-mon[28739]: Upgrade: Updating mgr.smithi169.eihczr 2024-03-04T13:44:09.841 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:09 smithi107 ceph-mon[28739]: Deploying daemon mgr.smithi169.eihczr on smithi169 2024-03-04T13:44:09.841 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:09 smithi107 ceph-mon[28739]: pgmap v18: 65 pgs: 65 active+clean; 292 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1017 KiB/s rd, 1.0 MiB/s wr, 81 op/s 2024-03-04T13:44:09.857 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:09 smithi169 ceph-mon[36313]: Upgrade: Updating mgr.smithi169.eihczr 2024-03-04T13:44:09.858 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:09 smithi169 ceph-mon[36313]: Deploying daemon mgr.smithi169.eihczr on smithi169 2024-03-04T13:44:09.858 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:09 smithi169 ceph-mon[36313]: pgmap v18: 65 pgs: 65 active+clean; 292 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1017 KiB/s rd, 1.0 MiB/s wr, 81 op/s 2024-03-04T13:44:11.841 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:11 smithi107 ceph-mon[28739]: pgmap v19: 65 pgs: 65 active+clean; 298 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 111 op/s 2024-03-04T13:44:11.858 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:11 smithi169 ceph-mon[36313]: pgmap v19: 65 pgs: 65 active+clean; 298 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 111 op/s 2024-03-04T13:44:11.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:11.987+0000 7f0669200700 1 -- 172.21.15.107:0/2308681748 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0664100380 msgr2=0x7f06641007d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:44:11.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:11.987+0000 7f0669200700 1 --2- 172.21.15.107:0/2308681748 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0664100380 0x7f06641007d0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f0650004940 tx=0x7f0650009e30 comp rx=0 tx=0).stop 2024-03-04T13:44:11.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:11.988+0000 7f0669200700 1 -- 172.21.15.107:0/2308681748 shutdown_connections 2024-03-04T13:44:11.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:11.988+0000 7f0669200700 1 --2- 172.21.15.107:0/2308681748 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0664100380 0x7f06641007d0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:44:11.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:11.988+0000 7f0669200700 1 --2- 172.21.15.107:0/2308681748 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f06640ff170 0x7f06640ff580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:44:11.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:11.988+0000 7f0669200700 1 -- 172.21.15.107:0/2308681748 >> 172.21.15.107:0/2308681748 conn(0x7f06640fa920 msgr2=0x7f06640fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:44:11.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:11.988+0000 7f0669200700 1 -- 172.21.15.107:0/2308681748 shutdown_connections 2024-03-04T13:44:11.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:11.988+0000 7f0669200700 1 -- 172.21.15.107:0/2308681748 wait complete. 2024-03-04T13:44:11.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:11.988+0000 7f0669200700 1 Processor -- start 2024-03-04T13:44:11.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:11.989+0000 7f0669200700 1 -- start start 2024-03-04T13:44:11.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:11.989+0000 7f0669200700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f06640ff170 0x7f066410fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:44:11.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:11.989+0000 7f0669200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0664100380 0x7f06641100d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:44:11.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:11.989+0000 7f0669200700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06641106d0 con 0x7f0664100380 2024-03-04T13:44:11.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:11.989+0000 7f0669200700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0664110f70 con 0x7f06640ff170 2024-03-04T13:44:11.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:11.989+0000 7f0661400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0664100380 0x7f06641100d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:44:11.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:11.989+0000 7f0661400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0664100380 0x7f06641100d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55814/0 (socket says 172.21.15.107:55814) 2024-03-04T13:44:11.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:11.989+0000 7f0661400700 1 -- 172.21.15.107:0/3290206662 learned_addr learned my addr 172.21.15.107:0/3290206662 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:44:11.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:11.990+0000 7f0661e00700 1 --2- 172.21.15.107:0/3290206662 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f06640ff170 0x7f066410fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:44:11.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:11.990+0000 7f0661400700 1 -- 172.21.15.107:0/3290206662 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f06640ff170 msgr2=0x7f066410fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:44:11.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:11.990+0000 7f0661400700 1 --2- 172.21.15.107:0/3290206662 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f06640ff170 0x7f066410fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:44:11.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:11.990+0000 7f0661400700 1 -- 172.21.15.107:0/3290206662 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0650005040 con 0x7f0664100380 2024-03-04T13:44:11.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:11.990+0000 7f0661400700 1 --2- 172.21.15.107:0/3290206662 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0664100380 0x7f06641100d0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f0650009fd0 tx=0x7f0650009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:44:11.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:11.991+0000 7f065a000700 1 -- 172.21.15.107:0/3290206662 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f065000e040 con 0x7f0664100380 2024-03-04T13:44:11.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:11.991+0000 7f065a000700 1 -- 172.21.15.107:0/3290206662 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f065000bcc0 con 0x7f0664100380 2024-03-04T13:44:11.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:11.991+0000 7f0669200700 1 -- 172.21.15.107:0/3290206662 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06641111f0 con 0x7f0664100380 2024-03-04T13:44:11.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:11.991+0000 7f065a000700 1 -- 172.21.15.107:0/3290206662 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f065001c630 con 0x7f0664100380 2024-03-04T13:44:11.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:11.991+0000 7f0669200700 1 -- 172.21.15.107:0/3290206662 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0664111730 con 0x7f0664100380 2024-03-04T13:44:11.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:11.992+0000 7f0669200700 1 -- 172.21.15.107:0/3290206662 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f066404bed0 con 0x7f0664100380 2024-03-04T13:44:11.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:11.993+0000 7f065a000700 1 -- 172.21.15.107:0/3290206662 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 31) v1 ==== 91301+0+0 (secure 0 0 0) 0x7f0650013420 con 0x7f0664100380 2024-03-04T13:44:11.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:11.994+0000 7f065a000700 1 --2- 172.21.15.107:0/3290206662 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f064806d200 0x7f064806f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:44:11.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:11.994+0000 7f065a000700 1 -- 172.21.15.107:0/3290206662 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f065008dbc0 con 0x7f0664100380 2024-03-04T13:44:11.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:11.994+0000 7f0661e00700 1 --2- 172.21.15.107:0/3290206662 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f064806d200 0x7f064806f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:44:11.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:11.995+0000 7f0661e00700 1 --2- 172.21.15.107:0/3290206662 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f064806d200 0x7f064806f6b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f06641001e0 tx=0x7f065400b040 comp rx=0 tx=0).ready entity=mgr.14696 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:44:11.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:11.999+0000 7f065a000700 1 -- 172.21.15.107:0/3290206662 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f065005b5d0 con 0x7f0664100380 2024-03-04T13:44:12.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:12.251+0000 7f0669200700 1 -- 172.21.15.107:0/3290206662 --> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0664111c50 con 0x7f064806d200 2024-03-04T13:44:12.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:12.254+0000 7f065a000700 1 -- 172.21.15.107:0/3290206662 <== mgr.14696 v2:172.21.15.107:6800/1613321046 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+360 (secure 0 0 0) 0x7f0664111c50 con 0x7f064806d200 2024-03-04T13:44:12.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:12.257+0000 7f0669200700 1 -- 172.21.15.107:0/3290206662 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f064806d200 msgr2=0x7f064806f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:44:12.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:12.257+0000 7f0669200700 1 --2- 172.21.15.107:0/3290206662 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f064806d200 0x7f064806f6b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f06641001e0 tx=0x7f065400b040 comp rx=0 tx=0).stop 2024-03-04T13:44:12.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:12.257+0000 7f0669200700 1 -- 172.21.15.107:0/3290206662 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0664100380 msgr2=0x7f06641100d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:44:12.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:12.257+0000 7f0669200700 1 --2- 172.21.15.107:0/3290206662 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0664100380 0x7f06641100d0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f0650009fd0 tx=0x7f0650009e30 comp rx=0 tx=0).stop 2024-03-04T13:44:12.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:12.257+0000 7f0669200700 1 -- 172.21.15.107:0/3290206662 shutdown_connections 2024-03-04T13:44:12.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:12.257+0000 7f0669200700 1 --2- 172.21.15.107:0/3290206662 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f06640ff170 0x7f066410fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:44:12.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:12.257+0000 7f0669200700 1 --2- 172.21.15.107:0/3290206662 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f064806d200 0x7f064806f6b0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:44:12.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:12.257+0000 7f0669200700 1 --2- 172.21.15.107:0/3290206662 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0664100380 0x7f06641100d0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:44:12.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:12.257+0000 7f0669200700 1 -- 172.21.15.107:0/3290206662 >> 172.21.15.107:0/3290206662 conn(0x7f06640fa920 msgr2=0x7f06641035b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:44:12.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:12.257+0000 7f0669200700 1 -- 172.21.15.107:0/3290206662 shutdown_connections 2024-03-04T13:44:12.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:12.258+0000 7f0669200700 1 -- 172.21.15.107:0/3290206662 wait complete. 2024-03-04T13:44:12.269 INFO:teuthology.orchestra.run.smithi107.stdout:true 2024-03-04T13:44:12.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:12.415+0000 7ff65e600700 1 -- 172.21.15.107:0/3936086933 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6580fcef0 msgr2=0x7ff6580fd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:44:12.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:12.415+0000 7ff65e600700 1 --2- 172.21.15.107:0/3936086933 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6580fcef0 0x7ff6580fd320 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7ff648004940 tx=0x7ff648009e30 comp rx=0 tx=0).stop 2024-03-04T13:44:12.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:12.415+0000 7ff65e600700 1 -- 172.21.15.107:0/3936086933 shutdown_connections 2024-03-04T13:44:12.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:12.415+0000 7ff65e600700 1 --2- 172.21.15.107:0/3936086933 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff6580fe0e0 0x7ff6580fe550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:44:12.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:12.415+0000 7ff65e600700 1 --2- 172.21.15.107:0/3936086933 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6580fcef0 0x7ff6580fd320 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:44:12.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:12.415+0000 7ff65e600700 1 -- 172.21.15.107:0/3936086933 >> 172.21.15.107:0/3936086933 conn(0x7ff6580f8620 msgr2=0x7ff6580faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:44:12.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:12.415+0000 7ff65e600700 1 -- 172.21.15.107:0/3936086933 shutdown_connections 2024-03-04T13:44:12.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:12.415+0000 7ff65e600700 1 -- 172.21.15.107:0/3936086933 wait complete. 2024-03-04T13:44:12.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:12.416+0000 7ff65e600700 1 Processor -- start 2024-03-04T13:44:12.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:12.416+0000 7ff65e600700 1 -- start start 2024-03-04T13:44:12.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:12.416+0000 7ff65e600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6580fcef0 0x7ff65810fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:44:12.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:12.416+0000 7ff65e600700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff6580fe0e0 0x7ff658110180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:44:12.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:12.416+0000 7ff65e600700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff658110780 con 0x7ff6580fcef0 2024-03-04T13:44:12.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:12.416+0000 7ff65e600700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff658110fd0 con 0x7ff6580fe0e0 2024-03-04T13:44:12.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:12.417+0000 7ff657400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6580fcef0 0x7ff65810fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:44:12.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:12.417+0000 7ff657400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6580fcef0 0x7ff65810fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55832/0 (socket says 172.21.15.107:55832) 2024-03-04T13:44:12.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:12.417+0000 7ff657400700 1 -- 172.21.15.107:0/278875020 learned_addr learned my addr 172.21.15.107:0/278875020 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:44:12.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:12.417+0000 7ff64fe00700 1 --2- 172.21.15.107:0/278875020 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff6580fe0e0 0x7ff658110180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:44:12.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:12.417+0000 7ff657400700 1 -- 172.21.15.107:0/278875020 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff6580fe0e0 msgr2=0x7ff658110180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:44:12.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:12.417+0000 7ff657400700 1 --2- 172.21.15.107:0/278875020 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff6580fe0e0 0x7ff658110180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:44:12.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:12.417+0000 7ff657400700 1 -- 172.21.15.107:0/278875020 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff648005040 con 0x7ff6580fcef0 2024-03-04T13:44:12.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:12.417+0000 7ff657400700 1 --2- 172.21.15.107:0/278875020 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6580fcef0 0x7ff65810fc40 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7ff648009360 tx=0x7ff648009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:44:12.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:12.418+0000 7ff64f400700 1 -- 172.21.15.107:0/278875020 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff64800e040 con 0x7ff6580fcef0 2024-03-04T13:44:12.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:12.418+0000 7ff64f400700 1 -- 172.21.15.107:0/278875020 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff648004290 con 0x7ff6580fcef0 2024-03-04T13:44:12.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:12.418+0000 7ff64f400700 1 -- 172.21.15.107:0/278875020 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff64801c630 con 0x7ff6580fcef0 2024-03-04T13:44:12.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:12.418+0000 7ff65e600700 1 -- 172.21.15.107:0/278875020 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff658111250 con 0x7ff6580fcef0 2024-03-04T13:44:12.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:12.418+0000 7ff65e600700 1 -- 172.21.15.107:0/278875020 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff658111790 con 0x7ff6580fcef0 2024-03-04T13:44:12.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:12.419+0000 7ff65e600700 1 -- 172.21.15.107:0/278875020 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff658102210 con 0x7ff6580fcef0 2024-03-04T13:44:12.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:12.420+0000 7ff64f400700 1 -- 172.21.15.107:0/278875020 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 31) v1 ==== 91301+0+0 (secure 0 0 0) 0x7ff648013420 con 0x7ff6580fcef0 2024-03-04T13:44:12.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:12.421+0000 7ff64f400700 1 --2- 172.21.15.107:0/278875020 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7ff63c06d200 0x7ff63c06f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:44:12.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:12.421+0000 7ff64f400700 1 -- 172.21.15.107:0/278875020 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7ff64808dbe0 con 0x7ff6580fcef0 2024-03-04T13:44:12.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:12.421+0000 7ff64fe00700 1 --2- 172.21.15.107:0/278875020 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7ff63c06d200 0x7ff63c06f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:44:12.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:12.422+0000 7ff64fe00700 1 --2- 172.21.15.107:0/278875020 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7ff63c06d200 0x7ff63c06f6b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff640004d60 tx=0x7ff64000b040 comp rx=0 tx=0).ready entity=mgr.14696 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:44:12.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:12.426+0000 7ff64f400700 1 -- 172.21.15.107:0/278875020 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7ff64805b5f0 con 0x7ff6580fcef0 2024-03-04T13:44:12.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:12.695+0000 7ff65e600700 1 -- 172.21.15.107:0/278875020 --> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff65802b310 con 0x7ff63c06d200 2024-03-04T13:44:12.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:12.698+0000 7ff64f400700 1 -- 172.21.15.107:0/278875020 <== mgr.14696 v2:172.21.15.107:6800/1613321046 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+360 (secure 0 0 0) 0x7ff65802b310 con 0x7ff63c06d200 2024-03-04T13:44:12.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:12.701+0000 7ff65e600700 1 -- 172.21.15.107:0/278875020 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7ff63c06d200 msgr2=0x7ff63c06f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:44:12.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:12.701+0000 7ff65e600700 1 --2- 172.21.15.107:0/278875020 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7ff63c06d200 0x7ff63c06f6b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff640004d60 tx=0x7ff64000b040 comp rx=0 tx=0).stop 2024-03-04T13:44:12.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:12.701+0000 7ff65e600700 1 -- 172.21.15.107:0/278875020 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6580fcef0 msgr2=0x7ff65810fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:44:12.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:12.701+0000 7ff65e600700 1 --2- 172.21.15.107:0/278875020 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6580fcef0 0x7ff65810fc40 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7ff648009360 tx=0x7ff648009e30 comp rx=0 tx=0).stop 2024-03-04T13:44:12.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:12.701+0000 7ff65e600700 1 -- 172.21.15.107:0/278875020 shutdown_connections 2024-03-04T13:44:12.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:12.701+0000 7ff65e600700 1 --2- 172.21.15.107:0/278875020 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7ff63c06d200 0x7ff63c06f6b0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:44:12.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:12.701+0000 7ff65e600700 1 --2- 172.21.15.107:0/278875020 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6580fcef0 0x7ff65810fc40 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:44:12.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:12.701+0000 7ff65e600700 1 --2- 172.21.15.107:0/278875020 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff6580fe0e0 0x7ff658110180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:44:12.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:12.701+0000 7ff65e600700 1 -- 172.21.15.107:0/278875020 >> 172.21.15.107:0/278875020 conn(0x7ff6580f8620 msgr2=0x7ff6580ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:44:12.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:12.701+0000 7ff65e600700 1 -- 172.21.15.107:0/278875020 shutdown_connections 2024-03-04T13:44:12.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:12.701+0000 7ff65e600700 1 -- 172.21.15.107:0/278875020 wait complete. 2024-03-04T13:44:12.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:12.889+0000 7fca6d600700 1 -- 172.21.15.107:0/2075763216 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fca680ff150 msgr2=0x7fca680ff560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:44:12.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:12.889+0000 7fca6d600700 1 --2- 172.21.15.107:0/2075763216 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fca680ff150 0x7fca680ff560 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fca58004940 tx=0x7fca58009e30 comp rx=0 tx=0).stop 2024-03-04T13:44:12.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:12.889+0000 7fca6d600700 1 -- 172.21.15.107:0/2075763216 shutdown_connections 2024-03-04T13:44:12.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:12.889+0000 7fca6d600700 1 --2- 172.21.15.107:0/2075763216 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca68100360 0x7fca681007b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:44:12.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:12.889+0000 7fca6d600700 1 --2- 172.21.15.107:0/2075763216 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fca680ff150 0x7fca680ff560 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:44:12.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:12.889+0000 7fca6d600700 1 -- 172.21.15.107:0/2075763216 >> 172.21.15.107:0/2075763216 conn(0x7fca680fa920 msgr2=0x7fca680fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:44:12.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:12.889+0000 7fca6d600700 1 -- 172.21.15.107:0/2075763216 shutdown_connections 2024-03-04T13:44:12.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:12.889+0000 7fca6d600700 1 -- 172.21.15.107:0/2075763216 wait complete. 2024-03-04T13:44:12.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:12.890+0000 7fca6d600700 1 Processor -- start 2024-03-04T13:44:12.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:12.890+0000 7fca6d600700 1 -- start start 2024-03-04T13:44:12.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:12.890+0000 7fca6d600700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca680ff150 0x7fca6810fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:44:12.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:12.890+0000 7fca6d600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fca68100360 0x7fca68110050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:44:12.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:12.890+0000 7fca6d600700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca68110650 con 0x7fca68100360 2024-03-04T13:44:12.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:12.890+0000 7fca6d600700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca68110ef0 con 0x7fca680ff150 2024-03-04T13:44:12.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:12.891+0000 7fca66a00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca680ff150 0x7fca6810fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:44:12.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:12.891+0000 7fca66a00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca680ff150 0x7fca6810fb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.107:51464/0 (socket says 172.21.15.107:51464) 2024-03-04T13:44:12.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:12.891+0000 7fca66a00700 1 -- 172.21.15.107:0/2402199557 learned_addr learned my addr 172.21.15.107:0/2402199557 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:44:12.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:12.891+0000 7fca66000700 1 --2- 172.21.15.107:0/2402199557 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fca68100360 0x7fca68110050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:44:12.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:12.891+0000 7fca66a00700 1 -- 172.21.15.107:0/2402199557 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fca68100360 msgr2=0x7fca68110050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:44:12.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:12.891+0000 7fca66a00700 1 --2- 172.21.15.107:0/2402199557 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fca68100360 0x7fca68110050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:44:12.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:12.891+0000 7fca66a00700 1 -- 172.21.15.107:0/2402199557 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca58005040 con 0x7fca680ff150 2024-03-04T13:44:12.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:12.892+0000 7fca66a00700 1 --2- 172.21.15.107:0/2402199557 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca680ff150 0x7fca6810fb10 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fca58009fd0 tx=0x7fca58009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:44:12.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:12.892+0000 7fca5ea00700 1 -- 172.21.15.107:0/2402199557 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fca58021070 con 0x7fca680ff150 2024-03-04T13:44:12.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:12.892+0000 7fca6d600700 1 -- 172.21.15.107:0/2402199557 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca68111170 con 0x7fca680ff150 2024-03-04T13:44:12.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:12.892+0000 7fca5ea00700 1 -- 172.21.15.107:0/2402199557 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fca5801c400 con 0x7fca680ff150 2024-03-04T13:44:12.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:12.892+0000 7fca5ea00700 1 -- 172.21.15.107:0/2402199557 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fca5800e070 con 0x7fca680ff150 2024-03-04T13:44:12.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:12.892+0000 7fca6d600700 1 -- 172.21.15.107:0/2402199557 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca68111650 con 0x7fca680ff150 2024-03-04T13:44:12.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:12.893+0000 7fca6d600700 1 -- 172.21.15.107:0/2402199557 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca6804bed0 con 0x7fca680ff150 2024-03-04T13:44:12.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:12.894+0000 7fca5ea00700 1 -- 172.21.15.107:0/2402199557 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 31) v1 ==== 91301+0+0 (secure 0 0 0) 0x7fca5801ca50 con 0x7fca680ff150 2024-03-04T13:44:12.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:12.895+0000 7fca5ea00700 1 --2- 172.21.15.107:0/2402199557 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7fca4c06d200 0x7fca4c06f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:44:12.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:12.895+0000 7fca5ea00700 1 -- 172.21.15.107:0/2402199557 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fca5808da30 con 0x7fca680ff150 2024-03-04T13:44:12.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:12.895+0000 7fca66000700 1 --2- 172.21.15.107:0/2402199557 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7fca4c06d200 0x7fca4c06f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:44:12.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:12.896+0000 7fca66000700 1 --2- 172.21.15.107:0/2402199557 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7fca4c06d200 0x7fca4c06f6b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fca54009b80 tx=0x7fca5400b040 comp rx=0 tx=0).ready entity=mgr.14696 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:44:12.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:12.903+0000 7fca5ea00700 1 -- 172.21.15.107:0/2402199557 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7fca5805b3c0 con 0x7fca680ff150 2024-03-04T13:44:13.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:13.152+0000 7fca6d600700 1 -- 172.21.15.107:0/2402199557 --> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fca68104cb0 con 0x7fca4c06d200 2024-03-04T13:44:13.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:13.167+0000 7fca5ea00700 1 -- 172.21.15.107:0/2402199557 <== mgr.14696 v2:172.21.15.107:6800/1613321046 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7fca68104cb0 con 0x7fca4c06d200 2024-03-04T13:44:13.165 INFO:teuthology.orchestra.run.smithi107.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-04T13:44:13.165 INFO:teuthology.orchestra.run.smithi107.stdout:alertmanager.smithi107 smithi107 *:9093,9094 running (2m) 26s ago 12m 15.2M - 0.25.0 c8568f914cd2 ac0e4f11c97a 2024-03-04T13:44:13.165 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi107 smithi107 running (12m) 26s ago 12m 6665k - 17.2.7-545-g42608bb1 9eec3f22859d 03e1570c2eff 2024-03-04T13:44:13.165 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi169 smithi169 running (11m) 25s ago 11m 6656k - 17.2.7-545-g42608bb1 9eec3f22859d 9c8881a5338e 2024-03-04T13:44:13.165 INFO:teuthology.orchestra.run.smithi107.stdout:grafana.smithi107 smithi107 *:3000 running (106s) 26s ago 12m 74.7M - 9.4.7 954c08fa6188 bd720c203473 2024-03-04T13:44:13.165 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.riexmm smithi107 running (5m) 26s ago 5m 12.5M - 17.2.7-545-g42608bb1 9eec3f22859d fc921f7d4328 2024-03-04T13:44:13.165 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.thycho smithi107 running (5m) 26s ago 5m 277M - 17.2.7-545-g42608bb1 9eec3f22859d 4069c6e0e063 2024-03-04T13:44:13.165 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi169.bxitzk smithi169 running (5m) 25s ago 5m 12.6M - 17.2.7-545-g42608bb1 9eec3f22859d 7d4115228c7c 2024-03-04T13:44:13.165 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi169.jusudl smithi169 running (5m) 25s ago 5m 243M - 17.2.7-545-g42608bb1 9eec3f22859d 2d2f2cc452e8 2024-03-04T13:44:13.165 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi107.wpzgwl smithi107 *:8443,9283,8765 running (62s) 26s ago 14m 475M - 19.0.0-1845-g717ce3cc d71b839abdf8 1c5fdee45eac 2024-03-04T13:44:13.165 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi169.eihczr smithi169 *:8443,9283 running (3m) 25s ago 11m 66.7M - 19.0.0-1845-g717ce3cc d71b839abdf8 18b4dfa896b5 2024-03-04T13:44:13.166 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi107 smithi107 running (14m) 26s ago 14m 50.0M 2048M 17.2.7-545-g42608bb1 9eec3f22859d 49f5da990d35 2024-03-04T13:44:13.166 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi169 smithi169 running (11m) 25s ago 11m 41.0M 2048M 17.2.7-545-g42608bb1 9eec3f22859d ba4ee916297f 2024-03-04T13:44:13.166 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi107 smithi107 *:9100 running (2m) 26s ago 12m 7256k - 1.5.0 0da6a335fe13 b31d3c803078 2024-03-04T13:44:13.166 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi169 smithi169 *:9100 running (84s) 25s ago 11m 7558k - 1.5.0 0da6a335fe13 043154b90aa2 2024-03-04T13:44:13.166 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0 smithi107 running (10m) 26s ago 10m 169M 1558M 17.2.7-545-g42608bb1 9eec3f22859d c0605b68f696 2024-03-04T13:44:13.166 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1 smithi107 running (9m) 26s ago 9m 181M 1558M 17.2.7-545-g42608bb1 9eec3f22859d 5fb2c0dd5769 2024-03-04T13:44:13.166 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2 smithi107 running (9m) 26s ago 9m 152M 1558M 17.2.7-545-g42608bb1 9eec3f22859d 61f30f643549 2024-03-04T13:44:13.166 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3 smithi169 running (8m) 25s ago 8m 205M 2582M 17.2.7-545-g42608bb1 9eec3f22859d cd91d8694764 2024-03-04T13:44:13.166 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4 smithi169 running (7m) 25s ago 7m 178M 2582M 17.2.7-545-g42608bb1 9eec3f22859d 08545e3d0659 2024-03-04T13:44:13.166 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5 smithi169 running (7m) 25s ago 7m 174M 2582M 17.2.7-545-g42608bb1 9eec3f22859d dcffe6cb3eac 2024-03-04T13:44:13.166 INFO:teuthology.orchestra.run.smithi107.stdout:prometheus.smithi107 smithi107 *:9095 starting - - - - 2024-03-04T13:44:13.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:13.171+0000 7fca6d600700 1 -- 172.21.15.107:0/2402199557 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7fca4c06d200 msgr2=0x7fca4c06f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:44:13.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:13.171+0000 7fca6d600700 1 --2- 172.21.15.107:0/2402199557 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7fca4c06d200 0x7fca4c06f6b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fca54009b80 tx=0x7fca5400b040 comp rx=0 tx=0).stop 2024-03-04T13:44:13.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:13.171+0000 7fca6d600700 1 -- 172.21.15.107:0/2402199557 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca680ff150 msgr2=0x7fca6810fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:44:13.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:13.171+0000 7fca6d600700 1 --2- 172.21.15.107:0/2402199557 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca680ff150 0x7fca6810fb10 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fca58009fd0 tx=0x7fca58009e30 comp rx=0 tx=0).stop 2024-03-04T13:44:13.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:13.171+0000 7fca6d600700 1 -- 172.21.15.107:0/2402199557 shutdown_connections 2024-03-04T13:44:13.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:13.171+0000 7fca6d600700 1 --2- 172.21.15.107:0/2402199557 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fca680ff150 0x7fca6810fb10 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:44:13.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:13.171+0000 7fca6d600700 1 --2- 172.21.15.107:0/2402199557 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7fca4c06d200 0x7fca4c06f6b0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:44:13.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:13.171+0000 7fca6d600700 1 --2- 172.21.15.107:0/2402199557 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fca68100360 0x7fca68110050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:44:13.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:13.171+0000 7fca6d600700 1 -- 172.21.15.107:0/2402199557 >> 172.21.15.107:0/2402199557 conn(0x7fca680fa920 msgr2=0x7fca68103590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:44:13.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:13.172+0000 7fca6d600700 1 -- 172.21.15.107:0/2402199557 shutdown_connections 2024-03-04T13:44:13.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:13.172+0000 7fca6d600700 1 -- 172.21.15.107:0/2402199557 wait complete. 2024-03-04T13:44:13.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:13.326+0000 7fbb8ca00700 1 -- 172.21.15.107:0/4292456397 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbb880663e0 msgr2=0x7fbb880ffaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:44:13.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:13.326+0000 7fbb8ca00700 1 --2- 172.21.15.107:0/4292456397 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbb880663e0 0x7fbb880ffaf0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fbb78004940 tx=0x7fbb78009e30 comp rx=0 tx=0).stop 2024-03-04T13:44:13.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:13.326+0000 7fbb8ca00700 1 -- 172.21.15.107:0/4292456397 shutdown_connections 2024-03-04T13:44:13.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:13.326+0000 7fbb8ca00700 1 --2- 172.21.15.107:0/4292456397 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb881000c0 0x7fbb88102520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:44:13.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:13.326+0000 7fbb8ca00700 1 --2- 172.21.15.107:0/4292456397 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbb880663e0 0x7fbb880ffaf0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:44:13.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:13.326+0000 7fbb8ca00700 1 -- 172.21.15.107:0/4292456397 >> 172.21.15.107:0/4292456397 conn(0x7fbb880f74e0 msgr2=0x7fbb880f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:44:13.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:13.326+0000 7fbb8ca00700 1 -- 172.21.15.107:0/4292456397 shutdown_connections 2024-03-04T13:44:13.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:13.326+0000 7fbb8ca00700 1 -- 172.21.15.107:0/4292456397 wait complete. 2024-03-04T13:44:13.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:13.327+0000 7fbb8ca00700 1 Processor -- start 2024-03-04T13:44:13.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:13.327+0000 7fbb8ca00700 1 -- start start 2024-03-04T13:44:13.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:13.327+0000 7fbb8ca00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb880663e0 0x7fbb8810fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:44:13.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:13.327+0000 7fbb8ca00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbb881000c0 0x7fbb88110060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:44:13.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:13.327+0000 7fbb8ca00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb88110660 con 0x7fbb881000c0 2024-03-04T13:44:13.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:13.328+0000 7fbb8ca00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb88110f00 con 0x7fbb880663e0 2024-03-04T13:44:13.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:13.328+0000 7fbb85400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbb881000c0 0x7fbb88110060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:44:13.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:13.328+0000 7fbb85400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbb881000c0 0x7fbb88110060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55862/0 (socket says 172.21.15.107:55862) 2024-03-04T13:44:13.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:13.328+0000 7fbb85400700 1 -- 172.21.15.107:0/1232825706 learned_addr learned my addr 172.21.15.107:0/1232825706 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:44:13.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:13.328+0000 7fbb85e00700 1 --2- 172.21.15.107:0/1232825706 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb880663e0 0x7fbb8810fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:44:13.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:13.328+0000 7fbb85400700 1 -- 172.21.15.107:0/1232825706 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb880663e0 msgr2=0x7fbb8810fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:44:13.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:13.328+0000 7fbb85400700 1 --2- 172.21.15.107:0/1232825706 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb880663e0 0x7fbb8810fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:44:13.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:13.328+0000 7fbb85400700 1 -- 172.21.15.107:0/1232825706 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb78005040 con 0x7fbb881000c0 2024-03-04T13:44:13.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:13.328+0000 7fbb85400700 1 --2- 172.21.15.107:0/1232825706 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbb881000c0 0x7fbb88110060 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fbb7400eb40 tx=0x7fbb7400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:44:13.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:13.329+0000 7fbb7e000700 1 -- 172.21.15.107:0/1232825706 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbb74007cc0 con 0x7fbb881000c0 2024-03-04T13:44:13.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:13.329+0000 7fbb7e000700 1 -- 172.21.15.107:0/1232825706 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbb7400f040 con 0x7fbb881000c0 2024-03-04T13:44:13.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:13.329+0000 7fbb7e000700 1 -- 172.21.15.107:0/1232825706 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbb7400cad0 con 0x7fbb881000c0 2024-03-04T13:44:13.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:13.329+0000 7fbb8ca00700 1 -- 172.21.15.107:0/1232825706 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb88111180 con 0x7fbb881000c0 2024-03-04T13:44:13.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:13.329+0000 7fbb8ca00700 1 -- 172.21.15.107:0/1232825706 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb881116c0 con 0x7fbb881000c0 2024-03-04T13:44:13.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:13.330+0000 7fbb8ca00700 1 -- 172.21.15.107:0/1232825706 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb8804bed0 con 0x7fbb881000c0 2024-03-04T13:44:13.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:13.331+0000 7fbb7e000700 1 -- 172.21.15.107:0/1232825706 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 31) v1 ==== 91301+0+0 (secure 0 0 0) 0x7fbb7400cc30 con 0x7fbb881000c0 2024-03-04T13:44:13.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:13.332+0000 7fbb7e000700 1 --2- 172.21.15.107:0/1232825706 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7fbb6c06d200 0x7fbb6c06f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:44:13.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:13.332+0000 7fbb7e000700 1 -- 172.21.15.107:0/1232825706 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fbb7408c3a0 con 0x7fbb881000c0 2024-03-04T13:44:13.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:13.332+0000 7fbb85e00700 1 --2- 172.21.15.107:0/1232825706 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7fbb6c06d200 0x7fbb6c06f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:44:13.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:13.333+0000 7fbb85e00700 1 --2- 172.21.15.107:0/1232825706 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7fbb6c06d200 0x7fbb6c06f6b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fbb780136a0 tx=0x7fbb78009e30 comp rx=0 tx=0).ready entity=mgr.14696 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:44:13.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:13.337+0000 7fbb7e000700 1 -- 172.21.15.107:0/1232825706 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7fbb74059cb0 con 0x7fbb881000c0 2024-03-04T13:44:13.725 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:13.727+0000 7fbb8ca00700 1 -- 172.21.15.107:0/1232825706 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fbb88105a50 con 0x7fbb881000c0 2024-03-04T13:44:13.725 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:13.728+0000 7fbb7e000700 1 -- 172.21.15.107:0/1232825706 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7fbb74059840 con 0x7fbb881000c0 2024-03-04T13:44:13.726 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-03-04T13:44:13.726 INFO:teuthology.orchestra.run.smithi107.stdout: "mon": { 2024-03-04T13:44:13.726 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-545-g42608bb1 (42608bb191117be10dba720e6bf8a02f3171cd34) quincy (stable)": 2 2024-03-04T13:44:13.726 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:44:13.726 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr": { 2024-03-04T13:44:13.726 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-1845-g717ce3cc (717ce3cce8d0166e4e577faa8004e6f9cb4128c0) squid (dev)": 2 2024-03-04T13:44:13.726 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:44:13.726 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": { 2024-03-04T13:44:13.726 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-545-g42608bb1 (42608bb191117be10dba720e6bf8a02f3171cd34) quincy (stable)": 6 2024-03-04T13:44:13.726 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:44:13.726 INFO:teuthology.orchestra.run.smithi107.stdout: "mds": { 2024-03-04T13:44:13.726 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-545-g42608bb1 (42608bb191117be10dba720e6bf8a02f3171cd34) quincy (stable)": 4 2024-03-04T13:44:13.726 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:44:13.727 INFO:teuthology.orchestra.run.smithi107.stdout: "overall": { 2024-03-04T13:44:13.727 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-545-g42608bb1 (42608bb191117be10dba720e6bf8a02f3171cd34) quincy (stable)": 12, 2024-03-04T13:44:13.727 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-1845-g717ce3cc (717ce3cce8d0166e4e577faa8004e6f9cb4128c0) squid (dev)": 2 2024-03-04T13:44:13.727 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-04T13:44:13.727 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-03-04T13:44:13.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:13.731+0000 7fbb8ca00700 1 -- 172.21.15.107:0/1232825706 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7fbb6c06d200 msgr2=0x7fbb6c06f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:44:13.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:13.731+0000 7fbb8ca00700 1 --2- 172.21.15.107:0/1232825706 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7fbb6c06d200 0x7fbb6c06f6b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fbb780136a0 tx=0x7fbb78009e30 comp rx=0 tx=0).stop 2024-03-04T13:44:13.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:13.731+0000 7fbb8ca00700 1 -- 172.21.15.107:0/1232825706 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbb881000c0 msgr2=0x7fbb88110060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:44:13.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:13.731+0000 7fbb8ca00700 1 --2- 172.21.15.107:0/1232825706 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbb881000c0 0x7fbb88110060 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fbb7400eb40 tx=0x7fbb7400b040 comp rx=0 tx=0).stop 2024-03-04T13:44:13.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:13.731+0000 7fbb8ca00700 1 -- 172.21.15.107:0/1232825706 shutdown_connections 2024-03-04T13:44:13.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:13.731+0000 7fbb8ca00700 1 --2- 172.21.15.107:0/1232825706 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb880663e0 0x7fbb8810fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:44:13.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:13.731+0000 7fbb8ca00700 1 --2- 172.21.15.107:0/1232825706 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7fbb6c06d200 0x7fbb6c06f6b0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:44:13.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:13.731+0000 7fbb8ca00700 1 --2- 172.21.15.107:0/1232825706 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbb881000c0 0x7fbb88110060 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:44:13.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:13.731+0000 7fbb8ca00700 1 -- 172.21.15.107:0/1232825706 >> 172.21.15.107:0/1232825706 conn(0x7fbb880f74e0 msgr2=0x7fbb880fb8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:44:13.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:13.731+0000 7fbb8ca00700 1 -- 172.21.15.107:0/1232825706 shutdown_connections 2024-03-04T13:44:13.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:13.732+0000 7fbb8ca00700 1 -- 172.21.15.107:0/1232825706 wait complete. 2024-03-04T13:44:13.856 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:13 smithi169 ceph-mon[36313]: pgmap v20: 65 pgs: 65 active+clean; 298 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 69 op/s 2024-03-04T13:44:13.857 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:13 smithi169 ceph-mon[36313]: from='client.14744 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:44:13.857 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:13 smithi169 ceph-mon[36313]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:44:13.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:13.892+0000 7fc122a00700 1 -- 172.21.15.107:0/3306119550 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc11c0fef60 msgr2=0x7fc11c0ff370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:44:13.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:13.892+0000 7fc122a00700 1 --2- 172.21.15.107:0/3306119550 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc11c0fef60 0x7fc11c0ff370 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fc110004940 tx=0x7fc110009e30 comp rx=0 tx=0).stop 2024-03-04T13:44:13.890 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:13 smithi107 ceph-mon[28739]: pgmap v20: 65 pgs: 65 active+clean; 298 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 69 op/s 2024-03-04T13:44:13.890 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:13 smithi107 ceph-mon[28739]: from='client.14744 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:44:13.890 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:13 smithi107 ceph-mon[28739]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:44:13.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:13.893+0000 7fc122a00700 1 -- 172.21.15.107:0/3306119550 shutdown_connections 2024-03-04T13:44:13.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:13.893+0000 7fc122a00700 1 --2- 172.21.15.107:0/3306119550 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc11c100170 0x7fc11c1005c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:44:13.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:13.893+0000 7fc122a00700 1 --2- 172.21.15.107:0/3306119550 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc11c0fef60 0x7fc11c0ff370 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:44:13.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:13.893+0000 7fc122a00700 1 -- 172.21.15.107:0/3306119550 >> 172.21.15.107:0/3306119550 conn(0x7fc11c0fa760 msgr2=0x7fc11c0fcb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:44:13.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:13.893+0000 7fc122a00700 1 -- 172.21.15.107:0/3306119550 shutdown_connections 2024-03-04T13:44:13.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:13.893+0000 7fc122a00700 1 -- 172.21.15.107:0/3306119550 wait complete. 2024-03-04T13:44:13.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:13.894+0000 7fc122a00700 1 Processor -- start 2024-03-04T13:44:13.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:13.894+0000 7fc122a00700 1 -- start start 2024-03-04T13:44:13.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:13.894+0000 7fc122a00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc11c0fef60 0x7fc11c10f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:44:13.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:13.894+0000 7fc122a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc11c100170 0x7fc11c10fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:44:13.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:13.894+0000 7fc122a00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc11c110440 con 0x7fc11c100170 2024-03-04T13:44:13.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:13.894+0000 7fc122a00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc11c110ce0 con 0x7fc11c0fef60 2024-03-04T13:44:13.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:13.895+0000 7fc120c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc11c100170 0x7fc11c10fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:44:13.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:13.895+0000 7fc120c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc11c100170 0x7fc11c10fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55874/0 (socket says 172.21.15.107:55874) 2024-03-04T13:44:13.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:13.895+0000 7fc120c00700 1 -- 172.21.15.107:0/3229409562 learned_addr learned my addr 172.21.15.107:0/3229409562 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:44:13.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:13.895+0000 7fc121600700 1 --2- 172.21.15.107:0/3229409562 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc11c0fef60 0x7fc11c10f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:44:13.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:13.895+0000 7fc120c00700 1 -- 172.21.15.107:0/3229409562 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc11c0fef60 msgr2=0x7fc11c10f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:44:13.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:13.895+0000 7fc120c00700 1 --2- 172.21.15.107:0/3229409562 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc11c0fef60 0x7fc11c10f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:44:13.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:13.895+0000 7fc120c00700 1 -- 172.21.15.107:0/3229409562 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc110005040 con 0x7fc11c100170 2024-03-04T13:44:13.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:13.895+0000 7fc121600700 1 --2- 172.21.15.107:0/3229409562 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc11c0fef60 0x7fc11c10f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T13:44:13.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:13.896+0000 7fc120c00700 1 --2- 172.21.15.107:0/3229409562 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc11c100170 0x7fc11c10fe40 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fc10800eb40 tx=0x7fc10800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:44:13.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:13.896+0000 7fc119600700 1 -- 172.21.15.107:0/3229409562 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc108007cc0 con 0x7fc11c100170 2024-03-04T13:44:13.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:13.896+0000 7fc119600700 1 -- 172.21.15.107:0/3229409562 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc10800f040 con 0x7fc11c100170 2024-03-04T13:44:13.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:13.896+0000 7fc122a00700 1 -- 172.21.15.107:0/3229409562 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc11c110f60 con 0x7fc11c100170 2024-03-04T13:44:13.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:13.896+0000 7fc119600700 1 -- 172.21.15.107:0/3229409562 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc108009620 con 0x7fc11c100170 2024-03-04T13:44:13.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:13.896+0000 7fc122a00700 1 -- 172.21.15.107:0/3229409562 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc11c1114a0 con 0x7fc11c100170 2024-03-04T13:44:13.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:13.897+0000 7fc122a00700 1 -- 172.21.15.107:0/3229409562 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc11c04bed0 con 0x7fc11c100170 2024-03-04T13:44:13.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:13.898+0000 7fc119600700 1 -- 172.21.15.107:0/3229409562 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 31) v1 ==== 91301+0+0 (secure 0 0 0) 0x7fc108009780 con 0x7fc11c100170 2024-03-04T13:44:13.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:13.899+0000 7fc119600700 1 --2- 172.21.15.107:0/3229409562 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7fc10406d200 0x7fc10406f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:44:13.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:13.899+0000 7fc119600700 1 -- 172.21.15.107:0/3229409562 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fc10808c460 con 0x7fc11c100170 2024-03-04T13:44:13.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:13.900+0000 7fc121600700 1 --2- 172.21.15.107:0/3229409562 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7fc10406d200 0x7fc10406f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:44:13.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:13.900+0000 7fc121600700 1 --2- 172.21.15.107:0/3229409562 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7fc10406d200 0x7fc10406f6b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc11000b0d0 tx=0x7fc110009e30 comp rx=0 tx=0).ready entity=mgr.14696 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:44:13.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:13.904+0000 7fc119600700 1 -- 172.21.15.107:0/3229409562 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7fc108059ea0 con 0x7fc11c100170 2024-03-04T13:44:14.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:14.164+0000 7fc122a00700 1 -- 172.21.15.107:0/3229409562 --> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc11c111ac0 con 0x7fc10406d200 2024-03-04T13:44:14.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:14.166+0000 7fc119600700 1 -- 172.21.15.107:0/3229409562 <== mgr.14696 v2:172.21.15.107:6800/1613321046 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+360 (secure 0 0 0) 0x7fc11c111ac0 con 0x7fc10406d200 2024-03-04T13:44:14.163 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-03-04T13:44:14.164 INFO:teuthology.orchestra.run.smithi107.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377", 2024-03-04T13:44:14.164 INFO:teuthology.orchestra.run.smithi107.stdout: "in_progress": true, 2024-03-04T13:44:14.164 INFO:teuthology.orchestra.run.smithi107.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-03-04T13:44:14.164 INFO:teuthology.orchestra.run.smithi107.stdout: "services_complete": [ 2024-03-04T13:44:14.164 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr" 2024-03-04T13:44:14.164 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-04T13:44:14.164 INFO:teuthology.orchestra.run.smithi107.stdout: "progress": "2/2 daemons upgraded", 2024-03-04T13:44:14.164 INFO:teuthology.orchestra.run.smithi107.stdout: "message": "Currently upgrading mgr daemons", 2024-03-04T13:44:14.164 INFO:teuthology.orchestra.run.smithi107.stdout: "is_paused": false 2024-03-04T13:44:14.164 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-03-04T13:44:14.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:14.169+0000 7fc122a00700 1 -- 172.21.15.107:0/3229409562 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7fc10406d200 msgr2=0x7fc10406f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:44:14.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:14.169+0000 7fc122a00700 1 --2- 172.21.15.107:0/3229409562 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7fc10406d200 0x7fc10406f6b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc11000b0d0 tx=0x7fc110009e30 comp rx=0 tx=0).stop 2024-03-04T13:44:14.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:14.169+0000 7fc122a00700 1 -- 172.21.15.107:0/3229409562 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc11c100170 msgr2=0x7fc11c10fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:44:14.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:14.169+0000 7fc122a00700 1 --2- 172.21.15.107:0/3229409562 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc11c100170 0x7fc11c10fe40 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fc10800eb40 tx=0x7fc10800b040 comp rx=0 tx=0).stop 2024-03-04T13:44:14.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:14.169+0000 7fc122a00700 1 -- 172.21.15.107:0/3229409562 shutdown_connections 2024-03-04T13:44:14.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:14.169+0000 7fc122a00700 1 --2- 172.21.15.107:0/3229409562 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc11c0fef60 0x7fc11c10f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:44:14.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:14.169+0000 7fc122a00700 1 --2- 172.21.15.107:0/3229409562 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7fc10406d200 0x7fc10406f6b0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:44:14.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:14.169+0000 7fc122a00700 1 --2- 172.21.15.107:0/3229409562 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc11c100170 0x7fc11c10fe40 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:44:14.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:14.169+0000 7fc122a00700 1 -- 172.21.15.107:0/3229409562 >> 172.21.15.107:0/3229409562 conn(0x7fc11c0fa760 msgr2=0x7fc11c1033a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:44:14.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:14.169+0000 7fc122a00700 1 -- 172.21.15.107:0/3229409562 shutdown_connections 2024-03-04T13:44:14.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:14.169+0000 7fc122a00700 1 -- 172.21.15.107:0/3229409562 wait complete. 2024-03-04T13:44:14.858 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:14 smithi169 ceph-mon[36313]: from='client.24483 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:44:14.858 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:14 smithi169 ceph-mon[36313]: from='client.? 172.21.15.107:0/1232825706' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:44:15.091 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:14 smithi107 ceph-mon[28739]: from='client.24483 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:44:15.091 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:14 smithi107 ceph-mon[28739]: from='client.? 172.21.15.107:0/1232825706' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:44:15.858 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:15 smithi169 ceph-mon[36313]: pgmap v21: 65 pgs: 65 active+clean; 293 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 85 op/s 2024-03-04T13:44:15.858 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:15 smithi169 ceph-mon[36313]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:44:15.858 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:15 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:44:15.858 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:15 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:44:15.858 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:15 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:44:16.091 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:15 smithi107 ceph-mon[28739]: pgmap v21: 65 pgs: 65 active+clean; 293 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 85 op/s 2024-03-04T13:44:16.091 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:15 smithi107 ceph-mon[28739]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:44:16.091 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:15 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:44:16.091 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:15 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:44:16.091 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:15 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:44:17.091 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:16 smithi107 ceph-mon[28739]: pgmap v22: 65 pgs: 65 active+clean; 297 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 105 op/s 2024-03-04T13:44:17.108 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:16 smithi169 ceph-mon[36313]: pgmap v22: 65 pgs: 65 active+clean; 297 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 105 op/s 2024-03-04T13:44:19.358 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:19 smithi169 ceph-mon[36313]: pgmap v23: 65 pgs: 65 active+clean; 297 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 67 op/s 2024-03-04T13:44:19.590 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:19 smithi107 ceph-mon[28739]: pgmap v23: 65 pgs: 65 active+clean; 297 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 67 op/s 2024-03-04T13:44:21.591 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:21 smithi107 ceph-mon[28739]: pgmap v24: 65 pgs: 65 active+clean; 298 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 110 op/s 2024-03-04T13:44:21.591 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:21 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:44:21.591 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:21 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:44:21.608 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:21 smithi169 ceph-mon[36313]: pgmap v24: 65 pgs: 65 active+clean; 298 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 110 op/s 2024-03-04T13:44:21.608 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:21 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:44:21.608 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:21 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:44:23.841 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:23 smithi107 ceph-mon[28739]: pgmap v25: 65 pgs: 65 active+clean; 298 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 80 op/s 2024-03-04T13:44:23.841 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:23 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:44:23.841 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:23 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:44:23.841 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:23 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:44:23.850 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:23 smithi169 ceph-mon[36313]: pgmap v25: 65 pgs: 65 active+clean; 298 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 80 op/s 2024-03-04T13:44:23.850 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:23 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:44:23.850 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:23 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:44:23.850 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:23 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:44:25.341 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:25 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:44:25.341 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:25 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:44:25.341 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:25 smithi107 ceph-mon[28739]: pgmap v26: 65 pgs: 65 active+clean; 303 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 111 op/s 2024-03-04T13:44:25.358 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:25 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:44:25.359 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:25 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:44:25.359 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:25 smithi169 ceph-mon[36313]: pgmap v26: 65 pgs: 65 active+clean; 303 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 111 op/s 2024-03-04T13:44:27.091 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:26 smithi107 ceph-mon[28739]: pgmap v27: 65 pgs: 65 active+clean; 302 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 100 op/s 2024-03-04T13:44:27.108 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:26 smithi169 ceph-mon[36313]: pgmap v27: 65 pgs: 65 active+clean; 302 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 100 op/s 2024-03-04T13:44:28.091 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:27 smithi107 ceph-mon[28739]: Standby manager daemon smithi169.eihczr restarted 2024-03-04T13:44:28.091 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:27 smithi107 ceph-mon[28739]: Standby manager daemon smithi169.eihczr started 2024-03-04T13:44:28.091 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:27 smithi107 ceph-mon[28739]: from='mgr.? 172.21.15.169:0/3747284671' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi169.eihczr/crt"}]: dispatch 2024-03-04T13:44:28.091 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:27 smithi107 ceph-mon[28739]: from='mgr.? 172.21.15.169:0/3747284671' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-04T13:44:28.091 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:27 smithi107 ceph-mon[28739]: from='mgr.? 172.21.15.169:0/3747284671' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi169.eihczr/key"}]: dispatch 2024-03-04T13:44:28.092 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:27 smithi107 ceph-mon[28739]: from='mgr.? 172.21.15.169:0/3747284671' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-04T13:44:28.108 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:27 smithi169 ceph-mon[36313]: Standby manager daemon smithi169.eihczr restarted 2024-03-04T13:44:28.108 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:27 smithi169 ceph-mon[36313]: Standby manager daemon smithi169.eihczr started 2024-03-04T13:44:28.108 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:27 smithi169 ceph-mon[36313]: from='mgr.? 172.21.15.169:0/3747284671' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi169.eihczr/crt"}]: dispatch 2024-03-04T13:44:28.108 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:27 smithi169 ceph-mon[36313]: from='mgr.? 172.21.15.169:0/3747284671' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-04T13:44:28.108 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:27 smithi169 ceph-mon[36313]: from='mgr.? 172.21.15.169:0/3747284671' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi169.eihczr/key"}]: dispatch 2024-03-04T13:44:28.108 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:27 smithi169 ceph-mon[36313]: from='mgr.? 172.21.15.169:0/3747284671' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-04T13:44:29.091 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:28 smithi107 ceph-mon[28739]: mgrmap e32: smithi107.wpzgwl(active, since 49s), standbys: smithi169.eihczr 2024-03-04T13:44:29.091 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:28 smithi107 ceph-mon[28739]: pgmap v28: 65 pgs: 65 active+clean; 302 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 78 op/s 2024-03-04T13:44:29.108 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:28 smithi169 ceph-mon[36313]: mgrmap e32: smithi107.wpzgwl(active, since 49s), standbys: smithi169.eihczr 2024-03-04T13:44:29.108 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:28 smithi169 ceph-mon[36313]: pgmap v28: 65 pgs: 65 active+clean; 302 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 78 op/s 2024-03-04T13:44:30.317 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:29 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:44:30.317 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:29 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:44:30.317 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:29 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:44:30.317 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:29 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:44:30.317 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:29 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:44:30.317 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:29 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:44:30.318 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:29 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:44:30.318 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:29 smithi107 ceph-mon[28739]: Upgrade: Setting container_image for all mgr 2024-03-04T13:44:30.318 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:29 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:44:30.318 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:29 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi107.wpzgwl"}]: dispatch 2024-03-04T13:44:30.318 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:29 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi107.wpzgwl"}]': finished 2024-03-04T13:44:30.318 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:29 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi169.eihczr"}]: dispatch 2024-03-04T13:44:30.318 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:29 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi169.eihczr"}]': finished 2024-03-04T13:44:30.318 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:29 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:44:30.318 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:29 smithi107 ceph-mon[28739]: Upgrade: Setting container_image for all rgw 2024-03-04T13:44:30.318 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:29 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:44:30.318 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:29 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:44:30.318 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:29 smithi107 ceph-mon[28739]: Upgrade: Setting container_image for all rbd-mirror 2024-03-04T13:44:30.318 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:29 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:44:30.318 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:29 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:44:30.318 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:29 smithi107 ceph-mon[28739]: Upgrade: Setting container_image for all cephfs-mirror 2024-03-04T13:44:30.319 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:29 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:44:30.319 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:29 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:44:30.319 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:29 smithi107 ceph-mon[28739]: Upgrade: Setting container_image for all ceph-exporter 2024-03-04T13:44:30.319 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:29 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:44:30.319 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:29 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:44:30.319 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:29 smithi107 ceph-mon[28739]: Upgrade: Setting container_image for all iscsi 2024-03-04T13:44:30.319 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:29 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:44:30.319 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:29 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:44:30.319 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:29 smithi107 ceph-mon[28739]: Upgrade: Setting container_image for all nfs 2024-03-04T13:44:30.319 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:29 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:44:30.319 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:29 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:44:30.319 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:29 smithi107 ceph-mon[28739]: Upgrade: Setting container_image for all nvmeof 2024-03-04T13:44:30.319 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:29 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:44:30.359 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:29 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:44:30.359 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:29 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:44:30.360 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:29 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:44:30.360 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:29 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:44:30.360 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:29 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:44:30.360 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:29 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:44:30.360 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:29 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:44:30.360 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:29 smithi169 ceph-mon[36313]: Upgrade: Setting container_image for all mgr 2024-03-04T13:44:30.360 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:29 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:44:30.360 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:29 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi107.wpzgwl"}]: dispatch 2024-03-04T13:44:30.360 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:29 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi107.wpzgwl"}]': finished 2024-03-04T13:44:30.360 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:29 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi169.eihczr"}]: dispatch 2024-03-04T13:44:30.360 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:29 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi169.eihczr"}]': finished 2024-03-04T13:44:30.360 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:29 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:44:30.360 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:29 smithi169 ceph-mon[36313]: Upgrade: Setting container_image for all rgw 2024-03-04T13:44:30.360 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:29 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:44:30.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:29 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:44:30.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:29 smithi169 ceph-mon[36313]: Upgrade: Setting container_image for all rbd-mirror 2024-03-04T13:44:30.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:29 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:44:30.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:29 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:44:30.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:29 smithi169 ceph-mon[36313]: Upgrade: Setting container_image for all cephfs-mirror 2024-03-04T13:44:30.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:29 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:44:30.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:29 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:44:30.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:29 smithi169 ceph-mon[36313]: Upgrade: Setting container_image for all ceph-exporter 2024-03-04T13:44:30.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:29 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:44:30.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:29 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:44:30.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:29 smithi169 ceph-mon[36313]: Upgrade: Setting container_image for all iscsi 2024-03-04T13:44:30.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:29 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:44:30.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:29 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:44:30.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:29 smithi169 ceph-mon[36313]: Upgrade: Setting container_image for all nfs 2024-03-04T13:44:30.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:29 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:44:30.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:29 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:44:30.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:29 smithi169 ceph-mon[36313]: Upgrade: Setting container_image for all nvmeof 2024-03-04T13:44:30.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:29 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:44:30.918 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:30 smithi107 ceph-mon[28739]: pgmap v29: 65 pgs: 65 active+clean; 307 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 125 op/s 2024-03-04T13:44:31.358 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:30 smithi169 ceph-mon[36313]: pgmap v29: 65 pgs: 65 active+clean; 307 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 125 op/s 2024-03-04T13:44:33.341 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:33 smithi107 ceph-mon[28739]: Upgrade: Updating node-exporter.smithi107 (1/2) 2024-03-04T13:44:33.341 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:33 smithi107 ceph-mon[28739]: Deploying daemon node-exporter.smithi107 on smithi107 2024-03-04T13:44:33.341 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:33 smithi107 ceph-mon[28739]: pgmap v30: 65 pgs: 65 active+clean; 307 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 83 op/s 2024-03-04T13:44:33.608 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:33 smithi169 ceph-mon[36313]: Upgrade: Updating node-exporter.smithi107 (1/2) 2024-03-04T13:44:33.608 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:33 smithi169 ceph-mon[36313]: Deploying daemon node-exporter.smithi107 on smithi107 2024-03-04T13:44:33.608 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:33 smithi169 ceph-mon[36313]: pgmap v30: 65 pgs: 65 active+clean; 307 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 83 op/s 2024-03-04T13:44:35.591 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:35 smithi107 ceph-mon[28739]: pgmap v31: 65 pgs: 65 active+clean; 307 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 122 op/s 2024-03-04T13:44:35.608 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:35 smithi169 ceph-mon[36313]: pgmap v31: 65 pgs: 65 active+clean; 307 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 122 op/s 2024-03-04T13:44:37.341 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:37 smithi107 ceph-mon[28739]: pgmap v32: 65 pgs: 65 active+clean; 307 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 90 op/s 2024-03-04T13:44:37.341 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:37 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:44:37.341 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:37 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:44:37.608 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:37 smithi169 ceph-mon[36313]: pgmap v32: 65 pgs: 65 active+clean; 307 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 90 op/s 2024-03-04T13:44:37.609 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:37 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:44:37.609 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:37 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:44:38.591 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:38 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:44:38.608 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:38 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:44:39.591 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:39 smithi107 ceph-mon[28739]: pgmap v33: 65 pgs: 65 active+clean; 307 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 86 op/s 2024-03-04T13:44:39.608 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:39 smithi169 ceph-mon[36313]: pgmap v33: 65 pgs: 65 active+clean; 307 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 86 op/s 2024-03-04T13:44:41.494 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:41 smithi169 ceph-mon[36313]: pgmap v34: 65 pgs: 65 active+clean; 304 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 186 op/s 2024-03-04T13:44:41.591 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:41 smithi107 ceph-mon[28739]: pgmap v34: 65 pgs: 65 active+clean; 304 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 186 op/s 2024-03-04T13:44:42.591 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:42 smithi107 ceph-mon[28739]: Upgrade: Updating node-exporter.smithi169 (2/2) 2024-03-04T13:44:42.591 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:42 smithi107 ceph-mon[28739]: Deploying daemon node-exporter.smithi169 on smithi169 2024-03-04T13:44:42.610 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:42 smithi169 ceph-mon[36313]: Upgrade: Updating node-exporter.smithi169 (2/2) 2024-03-04T13:44:42.611 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:42 smithi169 ceph-mon[36313]: Deploying daemon node-exporter.smithi169 on smithi169 2024-03-04T13:44:43.591 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:43 smithi107 ceph-mon[28739]: pgmap v35: 65 pgs: 65 active+clean; 304 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 982 KiB/s rd, 1.0 MiB/s wr, 138 op/s 2024-03-04T13:44:43.608 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:43 smithi169 ceph-mon[36313]: pgmap v35: 65 pgs: 65 active+clean; 304 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 982 KiB/s rd, 1.0 MiB/s wr, 138 op/s 2024-03-04T13:44:44.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:44.373+0000 7f4847a00700 1 -- 172.21.15.107:0/1463737273 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f48400ff150 msgr2=0x7f48400ff560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:44:44.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:44.373+0000 7f4847a00700 1 --2- 172.21.15.107:0/1463737273 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f48400ff150 0x7f48400ff560 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f4834004940 tx=0x7f4834009e30 comp rx=0 tx=0).stop 2024-03-04T13:44:44.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:44.374+0000 7f4847a00700 1 -- 172.21.15.107:0/1463737273 shutdown_connections 2024-03-04T13:44:44.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:44.374+0000 7f4847a00700 1 --2- 172.21.15.107:0/1463737273 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4840100360 0x7f48401007b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:44:44.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:44.374+0000 7f4847a00700 1 --2- 172.21.15.107:0/1463737273 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f48400ff150 0x7f48400ff560 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:44:44.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:44.374+0000 7f4847a00700 1 -- 172.21.15.107:0/1463737273 >> 172.21.15.107:0/1463737273 conn(0x7f48400fa920 msgr2=0x7f48400fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:44:44.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:44.374+0000 7f4847a00700 1 -- 172.21.15.107:0/1463737273 shutdown_connections 2024-03-04T13:44:44.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:44.374+0000 7f4847a00700 1 -- 172.21.15.107:0/1463737273 wait complete. 2024-03-04T13:44:44.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:44.374+0000 7f4847a00700 1 Processor -- start 2024-03-04T13:44:44.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:44.375+0000 7f4847a00700 1 -- start start 2024-03-04T13:44:44.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:44.375+0000 7f4847a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f48400ff150 0x7f484010fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:44:44.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:44.375+0000 7f4847a00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4840100360 0x7f48401100d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:44:44.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:44.375+0000 7f4847a00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48401106d0 con 0x7f48400ff150 2024-03-04T13:44:44.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:44.375+0000 7f4847a00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4840110f70 con 0x7f4840100360 2024-03-04T13:44:44.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:44.375+0000 7f4845000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f48400ff150 0x7f484010fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:44:44.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:44.375+0000 7f4845000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f48400ff150 0x7f484010fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48792/0 (socket says 172.21.15.107:48792) 2024-03-04T13:44:44.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:44.375+0000 7f4845000700 1 -- 172.21.15.107:0/2663765298 learned_addr learned my addr 172.21.15.107:0/2663765298 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:44:44.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:44.375+0000 7f483fe00700 1 --2- 172.21.15.107:0/2663765298 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4840100360 0x7f48401100d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:44:44.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:44.376+0000 7f4845000700 1 -- 172.21.15.107:0/2663765298 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4840100360 msgr2=0x7f48401100d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:44:44.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:44.376+0000 7f4845000700 1 --2- 172.21.15.107:0/2663765298 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4840100360 0x7f48401100d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:44:44.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:44.376+0000 7f4845000700 1 -- 172.21.15.107:0/2663765298 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4834005040 con 0x7f48400ff150 2024-03-04T13:44:44.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:44.376+0000 7f4845000700 1 --2- 172.21.15.107:0/2663765298 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f48400ff150 0x7f484010fb90 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f4834009fd0 tx=0x7f4834009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:44:44.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:44.377+0000 7f483cc00700 1 -- 172.21.15.107:0/2663765298 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4834021070 con 0x7f48400ff150 2024-03-04T13:44:44.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:44.377+0000 7f483cc00700 1 -- 172.21.15.107:0/2663765298 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4834013de0 con 0x7f48400ff150 2024-03-04T13:44:44.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:44.377+0000 7f4847a00700 1 -- 172.21.15.107:0/2663765298 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48401111f0 con 0x7f48400ff150 2024-03-04T13:44:44.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:44.377+0000 7f483cc00700 1 -- 172.21.15.107:0/2663765298 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f483400e070 con 0x7f48400ff150 2024-03-04T13:44:44.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:44.377+0000 7f4847a00700 1 -- 172.21.15.107:0/2663765298 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48401116d0 con 0x7f48400ff150 2024-03-04T13:44:44.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:44.378+0000 7f4847a00700 1 -- 172.21.15.107:0/2663765298 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f484004bed0 con 0x7f48400ff150 2024-03-04T13:44:44.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:44.379+0000 7f483cc00700 1 -- 172.21.15.107:0/2663765298 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 32) v1 ==== 91301+0+0 (secure 0 0 0) 0x7f4834013910 con 0x7f48400ff150 2024-03-04T13:44:44.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:44.380+0000 7f483cc00700 1 --2- 172.21.15.107:0/2663765298 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f482806d200 0x7f482806f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:44:44.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:44.381+0000 7f483cc00700 1 -- 172.21.15.107:0/2663765298 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f483408da40 con 0x7f48400ff150 2024-03-04T13:44:44.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:44.381+0000 7f483fe00700 1 --2- 172.21.15.107:0/2663765298 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f482806d200 0x7f482806f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:44:44.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:44.381+0000 7f483fe00700 1 --2- 172.21.15.107:0/2663765298 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f482806d200 0x7f482806f6b0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f482c009b80 tx=0x7f482c00b040 comp rx=0 tx=0).ready entity=mgr.14696 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:44:44.384 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:44.386+0000 7f483cc00700 1 -- 172.21.15.107:0/2663765298 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f483405b3d0 con 0x7f48400ff150 2024-03-04T13:44:44.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:44.641+0000 7f4847a00700 1 -- 172.21.15.107:0/2663765298 --> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4840111ac0 con 0x7f482806d200 2024-03-04T13:44:44.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:44.643+0000 7f483cc00700 1 -- 172.21.15.107:0/2663765298 <== mgr.14696 v2:172.21.15.107:6800/1613321046 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+370 (secure 0 0 0) 0x7f4840111ac0 con 0x7f482806d200 2024-03-04T13:44:44.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:44.646+0000 7f4847a00700 1 -- 172.21.15.107:0/2663765298 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f482806d200 msgr2=0x7f482806f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:44:44.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:44.646+0000 7f4847a00700 1 --2- 172.21.15.107:0/2663765298 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f482806d200 0x7f482806f6b0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f482c009b80 tx=0x7f482c00b040 comp rx=0 tx=0).stop 2024-03-04T13:44:44.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:44.646+0000 7f4847a00700 1 -- 172.21.15.107:0/2663765298 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f48400ff150 msgr2=0x7f484010fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:44:44.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:44.646+0000 7f4847a00700 1 --2- 172.21.15.107:0/2663765298 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f48400ff150 0x7f484010fb90 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f4834009fd0 tx=0x7f4834009e30 comp rx=0 tx=0).stop 2024-03-04T13:44:44.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:44.646+0000 7f4847a00700 1 -- 172.21.15.107:0/2663765298 shutdown_connections 2024-03-04T13:44:44.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:44.646+0000 7f4847a00700 1 --2- 172.21.15.107:0/2663765298 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f482806d200 0x7f482806f6b0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:44:44.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:44.646+0000 7f4847a00700 1 --2- 172.21.15.107:0/2663765298 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f48400ff150 0x7f484010fb90 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:44:44.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:44.646+0000 7f4847a00700 1 --2- 172.21.15.107:0/2663765298 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4840100360 0x7f48401100d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:44:44.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:44.646+0000 7f4847a00700 1 -- 172.21.15.107:0/2663765298 >> 172.21.15.107:0/2663765298 conn(0x7f48400fa920 msgr2=0x7f4840103590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:44:44.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:44.646+0000 7f4847a00700 1 -- 172.21.15.107:0/2663765298 shutdown_connections 2024-03-04T13:44:44.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:44.647+0000 7f4847a00700 1 -- 172.21.15.107:0/2663765298 wait complete. 2024-03-04T13:44:44.660 INFO:teuthology.orchestra.run.smithi107.stdout:true 2024-03-04T13:44:44.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:44.799+0000 7f2fff200700 1 -- 172.21.15.107:0/144182101 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3000071f10 msgr2=0x7f3000070380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:44:44.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:44.799+0000 7f2fff200700 1 --2- 172.21.15.107:0/144182101 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3000071f10 0x7f3000070380 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f2ff0004940 tx=0x7f2ff0009e30 comp rx=0 tx=0).stop 2024-03-04T13:44:44.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:44.799+0000 7f2fff200700 1 -- 172.21.15.107:0/144182101 shutdown_connections 2024-03-04T13:44:44.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:44.799+0000 7f2fff200700 1 --2- 172.21.15.107:0/144182101 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30000708c0 0x7f3000070d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:44:44.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:44.799+0000 7f2fff200700 1 --2- 172.21.15.107:0/144182101 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3000071f10 0x7f3000070380 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:44:44.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:44.799+0000 7f2fff200700 1 -- 172.21.15.107:0/144182101 >> 172.21.15.107:0/144182101 conn(0x7f30000f83e0 msgr2=0x7f30000fa830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:44:44.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:44.799+0000 7f2fff200700 1 -- 172.21.15.107:0/144182101 shutdown_connections 2024-03-04T13:44:44.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:44.799+0000 7f2fff200700 1 -- 172.21.15.107:0/144182101 wait complete. 2024-03-04T13:44:44.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:44.800+0000 7f2fff200700 1 Processor -- start 2024-03-04T13:44:44.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:44.800+0000 7f2fff200700 1 -- start start 2024-03-04T13:44:44.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:44.800+0000 7f2fff200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f30000708c0 0x7f300010f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:44:44.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:44.800+0000 7f2fff200700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3000071f10 0x7f300010fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:44:44.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:44.800+0000 7f2fff200700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3000110490 con 0x7f30000708c0 2024-03-04T13:44:44.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:44.800+0000 7f2fff200700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3000110d30 con 0x7f3000071f10 2024-03-04T13:44:44.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:44.801+0000 7f2ff5e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f30000708c0 0x7f300010f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:44:44.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:44.801+0000 7f2ff5e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f30000708c0 0x7f300010f950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48810/0 (socket says 172.21.15.107:48810) 2024-03-04T13:44:44.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:44.801+0000 7f2ff5e00700 1 -- 172.21.15.107:0/3368366518 learned_addr learned my addr 172.21.15.107:0/3368366518 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:44:44.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:44.801+0000 7f2ffde00700 1 --2- 172.21.15.107:0/3368366518 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3000071f10 0x7f300010fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:44:44.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:44.801+0000 7f2ff5e00700 1 -- 172.21.15.107:0/3368366518 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3000071f10 msgr2=0x7f300010fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:44:44.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:44.801+0000 7f2ff5e00700 1 --2- 172.21.15.107:0/3368366518 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3000071f10 0x7f300010fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:44:44.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:44.801+0000 7f2ff5e00700 1 -- 172.21.15.107:0/3368366518 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ff0005040 con 0x7f30000708c0 2024-03-04T13:44:44.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:44.801+0000 7f2ffde00700 1 --2- 172.21.15.107:0/3368366518 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3000071f10 0x7f300010fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T13:44:44.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:44.801+0000 7f2ff5e00700 1 --2- 172.21.15.107:0/3368366518 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f30000708c0 0x7f300010f950 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f2ff0009fd0 tx=0x7f2ff0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:44:44.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:44.802+0000 7f2ff6a00700 1 -- 172.21.15.107:0/3368366518 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2ff000e040 con 0x7f30000708c0 2024-03-04T13:44:44.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:44.802+0000 7f2ff6a00700 1 -- 172.21.15.107:0/3368366518 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2ff0004290 con 0x7f30000708c0 2024-03-04T13:44:44.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:44.802+0000 7f2ff6a00700 1 -- 172.21.15.107:0/3368366518 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2ff001c630 con 0x7f30000708c0 2024-03-04T13:44:44.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:44.802+0000 7f2fff200700 1 -- 172.21.15.107:0/3368366518 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3000110fb0 con 0x7f30000708c0 2024-03-04T13:44:44.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:44.802+0000 7f2fff200700 1 -- 172.21.15.107:0/3368366518 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30001114a0 con 0x7f30000708c0 2024-03-04T13:44:44.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:44.803+0000 7f2fff200700 1 -- 172.21.15.107:0/3368366518 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f300004bed0 con 0x7f30000708c0 2024-03-04T13:44:44.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:44.804+0000 7f2ff6a00700 1 -- 172.21.15.107:0/3368366518 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 32) v1 ==== 91301+0+0 (secure 0 0 0) 0x7f2ff000bc60 con 0x7f30000708c0 2024-03-04T13:44:44.804 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:44.805+0000 7f2ff6a00700 1 --2- 172.21.15.107:0/3368366518 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f2fe406d200 0x7f2fe406f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:44:44.804 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:44.805+0000 7f2ff6a00700 1 -- 172.21.15.107:0/3368366518 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f2ff008e770 con 0x7f30000708c0 2024-03-04T13:44:44.804 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:44.806+0000 7f2ffde00700 1 --2- 172.21.15.107:0/3368366518 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f2fe406d200 0x7f2fe406f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:44:44.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:44.806+0000 7f2ffde00700 1 --2- 172.21.15.107:0/3368366518 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f2fe406d200 0x7f2fe406f6b0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f2fe8009fd0 tx=0x7f2fe8009ee0 comp rx=0 tx=0).ready entity=mgr.14696 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:44:44.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:44.809+0000 7f2ff6a00700 1 -- 172.21.15.107:0/3368366518 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f2ff005c100 con 0x7f30000708c0 2024-03-04T13:44:45.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:45.138+0000 7f2fff200700 1 -- 172.21.15.107:0/3368366518 --> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3000111b50 con 0x7f2fe406d200 2024-03-04T13:44:45.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:45.140+0000 7f2ff6a00700 1 -- 172.21.15.107:0/3368366518 <== mgr.14696 v2:172.21.15.107:6800/1613321046 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+370 (secure 0 0 0) 0x7f3000111b50 con 0x7f2fe406d200 2024-03-04T13:44:45.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:45.142+0000 7f2fff200700 1 -- 172.21.15.107:0/3368366518 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f2fe406d200 msgr2=0x7f2fe406f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:44:45.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:45.142+0000 7f2fff200700 1 --2- 172.21.15.107:0/3368366518 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f2fe406d200 0x7f2fe406f6b0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f2fe8009fd0 tx=0x7f2fe8009ee0 comp rx=0 tx=0).stop 2024-03-04T13:44:45.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:45.142+0000 7f2fff200700 1 -- 172.21.15.107:0/3368366518 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f30000708c0 msgr2=0x7f300010f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:44:45.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:45.142+0000 7f2fff200700 1 --2- 172.21.15.107:0/3368366518 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f30000708c0 0x7f300010f950 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f2ff0009fd0 tx=0x7f2ff0009e30 comp rx=0 tx=0).stop 2024-03-04T13:44:45.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:45.142+0000 7f2fff200700 1 -- 172.21.15.107:0/3368366518 shutdown_connections 2024-03-04T13:44:45.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:45.142+0000 7f2fff200700 1 --2- 172.21.15.107:0/3368366518 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f2fe406d200 0x7f2fe406f6b0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:44:45.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:45.142+0000 7f2fff200700 1 --2- 172.21.15.107:0/3368366518 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f30000708c0 0x7f300010f950 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:44:45.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:45.142+0000 7f2fff200700 1 --2- 172.21.15.107:0/3368366518 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3000071f10 0x7f300010fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:44:45.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:45.142+0000 7f2fff200700 1 -- 172.21.15.107:0/3368366518 >> 172.21.15.107:0/3368366518 conn(0x7f30000f83e0 msgr2=0x7f3000102bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:44:45.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:45.142+0000 7f2fff200700 1 -- 172.21.15.107:0/3368366518 shutdown_connections 2024-03-04T13:44:45.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:45.142+0000 7f2fff200700 1 -- 172.21.15.107:0/3368366518 wait complete. 2024-03-04T13:44:45.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:45.296+0000 7f674ee00700 1 -- 172.21.15.107:0/7971912 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6748071e90 msgr2=0x7f6748070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:44:45.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:45.296+0000 7f674ee00700 1 --2- 172.21.15.107:0/7971912 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6748071e90 0x7f6748070300 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f673c004940 tx=0x7f673c009e30 comp rx=0 tx=0).stop 2024-03-04T13:44:45.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:45.296+0000 7f674ee00700 1 -- 172.21.15.107:0/7971912 shutdown_connections 2024-03-04T13:44:45.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:45.296+0000 7f674ee00700 1 --2- 172.21.15.107:0/7971912 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6748070840 0x7f6748070cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:44:45.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:45.296+0000 7f674ee00700 1 --2- 172.21.15.107:0/7971912 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6748071e90 0x7f6748070300 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:44:45.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:45.296+0000 7f674ee00700 1 -- 172.21.15.107:0/7971912 >> 172.21.15.107:0/7971912 conn(0x7f67480f8440 msgr2=0x7f67480fa890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:44:45.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:45.296+0000 7f674ee00700 1 -- 172.21.15.107:0/7971912 shutdown_connections 2024-03-04T13:44:45.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:45.297+0000 7f674ee00700 1 -- 172.21.15.107:0/7971912 wait complete. 2024-03-04T13:44:45.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:45.297+0000 7f674ee00700 1 Processor -- start 2024-03-04T13:44:45.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:45.298+0000 7f674ee00700 1 -- start start 2024-03-04T13:44:45.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:45.298+0000 7f674ee00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6748070840 0x7f674810f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:44:45.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:45.298+0000 7f674ee00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6748071e90 0x7f674810fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:44:45.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:45.298+0000 7f674ee00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67481103f0 con 0x7f6748070840 2024-03-04T13:44:45.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:45.298+0000 7f674ee00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6748110c90 con 0x7f6748071e90 2024-03-04T13:44:45.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:45.298+0000 7f674da00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6748070840 0x7f674810f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:44:45.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:45.298+0000 7f674da00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6748070840 0x7f674810f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48816/0 (socket says 172.21.15.107:48816) 2024-03-04T13:44:45.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:45.298+0000 7f674da00700 1 -- 172.21.15.107:0/1340158241 learned_addr learned my addr 172.21.15.107:0/1340158241 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:44:45.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:45.298+0000 7f674d000700 1 --2- 172.21.15.107:0/1340158241 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6748071e90 0x7f674810fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:44:45.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:45.299+0000 7f674da00700 1 -- 172.21.15.107:0/1340158241 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6748071e90 msgr2=0x7f674810fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:44:45.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:45.299+0000 7f674da00700 1 --2- 172.21.15.107:0/1340158241 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6748071e90 0x7f674810fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:44:45.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:45.299+0000 7f674da00700 1 -- 172.21.15.107:0/1340158241 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f673c005040 con 0x7f6748070840 2024-03-04T13:44:45.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:45.299+0000 7f674da00700 1 --2- 172.21.15.107:0/1340158241 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6748070840 0x7f674810f8b0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f673c009fd0 tx=0x7f673c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:44:45.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:45.299+0000 7f6745600700 1 -- 172.21.15.107:0/1340158241 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f673c021070 con 0x7f6748070840 2024-03-04T13:44:45.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:45.299+0000 7f6745600700 1 -- 172.21.15.107:0/1340158241 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f673c013de0 con 0x7f6748070840 2024-03-04T13:44:45.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:45.299+0000 7f674ee00700 1 -- 172.21.15.107:0/1340158241 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6748110f10 con 0x7f6748070840 2024-03-04T13:44:45.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:45.299+0000 7f6745600700 1 -- 172.21.15.107:0/1340158241 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f673c00e070 con 0x7f6748070840 2024-03-04T13:44:45.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:45.300+0000 7f674ee00700 1 -- 172.21.15.107:0/1340158241 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6748111450 con 0x7f6748070840 2024-03-04T13:44:45.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:45.302+0000 7f6745600700 1 -- 172.21.15.107:0/1340158241 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 32) v1 ==== 91301+0+0 (secure 0 0 0) 0x7f673c026410 con 0x7f6748070840 2024-03-04T13:44:45.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:45.302+0000 7f674ee00700 1 -- 172.21.15.107:0/1340158241 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f674804bed0 con 0x7f6748070840 2024-03-04T13:44:45.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:45.303+0000 7f6745600700 1 --2- 172.21.15.107:0/1340158241 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f673006d200 0x7f673006f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:44:45.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:45.303+0000 7f6745600700 1 -- 172.21.15.107:0/1340158241 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f673c013910 con 0x7f6748070840 2024-03-04T13:44:45.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:45.304+0000 7f674d000700 1 --2- 172.21.15.107:0/1340158241 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f673006d200 0x7f673006f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:44:45.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:45.305+0000 7f674d000700 1 --2- 172.21.15.107:0/1340158241 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f673006d200 0x7f673006f6b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f6734009b80 tx=0x7f673400b040 comp rx=0 tx=0).ready entity=mgr.14696 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:44:45.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:45.308+0000 7f6745600700 1 -- 172.21.15.107:0/1340158241 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f673c05c480 con 0x7f6748070840 2024-03-04T13:44:45.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:45.557+0000 7f674ee00700 1 -- 172.21.15.107:0/1340158241 --> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f67481043d0 con 0x7f673006d200 2024-03-04T13:44:45.556 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:45 smithi107 ceph-mon[28739]: pgmap v36: 65 pgs: 65 active+clean; 304 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 214 op/s 2024-03-04T13:44:45.556 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:45 smithi107 ceph-mon[28739]: from='client.14764 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:44:45.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:45.575+0000 7f6745600700 1 -- 172.21.15.107:0/1340158241 <== mgr.14696 v2:172.21.15.107:6800/1613321046 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f67481043d0 con 0x7f673006d200 2024-03-04T13:44:45.574 INFO:teuthology.orchestra.run.smithi107.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-04T13:44:45.574 INFO:teuthology.orchestra.run.smithi107.stdout:alertmanager.smithi107 smithi107 *:9093,9094 running (2m) 21s ago 13m 15.7M - 0.25.0 c8568f914cd2 ac0e4f11c97a 2024-03-04T13:44:45.575 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi107 smithi107 running (13m) 21s ago 13m 6665k - 17.2.7-545-g42608bb1 9eec3f22859d 03e1570c2eff 2024-03-04T13:44:45.575 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi169 smithi169 running (12m) 25s ago 12m 6656k - 17.2.7-545-g42608bb1 9eec3f22859d 9c8881a5338e 2024-03-04T13:44:45.575 INFO:teuthology.orchestra.run.smithi107.stdout:grafana.smithi107 smithi107 *:3000 running (2m) 21s ago 12m 74.7M - 9.4.7 954c08fa6188 bd720c203473 2024-03-04T13:44:45.575 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.riexmm smithi107 running (6m) 21s ago 6m 12.7M - 17.2.7-545-g42608bb1 9eec3f22859d fc921f7d4328 2024-03-04T13:44:45.575 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.thycho smithi107 running (6m) 21s ago 6m 278M - 17.2.7-545-g42608bb1 9eec3f22859d 4069c6e0e063 2024-03-04T13:44:45.575 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi169.bxitzk smithi169 running (5m) 25s ago 5m 12.8M - 17.2.7-545-g42608bb1 9eec3f22859d 7d4115228c7c 2024-03-04T13:44:45.575 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi169.jusudl smithi169 running (6m) 25s ago 6m 240M - 17.2.7-545-g42608bb1 9eec3f22859d 2d2f2cc452e8 2024-03-04T13:44:45.575 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi107.wpzgwl smithi107 *:8443,9283,8765 running (95s) 21s ago 14m 494M - 19.0.0-1845-g717ce3cc d71b839abdf8 1c5fdee45eac 2024-03-04T13:44:45.575 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi169.eihczr smithi169 *:8443,9283,8765 running (30s) 25s ago 11m 64.9M - 19.0.0-1845-g717ce3cc d71b839abdf8 b5cac2b7d22a 2024-03-04T13:44:45.575 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi107 smithi107 running (14m) 21s ago 15m 51.6M 2048M 17.2.7-545-g42608bb1 9eec3f22859d 49f5da990d35 2024-03-04T13:44:45.575 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi169 smithi169 running (11m) 25s ago 11m 40.3M 2048M 17.2.7-545-g42608bb1 9eec3f22859d ba4ee916297f 2024-03-04T13:44:45.576 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi107 smithi107 *:9100 starting - - - - 2024-03-04T13:44:45.576 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi169 smithi169 *:9100 running (117s) 25s ago 11m 7369k - 1.5.0 0da6a335fe13 043154b90aa2 2024-03-04T13:44:45.576 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0 smithi107 running (10m) 21s ago 10m 176M 1558M 17.2.7-545-g42608bb1 9eec3f22859d c0605b68f696 2024-03-04T13:44:45.576 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1 smithi107 running (10m) 21s ago 10m 184M 1558M 17.2.7-545-g42608bb1 9eec3f22859d 5fb2c0dd5769 2024-03-04T13:44:45.576 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2 smithi107 running (9m) 21s ago 9m 154M 1558M 17.2.7-545-g42608bb1 9eec3f22859d 61f30f643549 2024-03-04T13:44:45.576 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3 smithi169 running (9m) 25s ago 9m 204M 2582M 17.2.7-545-g42608bb1 9eec3f22859d cd91d8694764 2024-03-04T13:44:45.576 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4 smithi169 running (8m) 25s ago 8m 180M 2582M 17.2.7-545-g42608bb1 9eec3f22859d 08545e3d0659 2024-03-04T13:44:45.576 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5 smithi169 running (7m) 25s ago 7m 164M 2582M 17.2.7-545-g42608bb1 9eec3f22859d dcffe6cb3eac 2024-03-04T13:44:45.576 INFO:teuthology.orchestra.run.smithi107.stdout:prometheus.smithi107 smithi107 *:9095 running (40s) 21s ago 12m 42.7M - 2.43.0 a07b618ecd1d b59efdd6b13d 2024-03-04T13:44:45.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:45.578+0000 7f674ee00700 1 -- 172.21.15.107:0/1340158241 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f673006d200 msgr2=0x7f673006f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:44:45.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:45.578+0000 7f674ee00700 1 --2- 172.21.15.107:0/1340158241 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f673006d200 0x7f673006f6b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f6734009b80 tx=0x7f673400b040 comp rx=0 tx=0).stop 2024-03-04T13:44:45.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:45.578+0000 7f674ee00700 1 -- 172.21.15.107:0/1340158241 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6748070840 msgr2=0x7f674810f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:44:45.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:45.578+0000 7f674ee00700 1 --2- 172.21.15.107:0/1340158241 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6748070840 0x7f674810f8b0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f673c009fd0 tx=0x7f673c009e30 comp rx=0 tx=0).stop 2024-03-04T13:44:45.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:45.578+0000 7f674ee00700 1 -- 172.21.15.107:0/1340158241 shutdown_connections 2024-03-04T13:44:45.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:45.578+0000 7f674ee00700 1 --2- 172.21.15.107:0/1340158241 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f673006d200 0x7f673006f6b0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:44:45.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:45.578+0000 7f674ee00700 1 --2- 172.21.15.107:0/1340158241 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6748070840 0x7f674810f8b0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:44:45.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:45.578+0000 7f674ee00700 1 --2- 172.21.15.107:0/1340158241 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6748071e90 0x7f674810fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:44:45.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:45.578+0000 7f674ee00700 1 -- 172.21.15.107:0/1340158241 >> 172.21.15.107:0/1340158241 conn(0x7f67480f8440 msgr2=0x7f6748102cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:44:45.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:45.579+0000 7f674ee00700 1 -- 172.21.15.107:0/1340158241 shutdown_connections 2024-03-04T13:44:45.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:45.579+0000 7f674ee00700 1 -- 172.21.15.107:0/1340158241 wait complete. 2024-03-04T13:44:45.608 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:45 smithi169 ceph-mon[36313]: pgmap v36: 65 pgs: 65 active+clean; 304 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 214 op/s 2024-03-04T13:44:45.608 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:45 smithi169 ceph-mon[36313]: from='client.14764 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:44:45.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:45.736+0000 7f0c88400700 1 -- 172.21.15.107:0/2601489156 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0c800fcfe0 msgr2=0x7f0c800fd450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:44:45.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:45.736+0000 7f0c88400700 1 --2- 172.21.15.107:0/2601489156 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0c800fcfe0 0x7f0c800fd450 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c004940 tx=0x7f0c6c009e30 comp rx=0 tx=0).stop 2024-03-04T13:44:45.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:45.737+0000 7f0c88400700 1 -- 172.21.15.107:0/2601489156 shutdown_connections 2024-03-04T13:44:45.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:45.737+0000 7f0c88400700 1 --2- 172.21.15.107:0/2601489156 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0c800fcfe0 0x7f0c800fd450 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:44:45.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:45.737+0000 7f0c88400700 1 --2- 172.21.15.107:0/2601489156 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c800fbe70 0x7f0c800fc280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:44:45.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:45.737+0000 7f0c88400700 1 -- 172.21.15.107:0/2601489156 >> 172.21.15.107:0/2601489156 conn(0x7f0c800f7530 msgr2=0x7f0c800f9960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:44:45.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:45.737+0000 7f0c88400700 1 -- 172.21.15.107:0/2601489156 shutdown_connections 2024-03-04T13:44:45.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:45.737+0000 7f0c88400700 1 -- 172.21.15.107:0/2601489156 wait complete. 2024-03-04T13:44:45.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:45.738+0000 7f0c88400700 1 Processor -- start 2024-03-04T13:44:45.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:45.738+0000 7f0c88400700 1 -- start start 2024-03-04T13:44:45.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:45.739+0000 7f0c88400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0c800fbe70 0x7f0c8010f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:44:45.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:45.739+0000 7f0c88400700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c800fcfe0 0x7f0c8010ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:44:45.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:45.739+0000 7f0c88400700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c80110510 con 0x7f0c800fbe70 2024-03-04T13:44:45.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:45.739+0000 7f0c88400700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c80110db0 con 0x7f0c800fcfe0 2024-03-04T13:44:45.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:45.739+0000 7f0c85a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0c800fbe70 0x7f0c8010f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:44:45.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:45.739+0000 7f0c85a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0c800fbe70 0x7f0c8010f9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48824/0 (socket says 172.21.15.107:48824) 2024-03-04T13:44:45.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:45.739+0000 7f0c85a00700 1 -- 172.21.15.107:0/752074199 learned_addr learned my addr 172.21.15.107:0/752074199 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:44:45.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:45.739+0000 7f0c85000700 1 --2- 172.21.15.107:0/752074199 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c800fcfe0 0x7f0c8010ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:44:45.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:45.739+0000 7f0c85a00700 1 -- 172.21.15.107:0/752074199 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c800fcfe0 msgr2=0x7f0c8010ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:44:45.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:45.739+0000 7f0c85a00700 1 --2- 172.21.15.107:0/752074199 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c800fcfe0 0x7f0c8010ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:44:45.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:45.739+0000 7f0c85a00700 1 -- 172.21.15.107:0/752074199 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c6c005040 con 0x7f0c800fbe70 2024-03-04T13:44:45.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:45.739+0000 7f0c85000700 1 --2- 172.21.15.107:0/752074199 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c800fcfe0 0x7f0c8010ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T13:44:45.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:45.740+0000 7f0c85a00700 1 --2- 172.21.15.107:0/752074199 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0c800fbe70 0x7f0c8010f9d0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f0c7400eb40 tx=0x7f0c7400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:44:45.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:45.740+0000 7f0c7d600700 1 -- 172.21.15.107:0/752074199 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c74007cc0 con 0x7f0c800fbe70 2024-03-04T13:44:45.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:45.740+0000 7f0c7d600700 1 -- 172.21.15.107:0/752074199 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0c7400f040 con 0x7f0c800fbe70 2024-03-04T13:44:45.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:45.740+0000 7f0c7d600700 1 -- 172.21.15.107:0/752074199 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c74009620 con 0x7f0c800fbe70 2024-03-04T13:44:45.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:45.740+0000 7f0c88400700 1 -- 172.21.15.107:0/752074199 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c80111030 con 0x7f0c800fbe70 2024-03-04T13:44:45.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:45.740+0000 7f0c88400700 1 -- 172.21.15.107:0/752074199 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c80111570 con 0x7f0c800fbe70 2024-03-04T13:44:45.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:45.742+0000 7f0c88400700 1 -- 172.21.15.107:0/752074199 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c8004bed0 con 0x7f0c800fbe70 2024-03-04T13:44:45.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:45.743+0000 7f0c7d600700 1 -- 172.21.15.107:0/752074199 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 32) v1 ==== 91301+0+0 (secure 0 0 0) 0x7f0c74009780 con 0x7f0c800fbe70 2024-03-04T13:44:45.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:45.744+0000 7f0c7d600700 1 --2- 172.21.15.107:0/752074199 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f0c6806d200 0x7f0c6806f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:44:45.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:45.744+0000 7f0c7d600700 1 -- 172.21.15.107:0/752074199 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f0c7408c3c0 con 0x7f0c800fbe70 2024-03-04T13:44:45.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:45.745+0000 7f0c85000700 1 --2- 172.21.15.107:0/752074199 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f0c6806d200 0x7f0c6806f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:44:45.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:45.746+0000 7f0c85000700 1 --2- 172.21.15.107:0/752074199 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f0c6806d200 0x7f0c6806f6b0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c004560 tx=0x7f0c6c009e30 comp rx=0 tx=0).ready entity=mgr.14696 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:44:45.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:45.750+0000 7f0c7d600700 1 -- 172.21.15.107:0/752074199 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f0c74059ea0 con 0x7f0c800fbe70 2024-03-04T13:44:46.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:46.161+0000 7f0c88400700 1 -- 172.21.15.107:0/752074199 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f0c801117e0 con 0x7f0c800fbe70 2024-03-04T13:44:46.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:46.162+0000 7f0c7d600700 1 -- 172.21.15.107:0/752074199 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7f0c74059a30 con 0x7f0c800fbe70 2024-03-04T13:44:46.161 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-03-04T13:44:46.161 INFO:teuthology.orchestra.run.smithi107.stdout: "mon": { 2024-03-04T13:44:46.161 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-545-g42608bb1 (42608bb191117be10dba720e6bf8a02f3171cd34) quincy (stable)": 2 2024-03-04T13:44:46.161 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:44:46.162 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr": { 2024-03-04T13:44:46.162 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-1845-g717ce3cc (717ce3cce8d0166e4e577faa8004e6f9cb4128c0) squid (dev)": 2 2024-03-04T13:44:46.162 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:44:46.162 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": { 2024-03-04T13:44:46.162 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-545-g42608bb1 (42608bb191117be10dba720e6bf8a02f3171cd34) quincy (stable)": 6 2024-03-04T13:44:46.162 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:44:46.162 INFO:teuthology.orchestra.run.smithi107.stdout: "mds": { 2024-03-04T13:44:46.162 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-545-g42608bb1 (42608bb191117be10dba720e6bf8a02f3171cd34) quincy (stable)": 4 2024-03-04T13:44:46.162 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:44:46.162 INFO:teuthology.orchestra.run.smithi107.stdout: "overall": { 2024-03-04T13:44:46.162 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-545-g42608bb1 (42608bb191117be10dba720e6bf8a02f3171cd34) quincy (stable)": 12, 2024-03-04T13:44:46.162 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-1845-g717ce3cc (717ce3cce8d0166e4e577faa8004e6f9cb4128c0) squid (dev)": 2 2024-03-04T13:44:46.162 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-04T13:44:46.162 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-03-04T13:44:46.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:46.165+0000 7f0c88400700 1 -- 172.21.15.107:0/752074199 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f0c6806d200 msgr2=0x7f0c6806f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:44:46.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:46.165+0000 7f0c88400700 1 --2- 172.21.15.107:0/752074199 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f0c6806d200 0x7f0c6806f6b0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c004560 tx=0x7f0c6c009e30 comp rx=0 tx=0).stop 2024-03-04T13:44:46.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:46.165+0000 7f0c88400700 1 -- 172.21.15.107:0/752074199 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0c800fbe70 msgr2=0x7f0c8010f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:44:46.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:46.165+0000 7f0c88400700 1 --2- 172.21.15.107:0/752074199 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0c800fbe70 0x7f0c8010f9d0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f0c7400eb40 tx=0x7f0c7400b040 comp rx=0 tx=0).stop 2024-03-04T13:44:46.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:46.165+0000 7f0c88400700 1 -- 172.21.15.107:0/752074199 shutdown_connections 2024-03-04T13:44:46.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:46.165+0000 7f0c88400700 1 --2- 172.21.15.107:0/752074199 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f0c6806d200 0x7f0c6806f6b0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:44:46.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:46.165+0000 7f0c88400700 1 --2- 172.21.15.107:0/752074199 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0c800fbe70 0x7f0c8010f9d0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:44:46.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:46.165+0000 7f0c88400700 1 --2- 172.21.15.107:0/752074199 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c800fcfe0 0x7f0c8010ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:44:46.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:46.165+0000 7f0c88400700 1 -- 172.21.15.107:0/752074199 >> 172.21.15.107:0/752074199 conn(0x7f0c800f7530 msgr2=0x7f0c800656c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:44:46.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:46.166+0000 7f0c88400700 1 -- 172.21.15.107:0/752074199 shutdown_connections 2024-03-04T13:44:46.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:46.166+0000 7f0c88400700 1 -- 172.21.15.107:0/752074199 wait complete. 2024-03-04T13:44:46.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:46.328+0000 7f605fe00700 1 -- 172.21.15.107:0/3718715294 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60600fcd50 msgr2=0x7f60600fd160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:44:46.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:46.328+0000 7f605fe00700 1 --2- 172.21.15.107:0/3718715294 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60600fcd50 0x7f60600fd160 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f6050004940 tx=0x7f6050009e30 comp rx=0 tx=0).stop 2024-03-04T13:44:46.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:46.328+0000 7f605fe00700 1 -- 172.21.15.107:0/3718715294 shutdown_connections 2024-03-04T13:44:46.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:46.328+0000 7f605fe00700 1 --2- 172.21.15.107:0/3718715294 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f60600fdf60 0x7f60600fe3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:44:46.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:46.328+0000 7f605fe00700 1 --2- 172.21.15.107:0/3718715294 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60600fcd50 0x7f60600fd160 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:44:46.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:46.328+0000 7f605fe00700 1 -- 172.21.15.107:0/3718715294 >> 172.21.15.107:0/3718715294 conn(0x7f60600f8430 msgr2=0x7f60600fa840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:44:46.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:46.328+0000 7f605fe00700 1 -- 172.21.15.107:0/3718715294 shutdown_connections 2024-03-04T13:44:46.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:46.328+0000 7f605fe00700 1 -- 172.21.15.107:0/3718715294 wait complete. 2024-03-04T13:44:46.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:46.329+0000 7f605fe00700 1 Processor -- start 2024-03-04T13:44:46.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:46.329+0000 7f605fe00700 1 -- start start 2024-03-04T13:44:46.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:46.329+0000 7f605fe00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60600fcd50 0x7f606010d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:44:46.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:46.329+0000 7f605fe00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f60600fdf60 0x7f606010dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:44:46.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:46.329+0000 7f605fe00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f606010e270 con 0x7f60600fcd50 2024-03-04T13:44:46.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:46.329+0000 7f605fe00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f606010eb10 con 0x7f60600fdf60 2024-03-04T13:44:46.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:46.330+0000 7f605ea00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60600fcd50 0x7f606010d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:44:46.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:46.330+0000 7f605ea00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60600fcd50 0x7f606010d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48848/0 (socket says 172.21.15.107:48848) 2024-03-04T13:44:46.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:46.330+0000 7f605ea00700 1 -- 172.21.15.107:0/99477995 learned_addr learned my addr 172.21.15.107:0/99477995 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:44:46.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:46.330+0000 7f605e000700 1 --2- 172.21.15.107:0/99477995 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f60600fdf60 0x7f606010dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:44:46.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:46.330+0000 7f605ea00700 1 -- 172.21.15.107:0/99477995 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f60600fdf60 msgr2=0x7f606010dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:44:46.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:46.330+0000 7f605ea00700 1 --2- 172.21.15.107:0/99477995 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f60600fdf60 0x7f606010dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:44:46.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:46.330+0000 7f605ea00700 1 -- 172.21.15.107:0/99477995 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6050005040 con 0x7f60600fcd50 2024-03-04T13:44:46.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:46.331+0000 7f605ea00700 1 --2- 172.21.15.107:0/99477995 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60600fcd50 0x7f606010d730 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f6050000c00 tx=0x7f6050009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:44:46.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:46.331+0000 7f6056a00700 1 -- 172.21.15.107:0/99477995 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f605000e040 con 0x7f60600fcd50 2024-03-04T13:44:46.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:46.331+0000 7f6056a00700 1 -- 172.21.15.107:0/99477995 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f605000bcc0 con 0x7f60600fcd50 2024-03-04T13:44:46.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:46.331+0000 7f605fe00700 1 -- 172.21.15.107:0/99477995 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f606010ed90 con 0x7f60600fcd50 2024-03-04T13:44:46.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:46.331+0000 7f6056a00700 1 -- 172.21.15.107:0/99477995 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6050013ea0 con 0x7f60600fcd50 2024-03-04T13:44:46.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:46.331+0000 7f605fe00700 1 -- 172.21.15.107:0/99477995 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f606010f2d0 con 0x7f60600fcd50 2024-03-04T13:44:46.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:46.332+0000 7f605fe00700 1 -- 172.21.15.107:0/99477995 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f606004bed0 con 0x7f60600fcd50 2024-03-04T13:44:46.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:46.334+0000 7f6056a00700 1 -- 172.21.15.107:0/99477995 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 32) v1 ==== 91301+0+0 (secure 0 0 0) 0x7f6050013420 con 0x7f60600fcd50 2024-03-04T13:44:46.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:46.335+0000 7f6056a00700 1 --2- 172.21.15.107:0/99477995 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f604406d200 0x7f604406f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:44:46.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:46.335+0000 7f6056a00700 1 -- 172.21.15.107:0/99477995 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f605008db20 con 0x7f60600fcd50 2024-03-04T13:44:46.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:46.335+0000 7f605e000700 1 --2- 172.21.15.107:0/99477995 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f604406d200 0x7f604406f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:44:46.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:46.336+0000 7f605e000700 1 --2- 172.21.15.107:0/99477995 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f604406d200 0x7f604406f6b0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f604c004990 tx=0x7f604c009ee0 comp rx=0 tx=0).ready entity=mgr.14696 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:44:46.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:46.340+0000 7f6056a00700 1 -- 172.21.15.107:0/99477995 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f605005b4b0 con 0x7f60600fcd50 2024-03-04T13:44:46.591 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:46 smithi107 ceph-mon[28739]: from='client.14768 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:44:46.591 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:46 smithi107 ceph-mon[28739]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:44:46.591 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:46 smithi107 ceph-mon[28739]: from='client.? 172.21.15.107:0/752074199' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:44:46.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:46.596+0000 7f605fe00700 1 -- 172.21.15.107:0/99477995 --> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f60601028b0 con 0x7f604406d200 2024-03-04T13:44:46.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:46.599+0000 7f6056a00700 1 -- 172.21.15.107:0/99477995 <== mgr.14696 v2:172.21.15.107:6800/1613321046 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+370 (secure 0 0 0) 0x7f60601028b0 con 0x7f604406d200 2024-03-04T13:44:46.597 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-03-04T13:44:46.598 INFO:teuthology.orchestra.run.smithi107.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377", 2024-03-04T13:44:46.598 INFO:teuthology.orchestra.run.smithi107.stdout: "in_progress": true, 2024-03-04T13:44:46.598 INFO:teuthology.orchestra.run.smithi107.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-03-04T13:44:46.598 INFO:teuthology.orchestra.run.smithi107.stdout: "services_complete": [ 2024-03-04T13:44:46.598 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr" 2024-03-04T13:44:46.598 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-04T13:44:46.598 INFO:teuthology.orchestra.run.smithi107.stdout: "progress": "2/2 daemons upgraded", 2024-03-04T13:44:46.598 INFO:teuthology.orchestra.run.smithi107.stdout: "message": "Currently upgrading node-exporter daemons", 2024-03-04T13:44:46.598 INFO:teuthology.orchestra.run.smithi107.stdout: "is_paused": false 2024-03-04T13:44:46.598 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-03-04T13:44:46.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:46.601+0000 7f605fe00700 1 -- 172.21.15.107:0/99477995 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f604406d200 msgr2=0x7f604406f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:44:46.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:46.602+0000 7f605fe00700 1 --2- 172.21.15.107:0/99477995 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f604406d200 0x7f604406f6b0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f604c004990 tx=0x7f604c009ee0 comp rx=0 tx=0).stop 2024-03-04T13:44:46.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:46.602+0000 7f605fe00700 1 -- 172.21.15.107:0/99477995 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60600fcd50 msgr2=0x7f606010d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:44:46.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:46.602+0000 7f605fe00700 1 --2- 172.21.15.107:0/99477995 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60600fcd50 0x7f606010d730 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f6050000c00 tx=0x7f6050009e30 comp rx=0 tx=0).stop 2024-03-04T13:44:46.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:46.602+0000 7f605fe00700 1 -- 172.21.15.107:0/99477995 shutdown_connections 2024-03-04T13:44:46.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:46.602+0000 7f605fe00700 1 --2- 172.21.15.107:0/99477995 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f604406d200 0x7f604406f6b0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:44:46.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:46.602+0000 7f605fe00700 1 --2- 172.21.15.107:0/99477995 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60600fcd50 0x7f606010d730 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:44:46.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:46.602+0000 7f605fe00700 1 --2- 172.21.15.107:0/99477995 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f60600fdf60 0x7f606010dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:44:46.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:46.602+0000 7f605fe00700 1 -- 172.21.15.107:0/99477995 >> 172.21.15.107:0/99477995 conn(0x7f60600f8430 msgr2=0x7f6060101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:44:46.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:46.602+0000 7f605fe00700 1 -- 172.21.15.107:0/99477995 shutdown_connections 2024-03-04T13:44:46.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:44:46.602+0000 7f605fe00700 1 -- 172.21.15.107:0/99477995 wait complete. 2024-03-04T13:44:46.608 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:46 smithi169 ceph-mon[36313]: from='client.14768 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:44:46.608 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:46 smithi169 ceph-mon[36313]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:44:46.608 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:46 smithi169 ceph-mon[36313]: from='client.? 172.21.15.107:0/752074199' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:44:47.591 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:47 smithi107 ceph-mon[28739]: pgmap v37: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 260 op/s 2024-03-04T13:44:47.591 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:47 smithi107 ceph-mon[28739]: from='client.14780 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:44:47.608 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:47 smithi169 ceph-mon[36313]: pgmap v37: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 260 op/s 2024-03-04T13:44:47.608 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:47 smithi169 ceph-mon[36313]: from='client.14780 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:44:49.091 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:48 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:44:49.091 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:48 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:44:49.091 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:48 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:44:49.091 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:48 smithi107 ceph-mon[28739]: pgmap v38: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 260 op/s 2024-03-04T13:44:49.358 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:48 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:44:49.358 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:48 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:44:49.358 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:48 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:44:49.358 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:48 smithi169 ceph-mon[36313]: pgmap v38: 65 pgs: 65 active+clean; 307 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 260 op/s 2024-03-04T13:44:51.548 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:51 smithi107 ceph-mon[28739]: pgmap v39: 65 pgs: 65 active+clean; 302 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 350 op/s 2024-03-04T13:44:51.574 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:51 smithi169 ceph-mon[36313]: pgmap v39: 65 pgs: 65 active+clean; 302 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 350 op/s 2024-03-04T13:44:53.552 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:53 smithi107 ceph-mon[28739]: pgmap v40: 65 pgs: 65 active+clean; 302 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 251 op/s 2024-03-04T13:44:53.552 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:53 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:44:53.552 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:53 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:44:53.608 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:53 smithi169 ceph-mon[36313]: pgmap v40: 65 pgs: 65 active+clean; 302 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 251 op/s 2024-03-04T13:44:53.608 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:53 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:44:53.609 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:53 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:44:54.591 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:54 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:44:54.608 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:54 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:44:55.358 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:55 smithi169 ceph-mon[36313]: pgmap v41: 65 pgs: 65 active+clean; 296 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 283 op/s 2024-03-04T13:44:55.358 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:55 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:44:55.358 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:55 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:44:55.591 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:55 smithi107 ceph-mon[28739]: pgmap v41: 65 pgs: 65 active+clean; 296 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 283 op/s 2024-03-04T13:44:55.591 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:55 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:44:55.591 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:55 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:44:56.953 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:56 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:44:56.953 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:56 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:44:57.091 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:56 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:44:57.091 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:56 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:44:57.859 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:57 smithi169 ceph-mon[36313]: pgmap v42: 65 pgs: 65 active+clean; 295 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 266 op/s 2024-03-04T13:44:58.091 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:57 smithi107 ceph-mon[28739]: pgmap v42: 65 pgs: 65 active+clean; 295 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 266 op/s 2024-03-04T13:44:59.015 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:58 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:44:59.016 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:58 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:44:59.016 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:44:58 smithi107 ceph-mon[28739]: pgmap v43: 65 pgs: 65 active+clean; 295 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 181 op/s 2024-03-04T13:44:59.108 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:58 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:44:59.108 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:58 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:44:59.108 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:44:58 smithi169 ceph-mon[36313]: pgmap v43: 65 pgs: 65 active+clean; 295 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 181 op/s 2024-03-04T13:45:01.841 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:01 smithi107 ceph-mon[28739]: pgmap v44: 65 pgs: 65 active+clean; 284 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 275 op/s 2024-03-04T13:45:01.842 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:01 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:45:01.842 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:01 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:45:01.858 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:01 smithi169 ceph-mon[36313]: pgmap v44: 65 pgs: 65 active+clean; 284 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 275 op/s 2024-03-04T13:45:01.858 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:01 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:45:01.858 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:01 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:45:03.722 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:03 smithi107 ceph-mon[28739]: pgmap v45: 65 pgs: 65 active+clean; 284 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.0 MiB/s rd, 995 KiB/s wr, 185 op/s 2024-03-04T13:45:03.858 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:03 smithi169 ceph-mon[36313]: pgmap v45: 65 pgs: 65 active+clean; 284 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.0 MiB/s rd, 995 KiB/s wr, 185 op/s 2024-03-04T13:45:04.752 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:04 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:45:04.753 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:04 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:45:04.753 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:04 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:45:04.753 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:04 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:45:04.753 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:04 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:45:04.753 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:04 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:45:04.753 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:04 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:45:04.753 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:04 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:45:04.753 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:04 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:45:04.753 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:04 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:45:04.753 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:04 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:45:04.753 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:04 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:45:04.753 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:04 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:45:04.753 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:04 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:45:04.753 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:04 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:45:04.859 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:04 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:45:04.859 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:04 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:45:04.859 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:04 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:45:04.859 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:04 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:45:04.860 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:04 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:45:04.860 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:04 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:45:04.860 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:04 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:45:04.860 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:04 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:45:04.860 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:04 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:45:04.860 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:04 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:45:04.860 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:04 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:45:04.860 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:04 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:45:04.860 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:04 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:45:04.860 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:04 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:45:04.860 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:04 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:45:05.784 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:05 smithi107 ceph-mon[28739]: pgmap v46: 65 pgs: 65 active+clean; 285 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 201 op/s 2024-03-04T13:45:05.858 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:05 smithi169 ceph-mon[36313]: pgmap v46: 65 pgs: 65 active+clean; 285 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 201 op/s 2024-03-04T13:45:07.091 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:06 smithi107 ceph-mon[28739]: pgmap v47: 65 pgs: 65 active+clean; 282 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 232 op/s 2024-03-04T13:45:07.092 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:06 smithi107 ceph-mon[28739]: Upgrade: Updating prometheus.smithi107 2024-03-04T13:45:07.092 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:06 smithi107 ceph-mon[28739]: Deploying daemon prometheus.smithi107 on smithi107 2024-03-04T13:45:07.108 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:06 smithi169 ceph-mon[36313]: pgmap v47: 65 pgs: 65 active+clean; 282 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 232 op/s 2024-03-04T13:45:07.109 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:06 smithi169 ceph-mon[36313]: Upgrade: Updating prometheus.smithi107 2024-03-04T13:45:07.109 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:06 smithi169 ceph-mon[36313]: Deploying daemon prometheus.smithi107 on smithi107 2024-03-04T13:45:09.591 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:09 smithi107 ceph-mon[28739]: pgmap v48: 65 pgs: 65 active+clean; 282 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 812 KiB/s rd, 864 KiB/s wr, 173 op/s 2024-03-04T13:45:09.592 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:09 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:45:09.608 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:09 smithi169 ceph-mon[36313]: pgmap v48: 65 pgs: 65 active+clean; 282 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 812 KiB/s rd, 864 KiB/s wr, 173 op/s 2024-03-04T13:45:09.608 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:09 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:45:11.341 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:11 smithi107 ceph-mon[28739]: pgmap v49: 65 pgs: 65 active+clean; 278 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 230 op/s 2024-03-04T13:45:11.341 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:11 smithi107 ceph-mon[28739]: mgrmap e33: smithi107.wpzgwl(active, since 92s), standbys: smithi169.eihczr 2024-03-04T13:45:11.608 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:11 smithi169 ceph-mon[36313]: pgmap v49: 65 pgs: 65 active+clean; 278 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 230 op/s 2024-03-04T13:45:11.608 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:11 smithi169 ceph-mon[36313]: mgrmap e33: smithi107.wpzgwl(active, since 92s), standbys: smithi169.eihczr 2024-03-04T13:45:13.037 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.1... 2024-03-04T13:45:13.038 DEBUG:teuthology.orchestra.run.smithi169:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.1 /home/ubuntu/cephtest/clone.client.1 2024-03-04T13:45:13.358 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:13 smithi169 ceph-mon[36313]: pgmap v50: 65 pgs: 65 active+clean; 278 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 741 KiB/s rd, 789 KiB/s wr, 136 op/s 2024-03-04T13:45:13.591 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:13 smithi107 ceph-mon[28739]: pgmap v50: 65 pgs: 65 active+clean; 278 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 741 KiB/s rd, 789 KiB/s wr, 136 op/s 2024-03-04T13:45:14.472 DEBUG:teuthology.parallel:result is None 2024-03-04T13:45:15.503 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:15 smithi107 ceph-mon[28739]: pgmap v51: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 805 KiB/s rd, 825 KiB/s wr, 148 op/s 2024-03-04T13:45:15.609 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:15 smithi169 ceph-mon[36313]: pgmap v51: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 805 KiB/s rd, 825 KiB/s wr, 148 op/s 2024-03-04T13:45:16.797 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:16.797+0000 7fccc5e00700 1 -- 172.21.15.107:0/1570760975 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fccc00fbe40 msgr2=0x7fccc00fc250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:45:16.797 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:16.797+0000 7fccc5e00700 1 --2- 172.21.15.107:0/1570760975 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fccc00fbe40 0x7fccc00fc250 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fccb0004940 tx=0x7fccb0009e30 comp rx=0 tx=0).stop 2024-03-04T13:45:16.797 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:16.797+0000 7fccc5e00700 1 -- 172.21.15.107:0/1570760975 shutdown_connections 2024-03-04T13:45:16.797 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:16.797+0000 7fccc5e00700 1 --2- 172.21.15.107:0/1570760975 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fccc00fcfb0 0x7fccc00fd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:45:16.797 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:16.797+0000 7fccc5e00700 1 --2- 172.21.15.107:0/1570760975 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fccc00fbe40 0x7fccc00fc250 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:45:16.797 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:16.797+0000 7fccc5e00700 1 -- 172.21.15.107:0/1570760975 >> 172.21.15.107:0/1570760975 conn(0x7fccc00f74e0 msgr2=0x7fccc00f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:45:16.797 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:16.797+0000 7fccc5e00700 1 -- 172.21.15.107:0/1570760975 shutdown_connections 2024-03-04T13:45:16.797 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:16.797+0000 7fccc5e00700 1 -- 172.21.15.107:0/1570760975 wait complete. 2024-03-04T13:45:16.797 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:16.798+0000 7fccc5e00700 1 Processor -- start 2024-03-04T13:45:16.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:16.799+0000 7fccc5e00700 1 -- start start 2024-03-04T13:45:16.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:16.799+0000 7fccc5e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fccc00fbe40 0x7fccc010fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:45:16.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:16.800+0000 7fccc5e00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fccc00fcfb0 0x7fccc0110020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:45:16.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:16.800+0000 7fccc5e00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccc0110620 con 0x7fccc00fbe40 2024-03-04T13:45:16.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:16.800+0000 7fccc5e00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccc0110ec0 con 0x7fccc00fcfb0 2024-03-04T13:45:16.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:16.800+0000 7fccbea00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fccc00fbe40 0x7fccc010fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:45:16.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:16.800+0000 7fccbea00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fccc00fbe40 0x7fccc010fae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38512/0 (socket says 172.21.15.107:38512) 2024-03-04T13:45:16.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:16.800+0000 7fccbea00700 1 -- 172.21.15.107:0/204115933 learned_addr learned my addr 172.21.15.107:0/204115933 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:45:16.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:16.800+0000 7fccbe000700 1 --2- 172.21.15.107:0/204115933 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fccc00fcfb0 0x7fccc0110020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:45:16.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:16.800+0000 7fccbea00700 1 -- 172.21.15.107:0/204115933 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fccc00fcfb0 msgr2=0x7fccc0110020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:45:16.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:16.800+0000 7fccbea00700 1 --2- 172.21.15.107:0/204115933 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fccc00fcfb0 0x7fccc0110020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:45:16.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:16.800+0000 7fccbea00700 1 -- 172.21.15.107:0/204115933 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccb0005040 con 0x7fccc00fbe40 2024-03-04T13:45:16.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:16.801+0000 7fccbea00700 1 --2- 172.21.15.107:0/204115933 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fccc00fbe40 0x7fccc010fae0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fccb0009fd0 tx=0x7fccb0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:45:16.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:16.801+0000 7fccb6a00700 1 -- 172.21.15.107:0/204115933 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fccb0021070 con 0x7fccc00fbe40 2024-03-04T13:45:16.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:16.801+0000 7fccb6a00700 1 -- 172.21.15.107:0/204115933 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fccb001c400 con 0x7fccc00fbe40 2024-03-04T13:45:16.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:16.801+0000 7fccc5e00700 1 -- 172.21.15.107:0/204115933 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccc0111140 con 0x7fccc00fbe40 2024-03-04T13:45:16.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:16.801+0000 7fccb6a00700 1 -- 172.21.15.107:0/204115933 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fccb000e070 con 0x7fccc00fbe40 2024-03-04T13:45:16.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:16.802+0000 7fccc5e00700 1 -- 172.21.15.107:0/204115933 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccc0111620 con 0x7fccc00fbe40 2024-03-04T13:45:16.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:16.803+0000 7fccc5e00700 1 -- 172.21.15.107:0/204115933 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccc004bed0 con 0x7fccc00fbe40 2024-03-04T13:45:16.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:16.804+0000 7fccb6a00700 1 -- 172.21.15.107:0/204115933 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 33) v1 ==== 91199+0+0 (secure 0 0 0) 0x7fccb0013af0 con 0x7fccc00fbe40 2024-03-04T13:45:16.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:16.805+0000 7fccb6a00700 1 --2- 172.21.15.107:0/204115933 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7fcca406d0e0 0x7fcca406f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:45:16.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:16.805+0000 7fccb6a00700 1 -- 172.21.15.107:0/204115933 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fccb008d710 con 0x7fccc00fbe40 2024-03-04T13:45:16.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:16.806+0000 7fccbe000700 1 --2- 172.21.15.107:0/204115933 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7fcca406d0e0 0x7fcca406f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:45:16.804 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:16.807+0000 7fccbe000700 1 --2- 172.21.15.107:0/204115933 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7fcca406d0e0 0x7fcca406f590 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fccac009b80 tx=0x7fccac00b040 comp rx=0 tx=0).ready entity=mgr.14696 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:45:16.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:16.810+0000 7fccb6a00700 1 -- 172.21.15.107:0/204115933 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7fccb005b340 con 0x7fccc00fbe40 2024-03-04T13:45:17.070 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:17 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:45:17.070 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:17 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:45:17.070 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:17 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:45:17.070 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:17 smithi107 ceph-mon[28739]: pgmap v52: 65 pgs: 65 active+clean; 274 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 162 op/s 2024-03-04T13:45:17.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.096+0000 7fccc5e00700 1 -- 172.21.15.107:0/204115933 --> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fccc0101590 con 0x7fcca406d0e0 2024-03-04T13:45:17.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.098+0000 7fccb6a00700 1 -- 172.21.15.107:0/204115933 <== mgr.14696 v2:172.21.15.107:6800/1613321046 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+367 (secure 0 0 0) 0x7fccc0101590 con 0x7fcca406d0e0 2024-03-04T13:45:17.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.101+0000 7fccc5e00700 1 -- 172.21.15.107:0/204115933 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7fcca406d0e0 msgr2=0x7fcca406f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:45:17.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.101+0000 7fccc5e00700 1 --2- 172.21.15.107:0/204115933 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7fcca406d0e0 0x7fcca406f590 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fccac009b80 tx=0x7fccac00b040 comp rx=0 tx=0).stop 2024-03-04T13:45:17.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.101+0000 7fccc5e00700 1 -- 172.21.15.107:0/204115933 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fccc00fbe40 msgr2=0x7fccc010fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:45:17.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.101+0000 7fccc5e00700 1 --2- 172.21.15.107:0/204115933 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fccc00fbe40 0x7fccc010fae0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fccb0009fd0 tx=0x7fccb0009e30 comp rx=0 tx=0).stop 2024-03-04T13:45:17.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.101+0000 7fccc5e00700 1 -- 172.21.15.107:0/204115933 shutdown_connections 2024-03-04T13:45:17.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.101+0000 7fccc5e00700 1 --2- 172.21.15.107:0/204115933 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7fcca406d0e0 0x7fcca406f590 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:45:17.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.101+0000 7fccc5e00700 1 --2- 172.21.15.107:0/204115933 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fccc00fbe40 0x7fccc010fae0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:45:17.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.101+0000 7fccc5e00700 1 --2- 172.21.15.107:0/204115933 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fccc00fcfb0 0x7fccc0110020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:45:17.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.101+0000 7fccc5e00700 1 -- 172.21.15.107:0/204115933 >> 172.21.15.107:0/204115933 conn(0x7fccc00f74e0 msgr2=0x7fccc00f9710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:45:17.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.101+0000 7fccc5e00700 1 -- 172.21.15.107:0/204115933 shutdown_connections 2024-03-04T13:45:17.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.101+0000 7fccc5e00700 1 -- 172.21.15.107:0/204115933 wait complete. 2024-03-04T13:45:17.114 INFO:teuthology.orchestra.run.smithi107.stdout:true 2024-03-04T13:45:17.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.266+0000 7f0ef9200700 1 -- 172.21.15.107:0/1435487097 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0ef40708c0 msgr2=0x7f0ef4070d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:45:17.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.266+0000 7f0ef9200700 1 --2- 172.21.15.107:0/1435487097 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0ef40708c0 0x7f0ef4070d10 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f0ee0004940 tx=0x7f0ee0009e30 comp rx=0 tx=0).stop 2024-03-04T13:45:17.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.266+0000 7f0ef9200700 1 -- 172.21.15.107:0/1435487097 shutdown_connections 2024-03-04T13:45:17.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.266+0000 7f0ef9200700 1 --2- 172.21.15.107:0/1435487097 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0ef40708c0 0x7f0ef4070d10 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:45:17.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.266+0000 7f0ef9200700 1 --2- 172.21.15.107:0/1435487097 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ef4071f10 0x7f0ef4070380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:45:17.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.266+0000 7f0ef9200700 1 -- 172.21.15.107:0/1435487097 >> 172.21.15.107:0/1435487097 conn(0x7f0ef40f8620 msgr2=0x7f0ef40faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:45:17.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.266+0000 7f0ef9200700 1 -- 172.21.15.107:0/1435487097 shutdown_connections 2024-03-04T13:45:17.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.266+0000 7f0ef9200700 1 -- 172.21.15.107:0/1435487097 wait complete. 2024-03-04T13:45:17.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.267+0000 7f0ef9200700 1 Processor -- start 2024-03-04T13:45:17.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.267+0000 7f0ef9200700 1 -- start start 2024-03-04T13:45:17.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.267+0000 7f0ef9200700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ef40708c0 0x7f0ef4113fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:45:17.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.267+0000 7f0ef9200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0ef4071f10 0x7f0ef41144f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:45:17.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.267+0000 7f0ef9200700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef4114af0 con 0x7f0ef4071f10 2024-03-04T13:45:17.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.267+0000 7f0ef9200700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef4115390 con 0x7f0ef40708c0 2024-03-04T13:45:17.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.268+0000 7f0ef1400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0ef4071f10 0x7f0ef41144f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:45:17.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.268+0000 7f0ef1400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0ef4071f10 0x7f0ef41144f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38516/0 (socket says 172.21.15.107:38516) 2024-03-04T13:45:17.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.268+0000 7f0ef1400700 1 -- 172.21.15.107:0/1888593104 learned_addr learned my addr 172.21.15.107:0/1888593104 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:45:17.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.268+0000 7f0ef1400700 1 -- 172.21.15.107:0/1888593104 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ef40708c0 msgr2=0x7f0ef4113fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:45:17.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.268+0000 7f0ef1e00700 1 --2- 172.21.15.107:0/1888593104 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ef40708c0 0x7f0ef4113fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:45:17.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.268+0000 7f0ef1400700 1 --2- 172.21.15.107:0/1888593104 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ef40708c0 0x7f0ef4113fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:45:17.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.268+0000 7f0ef1400700 1 -- 172.21.15.107:0/1888593104 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ee0005040 con 0x7f0ef4071f10 2024-03-04T13:45:17.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.268+0000 7f0ef1e00700 1 --2- 172.21.15.107:0/1888593104 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ef40708c0 0x7f0ef4113fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T13:45:17.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.269+0000 7f0ef1400700 1 --2- 172.21.15.107:0/1888593104 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0ef4071f10 0x7f0ef41144f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f0ee000b3b0 tx=0x7f0ee0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:45:17.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.269+0000 7f0eea000700 1 -- 172.21.15.107:0/1888593104 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ee000e040 con 0x7f0ef4071f10 2024-03-04T13:45:17.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.269+0000 7f0eea000700 1 -- 172.21.15.107:0/1888593104 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0ee000bd90 con 0x7f0ef4071f10 2024-03-04T13:45:17.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.269+0000 7f0ef9200700 1 -- 172.21.15.107:0/1888593104 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ef4115610 con 0x7f0ef4071f10 2024-03-04T13:45:17.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.269+0000 7f0eea000700 1 -- 172.21.15.107:0/1888593104 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ee001c630 con 0x7f0ef4071f10 2024-03-04T13:45:17.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.269+0000 7f0ef9200700 1 -- 172.21.15.107:0/1888593104 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ef4115b50 con 0x7f0ef4071f10 2024-03-04T13:45:17.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.270+0000 7f0ef9200700 1 -- 172.21.15.107:0/1888593104 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ef404bed0 con 0x7f0ef4071f10 2024-03-04T13:45:17.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.271+0000 7f0eea000700 1 -- 172.21.15.107:0/1888593104 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 33) v1 ==== 91199+0+0 (secure 0 0 0) 0x7f0ee00139f0 con 0x7f0ef4071f10 2024-03-04T13:45:17.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.272+0000 7f0eea000700 1 --2- 172.21.15.107:0/1888593104 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f0ed806d0f0 0x7f0ed806f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:45:17.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.272+0000 7f0eea000700 1 -- 172.21.15.107:0/1888593104 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f0ee008db30 con 0x7f0ef4071f10 2024-03-04T13:45:17.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.273+0000 7f0ef1e00700 1 --2- 172.21.15.107:0/1888593104 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f0ed806d0f0 0x7f0ed806f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:45:17.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.273+0000 7f0ef1e00700 1 --2- 172.21.15.107:0/1888593104 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f0ed806d0f0 0x7f0ed806f5a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0ee4007fd0 tx=0x7f0ee4009040 comp rx=0 tx=0).ready entity=mgr.14696 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:45:17.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.277+0000 7f0eea000700 1 -- 172.21.15.107:0/1888593104 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f0ee005b6e0 con 0x7f0ef4071f10 2024-03-04T13:45:17.359 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:17 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:45:17.359 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:17 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:45:17.359 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:17 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:45:17.359 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:17 smithi169 ceph-mon[36313]: pgmap v52: 65 pgs: 65 active+clean; 274 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 162 op/s 2024-03-04T13:45:17.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.527+0000 7f0ef9200700 1 -- 172.21.15.107:0/1888593104 --> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0ef4115f70 con 0x7f0ed806d0f0 2024-03-04T13:45:17.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.529+0000 7f0eea000700 1 -- 172.21.15.107:0/1888593104 <== mgr.14696 v2:172.21.15.107:6800/1613321046 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+367 (secure 0 0 0) 0x7f0ef4115f70 con 0x7f0ed806d0f0 2024-03-04T13:45:17.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.532+0000 7f0ef9200700 1 -- 172.21.15.107:0/1888593104 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f0ed806d0f0 msgr2=0x7f0ed806f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:45:17.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.532+0000 7f0ef9200700 1 --2- 172.21.15.107:0/1888593104 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f0ed806d0f0 0x7f0ed806f5a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0ee4007fd0 tx=0x7f0ee4009040 comp rx=0 tx=0).stop 2024-03-04T13:45:17.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.532+0000 7f0ef9200700 1 -- 172.21.15.107:0/1888593104 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0ef4071f10 msgr2=0x7f0ef41144f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:45:17.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.532+0000 7f0ef9200700 1 --2- 172.21.15.107:0/1888593104 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0ef4071f10 0x7f0ef41144f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f0ee000b3b0 tx=0x7f0ee0009e30 comp rx=0 tx=0).stop 2024-03-04T13:45:17.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.532+0000 7f0ef9200700 1 -- 172.21.15.107:0/1888593104 shutdown_connections 2024-03-04T13:45:17.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.532+0000 7f0ef9200700 1 --2- 172.21.15.107:0/1888593104 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ef40708c0 0x7f0ef4113fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:45:17.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.532+0000 7f0ef9200700 1 --2- 172.21.15.107:0/1888593104 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f0ed806d0f0 0x7f0ed806f5a0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:45:17.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.532+0000 7f0ef9200700 1 --2- 172.21.15.107:0/1888593104 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0ef4071f10 0x7f0ef41144f0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:45:17.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.532+0000 7f0ef9200700 1 -- 172.21.15.107:0/1888593104 >> 172.21.15.107:0/1888593104 conn(0x7f0ef40f8620 msgr2=0x7f0ef40fa9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:45:17.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.533+0000 7f0ef9200700 1 -- 172.21.15.107:0/1888593104 shutdown_connections 2024-03-04T13:45:17.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.533+0000 7f0ef9200700 1 -- 172.21.15.107:0/1888593104 wait complete. 2024-03-04T13:45:17.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.693+0000 7f7eb3200700 1 -- 172.21.15.107:0/2473530092 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7eac1003a0 msgr2=0x7f7eac102780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:45:17.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.693+0000 7f7eb3200700 1 --2- 172.21.15.107:0/2473530092 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7eac1003a0 0x7f7eac102780 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f7e98013bf0 tx=0x7f7e98009e30 comp rx=0 tx=0).stop 2024-03-04T13:45:17.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.694+0000 7f7eb3200700 1 -- 172.21.15.107:0/2473530092 shutdown_connections 2024-03-04T13:45:17.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.694+0000 7f7eb3200700 1 --2- 172.21.15.107:0/2473530092 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7eac1003a0 0x7f7eac102780 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:45:17.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.694+0000 7f7eb3200700 1 --2- 172.21.15.107:0/2473530092 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7eac0fda80 0x7f7eac0ffe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:45:17.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.694+0000 7f7eb3200700 1 -- 172.21.15.107:0/2473530092 >> 172.21.15.107:0/2473530092 conn(0x7f7eac0f74e0 msgr2=0x7f7eac0f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:45:17.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.694+0000 7f7eb3200700 1 -- 172.21.15.107:0/2473530092 shutdown_connections 2024-03-04T13:45:17.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.695+0000 7f7eb3200700 1 -- 172.21.15.107:0/2473530092 wait complete. 2024-03-04T13:45:17.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.696+0000 7f7eb3200700 1 Processor -- start 2024-03-04T13:45:17.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.697+0000 7f7eb3200700 1 -- start start 2024-03-04T13:45:17.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.697+0000 7f7eb3200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7eac0fda80 0x7f7eac10b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:45:17.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.697+0000 7f7eb3200700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7eac1003a0 0x7f7eac10bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:45:17.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.697+0000 7f7eb3200700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eac10c380 con 0x7f7eac0fda80 2024-03-04T13:45:17.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.697+0000 7f7eb3200700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eac10cbc0 con 0x7f7eac1003a0 2024-03-04T13:45:17.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.698+0000 7f7eab400700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7eac1003a0 0x7f7eac10bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:45:17.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.699+0000 7f7eab400700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7eac1003a0 0x7f7eac10bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.107:44408/0 (socket says 172.21.15.107:44408) 2024-03-04T13:45:17.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.699+0000 7f7eab400700 1 -- 172.21.15.107:0/3941776956 learned_addr learned my addr 172.21.15.107:0/3941776956 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:45:17.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.700+0000 7f7eabe00700 1 --2- 172.21.15.107:0/3941776956 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7eac0fda80 0x7f7eac10b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:45:17.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.701+0000 7f7eabe00700 1 -- 172.21.15.107:0/3941776956 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7eac1003a0 msgr2=0x7f7eac10bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:45:17.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.701+0000 7f7eabe00700 1 --2- 172.21.15.107:0/3941776956 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7eac1003a0 0x7f7eac10bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:45:17.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.701+0000 7f7eabe00700 1 -- 172.21.15.107:0/3941776956 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e98005040 con 0x7f7eac0fda80 2024-03-04T13:45:17.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.701+0000 7f7eabe00700 1 --2- 172.21.15.107:0/3941776956 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7eac0fda80 0x7f7eac10b7b0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f7ea400ba50 tx=0x7f7ea4009760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:45:17.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.702+0000 7f7ea3e00700 1 -- 172.21.15.107:0/3941776956 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7ea400ca20 con 0x7f7eac0fda80 2024-03-04T13:45:17.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.702+0000 7f7eb3200700 1 -- 172.21.15.107:0/3941776956 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7eac10cea0 con 0x7f7eac0fda80 2024-03-04T13:45:17.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.702+0000 7f7eb3200700 1 -- 172.21.15.107:0/3941776956 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7eac10d3e0 con 0x7f7eac0fda80 2024-03-04T13:45:17.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.702+0000 7f7ea3e00700 1 -- 172.21.15.107:0/3941776956 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7ea40098a0 con 0x7f7eac0fda80 2024-03-04T13:45:17.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.702+0000 7f7ea3e00700 1 -- 172.21.15.107:0/3941776956 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7ea4013600 con 0x7f7eac0fda80 2024-03-04T13:45:17.703 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.705+0000 7f7eb3200700 1 -- 172.21.15.107:0/3941776956 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e8c0052e0 con 0x7f7eac0fda80 2024-03-04T13:45:17.703 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.705+0000 7f7ea3e00700 1 -- 172.21.15.107:0/3941776956 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 33) v1 ==== 91199+0+0 (secure 0 0 0) 0x7f7ea4013760 con 0x7f7eac0fda80 2024-03-04T13:45:17.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.707+0000 7f7ea3e00700 1 --2- 172.21.15.107:0/3941776956 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f7e9006d1c0 0x7f7e9006f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:45:17.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.707+0000 7f7ea3e00700 1 -- 172.21.15.107:0/3941776956 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f7ea408c770 con 0x7f7eac0fda80 2024-03-04T13:45:17.713 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.711+0000 7f7eab400700 1 --2- 172.21.15.107:0/3941776956 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f7e9006d1c0 0x7f7e9006f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:45:17.713 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.712+0000 7f7eab400700 1 --2- 172.21.15.107:0/3941776956 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f7e9006d1c0 0x7f7e9006f670 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7e98004a40 tx=0x7f7e98009e30 comp rx=0 tx=0).ready entity=mgr.14696 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:45:17.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.717+0000 7f7ea3e00700 1 -- 172.21.15.107:0/3941776956 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f7ea405b340 con 0x7f7eac0fda80 2024-03-04T13:45:17.969 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.971+0000 7f7eb3200700 1 -- 172.21.15.107:0/3941776956 --> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f7e8c000bf0 con 0x7f7e9006d1c0 2024-03-04T13:45:17.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.990+0000 7f7ea3e00700 1 -- 172.21.15.107:0/3941776956 <== mgr.14696 v2:172.21.15.107:6800/1613321046 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f7e8c000bf0 con 0x7f7e9006d1c0 2024-03-04T13:45:17.988 INFO:teuthology.orchestra.run.smithi107.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-04T13:45:17.988 INFO:teuthology.orchestra.run.smithi107.stdout:alertmanager.smithi107 smithi107 *:9093,9094 running (3m) 22s ago 13m 15.7M - 0.25.0 c8568f914cd2 ac0e4f11c97a 2024-03-04T13:45:17.988 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi107 smithi107 running (13m) 22s ago 13m 6665k - 17.2.7-545-g42608bb1 9eec3f22859d 03e1570c2eff 2024-03-04T13:45:17.988 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi169 smithi169 running (12m) 25s ago 12m 6656k - 17.2.7-545-g42608bb1 9eec3f22859d 9c8881a5338e 2024-03-04T13:45:17.988 INFO:teuthology.orchestra.run.smithi107.stdout:grafana.smithi107 smithi107 *:3000 running (2m) 22s ago 13m 74.8M - 9.4.7 954c08fa6188 bd720c203473 2024-03-04T13:45:17.988 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.riexmm smithi107 running (6m) 22s ago 6m 12.8M - 17.2.7-545-g42608bb1 9eec3f22859d fc921f7d4328 2024-03-04T13:45:17.988 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.thycho smithi107 running (6m) 22s ago 6m 233M - 17.2.7-545-g42608bb1 9eec3f22859d 4069c6e0e063 2024-03-04T13:45:17.988 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi169.bxitzk smithi169 running (6m) 25s ago 6m 13.0M - 17.2.7-545-g42608bb1 9eec3f22859d 7d4115228c7c 2024-03-04T13:45:17.989 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi169.jusudl smithi169 running (6m) 25s ago 6m 210M - 17.2.7-545-g42608bb1 9eec3f22859d 2d2f2cc452e8 2024-03-04T13:45:17.989 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi107.wpzgwl smithi107 *:8443,9283,8765 running (2m) 22s ago 15m 497M - 19.0.0-1845-g717ce3cc d71b839abdf8 1c5fdee45eac 2024-03-04T13:45:17.989 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi169.eihczr smithi169 *:8443,9283,8765 running (63s) 25s ago 12m 447M - 19.0.0-1845-g717ce3cc d71b839abdf8 b5cac2b7d22a 2024-03-04T13:45:17.989 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi107 smithi107 running (15m) 22s ago 15m 52.4M 2048M 17.2.7-545-g42608bb1 9eec3f22859d 49f5da990d35 2024-03-04T13:45:17.989 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi169 smithi169 running (12m) 25s ago 12m 42.4M 2048M 17.2.7-545-g42608bb1 9eec3f22859d ba4ee916297f 2024-03-04T13:45:17.989 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi107 smithi107 *:9100 running (42s) 22s ago 13m 6589k - 1.5.0 0da6a335fe13 8a8d3d4f724f 2024-03-04T13:45:17.989 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi169 smithi169 *:9100 running (32s) 25s ago 12m 2935k - 1.5.0 0da6a335fe13 f240f5a975bc 2024-03-04T13:45:17.989 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0 smithi107 running (11m) 22s ago 11m 186M 1558M 17.2.7-545-g42608bb1 9eec3f22859d c0605b68f696 2024-03-04T13:45:17.989 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1 smithi107 running (10m) 22s ago 10m 200M 1558M 17.2.7-545-g42608bb1 9eec3f22859d 5fb2c0dd5769 2024-03-04T13:45:17.989 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2 smithi107 running (10m) 22s ago 10m 168M 1558M 17.2.7-545-g42608bb1 9eec3f22859d 61f30f643549 2024-03-04T13:45:17.989 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3 smithi169 running (9m) 25s ago 9m 222M 2582M 17.2.7-545-g42608bb1 9eec3f22859d cd91d8694764 2024-03-04T13:45:17.989 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4 smithi169 running (9m) 25s ago 9m 194M 2582M 17.2.7-545-g42608bb1 9eec3f22859d 08545e3d0659 2024-03-04T13:45:17.989 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5 smithi169 running (8m) 25s ago 8m 180M 2582M 17.2.7-545-g42608bb1 9eec3f22859d dcffe6cb3eac 2024-03-04T13:45:17.989 INFO:teuthology.orchestra.run.smithi107.stdout:prometheus.smithi107 smithi107 *:9095 starting - - - - 2024-03-04T13:45:17.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.992+0000 7f7eb3200700 1 -- 172.21.15.107:0/3941776956 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f7e9006d1c0 msgr2=0x7f7e9006f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:45:17.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.992+0000 7f7eb3200700 1 --2- 172.21.15.107:0/3941776956 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f7e9006d1c0 0x7f7e9006f670 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7e98004a40 tx=0x7f7e98009e30 comp rx=0 tx=0).stop 2024-03-04T13:45:17.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.992+0000 7f7eb3200700 1 -- 172.21.15.107:0/3941776956 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7eac0fda80 msgr2=0x7f7eac10b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:45:17.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.992+0000 7f7eb3200700 1 --2- 172.21.15.107:0/3941776956 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7eac0fda80 0x7f7eac10b7b0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f7ea400ba50 tx=0x7f7ea4009760 comp rx=0 tx=0).stop 2024-03-04T13:45:17.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.993+0000 7f7eb3200700 1 -- 172.21.15.107:0/3941776956 shutdown_connections 2024-03-04T13:45:17.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.993+0000 7f7eb3200700 1 --2- 172.21.15.107:0/3941776956 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f7e9006d1c0 0x7f7e9006f670 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:45:17.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.993+0000 7f7eb3200700 1 --2- 172.21.15.107:0/3941776956 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7eac0fda80 0x7f7eac10b7b0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:45:17.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.993+0000 7f7eb3200700 1 --2- 172.21.15.107:0/3941776956 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7eac1003a0 0x7f7eac10bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:45:17.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.993+0000 7f7eb3200700 1 -- 172.21.15.107:0/3941776956 >> 172.21.15.107:0/3941776956 conn(0x7f7eac0f74e0 msgr2=0x7f7eac0f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:45:17.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.993+0000 7f7eb3200700 1 -- 172.21.15.107:0/3941776956 shutdown_connections 2024-03-04T13:45:17.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:17.993+0000 7f7eb3200700 1 -- 172.21.15.107:0/3941776956 wait complete. 2024-03-04T13:45:18.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:18.141+0000 7f63c0a00700 1 -- 172.21.15.107:0/3903246008 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63bc104750 msgr2=0x7f63bc104ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:45:18.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:18.141+0000 7f63c0a00700 1 --2- 172.21.15.107:0/3903246008 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63bc104750 0x7f63bc104ba0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f63a4004940 tx=0x7f63a4009e30 comp rx=0 tx=0).stop 2024-03-04T13:45:18.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:18.141+0000 7f63c0a00700 1 -- 172.21.15.107:0/3903246008 shutdown_connections 2024-03-04T13:45:18.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:18.141+0000 7f63c0a00700 1 --2- 172.21.15.107:0/3903246008 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63bc104750 0x7f63bc104ba0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:45:18.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:18.141+0000 7f63c0a00700 1 --2- 172.21.15.107:0/3903246008 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63bc103540 0x7f63bc103950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:45:18.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:18.141+0000 7f63c0a00700 1 -- 172.21.15.107:0/3903246008 >> 172.21.15.107:0/3903246008 conn(0x7f63bc0727d0 msgr2=0x7f63bc074c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:45:18.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:18.141+0000 7f63c0a00700 1 -- 172.21.15.107:0/3903246008 shutdown_connections 2024-03-04T13:45:18.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:18.141+0000 7f63c0a00700 1 -- 172.21.15.107:0/3903246008 wait complete. 2024-03-04T13:45:18.140 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:18 smithi107 ceph-mon[28739]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:45:18.140 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:18 smithi107 ceph-mon[28739]: from='client.14788 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:45:18.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:18.142+0000 7f63c0a00700 1 Processor -- start 2024-03-04T13:45:18.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:18.143+0000 7f63c0a00700 1 -- start start 2024-03-04T13:45:18.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:18.143+0000 7f63c0a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63bc103540 0x7f63bc113fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:45:18.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:18.143+0000 7f63c0a00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63bc104750 0x7f63bc1144e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:45:18.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:18.143+0000 7f63c0a00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63bc114b70 con 0x7f63bc103540 2024-03-04T13:45:18.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:18.143+0000 7f63c0a00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63bc1153b0 con 0x7f63bc104750 2024-03-04T13:45:18.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:18.143+0000 7f63b9e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63bc103540 0x7f63bc113fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:45:18.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:18.143+0000 7f63b9e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63bc103540 0x7f63bc113fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54880/0 (socket says 172.21.15.107:54880) 2024-03-04T13:45:18.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:18.143+0000 7f63b9e00700 1 -- 172.21.15.107:0/460637981 learned_addr learned my addr 172.21.15.107:0/460637981 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:45:18.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:18.144+0000 7f63b9400700 1 --2- 172.21.15.107:0/460637981 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63bc104750 0x7f63bc1144e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:45:18.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:18.144+0000 7f63b9e00700 1 -- 172.21.15.107:0/460637981 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63bc104750 msgr2=0x7f63bc1144e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:45:18.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:18.144+0000 7f63b9e00700 1 --2- 172.21.15.107:0/460637981 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63bc104750 0x7f63bc1144e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:45:18.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:18.144+0000 7f63b9e00700 1 -- 172.21.15.107:0/460637981 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63a4005040 con 0x7f63bc103540 2024-03-04T13:45:18.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:18.144+0000 7f63b9e00700 1 --2- 172.21.15.107:0/460637981 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63bc103540 0x7f63bc113fa0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f63ac00b790 tx=0x7f63ac009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:45:18.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:18.144+0000 7f63b2000700 1 -- 172.21.15.107:0/460637981 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f63ac00f6f0 con 0x7f63bc103540 2024-03-04T13:45:18.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:18.144+0000 7f63b2000700 1 -- 172.21.15.107:0/460637981 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f63ac00fd20 con 0x7f63bc103540 2024-03-04T13:45:18.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:18.144+0000 7f63b2000700 1 -- 172.21.15.107:0/460637981 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f63ac0103a0 con 0x7f63bc103540 2024-03-04T13:45:18.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:18.144+0000 7f63c0a00700 1 -- 172.21.15.107:0/460637981 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63bc115690 con 0x7f63bc103540 2024-03-04T13:45:18.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:18.145+0000 7f63c0a00700 1 -- 172.21.15.107:0/460637981 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63bc115bd0 con 0x7f63bc103540 2024-03-04T13:45:18.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:18.146+0000 7f63c0a00700 1 -- 172.21.15.107:0/460637981 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63bc04bed0 con 0x7f63bc103540 2024-03-04T13:45:18.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:18.147+0000 7f63b2000700 1 -- 172.21.15.107:0/460637981 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 33) v1 ==== 91199+0+0 (secure 0 0 0) 0x7f63ac011400 con 0x7f63bc103540 2024-03-04T13:45:18.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:18.147+0000 7f63b2000700 1 --2- 172.21.15.107:0/460637981 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f63a006d0f0 0x7f63a006f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:45:18.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:18.148+0000 7f63b2000700 1 -- 172.21.15.107:0/460637981 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f63ac08c450 con 0x7f63bc103540 2024-03-04T13:45:18.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:18.148+0000 7f63b9400700 1 --2- 172.21.15.107:0/460637981 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f63a006d0f0 0x7f63a006f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:45:18.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:18.148+0000 7f63b9400700 1 --2- 172.21.15.107:0/460637981 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f63a006d0f0 0x7f63a006f5a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f63a400b020 tx=0x7f63a4009e30 comp rx=0 tx=0).ready entity=mgr.14696 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:45:18.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:18.153+0000 7f63b2000700 1 -- 172.21.15.107:0/460637981 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f63ac059f80 con 0x7f63bc103540 2024-03-04T13:45:18.359 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:18 smithi169 ceph-mon[36313]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:45:18.359 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:18 smithi169 ceph-mon[36313]: from='client.14788 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:45:18.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:18.492+0000 7f63c0a00700 1 -- 172.21.15.107:0/460637981 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f63bc116010 con 0x7f63bc103540 2024-03-04T13:45:18.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:18.493+0000 7f63b2000700 1 -- 172.21.15.107:0/460637981 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7f63ac059b10 con 0x7f63bc103540 2024-03-04T13:45:18.492 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-03-04T13:45:18.492 INFO:teuthology.orchestra.run.smithi107.stdout: "mon": { 2024-03-04T13:45:18.492 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-545-g42608bb1 (42608bb191117be10dba720e6bf8a02f3171cd34) quincy (stable)": 2 2024-03-04T13:45:18.492 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:45:18.492 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr": { 2024-03-04T13:45:18.492 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-1845-g717ce3cc (717ce3cce8d0166e4e577faa8004e6f9cb4128c0) squid (dev)": 2 2024-03-04T13:45:18.492 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:45:18.492 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": { 2024-03-04T13:45:18.492 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-545-g42608bb1 (42608bb191117be10dba720e6bf8a02f3171cd34) quincy (stable)": 6 2024-03-04T13:45:18.492 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:45:18.492 INFO:teuthology.orchestra.run.smithi107.stdout: "mds": { 2024-03-04T13:45:18.493 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-545-g42608bb1 (42608bb191117be10dba720e6bf8a02f3171cd34) quincy (stable)": 4 2024-03-04T13:45:18.493 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:45:18.493 INFO:teuthology.orchestra.run.smithi107.stdout: "overall": { 2024-03-04T13:45:18.493 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-545-g42608bb1 (42608bb191117be10dba720e6bf8a02f3171cd34) quincy (stable)": 12, 2024-03-04T13:45:18.493 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-1845-g717ce3cc (717ce3cce8d0166e4e577faa8004e6f9cb4128c0) squid (dev)": 2 2024-03-04T13:45:18.493 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-04T13:45:18.493 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-03-04T13:45:18.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:18.496+0000 7f63c0a00700 1 -- 172.21.15.107:0/460637981 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f63a006d0f0 msgr2=0x7f63a006f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:45:18.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:18.496+0000 7f63c0a00700 1 --2- 172.21.15.107:0/460637981 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f63a006d0f0 0x7f63a006f5a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f63a400b020 tx=0x7f63a4009e30 comp rx=0 tx=0).stop 2024-03-04T13:45:18.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:18.496+0000 7f63c0a00700 1 -- 172.21.15.107:0/460637981 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63bc103540 msgr2=0x7f63bc113fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:45:18.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:18.496+0000 7f63c0a00700 1 --2- 172.21.15.107:0/460637981 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63bc103540 0x7f63bc113fa0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f63ac00b790 tx=0x7f63ac009220 comp rx=0 tx=0).stop 2024-03-04T13:45:18.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:18.496+0000 7f63c0a00700 1 -- 172.21.15.107:0/460637981 shutdown_connections 2024-03-04T13:45:18.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:18.496+0000 7f63c0a00700 1 --2- 172.21.15.107:0/460637981 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f63a006d0f0 0x7f63a006f5a0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:45:18.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:18.496+0000 7f63c0a00700 1 --2- 172.21.15.107:0/460637981 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63bc103540 0x7f63bc113fa0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:45:18.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:18.496+0000 7f63c0a00700 1 --2- 172.21.15.107:0/460637981 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63bc104750 0x7f63bc1144e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:45:18.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:18.496+0000 7f63c0a00700 1 -- 172.21.15.107:0/460637981 >> 172.21.15.107:0/460637981 conn(0x7f63bc0727d0 msgr2=0x7f63bc107980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:45:18.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:18.496+0000 7f63c0a00700 1 -- 172.21.15.107:0/460637981 shutdown_connections 2024-03-04T13:45:18.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:18.496+0000 7f63c0a00700 1 -- 172.21.15.107:0/460637981 wait complete. 2024-03-04T13:45:18.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:18.645+0000 7f7415400700 1 -- 172.21.15.107:0/760044182 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74100fb220 msgr2=0x7f74100fb630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:45:18.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:18.645+0000 7f7415400700 1 --2- 172.21.15.107:0/760044182 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74100fb220 0x7f74100fb630 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f7400004940 tx=0x7f7400009e30 comp rx=0 tx=0).stop 2024-03-04T13:45:18.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:18.646+0000 7f7415400700 1 -- 172.21.15.107:0/760044182 shutdown_connections 2024-03-04T13:45:18.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:18.646+0000 7f7415400700 1 --2- 172.21.15.107:0/760044182 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f74100fe2b0 0x7f74100fe700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:45:18.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:18.646+0000 7f7415400700 1 --2- 172.21.15.107:0/760044182 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74100fb220 0x7f74100fb630 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:45:18.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:18.646+0000 7f7415400700 1 -- 172.21.15.107:0/760044182 >> 172.21.15.107:0/760044182 conn(0x7f74100f6ca0 msgr2=0x7f74100f90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:45:18.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:18.646+0000 7f7415400700 1 -- 172.21.15.107:0/760044182 shutdown_connections 2024-03-04T13:45:18.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:18.646+0000 7f7415400700 1 -- 172.21.15.107:0/760044182 wait complete. 2024-03-04T13:45:18.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:18.646+0000 7f7415400700 1 Processor -- start 2024-03-04T13:45:18.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:18.647+0000 7f7415400700 1 -- start start 2024-03-04T13:45:18.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:18.647+0000 7f7415400700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f74100fb220 0x7f741010ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:45:18.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:18.647+0000 7f7415400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74100fe2b0 0x7f74101104f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:45:18.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:18.647+0000 7f7415400700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f741010b9c0 con 0x7f74100fe2b0 2024-03-04T13:45:18.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:18.647+0000 7f7415400700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f741010bb00 con 0x7f74100fb220 2024-03-04T13:45:18.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:18.647+0000 7f740de00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74100fe2b0 0x7f74101104f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:45:18.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:18.647+0000 7f740de00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74100fe2b0 0x7f74101104f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54910/0 (socket says 172.21.15.107:54910) 2024-03-04T13:45:18.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:18.647+0000 7f740de00700 1 -- 172.21.15.107:0/2720786950 learned_addr learned my addr 172.21.15.107:0/2720786950 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:45:18.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:18.647+0000 7f740e800700 1 --2- 172.21.15.107:0/2720786950 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f74100fb220 0x7f741010ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:45:18.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:18.648+0000 7f740de00700 1 -- 172.21.15.107:0/2720786950 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f74100fb220 msgr2=0x7f741010ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:45:18.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:18.648+0000 7f740de00700 1 --2- 172.21.15.107:0/2720786950 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f74100fb220 0x7f741010ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:45:18.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:18.648+0000 7f740de00700 1 -- 172.21.15.107:0/2720786950 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7400005040 con 0x7f74100fe2b0 2024-03-04T13:45:18.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:18.648+0000 7f740de00700 1 --2- 172.21.15.107:0/2720786950 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74100fe2b0 0x7f74101104f0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f73fc00b790 tx=0x7f73fc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:45:18.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:18.648+0000 7f7406a00700 1 -- 172.21.15.107:0/2720786950 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f73fc00f770 con 0x7f74100fe2b0 2024-03-04T13:45:18.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:18.648+0000 7f7406a00700 1 -- 172.21.15.107:0/2720786950 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f73fc00fda0 con 0x7f74100fe2b0 2024-03-04T13:45:18.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:18.648+0000 7f7415400700 1 -- 172.21.15.107:0/2720786950 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f741010bde0 con 0x7f74100fe2b0 2024-03-04T13:45:18.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:18.648+0000 7f7406a00700 1 -- 172.21.15.107:0/2720786950 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f73fc00d310 con 0x7f74100fe2b0 2024-03-04T13:45:18.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:18.648+0000 7f7415400700 1 -- 172.21.15.107:0/2720786950 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f741010c320 con 0x7f74100fe2b0 2024-03-04T13:45:18.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:18.649+0000 7f7415400700 1 -- 172.21.15.107:0/2720786950 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f741004bed0 con 0x7f74100fe2b0 2024-03-04T13:45:18.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:18.650+0000 7f7406a00700 1 -- 172.21.15.107:0/2720786950 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 33) v1 ==== 91199+0+0 (secure 0 0 0) 0x7f73fc010420 con 0x7f74100fe2b0 2024-03-04T13:45:18.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:18.651+0000 7f7406a00700 1 --2- 172.21.15.107:0/2720786950 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f73f406d0f0 0x7f73f406f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:45:18.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:18.651+0000 7f7406a00700 1 -- 172.21.15.107:0/2720786950 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f73fc08c5a0 con 0x7f74100fe2b0 2024-03-04T13:45:18.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:18.652+0000 7f740e800700 1 --2- 172.21.15.107:0/2720786950 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f73f406d0f0 0x7f73f406f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:45:18.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:18.652+0000 7f740e800700 1 --2- 172.21.15.107:0/2720786950 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f73f406d0f0 0x7f73f406f5a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f740000b020 tx=0x7f7400009e30 comp rx=0 tx=0).ready entity=mgr.14696 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:45:18.655 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:18.657+0000 7f7406a00700 1 -- 172.21.15.107:0/2720786950 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f73fc0566c0 con 0x7f74100fe2b0 2024-03-04T13:45:18.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:18.951+0000 7f7415400700 1 -- 172.21.15.107:0/2720786950 --> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7410105370 con 0x7f73f406d0f0 2024-03-04T13:45:18.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:18.952+0000 7f7406a00700 1 -- 172.21.15.107:0/2720786950 <== mgr.14696 v2:172.21.15.107:6800/1613321046 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+367 (secure 0 0 0) 0x7f7410105370 con 0x7f73f406d0f0 2024-03-04T13:45:18.950 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-03-04T13:45:18.950 INFO:teuthology.orchestra.run.smithi107.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377", 2024-03-04T13:45:18.950 INFO:teuthology.orchestra.run.smithi107.stdout: "in_progress": true, 2024-03-04T13:45:18.950 INFO:teuthology.orchestra.run.smithi107.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-03-04T13:45:18.950 INFO:teuthology.orchestra.run.smithi107.stdout: "services_complete": [ 2024-03-04T13:45:18.951 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr" 2024-03-04T13:45:18.951 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-04T13:45:18.951 INFO:teuthology.orchestra.run.smithi107.stdout: "progress": "2/2 daemons upgraded", 2024-03-04T13:45:18.951 INFO:teuthology.orchestra.run.smithi107.stdout: "message": "Currently upgrading prometheus daemons", 2024-03-04T13:45:18.951 INFO:teuthology.orchestra.run.smithi107.stdout: "is_paused": false 2024-03-04T13:45:18.951 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-03-04T13:45:18.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:18.954+0000 7f7415400700 1 -- 172.21.15.107:0/2720786950 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f73f406d0f0 msgr2=0x7f73f406f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:45:18.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:18.954+0000 7f7415400700 1 --2- 172.21.15.107:0/2720786950 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f73f406d0f0 0x7f73f406f5a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f740000b020 tx=0x7f7400009e30 comp rx=0 tx=0).stop 2024-03-04T13:45:18.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:18.955+0000 7f7415400700 1 -- 172.21.15.107:0/2720786950 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74100fe2b0 msgr2=0x7f74101104f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:45:18.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:18.955+0000 7f7415400700 1 --2- 172.21.15.107:0/2720786950 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74100fe2b0 0x7f74101104f0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f73fc00b790 tx=0x7f73fc009250 comp rx=0 tx=0).stop 2024-03-04T13:45:18.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:18.955+0000 7f7415400700 1 -- 172.21.15.107:0/2720786950 shutdown_connections 2024-03-04T13:45:18.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:18.955+0000 7f7415400700 1 --2- 172.21.15.107:0/2720786950 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f74100fb220 0x7f741010ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:45:18.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:18.955+0000 7f7415400700 1 --2- 172.21.15.107:0/2720786950 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f73f406d0f0 0x7f73f406f5a0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:45:18.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:18.955+0000 7f7415400700 1 --2- 172.21.15.107:0/2720786950 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74100fe2b0 0x7f74101104f0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:45:18.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:18.955+0000 7f7415400700 1 -- 172.21.15.107:0/2720786950 >> 172.21.15.107:0/2720786950 conn(0x7f74100f6ca0 msgr2=0x7f7410101d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:45:18.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:18.955+0000 7f7415400700 1 -- 172.21.15.107:0/2720786950 shutdown_connections 2024-03-04T13:45:18.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:18.955+0000 7f7415400700 1 -- 172.21.15.107:0/2720786950 wait complete. 2024-03-04T13:45:19.341 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:19 smithi107 ceph-mon[28739]: from='client.14792 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:45:19.341 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:19 smithi107 ceph-mon[28739]: pgmap v53: 65 pgs: 65 active+clean; 274 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 801 KiB/s rd, 761 KiB/s wr, 99 op/s 2024-03-04T13:45:19.341 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:19 smithi107 ceph-mon[28739]: from='client.? 172.21.15.107:0/460637981' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:45:19.359 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:19 smithi169 ceph-mon[36313]: from='client.14792 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:45:19.359 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:19 smithi169 ceph-mon[36313]: pgmap v53: 65 pgs: 65 active+clean; 274 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 801 KiB/s rd, 761 KiB/s wr, 99 op/s 2024-03-04T13:45:19.359 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:19 smithi169 ceph-mon[36313]: from='client.? 172.21.15.107:0/460637981' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:45:20.341 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:20 smithi107 ceph-mon[28739]: from='client.14800 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:45:20.358 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:20 smithi169 ceph-mon[36313]: from='client.14800 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:45:21.591 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:21 smithi107 ceph-mon[28739]: pgmap v54: 65 pgs: 65 active+clean; 272 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1008 KiB/s rd, 1003 KiB/s wr, 135 op/s 2024-03-04T13:45:21.608 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:21 smithi169 ceph-mon[36313]: pgmap v54: 65 pgs: 65 active+clean; 272 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1008 KiB/s rd, 1003 KiB/s wr, 135 op/s 2024-03-04T13:45:23.591 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:23 smithi107 ceph-mon[28739]: pgmap v55: 65 pgs: 65 active+clean; 272 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 611 KiB/s rd, 625 KiB/s wr, 78 op/s 2024-03-04T13:45:23.609 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:23 smithi169 ceph-mon[36313]: pgmap v55: 65 pgs: 65 active+clean; 272 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 611 KiB/s rd, 625 KiB/s wr, 78 op/s 2024-03-04T13:45:24.591 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:24 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:45:24.592 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:24 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:45:24.592 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:24 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:45:24.609 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:24 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:45:24.609 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:24 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:45:24.609 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:24 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:45:25.591 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:25 smithi107 ceph-mon[28739]: pgmap v56: 65 pgs: 65 active+clean; 273 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 797 KiB/s rd, 809 KiB/s wr, 107 op/s 2024-03-04T13:45:25.608 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:25 smithi169 ceph-mon[36313]: pgmap v56: 65 pgs: 65 active+clean; 273 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 797 KiB/s rd, 809 KiB/s wr, 107 op/s 2024-03-04T13:45:27.134 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:26 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:45:27.134 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:26 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:45:27.134 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:26 smithi107 ceph-mon[28739]: pgmap v57: 65 pgs: 65 active+clean; 266 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 734 KiB/s rd, 774 KiB/s wr, 107 op/s 2024-03-04T13:45:27.359 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:26 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:45:27.359 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:26 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:45:27.359 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:26 smithi169 ceph-mon[36313]: pgmap v57: 65 pgs: 65 active+clean; 266 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 734 KiB/s rd, 774 KiB/s wr, 107 op/s 2024-03-04T13:45:29.342 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:29 smithi107 ceph-mon[28739]: pgmap v58: 65 pgs: 65 active+clean; 266 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 392 KiB/s rd, 426 KiB/s wr, 76 op/s 2024-03-04T13:45:29.609 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:29 smithi169 ceph-mon[36313]: pgmap v58: 65 pgs: 65 active+clean; 266 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 392 KiB/s rd, 426 KiB/s wr, 76 op/s 2024-03-04T13:45:31.341 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:31 smithi107 ceph-mon[28739]: pgmap v59: 65 pgs: 65 active+clean; 262 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 557 KiB/s rd, 594 KiB/s wr, 113 op/s 2024-03-04T13:45:31.609 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:31 smithi169 ceph-mon[36313]: pgmap v59: 65 pgs: 65 active+clean; 262 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 557 KiB/s rd, 594 KiB/s wr, 113 op/s 2024-03-04T13:45:32.803 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:32 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:45:32.803 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:32 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:45:32.803 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:32 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:45:32.804 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:32 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:45:32.804 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:32 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:45:32.804 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:32 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-04T13:45:32.804 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:32 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:45:32.804 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:32 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:45:32.804 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:32 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:45:32.804 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:32 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:45:32.804 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:32 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:45:32.804 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:32 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:45:32.804 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:32 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:45:32.804 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:32 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:45:32.804 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:32 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:45:32.804 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:32 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:45:32.804 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:32 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:45:32.861 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:32 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:45:32.861 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:32 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:45:32.861 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:32 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:45:32.861 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:32 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:45:32.861 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:32 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:45:32.861 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:32 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-04T13:45:32.861 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:32 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:45:32.861 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:32 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:45:32.861 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:32 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:45:32.861 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:32 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:45:32.862 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:32 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:45:32.862 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:32 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:45:32.862 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:32 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:45:32.862 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:32 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:45:32.862 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:32 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:45:32.862 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:32 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:45:32.862 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:32 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:45:33.816 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:33 smithi107 ceph-mon[28739]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-04T13:45:33.816 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:33 smithi107 ceph-mon[28739]: pgmap v60: 65 pgs: 65 active+clean; 262 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 350 KiB/s rd, 352 KiB/s wr, 77 op/s 2024-03-04T13:45:33.858 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:33 smithi169 ceph-mon[36313]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-04T13:45:33.859 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:33 smithi169 ceph-mon[36313]: pgmap v60: 65 pgs: 65 active+clean; 262 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 350 KiB/s rd, 352 KiB/s wr, 77 op/s 2024-03-04T13:45:35.698 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:35 smithi107 ceph-mon[28739]: Upgrade: Updating alertmanager.smithi107 2024-03-04T13:45:35.698 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:35 smithi107 ceph-mon[28739]: Deploying daemon alertmanager.smithi107 on smithi107 2024-03-04T13:45:35.698 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:35 smithi107 ceph-mon[28739]: pgmap v61: 65 pgs: 65 active+clean; 263 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 497 KiB/s rd, 488 KiB/s wr, 101 op/s 2024-03-04T13:45:35.859 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:35 smithi169 ceph-mon[36313]: Upgrade: Updating alertmanager.smithi107 2024-03-04T13:45:35.859 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:35 smithi169 ceph-mon[36313]: Deploying daemon alertmanager.smithi107 on smithi107 2024-03-04T13:45:35.859 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:35 smithi169 ceph-mon[36313]: pgmap v61: 65 pgs: 65 active+clean; 263 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 497 KiB/s rd, 488 KiB/s wr, 101 op/s 2024-03-04T13:45:37.091 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:36 smithi107 ceph-mon[28739]: pgmap v62: 65 pgs: 65 active+clean; 263 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 347 KiB/s rd, 362 KiB/s wr, 89 op/s 2024-03-04T13:45:37.109 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:36 smithi169 ceph-mon[36313]: pgmap v62: 65 pgs: 65 active+clean; 263 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 347 KiB/s rd, 362 KiB/s wr, 89 op/s 2024-03-04T13:45:39.592 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:39 smithi107 ceph-mon[28739]: pgmap v63: 65 pgs: 65 active+clean; 263 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 347 KiB/s rd, 362 KiB/s wr, 77 op/s 2024-03-04T13:45:39.592 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:39 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:45:39.609 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:39 smithi169 ceph-mon[36313]: pgmap v63: 65 pgs: 65 active+clean; 263 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 347 KiB/s rd, 362 KiB/s wr, 77 op/s 2024-03-04T13:45:39.609 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:39 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:45:40.165 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.0... 2024-03-04T13:45:40.165 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-03-04T13:45:41.312 DEBUG:teuthology.parallel:result is None 2024-03-04T13:45:41.313 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-03-04T13:45:41.406 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-03-04T13:45:41.406 DEBUG:teuthology.orchestra.run.smithi169:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1 2024-03-04T13:45:41.486 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:41 smithi169 ceph-mon[36313]: pgmap v64: 65 pgs: 65 active+clean; 261 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 562 KiB/s rd, 585 KiB/s wr, 114 op/s 2024-03-04T13:45:41.520 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.1/client.1 2024-03-04T13:45:41.521 DEBUG:teuthology.parallel:result is None 2024-03-04T13:45:41.568 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:41 smithi107 ceph-mon[28739]: pgmap v64: 65 pgs: 65 active+clean; 261 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 562 KiB/s rd, 585 KiB/s wr, 114 op/s 2024-03-04T13:45:43.592 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:43 smithi107 ceph-mon[28739]: pgmap v65: 65 pgs: 65 active+clean; 261 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 397 KiB/s rd, 418 KiB/s wr, 77 op/s 2024-03-04T13:45:43.592 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:43 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:45:43.592 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:43 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:45:43.593 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:43 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:45:43.609 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:43 smithi169 ceph-mon[36313]: pgmap v65: 65 pgs: 65 active+clean; 261 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 397 KiB/s rd, 418 KiB/s wr, 77 op/s 2024-03-04T13:45:43.609 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:43 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:45:43.609 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:43 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:45:43.609 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:43 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:45:45.592 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:45 smithi107 ceph-mon[28739]: pgmap v66: 65 pgs: 65 active+clean; 256 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 401 KiB/s rd, 422 KiB/s wr, 84 op/s 2024-03-04T13:45:45.609 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:45 smithi169 ceph-mon[36313]: pgmap v66: 65 pgs: 65 active+clean; 256 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 401 KiB/s rd, 422 KiB/s wr, 84 op/s 2024-03-04T13:45:47.092 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:46 smithi107 ceph-mon[28739]: pgmap v67: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 332 KiB/s rd, 342 KiB/s wr, 70 op/s 2024-03-04T13:45:47.109 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:46 smithi169 ceph-mon[36313]: pgmap v67: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 332 KiB/s rd, 342 KiB/s wr, 70 op/s 2024-03-04T13:45:49.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.127+0000 7f1a0c800700 1 -- 172.21.15.107:0/1116693330 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a040fe0e0 msgr2=0x7f1a040fe550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:45:49.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.127+0000 7f1a0c800700 1 --2- 172.21.15.107:0/1116693330 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a040fe0e0 0x7f1a040fe550 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f19f4004940 tx=0x7f19f4009e30 comp rx=0 tx=0).stop 2024-03-04T13:45:49.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.127+0000 7f1a0c800700 1 -- 172.21.15.107:0/1116693330 shutdown_connections 2024-03-04T13:45:49.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.128+0000 7f1a0c800700 1 --2- 172.21.15.107:0/1116693330 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a040fe0e0 0x7f1a040fe550 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:45:49.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.128+0000 7f1a0c800700 1 --2- 172.21.15.107:0/1116693330 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a040fcef0 0x7f1a040fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:45:49.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.128+0000 7f1a0c800700 1 -- 172.21.15.107:0/1116693330 >> 172.21.15.107:0/1116693330 conn(0x7f1a040f8620 msgr2=0x7f1a040faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:45:49.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.128+0000 7f1a0c800700 1 -- 172.21.15.107:0/1116693330 shutdown_connections 2024-03-04T13:45:49.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.128+0000 7f1a0c800700 1 -- 172.21.15.107:0/1116693330 wait complete. 2024-03-04T13:45:49.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.128+0000 7f1a0c800700 1 Processor -- start 2024-03-04T13:45:49.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.129+0000 7f1a0c800700 1 -- start start 2024-03-04T13:45:49.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.129+0000 7f1a0c800700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a040fcef0 0x7f1a0410fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:45:49.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.129+0000 7f1a0c800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a040fe0e0 0x7f1a041100d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:45:49.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.129+0000 7f1a0c800700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a04110760 con 0x7f1a040fe0e0 2024-03-04T13:45:49.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.129+0000 7f1a0c800700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a04110fa0 con 0x7f1a040fcef0 2024-03-04T13:45:49.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.129+0000 7f1a09400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a040fe0e0 0x7f1a041100d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:45:49.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.129+0000 7f1a09400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a040fe0e0 0x7f1a041100d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55244/0 (socket says 172.21.15.107:55244) 2024-03-04T13:45:49.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.129+0000 7f1a09400700 1 -- 172.21.15.107:0/4072668215 learned_addr learned my addr 172.21.15.107:0/4072668215 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:45:49.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.130+0000 7f1a09e00700 1 --2- 172.21.15.107:0/4072668215 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a040fcef0 0x7f1a0410fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:45:49.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.130+0000 7f1a09400700 1 -- 172.21.15.107:0/4072668215 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a040fcef0 msgr2=0x7f1a0410fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:45:49.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.130+0000 7f1a09400700 1 --2- 172.21.15.107:0/4072668215 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a040fcef0 0x7f1a0410fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:45:49.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.130+0000 7f1a09400700 1 -- 172.21.15.107:0/4072668215 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19f4005040 con 0x7f1a040fe0e0 2024-03-04T13:45:49.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.130+0000 7f1a09e00700 1 --2- 172.21.15.107:0/4072668215 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a040fcef0 0x7f1a0410fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T13:45:49.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.130+0000 7f1a09400700 1 --2- 172.21.15.107:0/4072668215 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a040fe0e0 0x7f1a041100d0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f19f40093b0 tx=0x7f19f4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:45:49.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.130+0000 7f19fe000700 1 -- 172.21.15.107:0/4072668215 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f19f400e040 con 0x7f1a040fe0e0 2024-03-04T13:45:49.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.130+0000 7f19fe000700 1 -- 172.21.15.107:0/4072668215 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f19f400bcc0 con 0x7f1a040fe0e0 2024-03-04T13:45:49.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.130+0000 7f19fe000700 1 -- 172.21.15.107:0/4072668215 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f19f401c630 con 0x7f1a040fe0e0 2024-03-04T13:45:49.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.130+0000 7f1a0c800700 1 -- 172.21.15.107:0/4072668215 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a04111220 con 0x7f1a040fe0e0 2024-03-04T13:45:49.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.131+0000 7f1a0c800700 1 -- 172.21.15.107:0/4072668215 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a04111760 con 0x7f1a040fe0e0 2024-03-04T13:45:49.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.132+0000 7f1a0c800700 1 -- 172.21.15.107:0/4072668215 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a04102210 con 0x7f1a040fe0e0 2024-03-04T13:45:49.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.133+0000 7f19fe000700 1 -- 172.21.15.107:0/4072668215 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 33) v1 ==== 91199+0+0 (secure 0 0 0) 0x7f19f40139f0 con 0x7f1a040fe0e0 2024-03-04T13:45:49.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.134+0000 7f19fe000700 1 --2- 172.21.15.107:0/4072668215 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f19ec06d0e0 0x7f19ec06f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:45:49.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.135+0000 7f19fe000700 1 -- 172.21.15.107:0/4072668215 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f19f408da50 con 0x7f1a040fe0e0 2024-03-04T13:45:49.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.135+0000 7f1a09e00700 1 --2- 172.21.15.107:0/4072668215 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f19ec06d0e0 0x7f19ec06f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:45:49.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.136+0000 7f1a09e00700 1 --2- 172.21.15.107:0/4072668215 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f19ec06d0e0 0x7f19ec06f590 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f19f8006fd0 tx=0x7f19f8008040 comp rx=0 tx=0).ready entity=mgr.14696 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:45:49.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.140+0000 7f19fe000700 1 -- 172.21.15.107:0/4072668215 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f19f405b600 con 0x7f1a040fe0e0 2024-03-04T13:45:49.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.392+0000 7f1a0c800700 1 -- 172.21.15.107:0/4072668215 --> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1a04111b50 con 0x7f19ec06d0e0 2024-03-04T13:45:49.392 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:49 smithi107 ceph-mon[28739]: pgmap v68: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 297 KiB/s rd, 284 KiB/s wr, 53 op/s 2024-03-04T13:45:49.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.394+0000 7f19fe000700 1 -- 172.21.15.107:0/4072668215 <== mgr.14696 v2:172.21.15.107:6800/1613321046 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+369 (secure 0 0 0) 0x7f1a04111b50 con 0x7f19ec06d0e0 2024-03-04T13:45:49.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.397+0000 7f1a0c800700 1 -- 172.21.15.107:0/4072668215 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f19ec06d0e0 msgr2=0x7f19ec06f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:45:49.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.397+0000 7f1a0c800700 1 --2- 172.21.15.107:0/4072668215 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f19ec06d0e0 0x7f19ec06f590 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f19f8006fd0 tx=0x7f19f8008040 comp rx=0 tx=0).stop 2024-03-04T13:45:49.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.397+0000 7f1a0c800700 1 -- 172.21.15.107:0/4072668215 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a040fe0e0 msgr2=0x7f1a041100d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:45:49.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.397+0000 7f1a0c800700 1 --2- 172.21.15.107:0/4072668215 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a040fe0e0 0x7f1a041100d0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f19f40093b0 tx=0x7f19f4009e30 comp rx=0 tx=0).stop 2024-03-04T13:45:49.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.397+0000 7f1a0c800700 1 -- 172.21.15.107:0/4072668215 shutdown_connections 2024-03-04T13:45:49.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.397+0000 7f1a0c800700 1 --2- 172.21.15.107:0/4072668215 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a040fcef0 0x7f1a0410fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:45:49.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.397+0000 7f1a0c800700 1 --2- 172.21.15.107:0/4072668215 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f19ec06d0e0 0x7f19ec06f590 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:45:49.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.397+0000 7f1a0c800700 1 --2- 172.21.15.107:0/4072668215 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a040fe0e0 0x7f1a041100d0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:45:49.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.397+0000 7f1a0c800700 1 -- 172.21.15.107:0/4072668215 >> 172.21.15.107:0/4072668215 conn(0x7f1a040f8620 msgr2=0x7f1a040ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:45:49.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.397+0000 7f1a0c800700 1 -- 172.21.15.107:0/4072668215 shutdown_connections 2024-03-04T13:45:49.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.398+0000 7f1a0c800700 1 -- 172.21.15.107:0/4072668215 wait complete. 2024-03-04T13:45:49.412 INFO:teuthology.orchestra.run.smithi107.stdout:true 2024-03-04T13:45:49.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.550+0000 7f164bc00700 1 -- 172.21.15.107:0/2551178869 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1644071f10 msgr2=0x7f1644070380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:45:49.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.550+0000 7f164bc00700 1 --2- 172.21.15.107:0/2551178869 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1644071f10 0x7f1644070380 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f1638004940 tx=0x7f1638009e30 comp rx=0 tx=0).stop 2024-03-04T13:45:49.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.550+0000 7f164bc00700 1 -- 172.21.15.107:0/2551178869 shutdown_connections 2024-03-04T13:45:49.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.550+0000 7f164bc00700 1 --2- 172.21.15.107:0/2551178869 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16440708c0 0x7f1644070d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:45:49.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.550+0000 7f164bc00700 1 --2- 172.21.15.107:0/2551178869 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1644071f10 0x7f1644070380 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:45:49.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.550+0000 7f164bc00700 1 -- 172.21.15.107:0/2551178869 >> 172.21.15.107:0/2551178869 conn(0x7f16440f8620 msgr2=0x7f16440faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:45:49.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.550+0000 7f164bc00700 1 -- 172.21.15.107:0/2551178869 shutdown_connections 2024-03-04T13:45:49.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.550+0000 7f164bc00700 1 -- 172.21.15.107:0/2551178869 wait complete. 2024-03-04T13:45:49.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.551+0000 7f164bc00700 1 Processor -- start 2024-03-04T13:45:49.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.551+0000 7f164bc00700 1 -- start start 2024-03-04T13:45:49.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.551+0000 7f164bc00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16440708c0 0x7f164410fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:45:49.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.551+0000 7f164bc00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1644071f10 0x7f16441100d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:45:49.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.551+0000 7f164bc00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1644110760 con 0x7f1644071f10 2024-03-04T13:45:49.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.551+0000 7f164bc00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1644110fa0 con 0x7f16440708c0 2024-03-04T13:45:49.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.552+0000 7f1643e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1644071f10 0x7f16441100d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:45:49.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.552+0000 7f1643e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1644071f10 0x7f16441100d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55266/0 (socket says 172.21.15.107:55266) 2024-03-04T13:45:49.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.552+0000 7f1643e00700 1 -- 172.21.15.107:0/2048324955 learned_addr learned my addr 172.21.15.107:0/2048324955 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:45:49.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.552+0000 7f1649200700 1 --2- 172.21.15.107:0/2048324955 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16440708c0 0x7f164410fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:45:49.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.552+0000 7f1643e00700 1 -- 172.21.15.107:0/2048324955 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16440708c0 msgr2=0x7f164410fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:45:49.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.552+0000 7f1643e00700 1 --2- 172.21.15.107:0/2048324955 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16440708c0 0x7f164410fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:45:49.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.552+0000 7f1643e00700 1 -- 172.21.15.107:0/2048324955 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1638005040 con 0x7f1644071f10 2024-03-04T13:45:49.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.553+0000 7f1643e00700 1 --2- 172.21.15.107:0/2048324955 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1644071f10 0x7f16441100d0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f163000b790 tx=0x7f1630009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:45:49.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.553+0000 7f1640c00700 1 -- 172.21.15.107:0/2048324955 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f163000f770 con 0x7f1644071f10 2024-03-04T13:45:49.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.553+0000 7f1640c00700 1 -- 172.21.15.107:0/2048324955 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f163000fda0 con 0x7f1644071f10 2024-03-04T13:45:49.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.553+0000 7f164bc00700 1 -- 172.21.15.107:0/2048324955 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1644111280 con 0x7f1644071f10 2024-03-04T13:45:49.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.553+0000 7f1640c00700 1 -- 172.21.15.107:0/2048324955 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f163000d310 con 0x7f1644071f10 2024-03-04T13:45:49.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.553+0000 7f164bc00700 1 -- 172.21.15.107:0/2048324955 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16441117c0 con 0x7f1644071f10 2024-03-04T13:45:49.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.554+0000 7f164bc00700 1 -- 172.21.15.107:0/2048324955 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f164404bed0 con 0x7f1644071f10 2024-03-04T13:45:49.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.555+0000 7f1640c00700 1 -- 172.21.15.107:0/2048324955 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 33) v1 ==== 91199+0+0 (secure 0 0 0) 0x7f1630010420 con 0x7f1644071f10 2024-03-04T13:45:49.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.556+0000 7f1640c00700 1 --2- 172.21.15.107:0/2048324955 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f162c06d0f0 0x7f162c06f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:45:49.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.557+0000 7f1640c00700 1 -- 172.21.15.107:0/2048324955 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f163008cc60 con 0x7f1644071f10 2024-03-04T13:45:49.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.557+0000 7f1649200700 1 --2- 172.21.15.107:0/2048324955 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f162c06d0f0 0x7f162c06f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:45:49.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.557+0000 7f1649200700 1 --2- 172.21.15.107:0/2048324955 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f162c06d0f0 0x7f162c06f5a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1638004f90 tx=0x7f1638009e30 comp rx=0 tx=0).ready entity=mgr.14696 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:45:49.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.560+0000 7f1640c00700 1 -- 172.21.15.107:0/2048324955 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f163005a890 con 0x7f1644071f10 2024-03-04T13:45:49.609 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:49 smithi169 ceph-mon[36313]: pgmap v68: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 297 KiB/s rd, 284 KiB/s wr, 53 op/s 2024-03-04T13:45:49.790 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.790+0000 7f164bc00700 1 -- 172.21.15.107:0/2048324955 --> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1644111dd0 con 0x7f162c06d0f0 2024-03-04T13:45:49.791 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.792+0000 7f1640c00700 1 -- 172.21.15.107:0/2048324955 <== mgr.14696 v2:172.21.15.107:6800/1613321046 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+369 (secure 0 0 0) 0x7f1644111dd0 con 0x7f162c06d0f0 2024-03-04T13:45:49.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.795+0000 7f164bc00700 1 -- 172.21.15.107:0/2048324955 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f162c06d0f0 msgr2=0x7f162c06f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:45:49.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.795+0000 7f164bc00700 1 --2- 172.21.15.107:0/2048324955 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f162c06d0f0 0x7f162c06f5a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1638004f90 tx=0x7f1638009e30 comp rx=0 tx=0).stop 2024-03-04T13:45:49.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.795+0000 7f164bc00700 1 -- 172.21.15.107:0/2048324955 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1644071f10 msgr2=0x7f16441100d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:45:49.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.795+0000 7f164bc00700 1 --2- 172.21.15.107:0/2048324955 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1644071f10 0x7f16441100d0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f163000b790 tx=0x7f1630009250 comp rx=0 tx=0).stop 2024-03-04T13:45:49.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.795+0000 7f164bc00700 1 -- 172.21.15.107:0/2048324955 shutdown_connections 2024-03-04T13:45:49.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.795+0000 7f164bc00700 1 --2- 172.21.15.107:0/2048324955 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16440708c0 0x7f164410fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:45:49.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.795+0000 7f164bc00700 1 --2- 172.21.15.107:0/2048324955 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f162c06d0f0 0x7f162c06f5a0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:45:49.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.795+0000 7f164bc00700 1 --2- 172.21.15.107:0/2048324955 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1644071f10 0x7f16441100d0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:45:49.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.795+0000 7f164bc00700 1 -- 172.21.15.107:0/2048324955 >> 172.21.15.107:0/2048324955 conn(0x7f16440f8620 msgr2=0x7f1644102dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:45:49.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.795+0000 7f164bc00700 1 -- 172.21.15.107:0/2048324955 shutdown_connections 2024-03-04T13:45:49.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.795+0000 7f164bc00700 1 -- 172.21.15.107:0/2048324955 wait complete. 2024-03-04T13:45:49.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.949+0000 7fb08b200700 1 -- 172.21.15.107:0/769884682 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb08c071f10 msgr2=0x7fb08c070380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:45:49.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.949+0000 7fb08b200700 1 --2- 172.21.15.107:0/769884682 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb08c071f10 0x7fb08c070380 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fb07c004940 tx=0x7fb07c009e30 comp rx=0 tx=0).stop 2024-03-04T13:45:49.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.949+0000 7fb08b200700 1 -- 172.21.15.107:0/769884682 shutdown_connections 2024-03-04T13:45:49.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.949+0000 7fb08b200700 1 --2- 172.21.15.107:0/769884682 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb08c0708c0 0x7fb08c070d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:45:49.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.949+0000 7fb08b200700 1 --2- 172.21.15.107:0/769884682 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb08c071f10 0x7fb08c070380 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:45:49.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.949+0000 7fb08b200700 1 -- 172.21.15.107:0/769884682 >> 172.21.15.107:0/769884682 conn(0x7fb08c0f83e0 msgr2=0x7fb08c0fa830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:45:49.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.949+0000 7fb08b200700 1 -- 172.21.15.107:0/769884682 shutdown_connections 2024-03-04T13:45:49.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.949+0000 7fb08b200700 1 -- 172.21.15.107:0/769884682 wait complete. 2024-03-04T13:45:49.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.951+0000 7fb08b200700 1 Processor -- start 2024-03-04T13:45:49.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.951+0000 7fb08b200700 1 -- start start 2024-03-04T13:45:49.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.951+0000 7fb08b200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb08c0708c0 0x7fb08c113d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:45:49.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.951+0000 7fb08b200700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb08c071f10 0x7fb08c1142a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:45:49.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.951+0000 7fb08b200700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb08c115060 con 0x7fb08c0708c0 2024-03-04T13:45:49.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.951+0000 7fb08b200700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb08c1151d0 con 0x7fb08c071f10 2024-03-04T13:45:49.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.952+0000 7fb089e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb08c0708c0 0x7fb08c113d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:45:49.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.952+0000 7fb089e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb08c0708c0 0x7fb08c113d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55298/0 (socket says 172.21.15.107:55298) 2024-03-04T13:45:49.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.952+0000 7fb089e00700 1 -- 172.21.15.107:0/2339794961 learned_addr learned my addr 172.21.15.107:0/2339794961 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:45:49.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.952+0000 7fb089400700 1 --2- 172.21.15.107:0/2339794961 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb08c071f10 0x7fb08c1142a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:45:49.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.952+0000 7fb089e00700 1 -- 172.21.15.107:0/2339794961 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb08c071f10 msgr2=0x7fb08c1142a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:45:49.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.952+0000 7fb089e00700 1 --2- 172.21.15.107:0/2339794961 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb08c071f10 0x7fb08c1142a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:45:49.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.952+0000 7fb089e00700 1 -- 172.21.15.107:0/2339794961 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb07c005040 con 0x7fb08c0708c0 2024-03-04T13:45:49.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.953+0000 7fb089e00700 1 --2- 172.21.15.107:0/2339794961 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb08c0708c0 0x7fb08c113d60 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fb07c00b3b0 tx=0x7fb07c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:45:49.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.953+0000 7fb082000700 1 -- 172.21.15.107:0/2339794961 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb07c00e040 con 0x7fb08c0708c0 2024-03-04T13:45:49.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.953+0000 7fb082000700 1 -- 172.21.15.107:0/2339794961 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb07c00bd90 con 0x7fb08c0708c0 2024-03-04T13:45:49.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.953+0000 7fb082000700 1 -- 172.21.15.107:0/2339794961 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb07c01c630 con 0x7fb08c0708c0 2024-03-04T13:45:49.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.953+0000 7fb08b200700 1 -- 172.21.15.107:0/2339794961 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb08c1153f0 con 0x7fb08c0708c0 2024-03-04T13:45:49.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.954+0000 7fb08b200700 1 -- 172.21.15.107:0/2339794961 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb08c115960 con 0x7fb08c0708c0 2024-03-04T13:45:49.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.957+0000 7fb082000700 1 -- 172.21.15.107:0/2339794961 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 33) v1 ==== 91199+0+0 (secure 0 0 0) 0x7fb07c01c8f0 con 0x7fb08c0708c0 2024-03-04T13:45:49.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.958+0000 7fb08b200700 1 -- 172.21.15.107:0/2339794961 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb08c04bed0 con 0x7fb08c0708c0 2024-03-04T13:45:49.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.958+0000 7fb082000700 1 --2- 172.21.15.107:0/2339794961 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7fb07006d0f0 0x7fb07006f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:45:49.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.958+0000 7fb082000700 1 -- 172.21.15.107:0/2339794961 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fb07c08e990 con 0x7fb08c0708c0 2024-03-04T13:45:49.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.962+0000 7fb089400700 1 --2- 172.21.15.107:0/2339794961 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7fb07006d0f0 0x7fb07006f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:45:49.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.963+0000 7fb089400700 1 --2- 172.21.15.107:0/2339794961 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7fb07006d0f0 0x7fb07006f5a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb078004f40 tx=0x7fb07800b040 comp rx=0 tx=0).ready entity=mgr.14696 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:45:49.968 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:49.969+0000 7fb082000700 1 -- 172.21.15.107:0/2339794961 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7fb07c058a50 con 0x7fb08c0708c0 2024-03-04T13:45:50.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:50.198+0000 7fb08b200700 1 -- 172.21.15.107:0/2339794961 --> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fb08c0ffea0 con 0x7fb07006d0f0 2024-03-04T13:45:50.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:50.209+0000 7fb082000700 1 -- 172.21.15.107:0/2339794961 <== mgr.14696 v2:172.21.15.107:6800/1613321046 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fb08c0ffea0 con 0x7fb07006d0f0 2024-03-04T13:45:50.209 INFO:teuthology.orchestra.run.smithi107.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-04T13:45:50.209 INFO:teuthology.orchestra.run.smithi107.stdout:alertmanager.smithi107 smithi107 *:9093,9094 running (8s) 0s ago 14m 13.7M - 0.25.0 c8568f914cd2 e5ed7be32538 2024-03-04T13:45:50.209 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi107 smithi107 running (14m) 0s ago 14m 6665k - 17.2.7-545-g42608bb1 9eec3f22859d 03e1570c2eff 2024-03-04T13:45:50.209 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi169 smithi169 running (13m) 57s ago 13m 6656k - 17.2.7-545-g42608bb1 9eec3f22859d 9c8881a5338e 2024-03-04T13:45:50.209 INFO:teuthology.orchestra.run.smithi107.stdout:grafana.smithi107 smithi107 *:3000 running (3m) 0s ago 13m 72.7M - 9.4.7 954c08fa6188 bd720c203473 2024-03-04T13:45:50.209 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.riexmm smithi107 running (7m) 0s ago 7m 13.1M - 17.2.7-545-g42608bb1 9eec3f22859d fc921f7d4328 2024-03-04T13:45:50.209 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.thycho smithi107 running (7m) 0s ago 7m 129M - 17.2.7-545-g42608bb1 9eec3f22859d 4069c6e0e063 2024-03-04T13:45:50.209 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi169.bxitzk smithi169 running (7m) 57s ago 7m 13.0M - 17.2.7-545-g42608bb1 9eec3f22859d 7d4115228c7c 2024-03-04T13:45:50.209 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi169.jusudl smithi169 running (7m) 57s ago 7m 210M - 17.2.7-545-g42608bb1 9eec3f22859d 2d2f2cc452e8 2024-03-04T13:45:50.210 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi107.wpzgwl smithi107 *:8443,9283,8765 running (2m) 0s ago 15m 500M - 19.0.0-1845-g717ce3cc d71b839abdf8 1c5fdee45eac 2024-03-04T13:45:50.210 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi169.eihczr smithi169 *:8443,9283,8765 running (95s) 57s ago 12m 447M - 19.0.0-1845-g717ce3cc d71b839abdf8 b5cac2b7d22a 2024-03-04T13:45:50.210 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi107 smithi107 running (16m) 0s ago 16m 53.0M 2048M 17.2.7-545-g42608bb1 9eec3f22859d 49f5da990d35 2024-03-04T13:45:50.210 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi169 smithi169 running (12m) 57s ago 12m 42.4M 2048M 17.2.7-545-g42608bb1 9eec3f22859d ba4ee916297f 2024-03-04T13:45:50.210 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi107 smithi107 *:9100 running (74s) 0s ago 14m 7356k - 1.5.0 0da6a335fe13 8a8d3d4f724f 2024-03-04T13:45:50.210 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi169 smithi169 *:9100 running (64s) 57s ago 13m 2935k - 1.5.0 0da6a335fe13 f240f5a975bc 2024-03-04T13:45:50.210 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0 smithi107 running (11m) 0s ago 11m 205M 1558M 17.2.7-545-g42608bb1 9eec3f22859d c0605b68f696 2024-03-04T13:45:50.210 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1 smithi107 running (11m) 0s ago 11m 218M 1558M 17.2.7-545-g42608bb1 9eec3f22859d 5fb2c0dd5769 2024-03-04T13:45:50.210 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2 smithi107 running (10m) 0s ago 10m 180M 1558M 17.2.7-545-g42608bb1 9eec3f22859d 61f30f643549 2024-03-04T13:45:50.210 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3 smithi169 running (10m) 57s ago 10m 222M 2582M 17.2.7-545-g42608bb1 9eec3f22859d cd91d8694764 2024-03-04T13:45:50.210 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4 smithi169 running (9m) 57s ago 9m 194M 2582M 17.2.7-545-g42608bb1 9eec3f22859d 08545e3d0659 2024-03-04T13:45:50.211 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5 smithi169 running (9m) 57s ago 9m 180M 2582M 17.2.7-545-g42608bb1 9eec3f22859d dcffe6cb3eac 2024-03-04T13:45:50.211 INFO:teuthology.orchestra.run.smithi107.stdout:prometheus.smithi107 smithi107 *:9095 running (34s) 0s ago 13m 45.5M - 2.43.0 a07b618ecd1d 1eeb1ad5cb75 2024-03-04T13:45:50.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:50.212+0000 7fb08b200700 1 -- 172.21.15.107:0/2339794961 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7fb07006d0f0 msgr2=0x7fb07006f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:45:50.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:50.212+0000 7fb08b200700 1 --2- 172.21.15.107:0/2339794961 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7fb07006d0f0 0x7fb07006f5a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb078004f40 tx=0x7fb07800b040 comp rx=0 tx=0).stop 2024-03-04T13:45:50.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:50.212+0000 7fb08b200700 1 -- 172.21.15.107:0/2339794961 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb08c0708c0 msgr2=0x7fb08c113d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:45:50.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:50.212+0000 7fb08b200700 1 --2- 172.21.15.107:0/2339794961 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb08c0708c0 0x7fb08c113d60 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fb07c00b3b0 tx=0x7fb07c009e30 comp rx=0 tx=0).stop 2024-03-04T13:45:50.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:50.212+0000 7fb08b200700 1 -- 172.21.15.107:0/2339794961 shutdown_connections 2024-03-04T13:45:50.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:50.212+0000 7fb08b200700 1 --2- 172.21.15.107:0/2339794961 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7fb07006d0f0 0x7fb07006f5a0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:45:50.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:50.212+0000 7fb08b200700 1 --2- 172.21.15.107:0/2339794961 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb08c0708c0 0x7fb08c113d60 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:45:50.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:50.212+0000 7fb08b200700 1 --2- 172.21.15.107:0/2339794961 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb08c071f10 0x7fb08c1142a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:45:50.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:50.212+0000 7fb08b200700 1 -- 172.21.15.107:0/2339794961 >> 172.21.15.107:0/2339794961 conn(0x7fb08c0f83e0 msgr2=0x7fb08c0fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:45:50.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:50.212+0000 7fb08b200700 1 -- 172.21.15.107:0/2339794961 shutdown_connections 2024-03-04T13:45:50.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:50.212+0000 7fb08b200700 1 -- 172.21.15.107:0/2339794961 wait complete. 2024-03-04T13:45:50.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:50.349+0000 7ff615600700 1 -- 172.21.15.107:0/1880208167 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6100fe0e0 msgr2=0x7ff6100fe550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:45:50.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:50.349+0000 7ff615600700 1 --2- 172.21.15.107:0/1880208167 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6100fe0e0 0x7ff6100fe550 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc004940 tx=0x7ff5fc009e30 comp rx=0 tx=0).stop 2024-03-04T13:45:50.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:50.350+0000 7ff615600700 1 -- 172.21.15.107:0/1880208167 shutdown_connections 2024-03-04T13:45:50.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:50.350+0000 7ff615600700 1 --2- 172.21.15.107:0/1880208167 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6100fe0e0 0x7ff6100fe550 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:45:50.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:50.350+0000 7ff615600700 1 --2- 172.21.15.107:0/1880208167 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff6100fcef0 0x7ff6100fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:45:50.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:50.350+0000 7ff615600700 1 -- 172.21.15.107:0/1880208167 >> 172.21.15.107:0/1880208167 conn(0x7ff6100f8620 msgr2=0x7ff6100faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:45:50.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:50.350+0000 7ff615600700 1 -- 172.21.15.107:0/1880208167 shutdown_connections 2024-03-04T13:45:50.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:50.350+0000 7ff615600700 1 -- 172.21.15.107:0/1880208167 wait complete. 2024-03-04T13:45:50.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:50.351+0000 7ff615600700 1 Processor -- start 2024-03-04T13:45:50.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:50.351+0000 7ff615600700 1 -- start start 2024-03-04T13:45:50.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:50.351+0000 7ff615600700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff6100fcef0 0x7ff61010d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:45:50.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:50.351+0000 7ff615600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6100fe0e0 0x7ff61010deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:45:50.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:50.351+0000 7ff615600700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff61010e540 con 0x7ff6100fe0e0 2024-03-04T13:45:50.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:50.351+0000 7ff615600700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff61010ed80 con 0x7ff6100fcef0 2024-03-04T13:45:50.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:50.351+0000 7ff60e000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6100fe0e0 0x7ff61010deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:45:50.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:50.351+0000 7ff60e000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6100fe0e0 0x7ff61010deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55302/0 (socket says 172.21.15.107:55302) 2024-03-04T13:45:50.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:50.351+0000 7ff60e000700 1 -- 172.21.15.107:0/1091156407 learned_addr learned my addr 172.21.15.107:0/1091156407 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:45:50.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:50.352+0000 7ff60e000700 1 -- 172.21.15.107:0/1091156407 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff6100fcef0 msgr2=0x7ff61010d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:45:50.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:50.352+0000 7ff60ea00700 1 --2- 172.21.15.107:0/1091156407 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff6100fcef0 0x7ff61010d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:45:50.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:50.352+0000 7ff60e000700 1 --2- 172.21.15.107:0/1091156407 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff6100fcef0 0x7ff61010d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:45:50.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:50.352+0000 7ff60e000700 1 -- 172.21.15.107:0/1091156407 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5fc005040 con 0x7ff6100fe0e0 2024-03-04T13:45:50.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:50.352+0000 7ff60ea00700 1 --2- 172.21.15.107:0/1091156407 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff6100fcef0 0x7ff61010d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T13:45:50.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:50.352+0000 7ff60e000700 1 --2- 172.21.15.107:0/1091156407 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6100fe0e0 0x7ff61010deb0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc00b3b0 tx=0x7ff5fc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:45:50.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:50.352+0000 7ff606a00700 1 -- 172.21.15.107:0/1091156407 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff5fc00e040 con 0x7ff6100fe0e0 2024-03-04T13:45:50.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:50.353+0000 7ff606a00700 1 -- 172.21.15.107:0/1091156407 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff5fc00bd90 con 0x7ff6100fe0e0 2024-03-04T13:45:50.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:50.353+0000 7ff606a00700 1 -- 172.21.15.107:0/1091156407 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff5fc01c630 con 0x7ff6100fe0e0 2024-03-04T13:45:50.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:50.353+0000 7ff615600700 1 -- 172.21.15.107:0/1091156407 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff61010f000 con 0x7ff6100fe0e0 2024-03-04T13:45:50.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:50.353+0000 7ff615600700 1 -- 172.21.15.107:0/1091156407 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff61010f540 con 0x7ff6100fe0e0 2024-03-04T13:45:50.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:50.354+0000 7ff615600700 1 -- 172.21.15.107:0/1091156407 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff61004bed0 con 0x7ff6100fe0e0 2024-03-04T13:45:50.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:50.355+0000 7ff606a00700 1 -- 172.21.15.107:0/1091156407 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 33) v1 ==== 91199+0+0 (secure 0 0 0) 0x7ff5fc0139f0 con 0x7ff6100fe0e0 2024-03-04T13:45:50.355 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:50.356+0000 7ff606a00700 1 --2- 172.21.15.107:0/1091156407 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7ff5f406d0f0 0x7ff5f406f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:45:50.355 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:50.356+0000 7ff606a00700 1 -- 172.21.15.107:0/1091156407 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7ff5fc08da00 con 0x7ff6100fe0e0 2024-03-04T13:45:50.356 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:50.356+0000 7ff60ea00700 1 --2- 172.21.15.107:0/1091156407 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7ff5f406d0f0 0x7ff5f406f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:45:50.356 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:50.357+0000 7ff60ea00700 1 --2- 172.21.15.107:0/1091156407 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7ff5f406d0f0 0x7ff5f406f5a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff600006fd0 tx=0x7ff600008040 comp rx=0 tx=0).ready entity=mgr.14696 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:45:50.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:50.360+0000 7ff606a00700 1 -- 172.21.15.107:0/1091156407 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7ff5fc05b670 con 0x7ff6100fe0e0 2024-03-04T13:45:50.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:50.678+0000 7ff615600700 1 -- 172.21.15.107:0/1091156407 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff610105ad0 con 0x7ff6100fe0e0 2024-03-04T13:45:50.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:50.678+0000 7ff606a00700 1 -- 172.21.15.107:0/1091156407 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7ff5fc05b200 con 0x7ff6100fe0e0 2024-03-04T13:45:50.678 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-03-04T13:45:50.678 INFO:teuthology.orchestra.run.smithi107.stdout: "mon": { 2024-03-04T13:45:50.678 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-545-g42608bb1 (42608bb191117be10dba720e6bf8a02f3171cd34) quincy (stable)": 2 2024-03-04T13:45:50.678 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:45:50.678 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr": { 2024-03-04T13:45:50.678 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-1845-g717ce3cc (717ce3cce8d0166e4e577faa8004e6f9cb4128c0) squid (dev)": 2 2024-03-04T13:45:50.678 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:45:50.679 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": { 2024-03-04T13:45:50.679 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-545-g42608bb1 (42608bb191117be10dba720e6bf8a02f3171cd34) quincy (stable)": 6 2024-03-04T13:45:50.679 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:45:50.679 INFO:teuthology.orchestra.run.smithi107.stdout: "mds": { 2024-03-04T13:45:50.679 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-545-g42608bb1 (42608bb191117be10dba720e6bf8a02f3171cd34) quincy (stable)": 4 2024-03-04T13:45:50.679 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:45:50.679 INFO:teuthology.orchestra.run.smithi107.stdout: "overall": { 2024-03-04T13:45:50.679 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-545-g42608bb1 (42608bb191117be10dba720e6bf8a02f3171cd34) quincy (stable)": 12, 2024-03-04T13:45:50.679 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-1845-g717ce3cc (717ce3cce8d0166e4e577faa8004e6f9cb4128c0) squid (dev)": 2 2024-03-04T13:45:50.679 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-04T13:45:50.679 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-03-04T13:45:50.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:50.681+0000 7ff615600700 1 -- 172.21.15.107:0/1091156407 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7ff5f406d0f0 msgr2=0x7ff5f406f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:45:50.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:50.681+0000 7ff615600700 1 --2- 172.21.15.107:0/1091156407 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7ff5f406d0f0 0x7ff5f406f5a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff600006fd0 tx=0x7ff600008040 comp rx=0 tx=0).stop 2024-03-04T13:45:50.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:50.681+0000 7ff615600700 1 -- 172.21.15.107:0/1091156407 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6100fe0e0 msgr2=0x7ff61010deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:45:50.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:50.681+0000 7ff615600700 1 --2- 172.21.15.107:0/1091156407 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6100fe0e0 0x7ff61010deb0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc00b3b0 tx=0x7ff5fc009e30 comp rx=0 tx=0).stop 2024-03-04T13:45:50.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:50.681+0000 7ff615600700 1 -- 172.21.15.107:0/1091156407 shutdown_connections 2024-03-04T13:45:50.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:50.681+0000 7ff615600700 1 --2- 172.21.15.107:0/1091156407 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff6100fcef0 0x7ff61010d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:45:50.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:50.681+0000 7ff615600700 1 --2- 172.21.15.107:0/1091156407 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7ff5f406d0f0 0x7ff5f406f5a0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:45:50.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:50.681+0000 7ff615600700 1 --2- 172.21.15.107:0/1091156407 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6100fe0e0 0x7ff61010deb0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:45:50.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:50.681+0000 7ff615600700 1 -- 172.21.15.107:0/1091156407 >> 172.21.15.107:0/1091156407 conn(0x7ff6100f8620 msgr2=0x7ff6101013a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:45:50.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:50.681+0000 7ff615600700 1 -- 172.21.15.107:0/1091156407 shutdown_connections 2024-03-04T13:45:50.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:50.681+0000 7ff615600700 1 -- 172.21.15.107:0/1091156407 wait complete. 2024-03-04T13:45:50.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:50.820+0000 7f9be0a00700 1 -- 172.21.15.107:0/882767759 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9bdc0708c0 msgr2=0x7f9bdc070d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:45:50.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:50.820+0000 7f9be0a00700 1 --2- 172.21.15.107:0/882767759 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9bdc0708c0 0x7f9bdc070d10 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f9bc8004940 tx=0x7f9bc8009e30 comp rx=0 tx=0).stop 2024-03-04T13:45:50.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:50.820+0000 7f9be0a00700 1 -- 172.21.15.107:0/882767759 shutdown_connections 2024-03-04T13:45:50.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:50.820+0000 7f9be0a00700 1 --2- 172.21.15.107:0/882767759 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9bdc0708c0 0x7f9bdc070d10 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:45:50.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:50.820+0000 7f9be0a00700 1 --2- 172.21.15.107:0/882767759 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9bdc071f10 0x7f9bdc070380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:45:50.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:50.820+0000 7f9be0a00700 1 -- 172.21.15.107:0/882767759 >> 172.21.15.107:0/882767759 conn(0x7f9bdc0f8620 msgr2=0x7f9bdc0faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:45:50.820 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:50 smithi107 ceph-mon[28739]: from='client.14804 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:45:50.820 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:50 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:45:50.820 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:50 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:45:50.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:50.820+0000 7f9be0a00700 1 -- 172.21.15.107:0/882767759 shutdown_connections 2024-03-04T13:45:50.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:50.820+0000 7f9be0a00700 1 -- 172.21.15.107:0/882767759 wait complete. 2024-03-04T13:45:50.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:50.821+0000 7f9be0a00700 1 Processor -- start 2024-03-04T13:45:50.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:50.821+0000 7f9be0a00700 1 -- start start 2024-03-04T13:45:50.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:50.821+0000 7f9be0a00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9bdc0708c0 0x7f9bdc10fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:45:50.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:50.822+0000 7f9be0a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9bdc071f10 0x7f9bdc110130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:45:50.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:50.822+0000 7f9be0a00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bdc1107c0 con 0x7f9bdc071f10 2024-03-04T13:45:50.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:50.822+0000 7f9be0a00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bdc111000 con 0x7f9bdc0708c0 2024-03-04T13:45:50.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:50.822+0000 7f9bd9400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9bdc071f10 0x7f9bdc110130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:45:50.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:50.822+0000 7f9bd9400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9bdc071f10 0x7f9bdc110130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55324/0 (socket says 172.21.15.107:55324) 2024-03-04T13:45:50.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:50.822+0000 7f9bd9400700 1 -- 172.21.15.107:0/953603966 learned_addr learned my addr 172.21.15.107:0/953603966 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:45:50.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:50.822+0000 7f9bd9e00700 1 --2- 172.21.15.107:0/953603966 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9bdc0708c0 0x7f9bdc10fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:45:50.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:50.822+0000 7f9bd9400700 1 -- 172.21.15.107:0/953603966 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9bdc0708c0 msgr2=0x7f9bdc10fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:45:50.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:50.822+0000 7f9bd9400700 1 --2- 172.21.15.107:0/953603966 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9bdc0708c0 0x7f9bdc10fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:45:50.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:50.822+0000 7f9bd9400700 1 -- 172.21.15.107:0/953603966 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bc8005040 con 0x7f9bdc071f10 2024-03-04T13:45:50.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:50.822+0000 7f9bd9e00700 1 --2- 172.21.15.107:0/953603966 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9bdc0708c0 0x7f9bdc10fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T13:45:50.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:50.823+0000 7f9bd9400700 1 --2- 172.21.15.107:0/953603966 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9bdc071f10 0x7f9bdc110130 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f9bc8009380 tx=0x7f9bc8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:45:50.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:50.823+0000 7f9bd2000700 1 -- 172.21.15.107:0/953603966 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9bc800e040 con 0x7f9bdc071f10 2024-03-04T13:45:50.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:50.823+0000 7f9bd2000700 1 -- 172.21.15.107:0/953603966 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9bc800bc60 con 0x7f9bdc071f10 2024-03-04T13:45:50.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:50.823+0000 7f9be0a00700 1 -- 172.21.15.107:0/953603966 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bdc111280 con 0x7f9bdc071f10 2024-03-04T13:45:50.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:50.823+0000 7f9bd2000700 1 -- 172.21.15.107:0/953603966 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9bc801c630 con 0x7f9bdc071f10 2024-03-04T13:45:50.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:50.824+0000 7f9be0a00700 1 -- 172.21.15.107:0/953603966 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bdc1117c0 con 0x7f9bdc071f10 2024-03-04T13:45:50.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:50.825+0000 7f9be0a00700 1 -- 172.21.15.107:0/953603966 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bdc04bed0 con 0x7f9bdc071f10 2024-03-04T13:45:50.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:50.826+0000 7f9bd2000700 1 -- 172.21.15.107:0/953603966 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 33) v1 ==== 91199+0+0 (secure 0 0 0) 0x7f9bc80139f0 con 0x7f9bdc071f10 2024-03-04T13:45:50.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:50.827+0000 7f9bd2000700 1 --2- 172.21.15.107:0/953603966 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f9bc006d0f0 0x7f9bc006f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:45:50.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:50.827+0000 7f9bd2000700 1 -- 172.21.15.107:0/953603966 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f9bc808d0c0 con 0x7f9bdc071f10 2024-03-04T13:45:50.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:50.827+0000 7f9bd9e00700 1 --2- 172.21.15.107:0/953603966 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f9bc006d0f0 0x7f9bc006f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:45:50.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:50.828+0000 7f9bd9e00700 1 --2- 172.21.15.107:0/953603966 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f9bc006d0f0 0x7f9bc006f5a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9bcc006fd0 tx=0x7f9bcc008040 comp rx=0 tx=0).ready entity=mgr.14696 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:45:50.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:50.831+0000 7f9bd2000700 1 -- 172.21.15.107:0/953603966 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f9bc8092060 con 0x7f9bdc071f10 2024-03-04T13:45:50.859 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:50 smithi169 ceph-mon[36313]: from='client.14804 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:45:50.859 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:50 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:45:50.859 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:50 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:45:51.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:51.064+0000 7f9be0a00700 1 -- 172.21.15.107:0/953603966 --> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9bdc111d20 con 0x7f9bc006d0f0 2024-03-04T13:45:51.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:51.066+0000 7f9bd2000700 1 -- 172.21.15.107:0/953603966 <== mgr.14696 v2:172.21.15.107:6800/1613321046 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+369 (secure 0 0 0) 0x7f9bdc111d20 con 0x7f9bc006d0f0 2024-03-04T13:45:51.066 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-03-04T13:45:51.066 INFO:teuthology.orchestra.run.smithi107.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377", 2024-03-04T13:45:51.066 INFO:teuthology.orchestra.run.smithi107.stdout: "in_progress": true, 2024-03-04T13:45:51.066 INFO:teuthology.orchestra.run.smithi107.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-03-04T13:45:51.066 INFO:teuthology.orchestra.run.smithi107.stdout: "services_complete": [ 2024-03-04T13:45:51.066 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr" 2024-03-04T13:45:51.066 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-04T13:45:51.066 INFO:teuthology.orchestra.run.smithi107.stdout: "progress": "2/2 daemons upgraded", 2024-03-04T13:45:51.066 INFO:teuthology.orchestra.run.smithi107.stdout: "message": "Currently upgrading alertmanager daemons", 2024-03-04T13:45:51.066 INFO:teuthology.orchestra.run.smithi107.stdout: "is_paused": false 2024-03-04T13:45:51.067 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-03-04T13:45:51.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:51.069+0000 7f9be0a00700 1 -- 172.21.15.107:0/953603966 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f9bc006d0f0 msgr2=0x7f9bc006f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:45:51.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:51.069+0000 7f9be0a00700 1 --2- 172.21.15.107:0/953603966 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f9bc006d0f0 0x7f9bc006f5a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9bcc006fd0 tx=0x7f9bcc008040 comp rx=0 tx=0).stop 2024-03-04T13:45:51.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:51.069+0000 7f9be0a00700 1 -- 172.21.15.107:0/953603966 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9bdc071f10 msgr2=0x7f9bdc110130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:45:51.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:51.069+0000 7f9be0a00700 1 --2- 172.21.15.107:0/953603966 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9bdc071f10 0x7f9bdc110130 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f9bc8009380 tx=0x7f9bc8009e30 comp rx=0 tx=0).stop 2024-03-04T13:45:51.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:51.070+0000 7f9be0a00700 1 -- 172.21.15.107:0/953603966 shutdown_connections 2024-03-04T13:45:51.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:51.070+0000 7f9be0a00700 1 --2- 172.21.15.107:0/953603966 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9bdc0708c0 0x7f9bdc10fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:45:51.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:51.070+0000 7f9be0a00700 1 --2- 172.21.15.107:0/953603966 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f9bc006d0f0 0x7f9bc006f5a0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:45:51.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:51.070+0000 7f9be0a00700 1 --2- 172.21.15.107:0/953603966 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9bdc071f10 0x7f9bdc110130 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:45:51.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:51.070+0000 7f9be0a00700 1 -- 172.21.15.107:0/953603966 >> 172.21.15.107:0/953603966 conn(0x7f9bdc0f8620 msgr2=0x7f9bdc102dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:45:51.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:51.070+0000 7f9be0a00700 1 -- 172.21.15.107:0/953603966 shutdown_connections 2024-03-04T13:45:51.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:45:51.070+0000 7f9be0a00700 1 -- 172.21.15.107:0/953603966 wait complete. 2024-03-04T13:45:51.550 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:51 smithi169 ceph-mon[36313]: from='client.14808 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:45:51.550 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:51 smithi169 ceph-mon[36313]: pgmap v69: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 297 KiB/s rd, 284 KiB/s wr, 55 op/s 2024-03-04T13:45:51.550 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:51 smithi169 ceph-mon[36313]: from='client.14812 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:45:51.551 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:51 smithi169 ceph-mon[36313]: from='client.? 172.21.15.107:0/1091156407' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:45:51.551 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:51 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:45:51.551 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:51 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:45:51.592 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:51 smithi107 ceph-mon[28739]: from='client.14808 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:45:51.592 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:51 smithi107 ceph-mon[28739]: pgmap v69: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 297 KiB/s rd, 284 KiB/s wr, 55 op/s 2024-03-04T13:45:51.592 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:51 smithi107 ceph-mon[28739]: from='client.14812 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:45:51.592 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:51 smithi107 ceph-mon[28739]: from='client.? 172.21.15.107:0/1091156407' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:45:51.592 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:51 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:45:51.592 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:51 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:45:52.841 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:52 smithi107 ceph-mon[28739]: from='client.14820 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:45:52.859 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:52 smithi169 ceph-mon[36313]: from='client.14820 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:45:53.779 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:53 smithi107 ceph-mon[28739]: pgmap v70: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 82 KiB/s rd, 60 KiB/s wr, 18 op/s 2024-03-04T13:45:53.779 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:53 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:45:53.859 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:53 smithi169 ceph-mon[36313]: pgmap v70: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 82 KiB/s rd, 60 KiB/s wr, 18 op/s 2024-03-04T13:45:53.859 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:53 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:45:55.821 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:55 smithi107 ceph-mon[28739]: pgmap v71: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 82 KiB/s rd, 60 KiB/s wr, 18 op/s 2024-03-04T13:45:55.859 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:55 smithi169 ceph-mon[36313]: pgmap v71: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 82 KiB/s rd, 60 KiB/s wr, 18 op/s 2024-03-04T13:45:57.092 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:56 smithi107 ceph-mon[28739]: pgmap v72: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 79 KiB/s rd, 57 KiB/s wr, 12 op/s 2024-03-04T13:45:57.092 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:56 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:45:57.092 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:56 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:45:57.092 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:56 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:45:57.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:56 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:45:57.109 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:56 smithi169 ceph-mon[36313]: pgmap v72: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 79 KiB/s rd, 57 KiB/s wr, 12 op/s 2024-03-04T13:45:57.109 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:56 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:45:57.109 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:56 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:45:57.109 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:56 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:45:57.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:56 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:45:58.092 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:57 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:45:58.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:57 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-04T13:45:58.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:57 smithi107 ceph-mon[28739]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-04T13:45:58.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:57 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:45:58.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:57 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:45:58.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:57 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:45:58.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:57 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:45:58.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:57 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:45:58.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:57 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:45:58.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:57 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:45:58.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:57 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:45:58.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:57 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:45:58.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:57 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:45:58.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:57 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:45:58.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:57 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:45:58.109 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:57 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:45:58.109 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:57 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-04T13:45:58.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:57 smithi169 ceph-mon[36313]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-04T13:45:58.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:57 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:45:58.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:57 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:45:58.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:57 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:45:58.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:57 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:45:58.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:57 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:45:58.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:57 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:45:58.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:57 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:45:58.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:57 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:45:58.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:57 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:45:58.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:57 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:45:58.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:57 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:45:58.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:57 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:45:59.092 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:58 smithi107 ceph-mon[28739]: pgmap v73: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1023 B/s rd, 255 B/s wr, 2 op/s 2024-03-04T13:45:59.109 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:58 smithi169 ceph-mon[36313]: pgmap v73: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1023 B/s rd, 255 B/s wr, 2 op/s 2024-03-04T13:46:00.092 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:59 smithi107 ceph-mon[28739]: Upgrade: Updating grafana.smithi107 2024-03-04T13:46:00.092 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:45:59 smithi107 ceph-mon[28739]: Deploying daemon grafana.smithi107 on smithi107 2024-03-04T13:46:00.109 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:59 smithi169 ceph-mon[36313]: Upgrade: Updating grafana.smithi107 2024-03-04T13:46:00.109 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:45:59 smithi169 ceph-mon[36313]: Deploying daemon grafana.smithi107 on smithi107 2024-03-04T13:46:01.092 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:00 smithi107 ceph-mon[28739]: pgmap v74: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.4 KiB/s rd, 255 B/s wr, 3 op/s 2024-03-04T13:46:01.109 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:00 smithi169 ceph-mon[36313]: pgmap v74: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.4 KiB/s rd, 255 B/s wr, 3 op/s 2024-03-04T13:46:03.592 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:03 smithi107 ceph-mon[28739]: pgmap v75: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:46:03.609 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:03 smithi169 ceph-mon[36313]: pgmap v75: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:46:05.544 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:05 smithi107 ceph-mon[28739]: pgmap v76: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:46:05.609 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:05 smithi169 ceph-mon[36313]: pgmap v76: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:46:07.422 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:07 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:46:07.423 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:07 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:46:07.423 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:07 smithi107 ceph-mon[28739]: pgmap v77: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:46:07.423 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:07 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:46:07.610 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:07 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:46:07.610 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:07 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:46:07.610 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:07 smithi169 ceph-mon[36313]: pgmap v77: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:46:07.610 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:07 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:46:09.562 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:09 smithi107 ceph-mon[28739]: pgmap v78: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-03-04T13:46:09.562 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:09 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:46:09.609 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:09 smithi169 ceph-mon[36313]: pgmap v78: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-03-04T13:46:09.609 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:09 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:46:11.592 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:11 smithi107 ceph-mon[28739]: pgmap v79: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:46:11.609 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:11 smithi169 ceph-mon[36313]: pgmap v79: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:46:13.592 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:13 smithi107 ceph-mon[28739]: pgmap v80: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-03-04T13:46:13.609 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:13 smithi169 ceph-mon[36313]: pgmap v80: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-03-04T13:46:14.592 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:14 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:46:14.592 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:14 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:46:14.609 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:14 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:46:14.609 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:14 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:46:15.609 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:15 smithi169 ceph-mon[36313]: pgmap v81: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-03-04T13:46:15.842 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:15 smithi107 ceph-mon[28739]: pgmap v81: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-03-04T13:46:16.700 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:16 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:46:16.700 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:16 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:46:16.859 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:16 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:46:16.859 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:16 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:46:17.577 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:17 smithi107 ceph-mon[28739]: pgmap v82: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:46:17.859 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:17 smithi169 ceph-mon[36313]: pgmap v82: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:46:19.716 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:19 smithi107 ceph-mon[28739]: pgmap v83: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:46:19.859 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:19 smithi169 ceph-mon[36313]: pgmap v83: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:46:21.245 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:21.245+0000 7f1b80c00700 1 -- 172.21.15.107:0/1506030230 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b7c0fcef0 msgr2=0x7f1b7c0fd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:46:21.245 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:21.245+0000 7f1b80c00700 1 --2- 172.21.15.107:0/1506030230 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b7c0fcef0 0x7f1b7c0fd320 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c004940 tx=0x7f1b6c009e30 comp rx=0 tx=0).stop 2024-03-04T13:46:21.245 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:21.245+0000 7f1b80c00700 1 -- 172.21.15.107:0/1506030230 shutdown_connections 2024-03-04T13:46:21.245 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:21.245+0000 7f1b80c00700 1 --2- 172.21.15.107:0/1506030230 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b7c0fe0e0 0x7f1b7c0fe550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:46:21.245 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:21.245+0000 7f1b80c00700 1 --2- 172.21.15.107:0/1506030230 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b7c0fcef0 0x7f1b7c0fd320 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:46:21.245 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:21.245+0000 7f1b80c00700 1 -- 172.21.15.107:0/1506030230 >> 172.21.15.107:0/1506030230 conn(0x7f1b7c0f8620 msgr2=0x7f1b7c0faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:46:21.245 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:21.245+0000 7f1b80c00700 1 -- 172.21.15.107:0/1506030230 shutdown_connections 2024-03-04T13:46:21.246 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:21.245+0000 7f1b80c00700 1 -- 172.21.15.107:0/1506030230 wait complete. 2024-03-04T13:46:21.248 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:21.246+0000 7f1b80c00700 1 Processor -- start 2024-03-04T13:46:21.248 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:21.246+0000 7f1b80c00700 1 -- start start 2024-03-04T13:46:21.248 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:21.246+0000 7f1b80c00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b7c0fcef0 0x7f1b7c10fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:46:21.248 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:21.246+0000 7f1b80c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b7c0fe0e0 0x7f1b7c1100d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:46:21.248 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:21.246+0000 7f1b80c00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b7c1106d0 con 0x7f1b7c0fe0e0 2024-03-04T13:46:21.248 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:21.246+0000 7f1b80c00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b7c110f70 con 0x7f1b7c0fcef0 2024-03-04T13:46:21.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:21.247+0000 7f1b79400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b7c0fe0e0 0x7f1b7c1100d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:46:21.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:21.247+0000 7f1b79400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b7c0fe0e0 0x7f1b7c1100d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40266/0 (socket says 172.21.15.107:40266) 2024-03-04T13:46:21.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:21.247+0000 7f1b79400700 1 -- 172.21.15.107:0/1328340563 learned_addr learned my addr 172.21.15.107:0/1328340563 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:46:21.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:21.247+0000 7f1b79e00700 1 --2- 172.21.15.107:0/1328340563 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b7c0fcef0 0x7f1b7c10fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:46:21.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:21.247+0000 7f1b79400700 1 -- 172.21.15.107:0/1328340563 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b7c0fcef0 msgr2=0x7f1b7c10fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:46:21.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:21.247+0000 7f1b79400700 1 --2- 172.21.15.107:0/1328340563 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b7c0fcef0 0x7f1b7c10fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:46:21.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:21.247+0000 7f1b79400700 1 -- 172.21.15.107:0/1328340563 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b6c005040 con 0x7f1b7c0fe0e0 2024-03-04T13:46:21.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:21.247+0000 7f1b79e00700 1 --2- 172.21.15.107:0/1328340563 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b7c0fcef0 0x7f1b7c10fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T13:46:21.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:21.248+0000 7f1b79400700 1 --2- 172.21.15.107:0/1328340563 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b7c0fe0e0 0x7f1b7c1100d0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f1b6800c830 tx=0x7f1b68004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:46:21.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:21.248+0000 7f1b72000700 1 -- 172.21.15.107:0/1328340563 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1b6800a760 con 0x7f1b7c0fe0e0 2024-03-04T13:46:21.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:21.248+0000 7f1b80c00700 1 -- 172.21.15.107:0/1328340563 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b7c111250 con 0x7f1b7c0fe0e0 2024-03-04T13:46:21.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:21.248+0000 7f1b72000700 1 -- 172.21.15.107:0/1328340563 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1b6800ad90 con 0x7f1b7c0fe0e0 2024-03-04T13:46:21.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:21.248+0000 7f1b72000700 1 -- 172.21.15.107:0/1328340563 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1b680113a0 con 0x7f1b7c0fe0e0 2024-03-04T13:46:21.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:21.248+0000 7f1b80c00700 1 -- 172.21.15.107:0/1328340563 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b7c111790 con 0x7f1b7c0fe0e0 2024-03-04T13:46:21.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:21.249+0000 7f1b80c00700 1 -- 172.21.15.107:0/1328340563 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b7c102210 con 0x7f1b7c0fe0e0 2024-03-04T13:46:21.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:21.250+0000 7f1b72000700 1 -- 172.21.15.107:0/1328340563 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 33) v1 ==== 91199+0+0 (secure 0 0 0) 0x7f1b68012400 con 0x7f1b7c0fe0e0 2024-03-04T13:46:21.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:21.251+0000 7f1b72000700 1 --2- 172.21.15.107:0/1328340563 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f1b6006d0f0 0x7f1b6006f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:46:21.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:21.251+0000 7f1b72000700 1 -- 172.21.15.107:0/1328340563 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f1b6808c610 con 0x7f1b7c0fe0e0 2024-03-04T13:46:21.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:21.252+0000 7f1b79e00700 1 --2- 172.21.15.107:0/1328340563 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f1b6006d0f0 0x7f1b6006f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:46:21.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:21.252+0000 7f1b79e00700 1 --2- 172.21.15.107:0/1328340563 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f1b6006d0f0 0x7f1b6006f5a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c00b7a0 tx=0x7f1b6c009e30 comp rx=0 tx=0).ready entity=mgr.14696 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:46:21.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:21.258+0000 7f1b72000700 1 -- 172.21.15.107:0/1328340563 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f1b6805a1c0 con 0x7f1b7c0fe0e0 2024-03-04T13:46:21.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:21.501+0000 7f1b80c00700 1 -- 172.21.15.107:0/1328340563 --> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1b7c111b80 con 0x7f1b6006d0f0 2024-03-04T13:46:21.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:21.502+0000 7f1b72000700 1 -- 172.21.15.107:0/1328340563 <== mgr.14696 v2:172.21.15.107:6800/1613321046 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7f1b7c111b80 con 0x7f1b6006d0f0 2024-03-04T13:46:21.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:21.505+0000 7f1b80c00700 1 -- 172.21.15.107:0/1328340563 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f1b6006d0f0 msgr2=0x7f1b6006f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:46:21.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:21.505+0000 7f1b80c00700 1 --2- 172.21.15.107:0/1328340563 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f1b6006d0f0 0x7f1b6006f5a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c00b7a0 tx=0x7f1b6c009e30 comp rx=0 tx=0).stop 2024-03-04T13:46:21.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:21.505+0000 7f1b80c00700 1 -- 172.21.15.107:0/1328340563 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b7c0fe0e0 msgr2=0x7f1b7c1100d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:46:21.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:21.505+0000 7f1b80c00700 1 --2- 172.21.15.107:0/1328340563 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b7c0fe0e0 0x7f1b7c1100d0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f1b6800c830 tx=0x7f1b68004470 comp rx=0 tx=0).stop 2024-03-04T13:46:21.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:21.505+0000 7f1b80c00700 1 -- 172.21.15.107:0/1328340563 shutdown_connections 2024-03-04T13:46:21.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:21.505+0000 7f1b80c00700 1 --2- 172.21.15.107:0/1328340563 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b7c0fcef0 0x7f1b7c10fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:46:21.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:21.505+0000 7f1b80c00700 1 --2- 172.21.15.107:0/1328340563 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f1b6006d0f0 0x7f1b6006f5a0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:46:21.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:21.505+0000 7f1b80c00700 1 --2- 172.21.15.107:0/1328340563 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b7c0fe0e0 0x7f1b7c1100d0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:46:21.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:21.505+0000 7f1b80c00700 1 -- 172.21.15.107:0/1328340563 >> 172.21.15.107:0/1328340563 conn(0x7f1b7c0f8620 msgr2=0x7f1b7c0ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:46:21.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:21.505+0000 7f1b80c00700 1 -- 172.21.15.107:0/1328340563 shutdown_connections 2024-03-04T13:46:21.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:21.505+0000 7f1b80c00700 1 -- 172.21.15.107:0/1328340563 wait complete. 2024-03-04T13:46:21.843 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:21 smithi107 ceph-mon[28739]: pgmap v84: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:46:21.843 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:21 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:46:21.843 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:21 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:46:21.843 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:21 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:46:21.843 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:21 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:46:21.843 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:21 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:46:21.844 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:21 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-04T13:46:21.844 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:21 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:46:21.844 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:21 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:46:21.844 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:21 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:46:21.844 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:21 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:46:21.844 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:21 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:46:21.844 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:21 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:46:21.844 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:21 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:46:21.844 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:21 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:46:21.844 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:21 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:46:21.844 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:21 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:46:21.844 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:21 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:46:21.845 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:21 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:46:21.845 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:21 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:46:21.845 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:21 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:46:21.845 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:21 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:46:21.845 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:21 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:46:21.845 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:21 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-03-04T13:46:21.845 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:21 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-03-04T13:46:21.845 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:21 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-04T13:46:21.845 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:21 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-03-04T13:46:21.845 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:21 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-03-04T13:46:21.845 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:21 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-03-04T13:46:21.846 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:21 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-03-04T13:46:21.846 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:21 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-03-04T13:46:21.846 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:21 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-03-04T13:46:21.846 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:21 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-03-04T13:46:21.846 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:21 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-03-04T13:46:21.846 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:21 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-04T13:46:21.846 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:21 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-03-04T13:46:21.846 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:21 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-03-04T13:46:21.846 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:21 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-03-04T13:46:21.846 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:21 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-03-04T13:46:21.846 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:21 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-03-04T13:46:21.846 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:21 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-03-04T13:46:21.847 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:21 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-03-04T13:46:21.847 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:21 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-03-04T13:46:21.847 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:21 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-04T13:46:21.847 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:21 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-04T13:46:21.847 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:21 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-04T13:46:21.847 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:21 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-04T13:46:21.847 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:21 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-04T13:46:21.847 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:21 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-04T13:46:21.847 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:21 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-03-04T13:46:21.847 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:21 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-03-04T13:46:21.847 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:21 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:46:21.848 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:21 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:46:21.848 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:21 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:46:21.848 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:21 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:46:21.848 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:21 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:46:21.848 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:21 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:46:21.848 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:21 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:46:21.848 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:21 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:46:21.860 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:21 smithi169 ceph-mon[36313]: pgmap v84: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:46:21.860 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:21 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:46:21.860 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:21 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:46:21.860 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:21 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:46:21.860 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:21 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:46:21.861 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:21 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:46:21.861 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:21 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-04T13:46:21.861 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:21 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:46:21.861 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:21 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:46:21.861 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:21 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:46:21.861 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:21 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:46:21.861 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:21 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:46:21.861 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:21 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:46:21.861 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:21 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:46:21.861 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:21 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:46:21.861 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:21 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:46:21.862 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:21 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:46:21.862 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:21 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:46:21.862 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:21 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:46:21.862 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:21 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:46:21.862 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:21 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:46:21.862 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:21 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:46:21.862 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:21 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:46:21.862 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:21 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-03-04T13:46:21.862 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:21 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-03-04T13:46:21.863 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:21 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-04T13:46:21.863 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:21 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-03-04T13:46:21.863 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:21 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-03-04T13:46:21.863 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:21 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-03-04T13:46:21.863 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:21 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-03-04T13:46:21.863 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:21 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-03-04T13:46:21.863 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:21 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-03-04T13:46:21.863 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:21 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-03-04T13:46:21.864 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:21 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-03-04T13:46:21.864 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:21 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-04T13:46:21.864 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:21 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-03-04T13:46:21.864 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:21 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-03-04T13:46:21.864 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:21 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-03-04T13:46:21.864 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:21 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-03-04T13:46:21.864 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:21 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-03-04T13:46:21.864 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:21 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-03-04T13:46:21.864 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:21 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-03-04T13:46:21.864 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:21 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-03-04T13:46:21.864 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:21 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-04T13:46:21.864 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:21 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-04T13:46:21.865 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:21 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-04T13:46:21.865 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:21 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-04T13:46:21.865 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:21 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-04T13:46:21.865 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:21 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-04T13:46:21.865 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:21 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-03-04T13:46:21.865 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:21 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-03-04T13:46:21.865 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:21 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:46:21.865 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:21 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:46:21.865 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:21 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:46:21.865 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:21 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:46:21.865 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:21 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:46:21.866 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:21 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:46:21.866 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:21 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:46:21.866 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:21 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:46:22.394 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 -e sha1=717ce3cce8d0166e4e577faa8004e6f9cb4128c0 -- bash -c 'ceph versions | jq -e '"'"'.mgr | length == 1'"'"'' 2024-03-04T13:46:22.859 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:22 smithi169 ceph-mon[36313]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-04T13:46:22.860 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:22 smithi169 ceph-mon[36313]: Upgrade: Finalizing container_image settings 2024-03-04T13:46:22.860 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:22 smithi169 ceph-mon[36313]: Upgrade: Complete! 2024-03-04T13:46:22.860 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:22 smithi169 ceph-mon[36313]: from='client.14824 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:46:22.865 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/mon.smithi107/config 2024-03-04T13:46:23.016 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:22 smithi107 ceph-mon[28739]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-04T13:46:23.016 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:22 smithi107 ceph-mon[28739]: Upgrade: Finalizing container_image settings 2024-03-04T13:46:23.016 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:22 smithi107 ceph-mon[28739]: Upgrade: Complete! 2024-03-04T13:46:23.016 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:22 smithi107 ceph-mon[28739]: from='client.14824 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:46:23.919 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:23 smithi107 ceph-mon[28739]: pgmap v85: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:46:23.919 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:23 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:46:23.919 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:23 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:46:24.109 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:23 smithi169 ceph-mon[36313]: pgmap v85: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:46:24.109 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:23 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:46:24.109 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:23 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:46:25.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:25.324+0000 7f9aefa00700 1 -- 172.21.15.107:0/112087870 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ae81038f0 msgr2=0x7f9ae80fcf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:46:25.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:25.324+0000 7f9aefa00700 1 --2- 172.21.15.107:0/112087870 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ae81038f0 0x7f9ae80fcf60 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f9ad4004940 tx=0x7f9ad4009e30 comp rx=0 tx=0).stop 2024-03-04T13:46:25.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:25.324+0000 7f9aefa00700 1 -- 172.21.15.107:0/112087870 shutdown_connections 2024-03-04T13:46:25.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:25.324+0000 7f9aefa00700 1 --2- 172.21.15.107:0/112087870 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ae81038f0 0x7f9ae80fcf60 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:46:25.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:25.324+0000 7f9aefa00700 1 --2- 172.21.15.107:0/112087870 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ae8102f50 0x7f9ae8103320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:46:25.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:25.324+0000 7f9aefa00700 1 -- 172.21.15.107:0/112087870 >> 172.21.15.107:0/112087870 conn(0x7f9ae80f8ae0 msgr2=0x7f9ae80faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:46:25.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:25.325+0000 7f9aefa00700 1 -- 172.21.15.107:0/112087870 shutdown_connections 2024-03-04T13:46:25.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:25.325+0000 7f9aefa00700 1 -- 172.21.15.107:0/112087870 wait complete. 2024-03-04T13:46:25.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:25.325+0000 7f9aefa00700 1 Processor -- start 2024-03-04T13:46:25.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:25.326+0000 7f9aefa00700 1 -- start start 2024-03-04T13:46:25.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:25.326+0000 7f9aefa00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ae8102f50 0x7f9ae8070290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:46:25.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:25.326+0000 7f9aefa00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ae81038f0 0x7f9ae80707d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:46:25.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:25.326+0000 7f9aefa00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae8070dd0 con 0x7f9ae8102f50 2024-03-04T13:46:25.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:25.326+0000 7f9aefa00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae8070f40 con 0x7f9ae81038f0 2024-03-04T13:46:25.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:25.326+0000 7f9aed000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ae8102f50 0x7f9ae8070290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:46:25.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:25.326+0000 7f9aed000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ae8102f50 0x7f9ae8070290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40284/0 (socket says 172.21.15.107:40284) 2024-03-04T13:46:25.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:25.326+0000 7f9aed000700 1 -- 172.21.15.107:0/2941837157 learned_addr learned my addr 172.21.15.107:0/2941837157 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:46:25.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:25.327+0000 7f9aed000700 1 -- 172.21.15.107:0/2941837157 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ae81038f0 msgr2=0x7f9ae80707d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:46:25.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:25.327+0000 7f9aed000700 1 --2- 172.21.15.107:0/2941837157 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ae81038f0 0x7f9ae80707d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:46:25.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:25.327+0000 7f9aed000700 1 -- 172.21.15.107:0/2941837157 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ad4005040 con 0x7f9ae8102f50 2024-03-04T13:46:25.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:25.327+0000 7f9aed000700 1 --2- 172.21.15.107:0/2941837157 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ae8102f50 0x7f9ae8070290 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f9adc00c8c0 tx=0x7f9adc0044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:46:25.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:25.327+0000 7f9ae4c00700 1 -- 172.21.15.107:0/2941837157 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9adc00a980 con 0x7f9ae8102f50 2024-03-04T13:46:25.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:25.327+0000 7f9ae4c00700 1 -- 172.21.15.107:0/2941837157 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9adc00ce60 con 0x7f9ae8102f50 2024-03-04T13:46:25.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:25.327+0000 7f9ae4c00700 1 -- 172.21.15.107:0/2941837157 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9adc0115e0 con 0x7f9ae8102f50 2024-03-04T13:46:25.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:25.327+0000 7f9aefa00700 1 -- 172.21.15.107:0/2941837157 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ae8119cd0 con 0x7f9ae8102f50 2024-03-04T13:46:25.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:25.328+0000 7f9aefa00700 1 -- 172.21.15.107:0/2941837157 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ae811a1b0 con 0x7f9ae8102f50 2024-03-04T13:46:25.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:25.329+0000 7f9aefa00700 1 -- 172.21.15.107:0/2941837157 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ae8105420 con 0x7f9ae8102f50 2024-03-04T13:46:25.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:25.330+0000 7f9ae4c00700 1 -- 172.21.15.107:0/2941837157 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 33) v1 ==== 91199+0+0 (secure 0 0 0) 0x7f9adc012400 con 0x7f9ae8102f50 2024-03-04T13:46:25.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:25.331+0000 7f9ae4c00700 1 --2- 172.21.15.107:0/2941837157 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f9ad006d0f0 0x7f9ad006f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:46:25.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:25.331+0000 7f9ae4c00700 1 -- 172.21.15.107:0/2941837157 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f9adc08c600 con 0x7f9ae8102f50 2024-03-04T13:46:25.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:25.331+0000 7f9ae7e00700 1 --2- 172.21.15.107:0/2941837157 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f9ad006d0f0 0x7f9ad006f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:46:25.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:25.332+0000 7f9ae7e00700 1 --2- 172.21.15.107:0/2941837157 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f9ad006d0f0 0x7f9ad006f5a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9ad4004f90 tx=0x7f9ad4009e30 comp rx=0 tx=0).ready entity=mgr.14696 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:46:25.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:25.335+0000 7f9ae4c00700 1 -- 172.21.15.107:0/2941837157 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f9adc056700 con 0x7f9ae8102f50 2024-03-04T13:46:25.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:25.644+0000 7f9aefa00700 1 -- 172.21.15.107:0/2941837157 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f9ae80719d0 con 0x7f9ae8102f50 2024-03-04T13:46:25.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:25.645+0000 7f9ae4c00700 1 -- 172.21.15.107:0/2941837157 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7f9adc0126b0 con 0x7f9ae8102f50 2024-03-04T13:46:25.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:25.648+0000 7f9aefa00700 1 -- 172.21.15.107:0/2941837157 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f9ad006d0f0 msgr2=0x7f9ad006f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:46:25.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:25.648+0000 7f9aefa00700 1 --2- 172.21.15.107:0/2941837157 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f9ad006d0f0 0x7f9ad006f5a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9ad4004f90 tx=0x7f9ad4009e30 comp rx=0 tx=0).stop 2024-03-04T13:46:25.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:25.648+0000 7f9aefa00700 1 -- 172.21.15.107:0/2941837157 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ae8102f50 msgr2=0x7f9ae8070290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:46:25.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:25.648+0000 7f9aefa00700 1 --2- 172.21.15.107:0/2941837157 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ae8102f50 0x7f9ae8070290 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f9adc00c8c0 tx=0x7f9adc0044c0 comp rx=0 tx=0).stop 2024-03-04T13:46:25.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:25.648+0000 7f9aefa00700 1 -- 172.21.15.107:0/2941837157 shutdown_connections 2024-03-04T13:46:25.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:25.648+0000 7f9aefa00700 1 --2- 172.21.15.107:0/2941837157 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f9ad006d0f0 0x7f9ad006f5a0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:46:25.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:25.648+0000 7f9aefa00700 1 --2- 172.21.15.107:0/2941837157 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ae8102f50 0x7f9ae8070290 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:46:25.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:25.648+0000 7f9aefa00700 1 --2- 172.21.15.107:0/2941837157 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ae81038f0 0x7f9ae80707d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:46:25.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:25.648+0000 7f9aefa00700 1 -- 172.21.15.107:0/2941837157 >> 172.21.15.107:0/2941837157 conn(0x7f9ae80f8ae0 msgr2=0x7f9ae80fa5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:46:25.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:25.648+0000 7f9aefa00700 1 -- 172.21.15.107:0/2941837157 shutdown_connections 2024-03-04T13:46:25.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:25.649+0000 7f9aefa00700 1 -- 172.21.15.107:0/2941837157 wait complete. 2024-03-04T13:46:25.663 INFO:teuthology.orchestra.run.smithi107.stdout:true 2024-03-04T13:46:26.082 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:25 smithi107 ceph-mon[28739]: pgmap v86: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:46:26.109 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:25 smithi169 ceph-mon[36313]: pgmap v86: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:46:26.519 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 -e sha1=717ce3cce8d0166e4e577faa8004e6f9cb4128c0 -- bash -c 'ceph versions | jq -e '"'"'.mgr | keys'"'"' | grep $sha1' 2024-03-04T13:46:26.842 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:26 smithi107 ceph-mon[28739]: from='client.? 172.21.15.107:0/2941837157' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:46:27.002 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/mon.smithi107/config 2024-03-04T13:46:27.109 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:26 smithi169 ceph-mon[36313]: from='client.? 172.21.15.107:0/2941837157' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:46:27.842 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:27 smithi107 ceph-mon[28739]: pgmap v87: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:46:28.109 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:27 smithi169 ceph-mon[36313]: pgmap v87: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:46:29.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:29.203+0000 7fe702e00700 1 -- 172.21.15.107:0/3995361253 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe6fc105ab0 msgr2=0x7fe6fc0ff120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:46:29.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:29.203+0000 7fe702e00700 1 --2- 172.21.15.107:0/3995361253 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe6fc105ab0 0x7fe6fc0ff120 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fe6e8004940 tx=0x7fe6e8009e30 comp rx=0 tx=0).stop 2024-03-04T13:46:29.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:29.203+0000 7fe702e00700 1 -- 172.21.15.107:0/3995361253 shutdown_connections 2024-03-04T13:46:29.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:29.203+0000 7fe702e00700 1 --2- 172.21.15.107:0/3995361253 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe6fc105ab0 0x7fe6fc0ff120 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:46:29.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:29.203+0000 7fe702e00700 1 --2- 172.21.15.107:0/3995361253 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe6fc105110 0x7fe6fc1054e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:46:29.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:29.203+0000 7fe702e00700 1 -- 172.21.15.107:0/3995361253 >> 172.21.15.107:0/3995361253 conn(0x7fe6fc0fac40 msgr2=0x7fe6fc0fd050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:46:29.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:29.203+0000 7fe702e00700 1 -- 172.21.15.107:0/3995361253 shutdown_connections 2024-03-04T13:46:29.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:29.204+0000 7fe702e00700 1 -- 172.21.15.107:0/3995361253 wait complete. 2024-03-04T13:46:29.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:29.205+0000 7fe702e00700 1 Processor -- start 2024-03-04T13:46:29.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:29.205+0000 7fe702e00700 1 -- start start 2024-03-04T13:46:29.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:29.206+0000 7fe702e00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe6fc105110 0x7fe6fc115bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:46:29.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:29.206+0000 7fe702e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe6fc105ab0 0x7fe6fc1160f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:46:29.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:29.206+0000 7fe702e00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6fc116740 con 0x7fe6fc105ab0 2024-03-04T13:46:29.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:29.206+0000 7fe702e00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6fc10fa70 con 0x7fe6fc105110 2024-03-04T13:46:29.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:29.206+0000 7fe6fb400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe6fc105ab0 0x7fe6fc1160f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:46:29.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:29.206+0000 7fe6fb400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe6fc105ab0 0x7fe6fc1160f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42002/0 (socket says 172.21.15.107:42002) 2024-03-04T13:46:29.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:29.206+0000 7fe6fb400700 1 -- 172.21.15.107:0/2893532399 learned_addr learned my addr 172.21.15.107:0/2893532399 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:46:29.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:29.207+0000 7fe6fbe00700 1 --2- 172.21.15.107:0/2893532399 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe6fc105110 0x7fe6fc115bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:46:29.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:29.207+0000 7fe6fb400700 1 -- 172.21.15.107:0/2893532399 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe6fc105110 msgr2=0x7fe6fc115bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:46:29.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:29.207+0000 7fe6fb400700 1 --2- 172.21.15.107:0/2893532399 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe6fc105110 0x7fe6fc115bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:46:29.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:29.207+0000 7fe6fb400700 1 -- 172.21.15.107:0/2893532399 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6e8005040 con 0x7fe6fc105ab0 2024-03-04T13:46:29.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:29.207+0000 7fe6fb400700 1 --2- 172.21.15.107:0/2893532399 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe6fc105ab0 0x7fe6fc1160f0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fe6e8009100 tx=0x7fe6e8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:46:29.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:29.207+0000 7fe6f3e00700 1 -- 172.21.15.107:0/2893532399 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe6e800e040 con 0x7fe6fc105ab0 2024-03-04T13:46:29.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:29.207+0000 7fe6f3e00700 1 -- 172.21.15.107:0/2893532399 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe6e8004290 con 0x7fe6fc105ab0 2024-03-04T13:46:29.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:29.207+0000 7fe702e00700 1 -- 172.21.15.107:0/2893532399 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6fc10fcf0 con 0x7fe6fc105ab0 2024-03-04T13:46:29.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:29.207+0000 7fe6f3e00700 1 -- 172.21.15.107:0/2893532399 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe6e801c3f0 con 0x7fe6fc105ab0 2024-03-04T13:46:29.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:29.208+0000 7fe702e00700 1 -- 172.21.15.107:0/2893532399 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6fc1102b0 con 0x7fe6fc105ab0 2024-03-04T13:46:29.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:29.209+0000 7fe702e00700 1 -- 172.21.15.107:0/2893532399 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6fc1075e0 con 0x7fe6fc105ab0 2024-03-04T13:46:29.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:29.210+0000 7fe6f3e00700 1 -- 172.21.15.107:0/2893532399 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 33) v1 ==== 91199+0+0 (secure 0 0 0) 0x7fe6e800bbf0 con 0x7fe6fc105ab0 2024-03-04T13:46:29.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:29.211+0000 7fe6f3e00700 1 --2- 172.21.15.107:0/2893532399 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7fe6e006d0f0 0x7fe6e006f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:46:29.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:29.211+0000 7fe6f3e00700 1 -- 172.21.15.107:0/2893532399 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fe6e808d850 con 0x7fe6fc105ab0 2024-03-04T13:46:29.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:29.211+0000 7fe6fbe00700 1 --2- 172.21.15.107:0/2893532399 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7fe6e006d0f0 0x7fe6e006f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:46:29.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:29.212+0000 7fe6fbe00700 1 --2- 172.21.15.107:0/2893532399 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7fe6e006d0f0 0x7fe6e006f5a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec004990 tx=0x7fe6ec009ee0 comp rx=0 tx=0).ready entity=mgr.14696 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:46:29.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:29.216+0000 7fe6f3e00700 1 -- 172.21.15.107:0/2893532399 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7fe6e805b480 con 0x7fe6fc105ab0 2024-03-04T13:46:29.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:29.517+0000 7fe702e00700 1 -- 172.21.15.107:0/2893532399 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fe6fc10c1f0 con 0x7fe6fc105ab0 2024-03-04T13:46:29.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:29.518+0000 7fe6f3e00700 1 -- 172.21.15.107:0/2893532399 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7fe6e805b010 con 0x7fe6fc105ab0 2024-03-04T13:46:29.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:29.520+0000 7fe702e00700 1 -- 172.21.15.107:0/2893532399 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7fe6e006d0f0 msgr2=0x7fe6e006f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:46:29.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:29.520+0000 7fe702e00700 1 --2- 172.21.15.107:0/2893532399 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7fe6e006d0f0 0x7fe6e006f5a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec004990 tx=0x7fe6ec009ee0 comp rx=0 tx=0).stop 2024-03-04T13:46:29.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:29.520+0000 7fe702e00700 1 -- 172.21.15.107:0/2893532399 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe6fc105ab0 msgr2=0x7fe6fc1160f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:46:29.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:29.520+0000 7fe702e00700 1 --2- 172.21.15.107:0/2893532399 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe6fc105ab0 0x7fe6fc1160f0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fe6e8009100 tx=0x7fe6e8009e30 comp rx=0 tx=0).stop 2024-03-04T13:46:29.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:29.521+0000 7fe702e00700 1 -- 172.21.15.107:0/2893532399 shutdown_connections 2024-03-04T13:46:29.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:29.521+0000 7fe702e00700 1 --2- 172.21.15.107:0/2893532399 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe6fc105110 0x7fe6fc115bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:46:29.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:29.521+0000 7fe702e00700 1 --2- 172.21.15.107:0/2893532399 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7fe6e006d0f0 0x7fe6e006f5a0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:46:29.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:29.521+0000 7fe702e00700 1 --2- 172.21.15.107:0/2893532399 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe6fc105ab0 0x7fe6fc1160f0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:46:29.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:29.521+0000 7fe702e00700 1 -- 172.21.15.107:0/2893532399 >> 172.21.15.107:0/2893532399 conn(0x7fe6fc0fac40 msgr2=0x7fe6fc0fc650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:46:29.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:29.521+0000 7fe702e00700 1 -- 172.21.15.107:0/2893532399 shutdown_connections 2024-03-04T13:46:29.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:29.521+0000 7fe702e00700 1 -- 172.21.15.107:0/2893532399 wait complete. 2024-03-04T13:46:29.535 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-1845-g717ce3cc (717ce3cce8d0166e4e577faa8004e6f9cb4128c0) squid (dev)" 2024-03-04T13:46:29.842 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:29 smithi107 ceph-mon[28739]: pgmap v88: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:46:29.842 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:29 smithi107 ceph-mon[28739]: from='client.? 172.21.15.107:0/2893532399' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:46:30.109 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:29 smithi169 ceph-mon[36313]: pgmap v88: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:46:30.109 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:29 smithi169 ceph-mon[36313]: from='client.? 172.21.15.107:0/2893532399' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:46:30.306 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 -e sha1=717ce3cce8d0166e4e577faa8004e6f9cb4128c0 -- bash -c 'ceph versions | jq -e '"'"'.overall | length == 2'"'"'' 2024-03-04T13:46:30.808 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/mon.smithi107/config 2024-03-04T13:46:31.978 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:31 smithi107 ceph-mon[28739]: pgmap v89: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:46:32.109 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:31 smithi169 ceph-mon[36313]: pgmap v89: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:46:32.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:32.890+0000 7fef48800700 1 -- 172.21.15.107:0/924101152 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef401038f0 msgr2=0x7fef400fcf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:46:32.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:32.890+0000 7fef48800700 1 --2- 172.21.15.107:0/924101152 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef401038f0 0x7fef400fcf60 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fef30004940 tx=0x7fef30009e30 comp rx=0 tx=0).stop 2024-03-04T13:46:32.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:32.890+0000 7fef48800700 1 -- 172.21.15.107:0/924101152 shutdown_connections 2024-03-04T13:46:32.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:32.890+0000 7fef48800700 1 --2- 172.21.15.107:0/924101152 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef401038f0 0x7fef400fcf60 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:46:32.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:32.890+0000 7fef48800700 1 --2- 172.21.15.107:0/924101152 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef40102f50 0x7fef40103320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:46:32.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:32.890+0000 7fef48800700 1 -- 172.21.15.107:0/924101152 >> 172.21.15.107:0/924101152 conn(0x7fef400f8ae0 msgr2=0x7fef400faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:46:32.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:32.891+0000 7fef48800700 1 -- 172.21.15.107:0/924101152 shutdown_connections 2024-03-04T13:46:32.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:32.891+0000 7fef48800700 1 -- 172.21.15.107:0/924101152 wait complete. 2024-03-04T13:46:32.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:32.891+0000 7fef48800700 1 Processor -- start 2024-03-04T13:46:32.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:32.892+0000 7fef48800700 1 -- start start 2024-03-04T13:46:32.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:32.892+0000 7fef48800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef40102f50 0x7fef40115c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:46:32.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:32.892+0000 7fef48800700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef401038f0 0x7fef401161c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:46:32.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:32.892+0000 7fef48800700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef40116810 con 0x7fef40102f50 2024-03-04T13:46:32.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:32.892+0000 7fef48800700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef4010fb40 con 0x7fef401038f0 2024-03-04T13:46:32.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:32.893+0000 7fef45e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef40102f50 0x7fef40115c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:46:32.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:32.893+0000 7fef45e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef40102f50 0x7fef40115c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42018/0 (socket says 172.21.15.107:42018) 2024-03-04T13:46:32.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:32.893+0000 7fef45e00700 1 -- 172.21.15.107:0/3072931877 learned_addr learned my addr 172.21.15.107:0/3072931877 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:46:32.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:32.893+0000 7fef45400700 1 --2- 172.21.15.107:0/3072931877 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef401038f0 0x7fef401161c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:46:32.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:32.893+0000 7fef45e00700 1 -- 172.21.15.107:0/3072931877 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef401038f0 msgr2=0x7fef401161c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:46:32.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:32.893+0000 7fef45e00700 1 --2- 172.21.15.107:0/3072931877 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef401038f0 0x7fef401161c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:46:32.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:32.893+0000 7fef45e00700 1 -- 172.21.15.107:0/3072931877 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef30005040 con 0x7fef40102f50 2024-03-04T13:46:32.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:32.893+0000 7fef45e00700 1 --2- 172.21.15.107:0/3072931877 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef40102f50 0x7fef40115c80 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fef34002a50 tx=0x7fef340046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:46:32.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:32.894+0000 7fef3a000700 1 -- 172.21.15.107:0/3072931877 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fef34007cc0 con 0x7fef40102f50 2024-03-04T13:46:32.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:32.894+0000 7fef3a000700 1 -- 172.21.15.107:0/3072931877 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fef3400f040 con 0x7fef40102f50 2024-03-04T13:46:32.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:32.894+0000 7fef48800700 1 -- 172.21.15.107:0/3072931877 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef4010fe20 con 0x7fef40102f50 2024-03-04T13:46:32.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:32.894+0000 7fef3a000700 1 -- 172.21.15.107:0/3072931877 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fef34009640 con 0x7fef40102f50 2024-03-04T13:46:32.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:32.894+0000 7fef48800700 1 -- 172.21.15.107:0/3072931877 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef40110360 con 0x7fef40102f50 2024-03-04T13:46:32.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:32.895+0000 7fef48800700 1 -- 172.21.15.107:0/3072931877 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef400fc6e0 con 0x7fef40102f50 2024-03-04T13:46:32.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:32.896+0000 7fef3a000700 1 -- 172.21.15.107:0/3072931877 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 33) v1 ==== 91199+0+0 (secure 0 0 0) 0x7fef340097a0 con 0x7fef40102f50 2024-03-04T13:46:32.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:32.897+0000 7fef3a000700 1 --2- 172.21.15.107:0/3072931877 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7fef2806d0f0 0x7fef2806f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:46:32.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:32.897+0000 7fef3a000700 1 -- 172.21.15.107:0/3072931877 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fef3408c2f0 con 0x7fef40102f50 2024-03-04T13:46:32.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:32.897+0000 7fef45400700 1 --2- 172.21.15.107:0/3072931877 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7fef2806d0f0 0x7fef2806f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:46:32.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:32.898+0000 7fef45400700 1 --2- 172.21.15.107:0/3072931877 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7fef2806d0f0 0x7fef2806f5a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fef401114b0 tx=0x7fef30009e30 comp rx=0 tx=0).ready entity=mgr.14696 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:46:32.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:32.902+0000 7fef3a000700 1 -- 172.21.15.107:0/3072931877 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7fef34059ea0 con 0x7fef40102f50 2024-03-04T13:46:33.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:33.208+0000 7fef48800700 1 -- 172.21.15.107:0/3072931877 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fef40111220 con 0x7fef40102f50 2024-03-04T13:46:33.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:33.210+0000 7fef3a000700 1 -- 172.21.15.107:0/3072931877 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7fef34059a30 con 0x7fef40102f50 2024-03-04T13:46:33.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:33.213+0000 7fef48800700 1 -- 172.21.15.107:0/3072931877 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7fef2806d0f0 msgr2=0x7fef2806f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:46:33.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:33.213+0000 7fef48800700 1 --2- 172.21.15.107:0/3072931877 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7fef2806d0f0 0x7fef2806f5a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fef401114b0 tx=0x7fef30009e30 comp rx=0 tx=0).stop 2024-03-04T13:46:33.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:33.213+0000 7fef48800700 1 -- 172.21.15.107:0/3072931877 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef40102f50 msgr2=0x7fef40115c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:46:33.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:33.213+0000 7fef48800700 1 --2- 172.21.15.107:0/3072931877 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef40102f50 0x7fef40115c80 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fef34002a50 tx=0x7fef340046b0 comp rx=0 tx=0).stop 2024-03-04T13:46:33.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:33.213+0000 7fef48800700 1 -- 172.21.15.107:0/3072931877 shutdown_connections 2024-03-04T13:46:33.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:33.213+0000 7fef48800700 1 --2- 172.21.15.107:0/3072931877 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7fef2806d0f0 0x7fef2806f5a0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:46:33.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:33.213+0000 7fef48800700 1 --2- 172.21.15.107:0/3072931877 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef40102f50 0x7fef40115c80 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:46:33.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:33.213+0000 7fef48800700 1 --2- 172.21.15.107:0/3072931877 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef401038f0 0x7fef401161c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:46:33.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:33.213+0000 7fef48800700 1 -- 172.21.15.107:0/3072931877 >> 172.21.15.107:0/3072931877 conn(0x7fef400f8ae0 msgr2=0x7fef400fa470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:46:33.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:33.213+0000 7fef48800700 1 -- 172.21.15.107:0/3072931877 shutdown_connections 2024-03-04T13:46:33.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:33.213+0000 7fef48800700 1 -- 172.21.15.107:0/3072931877 wait complete. 2024-03-04T13:46:33.228 INFO:teuthology.orchestra.run.smithi107.stdout:true 2024-03-04T13:46:33.842 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:33 smithi107 ceph-mon[28739]: pgmap v90: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:46:33.842 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:33 smithi107 ceph-mon[28739]: from='client.? 172.21.15.107:0/3072931877' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:46:34.018 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 -e sha1=717ce3cce8d0166e4e577faa8004e6f9cb4128c0 -- bash -c 'ceph orch upgrade check quay.ceph.io/ceph-ci/ceph:$sha1 | jq -e '"'"'.up_to_date | length == 2'"'"'' 2024-03-04T13:46:34.109 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:33 smithi169 ceph-mon[36313]: pgmap v90: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:46:34.109 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:33 smithi169 ceph-mon[36313]: from='client.? 172.21.15.107:0/3072931877' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:46:34.499 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/mon.smithi107/config 2024-03-04T13:46:36.025 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:35 smithi107 ceph-mon[28739]: pgmap v91: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:46:36.109 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:35 smithi169 ceph-mon[36313]: pgmap v91: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:46:36.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:36.543+0000 7f5494a00700 1 -- 172.21.15.107:0/3141740674 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5490072dd0 msgr2=0x7f549010d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:46:36.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:36.543+0000 7f5494a00700 1 --2- 172.21.15.107:0/3141740674 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5490072dd0 0x7f549010d8e0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f547c004940 tx=0x7f547c009e30 comp rx=0 tx=0).stop 2024-03-04T13:46:36.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:36.543+0000 7f5494a00700 1 -- 172.21.15.107:0/3141740674 shutdown_connections 2024-03-04T13:46:36.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:36.543+0000 7f5494a00700 1 --2- 172.21.15.107:0/3141740674 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5490072dd0 0x7f549010d8e0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:46:36.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:36.543+0000 7f5494a00700 1 --2- 172.21.15.107:0/3141740674 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f54900724c0 0x7f5490072890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:46:36.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:36.543+0000 7f5494a00700 1 -- 172.21.15.107:0/3141740674 >> 172.21.15.107:0/3141740674 conn(0x7f54900face0 msgr2=0x7f54900fd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:46:36.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:36.543+0000 7f5494a00700 1 -- 172.21.15.107:0/3141740674 shutdown_connections 2024-03-04T13:46:36.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:36.543+0000 7f5494a00700 1 -- 172.21.15.107:0/3141740674 wait complete. 2024-03-04T13:46:36.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:36.544+0000 7f5494a00700 1 Processor -- start 2024-03-04T13:46:36.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:36.544+0000 7f5494a00700 1 -- start start 2024-03-04T13:46:36.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:36.544+0000 7f5494a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f54900724c0 0x7f549011a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:46:36.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:36.544+0000 7f5494a00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5490072dd0 0x7f549011a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:46:36.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:36.544+0000 7f5494a00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f549011acd0 con 0x7f54900724c0 2024-03-04T13:46:36.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:36.544+0000 7f5494a00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5490113fe0 con 0x7f5490072dd0 2024-03-04T13:46:36.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:36.545+0000 7f548de00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f54900724c0 0x7f549011a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:46:36.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:36.545+0000 7f548de00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f54900724c0 0x7f549011a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42028/0 (socket says 172.21.15.107:42028) 2024-03-04T13:46:36.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:36.545+0000 7f548de00700 1 -- 172.21.15.107:0/3596944132 learned_addr learned my addr 172.21.15.107:0/3596944132 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:46:36.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:36.545+0000 7f548d400700 1 --2- 172.21.15.107:0/3596944132 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5490072dd0 0x7f549011a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:46:36.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:36.545+0000 7f548de00700 1 -- 172.21.15.107:0/3596944132 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5490072dd0 msgr2=0x7f549011a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:46:36.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:36.545+0000 7f548de00700 1 --2- 172.21.15.107:0/3596944132 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5490072dd0 0x7f549011a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:46:36.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:36.545+0000 7f548de00700 1 -- 172.21.15.107:0/3596944132 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f547c005040 con 0x7f54900724c0 2024-03-04T13:46:36.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:36.545+0000 7f548d400700 1 --2- 172.21.15.107:0/3596944132 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5490072dd0 0x7f549011a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T13:46:36.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:36.545+0000 7f548de00700 1 --2- 172.21.15.107:0/3596944132 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f54900724c0 0x7f549011a140 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f548000c830 tx=0x7f5480004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:46:36.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:36.546+0000 7f5486000700 1 -- 172.21.15.107:0/3596944132 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f548000a760 con 0x7f54900724c0 2024-03-04T13:46:36.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:36.546+0000 7f5494a00700 1 -- 172.21.15.107:0/3596944132 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54901142c0 con 0x7f54900724c0 2024-03-04T13:46:36.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:36.546+0000 7f5486000700 1 -- 172.21.15.107:0/3596944132 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f548000ad90 con 0x7f54900724c0 2024-03-04T13:46:36.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:36.546+0000 7f5486000700 1 -- 172.21.15.107:0/3596944132 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f54800113a0 con 0x7f54900724c0 2024-03-04T13:46:36.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:36.546+0000 7f5494a00700 1 -- 172.21.15.107:0/3596944132 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5490114800 con 0x7f54900724c0 2024-03-04T13:46:36.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:36.547+0000 7f5494a00700 1 -- 172.21.15.107:0/3596944132 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f549010b010 con 0x7f54900724c0 2024-03-04T13:46:36.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:36.548+0000 7f5486000700 1 -- 172.21.15.107:0/3596944132 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 33) v1 ==== 91199+0+0 (secure 0 0 0) 0x7f5480012400 con 0x7f54900724c0 2024-03-04T13:46:36.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:36.549+0000 7f5486000700 1 --2- 172.21.15.107:0/3596944132 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f547406d0f0 0x7f547406f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:46:36.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:36.549+0000 7f5486000700 1 -- 172.21.15.107:0/3596944132 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f548008c630 con 0x7f54900724c0 2024-03-04T13:46:36.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:36.549+0000 7f548d400700 1 --2- 172.21.15.107:0/3596944132 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f547406d0f0 0x7f547406f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:46:36.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:36.550+0000 7f548d400700 1 --2- 172.21.15.107:0/3596944132 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f547406d0f0 0x7f547406f5a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f547c004f90 tx=0x7f547c009e30 comp rx=0 tx=0).ready entity=mgr.14696 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:46:36.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:36.554+0000 7f5486000700 1 -- 172.21.15.107:0/3596944132 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f548005a1e0 con 0x7f54900724c0 2024-03-04T13:46:36.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:36.786+0000 7f5494a00700 1 -- 172.21.15.107:0/3596944132 --> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] -- mgr_command(tid 0: {"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:717ce3cce8d0166e4e577faa8004e6f9cb4128c0", "target": ["mon-mgr", ""]}) v1 -- 0x7f549004bed0 con 0x7f547406d0f0 2024-03-04T13:46:37.092 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:36 smithi107 ceph-mon[28739]: pgmap v92: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:46:37.109 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:36 smithi169 ceph-mon[36313]: pgmap v92: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:46:38.088 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:37 smithi107 ceph-mon[28739]: from='client.14840 -' entity='client.admin' cmd=[{"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:717ce3cce8d0166e4e577faa8004e6f9cb4128c0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:46:38.109 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:37 smithi169 ceph-mon[36313]: from='client.14840 -' entity='client.admin' cmd=[{"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:717ce3cce8d0166e4e577faa8004e6f9cb4128c0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:46:39.092 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:38 smithi107 ceph-mon[28739]: pgmap v93: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:46:39.092 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:38 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:46:39.109 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:38 smithi169 ceph-mon[36313]: pgmap v93: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:46:39.109 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:38 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:46:39.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:39.617+0000 7f5486000700 1 -- 172.21.15.107:0/3596944132 <== mgr.14696 v2:172.21.15.107:6800/1613321046 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+5092 (secure 0 0 0) 0x7f549004bed0 con 0x7f547406d0f0 2024-03-04T13:46:39.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:39.621+0000 7f5494a00700 1 -- 172.21.15.107:0/3596944132 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f547406d0f0 msgr2=0x7f547406f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:46:39.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:39.621+0000 7f5494a00700 1 --2- 172.21.15.107:0/3596944132 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f547406d0f0 0x7f547406f5a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f547c004f90 tx=0x7f547c009e30 comp rx=0 tx=0).stop 2024-03-04T13:46:39.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:39.621+0000 7f5494a00700 1 -- 172.21.15.107:0/3596944132 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f54900724c0 msgr2=0x7f549011a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:46:39.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:39.621+0000 7f5494a00700 1 --2- 172.21.15.107:0/3596944132 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f54900724c0 0x7f549011a140 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f548000c830 tx=0x7f5480004470 comp rx=0 tx=0).stop 2024-03-04T13:46:39.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:39.621+0000 7f5494a00700 1 -- 172.21.15.107:0/3596944132 shutdown_connections 2024-03-04T13:46:39.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:39.621+0000 7f5494a00700 1 --2- 172.21.15.107:0/3596944132 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f547406d0f0 0x7f547406f5a0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:46:39.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:39.621+0000 7f5494a00700 1 --2- 172.21.15.107:0/3596944132 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f54900724c0 0x7f549011a140 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:46:39.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:39.621+0000 7f5494a00700 1 --2- 172.21.15.107:0/3596944132 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5490072dd0 0x7f549011a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:46:39.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:39.621+0000 7f5494a00700 1 -- 172.21.15.107:0/3596944132 >> 172.21.15.107:0/3596944132 conn(0x7f54900face0 msgr2=0x7f54900fb940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:46:39.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:39.622+0000 7f5494a00700 1 -- 172.21.15.107:0/3596944132 shutdown_connections 2024-03-04T13:46:39.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:39.622+0000 7f5494a00700 1 -- 172.21.15.107:0/3596944132 wait complete. 2024-03-04T13:46:39.639 INFO:teuthology.orchestra.run.smithi107.stdout:true 2024-03-04T13:46:40.476 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 -e sha1=717ce3cce8d0166e4e577faa8004e6f9cb4128c0 -- bash -c 'ceph orch ps' 2024-03-04T13:46:40.955 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/mon.smithi107/config 2024-03-04T13:46:41.407 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:41 smithi107 ceph-mon[28739]: pgmap v94: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:46:41.609 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:41 smithi169 ceph-mon[36313]: pgmap v94: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:46:43.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:43.133+0000 7febd3a00700 1 -- 172.21.15.107:0/366091340 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7febcc103dc0 msgr2=0x7febcc104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:46:43.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:43.133+0000 7febd3a00700 1 --2- 172.21.15.107:0/366091340 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7febcc103dc0 0x7febcc104210 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7febb8004940 tx=0x7febb8009e30 comp rx=0 tx=0).stop 2024-03-04T13:46:43.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:43.133+0000 7febd3a00700 1 -- 172.21.15.107:0/366091340 shutdown_connections 2024-03-04T13:46:43.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:43.133+0000 7febd3a00700 1 --2- 172.21.15.107:0/366091340 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7febcc103dc0 0x7febcc104210 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:46:43.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:43.133+0000 7febd3a00700 1 --2- 172.21.15.107:0/366091340 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7febcc1095f0 0x7febcc1099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:46:43.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:43.133+0000 7febd3a00700 1 -- 172.21.15.107:0/366091340 >> 172.21.15.107:0/366091340 conn(0x7febcc072d50 msgr2=0x7febcc075160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:46:43.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:43.134+0000 7febd3a00700 1 -- 172.21.15.107:0/366091340 shutdown_connections 2024-03-04T13:46:43.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:43.134+0000 7febd3a00700 1 -- 172.21.15.107:0/366091340 wait complete. 2024-03-04T13:46:43.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:43.134+0000 7febd3a00700 1 Processor -- start 2024-03-04T13:46:43.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:43.134+0000 7febd3a00700 1 -- start start 2024-03-04T13:46:43.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:43.135+0000 7febd3a00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7febcc103dc0 0x7febcc11a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:46:43.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:43.135+0000 7febd3a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7febcc1095f0 0x7febcc11a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:46:43.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:43.135+0000 7febd3a00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febcc11ac50 con 0x7febcc1095f0 2024-03-04T13:46:43.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:43.135+0000 7febd3a00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febcc113ed0 con 0x7febcc103dc0 2024-03-04T13:46:43.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:43.135+0000 7febcbe00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7febcc1095f0 0x7febcc11a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:46:43.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:43.135+0000 7febcbe00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7febcc1095f0 0x7febcc11a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52954/0 (socket says 172.21.15.107:52954) 2024-03-04T13:46:43.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:43.135+0000 7febcbe00700 1 -- 172.21.15.107:0/2438435141 learned_addr learned my addr 172.21.15.107:0/2438435141 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:46:43.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:43.136+0000 7febd1000700 1 --2- 172.21.15.107:0/2438435141 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7febcc103dc0 0x7febcc11a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:46:43.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:43.136+0000 7febcbe00700 1 -- 172.21.15.107:0/2438435141 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7febcc103dc0 msgr2=0x7febcc11a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:46:43.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:43.136+0000 7febcbe00700 1 --2- 172.21.15.107:0/2438435141 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7febcc103dc0 0x7febcc11a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:46:43.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:43.136+0000 7febcbe00700 1 -- 172.21.15.107:0/2438435141 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febb8005040 con 0x7febcc1095f0 2024-03-04T13:46:43.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:43.136+0000 7febcbe00700 1 --2- 172.21.15.107:0/2438435141 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7febcc1095f0 0x7febcc11a570 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7febb8009fd0 tx=0x7febb8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:46:43.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:43.136+0000 7febc8c00700 1 -- 172.21.15.107:0/2438435141 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7febb800e040 con 0x7febcc1095f0 2024-03-04T13:46:43.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:43.136+0000 7febd3a00700 1 -- 172.21.15.107:0/2438435141 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febcc1140f0 con 0x7febcc1095f0 2024-03-04T13:46:43.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:43.136+0000 7febc8c00700 1 -- 172.21.15.107:0/2438435141 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7febb800bbf0 con 0x7febcc1095f0 2024-03-04T13:46:43.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:43.136+0000 7febc8c00700 1 -- 172.21.15.107:0/2438435141 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7febb801c3f0 con 0x7febcc1095f0 2024-03-04T13:46:43.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:43.136+0000 7febd3a00700 1 -- 172.21.15.107:0/2438435141 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febcc114630 con 0x7febcc1095f0 2024-03-04T13:46:43.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:43.138+0000 7febd3a00700 1 -- 172.21.15.107:0/2438435141 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febcc075870 con 0x7febcc1095f0 2024-03-04T13:46:43.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:43.139+0000 7febc8c00700 1 -- 172.21.15.107:0/2438435141 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 33) v1 ==== 91199+0+0 (secure 0 0 0) 0x7febb800bd60 con 0x7febcc1095f0 2024-03-04T13:46:43.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:43.141+0000 7febc8c00700 1 --2- 172.21.15.107:0/2438435141 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7febb406d0f0 0x7febb406f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:46:43.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:43.141+0000 7febc8c00700 1 -- 172.21.15.107:0/2438435141 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7febb808d790 con 0x7febcc1095f0 2024-03-04T13:46:43.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:43.141+0000 7febd1000700 1 --2- 172.21.15.107:0/2438435141 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7febb406d0f0 0x7febb406f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:46:43.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:43.142+0000 7febd1000700 1 --2- 172.21.15.107:0/2438435141 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7febb406d0f0 0x7febb406f5a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7febc0006fd0 tx=0x7febc0008040 comp rx=0 tx=0).ready entity=mgr.14696 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:46:43.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:43.145+0000 7febc8c00700 1 -- 172.21.15.107:0/2438435141 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7febb805b480 con 0x7febcc1095f0 2024-03-04T13:46:43.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:43.376+0000 7febd3a00700 1 -- 172.21.15.107:0/2438435141 --> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7febcc1156b0 con 0x7febb406d0f0 2024-03-04T13:46:43.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:43.391+0000 7febc8c00700 1 -- 172.21.15.107:0/2438435141 <== mgr.14696 v2:172.21.15.107:6800/1613321046 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7febcc1156b0 con 0x7febb406d0f0 2024-03-04T13:46:43.392 INFO:teuthology.orchestra.run.smithi107.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-04T13:46:43.392 INFO:teuthology.orchestra.run.smithi107.stdout:alertmanager.smithi107 smithi107 *:9093,9094 running (61s) 30s ago 15m 13.8M - 0.25.0 c8568f914cd2 e5ed7be32538 2024-03-04T13:46:43.392 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi107 smithi107 running (15m) 30s ago 15m 6665k - 17.2.7-545-g42608bb1 9eec3f22859d 03e1570c2eff 2024-03-04T13:46:43.392 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi169 smithi169 running (14m) 110s ago 13m 6656k - 17.2.7-545-g42608bb1 9eec3f22859d 9c8881a5338e 2024-03-04T13:46:43.393 INFO:teuthology.orchestra.run.smithi107.stdout:grafana.smithi107 smithi107 *:3000 running (37s) 30s ago 14m 50.6M - 9.4.12 a0ea5e3c46d9 c5e4d33fb599 2024-03-04T13:46:43.393 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.riexmm smithi107 running (8m) 30s ago 8m 13.2M - 17.2.7-545-g42608bb1 9eec3f22859d fc921f7d4328 2024-03-04T13:46:43.393 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.thycho smithi107 running (8m) 30s ago 8m 98.1M - 17.2.7-545-g42608bb1 9eec3f22859d 4069c6e0e063 2024-03-04T13:46:43.393 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi169.bxitzk smithi169 running (7m) 110s ago 7m 13.0M - 17.2.7-545-g42608bb1 9eec3f22859d 7d4115228c7c 2024-03-04T13:46:43.393 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi169.jusudl smithi169 running (8m) 110s ago 8m 210M - 17.2.7-545-g42608bb1 9eec3f22859d 2d2f2cc452e8 2024-03-04T13:46:43.393 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi107.wpzgwl smithi107 *:8443,9283,8765 running (3m) 30s ago 16m 501M - 19.0.0-1845-g717ce3cc d71b839abdf8 1c5fdee45eac 2024-03-04T13:46:43.393 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi169.eihczr smithi169 *:8443,9283,8765 running (2m) 110s ago 13m 447M - 19.0.0-1845-g717ce3cc d71b839abdf8 b5cac2b7d22a 2024-03-04T13:46:43.393 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi107 smithi107 running (16m) 30s ago 17m 54.3M 2048M 17.2.7-545-g42608bb1 9eec3f22859d 49f5da990d35 2024-03-04T13:46:43.393 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi169 smithi169 running (13m) 110s ago 13m 42.4M 2048M 17.2.7-545-g42608bb1 9eec3f22859d ba4ee916297f 2024-03-04T13:46:43.393 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi107 smithi107 *:9100 running (2m) 30s ago 15m 7780k - 1.5.0 0da6a335fe13 8a8d3d4f724f 2024-03-04T13:46:43.393 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi169 smithi169 *:9100 running (118s) 110s ago 13m 2935k - 1.5.0 0da6a335fe13 f240f5a975bc 2024-03-04T13:46:43.393 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0 smithi107 running (12m) 30s ago 12m 206M 1558M 17.2.7-545-g42608bb1 9eec3f22859d c0605b68f696 2024-03-04T13:46:43.393 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1 smithi107 running (12m) 30s ago 12m 219M 1558M 17.2.7-545-g42608bb1 9eec3f22859d 5fb2c0dd5769 2024-03-04T13:46:43.393 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2 smithi107 running (11m) 30s ago 11m 181M 1558M 17.2.7-545-g42608bb1 9eec3f22859d 61f30f643549 2024-03-04T13:46:43.393 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3 smithi169 running (11m) 110s ago 11m 222M 2582M 17.2.7-545-g42608bb1 9eec3f22859d cd91d8694764 2024-03-04T13:46:43.394 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4 smithi169 running (10m) 110s ago 10m 194M 2582M 17.2.7-545-g42608bb1 9eec3f22859d 08545e3d0659 2024-03-04T13:46:43.394 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5 smithi169 running (9m) 110s ago 9m 180M 2582M 17.2.7-545-g42608bb1 9eec3f22859d dcffe6cb3eac 2024-03-04T13:46:43.394 INFO:teuthology.orchestra.run.smithi107.stdout:prometheus.smithi107 smithi107 *:9095 running (87s) 30s ago 14m 46.4M - 2.43.0 a07b618ecd1d 1eeb1ad5cb75 2024-03-04T13:46:43.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:43.395+0000 7febd3a00700 1 -- 172.21.15.107:0/2438435141 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7febb406d0f0 msgr2=0x7febb406f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:46:43.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:43.395+0000 7febd3a00700 1 --2- 172.21.15.107:0/2438435141 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7febb406d0f0 0x7febb406f5a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7febc0006fd0 tx=0x7febc0008040 comp rx=0 tx=0).stop 2024-03-04T13:46:43.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:43.395+0000 7febd3a00700 1 -- 172.21.15.107:0/2438435141 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7febcc1095f0 msgr2=0x7febcc11a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:46:43.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:43.395+0000 7febd3a00700 1 --2- 172.21.15.107:0/2438435141 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7febcc1095f0 0x7febcc11a570 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7febb8009fd0 tx=0x7febb8009e30 comp rx=0 tx=0).stop 2024-03-04T13:46:43.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:43.395+0000 7febd3a00700 1 -- 172.21.15.107:0/2438435141 shutdown_connections 2024-03-04T13:46:43.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:43.395+0000 7febd3a00700 1 --2- 172.21.15.107:0/2438435141 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7febcc103dc0 0x7febcc11a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:46:43.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:43.395+0000 7febd3a00700 1 --2- 172.21.15.107:0/2438435141 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7febb406d0f0 0x7febb406f5a0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:46:43.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:43.395+0000 7febd3a00700 1 --2- 172.21.15.107:0/2438435141 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7febcc1095f0 0x7febcc11a570 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:46:43.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:43.395+0000 7febd3a00700 1 -- 172.21.15.107:0/2438435141 >> 172.21.15.107:0/2438435141 conn(0x7febcc072d50 msgr2=0x7febcc0746b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:46:43.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:43.395+0000 7febd3a00700 1 -- 172.21.15.107:0/2438435141 shutdown_connections 2024-03-04T13:46:43.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:43.395+0000 7febd3a00700 1 -- 172.21.15.107:0/2438435141 wait complete. 2024-03-04T13:46:43.592 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:43 smithi107 ceph-mon[28739]: pgmap v95: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:46:43.609 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:43 smithi169 ceph-mon[36313]: pgmap v95: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:46:44.196 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-03-04T13:46:44.197 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi107.front.sepia.ceph.com 2024-03-04T13:46:44.198 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 -e sha1=717ce3cce8d0166e4e577faa8004e6f9cb4128c0 -- bash -c 'ceph config set mgr mgr/orchestrator/fail_fs true' 2024-03-04T13:46:44.592 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:44 smithi107 ceph-mon[28739]: from='client.14844 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:46:44.609 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:44 smithi169 ceph-mon[36313]: from='client.14844 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:46:44.680 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/mon.smithi107/config 2024-03-04T13:46:45.592 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:45 smithi107 ceph-mon[28739]: pgmap v96: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:46:45.609 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:45 smithi169 ceph-mon[36313]: pgmap v96: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:46:46.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:46.646+0000 7fe020a00700 1 -- 172.21.15.107:0/1304818944 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe01c072dd0 msgr2=0x7fe01c10d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:46:46.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:46.646+0000 7fe020a00700 1 --2- 172.21.15.107:0/1304818944 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe01c072dd0 0x7fe01c10d8e0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fe008004940 tx=0x7fe008009e30 comp rx=0 tx=0).stop 2024-03-04T13:46:46.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:46.646+0000 7fe020a00700 1 -- 172.21.15.107:0/1304818944 shutdown_connections 2024-03-04T13:46:46.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:46.646+0000 7fe020a00700 1 --2- 172.21.15.107:0/1304818944 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe01c072dd0 0x7fe01c10d8e0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:46:46.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:46.646+0000 7fe020a00700 1 --2- 172.21.15.107:0/1304818944 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe01c0724c0 0x7fe01c072890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:46:46.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:46.646+0000 7fe020a00700 1 -- 172.21.15.107:0/1304818944 >> 172.21.15.107:0/1304818944 conn(0x7fe01c0face0 msgr2=0x7fe01c0fd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:46:46.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:46.646+0000 7fe020a00700 1 -- 172.21.15.107:0/1304818944 shutdown_connections 2024-03-04T13:46:46.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:46.646+0000 7fe020a00700 1 -- 172.21.15.107:0/1304818944 wait complete. 2024-03-04T13:46:46.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:46.647+0000 7fe020a00700 1 Processor -- start 2024-03-04T13:46:46.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:46.648+0000 7fe020a00700 1 -- start start 2024-03-04T13:46:46.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:46.648+0000 7fe020a00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe01c0724c0 0x7fe01c11a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:46:46.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:46.648+0000 7fe020a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe01c072dd0 0x7fe01c11a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:46:46.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:46.648+0000 7fe020a00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe01c11ac40 con 0x7fe01c072dd0 2024-03-04T13:46:46.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:46.648+0000 7fe020a00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe01c113ec0 con 0x7fe01c0724c0 2024-03-04T13:46:46.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:46.648+0000 7fe019400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe01c072dd0 0x7fe01c11a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:46:46.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:46.648+0000 7fe019400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe01c072dd0 0x7fe01c11a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52980/0 (socket says 172.21.15.107:52980) 2024-03-04T13:46:46.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:46.648+0000 7fe019400700 1 -- 172.21.15.107:0/3846672435 learned_addr learned my addr 172.21.15.107:0/3846672435 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:46:46.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:46.649+0000 7fe019e00700 1 --2- 172.21.15.107:0/3846672435 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe01c0724c0 0x7fe01c11a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:46:46.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:46.649+0000 7fe019400700 1 -- 172.21.15.107:0/3846672435 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe01c0724c0 msgr2=0x7fe01c11a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:46:46.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:46.649+0000 7fe019400700 1 --2- 172.21.15.107:0/3846672435 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe01c0724c0 0x7fe01c11a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:46:46.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:46.649+0000 7fe019400700 1 -- 172.21.15.107:0/3846672435 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe008005040 con 0x7fe01c072dd0 2024-03-04T13:46:46.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:46.649+0000 7fe019e00700 1 --2- 172.21.15.107:0/3846672435 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe01c0724c0 0x7fe01c11a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T13:46:46.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:46.649+0000 7fe019400700 1 --2- 172.21.15.107:0/3846672435 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe01c072dd0 0x7fe01c11a560 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fe008000c00 tx=0x7fe008009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:46:46.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:46.649+0000 7fe012000700 1 -- 172.21.15.107:0/3846672435 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe00800e040 con 0x7fe01c072dd0 2024-03-04T13:46:46.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:46.649+0000 7fe012000700 1 -- 172.21.15.107:0/3846672435 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe008009af0 con 0x7fe01c072dd0 2024-03-04T13:46:46.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:46.649+0000 7fe020a00700 1 -- 172.21.15.107:0/3846672435 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe01c1140e0 con 0x7fe01c072dd0 2024-03-04T13:46:46.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:46.649+0000 7fe012000700 1 -- 172.21.15.107:0/3846672435 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe00801c630 con 0x7fe01c072dd0 2024-03-04T13:46:46.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:46.650+0000 7fe020a00700 1 -- 172.21.15.107:0/3846672435 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe01c114620 con 0x7fe01c072dd0 2024-03-04T13:46:46.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:46.651+0000 7fe020a00700 1 -- 172.21.15.107:0/3846672435 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe01c10b010 con 0x7fe01c072dd0 2024-03-04T13:46:46.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:46.651+0000 7fe012000700 1 -- 172.21.15.107:0/3846672435 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 33) v1 ==== 91199+0+0 (secure 0 0 0) 0x7fe00800bbf0 con 0x7fe01c072dd0 2024-03-04T13:46:46.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:46.652+0000 7fe012000700 1 --2- 172.21.15.107:0/3846672435 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7fe00006d0e0 0x7fe00006f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:46:46.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:46.652+0000 7fe012000700 1 -- 172.21.15.107:0/3846672435 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fe00808d980 con 0x7fe01c072dd0 2024-03-04T13:46:46.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:46.653+0000 7fe019e00700 1 --2- 172.21.15.107:0/3846672435 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7fe00006d0e0 0x7fe00006f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:46:46.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:46.653+0000 7fe019e00700 1 --2- 172.21.15.107:0/3846672435 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7fe00006d0e0 0x7fe00006f590 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe00c006fd0 tx=0x7fe00c008040 comp rx=0 tx=0).ready entity=mgr.14696 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:46:46.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:46.657+0000 7fe012000700 1 -- 172.21.15.107:0/3846672435 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7fe00805b530 con 0x7fe01c072dd0 2024-03-04T13:46:46.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:46.879+0000 7fe020a00700 1 -- 172.21.15.107:0/3846672435 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command([{prefix=config set, name=mgr/orchestrator/fail_fs}] v 0) v1 -- 0x7fe01c0fe900 con 0x7fe01c072dd0 2024-03-04T13:46:46.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:46.890+0000 7fe012000700 1 -- 172.21.15.107:0/3846672435 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/orchestrator/fail_fs}]=0 v47)=0 v47) v1 ==== 125+0+0 (secure 0 0 0) 0x7fe00805b0c0 con 0x7fe01c072dd0 2024-03-04T13:46:46.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:46.894+0000 7fe020a00700 1 -- 172.21.15.107:0/3846672435 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7fe00006d0e0 msgr2=0x7fe00006f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:46:46.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:46.894+0000 7fe020a00700 1 --2- 172.21.15.107:0/3846672435 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7fe00006d0e0 0x7fe00006f590 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe00c006fd0 tx=0x7fe00c008040 comp rx=0 tx=0).stop 2024-03-04T13:46:46.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:46.894+0000 7fe020a00700 1 -- 172.21.15.107:0/3846672435 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe01c072dd0 msgr2=0x7fe01c11a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:46:46.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:46.894+0000 7fe020a00700 1 --2- 172.21.15.107:0/3846672435 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe01c072dd0 0x7fe01c11a560 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fe008000c00 tx=0x7fe008009e30 comp rx=0 tx=0).stop 2024-03-04T13:46:46.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:46.894+0000 7fe020a00700 1 -- 172.21.15.107:0/3846672435 shutdown_connections 2024-03-04T13:46:46.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:46.894+0000 7fe020a00700 1 --2- 172.21.15.107:0/3846672435 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe01c0724c0 0x7fe01c11a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:46:46.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:46.894+0000 7fe020a00700 1 --2- 172.21.15.107:0/3846672435 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7fe00006d0e0 0x7fe00006f590 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:46:46.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:46.894+0000 7fe020a00700 1 --2- 172.21.15.107:0/3846672435 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe01c072dd0 0x7fe01c11a560 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:46:46.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:46.894+0000 7fe020a00700 1 -- 172.21.15.107:0/3846672435 >> 172.21.15.107:0/3846672435 conn(0x7fe01c0face0 msgr2=0x7fe01c0fc9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:46:46.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:46.894+0000 7fe020a00700 1 -- 172.21.15.107:0/3846672435 shutdown_connections 2024-03-04T13:46:46.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:46.894+0000 7fe020a00700 1 -- 172.21.15.107:0/3846672435 wait complete. 2024-03-04T13:46:47.092 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:46 smithi107 ceph-mon[28739]: pgmap v97: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:46:47.109 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:46 smithi169 ceph-mon[36313]: pgmap v97: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:46:47.671 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-03-04T13:46:47.671 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi107.front.sepia.ceph.com 2024-03-04T13:46:47.671 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 -e sha1=717ce3cce8d0166e4e577faa8004e6f9cb4128c0 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force' 2024-03-04T13:46:48.147 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/mon.smithi107/config 2024-03-04T13:46:48.295 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:47 smithi107 ceph-mon[28739]: from='client.? 172.21.15.107:0/3846672435' entity='client.admin' 2024-03-04T13:46:48.295 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:47 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:46:48.295 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:47 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:46:48.295 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:47 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:46:48.295 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:47 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:46:48.359 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:47 smithi169 ceph-mon[36313]: from='client.? 172.21.15.107:0/3846672435' entity='client.admin' 2024-03-04T13:46:48.360 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:47 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:46:48.360 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:47 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:46:48.360 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:47 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:46:48.360 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:47 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:46:49.307 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:48 smithi107 ceph-mon[28739]: pgmap v98: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:46:49.359 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:48 smithi169 ceph-mon[36313]: pgmap v98: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:46:50.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:50.309+0000 7f4501e00700 1 -- 172.21.15.107:0/1782727491 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44fc103140 msgr2=0x7f44fc103510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:46:50.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:50.309+0000 7f4501e00700 1 --2- 172.21.15.107:0/1782727491 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44fc103140 0x7f44fc103510 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f44ec004940 tx=0x7f44ec009e30 comp rx=0 tx=0).stop 2024-03-04T13:46:50.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:50.309+0000 7f4501e00700 1 -- 172.21.15.107:0/1782727491 shutdown_connections 2024-03-04T13:46:50.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:50.309+0000 7f4501e00700 1 --2- 172.21.15.107:0/1782727491 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44fc0fb570 0x7f44fc0fb9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:46:50.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:50.309+0000 7f4501e00700 1 --2- 172.21.15.107:0/1782727491 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44fc103140 0x7f44fc103510 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:46:50.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:50.309+0000 7f4501e00700 1 -- 172.21.15.107:0/1782727491 >> 172.21.15.107:0/1782727491 conn(0x7f44fc071560 msgr2=0x7f44fc071960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:46:50.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:50.309+0000 7f4501e00700 1 -- 172.21.15.107:0/1782727491 shutdown_connections 2024-03-04T13:46:50.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:50.309+0000 7f4501e00700 1 -- 172.21.15.107:0/1782727491 wait complete. 2024-03-04T13:46:50.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:50.310+0000 7f4501e00700 1 Processor -- start 2024-03-04T13:46:50.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:50.310+0000 7f4501e00700 1 -- start start 2024-03-04T13:46:50.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:50.310+0000 7f4501e00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44fc0fb570 0x7f44fc115da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:46:50.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:50.310+0000 7f4501e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44fc103140 0x7f44fc1162e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:46:50.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:50.310+0000 7f4501e00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44fc1169c0 con 0x7f44fc103140 2024-03-04T13:46:50.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:50.310+0000 7f4501e00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44fc10fc40 con 0x7f44fc0fb570 2024-03-04T13:46:50.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:50.311+0000 7f44fa000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44fc103140 0x7f44fc1162e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:46:50.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:50.311+0000 7f44fa000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44fc103140 0x7f44fc1162e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34680/0 (socket says 172.21.15.107:34680) 2024-03-04T13:46:50.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:50.311+0000 7f44fa000700 1 -- 172.21.15.107:0/3579737080 learned_addr learned my addr 172.21.15.107:0/3579737080 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:46:50.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:50.311+0000 7f44faa00700 1 --2- 172.21.15.107:0/3579737080 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44fc0fb570 0x7f44fc115da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:46:50.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:50.311+0000 7f44fa000700 1 -- 172.21.15.107:0/3579737080 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44fc0fb570 msgr2=0x7f44fc115da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:46:50.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:50.311+0000 7f44fa000700 1 --2- 172.21.15.107:0/3579737080 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44fc0fb570 0x7f44fc115da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:46:50.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:50.311+0000 7f44fa000700 1 -- 172.21.15.107:0/3579737080 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44ec005040 con 0x7f44fc103140 2024-03-04T13:46:50.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:50.312+0000 7f44fa000700 1 --2- 172.21.15.107:0/3579737080 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44fc103140 0x7f44fc1162e0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f44e800b790 tx=0x7f44e8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:46:50.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:50.312+0000 7f44f2a00700 1 -- 172.21.15.107:0/3579737080 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f44e800f770 con 0x7f44fc103140 2024-03-04T13:46:50.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:50.312+0000 7f44f2a00700 1 -- 172.21.15.107:0/3579737080 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f44e800fda0 con 0x7f44fc103140 2024-03-04T13:46:50.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:50.312+0000 7f4501e00700 1 -- 172.21.15.107:0/3579737080 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44fc10fec0 con 0x7f44fc103140 2024-03-04T13:46:50.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:50.312+0000 7f44f2a00700 1 -- 172.21.15.107:0/3579737080 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f44e800d310 con 0x7f44fc103140 2024-03-04T13:46:50.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:50.312+0000 7f4501e00700 1 -- 172.21.15.107:0/3579737080 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44fc110400 con 0x7f44fc103140 2024-03-04T13:46:50.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:50.313+0000 7f4501e00700 1 -- 172.21.15.107:0/3579737080 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44fc073080 con 0x7f44fc103140 2024-03-04T13:46:50.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:50.314+0000 7f44f2a00700 1 -- 172.21.15.107:0/3579737080 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 33) v1 ==== 91199+0+0 (secure 0 0 0) 0x7f44e8010420 con 0x7f44fc103140 2024-03-04T13:46:50.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:50.315+0000 7f44f2a00700 1 --2- 172.21.15.107:0/3579737080 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f44e006d0e0 0x7f44e006f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:46:50.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:50.315+0000 7f44f2a00700 1 -- 172.21.15.107:0/3579737080 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f44e808c5a0 con 0x7f44fc103140 2024-03-04T13:46:50.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:50.315+0000 7f44faa00700 1 --2- 172.21.15.107:0/3579737080 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f44e006d0e0 0x7f44e006f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:46:50.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:50.316+0000 7f44faa00700 1 --2- 172.21.15.107:0/3579737080 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f44e006d0e0 0x7f44e006f590 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f44ec004f90 tx=0x7f44ec009e30 comp rx=0 tx=0).ready entity=mgr.14696 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:46:50.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:50.320+0000 7f44f2a00700 1 -- 172.21.15.107:0/3579737080 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f44e805a1f0 con 0x7f44fc103140 2024-03-04T13:46:50.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:50.522+0000 7f4501e00700 1 -- 172.21.15.107:0/3579737080 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}] v 0) v1 -- 0x7f44fc111380 con 0x7f44fc103140 2024-03-04T13:46:50.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:50.523+0000 7f44f2a00700 1 -- 172.21.15.107:0/3579737080 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}]=0 v47)=0 v47) v1 ==== 155+0+0 (secure 0 0 0) 0x7f44e8059d80 con 0x7f44fc103140 2024-03-04T13:46:50.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:50.525+0000 7f4501e00700 1 -- 172.21.15.107:0/3579737080 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f44e006d0e0 msgr2=0x7f44e006f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:46:50.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:50.525+0000 7f4501e00700 1 --2- 172.21.15.107:0/3579737080 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f44e006d0e0 0x7f44e006f590 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f44ec004f90 tx=0x7f44ec009e30 comp rx=0 tx=0).stop 2024-03-04T13:46:50.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:50.525+0000 7f4501e00700 1 -- 172.21.15.107:0/3579737080 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44fc103140 msgr2=0x7f44fc1162e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:46:50.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:50.525+0000 7f4501e00700 1 --2- 172.21.15.107:0/3579737080 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44fc103140 0x7f44fc1162e0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f44e800b790 tx=0x7f44e8009250 comp rx=0 tx=0).stop 2024-03-04T13:46:50.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:50.526+0000 7f4501e00700 1 -- 172.21.15.107:0/3579737080 shutdown_connections 2024-03-04T13:46:50.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:50.526+0000 7f4501e00700 1 --2- 172.21.15.107:0/3579737080 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44fc0fb570 0x7f44fc115da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:46:50.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:50.526+0000 7f4501e00700 1 --2- 172.21.15.107:0/3579737080 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f44e006d0e0 0x7f44e006f590 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:46:50.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:50.526+0000 7f4501e00700 1 --2- 172.21.15.107:0/3579737080 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44fc103140 0x7f44fc1162e0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:46:50.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:50.526+0000 7f4501e00700 1 -- 172.21.15.107:0/3579737080 >> 172.21.15.107:0/3579737080 conn(0x7f44fc071560 msgr2=0x7f44fc0f9560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:46:50.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:50.526+0000 7f4501e00700 1 -- 172.21.15.107:0/3579737080 shutdown_connections 2024-03-04T13:46:50.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:50.526+0000 7f4501e00700 1 -- 172.21.15.107:0/3579737080 wait complete. 2024-03-04T13:46:51.306 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 -e sha1=717ce3cce8d0166e4e577faa8004e6f9cb4128c0 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force' 2024-03-04T13:46:51.592 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:51 smithi107 ceph-mon[28739]: pgmap v99: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:46:51.609 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:51 smithi169 ceph-mon[36313]: pgmap v99: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:46:51.792 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/mon.smithi107/config 2024-03-04T13:46:53.452 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:53 smithi107 ceph-mon[28739]: pgmap v100: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:46:53.452 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:53 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:46:53.609 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:53 smithi169 ceph-mon[36313]: pgmap v100: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:46:53.609 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:53 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:46:53.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:53.956+0000 7f6790a00700 1 -- 172.21.15.107:0/956827834 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f678c103dc0 msgr2=0x7f678c104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:46:53.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:53.956+0000 7f6790a00700 1 --2- 172.21.15.107:0/956827834 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f678c103dc0 0x7f678c104210 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f6774004940 tx=0x7f6774009e30 comp rx=0 tx=0).stop 2024-03-04T13:46:53.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:53.957+0000 7f6790a00700 1 -- 172.21.15.107:0/956827834 shutdown_connections 2024-03-04T13:46:53.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:53.957+0000 7f6790a00700 1 --2- 172.21.15.107:0/956827834 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f678c103dc0 0x7f678c104210 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:46:53.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:53.957+0000 7f6790a00700 1 --2- 172.21.15.107:0/956827834 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f678c1095f0 0x7f678c1099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:46:53.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:53.957+0000 7f6790a00700 1 -- 172.21.15.107:0/956827834 >> 172.21.15.107:0/956827834 conn(0x7f678c072d50 msgr2=0x7f678c075160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:46:53.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:53.957+0000 7f6790a00700 1 -- 172.21.15.107:0/956827834 shutdown_connections 2024-03-04T13:46:53.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:53.957+0000 7f6790a00700 1 -- 172.21.15.107:0/956827834 wait complete. 2024-03-04T13:46:53.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:53.957+0000 7f6790a00700 1 Processor -- start 2024-03-04T13:46:53.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:53.958+0000 7f6790a00700 1 -- start start 2024-03-04T13:46:53.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:53.958+0000 7f6790a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f678c103dc0 0x7f678c119fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:46:53.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:53.958+0000 7f6790a00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f678c1095f0 0x7f678c11a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:46:53.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:53.958+0000 7f6790a00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f678c11ab70 con 0x7f678c103dc0 2024-03-04T13:46:53.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:53.958+0000 7f6790a00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f678c113ef0 con 0x7f678c1095f0 2024-03-04T13:46:53.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:53.958+0000 7f6789e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f678c103dc0 0x7f678c119fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:46:53.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:53.958+0000 7f6789e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f678c103dc0 0x7f678c119fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34700/0 (socket says 172.21.15.107:34700) 2024-03-04T13:46:53.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:53.958+0000 7f6789e00700 1 -- 172.21.15.107:0/2438281826 learned_addr learned my addr 172.21.15.107:0/2438281826 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:46:53.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:53.959+0000 7f6789e00700 1 -- 172.21.15.107:0/2438281826 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f678c1095f0 msgr2=0x7f678c11a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:46:53.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:53.959+0000 7f6789400700 1 --2- 172.21.15.107:0/2438281826 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f678c1095f0 0x7f678c11a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:46:53.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:53.959+0000 7f6789e00700 1 --2- 172.21.15.107:0/2438281826 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f678c1095f0 0x7f678c11a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:46:53.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:53.959+0000 7f6789e00700 1 -- 172.21.15.107:0/2438281826 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6774005040 con 0x7f678c103dc0 2024-03-04T13:46:53.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:53.959+0000 7f6789e00700 1 --2- 172.21.15.107:0/2438281826 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f678c103dc0 0x7f678c119fe0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f677c00c810 tx=0x7f677c004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:46:53.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:53.959+0000 7f6782000700 1 -- 172.21.15.107:0/2438281826 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f677c012810 con 0x7f678c103dc0 2024-03-04T13:46:53.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:53.959+0000 7f6782000700 1 -- 172.21.15.107:0/2438281826 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f677c012e40 con 0x7f678c103dc0 2024-03-04T13:46:53.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:53.959+0000 7f6790a00700 1 -- 172.21.15.107:0/2438281826 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f678c1141d0 con 0x7f678c103dc0 2024-03-04T13:46:53.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:53.959+0000 7f6782000700 1 -- 172.21.15.107:0/2438281826 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f677c00a370 con 0x7f678c103dc0 2024-03-04T13:46:53.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:53.959+0000 7f6790a00700 1 -- 172.21.15.107:0/2438281826 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f678c114710 con 0x7f678c103dc0 2024-03-04T13:46:53.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:53.961+0000 7f6790a00700 1 -- 172.21.15.107:0/2438281826 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f678c10b9c0 con 0x7f678c103dc0 2024-03-04T13:46:53.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:53.962+0000 7f6782000700 1 -- 172.21.15.107:0/2438281826 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 33) v1 ==== 91199+0+0 (secure 0 0 0) 0x7f677c011810 con 0x7f678c103dc0 2024-03-04T13:46:53.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:53.963+0000 7f6782000700 1 --2- 172.21.15.107:0/2438281826 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f677006d0f0 0x7f677006f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:46:53.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:53.963+0000 7f6782000700 1 -- 172.21.15.107:0/2438281826 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f677c08ca50 con 0x7f678c103dc0 2024-03-04T13:46:53.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:53.963+0000 7f6789400700 1 --2- 172.21.15.107:0/2438281826 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f677006d0f0 0x7f677006f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:46:53.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:53.964+0000 7f6789400700 1 --2- 172.21.15.107:0/2438281826 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f677006d0f0 0x7f677006f5a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f678c115880 tx=0x7f6774009e30 comp rx=0 tx=0).ready entity=mgr.14696 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:46:53.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:53.968+0000 7f6782000700 1 -- 172.21.15.107:0/2438281826 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f677c05a580 con 0x7f678c103dc0 2024-03-04T13:46:54.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:54.186+0000 7f6790a00700 1 -- 172.21.15.107:0/2438281826 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}] v 0) v1 -- 0x7f678c075870 con 0x7f678c103dc0 2024-03-04T13:46:54.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:54.187+0000 7f6782000700 1 -- 172.21.15.107:0/2438281826 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}]=0 v47)=0 v47) v1 ==== 163+0+0 (secure 0 0 0) 0x7f677c05a110 con 0x7f678c103dc0 2024-03-04T13:46:54.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:54.190+0000 7f6790a00700 1 -- 172.21.15.107:0/2438281826 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f677006d0f0 msgr2=0x7f677006f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:46:54.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:54.190+0000 7f6790a00700 1 --2- 172.21.15.107:0/2438281826 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f677006d0f0 0x7f677006f5a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f678c115880 tx=0x7f6774009e30 comp rx=0 tx=0).stop 2024-03-04T13:46:54.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:54.190+0000 7f6790a00700 1 -- 172.21.15.107:0/2438281826 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f678c103dc0 msgr2=0x7f678c119fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:46:54.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:54.190+0000 7f6790a00700 1 --2- 172.21.15.107:0/2438281826 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f678c103dc0 0x7f678c119fe0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f677c00c810 tx=0x7f677c004470 comp rx=0 tx=0).stop 2024-03-04T13:46:54.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:54.190+0000 7f6790a00700 1 -- 172.21.15.107:0/2438281826 shutdown_connections 2024-03-04T13:46:54.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:54.190+0000 7f6790a00700 1 --2- 172.21.15.107:0/2438281826 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f677006d0f0 0x7f677006f5a0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:46:54.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:54.190+0000 7f6790a00700 1 --2- 172.21.15.107:0/2438281826 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f678c103dc0 0x7f678c119fe0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:46:54.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:54.190+0000 7f6790a00700 1 --2- 172.21.15.107:0/2438281826 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f678c1095f0 0x7f678c11a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:46:54.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:54.190+0000 7f6790a00700 1 -- 172.21.15.107:0/2438281826 >> 172.21.15.107:0/2438281826 conn(0x7f678c072d50 msgr2=0x7f678c074810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:46:54.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:54.190+0000 7f6790a00700 1 -- 172.21.15.107:0/2438281826 shutdown_connections 2024-03-04T13:46:54.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:54.191+0000 7f6790a00700 1 -- 172.21.15.107:0/2438281826 wait complete. 2024-03-04T13:46:55.102 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 -e sha1=717ce3cce8d0166e4e577faa8004e6f9cb4128c0 -- bash -c 'ceph config set global log_to_journald false --force' 2024-03-04T13:46:55.342 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:55 smithi107 ceph-mon[28739]: pgmap v101: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:46:55.572 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/mon.smithi107/config 2024-03-04T13:46:55.609 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:55 smithi169 ceph-mon[36313]: pgmap v101: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:46:57.092 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:56 smithi107 ceph-mon[28739]: pgmap v102: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:46:57.109 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:56 smithi169 ceph-mon[36313]: pgmap v102: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:46:58.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:58.034+0000 7fa860c00700 1 -- 172.21.15.107:0/1606016746 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa85c0fe960 msgr2=0x7fa85c106fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:46:58.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:58.034+0000 7fa860c00700 1 --2- 172.21.15.107:0/1606016746 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa85c0fe960 0x7fa85c106fa0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fa848004940 tx=0x7fa848009e30 comp rx=0 tx=0).stop 2024-03-04T13:46:58.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:58.034+0000 7fa860c00700 1 -- 172.21.15.107:0/1606016746 shutdown_connections 2024-03-04T13:46:58.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:58.034+0000 7fa860c00700 1 --2- 172.21.15.107:0/1606016746 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa85c0fe960 0x7fa85c106fa0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:46:58.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:58.034+0000 7fa860c00700 1 --2- 172.21.15.107:0/1606016746 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa85c0fdfc0 0x7fa85c0fe390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:46:58.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:58.034+0000 7fa860c00700 1 -- 172.21.15.107:0/1606016746 >> 172.21.15.107:0/1606016746 conn(0x7fa85c0f7a10 msgr2=0x7fa85c0f9e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:46:58.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:58.034+0000 7fa860c00700 1 -- 172.21.15.107:0/1606016746 shutdown_connections 2024-03-04T13:46:58.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:58.035+0000 7fa860c00700 1 -- 172.21.15.107:0/1606016746 wait complete. 2024-03-04T13:46:58.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:58.036+0000 7fa860c00700 1 Processor -- start 2024-03-04T13:46:58.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:58.036+0000 7fa860c00700 1 -- start start 2024-03-04T13:46:58.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:58.036+0000 7fa860c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa85c0fdfc0 0x7fa85c115cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:46:58.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:58.036+0000 7fa860c00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa85c0fe960 0x7fa85c116210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:46:58.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:58.036+0000 7fa860c00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa85c116860 con 0x7fa85c0fdfc0 2024-03-04T13:46:58.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:58.036+0000 7fa860c00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa85c10fb70 con 0x7fa85c0fe960 2024-03-04T13:46:58.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:58.037+0000 7fa859e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa85c0fdfc0 0x7fa85c115cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:46:58.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:58.037+0000 7fa859e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa85c0fdfc0 0x7fa85c115cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48188/0 (socket says 172.21.15.107:48188) 2024-03-04T13:46:58.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:58.037+0000 7fa859e00700 1 -- 172.21.15.107:0/3221848880 learned_addr learned my addr 172.21.15.107:0/3221848880 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:46:58.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:58.037+0000 7fa859e00700 1 -- 172.21.15.107:0/3221848880 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa85c0fe960 msgr2=0x7fa85c116210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:46:58.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:58.037+0000 7fa859400700 1 --2- 172.21.15.107:0/3221848880 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa85c0fe960 0x7fa85c116210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:46:58.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:58.037+0000 7fa859e00700 1 --2- 172.21.15.107:0/3221848880 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa85c0fe960 0x7fa85c116210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:46:58.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:58.037+0000 7fa859e00700 1 -- 172.21.15.107:0/3221848880 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa848005040 con 0x7fa85c0fdfc0 2024-03-04T13:46:58.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:58.037+0000 7fa859e00700 1 --2- 172.21.15.107:0/3221848880 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa85c0fdfc0 0x7fa85c115cd0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fa84c00c8c0 tx=0x7fa84c0044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:46:58.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:58.038+0000 7fa852000700 1 -- 172.21.15.107:0/3221848880 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa84c00a980 con 0x7fa85c0fdfc0 2024-03-04T13:46:58.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:58.038+0000 7fa852000700 1 -- 172.21.15.107:0/3221848880 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa84c00ce60 con 0x7fa85c0fdfc0 2024-03-04T13:46:58.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:58.038+0000 7fa852000700 1 -- 172.21.15.107:0/3221848880 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa84c0115e0 con 0x7fa85c0fdfc0 2024-03-04T13:46:58.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:58.038+0000 7fa860c00700 1 -- 172.21.15.107:0/3221848880 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa85c10fe50 con 0x7fa85c0fdfc0 2024-03-04T13:46:58.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:58.038+0000 7fa860c00700 1 -- 172.21.15.107:0/3221848880 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa85c110390 con 0x7fa85c0fdfc0 2024-03-04T13:46:58.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:58.039+0000 7fa860c00700 1 -- 172.21.15.107:0/3221848880 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa85c1069b0 con 0x7fa85c0fdfc0 2024-03-04T13:46:58.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:58.040+0000 7fa852000700 1 -- 172.21.15.107:0/3221848880 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 33) v1 ==== 91199+0+0 (secure 0 0 0) 0x7fa84c012400 con 0x7fa85c0fdfc0 2024-03-04T13:46:58.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:58.041+0000 7fa852000700 1 --2- 172.21.15.107:0/3221848880 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7fa84006d0e0 0x7fa84006f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:46:58.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:58.041+0000 7fa852000700 1 -- 172.21.15.107:0/3221848880 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fa84c08c5f0 con 0x7fa85c0fdfc0 2024-03-04T13:46:58.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:58.041+0000 7fa859400700 1 --2- 172.21.15.107:0/3221848880 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7fa84006d0e0 0x7fa84006f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:46:58.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:58.042+0000 7fa859400700 1 --2- 172.21.15.107:0/3221848880 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7fa84006d0e0 0x7fa84006f590 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa848004890 tx=0x7fa848009e30 comp rx=0 tx=0).ready entity=mgr.14696 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:46:58.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:58.050+0000 7fa852000700 1 -- 172.21.15.107:0/3221848880 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7fa84c05a220 con 0x7fa85c0fdfc0 2024-03-04T13:46:58.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:58.269+0000 7fa860c00700 1 -- 172.21.15.107:0/3221848880 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command([{prefix=config set, name=log_to_journald}] v 0) v1 -- 0x7fa85c0fb610 con 0x7fa85c0fdfc0 2024-03-04T13:46:58.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:58.270+0000 7fa852000700 1 -- 172.21.15.107:0/3221848880 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{prefix=config set, name=log_to_journald}]=0 v47)=0 v47) v1 ==== 135+0+0 (secure 0 0 0) 0x7fa84c0126b0 con 0x7fa85c0fdfc0 2024-03-04T13:46:58.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:58.273+0000 7fa860c00700 1 -- 172.21.15.107:0/3221848880 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7fa84006d0e0 msgr2=0x7fa84006f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:46:58.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:58.273+0000 7fa860c00700 1 --2- 172.21.15.107:0/3221848880 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7fa84006d0e0 0x7fa84006f590 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa848004890 tx=0x7fa848009e30 comp rx=0 tx=0).stop 2024-03-04T13:46:58.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:58.273+0000 7fa860c00700 1 -- 172.21.15.107:0/3221848880 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa85c0fdfc0 msgr2=0x7fa85c115cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:46:58.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:58.273+0000 7fa860c00700 1 --2- 172.21.15.107:0/3221848880 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa85c0fdfc0 0x7fa85c115cd0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fa84c00c8c0 tx=0x7fa84c0044c0 comp rx=0 tx=0).stop 2024-03-04T13:46:58.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:58.274+0000 7fa860c00700 1 -- 172.21.15.107:0/3221848880 shutdown_connections 2024-03-04T13:46:58.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:58.274+0000 7fa860c00700 1 --2- 172.21.15.107:0/3221848880 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7fa84006d0e0 0x7fa84006f590 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:46:58.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:58.274+0000 7fa860c00700 1 --2- 172.21.15.107:0/3221848880 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa85c0fdfc0 0x7fa85c115cd0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:46:58.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:58.274+0000 7fa860c00700 1 --2- 172.21.15.107:0/3221848880 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa85c0fe960 0x7fa85c116210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:46:58.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:58.274+0000 7fa860c00700 1 -- 172.21.15.107:0/3221848880 >> 172.21.15.107:0/3221848880 conn(0x7fa85c0f7a10 msgr2=0x7fa85c103b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:46:58.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:58.274+0000 7fa860c00700 1 -- 172.21.15.107:0/3221848880 shutdown_connections 2024-03-04T13:46:58.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:46:58.274+0000 7fa860c00700 1 -- 172.21.15.107:0/3221848880 wait complete. 2024-03-04T13:46:59.174 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 -e sha1=717ce3cce8d0166e4e577faa8004e6f9cb4128c0 -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1' 2024-03-04T13:46:59.481 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:46:59 smithi107 ceph-mon[28739]: pgmap v103: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:46:59.609 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:46:59 smithi169 ceph-mon[36313]: pgmap v103: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:46:59.660 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/mon.smithi107/config 2024-03-04T13:47:01.545 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:01 smithi107 ceph-mon[28739]: pgmap v104: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:47:01.609 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:01 smithi169 ceph-mon[36313]: pgmap v104: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:47:01.713 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:01.714+0000 7f3f27600700 1 -- 172.21.15.107:0/3880644584 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f200fdfe0 msgr2=0x7f3f200fe3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:47:01.713 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:01.714+0000 7f3f27600700 1 --2- 172.21.15.107:0/3880644584 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f200fdfe0 0x7f3f200fe3b0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f3f14004970 tx=0x7f3f14009e30 comp rx=0 tx=0).stop 2024-03-04T13:47:01.713 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:01.714+0000 7f3f27600700 1 -- 172.21.15.107:0/3880644584 shutdown_connections 2024-03-04T13:47:01.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:01.714+0000 7f3f27600700 1 --2- 172.21.15.107:0/3880644584 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3f200fe8f0 0x7f3f20107050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:47:01.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:01.714+0000 7f3f27600700 1 --2- 172.21.15.107:0/3880644584 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f200fdfe0 0x7f3f200fe3b0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:47:01.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:01.714+0000 7f3f27600700 1 -- 172.21.15.107:0/3880644584 >> 172.21.15.107:0/3880644584 conn(0x7f3f20071d90 msgr2=0x7f3f20072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:47:01.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:01.714+0000 7f3f27600700 1 -- 172.21.15.107:0/3880644584 shutdown_connections 2024-03-04T13:47:01.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:01.714+0000 7f3f27600700 1 -- 172.21.15.107:0/3880644584 wait complete. 2024-03-04T13:47:01.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:01.715+0000 7f3f27600700 1 Processor -- start 2024-03-04T13:47:01.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:01.715+0000 7f3f27600700 1 -- start start 2024-03-04T13:47:01.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:01.715+0000 7f3f27600700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3f200fdfe0 0x7f3f20113ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:47:01.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:01.715+0000 7f3f27600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f200fe8f0 0x7f3f20114010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:47:01.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:01.715+0000 7f3f27600700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f201146f0 con 0x7f3f200fe8f0 2024-03-04T13:47:01.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:01.715+0000 7f3f27600700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f2010d990 con 0x7f3f200fdfe0 2024-03-04T13:47:01.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:01.716+0000 7f3f1fe00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f200fe8f0 0x7f3f20114010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:47:01.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:01.716+0000 7f3f1fe00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f200fe8f0 0x7f3f20114010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48200/0 (socket says 172.21.15.107:48200) 2024-03-04T13:47:01.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:01.716+0000 7f3f1fe00700 1 -- 172.21.15.107:0/2422459335 learned_addr learned my addr 172.21.15.107:0/2422459335 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:47:01.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:01.716+0000 7f3f24c00700 1 --2- 172.21.15.107:0/2422459335 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3f200fdfe0 0x7f3f20113ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:47:01.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:01.716+0000 7f3f1fe00700 1 -- 172.21.15.107:0/2422459335 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3f200fdfe0 msgr2=0x7f3f20113ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:47:01.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:01.716+0000 7f3f1fe00700 1 --2- 172.21.15.107:0/2422459335 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3f200fdfe0 0x7f3f20113ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:47:01.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:01.716+0000 7f3f1fe00700 1 -- 172.21.15.107:0/2422459335 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f14005040 con 0x7f3f200fe8f0 2024-03-04T13:47:01.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:01.716+0000 7f3f1fe00700 1 --2- 172.21.15.107:0/2422459335 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f200fe8f0 0x7f3f20114010 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c00b790 tx=0x7f3f0c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:47:01.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:01.717+0000 7f3f1cc00700 1 -- 172.21.15.107:0/2422459335 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f0c00f770 con 0x7f3f200fe8f0 2024-03-04T13:47:01.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:01.717+0000 7f3f1cc00700 1 -- 172.21.15.107:0/2422459335 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3f0c00fda0 con 0x7f3f200fe8f0 2024-03-04T13:47:01.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:01.717+0000 7f3f1cc00700 1 -- 172.21.15.107:0/2422459335 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f0c00d310 con 0x7f3f200fe8f0 2024-03-04T13:47:01.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:01.717+0000 7f3f27600700 1 -- 172.21.15.107:0/2422459335 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f2010dc70 con 0x7f3f200fe8f0 2024-03-04T13:47:01.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:01.717+0000 7f3f27600700 1 -- 172.21.15.107:0/2422459335 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f2010e1b0 con 0x7f3f200fe8f0 2024-03-04T13:47:01.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:01.718+0000 7f3f27600700 1 -- 172.21.15.107:0/2422459335 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f201046c0 con 0x7f3f200fe8f0 2024-03-04T13:47:01.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:01.719+0000 7f3f1cc00700 1 -- 172.21.15.107:0/2422459335 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 33) v1 ==== 91199+0+0 (secure 0 0 0) 0x7f3f0c010420 con 0x7f3f200fe8f0 2024-03-04T13:47:01.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:01.720+0000 7f3f1cc00700 1 --2- 172.21.15.107:0/2422459335 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f3f0806d0e0 0x7f3f0806f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:47:01.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:01.720+0000 7f3f1cc00700 1 -- 172.21.15.107:0/2422459335 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f3f0c08c4c0 con 0x7f3f200fe8f0 2024-03-04T13:47:01.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:01.720+0000 7f3f24c00700 1 --2- 172.21.15.107:0/2422459335 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f3f0806d0e0 0x7f3f0806f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:47:01.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:01.721+0000 7f3f24c00700 1 --2- 172.21.15.107:0/2422459335 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f3f0806d0e0 0x7f3f0806f590 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3f14000c00 tx=0x7f3f14009e30 comp rx=0 tx=0).ready entity=mgr.14696 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:47:01.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:01.725+0000 7f3f1cc00700 1 -- 172.21.15.107:0/2422459335 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f3f0c05a1b0 con 0x7f3f200fe8f0 2024-03-04T13:47:01.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:01.963+0000 7f3f27600700 1 -- 172.21.15.107:0/2422459335 --> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] -- mgr_command(tid 0: {"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:717ce3cce8d0166e4e577faa8004e6f9cb4128c0", "target": ["mon-mgr", ""]}) v1 -- 0x7f3f2010a120 con 0x7f3f0806d0e0 2024-03-04T13:47:01.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:01.973+0000 7f3f1cc00700 1 -- 172.21.15.107:0/2422459335 <== mgr.14696 v2:172.21.15.107:6800/1613321046 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+89 (secure 0 0 0) 0x7f3f2010a120 con 0x7f3f0806d0e0 2024-03-04T13:47:01.972 INFO:teuthology.orchestra.run.smithi107.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:717ce3cce8d0166e4e577faa8004e6f9cb4128c0 2024-03-04T13:47:01.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:01.976+0000 7f3f27600700 1 -- 172.21.15.107:0/2422459335 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f3f0806d0e0 msgr2=0x7f3f0806f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:47:01.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:01.976+0000 7f3f27600700 1 --2- 172.21.15.107:0/2422459335 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f3f0806d0e0 0x7f3f0806f590 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3f14000c00 tx=0x7f3f14009e30 comp rx=0 tx=0).stop 2024-03-04T13:47:01.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:01.976+0000 7f3f27600700 1 -- 172.21.15.107:0/2422459335 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f200fe8f0 msgr2=0x7f3f20114010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:47:01.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:01.976+0000 7f3f27600700 1 --2- 172.21.15.107:0/2422459335 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f200fe8f0 0x7f3f20114010 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c00b790 tx=0x7f3f0c009250 comp rx=0 tx=0).stop 2024-03-04T13:47:01.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:01.976+0000 7f3f27600700 1 -- 172.21.15.107:0/2422459335 shutdown_connections 2024-03-04T13:47:01.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:01.976+0000 7f3f27600700 1 --2- 172.21.15.107:0/2422459335 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3f200fdfe0 0x7f3f20113ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:47:01.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:01.976+0000 7f3f27600700 1 --2- 172.21.15.107:0/2422459335 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f3f0806d0e0 0x7f3f0806f590 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:47:01.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:01.976+0000 7f3f27600700 1 --2- 172.21.15.107:0/2422459335 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f200fe8f0 0x7f3f20114010 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:47:01.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:01.976+0000 7f3f27600700 1 -- 172.21.15.107:0/2422459335 >> 172.21.15.107:0/2422459335 conn(0x7f3f20071d90 msgr2=0x7f3f20101840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:47:01.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:01.976+0000 7f3f27600700 1 -- 172.21.15.107:0/2422459335 shutdown_connections 2024-03-04T13:47:01.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:01.976+0000 7f3f27600700 1 -- 172.21.15.107:0/2422459335 wait complete. 2024-03-04T13:47:02.803 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-03-04T13:47:02.803 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi107.front.sepia.ceph.com 2024-03-04T13:47:02.803 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 -e sha1=717ce3cce8d0166e4e577faa8004e6f9cb4128c0 -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true && ! ceph orch upgrade status | jq '"'"'.message'"'"' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done' 2024-03-04T13:47:03.092 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:02 smithi107 ceph-mon[28739]: from='client.14864 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:717ce3cce8d0166e4e577faa8004e6f9cb4128c0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:47:03.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:02 smithi107 ceph-mon[28739]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:717ce3cce8d0166e4e577faa8004e6f9cb4128c0 2024-03-04T13:47:03.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:02 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:47:03.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:02 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:47:03.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:02 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:47:03.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:02 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:47:03.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:02 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:47:03.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:02 smithi107 ceph-mon[28739]: pgmap v105: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:47:03.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:02 smithi107 ceph-mon[28739]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:717ce3cce8d0166e4e577faa8004e6f9cb4128c0 2024-03-04T13:47:03.278 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/mon.smithi107/config 2024-03-04T13:47:03.360 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:02 smithi169 ceph-mon[36313]: from='client.14864 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:717ce3cce8d0166e4e577faa8004e6f9cb4128c0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:47:03.360 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:02 smithi169 ceph-mon[36313]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:717ce3cce8d0166e4e577faa8004e6f9cb4128c0 2024-03-04T13:47:03.360 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:02 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:47:03.360 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:02 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:47:03.360 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:02 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:47:03.360 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:02 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:47:03.360 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:02 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:47:03.360 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:02 smithi169 ceph-mon[36313]: pgmap v105: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:47:03.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:02 smithi169 ceph-mon[36313]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:717ce3cce8d0166e4e577faa8004e6f9cb4128c0 2024-03-04T13:47:05.363 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:05 smithi107 ceph-mon[28739]: pgmap v106: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:47:05.609 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:05 smithi169 ceph-mon[36313]: pgmap v106: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:47:06.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.027+0000 7f8996800700 1 -- 172.21.15.107:0/1326776747 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89900702d0 msgr2=0x7f89900706a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:47:06.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.027+0000 7f8996800700 1 --2- 172.21.15.107:0/1326776747 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89900702d0 0x7f89900706a0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f8980004970 tx=0x7f8980009e30 comp rx=0 tx=0).stop 2024-03-04T13:47:06.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.028+0000 7f8996800700 1 -- 172.21.15.107:0/1326776747 shutdown_connections 2024-03-04T13:47:06.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.028+0000 7f8996800700 1 --2- 172.21.15.107:0/1326776747 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8990070be0 0x7f89901092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:47:06.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.028+0000 7f8996800700 1 --2- 172.21.15.107:0/1326776747 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89900702d0 0x7f89900706a0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:47:06.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.028+0000 7f8996800700 1 -- 172.21.15.107:0/1326776747 >> 172.21.15.107:0/1326776747 conn(0x7f89900f8ae0 msgr2=0x7f89900faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:47:06.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.028+0000 7f8996800700 1 -- 172.21.15.107:0/1326776747 shutdown_connections 2024-03-04T13:47:06.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.028+0000 7f8996800700 1 -- 172.21.15.107:0/1326776747 wait complete. 2024-03-04T13:47:06.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.029+0000 7f8996800700 1 Processor -- start 2024-03-04T13:47:06.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.029+0000 7f8996800700 1 -- start start 2024-03-04T13:47:06.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.029+0000 7f8996800700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f89900702d0 0x7f8990115ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:47:06.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.029+0000 7f8996800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8990070be0 0x7f89901161e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:47:06.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.029+0000 7f8996800700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89901168c0 con 0x7f8990070be0 2024-03-04T13:47:06.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.029+0000 7f8996800700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f899010fb40 con 0x7f89900702d0 2024-03-04T13:47:06.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.029+0000 7f898ea00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8990070be0 0x7f89901161e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:47:06.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.029+0000 7f898ea00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8990070be0 0x7f89901161e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48230/0 (socket says 172.21.15.107:48230) 2024-03-04T13:47:06.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.030+0000 7f898ea00700 1 -- 172.21.15.107:0/281314178 learned_addr learned my addr 172.21.15.107:0/281314178 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:47:06.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.030+0000 7f898f400700 1 --2- 172.21.15.107:0/281314178 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f89900702d0 0x7f8990115ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:47:06.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.030+0000 7f898ea00700 1 -- 172.21.15.107:0/281314178 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f89900702d0 msgr2=0x7f8990115ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:47:06.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.030+0000 7f898ea00700 1 --2- 172.21.15.107:0/281314178 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f89900702d0 0x7f8990115ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:47:06.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.030+0000 7f898ea00700 1 -- 172.21.15.107:0/281314178 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8980005040 con 0x7f8990070be0 2024-03-04T13:47:06.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.030+0000 7f898ea00700 1 --2- 172.21.15.107:0/281314178 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8990070be0 0x7f89901161e0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f897c00b790 tx=0x7f897c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:47:06.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.030+0000 7f8987400700 1 -- 172.21.15.107:0/281314178 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f897c00f770 con 0x7f8990070be0 2024-03-04T13:47:06.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.030+0000 7f8996800700 1 -- 172.21.15.107:0/281314178 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f899010fdc0 con 0x7f8990070be0 2024-03-04T13:47:06.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.030+0000 7f8987400700 1 -- 172.21.15.107:0/281314178 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f897c00fda0 con 0x7f8990070be0 2024-03-04T13:47:06.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.031+0000 7f8987400700 1 -- 172.21.15.107:0/281314178 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f897c00d310 con 0x7f8990070be0 2024-03-04T13:47:06.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.031+0000 7f8996800700 1 -- 172.21.15.107:0/281314178 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8990110300 con 0x7f8990070be0 2024-03-04T13:47:06.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.032+0000 7f8996800700 1 -- 172.21.15.107:0/281314178 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8990106980 con 0x7f8990070be0 2024-03-04T13:47:06.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.033+0000 7f8987400700 1 -- 172.21.15.107:0/281314178 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 33) v1 ==== 91199+0+0 (secure 0 0 0) 0x7f897c010420 con 0x7f8990070be0 2024-03-04T13:47:06.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.034+0000 7f8987400700 1 --2- 172.21.15.107:0/281314178 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f897406d0f0 0x7f897406f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:47:06.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.034+0000 7f8987400700 1 -- 172.21.15.107:0/281314178 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f897c08c5a0 con 0x7f8990070be0 2024-03-04T13:47:06.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.034+0000 7f898f400700 1 --2- 172.21.15.107:0/281314178 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f897406d0f0 0x7f897406f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:47:06.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.035+0000 7f898f400700 1 --2- 172.21.15.107:0/281314178 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f897406d0f0 0x7f897406f5a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8980004ee0 tx=0x7f8980009e30 comp rx=0 tx=0).ready entity=mgr.14696 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:47:06.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.039+0000 7f8987400700 1 -- 172.21.15.107:0/281314178 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f897c05a1f0 con 0x7f8990070be0 2024-03-04T13:47:06.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.282+0000 7f8996800700 1 -- 172.21.15.107:0/281314178 --> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8990111280 con 0x7f897406d0f0 2024-03-04T13:47:06.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.284+0000 7f8987400700 1 -- 172.21.15.107:0/281314178 <== mgr.14696 v2:172.21.15.107:6800/1613321046 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7f8990111280 con 0x7f897406d0f0 2024-03-04T13:47:06.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.288+0000 7f8996800700 1 -- 172.21.15.107:0/281314178 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f897406d0f0 msgr2=0x7f897406f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:47:06.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.288+0000 7f8996800700 1 --2- 172.21.15.107:0/281314178 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f897406d0f0 0x7f897406f5a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8980004ee0 tx=0x7f8980009e30 comp rx=0 tx=0).stop 2024-03-04T13:47:06.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.288+0000 7f8996800700 1 -- 172.21.15.107:0/281314178 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8990070be0 msgr2=0x7f89901161e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:47:06.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.288+0000 7f8996800700 1 --2- 172.21.15.107:0/281314178 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8990070be0 0x7f89901161e0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f897c00b790 tx=0x7f897c009250 comp rx=0 tx=0).stop 2024-03-04T13:47:06.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.288+0000 7f8996800700 1 -- 172.21.15.107:0/281314178 shutdown_connections 2024-03-04T13:47:06.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.288+0000 7f8996800700 1 --2- 172.21.15.107:0/281314178 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f89900702d0 0x7f8990115ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:47:06.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.288+0000 7f8996800700 1 --2- 172.21.15.107:0/281314178 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f897406d0f0 0x7f897406f5a0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:47:06.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.288+0000 7f8996800700 1 --2- 172.21.15.107:0/281314178 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8990070be0 0x7f89901161e0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:47:06.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.288+0000 7f8996800700 1 -- 172.21.15.107:0/281314178 >> 172.21.15.107:0/281314178 conn(0x7f89900f8ae0 msgr2=0x7f8990103b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:47:06.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.288+0000 7f8996800700 1 -- 172.21.15.107:0/281314178 shutdown_connections 2024-03-04T13:47:06.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.288+0000 7f8996800700 1 -- 172.21.15.107:0/281314178 wait complete. 2024-03-04T13:47:06.302 INFO:teuthology.orchestra.run.smithi107.stdout:true 2024-03-04T13:47:06.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.444+0000 7f952be00700 1 -- 172.21.15.107:0/4130911913 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95240fbd40 msgr2=0x7f95240fc110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:47:06.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.444+0000 7f952be00700 1 --2- 172.21.15.107:0/4130911913 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95240fbd40 0x7f95240fc110 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f9514009ce0 tx=0x7f9514004d10 comp rx=0 tx=0).stop 2024-03-04T13:47:06.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.444+0000 7f952be00700 1 -- 172.21.15.107:0/4130911913 shutdown_connections 2024-03-04T13:47:06.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.444+0000 7f952be00700 1 --2- 172.21.15.107:0/4130911913 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95240fc6e0 0x7f9524100d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:47:06.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.444+0000 7f952be00700 1 --2- 172.21.15.107:0/4130911913 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95240fbd40 0x7f95240fc110 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:47:06.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.444+0000 7f952be00700 1 -- 172.21.15.107:0/4130911913 >> 172.21.15.107:0/4130911913 conn(0x7f95240f79e0 msgr2=0x7f95240f9df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:47:06.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.445+0000 7f952be00700 1 -- 172.21.15.107:0/4130911913 shutdown_connections 2024-03-04T13:47:06.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.445+0000 7f952be00700 1 -- 172.21.15.107:0/4130911913 wait complete. 2024-03-04T13:47:06.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.446+0000 7f952be00700 1 Processor -- start 2024-03-04T13:47:06.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.446+0000 7f952be00700 1 -- start start 2024-03-04T13:47:06.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.446+0000 7f952be00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95240fc6e0 0x7f952411a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:47:06.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.446+0000 7f952be00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f952411a920 0x7f95241142f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:47:06.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.446+0000 7f952be00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f952411aea0 con 0x7f95240fc6e0 2024-03-04T13:47:06.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.446+0000 7f952be00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f952411b010 con 0x7f952411a920 2024-03-04T13:47:06.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.447+0000 7f9529400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95240fc6e0 0x7f952411a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:47:06.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.447+0000 7f9529400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95240fc6e0 0x7f952411a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48250/0 (socket says 172.21.15.107:48250) 2024-03-04T13:47:06.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.447+0000 7f9529400700 1 -- 172.21.15.107:0/4159296787 learned_addr learned my addr 172.21.15.107:0/4159296787 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:47:06.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.447+0000 7f9528a00700 1 --2- 172.21.15.107:0/4159296787 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f952411a920 0x7f95241142f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:47:06.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.447+0000 7f9529400700 1 -- 172.21.15.107:0/4159296787 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f952411a920 msgr2=0x7f95241142f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:47:06.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.447+0000 7f9529400700 1 --2- 172.21.15.107:0/4159296787 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f952411a920 0x7f95241142f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:47:06.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.447+0000 7f9529400700 1 -- 172.21.15.107:0/4159296787 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9514009940 con 0x7f95240fc6e0 2024-03-04T13:47:06.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.448+0000 7f9529400700 1 --2- 172.21.15.107:0/4159296787 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95240fc6e0 0x7f952411a3e0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f9514015040 tx=0x7f9514004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:47:06.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.448+0000 7f9521600700 1 -- 172.21.15.107:0/4159296787 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f951400e040 con 0x7f95240fc6e0 2024-03-04T13:47:06.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.448+0000 7f9521600700 1 -- 172.21.15.107:0/4159296787 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f95140049b0 con 0x7f95240fc6e0 2024-03-04T13:47:06.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.448+0000 7f952be00700 1 -- 172.21.15.107:0/4159296787 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95241148f0 con 0x7f95240fc6e0 2024-03-04T13:47:06.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.448+0000 7f9521600700 1 -- 172.21.15.107:0/4159296787 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f951401c620 con 0x7f95240fc6e0 2024-03-04T13:47:06.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.448+0000 7f952be00700 1 -- 172.21.15.107:0/4159296787 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9524114e30 con 0x7f95240fc6e0 2024-03-04T13:47:06.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.449+0000 7f952be00700 1 -- 172.21.15.107:0/4159296787 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95240fe420 con 0x7f95240fc6e0 2024-03-04T13:47:06.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.450+0000 7f9521600700 1 -- 172.21.15.107:0/4159296787 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 33) v1 ==== 91199+0+0 (secure 0 0 0) 0x7f9514012a50 con 0x7f95240fc6e0 2024-03-04T13:47:06.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.452+0000 7f9521600700 1 --2- 172.21.15.107:0/4159296787 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f950806d0f0 0x7f950806f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:47:06.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.452+0000 7f9521600700 1 -- 172.21.15.107:0/4159296787 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f951408dfc0 con 0x7f95240fc6e0 2024-03-04T13:47:06.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.452+0000 7f9528a00700 1 --2- 172.21.15.107:0/4159296787 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f950806d0f0 0x7f950806f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:47:06.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.453+0000 7f9528a00700 1 --2- 172.21.15.107:0/4159296787 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f950806d0f0 0x7f950806f5a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9510004d60 tx=0x7f951000b040 comp rx=0 tx=0).ready entity=mgr.14696 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:47:06.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.456+0000 7f9521600700 1 -- 172.21.15.107:0/4159296787 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f95140c1620 con 0x7f95240fc6e0 2024-03-04T13:47:06.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.713+0000 7f952be00700 1 -- 172.21.15.107:0/4159296787 --> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9524115a30 con 0x7f950806d0f0 2024-03-04T13:47:06.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.715+0000 7f9521600700 1 -- 172.21.15.107:0/4159296787 <== mgr.14696 v2:172.21.15.107:6800/1613321046 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f9524115a30 con 0x7f950806d0f0 2024-03-04T13:47:06.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.718+0000 7f952be00700 1 -- 172.21.15.107:0/4159296787 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f950806d0f0 msgr2=0x7f950806f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:47:06.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.718+0000 7f952be00700 1 --2- 172.21.15.107:0/4159296787 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f950806d0f0 0x7f950806f5a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9510004d60 tx=0x7f951000b040 comp rx=0 tx=0).stop 2024-03-04T13:47:06.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.718+0000 7f952be00700 1 -- 172.21.15.107:0/4159296787 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95240fc6e0 msgr2=0x7f952411a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:47:06.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.718+0000 7f952be00700 1 --2- 172.21.15.107:0/4159296787 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95240fc6e0 0x7f952411a3e0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f9514015040 tx=0x7f9514004d10 comp rx=0 tx=0).stop 2024-03-04T13:47:06.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.718+0000 7f952be00700 1 -- 172.21.15.107:0/4159296787 shutdown_connections 2024-03-04T13:47:06.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.718+0000 7f952be00700 1 --2- 172.21.15.107:0/4159296787 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f950806d0f0 0x7f950806f5a0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:47:06.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.718+0000 7f952be00700 1 --2- 172.21.15.107:0/4159296787 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95240fc6e0 0x7f952411a3e0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:47:06.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.718+0000 7f952be00700 1 --2- 172.21.15.107:0/4159296787 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f952411a920 0x7f95241142f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:47:06.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.718+0000 7f952be00700 1 -- 172.21.15.107:0/4159296787 >> 172.21.15.107:0/4159296787 conn(0x7f95240f79e0 msgr2=0x7f952410e320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:47:06.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.718+0000 7f952be00700 1 -- 172.21.15.107:0/4159296787 shutdown_connections 2024-03-04T13:47:06.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.718+0000 7f952be00700 1 -- 172.21.15.107:0/4159296787 wait complete. 2024-03-04T13:47:06.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.871+0000 7f82d5600700 1 -- 172.21.15.107:0/3764755981 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82d00fc6f0 msgr2=0x7f82d0100d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:47:06.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.871+0000 7f82d5600700 1 --2- 172.21.15.107:0/3764755981 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82d00fc6f0 0x7f82d0100d10 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f82bc004940 tx=0x7f82bc009e30 comp rx=0 tx=0).stop 2024-03-04T13:47:06.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.871+0000 7f82d5600700 1 -- 172.21.15.107:0/3764755981 shutdown_connections 2024-03-04T13:47:06.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.871+0000 7f82d5600700 1 --2- 172.21.15.107:0/3764755981 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82d00fc6f0 0x7f82d0100d10 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:47:06.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.871+0000 7f82d5600700 1 --2- 172.21.15.107:0/3764755981 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f82d00fbd50 0x7f82d00fc120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:47:06.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.871+0000 7f82d5600700 1 -- 172.21.15.107:0/3764755981 >> 172.21.15.107:0/3764755981 conn(0x7f82d00f7a10 msgr2=0x7f82d00f9e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:47:06.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.871+0000 7f82d5600700 1 -- 172.21.15.107:0/3764755981 shutdown_connections 2024-03-04T13:47:06.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.871+0000 7f82d5600700 1 -- 172.21.15.107:0/3764755981 wait complete. 2024-03-04T13:47:06.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.872+0000 7f82d5600700 1 Processor -- start 2024-03-04T13:47:06.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.872+0000 7f82d5600700 1 -- start start 2024-03-04T13:47:06.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.872+0000 7f82d5600700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f82d00fbd50 0x7f82d011a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:47:06.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.872+0000 7f82d5600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82d00fc6f0 0x7f82d011a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:47:06.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.872+0000 7f82d5600700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82d011ad50 con 0x7f82d00fc6f0 2024-03-04T13:47:06.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.872+0000 7f82d5600700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82d0114060 con 0x7f82d00fbd50 2024-03-04T13:47:06.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.873+0000 7f82ce000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82d00fc6f0 0x7f82d011a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:47:06.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.873+0000 7f82ce000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82d00fc6f0 0x7f82d011a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48276/0 (socket says 172.21.15.107:48276) 2024-03-04T13:47:06.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.873+0000 7f82ce000700 1 -- 172.21.15.107:0/973089629 learned_addr learned my addr 172.21.15.107:0/973089629 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:47:06.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.873+0000 7f82cea00700 1 --2- 172.21.15.107:0/973089629 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f82d00fbd50 0x7f82d011a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:47:06.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.873+0000 7f82ce000700 1 -- 172.21.15.107:0/973089629 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f82d00fbd50 msgr2=0x7f82d011a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:47:06.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.873+0000 7f82ce000700 1 --2- 172.21.15.107:0/973089629 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f82d00fbd50 0x7f82d011a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:47:06.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.873+0000 7f82ce000700 1 -- 172.21.15.107:0/973089629 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82bc005040 con 0x7f82d00fc6f0 2024-03-04T13:47:06.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.874+0000 7f82ce000700 1 --2- 172.21.15.107:0/973089629 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82d00fc6f0 0x7f82d011a700 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f82bc009150 tx=0x7f82bc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:47:06.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.874+0000 7f82c6a00700 1 -- 172.21.15.107:0/973089629 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f82bc00e040 con 0x7f82d00fc6f0 2024-03-04T13:47:06.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.874+0000 7f82c6a00700 1 -- 172.21.15.107:0/973089629 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f82bc00bbf0 con 0x7f82d00fc6f0 2024-03-04T13:47:06.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.874+0000 7f82c6a00700 1 -- 172.21.15.107:0/973089629 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f82bc01c630 con 0x7f82d00fc6f0 2024-03-04T13:47:06.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.874+0000 7f82d5600700 1 -- 172.21.15.107:0/973089629 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82d01142e0 con 0x7f82d00fc6f0 2024-03-04T13:47:06.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.874+0000 7f82d5600700 1 -- 172.21.15.107:0/973089629 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82d0114820 con 0x7f82d00fc6f0 2024-03-04T13:47:06.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.875+0000 7f82d5600700 1 -- 172.21.15.107:0/973089629 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82d00fe430 con 0x7f82d00fc6f0 2024-03-04T13:47:06.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.876+0000 7f82c6a00700 1 -- 172.21.15.107:0/973089629 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 33) v1 ==== 91199+0+0 (secure 0 0 0) 0x7f82bc00bd60 con 0x7f82d00fc6f0 2024-03-04T13:47:06.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.877+0000 7f82c6a00700 1 --2- 172.21.15.107:0/973089629 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f82b406d0f0 0x7f82b406f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:47:06.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.877+0000 7f82c6a00700 1 -- 172.21.15.107:0/973089629 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f82bc08d920 con 0x7f82d00fc6f0 2024-03-04T13:47:06.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.878+0000 7f82cea00700 1 --2- 172.21.15.107:0/973089629 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f82b406d0f0 0x7f82b406f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:47:06.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.878+0000 7f82cea00700 1 --2- 172.21.15.107:0/973089629 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f82b406d0f0 0x7f82b406f5a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f82c00041d0 tx=0x7f82c0009040 comp rx=0 tx=0).ready entity=mgr.14696 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:47:06.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:06.882+0000 7f82c6a00700 1 -- 172.21.15.107:0/973089629 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f82bc05b550 con 0x7f82d00fc6f0 2024-03-04T13:47:07.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:07.110+0000 7f82d5600700 1 -- 172.21.15.107:0/973089629 --> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f82d004bed0 con 0x7f82b406d0f0 2024-03-04T13:47:07.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:07.133+0000 7f82c6a00700 1 -- 172.21.15.107:0/973089629 <== mgr.14696 v2:172.21.15.107:6800/1613321046 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f82d004bed0 con 0x7f82b406d0f0 2024-03-04T13:47:07.132 INFO:teuthology.orchestra.run.smithi107.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-04T13:47:07.132 INFO:teuthology.orchestra.run.smithi107.stdout:alertmanager.smithi107 smithi107 *:9093,9094 running (85s) 53s ago 15m 13.8M - 0.25.0 c8568f914cd2 e5ed7be32538 2024-03-04T13:47:07.132 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi107 smithi107 running (15m) 53s ago 15m 6665k - 17.2.7-545-g42608bb1 9eec3f22859d 03e1570c2eff 2024-03-04T13:47:07.133 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi169 smithi169 running (14m) 2m ago 14m 6656k - 17.2.7-545-g42608bb1 9eec3f22859d 9c8881a5338e 2024-03-04T13:47:07.133 INFO:teuthology.orchestra.run.smithi107.stdout:grafana.smithi107 smithi107 *:3000 running (61s) 53s ago 15m 50.6M - 9.4.12 a0ea5e3c46d9 c5e4d33fb599 2024-03-04T13:47:07.133 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.riexmm smithi107 running (8m) 53s ago 8m 13.2M - 17.2.7-545-g42608bb1 9eec3f22859d fc921f7d4328 2024-03-04T13:47:07.133 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.thycho smithi107 running (8m) 53s ago 8m 98.1M - 17.2.7-545-g42608bb1 9eec3f22859d 4069c6e0e063 2024-03-04T13:47:07.133 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi169.bxitzk smithi169 running (8m) 2m ago 8m 13.0M - 17.2.7-545-g42608bb1 9eec3f22859d 7d4115228c7c 2024-03-04T13:47:07.133 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi169.jusudl smithi169 running (8m) 2m ago 8m 210M - 17.2.7-545-g42608bb1 9eec3f22859d 2d2f2cc452e8 2024-03-04T13:47:07.133 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi107.wpzgwl smithi107 *:8443,9283,8765 running (3m) 53s ago 17m 501M - 19.0.0-1845-g717ce3cc d71b839abdf8 1c5fdee45eac 2024-03-04T13:47:07.133 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi169.eihczr smithi169 *:8443,9283,8765 running (2m) 2m ago 14m 447M - 19.0.0-1845-g717ce3cc d71b839abdf8 b5cac2b7d22a 2024-03-04T13:47:07.133 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi107 smithi107 running (17m) 53s ago 17m 54.3M 2048M 17.2.7-545-g42608bb1 9eec3f22859d 49f5da990d35 2024-03-04T13:47:07.133 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi169 smithi169 running (14m) 2m ago 14m 42.4M 2048M 17.2.7-545-g42608bb1 9eec3f22859d ba4ee916297f 2024-03-04T13:47:07.133 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi107 smithi107 *:9100 running (2m) 53s ago 15m 7780k - 1.5.0 0da6a335fe13 8a8d3d4f724f 2024-03-04T13:47:07.134 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi169 smithi169 *:9100 running (2m) 2m ago 14m 2935k - 1.5.0 0da6a335fe13 f240f5a975bc 2024-03-04T13:47:07.134 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0 smithi107 running (13m) 53s ago 13m 206M 1558M 17.2.7-545-g42608bb1 9eec3f22859d c0605b68f696 2024-03-04T13:47:07.134 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1 smithi107 running (12m) 53s ago 12m 219M 1558M 17.2.7-545-g42608bb1 9eec3f22859d 5fb2c0dd5769 2024-03-04T13:47:07.134 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2 smithi107 running (11m) 53s ago 11m 181M 1558M 17.2.7-545-g42608bb1 9eec3f22859d 61f30f643549 2024-03-04T13:47:07.134 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3 smithi169 running (11m) 2m ago 11m 222M 2582M 17.2.7-545-g42608bb1 9eec3f22859d cd91d8694764 2024-03-04T13:47:07.134 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4 smithi169 running (10m) 2m ago 10m 194M 2582M 17.2.7-545-g42608bb1 9eec3f22859d 08545e3d0659 2024-03-04T13:47:07.134 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5 smithi169 running (10m) 2m ago 10m 180M 2582M 17.2.7-545-g42608bb1 9eec3f22859d dcffe6cb3eac 2024-03-04T13:47:07.134 INFO:teuthology.orchestra.run.smithi107.stdout:prometheus.smithi107 smithi107 *:9095 running (111s) 53s ago 14m 46.4M - 2.43.0 a07b618ecd1d 1eeb1ad5cb75 2024-03-04T13:47:07.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:07.136+0000 7f82d5600700 1 -- 172.21.15.107:0/973089629 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f82b406d0f0 msgr2=0x7f82b406f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:47:07.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:07.136+0000 7f82d5600700 1 --2- 172.21.15.107:0/973089629 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f82b406d0f0 0x7f82b406f5a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f82c00041d0 tx=0x7f82c0009040 comp rx=0 tx=0).stop 2024-03-04T13:47:07.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:07.136+0000 7f82d5600700 1 -- 172.21.15.107:0/973089629 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82d00fc6f0 msgr2=0x7f82d011a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:47:07.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:07.136+0000 7f82d5600700 1 --2- 172.21.15.107:0/973089629 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82d00fc6f0 0x7f82d011a700 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f82bc009150 tx=0x7f82bc009e30 comp rx=0 tx=0).stop 2024-03-04T13:47:07.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:07.136+0000 7f82d5600700 1 -- 172.21.15.107:0/973089629 shutdown_connections 2024-03-04T13:47:07.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:07.136+0000 7f82d5600700 1 --2- 172.21.15.107:0/973089629 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f82d00fbd50 0x7f82d011a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:47:07.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:07.136+0000 7f82d5600700 1 --2- 172.21.15.107:0/973089629 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f82b406d0f0 0x7f82b406f5a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:47:07.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:07.136+0000 7f82d5600700 1 --2- 172.21.15.107:0/973089629 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82d00fc6f0 0x7f82d011a700 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:47:07.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:07.136+0000 7f82d5600700 1 -- 172.21.15.107:0/973089629 >> 172.21.15.107:0/973089629 conn(0x7f82d00f7a10 msgr2=0x7f82d010d390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:47:07.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:07.136+0000 7f82d5600700 1 -- 172.21.15.107:0/973089629 shutdown_connections 2024-03-04T13:47:07.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:07.137+0000 7f82d5600700 1 -- 172.21.15.107:0/973089629 wait complete. 2024-03-04T13:47:07.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:07.285+0000 7f111ee00700 1 -- 172.21.15.107:0/258131136 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11181038f0 msgr2=0x7f11180fcf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:47:07.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:07.285+0000 7f111ee00700 1 --2- 172.21.15.107:0/258131136 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11181038f0 0x7f11180fcf60 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f1104004940 tx=0x7f1104009e30 comp rx=0 tx=0).stop 2024-03-04T13:47:07.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:07.285+0000 7f111ee00700 1 -- 172.21.15.107:0/258131136 shutdown_connections 2024-03-04T13:47:07.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:07.285+0000 7f111ee00700 1 --2- 172.21.15.107:0/258131136 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11181038f0 0x7f11180fcf60 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:47:07.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:07.285+0000 7f111ee00700 1 --2- 172.21.15.107:0/258131136 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1118102f50 0x7f1118103320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:47:07.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:07.285+0000 7f111ee00700 1 -- 172.21.15.107:0/258131136 >> 172.21.15.107:0/258131136 conn(0x7f11180f8ae0 msgr2=0x7f11180faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:47:07.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:07.286+0000 7f111ee00700 1 -- 172.21.15.107:0/258131136 shutdown_connections 2024-03-04T13:47:07.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:07.286+0000 7f111ee00700 1 -- 172.21.15.107:0/258131136 wait complete. 2024-03-04T13:47:07.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:07.287+0000 7f111ee00700 1 Processor -- start 2024-03-04T13:47:07.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:07.287+0000 7f111ee00700 1 -- start start 2024-03-04T13:47:07.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:07.287+0000 7f111ee00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1118102f50 0x7f1118113a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:47:07.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:07.287+0000 7f111ee00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11181038f0 0x7f1118113fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:47:07.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:07.287+0000 7f111ee00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1118114600 con 0x7f11181038f0 2024-03-04T13:47:07.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:07.287+0000 7f111ee00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f111810d930 con 0x7f1118102f50 2024-03-04T13:47:07.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:07.288+0000 7f1117400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11181038f0 0x7f1118113fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:47:07.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:07.288+0000 7f1117400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11181038f0 0x7f1118113fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48292/0 (socket says 172.21.15.107:48292) 2024-03-04T13:47:07.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:07.288+0000 7f1117400700 1 -- 172.21.15.107:0/827644585 learned_addr learned my addr 172.21.15.107:0/827644585 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:47:07.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:07.288+0000 7f1117e00700 1 --2- 172.21.15.107:0/827644585 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1118102f50 0x7f1118113a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:47:07.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:07.288+0000 7f1117400700 1 -- 172.21.15.107:0/827644585 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1118102f50 msgr2=0x7f1118113a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:47:07.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:07.288+0000 7f1117400700 1 --2- 172.21.15.107:0/827644585 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1118102f50 0x7f1118113a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:47:07.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:07.288+0000 7f1117400700 1 -- 172.21.15.107:0/827644585 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1104005040 con 0x7f11181038f0 2024-03-04T13:47:07.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:07.288+0000 7f1117400700 1 --2- 172.21.15.107:0/827644585 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11181038f0 0x7f1118113fb0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f1104009100 tx=0x7f1104009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:47:07.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:07.289+0000 7f110fe00700 1 -- 172.21.15.107:0/827644585 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f110400e040 con 0x7f11181038f0 2024-03-04T13:47:07.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:07.289+0000 7f110fe00700 1 -- 172.21.15.107:0/827644585 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1104004290 con 0x7f11181038f0 2024-03-04T13:47:07.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:07.289+0000 7f111ee00700 1 -- 172.21.15.107:0/827644585 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f111810dbb0 con 0x7f11181038f0 2024-03-04T13:47:07.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:07.289+0000 7f110fe00700 1 -- 172.21.15.107:0/827644585 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f110401c3f0 con 0x7f11181038f0 2024-03-04T13:47:07.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:07.289+0000 7f111ee00700 1 -- 172.21.15.107:0/827644585 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f111810e0f0 con 0x7f11181038f0 2024-03-04T13:47:07.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:07.290+0000 7f110d600700 1 -- 172.21.15.107:0/827644585 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10f00052b0 con 0x7f11181038f0 2024-03-04T13:47:07.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:07.291+0000 7f110fe00700 1 -- 172.21.15.107:0/827644585 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 33) v1 ==== 91199+0+0 (secure 0 0 0) 0x7f110400bbf0 con 0x7f11181038f0 2024-03-04T13:47:07.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:07.292+0000 7f110fe00700 1 --2- 172.21.15.107:0/827644585 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f10fc06d0f0 0x7f10fc06f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:47:07.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:07.292+0000 7f110fe00700 1 -- 172.21.15.107:0/827644585 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f110408d920 con 0x7f11181038f0 2024-03-04T13:47:07.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:07.292+0000 7f1117e00700 1 --2- 172.21.15.107:0/827644585 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f10fc06d0f0 0x7f10fc06f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:47:07.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:07.293+0000 7f1117e00700 1 --2- 172.21.15.107:0/827644585 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f10fc06d0f0 0x7f10fc06f5a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1108004d60 tx=0x7f110800b040 comp rx=0 tx=0).ready entity=mgr.14696 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:47:07.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:07.300+0000 7f110fe00700 1 -- 172.21.15.107:0/827644585 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f110405b4d0 con 0x7f11181038f0 2024-03-04T13:47:07.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:07.621+0000 7f110d600700 1 -- 172.21.15.107:0/827644585 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f10f0005d70 con 0x7f11181038f0 2024-03-04T13:47:07.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:07.622+0000 7f110fe00700 1 -- 172.21.15.107:0/827644585 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7f110405b060 con 0x7f11181038f0 2024-03-04T13:47:07.621 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-03-04T13:47:07.621 INFO:teuthology.orchestra.run.smithi107.stdout: "mon": { 2024-03-04T13:47:07.622 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-545-g42608bb1 (42608bb191117be10dba720e6bf8a02f3171cd34) quincy (stable)": 2 2024-03-04T13:47:07.622 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:47:07.622 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr": { 2024-03-04T13:47:07.622 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-1845-g717ce3cc (717ce3cce8d0166e4e577faa8004e6f9cb4128c0) squid (dev)": 2 2024-03-04T13:47:07.622 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:47:07.622 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": { 2024-03-04T13:47:07.622 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-545-g42608bb1 (42608bb191117be10dba720e6bf8a02f3171cd34) quincy (stable)": 6 2024-03-04T13:47:07.622 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:47:07.622 INFO:teuthology.orchestra.run.smithi107.stdout: "mds": { 2024-03-04T13:47:07.622 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-545-g42608bb1 (42608bb191117be10dba720e6bf8a02f3171cd34) quincy (stable)": 4 2024-03-04T13:47:07.622 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:47:07.622 INFO:teuthology.orchestra.run.smithi107.stdout: "overall": { 2024-03-04T13:47:07.622 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-545-g42608bb1 (42608bb191117be10dba720e6bf8a02f3171cd34) quincy (stable)": 12, 2024-03-04T13:47:07.623 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-1845-g717ce3cc (717ce3cce8d0166e4e577faa8004e6f9cb4128c0) squid (dev)": 2 2024-03-04T13:47:07.623 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-04T13:47:07.623 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-03-04T13:47:07.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:07.625+0000 7f110d600700 1 -- 172.21.15.107:0/827644585 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f10fc06d0f0 msgr2=0x7f10fc06f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:47:07.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:07.625+0000 7f110d600700 1 --2- 172.21.15.107:0/827644585 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f10fc06d0f0 0x7f10fc06f5a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1108004d60 tx=0x7f110800b040 comp rx=0 tx=0).stop 2024-03-04T13:47:07.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:07.625+0000 7f110d600700 1 -- 172.21.15.107:0/827644585 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11181038f0 msgr2=0x7f1118113fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:47:07.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:07.625+0000 7f110d600700 1 --2- 172.21.15.107:0/827644585 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11181038f0 0x7f1118113fb0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f1104009100 tx=0x7f1104009e30 comp rx=0 tx=0).stop 2024-03-04T13:47:07.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:07.626+0000 7f110d600700 1 -- 172.21.15.107:0/827644585 shutdown_connections 2024-03-04T13:47:07.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:07.626+0000 7f110d600700 1 --2- 172.21.15.107:0/827644585 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1118102f50 0x7f1118113a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:47:07.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:07.626+0000 7f110d600700 1 --2- 172.21.15.107:0/827644585 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f10fc06d0f0 0x7f10fc06f5a0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:47:07.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:07.626+0000 7f110d600700 1 --2- 172.21.15.107:0/827644585 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11181038f0 0x7f1118113fb0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:47:07.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:07.626+0000 7f110d600700 1 -- 172.21.15.107:0/827644585 >> 172.21.15.107:0/827644585 conn(0x7f11180f8ae0 msgr2=0x7f11180fa570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:47:07.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:07.626+0000 7f110d600700 1 -- 172.21.15.107:0/827644585 shutdown_connections 2024-03-04T13:47:07.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:07.626+0000 7f110d600700 1 -- 172.21.15.107:0/827644585 wait complete. 2024-03-04T13:47:07.771 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:07 smithi107 ceph-mon[28739]: pgmap v107: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:47:07.772 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:07 smithi107 ceph-mon[28739]: from='client.14868 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:47:07.772 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:07 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:47:07.772 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:07 smithi107 ceph-mon[28739]: Upgrade: Target is version 19.0.0-1845-g717ce3cc (squid) 2024-03-04T13:47:07.772 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:07 smithi107 ceph-mon[28739]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, digests ['quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377'] 2024-03-04T13:47:07.772 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:07 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:47:07.772 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:07 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:47:07.772 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:07 smithi107 ceph-mon[28739]: Upgrade: Setting container_image for all mgr 2024-03-04T13:47:07.772 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:07 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:47:07.772 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:07 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-03-04T13:47:07.773 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:07 smithi107 ceph-mon[28739]: from='client.14872 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:47:07.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:07.772+0000 7f6903e00700 1 -- 172.21.15.107:0/260808255 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6904104f70 msgr2=0x7f6904105340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:47:07.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:07.772+0000 7f6903e00700 1 --2- 172.21.15.107:0/260808255 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6904104f70 0x7f6904105340 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f68f4004940 tx=0x7f68f4009e30 comp rx=0 tx=0).stop 2024-03-04T13:47:07.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:07.772+0000 7f6903e00700 1 -- 172.21.15.107:0/260808255 shutdown_connections 2024-03-04T13:47:07.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:07.772+0000 7f6903e00700 1 --2- 172.21.15.107:0/260808255 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6904105910 0x7f69040fef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:47:07.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:07.772+0000 7f6903e00700 1 --2- 172.21.15.107:0/260808255 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6904104f70 0x7f6904105340 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:47:07.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:07.772+0000 7f6903e00700 1 -- 172.21.15.107:0/260808255 >> 172.21.15.107:0/260808255 conn(0x7f69040faae0 msgr2=0x7f69040fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:47:07.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:07.772+0000 7f6903e00700 1 -- 172.21.15.107:0/260808255 shutdown_connections 2024-03-04T13:47:07.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:07.772+0000 7f6903e00700 1 -- 172.21.15.107:0/260808255 wait complete. 2024-03-04T13:47:07.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:07.773+0000 7f6903e00700 1 Processor -- start 2024-03-04T13:47:07.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:07.773+0000 7f6903e00700 1 -- start start 2024-03-04T13:47:07.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:07.774+0000 7f6903e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6904104f70 0x7f6904115a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:47:07.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:07.774+0000 7f6903e00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6904105910 0x7f6904115f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:47:07.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:07.774+0000 7f6903e00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6904116660 con 0x7f6904104f70 2024-03-04T13:47:07.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:07.774+0000 7f6903e00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f690410f900 con 0x7f6904105910 2024-03-04T13:47:07.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:07.774+0000 7f6902a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6904104f70 0x7f6904115a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:47:07.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:07.774+0000 7f6902a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6904104f70 0x7f6904115a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48316/0 (socket says 172.21.15.107:48316) 2024-03-04T13:47:07.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:07.774+0000 7f6902a00700 1 -- 172.21.15.107:0/608990760 learned_addr learned my addr 172.21.15.107:0/608990760 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:47:07.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:07.774+0000 7f6902000700 1 --2- 172.21.15.107:0/608990760 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6904105910 0x7f6904115f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:47:07.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:07.775+0000 7f6902a00700 1 -- 172.21.15.107:0/608990760 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6904105910 msgr2=0x7f6904115f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:47:07.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:07.775+0000 7f6902a00700 1 --2- 172.21.15.107:0/608990760 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6904105910 0x7f6904115f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:47:07.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:07.775+0000 7f6902a00700 1 -- 172.21.15.107:0/608990760 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68f4005040 con 0x7f6904104f70 2024-03-04T13:47:07.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:07.775+0000 7f6902a00700 1 --2- 172.21.15.107:0/608990760 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6904104f70 0x7f6904115a40 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f68f4009fd0 tx=0x7f68f4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:47:07.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:07.775+0000 7f68faa00700 1 -- 172.21.15.107:0/608990760 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f68f400e040 con 0x7f6904104f70 2024-03-04T13:47:07.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:07.775+0000 7f68faa00700 1 -- 172.21.15.107:0/608990760 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f68f400bbf0 con 0x7f6904104f70 2024-03-04T13:47:07.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:07.775+0000 7f68faa00700 1 -- 172.21.15.107:0/608990760 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f68f401c3f0 con 0x7f6904104f70 2024-03-04T13:47:07.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:07.775+0000 7f6903e00700 1 -- 172.21.15.107:0/608990760 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f690410fb80 con 0x7f6904104f70 2024-03-04T13:47:07.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:07.776+0000 7f6903e00700 1 -- 172.21.15.107:0/608990760 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6904110140 con 0x7f6904104f70 2024-03-04T13:47:07.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:07.777+0000 7f6903e00700 1 -- 172.21.15.107:0/608990760 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f690405f5c0 con 0x7f6904104f70 2024-03-04T13:47:07.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:07.778+0000 7f68faa00700 1 -- 172.21.15.107:0/608990760 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 33) v1 ==== 91199+0+0 (secure 0 0 0) 0x7f68f400bd60 con 0x7f6904104f70 2024-03-04T13:47:07.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:07.779+0000 7f68faa00700 1 --2- 172.21.15.107:0/608990760 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f68e806d0f0 0x7f68e806f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:47:07.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:07.779+0000 7f68faa00700 1 -- 172.21.15.107:0/608990760 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f68f408d790 con 0x7f6904104f70 2024-03-04T13:47:07.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:07.780+0000 7f6902000700 1 --2- 172.21.15.107:0/608990760 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f68e806d0f0 0x7f68e806f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:47:07.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:07.781+0000 7f6902000700 1 --2- 172.21.15.107:0/608990760 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f68e806d0f0 0x7f68e806f5a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f68f0006fd0 tx=0x7f68f0008040 comp rx=0 tx=0).ready entity=mgr.14696 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:47:07.784 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:07.786+0000 7f68faa00700 1 -- 172.21.15.107:0/608990760 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f68f405b480 con 0x7f6904104f70 2024-03-04T13:47:08.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.052+0000 7f6903e00700 1 -- 172.21.15.107:0/608990760 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f6904111020 con 0x7f6904104f70 2024-03-04T13:47:08.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.053+0000 7f68faa00700 1 -- 172.21.15.107:0/608990760 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 16 v16) v1 ==== 76+0+1842 (secure 0 0 0) 0x7f68f405b010 con 0x7f6904104f70 2024-03-04T13:47:08.053 INFO:teuthology.orchestra.run.smithi107.stdout:e16 2024-03-04T13:47:08.053 INFO:teuthology.orchestra.run.smithi107.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-04T13:47:08.053 INFO:teuthology.orchestra.run.smithi107.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-04T13:47:08.053 INFO:teuthology.orchestra.run.smithi107.stdout:legacy client fscid: 1 2024-03-04T13:47:08.053 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:47:08.053 INFO:teuthology.orchestra.run.smithi107.stdout:Filesystem 'cephfs' (1) 2024-03-04T13:47:08.053 INFO:teuthology.orchestra.run.smithi107.stdout:fs_name cephfs 2024-03-04T13:47:08.053 INFO:teuthology.orchestra.run.smithi107.stdout:epoch 15 2024-03-04T13:47:08.053 INFO:teuthology.orchestra.run.smithi107.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-04T13:47:08.054 INFO:teuthology.orchestra.run.smithi107.stdout:created 2024-03-04T13:38:30.854004+0000 2024-03-04T13:47:08.054 INFO:teuthology.orchestra.run.smithi107.stdout:modified 2024-03-04T13:38:50.015989+0000 2024-03-04T13:47:08.054 INFO:teuthology.orchestra.run.smithi107.stdout:tableserver 0 2024-03-04T13:47:08.054 INFO:teuthology.orchestra.run.smithi107.stdout:root 0 2024-03-04T13:47:08.054 INFO:teuthology.orchestra.run.smithi107.stdout:session_timeout 60 2024-03-04T13:47:08.054 INFO:teuthology.orchestra.run.smithi107.stdout:session_autoclose 300 2024-03-04T13:47:08.054 INFO:teuthology.orchestra.run.smithi107.stdout:max_file_size 1099511627776 2024-03-04T13:47:08.054 INFO:teuthology.orchestra.run.smithi107.stdout:required_client_features {} 2024-03-04T13:47:08.054 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure 0 2024-03-04T13:47:08.054 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure_osd_epoch 47 2024-03-04T13:47:08.054 INFO:teuthology.orchestra.run.smithi107.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-04T13:47:08.054 INFO:teuthology.orchestra.run.smithi107.stdout:max_mds 1 2024-03-04T13:47:08.055 INFO:teuthology.orchestra.run.smithi107.stdout:in 0 2024-03-04T13:47:08.055 INFO:teuthology.orchestra.run.smithi107.stdout:up {0=24247} 2024-03-04T13:47:08.055 INFO:teuthology.orchestra.run.smithi107.stdout:failed 2024-03-04T13:47:08.055 INFO:teuthology.orchestra.run.smithi107.stdout:damaged 2024-03-04T13:47:08.055 INFO:teuthology.orchestra.run.smithi107.stdout:stopped 2024-03-04T13:47:08.055 INFO:teuthology.orchestra.run.smithi107.stdout:data_pools [3] 2024-03-04T13:47:08.055 INFO:teuthology.orchestra.run.smithi107.stdout:metadata_pool 2 2024-03-04T13:47:08.055 INFO:teuthology.orchestra.run.smithi107.stdout:inline_data enabled 2024-03-04T13:47:08.055 INFO:teuthology.orchestra.run.smithi107.stdout:balancer 2024-03-04T13:47:08.055 INFO:teuthology.orchestra.run.smithi107.stdout:standby_count_wanted 1 2024-03-04T13:47:08.055 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi169.jusudl{0:24247} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.169:6824/2994866076,v1:172.21.15.169:6825/2994866076] compat {c=[1],r=[1],i=[7ff]}] 2024-03-04T13:47:08.055 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.thycho{0:14470} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.107:6828/1985577060,v1:172.21.15.107:6829/1985577060] compat {c=[1],r=[1],i=[7ff]}] 2024-03-04T13:47:08.055 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:47:08.055 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:47:08.056 INFO:teuthology.orchestra.run.smithi107.stdout:Standby daemons: 2024-03-04T13:47:08.056 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:47:08.056 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.riexmm{-1:14480} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.107:6826/3261622517,v1:172.21.15.107:6827/3261622517] compat {c=[1],r=[1],i=[7ff]}] 2024-03-04T13:47:08.056 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi169.bxitzk{-1:24259} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.169:6826/3696090493,v1:172.21.15.169:6827/3696090493] compat {c=[1],r=[1],i=[7ff]}] 2024-03-04T13:47:08.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.056+0000 7f6903e00700 1 -- 172.21.15.107:0/608990760 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f68e806d0f0 msgr2=0x7f68e806f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:47:08.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.056+0000 7f6903e00700 1 --2- 172.21.15.107:0/608990760 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f68e806d0f0 0x7f68e806f5a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f68f0006fd0 tx=0x7f68f0008040 comp rx=0 tx=0).stop 2024-03-04T13:47:08.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.056+0000 7f6903e00700 1 -- 172.21.15.107:0/608990760 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6904104f70 msgr2=0x7f6904115a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:47:08.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.056+0000 7f6903e00700 1 --2- 172.21.15.107:0/608990760 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6904104f70 0x7f6904115a40 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f68f4009fd0 tx=0x7f68f4009e30 comp rx=0 tx=0).stop 2024-03-04T13:47:08.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.056+0000 7f6903e00700 1 -- 172.21.15.107:0/608990760 shutdown_connections 2024-03-04T13:47:08.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.056+0000 7f6903e00700 1 --2- 172.21.15.107:0/608990760 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f68e806d0f0 0x7f68e806f5a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:47:08.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.056+0000 7f6903e00700 1 --2- 172.21.15.107:0/608990760 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6904104f70 0x7f6904115a40 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:47:08.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.056+0000 7f6903e00700 1 --2- 172.21.15.107:0/608990760 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6904105910 0x7f6904115f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:47:08.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.056+0000 7f6903e00700 1 -- 172.21.15.107:0/608990760 >> 172.21.15.107:0/608990760 conn(0x7f69040faae0 msgr2=0x7f69040fc400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:47:08.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.056+0000 7f6903e00700 1 -- 172.21.15.107:0/608990760 shutdown_connections 2024-03-04T13:47:08.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.056+0000 7f6903e00700 1 -- 172.21.15.107:0/608990760 wait complete. 2024-03-04T13:47:08.058 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 16 2024-03-04T13:47:08.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:07 smithi169 ceph-mon[36313]: pgmap v107: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:47:08.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:07 smithi169 ceph-mon[36313]: from='client.14868 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:47:08.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:07 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:47:08.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:07 smithi169 ceph-mon[36313]: Upgrade: Target is version 19.0.0-1845-g717ce3cc (squid) 2024-03-04T13:47:08.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:07 smithi169 ceph-mon[36313]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, digests ['quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377'] 2024-03-04T13:47:08.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:07 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:47:08.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:07 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:47:08.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:07 smithi169 ceph-mon[36313]: Upgrade: Setting container_image for all mgr 2024-03-04T13:47:08.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:07 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:47:08.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:07 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-03-04T13:47:08.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:07 smithi169 ceph-mon[36313]: from='client.14872 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:47:08.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.198+0000 7f50b2000700 1 -- 172.21.15.107:0/3610617914 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f50ac0fe730 msgr2=0x7f50ac106e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:47:08.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.198+0000 7f50b2000700 1 --2- 172.21.15.107:0/3610617914 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f50ac0fe730 0x7f50ac106e00 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f5098004940 tx=0x7f5098009e30 comp rx=0 tx=0).stop 2024-03-04T13:47:08.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.198+0000 7f50b2000700 1 -- 172.21.15.107:0/3610617914 shutdown_connections 2024-03-04T13:47:08.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.198+0000 7f50b2000700 1 --2- 172.21.15.107:0/3610617914 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f50ac0fe730 0x7f50ac106e00 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:47:08.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.198+0000 7f50b2000700 1 --2- 172.21.15.107:0/3610617914 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f50ac0fdd90 0x7f50ac0fe160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:47:08.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.198+0000 7f50b2000700 1 -- 172.21.15.107:0/3610617914 >> 172.21.15.107:0/3610617914 conn(0x7f50ac073ab0 msgr2=0x7f50ac073eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:47:08.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.198+0000 7f50b2000700 1 -- 172.21.15.107:0/3610617914 shutdown_connections 2024-03-04T13:47:08.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.198+0000 7f50b2000700 1 -- 172.21.15.107:0/3610617914 wait complete. 2024-03-04T13:47:08.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.199+0000 7f50b2000700 1 Processor -- start 2024-03-04T13:47:08.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.199+0000 7f50b2000700 1 -- start start 2024-03-04T13:47:08.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.200+0000 7f50b2000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f50ac0fdd90 0x7f50ac115af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:47:08.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.200+0000 7f50b2000700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f50ac0fe730 0x7f50ac116030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:47:08.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.200+0000 7f50b2000700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50ac116680 con 0x7f50ac0fdd90 2024-03-04T13:47:08.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.200+0000 7f50b2000700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50ac10f9b0 con 0x7f50ac0fe730 2024-03-04T13:47:08.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.200+0000 7f50b0c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f50ac0fdd90 0x7f50ac115af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:47:08.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.200+0000 7f50b0c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f50ac0fdd90 0x7f50ac115af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46454/0 (socket says 172.21.15.107:46454) 2024-03-04T13:47:08.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.200+0000 7f50b0c00700 1 -- 172.21.15.107:0/4170180287 learned_addr learned my addr 172.21.15.107:0/4170180287 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:47:08.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.200+0000 7f50a3e00700 1 --2- 172.21.15.107:0/4170180287 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f50ac0fe730 0x7f50ac116030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:47:08.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.201+0000 7f50b0c00700 1 -- 172.21.15.107:0/4170180287 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f50ac0fe730 msgr2=0x7f50ac116030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:47:08.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.201+0000 7f50b0c00700 1 --2- 172.21.15.107:0/4170180287 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f50ac0fe730 0x7f50ac116030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:47:08.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.201+0000 7f50b0c00700 1 -- 172.21.15.107:0/4170180287 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5098005040 con 0x7f50ac0fdd90 2024-03-04T13:47:08.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.201+0000 7f50b0c00700 1 --2- 172.21.15.107:0/4170180287 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f50ac0fdd90 0x7f50ac115af0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f509c002a50 tx=0x7f509c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:47:08.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.201+0000 7f50a9600700 1 -- 172.21.15.107:0/4170180287 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f509c007cc0 con 0x7f50ac0fdd90 2024-03-04T13:47:08.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.201+0000 7f50a9600700 1 -- 172.21.15.107:0/4170180287 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f509c00f040 con 0x7f50ac0fdd90 2024-03-04T13:47:08.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.201+0000 7f50b2000700 1 -- 172.21.15.107:0/4170180287 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50ac10fc90 con 0x7f50ac0fdd90 2024-03-04T13:47:08.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.202+0000 7f50a9600700 1 -- 172.21.15.107:0/4170180287 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f509c009640 con 0x7f50ac0fdd90 2024-03-04T13:47:08.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.202+0000 7f50b2000700 1 -- 172.21.15.107:0/4170180287 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50ac1101a0 con 0x7f50ac0fdd90 2024-03-04T13:47:08.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.203+0000 7f50b2000700 1 -- 172.21.15.107:0/4170180287 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50ac1068c0 con 0x7f50ac0fdd90 2024-03-04T13:47:08.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.204+0000 7f50a9600700 1 -- 172.21.15.107:0/4170180287 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 33) v1 ==== 91199+0+0 (secure 0 0 0) 0x7f509c0097c0 con 0x7f50ac0fdd90 2024-03-04T13:47:08.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.205+0000 7f50a9600700 1 --2- 172.21.15.107:0/4170180287 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f509006d0f0 0x7f509006f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:47:08.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.205+0000 7f50a9600700 1 -- 172.21.15.107:0/4170180287 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f509c08c2b0 con 0x7f50ac0fdd90 2024-03-04T13:47:08.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.205+0000 7f50a3e00700 1 --2- 172.21.15.107:0/4170180287 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f509006d0f0 0x7f509006f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:47:08.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.206+0000 7f50a3e00700 1 --2- 172.21.15.107:0/4170180287 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f509006d0f0 0x7f509006f5a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f50ac110ce0 tx=0x7f5098009e30 comp rx=0 tx=0).ready entity=mgr.14696 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:47:08.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.209+0000 7f50a9600700 1 -- 172.21.15.107:0/4170180287 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f509c059e60 con 0x7f50ac0fdd90 2024-03-04T13:47:08.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.437+0000 7f50b2000700 1 -- 172.21.15.107:0/4170180287 --> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f50ac10fe20 con 0x7f509006d0f0 2024-03-04T13:47:08.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.440+0000 7f50a9600700 1 -- 172.21.15.107:0/4170180287 <== mgr.14696 v2:172.21.15.107:6800/1613321046 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f50ac10fe20 con 0x7f509006d0f0 2024-03-04T13:47:08.439 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-03-04T13:47:08.439 INFO:teuthology.orchestra.run.smithi107.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377", 2024-03-04T13:47:08.439 INFO:teuthology.orchestra.run.smithi107.stdout: "in_progress": true, 2024-03-04T13:47:08.439 INFO:teuthology.orchestra.run.smithi107.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-04T13:47:08.439 INFO:teuthology.orchestra.run.smithi107.stdout: "services_complete": [ 2024-03-04T13:47:08.439 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr" 2024-03-04T13:47:08.439 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-04T13:47:08.439 INFO:teuthology.orchestra.run.smithi107.stdout: "progress": "2/21 daemons upgraded", 2024-03-04T13:47:08.440 INFO:teuthology.orchestra.run.smithi107.stdout: "message": "Currently upgrading mon daemons", 2024-03-04T13:47:08.440 INFO:teuthology.orchestra.run.smithi107.stdout: "is_paused": false 2024-03-04T13:47:08.440 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-03-04T13:47:08.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.443+0000 7f50b2000700 1 -- 172.21.15.107:0/4170180287 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f509006d0f0 msgr2=0x7f509006f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:47:08.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.443+0000 7f50b2000700 1 --2- 172.21.15.107:0/4170180287 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f509006d0f0 0x7f509006f5a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f50ac110ce0 tx=0x7f5098009e30 comp rx=0 tx=0).stop 2024-03-04T13:47:08.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.443+0000 7f50b2000700 1 -- 172.21.15.107:0/4170180287 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f50ac0fdd90 msgr2=0x7f50ac115af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:47:08.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.443+0000 7f50b2000700 1 --2- 172.21.15.107:0/4170180287 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f50ac0fdd90 0x7f50ac115af0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f509c002a50 tx=0x7f509c0046b0 comp rx=0 tx=0).stop 2024-03-04T13:47:08.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.443+0000 7f50b2000700 1 -- 172.21.15.107:0/4170180287 shutdown_connections 2024-03-04T13:47:08.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.443+0000 7f50b2000700 1 --2- 172.21.15.107:0/4170180287 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f509006d0f0 0x7f509006f5a0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:47:08.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.443+0000 7f50b2000700 1 --2- 172.21.15.107:0/4170180287 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f50ac0fdd90 0x7f50ac115af0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:47:08.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.443+0000 7f50b2000700 1 --2- 172.21.15.107:0/4170180287 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f50ac0fe730 0x7f50ac116030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:47:08.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.443+0000 7f50b2000700 1 -- 172.21.15.107:0/4170180287 >> 172.21.15.107:0/4170180287 conn(0x7f50ac073ab0 msgr2=0x7f50ac103a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:47:08.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.443+0000 7f50b2000700 1 -- 172.21.15.107:0/4170180287 shutdown_connections 2024-03-04T13:47:08.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.443+0000 7f50b2000700 1 -- 172.21.15.107:0/4170180287 wait complete. 2024-03-04T13:47:08.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.591+0000 7f5abd600700 1 -- 172.21.15.107:0/2630827758 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ab8103dc0 msgr2=0x7f5ab8104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:47:08.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.591+0000 7f5abd600700 1 --2- 172.21.15.107:0/2630827758 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ab8103dc0 0x7f5ab8104210 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f5aa4004940 tx=0x7f5aa4009e30 comp rx=0 tx=0).stop 2024-03-04T13:47:08.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.592+0000 7f5abd600700 1 -- 172.21.15.107:0/2630827758 shutdown_connections 2024-03-04T13:47:08.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.592+0000 7f5abd600700 1 --2- 172.21.15.107:0/2630827758 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ab8103dc0 0x7f5ab8104210 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:47:08.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.592+0000 7f5abd600700 1 --2- 172.21.15.107:0/2630827758 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5ab81095f0 0x7f5ab81099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:47:08.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.592+0000 7f5abd600700 1 -- 172.21.15.107:0/2630827758 >> 172.21.15.107:0/2630827758 conn(0x7f5ab8072d50 msgr2=0x7f5ab8075160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:47:08.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.592+0000 7f5abd600700 1 -- 172.21.15.107:0/2630827758 shutdown_connections 2024-03-04T13:47:08.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.592+0000 7f5abd600700 1 -- 172.21.15.107:0/2630827758 wait complete. 2024-03-04T13:47:08.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.593+0000 7f5abd600700 1 Processor -- start 2024-03-04T13:47:08.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.593+0000 7f5abd600700 1 -- start start 2024-03-04T13:47:08.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.593+0000 7f5abd600700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5ab8103dc0 0x7f5ab811a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:47:08.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.594+0000 7f5abd600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ab81095f0 0x7f5ab811a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:47:08.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.594+0000 7f5abd600700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab811ac50 con 0x7f5ab81095f0 2024-03-04T13:47:08.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.594+0000 7f5abd600700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab8113ed0 con 0x7f5ab8103dc0 2024-03-04T13:47:08.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.594+0000 7f5ab6000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ab81095f0 0x7f5ab811a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:47:08.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.594+0000 7f5ab6000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ab81095f0 0x7f5ab811a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46470/0 (socket says 172.21.15.107:46470) 2024-03-04T13:47:08.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.594+0000 7f5ab6000700 1 -- 172.21.15.107:0/2602785099 learned_addr learned my addr 172.21.15.107:0/2602785099 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:47:08.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.594+0000 7f5ab6a00700 1 --2- 172.21.15.107:0/2602785099 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5ab8103dc0 0x7f5ab811a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:47:08.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.594+0000 7f5ab6000700 1 -- 172.21.15.107:0/2602785099 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5ab8103dc0 msgr2=0x7f5ab811a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:47:08.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.594+0000 7f5ab6000700 1 --2- 172.21.15.107:0/2602785099 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5ab8103dc0 0x7f5ab811a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:47:08.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.594+0000 7f5ab6000700 1 -- 172.21.15.107:0/2602785099 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aa4005040 con 0x7f5ab81095f0 2024-03-04T13:47:08.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.595+0000 7f5ab6000700 1 --2- 172.21.15.107:0/2602785099 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ab81095f0 0x7f5ab811a570 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f5aa4009150 tx=0x7f5aa4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:47:08.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.595+0000 7f5aaea00700 1 -- 172.21.15.107:0/2602785099 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5aa400e040 con 0x7f5ab81095f0 2024-03-04T13:47:08.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.595+0000 7f5aaea00700 1 -- 172.21.15.107:0/2602785099 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5aa400bbf0 con 0x7f5ab81095f0 2024-03-04T13:47:08.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.595+0000 7f5aaea00700 1 -- 172.21.15.107:0/2602785099 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5aa401c630 con 0x7f5ab81095f0 2024-03-04T13:47:08.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.595+0000 7f5abd600700 1 -- 172.21.15.107:0/2602785099 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ab81140f0 con 0x7f5ab81095f0 2024-03-04T13:47:08.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.595+0000 7f5abd600700 1 -- 172.21.15.107:0/2602785099 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ab8114630 con 0x7f5ab81095f0 2024-03-04T13:47:08.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.596+0000 7f5abd600700 1 -- 172.21.15.107:0/2602785099 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ab8075870 con 0x7f5ab81095f0 2024-03-04T13:47:08.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.597+0000 7f5aaea00700 1 -- 172.21.15.107:0/2602785099 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 33) v1 ==== 91199+0+0 (secure 0 0 0) 0x7f5aa400bd60 con 0x7f5ab81095f0 2024-03-04T13:47:08.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.598+0000 7f5aaea00700 1 --2- 172.21.15.107:0/2602785099 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f5a9c06d0f0 0x7f5a9c06f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:47:08.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.598+0000 7f5aaea00700 1 -- 172.21.15.107:0/2602785099 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f5aa408d920 con 0x7f5ab81095f0 2024-03-04T13:47:08.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.598+0000 7f5ab6a00700 1 --2- 172.21.15.107:0/2602785099 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f5a9c06d0f0 0x7f5a9c06f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:47:08.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.599+0000 7f5ab6a00700 1 --2- 172.21.15.107:0/2602785099 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f5a9c06d0f0 0x7f5a9c06f5a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5aa8006fd0 tx=0x7f5aa8008040 comp rx=0 tx=0).ready entity=mgr.14696 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:47:08.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.603+0000 7f5aaea00700 1 -- 172.21.15.107:0/2602785099 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f5aa405b550 con 0x7f5ab81095f0 2024-03-04T13:47:08.842 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:08 smithi107 ceph-mon[28739]: from='client.14876 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:47:08.842 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:08 smithi107 ceph-mon[28739]: from='client.? 172.21.15.107:0/827644585' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:47:08.842 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:08 smithi107 ceph-mon[28739]: from='client.? 172.21.15.107:0/608990760' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-04T13:47:08.842 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:08 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:47:08.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.920+0000 7f5abd600700 1 -- 172.21.15.107:0/2602785099 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f5ab81106a0 con 0x7f5ab81095f0 2024-03-04T13:47:08.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.920+0000 7f5aaea00700 1 -- 172.21.15.107:0/2602785099 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f5aa405b0e0 con 0x7f5ab81095f0 2024-03-04T13:47:08.919 INFO:teuthology.orchestra.run.smithi107.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-03-04T13:47:08.919 INFO:teuthology.orchestra.run.smithi107.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-04T13:47:08.919 INFO:teuthology.orchestra.run.smithi107.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-04T13:47:08.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.923+0000 7f5abd600700 1 -- 172.21.15.107:0/2602785099 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f5a9c06d0f0 msgr2=0x7f5a9c06f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:47:08.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.923+0000 7f5abd600700 1 --2- 172.21.15.107:0/2602785099 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f5a9c06d0f0 0x7f5a9c06f5a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5aa8006fd0 tx=0x7f5aa8008040 comp rx=0 tx=0).stop 2024-03-04T13:47:08.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.923+0000 7f5abd600700 1 -- 172.21.15.107:0/2602785099 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ab81095f0 msgr2=0x7f5ab811a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:47:08.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.923+0000 7f5abd600700 1 --2- 172.21.15.107:0/2602785099 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ab81095f0 0x7f5ab811a570 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f5aa4009150 tx=0x7f5aa4009e30 comp rx=0 tx=0).stop 2024-03-04T13:47:08.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.923+0000 7f5abd600700 1 -- 172.21.15.107:0/2602785099 shutdown_connections 2024-03-04T13:47:08.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.923+0000 7f5abd600700 1 --2- 172.21.15.107:0/2602785099 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5ab8103dc0 0x7f5ab811a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:47:08.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.923+0000 7f5abd600700 1 --2- 172.21.15.107:0/2602785099 >> [v2:172.21.15.107:6800/1613321046,v1:172.21.15.107:6801/1613321046] conn(0x7f5a9c06d0f0 0x7f5a9c06f5a0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:47:08.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.923+0000 7f5abd600700 1 --2- 172.21.15.107:0/2602785099 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ab81095f0 0x7f5ab811a570 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:47:08.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.923+0000 7f5abd600700 1 -- 172.21.15.107:0/2602785099 >> 172.21.15.107:0/2602785099 conn(0x7f5ab8072d50 msgr2=0x7f5ab80746b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:47:08.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.923+0000 7f5abd600700 1 -- 172.21.15.107:0/2602785099 shutdown_connections 2024-03-04T13:47:08.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:08.923+0000 7f5abd600700 1 -- 172.21.15.107:0/2602785099 wait complete. 2024-03-04T13:47:09.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:08 smithi169 ceph-mon[36313]: from='client.14876 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:47:09.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:08 smithi169 ceph-mon[36313]: from='client.? 172.21.15.107:0/827644585' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:47:09.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:08 smithi169 ceph-mon[36313]: from='client.? 172.21.15.107:0/608990760' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-04T13:47:09.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:08 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:47:09.877 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:09 smithi107 ceph-mon[28739]: pgmap v108: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:47:09.877 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:09 smithi107 ceph-mon[28739]: from='client.14888 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:47:09.877 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:09 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:47:09.877 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:09 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-04T13:47:09.877 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:09 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-04T13:47:09.877 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:09 smithi107 ceph-mon[28739]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:47:09.877 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:09 smithi107 ceph-mon[28739]: from='client.? 172.21.15.107:0/2602785099' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-04T13:47:10.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:09 smithi169 ceph-mon[36313]: pgmap v108: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:47:10.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:09 smithi169 ceph-mon[36313]: from='client.14888 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:47:10.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:09 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:47:10.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:09 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-04T13:47:10.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:09 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-04T13:47:10.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:09 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:47:10.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:09 smithi169 ceph-mon[36313]: from='client.? 172.21.15.107:0/2602785099' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-04T13:47:10.843 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:10 smithi107 ceph-mon[28739]: Upgrade: Updating mon.smithi107 2024-03-04T13:47:10.843 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:10 smithi107 ceph-mon[28739]: Deploying daemon mon.smithi107 on smithi107 2024-03-04T13:47:11.109 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:10 smithi169 ceph-mon[36313]: Upgrade: Updating mon.smithi107 2024-03-04T13:47:11.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:10 smithi169 ceph-mon[36313]: Deploying daemon mon.smithi107 on smithi107 2024-03-04T13:47:11.843 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:11 smithi107 systemd[1]: Stopping Ceph mon.smithi107 for 1fc09a58-da2b-11ee-95c6-87774f69a715... 2024-03-04T13:47:11.843 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:11 smithi107 ceph-mon[28739]: pgmap v109: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:47:12.109 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:11 smithi169 ceph-mon[36313]: pgmap v109: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:47:12.286 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:11 smithi107 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-mon-smithi107[28734]: 2024-03-04T13:47:11.896+0000 7f1a3ce00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi107 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-04T13:47:12.286 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:11 smithi107 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-mon-smithi107[28734]: 2024-03-04T13:47:11.896+0000 7f1a3ce00700 -1 mon.smithi107@0(leader) e2 *** Got Signal Terminated *** 2024-03-04T13:47:12.286 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:11 smithi107 podman[112406]: 2024-03-04 13:47:11.968011935 +0000 UTC m=+0.260810654 container died 49f5da990d351f8884dee805009c3a572c63450c8a57344d930aedde1c6b4e6e (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-mon-smithi107, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, RELEASE=quincy-42608bb, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-03-04T13:47:12.593 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:12 smithi107 podman[112406]: 2024-03-04 13:47:12.422222141 +0000 UTC m=+0.715020916 container cleanup 49f5da990d351f8884dee805009c3a572c63450c8a57344d930aedde1c6b4e6e (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-mon-smithi107, org.label-schema.build-date=20240222, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=quincy-42608bb, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8) 2024-03-04T13:47:12.593 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:12 smithi107 bash[112406]: ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-mon-smithi107 2024-03-04T13:47:13.092 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:12 smithi107 podman[112431]: 2024-03-04 13:47:12.689931917 +0000 UTC m=+0.710460321 container remove 49f5da990d351f8884dee805009c3a572c63450c8a57344d930aedde1c6b4e6e (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-mon-smithi107, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=quincy-42608bb, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-03-04T13:47:13.531 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:13 smithi107 systemd[1]: ceph-1fc09a58-da2b-11ee-95c6-87774f69a715@mon.smithi107.service: Deactivated successfully. 2024-03-04T13:47:13.531 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:13 smithi107 systemd[1]: Stopped Ceph mon.smithi107 for 1fc09a58-da2b-11ee-95c6-87774f69a715. 2024-03-04T13:47:13.531 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:13 smithi107 systemd[1]: ceph-1fc09a58-da2b-11ee-95c6-87774f69a715@mon.smithi107.service: Consumed 30.561s CPU time. 2024-03-04T13:47:13.842 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:13 smithi107 systemd[1]: Starting Ceph mon.smithi107 for 1fc09a58-da2b-11ee-95c6-87774f69a715... 2024-03-04T13:47:14.177 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:13 smithi107 podman[112558]: 2024-03-04 13:47:13.780716556 +0000 UTC m=+0.032395741 image pull d71b839abdf892550525dad59fdf20a6b925b73dbcc7526f0d04da8380b9fabf quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377 2024-03-04T13:47:14.178 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:13 smithi107 podman[112558]: 2024-03-04 13:47:13.8990762 +0000 UTC m=+0.150755426 container create 32f07144274f6a2f547db4f34551424313f2e40c5d24141940b97c90ca4204eb (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-mon-smithi107, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, org.label-schema.vendor=CentOS, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240222, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-03-04T13:47:14.178 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 podman[112558]: 2024-03-04 13:47:14.177911003 +0000 UTC m=+0.429590231 container init 32f07144274f6a2f547db4f34551424313f2e40c5d24141940b97c90ca4204eb (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-mon-smithi107, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240222, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8) 2024-03-04T13:47:14.594 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 podman[112558]: 2024-03-04 13:47:14.189578877 +0000 UTC m=+0.441258122 container start 32f07144274f6a2f547db4f34551424313f2e40c5d24141940b97c90ca4204eb (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-mon-smithi107, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-04T13:47:14.594 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: set uid:gid to 167:167 (ceph:ceph) 2024-03-04T13:47:14.595 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: ceph version 19.0.0-1845-g717ce3cc (717ce3cce8d0166e4e577faa8004e6f9cb4128c0) squid (dev), process ceph-mon, pid 2 2024-03-04T13:47:14.595 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: pidfile_write: ignore empty --pid-file 2024-03-04T13:47:14.595 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: load: jerasure load: lrc 2024-03-04T13:47:14.595 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: RocksDB version: 7.9.2 2024-03-04T13:47:14.595 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Git sha 0 2024-03-04T13:47:14.595 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Compile date 2024-03-04 04:43:24 2024-03-04T13:47:14.595 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: DB SUMMARY 2024-03-04T13:47:14.595 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: DB Session ID: OUUSSCIQ0PWBY1ZXJD3D 2024-03-04T13:47:14.595 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: CURRENT file: CURRENT 2024-03-04T13:47:14.595 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: IDENTITY file: IDENTITY 2024-03-04T13:47:14.595 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: MANIFEST file: MANIFEST-000015 size: 1346 Bytes 2024-03-04T13:47:14.595 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi107/store.db dir, Total Num: 1, files: 000036.sst 2024-03-04T13:47:14.595 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi107/store.db: 000034.log size: 595730 ; 2024-03-04T13:47:14.595 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.error_if_exists: 0 2024-03-04T13:47:14.595 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.create_if_missing: 0 2024-03-04T13:47:14.595 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.paranoid_checks: 1 2024-03-04T13:47:14.596 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-04T13:47:14.596 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-04T13:47:14.596 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-04T13:47:14.596 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.env: 0x560d9d585b20 2024-03-04T13:47:14.596 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.fs: PosixFileSystem 2024-03-04T13:47:14.596 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.info_log: 0x560d9fa757c0 2024-03-04T13:47:14.596 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.max_file_opening_threads: 16 2024-03-04T13:47:14.596 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.statistics: (nil) 2024-03-04T13:47:14.596 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.use_fsync: 0 2024-03-04T13:47:14.596 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.max_log_file_size: 0 2024-03-04T13:47:14.596 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-04T13:47:14.596 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-04T13:47:14.596 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.keep_log_file_num: 1000 2024-03-04T13:47:14.597 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.recycle_log_file_num: 0 2024-03-04T13:47:14.597 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.allow_fallocate: 1 2024-03-04T13:47:14.597 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.allow_mmap_reads: 0 2024-03-04T13:47:14.597 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.allow_mmap_writes: 0 2024-03-04T13:47:14.597 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.use_direct_reads: 0 2024-03-04T13:47:14.597 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-04T13:47:14.597 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.create_missing_column_families: 0 2024-03-04T13:47:14.597 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.db_log_dir: 2024-03-04T13:47:14.597 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.wal_dir: 2024-03-04T13:47:14.597 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-04T13:47:14.597 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-04T13:47:14.598 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-04T13:47:14.598 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-04T13:47:14.598 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-04T13:47:14.598 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-04T13:47:14.598 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.advise_random_on_open: 1 2024-03-04T13:47:14.598 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.db_write_buffer_size: 0 2024-03-04T13:47:14.598 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.write_buffer_manager: 0x560d9fa49400 2024-03-04T13:47:14.598 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-04T13:47:14.598 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-04T13:47:14.598 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-04T13:47:14.598 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.rate_limiter: (nil) 2024-03-04T13:47:14.598 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-04T13:47:14.598 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.wal_recovery_mode: 2 2024-03-04T13:47:14.598 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.enable_thread_tracking: 0 2024-03-04T13:47:14.598 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.enable_pipelined_write: 0 2024-03-04T13:47:14.598 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.unordered_write: 0 2024-03-04T13:47:14.599 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-04T13:47:14.599 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-04T13:47:14.599 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-04T13:47:14.599 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-04T13:47:14.599 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.row_cache: None 2024-03-04T13:47:14.599 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.wal_filter: None 2024-03-04T13:47:14.599 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-04T13:47:14.599 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.allow_ingest_behind: 0 2024-03-04T13:47:14.599 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.two_write_queues: 0 2024-03-04T13:47:14.599 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.manual_wal_flush: 0 2024-03-04T13:47:14.599 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.wal_compression: 0 2024-03-04T13:47:14.599 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.atomic_flush: 0 2024-03-04T13:47:14.599 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-04T13:47:14.599 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-04T13:47:14.600 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-04T13:47:14.600 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.log_readahead_size: 0 2024-03-04T13:47:14.600 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-04T13:47:14.600 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.best_efforts_recovery: 0 2024-03-04T13:47:14.600 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-04T13:47:14.600 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-04T13:47:14.600 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.allow_data_in_errors: 0 2024-03-04T13:47:14.600 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.db_host_id: __hostname__ 2024-03-04T13:47:14.600 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-04T13:47:14.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.max_background_jobs: 2 2024-03-04T13:47:14.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.max_background_compactions: -1 2024-03-04T13:47:14.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.max_subcompactions: 1 2024-03-04T13:47:14.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-04T13:47:14.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-04T13:47:14.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-04T13:47:14.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.max_total_wal_size: 0 2024-03-04T13:47:14.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-04T13:47:14.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-04T13:47:14.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-04T13:47:14.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-04T13:47:14.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.max_open_files: -1 2024-03-04T13:47:14.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.bytes_per_sync: 0 2024-03-04T13:47:14.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-04T13:47:14.601 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-04T13:47:14.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.compaction_readahead_size: 0 2024-03-04T13:47:14.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.max_background_flushes: -1 2024-03-04T13:47:14.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Compression algorithms supported: 2024-03-04T13:47:14.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-04T13:47:14.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: kZSTD supported: 0 2024-03-04T13:47:14.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: kXpressCompression supported: 0 2024-03-04T13:47:14.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: kLZ4HCCompression supported: 1 2024-03-04T13:47:14.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: kZlibCompression supported: 1 2024-03-04T13:47:14.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: kSnappyCompression supported: 1 2024-03-04T13:47:14.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: kLZ4Compression supported: 1 2024-03-04T13:47:14.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: kBZip2Compression supported: 0 2024-03-04T13:47:14.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-04T13:47:14.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-04T13:47:14.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi107/store.db/MANIFEST-000015 2024-03-04T13:47:14.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-04T13:47:14.602 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-04T13:47:14.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.merge_operator: 2024-03-04T13:47:14.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.compaction_filter: None 2024-03-04T13:47:14.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.compaction_filter_factory: None 2024-03-04T13:47:14.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.sst_partitioner_factory: None 2024-03-04T13:47:14.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-04T13:47:14.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-04T13:47:14.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x560d9fa758e0) 2024-03-04T13:47:14.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: cache_index_and_filter_blocks: 1 2024-03-04T13:47:14.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-04T13:47:14.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-04T13:47:14.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: pin_top_level_index_and_filter: 1 2024-03-04T13:47:14.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: index_type: 0 2024-03-04T13:47:14.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: data_block_index_type: 0 2024-03-04T13:47:14.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: index_shortening: 1 2024-03-04T13:47:14.603 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-04T13:47:14.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: checksum: 4 2024-03-04T13:47:14.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: no_block_cache: 0 2024-03-04T13:47:14.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: block_cache: 0x560d9fa7b090 2024-03-04T13:47:14.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: block_cache_name: BinnedLRUCache 2024-03-04T13:47:14.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: block_cache_options: 2024-03-04T13:47:14.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: capacity : 536870912 2024-03-04T13:47:14.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: num_shard_bits : 4 2024-03-04T13:47:14.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: strict_capacity_limit : 0 2024-03-04T13:47:14.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: high_pri_pool_ratio: 0.000 2024-03-04T13:47:14.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: block_cache_compressed: (nil) 2024-03-04T13:47:14.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: persistent_cache: (nil) 2024-03-04T13:47:14.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: block_size: 4096 2024-03-04T13:47:14.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: block_size_deviation: 10 2024-03-04T13:47:14.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: block_restart_interval: 16 2024-03-04T13:47:14.604 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: index_block_restart_interval: 1 2024-03-04T13:47:14.605 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: metadata_block_size: 4096 2024-03-04T13:47:14.605 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: partition_filters: 0 2024-03-04T13:47:14.605 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: use_delta_encoding: 1 2024-03-04T13:47:14.605 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: filter_policy: bloomfilter 2024-03-04T13:47:14.605 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: whole_key_filtering: 1 2024-03-04T13:47:14.605 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: verify_compression: 0 2024-03-04T13:47:14.605 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: read_amp_bytes_per_bit: 0 2024-03-04T13:47:14.605 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: format_version: 5 2024-03-04T13:47:14.605 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: enable_index_compression: 1 2024-03-04T13:47:14.605 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: block_align: 0 2024-03-04T13:47:14.605 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: max_auto_readahead_size: 262144 2024-03-04T13:47:14.605 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: prepopulate_block_cache: 0 2024-03-04T13:47:14.605 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: initial_auto_readahead_size: 8192 2024-03-04T13:47:14.605 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: num_file_reads_for_auto_readahead: 2 2024-03-04T13:47:14.605 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.write_buffer_size: 33554432 2024-03-04T13:47:14.605 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.max_write_buffer_number: 2 2024-03-04T13:47:14.606 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.compression: NoCompression 2024-03-04T13:47:14.606 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.bottommost_compression: Disabled 2024-03-04T13:47:14.606 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.prefix_extractor: nullptr 2024-03-04T13:47:14.606 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-04T13:47:14.607 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.num_levels: 7 2024-03-04T13:47:14.607 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-04T13:47:14.607 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-04T13:47:14.607 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-04T13:47:14.607 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-04T13:47:14.607 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-04T13:47:14.607 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-04T13:47:14.607 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-04T13:47:14.607 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-04T13:47:14.607 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-04T13:47:14.607 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-04T13:47:14.608 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-04T13:47:14.608 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-04T13:47:14.608 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-04T13:47:14.608 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.compression_opts.level: 32767 2024-03-04T13:47:14.611 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.compression_opts.strategy: 0 2024-03-04T13:47:14.611 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-04T13:47:14.611 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-04T13:47:14.611 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-04T13:47:14.611 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-04T13:47:14.611 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.compression_opts.enabled: false 2024-03-04T13:47:14.611 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-04T13:47:14.611 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-04T13:47:14.611 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-04T13:47:14.611 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-04T13:47:14.611 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.target_file_size_base: 67108864 2024-03-04T13:47:14.611 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-04T13:47:14.611 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-04T13:47:14.612 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-04T13:47:14.612 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-04T13:47:14.612 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-04T13:47:14.612 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-04T13:47:14.612 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-04T13:47:14.612 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-04T13:47:14.612 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-04T13:47:14.612 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-04T13:47:14.612 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-04T13:47:14.612 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-04T13:47:14.612 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-04T13:47:14.612 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-04T13:47:14.612 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.arena_block_size: 1048576 2024-03-04T13:47:14.612 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-04T13:47:14.612 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-04T13:47:14.613 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.disable_auto_compactions: 0 2024-03-04T13:47:14.613 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-04T13:47:14.613 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-04T13:47:14.613 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-04T13:47:14.613 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-04T13:47:14.613 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-04T13:47:14.613 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-04T13:47:14.613 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-04T13:47:14.613 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-04T13:47:14.613 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-04T13:47:14.613 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-04T13:47:14.613 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.table_properties_collectors: 2024-03-04T13:47:14.613 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.inplace_update_support: 0 2024-03-04T13:47:14.613 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-04T13:47:14.613 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-04T13:47:14.613 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-04T13:47:14.614 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-04T13:47:14.614 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.bloom_locality: 0 2024-03-04T13:47:14.614 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.max_successive_merges: 0 2024-03-04T13:47:14.614 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-04T13:47:14.614 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.paranoid_file_checks: 0 2024-03-04T13:47:14.614 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.force_consistency_checks: 1 2024-03-04T13:47:14.614 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.report_bg_io_stats: 0 2024-03-04T13:47:14.614 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.ttl: 2592000 2024-03-04T13:47:14.614 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-04T13:47:14.614 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-04T13:47:14.614 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-04T13:47:14.614 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.enable_blob_files: false 2024-03-04T13:47:14.614 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.min_blob_size: 0 2024-03-04T13:47:14.614 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.blob_file_size: 268435456 2024-03-04T13:47:14.614 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-04T13:47:14.615 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-04T13:47:14.615 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-04T13:47:14.615 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-04T13:47:14.615 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-04T13:47:14.615 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.blob_file_starting_level: 0 2024-03-04T13:47:14.615 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-04T13:47:14.615 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: [table/block_based/block_based_table_reader.cc:721] At least one SST file opened without unique ID to verify: 36.sst 2024-03-04T13:47:14.615 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-03-04T13:47:14.615 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi107/store.db/MANIFEST-000015 succeeded,manifest_file_number is 15, next_file_number is 38, last_sequence is 16538, log_number is 34,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-03-04T13:47:14.615 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 34 2024-03-04T13:47:14.615 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: dcfaf532-0a93-46a9-8105-df3a14b94c0c 2024-03-04T13:47:14.615 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: EVENT_LOG_v1 {"time_micros": 1709560034249378, "job": 1, "event": "recovery_started", "wal_files": [34]} 2024-03-04T13:47:14.615 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #34 mode 2 2024-03-04T13:47:14.615 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: EVENT_LOG_v1 {"time_micros": 1709560034257884, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 39, "file_size": 565844, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 16532, "largest_seqno": 17421, "table_properties": {"data_size": 561814, "index_size": 1992, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 1029, "raw_key_size": 9773, "raw_average_key_size": 25, "raw_value_size": 553697, "raw_average_value_size": 1427, "num_data_blocks": 89, "num_entries": 388, "num_filter_entries": 388, "num_deletions": 7, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1709560034, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "dcfaf532-0a93-46a9-8105-df3a14b94c0c", "db_session_id": "OUUSSCIQ0PWBY1ZXJD3D", "orig_file_number": 39, "seqno_to_time_mapping": "N/A"}} 2024-03-04T13:47:14.616 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: EVENT_LOG_v1 {"time_micros": 1709560034258145, "job": 1, "event": "recovery_finished"} 2024-03-04T13:47:14.616 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: [db/version_set.cc:5047] Creating manifest 41 2024-03-04T13:47:14.616 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-03-04T13:47:14.616 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi107/store.db/000034.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-04T13:47:14.616 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x560d9fb6c000 2024-03-04T13:47:14.616 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: DB pointer 0x560d9fb58000 2024-03-04T13:47:14.616 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-04T13:47:14.616 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-04T13:47:14.616 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: ** DB Stats ** 2024-03-04T13:47:14.616 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-04T13:47:14.616 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-04T13:47:14.616 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-04T13:47:14.616 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-04T13:47:14.617 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-04T13:47:14.617 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-04T13:47:14.617 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-04T13:47:14.617 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: 2024-03-04T13:47:14.617 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: ** Compaction Stats [default] ** 2024-03-04T13:47:14.617 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-04T13:47:14.617 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-04T13:47:14.617 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: L0 1/0 552.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 110.2 0.00 0.00 1 0.005 0 0 0.0 0.0 2024-03-04T13:47:14.617 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: L6 1/0 7.46 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-03-04T13:47:14.617 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Sum 2/0 8.00 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 110.2 0.00 0.00 1 0.005 0 0 0.0 0.0 2024-03-04T13:47:14.617 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 110.2 0.00 0.00 1 0.005 0 0 0.0 0.0 2024-03-04T13:47:14.617 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: 2024-03-04T13:47:14.617 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: ** Compaction Stats [default] ** 2024-03-04T13:47:14.617 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-04T13:47:14.617 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-04T13:47:14.617 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 110.2 0.00 0.00 1 0.005 0 0 0.0 0.0 2024-03-04T13:47:14.618 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: 2024-03-04T13:47:14.618 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-04T13:47:14.618 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: 2024-03-04T13:47:14.618 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-04T13:47:14.618 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Flush(GB): cumulative 0.001, interval 0.001 2024-03-04T13:47:14.618 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-04T13:47:14.618 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-04T13:47:14.618 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-04T13:47:14.618 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-04T13:47:14.618 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Cumulative compaction: 0.00 GB write, 37.63 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-04T13:47:14.618 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Interval compaction: 0.00 GB write, 37.63 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-04T13:47:14.618 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-04T13:47:14.618 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Block cache BinnedLRUCache@0x560d9fa7b090#2 capacity: 512.00 MB usage: 3.23 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-03-04T13:47:14.618 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,1.16 KB,0.000220537%) IndexBlock(1,2.08 KB,0.000396371%) Misc(1,0.00 KB,0%) 2024-03-04T13:47:14.618 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: 2024-03-04T13:47:14.619 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-04T13:47:14.619 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: starting mon.smithi107 rank 0 at public addrs [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] at bind addrs [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi107 fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 2024-03-04T13:47:14.619 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: mon.smithi107@-1(???) e2 preinit fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 2024-03-04T13:47:14.619 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: mon.smithi107@-1(???).mds e16 new map 2024-03-04T13:47:14.619 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: mon.smithi107@-1(???).mds e16 print_map 2024-03-04T13:47:14.619 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: e16 2024-03-04T13:47:14.619 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-04T13:47:14.619 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-04T13:47:14.619 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: legacy client fscid: 1 2024-03-04T13:47:14.619 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: 2024-03-04T13:47:14.619 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Filesystem 'cephfs' (1) 2024-03-04T13:47:14.619 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: fs_name cephfs 2024-03-04T13:47:14.619 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: epoch 15 2024-03-04T13:47:14.619 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-04T13:47:14.619 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: created 2024-03-04T13:38:30.854004+0000 2024-03-04T13:47:14.620 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: modified 2024-03-04T13:38:50.015989+0000 2024-03-04T13:47:14.620 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: tableserver 0 2024-03-04T13:47:14.620 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: root 0 2024-03-04T13:47:14.620 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: session_timeout 60 2024-03-04T13:47:14.620 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: session_autoclose 300 2024-03-04T13:47:14.620 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: max_file_size 1099511627776 2024-03-04T13:47:14.620 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: max_xattr_size 65536 2024-03-04T13:47:14.620 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: required_client_features {} 2024-03-04T13:47:14.620 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: last_failure 0 2024-03-04T13:47:14.620 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: last_failure_osd_epoch 47 2024-03-04T13:47:14.620 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-04T13:47:14.620 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: max_mds 1 2024-03-04T13:47:14.620 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: in 0 2024-03-04T13:47:14.621 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: up {0=24247} 2024-03-04T13:47:14.621 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: failed 2024-03-04T13:47:14.621 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: damaged 2024-03-04T13:47:14.621 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: stopped 2024-03-04T13:47:14.621 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: data_pools [3] 2024-03-04T13:47:14.621 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: metadata_pool 2 2024-03-04T13:47:14.621 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: inline_data enabled 2024-03-04T13:47:14.621 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: balancer 2024-03-04T13:47:14.621 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: bal_rank_mask -1 2024-03-04T13:47:14.621 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: standby_count_wanted 1 2024-03-04T13:47:14.621 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: [mds.cephfs.smithi169.jusudl{0:24247} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.169:6824/2994866076,v1:172.21.15.169:6825/2994866076] compat {c=[1],r=[1],i=[7ff]}] 2024-03-04T13:47:14.621 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: [mds.cephfs.smithi107.thycho{0:14470} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.107:6828/1985577060,v1:172.21.15.107:6829/1985577060] compat {c=[1],r=[1],i=[7ff]}] 2024-03-04T13:47:14.621 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: 2024-03-04T13:47:14.621 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: 2024-03-04T13:47:14.621 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Standby daemons: 2024-03-04T13:47:14.621 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: 2024-03-04T13:47:14.622 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: [mds.cephfs.smithi107.riexmm{-1:14480} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.107:6826/3261622517,v1:172.21.15.107:6827/3261622517] compat {c=[1],r=[1],i=[7ff]}] 2024-03-04T13:47:14.622 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: [mds.cephfs.smithi169.bxitzk{-1:24259} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.169:6826/3696090493,v1:172.21.15.169:6827/3696090493] compat {c=[1],r=[1],i=[7ff]}] 2024-03-04T13:47:14.622 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: mon.smithi107@-1(???).osd e49 crush map has features 3314933000852226048, adjusting msgr requires 2024-03-04T13:47:14.622 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: mon.smithi107@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-03-04T13:47:14.622 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: mon.smithi107@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-03-04T13:47:14.622 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: mon.smithi107@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-03-04T13:47:14.622 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 ceph-mon[112578]: mon.smithi107@-1(???).paxosservice(auth 1..22) refresh upgraded, format 0 -> 3 2024-03-04T13:47:14.622 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 bash[112558]: 32f07144274f6a2f547db4f34551424313f2e40c5d24141940b97c90ca4204eb 2024-03-04T13:47:14.622 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:14 smithi107 systemd[1]: Started Ceph mon.smithi107 for 1fc09a58-da2b-11ee-95c6-87774f69a715. 2024-03-04T13:47:16.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:15 smithi107 ceph-mon[112578]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mon metadata", "id": "smithi107"}]: dispatch 2024-03-04T13:47:16.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:15 smithi107 ceph-mon[112578]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mon metadata", "id": "smithi169"}]: dispatch 2024-03-04T13:47:16.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:15 smithi107 ceph-mon[112578]: pgmap v110: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:47:16.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:15 smithi107 ceph-mon[112578]: pgmap v111: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:47:16.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:15 smithi107 ceph-mon[112578]: mon.smithi107 calling monitor election 2024-03-04T13:47:16.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:15 smithi107 ceph-mon[112578]: mon.smithi107 is new leader, mons smithi107,smithi169 in quorum (ranks 0,1) 2024-03-04T13:47:16.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:15 smithi107 ceph-mon[112578]: monmap e2: 2 mons at {smithi107=[v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0],smithi169=[v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-04T13:47:16.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:15 smithi107 ceph-mon[112578]: fsmap cephfs:1 {0=cephfs.smithi169.jusudl=up:active} 1 up:standby-replay 2 up:standby 2024-03-04T13:47:16.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:15 smithi107 ceph-mon[112578]: osdmap e49: 6 total, 6 up, 6 in 2024-03-04T13:47:16.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:15 smithi107 ceph-mon[112578]: mgrmap e33: smithi107.wpzgwl(active, since 3m), standbys: smithi169.eihczr 2024-03-04T13:47:16.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:15 smithi107 ceph-mon[112578]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-03-04T13:47:16.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:15 smithi107 ceph-mon[112578]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-04T13:47:16.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:15 smithi107 ceph-mon[112578]: fs cephfs has deprecated feature inline_data enabled. 2024-03-04T13:47:16.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:15 smithi107 ceph-mon[112578]: from='mgr.14696 ' entity='' 2024-03-04T13:47:16.095 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:15 smithi107 ceph-mon[112578]: mgrmap e34: smithi107.wpzgwl(active, since 3m), standbys: smithi169.eihczr 2024-03-04T13:47:16.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:15 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mon metadata", "id": "smithi107"}]: dispatch 2024-03-04T13:47:16.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:15 smithi169 ceph-mon[36313]: from='mgr.14696 172.21.15.107:0/4141126614' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mon metadata", "id": "smithi169"}]: dispatch 2024-03-04T13:47:16.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:15 smithi169 ceph-mon[36313]: pgmap v110: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:47:16.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:15 smithi169 ceph-mon[36313]: pgmap v111: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:47:16.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:15 smithi169 ceph-mon[36313]: mon.smithi107 calling monitor election 2024-03-04T13:47:16.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:15 smithi169 ceph-mon[36313]: mon.smithi107 is new leader, mons smithi107,smithi169 in quorum (ranks 0,1) 2024-03-04T13:47:16.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:15 smithi169 ceph-mon[36313]: monmap e2: 2 mons at {smithi107=[v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0],smithi169=[v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-04T13:47:16.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:15 smithi169 ceph-mon[36313]: fsmap cephfs:1 {0=cephfs.smithi169.jusudl=up:active} 1 up:standby-replay 2 up:standby 2024-03-04T13:47:16.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:15 smithi169 ceph-mon[36313]: osdmap e49: 6 total, 6 up, 6 in 2024-03-04T13:47:16.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:15 smithi169 ceph-mon[36313]: mgrmap e33: smithi107.wpzgwl(active, since 3m), standbys: smithi169.eihczr 2024-03-04T13:47:16.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:15 smithi169 ceph-mon[36313]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-03-04T13:47:16.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:15 smithi169 ceph-mon[36313]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-04T13:47:16.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:15 smithi169 ceph-mon[36313]: fs cephfs has deprecated feature inline_data enabled. 2024-03-04T13:47:16.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:15 smithi169 ceph-mon[36313]: from='mgr.14696 ' entity='' 2024-03-04T13:47:16.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:15 smithi169 ceph-mon[36313]: mgrmap e34: smithi107.wpzgwl(active, since 3m), standbys: smithi169.eihczr 2024-03-04T13:47:24.775 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:24 smithi169 ceph-mon[36313]: Standby manager daemon smithi169.eihczr restarted 2024-03-04T13:47:24.775 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:24 smithi169 ceph-mon[36313]: Standby manager daemon smithi169.eihczr started 2024-03-04T13:47:24.775 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:24 smithi169 ceph-mon[36313]: from='mgr.? 172.21.15.169:0/455588663' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi169.eihczr/crt"}]: dispatch 2024-03-04T13:47:24.775 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:24 smithi169 ceph-mon[36313]: from='mgr.? 172.21.15.169:0/455588663' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-04T13:47:24.775 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:24 smithi169 ceph-mon[36313]: from='mgr.? 172.21.15.169:0/455588663' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi169.eihczr/key"}]: dispatch 2024-03-04T13:47:24.775 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:24 smithi169 ceph-mon[36313]: from='mgr.? 172.21.15.169:0/455588663' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-04T13:47:24.842 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:24 smithi107 ceph-mon[112578]: Standby manager daemon smithi169.eihczr restarted 2024-03-04T13:47:24.842 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:24 smithi107 ceph-mon[112578]: Standby manager daemon smithi169.eihczr started 2024-03-04T13:47:24.842 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:24 smithi107 ceph-mon[112578]: from='mgr.? 172.21.15.169:0/455588663' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi169.eihczr/crt"}]: dispatch 2024-03-04T13:47:24.843 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:24 smithi107 ceph-mon[112578]: from='mgr.? 172.21.15.169:0/455588663' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-04T13:47:24.843 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:24 smithi107 ceph-mon[112578]: from='mgr.? 172.21.15.169:0/455588663' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi169.eihczr/key"}]: dispatch 2024-03-04T13:47:24.843 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:24 smithi107 ceph-mon[112578]: from='mgr.? 172.21.15.169:0/455588663' entity='mgr.smithi169.eihczr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-04T13:47:25.843 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:25 smithi107 ceph-mon[112578]: mgrmap e35: smithi107.wpzgwl(active, since 3m), standbys: smithi169.eihczr 2024-03-04T13:47:25.843 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:25 smithi107 ceph-mon[112578]: Active manager daemon smithi107.wpzgwl restarted 2024-03-04T13:47:25.843 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:25 smithi107 ceph-mon[112578]: Activating manager daemon smithi107.wpzgwl 2024-03-04T13:47:25.844 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:25 smithi107 ceph-mon[112578]: osdmap e50: 6 total, 6 up, 6 in 2024-03-04T13:47:25.844 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:25 smithi107 ceph-mon[112578]: mgrmap e36: smithi107.wpzgwl(active, starting, since 0.0164812s), standbys: smithi169.eihczr 2024-03-04T13:47:25.844 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:25 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mon metadata", "id": "smithi107"}]: dispatch 2024-03-04T13:47:25.844 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:25 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mon metadata", "id": "smithi169"}]: dispatch 2024-03-04T13:47:25.844 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:25 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi107.thycho"}]: dispatch 2024-03-04T13:47:25.844 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:25 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi107.riexmm"}]: dispatch 2024-03-04T13:47:25.844 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:25 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi169.jusudl"}]: dispatch 2024-03-04T13:47:25.844 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:25 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi169.bxitzk"}]: dispatch 2024-03-04T13:47:25.844 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:25 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mgr metadata", "who": "smithi107.wpzgwl", "id": "smithi107.wpzgwl"}]: dispatch 2024-03-04T13:47:25.844 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:25 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mgr metadata", "who": "smithi169.eihczr", "id": "smithi169.eihczr"}]: dispatch 2024-03-04T13:47:25.844 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:25 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-04T13:47:25.845 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:25 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-04T13:47:25.845 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:25 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-04T13:47:25.845 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:25 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-04T13:47:25.845 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:25 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-04T13:47:25.845 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:25 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-04T13:47:25.845 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:25 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-04T13:47:25.845 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:25 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-04T13:47:25.845 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:25 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-04T13:47:25.845 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:25 smithi107 ceph-mon[112578]: Manager daemon smithi107.wpzgwl is now available 2024-03-04T13:47:25.845 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:25 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:47:25.845 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:25 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:47:25.845 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:25 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:47:25.845 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:25 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi107.wpzgwl/mirror_snapshot_schedule"}]: dispatch 2024-03-04T13:47:25.846 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:25 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi107.wpzgwl/trash_purge_schedule"}]: dispatch 2024-03-04T13:47:25.861 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:25 smithi169 ceph-mon[36313]: mgrmap e35: smithi107.wpzgwl(active, since 3m), standbys: smithi169.eihczr 2024-03-04T13:47:25.861 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:25 smithi169 ceph-mon[36313]: Active manager daemon smithi107.wpzgwl restarted 2024-03-04T13:47:25.861 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:25 smithi169 ceph-mon[36313]: Activating manager daemon smithi107.wpzgwl 2024-03-04T13:47:25.861 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:25 smithi169 ceph-mon[36313]: osdmap e50: 6 total, 6 up, 6 in 2024-03-04T13:47:25.861 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:25 smithi169 ceph-mon[36313]: mgrmap e36: smithi107.wpzgwl(active, starting, since 0.0164812s), standbys: smithi169.eihczr 2024-03-04T13:47:25.861 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:25 smithi169 ceph-mon[36313]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mon metadata", "id": "smithi107"}]: dispatch 2024-03-04T13:47:25.861 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:25 smithi169 ceph-mon[36313]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mon metadata", "id": "smithi169"}]: dispatch 2024-03-04T13:47:25.861 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:25 smithi169 ceph-mon[36313]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi107.thycho"}]: dispatch 2024-03-04T13:47:25.862 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:25 smithi169 ceph-mon[36313]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi107.riexmm"}]: dispatch 2024-03-04T13:47:25.862 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:25 smithi169 ceph-mon[36313]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi169.jusudl"}]: dispatch 2024-03-04T13:47:25.862 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:25 smithi169 ceph-mon[36313]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi169.bxitzk"}]: dispatch 2024-03-04T13:47:25.862 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:25 smithi169 ceph-mon[36313]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mgr metadata", "who": "smithi107.wpzgwl", "id": "smithi107.wpzgwl"}]: dispatch 2024-03-04T13:47:25.862 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:25 smithi169 ceph-mon[36313]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mgr metadata", "who": "smithi169.eihczr", "id": "smithi169.eihczr"}]: dispatch 2024-03-04T13:47:25.862 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:25 smithi169 ceph-mon[36313]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-04T13:47:25.862 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:25 smithi169 ceph-mon[36313]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-04T13:47:25.862 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:25 smithi169 ceph-mon[36313]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-04T13:47:25.862 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:25 smithi169 ceph-mon[36313]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-04T13:47:25.862 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:25 smithi169 ceph-mon[36313]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-04T13:47:25.862 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:25 smithi169 ceph-mon[36313]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-04T13:47:25.862 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:25 smithi169 ceph-mon[36313]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-04T13:47:25.862 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:25 smithi169 ceph-mon[36313]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-04T13:47:25.862 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:25 smithi169 ceph-mon[36313]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-04T13:47:25.863 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:25 smithi169 ceph-mon[36313]: Manager daemon smithi107.wpzgwl is now available 2024-03-04T13:47:25.863 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:25 smithi169 ceph-mon[36313]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:47:25.863 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:25 smithi169 ceph-mon[36313]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:47:25.863 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:25 smithi169 ceph-mon[36313]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:47:25.863 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:25 smithi169 ceph-mon[36313]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi107.wpzgwl/mirror_snapshot_schedule"}]: dispatch 2024-03-04T13:47:25.863 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:25 smithi169 ceph-mon[36313]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi107.wpzgwl/trash_purge_schedule"}]: dispatch 2024-03-04T13:47:27.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:26 smithi107 ceph-mon[112578]: [04/Mar/2024:13:47:25] ENGINE Bus STARTING 2024-03-04T13:47:27.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:26 smithi107 ceph-mon[112578]: [04/Mar/2024:13:47:25] ENGINE Serving on https://172.21.15.107:7150 2024-03-04T13:47:27.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:26 smithi107 ceph-mon[112578]: mgrmap e37: smithi107.wpzgwl(active, since 1.0229s), standbys: smithi169.eihczr 2024-03-04T13:47:27.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:26 smithi107 ceph-mon[112578]: pgmap v3: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-04T13:47:27.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:26 smithi107 ceph-mon[112578]: [04/Mar/2024:13:47:25] ENGINE Serving on http://172.21.15.107:8765 2024-03-04T13:47:27.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:26 smithi107 ceph-mon[112578]: [04/Mar/2024:13:47:25] ENGINE Bus STARTED 2024-03-04T13:47:27.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:26 smithi169 ceph-mon[36313]: [04/Mar/2024:13:47:25] ENGINE Bus STARTING 2024-03-04T13:47:27.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:26 smithi169 ceph-mon[36313]: [04/Mar/2024:13:47:25] ENGINE Serving on https://172.21.15.107:7150 2024-03-04T13:47:27.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:26 smithi169 ceph-mon[36313]: mgrmap e37: smithi107.wpzgwl(active, since 1.0229s), standbys: smithi169.eihczr 2024-03-04T13:47:27.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:26 smithi169 ceph-mon[36313]: pgmap v3: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-04T13:47:27.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:26 smithi169 ceph-mon[36313]: [04/Mar/2024:13:47:25] ENGINE Serving on http://172.21.15.107:8765 2024-03-04T13:47:27.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:26 smithi169 ceph-mon[36313]: [04/Mar/2024:13:47:25] ENGINE Bus STARTED 2024-03-04T13:47:27.860 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:27 smithi169 ceph-mon[36313]: pgmap v4: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-04T13:47:27.860 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:27 smithi169 ceph-mon[36313]: mgrmap e38: smithi107.wpzgwl(active, since 2s), standbys: smithi169.eihczr 2024-03-04T13:47:28.092 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:27 smithi107 ceph-mon[112578]: pgmap v4: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-04T13:47:28.092 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:27 smithi107 ceph-mon[112578]: mgrmap e38: smithi107.wpzgwl(active, since 2s), standbys: smithi169.eihczr 2024-03-04T13:47:29.993 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:29 smithi107 ceph-mon[112578]: pgmap v5: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-04T13:47:29.993 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:29 smithi107 ceph-mon[112578]: mgrmap e39: smithi107.wpzgwl(active, since 4s), standbys: smithi169.eihczr 2024-03-04T13:47:29.993 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:29 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:47:29.993 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:29 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:47:30.109 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:29 smithi169 ceph-mon[36313]: pgmap v5: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-04T13:47:30.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:29 smithi169 ceph-mon[36313]: mgrmap e39: smithi107.wpzgwl(active, since 4s), standbys: smithi169.eihczr 2024-03-04T13:47:30.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:29 smithi169 ceph-mon[36313]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:47:30.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:29 smithi169 ceph-mon[36313]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:47:32.076 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:31 smithi107 ceph-mon[112578]: pgmap v6: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-04T13:47:32.076 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:31 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:47:32.076 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:31 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:47:32.109 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:31 smithi169 ceph-mon[36313]: pgmap v6: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-03-04T13:47:32.109 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:31 smithi169 ceph-mon[36313]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:47:32.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:31 smithi169 ceph-mon[36313]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:47:33.342 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:33 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:47:33.342 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:33 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:47:33.343 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:33 smithi107 ceph-mon[112578]: pgmap v7: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 30 KiB/s rd, 0 B/s wr, 18 op/s 2024-03-04T13:47:33.360 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:33 smithi169 ceph-mon[36313]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:47:33.360 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:33 smithi169 ceph-mon[36313]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:47:33.360 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:33 smithi169 ceph-mon[36313]: pgmap v7: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 30 KiB/s rd, 0 B/s wr, 18 op/s 2024-03-04T13:47:35.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:35 smithi169 ceph-mon[36313]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:47:35.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:35 smithi169 ceph-mon[36313]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:47:35.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:35 smithi169 ceph-mon[36313]: pgmap v8: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 23 KiB/s rd, 0 B/s wr, 14 op/s 2024-03-04T13:47:35.291 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:35 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:47:35.291 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:35 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:47:35.291 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:35 smithi107 ceph-mon[112578]: pgmap v8: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 23 KiB/s rd, 0 B/s wr, 14 op/s 2024-03-04T13:47:37.843 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:37 smithi107 ceph-mon[112578]: Detected new or changed devices on smithi169 2024-03-04T13:47:37.843 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:37 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:47:37.843 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:37 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:47:37.844 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:37 smithi107 ceph-mon[112578]: pgmap v9: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 20 KiB/s rd, 0 B/s wr, 12 op/s 2024-03-04T13:47:37.860 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:37 smithi169 ceph-mon[36313]: Detected new or changed devices on smithi169 2024-03-04T13:47:37.860 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:37 smithi169 ceph-mon[36313]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:47:37.860 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:37 smithi169 ceph-mon[36313]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:47:37.860 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:37 smithi169 ceph-mon[36313]: pgmap v9: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 20 KiB/s rd, 0 B/s wr, 12 op/s 2024-03-04T13:47:39.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.083+0000 7f42e6600700 1 -- 172.21.15.107:0/1567760550 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42e00fbdd0 msgr2=0x7f42e00fc1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:47:39.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.083+0000 7f42e6600700 1 --2- 172.21.15.107:0/1567760550 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42e00fbdd0 0x7f42e00fc1a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f42d0004940 tx=0x7f42d0009e30 comp rx=0 tx=0).stop 2024-03-04T13:47:39.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.084+0000 7f42e6600700 1 -- 172.21.15.107:0/1567760550 shutdown_connections 2024-03-04T13:47:39.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.084+0000 7f42e6600700 1 --2- 172.21.15.107:0/1567760550 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f42e00fc6e0 0x7f42e0109440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:47:39.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.084+0000 7f42e6600700 1 --2- 172.21.15.107:0/1567760550 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42e00fbdd0 0x7f42e00fc1a0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:47:39.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.084+0000 7f42e6600700 1 -- 172.21.15.107:0/1567760550 >> 172.21.15.107:0/1567760550 conn(0x7f42e0071d90 msgr2=0x7f42e0072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:47:39.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.084+0000 7f42e6600700 1 -- 172.21.15.107:0/1567760550 shutdown_connections 2024-03-04T13:47:39.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.084+0000 7f42e6600700 1 -- 172.21.15.107:0/1567760550 wait complete. 2024-03-04T13:47:39.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.085+0000 7f42e6600700 1 Processor -- start 2024-03-04T13:47:39.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.085+0000 7f42e6600700 1 -- start start 2024-03-04T13:47:39.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.085+0000 7f42e6600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42e00fbdd0 0x7f42e0115cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:47:39.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.085+0000 7f42e6600700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f42e00fc6e0 0x7f42e0116210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:47:39.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.086+0000 7f42e6600700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42e0116860 con 0x7f42e00fbdd0 2024-03-04T13:47:39.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.086+0000 7f42e6600700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42e010fb70 con 0x7f42e00fc6e0 2024-03-04T13:47:39.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.086+0000 7f42df400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42e00fbdd0 0x7f42e0115cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:47:39.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.086+0000 7f42df400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42e00fbdd0 0x7f42e0115cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37034/0 (socket says 172.21.15.107:37034) 2024-03-04T13:47:39.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.086+0000 7f42df400700 1 -- 172.21.15.107:0/4286810844 learned_addr learned my addr 172.21.15.107:0/4286810844 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:47:39.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.086+0000 7f42dea00700 1 --2- 172.21.15.107:0/4286810844 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f42e00fc6e0 0x7f42e0116210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:47:39.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.086+0000 7f42df400700 1 -- 172.21.15.107:0/4286810844 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f42e00fc6e0 msgr2=0x7f42e0116210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:47:39.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.086+0000 7f42df400700 1 --2- 172.21.15.107:0/4286810844 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f42e00fc6e0 0x7f42e0116210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:47:39.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.086+0000 7f42df400700 1 -- 172.21.15.107:0/4286810844 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42d0005040 con 0x7f42e00fbdd0 2024-03-04T13:47:39.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.087+0000 7f42df400700 1 --2- 172.21.15.107:0/4286810844 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42e00fbdd0 0x7f42e0115cd0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f42d0009360 tx=0x7f42d0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:47:39.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.087+0000 7f42d7400700 1 -- 172.21.15.107:0/4286810844 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f42d000e040 con 0x7f42e00fbdd0 2024-03-04T13:47:39.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.087+0000 7f42d7400700 1 -- 172.21.15.107:0/4286810844 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f42d000bb90 con 0x7f42e00fbdd0 2024-03-04T13:47:39.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.087+0000 7f42e6600700 1 -- 172.21.15.107:0/4286810844 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42e010fdf0 con 0x7f42e00fbdd0 2024-03-04T13:47:39.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.087+0000 7f42e6600700 1 -- 172.21.15.107:0/4286810844 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42e0110330 con 0x7f42e00fbdd0 2024-03-04T13:47:39.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.088+0000 7f42d7400700 1 -- 172.21.15.107:0/4286810844 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f42d0013ea0 con 0x7f42e00fbdd0 2024-03-04T13:47:39.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.089+0000 7f42e6600700 1 -- 172.21.15.107:0/4286810844 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42c00052e0 con 0x7f42e00fbdd0 2024-03-04T13:47:39.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.091+0000 7f42d7400700 1 -- 172.21.15.107:0/4286810844 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 39) v1 ==== 91587+0+0 (secure 0 0 0) 0x7f42d000bd00 con 0x7f42e00fbdd0 2024-03-04T13:47:39.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.092+0000 7f42d7400700 1 --2- 172.21.15.107:0/4286810844 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f42c406d900 0x7f42c406fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:47:39.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.092+0000 7f42d7400700 1 -- 172.21.15.107:0/4286810844 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f42d008ee50 con 0x7f42e00fbdd0 2024-03-04T13:47:39.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.092+0000 7f42dea00700 1 --2- 172.21.15.107:0/4286810844 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f42c406d900 0x7f42c406fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:47:39.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.093+0000 7f42dea00700 1 --2- 172.21.15.107:0/4286810844 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f42c406d900 0x7f42c406fdb0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f42cc007fd0 tx=0x7f42cc009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:47:39.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.096+0000 7f42d7400700 1 -- 172.21.15.107:0/4286810844 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f42d005c060 con 0x7f42e00fbdd0 2024-03-04T13:47:39.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.336+0000 7f42e6600700 1 -- 172.21.15.107:0/4286810844 --> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f42c0000bf0 con 0x7f42c406d900 2024-03-04T13:47:39.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.339+0000 7f42d7400700 1 -- 172.21.15.107:0/4286810844 <== mgr.34102 v2:172.21.15.107:6800/1223039812 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7f42c0000bf0 con 0x7f42c406d900 2024-03-04T13:47:39.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.342+0000 7f42e6600700 1 -- 172.21.15.107:0/4286810844 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f42c406d900 msgr2=0x7f42c406fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:47:39.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.342+0000 7f42e6600700 1 --2- 172.21.15.107:0/4286810844 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f42c406d900 0x7f42c406fdb0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f42cc007fd0 tx=0x7f42cc009040 comp rx=0 tx=0).stop 2024-03-04T13:47:39.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.342+0000 7f42e6600700 1 -- 172.21.15.107:0/4286810844 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42e00fbdd0 msgr2=0x7f42e0115cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:47:39.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.342+0000 7f42e6600700 1 --2- 172.21.15.107:0/4286810844 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42e00fbdd0 0x7f42e0115cd0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f42d0009360 tx=0x7f42d0009e30 comp rx=0 tx=0).stop 2024-03-04T13:47:39.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.342+0000 7f42e6600700 1 -- 172.21.15.107:0/4286810844 shutdown_connections 2024-03-04T13:47:39.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.342+0000 7f42e6600700 1 --2- 172.21.15.107:0/4286810844 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f42c406d900 0x7f42c406fdb0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:47:39.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.342+0000 7f42e6600700 1 --2- 172.21.15.107:0/4286810844 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42e00fbdd0 0x7f42e0115cd0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:47:39.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.342+0000 7f42e6600700 1 --2- 172.21.15.107:0/4286810844 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f42e00fc6e0 0x7f42e0116210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:47:39.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.342+0000 7f42e6600700 1 -- 172.21.15.107:0/4286810844 >> 172.21.15.107:0/4286810844 conn(0x7f42e0071d90 msgr2=0x7f42e00faa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:47:39.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.343+0000 7f42e6600700 1 -- 172.21.15.107:0/4286810844 shutdown_connections 2024-03-04T13:47:39.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.343+0000 7f42e6600700 1 -- 172.21.15.107:0/4286810844 wait complete. 2024-03-04T13:47:39.358 INFO:teuthology.orchestra.run.smithi107.stdout:true 2024-03-04T13:47:39.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.492+0000 7fd76ae00700 1 -- 172.21.15.107:0/3412557301 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd764105160 msgr2=0x7fd764105530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:47:39.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.492+0000 7fd76ae00700 1 --2- 172.21.15.107:0/3412557301 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd764105160 0x7fd764105530 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd754004940 tx=0x7fd754009e30 comp rx=0 tx=0).stop 2024-03-04T13:47:39.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.493+0000 7fd76ae00700 1 -- 172.21.15.107:0/3412557301 shutdown_connections 2024-03-04T13:47:39.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.493+0000 7fd76ae00700 1 --2- 172.21.15.107:0/3412557301 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd764105b00 0x7fd7640ff170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:47:39.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.493+0000 7fd76ae00700 1 --2- 172.21.15.107:0/3412557301 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd764105160 0x7fd764105530 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:47:39.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.493+0000 7fd76ae00700 1 -- 172.21.15.107:0/3412557301 >> 172.21.15.107:0/3412557301 conn(0x7fd7640fad30 msgr2=0x7fd7640fd140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:47:39.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.493+0000 7fd76ae00700 1 -- 172.21.15.107:0/3412557301 shutdown_connections 2024-03-04T13:47:39.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.493+0000 7fd76ae00700 1 -- 172.21.15.107:0/3412557301 wait complete. 2024-03-04T13:47:39.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.494+0000 7fd76ae00700 1 Processor -- start 2024-03-04T13:47:39.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.494+0000 7fd76ae00700 1 -- start start 2024-03-04T13:47:39.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.494+0000 7fd76ae00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd764105160 0x7fd764115c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:47:39.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.494+0000 7fd76ae00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd764105b00 0x7fd7641161a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:47:39.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.494+0000 7fd76ae00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7641167f0 con 0x7fd764105160 2024-03-04T13:47:39.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.494+0000 7fd76ae00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd76410fb40 con 0x7fd764105b00 2024-03-04T13:47:39.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.494+0000 7fd763e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd764105160 0x7fd764115c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:47:39.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.494+0000 7fd763e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd764105160 0x7fd764115c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37054/0 (socket says 172.21.15.107:37054) 2024-03-04T13:47:39.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.494+0000 7fd763e00700 1 -- 172.21.15.107:0/1330279026 learned_addr learned my addr 172.21.15.107:0/1330279026 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:47:39.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.495+0000 7fd763400700 1 --2- 172.21.15.107:0/1330279026 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd764105b00 0x7fd7641161a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:47:39.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.495+0000 7fd763e00700 1 -- 172.21.15.107:0/1330279026 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd764105b00 msgr2=0x7fd7641161a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:47:39.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.495+0000 7fd763e00700 1 --2- 172.21.15.107:0/1330279026 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd764105b00 0x7fd7641161a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:47:39.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.495+0000 7fd763e00700 1 -- 172.21.15.107:0/1330279026 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd754005040 con 0x7fd764105160 2024-03-04T13:47:39.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.495+0000 7fd763e00700 1 --2- 172.21.15.107:0/1330279026 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd764105160 0x7fd764115c60 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd754009150 tx=0x7fd754009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:47:39.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.495+0000 7fd75be00700 1 -- 172.21.15.107:0/1330279026 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd75400e040 con 0x7fd764105160 2024-03-04T13:47:39.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.496+0000 7fd76ae00700 1 -- 172.21.15.107:0/1330279026 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd76410fdc0 con 0x7fd764105160 2024-03-04T13:47:39.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.496+0000 7fd75be00700 1 -- 172.21.15.107:0/1330279026 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd75400bbf0 con 0x7fd764105160 2024-03-04T13:47:39.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.496+0000 7fd75be00700 1 -- 172.21.15.107:0/1330279026 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd75401c630 con 0x7fd764105160 2024-03-04T13:47:39.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.496+0000 7fd76ae00700 1 -- 172.21.15.107:0/1330279026 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd764110300 con 0x7fd764105160 2024-03-04T13:47:39.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.497+0000 7fd76ae00700 1 -- 172.21.15.107:0/1330279026 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd764102470 con 0x7fd764105160 2024-03-04T13:47:39.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.498+0000 7fd75be00700 1 -- 172.21.15.107:0/1330279026 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 39) v1 ==== 91587+0+0 (secure 0 0 0) 0x7fd75400bd60 con 0x7fd764105160 2024-03-04T13:47:39.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.499+0000 7fd75be00700 1 --2- 172.21.15.107:0/1330279026 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fd74806d900 0x7fd74806fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:47:39.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.499+0000 7fd75be00700 1 -- 172.21.15.107:0/1330279026 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fd75408e4d0 con 0x7fd764105160 2024-03-04T13:47:39.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.500+0000 7fd763400700 1 --2- 172.21.15.107:0/1330279026 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fd74806d900 0x7fd74806fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:47:39.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.500+0000 7fd763400700 1 --2- 172.21.15.107:0/1330279026 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fd74806d900 0x7fd74806fdb0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd750007fd0 tx=0x7fd750009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:47:39.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.506+0000 7fd75be00700 1 -- 172.21.15.107:0/1330279026 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd754057bb0 con 0x7fd764105160 2024-03-04T13:47:39.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.768+0000 7fd76ae00700 1 -- 172.21.15.107:0/1330279026 --> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd76404bf40 con 0x7fd74806d900 2024-03-04T13:47:39.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.770+0000 7fd75be00700 1 -- 172.21.15.107:0/1330279026 <== mgr.34102 v2:172.21.15.107:6800/1223039812 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7fd76404bf40 con 0x7fd74806d900 2024-03-04T13:47:39.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.775+0000 7fd76ae00700 1 -- 172.21.15.107:0/1330279026 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fd74806d900 msgr2=0x7fd74806fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:47:39.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.775+0000 7fd76ae00700 1 --2- 172.21.15.107:0/1330279026 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fd74806d900 0x7fd74806fdb0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd750007fd0 tx=0x7fd750009040 comp rx=0 tx=0).stop 2024-03-04T13:47:39.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.775+0000 7fd76ae00700 1 -- 172.21.15.107:0/1330279026 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd764105160 msgr2=0x7fd764115c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:47:39.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.775+0000 7fd76ae00700 1 --2- 172.21.15.107:0/1330279026 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd764105160 0x7fd764115c60 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd754009150 tx=0x7fd754009e30 comp rx=0 tx=0).stop 2024-03-04T13:47:39.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.775+0000 7fd76ae00700 1 -- 172.21.15.107:0/1330279026 shutdown_connections 2024-03-04T13:47:39.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.775+0000 7fd76ae00700 1 --2- 172.21.15.107:0/1330279026 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fd74806d900 0x7fd74806fdb0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:47:39.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.775+0000 7fd76ae00700 1 --2- 172.21.15.107:0/1330279026 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd764105160 0x7fd764115c60 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:47:39.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.775+0000 7fd76ae00700 1 --2- 172.21.15.107:0/1330279026 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd764105b00 0x7fd7641161a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:47:39.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.775+0000 7fd76ae00700 1 -- 172.21.15.107:0/1330279026 >> 172.21.15.107:0/1330279026 conn(0x7fd7640fad30 msgr2=0x7fd7640fc8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:47:39.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.775+0000 7fd76ae00700 1 -- 172.21.15.107:0/1330279026 shutdown_connections 2024-03-04T13:47:39.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.775+0000 7fd76ae00700 1 -- 172.21.15.107:0/1330279026 wait complete. 2024-03-04T13:47:39.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.927+0000 7fd1a6200700 1 -- 172.21.15.107:0/3799622407 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1a0102f50 msgr2=0x7fd1a0103320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:47:39.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.927+0000 7fd1a6200700 1 --2- 172.21.15.107:0/3799622407 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1a0102f50 0x7fd1a0103320 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fd190004940 tx=0x7fd190009e30 comp rx=0 tx=0).stop 2024-03-04T13:47:39.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.927+0000 7fd1a6200700 1 -- 172.21.15.107:0/3799622407 shutdown_connections 2024-03-04T13:47:39.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.927+0000 7fd1a6200700 1 --2- 172.21.15.107:0/3799622407 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd1a01038f0 0x7fd1a00fcf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:47:39.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.927+0000 7fd1a6200700 1 --2- 172.21.15.107:0/3799622407 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1a0102f50 0x7fd1a0103320 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:47:39.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.927+0000 7fd1a6200700 1 -- 172.21.15.107:0/3799622407 >> 172.21.15.107:0/3799622407 conn(0x7fd1a00f8ae0 msgr2=0x7fd1a00faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:47:39.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.927+0000 7fd1a6200700 1 -- 172.21.15.107:0/3799622407 shutdown_connections 2024-03-04T13:47:39.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.927+0000 7fd1a6200700 1 -- 172.21.15.107:0/3799622407 wait complete. 2024-03-04T13:47:39.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.928+0000 7fd1a6200700 1 Processor -- start 2024-03-04T13:47:39.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.928+0000 7fd1a6200700 1 -- start start 2024-03-04T13:47:39.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.929+0000 7fd1a6200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1a0102f50 0x7fd1a0070290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:47:39.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.929+0000 7fd1a6200700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd1a01038f0 0x7fd1a00707d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:47:39.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.929+0000 7fd1a6200700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a0070dd0 con 0x7fd1a0102f50 2024-03-04T13:47:39.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.929+0000 7fd1a6200700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a0070f40 con 0x7fd1a01038f0 2024-03-04T13:47:39.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.929+0000 7fd19f400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1a0102f50 0x7fd1a0070290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:47:39.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.929+0000 7fd19f400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1a0102f50 0x7fd1a0070290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37076/0 (socket says 172.21.15.107:37076) 2024-03-04T13:47:39.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.929+0000 7fd19f400700 1 -- 172.21.15.107:0/2133393605 learned_addr learned my addr 172.21.15.107:0/2133393605 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:47:39.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.930+0000 7fd19ea00700 1 --2- 172.21.15.107:0/2133393605 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd1a01038f0 0x7fd1a00707d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:47:39.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.930+0000 7fd19f400700 1 -- 172.21.15.107:0/2133393605 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd1a01038f0 msgr2=0x7fd1a00707d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:47:39.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.930+0000 7fd19f400700 1 --2- 172.21.15.107:0/2133393605 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd1a01038f0 0x7fd1a00707d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:47:39.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.930+0000 7fd19f400700 1 -- 172.21.15.107:0/2133393605 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd190005040 con 0x7fd1a0102f50 2024-03-04T13:47:39.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.930+0000 7fd19f400700 1 --2- 172.21.15.107:0/2133393605 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1a0102f50 0x7fd1a0070290 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd190009fd0 tx=0x7fd190009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:47:39.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.930+0000 7fd197400700 1 -- 172.21.15.107:0/2133393605 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd19000e040 con 0x7fd1a0102f50 2024-03-04T13:47:39.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.930+0000 7fd1a6200700 1 -- 172.21.15.107:0/2133393605 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1a0119c90 con 0x7fd1a0102f50 2024-03-04T13:47:39.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.930+0000 7fd197400700 1 -- 172.21.15.107:0/2133393605 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd19000bbf0 con 0x7fd1a0102f50 2024-03-04T13:47:39.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.930+0000 7fd197400700 1 -- 172.21.15.107:0/2133393605 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd19001c3f0 con 0x7fd1a0102f50 2024-03-04T13:47:39.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.930+0000 7fd1a6200700 1 -- 172.21.15.107:0/2133393605 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1a011a0f0 con 0x7fd1a0102f50 2024-03-04T13:47:39.932 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:39 smithi107 ceph-mon[112578]: pgmap v10: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 18 KiB/s rd, 0 B/s wr, 11 op/s 2024-03-04T13:47:39.932 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:39 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:47:39.932 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:39 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:47:39.932 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:39 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:47:39.932 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:39 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:47:39.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.932+0000 7fd1a6200700 1 -- 172.21.15.107:0/2133393605 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1a00fc6e0 con 0x7fd1a0102f50 2024-03-04T13:47:39.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.934+0000 7fd197400700 1 -- 172.21.15.107:0/2133393605 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 39) v1 ==== 91587+0+0 (secure 0 0 0) 0x7fd19000bd60 con 0x7fd1a0102f50 2024-03-04T13:47:39.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.935+0000 7fd197400700 1 --2- 172.21.15.107:0/2133393605 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fd18406d900 0x7fd18406fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:47:39.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.936+0000 7fd197400700 1 -- 172.21.15.107:0/2133393605 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fd19008e2e0 con 0x7fd1a0102f50 2024-03-04T13:47:39.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.936+0000 7fd19ea00700 1 --2- 172.21.15.107:0/2133393605 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fd18406d900 0x7fd18406fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:47:39.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.937+0000 7fd19ea00700 1 --2- 172.21.15.107:0/2133393605 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fd18406d900 0x7fd18406fdb0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd1a0071c90 tx=0x7fd18c009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:47:39.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:39.941+0000 7fd197400700 1 -- 172.21.15.107:0/2133393605 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd190057940 con 0x7fd1a0102f50 2024-03-04T13:47:40.109 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:39 smithi169 ceph-mon[36313]: pgmap v10: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 18 KiB/s rd, 0 B/s wr, 11 op/s 2024-03-04T13:47:40.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:39 smithi169 ceph-mon[36313]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:47:40.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:39 smithi169 ceph-mon[36313]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:47:40.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:39 smithi169 ceph-mon[36313]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:47:40.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:39 smithi169 ceph-mon[36313]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:47:40.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:40.167+0000 7fd1a6200700 1 -- 172.21.15.107:0/2133393605 --> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fd1a0071ae0 con 0x7fd18406d900 2024-03-04T13:47:40.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:40.179+0000 7fd197400700 1 -- 172.21.15.107:0/2133393605 <== mgr.34102 v2:172.21.15.107:6800/1223039812 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7fd1a0071ae0 con 0x7fd18406d900 2024-03-04T13:47:40.179 INFO:teuthology.orchestra.run.smithi107.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-04T13:47:40.179 INFO:teuthology.orchestra.run.smithi107.stdout:alertmanager.smithi107 smithi107 *:9093,9094 running (118s) 8s ago 16m 14.6M - 0.25.0 c8568f914cd2 e5ed7be32538 2024-03-04T13:47:40.179 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi107 smithi107 running (16m) 8s ago 16m 6665k - 17.2.7-545-g42608bb1 9eec3f22859d 03e1570c2eff 2024-03-04T13:47:40.179 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi169 smithi169 running (14m) 10s ago 14m 6656k - 17.2.7-545-g42608bb1 9eec3f22859d 9c8881a5338e 2024-03-04T13:47:40.179 INFO:teuthology.orchestra.run.smithi107.stdout:grafana.smithi107 smithi107 *:3000 running (94s) 8s ago 15m 74.2M - 9.4.12 a0ea5e3c46d9 c5e4d33fb599 2024-03-04T13:47:40.179 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.riexmm smithi107 running (9m) 8s ago 9m 14.1M - 17.2.7-545-g42608bb1 9eec3f22859d fc921f7d4328 2024-03-04T13:47:40.179 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.thycho smithi107 running (8m) 8s ago 8m 98.2M - 17.2.7-545-g42608bb1 9eec3f22859d 4069c6e0e063 2024-03-04T13:47:40.179 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi169.bxitzk smithi169 running (8m) 10s ago 8m 13.8M - 17.2.7-545-g42608bb1 9eec3f22859d 7d4115228c7c 2024-03-04T13:47:40.179 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi169.jusudl smithi169 running (9m) 10s ago 9m 146M - 17.2.7-545-g42608bb1 9eec3f22859d 2d2f2cc452e8 2024-03-04T13:47:40.179 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi107.wpzgwl smithi107 *:8443,9283,8765 running (4m) 8s ago 17m 476M - 19.0.0-1845-g717ce3cc d71b839abdf8 1c5fdee45eac 2024-03-04T13:47:40.180 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi169.eihczr smithi169 *:8443,9283,8765 running (3m) 10s ago 14m 446M - 19.0.0-1845-g717ce3cc d71b839abdf8 b5cac2b7d22a 2024-03-04T13:47:40.180 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi107 smithi107 running (26s) 8s ago 17m 28.3M 2048M 19.0.0-1845-g717ce3cc d71b839abdf8 32f07144274f 2024-03-04T13:47:40.180 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi169 smithi169 running (14m) 10s ago 14m 46.7M 2048M 17.2.7-545-g42608bb1 9eec3f22859d ba4ee916297f 2024-03-04T13:47:40.180 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi107 smithi107 *:9100 running (3m) 8s ago 16m 7650k - 1.5.0 0da6a335fe13 8a8d3d4f724f 2024-03-04T13:47:40.180 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi169 smithi169 *:9100 running (2m) 10s ago 14m 7662k - 1.5.0 0da6a335fe13 f240f5a975bc 2024-03-04T13:47:40.180 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0 smithi107 running (13m) 8s ago 13m 206M 1558M 17.2.7-545-g42608bb1 9eec3f22859d c0605b68f696 2024-03-04T13:47:40.180 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1 smithi107 running (13m) 8s ago 13m 220M 1558M 17.2.7-545-g42608bb1 9eec3f22859d 5fb2c0dd5769 2024-03-04T13:47:40.180 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2 smithi107 running (12m) 8s ago 12m 182M 1558M 17.2.7-545-g42608bb1 9eec3f22859d 61f30f643549 2024-03-04T13:47:40.180 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3 smithi169 running (11m) 10s ago 11m 235M 2582M 17.2.7-545-g42608bb1 9eec3f22859d cd91d8694764 2024-03-04T13:47:40.180 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4 smithi169 running (11m) 10s ago 11m 207M 2582M 17.2.7-545-g42608bb1 9eec3f22859d 08545e3d0659 2024-03-04T13:47:40.180 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5 smithi169 running (10m) 10s ago 10m 194M 2582M 17.2.7-545-g42608bb1 9eec3f22859d dcffe6cb3eac 2024-03-04T13:47:40.180 INFO:teuthology.orchestra.run.smithi107.stdout:prometheus.smithi107 smithi107 *:9095 running (2m) 8s ago 15m 50.2M - 2.43.0 a07b618ecd1d 1eeb1ad5cb75 2024-03-04T13:47:40.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:40.181+0000 7fd1a6200700 1 -- 172.21.15.107:0/2133393605 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fd18406d900 msgr2=0x7fd18406fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:47:40.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:40.181+0000 7fd1a6200700 1 --2- 172.21.15.107:0/2133393605 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fd18406d900 0x7fd18406fdb0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd1a0071c90 tx=0x7fd18c009040 comp rx=0 tx=0).stop 2024-03-04T13:47:40.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:40.181+0000 7fd1a6200700 1 -- 172.21.15.107:0/2133393605 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1a0102f50 msgr2=0x7fd1a0070290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:47:40.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:40.181+0000 7fd1a6200700 1 --2- 172.21.15.107:0/2133393605 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1a0102f50 0x7fd1a0070290 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd190009fd0 tx=0x7fd190009e30 comp rx=0 tx=0).stop 2024-03-04T13:47:40.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:40.181+0000 7fd1a6200700 1 -- 172.21.15.107:0/2133393605 shutdown_connections 2024-03-04T13:47:40.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:40.181+0000 7fd1a6200700 1 --2- 172.21.15.107:0/2133393605 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fd18406d900 0x7fd18406fdb0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:47:40.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:40.181+0000 7fd1a6200700 1 --2- 172.21.15.107:0/2133393605 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1a0102f50 0x7fd1a0070290 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:47:40.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:40.181+0000 7fd1a6200700 1 --2- 172.21.15.107:0/2133393605 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd1a01038f0 0x7fd1a00707d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:47:40.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:40.181+0000 7fd1a6200700 1 -- 172.21.15.107:0/2133393605 >> 172.21.15.107:0/2133393605 conn(0x7fd1a00f8ae0 msgr2=0x7fd1a00fa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:47:40.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:40.182+0000 7fd1a6200700 1 -- 172.21.15.107:0/2133393605 shutdown_connections 2024-03-04T13:47:40.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:40.182+0000 7fd1a6200700 1 -- 172.21.15.107:0/2133393605 wait complete. 2024-03-04T13:47:40.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:40.322+0000 7fcfe9800700 1 -- 172.21.15.107:0/1314796085 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcfe4102f50 msgr2=0x7fcfe4103320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:47:40.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:40.322+0000 7fcfe9800700 1 --2- 172.21.15.107:0/1314796085 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcfe4102f50 0x7fcfe4103320 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fcfd4004940 tx=0x7fcfd4009e30 comp rx=0 tx=0).stop 2024-03-04T13:47:40.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:40.322+0000 7fcfe9800700 1 -- 172.21.15.107:0/1314796085 shutdown_connections 2024-03-04T13:47:40.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:40.322+0000 7fcfe9800700 1 --2- 172.21.15.107:0/1314796085 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcfe41038f0 0x7fcfe40fcf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:47:40.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:40.322+0000 7fcfe9800700 1 --2- 172.21.15.107:0/1314796085 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcfe4102f50 0x7fcfe4103320 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:47:40.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:40.322+0000 7fcfe9800700 1 -- 172.21.15.107:0/1314796085 >> 172.21.15.107:0/1314796085 conn(0x7fcfe40f8ae0 msgr2=0x7fcfe40faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:47:40.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:40.322+0000 7fcfe9800700 1 -- 172.21.15.107:0/1314796085 shutdown_connections 2024-03-04T13:47:40.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:40.322+0000 7fcfe9800700 1 -- 172.21.15.107:0/1314796085 wait complete. 2024-03-04T13:47:40.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:40.323+0000 7fcfe9800700 1 Processor -- start 2024-03-04T13:47:40.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:40.323+0000 7fcfe9800700 1 -- start start 2024-03-04T13:47:40.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:40.324+0000 7fcfe9800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcfe4102f50 0x7fcfe41139f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:47:40.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:40.324+0000 7fcfe9800700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcfe41038f0 0x7fcfe4113f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:47:40.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:40.324+0000 7fcfe9800700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe4114580 con 0x7fcfe4102f50 2024-03-04T13:47:40.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:40.324+0000 7fcfe9800700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe410d8b0 con 0x7fcfe41038f0 2024-03-04T13:47:40.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:40.324+0000 7fcfe2a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcfe4102f50 0x7fcfe41139f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:47:40.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:40.324+0000 7fcfe2a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcfe4102f50 0x7fcfe41139f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37090/0 (socket says 172.21.15.107:37090) 2024-03-04T13:47:40.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:40.324+0000 7fcfe2a00700 1 -- 172.21.15.107:0/4114541423 learned_addr learned my addr 172.21.15.107:0/4114541423 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:47:40.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:40.324+0000 7fcfe2000700 1 --2- 172.21.15.107:0/4114541423 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcfe41038f0 0x7fcfe4113f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:47:40.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:40.325+0000 7fcfe2a00700 1 -- 172.21.15.107:0/4114541423 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcfe41038f0 msgr2=0x7fcfe4113f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:47:40.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:40.325+0000 7fcfe2a00700 1 --2- 172.21.15.107:0/4114541423 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcfe41038f0 0x7fcfe4113f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:47:40.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:40.325+0000 7fcfe2a00700 1 -- 172.21.15.107:0/4114541423 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfd4005040 con 0x7fcfe4102f50 2024-03-04T13:47:40.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:40.325+0000 7fcfe2a00700 1 --2- 172.21.15.107:0/4114541423 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcfe4102f50 0x7fcfe41139f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fcfd4000c00 tx=0x7fcfd4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:47:40.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:40.325+0000 7fcfdaa00700 1 -- 172.21.15.107:0/4114541423 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcfd4021070 con 0x7fcfe4102f50 2024-03-04T13:47:40.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:40.325+0000 7fcfdaa00700 1 -- 172.21.15.107:0/4114541423 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcfd4013d00 con 0x7fcfe4102f50 2024-03-04T13:47:40.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:40.325+0000 7fcfe9800700 1 -- 172.21.15.107:0/4114541423 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfe410db30 con 0x7fcfe4102f50 2024-03-04T13:47:40.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:40.325+0000 7fcfdaa00700 1 -- 172.21.15.107:0/4114541423 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcfd400e070 con 0x7fcfe4102f50 2024-03-04T13:47:40.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:40.326+0000 7fcfe9800700 1 -- 172.21.15.107:0/4114541423 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfe410e010 con 0x7fcfe4102f50 2024-03-04T13:47:40.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:40.327+0000 7fcfe9800700 1 -- 172.21.15.107:0/4114541423 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfe40fc6e0 con 0x7fcfe4102f50 2024-03-04T13:47:40.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:40.328+0000 7fcfdaa00700 1 -- 172.21.15.107:0/4114541423 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 39) v1 ==== 91587+0+0 (secure 0 0 0) 0x7fcfd401caf0 con 0x7fcfe4102f50 2024-03-04T13:47:40.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:40.329+0000 7fcfdaa00700 1 --2- 172.21.15.107:0/4114541423 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fcfc806d900 0x7fcfc806fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:47:40.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:40.329+0000 7fcfdaa00700 1 -- 172.21.15.107:0/4114541423 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fcfd408e5a0 con 0x7fcfe4102f50 2024-03-04T13:47:40.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:40.329+0000 7fcfe2000700 1 --2- 172.21.15.107:0/4114541423 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fcfc806d900 0x7fcfc806fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:47:40.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:40.330+0000 7fcfe2000700 1 --2- 172.21.15.107:0/4114541423 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fcfc806d900 0x7fcfc806fdb0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fcfe410f220 tx=0x7fcfd000b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:47:40.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:40.334+0000 7fcfdaa00700 1 -- 172.21.15.107:0/4114541423 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fcfd405b7b0 con 0x7fcfe4102f50 2024-03-04T13:47:40.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:40.642+0000 7fcfe9800700 1 -- 172.21.15.107:0/4114541423 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fcfe410a050 con 0x7fcfe4102f50 2024-03-04T13:47:40.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:40.643+0000 7fcfdaa00700 1 -- 172.21.15.107:0/4114541423 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+843 (secure 0 0 0) 0x7fcfd405b340 con 0x7fcfe4102f50 2024-03-04T13:47:40.643 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-03-04T13:47:40.644 INFO:teuthology.orchestra.run.smithi107.stdout: "mon": { 2024-03-04T13:47:40.644 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-545-g42608bb1 (42608bb191117be10dba720e6bf8a02f3171cd34) quincy (stable)": 1, 2024-03-04T13:47:40.644 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-1845-g717ce3cc (717ce3cce8d0166e4e577faa8004e6f9cb4128c0) squid (dev)": 1 2024-03-04T13:47:40.644 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:47:40.644 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr": { 2024-03-04T13:47:40.644 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-1845-g717ce3cc (717ce3cce8d0166e4e577faa8004e6f9cb4128c0) squid (dev)": 2 2024-03-04T13:47:40.644 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:47:40.644 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": { 2024-03-04T13:47:40.644 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-545-g42608bb1 (42608bb191117be10dba720e6bf8a02f3171cd34) quincy (stable)": 6 2024-03-04T13:47:40.644 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:47:40.644 INFO:teuthology.orchestra.run.smithi107.stdout: "mds": { 2024-03-04T13:47:40.644 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-545-g42608bb1 (42608bb191117be10dba720e6bf8a02f3171cd34) quincy (stable)": 4 2024-03-04T13:47:40.645 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:47:40.645 INFO:teuthology.orchestra.run.smithi107.stdout: "overall": { 2024-03-04T13:47:40.645 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-545-g42608bb1 (42608bb191117be10dba720e6bf8a02f3171cd34) quincy (stable)": 11, 2024-03-04T13:47:40.645 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-1845-g717ce3cc (717ce3cce8d0166e4e577faa8004e6f9cb4128c0) squid (dev)": 3 2024-03-04T13:47:40.645 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-04T13:47:40.645 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-03-04T13:47:40.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:40.646+0000 7fcfe9800700 1 -- 172.21.15.107:0/4114541423 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fcfc806d900 msgr2=0x7fcfc806fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:47:40.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:40.646+0000 7fcfe9800700 1 --2- 172.21.15.107:0/4114541423 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fcfc806d900 0x7fcfc806fdb0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fcfe410f220 tx=0x7fcfd000b040 comp rx=0 tx=0).stop 2024-03-04T13:47:40.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:40.646+0000 7fcfe9800700 1 -- 172.21.15.107:0/4114541423 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcfe4102f50 msgr2=0x7fcfe41139f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:47:40.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:40.646+0000 7fcfe9800700 1 --2- 172.21.15.107:0/4114541423 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcfe4102f50 0x7fcfe41139f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fcfd4000c00 tx=0x7fcfd4009e30 comp rx=0 tx=0).stop 2024-03-04T13:47:40.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:40.646+0000 7fcfe9800700 1 -- 172.21.15.107:0/4114541423 shutdown_connections 2024-03-04T13:47:40.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:40.646+0000 7fcfe9800700 1 --2- 172.21.15.107:0/4114541423 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fcfc806d900 0x7fcfc806fdb0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:47:40.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:40.646+0000 7fcfe9800700 1 --2- 172.21.15.107:0/4114541423 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcfe4102f50 0x7fcfe41139f0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:47:40.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:40.646+0000 7fcfe9800700 1 --2- 172.21.15.107:0/4114541423 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcfe41038f0 0x7fcfe4113f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:47:40.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:40.646+0000 7fcfe9800700 1 -- 172.21.15.107:0/4114541423 >> 172.21.15.107:0/4114541423 conn(0x7fcfe40f8ae0 msgr2=0x7fcfe40fa400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:47:40.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:40.646+0000 7fcfe9800700 1 -- 172.21.15.107:0/4114541423 shutdown_connections 2024-03-04T13:47:40.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:40.646+0000 7fcfe9800700 1 -- 172.21.15.107:0/4114541423 wait complete. 2024-03-04T13:47:40.784 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:40.784+0000 7f8af1a00700 1 -- 172.21.15.107:0/2938618873 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8aec1036e0 msgr2=0x7f8aec0fcd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:47:40.784 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:40.784+0000 7f8af1a00700 1 --2- 172.21.15.107:0/2938618873 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8aec1036e0 0x7f8aec0fcd50 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f8ad8004940 tx=0x7f8ad8009e30 comp rx=0 tx=0).stop 2024-03-04T13:47:40.784 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:40.784+0000 7f8af1a00700 1 -- 172.21.15.107:0/2938618873 shutdown_connections 2024-03-04T13:47:40.784 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:40.784+0000 7f8af1a00700 1 --2- 172.21.15.107:0/2938618873 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8aec1036e0 0x7f8aec0fcd50 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:47:40.784 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:40.784+0000 7f8af1a00700 1 --2- 172.21.15.107:0/2938618873 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8aec102d40 0x7f8aec103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:47:40.784 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:40.784+0000 7f8af1a00700 1 -- 172.21.15.107:0/2938618873 >> 172.21.15.107:0/2938618873 conn(0x7f8aec0f88b0 msgr2=0x7f8aec0facc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:47:40.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:40.784+0000 7f8af1a00700 1 -- 172.21.15.107:0/2938618873 shutdown_connections 2024-03-04T13:47:40.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:40.784+0000 7f8af1a00700 1 -- 172.21.15.107:0/2938618873 wait complete. 2024-03-04T13:47:40.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:40.785+0000 7f8af1a00700 1 Processor -- start 2024-03-04T13:47:40.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:40.785+0000 7f8af1a00700 1 -- start start 2024-03-04T13:47:40.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:40.785+0000 7f8af1a00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8aec102d40 0x7f8aec115a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:47:40.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:40.785+0000 7f8af1a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8aec1036e0 0x7f8aec115fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:47:40.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:40.785+0000 7f8af1a00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aec116600 con 0x7f8aec1036e0 2024-03-04T13:47:40.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:40.785+0000 7f8af1a00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aec10f930 con 0x7f8aec102d40 2024-03-04T13:47:40.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:40.786+0000 7f8aeb400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8aec1036e0 0x7f8aec115fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:47:40.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:40.786+0000 7f8aeb400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8aec1036e0 0x7f8aec115fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37118/0 (socket says 172.21.15.107:37118) 2024-03-04T13:47:40.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:40.786+0000 7f8aeb400700 1 -- 172.21.15.107:0/2163569592 learned_addr learned my addr 172.21.15.107:0/2163569592 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:47:40.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:40.786+0000 7f8aebe00700 1 --2- 172.21.15.107:0/2163569592 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8aec102d40 0x7f8aec115a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:47:40.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:40.786+0000 7f8aeb400700 1 -- 172.21.15.107:0/2163569592 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8aec102d40 msgr2=0x7f8aec115a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:47:40.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:40.786+0000 7f8aeb400700 1 --2- 172.21.15.107:0/2163569592 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8aec102d40 0x7f8aec115a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:47:40.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:40.786+0000 7f8aeb400700 1 -- 172.21.15.107:0/2163569592 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ad8005040 con 0x7f8aec1036e0 2024-03-04T13:47:40.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:40.786+0000 7f8aeb400700 1 --2- 172.21.15.107:0/2163569592 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8aec1036e0 0x7f8aec115fb0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f8ad80090d0 tx=0x7f8ad8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:47:40.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:40.787+0000 7f8ae3e00700 1 -- 172.21.15.107:0/2163569592 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8ad800e040 con 0x7f8aec1036e0 2024-03-04T13:47:40.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:40.787+0000 7f8ae3e00700 1 -- 172.21.15.107:0/2163569592 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8ad8009430 con 0x7f8aec1036e0 2024-03-04T13:47:40.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:40.787+0000 7f8ae3e00700 1 -- 172.21.15.107:0/2163569592 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8ad801c3f0 con 0x7f8aec1036e0 2024-03-04T13:47:40.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:40.787+0000 7f8af1a00700 1 -- 172.21.15.107:0/2163569592 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8aec10fbb0 con 0x7f8aec1036e0 2024-03-04T13:47:40.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:40.787+0000 7f8af1a00700 1 -- 172.21.15.107:0/2163569592 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8aec110170 con 0x7f8aec1036e0 2024-03-04T13:47:40.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:40.788+0000 7f8af1a00700 1 -- 172.21.15.107:0/2163569592 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8aec05f5c0 con 0x7f8aec1036e0 2024-03-04T13:47:40.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:40.789+0000 7f8ae3e00700 1 -- 172.21.15.107:0/2163569592 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 39) v1 ==== 91587+0+0 (secure 0 0 0) 0x7f8ad800bbf0 con 0x7f8aec1036e0 2024-03-04T13:47:40.790 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:40.790+0000 7f8ae3e00700 1 --2- 172.21.15.107:0/2163569592 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f8ad006d900 0x7f8ad006fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:47:40.790 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:40.790+0000 7f8ae3e00700 1 -- 172.21.15.107:0/2163569592 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f8ad808e370 con 0x7f8aec1036e0 2024-03-04T13:47:40.790 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:40.790+0000 7f8aebe00700 1 --2- 172.21.15.107:0/2163569592 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f8ad006d900 0x7f8ad006fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:47:40.791 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:40.791+0000 7f8aebe00700 1 --2- 172.21.15.107:0/2163569592 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f8ad006d900 0x7f8ad006fdb0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8adc004eb0 tx=0x7f8adc00b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:47:40.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:40.796+0000 7f8ae3e00700 1 -- 172.21.15.107:0/2163569592 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8ad805b580 con 0x7f8aec1036e0 2024-03-04T13:47:41.092 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:40 smithi107 ceph-mon[112578]: from='client.34130 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:47:41.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:40 smithi107 ceph-mon[112578]: Detected new or changed devices on smithi107 2024-03-04T13:47:41.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:40 smithi107 ceph-mon[112578]: Updating smithi107:/etc/ceph/ceph.conf 2024-03-04T13:47:41.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:40 smithi107 ceph-mon[112578]: Updating smithi169:/etc/ceph/ceph.conf 2024-03-04T13:47:41.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:40 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:47:41.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:40 smithi107 ceph-mon[112578]: from='client.34134 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:47:41.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:40 smithi107 ceph-mon[112578]: from='client.? 172.21.15.107:0/4114541423' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:47:41.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.097+0000 7f8af1a00700 1 -- 172.21.15.107:0/2163569592 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f8aec109e50 con 0x7f8aec1036e0 2024-03-04T13:47:41.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.098+0000 7f8ae3e00700 1 -- 172.21.15.107:0/2163569592 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 16 v16) v1 ==== 76+0+1880 (secure 0 0 0) 0x7f8ad805b110 con 0x7f8aec1036e0 2024-03-04T13:47:41.100 INFO:teuthology.orchestra.run.smithi107.stdout:e16 2024-03-04T13:47:41.100 INFO:teuthology.orchestra.run.smithi107.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-04T13:47:41.100 INFO:teuthology.orchestra.run.smithi107.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-04T13:47:41.100 INFO:teuthology.orchestra.run.smithi107.stdout:legacy client fscid: 1 2024-03-04T13:47:41.100 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:47:41.100 INFO:teuthology.orchestra.run.smithi107.stdout:Filesystem 'cephfs' (1) 2024-03-04T13:47:41.100 INFO:teuthology.orchestra.run.smithi107.stdout:fs_name cephfs 2024-03-04T13:47:41.100 INFO:teuthology.orchestra.run.smithi107.stdout:epoch 15 2024-03-04T13:47:41.100 INFO:teuthology.orchestra.run.smithi107.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-04T13:47:41.101 INFO:teuthology.orchestra.run.smithi107.stdout:created 2024-03-04T13:38:30.854004+0000 2024-03-04T13:47:41.101 INFO:teuthology.orchestra.run.smithi107.stdout:modified 2024-03-04T13:38:50.015989+0000 2024-03-04T13:47:41.101 INFO:teuthology.orchestra.run.smithi107.stdout:tableserver 0 2024-03-04T13:47:41.101 INFO:teuthology.orchestra.run.smithi107.stdout:root 0 2024-03-04T13:47:41.101 INFO:teuthology.orchestra.run.smithi107.stdout:session_timeout 60 2024-03-04T13:47:41.101 INFO:teuthology.orchestra.run.smithi107.stdout:session_autoclose 300 2024-03-04T13:47:41.101 INFO:teuthology.orchestra.run.smithi107.stdout:max_file_size 1099511627776 2024-03-04T13:47:41.101 INFO:teuthology.orchestra.run.smithi107.stdout:max_xattr_size 65536 2024-03-04T13:47:41.101 INFO:teuthology.orchestra.run.smithi107.stdout:required_client_features {} 2024-03-04T13:47:41.101 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure 0 2024-03-04T13:47:41.101 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure_osd_epoch 47 2024-03-04T13:47:41.101 INFO:teuthology.orchestra.run.smithi107.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-04T13:47:41.101 INFO:teuthology.orchestra.run.smithi107.stdout:max_mds 1 2024-03-04T13:47:41.101 INFO:teuthology.orchestra.run.smithi107.stdout:in 0 2024-03-04T13:47:41.102 INFO:teuthology.orchestra.run.smithi107.stdout:up {0=24247} 2024-03-04T13:47:41.102 INFO:teuthology.orchestra.run.smithi107.stdout:failed 2024-03-04T13:47:41.102 INFO:teuthology.orchestra.run.smithi107.stdout:damaged 2024-03-04T13:47:41.102 INFO:teuthology.orchestra.run.smithi107.stdout:stopped 2024-03-04T13:47:41.102 INFO:teuthology.orchestra.run.smithi107.stdout:data_pools [3] 2024-03-04T13:47:41.102 INFO:teuthology.orchestra.run.smithi107.stdout:metadata_pool 2 2024-03-04T13:47:41.102 INFO:teuthology.orchestra.run.smithi107.stdout:inline_data enabled 2024-03-04T13:47:41.102 INFO:teuthology.orchestra.run.smithi107.stdout:balancer 2024-03-04T13:47:41.102 INFO:teuthology.orchestra.run.smithi107.stdout:bal_rank_mask -1 2024-03-04T13:47:41.102 INFO:teuthology.orchestra.run.smithi107.stdout:standby_count_wanted 1 2024-03-04T13:47:41.102 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi169.jusudl{0:24247} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.169:6824/2994866076,v1:172.21.15.169:6825/2994866076] compat {c=[1],r=[1],i=[7ff]}] 2024-03-04T13:47:41.102 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.thycho{0:14470} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.107:6828/1985577060,v1:172.21.15.107:6829/1985577060] compat {c=[1],r=[1],i=[7ff]}] 2024-03-04T13:47:41.102 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:47:41.102 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:47:41.102 INFO:teuthology.orchestra.run.smithi107.stdout:Standby daemons: 2024-03-04T13:47:41.103 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:47:41.103 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.riexmm{-1:14480} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.107:6826/3261622517,v1:172.21.15.107:6827/3261622517] compat {c=[1],r=[1],i=[7ff]}] 2024-03-04T13:47:41.103 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi169.bxitzk{-1:24259} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.169:6826/3696090493,v1:172.21.15.169:6827/3696090493] compat {c=[1],r=[1],i=[7ff]}] 2024-03-04T13:47:41.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.102+0000 7f8af1a00700 1 -- 172.21.15.107:0/2163569592 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f8ad006d900 msgr2=0x7f8ad006fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:47:41.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.102+0000 7f8af1a00700 1 --2- 172.21.15.107:0/2163569592 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f8ad006d900 0x7f8ad006fdb0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8adc004eb0 tx=0x7f8adc00b040 comp rx=0 tx=0).stop 2024-03-04T13:47:41.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.102+0000 7f8af1a00700 1 -- 172.21.15.107:0/2163569592 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8aec1036e0 msgr2=0x7f8aec115fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:47:41.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.102+0000 7f8af1a00700 1 --2- 172.21.15.107:0/2163569592 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8aec1036e0 0x7f8aec115fb0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f8ad80090d0 tx=0x7f8ad8009e30 comp rx=0 tx=0).stop 2024-03-04T13:47:41.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.102+0000 7f8af1a00700 1 -- 172.21.15.107:0/2163569592 shutdown_connections 2024-03-04T13:47:41.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.102+0000 7f8af1a00700 1 --2- 172.21.15.107:0/2163569592 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f8ad006d900 0x7f8ad006fdb0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:47:41.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.102+0000 7f8af1a00700 1 --2- 172.21.15.107:0/2163569592 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8aec102d40 0x7f8aec115a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:47:41.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.102+0000 7f8af1a00700 1 --2- 172.21.15.107:0/2163569592 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8aec1036e0 0x7f8aec115fb0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:47:41.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.102+0000 7f8af1a00700 1 -- 172.21.15.107:0/2163569592 >> 172.21.15.107:0/2163569592 conn(0x7f8aec0f88b0 msgr2=0x7f8aec0fa1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:47:41.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.102+0000 7f8af1a00700 1 -- 172.21.15.107:0/2163569592 shutdown_connections 2024-03-04T13:47:41.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.102+0000 7f8af1a00700 1 -- 172.21.15.107:0/2163569592 wait complete. 2024-03-04T13:47:41.104 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 16 2024-03-04T13:47:41.109 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:40 smithi169 ceph-mon[36313]: from='client.34130 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:47:41.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:40 smithi169 ceph-mon[36313]: Detected new or changed devices on smithi107 2024-03-04T13:47:41.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:40 smithi169 ceph-mon[36313]: Updating smithi107:/etc/ceph/ceph.conf 2024-03-04T13:47:41.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:40 smithi169 ceph-mon[36313]: Updating smithi169:/etc/ceph/ceph.conf 2024-03-04T13:47:41.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:40 smithi169 ceph-mon[36313]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:47:41.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:40 smithi169 ceph-mon[36313]: from='client.34134 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:47:41.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:40 smithi169 ceph-mon[36313]: from='client.? 172.21.15.107:0/4114541423' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:47:41.248 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.248+0000 7f7c3aa00700 1 -- 172.21.15.107:0/1740692041 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c340fe8f0 msgr2=0x7f7c34107050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:47:41.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.248+0000 7f7c3aa00700 1 --2- 172.21.15.107:0/1740692041 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c340fe8f0 0x7f7c34107050 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c004970 tx=0x7f7c1c009e30 comp rx=0 tx=0).stop 2024-03-04T13:47:41.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.248+0000 7f7c3aa00700 1 -- 172.21.15.107:0/1740692041 shutdown_connections 2024-03-04T13:47:41.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.248+0000 7f7c3aa00700 1 --2- 172.21.15.107:0/1740692041 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c340fe8f0 0x7f7c34107050 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:47:41.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.248+0000 7f7c3aa00700 1 --2- 172.21.15.107:0/1740692041 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c340fdfe0 0x7f7c340fe3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:47:41.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.248+0000 7f7c3aa00700 1 -- 172.21.15.107:0/1740692041 >> 172.21.15.107:0/1740692041 conn(0x7f7c34071d90 msgr2=0x7f7c34072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:47:41.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.248+0000 7f7c3aa00700 1 -- 172.21.15.107:0/1740692041 shutdown_connections 2024-03-04T13:47:41.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.248+0000 7f7c3aa00700 1 -- 172.21.15.107:0/1740692041 wait complete. 2024-03-04T13:47:41.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.249+0000 7f7c3aa00700 1 Processor -- start 2024-03-04T13:47:41.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.249+0000 7f7c3aa00700 1 -- start start 2024-03-04T13:47:41.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.250+0000 7f7c3aa00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c340fdfe0 0x7f7c341139f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:47:41.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.250+0000 7f7c3aa00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c340fe8f0 0x7f7c34113f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:47:41.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.250+0000 7f7c3aa00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c34114580 con 0x7f7c340fe8f0 2024-03-04T13:47:41.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.250+0000 7f7c3aa00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c3410d8b0 con 0x7f7c340fdfe0 2024-03-04T13:47:41.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.250+0000 7f7c2be00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c340fe8f0 0x7f7c34113f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:47:41.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.250+0000 7f7c2be00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c340fe8f0 0x7f7c34113f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37142/0 (socket says 172.21.15.107:37142) 2024-03-04T13:47:41.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.250+0000 7f7c2be00700 1 -- 172.21.15.107:0/2513579029 learned_addr learned my addr 172.21.15.107:0/2513579029 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:47:41.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.250+0000 7f7c33e00700 1 --2- 172.21.15.107:0/2513579029 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c340fdfe0 0x7f7c341139f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:47:41.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.251+0000 7f7c2be00700 1 -- 172.21.15.107:0/2513579029 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c340fdfe0 msgr2=0x7f7c341139f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:47:41.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.251+0000 7f7c2be00700 1 --2- 172.21.15.107:0/2513579029 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c340fdfe0 0x7f7c341139f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:47:41.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.251+0000 7f7c2be00700 1 -- 172.21.15.107:0/2513579029 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c1c005040 con 0x7f7c340fe8f0 2024-03-04T13:47:41.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.251+0000 7f7c33e00700 1 --2- 172.21.15.107:0/2513579029 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c340fdfe0 0x7f7c341139f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T13:47:41.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.251+0000 7f7c2be00700 1 --2- 172.21.15.107:0/2513579029 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c340fe8f0 0x7f7c34113f30 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c005010 tx=0x7f7c1c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:47:41.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.251+0000 7f7c30c00700 1 -- 172.21.15.107:0/2513579029 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c1c00e040 con 0x7f7c340fe8f0 2024-03-04T13:47:41.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.251+0000 7f7c30c00700 1 -- 172.21.15.107:0/2513579029 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7c1c004290 con 0x7f7c340fe8f0 2024-03-04T13:47:41.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.251+0000 7f7c3aa00700 1 -- 172.21.15.107:0/2513579029 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c3410db30 con 0x7f7c340fe8f0 2024-03-04T13:47:41.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.251+0000 7f7c30c00700 1 -- 172.21.15.107:0/2513579029 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c1c01c3f0 con 0x7f7c340fe8f0 2024-03-04T13:47:41.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.251+0000 7f7c3aa00700 1 -- 172.21.15.107:0/2513579029 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c3410e070 con 0x7f7c340fe8f0 2024-03-04T13:47:41.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.252+0000 7f7c3aa00700 1 -- 172.21.15.107:0/2513579029 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c341046c0 con 0x7f7c340fe8f0 2024-03-04T13:47:41.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.254+0000 7f7c30c00700 1 -- 172.21.15.107:0/2513579029 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 39) v1 ==== 91587+0+0 (secure 0 0 0) 0x7f7c1c013820 con 0x7f7c340fe8f0 2024-03-04T13:47:41.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.255+0000 7f7c30c00700 1 --2- 172.21.15.107:0/2513579029 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f7c1806d900 0x7f7c1806fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:47:41.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.255+0000 7f7c30c00700 1 -- 172.21.15.107:0/2513579029 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f7c1c08e390 con 0x7f7c340fe8f0 2024-03-04T13:47:41.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.255+0000 7f7c33e00700 1 --2- 172.21.15.107:0/2513579029 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f7c1806d900 0x7f7c1806fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:47:41.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.256+0000 7f7c33e00700 1 --2- 172.21.15.107:0/2513579029 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f7c1806d900 0x7f7c1806fdb0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f7c24009fd0 tx=0x7f7c2400b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:47:41.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.260+0000 7f7c30c00700 1 -- 172.21.15.107:0/2513579029 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7c1c05b5a0 con 0x7f7c340fe8f0 2024-03-04T13:47:41.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.504+0000 7f7c3aa00700 1 -- 172.21.15.107:0/2513579029 --> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7c341136c0 con 0x7f7c1806d900 2024-03-04T13:47:41.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.505+0000 7f7c30c00700 1 -- 172.21.15.107:0/2513579029 <== mgr.34102 v2:172.21.15.107:6800/1223039812 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7f7c341136c0 con 0x7f7c1806d900 2024-03-04T13:47:41.505 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-03-04T13:47:41.506 INFO:teuthology.orchestra.run.smithi107.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377", 2024-03-04T13:47:41.506 INFO:teuthology.orchestra.run.smithi107.stdout: "in_progress": true, 2024-03-04T13:47:41.506 INFO:teuthology.orchestra.run.smithi107.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-04T13:47:41.506 INFO:teuthology.orchestra.run.smithi107.stdout: "services_complete": [ 2024-03-04T13:47:41.506 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr" 2024-03-04T13:47:41.506 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-04T13:47:41.506 INFO:teuthology.orchestra.run.smithi107.stdout: "progress": "3/21 daemons upgraded", 2024-03-04T13:47:41.506 INFO:teuthology.orchestra.run.smithi107.stdout: "message": "", 2024-03-04T13:47:41.506 INFO:teuthology.orchestra.run.smithi107.stdout: "is_paused": false 2024-03-04T13:47:41.506 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-03-04T13:47:41.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.508+0000 7f7c3aa00700 1 -- 172.21.15.107:0/2513579029 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f7c1806d900 msgr2=0x7f7c1806fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:47:41.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.508+0000 7f7c3aa00700 1 --2- 172.21.15.107:0/2513579029 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f7c1806d900 0x7f7c1806fdb0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f7c24009fd0 tx=0x7f7c2400b040 comp rx=0 tx=0).stop 2024-03-04T13:47:41.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.508+0000 7f7c3aa00700 1 -- 172.21.15.107:0/2513579029 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c340fe8f0 msgr2=0x7f7c34113f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:47:41.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.508+0000 7f7c3aa00700 1 --2- 172.21.15.107:0/2513579029 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c340fe8f0 0x7f7c34113f30 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c005010 tx=0x7f7c1c009e30 comp rx=0 tx=0).stop 2024-03-04T13:47:41.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.508+0000 7f7c3aa00700 1 -- 172.21.15.107:0/2513579029 shutdown_connections 2024-03-04T13:47:41.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.508+0000 7f7c3aa00700 1 --2- 172.21.15.107:0/2513579029 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f7c1806d900 0x7f7c1806fdb0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:47:41.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.508+0000 7f7c3aa00700 1 --2- 172.21.15.107:0/2513579029 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c340fdfe0 0x7f7c341139f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:47:41.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.508+0000 7f7c3aa00700 1 --2- 172.21.15.107:0/2513579029 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c340fe8f0 0x7f7c34113f30 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:47:41.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.508+0000 7f7c3aa00700 1 -- 172.21.15.107:0/2513579029 >> 172.21.15.107:0/2513579029 conn(0x7f7c34071d90 msgr2=0x7f7c34101840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:47:41.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.509+0000 7f7c3aa00700 1 -- 172.21.15.107:0/2513579029 shutdown_connections 2024-03-04T13:47:41.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.509+0000 7f7c3aa00700 1 -- 172.21.15.107:0/2513579029 wait complete. 2024-03-04T13:47:41.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.649+0000 7f4fa0600700 1 -- 172.21.15.107:0/146747667 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f980fdf40 msgr2=0x7f4f980fe310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:47:41.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.649+0000 7f4fa0600700 1 --2- 172.21.15.107:0/146747667 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f980fdf40 0x7f4f980fe310 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c004940 tx=0x7f4f8c009e30 comp rx=0 tx=0).stop 2024-03-04T13:47:41.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.649+0000 7f4fa0600700 1 -- 172.21.15.107:0/146747667 shutdown_connections 2024-03-04T13:47:41.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.649+0000 7f4fa0600700 1 --2- 172.21.15.107:0/146747667 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4f980fe8e0 0x7f4f98106f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:47:41.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.649+0000 7f4fa0600700 1 --2- 172.21.15.107:0/146747667 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f980fdf40 0x7f4f980fe310 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:47:41.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.649+0000 7f4fa0600700 1 -- 172.21.15.107:0/146747667 >> 172.21.15.107:0/146747667 conn(0x7f4f980f79d0 msgr2=0x7f4f980f9de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:47:41.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.649+0000 7f4fa0600700 1 -- 172.21.15.107:0/146747667 shutdown_connections 2024-03-04T13:47:41.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.649+0000 7f4fa0600700 1 -- 172.21.15.107:0/146747667 wait complete. 2024-03-04T13:47:41.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.650+0000 7f4fa0600700 1 Processor -- start 2024-03-04T13:47:41.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.650+0000 7f4fa0600700 1 -- start start 2024-03-04T13:47:41.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.651+0000 7f4fa0600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f980fdf40 0x7f4f98115bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:47:41.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.651+0000 7f4fa0600700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4f980fe8e0 0x7f4f98116100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:47:41.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.651+0000 7f4fa0600700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f98116750 con 0x7f4f980fdf40 2024-03-04T13:47:41.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.651+0000 7f4fa0600700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f9810fa80 con 0x7f4f980fe8e0 2024-03-04T13:47:41.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.651+0000 7f4f9dc00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f980fdf40 0x7f4f98115bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:47:41.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.651+0000 7f4f9dc00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f980fdf40 0x7f4f98115bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37162/0 (socket says 172.21.15.107:37162) 2024-03-04T13:47:41.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.651+0000 7f4f9dc00700 1 -- 172.21.15.107:0/3179209452 learned_addr learned my addr 172.21.15.107:0/3179209452 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:47:41.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.651+0000 7f4f9d200700 1 --2- 172.21.15.107:0/3179209452 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4f980fe8e0 0x7f4f98116100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:47:41.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.651+0000 7f4f9dc00700 1 -- 172.21.15.107:0/3179209452 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4f980fe8e0 msgr2=0x7f4f98116100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:47:41.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.651+0000 7f4f9dc00700 1 --2- 172.21.15.107:0/3179209452 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4f980fe8e0 0x7f4f98116100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:47:41.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.651+0000 7f4f9dc00700 1 -- 172.21.15.107:0/3179209452 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f8c005040 con 0x7f4f980fdf40 2024-03-04T13:47:41.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.652+0000 7f4f9dc00700 1 --2- 172.21.15.107:0/3179209452 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f980fdf40 0x7f4f98115bc0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c000c00 tx=0x7f4f8c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:47:41.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.652+0000 7f4f95600700 1 -- 172.21.15.107:0/3179209452 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4f8c021070 con 0x7f4f980fdf40 2024-03-04T13:47:41.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.652+0000 7f4f95600700 1 -- 172.21.15.107:0/3179209452 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4f8c013d00 con 0x7f4f980fdf40 2024-03-04T13:47:41.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.652+0000 7f4fa0600700 1 -- 172.21.15.107:0/3179209452 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f9810fd00 con 0x7f4f980fdf40 2024-03-04T13:47:41.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.652+0000 7f4f95600700 1 -- 172.21.15.107:0/3179209452 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4f8c00e070 con 0x7f4f980fdf40 2024-03-04T13:47:41.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.652+0000 7f4fa0600700 1 -- 172.21.15.107:0/3179209452 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f98110240 con 0x7f4f980fdf40 2024-03-04T13:47:41.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.654+0000 7f4fa0600700 1 -- 172.21.15.107:0/3179209452 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f98106990 con 0x7f4f980fdf40 2024-03-04T13:47:41.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.654+0000 7f4f95600700 1 -- 172.21.15.107:0/3179209452 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 39) v1 ==== 91587+0+0 (secure 0 0 0) 0x7f4f8c01cb60 con 0x7f4f980fdf40 2024-03-04T13:47:41.655 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.655+0000 7f4f95600700 1 --2- 172.21.15.107:0/3179209452 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f4f8006d900 0x7f4f8006fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:47:41.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.655+0000 7f4f95600700 1 -- 172.21.15.107:0/3179209452 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f4f8c08e540 con 0x7f4f980fdf40 2024-03-04T13:47:41.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.656+0000 7f4f9d200700 1 --2- 172.21.15.107:0/3179209452 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f4f8006d900 0x7f4f8006fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:47:41.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.656+0000 7f4f9d200700 1 --2- 172.21.15.107:0/3179209452 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f4f8006d900 0x7f4f8006fdb0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f4f84004600 tx=0x7f4f8400b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:47:41.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.661+0000 7f4f95600700 1 -- 172.21.15.107:0/3179209452 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4f8c05b810 con 0x7f4f980fdf40 2024-03-04T13:47:41.843 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:41 smithi107 ceph-mon[112578]: Updating smithi107:/var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-04T13:47:41.843 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:41 smithi107 ceph-mon[112578]: from='client.34138 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:47:41.843 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:41 smithi107 ceph-mon[112578]: Updating smithi169:/var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-04T13:47:41.843 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:41 smithi107 ceph-mon[112578]: pgmap v11: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 18 KiB/s rd, 0 B/s wr, 11 op/s 2024-03-04T13:47:41.843 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:41 smithi107 ceph-mon[112578]: Updating smithi107:/etc/ceph/ceph.client.admin.keyring 2024-03-04T13:47:41.843 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:41 smithi107 ceph-mon[112578]: from='client.? 172.21.15.107:0/2163569592' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-04T13:47:41.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.975+0000 7f4fa0600700 1 -- 172.21.15.107:0/3179209452 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f4f9810a030 con 0x7f4f980fdf40 2024-03-04T13:47:41.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.976+0000 7f4f95600700 1 -- 172.21.15.107:0/3179209452 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f4f8c05b3a0 con 0x7f4f980fdf40 2024-03-04T13:47:41.976 INFO:teuthology.orchestra.run.smithi107.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-03-04T13:47:41.976 INFO:teuthology.orchestra.run.smithi107.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-04T13:47:41.976 INFO:teuthology.orchestra.run.smithi107.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-04T13:47:41.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.978+0000 7f4fa0600700 1 -- 172.21.15.107:0/3179209452 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f4f8006d900 msgr2=0x7f4f8006fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:47:41.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.978+0000 7f4fa0600700 1 --2- 172.21.15.107:0/3179209452 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f4f8006d900 0x7f4f8006fdb0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f4f84004600 tx=0x7f4f8400b040 comp rx=0 tx=0).stop 2024-03-04T13:47:41.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.978+0000 7f4fa0600700 1 -- 172.21.15.107:0/3179209452 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f980fdf40 msgr2=0x7f4f98115bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:47:41.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.978+0000 7f4fa0600700 1 --2- 172.21.15.107:0/3179209452 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f980fdf40 0x7f4f98115bc0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c000c00 tx=0x7f4f8c009e30 comp rx=0 tx=0).stop 2024-03-04T13:47:41.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.978+0000 7f4fa0600700 1 -- 172.21.15.107:0/3179209452 shutdown_connections 2024-03-04T13:47:41.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.978+0000 7f4fa0600700 1 --2- 172.21.15.107:0/3179209452 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f4f8006d900 0x7f4f8006fdb0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:47:41.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.978+0000 7f4fa0600700 1 --2- 172.21.15.107:0/3179209452 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f980fdf40 0x7f4f98115bc0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:47:41.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.978+0000 7f4fa0600700 1 --2- 172.21.15.107:0/3179209452 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4f980fe8e0 0x7f4f98116100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:47:41.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.978+0000 7f4fa0600700 1 -- 172.21.15.107:0/3179209452 >> 172.21.15.107:0/3179209452 conn(0x7f4f980f79d0 msgr2=0x7f4f980fc530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:47:41.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.979+0000 7f4fa0600700 1 -- 172.21.15.107:0/3179209452 shutdown_connections 2024-03-04T13:47:41.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:47:41.979+0000 7f4fa0600700 1 -- 172.21.15.107:0/3179209452 wait complete. 2024-03-04T13:47:42.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:41 smithi169 ceph-mon[36313]: Updating smithi107:/var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-04T13:47:42.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:41 smithi169 ceph-mon[36313]: from='client.34138 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:47:42.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:41 smithi169 ceph-mon[36313]: Updating smithi169:/var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-04T13:47:42.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:41 smithi169 ceph-mon[36313]: pgmap v11: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 18 KiB/s rd, 0 B/s wr, 11 op/s 2024-03-04T13:47:42.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:41 smithi169 ceph-mon[36313]: Updating smithi107:/etc/ceph/ceph.client.admin.keyring 2024-03-04T13:47:42.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:41 smithi169 ceph-mon[36313]: from='client.? 172.21.15.107:0/2163569592' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-04T13:47:43.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:42 smithi107 ceph-mon[112578]: Updating smithi169:/etc/ceph/ceph.client.admin.keyring 2024-03-04T13:47:43.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:42 smithi107 ceph-mon[112578]: Updating smithi107:/var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/config/ceph.client.admin.keyring 2024-03-04T13:47:43.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:42 smithi107 ceph-mon[112578]: from='client.34150 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:47:43.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:42 smithi107 ceph-mon[112578]: Updating smithi169:/var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/config/ceph.client.admin.keyring 2024-03-04T13:47:43.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:42 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:47:43.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:42 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:47:43.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:42 smithi107 ceph-mon[112578]: from='client.? 172.21.15.107:0/3179209452' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-04T13:47:43.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:42 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:47:43.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:42 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:47:43.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:42 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:47:43.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:42 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:47:43.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:42 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:47:43.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:42 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-03-04T13:47:43.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:42 smithi169 ceph-mon[36313]: Updating smithi169:/etc/ceph/ceph.client.admin.keyring 2024-03-04T13:47:43.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:42 smithi169 ceph-mon[36313]: Updating smithi107:/var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/config/ceph.client.admin.keyring 2024-03-04T13:47:43.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:42 smithi169 ceph-mon[36313]: from='client.34150 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:47:43.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:42 smithi169 ceph-mon[36313]: Updating smithi169:/var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/config/ceph.client.admin.keyring 2024-03-04T13:47:43.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:42 smithi169 ceph-mon[36313]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:47:43.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:42 smithi169 ceph-mon[36313]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:47:43.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:42 smithi169 ceph-mon[36313]: from='client.? 172.21.15.107:0/3179209452' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-04T13:47:43.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:42 smithi169 ceph-mon[36313]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:47:43.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:42 smithi169 ceph-mon[36313]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:47:43.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:42 smithi169 ceph-mon[36313]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:47:43.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:42 smithi169 ceph-mon[36313]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:47:43.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:42 smithi169 ceph-mon[36313]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:47:43.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:42 smithi169 ceph-mon[36313]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-03-04T13:47:44.017 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:43 smithi169 ceph-mon[36313]: pgmap v12: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 18 KiB/s rd, 0 B/s wr, 12 op/s 2024-03-04T13:47:44.092 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:43 smithi107 ceph-mon[112578]: pgmap v12: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 18 KiB/s rd, 0 B/s wr, 12 op/s 2024-03-04T13:47:46.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:45 smithi107 ceph-mon[112578]: pgmap v13: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:47:46.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:45 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:47:46.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:45 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-04T13:47:46.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:45 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-04T13:47:46.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:45 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:47:46.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:45 smithi169 ceph-mon[36313]: pgmap v13: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:47:46.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:45 smithi169 ceph-mon[36313]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:47:46.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:45 smithi169 ceph-mon[36313]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-04T13:47:46.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:45 smithi169 ceph-mon[36313]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-04T13:47:46.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:45 smithi169 ceph-mon[36313]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:47:47.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:46 smithi107 ceph-mon[112578]: Upgrade: Updating mon.smithi169 2024-03-04T13:47:47.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:46 smithi107 ceph-mon[112578]: Deploying daemon mon.smithi169 on smithi169 2024-03-04T13:47:47.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:46 smithi169 ceph-mon[36313]: Upgrade: Updating mon.smithi169 2024-03-04T13:47:47.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:46 smithi169 ceph-mon[36313]: Deploying daemon mon.smithi169 on smithi169 2024-03-04T13:47:48.099 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:47 smithi169 systemd[1]: Stopping Ceph mon.smithi169 for 1fc09a58-da2b-11ee-95c6-87774f69a715... 2024-03-04T13:47:48.099 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:47 smithi169 ceph-mon[36313]: pgmap v14: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:47:48.331 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:47 smithi107 ceph-mon[112578]: pgmap v14: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:47:48.360 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:48 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-mon-smithi169[36298]: 2024-03-04T13:47:48.096+0000 7f685c200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi169 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-04T13:47:48.360 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:48 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-mon-smithi169[36298]: 2024-03-04T13:47:48.096+0000 7f685c200700 -1 mon.smithi169@1(peon) e2 *** Got Signal Terminated *** 2024-03-04T13:47:48.360 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:48 smithi169 podman[86021]: 2024-03-04 13:47:48.260761112 +0000 UTC m=+0.300129568 container died ba4ee916297fdd8fc62969689728b91425293dceddd7179ef90a69f022b94a0e (image=quay.ceph.io/ceph-ci/ceph@sha256:ee85afea73f504aaae524393a00a3b140e2a1cfd81d76da4525eaa6e73286a7d, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-mon-smithi169, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, RELEASE=quincy-42608bb, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-04T13:47:48.860 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:48 smithi169 podman[86021]: 2024-03-04 13:47:48.682124573 +0000 UTC m=+0.721493059 container cleanup ba4ee916297fdd8fc62969689728b91425293dceddd7179ef90a69f022b94a0e (image=quay.ceph.io/ceph-ci/ceph@sha256:ee85afea73f504aaae524393a00a3b140e2a1cfd81d76da4525eaa6e73286a7d, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-mon-smithi169, maintainer=Guillaume Abrioux , RELEASE=quincy-42608bb, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-03-04T13:47:48.861 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:48 smithi169 bash[86021]: ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-mon-smithi169 2024-03-04T13:47:49.360 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:48 smithi169 podman[86035]: 2024-03-04 13:47:48.983388898 +0000 UTC m=+0.711752813 container remove ba4ee916297fdd8fc62969689728b91425293dceddd7179ef90a69f022b94a0e (image=quay.ceph.io/ceph-ci/ceph@sha256:ee85afea73f504aaae524393a00a3b140e2a1cfd81d76da4525eaa6e73286a7d, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-mon-smithi169, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, GIT_CLEAN=True, RELEASE=quincy-42608bb, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-03-04T13:47:49.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:49 smithi169 systemd[1]: ceph-1fc09a58-da2b-11ee-95c6-87774f69a715@mon.smithi169.service: Deactivated successfully. 2024-03-04T13:47:49.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:49 smithi169 systemd[1]: Stopped Ceph mon.smithi169 for 1fc09a58-da2b-11ee-95c6-87774f69a715. 2024-03-04T13:47:49.776 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:49 smithi169 systemd[1]: ceph-1fc09a58-da2b-11ee-95c6-87774f69a715@mon.smithi169.service: Consumed 17.884s CPU time. 2024-03-04T13:47:50.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:49 smithi169 systemd[1]: Starting Ceph mon.smithi169 for 1fc09a58-da2b-11ee-95c6-87774f69a715... 2024-03-04T13:47:50.447 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 podman[86165]: 2024-03-04 13:47:50.032195683 +0000 UTC m=+0.035644603 image pull d71b839abdf892550525dad59fdf20a6b925b73dbcc7526f0d04da8380b9fabf quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377 2024-03-04T13:47:50.447 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 podman[86165]: 2024-03-04 13:47:50.19270161 +0000 UTC m=+0.196150555 container create e3527449381aee95c0af0351abb7d1250f92f63206a092eabd894a5a84545a78 (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-mon-smithi169, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-03-04T13:47:50.861 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 podman[86165]: 2024-03-04 13:47:50.446696111 +0000 UTC m=+0.450145102 container init e3527449381aee95c0af0351abb7d1250f92f63206a092eabd894a5a84545a78 (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-mon-smithi169, GIT_BRANCH=HEAD, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240222, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-03-04T13:47:50.861 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 podman[86165]: 2024-03-04 13:47:50.458598698 +0000 UTC m=+0.462047680 container start e3527449381aee95c0af0351abb7d1250f92f63206a092eabd894a5a84545a78 (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-mon-smithi169, io.buildah.version=1.33.5, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-03-04T13:47:50.861 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: set uid:gid to 167:167 (ceph:ceph) 2024-03-04T13:47:50.861 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: ceph version 19.0.0-1845-g717ce3cc (717ce3cce8d0166e4e577faa8004e6f9cb4128c0) squid (dev), process ceph-mon, pid 2 2024-03-04T13:47:50.862 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: pidfile_write: ignore empty --pid-file 2024-03-04T13:47:50.862 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: load: jerasure load: lrc 2024-03-04T13:47:50.862 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: RocksDB version: 7.9.2 2024-03-04T13:47:50.862 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Git sha 0 2024-03-04T13:47:50.862 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Compile date 2024-03-04 04:43:24 2024-03-04T13:47:50.862 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: DB SUMMARY 2024-03-04T13:47:50.862 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: DB Session ID: YKTJXVYQRRQEFUICWB5U 2024-03-04T13:47:50.862 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: CURRENT file: CURRENT 2024-03-04T13:47:50.862 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: IDENTITY file: IDENTITY 2024-03-04T13:47:50.862 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: MANIFEST file: MANIFEST-000009 size: 1271 Bytes 2024-03-04T13:47:50.863 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi169/store.db dir, Total Num: 1, files: 000030.sst 2024-03-04T13:47:50.863 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi169/store.db: 000028.log size: 3872467 ; 2024-03-04T13:47:50.863 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.error_if_exists: 0 2024-03-04T13:47:50.863 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.create_if_missing: 0 2024-03-04T13:47:50.863 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.paranoid_checks: 1 2024-03-04T13:47:50.863 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-04T13:47:50.863 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-04T13:47:50.863 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-04T13:47:50.863 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.env: 0x5597f8367b20 2024-03-04T13:47:50.863 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.fs: PosixFileSystem 2024-03-04T13:47:50.863 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.info_log: 0x5597f8f1b7c0 2024-03-04T13:47:50.864 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.max_file_opening_threads: 16 2024-03-04T13:47:50.864 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.statistics: (nil) 2024-03-04T13:47:50.865 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.use_fsync: 0 2024-03-04T13:47:50.865 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.max_log_file_size: 0 2024-03-04T13:47:50.865 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-04T13:47:50.865 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-04T13:47:50.865 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.keep_log_file_num: 1000 2024-03-04T13:47:50.865 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.recycle_log_file_num: 0 2024-03-04T13:47:50.865 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.allow_fallocate: 1 2024-03-04T13:47:50.865 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.allow_mmap_reads: 0 2024-03-04T13:47:50.865 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.allow_mmap_writes: 0 2024-03-04T13:47:50.866 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.use_direct_reads: 0 2024-03-04T13:47:50.866 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-04T13:47:50.866 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.create_missing_column_families: 0 2024-03-04T13:47:50.866 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.db_log_dir: 2024-03-04T13:47:50.866 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.wal_dir: 2024-03-04T13:47:50.866 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-04T13:47:50.866 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-04T13:47:50.866 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-04T13:47:50.866 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-04T13:47:50.866 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-04T13:47:50.866 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-04T13:47:50.867 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.advise_random_on_open: 1 2024-03-04T13:47:50.867 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.db_write_buffer_size: 0 2024-03-04T13:47:50.867 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.write_buffer_manager: 0x5597f8eef400 2024-03-04T13:47:50.867 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-04T13:47:50.867 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-04T13:47:50.867 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-04T13:47:50.867 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.rate_limiter: (nil) 2024-03-04T13:47:50.867 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-04T13:47:50.867 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.wal_recovery_mode: 2 2024-03-04T13:47:50.867 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.enable_thread_tracking: 0 2024-03-04T13:47:50.868 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.enable_pipelined_write: 0 2024-03-04T13:47:50.868 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.unordered_write: 0 2024-03-04T13:47:50.868 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-04T13:47:50.868 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-04T13:47:50.868 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-04T13:47:50.868 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-04T13:47:50.868 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.row_cache: None 2024-03-04T13:47:50.868 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.wal_filter: None 2024-03-04T13:47:50.868 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-04T13:47:50.868 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.allow_ingest_behind: 0 2024-03-04T13:47:50.868 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.two_write_queues: 0 2024-03-04T13:47:50.869 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.manual_wal_flush: 0 2024-03-04T13:47:50.869 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.wal_compression: 0 2024-03-04T13:47:50.869 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.atomic_flush: 0 2024-03-04T13:47:50.869 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-04T13:47:50.869 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-04T13:47:50.869 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-04T13:47:50.869 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.log_readahead_size: 0 2024-03-04T13:47:50.869 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-04T13:47:50.869 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.best_efforts_recovery: 0 2024-03-04T13:47:50.869 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-04T13:47:50.869 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-04T13:47:50.870 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.allow_data_in_errors: 0 2024-03-04T13:47:50.870 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.db_host_id: __hostname__ 2024-03-04T13:47:50.870 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-04T13:47:50.870 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.max_background_jobs: 2 2024-03-04T13:47:50.870 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.max_background_compactions: -1 2024-03-04T13:47:50.870 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.max_subcompactions: 1 2024-03-04T13:47:50.870 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-04T13:47:50.870 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-04T13:47:50.870 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-04T13:47:50.870 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.max_total_wal_size: 0 2024-03-04T13:47:50.870 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-04T13:47:50.870 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-04T13:47:50.871 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-04T13:47:50.871 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-04T13:47:50.871 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.max_open_files: -1 2024-03-04T13:47:50.871 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.bytes_per_sync: 0 2024-03-04T13:47:50.871 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-04T13:47:50.871 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-04T13:47:50.871 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.compaction_readahead_size: 0 2024-03-04T13:47:50.871 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.max_background_flushes: -1 2024-03-04T13:47:50.871 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Compression algorithms supported: 2024-03-04T13:47:50.871 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-04T13:47:50.872 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: kZSTD supported: 0 2024-03-04T13:47:50.872 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: kXpressCompression supported: 0 2024-03-04T13:47:50.872 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: kLZ4HCCompression supported: 1 2024-03-04T13:47:50.872 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: kZlibCompression supported: 1 2024-03-04T13:47:50.872 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: kSnappyCompression supported: 1 2024-03-04T13:47:50.872 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: kLZ4Compression supported: 1 2024-03-04T13:47:50.872 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: kBZip2Compression supported: 0 2024-03-04T13:47:50.872 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-04T13:47:50.872 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-04T13:47:50.872 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi169/store.db/MANIFEST-000009 2024-03-04T13:47:50.872 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-04T13:47:50.873 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-04T13:47:50.873 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.merge_operator: 2024-03-04T13:47:50.873 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.compaction_filter: None 2024-03-04T13:47:50.873 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.compaction_filter_factory: None 2024-03-04T13:47:50.873 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.sst_partitioner_factory: None 2024-03-04T13:47:50.873 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-04T13:47:50.873 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-04T13:47:50.873 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5597f8f1b8e0) 2024-03-04T13:47:50.873 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: cache_index_and_filter_blocks: 1 2024-03-04T13:47:50.873 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-04T13:47:50.873 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-04T13:47:50.874 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: pin_top_level_index_and_filter: 1 2024-03-04T13:47:50.874 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: index_type: 0 2024-03-04T13:47:50.874 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: data_block_index_type: 0 2024-03-04T13:47:50.874 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: index_shortening: 1 2024-03-04T13:47:50.874 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-04T13:47:50.874 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: checksum: 4 2024-03-04T13:47:50.874 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: no_block_cache: 0 2024-03-04T13:47:50.874 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: block_cache: 0x5597f8f21090 2024-03-04T13:47:50.874 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: block_cache_name: BinnedLRUCache 2024-03-04T13:47:50.874 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: block_cache_options: 2024-03-04T13:47:50.875 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: capacity : 536870912 2024-03-04T13:47:50.875 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: num_shard_bits : 4 2024-03-04T13:47:50.875 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: strict_capacity_limit : 0 2024-03-04T13:47:50.875 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: high_pri_pool_ratio: 0.000 2024-03-04T13:47:50.875 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: block_cache_compressed: (nil) 2024-03-04T13:47:50.875 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: persistent_cache: (nil) 2024-03-04T13:47:50.875 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: block_size: 4096 2024-03-04T13:47:50.875 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: block_size_deviation: 10 2024-03-04T13:47:50.875 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: block_restart_interval: 16 2024-03-04T13:47:50.875 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: index_block_restart_interval: 1 2024-03-04T13:47:50.876 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: metadata_block_size: 4096 2024-03-04T13:47:50.876 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: partition_filters: 0 2024-03-04T13:47:50.876 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: use_delta_encoding: 1 2024-03-04T13:47:50.876 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: filter_policy: bloomfilter 2024-03-04T13:47:50.876 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: whole_key_filtering: 1 2024-03-04T13:47:50.876 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: verify_compression: 0 2024-03-04T13:47:50.876 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: read_amp_bytes_per_bit: 0 2024-03-04T13:47:50.876 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: format_version: 5 2024-03-04T13:47:50.876 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: enable_index_compression: 1 2024-03-04T13:47:50.876 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: block_align: 0 2024-03-04T13:47:50.877 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: max_auto_readahead_size: 262144 2024-03-04T13:47:50.877 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: prepopulate_block_cache: 0 2024-03-04T13:47:50.877 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: initial_auto_readahead_size: 8192 2024-03-04T13:47:50.878 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: num_file_reads_for_auto_readahead: 2 2024-03-04T13:47:50.878 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.write_buffer_size: 33554432 2024-03-04T13:47:50.878 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.max_write_buffer_number: 2 2024-03-04T13:47:50.878 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.compression: NoCompression 2024-03-04T13:47:50.878 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.bottommost_compression: Disabled 2024-03-04T13:47:50.879 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.prefix_extractor: nullptr 2024-03-04T13:47:50.879 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-04T13:47:50.879 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.num_levels: 7 2024-03-04T13:47:50.879 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-04T13:47:50.879 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-04T13:47:50.879 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-04T13:47:50.879 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-04T13:47:50.879 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-04T13:47:50.879 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-04T13:47:50.879 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-04T13:47:50.879 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-04T13:47:50.879 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-04T13:47:50.880 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-04T13:47:50.880 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-04T13:47:50.880 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-04T13:47:50.880 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-04T13:47:50.880 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.compression_opts.level: 32767 2024-03-04T13:47:50.880 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.compression_opts.strategy: 0 2024-03-04T13:47:50.880 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-04T13:47:50.880 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-04T13:47:50.880 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-04T13:47:50.880 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-04T13:47:50.880 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.compression_opts.enabled: false 2024-03-04T13:47:50.881 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-04T13:47:50.881 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-04T13:47:50.881 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-04T13:47:50.881 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-04T13:47:50.881 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.target_file_size_base: 67108864 2024-03-04T13:47:50.881 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-04T13:47:50.881 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-04T13:47:50.881 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-04T13:47:50.881 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-04T13:47:50.881 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-04T13:47:50.881 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-04T13:47:50.882 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-04T13:47:50.882 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-04T13:47:50.882 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-04T13:47:50.882 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-04T13:47:50.882 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-04T13:47:50.882 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-04T13:47:50.882 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-04T13:47:50.882 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-04T13:47:50.882 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.arena_block_size: 1048576 2024-03-04T13:47:50.882 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-04T13:47:50.882 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-04T13:47:50.882 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.disable_auto_compactions: 0 2024-03-04T13:47:50.883 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-04T13:47:50.883 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-04T13:47:50.883 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-04T13:47:50.883 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-04T13:47:50.883 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-04T13:47:50.883 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-04T13:47:50.883 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-04T13:47:50.883 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-04T13:47:50.883 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-04T13:47:50.883 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-04T13:47:50.883 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.table_properties_collectors: 2024-03-04T13:47:50.883 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.inplace_update_support: 0 2024-03-04T13:47:50.884 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-04T13:47:50.884 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-04T13:47:50.884 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-04T13:47:50.884 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-04T13:47:50.884 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.bloom_locality: 0 2024-03-04T13:47:50.884 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.max_successive_merges: 0 2024-03-04T13:47:50.884 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-04T13:47:50.884 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.paranoid_file_checks: 0 2024-03-04T13:47:50.884 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.force_consistency_checks: 1 2024-03-04T13:47:50.884 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.report_bg_io_stats: 0 2024-03-04T13:47:50.885 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.ttl: 2592000 2024-03-04T13:47:50.885 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-04T13:47:50.885 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-04T13:47:50.885 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-04T13:47:50.885 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.enable_blob_files: false 2024-03-04T13:47:50.885 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.min_blob_size: 0 2024-03-04T13:47:50.885 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.blob_file_size: 268435456 2024-03-04T13:47:50.885 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-04T13:47:50.885 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-04T13:47:50.885 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-04T13:47:50.885 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-04T13:47:50.886 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-04T13:47:50.886 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.blob_file_starting_level: 0 2024-03-04T13:47:50.886 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-04T13:47:50.886 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: [table/block_based/block_based_table_reader.cc:721] At least one SST file opened without unique ID to verify: 30.sst 2024-03-04T13:47:50.886 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-03-04T13:47:50.886 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi169/store.db/MANIFEST-000009 succeeded,manifest_file_number is 9, next_file_number is 32, last_sequence is 16984, log_number is 28,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-03-04T13:47:50.886 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 28 2024-03-04T13:47:50.886 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 04792177-651b-4587-83fe-f963a16a352a 2024-03-04T13:47:50.886 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: EVENT_LOG_v1 {"time_micros": 1709560070517576, "job": 1, "event": "recovery_started", "wal_files": [28]} 2024-03-04T13:47:50.886 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #28 mode 2 2024-03-04T13:47:50.886 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: EVENT_LOG_v1 {"time_micros": 1709560070536375, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 33, "file_size": 2450754, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 16981, "largest_seqno": 18539, "table_properties": {"data_size": 2444340, "index_size": 3795, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 1605, "raw_key_size": 15607, "raw_average_key_size": 24, "raw_value_size": 2430802, "raw_average_value_size": 3816, "num_data_blocks": 171, "num_entries": 637, "num_filter_entries": 637, "num_deletions": 7, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1709560070, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "04792177-651b-4587-83fe-f963a16a352a", "db_session_id": "YKTJXVYQRRQEFUICWB5U", "orig_file_number": 33, "seqno_to_time_mapping": "N/A"}} 2024-03-04T13:47:50.886 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: EVENT_LOG_v1 {"time_micros": 1709560070536681, "job": 1, "event": "recovery_finished"} 2024-03-04T13:47:50.887 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: [db/version_set.cc:5047] Creating manifest 35 2024-03-04T13:47:50.887 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-03-04T13:47:50.887 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi169/store.db/000028.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-04T13:47:50.887 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5597f9012000 2024-03-04T13:47:50.887 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: DB pointer 0x5597f8ffe000 2024-03-04T13:47:50.887 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-04T13:47:50.887 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-04T13:47:50.887 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: ** DB Stats ** 2024-03-04T13:47:50.887 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-04T13:47:50.887 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-04T13:47:50.887 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-04T13:47:50.888 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-04T13:47:50.888 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-04T13:47:50.888 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-04T13:47:50.888 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-04T13:47:50.888 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: 2024-03-04T13:47:50.888 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: ** Compaction Stats [default] ** 2024-03-04T13:47:50.888 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-04T13:47:50.888 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-04T13:47:50.888 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: L0 1/0 2.34 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 496.3 0.00 0.00 1 0.005 0 0 0.0 0.0 2024-03-04T13:47:50.888 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: L6 1/0 7.46 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-03-04T13:47:50.888 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: Sum 2/0 9.79 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 496.3 0.00 0.00 1 0.005 0 0 0.0 0.0 2024-03-04T13:47:50.888 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 496.3 0.00 0.00 1 0.005 0 0 0.0 0.0 2024-03-04T13:47:50.889 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: 2024-03-04T13:47:50.889 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: ** Compaction Stats [default] ** 2024-03-04T13:47:50.889 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-04T13:47:50.889 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-04T13:47:50.889 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 496.3 0.00 0.00 1 0.005 0 0 0.0 0.0 2024-03-04T13:47:50.889 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: 2024-03-04T13:47:50.889 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-04T13:47:50.889 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: 2024-03-04T13:47:50.889 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-04T13:47:50.889 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: Flush(GB): cumulative 0.002, interval 0.002 2024-03-04T13:47:50.889 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-04T13:47:50.890 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-04T13:47:50.890 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-04T13:47:50.890 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-04T13:47:50.890 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: Cumulative compaction: 0.00 GB write, 92.37 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-04T13:47:50.890 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: Interval compaction: 0.00 GB write, 92.37 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-04T13:47:50.890 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-04T13:47:50.890 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: Block cache BinnedLRUCache@0x5597f8f21090#2 capacity: 512.00 MB usage: 5.86 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.6e-05 secs_since: 0 2024-03-04T13:47:50.890 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,1.78 KB,0.000339746%) IndexBlock(1,4.08 KB,0.000777841%) Misc(1,0.00 KB,0%) 2024-03-04T13:47:50.890 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: 2024-03-04T13:47:50.890 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-04T13:47:50.891 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: starting mon.smithi169 rank 1 at public addrs [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] at bind addrs [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi169 fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 2024-03-04T13:47:50.891 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: mon.smithi169@-1(???) e2 preinit fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 2024-03-04T13:47:50.891 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: mon.smithi169@-1(???).mds e16 new map 2024-03-04T13:47:50.891 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: mon.smithi169@-1(???).mds e16 print_map 2024-03-04T13:47:50.891 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: e16 2024-03-04T13:47:50.891 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-04T13:47:50.891 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-04T13:47:50.891 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: legacy client fscid: 1 2024-03-04T13:47:50.891 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: 2024-03-04T13:47:50.891 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: Filesystem 'cephfs' (1) 2024-03-04T13:47:50.891 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: fs_name cephfs 2024-03-04T13:47:50.892 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: epoch 15 2024-03-04T13:47:50.892 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-04T13:47:50.892 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: created 2024-03-04T13:38:30.854004+0000 2024-03-04T13:47:50.892 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: modified 2024-03-04T13:38:50.015989+0000 2024-03-04T13:47:50.892 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: tableserver 0 2024-03-04T13:47:50.892 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: root 0 2024-03-04T13:47:50.892 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: session_timeout 60 2024-03-04T13:47:50.892 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: session_autoclose 300 2024-03-04T13:47:50.892 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: max_file_size 1099511627776 2024-03-04T13:47:50.893 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: max_xattr_size 65536 2024-03-04T13:47:50.893 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: required_client_features {} 2024-03-04T13:47:50.893 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: last_failure 0 2024-03-04T13:47:50.893 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: last_failure_osd_epoch 47 2024-03-04T13:47:50.893 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-04T13:47:50.893 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: max_mds 1 2024-03-04T13:47:50.893 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: in 0 2024-03-04T13:47:50.893 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: up {0=24247} 2024-03-04T13:47:50.893 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: failed 2024-03-04T13:47:50.893 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: damaged 2024-03-04T13:47:50.893 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: stopped 2024-03-04T13:47:50.893 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: data_pools [3] 2024-03-04T13:47:50.893 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: metadata_pool 2 2024-03-04T13:47:50.894 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: inline_data enabled 2024-03-04T13:47:50.894 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: balancer 2024-03-04T13:47:50.894 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: bal_rank_mask -1 2024-03-04T13:47:50.894 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: standby_count_wanted 1 2024-03-04T13:47:50.894 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: [mds.cephfs.smithi169.jusudl{0:24247} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.169:6824/2994866076,v1:172.21.15.169:6825/2994866076] compat {c=[1],r=[1],i=[7ff]}] 2024-03-04T13:47:50.894 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: [mds.cephfs.smithi107.thycho{0:14470} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.107:6828/1985577060,v1:172.21.15.107:6829/1985577060] compat {c=[1],r=[1],i=[7ff]}] 2024-03-04T13:47:50.894 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: 2024-03-04T13:47:50.894 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: 2024-03-04T13:47:50.894 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: Standby daemons: 2024-03-04T13:47:50.894 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: 2024-03-04T13:47:50.895 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: [mds.cephfs.smithi107.riexmm{-1:14480} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.107:6826/3261622517,v1:172.21.15.107:6827/3261622517] compat {c=[1],r=[1],i=[7ff]}] 2024-03-04T13:47:50.895 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout: [mds.cephfs.smithi169.bxitzk{-1:24259} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.169:6826/3696090493,v1:172.21.15.169:6827/3696090493] compat {c=[1],r=[1],i=[7ff]}] 2024-03-04T13:47:50.895 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: mon.smithi169@-1(???).osd e50 crush map has features 3314933000852226048, adjusting msgr requires 2024-03-04T13:47:50.895 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: mon.smithi169@-1(???).osd e50 crush map has features 288514051259236352, adjusting msgr requires 2024-03-04T13:47:50.895 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: mon.smithi169@-1(???).osd e50 crush map has features 288514051259236352, adjusting msgr requires 2024-03-04T13:47:50.895 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: mon.smithi169@-1(???).osd e50 crush map has features 288514051259236352, adjusting msgr requires 2024-03-04T13:47:50.895 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 ceph-mon[86183]: mon.smithi169@-1(???).paxosservice(auth 1..23) refresh upgraded, format 0 -> 3 2024-03-04T13:47:50.895 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 bash[86165]: e3527449381aee95c0af0351abb7d1250f92f63206a092eabd894a5a84545a78 2024-03-04T13:47:50.895 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:50 smithi169 systemd[1]: Started Ceph mon.smithi169 for 1fc09a58-da2b-11ee-95c6-87774f69a715. 2024-03-04T13:47:53.022 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:52 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mon metadata", "id": "smithi107"}]: dispatch 2024-03-04T13:47:53.023 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:52 smithi169 ceph-mon[86183]: mon.smithi107 calling monitor election 2024-03-04T13:47:53.023 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:52 smithi169 ceph-mon[86183]: mon.smithi169 calling monitor election 2024-03-04T13:47:53.023 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:52 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mon metadata", "id": "smithi169"}]: dispatch 2024-03-04T13:47:53.023 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:52 smithi169 ceph-mon[86183]: mon.smithi107 is new leader, mons smithi107,smithi169 in quorum (ranks 0,1) 2024-03-04T13:47:53.023 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:52 smithi169 ceph-mon[86183]: monmap e3: 2 mons at {smithi107=[v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0],smithi169=[v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-04T13:47:53.023 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:52 smithi169 ceph-mon[86183]: fsmap cephfs:1 {0=cephfs.smithi169.jusudl=up:active} 1 up:standby-replay 2 up:standby 2024-03-04T13:47:53.023 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:52 smithi169 ceph-mon[86183]: osdmap e50: 6 total, 6 up, 6 in 2024-03-04T13:47:53.023 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:52 smithi169 ceph-mon[86183]: mgrmap e39: smithi107.wpzgwl(active, since 27s), standbys: smithi169.eihczr 2024-03-04T13:47:53.023 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:52 smithi169 ceph-mon[86183]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-03-04T13:47:53.023 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:52 smithi169 ceph-mon[86183]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-04T13:47:53.023 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:52 smithi169 ceph-mon[86183]: fs cephfs has deprecated feature inline_data enabled. 2024-03-04T13:47:53.023 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:52 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:47:53.023 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:52 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:47:53.024 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:52 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:47:53.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:52 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mon metadata", "id": "smithi107"}]: dispatch 2024-03-04T13:47:53.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:52 smithi107 ceph-mon[112578]: mon.smithi107 calling monitor election 2024-03-04T13:47:53.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:52 smithi107 ceph-mon[112578]: mon.smithi169 calling monitor election 2024-03-04T13:47:53.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:52 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mon metadata", "id": "smithi169"}]: dispatch 2024-03-04T13:47:53.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:52 smithi107 ceph-mon[112578]: mon.smithi107 is new leader, mons smithi107,smithi169 in quorum (ranks 0,1) 2024-03-04T13:47:53.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:52 smithi107 ceph-mon[112578]: monmap e3: 2 mons at {smithi107=[v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0],smithi169=[v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-04T13:47:53.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:52 smithi107 ceph-mon[112578]: fsmap cephfs:1 {0=cephfs.smithi169.jusudl=up:active} 1 up:standby-replay 2 up:standby 2024-03-04T13:47:53.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:52 smithi107 ceph-mon[112578]: osdmap e50: 6 total, 6 up, 6 in 2024-03-04T13:47:53.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:52 smithi107 ceph-mon[112578]: mgrmap e39: smithi107.wpzgwl(active, since 27s), standbys: smithi169.eihczr 2024-03-04T13:47:53.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:52 smithi107 ceph-mon[112578]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-03-04T13:47:53.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:52 smithi107 ceph-mon[112578]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-04T13:47:53.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:52 smithi107 ceph-mon[112578]: fs cephfs has deprecated feature inline_data enabled. 2024-03-04T13:47:53.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:52 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:47:53.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:52 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:47:53.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:52 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:47:54.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:53 smithi107 ceph-mon[112578]: pgmap v17: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:47:54.101 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:53 smithi169 ceph-mon[86183]: pgmap v17: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:47:55.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:54 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:47:55.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:54 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:47:56.095 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:55 smithi107 ceph-mon[112578]: pgmap v18: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:47:56.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:55 smithi169 ceph-mon[86183]: pgmap v18: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:47:57.343 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:56 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:47:57.343 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:56 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:47:57.343 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:56 smithi107 ceph-mon[112578]: pgmap v19: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:47:57.360 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:56 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:47:57.360 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:56 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:47:57.360 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:56 smithi169 ceph-mon[86183]: pgmap v19: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:47:59.327 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:59 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:47:59.327 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:59 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:47:59.327 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:47:59 smithi169 ceph-mon[86183]: pgmap v20: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:47:59.343 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:59 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:47:59.343 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:59 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:47:59.343 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:47:59 smithi107 ceph-mon[112578]: pgmap v20: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:48:02.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:01 smithi107 ceph-mon[112578]: pgmap v21: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:48:02.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:01 smithi169 ceph-mon[86183]: pgmap v21: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:48:04.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:03 smithi107 ceph-mon[112578]: pgmap v22: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:48:04.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:03 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:48:04.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:03 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:48:04.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:03 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:48:04.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:03 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:48:04.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:03 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:48:04.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:03 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-04T13:48:04.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:03 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-04T13:48:04.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:03 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:48:04.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:03 smithi169 ceph-mon[86183]: pgmap v22: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:48:04.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:03 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:48:04.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:03 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:48:04.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:03 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:48:04.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:03 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:48:04.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:03 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:48:04.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:03 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-04T13:48:04.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:03 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-04T13:48:04.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:03 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:48:04.843 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:04 smithi169 ceph-mon[86183]: Reconfiguring mon.smithi107 (monmap changed)... 2024-03-04T13:48:04.843 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:04 smithi169 ceph-mon[86183]: Reconfiguring daemon mon.smithi107 on smithi107 2024-03-04T13:48:05.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:04 smithi107 ceph-mon[112578]: Reconfiguring mon.smithi107 (monmap changed)... 2024-03-04T13:48:05.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:04 smithi107 ceph-mon[112578]: Reconfiguring daemon mon.smithi107 on smithi107 2024-03-04T13:48:05.843 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:05 smithi107 ceph-mon[112578]: pgmap v23: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:48:06.046 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:05 smithi169 ceph-mon[86183]: pgmap v23: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:48:06.997 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:06 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:48:06.997 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:06 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:48:06.997 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:06 smithi107 ceph-mon[112578]: Reconfiguring mgr.smithi107.wpzgwl (monmap changed)... 2024-03-04T13:48:06.997 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:06 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi107.wpzgwl", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-04T13:48:06.997 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:06 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-04T13:48:06.997 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:06 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:48:06.997 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:06 smithi107 ceph-mon[112578]: Reconfiguring daemon mgr.smithi107.wpzgwl on smithi107 2024-03-04T13:48:07.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:06 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:48:07.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:06 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:48:07.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:06 smithi169 ceph-mon[86183]: Reconfiguring mgr.smithi107.wpzgwl (monmap changed)... 2024-03-04T13:48:07.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:06 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi107.wpzgwl", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-04T13:48:07.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:06 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-04T13:48:07.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:06 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:48:07.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:06 smithi169 ceph-mon[86183]: Reconfiguring daemon mgr.smithi107.wpzgwl on smithi107 2024-03-04T13:48:08.049 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:07 smithi107 ceph-mon[112578]: pgmap v24: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:48:08.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:07 smithi169 ceph-mon[86183]: pgmap v24: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:48:09.469 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:09 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:48:09.469 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:09 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:48:09.469 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:09 smithi107 ceph-mon[112578]: Reconfiguring crash.smithi107 (monmap changed)... 2024-03-04T13:48:09.469 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:09 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi107", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-04T13:48:09.469 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:09 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:48:09.469 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:09 smithi107 ceph-mon[112578]: Reconfiguring daemon crash.smithi107 on smithi107 2024-03-04T13:48:09.469 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:09 smithi107 ceph-mon[112578]: pgmap v25: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:48:09.611 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:09 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:48:09.611 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:09 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:48:09.611 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:09 smithi169 ceph-mon[86183]: Reconfiguring crash.smithi107 (monmap changed)... 2024-03-04T13:48:09.611 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:09 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi107", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-04T13:48:09.611 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:09 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:48:09.611 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:09 smithi169 ceph-mon[86183]: Reconfiguring daemon crash.smithi107 on smithi107 2024-03-04T13:48:09.611 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:09 smithi169 ceph-mon[86183]: pgmap v25: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:48:10.593 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:10 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:48:10.610 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:10 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:48:11.525 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:11 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:48:11.525 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:11 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:48:11.525 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:11 smithi107 ceph-mon[112578]: Reconfiguring osd.0 (monmap changed)... 2024-03-04T13:48:11.525 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:11 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-04T13:48:11.525 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:11 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:48:11.525 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:11 smithi107 ceph-mon[112578]: Reconfiguring daemon osd.0 on smithi107 2024-03-04T13:48:11.525 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:11 smithi107 ceph-mon[112578]: pgmap v26: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:48:11.610 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:11 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:48:11.610 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:11 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:48:11.610 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:11 smithi169 ceph-mon[86183]: Reconfiguring osd.0 (monmap changed)... 2024-03-04T13:48:11.610 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:11 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-04T13:48:11.611 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:11 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:48:11.611 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:11 smithi169 ceph-mon[86183]: Reconfiguring daemon osd.0 on smithi107 2024-03-04T13:48:11.611 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:11 smithi169 ceph-mon[86183]: pgmap v26: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:48:12.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.145+0000 7fd3dac00700 1 -- 172.21.15.107:0/1925865468 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd3d4103dc0 msgr2=0x7fd3d4104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:48:12.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.145+0000 7fd3dac00700 1 --2- 172.21.15.107:0/1925865468 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd3d4103dc0 0x7fd3d4104210 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd3c0004940 tx=0x7fd3c0009e30 comp rx=0 tx=0).stop 2024-03-04T13:48:12.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.145+0000 7fd3dac00700 1 -- 172.21.15.107:0/1925865468 shutdown_connections 2024-03-04T13:48:12.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.145+0000 7fd3dac00700 1 --2- 172.21.15.107:0/1925865468 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd3d4103dc0 0x7fd3d4104210 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:48:12.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.145+0000 7fd3dac00700 1 --2- 172.21.15.107:0/1925865468 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd3d41095f0 0x7fd3d41099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:48:12.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.145+0000 7fd3dac00700 1 -- 172.21.15.107:0/1925865468 >> 172.21.15.107:0/1925865468 conn(0x7fd3d4072d50 msgr2=0x7fd3d4075160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:48:12.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.145+0000 7fd3dac00700 1 -- 172.21.15.107:0/1925865468 shutdown_connections 2024-03-04T13:48:12.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.145+0000 7fd3dac00700 1 -- 172.21.15.107:0/1925865468 wait complete. 2024-03-04T13:48:12.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.146+0000 7fd3dac00700 1 Processor -- start 2024-03-04T13:48:12.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.146+0000 7fd3dac00700 1 -- start start 2024-03-04T13:48:12.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.146+0000 7fd3dac00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd3d4103dc0 0x7fd3d411a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:48:12.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.146+0000 7fd3dac00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd3d41095f0 0x7fd3d411a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:48:12.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.146+0000 7fd3dac00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3d411ac50 con 0x7fd3d41095f0 2024-03-04T13:48:12.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.146+0000 7fd3dac00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3d4113ed0 con 0x7fd3d4103dc0 2024-03-04T13:48:12.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.147+0000 7fd3d3400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd3d41095f0 0x7fd3d411a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:48:12.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.147+0000 7fd3d3400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd3d41095f0 0x7fd3d411a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:53336/0 (socket says 172.21.15.107:53336) 2024-03-04T13:48:12.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.147+0000 7fd3d3400700 1 -- 172.21.15.107:0/4072434805 learned_addr learned my addr 172.21.15.107:0/4072434805 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:48:12.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.147+0000 7fd3d3e00700 1 --2- 172.21.15.107:0/4072434805 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd3d4103dc0 0x7fd3d411a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:48:12.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.147+0000 7fd3d3400700 1 -- 172.21.15.107:0/4072434805 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd3d4103dc0 msgr2=0x7fd3d411a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:48:12.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.147+0000 7fd3d3400700 1 --2- 172.21.15.107:0/4072434805 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd3d4103dc0 0x7fd3d411a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:48:12.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.147+0000 7fd3d3400700 1 -- 172.21.15.107:0/4072434805 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3c0005040 con 0x7fd3d41095f0 2024-03-04T13:48:12.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.147+0000 7fd3d3400700 1 --2- 172.21.15.107:0/4072434805 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd3d41095f0 0x7fd3d411a570 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd3c0009150 tx=0x7fd3c0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:48:12.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.148+0000 7fd3cbe00700 1 -- 172.21.15.107:0/4072434805 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd3c000e040 con 0x7fd3d41095f0 2024-03-04T13:48:12.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.148+0000 7fd3cbe00700 1 -- 172.21.15.107:0/4072434805 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd3c000bbf0 con 0x7fd3d41095f0 2024-03-04T13:48:12.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.148+0000 7fd3cbe00700 1 -- 172.21.15.107:0/4072434805 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd3c001c630 con 0x7fd3d41095f0 2024-03-04T13:48:12.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.148+0000 7fd3dac00700 1 -- 172.21.15.107:0/4072434805 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3d41140f0 con 0x7fd3d41095f0 2024-03-04T13:48:12.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.148+0000 7fd3dac00700 1 -- 172.21.15.107:0/4072434805 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3d4114630 con 0x7fd3d41095f0 2024-03-04T13:48:12.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.149+0000 7fd3dac00700 1 -- 172.21.15.107:0/4072434805 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3d4075870 con 0x7fd3d41095f0 2024-03-04T13:48:12.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.150+0000 7fd3cbe00700 1 -- 172.21.15.107:0/4072434805 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 39) v1 ==== 91587+0+0 (secure 0 0 0) 0x7fd3c000bd60 con 0x7fd3d41095f0 2024-03-04T13:48:12.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.151+0000 7fd3cbe00700 1 --2- 172.21.15.107:0/4072434805 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fd3b806d900 0x7fd3b806fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:48:12.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.151+0000 7fd3cbe00700 1 -- 172.21.15.107:0/4072434805 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fd3c008e4e0 con 0x7fd3d41095f0 2024-03-04T13:48:12.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.151+0000 7fd3d3e00700 1 --2- 172.21.15.107:0/4072434805 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fd3b806d900 0x7fd3b806fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:48:12.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.152+0000 7fd3d3e00700 1 --2- 172.21.15.107:0/4072434805 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fd3b806d900 0x7fd3b806fdb0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd3c4006fd0 tx=0x7fd3c4008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:48:12.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.158+0000 7fd3cbe00700 1 -- 172.21.15.107:0/4072434805 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd3c005b6f0 con 0x7fd3d41095f0 2024-03-04T13:48:12.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.401+0000 7fd3dac00700 1 -- 172.21.15.107:0/4072434805 --> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd3d4114960 con 0x7fd3b806d900 2024-03-04T13:48:12.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.404+0000 7fd3cbe00700 1 -- 172.21.15.107:0/4072434805 <== mgr.34102 v2:172.21.15.107:6800/1223039812 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7fd3d4114960 con 0x7fd3b806d900 2024-03-04T13:48:12.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.407+0000 7fd3dac00700 1 -- 172.21.15.107:0/4072434805 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fd3b806d900 msgr2=0x7fd3b806fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:48:12.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.407+0000 7fd3dac00700 1 --2- 172.21.15.107:0/4072434805 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fd3b806d900 0x7fd3b806fdb0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd3c4006fd0 tx=0x7fd3c4008040 comp rx=0 tx=0).stop 2024-03-04T13:48:12.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.407+0000 7fd3dac00700 1 -- 172.21.15.107:0/4072434805 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd3d41095f0 msgr2=0x7fd3d411a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:48:12.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.407+0000 7fd3dac00700 1 --2- 172.21.15.107:0/4072434805 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd3d41095f0 0x7fd3d411a570 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd3c0009150 tx=0x7fd3c0009e30 comp rx=0 tx=0).stop 2024-03-04T13:48:12.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.407+0000 7fd3dac00700 1 -- 172.21.15.107:0/4072434805 shutdown_connections 2024-03-04T13:48:12.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.407+0000 7fd3dac00700 1 --2- 172.21.15.107:0/4072434805 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fd3b806d900 0x7fd3b806fdb0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:48:12.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.407+0000 7fd3dac00700 1 --2- 172.21.15.107:0/4072434805 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd3d4103dc0 0x7fd3d411a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:48:12.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.407+0000 7fd3dac00700 1 --2- 172.21.15.107:0/4072434805 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd3d41095f0 0x7fd3d411a570 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:48:12.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.407+0000 7fd3dac00700 1 -- 172.21.15.107:0/4072434805 >> 172.21.15.107:0/4072434805 conn(0x7fd3d4072d50 msgr2=0x7fd3d40746b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:48:12.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.407+0000 7fd3dac00700 1 -- 172.21.15.107:0/4072434805 shutdown_connections 2024-03-04T13:48:12.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.408+0000 7fd3dac00700 1 -- 172.21.15.107:0/4072434805 wait complete. 2024-03-04T13:48:12.422 INFO:teuthology.orchestra.run.smithi107.stdout:true 2024-03-04T13:48:12.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.571+0000 7fc6c1000700 1 -- 172.21.15.107:0/2657543776 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6bc103710 msgr2=0x7fc6bc0fcd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:48:12.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.571+0000 7fc6c1000700 1 --2- 172.21.15.107:0/2657543776 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6bc103710 0x7fc6bc0fcd80 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc6a8004940 tx=0x7fc6a8009e30 comp rx=0 tx=0).stop 2024-03-04T13:48:12.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.571+0000 7fc6c1000700 1 -- 172.21.15.107:0/2657543776 shutdown_connections 2024-03-04T13:48:12.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.571+0000 7fc6c1000700 1 --2- 172.21.15.107:0/2657543776 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6bc103710 0x7fc6bc0fcd80 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:48:12.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.571+0000 7fc6c1000700 1 --2- 172.21.15.107:0/2657543776 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc6bc102d70 0x7fc6bc103140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:48:12.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.571+0000 7fc6c1000700 1 -- 172.21.15.107:0/2657543776 >> 172.21.15.107:0/2657543776 conn(0x7fc6bc0f88e0 msgr2=0x7fc6bc0facf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:48:12.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.571+0000 7fc6c1000700 1 -- 172.21.15.107:0/2657543776 shutdown_connections 2024-03-04T13:48:12.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.571+0000 7fc6c1000700 1 -- 172.21.15.107:0/2657543776 wait complete. 2024-03-04T13:48:12.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.572+0000 7fc6c1000700 1 Processor -- start 2024-03-04T13:48:12.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.572+0000 7fc6c1000700 1 -- start start 2024-03-04T13:48:12.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.573+0000 7fc6c1000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6bc102d70 0x7fc6bc115b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:48:12.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.573+0000 7fc6c1000700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc6bc103710 0x7fc6bc116090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:48:12.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.573+0000 7fc6c1000700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6bc116770 con 0x7fc6bc102d70 2024-03-04T13:48:12.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.573+0000 7fc6c1000700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6bc10fa10 con 0x7fc6bc103710 2024-03-04T13:48:12.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.573+0000 7fc6bb400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6bc102d70 0x7fc6bc115b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:48:12.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.573+0000 7fc6bb400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6bc102d70 0x7fc6bc115b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:53368/0 (socket says 172.21.15.107:53368) 2024-03-04T13:48:12.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.573+0000 7fc6bb400700 1 -- 172.21.15.107:0/1766623034 learned_addr learned my addr 172.21.15.107:0/1766623034 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:48:12.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.573+0000 7fc6baa00700 1 --2- 172.21.15.107:0/1766623034 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc6bc103710 0x7fc6bc116090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:48:12.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.573+0000 7fc6bb400700 1 -- 172.21.15.107:0/1766623034 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc6bc103710 msgr2=0x7fc6bc116090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:48:12.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.573+0000 7fc6bb400700 1 --2- 172.21.15.107:0/1766623034 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc6bc103710 0x7fc6bc116090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:48:12.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.573+0000 7fc6bb400700 1 -- 172.21.15.107:0/1766623034 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6a8005040 con 0x7fc6bc102d70 2024-03-04T13:48:12.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.574+0000 7fc6bb400700 1 --2- 172.21.15.107:0/1766623034 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6bc102d70 0x7fc6bc115b50 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac00b790 tx=0x7fc6ac009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:48:12.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.574+0000 7fc6b3400700 1 -- 172.21.15.107:0/1766623034 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc6ac00f770 con 0x7fc6bc102d70 2024-03-04T13:48:12.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.574+0000 7fc6b3400700 1 -- 172.21.15.107:0/1766623034 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc6ac00fda0 con 0x7fc6bc102d70 2024-03-04T13:48:12.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.574+0000 7fc6c1000700 1 -- 172.21.15.107:0/1766623034 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6bc10fcf0 con 0x7fc6bc102d70 2024-03-04T13:48:12.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.574+0000 7fc6b3400700 1 -- 172.21.15.107:0/1766623034 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc6ac00d310 con 0x7fc6bc102d70 2024-03-04T13:48:12.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.574+0000 7fc6c1000700 1 -- 172.21.15.107:0/1766623034 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6bc110230 con 0x7fc6bc102d70 2024-03-04T13:48:12.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.576+0000 7fc6c1000700 1 -- 172.21.15.107:0/1766623034 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6bc05f5c0 con 0x7fc6bc102d70 2024-03-04T13:48:12.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.577+0000 7fc6b3400700 1 -- 172.21.15.107:0/1766623034 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 39) v1 ==== 91587+0+0 (secure 0 0 0) 0x7fc6ac010420 con 0x7fc6bc102d70 2024-03-04T13:48:12.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.578+0000 7fc6b3400700 1 --2- 172.21.15.107:0/1766623034 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fc6a006d900 0x7fc6a006fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:48:12.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.578+0000 7fc6b3400700 1 -- 172.21.15.107:0/1766623034 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fc6ac08d090 con 0x7fc6bc102d70 2024-03-04T13:48:12.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.578+0000 7fc6baa00700 1 --2- 172.21.15.107:0/1766623034 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fc6a006d900 0x7fc6a006fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:48:12.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.579+0000 7fc6baa00700 1 --2- 172.21.15.107:0/1766623034 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fc6a006d900 0x7fc6a006fdb0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc6a800b3b0 tx=0x7fc6a8009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:48:12.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.583+0000 7fc6b3400700 1 -- 172.21.15.107:0/1766623034 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc6ac05a330 con 0x7fc6bc102d70 2024-03-04T13:48:12.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.828+0000 7fc6c1000700 1 -- 172.21.15.107:0/1766623034 --> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc6bc1111f0 con 0x7fc6a006d900 2024-03-04T13:48:12.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.831+0000 7fc6b3400700 1 -- 172.21.15.107:0/1766623034 <== mgr.34102 v2:172.21.15.107:6800/1223039812 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7fc6bc1111f0 con 0x7fc6a006d900 2024-03-04T13:48:12.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.833+0000 7fc6c1000700 1 -- 172.21.15.107:0/1766623034 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fc6a006d900 msgr2=0x7fc6a006fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:48:12.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.833+0000 7fc6c1000700 1 --2- 172.21.15.107:0/1766623034 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fc6a006d900 0x7fc6a006fdb0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc6a800b3b0 tx=0x7fc6a8009e30 comp rx=0 tx=0).stop 2024-03-04T13:48:12.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.833+0000 7fc6c1000700 1 -- 172.21.15.107:0/1766623034 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6bc102d70 msgr2=0x7fc6bc115b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:48:12.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.833+0000 7fc6c1000700 1 --2- 172.21.15.107:0/1766623034 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6bc102d70 0x7fc6bc115b50 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac00b790 tx=0x7fc6ac009250 comp rx=0 tx=0).stop 2024-03-04T13:48:12.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.834+0000 7fc6c1000700 1 -- 172.21.15.107:0/1766623034 shutdown_connections 2024-03-04T13:48:12.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.834+0000 7fc6c1000700 1 --2- 172.21.15.107:0/1766623034 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fc6a006d900 0x7fc6a006fdb0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:48:12.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.834+0000 7fc6c1000700 1 --2- 172.21.15.107:0/1766623034 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6bc102d70 0x7fc6bc115b50 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:48:12.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.834+0000 7fc6c1000700 1 --2- 172.21.15.107:0/1766623034 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc6bc103710 0x7fc6bc116090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:48:12.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.834+0000 7fc6c1000700 1 -- 172.21.15.107:0/1766623034 >> 172.21.15.107:0/1766623034 conn(0x7fc6bc0f88e0 msgr2=0x7fc6bc0fa110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:48:12.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.834+0000 7fc6c1000700 1 -- 172.21.15.107:0/1766623034 shutdown_connections 2024-03-04T13:48:12.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.834+0000 7fc6c1000700 1 -- 172.21.15.107:0/1766623034 wait complete. 2024-03-04T13:48:12.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.985+0000 7fc490e00700 1 -- 172.21.15.107:0/633090567 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc48c070330 msgr2=0x7fc48c070700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:48:12.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.985+0000 7fc490e00700 1 --2- 172.21.15.107:0/633090567 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc48c070330 0x7fc48c070700 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc47c004970 tx=0x7fc47c009e30 comp rx=0 tx=0).stop 2024-03-04T13:48:12.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.985+0000 7fc490e00700 1 -- 172.21.15.107:0/633090567 shutdown_connections 2024-03-04T13:48:12.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.985+0000 7fc490e00700 1 --2- 172.21.15.107:0/633090567 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc48c070c40 0x7fc48c109050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:48:12.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.985+0000 7fc490e00700 1 --2- 172.21.15.107:0/633090567 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc48c070330 0x7fc48c070700 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:48:12.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.985+0000 7fc490e00700 1 -- 172.21.15.107:0/633090567 >> 172.21.15.107:0/633090567 conn(0x7fc48c0f88b0 msgr2=0x7fc48c0facc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:48:12.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.985+0000 7fc490e00700 1 -- 172.21.15.107:0/633090567 shutdown_connections 2024-03-04T13:48:12.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.985+0000 7fc490e00700 1 -- 172.21.15.107:0/633090567 wait complete. 2024-03-04T13:48:12.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.986+0000 7fc490e00700 1 Processor -- start 2024-03-04T13:48:12.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.986+0000 7fc490e00700 1 -- start start 2024-03-04T13:48:12.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.987+0000 7fc490e00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc48c070330 0x7fc48c1159d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:48:12.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.987+0000 7fc490e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc48c070c40 0x7fc48c115f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:48:12.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.987+0000 7fc490e00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc48c116560 con 0x7fc48c070c40 2024-03-04T13:48:12.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.987+0000 7fc490e00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc48c10f890 con 0x7fc48c070330 2024-03-04T13:48:12.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.987+0000 7fc48aa00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc48c070c40 0x7fc48c115f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:48:12.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.987+0000 7fc48aa00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc48c070c40 0x7fc48c115f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:53384/0 (socket says 172.21.15.107:53384) 2024-03-04T13:48:12.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.987+0000 7fc48aa00700 1 -- 172.21.15.107:0/645572348 learned_addr learned my addr 172.21.15.107:0/645572348 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:48:12.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.987+0000 7fc48b400700 1 --2- 172.21.15.107:0/645572348 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc48c070330 0x7fc48c1159d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:48:12.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.987+0000 7fc48aa00700 1 -- 172.21.15.107:0/645572348 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc48c070330 msgr2=0x7fc48c1159d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:48:12.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.987+0000 7fc48aa00700 1 --2- 172.21.15.107:0/645572348 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc48c070330 0x7fc48c1159d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:48:12.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.987+0000 7fc48aa00700 1 -- 172.21.15.107:0/645572348 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc47c005040 con 0x7fc48c070c40 2024-03-04T13:48:12.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.988+0000 7fc48aa00700 1 --2- 172.21.15.107:0/645572348 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc48c070c40 0x7fc48c115f10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc47800c830 tx=0x7fc478004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:48:12.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.988+0000 7fc483400700 1 -- 172.21.15.107:0/645572348 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc47800a760 con 0x7fc48c070c40 2024-03-04T13:48:12.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.988+0000 7fc483400700 1 -- 172.21.15.107:0/645572348 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc47800ad90 con 0x7fc48c070c40 2024-03-04T13:48:12.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.988+0000 7fc490e00700 1 -- 172.21.15.107:0/645572348 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc48c10fb70 con 0x7fc48c070c40 2024-03-04T13:48:12.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.988+0000 7fc483400700 1 -- 172.21.15.107:0/645572348 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc4780113a0 con 0x7fc48c070c40 2024-03-04T13:48:12.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.988+0000 7fc490e00700 1 -- 172.21.15.107:0/645572348 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc48c1100b0 con 0x7fc48c070c40 2024-03-04T13:48:12.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.990+0000 7fc490e00700 1 -- 172.21.15.107:0/645572348 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc48c106710 con 0x7fc48c070c40 2024-03-04T13:48:12.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.990+0000 7fc483400700 1 -- 172.21.15.107:0/645572348 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 39) v1 ==== 91587+0+0 (secure 0 0 0) 0x7fc478012400 con 0x7fc48c070c40 2024-03-04T13:48:12.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.991+0000 7fc483400700 1 --2- 172.21.15.107:0/645572348 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fc47006d900 0x7fc47006fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:48:12.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.991+0000 7fc483400700 1 -- 172.21.15.107:0/645572348 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fc47808cff0 con 0x7fc48c070c40 2024-03-04T13:48:12.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.992+0000 7fc48b400700 1 --2- 172.21.15.107:0/645572348 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fc47006d900 0x7fc47006fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:48:12.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.992+0000 7fc48b400700 1 --2- 172.21.15.107:0/645572348 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fc47006d900 0x7fc47006fdb0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc47c000c00 tx=0x7fc47c009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:48:12.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:12.996+0000 7fc483400700 1 -- 172.21.15.107:0/645572348 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc47805a350 con 0x7fc48c070c40 2024-03-04T13:48:13.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:13.218+0000 7fc490e00700 1 -- 172.21.15.107:0/645572348 --> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fc48c1156a0 con 0x7fc47006d900 2024-03-04T13:48:13.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:13.229+0000 7fc483400700 1 -- 172.21.15.107:0/645572348 <== mgr.34102 v2:172.21.15.107:6800/1223039812 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fc48c1156a0 con 0x7fc47006d900 2024-03-04T13:48:13.229 INFO:teuthology.orchestra.run.smithi107.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-04T13:48:13.229 INFO:teuthology.orchestra.run.smithi107.stdout:alertmanager.smithi107 smithi107 *:9093,9094 running (2m) 41s ago 16m 14.6M - 0.25.0 c8568f914cd2 e5ed7be32538 2024-03-04T13:48:13.229 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi107 smithi107 running (16m) 41s ago 16m 6665k - 17.2.7-545-g42608bb1 9eec3f22859d 03e1570c2eff 2024-03-04T13:48:13.229 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi169 smithi169 running (15m) 17s ago 15m 6656k - 17.2.7-545-g42608bb1 9eec3f22859d 9c8881a5338e 2024-03-04T13:48:13.229 INFO:teuthology.orchestra.run.smithi107.stdout:grafana.smithi107 smithi107 *:3000 running (2m) 41s ago 16m 74.2M - 9.4.12 a0ea5e3c46d9 c5e4d33fb599 2024-03-04T13:48:13.229 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.riexmm smithi107 running (9m) 41s ago 9m 14.1M - 17.2.7-545-g42608bb1 9eec3f22859d fc921f7d4328 2024-03-04T13:48:13.229 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.thycho smithi107 running (9m) 41s ago 9m 98.2M - 17.2.7-545-g42608bb1 9eec3f22859d 4069c6e0e063 2024-03-04T13:48:13.229 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi169.bxitzk smithi169 running (9m) 17s ago 9m 14.0M - 17.2.7-545-g42608bb1 9eec3f22859d 7d4115228c7c 2024-03-04T13:48:13.229 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi169.jusudl smithi169 running (9m) 17s ago 9m 145M - 17.2.7-545-g42608bb1 9eec3f22859d 2d2f2cc452e8 2024-03-04T13:48:13.229 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi107.wpzgwl smithi107 *:8443,9283,8765 running (5m) 41s ago 18m 476M - 19.0.0-1845-g717ce3cc d71b839abdf8 1c5fdee45eac 2024-03-04T13:48:13.229 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi169.eihczr smithi169 *:8443,9283,8765 running (3m) 17s ago 15m 448M - 19.0.0-1845-g717ce3cc d71b839abdf8 b5cac2b7d22a 2024-03-04T13:48:13.230 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi107 smithi107 running (59s) 41s ago 18m 28.3M 2048M 19.0.0-1845-g717ce3cc d71b839abdf8 32f07144274f 2024-03-04T13:48:13.230 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi169 smithi169 running (23s) 17s ago 15m 18.5M 2048M 19.0.0-1845-g717ce3cc d71b839abdf8 e3527449381a 2024-03-04T13:48:13.230 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi107 smithi107 *:9100 running (3m) 41s ago 16m 7650k - 1.5.0 0da6a335fe13 8a8d3d4f724f 2024-03-04T13:48:13.230 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi169 smithi169 *:9100 running (3m) 17s ago 15m 7457k - 1.5.0 0da6a335fe13 f240f5a975bc 2024-03-04T13:48:13.230 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0 smithi107 running (14m) 41s ago 14m 206M 1558M 17.2.7-545-g42608bb1 9eec3f22859d c0605b68f696 2024-03-04T13:48:13.230 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1 smithi107 running (13m) 41s ago 13m 220M 1558M 17.2.7-545-g42608bb1 9eec3f22859d 5fb2c0dd5769 2024-03-04T13:48:13.230 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2 smithi107 running (13m) 41s ago 13m 182M 1558M 17.2.7-545-g42608bb1 9eec3f22859d 61f30f643549 2024-03-04T13:48:13.230 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3 smithi169 running (12m) 17s ago 12m 235M 2582M 17.2.7-545-g42608bb1 9eec3f22859d cd91d8694764 2024-03-04T13:48:13.230 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4 smithi169 running (11m) 17s ago 11m 207M 2582M 17.2.7-545-g42608bb1 9eec3f22859d 08545e3d0659 2024-03-04T13:48:13.230 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5 smithi169 running (11m) 17s ago 11m 194M 2582M 17.2.7-545-g42608bb1 9eec3f22859d dcffe6cb3eac 2024-03-04T13:48:13.230 INFO:teuthology.orchestra.run.smithi107.stdout:prometheus.smithi107 smithi107 *:9095 running (2m) 41s ago 15m 50.2M - 2.43.0 a07b618ecd1d 1eeb1ad5cb75 2024-03-04T13:48:13.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:13.232+0000 7fc490e00700 1 -- 172.21.15.107:0/645572348 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fc47006d900 msgr2=0x7fc47006fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:48:13.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:13.232+0000 7fc490e00700 1 --2- 172.21.15.107:0/645572348 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fc47006d900 0x7fc47006fdb0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc47c000c00 tx=0x7fc47c009e30 comp rx=0 tx=0).stop 2024-03-04T13:48:13.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:13.232+0000 7fc490e00700 1 -- 172.21.15.107:0/645572348 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc48c070c40 msgr2=0x7fc48c115f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:48:13.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:13.232+0000 7fc490e00700 1 --2- 172.21.15.107:0/645572348 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc48c070c40 0x7fc48c115f10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc47800c830 tx=0x7fc478004470 comp rx=0 tx=0).stop 2024-03-04T13:48:13.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:13.232+0000 7fc490e00700 1 -- 172.21.15.107:0/645572348 shutdown_connections 2024-03-04T13:48:13.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:13.232+0000 7fc490e00700 1 --2- 172.21.15.107:0/645572348 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fc47006d900 0x7fc47006fdb0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:48:13.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:13.232+0000 7fc490e00700 1 --2- 172.21.15.107:0/645572348 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc48c070330 0x7fc48c1159d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:48:13.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:13.232+0000 7fc490e00700 1 --2- 172.21.15.107:0/645572348 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc48c070c40 0x7fc48c115f10 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:48:13.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:13.232+0000 7fc490e00700 1 -- 172.21.15.107:0/645572348 >> 172.21.15.107:0/645572348 conn(0x7fc48c0f88b0 msgr2=0x7fc48c103890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:48:13.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:13.233+0000 7fc490e00700 1 -- 172.21.15.107:0/645572348 shutdown_connections 2024-03-04T13:48:13.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:13.233+0000 7fc490e00700 1 -- 172.21.15.107:0/645572348 wait complete. 2024-03-04T13:48:13.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:13.374+0000 7fafd4e00700 1 -- 172.21.15.107:0/2934786880 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fafd00702d0 msgr2=0x7fafd00706a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:48:13.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:13.374+0000 7fafd4e00700 1 --2- 172.21.15.107:0/2934786880 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fafd00702d0 0x7fafd00706a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fafc0004970 tx=0x7fafc0009e30 comp rx=0 tx=0).stop 2024-03-04T13:48:13.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:13.374+0000 7fafd4e00700 1 -- 172.21.15.107:0/2934786880 shutdown_connections 2024-03-04T13:48:13.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:13.374+0000 7fafd4e00700 1 --2- 172.21.15.107:0/2934786880 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fafd0070be0 0x7fafd01092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:48:13.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:13.374+0000 7fafd4e00700 1 --2- 172.21.15.107:0/2934786880 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fafd00702d0 0x7fafd00706a0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:48:13.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:13.374+0000 7fafd4e00700 1 -- 172.21.15.107:0/2934786880 >> 172.21.15.107:0/2934786880 conn(0x7fafd00f8ae0 msgr2=0x7fafd00faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:48:13.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:13.374+0000 7fafd4e00700 1 -- 172.21.15.107:0/2934786880 shutdown_connections 2024-03-04T13:48:13.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:13.374+0000 7fafd4e00700 1 -- 172.21.15.107:0/2934786880 wait complete. 2024-03-04T13:48:13.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:13.375+0000 7fafd4e00700 1 Processor -- start 2024-03-04T13:48:13.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:13.375+0000 7fafd4e00700 1 -- start start 2024-03-04T13:48:13.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:13.375+0000 7fafd4e00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fafd00702d0 0x7fafd0115ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:48:13.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:13.376+0000 7fafd4e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fafd0070be0 0x7fafd01161e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:48:13.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:13.376+0000 7fafd4e00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafd01168c0 con 0x7fafd0070be0 2024-03-04T13:48:13.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:13.376+0000 7fafd4e00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafd010fb40 con 0x7fafd00702d0 2024-03-04T13:48:13.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:13.376+0000 7fafcd400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fafd0070be0 0x7fafd01161e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:48:13.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:13.376+0000 7fafcd400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fafd0070be0 0x7fafd01161e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:53400/0 (socket says 172.21.15.107:53400) 2024-03-04T13:48:13.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:13.376+0000 7fafcd400700 1 -- 172.21.15.107:0/4137779195 learned_addr learned my addr 172.21.15.107:0/4137779195 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:48:13.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:13.376+0000 7fafcde00700 1 --2- 172.21.15.107:0/4137779195 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fafd00702d0 0x7fafd0115ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:48:13.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:13.376+0000 7fafcd400700 1 -- 172.21.15.107:0/4137779195 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fafd00702d0 msgr2=0x7fafd0115ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:48:13.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:13.376+0000 7fafcd400700 1 --2- 172.21.15.107:0/4137779195 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fafd00702d0 0x7fafd0115ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:48:13.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:13.376+0000 7fafcd400700 1 -- 172.21.15.107:0/4137779195 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafc0005040 con 0x7fafd0070be0 2024-03-04T13:48:13.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:13.377+0000 7fafcd400700 1 --2- 172.21.15.107:0/4137779195 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fafd0070be0 0x7fafd01161e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fafbc00b790 tx=0x7fafbc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:48:13.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:13.377+0000 7fafc6000700 1 -- 172.21.15.107:0/4137779195 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fafbc00f770 con 0x7fafd0070be0 2024-03-04T13:48:13.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:13.377+0000 7fafd4e00700 1 -- 172.21.15.107:0/4137779195 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafd010fdc0 con 0x7fafd0070be0 2024-03-04T13:48:13.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:13.377+0000 7fafc6000700 1 -- 172.21.15.107:0/4137779195 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fafbc00fda0 con 0x7fafd0070be0 2024-03-04T13:48:13.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:13.377+0000 7fafc6000700 1 -- 172.21.15.107:0/4137779195 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fafbc00d310 con 0x7fafd0070be0 2024-03-04T13:48:13.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:13.377+0000 7fafd4e00700 1 -- 172.21.15.107:0/4137779195 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafd0110300 con 0x7fafd0070be0 2024-03-04T13:48:13.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:13.378+0000 7fafd4e00700 1 -- 172.21.15.107:0/4137779195 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafd0106980 con 0x7fafd0070be0 2024-03-04T13:48:13.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:13.379+0000 7fafc6000700 1 -- 172.21.15.107:0/4137779195 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 39) v1 ==== 91587+0+0 (secure 0 0 0) 0x7fafbc010420 con 0x7fafd0070be0 2024-03-04T13:48:13.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:13.380+0000 7fafc6000700 1 --2- 172.21.15.107:0/4137779195 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fafb406d900 0x7fafb406fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:48:13.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:13.380+0000 7fafc6000700 1 -- 172.21.15.107:0/4137779195 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fafbc08d040 con 0x7fafd0070be0 2024-03-04T13:48:13.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:13.380+0000 7fafcde00700 1 --2- 172.21.15.107:0/4137779195 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fafb406d900 0x7fafb406fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:48:13.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:13.381+0000 7fafcde00700 1 --2- 172.21.15.107:0/4137779195 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fafb406d900 0x7fafb406fdb0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fafc0000c00 tx=0x7fafc0009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:48:13.384 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:13.384+0000 7fafc6000700 1 -- 172.21.15.107:0/4137779195 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fafbc05a380 con 0x7fafd0070be0 2024-03-04T13:48:13.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:13.691+0000 7fafd4e00700 1 -- 172.21.15.107:0/4137779195 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fafd00fc6e0 con 0x7fafd0070be0 2024-03-04T13:48:13.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:13.692+0000 7fafc6000700 1 -- 172.21.15.107:0/4137779195 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+736 (secure 0 0 0) 0x7fafbc059f10 con 0x7fafd0070be0 2024-03-04T13:48:13.692 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-03-04T13:48:13.692 INFO:teuthology.orchestra.run.smithi107.stdout: "mon": { 2024-03-04T13:48:13.692 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-1845-g717ce3cc (717ce3cce8d0166e4e577faa8004e6f9cb4128c0) squid (dev)": 2 2024-03-04T13:48:13.693 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:48:13.693 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr": { 2024-03-04T13:48:13.693 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-1845-g717ce3cc (717ce3cce8d0166e4e577faa8004e6f9cb4128c0) squid (dev)": 2 2024-03-04T13:48:13.693 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:48:13.693 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": { 2024-03-04T13:48:13.693 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-545-g42608bb1 (42608bb191117be10dba720e6bf8a02f3171cd34) quincy (stable)": 6 2024-03-04T13:48:13.693 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:48:13.693 INFO:teuthology.orchestra.run.smithi107.stdout: "mds": { 2024-03-04T13:48:13.693 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-545-g42608bb1 (42608bb191117be10dba720e6bf8a02f3171cd34) quincy (stable)": 4 2024-03-04T13:48:13.693 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:48:13.693 INFO:teuthology.orchestra.run.smithi107.stdout: "overall": { 2024-03-04T13:48:13.693 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-545-g42608bb1 (42608bb191117be10dba720e6bf8a02f3171cd34) quincy (stable)": 10, 2024-03-04T13:48:13.693 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-1845-g717ce3cc (717ce3cce8d0166e4e577faa8004e6f9cb4128c0) squid (dev)": 4 2024-03-04T13:48:13.694 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-04T13:48:13.694 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-03-04T13:48:13.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:13.695+0000 7fafd4e00700 1 -- 172.21.15.107:0/4137779195 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fafb406d900 msgr2=0x7fafb406fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:48:13.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:13.695+0000 7fafd4e00700 1 --2- 172.21.15.107:0/4137779195 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fafb406d900 0x7fafb406fdb0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fafc0000c00 tx=0x7fafc0009e30 comp rx=0 tx=0).stop 2024-03-04T13:48:13.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:13.695+0000 7fafd4e00700 1 -- 172.21.15.107:0/4137779195 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fafd0070be0 msgr2=0x7fafd01161e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:48:13.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:13.695+0000 7fafd4e00700 1 --2- 172.21.15.107:0/4137779195 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fafd0070be0 0x7fafd01161e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fafbc00b790 tx=0x7fafbc009250 comp rx=0 tx=0).stop 2024-03-04T13:48:13.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:13.695+0000 7fafd4e00700 1 -- 172.21.15.107:0/4137779195 shutdown_connections 2024-03-04T13:48:13.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:13.695+0000 7fafd4e00700 1 --2- 172.21.15.107:0/4137779195 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fafb406d900 0x7fafb406fdb0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:48:13.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:13.695+0000 7fafd4e00700 1 --2- 172.21.15.107:0/4137779195 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fafd00702d0 0x7fafd0115ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:48:13.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:13.695+0000 7fafd4e00700 1 --2- 172.21.15.107:0/4137779195 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fafd0070be0 0x7fafd01161e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:48:13.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:13.695+0000 7fafd4e00700 1 -- 172.21.15.107:0/4137779195 >> 172.21.15.107:0/4137779195 conn(0x7fafd00f8ae0 msgr2=0x7fafd0103b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:48:13.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:13.695+0000 7fafd4e00700 1 -- 172.21.15.107:0/4137779195 shutdown_connections 2024-03-04T13:48:13.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:13.695+0000 7fafd4e00700 1 -- 172.21.15.107:0/4137779195 wait complete. 2024-03-04T13:48:13.821 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:13 smithi107 ceph-mon[112578]: from='client.34158 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:48:13.822 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:13 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:48:13.822 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:13 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:48:13.822 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:13 smithi107 ceph-mon[112578]: Reconfiguring osd.1 (monmap changed)... 2024-03-04T13:48:13.822 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:13 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-04T13:48:13.822 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:13 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:48:13.822 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:13 smithi107 ceph-mon[112578]: Reconfiguring daemon osd.1 on smithi107 2024-03-04T13:48:13.822 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:13 smithi107 ceph-mon[112578]: pgmap v27: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:48:13.822 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:13 smithi107 ceph-mon[112578]: from='client.34162 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:48:13.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:13.838+0000 7f3ff5800700 1 -- 172.21.15.107:0/2034326856 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ff00fe710 msgr2=0x7f3ff0106d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:48:13.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:13.838+0000 7f3ff5800700 1 --2- 172.21.15.107:0/2034326856 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ff00fe710 0x7f3ff0106d50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc004940 tx=0x7f3fdc009e30 comp rx=0 tx=0).stop 2024-03-04T13:48:13.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:13.838+0000 7f3ff5800700 1 -- 172.21.15.107:0/2034326856 shutdown_connections 2024-03-04T13:48:13.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:13.838+0000 7f3ff5800700 1 --2- 172.21.15.107:0/2034326856 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ff00fe710 0x7f3ff0106d50 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:48:13.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:13.838+0000 7f3ff5800700 1 --2- 172.21.15.107:0/2034326856 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ff00fdd70 0x7f3ff00fe140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:48:13.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:13.838+0000 7f3ff5800700 1 -- 172.21.15.107:0/2034326856 >> 172.21.15.107:0/2034326856 conn(0x7f3ff00f77e0 msgr2=0x7f3ff00f9bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:48:13.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:13.838+0000 7f3ff5800700 1 -- 172.21.15.107:0/2034326856 shutdown_connections 2024-03-04T13:48:13.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:13.838+0000 7f3ff5800700 1 -- 172.21.15.107:0/2034326856 wait complete. 2024-03-04T13:48:13.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:13.839+0000 7f3ff5800700 1 Processor -- start 2024-03-04T13:48:13.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:13.839+0000 7f3ff5800700 1 -- start start 2024-03-04T13:48:13.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:13.839+0000 7f3ff5800700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ff00fdd70 0x7f3ff00fbba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:48:13.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:13.839+0000 7f3ff5800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ff00fe710 0x7f3ff00fc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:48:13.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:13.839+0000 7f3ff5800700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff00fc770 con 0x7f3ff00fe710 2024-03-04T13:48:13.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:13.839+0000 7f3ff5800700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff00fc8e0 con 0x7f3ff00fdd70 2024-03-04T13:48:13.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:13.839+0000 7f3fef400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ff00fe710 0x7f3ff00fc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:48:13.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:13.840+0000 7f3fef400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ff00fe710 0x7f3ff00fc0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:53422/0 (socket says 172.21.15.107:53422) 2024-03-04T13:48:13.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:13.840+0000 7f3fef400700 1 -- 172.21.15.107:0/3448348634 learned_addr learned my addr 172.21.15.107:0/3448348634 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:48:13.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:13.840+0000 7f3fef400700 1 -- 172.21.15.107:0/3448348634 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ff00fdd70 msgr2=0x7f3ff00fbba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:48:13.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:13.840+0000 7f3fef400700 1 --2- 172.21.15.107:0/3448348634 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ff00fdd70 0x7f3ff00fbba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:48:13.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:13.840+0000 7f3fef400700 1 -- 172.21.15.107:0/3448348634 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fdc005040 con 0x7f3ff00fe710 2024-03-04T13:48:13.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:13.840+0000 7f3fef400700 1 --2- 172.21.15.107:0/3448348634 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ff00fe710 0x7f3ff00fc0e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc000c00 tx=0x7f3fdc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:48:13.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:13.840+0000 7f3fe7e00700 1 -- 172.21.15.107:0/3448348634 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3fdc00e040 con 0x7f3ff00fe710 2024-03-04T13:48:13.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:13.840+0000 7f3fe7e00700 1 -- 172.21.15.107:0/3448348634 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3fdc009af0 con 0x7f3ff00fe710 2024-03-04T13:48:13.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:13.840+0000 7f3ff5800700 1 -- 172.21.15.107:0/3448348634 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ff0119b00 con 0x7f3ff00fe710 2024-03-04T13:48:13.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:13.840+0000 7f3fe7e00700 1 -- 172.21.15.107:0/3448348634 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3fdc01c630 con 0x7f3ff00fe710 2024-03-04T13:48:13.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:13.841+0000 7f3ff5800700 1 -- 172.21.15.107:0/3448348634 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ff0119ff0 con 0x7f3ff00fe710 2024-03-04T13:48:13.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:13.842+0000 7f3ff5800700 1 -- 172.21.15.107:0/3448348634 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ff0104360 con 0x7f3ff00fe710 2024-03-04T13:48:13.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:13.842+0000 7f3fe7e00700 1 -- 172.21.15.107:0/3448348634 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 39) v1 ==== 91587+0+0 (secure 0 0 0) 0x7f3fdc00bbf0 con 0x7f3ff00fe710 2024-03-04T13:48:13.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:13.843+0000 7f3fe7e00700 1 --2- 172.21.15.107:0/3448348634 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f3fd406d900 0x7f3fd406fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:48:13.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:13.843+0000 7f3fe7e00700 1 -- 172.21.15.107:0/3448348634 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f3fdc08e530 con 0x7f3ff00fe710 2024-03-04T13:48:13.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:13.844+0000 7f3fefe00700 1 --2- 172.21.15.107:0/3448348634 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f3fd406d900 0x7f3fd406fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:48:13.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:13.844+0000 7f3fefe00700 1 --2- 172.21.15.107:0/3448348634 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f3fd406d900 0x7f3fd406fdb0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f3fe0006fd0 tx=0x7f3fe0006db0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:48:13.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:13.848+0000 7f3fe7e00700 1 -- 172.21.15.107:0/3448348634 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3fdc057b90 con 0x7f3ff00fe710 2024-03-04T13:48:13.861 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:13 smithi169 ceph-mon[86183]: from='client.34158 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:48:13.861 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:13 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:48:13.861 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:13 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:48:13.861 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:13 smithi169 ceph-mon[86183]: Reconfiguring osd.1 (monmap changed)... 2024-03-04T13:48:13.861 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:13 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-04T13:48:13.861 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:13 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:48:13.861 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:13 smithi169 ceph-mon[86183]: Reconfiguring daemon osd.1 on smithi107 2024-03-04T13:48:13.861 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:13 smithi169 ceph-mon[86183]: pgmap v27: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:48:13.861 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:13 smithi169 ceph-mon[86183]: from='client.34162 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:48:14.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.096+0000 7f3ff5800700 1 -- 172.21.15.107:0/3448348634 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f3ff004bed0 con 0x7f3ff00fe710 2024-03-04T13:48:14.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.097+0000 7f3fe7e00700 1 -- 172.21.15.107:0/3448348634 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 16 v16) v1 ==== 76+0+1880 (secure 0 0 0) 0x7f3fdc05b250 con 0x7f3ff00fe710 2024-03-04T13:48:14.097 INFO:teuthology.orchestra.run.smithi107.stdout:e16 2024-03-04T13:48:14.097 INFO:teuthology.orchestra.run.smithi107.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-04T13:48:14.097 INFO:teuthology.orchestra.run.smithi107.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-04T13:48:14.098 INFO:teuthology.orchestra.run.smithi107.stdout:legacy client fscid: 1 2024-03-04T13:48:14.098 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:48:14.098 INFO:teuthology.orchestra.run.smithi107.stdout:Filesystem 'cephfs' (1) 2024-03-04T13:48:14.098 INFO:teuthology.orchestra.run.smithi107.stdout:fs_name cephfs 2024-03-04T13:48:14.098 INFO:teuthology.orchestra.run.smithi107.stdout:epoch 15 2024-03-04T13:48:14.098 INFO:teuthology.orchestra.run.smithi107.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-04T13:48:14.098 INFO:teuthology.orchestra.run.smithi107.stdout:created 2024-03-04T13:38:30.854004+0000 2024-03-04T13:48:14.098 INFO:teuthology.orchestra.run.smithi107.stdout:modified 2024-03-04T13:38:50.015989+0000 2024-03-04T13:48:14.098 INFO:teuthology.orchestra.run.smithi107.stdout:tableserver 0 2024-03-04T13:48:14.098 INFO:teuthology.orchestra.run.smithi107.stdout:root 0 2024-03-04T13:48:14.098 INFO:teuthology.orchestra.run.smithi107.stdout:session_timeout 60 2024-03-04T13:48:14.098 INFO:teuthology.orchestra.run.smithi107.stdout:session_autoclose 300 2024-03-04T13:48:14.098 INFO:teuthology.orchestra.run.smithi107.stdout:max_file_size 1099511627776 2024-03-04T13:48:14.098 INFO:teuthology.orchestra.run.smithi107.stdout:max_xattr_size 65536 2024-03-04T13:48:14.099 INFO:teuthology.orchestra.run.smithi107.stdout:required_client_features {} 2024-03-04T13:48:14.099 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure 0 2024-03-04T13:48:14.099 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure_osd_epoch 47 2024-03-04T13:48:14.099 INFO:teuthology.orchestra.run.smithi107.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-04T13:48:14.099 INFO:teuthology.orchestra.run.smithi107.stdout:max_mds 1 2024-03-04T13:48:14.099 INFO:teuthology.orchestra.run.smithi107.stdout:in 0 2024-03-04T13:48:14.099 INFO:teuthology.orchestra.run.smithi107.stdout:up {0=24247} 2024-03-04T13:48:14.099 INFO:teuthology.orchestra.run.smithi107.stdout:failed 2024-03-04T13:48:14.099 INFO:teuthology.orchestra.run.smithi107.stdout:damaged 2024-03-04T13:48:14.099 INFO:teuthology.orchestra.run.smithi107.stdout:stopped 2024-03-04T13:48:14.099 INFO:teuthology.orchestra.run.smithi107.stdout:data_pools [3] 2024-03-04T13:48:14.099 INFO:teuthology.orchestra.run.smithi107.stdout:metadata_pool 2 2024-03-04T13:48:14.099 INFO:teuthology.orchestra.run.smithi107.stdout:inline_data enabled 2024-03-04T13:48:14.099 INFO:teuthology.orchestra.run.smithi107.stdout:balancer 2024-03-04T13:48:14.100 INFO:teuthology.orchestra.run.smithi107.stdout:bal_rank_mask -1 2024-03-04T13:48:14.100 INFO:teuthology.orchestra.run.smithi107.stdout:standby_count_wanted 1 2024-03-04T13:48:14.100 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi169.jusudl{0:24247} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.169:6824/2994866076,v1:172.21.15.169:6825/2994866076] compat {c=[1],r=[1],i=[7ff]}] 2024-03-04T13:48:14.100 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.thycho{0:14470} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.107:6828/1985577060,v1:172.21.15.107:6829/1985577060] compat {c=[1],r=[1],i=[7ff]}] 2024-03-04T13:48:14.100 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:48:14.100 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:48:14.100 INFO:teuthology.orchestra.run.smithi107.stdout:Standby daemons: 2024-03-04T13:48:14.100 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:48:14.100 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.riexmm{-1:14480} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.107:6826/3261622517,v1:172.21.15.107:6827/3261622517] compat {c=[1],r=[1],i=[7ff]}] 2024-03-04T13:48:14.100 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi169.bxitzk{-1:24259} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.169:6826/3696090493,v1:172.21.15.169:6827/3696090493] compat {c=[1],r=[1],i=[7ff]}] 2024-03-04T13:48:14.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.100+0000 7f3ff5800700 1 -- 172.21.15.107:0/3448348634 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f3fd406d900 msgr2=0x7f3fd406fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:48:14.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.100+0000 7f3ff5800700 1 --2- 172.21.15.107:0/3448348634 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f3fd406d900 0x7f3fd406fdb0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f3fe0006fd0 tx=0x7f3fe0006db0 comp rx=0 tx=0).stop 2024-03-04T13:48:14.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.100+0000 7f3ff5800700 1 -- 172.21.15.107:0/3448348634 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ff00fe710 msgr2=0x7f3ff00fc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:48:14.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.100+0000 7f3ff5800700 1 --2- 172.21.15.107:0/3448348634 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ff00fe710 0x7f3ff00fc0e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc000c00 tx=0x7f3fdc009e30 comp rx=0 tx=0).stop 2024-03-04T13:48:14.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.100+0000 7f3ff5800700 1 -- 172.21.15.107:0/3448348634 shutdown_connections 2024-03-04T13:48:14.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.100+0000 7f3ff5800700 1 --2- 172.21.15.107:0/3448348634 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f3fd406d900 0x7f3fd406fdb0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:48:14.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.100+0000 7f3ff5800700 1 --2- 172.21.15.107:0/3448348634 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ff00fdd70 0x7f3ff00fbba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:48:14.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.100+0000 7f3ff5800700 1 --2- 172.21.15.107:0/3448348634 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ff00fe710 0x7f3ff00fc0e0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:48:14.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.100+0000 7f3ff5800700 1 -- 172.21.15.107:0/3448348634 >> 172.21.15.107:0/3448348634 conn(0x7f3ff00f77e0 msgr2=0x7f3ff01014e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:48:14.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.100+0000 7f3ff5800700 1 -- 172.21.15.107:0/3448348634 shutdown_connections 2024-03-04T13:48:14.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.100+0000 7f3ff5800700 1 -- 172.21.15.107:0/3448348634 wait complete. 2024-03-04T13:48:14.102 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 16 2024-03-04T13:48:14.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.250+0000 7f293a800700 1 -- 172.21.15.107:0/395243114 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2934105920 msgr2=0x7f29340fef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:48:14.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.250+0000 7f293a800700 1 --2- 172.21.15.107:0/395243114 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2934105920 0x7f29340fef90 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2920004940 tx=0x7f2920009e30 comp rx=0 tx=0).stop 2024-03-04T13:48:14.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.250+0000 7f293a800700 1 -- 172.21.15.107:0/395243114 shutdown_connections 2024-03-04T13:48:14.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.250+0000 7f293a800700 1 --2- 172.21.15.107:0/395243114 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2934105920 0x7f29340fef90 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:48:14.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.250+0000 7f293a800700 1 --2- 172.21.15.107:0/395243114 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2934104f80 0x7f2934105350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:48:14.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.250+0000 7f293a800700 1 -- 172.21.15.107:0/395243114 >> 172.21.15.107:0/395243114 conn(0x7f29340fab10 msgr2=0x7f29340fcf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:48:14.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.250+0000 7f293a800700 1 -- 172.21.15.107:0/395243114 shutdown_connections 2024-03-04T13:48:14.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.250+0000 7f293a800700 1 -- 172.21.15.107:0/395243114 wait complete. 2024-03-04T13:48:14.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.251+0000 7f293a800700 1 Processor -- start 2024-03-04T13:48:14.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.251+0000 7f293a800700 1 -- start start 2024-03-04T13:48:14.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.251+0000 7f293a800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2934104f80 0x7f2934115a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:48:14.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.251+0000 7f293a800700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2934105920 0x7f2934115fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:48:14.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.251+0000 7f293a800700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2934116600 con 0x7f2934104f80 2024-03-04T13:48:14.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.251+0000 7f293a800700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f293410f930 con 0x7f2934105920 2024-03-04T13:48:14.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.252+0000 7f2939400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2934104f80 0x7f2934115a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:48:14.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.252+0000 7f2939400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2934104f80 0x7f2934115a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:53434/0 (socket says 172.21.15.107:53434) 2024-03-04T13:48:14.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.252+0000 7f2939400700 1 -- 172.21.15.107:0/4269471521 learned_addr learned my addr 172.21.15.107:0/4269471521 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:48:14.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.252+0000 7f2939400700 1 -- 172.21.15.107:0/4269471521 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2934105920 msgr2=0x7f2934115fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:48:14.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.252+0000 7f2938a00700 1 --2- 172.21.15.107:0/4269471521 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2934105920 0x7f2934115fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:48:14.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.252+0000 7f2939400700 1 --2- 172.21.15.107:0/4269471521 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2934105920 0x7f2934115fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:48:14.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.252+0000 7f2939400700 1 -- 172.21.15.107:0/4269471521 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2920005040 con 0x7f2934104f80 2024-03-04T13:48:14.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.253+0000 7f2939400700 1 --2- 172.21.15.107:0/4269471521 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2934104f80 0x7f2934115a70 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f292800c810 tx=0x7f2928004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:48:14.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.253+0000 7f2931600700 1 -- 172.21.15.107:0/4269471521 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2928012810 con 0x7f2934104f80 2024-03-04T13:48:14.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.253+0000 7f2931600700 1 -- 172.21.15.107:0/4269471521 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2928012e40 con 0x7f2934104f80 2024-03-04T13:48:14.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.253+0000 7f2931600700 1 -- 172.21.15.107:0/4269471521 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f292800a370 con 0x7f2934104f80 2024-03-04T13:48:14.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.253+0000 7f293a800700 1 -- 172.21.15.107:0/4269471521 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f293410fc10 con 0x7f2934104f80 2024-03-04T13:48:14.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.253+0000 7f293a800700 1 -- 172.21.15.107:0/4269471521 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29341101d0 con 0x7f2934104f80 2024-03-04T13:48:14.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.254+0000 7f293a800700 1 -- 172.21.15.107:0/4269471521 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2934107450 con 0x7f2934104f80 2024-03-04T13:48:14.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.255+0000 7f2931600700 1 -- 172.21.15.107:0/4269471521 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 39) v1 ==== 91587+0+0 (secure 0 0 0) 0x7f2928011810 con 0x7f2934104f80 2024-03-04T13:48:14.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.256+0000 7f2931600700 1 --2- 172.21.15.107:0/4269471521 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f291c06d900 0x7f291c06fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:48:14.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.257+0000 7f2931600700 1 -- 172.21.15.107:0/4269471521 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f292808d420 con 0x7f2934104f80 2024-03-04T13:48:14.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.257+0000 7f2938a00700 1 --2- 172.21.15.107:0/4269471521 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f291c06d900 0x7f291c06fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:48:14.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.257+0000 7f2938a00700 1 --2- 172.21.15.107:0/4269471521 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f291c06d900 0x7f291c06fdb0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f29341112a0 tx=0x7f2920009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:48:14.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.261+0000 7f2931600700 1 -- 172.21.15.107:0/4269471521 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f292805a6c0 con 0x7f2934104f80 2024-03-04T13:48:14.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.489+0000 7f293a800700 1 -- 172.21.15.107:0/4269471521 --> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f293404bf40 con 0x7f291c06d900 2024-03-04T13:48:14.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.490+0000 7f2931600700 1 -- 172.21.15.107:0/4269471521 <== mgr.34102 v2:172.21.15.107:6800/1223039812 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7f293404bf40 con 0x7f291c06d900 2024-03-04T13:48:14.490 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-03-04T13:48:14.490 INFO:teuthology.orchestra.run.smithi107.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377", 2024-03-04T13:48:14.490 INFO:teuthology.orchestra.run.smithi107.stdout: "in_progress": true, 2024-03-04T13:48:14.490 INFO:teuthology.orchestra.run.smithi107.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-04T13:48:14.490 INFO:teuthology.orchestra.run.smithi107.stdout: "services_complete": [ 2024-03-04T13:48:14.490 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr", 2024-03-04T13:48:14.490 INFO:teuthology.orchestra.run.smithi107.stdout: "mon" 2024-03-04T13:48:14.490 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-04T13:48:14.490 INFO:teuthology.orchestra.run.smithi107.stdout: "progress": "4/21 daemons upgraded", 2024-03-04T13:48:14.490 INFO:teuthology.orchestra.run.smithi107.stdout: "message": "Currently upgrading mon daemons", 2024-03-04T13:48:14.491 INFO:teuthology.orchestra.run.smithi107.stdout: "is_paused": false 2024-03-04T13:48:14.491 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-03-04T13:48:14.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.494+0000 7f293a800700 1 -- 172.21.15.107:0/4269471521 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f291c06d900 msgr2=0x7f291c06fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:48:14.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.494+0000 7f293a800700 1 --2- 172.21.15.107:0/4269471521 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f291c06d900 0x7f291c06fdb0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f29341112a0 tx=0x7f2920009e30 comp rx=0 tx=0).stop 2024-03-04T13:48:14.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.494+0000 7f293a800700 1 -- 172.21.15.107:0/4269471521 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2934104f80 msgr2=0x7f2934115a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:48:14.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.494+0000 7f293a800700 1 --2- 172.21.15.107:0/4269471521 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2934104f80 0x7f2934115a70 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f292800c810 tx=0x7f2928004470 comp rx=0 tx=0).stop 2024-03-04T13:48:14.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.494+0000 7f293a800700 1 -- 172.21.15.107:0/4269471521 shutdown_connections 2024-03-04T13:48:14.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.494+0000 7f293a800700 1 --2- 172.21.15.107:0/4269471521 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f291c06d900 0x7f291c06fdb0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:48:14.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.494+0000 7f293a800700 1 --2- 172.21.15.107:0/4269471521 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2934104f80 0x7f2934115a70 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:48:14.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.494+0000 7f293a800700 1 --2- 172.21.15.107:0/4269471521 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2934105920 0x7f2934115fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:48:14.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.494+0000 7f293a800700 1 -- 172.21.15.107:0/4269471521 >> 172.21.15.107:0/4269471521 conn(0x7f29340fab10 msgr2=0x7f29340fc520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:48:14.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.494+0000 7f293a800700 1 -- 172.21.15.107:0/4269471521 shutdown_connections 2024-03-04T13:48:14.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.494+0000 7f293a800700 1 -- 172.21.15.107:0/4269471521 wait complete. 2024-03-04T13:48:14.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.643+0000 7f649e600700 1 -- 172.21.15.107:0/1633431076 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6498070c40 msgr2=0x7f6498109260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:48:14.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.643+0000 7f649e600700 1 --2- 172.21.15.107:0/1633431076 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6498070c40 0x7f6498109260 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6484004970 tx=0x7f6484009e30 comp rx=0 tx=0).stop 2024-03-04T13:48:14.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.643+0000 7f649e600700 1 -- 172.21.15.107:0/1633431076 shutdown_connections 2024-03-04T13:48:14.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.643+0000 7f649e600700 1 --2- 172.21.15.107:0/1633431076 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6498070c40 0x7f6498109260 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:48:14.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.643+0000 7f649e600700 1 --2- 172.21.15.107:0/1633431076 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6498070330 0x7f6498070700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:48:14.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.643+0000 7f649e600700 1 -- 172.21.15.107:0/1633431076 >> 172.21.15.107:0/1633431076 conn(0x7f64980f8ae0 msgr2=0x7f64980faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:48:14.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.643+0000 7f649e600700 1 -- 172.21.15.107:0/1633431076 shutdown_connections 2024-03-04T13:48:14.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.643+0000 7f649e600700 1 -- 172.21.15.107:0/1633431076 wait complete. 2024-03-04T13:48:14.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.644+0000 7f649e600700 1 Processor -- start 2024-03-04T13:48:14.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.644+0000 7f649e600700 1 -- start start 2024-03-04T13:48:14.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.645+0000 7f649e600700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6498070330 0x7f6498115ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:48:14.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.645+0000 7f649e600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6498070c40 0x7f64981161e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:48:14.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.645+0000 7f649e600700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64981168c0 con 0x7f6498070c40 2024-03-04T13:48:14.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.645+0000 7f649e600700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f649810fb40 con 0x7f6498070330 2024-03-04T13:48:14.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.645+0000 7f6496a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6498070c40 0x7f64981161e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:48:14.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.645+0000 7f6496a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6498070c40 0x7f64981161e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:53454/0 (socket says 172.21.15.107:53454) 2024-03-04T13:48:14.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.645+0000 7f6496a00700 1 -- 172.21.15.107:0/4098057199 learned_addr learned my addr 172.21.15.107:0/4098057199 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:48:14.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.645+0000 7f6497400700 1 --2- 172.21.15.107:0/4098057199 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6498070330 0x7f6498115ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:48:14.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.645+0000 7f6496a00700 1 -- 172.21.15.107:0/4098057199 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6498070330 msgr2=0x7f6498115ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:48:14.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.645+0000 7f6496a00700 1 --2- 172.21.15.107:0/4098057199 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6498070330 0x7f6498115ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:48:14.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.646+0000 7f6496a00700 1 -- 172.21.15.107:0/4098057199 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6484005040 con 0x7f6498070c40 2024-03-04T13:48:14.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.646+0000 7f6496a00700 1 --2- 172.21.15.107:0/4098057199 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6498070c40 0x7f64981161e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6484004f50 tx=0x7f6484009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:48:14.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.646+0000 7f648f400700 1 -- 172.21.15.107:0/4098057199 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f648400e040 con 0x7f6498070c40 2024-03-04T13:48:14.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.646+0000 7f648f400700 1 -- 172.21.15.107:0/4098057199 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6484004290 con 0x7f6498070c40 2024-03-04T13:48:14.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.646+0000 7f648f400700 1 -- 172.21.15.107:0/4098057199 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f648401c630 con 0x7f6498070c40 2024-03-04T13:48:14.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.646+0000 7f649e600700 1 -- 172.21.15.107:0/4098057199 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f649810fd60 con 0x7f6498070c40 2024-03-04T13:48:14.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.646+0000 7f649e600700 1 -- 172.21.15.107:0/4098057199 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64981102a0 con 0x7f6498070c40 2024-03-04T13:48:14.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.648+0000 7f649e600700 1 -- 172.21.15.107:0/4098057199 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6498106920 con 0x7f6498070c40 2024-03-04T13:48:14.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.649+0000 7f648f400700 1 -- 172.21.15.107:0/4098057199 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 39) v1 ==== 91587+0+0 (secure 0 0 0) 0x7f648400bc10 con 0x7f6498070c40 2024-03-04T13:48:14.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.649+0000 7f648f400700 1 --2- 172.21.15.107:0/4098057199 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f647c06d900 0x7f647c06fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:48:14.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.649+0000 7f648f400700 1 -- 172.21.15.107:0/4098057199 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f648408e3a0 con 0x7f6498070c40 2024-03-04T13:48:14.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.650+0000 7f6497400700 1 --2- 172.21.15.107:0/4098057199 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f647c06d900 0x7f647c06fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:48:14.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.650+0000 7f6497400700 1 --2- 172.21.15.107:0/4098057199 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f647c06d900 0x7f647c06fdb0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f6488006fd0 tx=0x7f6488008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:48:14.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.654+0000 7f648f400700 1 -- 172.21.15.107:0/4098057199 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f648405b670 con 0x7f6498070c40 2024-03-04T13:48:14.842 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:14 smithi107 ceph-mon[112578]: from='client.34166 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:48:14.842 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:14 smithi107 ceph-mon[112578]: from='client.? 172.21.15.107:0/4137779195' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:48:14.843 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:14 smithi107 ceph-mon[112578]: from='client.? 172.21.15.107:0/3448348634' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-04T13:48:14.860 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:14 smithi169 ceph-mon[86183]: from='client.34166 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:48:14.860 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:14 smithi169 ceph-mon[86183]: from='client.? 172.21.15.107:0/4137779195' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:48:14.860 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:14 smithi169 ceph-mon[86183]: from='client.? 172.21.15.107:0/3448348634' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-04T13:48:14.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.979+0000 7f649e600700 1 -- 172.21.15.107:0/4098057199 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f649804bed0 con 0x7f6498070c40 2024-03-04T13:48:14.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.980+0000 7f648f400700 1 -- 172.21.15.107:0/4098057199 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f6484013e60 con 0x7f6498070c40 2024-03-04T13:48:14.979 INFO:teuthology.orchestra.run.smithi107.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-03-04T13:48:14.980 INFO:teuthology.orchestra.run.smithi107.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-04T13:48:14.980 INFO:teuthology.orchestra.run.smithi107.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-04T13:48:14.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.982+0000 7f649e600700 1 -- 172.21.15.107:0/4098057199 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f647c06d900 msgr2=0x7f647c06fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:48:14.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.982+0000 7f649e600700 1 --2- 172.21.15.107:0/4098057199 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f647c06d900 0x7f647c06fdb0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f6488006fd0 tx=0x7f6488008040 comp rx=0 tx=0).stop 2024-03-04T13:48:14.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.982+0000 7f649e600700 1 -- 172.21.15.107:0/4098057199 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6498070c40 msgr2=0x7f64981161e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:48:14.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.982+0000 7f649e600700 1 --2- 172.21.15.107:0/4098057199 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6498070c40 0x7f64981161e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6484004f50 tx=0x7f6484009e30 comp rx=0 tx=0).stop 2024-03-04T13:48:14.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.982+0000 7f649e600700 1 -- 172.21.15.107:0/4098057199 shutdown_connections 2024-03-04T13:48:14.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.982+0000 7f649e600700 1 --2- 172.21.15.107:0/4098057199 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f647c06d900 0x7f647c06fdb0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:48:14.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.982+0000 7f649e600700 1 --2- 172.21.15.107:0/4098057199 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6498070330 0x7f6498115ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:48:14.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.982+0000 7f649e600700 1 --2- 172.21.15.107:0/4098057199 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6498070c40 0x7f64981161e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:48:14.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.982+0000 7f649e600700 1 -- 172.21.15.107:0/4098057199 >> 172.21.15.107:0/4098057199 conn(0x7f64980f8ae0 msgr2=0x7f6498103aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:48:14.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.983+0000 7f649e600700 1 -- 172.21.15.107:0/4098057199 shutdown_connections 2024-03-04T13:48:14.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:14.983+0000 7f649e600700 1 -- 172.21.15.107:0/4098057199 wait complete. 2024-03-04T13:48:15.893 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:15 smithi107 ceph-mon[112578]: from='client.34178 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:48:15.893 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:15 smithi107 ceph-mon[112578]: pgmap v28: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:48:15.893 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:15 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:48:15.893 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:15 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:48:15.894 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:15 smithi107 ceph-mon[112578]: Reconfiguring osd.2 (monmap changed)... 2024-03-04T13:48:15.894 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:15 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-04T13:48:15.894 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:15 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:48:15.894 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:15 smithi107 ceph-mon[112578]: Reconfiguring daemon osd.2 on smithi107 2024-03-04T13:48:15.894 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:15 smithi107 ceph-mon[112578]: from='client.? 172.21.15.107:0/4098057199' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-04T13:48:16.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:15 smithi169 ceph-mon[86183]: from='client.34178 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:48:16.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:15 smithi169 ceph-mon[86183]: pgmap v28: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:48:16.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:15 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:48:16.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:15 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:48:16.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:15 smithi169 ceph-mon[86183]: Reconfiguring osd.2 (monmap changed)... 2024-03-04T13:48:16.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:15 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-04T13:48:16.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:15 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:48:16.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:15 smithi169 ceph-mon[86183]: Reconfiguring daemon osd.2 on smithi107 2024-03-04T13:48:16.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:15 smithi169 ceph-mon[86183]: from='client.? 172.21.15.107:0/4098057199' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-04T13:48:18.070 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:17 smithi107 ceph-mon[112578]: pgmap v29: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:48:18.070 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:17 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:48:18.070 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:17 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:48:18.070 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:17 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi107.riexmm", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-04T13:48:18.070 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:17 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:48:18.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:17 smithi169 ceph-mon[86183]: pgmap v29: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:48:18.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:17 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:48:18.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:17 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:48:18.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:17 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi107.riexmm", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-04T13:48:18.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:17 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:48:19.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:18 smithi107 ceph-mon[112578]: Reconfiguring mds.cephfs.smithi107.riexmm (monmap changed)... 2024-03-04T13:48:19.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:18 smithi107 ceph-mon[112578]: Reconfiguring daemon mds.cephfs.smithi107.riexmm on smithi107 2024-03-04T13:48:19.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:18 smithi169 ceph-mon[86183]: Reconfiguring mds.cephfs.smithi107.riexmm (monmap changed)... 2024-03-04T13:48:19.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:18 smithi169 ceph-mon[86183]: Reconfiguring daemon mds.cephfs.smithi107.riexmm on smithi107 2024-03-04T13:48:20.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:19 smithi107 ceph-mon[112578]: pgmap v30: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:48:20.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:19 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:48:20.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:19 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:48:20.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:19 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi107.thycho", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-04T13:48:20.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:19 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:48:20.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:19 smithi169 ceph-mon[86183]: pgmap v30: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:48:20.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:19 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:48:20.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:19 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:48:20.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:19 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi107.thycho", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-04T13:48:20.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:19 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:48:21.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:20 smithi107 ceph-mon[112578]: Reconfiguring mds.cephfs.smithi107.thycho (monmap changed)... 2024-03-04T13:48:21.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:20 smithi107 ceph-mon[112578]: Reconfiguring daemon mds.cephfs.smithi107.thycho on smithi107 2024-03-04T13:48:21.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:20 smithi169 ceph-mon[86183]: Reconfiguring mds.cephfs.smithi107.thycho (monmap changed)... 2024-03-04T13:48:21.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:20 smithi169 ceph-mon[86183]: Reconfiguring daemon mds.cephfs.smithi107.thycho on smithi107 2024-03-04T13:48:22.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:21 smithi107 ceph-mon[112578]: pgmap v31: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:48:22.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:21 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:48:22.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:21 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:48:22.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:21 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi169", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-04T13:48:22.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:21 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:48:22.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:21 smithi169 ceph-mon[86183]: pgmap v31: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:48:22.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:21 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:48:22.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:21 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:48:22.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:21 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi169", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-04T13:48:22.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:21 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:48:22.860 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:22 smithi169 ceph-mon[86183]: Reconfiguring crash.smithi169 (monmap changed)... 2024-03-04T13:48:22.860 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:22 smithi169 ceph-mon[86183]: Reconfiguring daemon crash.smithi169 on smithi169 2024-03-04T13:48:23.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:22 smithi107 ceph-mon[112578]: Reconfiguring crash.smithi169 (monmap changed)... 2024-03-04T13:48:23.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:22 smithi107 ceph-mon[112578]: Reconfiguring daemon crash.smithi169 on smithi169 2024-03-04T13:48:24.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:23 smithi107 ceph-mon[112578]: pgmap v32: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:48:24.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:23 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:48:24.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:23 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:48:24.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:23 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi169.eihczr", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-04T13:48:24.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:23 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-04T13:48:24.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:23 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:48:24.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:23 smithi169 ceph-mon[86183]: pgmap v32: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:48:24.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:23 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:48:24.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:23 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:48:24.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:23 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi169.eihczr", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-04T13:48:24.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:23 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-04T13:48:24.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:23 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:48:25.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:24 smithi107 ceph-mon[112578]: Reconfiguring mgr.smithi169.eihczr (monmap changed)... 2024-03-04T13:48:25.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:24 smithi107 ceph-mon[112578]: Reconfiguring daemon mgr.smithi169.eihczr on smithi169 2024-03-04T13:48:25.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:24 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:48:25.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:24 smithi169 ceph-mon[86183]: Reconfiguring mgr.smithi169.eihczr (monmap changed)... 2024-03-04T13:48:25.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:24 smithi169 ceph-mon[86183]: Reconfiguring daemon mgr.smithi169.eihczr on smithi169 2024-03-04T13:48:25.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:24 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:48:26.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:25 smithi169 ceph-mon[86183]: pgmap v33: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:48:26.343 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:25 smithi107 ceph-mon[112578]: pgmap v33: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:48:27.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:26 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:48:27.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:26 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:48:27.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:26 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-04T13:48:27.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:26 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-04T13:48:27.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:26 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:48:27.343 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:26 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:48:27.343 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:26 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:48:27.343 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:26 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-04T13:48:27.343 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:26 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-04T13:48:27.343 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:26 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:48:28.334 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:27 smithi107 ceph-mon[112578]: Reconfiguring mon.smithi169 (monmap changed)... 2024-03-04T13:48:28.334 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:27 smithi107 ceph-mon[112578]: Reconfiguring daemon mon.smithi169 on smithi169 2024-03-04T13:48:28.334 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:27 smithi107 ceph-mon[112578]: pgmap v34: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:48:28.360 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:27 smithi169 ceph-mon[86183]: Reconfiguring mon.smithi169 (monmap changed)... 2024-03-04T13:48:28.360 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:27 smithi169 ceph-mon[86183]: Reconfiguring daemon mon.smithi169 on smithi169 2024-03-04T13:48:28.360 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:27 smithi169 ceph-mon[86183]: pgmap v34: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:48:29.360 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:29 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:48:29.360 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:29 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:48:29.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:29 smithi169 ceph-mon[86183]: Reconfiguring osd.3 (monmap changed)... 2024-03-04T13:48:29.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:29 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-04T13:48:29.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:29 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:48:29.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:29 smithi169 ceph-mon[86183]: Reconfiguring daemon osd.3 on smithi169 2024-03-04T13:48:29.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:29 smithi169 ceph-mon[86183]: pgmap v35: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:48:29.593 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:29 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:48:29.593 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:29 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:48:29.593 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:29 smithi107 ceph-mon[112578]: Reconfiguring osd.3 (monmap changed)... 2024-03-04T13:48:29.593 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:29 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-04T13:48:29.594 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:29 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:48:29.594 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:29 smithi107 ceph-mon[112578]: Reconfiguring daemon osd.3 on smithi169 2024-03-04T13:48:29.594 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:29 smithi107 ceph-mon[112578]: pgmap v35: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:48:31.657 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:31 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:48:31.657 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:31 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:48:31.657 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:31 smithi169 ceph-mon[86183]: Reconfiguring osd.4 (monmap changed)... 2024-03-04T13:48:31.657 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:31 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-04T13:48:31.657 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:31 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:48:31.657 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:31 smithi169 ceph-mon[86183]: Reconfiguring daemon osd.4 on smithi169 2024-03-04T13:48:31.657 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:31 smithi169 ceph-mon[86183]: pgmap v36: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:48:31.843 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:31 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:48:31.843 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:31 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:48:31.843 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:31 smithi107 ceph-mon[112578]: Reconfiguring osd.4 (monmap changed)... 2024-03-04T13:48:31.843 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:31 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-04T13:48:31.844 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:31 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:48:31.844 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:31 smithi107 ceph-mon[112578]: Reconfiguring daemon osd.4 on smithi169 2024-03-04T13:48:31.844 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:31 smithi107 ceph-mon[112578]: pgmap v36: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:48:34.088 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:33 smithi169 ceph-mon[86183]: pgmap v37: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:48:34.088 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:33 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:48:34.088 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:33 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:48:34.089 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:33 smithi169 ceph-mon[86183]: Reconfiguring osd.5 (monmap changed)... 2024-03-04T13:48:34.089 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:33 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-04T13:48:34.089 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:33 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:48:34.089 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:33 smithi169 ceph-mon[86183]: Reconfiguring daemon osd.5 on smithi169 2024-03-04T13:48:34.151 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:33 smithi107 ceph-mon[112578]: pgmap v37: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:48:34.151 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:33 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:48:34.151 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:33 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:48:34.151 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:33 smithi107 ceph-mon[112578]: Reconfiguring osd.5 (monmap changed)... 2024-03-04T13:48:34.151 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:33 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-04T13:48:34.151 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:33 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:48:34.151 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:33 smithi107 ceph-mon[112578]: Reconfiguring daemon osd.5 on smithi169 2024-03-04T13:48:35.593 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:35 smithi107 ceph-mon[112578]: pgmap v38: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:48:35.593 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:35 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:48:35.593 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:35 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:48:35.610 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:35 smithi169 ceph-mon[86183]: pgmap v38: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:48:35.610 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:35 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:48:35.610 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:35 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:48:36.536 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:36 smithi169 ceph-mon[86183]: Reconfiguring mds.cephfs.smithi169.jusudl (monmap changed)... 2024-03-04T13:48:36.536 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:36 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi169.jusudl", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-04T13:48:36.536 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:36 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:48:36.536 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:36 smithi169 ceph-mon[86183]: Reconfiguring daemon mds.cephfs.smithi169.jusudl on smithi169 2024-03-04T13:48:36.593 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:36 smithi107 ceph-mon[112578]: Reconfiguring mds.cephfs.smithi169.jusudl (monmap changed)... 2024-03-04T13:48:36.593 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:36 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi169.jusudl", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-04T13:48:36.593 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:36 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:48:36.593 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:36 smithi107 ceph-mon[112578]: Reconfiguring daemon mds.cephfs.smithi169.jusudl on smithi169 2024-03-04T13:48:37.593 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:37 smithi107 ceph-mon[112578]: pgmap v39: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:48:37.610 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:37 smithi169 ceph-mon[86183]: pgmap v39: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:48:38.593 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:38 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:48:38.593 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:38 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:48:38.593 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:38 smithi107 ceph-mon[112578]: Reconfiguring mds.cephfs.smithi169.bxitzk (monmap changed)... 2024-03-04T13:48:38.593 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:38 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi169.bxitzk", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-04T13:48:38.594 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:38 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:48:38.594 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:38 smithi107 ceph-mon[112578]: Reconfiguring daemon mds.cephfs.smithi169.bxitzk on smithi169 2024-03-04T13:48:38.611 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:38 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:48:38.611 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:38 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:48:38.611 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:38 smithi169 ceph-mon[86183]: Reconfiguring mds.cephfs.smithi169.bxitzk (monmap changed)... 2024-03-04T13:48:38.611 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:38 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi169.bxitzk", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-04T13:48:38.611 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:38 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:48:38.611 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:38 smithi169 ceph-mon[86183]: Reconfiguring daemon mds.cephfs.smithi169.bxitzk on smithi169 2024-03-04T13:48:39.843 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:39 smithi107 ceph-mon[112578]: pgmap v40: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:48:39.860 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:39 smithi169 ceph-mon[86183]: pgmap v40: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:48:40.772 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:40 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:48:40.772 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:40 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:48:40.772 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:40 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:48:40.773 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:40 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:48:40.773 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:40 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:48:40.773 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:40 smithi107 ceph-mon[112578]: Upgrade: Setting container_image for all mon 2024-03-04T13:48:40.773 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:40 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:48:40.773 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:40 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi107"}]: dispatch 2024-03-04T13:48:40.773 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:40 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi107"}]': finished 2024-03-04T13:48:40.773 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:40 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi169"}]: dispatch 2024-03-04T13:48:40.773 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:40 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi169"}]': finished 2024-03-04T13:48:40.773 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:40 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:48:40.861 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:40 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:48:40.861 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:40 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:48:40.861 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:40 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:48:40.861 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:40 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:48:40.861 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:40 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:48:40.861 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:40 smithi169 ceph-mon[86183]: Upgrade: Setting container_image for all mon 2024-03-04T13:48:40.861 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:40 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:48:40.861 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:40 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi107"}]: dispatch 2024-03-04T13:48:40.861 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:40 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi107"}]': finished 2024-03-04T13:48:40.861 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:40 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi169"}]: dispatch 2024-03-04T13:48:40.861 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:40 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi169"}]': finished 2024-03-04T13:48:40.861 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:40 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:48:41.593 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:41 smithi107 ceph-mon[112578]: pgmap v41: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:48:41.860 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:41 smithi169 ceph-mon[86183]: pgmap v41: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:48:42.843 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:42 smithi107 ceph-mon[112578]: Upgrade: Updating crash.smithi107 (1/2) 2024-03-04T13:48:42.843 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:42 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:48:42.844 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:42 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi107", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-04T13:48:42.844 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:42 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:48:42.860 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:42 smithi169 ceph-mon[86183]: Upgrade: Updating crash.smithi107 (1/2) 2024-03-04T13:48:42.860 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:42 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:48:42.860 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:42 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi107", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-04T13:48:42.860 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:42 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:48:43.843 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:43 smithi107 ceph-mon[112578]: Deploying daemon crash.smithi107 on smithi107 2024-03-04T13:48:43.843 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:43 smithi107 ceph-mon[112578]: pgmap v42: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:48:43.860 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:43 smithi169 ceph-mon[86183]: Deploying daemon crash.smithi107 on smithi107 2024-03-04T13:48:43.860 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:43 smithi169 ceph-mon[86183]: pgmap v42: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:48:45.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.144+0000 7f6143200700 1 -- 172.21.15.107:0/4060648459 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f613c070be0 msgr2=0x7f613c1092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:48:45.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.144+0000 7f6143200700 1 --2- 172.21.15.107:0/4060648459 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f613c070be0 0x7f613c1092c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6124009d10 tx=0x7f6124004d10 comp rx=0 tx=0).stop 2024-03-04T13:48:45.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.145+0000 7f6143200700 1 -- 172.21.15.107:0/4060648459 shutdown_connections 2024-03-04T13:48:45.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.145+0000 7f6143200700 1 --2- 172.21.15.107:0/4060648459 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f613c070be0 0x7f613c1092c0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:48:45.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.145+0000 7f6143200700 1 --2- 172.21.15.107:0/4060648459 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f613c0702d0 0x7f613c0706a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:48:45.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.145+0000 7f6143200700 1 -- 172.21.15.107:0/4060648459 >> 172.21.15.107:0/4060648459 conn(0x7f613c0f8ae0 msgr2=0x7f613c0faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:48:45.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.145+0000 7f6143200700 1 -- 172.21.15.107:0/4060648459 shutdown_connections 2024-03-04T13:48:45.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.145+0000 7f6143200700 1 -- 172.21.15.107:0/4060648459 wait complete. 2024-03-04T13:48:45.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.146+0000 7f6143200700 1 Processor -- start 2024-03-04T13:48:45.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.146+0000 7f6143200700 1 -- start start 2024-03-04T13:48:45.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.146+0000 7f6143200700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f613c0702d0 0x7f613c115f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:48:45.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.146+0000 7f6143200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f613c070be0 0x7f613c116440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:48:45.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.146+0000 7f6143200700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f613c116b20 con 0x7f613c070be0 2024-03-04T13:48:45.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.146+0000 7f6143200700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f613c10fda0 con 0x7f613c0702d0 2024-03-04T13:48:45.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.146+0000 7f6133400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f613c070be0 0x7f613c116440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:48:45.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.147+0000 7f6133400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f613c070be0 0x7f613c116440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47180/0 (socket says 172.21.15.107:47180) 2024-03-04T13:48:45.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.147+0000 7f6133400700 1 -- 172.21.15.107:0/297679021 learned_addr learned my addr 172.21.15.107:0/297679021 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:48:45.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.147+0000 7f613be00700 1 --2- 172.21.15.107:0/297679021 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f613c0702d0 0x7f613c115f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:48:45.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.147+0000 7f6133400700 1 -- 172.21.15.107:0/297679021 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f613c0702d0 msgr2=0x7f613c115f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:48:45.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.147+0000 7f6133400700 1 --2- 172.21.15.107:0/297679021 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f613c0702d0 0x7f613c115f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:48:45.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.147+0000 7f6133400700 1 -- 172.21.15.107:0/297679021 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6124009940 con 0x7f613c070be0 2024-03-04T13:48:45.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.147+0000 7f6133400700 1 --2- 172.21.15.107:0/297679021 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f613c070be0 0x7f613c116440 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6124008dd0 tx=0x7f6124004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:48:45.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.148+0000 7f6138c00700 1 -- 172.21.15.107:0/297679021 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f612400e040 con 0x7f613c070be0 2024-03-04T13:48:45.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.148+0000 7f6138c00700 1 -- 172.21.15.107:0/297679021 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f61240128e0 con 0x7f613c070be0 2024-03-04T13:48:45.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.148+0000 7f6143200700 1 -- 172.21.15.107:0/297679021 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f613c10ffc0 con 0x7f613c070be0 2024-03-04T13:48:45.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.148+0000 7f6138c00700 1 -- 172.21.15.107:0/297679021 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f612401c8f0 con 0x7f613c070be0 2024-03-04T13:48:45.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.148+0000 7f6143200700 1 -- 172.21.15.107:0/297679021 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f613c110500 con 0x7f613c070be0 2024-03-04T13:48:45.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.149+0000 7f6143200700 1 -- 172.21.15.107:0/297679021 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f613c106980 con 0x7f613c070be0 2024-03-04T13:48:45.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.150+0000 7f6138c00700 1 -- 172.21.15.107:0/297679021 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 39) v1 ==== 91587+0+0 (secure 0 0 0) 0x7f6124012a50 con 0x7f613c070be0 2024-03-04T13:48:45.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.151+0000 7f6138c00700 1 --2- 172.21.15.107:0/297679021 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f612006d900 0x7f612006fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:48:45.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.151+0000 7f6138c00700 1 -- 172.21.15.107:0/297679021 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f612408edf0 con 0x7f613c070be0 2024-03-04T13:48:45.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.151+0000 7f613be00700 1 --2- 172.21.15.107:0/297679021 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f612006d900 0x7f612006fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:48:45.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.152+0000 7f613be00700 1 --2- 172.21.15.107:0/297679021 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f612006d900 0x7f612006fdb0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f612c006fd0 tx=0x7f612c008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:48:45.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.156+0000 7f6138c00700 1 -- 172.21.15.107:0/297679021 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f612405bf80 con 0x7f613c070be0 2024-03-04T13:48:45.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.401+0000 7f6143200700 1 -- 172.21.15.107:0/297679021 --> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f613c04bf40 con 0x7f612006d900 2024-03-04T13:48:45.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.403+0000 7f6138c00700 1 -- 172.21.15.107:0/297679021 <== mgr.34102 v2:172.21.15.107:6800/1223039812 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+385 (secure 0 0 0) 0x7f613c04bf40 con 0x7f612006d900 2024-03-04T13:48:45.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.405+0000 7f6143200700 1 -- 172.21.15.107:0/297679021 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f612006d900 msgr2=0x7f612006fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:48:45.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.405+0000 7f6143200700 1 --2- 172.21.15.107:0/297679021 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f612006d900 0x7f612006fdb0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f612c006fd0 tx=0x7f612c008040 comp rx=0 tx=0).stop 2024-03-04T13:48:45.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.406+0000 7f6143200700 1 -- 172.21.15.107:0/297679021 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f613c070be0 msgr2=0x7f613c116440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:48:45.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.406+0000 7f6143200700 1 --2- 172.21.15.107:0/297679021 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f613c070be0 0x7f613c116440 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6124008dd0 tx=0x7f6124004d10 comp rx=0 tx=0).stop 2024-03-04T13:48:45.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.406+0000 7f6143200700 1 -- 172.21.15.107:0/297679021 shutdown_connections 2024-03-04T13:48:45.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.406+0000 7f6143200700 1 --2- 172.21.15.107:0/297679021 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f612006d900 0x7f612006fdb0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:48:45.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.406+0000 7f6143200700 1 --2- 172.21.15.107:0/297679021 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f613c0702d0 0x7f613c115f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:48:45.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.406+0000 7f6143200700 1 --2- 172.21.15.107:0/297679021 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f613c070be0 0x7f613c116440 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:48:45.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.406+0000 7f6143200700 1 -- 172.21.15.107:0/297679021 >> 172.21.15.107:0/297679021 conn(0x7f613c0f8ae0 msgr2=0x7f613c103b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:48:45.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.406+0000 7f6143200700 1 -- 172.21.15.107:0/297679021 shutdown_connections 2024-03-04T13:48:45.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.406+0000 7f6143200700 1 -- 172.21.15.107:0/297679021 wait complete. 2024-03-04T13:48:45.424 INFO:teuthology.orchestra.run.smithi107.stdout:true 2024-03-04T13:48:45.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.556+0000 7f3f5c400700 1 -- 172.21.15.107:0/2853671236 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f54103dc0 msgr2=0x7f3f54104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:48:45.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.556+0000 7f3f5c400700 1 --2- 172.21.15.107:0/2853671236 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f54103dc0 0x7f3f54104210 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3f40004940 tx=0x7f3f40009e30 comp rx=0 tx=0).stop 2024-03-04T13:48:45.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.556+0000 7f3f5c400700 1 -- 172.21.15.107:0/2853671236 shutdown_connections 2024-03-04T13:48:45.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.556+0000 7f3f5c400700 1 --2- 172.21.15.107:0/2853671236 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f54103dc0 0x7f3f54104210 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:48:45.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.557+0000 7f3f5c400700 1 --2- 172.21.15.107:0/2853671236 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3f541095f0 0x7f3f541099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:48:45.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.557+0000 7f3f5c400700 1 -- 172.21.15.107:0/2853671236 >> 172.21.15.107:0/2853671236 conn(0x7f3f54072d50 msgr2=0x7f3f54075160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:48:45.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.557+0000 7f3f5c400700 1 -- 172.21.15.107:0/2853671236 shutdown_connections 2024-03-04T13:48:45.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.557+0000 7f3f5c400700 1 -- 172.21.15.107:0/2853671236 wait complete. 2024-03-04T13:48:45.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.557+0000 7f3f5c400700 1 Processor -- start 2024-03-04T13:48:45.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.558+0000 7f3f5c400700 1 -- start start 2024-03-04T13:48:45.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.558+0000 7f3f5c400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f54103dc0 0x7f3f5411a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:48:45.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.558+0000 7f3f5c400700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3f541095f0 0x7f3f5411a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:48:45.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.558+0000 7f3f5c400700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f5411abd0 con 0x7f3f54103dc0 2024-03-04T13:48:45.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.558+0000 7f3f5c400700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f54113ee0 con 0x7f3f541095f0 2024-03-04T13:48:45.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.558+0000 7f3f59a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f54103dc0 0x7f3f5411a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:48:45.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.558+0000 7f3f59a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f54103dc0 0x7f3f5411a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47204/0 (socket says 172.21.15.107:47204) 2024-03-04T13:48:45.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.558+0000 7f3f59a00700 1 -- 172.21.15.107:0/2491994753 learned_addr learned my addr 172.21.15.107:0/2491994753 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:48:45.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.559+0000 7f3f59000700 1 --2- 172.21.15.107:0/2491994753 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3f541095f0 0x7f3f5411a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:48:45.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.559+0000 7f3f59a00700 1 -- 172.21.15.107:0/2491994753 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3f541095f0 msgr2=0x7f3f5411a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:48:45.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.559+0000 7f3f59a00700 1 --2- 172.21.15.107:0/2491994753 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3f541095f0 0x7f3f5411a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:48:45.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.559+0000 7f3f59a00700 1 -- 172.21.15.107:0/2491994753 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f40005040 con 0x7f3f54103dc0 2024-03-04T13:48:45.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.559+0000 7f3f59000700 1 --2- 172.21.15.107:0/2491994753 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3f541095f0 0x7f3f5411a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T13:48:45.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.559+0000 7f3f59a00700 1 --2- 172.21.15.107:0/2491994753 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f54103dc0 0x7f3f5411a040 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3f4800cb10 tx=0x7f3f48007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:48:45.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.559+0000 7f3f51600700 1 -- 172.21.15.107:0/2491994753 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f48004200 con 0x7f3f54103dc0 2024-03-04T13:48:45.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.559+0000 7f3f51600700 1 -- 172.21.15.107:0/2491994753 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3f4800acc0 con 0x7f3f54103dc0 2024-03-04T13:48:45.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.559+0000 7f3f51600700 1 -- 172.21.15.107:0/2491994753 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f48003d90 con 0x7f3f54103dc0 2024-03-04T13:48:45.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.559+0000 7f3f5c400700 1 -- 172.21.15.107:0/2491994753 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f541141c0 con 0x7f3f54103dc0 2024-03-04T13:48:45.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.560+0000 7f3f5c400700 1 -- 172.21.15.107:0/2491994753 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f54114700 con 0x7f3f54103dc0 2024-03-04T13:48:45.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.561+0000 7f3f5c400700 1 -- 172.21.15.107:0/2491994753 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f5410b9c0 con 0x7f3f54103dc0 2024-03-04T13:48:45.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.562+0000 7f3f51600700 1 -- 172.21.15.107:0/2491994753 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 39) v1 ==== 91587+0+0 (secure 0 0 0) 0x7f3f48003430 con 0x7f3f54103dc0 2024-03-04T13:48:45.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.562+0000 7f3f51600700 1 --2- 172.21.15.107:0/2491994753 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f3f3c06d900 0x7f3f3c06fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:48:45.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.563+0000 7f3f51600700 1 -- 172.21.15.107:0/2491994753 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f3f4808d5c0 con 0x7f3f54103dc0 2024-03-04T13:48:45.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.563+0000 7f3f59000700 1 --2- 172.21.15.107:0/2491994753 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f3f3c06d900 0x7f3f3c06fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:48:45.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.563+0000 7f3f59000700 1 --2- 172.21.15.107:0/2491994753 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f3f3c06d900 0x7f3f3c06fdb0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f3f4000b3b0 tx=0x7f3f40009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:48:45.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.567+0000 7f3f51600700 1 -- 172.21.15.107:0/2491994753 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3f4805a860 con 0x7f3f54103dc0 2024-03-04T13:48:45.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.815+0000 7f3f5c400700 1 -- 172.21.15.107:0/2491994753 --> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3f5404bf70 con 0x7f3f3c06d900 2024-03-04T13:48:45.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.818+0000 7f3f51600700 1 -- 172.21.15.107:0/2491994753 <== mgr.34102 v2:172.21.15.107:6800/1223039812 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+385 (secure 0 0 0) 0x7f3f40008e30 con 0x7f3f3c06d900 2024-03-04T13:48:45.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.821+0000 7f3f5c400700 1 -- 172.21.15.107:0/2491994753 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f3f3c06d900 msgr2=0x7f3f3c06fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:48:45.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.821+0000 7f3f5c400700 1 --2- 172.21.15.107:0/2491994753 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f3f3c06d900 0x7f3f3c06fdb0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f3f4000b3b0 tx=0x7f3f40009e30 comp rx=0 tx=0).stop 2024-03-04T13:48:45.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.821+0000 7f3f5c400700 1 -- 172.21.15.107:0/2491994753 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f54103dc0 msgr2=0x7f3f5411a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:48:45.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.821+0000 7f3f5c400700 1 --2- 172.21.15.107:0/2491994753 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f54103dc0 0x7f3f5411a040 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3f4800cb10 tx=0x7f3f48007f90 comp rx=0 tx=0).stop 2024-03-04T13:48:45.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.821+0000 7f3f5c400700 1 -- 172.21.15.107:0/2491994753 shutdown_connections 2024-03-04T13:48:45.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.821+0000 7f3f5c400700 1 --2- 172.21.15.107:0/2491994753 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f3f3c06d900 0x7f3f3c06fdb0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:48:45.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.821+0000 7f3f5c400700 1 --2- 172.21.15.107:0/2491994753 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f54103dc0 0x7f3f5411a040 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:48:45.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.821+0000 7f3f5c400700 1 --2- 172.21.15.107:0/2491994753 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3f541095f0 0x7f3f5411a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:48:45.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.821+0000 7f3f5c400700 1 -- 172.21.15.107:0/2491994753 >> 172.21.15.107:0/2491994753 conn(0x7f3f54072d50 msgr2=0x7f3f54074850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:48:45.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.821+0000 7f3f5c400700 1 -- 172.21.15.107:0/2491994753 shutdown_connections 2024-03-04T13:48:45.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.821+0000 7f3f5c400700 1 -- 172.21.15.107:0/2491994753 wait complete. 2024-03-04T13:48:45.978 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:45 smithi107 ceph-mon[112578]: pgmap v43: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:48:45.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.977+0000 7f11fda00700 1 -- 172.21.15.107:0/3843846971 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11f81038f0 msgr2=0x7f11f80fcf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:48:45.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.977+0000 7f11fda00700 1 --2- 172.21.15.107:0/3843846971 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11f81038f0 0x7f11f80fcf60 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f11e4004940 tx=0x7f11e4009e30 comp rx=0 tx=0).stop 2024-03-04T13:48:45.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.977+0000 7f11fda00700 1 -- 172.21.15.107:0/3843846971 shutdown_connections 2024-03-04T13:48:45.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.977+0000 7f11fda00700 1 --2- 172.21.15.107:0/3843846971 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11f81038f0 0x7f11f80fcf60 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:48:45.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.977+0000 7f11fda00700 1 --2- 172.21.15.107:0/3843846971 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11f8102f50 0x7f11f8103320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:48:45.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.977+0000 7f11fda00700 1 -- 172.21.15.107:0/3843846971 >> 172.21.15.107:0/3843846971 conn(0x7f11f80f8ae0 msgr2=0x7f11f80faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:48:45.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.977+0000 7f11fda00700 1 -- 172.21.15.107:0/3843846971 shutdown_connections 2024-03-04T13:48:45.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.977+0000 7f11fda00700 1 -- 172.21.15.107:0/3843846971 wait complete. 2024-03-04T13:48:45.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.978+0000 7f11fda00700 1 Processor -- start 2024-03-04T13:48:45.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.979+0000 7f11fda00700 1 -- start start 2024-03-04T13:48:45.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.979+0000 7f11fda00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11f8102f50 0x7f11f8115c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:48:45.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.979+0000 7f11fda00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11f81038f0 0x7f11f81161c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:48:45.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.979+0000 7f11fda00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11f8116810 con 0x7f11f8102f50 2024-03-04T13:48:45.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.979+0000 7f11fda00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11f810fb40 con 0x7f11f81038f0 2024-03-04T13:48:45.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.979+0000 7f11f6a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11f8102f50 0x7f11f8115c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:48:45.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.979+0000 7f11f6a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11f8102f50 0x7f11f8115c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47214/0 (socket says 172.21.15.107:47214) 2024-03-04T13:48:45.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.979+0000 7f11f6a00700 1 -- 172.21.15.107:0/4082736626 learned_addr learned my addr 172.21.15.107:0/4082736626 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:48:45.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.980+0000 7f11f6000700 1 --2- 172.21.15.107:0/4082736626 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11f81038f0 0x7f11f81161c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:48:45.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.980+0000 7f11f6a00700 1 -- 172.21.15.107:0/4082736626 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11f81038f0 msgr2=0x7f11f81161c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:48:45.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.980+0000 7f11f6a00700 1 --2- 172.21.15.107:0/4082736626 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11f81038f0 0x7f11f81161c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:48:45.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.980+0000 7f11f6a00700 1 -- 172.21.15.107:0/4082736626 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11e4005040 con 0x7f11f8102f50 2024-03-04T13:48:45.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.980+0000 7f11f6a00700 1 --2- 172.21.15.107:0/4082736626 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11f8102f50 0x7f11f8115c80 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f11e800c8c0 tx=0x7f11e80044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:48:45.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.981+0000 7f11eea00700 1 -- 172.21.15.107:0/4082736626 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f11e800a980 con 0x7f11f8102f50 2024-03-04T13:48:45.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.981+0000 7f11eea00700 1 -- 172.21.15.107:0/4082736626 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f11e800ce60 con 0x7f11f8102f50 2024-03-04T13:48:45.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.981+0000 7f11fda00700 1 -- 172.21.15.107:0/4082736626 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11f810fe20 con 0x7f11f8102f50 2024-03-04T13:48:45.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.981+0000 7f11eea00700 1 -- 172.21.15.107:0/4082736626 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f11e80115e0 con 0x7f11f8102f50 2024-03-04T13:48:45.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.981+0000 7f11fda00700 1 -- 172.21.15.107:0/4082736626 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11f8110360 con 0x7f11f8102f50 2024-03-04T13:48:45.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.982+0000 7f11fda00700 1 -- 172.21.15.107:0/4082736626 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11f80fc6e0 con 0x7f11f8102f50 2024-03-04T13:48:45.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.983+0000 7f11eea00700 1 -- 172.21.15.107:0/4082736626 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 39) v1 ==== 91587+0+0 (secure 0 0 0) 0x7f11e8012400 con 0x7f11f8102f50 2024-03-04T13:48:45.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.985+0000 7f11eea00700 1 --2- 172.21.15.107:0/4082736626 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f11dc06d900 0x7f11dc06fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:48:45.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.985+0000 7f11eea00700 1 -- 172.21.15.107:0/4082736626 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f11e808cfe0 con 0x7f11f8102f50 2024-03-04T13:48:45.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.985+0000 7f11f6000700 1 --2- 172.21.15.107:0/4082736626 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f11dc06d900 0x7f11dc06fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:48:45.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.986+0000 7f11f6000700 1 --2- 172.21.15.107:0/4082736626 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f11dc06d900 0x7f11dc06fdb0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f11f81114b0 tx=0x7f11e4009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:48:45.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:45.989+0000 7f11eea00700 1 -- 172.21.15.107:0/4082736626 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f11e805a3c0 con 0x7f11f8102f50 2024-03-04T13:48:46.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:45 smithi169 ceph-mon[86183]: pgmap v43: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:48:46.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:46.274+0000 7f11fda00700 1 -- 172.21.15.107:0/4082736626 --> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f11f81111f0 con 0x7f11dc06d900 2024-03-04T13:48:46.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:46.311+0000 7f11eea00700 1 -- 172.21.15.107:0/4082736626 <== mgr.34102 v2:172.21.15.107:6800/1223039812 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f11f81111f0 con 0x7f11dc06d900 2024-03-04T13:48:46.312 INFO:teuthology.orchestra.run.smithi107.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-04T13:48:46.312 INFO:teuthology.orchestra.run.smithi107.stdout:alertmanager.smithi107 smithi107 *:9093,9094 running (3m) 74s ago 17m 14.6M - 0.25.0 c8568f914cd2 e5ed7be32538 2024-03-04T13:48:46.312 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi107 smithi107 running (17m) 74s ago 17m 6665k - 17.2.7-545-g42608bb1 9eec3f22859d 03e1570c2eff 2024-03-04T13:48:46.313 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi169 smithi169 running (16m) 50s ago 16m 6656k - 17.2.7-545-g42608bb1 9eec3f22859d 9c8881a5338e 2024-03-04T13:48:46.313 INFO:teuthology.orchestra.run.smithi107.stdout:grafana.smithi107 smithi107 *:3000 running (2m) 74s ago 16m 74.2M - 9.4.12 a0ea5e3c46d9 c5e4d33fb599 2024-03-04T13:48:46.313 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.riexmm smithi107 running (10m) 74s ago 10m 14.1M - 17.2.7-545-g42608bb1 9eec3f22859d fc921f7d4328 2024-03-04T13:48:46.313 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.thycho smithi107 running (10m) 74s ago 10m 98.2M - 17.2.7-545-g42608bb1 9eec3f22859d 4069c6e0e063 2024-03-04T13:48:46.313 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi169.bxitzk smithi169 running (10m) 50s ago 10m 14.0M - 17.2.7-545-g42608bb1 9eec3f22859d 7d4115228c7c 2024-03-04T13:48:46.313 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi169.jusudl smithi169 running (10m) 50s ago 10m 145M - 17.2.7-545-g42608bb1 9eec3f22859d 2d2f2cc452e8 2024-03-04T13:48:46.313 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi107.wpzgwl smithi107 *:8443,9283,8765 running (5m) 74s ago 18m 476M - 19.0.0-1845-g717ce3cc d71b839abdf8 1c5fdee45eac 2024-03-04T13:48:46.313 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi169.eihczr smithi169 *:8443,9283,8765 running (4m) 50s ago 15m 448M - 19.0.0-1845-g717ce3cc d71b839abdf8 b5cac2b7d22a 2024-03-04T13:48:46.313 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi107 smithi107 running (92s) 74s ago 19m 28.3M 2048M 19.0.0-1845-g717ce3cc d71b839abdf8 32f07144274f 2024-03-04T13:48:46.313 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi169 smithi169 running (56s) 50s ago 15m 18.5M 2048M 19.0.0-1845-g717ce3cc d71b839abdf8 e3527449381a 2024-03-04T13:48:46.313 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi107 smithi107 *:9100 running (4m) 74s ago 17m 7650k - 1.5.0 0da6a335fe13 8a8d3d4f724f 2024-03-04T13:48:46.313 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi169 smithi169 *:9100 running (4m) 50s ago 15m 7457k - 1.5.0 0da6a335fe13 f240f5a975bc 2024-03-04T13:48:46.313 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0 smithi107 running (14m) 74s ago 14m 206M 1558M 17.2.7-545-g42608bb1 9eec3f22859d c0605b68f696 2024-03-04T13:48:46.313 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1 smithi107 running (14m) 74s ago 14m 220M 1558M 17.2.7-545-g42608bb1 9eec3f22859d 5fb2c0dd5769 2024-03-04T13:48:46.314 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2 smithi107 running (13m) 74s ago 13m 182M 1558M 17.2.7-545-g42608bb1 9eec3f22859d 61f30f643549 2024-03-04T13:48:46.314 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3 smithi169 running (13m) 50s ago 13m 235M 2582M 17.2.7-545-g42608bb1 9eec3f22859d cd91d8694764 2024-03-04T13:48:46.314 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4 smithi169 running (12m) 50s ago 12m 207M 2582M 17.2.7-545-g42608bb1 9eec3f22859d 08545e3d0659 2024-03-04T13:48:46.314 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5 smithi169 running (11m) 50s ago 11m 194M 2582M 17.2.7-545-g42608bb1 9eec3f22859d dcffe6cb3eac 2024-03-04T13:48:46.314 INFO:teuthology.orchestra.run.smithi107.stdout:prometheus.smithi107 smithi107 *:9095 running (3m) 74s ago 16m 50.2M - 2.43.0 a07b618ecd1d 1eeb1ad5cb75 2024-03-04T13:48:46.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:46.315+0000 7f11fda00700 1 -- 172.21.15.107:0/4082736626 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f11dc06d900 msgr2=0x7f11dc06fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:48:46.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:46.315+0000 7f11fda00700 1 --2- 172.21.15.107:0/4082736626 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f11dc06d900 0x7f11dc06fdb0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f11f81114b0 tx=0x7f11e4009e30 comp rx=0 tx=0).stop 2024-03-04T13:48:46.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:46.315+0000 7f11fda00700 1 -- 172.21.15.107:0/4082736626 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11f8102f50 msgr2=0x7f11f8115c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:48:46.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:46.315+0000 7f11fda00700 1 --2- 172.21.15.107:0/4082736626 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11f8102f50 0x7f11f8115c80 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f11e800c8c0 tx=0x7f11e80044c0 comp rx=0 tx=0).stop 2024-03-04T13:48:46.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:46.315+0000 7f11fda00700 1 -- 172.21.15.107:0/4082736626 shutdown_connections 2024-03-04T13:48:46.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:46.315+0000 7f11fda00700 1 --2- 172.21.15.107:0/4082736626 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f11dc06d900 0x7f11dc06fdb0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:48:46.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:46.315+0000 7f11fda00700 1 --2- 172.21.15.107:0/4082736626 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11f8102f50 0x7f11f8115c80 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:48:46.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:46.315+0000 7f11fda00700 1 --2- 172.21.15.107:0/4082736626 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11f81038f0 0x7f11f81161c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:48:46.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:46.315+0000 7f11fda00700 1 -- 172.21.15.107:0/4082736626 >> 172.21.15.107:0/4082736626 conn(0x7f11f80f8ae0 msgr2=0x7f11f80fa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:48:46.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:46.315+0000 7f11fda00700 1 -- 172.21.15.107:0/4082736626 shutdown_connections 2024-03-04T13:48:46.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:46.315+0000 7f11fda00700 1 -- 172.21.15.107:0/4082736626 wait complete. 2024-03-04T13:48:46.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:46.466+0000 7fd5d3600700 1 -- 172.21.15.107:0/2442933736 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd5cc102f50 msgr2=0x7fd5cc103320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:48:46.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:46.466+0000 7fd5d3600700 1 --2- 172.21.15.107:0/2442933736 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd5cc102f50 0x7fd5cc103320 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd5c0004940 tx=0x7fd5c0009e30 comp rx=0 tx=0).stop 2024-03-04T13:48:46.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:46.466+0000 7fd5d3600700 1 -- 172.21.15.107:0/2442933736 shutdown_connections 2024-03-04T13:48:46.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:46.466+0000 7fd5d3600700 1 --2- 172.21.15.107:0/2442933736 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5cc1038f0 0x7fd5cc0fcf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:48:46.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:46.466+0000 7fd5d3600700 1 --2- 172.21.15.107:0/2442933736 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd5cc102f50 0x7fd5cc103320 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:48:46.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:46.466+0000 7fd5d3600700 1 -- 172.21.15.107:0/2442933736 >> 172.21.15.107:0/2442933736 conn(0x7fd5cc0f8ae0 msgr2=0x7fd5cc0faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:48:46.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:46.466+0000 7fd5d3600700 1 -- 172.21.15.107:0/2442933736 shutdown_connections 2024-03-04T13:48:46.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:46.466+0000 7fd5d3600700 1 -- 172.21.15.107:0/2442933736 wait complete. 2024-03-04T13:48:46.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:46.467+0000 7fd5d3600700 1 Processor -- start 2024-03-04T13:48:46.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:46.467+0000 7fd5d3600700 1 -- start start 2024-03-04T13:48:46.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:46.467+0000 7fd5d3600700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5cc102f50 0x7fd5cc115c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:48:46.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:46.467+0000 7fd5d3600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd5cc1038f0 0x7fd5cc1161b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:48:46.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:46.467+0000 7fd5d3600700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5cc116800 con 0x7fd5cc1038f0 2024-03-04T13:48:46.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:46.467+0000 7fd5d3600700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5cc10fb30 con 0x7fd5cc102f50 2024-03-04T13:48:46.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:46.468+0000 7fd5cbe00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd5cc1038f0 0x7fd5cc1161b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:48:46.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:46.468+0000 7fd5cbe00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd5cc1038f0 0x7fd5cc1161b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47246/0 (socket says 172.21.15.107:47246) 2024-03-04T13:48:46.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:46.468+0000 7fd5cbe00700 1 -- 172.21.15.107:0/338796288 learned_addr learned my addr 172.21.15.107:0/338796288 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:48:46.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:46.468+0000 7fd5d0c00700 1 --2- 172.21.15.107:0/338796288 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5cc102f50 0x7fd5cc115c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:48:46.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:46.468+0000 7fd5cbe00700 1 -- 172.21.15.107:0/338796288 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5cc102f50 msgr2=0x7fd5cc115c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:48:46.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:46.468+0000 7fd5cbe00700 1 --2- 172.21.15.107:0/338796288 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5cc102f50 0x7fd5cc115c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:48:46.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:46.468+0000 7fd5cbe00700 1 -- 172.21.15.107:0/338796288 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5c0005040 con 0x7fd5cc1038f0 2024-03-04T13:48:46.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:46.468+0000 7fd5cbe00700 1 --2- 172.21.15.107:0/338796288 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd5cc1038f0 0x7fd5cc1161b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd5b8002a50 tx=0x7fd5b8004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:48:46.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:46.469+0000 7fd5c8c00700 1 -- 172.21.15.107:0/338796288 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd5b8007cc0 con 0x7fd5cc1038f0 2024-03-04T13:48:46.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:46.469+0000 7fd5c8c00700 1 -- 172.21.15.107:0/338796288 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd5b800f040 con 0x7fd5cc1038f0 2024-03-04T13:48:46.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:46.469+0000 7fd5c8c00700 1 -- 172.21.15.107:0/338796288 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd5b800daf0 con 0x7fd5cc1038f0 2024-03-04T13:48:46.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:46.469+0000 7fd5d3600700 1 -- 172.21.15.107:0/338796288 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5cc10fe10 con 0x7fd5cc1038f0 2024-03-04T13:48:46.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:46.469+0000 7fd5d3600700 1 -- 172.21.15.107:0/338796288 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5cc110350 con 0x7fd5cc1038f0 2024-03-04T13:48:46.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:46.470+0000 7fd5d3600700 1 -- 172.21.15.107:0/338796288 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5cc0fc6e0 con 0x7fd5cc1038f0 2024-03-04T13:48:46.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:46.471+0000 7fd5c8c00700 1 -- 172.21.15.107:0/338796288 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 39) v1 ==== 91587+0+0 (secure 0 0 0) 0x7fd5b8009810 con 0x7fd5cc1038f0 2024-03-04T13:48:46.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:46.473+0000 7fd5c8c00700 1 --2- 172.21.15.107:0/338796288 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fd5b406d900 0x7fd5b406fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:48:46.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:46.473+0000 7fd5c8c00700 1 -- 172.21.15.107:0/338796288 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fd5b808ce20 con 0x7fd5cc1038f0 2024-03-04T13:48:46.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:46.473+0000 7fd5d0c00700 1 --2- 172.21.15.107:0/338796288 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fd5b406d900 0x7fd5b406fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:48:46.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:46.474+0000 7fd5d0c00700 1 --2- 172.21.15.107:0/338796288 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fd5b406d900 0x7fd5b406fdb0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd5c000b020 tx=0x7fd5c0009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:48:46.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:46.478+0000 7fd5c8c00700 1 -- 172.21.15.107:0/338796288 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd5b8059ff0 con 0x7fd5cc1038f0 2024-03-04T13:48:46.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:46.820+0000 7fd5d3600700 1 -- 172.21.15.107:0/338796288 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fd5cc1111e0 con 0x7fd5cc1038f0 2024-03-04T13:48:46.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:46.821+0000 7fd5c8c00700 1 -- 172.21.15.107:0/338796288 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+736 (secure 0 0 0) 0x7fd5b8059b80 con 0x7fd5cc1038f0 2024-03-04T13:48:46.822 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-03-04T13:48:46.822 INFO:teuthology.orchestra.run.smithi107.stdout: "mon": { 2024-03-04T13:48:46.822 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-1845-g717ce3cc (717ce3cce8d0166e4e577faa8004e6f9cb4128c0) squid (dev)": 2 2024-03-04T13:48:46.822 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:48:46.822 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr": { 2024-03-04T13:48:46.822 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-1845-g717ce3cc (717ce3cce8d0166e4e577faa8004e6f9cb4128c0) squid (dev)": 2 2024-03-04T13:48:46.822 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:48:46.822 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": { 2024-03-04T13:48:46.822 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-545-g42608bb1 (42608bb191117be10dba720e6bf8a02f3171cd34) quincy (stable)": 6 2024-03-04T13:48:46.822 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:48:46.822 INFO:teuthology.orchestra.run.smithi107.stdout: "mds": { 2024-03-04T13:48:46.822 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-545-g42608bb1 (42608bb191117be10dba720e6bf8a02f3171cd34) quincy (stable)": 4 2024-03-04T13:48:46.822 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:48:46.822 INFO:teuthology.orchestra.run.smithi107.stdout: "overall": { 2024-03-04T13:48:46.822 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-545-g42608bb1 (42608bb191117be10dba720e6bf8a02f3171cd34) quincy (stable)": 10, 2024-03-04T13:48:46.822 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-1845-g717ce3cc (717ce3cce8d0166e4e577faa8004e6f9cb4128c0) squid (dev)": 4 2024-03-04T13:48:46.823 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-04T13:48:46.823 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-03-04T13:48:46.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:46.824+0000 7fd5d3600700 1 -- 172.21.15.107:0/338796288 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fd5b406d900 msgr2=0x7fd5b406fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:48:46.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:46.824+0000 7fd5d3600700 1 --2- 172.21.15.107:0/338796288 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fd5b406d900 0x7fd5b406fdb0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd5c000b020 tx=0x7fd5c0009e30 comp rx=0 tx=0).stop 2024-03-04T13:48:46.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:46.824+0000 7fd5d3600700 1 -- 172.21.15.107:0/338796288 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd5cc1038f0 msgr2=0x7fd5cc1161b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:48:46.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:46.824+0000 7fd5d3600700 1 --2- 172.21.15.107:0/338796288 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd5cc1038f0 0x7fd5cc1161b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd5b8002a50 tx=0x7fd5b8004600 comp rx=0 tx=0).stop 2024-03-04T13:48:46.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:46.824+0000 7fd5d3600700 1 -- 172.21.15.107:0/338796288 shutdown_connections 2024-03-04T13:48:46.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:46.824+0000 7fd5d3600700 1 --2- 172.21.15.107:0/338796288 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fd5b406d900 0x7fd5b406fdb0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:48:46.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:46.824+0000 7fd5d3600700 1 --2- 172.21.15.107:0/338796288 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5cc102f50 0x7fd5cc115c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:48:46.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:46.824+0000 7fd5d3600700 1 --2- 172.21.15.107:0/338796288 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd5cc1038f0 0x7fd5cc1161b0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:48:46.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:46.824+0000 7fd5d3600700 1 -- 172.21.15.107:0/338796288 >> 172.21.15.107:0/338796288 conn(0x7fd5cc0f8ae0 msgr2=0x7fd5cc070510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:48:46.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:46.824+0000 7fd5d3600700 1 -- 172.21.15.107:0/338796288 shutdown_connections 2024-03-04T13:48:46.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:46.824+0000 7fd5d3600700 1 -- 172.21.15.107:0/338796288 wait complete. 2024-03-04T13:48:46.980 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:46 smithi107 ceph-mon[112578]: from='client.34186 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:48:46.981 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:46 smithi107 ceph-mon[112578]: from='client.34190 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:48:46.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:46.979+0000 7fec88a00700 1 -- 172.21.15.107:0/2018692712 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec84102d40 msgr2=0x7fec84103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:48:46.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:46.979+0000 7fec88a00700 1 --2- 172.21.15.107:0/2018692712 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec84102d40 0x7fec84103110 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fec74004940 tx=0x7fec74009e30 comp rx=0 tx=0).stop 2024-03-04T13:48:46.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:46.979+0000 7fec88a00700 1 -- 172.21.15.107:0/2018692712 shutdown_connections 2024-03-04T13:48:46.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:46.979+0000 7fec88a00700 1 --2- 172.21.15.107:0/2018692712 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec841036e0 0x7fec840fcd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:48:46.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:46.979+0000 7fec88a00700 1 --2- 172.21.15.107:0/2018692712 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec84102d40 0x7fec84103110 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:48:46.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:46.980+0000 7fec88a00700 1 -- 172.21.15.107:0/2018692712 >> 172.21.15.107:0/2018692712 conn(0x7fec840f88b0 msgr2=0x7fec840facc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:48:46.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:46.980+0000 7fec88a00700 1 -- 172.21.15.107:0/2018692712 shutdown_connections 2024-03-04T13:48:46.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:46.980+0000 7fec88a00700 1 -- 172.21.15.107:0/2018692712 wait complete. 2024-03-04T13:48:46.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:46.981+0000 7fec88a00700 1 Processor -- start 2024-03-04T13:48:46.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:46.981+0000 7fec88a00700 1 -- start start 2024-03-04T13:48:46.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:46.981+0000 7fec88a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec84102d40 0x7fec841159f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:48:46.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:46.981+0000 7fec88a00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec841036e0 0x7fec84115f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:48:46.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:46.981+0000 7fec88a00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec84116580 con 0x7fec84102d40 2024-03-04T13:48:46.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:46.981+0000 7fec88a00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec8410f8b0 con 0x7fec841036e0 2024-03-04T13:48:46.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:46.981+0000 7fec83400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec84102d40 0x7fec841159f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:48:46.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:46.981+0000 7fec83400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec84102d40 0x7fec841159f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47256/0 (socket says 172.21.15.107:47256) 2024-03-04T13:48:46.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:46.981+0000 7fec83400700 1 -- 172.21.15.107:0/1566472480 learned_addr learned my addr 172.21.15.107:0/1566472480 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:48:46.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:46.982+0000 7fec82a00700 1 --2- 172.21.15.107:0/1566472480 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec841036e0 0x7fec84115f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:48:46.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:46.982+0000 7fec83400700 1 -- 172.21.15.107:0/1566472480 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec841036e0 msgr2=0x7fec84115f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:48:46.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:46.982+0000 7fec83400700 1 --2- 172.21.15.107:0/1566472480 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec841036e0 0x7fec84115f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:48:46.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:46.982+0000 7fec83400700 1 -- 172.21.15.107:0/1566472480 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec74005040 con 0x7fec84102d40 2024-03-04T13:48:46.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:46.982+0000 7fec82a00700 1 --2- 172.21.15.107:0/1566472480 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec841036e0 0x7fec84115f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T13:48:46.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:46.982+0000 7fec83400700 1 --2- 172.21.15.107:0/1566472480 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec84102d40 0x7fec841159f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fec74009fd0 tx=0x7fec74009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:48:46.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:46.982+0000 7fec7b400700 1 -- 172.21.15.107:0/1566472480 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fec7400e040 con 0x7fec84102d40 2024-03-04T13:48:46.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:46.982+0000 7fec7b400700 1 -- 172.21.15.107:0/1566472480 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fec7400bbf0 con 0x7fec84102d40 2024-03-04T13:48:46.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:46.982+0000 7fec88a00700 1 -- 172.21.15.107:0/1566472480 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec8410fb30 con 0x7fec84102d40 2024-03-04T13:48:46.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:46.982+0000 7fec7b400700 1 -- 172.21.15.107:0/1566472480 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fec7401c3f0 con 0x7fec84102d40 2024-03-04T13:48:46.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:46.983+0000 7fec88a00700 1 -- 172.21.15.107:0/1566472480 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec841100f0 con 0x7fec84102d40 2024-03-04T13:48:46.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:46.984+0000 7fec88a00700 1 -- 172.21.15.107:0/1566472480 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec8405f5c0 con 0x7fec84102d40 2024-03-04T13:48:46.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:46.985+0000 7fec7b400700 1 -- 172.21.15.107:0/1566472480 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 39) v1 ==== 91587+0+0 (secure 0 0 0) 0x7fec7400bd60 con 0x7fec84102d40 2024-03-04T13:48:46.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:46.985+0000 7fec7b400700 1 --2- 172.21.15.107:0/1566472480 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fec6806d900 0x7fec6806fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:48:46.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:46.986+0000 7fec7b400700 1 -- 172.21.15.107:0/1566472480 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fec7408e4f0 con 0x7fec84102d40 2024-03-04T13:48:46.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:46.986+0000 7fec82a00700 1 --2- 172.21.15.107:0/1566472480 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fec6806d900 0x7fec6806fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:48:46.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:46.986+0000 7fec82a00700 1 --2- 172.21.15.107:0/1566472480 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fec6806d900 0x7fec6806fdb0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fec70007fd0 tx=0x7fec70009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:48:46.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:46.990+0000 7fec7b400700 1 -- 172.21.15.107:0/1566472480 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fec7405b680 con 0x7fec84102d40 2024-03-04T13:48:47.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:46 smithi169 ceph-mon[86183]: from='client.34186 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:48:47.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:46 smithi169 ceph-mon[86183]: from='client.34190 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:48:47.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:47.274+0000 7fec88a00700 1 -- 172.21.15.107:0/1566472480 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fec84109ec0 con 0x7fec84102d40 2024-03-04T13:48:47.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:47.275+0000 7fec7b400700 1 -- 172.21.15.107:0/1566472480 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 16 v16) v1 ==== 76+0+1880 (secure 0 0 0) 0x7fec84109ec0 con 0x7fec84102d40 2024-03-04T13:48:47.276 INFO:teuthology.orchestra.run.smithi107.stdout:e16 2024-03-04T13:48:47.276 INFO:teuthology.orchestra.run.smithi107.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-04T13:48:47.276 INFO:teuthology.orchestra.run.smithi107.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-04T13:48:47.276 INFO:teuthology.orchestra.run.smithi107.stdout:legacy client fscid: 1 2024-03-04T13:48:47.276 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:48:47.276 INFO:teuthology.orchestra.run.smithi107.stdout:Filesystem 'cephfs' (1) 2024-03-04T13:48:47.276 INFO:teuthology.orchestra.run.smithi107.stdout:fs_name cephfs 2024-03-04T13:48:47.276 INFO:teuthology.orchestra.run.smithi107.stdout:epoch 15 2024-03-04T13:48:47.276 INFO:teuthology.orchestra.run.smithi107.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-04T13:48:47.277 INFO:teuthology.orchestra.run.smithi107.stdout:created 2024-03-04T13:38:30.854004+0000 2024-03-04T13:48:47.277 INFO:teuthology.orchestra.run.smithi107.stdout:modified 2024-03-04T13:38:50.015989+0000 2024-03-04T13:48:47.277 INFO:teuthology.orchestra.run.smithi107.stdout:tableserver 0 2024-03-04T13:48:47.277 INFO:teuthology.orchestra.run.smithi107.stdout:root 0 2024-03-04T13:48:47.277 INFO:teuthology.orchestra.run.smithi107.stdout:session_timeout 60 2024-03-04T13:48:47.277 INFO:teuthology.orchestra.run.smithi107.stdout:session_autoclose 300 2024-03-04T13:48:47.277 INFO:teuthology.orchestra.run.smithi107.stdout:max_file_size 1099511627776 2024-03-04T13:48:47.277 INFO:teuthology.orchestra.run.smithi107.stdout:max_xattr_size 65536 2024-03-04T13:48:47.277 INFO:teuthology.orchestra.run.smithi107.stdout:required_client_features {} 2024-03-04T13:48:47.277 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure 0 2024-03-04T13:48:47.277 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure_osd_epoch 47 2024-03-04T13:48:47.277 INFO:teuthology.orchestra.run.smithi107.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-04T13:48:47.277 INFO:teuthology.orchestra.run.smithi107.stdout:max_mds 1 2024-03-04T13:48:47.277 INFO:teuthology.orchestra.run.smithi107.stdout:in 0 2024-03-04T13:48:47.277 INFO:teuthology.orchestra.run.smithi107.stdout:up {0=24247} 2024-03-04T13:48:47.278 INFO:teuthology.orchestra.run.smithi107.stdout:failed 2024-03-04T13:48:47.278 INFO:teuthology.orchestra.run.smithi107.stdout:damaged 2024-03-04T13:48:47.278 INFO:teuthology.orchestra.run.smithi107.stdout:stopped 2024-03-04T13:48:47.278 INFO:teuthology.orchestra.run.smithi107.stdout:data_pools [3] 2024-03-04T13:48:47.278 INFO:teuthology.orchestra.run.smithi107.stdout:metadata_pool 2 2024-03-04T13:48:47.278 INFO:teuthology.orchestra.run.smithi107.stdout:inline_data enabled 2024-03-04T13:48:47.278 INFO:teuthology.orchestra.run.smithi107.stdout:balancer 2024-03-04T13:48:47.278 INFO:teuthology.orchestra.run.smithi107.stdout:bal_rank_mask -1 2024-03-04T13:48:47.278 INFO:teuthology.orchestra.run.smithi107.stdout:standby_count_wanted 1 2024-03-04T13:48:47.278 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi169.jusudl{0:24247} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.169:6824/2994866076,v1:172.21.15.169:6825/2994866076] compat {c=[1],r=[1],i=[7ff]}] 2024-03-04T13:48:47.278 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.thycho{0:14470} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.107:6828/1985577060,v1:172.21.15.107:6829/1985577060] compat {c=[1],r=[1],i=[7ff]}] 2024-03-04T13:48:47.278 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:48:47.278 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:48:47.278 INFO:teuthology.orchestra.run.smithi107.stdout:Standby daemons: 2024-03-04T13:48:47.278 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:48:47.279 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.riexmm{-1:14480} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.107:6826/3261622517,v1:172.21.15.107:6827/3261622517] compat {c=[1],r=[1],i=[7ff]}] 2024-03-04T13:48:47.279 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi169.bxitzk{-1:24259} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.169:6826/3696090493,v1:172.21.15.169:6827/3696090493] compat {c=[1],r=[1],i=[7ff]}] 2024-03-04T13:48:47.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:47.279+0000 7fec88a00700 1 -- 172.21.15.107:0/1566472480 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fec6806d900 msgr2=0x7fec6806fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:48:47.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:47.279+0000 7fec88a00700 1 --2- 172.21.15.107:0/1566472480 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fec6806d900 0x7fec6806fdb0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fec70007fd0 tx=0x7fec70009040 comp rx=0 tx=0).stop 2024-03-04T13:48:47.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:47.279+0000 7fec88a00700 1 -- 172.21.15.107:0/1566472480 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec84102d40 msgr2=0x7fec841159f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:48:47.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:47.279+0000 7fec88a00700 1 --2- 172.21.15.107:0/1566472480 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec84102d40 0x7fec841159f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fec74009fd0 tx=0x7fec74009e30 comp rx=0 tx=0).stop 2024-03-04T13:48:47.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:47.279+0000 7fec88a00700 1 -- 172.21.15.107:0/1566472480 shutdown_connections 2024-03-04T13:48:47.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:47.279+0000 7fec88a00700 1 --2- 172.21.15.107:0/1566472480 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fec6806d900 0x7fec6806fdb0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:48:47.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:47.279+0000 7fec88a00700 1 --2- 172.21.15.107:0/1566472480 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec84102d40 0x7fec841159f0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:48:47.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:47.279+0000 7fec88a00700 1 --2- 172.21.15.107:0/1566472480 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec841036e0 0x7fec84115f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:48:47.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:47.279+0000 7fec88a00700 1 -- 172.21.15.107:0/1566472480 >> 172.21.15.107:0/1566472480 conn(0x7fec840f88b0 msgr2=0x7fec840fa140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:48:47.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:47.279+0000 7fec88a00700 1 -- 172.21.15.107:0/1566472480 shutdown_connections 2024-03-04T13:48:47.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:47.279+0000 7fec88a00700 1 -- 172.21.15.107:0/1566472480 wait complete. 2024-03-04T13:48:47.281 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 16 2024-03-04T13:48:47.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:47.431+0000 7f0513e00700 1 -- 172.21.15.107:0/1987211749 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05141036e0 msgr2=0x7f05140fcd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:48:47.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:47.431+0000 7f0513e00700 1 --2- 172.21.15.107:0/1987211749 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05141036e0 0x7f05140fcd50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f04fc004940 tx=0x7f04fc009e30 comp rx=0 tx=0).stop 2024-03-04T13:48:47.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:47.431+0000 7f0513e00700 1 -- 172.21.15.107:0/1987211749 shutdown_connections 2024-03-04T13:48:47.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:47.431+0000 7f0513e00700 1 --2- 172.21.15.107:0/1987211749 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05141036e0 0x7f05140fcd50 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:48:47.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:47.431+0000 7f0513e00700 1 --2- 172.21.15.107:0/1987211749 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0514102d40 0x7f0514103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:48:47.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:47.431+0000 7f0513e00700 1 -- 172.21.15.107:0/1987211749 >> 172.21.15.107:0/1987211749 conn(0x7f05140f88b0 msgr2=0x7f05140facc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:48:47.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:47.431+0000 7f0513e00700 1 -- 172.21.15.107:0/1987211749 shutdown_connections 2024-03-04T13:48:47.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:47.432+0000 7f0513e00700 1 -- 172.21.15.107:0/1987211749 wait complete. 2024-03-04T13:48:47.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:47.432+0000 7f0513e00700 1 Processor -- start 2024-03-04T13:48:47.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:47.432+0000 7f0513e00700 1 -- start start 2024-03-04T13:48:47.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:47.433+0000 7f0513e00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0514102d40 0x7f05141159a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:48:47.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:47.433+0000 7f0513e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05141036e0 0x7f0514115ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:48:47.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:47.433+0000 7f0513e00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0514116530 con 0x7f05141036e0 2024-03-04T13:48:47.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:47.433+0000 7f0513e00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f051410f860 con 0x7f0514102d40 2024-03-04T13:48:47.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:47.433+0000 7f050a000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05141036e0 0x7f0514115ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:48:47.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:47.433+0000 7f050a000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05141036e0 0x7f0514115ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47286/0 (socket says 172.21.15.107:47286) 2024-03-04T13:48:47.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:47.433+0000 7f050a000700 1 -- 172.21.15.107:0/1293215848 learned_addr learned my addr 172.21.15.107:0/1293215848 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:48:47.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:47.433+0000 7f0512a00700 1 --2- 172.21.15.107:0/1293215848 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0514102d40 0x7f05141159a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:48:47.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:47.433+0000 7f050a000700 1 -- 172.21.15.107:0/1293215848 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0514102d40 msgr2=0x7f05141159a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:48:47.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:47.433+0000 7f050a000700 1 --2- 172.21.15.107:0/1293215848 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0514102d40 0x7f05141159a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:48:47.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:47.434+0000 7f050a000700 1 -- 172.21.15.107:0/1293215848 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04fc005040 con 0x7f05141036e0 2024-03-04T13:48:47.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:47.434+0000 7f050a000700 1 --2- 172.21.15.107:0/1293215848 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05141036e0 0x7f0514115ee0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f04fc009fd0 tx=0x7f04fc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:48:47.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:47.434+0000 7f050b400700 1 -- 172.21.15.107:0/1293215848 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f04fc021070 con 0x7f05141036e0 2024-03-04T13:48:47.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:47.434+0000 7f050b400700 1 -- 172.21.15.107:0/1293215848 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f04fc013d00 con 0x7f05141036e0 2024-03-04T13:48:47.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:47.434+0000 7f050b400700 1 -- 172.21.15.107:0/1293215848 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f04fc00e070 con 0x7f05141036e0 2024-03-04T13:48:47.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:47.434+0000 7f0513e00700 1 -- 172.21.15.107:0/1293215848 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f051410fae0 con 0x7f05141036e0 2024-03-04T13:48:47.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:47.434+0000 7f0513e00700 1 -- 172.21.15.107:0/1293215848 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f051410fff0 con 0x7f05141036e0 2024-03-04T13:48:47.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:47.435+0000 7f0513e00700 1 -- 172.21.15.107:0/1293215848 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f051405f5c0 con 0x7f05141036e0 2024-03-04T13:48:47.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:47.436+0000 7f050b400700 1 -- 172.21.15.107:0/1293215848 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 39) v1 ==== 91587+0+0 (secure 0 0 0) 0x7f04fc01c400 con 0x7f05141036e0 2024-03-04T13:48:47.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:47.437+0000 7f050b400700 1 --2- 172.21.15.107:0/1293215848 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f04f806d900 0x7f04f806fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:48:47.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:47.437+0000 7f050b400700 1 -- 172.21.15.107:0/1293215848 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f04fc08e5f0 con 0x7f05141036e0 2024-03-04T13:48:47.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:47.437+0000 7f0512a00700 1 --2- 172.21.15.107:0/1293215848 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f04f806d900 0x7f04f806fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:48:47.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:47.438+0000 7f0512a00700 1 --2- 172.21.15.107:0/1293215848 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f04f806d900 0x7f04f806fdb0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0504001450 tx=0x7f050400b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:48:47.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:47.442+0000 7f050b400700 1 -- 172.21.15.107:0/1293215848 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f04fc05b820 con 0x7f05141036e0 2024-03-04T13:48:47.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:47.688+0000 7f0513e00700 1 -- 172.21.15.107:0/1293215848 --> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0514115670 con 0x7f04f806d900 2024-03-04T13:48:47.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:47.690+0000 7f050b400700 1 -- 172.21.15.107:0/1293215848 <== mgr.34102 v2:172.21.15.107:6800/1223039812 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+385 (secure 0 0 0) 0x7f0514115670 con 0x7f04f806d900 2024-03-04T13:48:47.691 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-03-04T13:48:47.691 INFO:teuthology.orchestra.run.smithi107.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377", 2024-03-04T13:48:47.691 INFO:teuthology.orchestra.run.smithi107.stdout: "in_progress": true, 2024-03-04T13:48:47.691 INFO:teuthology.orchestra.run.smithi107.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-04T13:48:47.691 INFO:teuthology.orchestra.run.smithi107.stdout: "services_complete": [ 2024-03-04T13:48:47.691 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr", 2024-03-04T13:48:47.691 INFO:teuthology.orchestra.run.smithi107.stdout: "mon" 2024-03-04T13:48:47.691 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-04T13:48:47.691 INFO:teuthology.orchestra.run.smithi107.stdout: "progress": "4/21 daemons upgraded", 2024-03-04T13:48:47.691 INFO:teuthology.orchestra.run.smithi107.stdout: "message": "Currently upgrading crash daemons", 2024-03-04T13:48:47.691 INFO:teuthology.orchestra.run.smithi107.stdout: "is_paused": false 2024-03-04T13:48:47.691 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-03-04T13:48:47.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:47.693+0000 7f0513e00700 1 -- 172.21.15.107:0/1293215848 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f04f806d900 msgr2=0x7f04f806fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:48:47.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:47.693+0000 7f0513e00700 1 --2- 172.21.15.107:0/1293215848 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f04f806d900 0x7f04f806fdb0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0504001450 tx=0x7f050400b040 comp rx=0 tx=0).stop 2024-03-04T13:48:47.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:47.693+0000 7f0513e00700 1 -- 172.21.15.107:0/1293215848 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05141036e0 msgr2=0x7f0514115ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:48:47.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:47.693+0000 7f0513e00700 1 --2- 172.21.15.107:0/1293215848 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05141036e0 0x7f0514115ee0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f04fc009fd0 tx=0x7f04fc009e30 comp rx=0 tx=0).stop 2024-03-04T13:48:47.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:47.693+0000 7f0513e00700 1 -- 172.21.15.107:0/1293215848 shutdown_connections 2024-03-04T13:48:47.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:47.693+0000 7f0513e00700 1 --2- 172.21.15.107:0/1293215848 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f04f806d900 0x7f04f806fdb0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:48:47.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:47.693+0000 7f0513e00700 1 --2- 172.21.15.107:0/1293215848 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0514102d40 0x7f05141159a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:48:47.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:47.693+0000 7f0513e00700 1 --2- 172.21.15.107:0/1293215848 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05141036e0 0x7f0514115ee0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:48:47.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:47.693+0000 7f0513e00700 1 -- 172.21.15.107:0/1293215848 >> 172.21.15.107:0/1293215848 conn(0x7f05140f88b0 msgr2=0x7f05140fa0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:48:47.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:47.693+0000 7f0513e00700 1 -- 172.21.15.107:0/1293215848 shutdown_connections 2024-03-04T13:48:47.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:47.693+0000 7f0513e00700 1 -- 172.21.15.107:0/1293215848 wait complete. 2024-03-04T13:48:47.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:47.842+0000 7f92b3600700 1 -- 172.21.15.107:0/2922828793 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92ac105170 msgr2=0x7f92ac105540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:48:47.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:47.842+0000 7f92b3600700 1 --2- 172.21.15.107:0/2922828793 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92ac105170 0x7f92ac105540 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f92a0004940 tx=0x7f92a0009e30 comp rx=0 tx=0).stop 2024-03-04T13:48:47.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:47.842+0000 7f92b3600700 1 -- 172.21.15.107:0/2922828793 shutdown_connections 2024-03-04T13:48:47.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:47.842+0000 7f92b3600700 1 --2- 172.21.15.107:0/2922828793 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f92ac105b10 0x7f92ac0ff180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:48:47.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:47.842+0000 7f92b3600700 1 --2- 172.21.15.107:0/2922828793 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92ac105170 0x7f92ac105540 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:48:47.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:47.842+0000 7f92b3600700 1 -- 172.21.15.107:0/2922828793 >> 172.21.15.107:0/2922828793 conn(0x7f92ac0face0 msgr2=0x7f92ac0fd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:48:47.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:47.842+0000 7f92b3600700 1 -- 172.21.15.107:0/2922828793 shutdown_connections 2024-03-04T13:48:47.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:47.842+0000 7f92b3600700 1 -- 172.21.15.107:0/2922828793 wait complete. 2024-03-04T13:48:47.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:47.843+0000 7f92b3600700 1 Processor -- start 2024-03-04T13:48:47.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:47.843+0000 7f92b3600700 1 -- start start 2024-03-04T13:48:47.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:47.843+0000 7f92b3600700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f92ac105170 0x7f92ac115c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:48:47.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:47.843+0000 7f92b3600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92ac105b10 0x7f92ac1161a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:48:47.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:47.844+0000 7f92b3600700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92ac1167f0 con 0x7f92ac105b10 2024-03-04T13:48:47.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:47.844+0000 7f92b3600700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92ac10fb40 con 0x7f92ac105170 2024-03-04T13:48:47.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:47.844+0000 7f92abe00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92ac105b10 0x7f92ac1161a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:48:47.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:47.844+0000 7f92abe00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92ac105b10 0x7f92ac1161a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47302/0 (socket says 172.21.15.107:47302) 2024-03-04T13:48:47.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:47.844+0000 7f92abe00700 1 -- 172.21.15.107:0/2839087523 learned_addr learned my addr 172.21.15.107:0/2839087523 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:48:47.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:47.844+0000 7f92b0c00700 1 --2- 172.21.15.107:0/2839087523 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f92ac105170 0x7f92ac115c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:48:47.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:47.844+0000 7f92abe00700 1 -- 172.21.15.107:0/2839087523 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f92ac105170 msgr2=0x7f92ac115c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:48:47.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:47.844+0000 7f92abe00700 1 --2- 172.21.15.107:0/2839087523 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f92ac105170 0x7f92ac115c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:48:47.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:47.844+0000 7f92abe00700 1 -- 172.21.15.107:0/2839087523 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92a0005040 con 0x7f92ac105b10 2024-03-04T13:48:47.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:47.845+0000 7f92abe00700 1 --2- 172.21.15.107:0/2839087523 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92ac105b10 0x7f92ac1161a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9298002a50 tx=0x7f9298004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:48:47.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:47.845+0000 7f92a8c00700 1 -- 172.21.15.107:0/2839087523 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9298007cc0 con 0x7f92ac105b10 2024-03-04T13:48:47.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:47.845+0000 7f92a8c00700 1 -- 172.21.15.107:0/2839087523 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f929800f040 con 0x7f92ac105b10 2024-03-04T13:48:47.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:47.845+0000 7f92a8c00700 1 -- 172.21.15.107:0/2839087523 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f929800daf0 con 0x7f92ac105b10 2024-03-04T13:48:47.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:47.845+0000 7f92b3600700 1 -- 172.21.15.107:0/2839087523 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92ac10fe20 con 0x7f92ac105b10 2024-03-04T13:48:47.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:47.845+0000 7f92b3600700 1 -- 172.21.15.107:0/2839087523 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92ac110360 con 0x7f92ac105b10 2024-03-04T13:48:47.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:47.846+0000 7f92b3600700 1 -- 172.21.15.107:0/2839087523 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92ac102480 con 0x7f92ac105b10 2024-03-04T13:48:47.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:47.847+0000 7f92a8c00700 1 -- 172.21.15.107:0/2839087523 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 39) v1 ==== 91587+0+0 (secure 0 0 0) 0x7f9298009810 con 0x7f92ac105b10 2024-03-04T13:48:47.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:47.849+0000 7f92a8c00700 1 --2- 172.21.15.107:0/2839087523 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f929406d900 0x7f929406fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:48:47.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:47.849+0000 7f92a8c00700 1 -- 172.21.15.107:0/2839087523 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f929808ce60 con 0x7f92ac105b10 2024-03-04T13:48:47.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:47.850+0000 7f92b0c00700 1 --2- 172.21.15.107:0/2839087523 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f929406d900 0x7f929406fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:48:47.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:47.850+0000 7f92b0c00700 1 --2- 172.21.15.107:0/2839087523 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f929406d900 0x7f929406fdb0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f92a000b3b0 tx=0x7f92a0009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:48:47.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:47.854+0000 7f92a8c00700 1 -- 172.21.15.107:0/2839087523 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f92980564c0 con 0x7f92ac105b10 2024-03-04T13:48:48.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:47 smithi107 ceph-mon[112578]: from='client.34194 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:48:48.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:47 smithi107 ceph-mon[112578]: pgmap v44: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:48:48.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:47 smithi107 ceph-mon[112578]: from='client.? 172.21.15.107:0/338796288' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:48:48.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:47 smithi107 ceph-mon[112578]: from='client.? 172.21.15.107:0/1566472480' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-04T13:48:48.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:47 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:48:48.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:47 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:48:48.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:47 smithi169 ceph-mon[86183]: from='client.34194 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:48:48.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:47 smithi169 ceph-mon[86183]: pgmap v44: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:48:48.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:47 smithi169 ceph-mon[86183]: from='client.? 172.21.15.107:0/338796288' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:48:48.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:47 smithi169 ceph-mon[86183]: from='client.? 172.21.15.107:0/1566472480' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-04T13:48:48.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:47 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:48:48.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:47 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:48:48.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:48.144+0000 7f92b3600700 1 -- 172.21.15.107:0/2839087523 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f92ac04bed0 con 0x7f92ac105b10 2024-03-04T13:48:48.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:48.145+0000 7f92a8c00700 1 -- 172.21.15.107:0/2839087523 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f9298059b80 con 0x7f92ac105b10 2024-03-04T13:48:48.146 INFO:teuthology.orchestra.run.smithi107.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-03-04T13:48:48.146 INFO:teuthology.orchestra.run.smithi107.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-04T13:48:48.146 INFO:teuthology.orchestra.run.smithi107.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-04T13:48:48.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:48.148+0000 7f92b3600700 1 -- 172.21.15.107:0/2839087523 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f929406d900 msgr2=0x7f929406fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:48:48.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:48.148+0000 7f92b3600700 1 --2- 172.21.15.107:0/2839087523 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f929406d900 0x7f929406fdb0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f92a000b3b0 tx=0x7f92a0009e30 comp rx=0 tx=0).stop 2024-03-04T13:48:48.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:48.148+0000 7f92b3600700 1 -- 172.21.15.107:0/2839087523 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92ac105b10 msgr2=0x7f92ac1161a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:48:48.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:48.148+0000 7f92b3600700 1 --2- 172.21.15.107:0/2839087523 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92ac105b10 0x7f92ac1161a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9298002a50 tx=0x7f9298004600 comp rx=0 tx=0).stop 2024-03-04T13:48:48.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:48.149+0000 7f92b3600700 1 -- 172.21.15.107:0/2839087523 shutdown_connections 2024-03-04T13:48:48.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:48.149+0000 7f92b3600700 1 --2- 172.21.15.107:0/2839087523 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f929406d900 0x7f929406fdb0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:48:48.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:48.149+0000 7f92b3600700 1 --2- 172.21.15.107:0/2839087523 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f92ac105170 0x7f92ac115c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:48:48.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:48.149+0000 7f92b3600700 1 --2- 172.21.15.107:0/2839087523 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92ac105b10 0x7f92ac1161a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:48:48.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:48.149+0000 7f92b3600700 1 -- 172.21.15.107:0/2839087523 >> 172.21.15.107:0/2839087523 conn(0x7f92ac0face0 msgr2=0x7f92ac0fc810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:48:48.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:48.149+0000 7f92b3600700 1 -- 172.21.15.107:0/2839087523 shutdown_connections 2024-03-04T13:48:48.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:48:48.149+0000 7f92b3600700 1 -- 172.21.15.107:0/2839087523 wait complete. 2024-03-04T13:48:48.861 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:48 smithi169 ceph-mon[86183]: from='client.34206 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:48:48.862 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:48 smithi169 ceph-mon[86183]: from='client.? 172.21.15.107:0/2839087523' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-04T13:48:49.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:48 smithi107 ceph-mon[112578]: from='client.34206 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:48:49.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:48 smithi107 ceph-mon[112578]: from='client.? 172.21.15.107:0/2839087523' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-04T13:48:50.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:49 smithi107 ceph-mon[112578]: pgmap v45: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:48:50.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:49 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:48:50.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:49 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi169", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-04T13:48:50.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:49 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:48:50.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:49 smithi169 ceph-mon[86183]: pgmap v45: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:48:50.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:49 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:48:50.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:49 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi169", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-04T13:48:50.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:49 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:48:51.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:50 smithi107 ceph-mon[112578]: Upgrade: Updating crash.smithi169 (2/2) 2024-03-04T13:48:51.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:50 smithi107 ceph-mon[112578]: Deploying daemon crash.smithi169 on smithi169 2024-03-04T13:48:51.093 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:50 smithi169 ceph-mon[86183]: Upgrade: Updating crash.smithi169 (2/2) 2024-03-04T13:48:51.093 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:50 smithi169 ceph-mon[86183]: Deploying daemon crash.smithi169 on smithi169 2024-03-04T13:48:52.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:51 smithi107 ceph-mon[112578]: pgmap v46: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:48:52.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:51 smithi169 ceph-mon[86183]: pgmap v46: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:48:54.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:53 smithi107 ceph-mon[112578]: pgmap v47: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-03-04T13:48:54.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:53 smithi169 ceph-mon[86183]: pgmap v47: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-03-04T13:48:55.843 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:55 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:48:55.843 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:55 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:48:55.843 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:55 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:48:55.844 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:55 smithi107 ceph-mon[112578]: pgmap v48: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-03-04T13:48:55.844 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:55 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:48:55.860 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:55 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:48:55.860 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:55 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:48:55.861 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:55 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:48:55.861 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:55 smithi169 ceph-mon[86183]: pgmap v48: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-03-04T13:48:55.861 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:55 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:48:58.007 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:57 smithi169 ceph-mon[86183]: pgmap v49: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-04T13:48:58.008 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:57 smithi169 ceph-mon[86183]: mgrmap e40: smithi107.wpzgwl(active, since 92s), standbys: smithi169.eihczr 2024-03-04T13:48:58.070 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:57 smithi107 ceph-mon[112578]: pgmap v49: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-04T13:48:58.070 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:57 smithi107 ceph-mon[112578]: mgrmap e40: smithi107.wpzgwl(active, since 92s), standbys: smithi169.eihczr 2024-03-04T13:49:00.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:59 smithi169 ceph-mon[86183]: pgmap v50: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 767 B/s rd, 255 B/s wr, 1 op/s 2024-03-04T13:49:00.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:59 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:49:00.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:48:59 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:49:00.171 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:59 smithi107 ceph-mon[112578]: pgmap v50: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 767 B/s rd, 255 B/s wr, 1 op/s 2024-03-04T13:49:00.171 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:59 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:49:00.171 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:48:59 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:49:02.105 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:01 smithi169 ceph-mon[86183]: pgmap v51: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-04T13:49:02.105 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:01 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:49:02.105 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:01 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:49:02.106 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:01 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:49:02.106 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:01 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:49:02.343 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:01 smithi107 ceph-mon[112578]: pgmap v51: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-04T13:49:02.343 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:01 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:49:02.343 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:01 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:49:02.343 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:01 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:49:02.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:01 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:49:04.343 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:03 smithi107 ceph-mon[112578]: pgmap v52: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 938 B/s rd, 255 B/s wr, 1 op/s 2024-03-04T13:49:04.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:03 smithi169 ceph-mon[86183]: pgmap v52: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 938 B/s rd, 255 B/s wr, 1 op/s 2024-03-04T13:49:05.244 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:04 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:49:05.244 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:04 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:49:05.246 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:04 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:49:05.246 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:04 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:49:05.915 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:05 smithi107 ceph-mon[112578]: pgmap v53: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 938 B/s rd, 255 B/s wr, 1 op/s 2024-03-04T13:49:06.360 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:05 smithi169 ceph-mon[86183]: pgmap v53: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 938 B/s rd, 255 B/s wr, 1 op/s 2024-03-04T13:49:07.343 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:07 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:49:07.343 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:07 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:49:07.343 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:07 smithi107 ceph-mon[112578]: pgmap v54: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-04T13:49:07.360 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:07 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:49:07.360 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:07 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:49:07.360 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:07 smithi169 ceph-mon[86183]: pgmap v54: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-04T13:49:10.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:09 smithi107 ceph-mon[112578]: pgmap v55: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 938 B/s rd, 1 op/s 2024-03-04T13:49:10.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:09 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:49:10.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:09 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:49:10.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:09 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:49:10.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:09 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:49:10.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:09 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:49:10.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:09 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:49:10.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:09 smithi169 ceph-mon[86183]: pgmap v55: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 938 B/s rd, 1 op/s 2024-03-04T13:49:10.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:09 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:49:10.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:09 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:49:10.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:09 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:49:10.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:09 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:49:10.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:09 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:49:10.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:09 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:49:11.057 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:10 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:49:11.057 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:10 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:49:11.057 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:10 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:49:11.057 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:10 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:49:11.057 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:10 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:49:11.057 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:10 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi107"}]: dispatch 2024-03-04T13:49:11.057 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:10 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi107"}]': finished 2024-03-04T13:49:11.057 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:10 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi169"}]: dispatch 2024-03-04T13:49:11.057 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:10 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi169"}]': finished 2024-03-04T13:49:11.057 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:10 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-03-04T13:49:11.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:10 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:49:11.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:10 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:49:11.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:10 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:49:11.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:10 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:49:11.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:10 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:49:11.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:10 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi107"}]: dispatch 2024-03-04T13:49:11.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:10 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi107"}]': finished 2024-03-04T13:49:11.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:10 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi169"}]: dispatch 2024-03-04T13:49:11.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:10 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi169"}]': finished 2024-03-04T13:49:11.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:10 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-03-04T13:49:12.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:11 smithi107 ceph-mon[112578]: Upgrade: Setting container_image for all crash 2024-03-04T13:49:12.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:11 smithi107 ceph-mon[112578]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-03-04T13:49:12.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:11 smithi107 ceph-mon[112578]: Upgrade: osd.0 is safe to restart 2024-03-04T13:49:12.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:11 smithi107 ceph-mon[112578]: pgmap v56: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.3 KiB/s rd, 2 op/s 2024-03-04T13:49:12.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:11 smithi169 ceph-mon[86183]: Upgrade: Setting container_image for all crash 2024-03-04T13:49:12.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:11 smithi169 ceph-mon[86183]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-03-04T13:49:12.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:11 smithi169 ceph-mon[86183]: Upgrade: osd.0 is safe to restart 2024-03-04T13:49:12.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:11 smithi169 ceph-mon[86183]: pgmap v56: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.3 KiB/s rd, 2 op/s 2024-03-04T13:49:13.458 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:13 smithi107 ceph-mon[112578]: Upgrade: Updating osd.0 2024-03-04T13:49:13.458 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:13 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:49:13.458 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:13 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-04T13:49:13.458 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:13 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:49:13.458 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:13 smithi107 ceph-mon[112578]: Deploying daemon osd.0 on smithi107 2024-03-04T13:49:13.458 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:13 smithi107 ceph-mon[112578]: pgmap v57: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:49:13.611 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:13 smithi169 ceph-mon[86183]: Upgrade: Updating osd.0 2024-03-04T13:49:13.611 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:13 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:49:13.611 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:13 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-04T13:49:13.611 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:13 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:49:13.611 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:13 smithi169 ceph-mon[86183]: Deploying daemon osd.0 on smithi107 2024-03-04T13:49:13.611 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:13 smithi169 ceph-mon[86183]: pgmap v57: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:49:15.439 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:15 smithi107 ceph-mon[112578]: pgmap v58: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:49:15.610 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:15 smithi169 ceph-mon[86183]: pgmap v58: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:49:17.327 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 04 13:49:16 smithi107 systemd[1]: Stopping Ceph osd.0 for 1fc09a58-da2b-11ee-95c6-87774f69a715... 2024-03-04T13:49:17.594 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 04 13:49:17 smithi107 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-0[46870]: 2024-03-04T13:49:17.325+0000 7f5f84c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-04T13:49:17.594 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 04 13:49:17 smithi107 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-0[46870]: 2024-03-04T13:49:17.325+0000 7f5f84c00700 -1 osd.0 50 *** Got signal Terminated *** 2024-03-04T13:49:17.594 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 04 13:49:17 smithi107 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-0[46870]: 2024-03-04T13:49:17.325+0000 7f5f84c00700 -1 osd.0 50 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-04T13:49:18.047 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:17 smithi169 ceph-mon[86183]: pgmap v59: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:49:18.047 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:17 smithi169 ceph-mon[86183]: osd.0 marked itself down and dead 2024-03-04T13:49:18.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:17 smithi107 ceph-mon[112578]: pgmap v59: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:49:18.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:17 smithi107 ceph-mon[112578]: osd.0 marked itself down and dead 2024-03-04T13:49:18.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:18.314+0000 7f4847400700 1 -- 172.21.15.107:0/3935397380 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4840103dc0 msgr2=0x7f4840104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:49:18.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:18.314+0000 7f4847400700 1 --2- 172.21.15.107:0/3935397380 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4840103dc0 0x7f4840104210 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f482c004940 tx=0x7f482c009e30 comp rx=0 tx=0).stop 2024-03-04T13:49:18.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:18.314+0000 7f4847400700 1 -- 172.21.15.107:0/3935397380 shutdown_connections 2024-03-04T13:49:18.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:18.314+0000 7f4847400700 1 --2- 172.21.15.107:0/3935397380 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4840103dc0 0x7f4840104210 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:49:18.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:18.314+0000 7f4847400700 1 --2- 172.21.15.107:0/3935397380 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f48401095f0 0x7f48401099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:49:18.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:18.314+0000 7f4847400700 1 -- 172.21.15.107:0/3935397380 >> 172.21.15.107:0/3935397380 conn(0x7f4840072d50 msgr2=0x7f4840075160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:49:18.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:18.314+0000 7f4847400700 1 -- 172.21.15.107:0/3935397380 shutdown_connections 2024-03-04T13:49:18.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:18.314+0000 7f4847400700 1 -- 172.21.15.107:0/3935397380 wait complete. 2024-03-04T13:49:18.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:18.315+0000 7f4847400700 1 Processor -- start 2024-03-04T13:49:18.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:18.315+0000 7f4847400700 1 -- start start 2024-03-04T13:49:18.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:18.316+0000 7f4847400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4840103dc0 0x7f484011a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:49:18.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:18.316+0000 7f4847400700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f48401095f0 0x7f484011a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:49:18.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:18.316+0000 7f4847400700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f484011ac50 con 0x7f4840103dc0 2024-03-04T13:49:18.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:18.316+0000 7f4847400700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4840113ed0 con 0x7f48401095f0 2024-03-04T13:49:18.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:18.316+0000 7f4844a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4840103dc0 0x7f484011a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:49:18.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:18.316+0000 7f4844a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4840103dc0 0x7f484011a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36196/0 (socket says 172.21.15.107:36196) 2024-03-04T13:49:18.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:18.316+0000 7f4844a00700 1 -- 172.21.15.107:0/2568721708 learned_addr learned my addr 172.21.15.107:0/2568721708 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:49:18.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:18.316+0000 7f483fe00700 1 --2- 172.21.15.107:0/2568721708 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f48401095f0 0x7f484011a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:49:18.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:18.316+0000 7f4844a00700 1 -- 172.21.15.107:0/2568721708 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f48401095f0 msgr2=0x7f484011a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:49:18.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:18.316+0000 7f4844a00700 1 --2- 172.21.15.107:0/2568721708 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f48401095f0 0x7f484011a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:49:18.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:18.316+0000 7f4844a00700 1 -- 172.21.15.107:0/2568721708 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f482c005040 con 0x7f4840103dc0 2024-03-04T13:49:18.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:18.316+0000 7f483fe00700 1 --2- 172.21.15.107:0/2568721708 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f48401095f0 0x7f484011a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T13:49:18.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:18.317+0000 7f4844a00700 1 --2- 172.21.15.107:0/2568721708 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4840103dc0 0x7f484011a030 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f483400b790 tx=0x7f4834009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:49:18.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:18.317+0000 7f483cc00700 1 -- 172.21.15.107:0/2568721708 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f483400f770 con 0x7f4840103dc0 2024-03-04T13:49:18.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:18.317+0000 7f483cc00700 1 -- 172.21.15.107:0/2568721708 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f483400fda0 con 0x7f4840103dc0 2024-03-04T13:49:18.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:18.317+0000 7f4847400700 1 -- 172.21.15.107:0/2568721708 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4840114150 con 0x7f4840103dc0 2024-03-04T13:49:18.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:18.317+0000 7f483cc00700 1 -- 172.21.15.107:0/2568721708 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f483400d310 con 0x7f4840103dc0 2024-03-04T13:49:18.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:18.317+0000 7f4847400700 1 -- 172.21.15.107:0/2568721708 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4840114690 con 0x7f4840103dc0 2024-03-04T13:49:18.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:18.318+0000 7f4847400700 1 -- 172.21.15.107:0/2568721708 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4840075870 con 0x7f4840103dc0 2024-03-04T13:49:18.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:18.319+0000 7f483cc00700 1 -- 172.21.15.107:0/2568721708 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f4834010420 con 0x7f4840103dc0 2024-03-04T13:49:18.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:18.320+0000 7f483cc00700 1 --2- 172.21.15.107:0/2568721708 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f482806d7f0 0x7f482806fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:49:18.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:18.320+0000 7f483cc00700 1 -- 172.21.15.107:0/2568721708 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f483408cf20 con 0x7f4840103dc0 2024-03-04T13:49:18.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:18.321+0000 7f483fe00700 1 --2- 172.21.15.107:0/2568721708 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f482806d7f0 0x7f482806fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:49:18.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:18.321+0000 7f483fe00700 1 --2- 172.21.15.107:0/2568721708 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f482806d7f0 0x7f482806fca0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f482c00b3b0 tx=0x7f482c009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:49:18.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:18.325+0000 7f483cc00700 1 -- 172.21.15.107:0/2568721708 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f483405a310 con 0x7f4840103dc0 2024-03-04T13:49:18.550 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 04 13:49:18 smithi107 podman[123643]: 2024-03-04 13:49:18.200552441 +0000 UTC m=+1.005149056 container died c0605b68f69680598a9e0a4ef25a082b589a0cb9fbd73ce641fe050eea163df3 (image=quay.ceph.io/ceph-ci/ceph@sha256:ee85afea73f504aaae524393a00a3b140e2a1cfd81d76da4525eaa6e73286a7d, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=quincy-42608bb, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-03-04T13:49:18.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:18.569+0000 7f4847400700 1 -- 172.21.15.107:0/2568721708 --> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4840115720 con 0x7f482806d7f0 2024-03-04T13:49:18.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:18.571+0000 7f483cc00700 1 -- 172.21.15.107:0/2568721708 <== mgr.34102 v2:172.21.15.107:6800/1223039812 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f4840115720 con 0x7f482806d7f0 2024-03-04T13:49:18.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:18.574+0000 7f4847400700 1 -- 172.21.15.107:0/2568721708 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f482806d7f0 msgr2=0x7f482806fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:49:18.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:18.574+0000 7f4847400700 1 --2- 172.21.15.107:0/2568721708 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f482806d7f0 0x7f482806fca0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f482c00b3b0 tx=0x7f482c009e30 comp rx=0 tx=0).stop 2024-03-04T13:49:18.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:18.574+0000 7f4847400700 1 -- 172.21.15.107:0/2568721708 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4840103dc0 msgr2=0x7f484011a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:49:18.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:18.574+0000 7f4847400700 1 --2- 172.21.15.107:0/2568721708 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4840103dc0 0x7f484011a030 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f483400b790 tx=0x7f4834009250 comp rx=0 tx=0).stop 2024-03-04T13:49:18.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:18.574+0000 7f4847400700 1 -- 172.21.15.107:0/2568721708 shutdown_connections 2024-03-04T13:49:18.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:18.574+0000 7f4847400700 1 --2- 172.21.15.107:0/2568721708 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f482806d7f0 0x7f482806fca0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:49:18.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:18.574+0000 7f4847400700 1 --2- 172.21.15.107:0/2568721708 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4840103dc0 0x7f484011a030 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:49:18.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:18.574+0000 7f4847400700 1 --2- 172.21.15.107:0/2568721708 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f48401095f0 0x7f484011a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:49:18.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:18.574+0000 7f4847400700 1 -- 172.21.15.107:0/2568721708 >> 172.21.15.107:0/2568721708 conn(0x7f4840072d50 msgr2=0x7f48400746b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:49:18.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:18.574+0000 7f4847400700 1 -- 172.21.15.107:0/2568721708 shutdown_connections 2024-03-04T13:49:18.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:18.574+0000 7f4847400700 1 -- 172.21.15.107:0/2568721708 wait complete. 2024-03-04T13:49:18.590 INFO:teuthology.orchestra.run.smithi107.stdout:true 2024-03-04T13:49:18.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:18.725+0000 7f9b55600700 1 -- 172.21.15.107:0/645371262 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b501038f0 msgr2=0x7f9b500fcf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:49:18.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:18.725+0000 7f9b55600700 1 --2- 172.21.15.107:0/645371262 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b501038f0 0x7f9b500fcf60 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c004940 tx=0x7f9b3c009e30 comp rx=0 tx=0).stop 2024-03-04T13:49:18.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:18.726+0000 7f9b55600700 1 -- 172.21.15.107:0/645371262 shutdown_connections 2024-03-04T13:49:18.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:18.726+0000 7f9b55600700 1 --2- 172.21.15.107:0/645371262 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b501038f0 0x7f9b500fcf60 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:49:18.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:18.726+0000 7f9b55600700 1 --2- 172.21.15.107:0/645371262 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b50102f50 0x7f9b50103320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:49:18.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:18.726+0000 7f9b55600700 1 -- 172.21.15.107:0/645371262 >> 172.21.15.107:0/645371262 conn(0x7f9b500f8ae0 msgr2=0x7f9b500faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:49:18.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:18.727+0000 7f9b55600700 1 -- 172.21.15.107:0/645371262 shutdown_connections 2024-03-04T13:49:18.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:18.727+0000 7f9b55600700 1 -- 172.21.15.107:0/645371262 wait complete. 2024-03-04T13:49:18.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:18.728+0000 7f9b55600700 1 Processor -- start 2024-03-04T13:49:18.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:18.729+0000 7f9b55600700 1 -- start start 2024-03-04T13:49:18.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:18.729+0000 7f9b55600700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b50102f50 0x7f9b50070290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:49:18.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:18.729+0000 7f9b55600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b501038f0 0x7f9b500707d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:49:18.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:18.729+0000 7f9b55600700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b50070dd0 con 0x7f9b501038f0 2024-03-04T13:49:18.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:18.729+0000 7f9b55600700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b50070f40 con 0x7f9b50102f50 2024-03-04T13:49:18.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:18.730+0000 7f9b4e000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b501038f0 0x7f9b500707d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:49:18.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:18.731+0000 7f9b4ea00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b50102f50 0x7f9b50070290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:49:18.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:18.731+0000 7f9b4ea00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b50102f50 0x7f9b50070290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.107:36670/0 (socket says 172.21.15.107:36670) 2024-03-04T13:49:18.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:18.731+0000 7f9b4ea00700 1 -- 172.21.15.107:0/3967936813 learned_addr learned my addr 172.21.15.107:0/3967936813 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:49:18.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:18.730+0000 7f9b4e000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b501038f0 0x7f9b500707d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36204/0 (socket says 172.21.15.107:36204) 2024-03-04T13:49:18.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:18.732+0000 7f9b4e000700 1 -- 172.21.15.107:0/3967936813 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b50102f50 msgr2=0x7f9b50070290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:49:18.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:18.732+0000 7f9b4e000700 1 --2- 172.21.15.107:0/3967936813 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b50102f50 0x7f9b50070290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:49:18.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:18.732+0000 7f9b4e000700 1 -- 172.21.15.107:0/3967936813 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b3c005040 con 0x7f9b501038f0 2024-03-04T13:49:18.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:18.732+0000 7f9b4e000700 1 --2- 172.21.15.107:0/3967936813 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b501038f0 0x7f9b500707d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c00b3b0 tx=0x7f9b3c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:49:18.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:18.733+0000 7f9b46a00700 1 -- 172.21.15.107:0/3967936813 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b3c00e040 con 0x7f9b501038f0 2024-03-04T13:49:18.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:18.733+0000 7f9b55600700 1 -- 172.21.15.107:0/3967936813 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b50119cb0 con 0x7f9b501038f0 2024-03-04T13:49:18.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:18.733+0000 7f9b46a00700 1 -- 172.21.15.107:0/3967936813 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9b3c0093d0 con 0x7f9b501038f0 2024-03-04T13:49:18.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:18.733+0000 7f9b46a00700 1 -- 172.21.15.107:0/3967936813 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b3c01c3f0 con 0x7f9b501038f0 2024-03-04T13:49:18.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:18.733+0000 7f9b55600700 1 -- 172.21.15.107:0/3967936813 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b5011a110 con 0x7f9b501038f0 2024-03-04T13:49:18.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:18.735+0000 7f9b55600700 1 -- 172.21.15.107:0/3967936813 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b50105420 con 0x7f9b501038f0 2024-03-04T13:49:18.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:18.739+0000 7f9b46a00700 1 -- 172.21.15.107:0/3967936813 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f9b3c00bbf0 con 0x7f9b501038f0 2024-03-04T13:49:18.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:18.740+0000 7f9b46a00700 1 --2- 172.21.15.107:0/3967936813 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f9b3406d7f0 0x7f9b3406fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:49:18.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:18.740+0000 7f9b46a00700 1 -- 172.21.15.107:0/3967936813 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f9b3c08dfd0 con 0x7f9b501038f0 2024-03-04T13:49:18.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:18.741+0000 7f9b4ea00700 1 --2- 172.21.15.107:0/3967936813 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f9b3406d7f0 0x7f9b3406fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:49:18.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:18.742+0000 7f9b4ea00700 1 --2- 172.21.15.107:0/3967936813 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f9b3406d7f0 0x7f9b3406fca0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9b400041d0 tx=0x7f9b40004470 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:49:18.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:18.751+0000 7f9b46a00700 1 -- 172.21.15.107:0/3967936813 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9b3c057910 con 0x7f9b501038f0 2024-03-04T13:49:18.843 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:18 smithi107 ceph-mon[112578]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-04T13:49:18.843 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:18 smithi107 ceph-mon[112578]: osdmap e51: 6 total, 5 up, 6 in 2024-03-04T13:49:18.843 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:18 smithi107 ceph-mon[112578]: osdmap e52: 6 total, 5 up, 6 in 2024-03-04T13:49:18.844 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 04 13:49:18 smithi107 podman[123643]: 2024-03-04 13:49:18.678091238 +0000 UTC m=+1.482687891 container cleanup c0605b68f69680598a9e0a4ef25a082b589a0cb9fbd73ce641fe050eea163df3 (image=quay.ceph.io/ceph-ci/ceph@sha256:ee85afea73f504aaae524393a00a3b140e2a1cfd81d76da4525eaa6e73286a7d, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=quincy-42608bb, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222) 2024-03-04T13:49:18.844 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 04 13:49:18 smithi107 bash[123643]: ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-0 2024-03-04T13:49:19.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:18.999+0000 7f9b55600700 1 -- 172.21.15.107:0/3967936813 --> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9b5004bf70 con 0x7f9b3406d7f0 2024-03-04T13:49:19.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.002+0000 7f9b46a00700 1 -- 172.21.15.107:0/3967936813 <== mgr.34102 v2:172.21.15.107:6800/1223039812 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f9b4000f9e0 con 0x7f9b3406d7f0 2024-03-04T13:49:19.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.005+0000 7f9b55600700 1 -- 172.21.15.107:0/3967936813 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f9b3406d7f0 msgr2=0x7f9b3406fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:49:19.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.005+0000 7f9b55600700 1 --2- 172.21.15.107:0/3967936813 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f9b3406d7f0 0x7f9b3406fca0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9b400041d0 tx=0x7f9b40004470 comp rx=0 tx=0).stop 2024-03-04T13:49:19.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.005+0000 7f9b55600700 1 -- 172.21.15.107:0/3967936813 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b501038f0 msgr2=0x7f9b500707d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:49:19.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.005+0000 7f9b55600700 1 --2- 172.21.15.107:0/3967936813 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b501038f0 0x7f9b500707d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c00b3b0 tx=0x7f9b3c009e30 comp rx=0 tx=0).stop 2024-03-04T13:49:19.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.005+0000 7f9b55600700 1 -- 172.21.15.107:0/3967936813 shutdown_connections 2024-03-04T13:49:19.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.005+0000 7f9b55600700 1 --2- 172.21.15.107:0/3967936813 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f9b3406d7f0 0x7f9b3406fca0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:49:19.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.005+0000 7f9b55600700 1 --2- 172.21.15.107:0/3967936813 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b50102f50 0x7f9b50070290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:49:19.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.005+0000 7f9b55600700 1 --2- 172.21.15.107:0/3967936813 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b501038f0 0x7f9b500707d0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:49:19.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.005+0000 7f9b55600700 1 -- 172.21.15.107:0/3967936813 >> 172.21.15.107:0/3967936813 conn(0x7f9b500f8ae0 msgr2=0x7f9b500fa570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:49:19.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.005+0000 7f9b55600700 1 -- 172.21.15.107:0/3967936813 shutdown_connections 2024-03-04T13:49:19.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.005+0000 7f9b55600700 1 -- 172.21.15.107:0/3967936813 wait complete. 2024-03-04T13:49:19.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:18 smithi169 ceph-mon[86183]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-04T13:49:19.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:18 smithi169 ceph-mon[86183]: osdmap e51: 6 total, 5 up, 6 in 2024-03-04T13:49:19.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:18 smithi169 ceph-mon[86183]: osdmap e52: 6 total, 5 up, 6 in 2024-03-04T13:49:19.169 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 04 13:49:18 smithi107 podman[123665]: 2024-03-04 13:49:18.904443075 +0000 UTC m=+0.697357035 container remove c0605b68f69680598a9e0a4ef25a082b589a0cb9fbd73ce641fe050eea163df3 (image=quay.ceph.io/ceph-ci/ceph@sha256:ee85afea73f504aaae524393a00a3b140e2a1cfd81d76da4525eaa6e73286a7d, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-0, io.buildah.version=1.33.5, org.label-schema.build-date=20240222, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=quincy-42608bb, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-04T13:49:19.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.168+0000 7eff31a00700 1 -- 172.21.15.107:0/1394621394 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff2c0fdfc0 msgr2=0x7eff2c0fe390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:49:19.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.168+0000 7eff31a00700 1 --2- 172.21.15.107:0/1394621394 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff2c0fdfc0 0x7eff2c0fe390 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7eff1c004940 tx=0x7eff1c009e30 comp rx=0 tx=0).stop 2024-03-04T13:49:19.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.169+0000 7eff31a00700 1 -- 172.21.15.107:0/1394621394 shutdown_connections 2024-03-04T13:49:19.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.169+0000 7eff31a00700 1 --2- 172.21.15.107:0/1394621394 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff2c0fe960 0x7eff2c106fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:49:19.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.169+0000 7eff31a00700 1 --2- 172.21.15.107:0/1394621394 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff2c0fdfc0 0x7eff2c0fe390 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:49:19.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.169+0000 7eff31a00700 1 -- 172.21.15.107:0/1394621394 >> 172.21.15.107:0/1394621394 conn(0x7eff2c0f7a10 msgr2=0x7eff2c0f9e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:49:19.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.169+0000 7eff31a00700 1 -- 172.21.15.107:0/1394621394 shutdown_connections 2024-03-04T13:49:19.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.169+0000 7eff31a00700 1 -- 172.21.15.107:0/1394621394 wait complete. 2024-03-04T13:49:19.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.170+0000 7eff31a00700 1 Processor -- start 2024-03-04T13:49:19.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.170+0000 7eff31a00700 1 -- start start 2024-03-04T13:49:19.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.171+0000 7eff31a00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff2c0fdfc0 0x7eff2c115bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:49:19.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.171+0000 7eff31a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff2c0fe960 0x7eff2c116130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:49:19.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.171+0000 7eff31a00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff2c116810 con 0x7eff2c0fe960 2024-03-04T13:49:19.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.171+0000 7eff31a00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff2c10fab0 con 0x7eff2c0fdfc0 2024-03-04T13:49:19.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.171+0000 7eff2a000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff2c0fe960 0x7eff2c116130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:49:19.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.171+0000 7eff2a000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff2c0fe960 0x7eff2c116130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36222/0 (socket says 172.21.15.107:36222) 2024-03-04T13:49:19.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.171+0000 7eff2a000700 1 -- 172.21.15.107:0/2571403714 learned_addr learned my addr 172.21.15.107:0/2571403714 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:49:19.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.171+0000 7eff2aa00700 1 --2- 172.21.15.107:0/2571403714 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff2c0fdfc0 0x7eff2c115bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:49:19.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.172+0000 7eff2a000700 1 -- 172.21.15.107:0/2571403714 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff2c0fdfc0 msgr2=0x7eff2c115bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:49:19.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.172+0000 7eff2a000700 1 --2- 172.21.15.107:0/2571403714 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff2c0fdfc0 0x7eff2c115bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:49:19.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.172+0000 7eff2a000700 1 -- 172.21.15.107:0/2571403714 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff1c005040 con 0x7eff2c0fe960 2024-03-04T13:49:19.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.172+0000 7eff2a000700 1 --2- 172.21.15.107:0/2571403714 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff2c0fe960 0x7eff2c116130 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7eff1800b790 tx=0x7eff18009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:49:19.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.172+0000 7eff22a00700 1 -- 172.21.15.107:0/2571403714 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7eff1800f770 con 0x7eff2c0fe960 2024-03-04T13:49:19.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.172+0000 7eff31a00700 1 -- 172.21.15.107:0/2571403714 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff2c10fd90 con 0x7eff2c0fe960 2024-03-04T13:49:19.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.172+0000 7eff22a00700 1 -- 172.21.15.107:0/2571403714 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7eff1800fda0 con 0x7eff2c0fe960 2024-03-04T13:49:19.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.172+0000 7eff22a00700 1 -- 172.21.15.107:0/2571403714 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7eff1800d310 con 0x7eff2c0fe960 2024-03-04T13:49:19.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.173+0000 7eff31a00700 1 -- 172.21.15.107:0/2571403714 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff2c1102d0 con 0x7eff2c0fe960 2024-03-04T13:49:19.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.174+0000 7eff31a00700 1 -- 172.21.15.107:0/2571403714 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff2c106a10 con 0x7eff2c0fe960 2024-03-04T13:49:19.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.175+0000 7eff22a00700 1 -- 172.21.15.107:0/2571403714 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7eff18010420 con 0x7eff2c0fe960 2024-03-04T13:49:19.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.176+0000 7eff22a00700 1 --2- 172.21.15.107:0/2571403714 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7eff1006d7f0 0x7eff1006fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:49:19.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.176+0000 7eff22a00700 1 -- 172.21.15.107:0/2571403714 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6324+0+0 (secure 0 0 0) 0x7eff1808cd80 con 0x7eff2c0fe960 2024-03-04T13:49:19.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.177+0000 7eff2aa00700 1 --2- 172.21.15.107:0/2571403714 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7eff1006d7f0 0x7eff1006fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:49:19.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.177+0000 7eff2aa00700 1 --2- 172.21.15.107:0/2571403714 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7eff1006d7f0 0x7eff1006fca0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7eff1c00b3b0 tx=0x7eff1c009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:49:19.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.181+0000 7eff22a00700 1 -- 172.21.15.107:0/2571403714 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7eff1805a270 con 0x7eff2c0fe960 2024-03-04T13:49:19.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.416+0000 7eff31a00700 1 -- 172.21.15.107:0/2571403714 --> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7eff2c10a000 con 0x7eff1006d7f0 2024-03-04T13:49:19.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.427+0000 7eff22a00700 1 -- 172.21.15.107:0/2571403714 <== mgr.34102 v2:172.21.15.107:6800/1223039812 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7eff2c10a000 con 0x7eff1006d7f0 2024-03-04T13:49:19.428 INFO:teuthology.orchestra.run.smithi107.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-04T13:49:19.429 INFO:teuthology.orchestra.run.smithi107.stdout:alertmanager.smithi107 smithi107 *:9093,9094 running (3m) 17s ago 17m 14.0M - 0.25.0 c8568f914cd2 e5ed7be32538 2024-03-04T13:49:19.429 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi107 smithi107 running (32s) 17s ago 18m 6660k - 19.0.0-1845-g717ce3cc d71b839abdf8 0558022b0d84 2024-03-04T13:49:19.429 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi169 smithi169 running (25s) 20s ago 16m 6665k - 19.0.0-1845-g717ce3cc d71b839abdf8 221ca53aa0d3 2024-03-04T13:49:19.429 INFO:teuthology.orchestra.run.smithi107.stdout:grafana.smithi107 smithi107 *:3000 running (3m) 17s ago 17m 74.5M - 9.4.12 a0ea5e3c46d9 c5e4d33fb599 2024-03-04T13:49:19.429 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.riexmm smithi107 running (10m) 17s ago 10m 14.6M - 17.2.7-545-g42608bb1 9eec3f22859d fc921f7d4328 2024-03-04T13:49:19.429 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.thycho smithi107 running (10m) 17s ago 10m 98.2M - 17.2.7-545-g42608bb1 9eec3f22859d 4069c6e0e063 2024-03-04T13:49:19.429 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi169.bxitzk smithi169 running (10m) 20s ago 10m 14.2M - 17.2.7-545-g42608bb1 9eec3f22859d 7d4115228c7c 2024-03-04T13:49:19.429 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi169.jusudl smithi169 running (10m) 20s ago 10m 145M - 17.2.7-545-g42608bb1 9eec3f22859d 2d2f2cc452e8 2024-03-04T13:49:19.429 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi107.wpzgwl smithi107 *:8443,9283,8765 running (6m) 17s ago 19m 495M - 19.0.0-1845-g717ce3cc d71b839abdf8 1c5fdee45eac 2024-03-04T13:49:19.429 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi169.eihczr smithi169 *:8443,9283,8765 running (5m) 20s ago 16m 449M - 19.0.0-1845-g717ce3cc d71b839abdf8 b5cac2b7d22a 2024-03-04T13:49:19.429 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi107 smithi107 running (2m) 17s ago 19m 40.7M 2048M 19.0.0-1845-g717ce3cc d71b839abdf8 32f07144274f 2024-03-04T13:49:19.429 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi169 smithi169 running (89s) 20s ago 16m 33.9M 2048M 19.0.0-1845-g717ce3cc d71b839abdf8 e3527449381a 2024-03-04T13:49:19.429 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi107 smithi107 *:9100 running (4m) 17s ago 17m 7801k - 1.5.0 0da6a335fe13 8a8d3d4f724f 2024-03-04T13:49:19.430 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi169 smithi169 *:9100 running (4m) 20s ago 16m 7688k - 1.5.0 0da6a335fe13 f240f5a975bc 2024-03-04T13:49:19.430 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0 smithi107 running (15m) 17s ago 15m 207M 1558M 17.2.7-545-g42608bb1 9eec3f22859d c0605b68f696 2024-03-04T13:49:19.430 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1 smithi107 running (14m) 17s ago 14m 220M 1558M 17.2.7-545-g42608bb1 9eec3f22859d 5fb2c0dd5769 2024-03-04T13:49:19.430 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2 smithi107 running (14m) 17s ago 14m 182M 1558M 17.2.7-545-g42608bb1 9eec3f22859d 61f30f643549 2024-03-04T13:49:19.430 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3 smithi169 running (13m) 20s ago 13m 236M 2582M 17.2.7-545-g42608bb1 9eec3f22859d cd91d8694764 2024-03-04T13:49:19.430 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4 smithi169 running (13m) 20s ago 13m 207M 2582M 17.2.7-545-g42608bb1 9eec3f22859d 08545e3d0659 2024-03-04T13:49:19.430 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5 smithi169 running (12m) 20s ago 12m 194M 2582M 17.2.7-545-g42608bb1 9eec3f22859d dcffe6cb3eac 2024-03-04T13:49:19.430 INFO:teuthology.orchestra.run.smithi107.stdout:prometheus.smithi107 smithi107 *:9095 running (4m) 17s ago 17m 52.3M - 2.43.0 a07b618ecd1d 1eeb1ad5cb75 2024-03-04T13:49:19.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.430+0000 7eff31a00700 1 -- 172.21.15.107:0/2571403714 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7eff1006d7f0 msgr2=0x7eff1006fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:49:19.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.430+0000 7eff31a00700 1 --2- 172.21.15.107:0/2571403714 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7eff1006d7f0 0x7eff1006fca0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7eff1c00b3b0 tx=0x7eff1c009e30 comp rx=0 tx=0).stop 2024-03-04T13:49:19.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.430+0000 7eff31a00700 1 -- 172.21.15.107:0/2571403714 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff2c0fe960 msgr2=0x7eff2c116130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:49:19.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.430+0000 7eff31a00700 1 --2- 172.21.15.107:0/2571403714 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff2c0fe960 0x7eff2c116130 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7eff1800b790 tx=0x7eff18009250 comp rx=0 tx=0).stop 2024-03-04T13:49:19.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.430+0000 7eff31a00700 1 -- 172.21.15.107:0/2571403714 shutdown_connections 2024-03-04T13:49:19.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.430+0000 7eff31a00700 1 --2- 172.21.15.107:0/2571403714 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7eff1006d7f0 0x7eff1006fca0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:49:19.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.430+0000 7eff31a00700 1 --2- 172.21.15.107:0/2571403714 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7eff2c0fdfc0 0x7eff2c115bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:49:19.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.430+0000 7eff31a00700 1 --2- 172.21.15.107:0/2571403714 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff2c0fe960 0x7eff2c116130 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:49:19.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.431+0000 7eff31a00700 1 -- 172.21.15.107:0/2571403714 >> 172.21.15.107:0/2571403714 conn(0x7eff2c0f7a10 msgr2=0x7eff2c0fc5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:49:19.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.431+0000 7eff31a00700 1 -- 172.21.15.107:0/2571403714 shutdown_connections 2024-03-04T13:49:19.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.431+0000 7eff31a00700 1 -- 172.21.15.107:0/2571403714 wait complete. 2024-03-04T13:49:19.585 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.584+0000 7fafdfc00700 1 -- 172.21.15.107:0/1598075344 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fafd8102f50 msgr2=0x7fafd8103320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:49:19.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.584+0000 7fafdfc00700 1 --2- 172.21.15.107:0/1598075344 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fafd8102f50 0x7fafd8103320 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fafc8004940 tx=0x7fafc8009e30 comp rx=0 tx=0).stop 2024-03-04T13:49:19.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.585+0000 7fafdfc00700 1 -- 172.21.15.107:0/1598075344 shutdown_connections 2024-03-04T13:49:19.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.585+0000 7fafdfc00700 1 --2- 172.21.15.107:0/1598075344 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fafd81038f0 0x7fafd80fcf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:49:19.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.585+0000 7fafdfc00700 1 --2- 172.21.15.107:0/1598075344 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fafd8102f50 0x7fafd8103320 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:49:19.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.585+0000 7fafdfc00700 1 -- 172.21.15.107:0/1598075344 >> 172.21.15.107:0/1598075344 conn(0x7fafd80f8ae0 msgr2=0x7fafd80faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:49:19.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.585+0000 7fafdfc00700 1 -- 172.21.15.107:0/1598075344 shutdown_connections 2024-03-04T13:49:19.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.585+0000 7fafdfc00700 1 -- 172.21.15.107:0/1598075344 wait complete. 2024-03-04T13:49:19.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.586+0000 7fafdfc00700 1 Processor -- start 2024-03-04T13:49:19.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.586+0000 7fafdfc00700 1 -- start start 2024-03-04T13:49:19.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.586+0000 7fafdfc00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fafd8102f50 0x7fafd8070290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:49:19.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.586+0000 7fafdfc00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fafd81038f0 0x7fafd80707d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:49:19.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.586+0000 7fafdfc00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafd8070e60 con 0x7fafd8102f50 2024-03-04T13:49:19.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.586+0000 7fafdfc00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafd8070fd0 con 0x7fafd81038f0 2024-03-04T13:49:19.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.586+0000 7fafdd200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fafd8102f50 0x7fafd8070290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:49:19.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.586+0000 7fafdd200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fafd8102f50 0x7fafd8070290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36244/0 (socket says 172.21.15.107:36244) 2024-03-04T13:49:19.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.586+0000 7fafdd200700 1 -- 172.21.15.107:0/4140869443 learned_addr learned my addr 172.21.15.107:0/4140869443 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:49:19.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.587+0000 7fafcfe00700 1 --2- 172.21.15.107:0/4140869443 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fafd81038f0 0x7fafd80707d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:49:19.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.587+0000 7fafdd200700 1 -- 172.21.15.107:0/4140869443 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fafd81038f0 msgr2=0x7fafd80707d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:49:19.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.587+0000 7fafdd200700 1 --2- 172.21.15.107:0/4140869443 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fafd81038f0 0x7fafd80707d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:49:19.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.587+0000 7fafdd200700 1 -- 172.21.15.107:0/4140869443 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafc8005040 con 0x7fafd8102f50 2024-03-04T13:49:19.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.587+0000 7fafcfe00700 1 --2- 172.21.15.107:0/4140869443 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fafd81038f0 0x7fafd80707d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T13:49:19.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.587+0000 7fafdd200700 1 --2- 172.21.15.107:0/4140869443 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fafd8102f50 0x7fafd8070290 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fafc8009100 tx=0x7fafc8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:49:19.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.587+0000 7fafd5600700 1 -- 172.21.15.107:0/4140869443 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fafc800e040 con 0x7fafd8102f50 2024-03-04T13:49:19.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.587+0000 7fafd5600700 1 -- 172.21.15.107:0/4140869443 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fafc8004290 con 0x7fafd8102f50 2024-03-04T13:49:19.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.587+0000 7fafdfc00700 1 -- 172.21.15.107:0/4140869443 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafd8119ce0 con 0x7fafd8102f50 2024-03-04T13:49:19.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.587+0000 7fafd5600700 1 -- 172.21.15.107:0/4140869443 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fafc801c630 con 0x7fafd8102f50 2024-03-04T13:49:19.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.588+0000 7fafdfc00700 1 -- 172.21.15.107:0/4140869443 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafd811a1d0 con 0x7fafd8102f50 2024-03-04T13:49:19.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.589+0000 7fafdfc00700 1 -- 172.21.15.107:0/4140869443 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafd80fc6e0 con 0x7fafd8102f50 2024-03-04T13:49:19.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.590+0000 7fafd5600700 1 -- 172.21.15.107:0/4140869443 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fafc800bbf0 con 0x7fafd8102f50 2024-03-04T13:49:19.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.591+0000 7fafd5600700 1 --2- 172.21.15.107:0/4140869443 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fafbc06d7e0 0x7fafbc06fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:49:19.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.591+0000 7fafd5600700 1 -- 172.21.15.107:0/4140869443 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fafc808e1e0 con 0x7fafd8102f50 2024-03-04T13:49:19.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.591+0000 7fafcfe00700 1 --2- 172.21.15.107:0/4140869443 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fafbc06d7e0 0x7fafbc06fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:49:19.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.592+0000 7fafcfe00700 1 --2- 172.21.15.107:0/4140869443 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fafbc06d7e0 0x7fafbc06fc90 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fafc4006fd0 tx=0x7fafc4008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:49:19.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.596+0000 7fafd5600700 1 -- 172.21.15.107:0/4140869443 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fafc8057b00 con 0x7fafd8102f50 2024-03-04T13:49:19.757 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 04 13:49:19 smithi107 podman[123823]: 2024-03-04 13:49:19.508256547 +0000 UTC m=+0.031334678 image pull d71b839abdf892550525dad59fdf20a6b925b73dbcc7526f0d04da8380b9fabf quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377 2024-03-04T13:49:19.757 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 04 13:49:19 smithi107 podman[123823]: 2024-03-04 13:49:19.638754849 +0000 UTC m=+0.161833044 container create b9a84bb7360801575c02ede6c3806723da18b8d8c330a6206426618bb076bd1b (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-0-deactivate, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c) 2024-03-04T13:49:19.758 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:19 smithi107 ceph-mon[112578]: from='client.34214 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:49:19.758 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:19 smithi107 ceph-mon[112578]: pgmap v61: 65 pgs: 9 stale+active+clean, 56 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s 2024-03-04T13:49:19.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.928+0000 7fafdfc00700 1 -- 172.21.15.107:0/4140869443 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fafd810a0c0 con 0x7fafd8102f50 2024-03-04T13:49:19.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.929+0000 7fafd5600700 1 -- 172.21.15.107:0/4140869443 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+735 (secure 0 0 0) 0x7fafc805b1c0 con 0x7fafd8102f50 2024-03-04T13:49:19.931 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-03-04T13:49:19.931 INFO:teuthology.orchestra.run.smithi107.stdout: "mon": { 2024-03-04T13:49:19.931 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-1845-g717ce3cc (717ce3cce8d0166e4e577faa8004e6f9cb4128c0) squid (dev)": 2 2024-03-04T13:49:19.931 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:49:19.931 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr": { 2024-03-04T13:49:19.931 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-1845-g717ce3cc (717ce3cce8d0166e4e577faa8004e6f9cb4128c0) squid (dev)": 2 2024-03-04T13:49:19.931 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:49:19.931 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": { 2024-03-04T13:49:19.932 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-545-g42608bb1 (42608bb191117be10dba720e6bf8a02f3171cd34) quincy (stable)": 5 2024-03-04T13:49:19.932 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:49:19.932 INFO:teuthology.orchestra.run.smithi107.stdout: "mds": { 2024-03-04T13:49:19.932 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-545-g42608bb1 (42608bb191117be10dba720e6bf8a02f3171cd34) quincy (stable)": 4 2024-03-04T13:49:19.932 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:49:19.932 INFO:teuthology.orchestra.run.smithi107.stdout: "overall": { 2024-03-04T13:49:19.932 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-545-g42608bb1 (42608bb191117be10dba720e6bf8a02f3171cd34) quincy (stable)": 9, 2024-03-04T13:49:19.932 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-1845-g717ce3cc (717ce3cce8d0166e4e577faa8004e6f9cb4128c0) squid (dev)": 4 2024-03-04T13:49:19.932 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-04T13:49:19.932 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-03-04T13:49:19.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.932+0000 7fafdfc00700 1 -- 172.21.15.107:0/4140869443 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fafbc06d7e0 msgr2=0x7fafbc06fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:49:19.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.932+0000 7fafdfc00700 1 --2- 172.21.15.107:0/4140869443 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fafbc06d7e0 0x7fafbc06fc90 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fafc4006fd0 tx=0x7fafc4008040 comp rx=0 tx=0).stop 2024-03-04T13:49:19.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.932+0000 7fafdfc00700 1 -- 172.21.15.107:0/4140869443 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fafd8102f50 msgr2=0x7fafd8070290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:49:19.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.932+0000 7fafdfc00700 1 --2- 172.21.15.107:0/4140869443 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fafd8102f50 0x7fafd8070290 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fafc8009100 tx=0x7fafc8009e30 comp rx=0 tx=0).stop 2024-03-04T13:49:19.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.932+0000 7fafdfc00700 1 -- 172.21.15.107:0/4140869443 shutdown_connections 2024-03-04T13:49:19.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.932+0000 7fafdfc00700 1 --2- 172.21.15.107:0/4140869443 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fafbc06d7e0 0x7fafbc06fc90 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:49:19.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.932+0000 7fafdfc00700 1 --2- 172.21.15.107:0/4140869443 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fafd8102f50 0x7fafd8070290 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:49:19.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.932+0000 7fafdfc00700 1 --2- 172.21.15.107:0/4140869443 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fafd81038f0 0x7fafd80707d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:49:19.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.932+0000 7fafdfc00700 1 -- 172.21.15.107:0/4140869443 >> 172.21.15.107:0/4140869443 conn(0x7fafd80f8ae0 msgr2=0x7fafd80fa460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:49:19.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.932+0000 7fafdfc00700 1 -- 172.21.15.107:0/4140869443 shutdown_connections 2024-03-04T13:49:19.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:19.932+0000 7fafdfc00700 1 -- 172.21.15.107:0/4140869443 wait complete. 2024-03-04T13:49:20.044 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 04 13:49:19 smithi107 podman[123823]: 2024-03-04 13:49:19.880494589 +0000 UTC m=+0.403572763 container init b9a84bb7360801575c02ede6c3806723da18b8d8c330a6206426618bb076bd1b (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-0-deactivate, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, io.buildah.version=1.33.5, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-03-04T13:49:20.044 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 04 13:49:19 smithi107 podman[123823]: 2024-03-04 13:49:19.89012915 +0000 UTC m=+0.413207334 container start b9a84bb7360801575c02ede6c3806723da18b8d8c330a6206426618bb076bd1b (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-0-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240222, io.buildah.version=1.33.5, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, GIT_CLEAN=True) 2024-03-04T13:49:20.044 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 04 13:49:20 smithi107 podman[123823]: 2024-03-04 13:49:20.043853361 +0000 UTC m=+0.566931568 container attach b9a84bb7360801575c02ede6c3806723da18b8d8c330a6206426618bb076bd1b (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-0-deactivate, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-03-04T13:49:20.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.096+0000 7fed20a00700 1 -- 172.21.15.107:0/1875465057 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed1c102f50 msgr2=0x7fed1c103320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:49:20.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.096+0000 7fed20a00700 1 --2- 172.21.15.107:0/1875465057 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed1c102f50 0x7fed1c103320 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fed0c004940 tx=0x7fed0c009e30 comp rx=0 tx=0).stop 2024-03-04T13:49:20.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.096+0000 7fed20a00700 1 -- 172.21.15.107:0/1875465057 shutdown_connections 2024-03-04T13:49:20.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.096+0000 7fed20a00700 1 --2- 172.21.15.107:0/1875465057 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed1c1038f0 0x7fed1c0fcf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:49:20.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.096+0000 7fed20a00700 1 --2- 172.21.15.107:0/1875465057 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed1c102f50 0x7fed1c103320 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:49:20.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.096+0000 7fed20a00700 1 -- 172.21.15.107:0/1875465057 >> 172.21.15.107:0/1875465057 conn(0x7fed1c0f8ae0 msgr2=0x7fed1c0faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:49:20.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.096+0000 7fed20a00700 1 -- 172.21.15.107:0/1875465057 shutdown_connections 2024-03-04T13:49:20.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.096+0000 7fed20a00700 1 -- 172.21.15.107:0/1875465057 wait complete. 2024-03-04T13:49:20.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.097+0000 7fed20a00700 1 Processor -- start 2024-03-04T13:49:20.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.097+0000 7fed20a00700 1 -- start start 2024-03-04T13:49:20.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.098+0000 7fed20a00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed1c102f50 0x7fed1c115c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:49:20.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.098+0000 7fed20a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed1c1038f0 0x7fed1c116150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:49:20.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.098+0000 7fed20a00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed1c1167a0 con 0x7fed1c1038f0 2024-03-04T13:49:20.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.098+0000 7fed20a00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed1c10faf0 con 0x7fed1c102f50 2024-03-04T13:49:20.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.098+0000 7fed19400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed1c1038f0 0x7fed1c116150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:49:20.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.098+0000 7fed19400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed1c1038f0 0x7fed1c116150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36252/0 (socket says 172.21.15.107:36252) 2024-03-04T13:49:20.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.098+0000 7fed19400700 1 -- 172.21.15.107:0/2847416267 learned_addr learned my addr 172.21.15.107:0/2847416267 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:49:20.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.098+0000 7fed19e00700 1 --2- 172.21.15.107:0/2847416267 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed1c102f50 0x7fed1c115c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:49:20.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.098+0000 7fed19400700 1 -- 172.21.15.107:0/2847416267 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed1c102f50 msgr2=0x7fed1c115c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:49:20.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.098+0000 7fed19400700 1 --2- 172.21.15.107:0/2847416267 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed1c102f50 0x7fed1c115c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:49:20.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.098+0000 7fed19400700 1 -- 172.21.15.107:0/2847416267 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed0c005040 con 0x7fed1c1038f0 2024-03-04T13:49:20.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.099+0000 7fed19400700 1 --2- 172.21.15.107:0/2847416267 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed1c1038f0 0x7fed1c116150 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fed0800c890 tx=0x7fed080044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:49:20.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.099+0000 7fed12000700 1 -- 172.21.15.107:0/2847416267 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fed0800a7d0 con 0x7fed1c1038f0 2024-03-04T13:49:20.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.099+0000 7fed20a00700 1 -- 172.21.15.107:0/2847416267 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed1c10fdd0 con 0x7fed1c1038f0 2024-03-04T13:49:20.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.099+0000 7fed12000700 1 -- 172.21.15.107:0/2847416267 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fed0800ae00 con 0x7fed1c1038f0 2024-03-04T13:49:20.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.099+0000 7fed20a00700 1 -- 172.21.15.107:0/2847416267 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed1c110310 con 0x7fed1c1038f0 2024-03-04T13:49:20.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.099+0000 7fed12000700 1 -- 172.21.15.107:0/2847416267 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fed0801b400 con 0x7fed1c1038f0 2024-03-04T13:49:20.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.100+0000 7fed20a00700 1 -- 172.21.15.107:0/2847416267 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecfc0052e0 con 0x7fed1c1038f0 2024-03-04T13:49:20.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.101+0000 7fed12000700 1 -- 172.21.15.107:0/2847416267 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fed0801b560 con 0x7fed1c1038f0 2024-03-04T13:49:20.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.102+0000 7fed12000700 1 --2- 172.21.15.107:0/2847416267 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fed0006d7e0 0x7fed0006fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:49:20.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.102+0000 7fed12000700 1 -- 172.21.15.107:0/2847416267 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fed0808cf20 con 0x7fed1c1038f0 2024-03-04T13:49:20.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.103+0000 7fed19e00700 1 --2- 172.21.15.107:0/2847416267 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fed0006d7e0 0x7fed0006fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:49:20.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.103+0000 7fed19e00700 1 --2- 172.21.15.107:0/2847416267 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fed0006d7e0 0x7fed0006fc90 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fed0c004560 tx=0x7fed0c009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:49:20.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:19 smithi169 ceph-mon[86183]: from='client.34214 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:49:20.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:19 smithi169 ceph-mon[86183]: pgmap v61: 65 pgs: 9 stale+active+clean, 56 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s 2024-03-04T13:49:20.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.111+0000 7fed12000700 1 -- 172.21.15.107:0/2847416267 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fed08056960 con 0x7fed1c1038f0 2024-03-04T13:49:20.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.379+0000 7fed20a00700 1 -- 172.21.15.107:0/2847416267 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fecfc005da0 con 0x7fed1c1038f0 2024-03-04T13:49:20.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.380+0000 7fed12000700 1 -- 172.21.15.107:0/2847416267 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 16 v16) v1 ==== 76+0+1880 (secure 0 0 0) 0x7fed0805a020 con 0x7fed1c1038f0 2024-03-04T13:49:20.381 INFO:teuthology.orchestra.run.smithi107.stdout:e16 2024-03-04T13:49:20.381 INFO:teuthology.orchestra.run.smithi107.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-04T13:49:20.381 INFO:teuthology.orchestra.run.smithi107.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-04T13:49:20.381 INFO:teuthology.orchestra.run.smithi107.stdout:legacy client fscid: 1 2024-03-04T13:49:20.382 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:49:20.382 INFO:teuthology.orchestra.run.smithi107.stdout:Filesystem 'cephfs' (1) 2024-03-04T13:49:20.382 INFO:teuthology.orchestra.run.smithi107.stdout:fs_name cephfs 2024-03-04T13:49:20.382 INFO:teuthology.orchestra.run.smithi107.stdout:epoch 15 2024-03-04T13:49:20.382 INFO:teuthology.orchestra.run.smithi107.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-04T13:49:20.382 INFO:teuthology.orchestra.run.smithi107.stdout:created 2024-03-04T13:38:30.854004+0000 2024-03-04T13:49:20.382 INFO:teuthology.orchestra.run.smithi107.stdout:modified 2024-03-04T13:38:50.015989+0000 2024-03-04T13:49:20.382 INFO:teuthology.orchestra.run.smithi107.stdout:tableserver 0 2024-03-04T13:49:20.382 INFO:teuthology.orchestra.run.smithi107.stdout:root 0 2024-03-04T13:49:20.382 INFO:teuthology.orchestra.run.smithi107.stdout:session_timeout 60 2024-03-04T13:49:20.382 INFO:teuthology.orchestra.run.smithi107.stdout:session_autoclose 300 2024-03-04T13:49:20.382 INFO:teuthology.orchestra.run.smithi107.stdout:max_file_size 1099511627776 2024-03-04T13:49:20.382 INFO:teuthology.orchestra.run.smithi107.stdout:max_xattr_size 65536 2024-03-04T13:49:20.382 INFO:teuthology.orchestra.run.smithi107.stdout:required_client_features {} 2024-03-04T13:49:20.383 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure 0 2024-03-04T13:49:20.383 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure_osd_epoch 47 2024-03-04T13:49:20.383 INFO:teuthology.orchestra.run.smithi107.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-04T13:49:20.383 INFO:teuthology.orchestra.run.smithi107.stdout:max_mds 1 2024-03-04T13:49:20.383 INFO:teuthology.orchestra.run.smithi107.stdout:in 0 2024-03-04T13:49:20.383 INFO:teuthology.orchestra.run.smithi107.stdout:up {0=24247} 2024-03-04T13:49:20.383 INFO:teuthology.orchestra.run.smithi107.stdout:failed 2024-03-04T13:49:20.383 INFO:teuthology.orchestra.run.smithi107.stdout:damaged 2024-03-04T13:49:20.383 INFO:teuthology.orchestra.run.smithi107.stdout:stopped 2024-03-04T13:49:20.383 INFO:teuthology.orchestra.run.smithi107.stdout:data_pools [3] 2024-03-04T13:49:20.383 INFO:teuthology.orchestra.run.smithi107.stdout:metadata_pool 2 2024-03-04T13:49:20.383 INFO:teuthology.orchestra.run.smithi107.stdout:inline_data enabled 2024-03-04T13:49:20.383 INFO:teuthology.orchestra.run.smithi107.stdout:balancer 2024-03-04T13:49:20.383 INFO:teuthology.orchestra.run.smithi107.stdout:bal_rank_mask -1 2024-03-04T13:49:20.384 INFO:teuthology.orchestra.run.smithi107.stdout:standby_count_wanted 1 2024-03-04T13:49:20.384 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi169.jusudl{0:24247} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.169:6824/2994866076,v1:172.21.15.169:6825/2994866076] compat {c=[1],r=[1],i=[7ff]}] 2024-03-04T13:49:20.384 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.thycho{0:14470} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.107:6828/1985577060,v1:172.21.15.107:6829/1985577060] compat {c=[1],r=[1],i=[7ff]}] 2024-03-04T13:49:20.384 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:49:20.384 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:49:20.384 INFO:teuthology.orchestra.run.smithi107.stdout:Standby daemons: 2024-03-04T13:49:20.384 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:49:20.384 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.riexmm{-1:14480} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.107:6826/3261622517,v1:172.21.15.107:6827/3261622517] compat {c=[1],r=[1],i=[7ff]}] 2024-03-04T13:49:20.384 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi169.bxitzk{-1:24259} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.169:6826/3696090493,v1:172.21.15.169:6827/3696090493] compat {c=[1],r=[1],i=[7ff]}] 2024-03-04T13:49:20.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.383+0000 7fed20a00700 1 -- 172.21.15.107:0/2847416267 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fed0006d7e0 msgr2=0x7fed0006fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:49:20.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.383+0000 7fed20a00700 1 --2- 172.21.15.107:0/2847416267 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fed0006d7e0 0x7fed0006fc90 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fed0c004560 tx=0x7fed0c009e30 comp rx=0 tx=0).stop 2024-03-04T13:49:20.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.383+0000 7fed20a00700 1 -- 172.21.15.107:0/2847416267 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed1c1038f0 msgr2=0x7fed1c116150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:49:20.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.383+0000 7fed20a00700 1 --2- 172.21.15.107:0/2847416267 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed1c1038f0 0x7fed1c116150 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fed0800c890 tx=0x7fed080044c0 comp rx=0 tx=0).stop 2024-03-04T13:49:20.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.383+0000 7fed20a00700 1 -- 172.21.15.107:0/2847416267 shutdown_connections 2024-03-04T13:49:20.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.383+0000 7fed20a00700 1 --2- 172.21.15.107:0/2847416267 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fed0006d7e0 0x7fed0006fc90 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:49:20.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.383+0000 7fed20a00700 1 --2- 172.21.15.107:0/2847416267 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed1c102f50 0x7fed1c115c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:49:20.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.383+0000 7fed20a00700 1 --2- 172.21.15.107:0/2847416267 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed1c1038f0 0x7fed1c116150 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:49:20.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.383+0000 7fed20a00700 1 -- 172.21.15.107:0/2847416267 >> 172.21.15.107:0/2847416267 conn(0x7fed1c0f8ae0 msgr2=0x7fed1c0fa3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:49:20.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.383+0000 7fed20a00700 1 -- 172.21.15.107:0/2847416267 shutdown_connections 2024-03-04T13:49:20.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.383+0000 7fed20a00700 1 -- 172.21.15.107:0/2847416267 wait complete. 2024-03-04T13:49:20.386 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 16 2024-03-04T13:49:20.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.522+0000 7fe5c8a00700 1 -- 172.21.15.107:0/3949409079 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe5c4102f50 msgr2=0x7fe5c4103320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:49:20.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.522+0000 7fe5c8a00700 1 --2- 172.21.15.107:0/3949409079 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe5c4102f50 0x7fe5c4103320 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe5b4004940 tx=0x7fe5b4009e30 comp rx=0 tx=0).stop 2024-03-04T13:49:20.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.522+0000 7fe5c8a00700 1 -- 172.21.15.107:0/3949409079 shutdown_connections 2024-03-04T13:49:20.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.522+0000 7fe5c8a00700 1 --2- 172.21.15.107:0/3949409079 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5c41038f0 0x7fe5c40fcf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:49:20.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.522+0000 7fe5c8a00700 1 --2- 172.21.15.107:0/3949409079 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe5c4102f50 0x7fe5c4103320 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:49:20.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.522+0000 7fe5c8a00700 1 -- 172.21.15.107:0/3949409079 >> 172.21.15.107:0/3949409079 conn(0x7fe5c40f8ae0 msgr2=0x7fe5c40faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:49:20.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.522+0000 7fe5c8a00700 1 -- 172.21.15.107:0/3949409079 shutdown_connections 2024-03-04T13:49:20.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.522+0000 7fe5c8a00700 1 -- 172.21.15.107:0/3949409079 wait complete. 2024-03-04T13:49:20.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.523+0000 7fe5c8a00700 1 Processor -- start 2024-03-04T13:49:20.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.523+0000 7fe5c8a00700 1 -- start start 2024-03-04T13:49:20.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.524+0000 7fe5c8a00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5c4102f50 0x7fe5c4113a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:49:20.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.524+0000 7fe5c8a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe5c41038f0 0x7fe5c4113f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:49:20.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.524+0000 7fe5c8a00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c41145e0 con 0x7fe5c41038f0 2024-03-04T13:49:20.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.524+0000 7fe5c8a00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c410d910 con 0x7fe5c4102f50 2024-03-04T13:49:20.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.524+0000 7fe5bbe00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe5c41038f0 0x7fe5c4113f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:49:20.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.524+0000 7fe5bbe00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe5c41038f0 0x7fe5c4113f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36270/0 (socket says 172.21.15.107:36270) 2024-03-04T13:49:20.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.524+0000 7fe5bbe00700 1 -- 172.21.15.107:0/84417428 learned_addr learned my addr 172.21.15.107:0/84417428 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:49:20.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.524+0000 7fe5c1e00700 1 --2- 172.21.15.107:0/84417428 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5c4102f50 0x7fe5c4113a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:49:20.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.524+0000 7fe5bbe00700 1 -- 172.21.15.107:0/84417428 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5c4102f50 msgr2=0x7fe5c4113a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:49:20.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.524+0000 7fe5bbe00700 1 --2- 172.21.15.107:0/84417428 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5c4102f50 0x7fe5c4113a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:49:20.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.524+0000 7fe5bbe00700 1 -- 172.21.15.107:0/84417428 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5b4005040 con 0x7fe5c41038f0 2024-03-04T13:49:20.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.525+0000 7fe5bbe00700 1 --2- 172.21.15.107:0/84417428 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe5c41038f0 0x7fe5c4113f90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac002a50 tx=0x7fe5ac004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:49:20.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.525+0000 7fe5ba000700 1 -- 172.21.15.107:0/84417428 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe5ac007cc0 con 0x7fe5c41038f0 2024-03-04T13:49:20.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.525+0000 7fe5ba000700 1 -- 172.21.15.107:0/84417428 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe5ac00f040 con 0x7fe5c41038f0 2024-03-04T13:49:20.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.525+0000 7fe5c8a00700 1 -- 172.21.15.107:0/84417428 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5c410dbf0 con 0x7fe5c41038f0 2024-03-04T13:49:20.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.525+0000 7fe5ba000700 1 -- 172.21.15.107:0/84417428 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe5ac00daf0 con 0x7fe5c41038f0 2024-03-04T13:49:20.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.525+0000 7fe5c8a00700 1 -- 172.21.15.107:0/84417428 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5c410e130 con 0x7fe5c41038f0 2024-03-04T13:49:20.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.526+0000 7fe5c8a00700 1 -- 172.21.15.107:0/84417428 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5c40fc6e0 con 0x7fe5c41038f0 2024-03-04T13:49:20.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.528+0000 7fe5ba000700 1 -- 172.21.15.107:0/84417428 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fe5ac009810 con 0x7fe5c41038f0 2024-03-04T13:49:20.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.529+0000 7fe5ba000700 1 --2- 172.21.15.107:0/84417428 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fe5a806d7f0 0x7fe5a806fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:49:20.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.529+0000 7fe5ba000700 1 -- 172.21.15.107:0/84417428 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fe5ac08cca0 con 0x7fe5c41038f0 2024-03-04T13:49:20.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.529+0000 7fe5c1e00700 1 --2- 172.21.15.107:0/84417428 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fe5a806d7f0 0x7fe5a806fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:49:20.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.529+0000 7fe5c1e00700 1 --2- 172.21.15.107:0/84417428 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fe5a806d7f0 0x7fe5a806fca0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe5b400b020 tx=0x7fe5b4009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:49:20.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.533+0000 7fe5ba000700 1 -- 172.21.15.107:0/84417428 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe5ac059fd0 con 0x7fe5c41038f0 2024-03-04T13:49:20.593 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 04 13:49:20 smithi107 podman[123892]: 2024-03-04 13:49:20.383732064 +0000 UTC m=+0.031526720 container died b9a84bb7360801575c02ede6c3806723da18b8d8c330a6206426618bb076bd1b (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-0-deactivate, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, org.label-schema.build-date=20240222, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-04T13:49:20.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.788+0000 7fe5c8a00700 1 -- 172.21.15.107:0/84417428 --> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe5c410e350 con 0x7fe5a806d7f0 2024-03-04T13:49:20.791 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.790+0000 7fe5ba000700 1 -- 172.21.15.107:0/84417428 <== mgr.34102 v2:172.21.15.107:6800/1223039812 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fe5c410e350 con 0x7fe5a806d7f0 2024-03-04T13:49:20.791 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-03-04T13:49:20.791 INFO:teuthology.orchestra.run.smithi107.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377", 2024-03-04T13:49:20.791 INFO:teuthology.orchestra.run.smithi107.stdout: "in_progress": true, 2024-03-04T13:49:20.791 INFO:teuthology.orchestra.run.smithi107.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-04T13:49:20.791 INFO:teuthology.orchestra.run.smithi107.stdout: "services_complete": [ 2024-03-04T13:49:20.792 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr", 2024-03-04T13:49:20.792 INFO:teuthology.orchestra.run.smithi107.stdout: "mon", 2024-03-04T13:49:20.792 INFO:teuthology.orchestra.run.smithi107.stdout: "crash" 2024-03-04T13:49:20.792 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-04T13:49:20.792 INFO:teuthology.orchestra.run.smithi107.stdout: "progress": "6/21 daemons upgraded", 2024-03-04T13:49:20.792 INFO:teuthology.orchestra.run.smithi107.stdout: "message": "Currently upgrading osd daemons", 2024-03-04T13:49:20.792 INFO:teuthology.orchestra.run.smithi107.stdout: "is_paused": false 2024-03-04T13:49:20.792 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-03-04T13:49:20.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.793+0000 7fe5c8a00700 1 -- 172.21.15.107:0/84417428 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fe5a806d7f0 msgr2=0x7fe5a806fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:49:20.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.793+0000 7fe5c8a00700 1 --2- 172.21.15.107:0/84417428 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fe5a806d7f0 0x7fe5a806fca0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe5b400b020 tx=0x7fe5b4009e30 comp rx=0 tx=0).stop 2024-03-04T13:49:20.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.793+0000 7fe5c8a00700 1 -- 172.21.15.107:0/84417428 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe5c41038f0 msgr2=0x7fe5c4113f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:49:20.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.793+0000 7fe5c8a00700 1 --2- 172.21.15.107:0/84417428 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe5c41038f0 0x7fe5c4113f90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac002a50 tx=0x7fe5ac004600 comp rx=0 tx=0).stop 2024-03-04T13:49:20.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.793+0000 7fe5c8a00700 1 -- 172.21.15.107:0/84417428 shutdown_connections 2024-03-04T13:49:20.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.793+0000 7fe5c8a00700 1 --2- 172.21.15.107:0/84417428 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fe5a806d7f0 0x7fe5a806fca0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:49:20.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.793+0000 7fe5c8a00700 1 --2- 172.21.15.107:0/84417428 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5c4102f50 0x7fe5c4113a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:49:20.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.793+0000 7fe5c8a00700 1 --2- 172.21.15.107:0/84417428 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe5c41038f0 0x7fe5c4113f90 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:49:20.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.793+0000 7fe5c8a00700 1 -- 172.21.15.107:0/84417428 >> 172.21.15.107:0/84417428 conn(0x7fe5c40f8ae0 msgr2=0x7fe5c40fa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:49:20.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.793+0000 7fe5c8a00700 1 -- 172.21.15.107:0/84417428 shutdown_connections 2024-03-04T13:49:20.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.793+0000 7fe5c8a00700 1 -- 172.21.15.107:0/84417428 wait complete. 2024-03-04T13:49:20.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.939+0000 7f7fed400700 1 -- 172.21.15.107:0/1627316479 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7fe80fe8f0 msgr2=0x7f7fe8107050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:49:20.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.939+0000 7f7fed400700 1 --2- 172.21.15.107:0/1627316479 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7fe80fe8f0 0x7f7fe8107050 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7fd4004970 tx=0x7f7fd4009e30 comp rx=0 tx=0).stop 2024-03-04T13:49:20.940 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:20 smithi107 ceph-mon[112578]: from='client.34218 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:49:20.941 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:20 smithi107 ceph-mon[112578]: from='client.34222 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:49:20.941 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:20 smithi107 ceph-mon[112578]: from='client.? 172.21.15.107:0/4140869443' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:49:20.941 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:20 smithi107 ceph-mon[112578]: from='client.? 172.21.15.107:0/2847416267' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-04T13:49:20.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.940+0000 7f7fed400700 1 -- 172.21.15.107:0/1627316479 shutdown_connections 2024-03-04T13:49:20.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.940+0000 7f7fed400700 1 --2- 172.21.15.107:0/1627316479 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7fe80fe8f0 0x7f7fe8107050 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:49:20.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.940+0000 7f7fed400700 1 --2- 172.21.15.107:0/1627316479 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7fe80fdfe0 0x7f7fe80fe3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:49:20.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.940+0000 7f7fed400700 1 -- 172.21.15.107:0/1627316479 >> 172.21.15.107:0/1627316479 conn(0x7f7fe8071d90 msgr2=0x7f7fe8072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:49:20.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.940+0000 7f7fed400700 1 -- 172.21.15.107:0/1627316479 shutdown_connections 2024-03-04T13:49:20.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.940+0000 7f7fed400700 1 -- 172.21.15.107:0/1627316479 wait complete. 2024-03-04T13:49:20.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.941+0000 7f7fed400700 1 Processor -- start 2024-03-04T13:49:20.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.941+0000 7f7fed400700 1 -- start start 2024-03-04T13:49:20.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.941+0000 7f7fed400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7fe80fdfe0 0x7f7fe81139f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:49:20.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.941+0000 7f7fed400700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7fe80fe8f0 0x7f7fe8113f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:49:20.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.941+0000 7f7fed400700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe8114580 con 0x7f7fe80fdfe0 2024-03-04T13:49:20.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.941+0000 7f7fed400700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe810d8b0 con 0x7f7fe80fe8f0 2024-03-04T13:49:20.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.942+0000 7f7fe6800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7fe80fdfe0 0x7f7fe81139f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:49:20.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.942+0000 7f7fe6800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7fe80fdfe0 0x7f7fe81139f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36282/0 (socket says 172.21.15.107:36282) 2024-03-04T13:49:20.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.942+0000 7f7fe6800700 1 -- 172.21.15.107:0/346316224 learned_addr learned my addr 172.21.15.107:0/346316224 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:49:20.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.942+0000 7f7fe6800700 1 -- 172.21.15.107:0/346316224 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7fe80fe8f0 msgr2=0x7f7fe8113f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:49:20.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.942+0000 7f7fe5e00700 1 --2- 172.21.15.107:0/346316224 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7fe80fe8f0 0x7f7fe8113f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:49:20.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.942+0000 7f7fe6800700 1 --2- 172.21.15.107:0/346316224 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7fe80fe8f0 0x7f7fe8113f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:49:20.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.942+0000 7f7fe6800700 1 -- 172.21.15.107:0/346316224 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fd4005040 con 0x7f7fe80fdfe0 2024-03-04T13:49:20.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.942+0000 7f7fe5e00700 1 --2- 172.21.15.107:0/346316224 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7fe80fe8f0 0x7f7fe8113f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T13:49:20.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.942+0000 7f7fe6800700 1 --2- 172.21.15.107:0/346316224 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7fe80fdfe0 0x7f7fe81139f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7fd800c8c0 tx=0x7f7fd80044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:49:20.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.943+0000 7f7fdea00700 1 -- 172.21.15.107:0/346316224 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7fd800a980 con 0x7f7fe80fdfe0 2024-03-04T13:49:20.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.943+0000 7f7fed400700 1 -- 172.21.15.107:0/346316224 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fe810db90 con 0x7f7fe80fdfe0 2024-03-04T13:49:20.945 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.943+0000 7f7fdea00700 1 -- 172.21.15.107:0/346316224 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7fd800ce60 con 0x7f7fe80fdfe0 2024-03-04T13:49:20.945 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.943+0000 7f7fdea00700 1 -- 172.21.15.107:0/346316224 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7fd80115e0 con 0x7f7fe80fdfe0 2024-03-04T13:49:20.945 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.943+0000 7f7fed400700 1 -- 172.21.15.107:0/346316224 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fe810e0d0 con 0x7f7fe80fdfe0 2024-03-04T13:49:20.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.945+0000 7f7fed400700 1 -- 172.21.15.107:0/346316224 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fe81046c0 con 0x7f7fe80fdfe0 2024-03-04T13:49:20.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.945+0000 7f7fdea00700 1 -- 172.21.15.107:0/346316224 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f7fd8012400 con 0x7f7fe80fdfe0 2024-03-04T13:49:20.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.946+0000 7f7fdea00700 1 --2- 172.21.15.107:0/346316224 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f7fcc06d7f0 0x7f7fcc06fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:49:20.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.946+0000 7f7fdea00700 1 -- 172.21.15.107:0/346316224 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f7fd808ced0 con 0x7f7fe80fdfe0 2024-03-04T13:49:20.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.946+0000 7f7fe5e00700 1 --2- 172.21.15.107:0/346316224 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f7fcc06d7f0 0x7f7fcc06fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:49:20.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.947+0000 7f7fe5e00700 1 --2- 172.21.15.107:0/346316224 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f7fcc06d7f0 0x7f7fcc06fca0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7fd400b370 tx=0x7f7fd4009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:49:20.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:20.951+0000 7f7fdea00700 1 -- 172.21.15.107:0/346316224 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7fd805a340 con 0x7f7fe80fdfe0 2024-03-04T13:49:21.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:20 smithi169 ceph-mon[86183]: from='client.34218 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:49:21.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:20 smithi169 ceph-mon[86183]: from='client.34222 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:49:21.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:20 smithi169 ceph-mon[86183]: from='client.? 172.21.15.107:0/4140869443' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:49:21.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:20 smithi169 ceph-mon[86183]: from='client.? 172.21.15.107:0/2847416267' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-04T13:49:21.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:21.289+0000 7f7fed400700 1 -- 172.21.15.107:0/346316224 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f7fe804bed0 con 0x7f7fe80fdfe0 2024-03-04T13:49:21.291 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 04 13:49:21 smithi107 podman[123892]: 2024-03-04 13:49:21.073866477 +0000 UTC m=+0.721661176 container remove b9a84bb7360801575c02ede6c3806723da18b8d8c330a6206426618bb076bd1b (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-0-deactivate, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240222, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-03-04T13:49:21.292 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 04 13:49:21 smithi107 systemd[1]: ceph-1fc09a58-da2b-11ee-95c6-87774f69a715@osd.0.service: Deactivated successfully. 2024-03-04T13:49:21.292 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 04 13:49:21 smithi107 systemd[1]: Stopped Ceph osd.0 for 1fc09a58-da2b-11ee-95c6-87774f69a715. 2024-03-04T13:49:21.292 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 04 13:49:21 smithi107 systemd[1]: ceph-1fc09a58-da2b-11ee-95c6-87774f69a715@osd.0.service: Consumed 1min 40.868s CPU time. 2024-03-04T13:49:21.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:21.291+0000 7f7fdea00700 1 -- 172.21.15.107:0/346316224 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+1517 (secure 0 0 0) 0x7f7fd80126b0 con 0x7f7fe80fdfe0 2024-03-04T13:49:21.292 INFO:teuthology.orchestra.run.smithi107.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data; 1 osds down; Degraded data redundancy: 50/264 objects degraded (18.939%), 19 pgs degraded 2024-03-04T13:49:21.292 INFO:teuthology.orchestra.run.smithi107.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-04T13:49:21.292 INFO:teuthology.orchestra.run.smithi107.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-04T13:49:21.293 INFO:teuthology.orchestra.run.smithi107.stdout:[WRN] OSD_DOWN: 1 osds down 2024-03-04T13:49:21.293 INFO:teuthology.orchestra.run.smithi107.stdout: osd.0 (root=default,host=smithi107) is down 2024-03-04T13:49:21.293 INFO:teuthology.orchestra.run.smithi107.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 50/264 objects degraded (18.939%), 19 pgs degraded 2024-03-04T13:49:21.293 INFO:teuthology.orchestra.run.smithi107.stdout: pg 1.0 is active+undersized+degraded, acting [3,1] 2024-03-04T13:49:21.293 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.0 is active+undersized+degraded, acting [3,1] 2024-03-04T13:49:21.293 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.1 is active+undersized+degraded, acting [2,1] 2024-03-04T13:49:21.293 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.2 is active+undersized+degraded, acting [5,1] 2024-03-04T13:49:21.293 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.4 is active+undersized+degraded, acting [1,4] 2024-03-04T13:49:21.293 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.5 is active+undersized+degraded, acting [3,4] 2024-03-04T13:49:21.293 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.8 is active+undersized+degraded, acting [3,5] 2024-03-04T13:49:21.293 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.9 is active+undersized+degraded, acting [1,4] 2024-03-04T13:49:21.293 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.c is active+undersized+degraded, acting [2,3] 2024-03-04T13:49:21.293 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.e is active+undersized+degraded, acting [2,3] 2024-03-04T13:49:21.293 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.f is active+undersized+degraded, acting [4,5] 2024-03-04T13:49:21.294 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.10 is active+undersized+degraded, acting [2,1] 2024-03-04T13:49:21.294 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.13 is active+undersized+degraded, acting [4,2] 2024-03-04T13:49:21.294 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.15 is active+undersized+degraded, acting [1,3] 2024-03-04T13:49:21.294 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.19 is active+undersized+degraded, acting [4,2] 2024-03-04T13:49:21.294 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.1b is active+undersized+degraded, acting [1,5] 2024-03-04T13:49:21.294 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.1d is active+undersized+degraded, acting [3,5] 2024-03-04T13:49:21.294 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.1e is active+undersized+degraded, acting [2,5] 2024-03-04T13:49:21.294 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.1f is active+undersized+degraded, acting [3,4] 2024-03-04T13:49:21.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:21.294+0000 7f7fed400700 1 -- 172.21.15.107:0/346316224 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f7fcc06d7f0 msgr2=0x7f7fcc06fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:49:21.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:21.294+0000 7f7fed400700 1 --2- 172.21.15.107:0/346316224 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f7fcc06d7f0 0x7f7fcc06fca0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7fd400b370 tx=0x7f7fd4009e30 comp rx=0 tx=0).stop 2024-03-04T13:49:21.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:21.294+0000 7f7fed400700 1 -- 172.21.15.107:0/346316224 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7fe80fdfe0 msgr2=0x7f7fe81139f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:49:21.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:21.294+0000 7f7fed400700 1 --2- 172.21.15.107:0/346316224 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7fe80fdfe0 0x7f7fe81139f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7fd800c8c0 tx=0x7f7fd80044c0 comp rx=0 tx=0).stop 2024-03-04T13:49:21.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:21.294+0000 7f7fed400700 1 -- 172.21.15.107:0/346316224 shutdown_connections 2024-03-04T13:49:21.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:21.294+0000 7f7fed400700 1 --2- 172.21.15.107:0/346316224 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f7fcc06d7f0 0x7f7fcc06fca0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:49:21.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:21.294+0000 7f7fed400700 1 --2- 172.21.15.107:0/346316224 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7fe80fdfe0 0x7f7fe81139f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:49:21.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:21.294+0000 7f7fed400700 1 --2- 172.21.15.107:0/346316224 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7fe80fe8f0 0x7f7fe8113f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:49:21.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:21.294+0000 7f7fed400700 1 -- 172.21.15.107:0/346316224 >> 172.21.15.107:0/346316224 conn(0x7f7fe8071d90 msgr2=0x7f7fe8101840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:49:21.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:21.295+0000 7f7fed400700 1 -- 172.21.15.107:0/346316224 shutdown_connections 2024-03-04T13:49:21.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:21.295+0000 7f7fed400700 1 -- 172.21.15.107:0/346316224 wait complete. 2024-03-04T13:49:21.593 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 04 13:49:21 smithi107 systemd[1]: Starting Ceph osd.0 for 1fc09a58-da2b-11ee-95c6-87774f69a715... 2024-03-04T13:49:22.054 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:21 smithi107 ceph-mon[112578]: pgmap v63: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s; 50/264 objects degraded (18.939%) 2024-03-04T13:49:22.054 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:21 smithi107 ceph-mon[112578]: Health check failed: Degraded data redundancy: 50/264 objects degraded (18.939%), 19 pgs degraded (PG_DEGRADED) 2024-03-04T13:49:22.054 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:21 smithi107 ceph-mon[112578]: from='client.34234 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:49:22.054 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:21 smithi107 ceph-mon[112578]: from='client.? 172.21.15.107:0/346316224' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-04T13:49:22.054 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 04 13:49:21 smithi107 podman[124041]: 2024-03-04 13:49:21.716820822 +0000 UTC m=+0.031687809 image pull d71b839abdf892550525dad59fdf20a6b925b73dbcc7526f0d04da8380b9fabf quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377 2024-03-04T13:49:22.054 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 04 13:49:22 smithi107 podman[124041]: 2024-03-04 13:49:22.053771807 +0000 UTC m=+0.368638830 container create 072bf8f7413bc5184e44553314757fad50ef451d28e49b75433e7e86bc52e850 (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-0-activate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240222, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-03-04T13:49:22.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:21 smithi169 ceph-mon[86183]: pgmap v63: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s; 50/264 objects degraded (18.939%) 2024-03-04T13:49:22.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:21 smithi169 ceph-mon[86183]: Health check failed: Degraded data redundancy: 50/264 objects degraded (18.939%), 19 pgs degraded (PG_DEGRADED) 2024-03-04T13:49:22.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:21 smithi169 ceph-mon[86183]: from='client.34234 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:49:22.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:21 smithi169 ceph-mon[86183]: from='client.? 172.21.15.107:0/346316224' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-04T13:49:22.596 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 04 13:49:22 smithi107 podman[124041]: 2024-03-04 13:49:22.338149106 +0000 UTC m=+0.653016152 container init 072bf8f7413bc5184e44553314757fad50ef451d28e49b75433e7e86bc52e850 (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-0-activate, GIT_CLEAN=True, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-03-04T13:49:22.596 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 04 13:49:22 smithi107 podman[124041]: 2024-03-04 13:49:22.351137739 +0000 UTC m=+0.666004779 container start 072bf8f7413bc5184e44553314757fad50ef451d28e49b75433e7e86bc52e850 (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-0-activate, org.label-schema.build-date=20240222, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-03-04T13:49:22.596 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 04 13:49:22 smithi107 podman[124041]: 2024-03-04 13:49:22.428262976 +0000 UTC m=+0.743130045 container attach 072bf8f7413bc5184e44553314757fad50ef451d28e49b75433e7e86bc52e850 (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-0-activate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.license=GPLv2, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-03-04T13:49:23.497 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 04 13:49:23 smithi107 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-0-activate[124055]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-04T13:49:23.497 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 04 13:49:23 smithi107 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-0-activate[124055]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-03-04T13:49:23.497 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 04 13:49:23 smithi107 bash[124041]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-04T13:49:23.497 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 04 13:49:23 smithi107 bash[124041]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-03-04T13:49:23.497 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 04 13:49:23 smithi107 bash[124041]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-03-04T13:49:23.497 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 04 13:49:23 smithi107 bash[124041]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-03-04T13:49:23.497 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 04 13:49:23 smithi107 bash[124041]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-03-04T13:49:23.497 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 04 13:49:23 smithi107 bash[124041]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-04T13:49:23.498 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 04 13:49:23 smithi107 bash[124041]: --> ceph-volume raw activate successful for osd ID: 0 2024-03-04T13:49:23.498 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 04 13:49:23 smithi107 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-0-activate[124055]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-03-04T13:49:23.498 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 04 13:49:23 smithi107 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-0-activate[124055]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-03-04T13:49:23.498 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 04 13:49:23 smithi107 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-0-activate[124055]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-03-04T13:49:23.498 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 04 13:49:23 smithi107 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-0-activate[124055]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-04T13:49:23.498 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 04 13:49:23 smithi107 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-0-activate[124055]: --> ceph-volume raw activate successful for osd ID: 0 2024-03-04T13:49:23.498 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 04 13:49:23 smithi107 podman[124041]: 2024-03-04 13:49:23.270520406 +0000 UTC m=+1.585387473 container died 072bf8f7413bc5184e44553314757fad50ef451d28e49b75433e7e86bc52e850 (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-0-activate, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240222, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-03-04T13:49:24.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:23 smithi107 ceph-mon[112578]: pgmap v64: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s; 50/264 objects degraded (18.939%) 2024-03-04T13:49:24.093 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 04 13:49:23 smithi107 podman[124183]: 2024-03-04 13:49:23.965416972 +0000 UTC m=+0.675520940 container remove 072bf8f7413bc5184e44553314757fad50ef451d28e49b75433e7e86bc52e850 (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-0-activate, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, io.buildah.version=1.33.5, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-03-04T13:49:24.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:23 smithi169 ceph-mon[86183]: pgmap v64: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s; 50/264 objects degraded (18.939%) 2024-03-04T13:49:24.594 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 04 13:49:24 smithi107 podman[124245]: 2024-03-04 13:49:24.235132819 +0000 UTC m=+0.038646832 image pull d71b839abdf892550525dad59fdf20a6b925b73dbcc7526f0d04da8380b9fabf quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377 2024-03-04T13:49:24.594 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 04 13:49:24 smithi107 podman[124245]: 2024-03-04 13:49:24.350041538 +0000 UTC m=+0.153555618 container create 5801a5540dc65ce77698d452b23e13632a4bf736fa4f5f30f57df1379a41f32a (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240222) 2024-03-04T13:49:25.017 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 04 13:49:24 smithi107 podman[124245]: 2024-03-04 13:49:24.638023249 +0000 UTC m=+0.441537295 container init 5801a5540dc65ce77698d452b23e13632a4bf736fa4f5f30f57df1379a41f32a (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-0, GIT_CLEAN=True, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240222, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-03-04T13:49:25.017 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 04 13:49:24 smithi107 podman[124245]: 2024-03-04 13:49:24.649030513 +0000 UTC m=+0.452544517 container start 5801a5540dc65ce77698d452b23e13632a4bf736fa4f5f30f57df1379a41f32a (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-0, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, GIT_BRANCH=HEAD, org.label-schema.build-date=20240222, CEPH_POINT_RELEASE=, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-03-04T13:49:25.017 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 04 13:49:24 smithi107 ceph-osd[124265]: -- 172.21.15.107:0/2646399192 <== mon.0 v2:172.21.15.107:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5640a2fe9680 con 0x5640a2fb3000 2024-03-04T13:49:25.017 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 04 13:49:24 smithi107 bash[124245]: 5801a5540dc65ce77698d452b23e13632a4bf736fa4f5f30f57df1379a41f32a 2024-03-04T13:49:25.017 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 04 13:49:24 smithi107 systemd[1]: Started Ceph osd.0 for 1fc09a58-da2b-11ee-95c6-87774f69a715. 2024-03-04T13:49:25.343 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 04 13:49:25 smithi107 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-0[124261]: 2024-03-04T13:49:25.210+0000 7fa2aac697c0 -1 Falling back to public interface 2024-03-04T13:49:26.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:25 smithi107 ceph-mon[112578]: pgmap v65: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 50/264 objects degraded (18.939%) 2024-03-04T13:49:26.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:25 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:49:26.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:25 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:49:26.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:25 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:49:26.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:25 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:49:26.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:25 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:49:26.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:25 smithi169 ceph-mon[86183]: pgmap v65: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 50/264 objects degraded (18.939%) 2024-03-04T13:49:26.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:25 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:49:26.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:25 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:49:26.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:25 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:49:26.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:25 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:49:26.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:25 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:49:27.843 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:27 smithi107 ceph-mon[112578]: pgmap v66: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.1 KiB/s rd, 2 op/s; 50/264 objects degraded (18.939%) 2024-03-04T13:49:28.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:27 smithi169 ceph-mon[86183]: pgmap v66: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.1 KiB/s rd, 2 op/s; 50/264 objects degraded (18.939%) 2024-03-04T13:49:30.032 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:29 smithi107 ceph-mon[112578]: pgmap v67: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s; 50/264 objects degraded (18.939%) 2024-03-04T13:49:30.033 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 04 13:49:29 smithi107 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-0[124261]: 2024-03-04T13:49:29.803+0000 7fa2aac697c0 -1 osd.0 0 read_superblock omap replica is missing. 2024-03-04T13:49:30.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:29 smithi169 ceph-mon[86183]: pgmap v67: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s; 50/264 objects degraded (18.939%) 2024-03-04T13:49:30.298 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 04 13:49:30 smithi107 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-0[124261]: 2024-03-04T13:49:30.203+0000 7fa2aac697c0 -1 osd.0 50 log_to_monitors true 2024-03-04T13:49:31.022 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:30 smithi107 ceph-mon[112578]: from='osd.0 [v2:172.21.15.107:6802/3996509230,v1:172.21.15.107:6803/3996509230]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-04T13:49:31.023 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 04 13:49:30 smithi107 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-0[124261]: 2024-03-04T13:49:30.750+0000 7fa29a000700 -1 osd.0 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-04T13:49:31.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:30 smithi169 ceph-mon[86183]: from='osd.0 [v2:172.21.15.107:6802/3996509230,v1:172.21.15.107:6803/3996509230]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-04T13:49:32.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:31 smithi107 ceph-mon[112578]: pgmap v68: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.3 KiB/s rd, 2 op/s; 50/264 objects degraded (18.939%) 2024-03-04T13:49:32.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:31 smithi107 ceph-mon[112578]: from='osd.0 [v2:172.21.15.107:6802/3996509230,v1:172.21.15.107:6803/3996509230]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-04T13:49:32.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:31 smithi107 ceph-mon[112578]: osdmap e53: 6 total, 5 up, 6 in 2024-03-04T13:49:32.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:31 smithi107 ceph-mon[112578]: from='osd.0 [v2:172.21.15.107:6802/3996509230,v1:172.21.15.107:6803/3996509230]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-03-04T13:49:32.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:31 smithi169 ceph-mon[86183]: pgmap v68: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.3 KiB/s rd, 2 op/s; 50/264 objects degraded (18.939%) 2024-03-04T13:49:32.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:31 smithi169 ceph-mon[86183]: from='osd.0 [v2:172.21.15.107:6802/3996509230,v1:172.21.15.107:6803/3996509230]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-04T13:49:32.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:31 smithi169 ceph-mon[86183]: osdmap e53: 6 total, 5 up, 6 in 2024-03-04T13:49:32.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:31 smithi169 ceph-mon[86183]: from='osd.0 [v2:172.21.15.107:6802/3996509230,v1:172.21.15.107:6803/3996509230]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-03-04T13:49:33.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:32 smithi107 ceph-mon[112578]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-04T13:49:33.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:32 smithi107 ceph-mon[112578]: osd.0 [v2:172.21.15.107:6802/3996509230,v1:172.21.15.107:6803/3996509230] boot 2024-03-04T13:49:33.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:32 smithi107 ceph-mon[112578]: osdmap e54: 6 total, 6 up, 6 in 2024-03-04T13:49:33.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:32 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-04T13:49:33.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:32 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:49:33.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:32 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:49:33.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:32 smithi107 ceph-mon[112578]: osdmap e55: 6 total, 6 up, 6 in 2024-03-04T13:49:33.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:32 smithi169 ceph-mon[86183]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-04T13:49:33.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:32 smithi169 ceph-mon[86183]: osd.0 [v2:172.21.15.107:6802/3996509230,v1:172.21.15.107:6803/3996509230] boot 2024-03-04T13:49:33.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:32 smithi169 ceph-mon[86183]: osdmap e54: 6 total, 6 up, 6 in 2024-03-04T13:49:33.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:32 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-04T13:49:33.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:32 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:49:33.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:32 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:49:33.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:32 smithi169 ceph-mon[86183]: osdmap e55: 6 total, 6 up, 6 in 2024-03-04T13:49:34.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:33 smithi107 ceph-mon[112578]: pgmap v71: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 255 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s; 50/264 objects degraded (18.939%) 2024-03-04T13:49:34.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:33 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:49:34.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:33 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:49:34.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:33 smithi169 ceph-mon[86183]: pgmap v71: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 255 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s; 50/264 objects degraded (18.939%) 2024-03-04T13:49:34.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:33 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:49:34.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:33 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:49:36.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:35 smithi107 ceph-mon[112578]: pgmap v73: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 255 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 50/264 objects degraded (18.939%) 2024-03-04T13:49:36.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:35 smithi169 ceph-mon[86183]: pgmap v73: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 255 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 50/264 objects degraded (18.939%) 2024-03-04T13:49:37.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:36 smithi107 ceph-mon[112578]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 50/264 objects degraded (18.939%), 19 pgs degraded) 2024-03-04T13:49:37.360 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:36 smithi169 ceph-mon[86183]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 50/264 objects degraded (18.939%), 19 pgs degraded) 2024-03-04T13:49:38.332 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:37 smithi107 ceph-mon[112578]: pgmap v74: 65 pgs: 65 active+clean; 255 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 682 B/s rd, 1 op/s 2024-03-04T13:49:38.360 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:37 smithi169 ceph-mon[86183]: pgmap v74: 65 pgs: 65 active+clean; 255 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 682 B/s rd, 1 op/s 2024-03-04T13:49:40.343 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:39 smithi107 ceph-mon[112578]: pgmap v75: 65 pgs: 65 active+clean; 255 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 516 B/s rd, 1 op/s 2024-03-04T13:49:40.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:39 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:49:40.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:39 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:49:40.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:39 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:49:40.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:39 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:49:40.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:39 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:49:40.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:39 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:49:40.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:39 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:49:40.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:39 smithi169 ceph-mon[86183]: pgmap v75: 65 pgs: 65 active+clean; 255 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 516 B/s rd, 1 op/s 2024-03-04T13:49:40.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:39 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:49:40.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:39 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:49:40.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:39 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:49:40.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:39 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:49:40.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:39 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:49:40.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:39 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:49:40.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:39 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:49:41.071 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:40 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:49:41.071 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:40 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:49:41.072 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:40 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:49:41.072 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:40 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:49:41.072 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:40 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-04T13:49:41.360 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:40 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:49:41.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:40 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:49:41.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:40 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:49:41.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:40 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:49:41.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:40 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-04T13:49:42.343 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:41 smithi107 ceph-mon[112578]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-04T13:49:42.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:41 smithi107 ceph-mon[112578]: Upgrade: osd.1 is safe to restart 2024-03-04T13:49:42.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:41 smithi107 ceph-mon[112578]: pgmap v76: 65 pgs: 65 active+clean; 255 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.0 KiB/s rd, 2 op/s 2024-03-04T13:49:42.360 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:41 smithi169 ceph-mon[86183]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-04T13:49:42.360 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:41 smithi169 ceph-mon[86183]: Upgrade: osd.1 is safe to restart 2024-03-04T13:49:42.360 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:41 smithi169 ceph-mon[86183]: pgmap v76: 65 pgs: 65 active+clean; 255 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.0 KiB/s rd, 2 op/s 2024-03-04T13:49:43.194 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:43 smithi107 ceph-mon[112578]: Upgrade: Updating osd.1 2024-03-04T13:49:43.194 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:43 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:49:43.194 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:43 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-04T13:49:43.194 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:43 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:49:43.194 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:43 smithi107 ceph-mon[112578]: Deploying daemon osd.1 on smithi107 2024-03-04T13:49:43.194 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:43 smithi107 ceph-mon[112578]: pgmap v77: 65 pgs: 65 active+clean; 255 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 921 B/s rd, 1 op/s 2024-03-04T13:49:43.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:43 smithi169 ceph-mon[86183]: Upgrade: Updating osd.1 2024-03-04T13:49:43.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:43 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:49:43.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:43 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-04T13:49:43.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:43 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:49:43.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:43 smithi169 ceph-mon[86183]: Deploying daemon osd.1 on smithi107 2024-03-04T13:49:43.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:43 smithi169 ceph-mon[86183]: pgmap v77: 65 pgs: 65 active+clean; 255 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 921 B/s rd, 1 op/s 2024-03-04T13:49:45.844 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:45 smithi107 ceph-mon[112578]: pgmap v78: 65 pgs: 65 active+clean; 255 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 772 B/s rd, 1 op/s 2024-03-04T13:49:45.844 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:45 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:49:46.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:45 smithi169 ceph-mon[86183]: pgmap v78: 65 pgs: 65 active+clean; 255 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 772 B/s rd, 1 op/s 2024-03-04T13:49:46.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:45 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:49:47.093 INFO:journalctl@ceph.osd.1.smithi107.stdout:Mar 04 13:49:46 smithi107 systemd[1]: Stopping Ceph osd.1 for 1fc09a58-da2b-11ee-95c6-87774f69a715... 2024-03-04T13:49:47.595 INFO:journalctl@ceph.osd.1.smithi107.stdout:Mar 04 13:49:47 smithi107 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-1[54727]: 2024-03-04T13:49:47.264+0000 7f4258a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-04T13:49:47.595 INFO:journalctl@ceph.osd.1.smithi107.stdout:Mar 04 13:49:47 smithi107 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-1[54727]: 2024-03-04T13:49:47.264+0000 7f4258a00700 -1 osd.1 55 *** Got signal Terminated *** 2024-03-04T13:49:47.595 INFO:journalctl@ceph.osd.1.smithi107.stdout:Mar 04 13:49:47 smithi107 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-1[54727]: 2024-03-04T13:49:47.264+0000 7f4258a00700 -1 osd.1 55 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-04T13:49:48.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:47 smithi107 ceph-mon[112578]: pgmap v79: 65 pgs: 65 active+clean; 255 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:49:48.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:47 smithi107 ceph-mon[112578]: osd.1 marked itself down and dead 2024-03-04T13:49:48.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:47 smithi169 ceph-mon[86183]: pgmap v79: 65 pgs: 65 active+clean; 255 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:49:48.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:47 smithi169 ceph-mon[86183]: osd.1 marked itself down and dead 2024-03-04T13:49:48.593 INFO:journalctl@ceph.osd.1.smithi107.stdout:Mar 04 13:49:48 smithi107 podman[129469]: 2024-03-04 13:49:48.31358694 +0000 UTC m=+1.199629082 container died 5fb2c0dd5769cae5dd99807e1f2306031f453f251b3546ecb42ca965e68024a7 (image=quay.ceph.io/ceph-ci/ceph@sha256:ee85afea73f504aaae524393a00a3b140e2a1cfd81d76da4525eaa6e73286a7d, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-1, org.label-schema.build-date=20240222, RELEASE=quincy-42608bb, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-03-04T13:49:49.028 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:48 smithi107 ceph-mon[112578]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-04T13:49:49.028 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:48 smithi107 ceph-mon[112578]: osdmap e56: 6 total, 5 up, 6 in 2024-03-04T13:49:49.028 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:48 smithi107 ceph-mon[112578]: osdmap e57: 6 total, 5 up, 6 in 2024-03-04T13:49:49.029 INFO:journalctl@ceph.osd.1.smithi107.stdout:Mar 04 13:49:48 smithi107 podman[129469]: 2024-03-04 13:49:48.769064512 +0000 UTC m=+1.655106730 container cleanup 5fb2c0dd5769cae5dd99807e1f2306031f453f251b3546ecb42ca965e68024a7 (image=quay.ceph.io/ceph-ci/ceph@sha256:ee85afea73f504aaae524393a00a3b140e2a1cfd81d76da4525eaa6e73286a7d, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-1, RELEASE=quincy-42608bb, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-03-04T13:49:49.029 INFO:journalctl@ceph.osd.1.smithi107.stdout:Mar 04 13:49:48 smithi107 bash[129469]: ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-1 2024-03-04T13:49:49.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:48 smithi169 ceph-mon[86183]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-04T13:49:49.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:48 smithi169 ceph-mon[86183]: osdmap e56: 6 total, 5 up, 6 in 2024-03-04T13:49:49.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:48 smithi169 ceph-mon[86183]: osdmap e57: 6 total, 5 up, 6 in 2024-03-04T13:49:49.344 INFO:journalctl@ceph.osd.1.smithi107.stdout:Mar 04 13:49:49 smithi107 podman[129488]: 2024-03-04 13:49:49.027256835 +0000 UTC m=+0.703509130 container remove 5fb2c0dd5769cae5dd99807e1f2306031f453f251b3546ecb42ca965e68024a7 (image=quay.ceph.io/ceph-ci/ceph@sha256:ee85afea73f504aaae524393a00a3b140e2a1cfd81d76da4525eaa6e73286a7d, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, RELEASE=quincy-42608bb, maintainer=Guillaume Abrioux ) 2024-03-04T13:49:50.059 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:49 smithi107 ceph-mon[112578]: pgmap v81: 65 pgs: 12 stale+active+clean, 53 active+clean; 255 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s 2024-03-04T13:49:50.059 INFO:journalctl@ceph.osd.1.smithi107.stdout:Mar 04 13:49:49 smithi107 podman[129584]: 2024-03-04 13:49:49.760661851 +0000 UTC m=+0.033945092 image pull d71b839abdf892550525dad59fdf20a6b925b73dbcc7526f0d04da8380b9fabf quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377 2024-03-04T13:49:50.060 INFO:journalctl@ceph.osd.1.smithi107.stdout:Mar 04 13:49:49 smithi107 podman[129584]: 2024-03-04 13:49:49.878480213 +0000 UTC m=+0.151763481 container create 682c78e76ec0bf15f3aa21ae434b13f302cd061d835c2b6740375a680da845e2 (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-1-deactivate, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, org.label-schema.build-date=20240222, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-03-04T13:49:50.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:49 smithi169 ceph-mon[86183]: pgmap v81: 65 pgs: 12 stale+active+clean, 53 active+clean; 255 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s 2024-03-04T13:49:50.344 INFO:journalctl@ceph.osd.1.smithi107.stdout:Mar 04 13:49:50 smithi107 podman[129584]: 2024-03-04 13:49:50.232451991 +0000 UTC m=+0.505735298 container init 682c78e76ec0bf15f3aa21ae434b13f302cd061d835c2b6740375a680da845e2 (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-1-deactivate, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, org.label-schema.build-date=20240222) 2024-03-04T13:49:50.344 INFO:journalctl@ceph.osd.1.smithi107.stdout:Mar 04 13:49:50 smithi107 podman[129584]: 2024-03-04 13:49:50.243849832 +0000 UTC m=+0.517133086 container start 682c78e76ec0bf15f3aa21ae434b13f302cd061d835c2b6740375a680da845e2 (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-1-deactivate, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, CEPH_POINT_RELEASE=) 2024-03-04T13:49:50.686 INFO:journalctl@ceph.osd.1.smithi107.stdout:Mar 04 13:49:50 smithi107 podman[129584]: 2024-03-04 13:49:50.347866777 +0000 UTC m=+0.621150091 container attach 682c78e76ec0bf15f3aa21ae434b13f302cd061d835c2b6740375a680da845e2 (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-1-deactivate, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240222, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240304.042522-717ce3c) 2024-03-04T13:49:51.083 INFO:journalctl@ceph.osd.1.smithi107.stdout:Mar 04 13:49:50 smithi107 podman[129584]: 2024-03-04 13:49:50.687057938 +0000 UTC m=+0.960341239 container died 682c78e76ec0bf15f3aa21ae434b13f302cd061d835c2b6740375a680da845e2 (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-1-deactivate, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, CEPH_POINT_RELEASE=, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8) 2024-03-04T13:49:51.083 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:50 smithi107 ceph-mon[112578]: Health check failed: Degraded data redundancy: 33/264 objects degraded (12.500%), 11 pgs degraded (PG_DEGRADED) 2024-03-04T13:49:51.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:50 smithi169 ceph-mon[86183]: Health check failed: Degraded data redundancy: 33/264 objects degraded (12.500%), 11 pgs degraded (PG_DEGRADED) 2024-03-04T13:49:51.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:51.461+0000 7fabff400700 1 -- 172.21.15.107:0/3878659914 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fabf8102f50 msgr2=0x7fabf8103320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:49:51.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:51.461+0000 7fabff400700 1 --2- 172.21.15.107:0/3878659914 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fabf8102f50 0x7fabf8103320 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fabec009ce0 tx=0x7fabec004d10 comp rx=0 tx=0).stop 2024-03-04T13:49:51.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:51.461+0000 7fabff400700 1 -- 172.21.15.107:0/3878659914 shutdown_connections 2024-03-04T13:49:51.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:51.461+0000 7fabff400700 1 --2- 172.21.15.107:0/3878659914 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fabf81038f0 0x7fabf80fcf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:49:51.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:51.461+0000 7fabff400700 1 --2- 172.21.15.107:0/3878659914 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fabf8102f50 0x7fabf8103320 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:49:51.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:51.461+0000 7fabff400700 1 -- 172.21.15.107:0/3878659914 >> 172.21.15.107:0/3878659914 conn(0x7fabf80f8ae0 msgr2=0x7fabf80faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:49:51.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:51.461+0000 7fabff400700 1 -- 172.21.15.107:0/3878659914 shutdown_connections 2024-03-04T13:49:51.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:51.462+0000 7fabff400700 1 -- 172.21.15.107:0/3878659914 wait complete. 2024-03-04T13:49:51.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:51.462+0000 7fabff400700 1 Processor -- start 2024-03-04T13:49:51.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:51.463+0000 7fabff400700 1 -- start start 2024-03-04T13:49:51.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:51.463+0000 7fabff400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fabf8102f50 0x7fabf8115e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:49:51.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:51.463+0000 7fabff400700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fabf81038f0 0x7fabf8116340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:49:51.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:51.463+0000 7fabff400700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf8116990 con 0x7fabf8102f50 2024-03-04T13:49:51.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:51.463+0000 7fabff400700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf810fce0 con 0x7fabf81038f0 2024-03-04T13:49:51.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:51.463+0000 7fabfca00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fabf8102f50 0x7fabf8115e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:49:51.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:51.463+0000 7fabfca00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fabf8102f50 0x7fabf8115e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38572/0 (socket says 172.21.15.107:38572) 2024-03-04T13:49:51.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:51.463+0000 7fabfca00700 1 -- 172.21.15.107:0/1532643221 learned_addr learned my addr 172.21.15.107:0/1532643221 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:49:51.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:51.464+0000 7fabf7e00700 1 --2- 172.21.15.107:0/1532643221 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fabf81038f0 0x7fabf8116340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:49:51.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:51.464+0000 7fabfca00700 1 -- 172.21.15.107:0/1532643221 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fabf81038f0 msgr2=0x7fabf8116340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:49:51.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:51.464+0000 7fabfca00700 1 --2- 172.21.15.107:0/1532643221 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fabf81038f0 0x7fabf8116340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:49:51.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:51.464+0000 7fabfca00700 1 -- 172.21.15.107:0/1532643221 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabec009940 con 0x7fabf8102f50 2024-03-04T13:49:51.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:51.464+0000 7fabfca00700 1 --2- 172.21.15.107:0/1532643221 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fabf8102f50 0x7fabf8115e00 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fabec004b70 tx=0x7fabec004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:49:51.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:51.465+0000 7fabf4c00700 1 -- 172.21.15.107:0/1532643221 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fabec00e040 con 0x7fabf8102f50 2024-03-04T13:49:51.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:51.465+0000 7fabf4c00700 1 -- 172.21.15.107:0/1532643221 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fabec004610 con 0x7fabf8102f50 2024-03-04T13:49:51.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:51.465+0000 7fabf4c00700 1 -- 172.21.15.107:0/1532643221 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fabec01c810 con 0x7fabf8102f50 2024-03-04T13:49:51.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:51.465+0000 7fabff400700 1 -- 172.21.15.107:0/1532643221 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabf810ff60 con 0x7fabf8102f50 2024-03-04T13:49:51.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:51.465+0000 7fabff400700 1 -- 172.21.15.107:0/1532643221 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabf81104a0 con 0x7fabf8102f50 2024-03-04T13:49:51.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:51.466+0000 7fabff400700 1 -- 172.21.15.107:0/1532643221 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabf80fc6e0 con 0x7fabf8102f50 2024-03-04T13:49:51.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:51.467+0000 7fabf4c00700 1 -- 172.21.15.107:0/1532643221 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fabec012c00 con 0x7fabf8102f50 2024-03-04T13:49:51.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:51.468+0000 7fabf4c00700 1 --2- 172.21.15.107:0/1532643221 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fabe006d7f0 0x7fabe006fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:49:51.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:51.468+0000 7fabf4c00700 1 -- 172.21.15.107:0/1532643221 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fabec08ea80 con 0x7fabf8102f50 2024-03-04T13:49:51.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:51.469+0000 7fabf7e00700 1 --2- 172.21.15.107:0/1532643221 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fabe006d7f0 0x7fabe006fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:49:51.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:51.469+0000 7fabf7e00700 1 --2- 172.21.15.107:0/1532643221 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fabe006d7f0 0x7fabe006fca0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fabe4007fd0 tx=0x7fabe4009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:49:51.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:51.474+0000 7fabf4c00700 1 -- 172.21.15.107:0/1532643221 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fabec058300 con 0x7fabf8102f50 2024-03-04T13:49:51.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:51.734+0000 7fabff400700 1 -- 172.21.15.107:0/1532643221 --> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fabf8111490 con 0x7fabe006d7f0 2024-03-04T13:49:51.737 INFO:journalctl@ceph.osd.1.smithi107.stdout:Mar 04 13:49:51 smithi107 podman[129610]: 2024-03-04 13:49:51.451867414 +0000 UTC m=+0.749833855 container remove 682c78e76ec0bf15f3aa21ae434b13f302cd061d835c2b6740375a680da845e2 (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-1-deactivate, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-04T13:49:51.737 INFO:journalctl@ceph.osd.1.smithi107.stdout:Mar 04 13:49:51 smithi107 systemd[1]: ceph-1fc09a58-da2b-11ee-95c6-87774f69a715@osd.1.service: Deactivated successfully. 2024-03-04T13:49:51.737 INFO:journalctl@ceph.osd.1.smithi107.stdout:Mar 04 13:49:51 smithi107 systemd[1]: Stopped Ceph osd.1 for 1fc09a58-da2b-11ee-95c6-87774f69a715. 2024-03-04T13:49:51.737 INFO:journalctl@ceph.osd.1.smithi107.stdout:Mar 04 13:49:51 smithi107 systemd[1]: ceph-1fc09a58-da2b-11ee-95c6-87774f69a715@osd.1.service: Consumed 1min 48.259s CPU time. 2024-03-04T13:49:51.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:51.738+0000 7fabf4c00700 1 -- 172.21.15.107:0/1532643221 <== mgr.34102 v2:172.21.15.107:6800/1223039812 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fabf8111490 con 0x7fabe006d7f0 2024-03-04T13:49:51.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:51.741+0000 7fabff400700 1 -- 172.21.15.107:0/1532643221 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fabe006d7f0 msgr2=0x7fabe006fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:49:51.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:51.741+0000 7fabff400700 1 --2- 172.21.15.107:0/1532643221 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fabe006d7f0 0x7fabe006fca0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fabe4007fd0 tx=0x7fabe4009040 comp rx=0 tx=0).stop 2024-03-04T13:49:51.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:51.741+0000 7fabff400700 1 -- 172.21.15.107:0/1532643221 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fabf8102f50 msgr2=0x7fabf8115e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:49:51.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:51.741+0000 7fabff400700 1 --2- 172.21.15.107:0/1532643221 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fabf8102f50 0x7fabf8115e00 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fabec004b70 tx=0x7fabec004d10 comp rx=0 tx=0).stop 2024-03-04T13:49:51.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:51.741+0000 7fabff400700 1 -- 172.21.15.107:0/1532643221 shutdown_connections 2024-03-04T13:49:51.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:51.741+0000 7fabff400700 1 --2- 172.21.15.107:0/1532643221 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fabe006d7f0 0x7fabe006fca0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:49:51.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:51.741+0000 7fabff400700 1 --2- 172.21.15.107:0/1532643221 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fabf8102f50 0x7fabf8115e00 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:49:51.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:51.741+0000 7fabff400700 1 --2- 172.21.15.107:0/1532643221 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fabf81038f0 0x7fabf8116340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:49:51.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:51.741+0000 7fabff400700 1 -- 172.21.15.107:0/1532643221 >> 172.21.15.107:0/1532643221 conn(0x7fabf80f8ae0 msgr2=0x7fabf80fa420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:49:51.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:51.741+0000 7fabff400700 1 -- 172.21.15.107:0/1532643221 shutdown_connections 2024-03-04T13:49:51.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:51.741+0000 7fabff400700 1 -- 172.21.15.107:0/1532643221 wait complete. 2024-03-04T13:49:51.758 INFO:teuthology.orchestra.run.smithi107.stdout:true 2024-03-04T13:49:51.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:51.899+0000 7f6217a00700 1 -- 172.21.15.107:0/3038952855 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6210102f50 msgr2=0x7f6210103320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:49:51.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:51.899+0000 7f6217a00700 1 --2- 172.21.15.107:0/3038952855 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6210102f50 0x7f6210103320 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f61f8004940 tx=0x7f61f8009e30 comp rx=0 tx=0).stop 2024-03-04T13:49:51.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:51.899+0000 7f6217a00700 1 -- 172.21.15.107:0/3038952855 shutdown_connections 2024-03-04T13:49:51.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:51.899+0000 7f6217a00700 1 --2- 172.21.15.107:0/3038952855 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62101038f0 0x7f62100fcf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:49:51.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:51.899+0000 7f6217a00700 1 --2- 172.21.15.107:0/3038952855 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6210102f50 0x7f6210103320 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:49:51.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:51.899+0000 7f6217a00700 1 -- 172.21.15.107:0/3038952855 >> 172.21.15.107:0/3038952855 conn(0x7f62100f8ae0 msgr2=0x7f62100faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:49:51.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:51.900+0000 7f6217a00700 1 -- 172.21.15.107:0/3038952855 shutdown_connections 2024-03-04T13:49:51.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:51.900+0000 7f6217a00700 1 -- 172.21.15.107:0/3038952855 wait complete. 2024-03-04T13:49:51.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:51.900+0000 7f6217a00700 1 Processor -- start 2024-03-04T13:49:51.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:51.901+0000 7f6217a00700 1 -- start start 2024-03-04T13:49:51.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:51.901+0000 7f6217a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6210102f50 0x7f6210070290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:49:51.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:51.901+0000 7f6217a00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62101038f0 0x7f62100707d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:49:51.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:51.901+0000 7f6217a00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6210070dd0 con 0x7f6210102f50 2024-03-04T13:49:51.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:51.901+0000 7f6217a00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6210070f40 con 0x7f62101038f0 2024-03-04T13:49:51.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:51.901+0000 7f6215000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6210102f50 0x7f6210070290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:49:51.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:51.901+0000 7f6215000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6210102f50 0x7f6210070290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38580/0 (socket says 172.21.15.107:38580) 2024-03-04T13:49:51.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:51.901+0000 7f6215000700 1 -- 172.21.15.107:0/1725956788 learned_addr learned my addr 172.21.15.107:0/1725956788 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:49:51.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:51.902+0000 7f620fe00700 1 --2- 172.21.15.107:0/1725956788 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62101038f0 0x7f62100707d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:49:51.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:51.902+0000 7f6215000700 1 -- 172.21.15.107:0/1725956788 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62101038f0 msgr2=0x7f62100707d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:49:51.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:51.902+0000 7f6215000700 1 --2- 172.21.15.107:0/1725956788 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62101038f0 0x7f62100707d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:49:51.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:51.902+0000 7f6215000700 1 -- 172.21.15.107:0/1725956788 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61f8005040 con 0x7f6210102f50 2024-03-04T13:49:51.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:51.902+0000 7f6215000700 1 --2- 172.21.15.107:0/1725956788 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6210102f50 0x7f6210070290 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f61f8009150 tx=0x7f61f8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:49:51.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:51.903+0000 7f620cc00700 1 -- 172.21.15.107:0/1725956788 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f61f800e040 con 0x7f6210102f50 2024-03-04T13:49:51.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:51.903+0000 7f620cc00700 1 -- 172.21.15.107:0/1725956788 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f61f800bbf0 con 0x7f6210102f50 2024-03-04T13:49:51.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:51.903+0000 7f6217a00700 1 -- 172.21.15.107:0/1725956788 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6210119c90 con 0x7f6210102f50 2024-03-04T13:49:51.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:51.903+0000 7f620cc00700 1 -- 172.21.15.107:0/1725956788 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f61f801c630 con 0x7f6210102f50 2024-03-04T13:49:51.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:51.903+0000 7f6217a00700 1 -- 172.21.15.107:0/1725956788 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f621011a0f0 con 0x7f6210102f50 2024-03-04T13:49:51.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:51.904+0000 7f6217a00700 1 -- 172.21.15.107:0/1725956788 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62100fc6e0 con 0x7f6210102f50 2024-03-04T13:49:51.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:51.905+0000 7f620cc00700 1 -- 172.21.15.107:0/1725956788 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f61f800bd60 con 0x7f6210102f50 2024-03-04T13:49:51.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:51.907+0000 7f620cc00700 1 --2- 172.21.15.107:0/1725956788 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f61f406d7f0 0x7f61f406fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:49:51.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:51.907+0000 7f620cc00700 1 -- 172.21.15.107:0/1725956788 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f61f808e290 con 0x7f6210102f50 2024-03-04T13:49:51.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:51.907+0000 7f620fe00700 1 --2- 172.21.15.107:0/1725956788 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f61f406d7f0 0x7f61f406fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:49:51.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:51.908+0000 7f620fe00700 1 --2- 172.21.15.107:0/1725956788 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f61f406d7f0 0x7f61f406fca0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6200007fd0 tx=0x7f6200009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:49:51.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:51.914+0000 7f620cc00700 1 -- 172.21.15.107:0/1725956788 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f61f8057bb0 con 0x7f6210102f50 2024-03-04T13:49:52.093 INFO:journalctl@ceph.osd.1.smithi107.stdout:Mar 04 13:49:51 smithi107 systemd[1]: Starting Ceph osd.1 for 1fc09a58-da2b-11ee-95c6-87774f69a715... 2024-03-04T13:49:52.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:51 smithi107 ceph-mon[112578]: pgmap v83: 65 pgs: 11 active+undersized+degraded, 3 stale+active+clean, 18 active+undersized, 33 active+clean; 255 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 33/264 objects degraded (12.500%) 2024-03-04T13:49:52.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:51 smithi169 ceph-mon[86183]: pgmap v83: 65 pgs: 11 active+undersized+degraded, 3 stale+active+clean, 18 active+undersized, 33 active+clean; 255 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 33/264 objects degraded (12.500%) 2024-03-04T13:49:52.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:52.182+0000 7f6217a00700 1 -- 172.21.15.107:0/1725956788 --> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6210071ae0 con 0x7f61f406d7f0 2024-03-04T13:49:52.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:52.184+0000 7f620cc00700 1 -- 172.21.15.107:0/1725956788 <== mgr.34102 v2:172.21.15.107:6800/1223039812 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f6210071ae0 con 0x7f61f406d7f0 2024-03-04T13:49:52.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:52.187+0000 7f6217a00700 1 -- 172.21.15.107:0/1725956788 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f61f406d7f0 msgr2=0x7f61f406fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:49:52.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:52.187+0000 7f6217a00700 1 --2- 172.21.15.107:0/1725956788 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f61f406d7f0 0x7f61f406fca0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6200007fd0 tx=0x7f6200009040 comp rx=0 tx=0).stop 2024-03-04T13:49:52.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:52.187+0000 7f6217a00700 1 -- 172.21.15.107:0/1725956788 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6210102f50 msgr2=0x7f6210070290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:49:52.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:52.187+0000 7f6217a00700 1 --2- 172.21.15.107:0/1725956788 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6210102f50 0x7f6210070290 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f61f8009150 tx=0x7f61f8009e30 comp rx=0 tx=0).stop 2024-03-04T13:49:52.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:52.187+0000 7f6217a00700 1 -- 172.21.15.107:0/1725956788 shutdown_connections 2024-03-04T13:49:52.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:52.187+0000 7f6217a00700 1 --2- 172.21.15.107:0/1725956788 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f61f406d7f0 0x7f61f406fca0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:49:52.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:52.187+0000 7f6217a00700 1 --2- 172.21.15.107:0/1725956788 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6210102f50 0x7f6210070290 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:49:52.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:52.187+0000 7f6217a00700 1 --2- 172.21.15.107:0/1725956788 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62101038f0 0x7f62100707d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:49:52.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:52.187+0000 7f6217a00700 1 -- 172.21.15.107:0/1725956788 >> 172.21.15.107:0/1725956788 conn(0x7f62100f8ae0 msgr2=0x7f62100fa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:49:52.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:52.188+0000 7f6217a00700 1 -- 172.21.15.107:0/1725956788 shutdown_connections 2024-03-04T13:49:52.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:52.188+0000 7f6217a00700 1 -- 172.21.15.107:0/1725956788 wait complete. 2024-03-04T13:49:52.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:52.338+0000 7f8ebc800700 1 -- 172.21.15.107:0/1508879871 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8eb40fe900 msgr2=0x7f8eb4106f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:49:52.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:52.338+0000 7f8ebc800700 1 --2- 172.21.15.107:0/1508879871 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8eb40fe900 0x7f8eb4106f40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8ea4004940 tx=0x7f8ea4009e30 comp rx=0 tx=0).stop 2024-03-04T13:49:52.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:52.339+0000 7f8ebc800700 1 -- 172.21.15.107:0/1508879871 shutdown_connections 2024-03-04T13:49:52.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:52.339+0000 7f8ebc800700 1 --2- 172.21.15.107:0/1508879871 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8eb40fe900 0x7f8eb4106f40 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:49:52.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:52.339+0000 7f8ebc800700 1 --2- 172.21.15.107:0/1508879871 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8eb40fdf60 0x7f8eb40fe330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:49:52.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:52.339+0000 7f8ebc800700 1 -- 172.21.15.107:0/1508879871 >> 172.21.15.107:0/1508879871 conn(0x7f8eb40f7a10 msgr2=0x7f8eb40f9e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:49:52.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:52.339+0000 7f8ebc800700 1 -- 172.21.15.107:0/1508879871 shutdown_connections 2024-03-04T13:49:52.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:52.339+0000 7f8ebc800700 1 -- 172.21.15.107:0/1508879871 wait complete. 2024-03-04T13:49:52.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:52.340+0000 7f8ebc800700 1 Processor -- start 2024-03-04T13:49:52.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:52.340+0000 7f8ebc800700 1 -- start start 2024-03-04T13:49:52.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:52.341+0000 7f8ebc800700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8eb40fdf60 0x7f8eb4115d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:49:52.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:52.341+0000 7f8ebc800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8eb40fe900 0x7f8eb4116270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:49:52.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:52.341+0000 7f8ebc800700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb41168c0 con 0x7f8eb40fe900 2024-03-04T13:49:52.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:52.341+0000 7f8ebc800700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb410fbd0 con 0x7f8eb40fdf60 2024-03-04T13:49:52.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:52.341+0000 7f8eb9400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8eb40fe900 0x7f8eb4116270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:49:52.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:52.341+0000 7f8eb9400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8eb40fe900 0x7f8eb4116270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38594/0 (socket says 172.21.15.107:38594) 2024-03-04T13:49:52.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:52.341+0000 7f8eb9400700 1 -- 172.21.15.107:0/3574690824 learned_addr learned my addr 172.21.15.107:0/3574690824 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:49:52.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:52.341+0000 7f8eb9e00700 1 --2- 172.21.15.107:0/3574690824 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8eb40fdf60 0x7f8eb4115d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:49:52.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:52.342+0000 7f8eb9400700 1 -- 172.21.15.107:0/3574690824 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8eb40fdf60 msgr2=0x7f8eb4115d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:49:52.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:52.342+0000 7f8eb9400700 1 --2- 172.21.15.107:0/3574690824 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8eb40fdf60 0x7f8eb4115d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:49:52.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:52.342+0000 7f8eb9400700 1 -- 172.21.15.107:0/3574690824 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ea4005040 con 0x7f8eb40fe900 2024-03-04T13:49:52.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:52.343+0000 7f8eb9400700 1 --2- 172.21.15.107:0/3574690824 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8eb40fe900 0x7f8eb4116270 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8ea4009180 tx=0x7f8ea4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:49:52.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:52.343+0000 7f8eae000700 1 -- 172.21.15.107:0/3574690824 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8ea400e040 con 0x7f8eb40fe900 2024-03-04T13:49:52.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:52.343+0000 7f8eae000700 1 -- 172.21.15.107:0/3574690824 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8ea4004290 con 0x7f8eb40fe900 2024-03-04T13:49:52.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:52.343+0000 7f8ebc800700 1 -- 172.21.15.107:0/3574690824 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8eb410fe50 con 0x7f8eb40fe900 2024-03-04T13:49:52.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:52.343+0000 7f8eae000700 1 -- 172.21.15.107:0/3574690824 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8ea401c630 con 0x7f8eb40fe900 2024-03-04T13:49:52.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:52.344+0000 7f8ebc800700 1 -- 172.21.15.107:0/3574690824 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8eb4110390 con 0x7f8eb40fe900 2024-03-04T13:49:52.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:52.345+0000 7f8ebc800700 1 -- 172.21.15.107:0/3574690824 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8eb4106950 con 0x7f8eb40fe900 2024-03-04T13:49:52.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:52.347+0000 7f8eae000700 1 -- 172.21.15.107:0/3574690824 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f8ea400bbf0 con 0x7f8eb40fe900 2024-03-04T13:49:52.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:52.348+0000 7f8eae000700 1 --2- 172.21.15.107:0/3574690824 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f8e9c06d7f0 0x7f8e9c06fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:49:52.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:52.349+0000 7f8eae000700 1 -- 172.21.15.107:0/3574690824 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f8ea408ebf0 con 0x7f8eb40fe900 2024-03-04T13:49:52.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:52.349+0000 7f8eb9e00700 1 --2- 172.21.15.107:0/3574690824 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f8e9c06d7f0 0x7f8e9c06fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:49:52.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:52.349+0000 7f8eb9e00700 1 --2- 172.21.15.107:0/3574690824 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f8e9c06d7f0 0x7f8e9c06fca0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8ea8004f40 tx=0x7f8ea800b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:49:52.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:52.352+0000 7f8eae000700 1 -- 172.21.15.107:0/3574690824 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8ea405c020 con 0x7f8eb40fe900 2024-03-04T13:49:52.458 INFO:journalctl@ceph.osd.1.smithi107.stdout:Mar 04 13:49:52 smithi107 podman[129758]: 2024-03-04 13:49:52.109051956 +0000 UTC m=+0.045729561 image pull d71b839abdf892550525dad59fdf20a6b925b73dbcc7526f0d04da8380b9fabf quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377 2024-03-04T13:49:52.458 INFO:journalctl@ceph.osd.1.smithi107.stdout:Mar 04 13:49:52 smithi107 podman[129758]: 2024-03-04 13:49:52.228531899 +0000 UTC m=+0.165209529 container create f7a36f2809e6c9a540573da6b07de93f0d0db6f8d565c0ed9b7060a67edb3c0f (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-1-activate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-03-04T13:49:52.458 INFO:journalctl@ceph.osd.1.smithi107.stdout:Mar 04 13:49:52 smithi107 podman[129758]: 2024-03-04 13:49:52.457398978 +0000 UTC m=+0.394076614 container init f7a36f2809e6c9a540573da6b07de93f0d0db6f8d565c0ed9b7060a67edb3c0f (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-1-activate, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2) 2024-03-04T13:49:52.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:52.588+0000 7f8ebc800700 1 -- 172.21.15.107:0/3574690824 --> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f8eb404bf40 con 0x7f8e9c06d7f0 2024-03-04T13:49:52.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:52.600+0000 7f8eae000700 1 -- 172.21.15.107:0/3574690824 <== mgr.34102 v2:172.21.15.107:6800/1223039812 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f8eb404bf40 con 0x7f8e9c06d7f0 2024-03-04T13:49:52.601 INFO:teuthology.orchestra.run.smithi107.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-04T13:49:52.601 INFO:teuthology.orchestra.run.smithi107.stdout:alertmanager.smithi107 smithi107 *:9093,9094 running (4m) 20s ago 18m 14.0M - 0.25.0 c8568f914cd2 e5ed7be32538 2024-03-04T13:49:52.602 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi107 smithi107 running (65s) 20s ago 18m 6660k - 19.0.0-1845-g717ce3cc d71b839abdf8 0558022b0d84 2024-03-04T13:49:52.602 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi169 smithi169 running (58s) 53s ago 17m 6665k - 19.0.0-1845-g717ce3cc d71b839abdf8 221ca53aa0d3 2024-03-04T13:49:52.602 INFO:teuthology.orchestra.run.smithi107.stdout:grafana.smithi107 smithi107 *:3000 running (3m) 20s ago 17m 74.5M - 9.4.12 a0ea5e3c46d9 c5e4d33fb599 2024-03-04T13:49:52.602 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.riexmm smithi107 running (11m) 20s ago 11m 14.7M - 17.2.7-545-g42608bb1 9eec3f22859d fc921f7d4328 2024-03-04T13:49:52.602 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.thycho smithi107 running (11m) 20s ago 11m 98.3M - 17.2.7-545-g42608bb1 9eec3f22859d 4069c6e0e063 2024-03-04T13:49:52.602 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi169.bxitzk smithi169 running (11m) 53s ago 11m 14.2M - 17.2.7-545-g42608bb1 9eec3f22859d 7d4115228c7c 2024-03-04T13:49:52.602 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi169.jusudl smithi169 running (11m) 53s ago 11m 145M - 17.2.7-545-g42608bb1 9eec3f22859d 2d2f2cc452e8 2024-03-04T13:49:52.602 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi107.wpzgwl smithi107 *:8443,9283,8765 running (6m) 20s ago 19m 498M - 19.0.0-1845-g717ce3cc d71b839abdf8 1c5fdee45eac 2024-03-04T13:49:52.602 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi169.eihczr smithi169 *:8443,9283,8765 running (5m) 53s ago 16m 449M - 19.0.0-1845-g717ce3cc d71b839abdf8 b5cac2b7d22a 2024-03-04T13:49:52.602 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi107 smithi107 running (2m) 20s ago 20m 44.3M 2048M 19.0.0-1845-g717ce3cc d71b839abdf8 32f07144274f 2024-03-04T13:49:52.602 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi169 smithi169 running (2m) 53s ago 16m 33.9M 2048M 19.0.0-1845-g717ce3cc d71b839abdf8 e3527449381a 2024-03-04T13:49:52.602 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi107 smithi107 *:9100 running (5m) 20s ago 18m 7847k - 1.5.0 0da6a335fe13 8a8d3d4f724f 2024-03-04T13:49:52.602 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi169 smithi169 *:9100 running (5m) 53s ago 17m 7688k - 1.5.0 0da6a335fe13 f240f5a975bc 2024-03-04T13:49:52.603 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0 smithi107 running (28s) 20s ago 15m 11.2M 1558M 19.0.0-1845-g717ce3cc d71b839abdf8 5801a5540dc6 2024-03-04T13:49:52.603 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1 smithi107 running (15m) 20s ago 15m 220M 1558M 17.2.7-545-g42608bb1 9eec3f22859d 5fb2c0dd5769 2024-03-04T13:49:52.603 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2 smithi107 running (14m) 20s ago 14m 181M 1558M 17.2.7-545-g42608bb1 9eec3f22859d 61f30f643549 2024-03-04T13:49:52.603 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3 smithi169 running (14m) 53s ago 14m 236M 2582M 17.2.7-545-g42608bb1 9eec3f22859d cd91d8694764 2024-03-04T13:49:52.603 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4 smithi169 running (13m) 53s ago 13m 207M 2582M 17.2.7-545-g42608bb1 9eec3f22859d 08545e3d0659 2024-03-04T13:49:52.603 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5 smithi169 running (13m) 53s ago 13m 194M 2582M 17.2.7-545-g42608bb1 9eec3f22859d dcffe6cb3eac 2024-03-04T13:49:52.603 INFO:teuthology.orchestra.run.smithi107.stdout:prometheus.smithi107 smithi107 *:9095 running (4m) 20s ago 17m 49.6M - 2.43.0 a07b618ecd1d 1eeb1ad5cb75 2024-03-04T13:49:52.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:52.603+0000 7f8ebc800700 1 -- 172.21.15.107:0/3574690824 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f8e9c06d7f0 msgr2=0x7f8e9c06fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:49:52.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:52.603+0000 7f8ebc800700 1 --2- 172.21.15.107:0/3574690824 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f8e9c06d7f0 0x7f8e9c06fca0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8ea8004f40 tx=0x7f8ea800b040 comp rx=0 tx=0).stop 2024-03-04T13:49:52.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:52.603+0000 7f8ebc800700 1 -- 172.21.15.107:0/3574690824 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8eb40fe900 msgr2=0x7f8eb4116270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:49:52.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:52.603+0000 7f8ebc800700 1 --2- 172.21.15.107:0/3574690824 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8eb40fe900 0x7f8eb4116270 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8ea4009180 tx=0x7f8ea4009e30 comp rx=0 tx=0).stop 2024-03-04T13:49:52.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:52.603+0000 7f8ebc800700 1 -- 172.21.15.107:0/3574690824 shutdown_connections 2024-03-04T13:49:52.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:52.603+0000 7f8ebc800700 1 --2- 172.21.15.107:0/3574690824 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f8e9c06d7f0 0x7f8e9c06fca0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:49:52.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:52.603+0000 7f8ebc800700 1 --2- 172.21.15.107:0/3574690824 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8eb40fdf60 0x7f8eb4115d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:49:52.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:52.603+0000 7f8ebc800700 1 --2- 172.21.15.107:0/3574690824 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8eb40fe900 0x7f8eb4116270 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:49:52.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:52.603+0000 7f8ebc800700 1 -- 172.21.15.107:0/3574690824 >> 172.21.15.107:0/3574690824 conn(0x7f8eb40f7a10 msgr2=0x7f8eb4103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:49:52.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:52.604+0000 7f8ebc800700 1 -- 172.21.15.107:0/3574690824 shutdown_connections 2024-03-04T13:49:52.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:52.604+0000 7f8ebc800700 1 -- 172.21.15.107:0/3574690824 wait complete. 2024-03-04T13:49:52.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:52.752+0000 7f8afe000700 1 -- 172.21.15.107:0/2881399689 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8af8102f50 msgr2=0x7f8af8103320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:49:52.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:52.752+0000 7f8afe000700 1 --2- 172.21.15.107:0/2881399689 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8af8102f50 0x7f8af8103320 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8ae8004940 tx=0x7f8ae8009e30 comp rx=0 tx=0).stop 2024-03-04T13:49:52.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:52.752+0000 7f8afe000700 1 -- 172.21.15.107:0/2881399689 shutdown_connections 2024-03-04T13:49:52.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:52.752+0000 7f8afe000700 1 --2- 172.21.15.107:0/2881399689 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8af81038f0 0x7f8af80fcf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:49:52.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:52.752+0000 7f8afe000700 1 --2- 172.21.15.107:0/2881399689 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8af8102f50 0x7f8af8103320 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:49:52.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:52.752+0000 7f8afe000700 1 -- 172.21.15.107:0/2881399689 >> 172.21.15.107:0/2881399689 conn(0x7f8af80f8ae0 msgr2=0x7f8af80faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:49:52.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:52.752+0000 7f8afe000700 1 -- 172.21.15.107:0/2881399689 shutdown_connections 2024-03-04T13:49:52.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:52.752+0000 7f8afe000700 1 -- 172.21.15.107:0/2881399689 wait complete. 2024-03-04T13:49:52.754 INFO:journalctl@ceph.osd.1.smithi107.stdout:Mar 04 13:49:52 smithi107 podman[129758]: 2024-03-04 13:49:52.469553509 +0000 UTC m=+0.406231108 container start f7a36f2809e6c9a540573da6b07de93f0d0db6f8d565c0ed9b7060a67edb3c0f (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-1-activate, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-04T13:49:52.754 INFO:journalctl@ceph.osd.1.smithi107.stdout:Mar 04 13:49:52 smithi107 podman[129758]: 2024-03-04 13:49:52.622996102 +0000 UTC m=+0.559673739 container attach f7a36f2809e6c9a540573da6b07de93f0d0db6f8d565c0ed9b7060a67edb3c0f (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-1-activate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-03-04T13:49:52.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:52.753+0000 7f8afe000700 1 Processor -- start 2024-03-04T13:49:52.756 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:52.753+0000 7f8afe000700 1 -- start start 2024-03-04T13:49:52.756 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:52.753+0000 7f8afe000700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8af8102f50 0x7f8af8070290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:49:52.756 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:52.754+0000 7f8afe000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8af81038f0 0x7f8af80707d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:49:52.756 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:52.754+0000 7f8afe000700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8af8070dd0 con 0x7f8af81038f0 2024-03-04T13:49:52.756 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:52.754+0000 7f8afe000700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8af8070f40 con 0x7f8af8102f50 2024-03-04T13:49:52.756 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:52.754+0000 7f8af6a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8af81038f0 0x7f8af80707d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:49:52.756 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:52.754+0000 7f8af6a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8af81038f0 0x7f8af80707d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38614/0 (socket says 172.21.15.107:38614) 2024-03-04T13:49:52.756 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:52.754+0000 7f8af6a00700 1 -- 172.21.15.107:0/2341350502 learned_addr learned my addr 172.21.15.107:0/2341350502 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:49:52.756 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:52.754+0000 7f8af7400700 1 --2- 172.21.15.107:0/2341350502 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8af8102f50 0x7f8af8070290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:49:52.756 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:52.755+0000 7f8af6a00700 1 -- 172.21.15.107:0/2341350502 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8af8102f50 msgr2=0x7f8af8070290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:49:52.756 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:52.755+0000 7f8af6a00700 1 --2- 172.21.15.107:0/2341350502 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8af8102f50 0x7f8af8070290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:49:52.756 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:52.755+0000 7f8af6a00700 1 -- 172.21.15.107:0/2341350502 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ae8005040 con 0x7f8af81038f0 2024-03-04T13:49:52.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:52.755+0000 7f8af6a00700 1 --2- 172.21.15.107:0/2341350502 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8af81038f0 0x7f8af80707d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8ae400c890 tx=0x7f8ae40044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:49:52.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:52.755+0000 7f8aef400700 1 -- 172.21.15.107:0/2341350502 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8ae400a700 con 0x7f8af81038f0 2024-03-04T13:49:52.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:52.755+0000 7f8aef400700 1 -- 172.21.15.107:0/2341350502 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8ae400ad30 con 0x7f8af81038f0 2024-03-04T13:49:52.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:52.755+0000 7f8afe000700 1 -- 172.21.15.107:0/2341350502 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8af8119c90 con 0x7f8af81038f0 2024-03-04T13:49:52.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:52.755+0000 7f8aef400700 1 -- 172.21.15.107:0/2341350502 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8ae40113a0 con 0x7f8af81038f0 2024-03-04T13:49:52.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:52.755+0000 7f8afe000700 1 -- 172.21.15.107:0/2341350502 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8af811a170 con 0x7f8af81038f0 2024-03-04T13:49:52.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:52.757+0000 7f8afe000700 1 -- 172.21.15.107:0/2341350502 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8af80fc6e0 con 0x7f8af81038f0 2024-03-04T13:49:52.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:52.758+0000 7f8aef400700 1 -- 172.21.15.107:0/2341350502 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f8ae4012400 con 0x7f8af81038f0 2024-03-04T13:49:52.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:52.759+0000 7f8aef400700 1 --2- 172.21.15.107:0/2341350502 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f8adc06d7f0 0x7f8adc06fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:49:52.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:52.759+0000 7f8aef400700 1 -- 172.21.15.107:0/2341350502 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f8ae408ceb0 con 0x7f8af81038f0 2024-03-04T13:49:52.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:52.760+0000 7f8af7400700 1 --2- 172.21.15.107:0/2341350502 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f8adc06d7f0 0x7f8adc06fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:49:52.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:52.761+0000 7f8af7400700 1 --2- 172.21.15.107:0/2341350502 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f8adc06d7f0 0x7f8adc06fca0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8ae80043d0 tx=0x7f8ae8009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:49:52.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:52.764+0000 7f8aef400700 1 -- 172.21.15.107:0/2341350502 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8ae40567b0 con 0x7f8af81038f0 2024-03-04T13:49:53.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.064+0000 7f8afe000700 1 -- 172.21.15.107:0/2341350502 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f8af810a050 con 0x7f8af81038f0 2024-03-04T13:49:53.066 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:52 smithi107 ceph-mon[112578]: from='client.34246 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:49:53.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.065+0000 7f8aef400700 1 -- 172.21.15.107:0/2341350502 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+839 (secure 0 0 0) 0x7f8ae4059e70 con 0x7f8af81038f0 2024-03-04T13:49:53.067 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-03-04T13:49:53.067 INFO:teuthology.orchestra.run.smithi107.stdout: "mon": { 2024-03-04T13:49:53.067 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-1845-g717ce3cc (717ce3cce8d0166e4e577faa8004e6f9cb4128c0) squid (dev)": 2 2024-03-04T13:49:53.067 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:49:53.067 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr": { 2024-03-04T13:49:53.067 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-1845-g717ce3cc (717ce3cce8d0166e4e577faa8004e6f9cb4128c0) squid (dev)": 2 2024-03-04T13:49:53.067 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:49:53.067 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": { 2024-03-04T13:49:53.067 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-545-g42608bb1 (42608bb191117be10dba720e6bf8a02f3171cd34) quincy (stable)": 4, 2024-03-04T13:49:53.068 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-1845-g717ce3cc (717ce3cce8d0166e4e577faa8004e6f9cb4128c0) squid (dev)": 1 2024-03-04T13:49:53.068 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:49:53.068 INFO:teuthology.orchestra.run.smithi107.stdout: "mds": { 2024-03-04T13:49:53.068 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-545-g42608bb1 (42608bb191117be10dba720e6bf8a02f3171cd34) quincy (stable)": 4 2024-03-04T13:49:53.068 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:49:53.068 INFO:teuthology.orchestra.run.smithi107.stdout: "overall": { 2024-03-04T13:49:53.068 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-545-g42608bb1 (42608bb191117be10dba720e6bf8a02f3171cd34) quincy (stable)": 8, 2024-03-04T13:49:53.068 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-1845-g717ce3cc (717ce3cce8d0166e4e577faa8004e6f9cb4128c0) squid (dev)": 5 2024-03-04T13:49:53.068 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-04T13:49:53.068 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-03-04T13:49:53.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.069+0000 7f8afe000700 1 -- 172.21.15.107:0/2341350502 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f8adc06d7f0 msgr2=0x7f8adc06fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:49:53.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.069+0000 7f8afe000700 1 --2- 172.21.15.107:0/2341350502 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f8adc06d7f0 0x7f8adc06fca0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8ae80043d0 tx=0x7f8ae8009e30 comp rx=0 tx=0).stop 2024-03-04T13:49:53.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.069+0000 7f8afe000700 1 -- 172.21.15.107:0/2341350502 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8af81038f0 msgr2=0x7f8af80707d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:49:53.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.069+0000 7f8afe000700 1 --2- 172.21.15.107:0/2341350502 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8af81038f0 0x7f8af80707d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8ae400c890 tx=0x7f8ae40044c0 comp rx=0 tx=0).stop 2024-03-04T13:49:53.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.069+0000 7f8afe000700 1 -- 172.21.15.107:0/2341350502 shutdown_connections 2024-03-04T13:49:53.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.069+0000 7f8afe000700 1 --2- 172.21.15.107:0/2341350502 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f8adc06d7f0 0x7f8adc06fca0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:49:53.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.069+0000 7f8afe000700 1 --2- 172.21.15.107:0/2341350502 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8af8102f50 0x7f8af8070290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:49:53.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.069+0000 7f8afe000700 1 --2- 172.21.15.107:0/2341350502 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8af81038f0 0x7f8af80707d0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:49:53.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.069+0000 7f8afe000700 1 -- 172.21.15.107:0/2341350502 >> 172.21.15.107:0/2341350502 conn(0x7f8af80f8ae0 msgr2=0x7f8af80fa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:49:53.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.069+0000 7f8afe000700 1 -- 172.21.15.107:0/2341350502 shutdown_connections 2024-03-04T13:49:53.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.069+0000 7f8afe000700 1 -- 172.21.15.107:0/2341350502 wait complete. 2024-03-04T13:49:53.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:52 smithi169 ceph-mon[86183]: from='client.34246 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:49:53.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.214+0000 7f7c9d200700 1 -- 172.21.15.107:0/3687444833 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c98101e40 msgr2=0x7f7c98102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:49:53.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.214+0000 7f7c9d200700 1 --2- 172.21.15.107:0/3687444833 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c98101e40 0x7f7c98102210 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7c88004940 tx=0x7f7c88009e30 comp rx=0 tx=0).stop 2024-03-04T13:49:53.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.214+0000 7f7c9d200700 1 -- 172.21.15.107:0/3687444833 shutdown_connections 2024-03-04T13:49:53.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.214+0000 7f7c9d200700 1 --2- 172.21.15.107:0/3687444833 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c981027e0 0x7f7c980fbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:49:53.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.214+0000 7f7c9d200700 1 --2- 172.21.15.107:0/3687444833 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c98101e40 0x7f7c98102210 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:49:53.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.214+0000 7f7c9d200700 1 -- 172.21.15.107:0/3687444833 >> 172.21.15.107:0/3687444833 conn(0x7f7c98071d90 msgr2=0x7f7c98072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:49:53.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.214+0000 7f7c9d200700 1 -- 172.21.15.107:0/3687444833 shutdown_connections 2024-03-04T13:49:53.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.214+0000 7f7c9d200700 1 -- 172.21.15.107:0/3687444833 wait complete. 2024-03-04T13:49:53.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.215+0000 7f7c9d200700 1 Processor -- start 2024-03-04T13:49:53.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.215+0000 7f7c9d200700 1 -- start start 2024-03-04T13:49:53.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.215+0000 7f7c9d200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c98101e40 0x7f7c9810c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:49:53.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.215+0000 7f7c9d200700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c981027e0 0x7f7c9810cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:49:53.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.215+0000 7f7c9d200700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c9810d400 con 0x7f7c98101e40 2024-03-04T13:49:53.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.215+0000 7f7c9d200700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c9810d570 con 0x7f7c981027e0 2024-03-04T13:49:53.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.216+0000 7f7c95e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c98101e40 0x7f7c9810c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:49:53.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.216+0000 7f7c95e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c98101e40 0x7f7c9810c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38630/0 (socket says 172.21.15.107:38630) 2024-03-04T13:49:53.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.216+0000 7f7c95e00700 1 -- 172.21.15.107:0/1822267718 learned_addr learned my addr 172.21.15.107:0/1822267718 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:49:53.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.216+0000 7f7c95e00700 1 -- 172.21.15.107:0/1822267718 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c981027e0 msgr2=0x7f7c9810cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:49:53.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.216+0000 7f7c95400700 1 --2- 172.21.15.107:0/1822267718 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c981027e0 0x7f7c9810cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:49:53.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.216+0000 7f7c95e00700 1 --2- 172.21.15.107:0/1822267718 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c981027e0 0x7f7c9810cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:49:53.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.216+0000 7f7c95e00700 1 -- 172.21.15.107:0/1822267718 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c88005040 con 0x7f7c98101e40 2024-03-04T13:49:53.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.217+0000 7f7c95e00700 1 --2- 172.21.15.107:0/1822267718 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c98101e40 0x7f7c9810c8a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7c88009150 tx=0x7f7c88009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:49:53.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.217+0000 7f7c8e000700 1 -- 172.21.15.107:0/1822267718 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c8800e040 con 0x7f7c98101e40 2024-03-04T13:49:53.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.217+0000 7f7c8e000700 1 -- 172.21.15.107:0/1822267718 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7c8800bbf0 con 0x7f7c98101e40 2024-03-04T13:49:53.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.217+0000 7f7c9d200700 1 -- 172.21.15.107:0/1822267718 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c98111870 con 0x7f7c98101e40 2024-03-04T13:49:53.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.217+0000 7f7c8e000700 1 -- 172.21.15.107:0/1822267718 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c8801c630 con 0x7f7c98101e40 2024-03-04T13:49:53.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.217+0000 7f7c9d200700 1 -- 172.21.15.107:0/1822267718 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c98111db0 con 0x7f7c98101e40 2024-03-04T13:49:53.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.218+0000 7f7c9d200700 1 -- 172.21.15.107:0/1822267718 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c980738b0 con 0x7f7c98101e40 2024-03-04T13:49:53.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.219+0000 7f7c8e000700 1 -- 172.21.15.107:0/1822267718 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f7c8800bd60 con 0x7f7c98101e40 2024-03-04T13:49:53.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.220+0000 7f7c8e000700 1 --2- 172.21.15.107:0/1822267718 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f7c7c06d7f0 0x7f7c7c06fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:49:53.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.220+0000 7f7c8e000700 1 -- 172.21.15.107:0/1822267718 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f7c8808e240 con 0x7f7c98101e40 2024-03-04T13:49:53.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.220+0000 7f7c95400700 1 --2- 172.21.15.107:0/1822267718 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f7c7c06d7f0 0x7f7c7c06fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:49:53.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.221+0000 7f7c95400700 1 --2- 172.21.15.107:0/1822267718 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f7c7c06d7f0 0x7f7c7c06fca0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7c84006fd0 tx=0x7f7c84008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:49:53.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.225+0000 7f7c8e000700 1 -- 172.21.15.107:0/1822267718 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7c88057b40 con 0x7f7c98101e40 2024-03-04T13:49:53.324 INFO:journalctl@ceph.osd.1.smithi107.stdout:Mar 04 13:49:53 smithi107 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-1-activate[129781]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-04T13:49:53.324 INFO:journalctl@ceph.osd.1.smithi107.stdout:Mar 04 13:49:53 smithi107 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-1-activate[129781]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-03-04T13:49:53.324 INFO:journalctl@ceph.osd.1.smithi107.stdout:Mar 04 13:49:53 smithi107 bash[129758]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-04T13:49:53.324 INFO:journalctl@ceph.osd.1.smithi107.stdout:Mar 04 13:49:53 smithi107 bash[129758]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-03-04T13:49:53.324 INFO:journalctl@ceph.osd.1.smithi107.stdout:Mar 04 13:49:53 smithi107 bash[129758]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-03-04T13:49:53.324 INFO:journalctl@ceph.osd.1.smithi107.stdout:Mar 04 13:49:53 smithi107 bash[129758]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-03-04T13:49:53.324 INFO:journalctl@ceph.osd.1.smithi107.stdout:Mar 04 13:49:53 smithi107 bash[129758]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-03-04T13:49:53.324 INFO:journalctl@ceph.osd.1.smithi107.stdout:Mar 04 13:49:53 smithi107 bash[129758]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-04T13:49:53.324 INFO:journalctl@ceph.osd.1.smithi107.stdout:Mar 04 13:49:53 smithi107 bash[129758]: --> ceph-volume raw activate successful for osd ID: 1 2024-03-04T13:49:53.324 INFO:journalctl@ceph.osd.1.smithi107.stdout:Mar 04 13:49:53 smithi107 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-1-activate[129781]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-03-04T13:49:53.324 INFO:journalctl@ceph.osd.1.smithi107.stdout:Mar 04 13:49:53 smithi107 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-1-activate[129781]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-03-04T13:49:53.324 INFO:journalctl@ceph.osd.1.smithi107.stdout:Mar 04 13:49:53 smithi107 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-1-activate[129781]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-03-04T13:49:53.324 INFO:journalctl@ceph.osd.1.smithi107.stdout:Mar 04 13:49:53 smithi107 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-1-activate[129781]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-04T13:49:53.325 INFO:journalctl@ceph.osd.1.smithi107.stdout:Mar 04 13:49:53 smithi107 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-1-activate[129781]: --> ceph-volume raw activate successful for osd ID: 1 2024-03-04T13:49:53.325 INFO:journalctl@ceph.osd.1.smithi107.stdout:Mar 04 13:49:53 smithi107 podman[129758]: 2024-03-04 13:49:53.286303985 +0000 UTC m=+1.222981580 container died f7a36f2809e6c9a540573da6b07de93f0d0db6f8d565c0ed9b7060a67edb3c0f (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-1-activate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-04T13:49:53.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.496+0000 7f7c9d200700 1 -- 172.21.15.107:0/1822267718 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f7c98065fe0 con 0x7f7c98101e40 2024-03-04T13:49:53.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.497+0000 7f7c8e000700 1 -- 172.21.15.107:0/1822267718 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 16 v16) v1 ==== 76+0+1880 (secure 0 0 0) 0x7f7c8805b200 con 0x7f7c98101e40 2024-03-04T13:49:53.499 INFO:teuthology.orchestra.run.smithi107.stdout:e16 2024-03-04T13:49:53.499 INFO:teuthology.orchestra.run.smithi107.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-04T13:49:53.499 INFO:teuthology.orchestra.run.smithi107.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-04T13:49:53.499 INFO:teuthology.orchestra.run.smithi107.stdout:legacy client fscid: 1 2024-03-04T13:49:53.500 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:49:53.500 INFO:teuthology.orchestra.run.smithi107.stdout:Filesystem 'cephfs' (1) 2024-03-04T13:49:53.500 INFO:teuthology.orchestra.run.smithi107.stdout:fs_name cephfs 2024-03-04T13:49:53.500 INFO:teuthology.orchestra.run.smithi107.stdout:epoch 15 2024-03-04T13:49:53.500 INFO:teuthology.orchestra.run.smithi107.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-04T13:49:53.500 INFO:teuthology.orchestra.run.smithi107.stdout:created 2024-03-04T13:38:30.854004+0000 2024-03-04T13:49:53.500 INFO:teuthology.orchestra.run.smithi107.stdout:modified 2024-03-04T13:38:50.015989+0000 2024-03-04T13:49:53.500 INFO:teuthology.orchestra.run.smithi107.stdout:tableserver 0 2024-03-04T13:49:53.500 INFO:teuthology.orchestra.run.smithi107.stdout:root 0 2024-03-04T13:49:53.500 INFO:teuthology.orchestra.run.smithi107.stdout:session_timeout 60 2024-03-04T13:49:53.500 INFO:teuthology.orchestra.run.smithi107.stdout:session_autoclose 300 2024-03-04T13:49:53.500 INFO:teuthology.orchestra.run.smithi107.stdout:max_file_size 1099511627776 2024-03-04T13:49:53.500 INFO:teuthology.orchestra.run.smithi107.stdout:max_xattr_size 65536 2024-03-04T13:49:53.501 INFO:teuthology.orchestra.run.smithi107.stdout:required_client_features {} 2024-03-04T13:49:53.501 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure 0 2024-03-04T13:49:53.501 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure_osd_epoch 47 2024-03-04T13:49:53.501 INFO:teuthology.orchestra.run.smithi107.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-04T13:49:53.501 INFO:teuthology.orchestra.run.smithi107.stdout:max_mds 1 2024-03-04T13:49:53.501 INFO:teuthology.orchestra.run.smithi107.stdout:in 0 2024-03-04T13:49:53.501 INFO:teuthology.orchestra.run.smithi107.stdout:up {0=24247} 2024-03-04T13:49:53.501 INFO:teuthology.orchestra.run.smithi107.stdout:failed 2024-03-04T13:49:53.501 INFO:teuthology.orchestra.run.smithi107.stdout:damaged 2024-03-04T13:49:53.501 INFO:teuthology.orchestra.run.smithi107.stdout:stopped 2024-03-04T13:49:53.501 INFO:teuthology.orchestra.run.smithi107.stdout:data_pools [3] 2024-03-04T13:49:53.501 INFO:teuthology.orchestra.run.smithi107.stdout:metadata_pool 2 2024-03-04T13:49:53.501 INFO:teuthology.orchestra.run.smithi107.stdout:inline_data enabled 2024-03-04T13:49:53.501 INFO:teuthology.orchestra.run.smithi107.stdout:balancer 2024-03-04T13:49:53.502 INFO:teuthology.orchestra.run.smithi107.stdout:bal_rank_mask -1 2024-03-04T13:49:53.502 INFO:teuthology.orchestra.run.smithi107.stdout:standby_count_wanted 1 2024-03-04T13:49:53.502 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi169.jusudl{0:24247} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.169:6824/2994866076,v1:172.21.15.169:6825/2994866076] compat {c=[1],r=[1],i=[7ff]}] 2024-03-04T13:49:53.502 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.thycho{0:14470} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.107:6828/1985577060,v1:172.21.15.107:6829/1985577060] compat {c=[1],r=[1],i=[7ff]}] 2024-03-04T13:49:53.502 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:49:53.502 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:49:53.502 INFO:teuthology.orchestra.run.smithi107.stdout:Standby daemons: 2024-03-04T13:49:53.502 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:49:53.502 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.riexmm{-1:14480} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.107:6826/3261622517,v1:172.21.15.107:6827/3261622517] compat {c=[1],r=[1],i=[7ff]}] 2024-03-04T13:49:53.502 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi169.bxitzk{-1:24259} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.169:6826/3696090493,v1:172.21.15.169:6827/3696090493] compat {c=[1],r=[1],i=[7ff]}] 2024-03-04T13:49:53.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.500+0000 7f7c9d200700 1 -- 172.21.15.107:0/1822267718 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f7c7c06d7f0 msgr2=0x7f7c7c06fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:49:53.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.500+0000 7f7c9d200700 1 --2- 172.21.15.107:0/1822267718 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f7c7c06d7f0 0x7f7c7c06fca0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7c84006fd0 tx=0x7f7c84008040 comp rx=0 tx=0).stop 2024-03-04T13:49:53.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.500+0000 7f7c9d200700 1 -- 172.21.15.107:0/1822267718 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c98101e40 msgr2=0x7f7c9810c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:49:53.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.500+0000 7f7c9d200700 1 --2- 172.21.15.107:0/1822267718 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c98101e40 0x7f7c9810c8a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7c88009150 tx=0x7f7c88009e30 comp rx=0 tx=0).stop 2024-03-04T13:49:53.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.500+0000 7f7c9d200700 1 -- 172.21.15.107:0/1822267718 shutdown_connections 2024-03-04T13:49:53.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.500+0000 7f7c9d200700 1 --2- 172.21.15.107:0/1822267718 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f7c7c06d7f0 0x7f7c7c06fca0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:49:53.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.500+0000 7f7c9d200700 1 --2- 172.21.15.107:0/1822267718 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c98101e40 0x7f7c9810c8a0 secure :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7c88009150 tx=0x7f7c88009e30 comp rx=0 tx=0).stop 2024-03-04T13:49:53.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.500+0000 7f7c9d200700 1 --2- 172.21.15.107:0/1822267718 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7c981027e0 0x7f7c9810cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:49:53.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.500+0000 7f7c9d200700 1 -- 172.21.15.107:0/1822267718 >> 172.21.15.107:0/1822267718 conn(0x7f7c98071d90 msgr2=0x7f7c980fad10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:49:53.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.500+0000 7f7c9d200700 1 -- 172.21.15.107:0/1822267718 shutdown_connections 2024-03-04T13:49:53.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.500+0000 7f7c9d200700 1 -- 172.21.15.107:0/1822267718 wait complete. 2024-03-04T13:49:53.504 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 16 2024-03-04T13:49:53.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.661+0000 7fa27ac00700 1 -- 172.21.15.107:0/2519796744 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa2740fdfe0 msgr2=0x7fa2740fe3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:49:53.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.661+0000 7fa27ac00700 1 --2- 172.21.15.107:0/2519796744 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa2740fdfe0 0x7fa2740fe3b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa264004970 tx=0x7fa264009e30 comp rx=0 tx=0).stop 2024-03-04T13:49:53.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.661+0000 7fa27ac00700 1 -- 172.21.15.107:0/2519796744 shutdown_connections 2024-03-04T13:49:53.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.661+0000 7fa27ac00700 1 --2- 172.21.15.107:0/2519796744 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2740fe8f0 0x7fa274107050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:49:53.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.661+0000 7fa27ac00700 1 --2- 172.21.15.107:0/2519796744 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa2740fdfe0 0x7fa2740fe3b0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:49:53.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.661+0000 7fa27ac00700 1 -- 172.21.15.107:0/2519796744 >> 172.21.15.107:0/2519796744 conn(0x7fa274071d90 msgr2=0x7fa274072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:49:53.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.661+0000 7fa27ac00700 1 -- 172.21.15.107:0/2519796744 shutdown_connections 2024-03-04T13:49:53.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.661+0000 7fa27ac00700 1 -- 172.21.15.107:0/2519796744 wait complete. 2024-03-04T13:49:53.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.662+0000 7fa27ac00700 1 Processor -- start 2024-03-04T13:49:53.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.662+0000 7fa27ac00700 1 -- start start 2024-03-04T13:49:53.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.663+0000 7fa27ac00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa2740fdfe0 0x7fa274113a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:49:53.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.663+0000 7fa27ac00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2740fe8f0 0x7fa274113f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:49:53.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.663+0000 7fa27ac00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2741145e0 con 0x7fa2740fdfe0 2024-03-04T13:49:53.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.663+0000 7fa27ac00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa27410d910 con 0x7fa2740fe8f0 2024-03-04T13:49:53.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.663+0000 7fa273e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa2740fdfe0 0x7fa274113a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:49:53.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.663+0000 7fa273e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa2740fdfe0 0x7fa274113a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38636/0 (socket says 172.21.15.107:38636) 2024-03-04T13:49:53.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.663+0000 7fa273e00700 1 -- 172.21.15.107:0/2276466758 learned_addr learned my addr 172.21.15.107:0/2276466758 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:49:53.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.663+0000 7fa273400700 1 --2- 172.21.15.107:0/2276466758 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2740fe8f0 0x7fa274113f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:49:53.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.663+0000 7fa273e00700 1 -- 172.21.15.107:0/2276466758 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2740fe8f0 msgr2=0x7fa274113f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:49:53.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.663+0000 7fa273e00700 1 --2- 172.21.15.107:0/2276466758 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2740fe8f0 0x7fa274113f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:49:53.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.663+0000 7fa273e00700 1 -- 172.21.15.107:0/2276466758 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa264005040 con 0x7fa2740fdfe0 2024-03-04T13:49:53.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.663+0000 7fa273400700 1 --2- 172.21.15.107:0/2276466758 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2740fe8f0 0x7fa274113f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T13:49:53.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.664+0000 7fa273e00700 1 --2- 172.21.15.107:0/2276466758 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa2740fdfe0 0x7fa274113a50 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa264004f50 tx=0x7fa264009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:49:53.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.664+0000 7fa26be00700 1 -- 172.21.15.107:0/2276466758 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa26400e040 con 0x7fa2740fdfe0 2024-03-04T13:49:53.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.664+0000 7fa26be00700 1 -- 172.21.15.107:0/2276466758 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa264004290 con 0x7fa2740fdfe0 2024-03-04T13:49:53.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.664+0000 7fa26be00700 1 -- 172.21.15.107:0/2276466758 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa26401c630 con 0x7fa2740fdfe0 2024-03-04T13:49:53.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.664+0000 7fa27ac00700 1 -- 172.21.15.107:0/2276466758 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa27410db90 con 0x7fa2740fdfe0 2024-03-04T13:49:53.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.664+0000 7fa27ac00700 1 -- 172.21.15.107:0/2276466758 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa27410e0d0 con 0x7fa2740fdfe0 2024-03-04T13:49:53.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.666+0000 7fa27ac00700 1 -- 172.21.15.107:0/2276466758 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2741046c0 con 0x7fa2740fdfe0 2024-03-04T13:49:53.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.667+0000 7fa26be00700 1 -- 172.21.15.107:0/2276466758 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fa26400bc10 con 0x7fa2740fdfe0 2024-03-04T13:49:53.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.671+0000 7fa26be00700 1 --2- 172.21.15.107:0/2276466758 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fa25806d7f0 0x7fa25806fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:49:53.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.671+0000 7fa26be00700 1 -- 172.21.15.107:0/2276466758 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fa26408ec90 con 0x7fa2740fdfe0 2024-03-04T13:49:53.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.671+0000 7fa273400700 1 --2- 172.21.15.107:0/2276466758 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fa25806d7f0 0x7fa25806fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:49:53.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.671+0000 7fa273400700 1 --2- 172.21.15.107:0/2276466758 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fa25806d7f0 0x7fa25806fca0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa260007fd0 tx=0x7fa260009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:49:53.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.672+0000 7fa26be00700 1 -- 172.21.15.107:0/2276466758 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa26405c180 con 0x7fa2740fdfe0 2024-03-04T13:49:53.913 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.912+0000 7fa27ac00700 1 -- 172.21.15.107:0/2276466758 --> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa27404bed0 con 0x7fa25806d7f0 2024-03-04T13:49:53.913 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:53 smithi107 ceph-mon[112578]: from='client.34250 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:49:53.913 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:53 smithi107 ceph-mon[112578]: from='client.34254 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:49:53.913 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:53 smithi107 ceph-mon[112578]: pgmap v84: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 255 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 35/264 objects degraded (13.258%) 2024-03-04T13:49:53.913 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:53 smithi107 ceph-mon[112578]: from='client.? 172.21.15.107:0/2341350502' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:49:53.914 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:53 smithi107 ceph-mon[112578]: from='client.? 172.21.15.107:0/1822267718' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-04T13:49:53.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.913+0000 7fa26be00700 1 -- 172.21.15.107:0/2276466758 <== mgr.34102 v2:172.21.15.107:6800/1223039812 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fa27404bed0 con 0x7fa25806d7f0 2024-03-04T13:49:53.915 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-03-04T13:49:53.915 INFO:teuthology.orchestra.run.smithi107.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377", 2024-03-04T13:49:53.915 INFO:teuthology.orchestra.run.smithi107.stdout: "in_progress": true, 2024-03-04T13:49:53.915 INFO:teuthology.orchestra.run.smithi107.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-04T13:49:53.915 INFO:teuthology.orchestra.run.smithi107.stdout: "services_complete": [ 2024-03-04T13:49:53.915 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr", 2024-03-04T13:49:53.915 INFO:teuthology.orchestra.run.smithi107.stdout: "mon", 2024-03-04T13:49:53.915 INFO:teuthology.orchestra.run.smithi107.stdout: "crash" 2024-03-04T13:49:53.915 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-04T13:49:53.915 INFO:teuthology.orchestra.run.smithi107.stdout: "progress": "7/21 daemons upgraded", 2024-03-04T13:49:53.915 INFO:teuthology.orchestra.run.smithi107.stdout: "message": "Currently upgrading osd daemons", 2024-03-04T13:49:53.915 INFO:teuthology.orchestra.run.smithi107.stdout: "is_paused": false 2024-03-04T13:49:53.915 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-03-04T13:49:53.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.917+0000 7fa27ac00700 1 -- 172.21.15.107:0/2276466758 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fa25806d7f0 msgr2=0x7fa25806fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:49:53.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.917+0000 7fa27ac00700 1 --2- 172.21.15.107:0/2276466758 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fa25806d7f0 0x7fa25806fca0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa260007fd0 tx=0x7fa260009040 comp rx=0 tx=0).stop 2024-03-04T13:49:53.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.917+0000 7fa27ac00700 1 -- 172.21.15.107:0/2276466758 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa2740fdfe0 msgr2=0x7fa274113a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:49:53.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.917+0000 7fa27ac00700 1 --2- 172.21.15.107:0/2276466758 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa2740fdfe0 0x7fa274113a50 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa264004f50 tx=0x7fa264009e30 comp rx=0 tx=0).stop 2024-03-04T13:49:53.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.917+0000 7fa27ac00700 1 -- 172.21.15.107:0/2276466758 shutdown_connections 2024-03-04T13:49:53.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.917+0000 7fa27ac00700 1 --2- 172.21.15.107:0/2276466758 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fa25806d7f0 0x7fa25806fca0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:49:53.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.917+0000 7fa27ac00700 1 --2- 172.21.15.107:0/2276466758 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa2740fdfe0 0x7fa274113a50 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:49:53.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.917+0000 7fa27ac00700 1 --2- 172.21.15.107:0/2276466758 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2740fe8f0 0x7fa274113f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:49:53.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.917+0000 7fa27ac00700 1 -- 172.21.15.107:0/2276466758 >> 172.21.15.107:0/2276466758 conn(0x7fa274071d90 msgr2=0x7fa274101840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:49:53.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.917+0000 7fa27ac00700 1 -- 172.21.15.107:0/2276466758 shutdown_connections 2024-03-04T13:49:53.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:53.917+0000 7fa27ac00700 1 -- 172.21.15.107:0/2276466758 wait complete. 2024-03-04T13:49:54.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:54.065+0000 7f16bb400700 1 -- 172.21.15.107:0/1310331724 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f16b4070330 msgr2=0x7f16b4070700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:49:54.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:54.065+0000 7f16bb400700 1 --2- 172.21.15.107:0/1310331724 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f16b4070330 0x7f16b4070700 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f16a8004970 tx=0x7f16a8009e30 comp rx=0 tx=0).stop 2024-03-04T13:49:54.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:54.065+0000 7f16bb400700 1 -- 172.21.15.107:0/1310331724 shutdown_connections 2024-03-04T13:49:54.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:54.065+0000 7f16bb400700 1 --2- 172.21.15.107:0/1310331724 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16b4070c40 0x7f16b4109260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:49:54.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:54.065+0000 7f16bb400700 1 --2- 172.21.15.107:0/1310331724 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f16b4070330 0x7f16b4070700 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:49:54.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:54.065+0000 7f16bb400700 1 -- 172.21.15.107:0/1310331724 >> 172.21.15.107:0/1310331724 conn(0x7f16b40f8ae0 msgr2=0x7f16b40faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:49:54.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:54.065+0000 7f16bb400700 1 -- 172.21.15.107:0/1310331724 shutdown_connections 2024-03-04T13:49:54.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:54.065+0000 7f16bb400700 1 -- 172.21.15.107:0/1310331724 wait complete. 2024-03-04T13:49:54.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:54.066+0000 7f16bb400700 1 Processor -- start 2024-03-04T13:49:54.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:54.066+0000 7f16bb400700 1 -- start start 2024-03-04T13:49:54.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:54.067+0000 7f16bb400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f16b4070330 0x7f16b4115c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:49:54.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:54.067+0000 7f16bb400700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16b4070c40 0x7f16b4116160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:49:54.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:54.067+0000 7f16bb400700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16b41167b0 con 0x7f16b4070330 2024-03-04T13:49:54.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:54.067+0000 7f16bb400700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16b410fae0 con 0x7f16b4070c40 2024-03-04T13:49:54.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:54.067+0000 7f16b8a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f16b4070330 0x7f16b4115c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:49:54.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:54.067+0000 7f16b8a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f16b4070330 0x7f16b4115c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38644/0 (socket says 172.21.15.107:38644) 2024-03-04T13:49:54.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:54.067+0000 7f16b8a00700 1 -- 172.21.15.107:0/4260999930 learned_addr learned my addr 172.21.15.107:0/4260999930 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:49:54.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:54.067+0000 7f16b3e00700 1 --2- 172.21.15.107:0/4260999930 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16b4070c40 0x7f16b4116160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:49:54.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:54.068+0000 7f16b8a00700 1 -- 172.21.15.107:0/4260999930 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16b4070c40 msgr2=0x7f16b4116160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:49:54.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:54.068+0000 7f16b8a00700 1 --2- 172.21.15.107:0/4260999930 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16b4070c40 0x7f16b4116160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:49:54.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:54.068+0000 7f16b8a00700 1 -- 172.21.15.107:0/4260999930 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16a8005040 con 0x7f16b4070330 2024-03-04T13:49:54.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:54.068+0000 7f16b8a00700 1 --2- 172.21.15.107:0/4260999930 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f16b4070330 0x7f16b4115c20 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f16a8004f50 tx=0x7f16a8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:49:54.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:54.068+0000 7f16b0c00700 1 -- 172.21.15.107:0/4260999930 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f16a800e040 con 0x7f16b4070330 2024-03-04T13:49:54.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:54.068+0000 7f16bb400700 1 -- 172.21.15.107:0/4260999930 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16b410fd60 con 0x7f16b4070330 2024-03-04T13:49:54.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:54.068+0000 7f16b0c00700 1 -- 172.21.15.107:0/4260999930 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f16a8004290 con 0x7f16b4070330 2024-03-04T13:49:54.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:54.068+0000 7f16b0c00700 1 -- 172.21.15.107:0/4260999930 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f16a801c630 con 0x7f16b4070330 2024-03-04T13:49:54.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:54.068+0000 7f16bb400700 1 -- 172.21.15.107:0/4260999930 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16b41102a0 con 0x7f16b4070330 2024-03-04T13:49:54.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:54.070+0000 7f16bb400700 1 -- 172.21.15.107:0/4260999930 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16b4106920 con 0x7f16b4070330 2024-03-04T13:49:54.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:54.071+0000 7f16b0c00700 1 -- 172.21.15.107:0/4260999930 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f16a800bc10 con 0x7f16b4070330 2024-03-04T13:49:54.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:54.072+0000 7f16b0c00700 1 --2- 172.21.15.107:0/4260999930 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f169c06d7f0 0x7f169c06fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:49:54.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:54.072+0000 7f16b0c00700 1 -- 172.21.15.107:0/4260999930 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f16a808e1a0 con 0x7f16b4070330 2024-03-04T13:49:54.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:54.072+0000 7f16b3e00700 1 --2- 172.21.15.107:0/4260999930 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f169c06d7f0 0x7f169c06fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:49:54.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:54.073+0000 7f16b3e00700 1 --2- 172.21.15.107:0/4260999930 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f169c06d7f0 0x7f169c06fca0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f16b4111450 tx=0x7f16a0009ee0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:49:54.077 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:54.077+0000 7f16b0c00700 1 -- 172.21.15.107:0/4260999930 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f16a805b690 con 0x7f16b4070330 2024-03-04T13:49:54.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:53 smithi169 ceph-mon[86183]: from='client.34250 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:49:54.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:53 smithi169 ceph-mon[86183]: from='client.34254 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:49:54.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:53 smithi169 ceph-mon[86183]: pgmap v84: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 255 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 35/264 objects degraded (13.258%) 2024-03-04T13:49:54.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:53 smithi169 ceph-mon[86183]: from='client.? 172.21.15.107:0/2341350502' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:49:54.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:53 smithi169 ceph-mon[86183]: from='client.? 172.21.15.107:0/1822267718' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-04T13:49:54.331 INFO:journalctl@ceph.osd.1.smithi107.stdout:Mar 04 13:49:54 smithi107 podman[129965]: 2024-03-04 13:49:54.010913136 +0000 UTC m=+0.706970949 container remove f7a36f2809e6c9a540573da6b07de93f0d0db6f8d565c0ed9b7060a67edb3c0f (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-1-activate, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True) 2024-03-04T13:49:54.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:54.382+0000 7f16bb400700 1 -- 172.21.15.107:0/4260999930 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f16b404bed0 con 0x7f16b4070330 2024-03-04T13:49:54.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:54.382+0000 7f16b0c00700 1 -- 172.21.15.107:0/4260999930 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+1182 (secure 0 0 0) 0x7f16a8013e60 con 0x7f16b4070330 2024-03-04T13:49:54.384 INFO:teuthology.orchestra.run.smithi107.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data; 1 osds down; Degraded data redundancy: 35/264 objects degraded (13.258%), 13 pgs degraded 2024-03-04T13:49:54.384 INFO:teuthology.orchestra.run.smithi107.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-04T13:49:54.384 INFO:teuthology.orchestra.run.smithi107.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-04T13:49:54.384 INFO:teuthology.orchestra.run.smithi107.stdout:[WRN] OSD_DOWN: 1 osds down 2024-03-04T13:49:54.384 INFO:teuthology.orchestra.run.smithi107.stdout: osd.1 (root=default,host=smithi107) is down 2024-03-04T13:49:54.384 INFO:teuthology.orchestra.run.smithi107.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 35/264 objects degraded (13.258%), 13 pgs degraded 2024-03-04T13:49:54.384 INFO:teuthology.orchestra.run.smithi107.stdout: pg 1.0 is active+undersized+degraded, acting [3,0] 2024-03-04T13:49:54.384 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.0 is active+undersized+degraded, acting [3,0] 2024-03-04T13:49:54.384 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.1 is active+undersized+degraded, acting [2,0] 2024-03-04T13:49:54.385 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.2 is active+undersized+degraded, acting [5,0] 2024-03-04T13:49:54.385 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.3 is active+undersized+degraded, acting [5,2] 2024-03-04T13:49:54.385 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.4 is active+undersized+degraded, acting [0,4] 2024-03-04T13:49:54.385 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.6 is active+undersized+degraded, acting [3,4] 2024-03-04T13:49:54.385 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.9 is active+undersized+degraded, acting [4,0] 2024-03-04T13:49:54.385 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.a is active+undersized+degraded, acting [4,3] 2024-03-04T13:49:54.385 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.d is active+undersized+degraded, acting [3,2] 2024-03-04T13:49:54.385 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.10 is active+undersized+degraded, acting [2,0] 2024-03-04T13:49:54.385 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.15 is active+undersized+degraded, acting [3,0] 2024-03-04T13:49:54.385 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.1b is active+undersized+degraded, acting [0,5] 2024-03-04T13:49:54.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:54.385+0000 7f16bb400700 1 -- 172.21.15.107:0/4260999930 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f169c06d7f0 msgr2=0x7f169c06fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:49:54.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:54.385+0000 7f16bb400700 1 --2- 172.21.15.107:0/4260999930 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f169c06d7f0 0x7f169c06fca0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f16b4111450 tx=0x7f16a0009ee0 comp rx=0 tx=0).stop 2024-03-04T13:49:54.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:54.385+0000 7f16bb400700 1 -- 172.21.15.107:0/4260999930 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f16b4070330 msgr2=0x7f16b4115c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:49:54.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:54.385+0000 7f16bb400700 1 --2- 172.21.15.107:0/4260999930 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f16b4070330 0x7f16b4115c20 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f16a8004f50 tx=0x7f16a8009e30 comp rx=0 tx=0).stop 2024-03-04T13:49:54.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:54.386+0000 7f16bb400700 1 -- 172.21.15.107:0/4260999930 shutdown_connections 2024-03-04T13:49:54.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:54.386+0000 7f16bb400700 1 --2- 172.21.15.107:0/4260999930 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f169c06d7f0 0x7f169c06fca0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:49:54.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:54.386+0000 7f16bb400700 1 --2- 172.21.15.107:0/4260999930 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f16b4070330 0x7f16b4115c20 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:49:54.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:54.386+0000 7f16bb400700 1 --2- 172.21.15.107:0/4260999930 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16b4070c40 0x7f16b4116160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:49:54.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:54.386+0000 7f16bb400700 1 -- 172.21.15.107:0/4260999930 >> 172.21.15.107:0/4260999930 conn(0x7f16b40f8ae0 msgr2=0x7f16b4103aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:49:54.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:54.386+0000 7f16bb400700 1 -- 172.21.15.107:0/4260999930 shutdown_connections 2024-03-04T13:49:54.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:49:54.386+0000 7f16bb400700 1 -- 172.21.15.107:0/4260999930 wait complete. 2024-03-04T13:49:54.594 INFO:journalctl@ceph.osd.1.smithi107.stdout:Mar 04 13:49:54 smithi107 podman[130072]: 2024-03-04 13:49:54.237665972 +0000 UTC m=+0.033107169 image pull d71b839abdf892550525dad59fdf20a6b925b73dbcc7526f0d04da8380b9fabf quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377 2024-03-04T13:49:54.594 INFO:journalctl@ceph.osd.1.smithi107.stdout:Mar 04 13:49:54 smithi107 podman[130072]: 2024-03-04 13:49:54.397483721 +0000 UTC m=+0.192924926 container create b614af6588548b0b59096d1f9d8cc60aeb4e06c756b823e4092c9a2507fa20d6 (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-1, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8) 2024-03-04T13:49:55.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:54 smithi107 ceph-mon[112578]: from='client.? 172.21.15.107:0/4260999930' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-04T13:49:55.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:54 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:49:55.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:54 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:49:55.094 INFO:journalctl@ceph.osd.1.smithi107.stdout:Mar 04 13:49:54 smithi107 podman[130072]: 2024-03-04 13:49:54.699968423 +0000 UTC m=+0.495409656 container init b614af6588548b0b59096d1f9d8cc60aeb4e06c756b823e4092c9a2507fa20d6 (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-1, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240222, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-03-04T13:49:55.094 INFO:journalctl@ceph.osd.1.smithi107.stdout:Mar 04 13:49:54 smithi107 podman[130072]: 2024-03-04 13:49:54.711701586 +0000 UTC m=+0.507142781 container start b614af6588548b0b59096d1f9d8cc60aeb4e06c756b823e4092c9a2507fa20d6 (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-1, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8) 2024-03-04T13:49:55.094 INFO:journalctl@ceph.osd.1.smithi107.stdout:Mar 04 13:49:54 smithi107 ceph-osd[130093]: -- 172.21.15.107:0/3919322712 <== mon.0 v2:172.21.15.107:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a3f50f1680 con 0x55a3f50bac00 2024-03-04T13:49:55.094 INFO:journalctl@ceph.osd.1.smithi107.stdout:Mar 04 13:49:54 smithi107 bash[130072]: b614af6588548b0b59096d1f9d8cc60aeb4e06c756b823e4092c9a2507fa20d6 2024-03-04T13:49:55.094 INFO:journalctl@ceph.osd.1.smithi107.stdout:Mar 04 13:49:54 smithi107 systemd[1]: Started Ceph osd.1 for 1fc09a58-da2b-11ee-95c6-87774f69a715. 2024-03-04T13:49:55.360 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:54 smithi169 ceph-mon[86183]: from='client.? 172.21.15.107:0/4260999930' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-04T13:49:55.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:54 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:49:55.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:54 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:49:55.593 INFO:journalctl@ceph.osd.1.smithi107.stdout:Mar 04 13:49:55 smithi107 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-1[130089]: 2024-03-04T13:49:55.295+0000 7f3652efe7c0 -1 Falling back to public interface 2024-03-04T13:49:56.202 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:55 smithi107 ceph-mon[112578]: from='client.34266 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:49:56.203 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:55 smithi107 ceph-mon[112578]: pgmap v85: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 255 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 35/264 objects degraded (13.258%) 2024-03-04T13:49:56.203 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:55 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:49:56.203 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:55 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:49:56.203 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:55 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:49:56.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:55 smithi169 ceph-mon[86183]: from='client.34266 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:49:56.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:55 smithi169 ceph-mon[86183]: pgmap v85: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 255 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 35/264 objects degraded (13.258%) 2024-03-04T13:49:56.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:55 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:49:56.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:55 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:49:56.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:55 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:49:58.161 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:57 smithi107 ceph-mon[112578]: pgmap v86: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 255 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 35/264 objects degraded (13.258%) 2024-03-04T13:49:58.360 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:57 smithi169 ceph-mon[86183]: pgmap v86: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 255 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 35/264 objects degraded (13.258%) 2024-03-04T13:50:00.175 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:59 smithi107 ceph-mon[112578]: pgmap v87: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 255 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 35/264 objects degraded (13.258%) 2024-03-04T13:50:00.175 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:49:59 smithi107 ceph-mon[112578]: Health check update: Degraded data redundancy: 35/264 objects degraded (13.258%), 13 pgs degraded (PG_DEGRADED) 2024-03-04T13:50:00.360 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:59 smithi169 ceph-mon[86183]: pgmap v87: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 255 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 35/264 objects degraded (13.258%) 2024-03-04T13:50:00.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:49:59 smithi169 ceph-mon[86183]: Health check update: Degraded data redundancy: 35/264 objects degraded (13.258%), 13 pgs degraded (PG_DEGRADED) 2024-03-04T13:50:00.593 INFO:journalctl@ceph.osd.1.smithi107.stdout:Mar 04 13:50:00 smithi107 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-1[130089]: 2024-03-04T13:50:00.229+0000 7f3652efe7c0 -1 osd.1 0 read_superblock omap replica is missing. 2024-03-04T13:50:01.077 INFO:journalctl@ceph.osd.1.smithi107.stdout:Mar 04 13:50:00 smithi107 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-1[130089]: 2024-03-04T13:50:00.680+0000 7f3652efe7c0 -1 osd.1 55 log_to_monitors true 2024-03-04T13:50:01.077 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:00 smithi107 ceph-mon[112578]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data; 1 osds down; Degraded data redundancy: 35/264 objects degraded (13.258%), 13 pgs degraded 2024-03-04T13:50:01.077 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:00 smithi107 ceph-mon[112578]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-04T13:50:01.077 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:00 smithi107 ceph-mon[112578]: fs cephfs has deprecated feature inline_data enabled. 2024-03-04T13:50:01.078 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:00 smithi107 ceph-mon[112578]: [WRN] OSD_DOWN: 1 osds down 2024-03-04T13:50:01.078 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:00 smithi107 ceph-mon[112578]: osd.1 (root=default,host=smithi107) is down 2024-03-04T13:50:01.078 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:00 smithi107 ceph-mon[112578]: [WRN] PG_DEGRADED: Degraded data redundancy: 35/264 objects degraded (13.258%), 13 pgs degraded 2024-03-04T13:50:01.078 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:00 smithi107 ceph-mon[112578]: pg 1.0 is active+undersized+degraded, acting [3,0] 2024-03-04T13:50:01.078 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:00 smithi107 ceph-mon[112578]: pg 2.0 is active+undersized+degraded, acting [3,0] 2024-03-04T13:50:01.078 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:00 smithi107 ceph-mon[112578]: pg 2.1 is active+undersized+degraded, acting [2,0] 2024-03-04T13:50:01.078 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:00 smithi107 ceph-mon[112578]: pg 2.2 is active+undersized+degraded, acting [5,0] 2024-03-04T13:50:01.078 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:00 smithi107 ceph-mon[112578]: pg 2.3 is active+undersized+degraded, acting [5,2] 2024-03-04T13:50:01.078 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:00 smithi107 ceph-mon[112578]: pg 2.4 is active+undersized+degraded, acting [0,4] 2024-03-04T13:50:01.078 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:00 smithi107 ceph-mon[112578]: pg 2.6 is active+undersized+degraded, acting [3,4] 2024-03-04T13:50:01.078 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:00 smithi107 ceph-mon[112578]: pg 2.9 is active+undersized+degraded, acting [4,0] 2024-03-04T13:50:01.078 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:00 smithi107 ceph-mon[112578]: pg 2.a is active+undersized+degraded, acting [4,3] 2024-03-04T13:50:01.078 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:00 smithi107 ceph-mon[112578]: pg 2.d is active+undersized+degraded, acting [3,2] 2024-03-04T13:50:01.079 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:00 smithi107 ceph-mon[112578]: pg 2.10 is active+undersized+degraded, acting [2,0] 2024-03-04T13:50:01.079 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:00 smithi107 ceph-mon[112578]: pg 2.15 is active+undersized+degraded, acting [3,0] 2024-03-04T13:50:01.079 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:00 smithi107 ceph-mon[112578]: pg 2.1b is active+undersized+degraded, acting [0,5] 2024-03-04T13:50:01.079 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:00 smithi107 ceph-mon[112578]: from='osd.1 [v2:172.21.15.107:6810/1164427580,v1:172.21.15.107:6811/1164427580]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-04T13:50:01.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:00 smithi169 ceph-mon[86183]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data; 1 osds down; Degraded data redundancy: 35/264 objects degraded (13.258%), 13 pgs degraded 2024-03-04T13:50:01.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:00 smithi169 ceph-mon[86183]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-04T13:50:01.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:00 smithi169 ceph-mon[86183]: fs cephfs has deprecated feature inline_data enabled. 2024-03-04T13:50:01.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:00 smithi169 ceph-mon[86183]: [WRN] OSD_DOWN: 1 osds down 2024-03-04T13:50:01.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:00 smithi169 ceph-mon[86183]: osd.1 (root=default,host=smithi107) is down 2024-03-04T13:50:01.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:00 smithi169 ceph-mon[86183]: [WRN] PG_DEGRADED: Degraded data redundancy: 35/264 objects degraded (13.258%), 13 pgs degraded 2024-03-04T13:50:01.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:00 smithi169 ceph-mon[86183]: pg 1.0 is active+undersized+degraded, acting [3,0] 2024-03-04T13:50:01.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:00 smithi169 ceph-mon[86183]: pg 2.0 is active+undersized+degraded, acting [3,0] 2024-03-04T13:50:01.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:00 smithi169 ceph-mon[86183]: pg 2.1 is active+undersized+degraded, acting [2,0] 2024-03-04T13:50:01.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:00 smithi169 ceph-mon[86183]: pg 2.2 is active+undersized+degraded, acting [5,0] 2024-03-04T13:50:01.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:00 smithi169 ceph-mon[86183]: pg 2.3 is active+undersized+degraded, acting [5,2] 2024-03-04T13:50:01.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:00 smithi169 ceph-mon[86183]: pg 2.4 is active+undersized+degraded, acting [0,4] 2024-03-04T13:50:01.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:00 smithi169 ceph-mon[86183]: pg 2.6 is active+undersized+degraded, acting [3,4] 2024-03-04T13:50:01.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:00 smithi169 ceph-mon[86183]: pg 2.9 is active+undersized+degraded, acting [4,0] 2024-03-04T13:50:01.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:00 smithi169 ceph-mon[86183]: pg 2.a is active+undersized+degraded, acting [4,3] 2024-03-04T13:50:01.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:00 smithi169 ceph-mon[86183]: pg 2.d is active+undersized+degraded, acting [3,2] 2024-03-04T13:50:01.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:00 smithi169 ceph-mon[86183]: pg 2.10 is active+undersized+degraded, acting [2,0] 2024-03-04T13:50:01.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:00 smithi169 ceph-mon[86183]: pg 2.15 is active+undersized+degraded, acting [3,0] 2024-03-04T13:50:01.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:00 smithi169 ceph-mon[86183]: pg 2.1b is active+undersized+degraded, acting [0,5] 2024-03-04T13:50:01.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:00 smithi169 ceph-mon[86183]: from='osd.1 [v2:172.21.15.107:6810/1164427580,v1:172.21.15.107:6811/1164427580]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-04T13:50:02.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:01 smithi107 ceph-mon[112578]: pgmap v88: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 255 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 35/264 objects degraded (13.258%) 2024-03-04T13:50:02.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:01 smithi107 ceph-mon[112578]: from='osd.1 [v2:172.21.15.107:6810/1164427580,v1:172.21.15.107:6811/1164427580]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-04T13:50:02.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:01 smithi107 ceph-mon[112578]: osdmap e58: 6 total, 5 up, 6 in 2024-03-04T13:50:02.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:01 smithi107 ceph-mon[112578]: from='osd.1 [v2:172.21.15.107:6810/1164427580,v1:172.21.15.107:6811/1164427580]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-03-04T13:50:02.344 INFO:journalctl@ceph.osd.1.smithi107.stdout:Mar 04 13:50:01 smithi107 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-1[130089]: 2024-03-04T13:50:01.950+0000 7f363d200700 -1 osd.1 55 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-04T13:50:02.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:01 smithi169 ceph-mon[86183]: pgmap v88: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 255 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 35/264 objects degraded (13.258%) 2024-03-04T13:50:02.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:01 smithi169 ceph-mon[86183]: from='osd.1 [v2:172.21.15.107:6810/1164427580,v1:172.21.15.107:6811/1164427580]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-04T13:50:02.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:01 smithi169 ceph-mon[86183]: osdmap e58: 6 total, 5 up, 6 in 2024-03-04T13:50:02.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:01 smithi169 ceph-mon[86183]: from='osd.1 [v2:172.21.15.107:6810/1164427580,v1:172.21.15.107:6811/1164427580]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-03-04T13:50:03.343 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:02 smithi107 ceph-mon[112578]: from='osd.1 [v2:172.21.15.107:6810/1164427580,v1:172.21.15.107:6811/1164427580]' entity='osd.1' 2024-03-04T13:50:03.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:02 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:50:03.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:02 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:50:03.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:02 smithi107 ceph-mon[112578]: pgmap v90: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 255 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s; 35/264 objects degraded (13.258%) 2024-03-04T13:50:03.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:02 smithi169 ceph-mon[86183]: from='osd.1 [v2:172.21.15.107:6810/1164427580,v1:172.21.15.107:6811/1164427580]' entity='osd.1' 2024-03-04T13:50:03.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:02 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:50:03.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:02 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:50:03.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:02 smithi169 ceph-mon[86183]: pgmap v90: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 255 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s; 35/264 objects degraded (13.258%) 2024-03-04T13:50:04.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:03 smithi107 ceph-mon[112578]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-04T13:50:04.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:03 smithi107 ceph-mon[112578]: osd.1 [v2:172.21.15.107:6810/1164427580,v1:172.21.15.107:6811/1164427580] boot 2024-03-04T13:50:04.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:03 smithi107 ceph-mon[112578]: osdmap e59: 6 total, 6 up, 6 in 2024-03-04T13:50:04.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:03 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-04T13:50:04.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:03 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:50:04.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:03 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:50:04.364 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:03 smithi169 ceph-mon[86183]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-04T13:50:04.364 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:03 smithi169 ceph-mon[86183]: osd.1 [v2:172.21.15.107:6810/1164427580,v1:172.21.15.107:6811/1164427580] boot 2024-03-04T13:50:04.364 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:03 smithi169 ceph-mon[86183]: osdmap e59: 6 total, 6 up, 6 in 2024-03-04T13:50:04.364 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:03 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-04T13:50:04.364 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:03 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:50:04.364 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:03 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:50:05.185 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:04 smithi107 ceph-mon[112578]: osdmap e60: 6 total, 6 up, 6 in 2024-03-04T13:50:05.185 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:04 smithi107 ceph-mon[112578]: pgmap v93: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 255 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 35/264 objects degraded (13.258%) 2024-03-04T13:50:05.360 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:04 smithi169 ceph-mon[86183]: osdmap e60: 6 total, 6 up, 6 in 2024-03-04T13:50:05.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:04 smithi169 ceph-mon[86183]: pgmap v93: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 255 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 35/264 objects degraded (13.258%) 2024-03-04T13:50:06.843 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:06 smithi107 ceph-mon[112578]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 35/264 objects degraded (13.258%), 13 pgs degraded) 2024-03-04T13:50:07.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:06 smithi169 ceph-mon[86183]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 35/264 objects degraded (13.258%), 13 pgs degraded) 2024-03-04T13:50:08.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:07 smithi107 ceph-mon[112578]: pgmap v94: 65 pgs: 65 active+clean; 255 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:50:08.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:07 smithi169 ceph-mon[86183]: pgmap v94: 65 pgs: 65 active+clean; 255 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:50:09.843 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:09 smithi107 ceph-mon[112578]: pgmap v95: 65 pgs: 65 active+clean; 255 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 657 B/s rd, 1 op/s 2024-03-04T13:50:09.844 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:09 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:50:09.844 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:09 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:50:09.844 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:09 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:50:09.844 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:09 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:50:09.844 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:09 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:50:09.844 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:09 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:50:09.844 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:09 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:50:10.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:09 smithi169 ceph-mon[86183]: pgmap v95: 65 pgs: 65 active+clean; 255 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 657 B/s rd, 1 op/s 2024-03-04T13:50:10.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:09 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:50:10.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:09 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:50:10.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:09 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:50:10.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:09 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:50:10.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:09 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:50:10.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:09 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:50:10.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:09 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:50:11.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:10 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:50:11.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:10 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:50:11.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:10 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:50:11.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:10 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:50:11.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:10 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-04T13:50:11.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:10 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:50:11.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:10 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:50:11.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:10 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:50:11.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:10 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:50:11.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:10 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-04T13:50:12.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:11 smithi107 ceph-mon[112578]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-04T13:50:12.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:11 smithi107 ceph-mon[112578]: Upgrade: osd.2 is safe to restart 2024-03-04T13:50:12.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:11 smithi107 ceph-mon[112578]: pgmap v96: 65 pgs: 65 active+clean; 255 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:50:12.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:11 smithi169 ceph-mon[86183]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-04T13:50:12.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:11 smithi169 ceph-mon[86183]: Upgrade: osd.2 is safe to restart 2024-03-04T13:50:12.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:11 smithi169 ceph-mon[86183]: pgmap v96: 65 pgs: 65 active+clean; 255 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:50:13.675 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:13 smithi107 ceph-mon[112578]: Upgrade: Updating osd.2 2024-03-04T13:50:13.675 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:13 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:50:13.675 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:13 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-04T13:50:13.676 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:13 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:50:13.676 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:13 smithi107 ceph-mon[112578]: Deploying daemon osd.2 on smithi107 2024-03-04T13:50:13.676 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:13 smithi107 ceph-mon[112578]: pgmap v97: 65 pgs: 65 active+clean; 255 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.0 KiB/s rd, 2 op/s 2024-03-04T13:50:13.861 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:13 smithi169 ceph-mon[86183]: Upgrade: Updating osd.2 2024-03-04T13:50:13.861 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:13 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:50:13.861 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:13 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-04T13:50:13.861 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:13 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:50:13.862 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:13 smithi169 ceph-mon[86183]: Deploying daemon osd.2 on smithi107 2024-03-04T13:50:13.862 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:13 smithi169 ceph-mon[86183]: pgmap v97: 65 pgs: 65 active+clean; 255 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.0 KiB/s rd, 2 op/s 2024-03-04T13:50:16.013 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:15 smithi107 ceph-mon[112578]: pgmap v98: 65 pgs: 65 active+clean; 255 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 956 B/s rd, 1 op/s 2024-03-04T13:50:16.013 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:15 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:50:16.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:15 smithi169 ceph-mon[86183]: pgmap v98: 65 pgs: 65 active+clean; 255 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 956 B/s rd, 1 op/s 2024-03-04T13:50:16.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:15 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:50:17.594 INFO:journalctl@ceph.osd.2.smithi107.stdout:Mar 04 13:50:17 smithi107 systemd[1]: Stopping Ceph osd.2 for 1fc09a58-da2b-11ee-95c6-87774f69a715... 2024-03-04T13:50:18.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:17 smithi107 ceph-mon[112578]: pgmap v99: 65 pgs: 65 active+clean; 255 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:50:18.094 INFO:journalctl@ceph.osd.2.smithi107.stdout:Mar 04 13:50:17 smithi107 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-2[62189]: 2024-03-04T13:50:17.764+0000 7f679fa00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-04T13:50:18.095 INFO:journalctl@ceph.osd.2.smithi107.stdout:Mar 04 13:50:17 smithi107 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-2[62189]: 2024-03-04T13:50:17.764+0000 7f679fa00700 -1 osd.2 60 *** Got signal Terminated *** 2024-03-04T13:50:18.095 INFO:journalctl@ceph.osd.2.smithi107.stdout:Mar 04 13:50:17 smithi107 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-2[62189]: 2024-03-04T13:50:17.764+0000 7f679fa00700 -1 osd.2 60 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-04T13:50:18.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:17 smithi169 ceph-mon[86183]: pgmap v99: 65 pgs: 65 active+clean; 255 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:50:19.013 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:18 smithi169 ceph-mon[86183]: osd.2 marked itself down and dead 2024-03-04T13:50:19.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:18 smithi107 ceph-mon[112578]: osd.2 marked itself down and dead 2024-03-04T13:50:19.594 INFO:journalctl@ceph.osd.2.smithi107.stdout:Mar 04 13:50:19 smithi107 podman[135263]: 2024-03-04 13:50:19.304955829 +0000 UTC m=+1.678292019 container died 61f30f643549ee7aa269457d81147a05de50c28863967f41a7b52b90b907626d (image=quay.ceph.io/ceph-ci/ceph@sha256:ee85afea73f504aaae524393a00a3b140e2a1cfd81d76da4525eaa6e73286a7d, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-2, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, RELEASE=quincy-42608bb, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-03-04T13:50:20.001 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:19 smithi107 ceph-mon[112578]: pgmap v100: 65 pgs: 65 active+clean; 255 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:50:20.002 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:19 smithi107 ceph-mon[112578]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-04T13:50:20.002 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:19 smithi107 ceph-mon[112578]: osdmap e61: 6 total, 5 up, 6 in 2024-03-04T13:50:20.002 INFO:journalctl@ceph.osd.2.smithi107.stdout:Mar 04 13:50:19 smithi107 podman[135263]: 2024-03-04 13:50:19.738405485 +0000 UTC m=+2.111741699 container cleanup 61f30f643549ee7aa269457d81147a05de50c28863967f41a7b52b90b907626d (image=quay.ceph.io/ceph-ci/ceph@sha256:ee85afea73f504aaae524393a00a3b140e2a1cfd81d76da4525eaa6e73286a7d, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-2, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=quincy-42608bb, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0) 2024-03-04T13:50:20.002 INFO:journalctl@ceph.osd.2.smithi107.stdout:Mar 04 13:50:19 smithi107 bash[135263]: ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-2 2024-03-04T13:50:20.038 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:19 smithi169 ceph-mon[86183]: pgmap v100: 65 pgs: 65 active+clean; 255 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:50:20.038 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:19 smithi169 ceph-mon[86183]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-04T13:50:20.038 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:19 smithi169 ceph-mon[86183]: osdmap e61: 6 total, 5 up, 6 in 2024-03-04T13:50:20.344 INFO:journalctl@ceph.osd.2.smithi107.stdout:Mar 04 13:50:20 smithi107 podman[135281]: 2024-03-04 13:50:19.999638713 +0000 UTC m=+0.685967656 container remove 61f30f643549ee7aa269457d81147a05de50c28863967f41a7b52b90b907626d (image=quay.ceph.io/ceph-ci/ceph@sha256:ee85afea73f504aaae524393a00a3b140e2a1cfd81d76da4525eaa6e73286a7d, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-2, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, GIT_BRANCH=HEAD, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, RELEASE=quincy-42608bb, org.label-schema.vendor=CentOS) 2024-03-04T13:50:21.057 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:20 smithi107 ceph-mon[112578]: osdmap e62: 6 total, 5 up, 6 in 2024-03-04T13:50:21.058 INFO:journalctl@ceph.osd.2.smithi107.stdout:Mar 04 13:50:20 smithi107 podman[135370]: 2024-03-04 13:50:20.639717316 +0000 UTC m=+0.033107997 image pull d71b839abdf892550525dad59fdf20a6b925b73dbcc7526f0d04da8380b9fabf quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377 2024-03-04T13:50:21.058 INFO:journalctl@ceph.osd.2.smithi107.stdout:Mar 04 13:50:20 smithi107 podman[135370]: 2024-03-04 13:50:20.792438946 +0000 UTC m=+0.185829657 container create 7a37ead732252c704b928fa4f9b6835a417ef56719cb82e2393f24732f62f421 (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-2-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240222, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-03-04T13:50:21.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:20 smithi169 ceph-mon[86183]: osdmap e62: 6 total, 5 up, 6 in 2024-03-04T13:50:21.344 INFO:journalctl@ceph.osd.2.smithi107.stdout:Mar 04 13:50:21 smithi107 podman[135370]: 2024-03-04 13:50:21.05705345 +0000 UTC m=+0.450444168 container init 7a37ead732252c704b928fa4f9b6835a417ef56719cb82e2393f24732f62f421 (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.build-date=20240222, ceph=True) 2024-03-04T13:50:21.344 INFO:journalctl@ceph.osd.2.smithi107.stdout:Mar 04 13:50:21 smithi107 podman[135370]: 2024-03-04 13:50:21.070083229 +0000 UTC m=+0.463473977 container start 7a37ead732252c704b928fa4f9b6835a417ef56719cb82e2393f24732f62f421 (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-2-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240222, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-03-04T13:50:21.344 INFO:journalctl@ceph.osd.2.smithi107.stdout:Mar 04 13:50:21 smithi107 podman[135370]: 2024-03-04 13:50:21.145122038 +0000 UTC m=+0.538512729 container attach 7a37ead732252c704b928fa4f9b6835a417ef56719cb82e2393f24732f62f421 (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-2-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-04T13:50:21.789 INFO:journalctl@ceph.osd.2.smithi107.stdout:Mar 04 13:50:21 smithi107 podman[135370]: 2024-03-04 13:50:21.509680421 +0000 UTC m=+0.903071139 container died 7a37ead732252c704b928fa4f9b6835a417ef56719cb82e2393f24732f62f421 (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-2-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True) 2024-03-04T13:50:21.861 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:21 smithi169 ceph-mon[86183]: pgmap v103: 65 pgs: 5 active+undersized+degraded, 7 stale+active+clean, 7 active+undersized, 46 active+clean; 255 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.1 KiB/s rd, 2 op/s; 10/264 objects degraded (3.788%) 2024-03-04T13:50:21.861 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:21 smithi169 ceph-mon[86183]: Health check failed: Degraded data redundancy: 10/264 objects degraded (3.788%), 5 pgs degraded (PG_DEGRADED) 2024-03-04T13:50:22.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:21 smithi107 ceph-mon[112578]: pgmap v103: 65 pgs: 5 active+undersized+degraded, 7 stale+active+clean, 7 active+undersized, 46 active+clean; 255 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.1 KiB/s rd, 2 op/s; 10/264 objects degraded (3.788%) 2024-03-04T13:50:22.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:21 smithi107 ceph-mon[112578]: Health check failed: Degraded data redundancy: 10/264 objects degraded (3.788%), 5 pgs degraded (PG_DEGRADED) 2024-03-04T13:50:22.595 INFO:journalctl@ceph.osd.2.smithi107.stdout:Mar 04 13:50:22 smithi107 podman[135395]: 2024-03-04 13:50:22.245085739 +0000 UTC m=+0.718574154 container remove 7a37ead732252c704b928fa4f9b6835a417ef56719cb82e2393f24732f62f421 (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-2-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240222, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-04T13:50:22.595 INFO:journalctl@ceph.osd.2.smithi107.stdout:Mar 04 13:50:22 smithi107 systemd[1]: ceph-1fc09a58-da2b-11ee-95c6-87774f69a715@osd.2.service: Deactivated successfully. 2024-03-04T13:50:22.595 INFO:journalctl@ceph.osd.2.smithi107.stdout:Mar 04 13:50:22 smithi107 systemd[1]: Stopped Ceph osd.2 for 1fc09a58-da2b-11ee-95c6-87774f69a715. 2024-03-04T13:50:22.595 INFO:journalctl@ceph.osd.2.smithi107.stdout:Mar 04 13:50:22 smithi107 systemd[1]: ceph-1fc09a58-da2b-11ee-95c6-87774f69a715@osd.2.service: Consumed 1min 34.397s CPU time. 2024-03-04T13:50:22.998 INFO:journalctl@ceph.osd.2.smithi107.stdout:Mar 04 13:50:22 smithi107 systemd[1]: Starting Ceph osd.2 for 1fc09a58-da2b-11ee-95c6-87774f69a715... 2024-03-04T13:50:23.305 INFO:journalctl@ceph.osd.2.smithi107.stdout:Mar 04 13:50:22 smithi107 podman[135497]: 2024-03-04 13:50:22.903731171 +0000 UTC m=+0.034856457 image pull d71b839abdf892550525dad59fdf20a6b925b73dbcc7526f0d04da8380b9fabf quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377 2024-03-04T13:50:23.305 INFO:journalctl@ceph.osd.2.smithi107.stdout:Mar 04 13:50:23 smithi107 podman[135497]: 2024-03-04 13:50:23.059003573 +0000 UTC m=+0.190128896 container create 1477327054eafedd6babec35a39a6d48d78dc814dc1717529c4b11a66ef5977e (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-2-activate, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240222, CEPH_POINT_RELEASE=, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-03-04T13:50:23.305 INFO:journalctl@ceph.osd.2.smithi107.stdout:Mar 04 13:50:23 smithi107 podman[135497]: 2024-03-04 13:50:23.304137395 +0000 UTC m=+0.435262765 container init 1477327054eafedd6babec35a39a6d48d78dc814dc1717529c4b11a66ef5977e (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-2-activate, ceph=True, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240222, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-03-04T13:50:23.594 INFO:journalctl@ceph.osd.2.smithi107.stdout:Mar 04 13:50:23 smithi107 podman[135497]: 2024-03-04 13:50:23.316536213 +0000 UTC m=+0.447661509 container start 1477327054eafedd6babec35a39a6d48d78dc814dc1717529c4b11a66ef5977e (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-2-activate, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222) 2024-03-04T13:50:23.594 INFO:journalctl@ceph.osd.2.smithi107.stdout:Mar 04 13:50:23 smithi107 podman[135497]: 2024-03-04 13:50:23.420809488 +0000 UTC m=+0.551934795 container attach 1477327054eafedd6babec35a39a6d48d78dc814dc1717529c4b11a66ef5977e (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-2-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240222, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-03-04T13:50:24.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:23 smithi107 ceph-mon[112578]: pgmap v104: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 255 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 895 B/s rd, 1 op/s; 34/264 objects degraded (12.879%) 2024-03-04T13:50:24.110 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:23 smithi169 ceph-mon[86183]: pgmap v104: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 255 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 895 B/s rd, 1 op/s; 34/264 objects degraded (12.879%) 2024-03-04T13:50:24.532 INFO:journalctl@ceph.osd.2.smithi107.stdout:Mar 04 13:50:24 smithi107 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-2-activate[135512]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-04T13:50:24.532 INFO:journalctl@ceph.osd.2.smithi107.stdout:Mar 04 13:50:24 smithi107 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-2-activate[135512]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-03-04T13:50:24.532 INFO:journalctl@ceph.osd.2.smithi107.stdout:Mar 04 13:50:24 smithi107 bash[135497]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-04T13:50:24.532 INFO:journalctl@ceph.osd.2.smithi107.stdout:Mar 04 13:50:24 smithi107 bash[135497]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-03-04T13:50:24.532 INFO:journalctl@ceph.osd.2.smithi107.stdout:Mar 04 13:50:24 smithi107 bash[135497]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-03-04T13:50:24.533 INFO:journalctl@ceph.osd.2.smithi107.stdout:Mar 04 13:50:24 smithi107 bash[135497]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-03-04T13:50:24.533 INFO:journalctl@ceph.osd.2.smithi107.stdout:Mar 04 13:50:24 smithi107 bash[135497]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-03-04T13:50:24.533 INFO:journalctl@ceph.osd.2.smithi107.stdout:Mar 04 13:50:24 smithi107 bash[135497]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-04T13:50:24.533 INFO:journalctl@ceph.osd.2.smithi107.stdout:Mar 04 13:50:24 smithi107 bash[135497]: --> ceph-volume raw activate successful for osd ID: 2 2024-03-04T13:50:24.533 INFO:journalctl@ceph.osd.2.smithi107.stdout:Mar 04 13:50:24 smithi107 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-2-activate[135512]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-03-04T13:50:24.533 INFO:journalctl@ceph.osd.2.smithi107.stdout:Mar 04 13:50:24 smithi107 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-2-activate[135512]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-03-04T13:50:24.533 INFO:journalctl@ceph.osd.2.smithi107.stdout:Mar 04 13:50:24 smithi107 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-2-activate[135512]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-03-04T13:50:24.533 INFO:journalctl@ceph.osd.2.smithi107.stdout:Mar 04 13:50:24 smithi107 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-2-activate[135512]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-04T13:50:24.533 INFO:journalctl@ceph.osd.2.smithi107.stdout:Mar 04 13:50:24 smithi107 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-2-activate[135512]: --> ceph-volume raw activate successful for osd ID: 2 2024-03-04T13:50:24.533 INFO:journalctl@ceph.osd.2.smithi107.stdout:Mar 04 13:50:24 smithi107 podman[135497]: 2024-03-04 13:50:24.228781947 +0000 UTC m=+1.359907293 container died 1477327054eafedd6babec35a39a6d48d78dc814dc1717529c4b11a66ef5977e (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-2-activate, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240222, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-03-04T13:50:24.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:24.571+0000 7fb65ac00700 1 -- 172.21.15.107:0/3610999800 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb654102d70 msgr2=0x7fb654103140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:50:24.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:24.571+0000 7fb65ac00700 1 --2- 172.21.15.107:0/3610999800 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb654102d70 0x7fb654103140 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb648004940 tx=0x7fb648009e30 comp rx=0 tx=0).stop 2024-03-04T13:50:24.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:24.571+0000 7fb65ac00700 1 -- 172.21.15.107:0/3610999800 shutdown_connections 2024-03-04T13:50:24.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:24.571+0000 7fb65ac00700 1 --2- 172.21.15.107:0/3610999800 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb654103710 0x7fb6540fcd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:50:24.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:24.571+0000 7fb65ac00700 1 --2- 172.21.15.107:0/3610999800 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb654102d70 0x7fb654103140 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:50:24.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:24.571+0000 7fb65ac00700 1 -- 172.21.15.107:0/3610999800 >> 172.21.15.107:0/3610999800 conn(0x7fb6540f88e0 msgr2=0x7fb6540facf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:50:24.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:24.571+0000 7fb65ac00700 1 -- 172.21.15.107:0/3610999800 shutdown_connections 2024-03-04T13:50:24.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:24.571+0000 7fb65ac00700 1 -- 172.21.15.107:0/3610999800 wait complete. 2024-03-04T13:50:24.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:24.572+0000 7fb65ac00700 1 Processor -- start 2024-03-04T13:50:24.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:24.572+0000 7fb65ac00700 1 -- start start 2024-03-04T13:50:24.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:24.573+0000 7fb65ac00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb654102d70 0x7fb654070290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:50:24.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:24.573+0000 7fb65ac00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb654103710 0x7fb6540707d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:50:24.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:24.573+0000 7fb65ac00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6541118c0 con 0x7fb654103710 2024-03-04T13:50:24.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:24.573+0000 7fb65ac00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb654070d10 con 0x7fb654102d70 2024-03-04T13:50:24.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:24.573+0000 7fb658e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb654103710 0x7fb6540707d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:50:24.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:24.573+0000 7fb658e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb654103710 0x7fb6540707d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54680/0 (socket says 172.21.15.107:54680) 2024-03-04T13:50:24.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:24.573+0000 7fb658e00700 1 -- 172.21.15.107:0/2176564886 learned_addr learned my addr 172.21.15.107:0/2176564886 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:50:24.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:24.573+0000 7fb659800700 1 --2- 172.21.15.107:0/2176564886 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb654102d70 0x7fb654070290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:50:24.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:24.574+0000 7fb658e00700 1 -- 172.21.15.107:0/2176564886 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb654102d70 msgr2=0x7fb654070290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:50:24.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:24.574+0000 7fb658e00700 1 --2- 172.21.15.107:0/2176564886 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb654102d70 0x7fb654070290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:50:24.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:24.574+0000 7fb658e00700 1 -- 172.21.15.107:0/2176564886 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb648005040 con 0x7fb654103710 2024-03-04T13:50:24.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:24.574+0000 7fb658e00700 1 --2- 172.21.15.107:0/2176564886 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb654103710 0x7fb6540707d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb64000b790 tx=0x7fb640009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:50:24.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:24.574+0000 7fb651600700 1 -- 172.21.15.107:0/2176564886 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb64000f6f0 con 0x7fb654103710 2024-03-04T13:50:24.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:24.574+0000 7fb651600700 1 -- 172.21.15.107:0/2176564886 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb64000fd20 con 0x7fb654103710 2024-03-04T13:50:24.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:24.574+0000 7fb651600700 1 -- 172.21.15.107:0/2176564886 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb6400103a0 con 0x7fb654103710 2024-03-04T13:50:24.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:24.574+0000 7fb65ac00700 1 -- 172.21.15.107:0/2176564886 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb654070ff0 con 0x7fb654103710 2024-03-04T13:50:24.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:24.574+0000 7fb65ac00700 1 -- 172.21.15.107:0/2176564886 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb65411e570 con 0x7fb654103710 2024-03-04T13:50:24.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:24.576+0000 7fb65ac00700 1 -- 172.21.15.107:0/2176564886 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb65405f5c0 con 0x7fb654103710 2024-03-04T13:50:24.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:24.577+0000 7fb651600700 1 -- 172.21.15.107:0/2176564886 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fb640011400 con 0x7fb654103710 2024-03-04T13:50:24.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:24.578+0000 7fb651600700 1 --2- 172.21.15.107:0/2176564886 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fb63c06d7f0 0x7fb63c06fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:50:24.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:24.578+0000 7fb651600700 1 -- 172.21.15.107:0/2176564886 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fb64008ccb0 con 0x7fb654103710 2024-03-04T13:50:24.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:24.578+0000 7fb659800700 1 --2- 172.21.15.107:0/2176564886 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fb63c06d7f0 0x7fb63c06fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:50:24.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:24.579+0000 7fb659800700 1 --2- 172.21.15.107:0/2176564886 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fb63c06d7f0 0x7fb63c06fca0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb64800b020 tx=0x7fb648009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:50:24.584 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:24.583+0000 7fb651600700 1 -- 172.21.15.107:0/2176564886 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb640056530 con 0x7fb654103710 2024-03-04T13:50:24.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:24.825+0000 7fb65ac00700 1 -- 172.21.15.107:0/2176564886 --> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb654071ae0 con 0x7fb63c06d7f0 2024-03-04T13:50:24.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:24.827+0000 7fb651600700 1 -- 172.21.15.107:0/2176564886 <== mgr.34102 v2:172.21.15.107:6800/1223039812 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fb654071ae0 con 0x7fb63c06d7f0 2024-03-04T13:50:24.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:24.831+0000 7fb65ac00700 1 -- 172.21.15.107:0/2176564886 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fb63c06d7f0 msgr2=0x7fb63c06fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:50:24.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:24.831+0000 7fb65ac00700 1 --2- 172.21.15.107:0/2176564886 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fb63c06d7f0 0x7fb63c06fca0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb64800b020 tx=0x7fb648009e30 comp rx=0 tx=0).stop 2024-03-04T13:50:24.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:24.831+0000 7fb65ac00700 1 -- 172.21.15.107:0/2176564886 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb654103710 msgr2=0x7fb6540707d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:50:24.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:24.831+0000 7fb65ac00700 1 --2- 172.21.15.107:0/2176564886 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb654103710 0x7fb6540707d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb64000b790 tx=0x7fb640009220 comp rx=0 tx=0).stop 2024-03-04T13:50:24.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:24.831+0000 7fb65ac00700 1 -- 172.21.15.107:0/2176564886 shutdown_connections 2024-03-04T13:50:24.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:24.831+0000 7fb65ac00700 1 --2- 172.21.15.107:0/2176564886 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fb63c06d7f0 0x7fb63c06fca0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:50:24.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:24.831+0000 7fb65ac00700 1 --2- 172.21.15.107:0/2176564886 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb654102d70 0x7fb654070290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:50:24.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:24.831+0000 7fb65ac00700 1 --2- 172.21.15.107:0/2176564886 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb654103710 0x7fb6540707d0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:50:24.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:24.831+0000 7fb65ac00700 1 -- 172.21.15.107:0/2176564886 >> 172.21.15.107:0/2176564886 conn(0x7fb6540f88e0 msgr2=0x7fb6540fa2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:50:24.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:24.831+0000 7fb65ac00700 1 -- 172.21.15.107:0/2176564886 shutdown_connections 2024-03-04T13:50:24.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:24.831+0000 7fb65ac00700 1 -- 172.21.15.107:0/2176564886 wait complete. 2024-03-04T13:50:24.848 INFO:teuthology.orchestra.run.smithi107.stdout:true 2024-03-04T13:50:24.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:24.986+0000 7fac4aa00700 1 -- 172.21.15.107:0/2829870065 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac44102f50 msgr2=0x7fac44103320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:50:24.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:24.986+0000 7fac4aa00700 1 --2- 172.21.15.107:0/2829870065 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac44102f50 0x7fac44103320 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fac34004940 tx=0x7fac34009e30 comp rx=0 tx=0).stop 2024-03-04T13:50:24.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:24.987+0000 7fac4aa00700 1 -- 172.21.15.107:0/2829870065 shutdown_connections 2024-03-04T13:50:24.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:24.987+0000 7fac4aa00700 1 --2- 172.21.15.107:0/2829870065 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac441038f0 0x7fac440fcf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:50:24.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:24.987+0000 7fac4aa00700 1 --2- 172.21.15.107:0/2829870065 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac44102f50 0x7fac44103320 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:50:24.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:24.987+0000 7fac4aa00700 1 -- 172.21.15.107:0/2829870065 >> 172.21.15.107:0/2829870065 conn(0x7fac440f8ae0 msgr2=0x7fac440faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:50:24.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:24.987+0000 7fac4aa00700 1 -- 172.21.15.107:0/2829870065 shutdown_connections 2024-03-04T13:50:24.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:24.987+0000 7fac4aa00700 1 -- 172.21.15.107:0/2829870065 wait complete. 2024-03-04T13:50:24.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:24.988+0000 7fac4aa00700 1 Processor -- start 2024-03-04T13:50:24.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:24.988+0000 7fac4aa00700 1 -- start start 2024-03-04T13:50:24.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:24.988+0000 7fac4aa00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac44102f50 0x7fac44070290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:50:24.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:24.988+0000 7fac4aa00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac441038f0 0x7fac440707d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:50:24.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:24.988+0000 7fac4aa00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac44111920 con 0x7fac441038f0 2024-03-04T13:50:24.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:24.988+0000 7fac4aa00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac44111a90 con 0x7fac44102f50 2024-03-04T13:50:24.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:24.989+0000 7fac43400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac441038f0 0x7fac440707d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:50:24.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:24.989+0000 7fac43400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac441038f0 0x7fac440707d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54696/0 (socket says 172.21.15.107:54696) 2024-03-04T13:50:24.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:24.989+0000 7fac43400700 1 -- 172.21.15.107:0/743364396 learned_addr learned my addr 172.21.15.107:0/743364396 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:50:24.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:24.989+0000 7fac43e00700 1 --2- 172.21.15.107:0/743364396 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac44102f50 0x7fac44070290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:50:24.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:24.989+0000 7fac43400700 1 -- 172.21.15.107:0/743364396 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac44102f50 msgr2=0x7fac44070290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:50:24.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:24.989+0000 7fac43400700 1 --2- 172.21.15.107:0/743364396 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac44102f50 0x7fac44070290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:50:24.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:24.989+0000 7fac43400700 1 -- 172.21.15.107:0/743364396 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac34005040 con 0x7fac441038f0 2024-03-04T13:50:24.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:24.990+0000 7fac43400700 1 --2- 172.21.15.107:0/743364396 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac441038f0 0x7fac440707d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fac3000c800 tx=0x7fac30004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:50:24.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:24.990+0000 7fac3be00700 1 -- 172.21.15.107:0/743364396 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fac3000a700 con 0x7fac441038f0 2024-03-04T13:50:24.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:24.990+0000 7fac3be00700 1 -- 172.21.15.107:0/743364396 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fac3000ad30 con 0x7fac441038f0 2024-03-04T13:50:24.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:24.990+0000 7fac4aa00700 1 -- 172.21.15.107:0/743364396 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac44070e60 con 0x7fac441038f0 2024-03-04T13:50:24.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:24.990+0000 7fac3be00700 1 -- 172.21.15.107:0/743364396 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fac300113a0 con 0x7fac441038f0 2024-03-04T13:50:24.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:24.990+0000 7fac4aa00700 1 -- 172.21.15.107:0/743364396 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac4411e620 con 0x7fac441038f0 2024-03-04T13:50:24.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:24.991+0000 7fac4aa00700 1 -- 172.21.15.107:0/743364396 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac440fc6e0 con 0x7fac441038f0 2024-03-04T13:50:24.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:24.993+0000 7fac3be00700 1 -- 172.21.15.107:0/743364396 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fac30012400 con 0x7fac441038f0 2024-03-04T13:50:24.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:24.994+0000 7fac3be00700 1 --2- 172.21.15.107:0/743364396 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fac2806d7f0 0x7fac2806fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:50:24.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:24.994+0000 7fac3be00700 1 -- 172.21.15.107:0/743364396 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fac3008ceb0 con 0x7fac441038f0 2024-03-04T13:50:24.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:24.994+0000 7fac43e00700 1 --2- 172.21.15.107:0/743364396 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fac2806d7f0 0x7fac2806fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:50:24.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:24.995+0000 7fac43e00700 1 --2- 172.21.15.107:0/743364396 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fac2806d7f0 0x7fac2806fca0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fac3400b020 tx=0x7fac34009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:50:25.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:25.000+0000 7fac3be00700 1 -- 172.21.15.107:0/743364396 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fac300567b0 con 0x7fac441038f0 2024-03-04T13:50:25.093 INFO:journalctl@ceph.osd.2.smithi107.stdout:Mar 04 13:50:24 smithi107 podman[135635]: 2024-03-04 13:50:24.94092262 +0000 UTC m=+0.688795111 container remove 1477327054eafedd6babec35a39a6d48d78dc814dc1717529c4b11a66ef5977e (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-2-activate, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8) 2024-03-04T13:50:25.246 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:25.245+0000 7fac4aa00700 1 -- 172.21.15.107:0/743364396 --> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fac440fac10 con 0x7fac2806d7f0 2024-03-04T13:50:25.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:25.248+0000 7fac3be00700 1 -- 172.21.15.107:0/743364396 <== mgr.34102 v2:172.21.15.107:6800/1223039812 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fac440fac10 con 0x7fac2806d7f0 2024-03-04T13:50:25.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:25.251+0000 7fac4aa00700 1 -- 172.21.15.107:0/743364396 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fac2806d7f0 msgr2=0x7fac2806fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:50:25.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:25.251+0000 7fac4aa00700 1 --2- 172.21.15.107:0/743364396 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fac2806d7f0 0x7fac2806fca0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fac3400b020 tx=0x7fac34009e30 comp rx=0 tx=0).stop 2024-03-04T13:50:25.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:25.251+0000 7fac4aa00700 1 -- 172.21.15.107:0/743364396 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac441038f0 msgr2=0x7fac440707d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:50:25.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:25.251+0000 7fac4aa00700 1 --2- 172.21.15.107:0/743364396 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac441038f0 0x7fac440707d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fac3000c800 tx=0x7fac30004470 comp rx=0 tx=0).stop 2024-03-04T13:50:25.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:25.251+0000 7fac4aa00700 1 -- 172.21.15.107:0/743364396 shutdown_connections 2024-03-04T13:50:25.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:25.251+0000 7fac4aa00700 1 --2- 172.21.15.107:0/743364396 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fac2806d7f0 0x7fac2806fca0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:50:25.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:25.251+0000 7fac4aa00700 1 --2- 172.21.15.107:0/743364396 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac44102f50 0x7fac44070290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:50:25.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:25.251+0000 7fac4aa00700 1 --2- 172.21.15.107:0/743364396 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac441038f0 0x7fac440707d0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:50:25.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:25.251+0000 7fac4aa00700 1 -- 172.21.15.107:0/743364396 >> 172.21.15.107:0/743364396 conn(0x7fac440f8ae0 msgr2=0x7fac440fa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:50:25.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:25.251+0000 7fac4aa00700 1 -- 172.21.15.107:0/743364396 shutdown_connections 2024-03-04T13:50:25.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:25.251+0000 7fac4aa00700 1 -- 172.21.15.107:0/743364396 wait complete. 2024-03-04T13:50:25.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:25.402+0000 7fbde6600700 1 -- 172.21.15.107:0/2532061314 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbde0102f50 msgr2=0x7fbde0103320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:50:25.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:25.402+0000 7fbde6600700 1 --2- 172.21.15.107:0/2532061314 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbde0102f50 0x7fbde0103320 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbdd0004940 tx=0x7fbdd0009e30 comp rx=0 tx=0).stop 2024-03-04T13:50:25.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:25.402+0000 7fbde6600700 1 -- 172.21.15.107:0/2532061314 shutdown_connections 2024-03-04T13:50:25.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:25.402+0000 7fbde6600700 1 --2- 172.21.15.107:0/2532061314 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbde01038f0 0x7fbde00fcf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:50:25.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:25.402+0000 7fbde6600700 1 --2- 172.21.15.107:0/2532061314 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbde0102f50 0x7fbde0103320 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:50:25.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:25.402+0000 7fbde6600700 1 -- 172.21.15.107:0/2532061314 >> 172.21.15.107:0/2532061314 conn(0x7fbde00f8ae0 msgr2=0x7fbde00faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:50:25.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:25.402+0000 7fbde6600700 1 -- 172.21.15.107:0/2532061314 shutdown_connections 2024-03-04T13:50:25.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:25.402+0000 7fbde6600700 1 -- 172.21.15.107:0/2532061314 wait complete. 2024-03-04T13:50:25.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:25.403+0000 7fbde6600700 1 Processor -- start 2024-03-04T13:50:25.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:25.403+0000 7fbde6600700 1 -- start start 2024-03-04T13:50:25.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:25.404+0000 7fbde6600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbde0102f50 0x7fbde0113a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:50:25.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:25.404+0000 7fbde6600700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbde01038f0 0x7fbde0113f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:50:25.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:25.404+0000 7fbde6600700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbde01145e0 con 0x7fbde0102f50 2024-03-04T13:50:25.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:25.404+0000 7fbde6600700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbde010d910 con 0x7fbde01038f0 2024-03-04T13:50:25.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:25.404+0000 7fbddf400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbde0102f50 0x7fbde0113a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:50:25.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:25.404+0000 7fbddf400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbde0102f50 0x7fbde0113a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54722/0 (socket says 172.21.15.107:54722) 2024-03-04T13:50:25.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:25.404+0000 7fbddf400700 1 -- 172.21.15.107:0/915893143 learned_addr learned my addr 172.21.15.107:0/915893143 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:50:25.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:25.404+0000 7fbddea00700 1 --2- 172.21.15.107:0/915893143 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbde01038f0 0x7fbde0113f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:50:25.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:25.405+0000 7fbddf400700 1 -- 172.21.15.107:0/915893143 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbde01038f0 msgr2=0x7fbde0113f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:50:25.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:25.405+0000 7fbddf400700 1 --2- 172.21.15.107:0/915893143 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbde01038f0 0x7fbde0113f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:50:25.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:25.405+0000 7fbddf400700 1 -- 172.21.15.107:0/915893143 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdd0005040 con 0x7fbde0102f50 2024-03-04T13:50:25.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:25.405+0000 7fbddf400700 1 --2- 172.21.15.107:0/915893143 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbde0102f50 0x7fbde0113a50 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fbdd0009150 tx=0x7fbdd0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:50:25.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:25.405+0000 7fbdd7400700 1 -- 172.21.15.107:0/915893143 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbdd000e040 con 0x7fbde0102f50 2024-03-04T13:50:25.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:25.405+0000 7fbdd7400700 1 -- 172.21.15.107:0/915893143 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbdd000bbf0 con 0x7fbde0102f50 2024-03-04T13:50:25.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:25.405+0000 7fbde6600700 1 -- 172.21.15.107:0/915893143 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbde010db90 con 0x7fbde0102f50 2024-03-04T13:50:25.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:25.405+0000 7fbdd7400700 1 -- 172.21.15.107:0/915893143 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbdd001c630 con 0x7fbde0102f50 2024-03-04T13:50:25.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:25.405+0000 7fbde6600700 1 -- 172.21.15.107:0/915893143 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbde010e0d0 con 0x7fbde0102f50 2024-03-04T13:50:25.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:25.407+0000 7fbde6600700 1 -- 172.21.15.107:0/915893143 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbde00fc6e0 con 0x7fbde0102f50 2024-03-04T13:50:25.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:25.409+0000 7fbdd7400700 1 -- 172.21.15.107:0/915893143 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fbdd000bd60 con 0x7fbde0102f50 2024-03-04T13:50:25.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:25.409+0000 7fbdd7400700 1 --2- 172.21.15.107:0/915893143 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fbdc406d7f0 0x7fbdc406fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:50:25.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:25.410+0000 7fbdd7400700 1 -- 172.21.15.107:0/915893143 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fbdd008e250 con 0x7fbde0102f50 2024-03-04T13:50:25.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:25.410+0000 7fbddea00700 1 --2- 172.21.15.107:0/915893143 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fbdc406d7f0 0x7fbdc406fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:50:25.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:25.411+0000 7fbddea00700 1 --2- 172.21.15.107:0/915893143 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fbdc406d7f0 0x7fbdc406fca0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbde010f280 tx=0x7fbdcc009ee0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:50:25.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:25.415+0000 7fbdd7400700 1 -- 172.21.15.107:0/915893143 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbdd005b680 con 0x7fbde0102f50 2024-03-04T13:50:25.554 INFO:journalctl@ceph.osd.2.smithi107.stdout:Mar 04 13:50:25 smithi107 podman[135738]: 2024-03-04 13:50:25.172791998 +0000 UTC m=+0.030934059 image pull d71b839abdf892550525dad59fdf20a6b925b73dbcc7526f0d04da8380b9fabf quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377 2024-03-04T13:50:25.554 INFO:journalctl@ceph.osd.2.smithi107.stdout:Mar 04 13:50:25 smithi107 podman[135738]: 2024-03-04 13:50:25.285348812 +0000 UTC m=+0.143490924 container create 6e4da4f6f146276914ee5dbafa3f0c124a08bea50684b074bf446e26f95314dd (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-2, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8) 2024-03-04T13:50:25.554 INFO:journalctl@ceph.osd.2.smithi107.stdout:Mar 04 13:50:25 smithi107 podman[135738]: 2024-03-04 13:50:25.553991608 +0000 UTC m=+0.412133750 container init 6e4da4f6f146276914ee5dbafa3f0c124a08bea50684b074bf446e26f95314dd (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c) 2024-03-04T13:50:25.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:25.636+0000 7fbde6600700 1 -- 172.21.15.107:0/915893143 --> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fbde010f0a0 con 0x7fbdc406d7f0 2024-03-04T13:50:25.655 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:25.654+0000 7fbdd7400700 1 -- 172.21.15.107:0/915893143 <== mgr.34102 v2:172.21.15.107:6800/1223039812 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fbde010f0a0 con 0x7fbdc406d7f0 2024-03-04T13:50:25.656 INFO:teuthology.orchestra.run.smithi107.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-04T13:50:25.656 INFO:teuthology.orchestra.run.smithi107.stdout:alertmanager.smithi107 smithi107 *:9093,9094 running (4m) 23s ago 18m 14.4M - 0.25.0 c8568f914cd2 e5ed7be32538 2024-03-04T13:50:25.656 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi107 smithi107 running (98s) 23s ago 19m 6660k - 19.0.0-1845-g717ce3cc d71b839abdf8 0558022b0d84 2024-03-04T13:50:25.656 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi169 smithi169 running (91s) 86s ago 17m 6665k - 19.0.0-1845-g717ce3cc d71b839abdf8 221ca53aa0d3 2024-03-04T13:50:25.656 INFO:teuthology.orchestra.run.smithi107.stdout:grafana.smithi107 smithi107 *:3000 running (4m) 23s ago 18m 74.5M - 9.4.12 a0ea5e3c46d9 c5e4d33fb599 2024-03-04T13:50:25.656 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.riexmm smithi107 running (11m) 23s ago 11m 14.8M - 17.2.7-545-g42608bb1 9eec3f22859d fc921f7d4328 2024-03-04T13:50:25.656 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.thycho smithi107 running (11m) 23s ago 11m 98.3M - 17.2.7-545-g42608bb1 9eec3f22859d 4069c6e0e063 2024-03-04T13:50:25.656 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi169.bxitzk smithi169 running (11m) 86s ago 11m 14.2M - 17.2.7-545-g42608bb1 9eec3f22859d 7d4115228c7c 2024-03-04T13:50:25.656 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi169.jusudl smithi169 running (11m) 86s ago 11m 145M - 17.2.7-545-g42608bb1 9eec3f22859d 2d2f2cc452e8 2024-03-04T13:50:25.656 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi107.wpzgwl smithi107 *:8443,9283,8765 running (7m) 23s ago 20m 503M - 19.0.0-1845-g717ce3cc d71b839abdf8 1c5fdee45eac 2024-03-04T13:50:25.656 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi169.eihczr smithi169 *:8443,9283,8765 running (6m) 86s ago 17m 449M - 19.0.0-1845-g717ce3cc d71b839abdf8 b5cac2b7d22a 2024-03-04T13:50:25.657 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi107 smithi107 running (3m) 23s ago 20m 44.9M 2048M 19.0.0-1845-g717ce3cc d71b839abdf8 32f07144274f 2024-03-04T13:50:25.657 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi169 smithi169 running (2m) 86s ago 17m 33.9M 2048M 19.0.0-1845-g717ce3cc d71b839abdf8 e3527449381a 2024-03-04T13:50:25.657 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi107 smithi107 *:9100 running (5m) 23s ago 19m 7898k - 1.5.0 0da6a335fe13 8a8d3d4f724f 2024-03-04T13:50:25.657 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi169 smithi169 *:9100 running (5m) 86s ago 17m 7688k - 1.5.0 0da6a335fe13 f240f5a975bc 2024-03-04T13:50:25.657 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0 smithi107 running (61s) 23s ago 16m 79.6M 1558M 19.0.0-1845-g717ce3cc d71b839abdf8 5801a5540dc6 2024-03-04T13:50:25.657 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1 smithi107 running (31s) 23s ago 15m 11.2M 1558M 19.0.0-1845-g717ce3cc d71b839abdf8 b614af658854 2024-03-04T13:50:25.657 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2 smithi107 running (15m) 23s ago 15m 181M 1558M 17.2.7-545-g42608bb1 9eec3f22859d 61f30f643549 2024-03-04T13:50:25.657 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3 smithi169 running (14m) 86s ago 14m 236M 2582M 17.2.7-545-g42608bb1 9eec3f22859d cd91d8694764 2024-03-04T13:50:25.657 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4 smithi169 running (14m) 86s ago 14m 207M 2582M 17.2.7-545-g42608bb1 9eec3f22859d 08545e3d0659 2024-03-04T13:50:25.657 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5 smithi169 running (13m) 86s ago 13m 194M 2582M 17.2.7-545-g42608bb1 9eec3f22859d dcffe6cb3eac 2024-03-04T13:50:25.657 INFO:teuthology.orchestra.run.smithi107.stdout:prometheus.smithi107 smithi107 *:9095 running (5m) 23s ago 18m 49.7M - 2.43.0 a07b618ecd1d 1eeb1ad5cb75 2024-03-04T13:50:25.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:25.657+0000 7fbde6600700 1 -- 172.21.15.107:0/915893143 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fbdc406d7f0 msgr2=0x7fbdc406fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:50:25.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:25.657+0000 7fbde6600700 1 --2- 172.21.15.107:0/915893143 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fbdc406d7f0 0x7fbdc406fca0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbde010f280 tx=0x7fbdcc009ee0 comp rx=0 tx=0).stop 2024-03-04T13:50:25.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:25.657+0000 7fbde6600700 1 -- 172.21.15.107:0/915893143 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbde0102f50 msgr2=0x7fbde0113a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:50:25.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:25.657+0000 7fbde6600700 1 --2- 172.21.15.107:0/915893143 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbde0102f50 0x7fbde0113a50 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fbdd0009150 tx=0x7fbdd0009e30 comp rx=0 tx=0).stop 2024-03-04T13:50:25.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:25.657+0000 7fbde6600700 1 -- 172.21.15.107:0/915893143 shutdown_connections 2024-03-04T13:50:25.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:25.657+0000 7fbde6600700 1 --2- 172.21.15.107:0/915893143 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fbdc406d7f0 0x7fbdc406fca0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:50:25.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:25.657+0000 7fbde6600700 1 --2- 172.21.15.107:0/915893143 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbde0102f50 0x7fbde0113a50 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:50:25.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:25.657+0000 7fbde6600700 1 --2- 172.21.15.107:0/915893143 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbde01038f0 0x7fbde0113f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:50:25.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:25.657+0000 7fbde6600700 1 -- 172.21.15.107:0/915893143 >> 172.21.15.107:0/915893143 conn(0x7fbde00f8ae0 msgr2=0x7fbde00fa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:50:25.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:25.658+0000 7fbde6600700 1 -- 172.21.15.107:0/915893143 shutdown_connections 2024-03-04T13:50:25.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:25.658+0000 7fbde6600700 1 -- 172.21.15.107:0/915893143 wait complete. 2024-03-04T13:50:25.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:25.806+0000 7f16c8000700 1 -- 172.21.15.107:0/184143394 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f16c0105b10 msgr2=0x7f16c00ff180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:50:25.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:25.806+0000 7f16c8000700 1 --2- 172.21.15.107:0/184143394 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f16c0105b10 0x7f16c00ff180 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f16ac004940 tx=0x7f16ac009e30 comp rx=0 tx=0).stop 2024-03-04T13:50:25.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:25.807+0000 7f16c8000700 1 -- 172.21.15.107:0/184143394 shutdown_connections 2024-03-04T13:50:25.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:25.807+0000 7f16c8000700 1 --2- 172.21.15.107:0/184143394 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f16c0105b10 0x7f16c00ff180 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:50:25.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:25.807+0000 7f16c8000700 1 --2- 172.21.15.107:0/184143394 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16c0105170 0x7f16c0105540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:50:25.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:25.807+0000 7f16c8000700 1 -- 172.21.15.107:0/184143394 >> 172.21.15.107:0/184143394 conn(0x7f16c00face0 msgr2=0x7f16c00fd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:50:25.808 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:25 smithi107 ceph-mon[112578]: pgmap v105: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 255 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 255 B/s rd, 0 op/s; 34/264 objects degraded (12.879%) 2024-03-04T13:50:25.809 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:25 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:50:25.809 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:25 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:50:25.809 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:25 smithi107 ceph-mon[112578]: from='client.34278 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:50:25.809 INFO:journalctl@ceph.osd.2.smithi107.stdout:Mar 04 13:50:25 smithi107 podman[135738]: 2024-03-04 13:50:25.563939533 +0000 UTC m=+0.422081603 container start 6e4da4f6f146276914ee5dbafa3f0c124a08bea50684b074bf446e26f95314dd (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-2, org.label-schema.build-date=20240222, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-04T13:50:25.809 INFO:journalctl@ceph.osd.2.smithi107.stdout:Mar 04 13:50:25 smithi107 ceph-osd[135780]: -- 172.21.15.107:0/1611446366 <== mon.0 v2:172.21.15.107:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5647fa56d680 con 0x5647fa537000 2024-03-04T13:50:25.809 INFO:journalctl@ceph.osd.2.smithi107.stdout:Mar 04 13:50:25 smithi107 bash[135738]: 6e4da4f6f146276914ee5dbafa3f0c124a08bea50684b074bf446e26f95314dd 2024-03-04T13:50:25.809 INFO:journalctl@ceph.osd.2.smithi107.stdout:Mar 04 13:50:25 smithi107 systemd[1]: Started Ceph osd.2 for 1fc09a58-da2b-11ee-95c6-87774f69a715. 2024-03-04T13:50:25.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:25.807+0000 7f16c8000700 1 -- 172.21.15.107:0/184143394 shutdown_connections 2024-03-04T13:50:25.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:25.807+0000 7f16c8000700 1 -- 172.21.15.107:0/184143394 wait complete. 2024-03-04T13:50:25.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:25.808+0000 7f16c8000700 1 Processor -- start 2024-03-04T13:50:25.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:25.809+0000 7f16c8000700 1 -- start start 2024-03-04T13:50:25.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:25.809+0000 7f16c8000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f16c0105170 0x7f16c0115de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:50:25.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:25.809+0000 7f16c8000700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16c0105b10 0x7f16c0116320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:50:25.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:25.809+0000 7f16c8000700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c0116a00 con 0x7f16c0105170 2024-03-04T13:50:25.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:25.809+0000 7f16c8000700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c010fc80 con 0x7f16c0105b10 2024-03-04T13:50:25.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:25.810+0000 7f16c5600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f16c0105170 0x7f16c0115de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:50:25.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:25.810+0000 7f16c5600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f16c0105170 0x7f16c0115de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54742/0 (socket says 172.21.15.107:54742) 2024-03-04T13:50:25.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:25.810+0000 7f16c5600700 1 -- 172.21.15.107:0/2093758852 learned_addr learned my addr 172.21.15.107:0/2093758852 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:50:25.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:25.810+0000 7f16c4c00700 1 --2- 172.21.15.107:0/2093758852 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16c0105b10 0x7f16c0116320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:50:25.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:25.810+0000 7f16c5600700 1 -- 172.21.15.107:0/2093758852 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16c0105b10 msgr2=0x7f16c0116320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:50:25.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:25.810+0000 7f16c5600700 1 --2- 172.21.15.107:0/2093758852 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16c0105b10 0x7f16c0116320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:50:25.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:25.810+0000 7f16c5600700 1 -- 172.21.15.107:0/2093758852 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16ac005040 con 0x7f16c0105170 2024-03-04T13:50:25.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:25.810+0000 7f16c5600700 1 --2- 172.21.15.107:0/2093758852 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f16c0105170 0x7f16c0115de0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f16b400b790 tx=0x7f16b4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:50:25.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:25.811+0000 7f16bd600700 1 -- 172.21.15.107:0/2093758852 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f16b400f770 con 0x7f16c0105170 2024-03-04T13:50:25.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:25.811+0000 7f16bd600700 1 -- 172.21.15.107:0/2093758852 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f16b400fda0 con 0x7f16c0105170 2024-03-04T13:50:25.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:25.811+0000 7f16c8000700 1 -- 172.21.15.107:0/2093758852 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16c010ff00 con 0x7f16c0105170 2024-03-04T13:50:25.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:25.811+0000 7f16bd600700 1 -- 172.21.15.107:0/2093758852 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f16b400d310 con 0x7f16c0105170 2024-03-04T13:50:25.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:25.811+0000 7f16c8000700 1 -- 172.21.15.107:0/2093758852 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16c0110440 con 0x7f16c0105170 2024-03-04T13:50:25.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:25.812+0000 7f16c8000700 1 -- 172.21.15.107:0/2093758852 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16a00052e0 con 0x7f16c0105170 2024-03-04T13:50:25.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:25.813+0000 7f16bd600700 1 -- 172.21.15.107:0/2093758852 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f16b400f8d0 con 0x7f16c0105170 2024-03-04T13:50:25.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:25.814+0000 7f16bd600700 1 --2- 172.21.15.107:0/2093758852 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f16a806d7f0 0x7f16a806fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:50:25.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:25.814+0000 7f16bd600700 1 -- 172.21.15.107:0/2093758852 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f16b408ce70 con 0x7f16c0105170 2024-03-04T13:50:25.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:25.814+0000 7f16c4c00700 1 --2- 172.21.15.107:0/2093758852 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f16a806d7f0 0x7f16a806fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:50:25.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:25.815+0000 7f16c4c00700 1 --2- 172.21.15.107:0/2093758852 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f16a806d7f0 0x7f16a806fca0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f16ac00b020 tx=0x7f16ac009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:50:25.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:25.821+0000 7f16bd600700 1 -- 172.21.15.107:0/2093758852 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f16b405a220 con 0x7f16c0105170 2024-03-04T13:50:26.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:25 smithi169 ceph-mon[86183]: pgmap v105: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 255 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 255 B/s rd, 0 op/s; 34/264 objects degraded (12.879%) 2024-03-04T13:50:26.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:25 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:50:26.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:25 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:50:26.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:25 smithi169 ceph-mon[86183]: from='client.34278 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:50:26.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:26.145+0000 7f16c8000700 1 -- 172.21.15.107:0/2093758852 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f16a0005da0 con 0x7f16c0105170 2024-03-04T13:50:26.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:26.146+0000 7f16bd600700 1 -- 172.21.15.107:0/2093758852 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+839 (secure 0 0 0) 0x7f16b4059db0 con 0x7f16c0105170 2024-03-04T13:50:26.148 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-03-04T13:50:26.148 INFO:teuthology.orchestra.run.smithi107.stdout: "mon": { 2024-03-04T13:50:26.148 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-1845-g717ce3cc (717ce3cce8d0166e4e577faa8004e6f9cb4128c0) squid (dev)": 2 2024-03-04T13:50:26.148 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:50:26.148 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr": { 2024-03-04T13:50:26.148 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-1845-g717ce3cc (717ce3cce8d0166e4e577faa8004e6f9cb4128c0) squid (dev)": 2 2024-03-04T13:50:26.148 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:50:26.148 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": { 2024-03-04T13:50:26.148 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-545-g42608bb1 (42608bb191117be10dba720e6bf8a02f3171cd34) quincy (stable)": 3, 2024-03-04T13:50:26.148 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-1845-g717ce3cc (717ce3cce8d0166e4e577faa8004e6f9cb4128c0) squid (dev)": 2 2024-03-04T13:50:26.148 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:50:26.148 INFO:teuthology.orchestra.run.smithi107.stdout: "mds": { 2024-03-04T13:50:26.148 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-545-g42608bb1 (42608bb191117be10dba720e6bf8a02f3171cd34) quincy (stable)": 4 2024-03-04T13:50:26.149 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:50:26.149 INFO:teuthology.orchestra.run.smithi107.stdout: "overall": { 2024-03-04T13:50:26.149 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-545-g42608bb1 (42608bb191117be10dba720e6bf8a02f3171cd34) quincy (stable)": 7, 2024-03-04T13:50:26.149 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-1845-g717ce3cc (717ce3cce8d0166e4e577faa8004e6f9cb4128c0) squid (dev)": 6 2024-03-04T13:50:26.149 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-04T13:50:26.149 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-03-04T13:50:26.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:26.149+0000 7f16c8000700 1 -- 172.21.15.107:0/2093758852 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f16a806d7f0 msgr2=0x7f16a806fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:50:26.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:26.149+0000 7f16c8000700 1 --2- 172.21.15.107:0/2093758852 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f16a806d7f0 0x7f16a806fca0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f16ac00b020 tx=0x7f16ac009e30 comp rx=0 tx=0).stop 2024-03-04T13:50:26.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:26.149+0000 7f16c8000700 1 -- 172.21.15.107:0/2093758852 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f16c0105170 msgr2=0x7f16c0115de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:50:26.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:26.149+0000 7f16c8000700 1 --2- 172.21.15.107:0/2093758852 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f16c0105170 0x7f16c0115de0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f16b400b790 tx=0x7f16b4009250 comp rx=0 tx=0).stop 2024-03-04T13:50:26.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:26.149+0000 7f16c8000700 1 -- 172.21.15.107:0/2093758852 shutdown_connections 2024-03-04T13:50:26.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:26.149+0000 7f16c8000700 1 --2- 172.21.15.107:0/2093758852 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f16a806d7f0 0x7f16a806fca0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:50:26.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:26.149+0000 7f16c8000700 1 --2- 172.21.15.107:0/2093758852 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f16c0105170 0x7f16c0115de0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:50:26.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:26.149+0000 7f16c8000700 1 --2- 172.21.15.107:0/2093758852 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16c0105b10 0x7f16c0116320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:50:26.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:26.149+0000 7f16c8000700 1 -- 172.21.15.107:0/2093758852 >> 172.21.15.107:0/2093758852 conn(0x7f16c00face0 msgr2=0x7f16c00fc710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:50:26.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:26.150+0000 7f16c8000700 1 -- 172.21.15.107:0/2093758852 shutdown_connections 2024-03-04T13:50:26.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:26.150+0000 7f16c8000700 1 -- 172.21.15.107:0/2093758852 wait complete. 2024-03-04T13:50:26.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:26.295+0000 7f2fabe00700 1 -- 172.21.15.107:0/4197698348 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2fa40fc6e0 msgr2=0x7f2fa4109440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:50:26.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:26.295+0000 7f2fabe00700 1 --2- 172.21.15.107:0/4197698348 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2fa40fc6e0 0x7f2fa4109440 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2f90004940 tx=0x7f2f90009e30 comp rx=0 tx=0).stop 2024-03-04T13:50:26.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:26.295+0000 7f2fabe00700 1 -- 172.21.15.107:0/4197698348 shutdown_connections 2024-03-04T13:50:26.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:26.295+0000 7f2fabe00700 1 --2- 172.21.15.107:0/4197698348 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2fa40fc6e0 0x7f2fa4109440 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:50:26.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:26.295+0000 7f2fabe00700 1 --2- 172.21.15.107:0/4197698348 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2fa40fbdd0 0x7f2fa40fc1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:50:26.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:26.295+0000 7f2fabe00700 1 -- 172.21.15.107:0/4197698348 >> 172.21.15.107:0/4197698348 conn(0x7f2fa4071d90 msgr2=0x7f2fa4072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:50:26.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:26.295+0000 7f2fabe00700 1 -- 172.21.15.107:0/4197698348 shutdown_connections 2024-03-04T13:50:26.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:26.295+0000 7f2fabe00700 1 -- 172.21.15.107:0/4197698348 wait complete. 2024-03-04T13:50:26.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:26.296+0000 7f2fabe00700 1 Processor -- start 2024-03-04T13:50:26.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:26.296+0000 7f2fabe00700 1 -- start start 2024-03-04T13:50:26.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:26.296+0000 7f2fabe00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2fa40fbdd0 0x7f2fa4115d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:50:26.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:26.296+0000 7f2fabe00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2fa40fc6e0 0x7f2fa4116280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:50:26.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:26.296+0000 7f2fabe00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa41168d0 con 0x7f2fa40fbdd0 2024-03-04T13:50:26.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:26.296+0000 7f2fabe00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa410fbe0 con 0x7f2fa40fc6e0 2024-03-04T13:50:26.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:26.297+0000 7f2fa9400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2fa40fbdd0 0x7f2fa4115d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:50:26.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:26.297+0000 7f2fa9400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2fa40fbdd0 0x7f2fa4115d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54752/0 (socket says 172.21.15.107:54752) 2024-03-04T13:50:26.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:26.297+0000 7f2fa9400700 1 -- 172.21.15.107:0/920972970 learned_addr learned my addr 172.21.15.107:0/920972970 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:50:26.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:26.297+0000 7f2fa8a00700 1 --2- 172.21.15.107:0/920972970 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2fa40fc6e0 0x7f2fa4116280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:50:26.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:26.297+0000 7f2fa9400700 1 -- 172.21.15.107:0/920972970 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2fa40fc6e0 msgr2=0x7f2fa4116280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:50:26.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:26.297+0000 7f2fa9400700 1 --2- 172.21.15.107:0/920972970 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2fa40fc6e0 0x7f2fa4116280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:50:26.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:26.297+0000 7f2fa9400700 1 -- 172.21.15.107:0/920972970 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f90005040 con 0x7f2fa40fbdd0 2024-03-04T13:50:26.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:26.297+0000 7f2fa8a00700 1 --2- 172.21.15.107:0/920972970 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2fa40fc6e0 0x7f2fa4116280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T13:50:26.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:26.297+0000 7f2fa9400700 1 --2- 172.21.15.107:0/920972970 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2fa40fbdd0 0x7f2fa4115d40 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2f98002a50 tx=0x7f2f980046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:50:26.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:26.298+0000 7f2fa1600700 1 -- 172.21.15.107:0/920972970 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2f98007cc0 con 0x7f2fa40fbdd0 2024-03-04T13:50:26.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:26.298+0000 7f2fa1600700 1 -- 172.21.15.107:0/920972970 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2f9800f040 con 0x7f2fa40fbdd0 2024-03-04T13:50:26.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:26.298+0000 7f2fabe00700 1 -- 172.21.15.107:0/920972970 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fa410fec0 con 0x7f2fa40fbdd0 2024-03-04T13:50:26.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:26.298+0000 7f2fa1600700 1 -- 172.21.15.107:0/920972970 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2f98009640 con 0x7f2fa40fbdd0 2024-03-04T13:50:26.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:26.298+0000 7f2fabe00700 1 -- 172.21.15.107:0/920972970 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fa4110400 con 0x7f2fa40fbdd0 2024-03-04T13:50:26.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:26.299+0000 7f2fabe00700 1 -- 172.21.15.107:0/920972970 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fa4106b70 con 0x7f2fa40fbdd0 2024-03-04T13:50:26.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:26.300+0000 7f2fa1600700 1 -- 172.21.15.107:0/920972970 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f2f980097a0 con 0x7f2fa40fbdd0 2024-03-04T13:50:26.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:26.301+0000 7f2fa1600700 1 --2- 172.21.15.107:0/920972970 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f2f8c06d7f0 0x7f2f8c06fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:50:26.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:26.301+0000 7f2fa1600700 1 -- 172.21.15.107:0/920972970 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f2f9808cc10 con 0x7f2fa40fbdd0 2024-03-04T13:50:26.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:26.301+0000 7f2fa8a00700 1 --2- 172.21.15.107:0/920972970 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f2f8c06d7f0 0x7f2f8c06fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:50:26.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:26.302+0000 7f2fa8a00700 1 --2- 172.21.15.107:0/920972970 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f2f8c06d7f0 0x7f2f8c06fca0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2f9000b3b0 tx=0x7f2f90009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:50:26.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:26.306+0000 7f2fa1600700 1 -- 172.21.15.107:0/920972970 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2f98059fc0 con 0x7f2fa40fbdd0 2024-03-04T13:50:26.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:26.582+0000 7f2fabe00700 1 -- 172.21.15.107:0/920972970 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f2fa404bed0 con 0x7f2fa40fbdd0 2024-03-04T13:50:26.584 INFO:journalctl@ceph.osd.2.smithi107.stdout:Mar 04 13:50:26 smithi107 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-2[135773]: 2024-03-04T13:50:26.127+0000 7f60794f17c0 -1 Falling back to public interface 2024-03-04T13:50:26.584 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:26.583+0000 7f2fa1600700 1 -- 172.21.15.107:0/920972970 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 16 v16) v1 ==== 76+0+1880 (secure 0 0 0) 0x7f2f98059b50 con 0x7f2fa40fbdd0 2024-03-04T13:50:26.584 INFO:teuthology.orchestra.run.smithi107.stdout:e16 2024-03-04T13:50:26.584 INFO:teuthology.orchestra.run.smithi107.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-04T13:50:26.584 INFO:teuthology.orchestra.run.smithi107.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-04T13:50:26.584 INFO:teuthology.orchestra.run.smithi107.stdout:legacy client fscid: 1 2024-03-04T13:50:26.585 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:50:26.585 INFO:teuthology.orchestra.run.smithi107.stdout:Filesystem 'cephfs' (1) 2024-03-04T13:50:26.585 INFO:teuthology.orchestra.run.smithi107.stdout:fs_name cephfs 2024-03-04T13:50:26.585 INFO:teuthology.orchestra.run.smithi107.stdout:epoch 15 2024-03-04T13:50:26.585 INFO:teuthology.orchestra.run.smithi107.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-04T13:50:26.585 INFO:teuthology.orchestra.run.smithi107.stdout:created 2024-03-04T13:38:30.854004+0000 2024-03-04T13:50:26.585 INFO:teuthology.orchestra.run.smithi107.stdout:modified 2024-03-04T13:38:50.015989+0000 2024-03-04T13:50:26.585 INFO:teuthology.orchestra.run.smithi107.stdout:tableserver 0 2024-03-04T13:50:26.585 INFO:teuthology.orchestra.run.smithi107.stdout:root 0 2024-03-04T13:50:26.585 INFO:teuthology.orchestra.run.smithi107.stdout:session_timeout 60 2024-03-04T13:50:26.585 INFO:teuthology.orchestra.run.smithi107.stdout:session_autoclose 300 2024-03-04T13:50:26.585 INFO:teuthology.orchestra.run.smithi107.stdout:max_file_size 1099511627776 2024-03-04T13:50:26.585 INFO:teuthology.orchestra.run.smithi107.stdout:max_xattr_size 65536 2024-03-04T13:50:26.585 INFO:teuthology.orchestra.run.smithi107.stdout:required_client_features {} 2024-03-04T13:50:26.585 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure 0 2024-03-04T13:50:26.586 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure_osd_epoch 47 2024-03-04T13:50:26.586 INFO:teuthology.orchestra.run.smithi107.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-04T13:50:26.586 INFO:teuthology.orchestra.run.smithi107.stdout:max_mds 1 2024-03-04T13:50:26.586 INFO:teuthology.orchestra.run.smithi107.stdout:in 0 2024-03-04T13:50:26.586 INFO:teuthology.orchestra.run.smithi107.stdout:up {0=24247} 2024-03-04T13:50:26.586 INFO:teuthology.orchestra.run.smithi107.stdout:failed 2024-03-04T13:50:26.586 INFO:teuthology.orchestra.run.smithi107.stdout:damaged 2024-03-04T13:50:26.586 INFO:teuthology.orchestra.run.smithi107.stdout:stopped 2024-03-04T13:50:26.586 INFO:teuthology.orchestra.run.smithi107.stdout:data_pools [3] 2024-03-04T13:50:26.586 INFO:teuthology.orchestra.run.smithi107.stdout:metadata_pool 2 2024-03-04T13:50:26.586 INFO:teuthology.orchestra.run.smithi107.stdout:inline_data enabled 2024-03-04T13:50:26.586 INFO:teuthology.orchestra.run.smithi107.stdout:balancer 2024-03-04T13:50:26.586 INFO:teuthology.orchestra.run.smithi107.stdout:bal_rank_mask -1 2024-03-04T13:50:26.586 INFO:teuthology.orchestra.run.smithi107.stdout:standby_count_wanted 1 2024-03-04T13:50:26.587 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi169.jusudl{0:24247} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.169:6824/2994866076,v1:172.21.15.169:6825/2994866076] compat {c=[1],r=[1],i=[7ff]}] 2024-03-04T13:50:26.587 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.thycho{0:14470} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.107:6828/1985577060,v1:172.21.15.107:6829/1985577060] compat {c=[1],r=[1],i=[7ff]}] 2024-03-04T13:50:26.587 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:50:26.587 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:50:26.587 INFO:teuthology.orchestra.run.smithi107.stdout:Standby daemons: 2024-03-04T13:50:26.587 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:50:26.587 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.riexmm{-1:14480} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.107:6826/3261622517,v1:172.21.15.107:6827/3261622517] compat {c=[1],r=[1],i=[7ff]}] 2024-03-04T13:50:26.587 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi169.bxitzk{-1:24259} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.169:6826/3696090493,v1:172.21.15.169:6827/3696090493] compat {c=[1],r=[1],i=[7ff]}] 2024-03-04T13:50:26.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:26.587+0000 7f2fabe00700 1 -- 172.21.15.107:0/920972970 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f2f8c06d7f0 msgr2=0x7f2f8c06fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:50:26.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:26.587+0000 7f2fabe00700 1 --2- 172.21.15.107:0/920972970 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f2f8c06d7f0 0x7f2f8c06fca0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2f9000b3b0 tx=0x7f2f90009e30 comp rx=0 tx=0).stop 2024-03-04T13:50:26.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:26.587+0000 7f2fabe00700 1 -- 172.21.15.107:0/920972970 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2fa40fbdd0 msgr2=0x7f2fa4115d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:50:26.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:26.587+0000 7f2fabe00700 1 --2- 172.21.15.107:0/920972970 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2fa40fbdd0 0x7f2fa4115d40 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2f98002a50 tx=0x7f2f980046b0 comp rx=0 tx=0).stop 2024-03-04T13:50:26.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:26.587+0000 7f2fabe00700 1 -- 172.21.15.107:0/920972970 shutdown_connections 2024-03-04T13:50:26.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:26.587+0000 7f2fabe00700 1 --2- 172.21.15.107:0/920972970 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f2f8c06d7f0 0x7f2f8c06fca0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:50:26.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:26.587+0000 7f2fabe00700 1 --2- 172.21.15.107:0/920972970 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2fa40fbdd0 0x7f2fa4115d40 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:50:26.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:26.587+0000 7f2fabe00700 1 --2- 172.21.15.107:0/920972970 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2fa40fc6e0 0x7f2fa4116280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:50:26.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:26.587+0000 7f2fabe00700 1 -- 172.21.15.107:0/920972970 >> 172.21.15.107:0/920972970 conn(0x7f2fa4071d90 msgr2=0x7f2fa40fa9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:50:26.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:26.587+0000 7f2fabe00700 1 -- 172.21.15.107:0/920972970 shutdown_connections 2024-03-04T13:50:26.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:26.587+0000 7f2fabe00700 1 -- 172.21.15.107:0/920972970 wait complete. 2024-03-04T13:50:26.590 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 16 2024-03-04T13:50:26.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:26.738+0000 7f4158600700 1 -- 172.21.15.107:0/2124275787 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f41501038f0 msgr2=0x7f41500fcf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:50:26.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:26.738+0000 7f4158600700 1 --2- 172.21.15.107:0/2124275787 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f41501038f0 0x7f41500fcf60 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f413c004940 tx=0x7f413c009e30 comp rx=0 tx=0).stop 2024-03-04T13:50:26.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:26.739+0000 7f4158600700 1 -- 172.21.15.107:0/2124275787 shutdown_connections 2024-03-04T13:50:26.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:26.739+0000 7f4158600700 1 --2- 172.21.15.107:0/2124275787 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f41501038f0 0x7f41500fcf60 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:50:26.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:26.739+0000 7f4158600700 1 --2- 172.21.15.107:0/2124275787 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4150102f50 0x7f4150103320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:50:26.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:26.739+0000 7f4158600700 1 -- 172.21.15.107:0/2124275787 >> 172.21.15.107:0/2124275787 conn(0x7f41500f8ae0 msgr2=0x7f41500faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:50:26.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:26.739+0000 7f4158600700 1 -- 172.21.15.107:0/2124275787 shutdown_connections 2024-03-04T13:50:26.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:26.739+0000 7f4158600700 1 -- 172.21.15.107:0/2124275787 wait complete. 2024-03-04T13:50:26.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:26.740+0000 7f4158600700 1 Processor -- start 2024-03-04T13:50:26.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:26.740+0000 7f4158600700 1 -- start start 2024-03-04T13:50:26.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:26.740+0000 7f4158600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4150102f50 0x7f4150115c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:50:26.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:26.740+0000 7f4158600700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f41501038f0 0x7f41501161c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:50:26.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:26.740+0000 7f4158600700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4150116810 con 0x7f4150102f50 2024-03-04T13:50:26.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:26.740+0000 7f4158600700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f415010fb40 con 0x7f41501038f0 2024-03-04T13:50:26.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:26.741+0000 7f4155c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4150102f50 0x7f4150115c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:50:26.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:26.741+0000 7f4155c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4150102f50 0x7f4150115c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54762/0 (socket says 172.21.15.107:54762) 2024-03-04T13:50:26.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:26.741+0000 7f4155c00700 1 -- 172.21.15.107:0/1956764669 learned_addr learned my addr 172.21.15.107:0/1956764669 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:50:26.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:26.741+0000 7f4155200700 1 --2- 172.21.15.107:0/1956764669 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f41501038f0 0x7f41501161c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:50:26.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:26.741+0000 7f4155c00700 1 -- 172.21.15.107:0/1956764669 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f41501038f0 msgr2=0x7f41501161c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:50:26.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:26.741+0000 7f4155c00700 1 --2- 172.21.15.107:0/1956764669 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f41501038f0 0x7f41501161c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:50:26.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:26.741+0000 7f4155c00700 1 -- 172.21.15.107:0/1956764669 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f413c005040 con 0x7f4150102f50 2024-03-04T13:50:26.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:26.741+0000 7f4155200700 1 --2- 172.21.15.107:0/1956764669 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f41501038f0 0x7f41501161c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T13:50:26.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:26.741+0000 7f4155c00700 1 --2- 172.21.15.107:0/1956764669 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4150102f50 0x7f4150115c80 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4144002a50 tx=0x7f41440046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:50:26.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:26.742+0000 7f414d600700 1 -- 172.21.15.107:0/1956764669 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4144007cc0 con 0x7f4150102f50 2024-03-04T13:50:26.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:26.742+0000 7f414d600700 1 -- 172.21.15.107:0/1956764669 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f414400f040 con 0x7f4150102f50 2024-03-04T13:50:26.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:26.742+0000 7f4158600700 1 -- 172.21.15.107:0/1956764669 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f415010fe20 con 0x7f4150102f50 2024-03-04T13:50:26.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:26.742+0000 7f414d600700 1 -- 172.21.15.107:0/1956764669 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4144009640 con 0x7f4150102f50 2024-03-04T13:50:26.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:26.742+0000 7f4158600700 1 -- 172.21.15.107:0/1956764669 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4150110360 con 0x7f4150102f50 2024-03-04T13:50:26.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:26.743+0000 7f4158600700 1 -- 172.21.15.107:0/1956764669 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41500fc6e0 con 0x7f4150102f50 2024-03-04T13:50:26.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:26.744+0000 7f414d600700 1 -- 172.21.15.107:0/1956764669 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f41440097a0 con 0x7f4150102f50 2024-03-04T13:50:26.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:26.745+0000 7f414d600700 1 --2- 172.21.15.107:0/1956764669 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f413806d7f0 0x7f413806fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:50:26.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:26.745+0000 7f414d600700 1 -- 172.21.15.107:0/1956764669 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f414408cc10 con 0x7f4150102f50 2024-03-04T13:50:26.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:26.745+0000 7f4155200700 1 --2- 172.21.15.107:0/1956764669 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f413806d7f0 0x7f413806fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:50:26.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:26.746+0000 7f4155200700 1 --2- 172.21.15.107:0/1956764669 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f413806d7f0 0x7f413806fca0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f413c00b3b0 tx=0x7f413c009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:50:26.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:26.750+0000 7f414d600700 1 -- 172.21.15.107:0/1956764669 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4144059fc0 con 0x7f4150102f50 2024-03-04T13:50:26.843 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:26 smithi107 ceph-mon[112578]: from='client.34282 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:50:26.844 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:26 smithi107 ceph-mon[112578]: from='client.34286 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:50:26.844 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:26 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:50:26.844 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:26 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:50:26.844 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:26 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:50:26.844 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:26 smithi107 ceph-mon[112578]: from='client.? 172.21.15.107:0/2093758852' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:50:26.844 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:26 smithi107 ceph-mon[112578]: from='client.? 172.21.15.107:0/920972970' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-04T13:50:26.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:26.994+0000 7f4158600700 1 -- 172.21.15.107:0/1956764669 --> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4150111290 con 0x7f413806d7f0 2024-03-04T13:50:27.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:27.001+0000 7f414d600700 1 -- 172.21.15.107:0/1956764669 <== mgr.34102 v2:172.21.15.107:6800/1223039812 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f4150111290 con 0x7f413806d7f0 2024-03-04T13:50:27.003 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-03-04T13:50:27.003 INFO:teuthology.orchestra.run.smithi107.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377", 2024-03-04T13:50:27.003 INFO:teuthology.orchestra.run.smithi107.stdout: "in_progress": true, 2024-03-04T13:50:27.003 INFO:teuthology.orchestra.run.smithi107.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-04T13:50:27.003 INFO:teuthology.orchestra.run.smithi107.stdout: "services_complete": [ 2024-03-04T13:50:27.003 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr", 2024-03-04T13:50:27.003 INFO:teuthology.orchestra.run.smithi107.stdout: "mon", 2024-03-04T13:50:27.003 INFO:teuthology.orchestra.run.smithi107.stdout: "crash" 2024-03-04T13:50:27.003 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-04T13:50:27.003 INFO:teuthology.orchestra.run.smithi107.stdout: "progress": "8/21 daemons upgraded", 2024-03-04T13:50:27.003 INFO:teuthology.orchestra.run.smithi107.stdout: "message": "Currently upgrading osd daemons", 2024-03-04T13:50:27.003 INFO:teuthology.orchestra.run.smithi107.stdout: "is_paused": false 2024-03-04T13:50:27.003 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-03-04T13:50:27.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:27.004+0000 7f4158600700 1 -- 172.21.15.107:0/1956764669 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f413806d7f0 msgr2=0x7f413806fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:50:27.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:27.004+0000 7f4158600700 1 --2- 172.21.15.107:0/1956764669 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f413806d7f0 0x7f413806fca0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f413c00b3b0 tx=0x7f413c009e30 comp rx=0 tx=0).stop 2024-03-04T13:50:27.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:27.004+0000 7f4158600700 1 -- 172.21.15.107:0/1956764669 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4150102f50 msgr2=0x7f4150115c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:50:27.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:27.004+0000 7f4158600700 1 --2- 172.21.15.107:0/1956764669 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4150102f50 0x7f4150115c80 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4144002a50 tx=0x7f41440046b0 comp rx=0 tx=0).stop 2024-03-04T13:50:27.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:27.004+0000 7f4158600700 1 -- 172.21.15.107:0/1956764669 shutdown_connections 2024-03-04T13:50:27.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:27.004+0000 7f4158600700 1 --2- 172.21.15.107:0/1956764669 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f413806d7f0 0x7f413806fca0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:50:27.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:27.005+0000 7f4158600700 1 --2- 172.21.15.107:0/1956764669 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4150102f50 0x7f4150115c80 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:50:27.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:27.005+0000 7f4158600700 1 --2- 172.21.15.107:0/1956764669 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f41501038f0 0x7f41501161c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:50:27.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:27.005+0000 7f4158600700 1 -- 172.21.15.107:0/1956764669 >> 172.21.15.107:0/1956764669 conn(0x7f41500f8ae0 msgr2=0x7f41500fa470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:50:27.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:27.005+0000 7f4158600700 1 -- 172.21.15.107:0/1956764669 shutdown_connections 2024-03-04T13:50:27.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:27.005+0000 7f4158600700 1 -- 172.21.15.107:0/1956764669 wait complete. 2024-03-04T13:50:27.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:26 smithi169 ceph-mon[86183]: from='client.34282 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:50:27.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:26 smithi169 ceph-mon[86183]: from='client.34286 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:50:27.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:26 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:50:27.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:26 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:50:27.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:26 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:50:27.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:26 smithi169 ceph-mon[86183]: from='client.? 172.21.15.107:0/2093758852' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:50:27.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:26 smithi169 ceph-mon[86183]: from='client.? 172.21.15.107:0/920972970' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-04T13:50:27.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:27.149+0000 7f1a95800700 1 -- 172.21.15.107:0/2260937276 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a901038f0 msgr2=0x7f1a900fcf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:50:27.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:27.149+0000 7f1a95800700 1 --2- 172.21.15.107:0/2260937276 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a901038f0 0x7f1a900fcf60 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1a7c004940 tx=0x7f1a7c009e30 comp rx=0 tx=0).stop 2024-03-04T13:50:27.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:27.149+0000 7f1a95800700 1 -- 172.21.15.107:0/2260937276 shutdown_connections 2024-03-04T13:50:27.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:27.149+0000 7f1a95800700 1 --2- 172.21.15.107:0/2260937276 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a901038f0 0x7f1a900fcf60 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:50:27.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:27.149+0000 7f1a95800700 1 --2- 172.21.15.107:0/2260937276 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a90102f50 0x7f1a90103320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:50:27.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:27.149+0000 7f1a95800700 1 -- 172.21.15.107:0/2260937276 >> 172.21.15.107:0/2260937276 conn(0x7f1a900f8ae0 msgr2=0x7f1a900faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:50:27.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:27.149+0000 7f1a95800700 1 -- 172.21.15.107:0/2260937276 shutdown_connections 2024-03-04T13:50:27.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:27.150+0000 7f1a95800700 1 -- 172.21.15.107:0/2260937276 wait complete. 2024-03-04T13:50:27.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:27.150+0000 7f1a95800700 1 Processor -- start 2024-03-04T13:50:27.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:27.150+0000 7f1a95800700 1 -- start start 2024-03-04T13:50:27.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:27.151+0000 7f1a95800700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a90102f50 0x7f1a90115cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:50:27.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:27.151+0000 7f1a95800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a901038f0 0x7f1a90116210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:50:27.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:27.151+0000 7f1a95800700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a901168f0 con 0x7f1a901038f0 2024-03-04T13:50:27.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:27.151+0000 7f1a95800700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a9010fb70 con 0x7f1a90102f50 2024-03-04T13:50:27.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:27.151+0000 7f1a8e000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a901038f0 0x7f1a90116210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:50:27.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:27.151+0000 7f1a8e000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a901038f0 0x7f1a90116210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54772/0 (socket says 172.21.15.107:54772) 2024-03-04T13:50:27.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:27.151+0000 7f1a8e000700 1 -- 172.21.15.107:0/775475315 learned_addr learned my addr 172.21.15.107:0/775475315 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:50:27.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:27.151+0000 7f1a8ea00700 1 --2- 172.21.15.107:0/775475315 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a90102f50 0x7f1a90115cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:50:27.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:27.152+0000 7f1a8e000700 1 -- 172.21.15.107:0/775475315 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a90102f50 msgr2=0x7f1a90115cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:50:27.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:27.152+0000 7f1a8e000700 1 --2- 172.21.15.107:0/775475315 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a90102f50 0x7f1a90115cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:50:27.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:27.152+0000 7f1a8e000700 1 -- 172.21.15.107:0/775475315 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a7c005040 con 0x7f1a901038f0 2024-03-04T13:50:27.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:27.152+0000 7f1a8e000700 1 --2- 172.21.15.107:0/775475315 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a901038f0 0x7f1a90116210 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1a7c009150 tx=0x7f1a7c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:50:27.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:27.152+0000 7f1a86a00700 1 -- 172.21.15.107:0/775475315 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a7c00e040 con 0x7f1a901038f0 2024-03-04T13:50:27.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:27.152+0000 7f1a86a00700 1 -- 172.21.15.107:0/775475315 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1a7c00bbf0 con 0x7f1a901038f0 2024-03-04T13:50:27.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:27.152+0000 7f1a95800700 1 -- 172.21.15.107:0/775475315 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a9010fd90 con 0x7f1a901038f0 2024-03-04T13:50:27.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:27.152+0000 7f1a86a00700 1 -- 172.21.15.107:0/775475315 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a7c01c630 con 0x7f1a901038f0 2024-03-04T13:50:27.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:27.152+0000 7f1a95800700 1 -- 172.21.15.107:0/775475315 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a901102d0 con 0x7f1a901038f0 2024-03-04T13:50:27.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:27.154+0000 7f1a95800700 1 -- 172.21.15.107:0/775475315 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a900fc6e0 con 0x7f1a901038f0 2024-03-04T13:50:27.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:27.155+0000 7f1a86a00700 1 -- 172.21.15.107:0/775475315 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f1a7c00bd60 con 0x7f1a901038f0 2024-03-04T13:50:27.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:27.156+0000 7f1a86a00700 1 --2- 172.21.15.107:0/775475315 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f1a7406d7e0 0x7f1a7406fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:50:27.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:27.156+0000 7f1a86a00700 1 -- 172.21.15.107:0/775475315 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f1a7c08e300 con 0x7f1a901038f0 2024-03-04T13:50:27.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:27.156+0000 7f1a8ea00700 1 --2- 172.21.15.107:0/775475315 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f1a7406d7e0 0x7f1a7406fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:50:27.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:27.157+0000 7f1a8ea00700 1 --2- 172.21.15.107:0/775475315 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f1a7406d7e0 0x7f1a7406fc90 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1a80006fd0 tx=0x7f1a80008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:50:27.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:27.161+0000 7f1a86a00700 1 -- 172.21.15.107:0/775475315 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1a7c05b6b0 con 0x7f1a901038f0 2024-03-04T13:50:27.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:27.468+0000 7f1a95800700 1 -- 172.21.15.107:0/775475315 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f1a9010a0a0 con 0x7f1a901038f0 2024-03-04T13:50:27.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:27.469+0000 7f1a86a00700 1 -- 172.21.15.107:0/775475315 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+1130 (secure 0 0 0) 0x7f1a7c05b240 con 0x7f1a901038f0 2024-03-04T13:50:27.471 INFO:teuthology.orchestra.run.smithi107.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data; 1 osds down; Degraded data redundancy: 34/264 objects degraded (12.879%), 12 pgs degraded 2024-03-04T13:50:27.471 INFO:teuthology.orchestra.run.smithi107.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-04T13:50:27.471 INFO:teuthology.orchestra.run.smithi107.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-04T13:50:27.471 INFO:teuthology.orchestra.run.smithi107.stdout:[WRN] OSD_DOWN: 1 osds down 2024-03-04T13:50:27.471 INFO:teuthology.orchestra.run.smithi107.stdout: osd.2 (root=default,host=smithi107) is down 2024-03-04T13:50:27.471 INFO:teuthology.orchestra.run.smithi107.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 34/264 objects degraded (12.879%), 12 pgs degraded 2024-03-04T13:50:27.471 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.1 is active+undersized+degraded, acting [1,0] 2024-03-04T13:50:27.471 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.3 is active+undersized+degraded, acting [5,1] 2024-03-04T13:50:27.471 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.7 is active+undersized+degraded, acting [3,4] 2024-03-04T13:50:27.471 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.c is active+undersized+degraded, acting [3,0] 2024-03-04T13:50:27.471 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.d is active+undersized+degraded, acting [1,3] 2024-03-04T13:50:27.471 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.e is active+undersized+degraded, acting [0,3] 2024-03-04T13:50:27.471 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.10 is active+undersized+degraded, acting [1,0] 2024-03-04T13:50:27.472 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.13 is active+undersized+degraded, acting [0,4] 2024-03-04T13:50:27.472 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.16 is active+undersized+degraded, acting [5,3] 2024-03-04T13:50:27.472 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.19 is active+undersized+degraded, acting [0,4] 2024-03-04T13:50:27.472 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.1c is active+undersized+degraded, acting [4,5] 2024-03-04T13:50:27.472 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.1e is active+undersized+degraded, acting [0,5] 2024-03-04T13:50:27.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:27.471+0000 7f1a95800700 1 -- 172.21.15.107:0/775475315 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f1a7406d7e0 msgr2=0x7f1a7406fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:50:27.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:27.471+0000 7f1a95800700 1 --2- 172.21.15.107:0/775475315 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f1a7406d7e0 0x7f1a7406fc90 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1a80006fd0 tx=0x7f1a80008040 comp rx=0 tx=0).stop 2024-03-04T13:50:27.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:27.471+0000 7f1a95800700 1 -- 172.21.15.107:0/775475315 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a901038f0 msgr2=0x7f1a90116210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:50:27.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:27.471+0000 7f1a95800700 1 --2- 172.21.15.107:0/775475315 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a901038f0 0x7f1a90116210 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1a7c009150 tx=0x7f1a7c009e30 comp rx=0 tx=0).stop 2024-03-04T13:50:27.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:27.472+0000 7f1a95800700 1 -- 172.21.15.107:0/775475315 shutdown_connections 2024-03-04T13:50:27.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:27.472+0000 7f1a95800700 1 --2- 172.21.15.107:0/775475315 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f1a7406d7e0 0x7f1a7406fc90 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:50:27.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:27.472+0000 7f1a95800700 1 --2- 172.21.15.107:0/775475315 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a90102f50 0x7f1a90115cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:50:27.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:27.472+0000 7f1a95800700 1 --2- 172.21.15.107:0/775475315 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a901038f0 0x7f1a90116210 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:50:27.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:27.472+0000 7f1a95800700 1 -- 172.21.15.107:0/775475315 >> 172.21.15.107:0/775475315 conn(0x7f1a900f8ae0 msgr2=0x7f1a900fa360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:50:27.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:27.472+0000 7f1a95800700 1 -- 172.21.15.107:0/775475315 shutdown_connections 2024-03-04T13:50:27.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:27.472+0000 7f1a95800700 1 -- 172.21.15.107:0/775475315 wait complete. 2024-03-04T13:50:28.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:27 smithi107 ceph-mon[112578]: pgmap v106: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 255 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 895 B/s rd, 1 op/s; 34/264 objects degraded (12.879%) 2024-03-04T13:50:28.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:27 smithi107 ceph-mon[112578]: from='client.? 172.21.15.107:0/775475315' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-04T13:50:28.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:27 smithi169 ceph-mon[86183]: pgmap v106: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 255 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 895 B/s rd, 1 op/s; 34/264 objects degraded (12.879%) 2024-03-04T13:50:28.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:27 smithi169 ceph-mon[86183]: from='client.? 172.21.15.107:0/775475315' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-04T13:50:29.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:28 smithi107 ceph-mon[112578]: from='client.34300 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:50:29.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:28 smithi169 ceph-mon[86183]: from='client.34300 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:50:30.095 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:29 smithi107 ceph-mon[112578]: pgmap v107: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 255 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 723 B/s rd, 1 op/s; 34/264 objects degraded (12.879%) 2024-03-04T13:50:30.095 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:29 smithi107 ceph-mon[112578]: Health check update: Degraded data redundancy: 34/264 objects degraded (12.879%), 12 pgs degraded (PG_DEGRADED) 2024-03-04T13:50:30.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:29 smithi169 ceph-mon[86183]: pgmap v107: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 255 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 723 B/s rd, 1 op/s; 34/264 objects degraded (12.879%) 2024-03-04T13:50:30.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:29 smithi169 ceph-mon[86183]: Health check update: Degraded data redundancy: 34/264 objects degraded (12.879%), 12 pgs degraded (PG_DEGRADED) 2024-03-04T13:50:30.978 INFO:journalctl@ceph.osd.2.smithi107.stdout:Mar 04 13:50:30 smithi107 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-2[135773]: 2024-03-04T13:50:30.620+0000 7f60794f17c0 -1 osd.2 0 read_superblock omap replica is missing. 2024-03-04T13:50:30.978 INFO:journalctl@ceph.osd.2.smithi107.stdout:Mar 04 13:50:30 smithi107 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-2[135773]: 2024-03-04T13:50:30.977+0000 7f60794f17c0 -1 osd.2 60 log_to_monitors true 2024-03-04T13:50:32.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:31 smithi107 ceph-mon[112578]: pgmap v108: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 255 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.1 KiB/s rd, 2 op/s; 34/264 objects degraded (12.879%) 2024-03-04T13:50:32.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:31 smithi107 ceph-mon[112578]: from='osd.2 [v2:172.21.15.107:6818/150828071,v1:172.21.15.107:6819/150828071]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-04T13:50:32.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:31 smithi169 ceph-mon[86183]: pgmap v108: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 255 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.1 KiB/s rd, 2 op/s; 34/264 objects degraded (12.879%) 2024-03-04T13:50:32.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:31 smithi169 ceph-mon[86183]: from='osd.2 [v2:172.21.15.107:6818/150828071,v1:172.21.15.107:6819/150828071]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-04T13:50:33.094 INFO:journalctl@ceph.osd.2.smithi107.stdout:Mar 04 13:50:32 smithi107 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-2[135773]: 2024-03-04T13:50:32.839+0000 7f6063800700 -1 osd.2 60 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-04T13:50:33.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:32 smithi107 ceph-mon[112578]: from='osd.2 [v2:172.21.15.107:6818/150828071,v1:172.21.15.107:6819/150828071]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-04T13:50:33.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:32 smithi107 ceph-mon[112578]: osdmap e63: 6 total, 5 up, 6 in 2024-03-04T13:50:33.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:32 smithi107 ceph-mon[112578]: from='osd.2 [v2:172.21.15.107:6818/150828071,v1:172.21.15.107:6819/150828071]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-03-04T13:50:33.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:32 smithi169 ceph-mon[86183]: from='osd.2 [v2:172.21.15.107:6818/150828071,v1:172.21.15.107:6819/150828071]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-04T13:50:33.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:32 smithi169 ceph-mon[86183]: osdmap e63: 6 total, 5 up, 6 in 2024-03-04T13:50:33.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:32 smithi169 ceph-mon[86183]: from='osd.2 [v2:172.21.15.107:6818/150828071,v1:172.21.15.107:6819/150828071]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-03-04T13:50:34.305 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:33 smithi107 ceph-mon[112578]: pgmap v110: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 255 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s; 34/264 objects degraded (12.879%) 2024-03-04T13:50:34.305 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:33 smithi107 ceph-mon[112578]: from='osd.2 [v2:172.21.15.107:6818/150828071,v1:172.21.15.107:6819/150828071]' entity='osd.2' 2024-03-04T13:50:34.305 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:33 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:50:34.305 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:33 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:50:34.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:33 smithi169 ceph-mon[86183]: pgmap v110: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 255 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s; 34/264 objects degraded (12.879%) 2024-03-04T13:50:34.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:33 smithi169 ceph-mon[86183]: from='osd.2 [v2:172.21.15.107:6818/150828071,v1:172.21.15.107:6819/150828071]' entity='osd.2' 2024-03-04T13:50:34.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:33 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:50:34.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:33 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:50:35.343 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:34 smithi107 ceph-mon[112578]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-04T13:50:35.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:34 smithi107 ceph-mon[112578]: osd.2 [v2:172.21.15.107:6818/150828071,v1:172.21.15.107:6819/150828071] boot 2024-03-04T13:50:35.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:34 smithi107 ceph-mon[112578]: osdmap e64: 6 total, 6 up, 6 in 2024-03-04T13:50:35.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:34 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-04T13:50:35.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:34 smithi107 ceph-mon[112578]: osdmap e65: 6 total, 6 up, 6 in 2024-03-04T13:50:35.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:34 smithi169 ceph-mon[86183]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-04T13:50:35.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:34 smithi169 ceph-mon[86183]: osd.2 [v2:172.21.15.107:6818/150828071,v1:172.21.15.107:6819/150828071] boot 2024-03-04T13:50:35.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:34 smithi169 ceph-mon[86183]: osdmap e64: 6 total, 6 up, 6 in 2024-03-04T13:50:35.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:34 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-04T13:50:35.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:34 smithi169 ceph-mon[86183]: osdmap e65: 6 total, 6 up, 6 in 2024-03-04T13:50:36.154 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:35 smithi107 ceph-mon[112578]: pgmap v112: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 255 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 34/264 objects degraded (12.879%) 2024-03-04T13:50:36.154 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:35 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:50:36.154 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:35 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:50:36.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:35 smithi169 ceph-mon[86183]: pgmap v112: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 255 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 34/264 objects degraded (12.879%) 2024-03-04T13:50:36.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:35 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:50:36.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:35 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:50:37.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:36 smithi107 ceph-mon[112578]: pgmap v114: 65 pgs: 5 active+undersized+degraded, 4 active+undersized, 56 active+clean; 255 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 18/264 objects degraded (6.818%) 2024-03-04T13:50:37.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:36 smithi107 ceph-mon[112578]: Health check update: Degraded data redundancy: 18/264 objects degraded (6.818%), 5 pgs degraded (PG_DEGRADED) 2024-03-04T13:50:37.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:36 smithi169 ceph-mon[86183]: pgmap v114: 65 pgs: 5 active+undersized+degraded, 4 active+undersized, 56 active+clean; 255 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 18/264 objects degraded (6.818%) 2024-03-04T13:50:37.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:36 smithi169 ceph-mon[86183]: Health check update: Degraded data redundancy: 18/264 objects degraded (6.818%), 5 pgs degraded (PG_DEGRADED) 2024-03-04T13:50:39.093 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:38 smithi107 ceph-mon[112578]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 18/264 objects degraded (6.818%), 5 pgs degraded) 2024-03-04T13:50:39.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:38 smithi169 ceph-mon[86183]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 18/264 objects degraded (6.818%), 5 pgs degraded) 2024-03-04T13:50:40.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:39 smithi107 ceph-mon[112578]: pgmap v115: 65 pgs: 65 active+clean; 255 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 745 B/s rd, 1 op/s 2024-03-04T13:50:40.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:39 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:50:40.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:39 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:50:40.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:39 smithi169 ceph-mon[86183]: pgmap v115: 65 pgs: 65 active+clean; 255 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 745 B/s rd, 1 op/s 2024-03-04T13:50:40.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:39 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:50:40.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:39 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:50:42.172 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:41 smithi169 ceph-mon[86183]: pgmap v116: 65 pgs: 65 active+clean; 255 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:50:42.172 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:41 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:50:42.173 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:41 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:50:42.173 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:41 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:50:42.173 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:41 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:50:42.173 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:41 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:50:42.173 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:41 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:50:42.173 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:41 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:50:42.173 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:41 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:50:42.173 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:41 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:50:42.173 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:41 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-04T13:50:42.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:41 smithi107 ceph-mon[112578]: pgmap v116: 65 pgs: 65 active+clean; 255 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:50:42.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:41 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:50:42.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:41 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:50:42.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:41 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:50:42.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:41 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:50:42.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:41 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:50:42.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:41 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:50:42.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:41 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:50:42.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:41 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:50:42.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:41 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:50:42.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:41 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-04T13:50:43.195 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:42 smithi169 ceph-mon[86183]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-04T13:50:43.195 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:42 smithi169 ceph-mon[86183]: Upgrade: osd.3 is safe to restart 2024-03-04T13:50:43.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:42 smithi107 ceph-mon[112578]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-04T13:50:43.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:42 smithi107 ceph-mon[112578]: Upgrade: osd.3 is safe to restart 2024-03-04T13:50:44.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:43 smithi107 ceph-mon[112578]: pgmap v117: 65 pgs: 65 active+clean; 255 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 2 op/s 2024-03-04T13:50:44.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:43 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:50:44.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:43 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-04T13:50:44.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:43 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:50:44.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:43 smithi169 ceph-mon[86183]: pgmap v117: 65 pgs: 65 active+clean; 255 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 2 op/s 2024-03-04T13:50:44.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:43 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:50:44.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:43 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-04T13:50:44.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:43 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:50:45.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:44 smithi169 ceph-mon[86183]: Upgrade: Updating osd.3 2024-03-04T13:50:45.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:44 smithi169 ceph-mon[86183]: Deploying daemon osd.3 on smithi169 2024-03-04T13:50:45.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:44 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:50:45.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:44 smithi107 ceph-mon[112578]: Upgrade: Updating osd.3 2024-03-04T13:50:45.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:44 smithi107 ceph-mon[112578]: Deploying daemon osd.3 on smithi169 2024-03-04T13:50:45.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:44 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:50:46.232 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:45 smithi169 ceph-mon[86183]: pgmap v118: 65 pgs: 65 active+clean; 255 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s 2024-03-04T13:50:46.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:45 smithi107 ceph-mon[112578]: pgmap v118: 65 pgs: 65 active+clean; 255 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s 2024-03-04T13:50:48.182 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:47 smithi169 ceph-mon[86183]: pgmap v119: 65 pgs: 65 active+clean; 255 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 2 op/s 2024-03-04T13:50:48.333 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:47 smithi107 ceph-mon[112578]: pgmap v119: 65 pgs: 65 active+clean; 255 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 2 op/s 2024-03-04T13:50:48.611 INFO:journalctl@ceph.osd.3.smithi169.stdout:Mar 04 13:50:48 smithi169 systemd[1]: Stopping Ceph osd.3 for 1fc09a58-da2b-11ee-95c6-87774f69a715... 2024-03-04T13:50:48.917 INFO:journalctl@ceph.osd.3.smithi169.stdout:Mar 04 13:50:48 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-3[41844]: 2024-03-04T13:50:48.655+0000 7f125f200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-04T13:50:48.917 INFO:journalctl@ceph.osd.3.smithi169.stdout:Mar 04 13:50:48 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-3[41844]: 2024-03-04T13:50:48.655+0000 7f125f200700 -1 osd.3 65 *** Got signal Terminated *** 2024-03-04T13:50:48.917 INFO:journalctl@ceph.osd.3.smithi169.stdout:Mar 04 13:50:48 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-3[41844]: 2024-03-04T13:50:48.655+0000 7f125f200700 -1 osd.3 65 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-04T13:50:49.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:48 smithi107 ceph-mon[112578]: osd.3 marked itself down and dead 2024-03-04T13:50:49.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:48 smithi169 ceph-mon[86183]: osd.3 marked itself down and dead 2024-03-04T13:50:49.920 INFO:journalctl@ceph.osd.3.smithi169.stdout:Mar 04 13:50:49 smithi169 podman[94791]: 2024-03-04 13:50:49.623529056 +0000 UTC m=+1.163503146 container died cd91d86947641098c52c6a44d7660a957bcfcaffc7ec51194223dda74569c27c (image=quay.ceph.io/ceph-ci/ceph@sha256:ee85afea73f504aaae524393a00a3b140e2a1cfd81d76da4525eaa6e73286a7d, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-3, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=quincy-42608bb, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-03-04T13:50:50.321 INFO:journalctl@ceph.osd.3.smithi169.stdout:Mar 04 13:50:50 smithi169 podman[94791]: 2024-03-04 13:50:50.086089233 +0000 UTC m=+1.626063346 container cleanup cd91d86947641098c52c6a44d7660a957bcfcaffc7ec51194223dda74569c27c (image=quay.ceph.io/ceph-ci/ceph@sha256:ee85afea73f504aaae524393a00a3b140e2a1cfd81d76da4525eaa6e73286a7d, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-3, io.buildah.version=1.33.5, RELEASE=quincy-42608bb, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, ceph=True) 2024-03-04T13:50:50.321 INFO:journalctl@ceph.osd.3.smithi169.stdout:Mar 04 13:50:50 smithi169 bash[94791]: ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-3 2024-03-04T13:50:50.322 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:49 smithi169 ceph-mon[86183]: pgmap v120: 65 pgs: 65 active+clean; 255 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:50:50.322 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:49 smithi169 ceph-mon[86183]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-04T13:50:50.322 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:49 smithi169 ceph-mon[86183]: osdmap e66: 6 total, 5 up, 6 in 2024-03-04T13:50:50.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:49 smithi107 ceph-mon[112578]: pgmap v120: 65 pgs: 65 active+clean; 255 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:50:50.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:49 smithi107 ceph-mon[112578]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-04T13:50:50.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:49 smithi107 ceph-mon[112578]: osdmap e66: 6 total, 5 up, 6 in 2024-03-04T13:50:50.611 INFO:journalctl@ceph.osd.3.smithi169.stdout:Mar 04 13:50:50 smithi169 podman[94809]: 2024-03-04 13:50:50.320544017 +0000 UTC m=+0.672509705 container remove cd91d86947641098c52c6a44d7660a957bcfcaffc7ec51194223dda74569c27c (image=quay.ceph.io/ceph-ci/ceph@sha256:ee85afea73f504aaae524393a00a3b140e2a1cfd81d76da4525eaa6e73286a7d, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-3, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240222, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=quincy-42608bb, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-03-04T13:50:51.122 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:50 smithi169 ceph-mon[86183]: osdmap e67: 6 total, 5 up, 6 in 2024-03-04T13:50:51.122 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:50 smithi169 ceph-mon[86183]: pgmap v123: 65 pgs: 8 active+undersized+degraded, 6 stale+active+clean, 10 active+undersized, 41 active+clean; 255 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 31/264 objects degraded (11.742%) 2024-03-04T13:50:51.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:50 smithi107 ceph-mon[112578]: osdmap e67: 6 total, 5 up, 6 in 2024-03-04T13:50:51.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:50 smithi107 ceph-mon[112578]: pgmap v123: 65 pgs: 8 active+undersized+degraded, 6 stale+active+clean, 10 active+undersized, 41 active+clean; 255 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 31/264 objects degraded (11.742%) 2024-03-04T13:50:51.540 INFO:journalctl@ceph.osd.3.smithi169.stdout:Mar 04 13:50:51 smithi169 podman[94896]: 2024-03-04 13:50:51.02810046 +0000 UTC m=+0.039796542 image pull d71b839abdf892550525dad59fdf20a6b925b73dbcc7526f0d04da8380b9fabf quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377 2024-03-04T13:50:51.540 INFO:journalctl@ceph.osd.3.smithi169.stdout:Mar 04 13:50:51 smithi169 podman[94896]: 2024-03-04 13:50:51.152624597 +0000 UTC m=+0.164320700 container create e9b7ac12c4e3106e20a2670e20e2ca9c3628616642722703602a1036bf208035 (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-3-deactivate, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-03-04T13:50:51.861 INFO:journalctl@ceph.osd.3.smithi169.stdout:Mar 04 13:50:51 smithi169 podman[94896]: 2024-03-04 13:50:51.558985854 +0000 UTC m=+0.570681980 container init e9b7ac12c4e3106e20a2670e20e2ca9c3628616642722703602a1036bf208035 (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-3-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.build-date=20240222, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-03-04T13:50:51.861 INFO:journalctl@ceph.osd.3.smithi169.stdout:Mar 04 13:50:51 smithi169 podman[94896]: 2024-03-04 13:50:51.572667541 +0000 UTC m=+0.584363675 container start e9b7ac12c4e3106e20a2670e20e2ca9c3628616642722703602a1036bf208035 (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-3-deactivate, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240222, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8) 2024-03-04T13:50:51.861 INFO:journalctl@ceph.osd.3.smithi169.stdout:Mar 04 13:50:51 smithi169 podman[94896]: 2024-03-04 13:50:51.640947155 +0000 UTC m=+0.652643302 container attach e9b7ac12c4e3106e20a2670e20e2ca9c3628616642722703602a1036bf208035 (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-3-deactivate, io.buildah.version=1.33.5, org.label-schema.build-date=20240222, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-03-04T13:50:52.290 INFO:journalctl@ceph.osd.3.smithi169.stdout:Mar 04 13:50:52 smithi169 podman[94896]: 2024-03-04 13:50:52.003443689 +0000 UTC m=+1.015139761 container died e9b7ac12c4e3106e20a2670e20e2ca9c3628616642722703602a1036bf208035 (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-3-deactivate, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240222, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-04T13:50:52.290 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:51 smithi169 ceph-mon[86183]: Health check failed: Degraded data redundancy: 31/264 objects degraded (11.742%), 8 pgs degraded (PG_DEGRADED) 2024-03-04T13:50:52.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:51 smithi107 ceph-mon[112578]: Health check failed: Degraded data redundancy: 31/264 objects degraded (11.742%), 8 pgs degraded (PG_DEGRADED) 2024-03-04T13:50:52.958 INFO:journalctl@ceph.osd.3.smithi169.stdout:Mar 04 13:50:52 smithi169 podman[94931]: 2024-03-04 13:50:52.686301428 +0000 UTC m=+0.661657082 container remove e9b7ac12c4e3106e20a2670e20e2ca9c3628616642722703602a1036bf208035 (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-3-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240222, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, ceph=True, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, io.buildah.version=1.33.5) 2024-03-04T13:50:52.958 INFO:journalctl@ceph.osd.3.smithi169.stdout:Mar 04 13:50:52 smithi169 systemd[1]: ceph-1fc09a58-da2b-11ee-95c6-87774f69a715@osd.3.service: Deactivated successfully. 2024-03-04T13:50:52.958 INFO:journalctl@ceph.osd.3.smithi169.stdout:Mar 04 13:50:52 smithi169 systemd[1]: Stopped Ceph osd.3 for 1fc09a58-da2b-11ee-95c6-87774f69a715. 2024-03-04T13:50:52.958 INFO:journalctl@ceph.osd.3.smithi169.stdout:Mar 04 13:50:52 smithi169 systemd[1]: ceph-1fc09a58-da2b-11ee-95c6-87774f69a715@osd.3.service: Consumed 1min 48.041s CPU time. 2024-03-04T13:50:53.246 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:52 smithi107 ceph-mon[112578]: pgmap v124: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 255 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 59/264 objects degraded (22.348%) 2024-03-04T13:50:53.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:52 smithi169 ceph-mon[86183]: pgmap v124: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 255 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 59/264 objects degraded (22.348%) 2024-03-04T13:50:53.361 INFO:journalctl@ceph.osd.3.smithi169.stdout:Mar 04 13:50:53 smithi169 systemd[1]: Starting Ceph osd.3 for 1fc09a58-da2b-11ee-95c6-87774f69a715... 2024-03-04T13:50:53.725 INFO:journalctl@ceph.osd.3.smithi169.stdout:Mar 04 13:50:53 smithi169 podman[95035]: 2024-03-04 13:50:53.335620567 +0000 UTC m=+0.031750776 image pull d71b839abdf892550525dad59fdf20a6b925b73dbcc7526f0d04da8380b9fabf quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377 2024-03-04T13:50:53.725 INFO:journalctl@ceph.osd.3.smithi169.stdout:Mar 04 13:50:53 smithi169 podman[95035]: 2024-03-04 13:50:53.445638961 +0000 UTC m=+0.141769195 container create d29d4676357cd892d16872c02dbf8908310f900f071317f741465283389e47c3 (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-3-activate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2) 2024-03-04T13:50:53.726 INFO:journalctl@ceph.osd.3.smithi169.stdout:Mar 04 13:50:53 smithi169 podman[95035]: 2024-03-04 13:50:53.724487129 +0000 UTC m=+0.420617393 container init d29d4676357cd892d16872c02dbf8908310f900f071317f741465283389e47c3 (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-3-activate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, GIT_CLEAN=True, org.label-schema.build-date=20240222, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-03-04T13:50:54.111 INFO:journalctl@ceph.osd.3.smithi169.stdout:Mar 04 13:50:53 smithi169 podman[95035]: 2024-03-04 13:50:53.735966508 +0000 UTC m=+0.432096708 container start d29d4676357cd892d16872c02dbf8908310f900f071317f741465283389e47c3 (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-3-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240222, ceph=True, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-03-04T13:50:54.111 INFO:journalctl@ceph.osd.3.smithi169.stdout:Mar 04 13:50:53 smithi169 podman[95035]: 2024-03-04 13:50:53.84852309 +0000 UTC m=+0.544653336 container attach d29d4676357cd892d16872c02dbf8908310f900f071317f741465283389e47c3 (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-3-activate, ceph=True, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, maintainer=Guillaume Abrioux , GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-03-04T13:50:54.862 INFO:journalctl@ceph.osd.3.smithi169.stdout:Mar 04 13:50:54 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-3-activate[95050]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-04T13:50:54.863 INFO:journalctl@ceph.osd.3.smithi169.stdout:Mar 04 13:50:54 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-3-activate[95050]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-03-04T13:50:54.863 INFO:journalctl@ceph.osd.3.smithi169.stdout:Mar 04 13:50:54 smithi169 bash[95035]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-04T13:50:54.863 INFO:journalctl@ceph.osd.3.smithi169.stdout:Mar 04 13:50:54 smithi169 bash[95035]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-03-04T13:50:54.863 INFO:journalctl@ceph.osd.3.smithi169.stdout:Mar 04 13:50:54 smithi169 bash[95035]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-03-04T13:50:54.863 INFO:journalctl@ceph.osd.3.smithi169.stdout:Mar 04 13:50:54 smithi169 bash[95035]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-03-04T13:50:54.863 INFO:journalctl@ceph.osd.3.smithi169.stdout:Mar 04 13:50:54 smithi169 bash[95035]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2024-03-04T13:50:54.863 INFO:journalctl@ceph.osd.3.smithi169.stdout:Mar 04 13:50:54 smithi169 bash[95035]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-04T13:50:54.863 INFO:journalctl@ceph.osd.3.smithi169.stdout:Mar 04 13:50:54 smithi169 bash[95035]: --> ceph-volume raw activate successful for osd ID: 3 2024-03-04T13:50:54.863 INFO:journalctl@ceph.osd.3.smithi169.stdout:Mar 04 13:50:54 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-3-activate[95050]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-03-04T13:50:54.863 INFO:journalctl@ceph.osd.3.smithi169.stdout:Mar 04 13:50:54 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-3-activate[95050]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-03-04T13:50:54.863 INFO:journalctl@ceph.osd.3.smithi169.stdout:Mar 04 13:50:54 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-3-activate[95050]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2024-03-04T13:50:54.863 INFO:journalctl@ceph.osd.3.smithi169.stdout:Mar 04 13:50:54 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-3-activate[95050]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-04T13:50:54.863 INFO:journalctl@ceph.osd.3.smithi169.stdout:Mar 04 13:50:54 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-3-activate[95050]: --> ceph-volume raw activate successful for osd ID: 3 2024-03-04T13:50:54.864 INFO:journalctl@ceph.osd.3.smithi169.stdout:Mar 04 13:50:54 smithi169 podman[95035]: 2024-03-04 13:50:54.64198268 +0000 UTC m=+1.338112967 container died d29d4676357cd892d16872c02dbf8908310f900f071317f741465283389e47c3 (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-3-activate, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.build-date=20240222, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-03-04T13:50:55.611 INFO:journalctl@ceph.osd.3.smithi169.stdout:Mar 04 13:50:55 smithi169 podman[95175]: 2024-03-04 13:50:55.318714228 +0000 UTC m=+0.656403245 container remove d29d4676357cd892d16872c02dbf8908310f900f071317f741465283389e47c3 (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-3-activate, io.buildah.version=1.33.5, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, GIT_CLEAN=True, ceph=True) 2024-03-04T13:50:55.966 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:55 smithi169 ceph-mon[86183]: pgmap v125: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 255 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 59/264 objects degraded (22.348%) 2024-03-04T13:50:55.966 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:55 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:50:55.966 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:55 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:50:55.967 INFO:journalctl@ceph.osd.3.smithi169.stdout:Mar 04 13:50:55 smithi169 podman[95237]: 2024-03-04 13:50:55.569763325 +0000 UTC m=+0.035558197 image pull d71b839abdf892550525dad59fdf20a6b925b73dbcc7526f0d04da8380b9fabf quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377 2024-03-04T13:50:55.967 INFO:journalctl@ceph.osd.3.smithi169.stdout:Mar 04 13:50:55 smithi169 podman[95237]: 2024-03-04 13:50:55.695073005 +0000 UTC m=+0.160867907 container create d31baf764d7ac1ac25131ad812b587e6790aa93e48d4e463bf412a4d705954e4 (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, org.label-schema.license=GPLv2) 2024-03-04T13:50:55.967 INFO:journalctl@ceph.osd.3.smithi169.stdout:Mar 04 13:50:55 smithi169 podman[95237]: 2024-03-04 13:50:55.965996172 +0000 UTC m=+0.431791094 container init d31baf764d7ac1ac25131ad812b587e6790aa93e48d4e463bf412a4d705954e4 (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-3, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, GIT_BRANCH=HEAD, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222) 2024-03-04T13:50:56.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:55 smithi107 ceph-mon[112578]: pgmap v125: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 255 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 59/264 objects degraded (22.348%) 2024-03-04T13:50:56.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:55 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:50:56.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:55 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:50:56.361 INFO:journalctl@ceph.osd.3.smithi169.stdout:Mar 04 13:50:55 smithi169 podman[95237]: 2024-03-04 13:50:55.977604342 +0000 UTC m=+0.443399219 container start d31baf764d7ac1ac25131ad812b587e6790aa93e48d4e463bf412a4d705954e4 (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-3, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240222, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-03-04T13:50:56.361 INFO:journalctl@ceph.osd.3.smithi169.stdout:Mar 04 13:50:56 smithi169 ceph-osd[95256]: -- 172.21.15.169:0/3438753374 <== mon.1 v2:172.21.15.169:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55cee5a15680 con 0x55cee59dec00 2024-03-04T13:50:56.361 INFO:journalctl@ceph.osd.3.smithi169.stdout:Mar 04 13:50:56 smithi169 bash[95237]: d31baf764d7ac1ac25131ad812b587e6790aa93e48d4e463bf412a4d705954e4 2024-03-04T13:50:56.361 INFO:journalctl@ceph.osd.3.smithi169.stdout:Mar 04 13:50:56 smithi169 systemd[1]: Started Ceph osd.3 for 1fc09a58-da2b-11ee-95c6-87774f69a715. 2024-03-04T13:50:56.861 INFO:journalctl@ceph.osd.3.smithi169.stdout:Mar 04 13:50:56 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-3[95252]: 2024-03-04T13:50:56.551+0000 7f384c83d7c0 -1 Falling back to public interface 2024-03-04T13:50:57.594 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:57 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:50:57.594 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:57 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:50:57.594 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:57 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:50:57.594 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:57 smithi107 ceph-mon[112578]: pgmap v126: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 255 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 59/264 objects degraded (22.348%) 2024-03-04T13:50:57.611 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:57 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:50:57.611 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:57 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:50:57.611 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:57 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:50:57.611 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:57 smithi169 ceph-mon[86183]: pgmap v126: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 255 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 59/264 objects degraded (22.348%) 2024-03-04T13:50:57.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:57.640+0000 7f1b11e00700 1 -- 172.21.15.107:0/1270396962 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b0c103dc0 msgr2=0x7f1b0c104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:50:57.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:57.640+0000 7f1b11e00700 1 --2- 172.21.15.107:0/1270396962 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b0c103dc0 0x7f1b0c104210 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1af4004940 tx=0x7f1af4009e30 comp rx=0 tx=0).stop 2024-03-04T13:50:57.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:57.640+0000 7f1b11e00700 1 -- 172.21.15.107:0/1270396962 shutdown_connections 2024-03-04T13:50:57.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:57.640+0000 7f1b11e00700 1 --2- 172.21.15.107:0/1270396962 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b0c103dc0 0x7f1b0c104210 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:50:57.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:57.640+0000 7f1b11e00700 1 --2- 172.21.15.107:0/1270396962 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b0c1095f0 0x7f1b0c1099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:50:57.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:57.640+0000 7f1b11e00700 1 -- 172.21.15.107:0/1270396962 >> 172.21.15.107:0/1270396962 conn(0x7f1b0c072d50 msgr2=0x7f1b0c075160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:50:57.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:57.640+0000 7f1b11e00700 1 -- 172.21.15.107:0/1270396962 shutdown_connections 2024-03-04T13:50:57.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:57.640+0000 7f1b11e00700 1 -- 172.21.15.107:0/1270396962 wait complete. 2024-03-04T13:50:57.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:57.641+0000 7f1b11e00700 1 Processor -- start 2024-03-04T13:50:57.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:57.641+0000 7f1b11e00700 1 -- start start 2024-03-04T13:50:57.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:57.642+0000 7f1b11e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b0c103dc0 0x7f1b0c11a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:50:57.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:57.642+0000 7f1b11e00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b0c1095f0 0x7f1b0c11a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:50:57.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:57.642+0000 7f1b11e00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b0c11ab90 con 0x7f1b0c103dc0 2024-03-04T13:50:57.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:57.642+0000 7f1b11e00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b0c113ee0 con 0x7f1b0c1095f0 2024-03-04T13:50:57.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:57.642+0000 7f1b0aa00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b0c103dc0 0x7f1b0c11a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:50:57.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:57.642+0000 7f1b0aa00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b0c103dc0 0x7f1b0c11a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33482/0 (socket says 172.21.15.107:33482) 2024-03-04T13:50:57.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:57.642+0000 7f1b0aa00700 1 -- 172.21.15.107:0/20561669 learned_addr learned my addr 172.21.15.107:0/20561669 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:50:57.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:57.642+0000 7f1b03e00700 1 --2- 172.21.15.107:0/20561669 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b0c1095f0 0x7f1b0c11a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:50:57.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:57.643+0000 7f1b0aa00700 1 -- 172.21.15.107:0/20561669 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b0c1095f0 msgr2=0x7f1b0c11a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:50:57.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:57.643+0000 7f1b0aa00700 1 --2- 172.21.15.107:0/20561669 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b0c1095f0 0x7f1b0c11a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:50:57.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:57.643+0000 7f1b0aa00700 1 -- 172.21.15.107:0/20561669 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1af4005040 con 0x7f1b0c103dc0 2024-03-04T13:50:57.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:57.643+0000 7f1b0aa00700 1 --2- 172.21.15.107:0/20561669 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b0c103dc0 0x7f1b0c11a000 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1afc00eb40 tx=0x7f1afc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:50:57.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:57.643+0000 7f1b02a00700 1 -- 172.21.15.107:0/20561669 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1afc007cc0 con 0x7f1b0c103dc0 2024-03-04T13:50:57.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:57.643+0000 7f1b02a00700 1 -- 172.21.15.107:0/20561669 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1afc00f040 con 0x7f1b0c103dc0 2024-03-04T13:50:57.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:57.643+0000 7f1b11e00700 1 -- 172.21.15.107:0/20561669 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b0c114160 con 0x7f1b0c103dc0 2024-03-04T13:50:57.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:57.643+0000 7f1b02a00700 1 -- 172.21.15.107:0/20561669 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1afc009620 con 0x7f1b0c103dc0 2024-03-04T13:50:57.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:57.643+0000 7f1b11e00700 1 -- 172.21.15.107:0/20561669 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b0c1146a0 con 0x7f1b0c103dc0 2024-03-04T13:50:57.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:57.645+0000 7f1b11e00700 1 -- 172.21.15.107:0/20561669 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b0c075870 con 0x7f1b0c103dc0 2024-03-04T13:50:57.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:57.646+0000 7f1b02a00700 1 -- 172.21.15.107:0/20561669 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f1afc009780 con 0x7f1b0c103dc0 2024-03-04T13:50:57.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:57.646+0000 7f1b02a00700 1 --2- 172.21.15.107:0/20561669 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f1af006d7f0 0x7f1af006fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:50:57.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:57.647+0000 7f1b02a00700 1 -- 172.21.15.107:0/20561669 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f1afc08cb10 con 0x7f1b0c103dc0 2024-03-04T13:50:57.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:57.647+0000 7f1b03e00700 1 --2- 172.21.15.107:0/20561669 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f1af006d7f0 0x7f1af006fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:50:57.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:57.647+0000 7f1b03e00700 1 --2- 172.21.15.107:0/20561669 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f1af006d7f0 0x7f1af006fca0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1af400b0d0 tx=0x7f1af4009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:50:57.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:57.652+0000 7f1b02a00700 1 -- 172.21.15.107:0/20561669 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1afc056460 con 0x7f1b0c103dc0 2024-03-04T13:50:57.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:57.880+0000 7f1b11e00700 1 -- 172.21.15.107:0/20561669 --> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1b0c115690 con 0x7f1af006d7f0 2024-03-04T13:50:57.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:57.882+0000 7f1b02a00700 1 -- 172.21.15.107:0/20561669 <== mgr.34102 v2:172.21.15.107:6800/1223039812 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f1b0c115690 con 0x7f1af006d7f0 2024-03-04T13:50:57.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:57.885+0000 7f1b11e00700 1 -- 172.21.15.107:0/20561669 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f1af006d7f0 msgr2=0x7f1af006fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:50:57.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:57.885+0000 7f1b11e00700 1 --2- 172.21.15.107:0/20561669 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f1af006d7f0 0x7f1af006fca0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1af400b0d0 tx=0x7f1af4009e30 comp rx=0 tx=0).stop 2024-03-04T13:50:57.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:57.885+0000 7f1b11e00700 1 -- 172.21.15.107:0/20561669 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b0c103dc0 msgr2=0x7f1b0c11a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:50:57.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:57.885+0000 7f1b11e00700 1 --2- 172.21.15.107:0/20561669 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b0c103dc0 0x7f1b0c11a000 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1afc00eb40 tx=0x7f1afc00b040 comp rx=0 tx=0).stop 2024-03-04T13:50:57.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:57.885+0000 7f1b11e00700 1 -- 172.21.15.107:0/20561669 shutdown_connections 2024-03-04T13:50:57.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:57.885+0000 7f1b11e00700 1 --2- 172.21.15.107:0/20561669 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f1af006d7f0 0x7f1af006fca0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:50:57.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:57.885+0000 7f1b11e00700 1 --2- 172.21.15.107:0/20561669 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b0c103dc0 0x7f1b0c11a000 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:50:57.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:57.885+0000 7f1b11e00700 1 --2- 172.21.15.107:0/20561669 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b0c1095f0 0x7f1b0c11a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:50:57.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:57.885+0000 7f1b11e00700 1 -- 172.21.15.107:0/20561669 >> 172.21.15.107:0/20561669 conn(0x7f1b0c072d50 msgr2=0x7f1b0c074790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:50:57.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:57.885+0000 7f1b11e00700 1 -- 172.21.15.107:0/20561669 shutdown_connections 2024-03-04T13:50:57.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:57.885+0000 7f1b11e00700 1 -- 172.21.15.107:0/20561669 wait complete. 2024-03-04T13:50:57.900 INFO:teuthology.orchestra.run.smithi107.stdout:true 2024-03-04T13:50:58.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.038+0000 7f091e400700 1 -- 172.21.15.107:0/185000454 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0918103dc0 msgr2=0x7f0918104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:50:58.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.038+0000 7f091e400700 1 --2- 172.21.15.107:0/185000454 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0918103dc0 0x7f0918104210 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0904004940 tx=0x7f0904009e30 comp rx=0 tx=0).stop 2024-03-04T13:50:58.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.039+0000 7f091e400700 1 -- 172.21.15.107:0/185000454 shutdown_connections 2024-03-04T13:50:58.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.039+0000 7f091e400700 1 --2- 172.21.15.107:0/185000454 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0918103dc0 0x7f0918104210 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:50:58.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.039+0000 7f091e400700 1 --2- 172.21.15.107:0/185000454 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f09181095f0 0x7f09181099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:50:58.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.039+0000 7f091e400700 1 -- 172.21.15.107:0/185000454 >> 172.21.15.107:0/185000454 conn(0x7f0918072d50 msgr2=0x7f0918075160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:50:58.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.039+0000 7f091e400700 1 -- 172.21.15.107:0/185000454 shutdown_connections 2024-03-04T13:50:58.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.039+0000 7f091e400700 1 -- 172.21.15.107:0/185000454 wait complete. 2024-03-04T13:50:58.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.040+0000 7f091e400700 1 Processor -- start 2024-03-04T13:50:58.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.040+0000 7f091e400700 1 -- start start 2024-03-04T13:50:58.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.040+0000 7f091e400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0918103dc0 0x7f091811a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:50:58.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.040+0000 7f091e400700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f09181095f0 0x7f091811a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:50:58.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.040+0000 7f091e400700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f091811ac50 con 0x7f0918103dc0 2024-03-04T13:50:58.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.040+0000 7f091e400700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0918113ed0 con 0x7f09181095f0 2024-03-04T13:50:58.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.040+0000 7f0917400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0918103dc0 0x7f091811a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:50:58.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.041+0000 7f0917400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0918103dc0 0x7f091811a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:35354/0 (socket says 172.21.15.107:35354) 2024-03-04T13:50:58.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.041+0000 7f0917400700 1 -- 172.21.15.107:0/1826326460 learned_addr learned my addr 172.21.15.107:0/1826326460 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:50:58.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.041+0000 7f0916a00700 1 --2- 172.21.15.107:0/1826326460 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f09181095f0 0x7f091811a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:50:58.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.041+0000 7f0917400700 1 -- 172.21.15.107:0/1826326460 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f09181095f0 msgr2=0x7f091811a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:50:58.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.041+0000 7f0917400700 1 --2- 172.21.15.107:0/1826326460 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f09181095f0 0x7f091811a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:50:58.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.041+0000 7f0917400700 1 -- 172.21.15.107:0/1826326460 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0904005040 con 0x7f0918103dc0 2024-03-04T13:50:58.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.041+0000 7f0917400700 1 --2- 172.21.15.107:0/1826326460 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0918103dc0 0x7f091811a030 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f090800b790 tx=0x7f0908009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:50:58.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.041+0000 7f090f400700 1 -- 172.21.15.107:0/1826326460 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f090800f770 con 0x7f0918103dc0 2024-03-04T13:50:58.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.041+0000 7f091e400700 1 -- 172.21.15.107:0/1826326460 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0918114150 con 0x7f0918103dc0 2024-03-04T13:50:58.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.041+0000 7f090f400700 1 -- 172.21.15.107:0/1826326460 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f090800fda0 con 0x7f0918103dc0 2024-03-04T13:50:58.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.041+0000 7f090f400700 1 -- 172.21.15.107:0/1826326460 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f090800d310 con 0x7f0918103dc0 2024-03-04T13:50:58.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.042+0000 7f091e400700 1 -- 172.21.15.107:0/1826326460 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0918114690 con 0x7f0918103dc0 2024-03-04T13:50:58.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.043+0000 7f091e400700 1 -- 172.21.15.107:0/1826326460 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0918075870 con 0x7f0918103dc0 2024-03-04T13:50:58.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.044+0000 7f090f400700 1 -- 172.21.15.107:0/1826326460 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f0908010420 con 0x7f0918103dc0 2024-03-04T13:50:58.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.045+0000 7f090f400700 1 --2- 172.21.15.107:0/1826326460 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f08fc06d7f0 0x7f08fc06fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:50:58.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.045+0000 7f090f400700 1 -- 172.21.15.107:0/1826326460 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f090808cec0 con 0x7f0918103dc0 2024-03-04T13:50:58.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.045+0000 7f0916a00700 1 --2- 172.21.15.107:0/1826326460 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f08fc06d7f0 0x7f08fc06fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:50:58.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.045+0000 7f0916a00700 1 --2- 172.21.15.107:0/1826326460 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f08fc06d7f0 0x7f08fc06fca0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f090400b020 tx=0x7f0904009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:50:58.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.049+0000 7f090f400700 1 -- 172.21.15.107:0/1826326460 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f090805a310 con 0x7f0918103dc0 2024-03-04T13:50:58.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.285+0000 7f091e400700 1 -- 172.21.15.107:0/1826326460 --> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0918114910 con 0x7f08fc06d7f0 2024-03-04T13:50:58.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.286+0000 7f090f400700 1 -- 172.21.15.107:0/1826326460 <== mgr.34102 v2:172.21.15.107:6800/1223039812 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f0918114910 con 0x7f08fc06d7f0 2024-03-04T13:50:58.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.289+0000 7f091e400700 1 -- 172.21.15.107:0/1826326460 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f08fc06d7f0 msgr2=0x7f08fc06fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:50:58.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.289+0000 7f091e400700 1 --2- 172.21.15.107:0/1826326460 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f08fc06d7f0 0x7f08fc06fca0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f090400b020 tx=0x7f0904009e30 comp rx=0 tx=0).stop 2024-03-04T13:50:58.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.289+0000 7f091e400700 1 -- 172.21.15.107:0/1826326460 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0918103dc0 msgr2=0x7f091811a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:50:58.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.289+0000 7f091e400700 1 --2- 172.21.15.107:0/1826326460 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0918103dc0 0x7f091811a030 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f090800b790 tx=0x7f0908009250 comp rx=0 tx=0).stop 2024-03-04T13:50:58.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.290+0000 7f091e400700 1 -- 172.21.15.107:0/1826326460 shutdown_connections 2024-03-04T13:50:58.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.290+0000 7f091e400700 1 --2- 172.21.15.107:0/1826326460 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f08fc06d7f0 0x7f08fc06fca0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:50:58.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.290+0000 7f091e400700 1 --2- 172.21.15.107:0/1826326460 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0918103dc0 0x7f091811a030 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:50:58.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.290+0000 7f091e400700 1 --2- 172.21.15.107:0/1826326460 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f09181095f0 0x7f091811a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:50:58.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.290+0000 7f091e400700 1 -- 172.21.15.107:0/1826326460 >> 172.21.15.107:0/1826326460 conn(0x7f0918072d50 msgr2=0x7f09180746b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:50:58.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.290+0000 7f091e400700 1 -- 172.21.15.107:0/1826326460 shutdown_connections 2024-03-04T13:50:58.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.290+0000 7f091e400700 1 -- 172.21.15.107:0/1826326460 wait complete. 2024-03-04T13:50:58.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.440+0000 7fd143400700 1 -- 172.21.15.107:0/3068649007 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd13c102f50 msgr2=0x7fd13c103320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:50:58.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.440+0000 7fd143400700 1 --2- 172.21.15.107:0/3068649007 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd13c102f50 0x7fd13c103320 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd130009ce0 tx=0x7fd130004d10 comp rx=0 tx=0).stop 2024-03-04T13:50:58.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.440+0000 7fd143400700 1 -- 172.21.15.107:0/3068649007 shutdown_connections 2024-03-04T13:50:58.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.440+0000 7fd143400700 1 --2- 172.21.15.107:0/3068649007 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd13c1038f0 0x7fd13c0fcf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:50:58.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.440+0000 7fd143400700 1 --2- 172.21.15.107:0/3068649007 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd13c102f50 0x7fd13c103320 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:50:58.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.440+0000 7fd143400700 1 -- 172.21.15.107:0/3068649007 >> 172.21.15.107:0/3068649007 conn(0x7fd13c0f8ae0 msgr2=0x7fd13c0faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:50:58.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.440+0000 7fd143400700 1 -- 172.21.15.107:0/3068649007 shutdown_connections 2024-03-04T13:50:58.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.440+0000 7fd143400700 1 -- 172.21.15.107:0/3068649007 wait complete. 2024-03-04T13:50:58.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.441+0000 7fd143400700 1 Processor -- start 2024-03-04T13:50:58.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.441+0000 7fd143400700 1 -- start start 2024-03-04T13:50:58.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.442+0000 7fd143400700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd13c102f50 0x7fd13c070290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:50:58.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.442+0000 7fd143400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd13c1038f0 0x7fd13c0707f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:50:58.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.442+0000 7fd143400700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd13c070e40 con 0x7fd13c1038f0 2024-03-04T13:50:58.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.442+0000 7fd143400700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd13c070fb0 con 0x7fd13c102f50 2024-03-04T13:50:58.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.442+0000 7fd13be00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd13c1038f0 0x7fd13c0707f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:50:58.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.442+0000 7fd13be00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd13c1038f0 0x7fd13c0707f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:35360/0 (socket says 172.21.15.107:35360) 2024-03-04T13:50:58.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.442+0000 7fd13be00700 1 -- 172.21.15.107:0/2799511100 learned_addr learned my addr 172.21.15.107:0/2799511100 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:50:58.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.442+0000 7fd140a00700 1 --2- 172.21.15.107:0/2799511100 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd13c102f50 0x7fd13c070290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:50:58.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.442+0000 7fd13be00700 1 -- 172.21.15.107:0/2799511100 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd13c102f50 msgr2=0x7fd13c070290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:50:58.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.442+0000 7fd13be00700 1 --2- 172.21.15.107:0/2799511100 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd13c102f50 0x7fd13c070290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:50:58.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.442+0000 7fd13be00700 1 -- 172.21.15.107:0/2799511100 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd130009940 con 0x7fd13c1038f0 2024-03-04T13:50:58.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.443+0000 7fd13be00700 1 --2- 172.21.15.107:0/2799511100 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd13c1038f0 0x7fd13c0707f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd128002a50 tx=0x7fd128004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:50:58.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.443+0000 7fd138c00700 1 -- 172.21.15.107:0/2799511100 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd128007cc0 con 0x7fd13c1038f0 2024-03-04T13:50:58.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.443+0000 7fd138c00700 1 -- 172.21.15.107:0/2799511100 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd12800f040 con 0x7fd13c1038f0 2024-03-04T13:50:58.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.443+0000 7fd143400700 1 -- 172.21.15.107:0/2799511100 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd13c119e70 con 0x7fd13c1038f0 2024-03-04T13:50:58.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.443+0000 7fd138c00700 1 -- 172.21.15.107:0/2799511100 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd12800daf0 con 0x7fd13c1038f0 2024-03-04T13:50:58.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.443+0000 7fd143400700 1 -- 172.21.15.107:0/2799511100 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd13c11a380 con 0x7fd13c1038f0 2024-03-04T13:50:58.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.444+0000 7fd143400700 1 -- 172.21.15.107:0/2799511100 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd13c100260 con 0x7fd13c1038f0 2024-03-04T13:50:58.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.445+0000 7fd138c00700 1 -- 172.21.15.107:0/2799511100 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fd128009810 con 0x7fd13c1038f0 2024-03-04T13:50:58.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.446+0000 7fd138c00700 1 --2- 172.21.15.107:0/2799511100 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fd12406d7f0 0x7fd12406fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:50:58.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.447+0000 7fd138c00700 1 -- 172.21.15.107:0/2799511100 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fd12808cbb0 con 0x7fd13c1038f0 2024-03-04T13:50:58.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.447+0000 7fd140a00700 1 --2- 172.21.15.107:0/2799511100 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fd12406d7f0 0x7fd12406fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:50:58.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.448+0000 7fd140a00700 1 --2- 172.21.15.107:0/2799511100 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fd12406d7f0 0x7fd12406fca0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd130005010 tx=0x7fd130004d10 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:50:58.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.451+0000 7fd138c00700 1 -- 172.21.15.107:0/2799511100 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd128056430 con 0x7fd13c1038f0 2024-03-04T13:50:58.594 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:58 smithi107 ceph-mon[112578]: from='client.34310 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:50:58.611 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:58 smithi169 ceph-mon[86183]: from='client.34310 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:50:58.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.680+0000 7fd143400700 1 -- 172.21.15.107:0/2799511100 --> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fd13c04bf70 con 0x7fd12406d7f0 2024-03-04T13:50:58.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.709+0000 7fd138c00700 1 -- 172.21.15.107:0/2799511100 <== mgr.34102 v2:172.21.15.107:6800/1223039812 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fd130009640 con 0x7fd12406d7f0 2024-03-04T13:50:58.710 INFO:teuthology.orchestra.run.smithi107.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-04T13:50:58.710 INFO:teuthology.orchestra.run.smithi107.stdout:alertmanager.smithi107 smithi107 *:9093,9094 running (5m) 25s ago 19m 14.6M - 0.25.0 c8568f914cd2 e5ed7be32538 2024-03-04T13:50:58.710 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi107 smithi107 running (2m) 25s ago 19m 6660k - 19.0.0-1845-g717ce3cc d71b839abdf8 0558022b0d84 2024-03-04T13:50:58.711 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi169 smithi169 running (2m) 119s ago 18m 6665k - 19.0.0-1845-g717ce3cc d71b839abdf8 221ca53aa0d3 2024-03-04T13:50:58.711 INFO:teuthology.orchestra.run.smithi107.stdout:grafana.smithi107 smithi107 *:3000 running (4m) 25s ago 18m 74.5M - 9.4.12 a0ea5e3c46d9 c5e4d33fb599 2024-03-04T13:50:58.711 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.riexmm smithi107 running (12m) 25s ago 12m 14.9M - 17.2.7-545-g42608bb1 9eec3f22859d fc921f7d4328 2024-03-04T13:50:58.711 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.thycho smithi107 running (12m) 25s ago 12m 98.3M - 17.2.7-545-g42608bb1 9eec3f22859d 4069c6e0e063 2024-03-04T13:50:58.711 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi169.bxitzk smithi169 running (12m) 119s ago 12m 14.2M - 17.2.7-545-g42608bb1 9eec3f22859d 7d4115228c7c 2024-03-04T13:50:58.711 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi169.jusudl smithi169 running (12m) 119s ago 12m 145M - 17.2.7-545-g42608bb1 9eec3f22859d 2d2f2cc452e8 2024-03-04T13:50:58.711 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi107.wpzgwl smithi107 *:8443,9283,8765 running (7m) 25s ago 21m 504M - 19.0.0-1845-g717ce3cc d71b839abdf8 1c5fdee45eac 2024-03-04T13:50:58.711 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi169.eihczr smithi169 *:8443,9283,8765 running (6m) 119s ago 18m 449M - 19.0.0-1845-g717ce3cc d71b839abdf8 b5cac2b7d22a 2024-03-04T13:50:58.711 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi107 smithi107 running (3m) 25s ago 21m 46.5M 2048M 19.0.0-1845-g717ce3cc d71b839abdf8 32f07144274f 2024-03-04T13:50:58.711 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi169 smithi169 running (3m) 119s ago 17m 33.9M 2048M 19.0.0-1845-g717ce3cc d71b839abdf8 e3527449381a 2024-03-04T13:50:58.711 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi107 smithi107 *:9100 running (6m) 25s ago 19m 7680k - 1.5.0 0da6a335fe13 8a8d3d4f724f 2024-03-04T13:50:58.712 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi169 smithi169 *:9100 running (6m) 119s ago 18m 7688k - 1.5.0 0da6a335fe13 f240f5a975bc 2024-03-04T13:50:58.712 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0 smithi107 running (94s) 25s ago 17m 80.5M 1558M 19.0.0-1845-g717ce3cc d71b839abdf8 5801a5540dc6 2024-03-04T13:50:58.712 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1 smithi107 running (64s) 25s ago 16m 91.3M 1558M 19.0.0-1845-g717ce3cc d71b839abdf8 b614af658854 2024-03-04T13:50:58.712 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2 smithi107 running (33s) 25s ago 15m 11.1M 1558M 19.0.0-1845-g717ce3cc d71b839abdf8 6e4da4f6f146 2024-03-04T13:50:58.712 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3 smithi169 starting - - - 2582M 2024-03-04T13:50:58.712 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4 smithi169 running (14m) 119s ago 14m 207M 2582M 17.2.7-545-g42608bb1 9eec3f22859d 08545e3d0659 2024-03-04T13:50:58.712 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5 smithi169 running (14m) 119s ago 14m 194M 2582M 17.2.7-545-g42608bb1 9eec3f22859d dcffe6cb3eac 2024-03-04T13:50:58.712 INFO:teuthology.orchestra.run.smithi107.stdout:prometheus.smithi107 smithi107 *:9095 running (5m) 25s ago 18m 51.0M - 2.43.0 a07b618ecd1d 1eeb1ad5cb75 2024-03-04T13:50:58.713 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.712+0000 7fd143400700 1 -- 172.21.15.107:0/2799511100 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fd12406d7f0 msgr2=0x7fd12406fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:50:58.713 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.712+0000 7fd143400700 1 --2- 172.21.15.107:0/2799511100 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fd12406d7f0 0x7fd12406fca0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd130005010 tx=0x7fd130004d10 comp rx=0 tx=0).stop 2024-03-04T13:50:58.713 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.712+0000 7fd143400700 1 -- 172.21.15.107:0/2799511100 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd13c1038f0 msgr2=0x7fd13c0707f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:50:58.713 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.712+0000 7fd143400700 1 --2- 172.21.15.107:0/2799511100 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd13c1038f0 0x7fd13c0707f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd128002a50 tx=0x7fd128004600 comp rx=0 tx=0).stop 2024-03-04T13:50:58.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.713+0000 7fd143400700 1 -- 172.21.15.107:0/2799511100 shutdown_connections 2024-03-04T13:50:58.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.713+0000 7fd143400700 1 --2- 172.21.15.107:0/2799511100 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fd12406d7f0 0x7fd12406fca0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:50:58.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.713+0000 7fd143400700 1 --2- 172.21.15.107:0/2799511100 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd13c102f50 0x7fd13c070290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:50:58.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.713+0000 7fd143400700 1 --2- 172.21.15.107:0/2799511100 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd13c1038f0 0x7fd13c0707f0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:50:58.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.713+0000 7fd143400700 1 -- 172.21.15.107:0/2799511100 >> 172.21.15.107:0/2799511100 conn(0x7fd13c0f8ae0 msgr2=0x7fd13c0fa5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:50:58.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.713+0000 7fd143400700 1 -- 172.21.15.107:0/2799511100 shutdown_connections 2024-03-04T13:50:58.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.713+0000 7fd143400700 1 -- 172.21.15.107:0/2799511100 wait complete. 2024-03-04T13:50:58.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.854+0000 7f4598800700 1 -- 172.21.15.107:0/1168446591 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4590070870 msgr2=0x7f4590070ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:50:58.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.854+0000 7f4598800700 1 --2- 172.21.15.107:0/1168446591 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4590070870 0x7f4590070ce0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4580004970 tx=0x7f4580009e30 comp rx=0 tx=0).stop 2024-03-04T13:50:58.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.854+0000 7f4598800700 1 -- 172.21.15.107:0/1168446591 shutdown_connections 2024-03-04T13:50:58.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.854+0000 7f4598800700 1 --2- 172.21.15.107:0/1168446591 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4590070870 0x7f4590070ce0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:50:58.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.854+0000 7f4598800700 1 --2- 172.21.15.107:0/1168446591 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4590071f60 0x7f4590070330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:50:58.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.854+0000 7f4598800700 1 -- 172.21.15.107:0/1168446591 >> 172.21.15.107:0/1168446591 conn(0x7f45900f8ae0 msgr2=0x7f45900faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:50:58.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.854+0000 7f4598800700 1 -- 172.21.15.107:0/1168446591 shutdown_connections 2024-03-04T13:50:58.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.854+0000 7f4598800700 1 -- 172.21.15.107:0/1168446591 wait complete. 2024-03-04T13:50:58.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.855+0000 7f4598800700 1 Processor -- start 2024-03-04T13:50:58.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.855+0000 7f4598800700 1 -- start start 2024-03-04T13:50:58.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.855+0000 7f4598800700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4590070870 0x7f459011a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:50:58.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.855+0000 7f4598800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4590071f60 0x7f459011a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:50:58.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.855+0000 7f4598800700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f459011ad50 con 0x7f4590071f60 2024-03-04T13:50:58.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.855+0000 7f4598800700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4590113fd0 con 0x7f4590070870 2024-03-04T13:50:58.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.856+0000 7f4595400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4590071f60 0x7f459011a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:50:58.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.856+0000 7f4595400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4590071f60 0x7f459011a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:35378/0 (socket says 172.21.15.107:35378) 2024-03-04T13:50:58.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.856+0000 7f4595400700 1 -- 172.21.15.107:0/1193639395 learned_addr learned my addr 172.21.15.107:0/1193639395 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:50:58.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.856+0000 7f4595e00700 1 --2- 172.21.15.107:0/1193639395 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4590070870 0x7f459011a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:50:58.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.856+0000 7f4595400700 1 -- 172.21.15.107:0/1193639395 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4590070870 msgr2=0x7f459011a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:50:58.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.856+0000 7f4595400700 1 --2- 172.21.15.107:0/1193639395 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4590070870 0x7f459011a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:50:58.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.856+0000 7f4595400700 1 -- 172.21.15.107:0/1193639395 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4580005040 con 0x7f4590071f60 2024-03-04T13:50:58.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.856+0000 7f4595e00700 1 --2- 172.21.15.107:0/1193639395 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4590070870 0x7f459011a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T13:50:58.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.856+0000 7f4595400700 1 --2- 172.21.15.107:0/1193639395 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4590071f60 0x7f459011a670 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4580004f50 tx=0x7f4580009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:50:58.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.857+0000 7f458a000700 1 -- 172.21.15.107:0/1193639395 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f458000e040 con 0x7f4590071f60 2024-03-04T13:50:58.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.857+0000 7f458a000700 1 -- 172.21.15.107:0/1193639395 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f458000bc10 con 0x7f4590071f60 2024-03-04T13:50:58.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.857+0000 7f458a000700 1 -- 172.21.15.107:0/1193639395 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f458001c630 con 0x7f4590071f60 2024-03-04T13:50:58.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.857+0000 7f4598800700 1 -- 172.21.15.107:0/1193639395 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45901141f0 con 0x7f4590071f60 2024-03-04T13:50:58.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.857+0000 7f4598800700 1 -- 172.21.15.107:0/1193639395 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4590114730 con 0x7f4590071f60 2024-03-04T13:50:58.859 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.858+0000 7f4598800700 1 -- 172.21.15.107:0/1193639395 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45900fe400 con 0x7f4590071f60 2024-03-04T13:50:58.859 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.859+0000 7f458a000700 1 -- 172.21.15.107:0/1193639395 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f458000bd80 con 0x7f4590071f60 2024-03-04T13:50:58.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.860+0000 7f458a000700 1 --2- 172.21.15.107:0/1193639395 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f457806d7f0 0x7f457806fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:50:58.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.860+0000 7f458a000700 1 -- 172.21.15.107:0/1193639395 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f458008e130 con 0x7f4590071f60 2024-03-04T13:50:58.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.860+0000 7f4595e00700 1 --2- 172.21.15.107:0/1193639395 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f457806d7f0 0x7f457806fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:50:58.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.861+0000 7f4595e00700 1 --2- 172.21.15.107:0/1193639395 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f457806d7f0 0x7f457806fca0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4584006fd0 tx=0x7f4584008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:50:58.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:58.865+0000 7f458a000700 1 -- 172.21.15.107:0/1193639395 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f458005b620 con 0x7f4590071f60 2024-03-04T13:50:59.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.168+0000 7f4598800700 1 -- 172.21.15.107:0/1193639395 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f459004bf00 con 0x7f4590071f60 2024-03-04T13:50:59.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.169+0000 7f458a000700 1 -- 172.21.15.107:0/1193639395 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+839 (secure 0 0 0) 0x7f458005b1b0 con 0x7f4590071f60 2024-03-04T13:50:59.170 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-03-04T13:50:59.170 INFO:teuthology.orchestra.run.smithi107.stdout: "mon": { 2024-03-04T13:50:59.170 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-1845-g717ce3cc (717ce3cce8d0166e4e577faa8004e6f9cb4128c0) squid (dev)": 2 2024-03-04T13:50:59.170 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:50:59.170 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr": { 2024-03-04T13:50:59.170 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-1845-g717ce3cc (717ce3cce8d0166e4e577faa8004e6f9cb4128c0) squid (dev)": 2 2024-03-04T13:50:59.170 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:50:59.170 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": { 2024-03-04T13:50:59.171 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-545-g42608bb1 (42608bb191117be10dba720e6bf8a02f3171cd34) quincy (stable)": 2, 2024-03-04T13:50:59.171 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-1845-g717ce3cc (717ce3cce8d0166e4e577faa8004e6f9cb4128c0) squid (dev)": 3 2024-03-04T13:50:59.171 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:50:59.171 INFO:teuthology.orchestra.run.smithi107.stdout: "mds": { 2024-03-04T13:50:59.171 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-545-g42608bb1 (42608bb191117be10dba720e6bf8a02f3171cd34) quincy (stable)": 4 2024-03-04T13:50:59.171 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:50:59.171 INFO:teuthology.orchestra.run.smithi107.stdout: "overall": { 2024-03-04T13:50:59.171 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-545-g42608bb1 (42608bb191117be10dba720e6bf8a02f3171cd34) quincy (stable)": 6, 2024-03-04T13:50:59.171 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-1845-g717ce3cc (717ce3cce8d0166e4e577faa8004e6f9cb4128c0) squid (dev)": 7 2024-03-04T13:50:59.171 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-04T13:50:59.171 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-03-04T13:50:59.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.172+0000 7f4598800700 1 -- 172.21.15.107:0/1193639395 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f457806d7f0 msgr2=0x7f457806fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:50:59.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.172+0000 7f4598800700 1 --2- 172.21.15.107:0/1193639395 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f457806d7f0 0x7f457806fca0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4584006fd0 tx=0x7f4584008040 comp rx=0 tx=0).stop 2024-03-04T13:50:59.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.172+0000 7f4598800700 1 -- 172.21.15.107:0/1193639395 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4590071f60 msgr2=0x7f459011a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:50:59.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.172+0000 7f4598800700 1 --2- 172.21.15.107:0/1193639395 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4590071f60 0x7f459011a670 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4580004f50 tx=0x7f4580009e30 comp rx=0 tx=0).stop 2024-03-04T13:50:59.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.172+0000 7f4598800700 1 -- 172.21.15.107:0/1193639395 shutdown_connections 2024-03-04T13:50:59.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.172+0000 7f4598800700 1 --2- 172.21.15.107:0/1193639395 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f457806d7f0 0x7f457806fca0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:50:59.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.172+0000 7f4598800700 1 --2- 172.21.15.107:0/1193639395 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4590070870 0x7f459011a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:50:59.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.172+0000 7f4598800700 1 --2- 172.21.15.107:0/1193639395 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4590071f60 0x7f459011a670 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:50:59.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.172+0000 7f4598800700 1 -- 172.21.15.107:0/1193639395 >> 172.21.15.107:0/1193639395 conn(0x7f45900f8ae0 msgr2=0x7f45900fa850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:50:59.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.172+0000 7f4598800700 1 -- 172.21.15.107:0/1193639395 shutdown_connections 2024-03-04T13:50:59.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.172+0000 7f4598800700 1 -- 172.21.15.107:0/1193639395 wait complete. 2024-03-04T13:50:59.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.320+0000 7f5dfb200700 1 -- 172.21.15.107:0/351841 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5dfc109380 msgr2=0x7f5dfc109750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:50:59.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.320+0000 7f5dfb200700 1 --2- 172.21.15.107:0/351841 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5dfc109380 0x7f5dfc109750 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5dec004940 tx=0x7f5dec009e30 comp rx=0 tx=0).stop 2024-03-04T13:50:59.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.320+0000 7f5dfb200700 1 -- 172.21.15.107:0/351841 shutdown_connections 2024-03-04T13:50:59.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.320+0000 7f5dfb200700 1 --2- 172.21.15.107:0/351841 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5dfc109d20 0x7f5dfc1033a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:50:59.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.320+0000 7f5dfb200700 1 --2- 172.21.15.107:0/351841 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5dfc109380 0x7f5dfc109750 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:50:59.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.320+0000 7f5dfb200700 1 -- 172.21.15.107:0/351841 >> 172.21.15.107:0/351841 conn(0x7f5dfc072b50 msgr2=0x7f5dfc074f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:50:59.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.320+0000 7f5dfb200700 1 -- 172.21.15.107:0/351841 shutdown_connections 2024-03-04T13:50:59.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.320+0000 7f5dfb200700 1 -- 172.21.15.107:0/351841 wait complete. 2024-03-04T13:50:59.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.321+0000 7f5dfb200700 1 Processor -- start 2024-03-04T13:50:59.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.321+0000 7f5dfb200700 1 -- start start 2024-03-04T13:50:59.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.321+0000 7f5dfb200700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5dfc109380 0x7f5dfc119ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:50:59.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.321+0000 7f5dfb200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5dfc109d20 0x7f5dfc11a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:50:59.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.321+0000 7f5dfb200700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dfc11aa30 con 0x7f5dfc109d20 2024-03-04T13:50:59.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.321+0000 7f5dfb200700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dfc113d40 con 0x7f5dfc109380 2024-03-04T13:50:59.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.321+0000 7f5df9400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5dfc109d20 0x7f5dfc11a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:50:59.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.321+0000 7f5df9400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5dfc109d20 0x7f5dfc11a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:35390/0 (socket says 172.21.15.107:35390) 2024-03-04T13:50:59.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.321+0000 7f5df9400700 1 -- 172.21.15.107:0/2478496375 learned_addr learned my addr 172.21.15.107:0/2478496375 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:50:59.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.322+0000 7f5df9e00700 1 --2- 172.21.15.107:0/2478496375 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5dfc109380 0x7f5dfc119ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:50:59.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.322+0000 7f5df9400700 1 -- 172.21.15.107:0/2478496375 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5dfc109380 msgr2=0x7f5dfc119ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:50:59.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.322+0000 7f5df9400700 1 --2- 172.21.15.107:0/2478496375 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5dfc109380 0x7f5dfc119ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:50:59.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.322+0000 7f5df9400700 1 -- 172.21.15.107:0/2478496375 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dec005040 con 0x7f5dfc109d20 2024-03-04T13:50:59.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.322+0000 7f5df9400700 1 --2- 172.21.15.107:0/2478496375 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5dfc109d20 0x7f5dfc11a3e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5de8002a50 tx=0x7f5de8004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:50:59.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.322+0000 7f5df2000700 1 -- 172.21.15.107:0/2478496375 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5de8007cc0 con 0x7f5dfc109d20 2024-03-04T13:50:59.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.322+0000 7f5df2000700 1 -- 172.21.15.107:0/2478496375 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5de800f040 con 0x7f5dfc109d20 2024-03-04T13:50:59.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.322+0000 7f5dfb200700 1 -- 172.21.15.107:0/2478496375 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dfc114020 con 0x7f5dfc109d20 2024-03-04T13:50:59.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.323+0000 7f5df2000700 1 -- 172.21.15.107:0/2478496375 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5de800daf0 con 0x7f5dfc109d20 2024-03-04T13:50:59.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.323+0000 7f5dfb200700 1 -- 172.21.15.107:0/2478496375 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dfc114560 con 0x7f5dfc109d20 2024-03-04T13:50:59.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.324+0000 7f5dfb200700 1 -- 172.21.15.107:0/2478496375 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dfc106690 con 0x7f5dfc109d20 2024-03-04T13:50:59.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.325+0000 7f5df2000700 1 -- 172.21.15.107:0/2478496375 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f5de8009810 con 0x7f5dfc109d20 2024-03-04T13:50:59.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.325+0000 7f5df2000700 1 --2- 172.21.15.107:0/2478496375 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f5de006d7f0 0x7f5de006fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:50:59.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.326+0000 7f5df2000700 1 -- 172.21.15.107:0/2478496375 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f5de808cca0 con 0x7f5dfc109d20 2024-03-04T13:50:59.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.326+0000 7f5df9e00700 1 --2- 172.21.15.107:0/2478496375 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f5de006d7f0 0x7f5de006fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:50:59.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.326+0000 7f5df9e00700 1 --2- 172.21.15.107:0/2478496375 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f5de006d7f0 0x7f5de006fca0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5dec00b3b0 tx=0x7f5dec009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:50:59.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.330+0000 7f5df2000700 1 -- 172.21.15.107:0/2478496375 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5de8059fd0 con 0x7f5dfc109d20 2024-03-04T13:50:59.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.575+0000 7f5dfb200700 1 -- 172.21.15.107:0/2478496375 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f5dfc05f5c0 con 0x7f5dfc109d20 2024-03-04T13:50:59.576 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:59 smithi107 ceph-mon[112578]: from='client.34314 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:50:59.576 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:59 smithi107 ceph-mon[112578]: from='client.34318 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:50:59.576 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:59 smithi107 ceph-mon[112578]: pgmap v127: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 255 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 59/264 objects degraded (22.348%) 2024-03-04T13:50:59.576 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:50:59 smithi107 ceph-mon[112578]: from='client.? 172.21.15.107:0/1193639395' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:50:59.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.576+0000 7f5df2000700 1 -- 172.21.15.107:0/2478496375 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 16 v16) v1 ==== 76+0+1880 (secure 0 0 0) 0x7f5de8059b60 con 0x7f5dfc109d20 2024-03-04T13:50:59.577 INFO:teuthology.orchestra.run.smithi107.stdout:e16 2024-03-04T13:50:59.577 INFO:teuthology.orchestra.run.smithi107.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-04T13:50:59.577 INFO:teuthology.orchestra.run.smithi107.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-04T13:50:59.577 INFO:teuthology.orchestra.run.smithi107.stdout:legacy client fscid: 1 2024-03-04T13:50:59.577 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:50:59.578 INFO:teuthology.orchestra.run.smithi107.stdout:Filesystem 'cephfs' (1) 2024-03-04T13:50:59.578 INFO:teuthology.orchestra.run.smithi107.stdout:fs_name cephfs 2024-03-04T13:50:59.578 INFO:teuthology.orchestra.run.smithi107.stdout:epoch 15 2024-03-04T13:50:59.578 INFO:teuthology.orchestra.run.smithi107.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-04T13:50:59.578 INFO:teuthology.orchestra.run.smithi107.stdout:created 2024-03-04T13:38:30.854004+0000 2024-03-04T13:50:59.578 INFO:teuthology.orchestra.run.smithi107.stdout:modified 2024-03-04T13:38:50.015989+0000 2024-03-04T13:50:59.578 INFO:teuthology.orchestra.run.smithi107.stdout:tableserver 0 2024-03-04T13:50:59.578 INFO:teuthology.orchestra.run.smithi107.stdout:root 0 2024-03-04T13:50:59.578 INFO:teuthology.orchestra.run.smithi107.stdout:session_timeout 60 2024-03-04T13:50:59.578 INFO:teuthology.orchestra.run.smithi107.stdout:session_autoclose 300 2024-03-04T13:50:59.578 INFO:teuthology.orchestra.run.smithi107.stdout:max_file_size 1099511627776 2024-03-04T13:50:59.579 INFO:teuthology.orchestra.run.smithi107.stdout:max_xattr_size 65536 2024-03-04T13:50:59.579 INFO:teuthology.orchestra.run.smithi107.stdout:required_client_features {} 2024-03-04T13:50:59.579 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure 0 2024-03-04T13:50:59.579 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure_osd_epoch 47 2024-03-04T13:50:59.579 INFO:teuthology.orchestra.run.smithi107.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-04T13:50:59.579 INFO:teuthology.orchestra.run.smithi107.stdout:max_mds 1 2024-03-04T13:50:59.579 INFO:teuthology.orchestra.run.smithi107.stdout:in 0 2024-03-04T13:50:59.579 INFO:teuthology.orchestra.run.smithi107.stdout:up {0=24247} 2024-03-04T13:50:59.579 INFO:teuthology.orchestra.run.smithi107.stdout:failed 2024-03-04T13:50:59.579 INFO:teuthology.orchestra.run.smithi107.stdout:damaged 2024-03-04T13:50:59.579 INFO:teuthology.orchestra.run.smithi107.stdout:stopped 2024-03-04T13:50:59.579 INFO:teuthology.orchestra.run.smithi107.stdout:data_pools [3] 2024-03-04T13:50:59.580 INFO:teuthology.orchestra.run.smithi107.stdout:metadata_pool 2 2024-03-04T13:50:59.580 INFO:teuthology.orchestra.run.smithi107.stdout:inline_data enabled 2024-03-04T13:50:59.580 INFO:teuthology.orchestra.run.smithi107.stdout:balancer 2024-03-04T13:50:59.580 INFO:teuthology.orchestra.run.smithi107.stdout:bal_rank_mask -1 2024-03-04T13:50:59.580 INFO:teuthology.orchestra.run.smithi107.stdout:standby_count_wanted 1 2024-03-04T13:50:59.580 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi169.jusudl{0:24247} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.169:6824/2994866076,v1:172.21.15.169:6825/2994866076] compat {c=[1],r=[1],i=[7ff]}] 2024-03-04T13:50:59.580 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.thycho{0:14470} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.107:6828/1985577060,v1:172.21.15.107:6829/1985577060] compat {c=[1],r=[1],i=[7ff]}] 2024-03-04T13:50:59.580 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:50:59.580 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:50:59.580 INFO:teuthology.orchestra.run.smithi107.stdout:Standby daemons: 2024-03-04T13:50:59.581 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:50:59.581 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.riexmm{-1:14480} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.107:6826/3261622517,v1:172.21.15.107:6827/3261622517] compat {c=[1],r=[1],i=[7ff]}] 2024-03-04T13:50:59.581 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi169.bxitzk{-1:24259} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.169:6826/3696090493,v1:172.21.15.169:6827/3696090493] compat {c=[1],r=[1],i=[7ff]}] 2024-03-04T13:50:59.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.579+0000 7f5dfb200700 1 -- 172.21.15.107:0/2478496375 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f5de006d7f0 msgr2=0x7f5de006fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:50:59.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.579+0000 7f5dfb200700 1 --2- 172.21.15.107:0/2478496375 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f5de006d7f0 0x7f5de006fca0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5dec00b3b0 tx=0x7f5dec009e30 comp rx=0 tx=0).stop 2024-03-04T13:50:59.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.579+0000 7f5dfb200700 1 -- 172.21.15.107:0/2478496375 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5dfc109d20 msgr2=0x7f5dfc11a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:50:59.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.579+0000 7f5dfb200700 1 --2- 172.21.15.107:0/2478496375 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5dfc109d20 0x7f5dfc11a3e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5de8002a50 tx=0x7f5de8004600 comp rx=0 tx=0).stop 2024-03-04T13:50:59.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.580+0000 7f5dfb200700 1 -- 172.21.15.107:0/2478496375 shutdown_connections 2024-03-04T13:50:59.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.580+0000 7f5dfb200700 1 --2- 172.21.15.107:0/2478496375 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f5de006d7f0 0x7f5de006fca0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:50:59.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.580+0000 7f5dfb200700 1 --2- 172.21.15.107:0/2478496375 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5dfc109380 0x7f5dfc119ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:50:59.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.580+0000 7f5dfb200700 1 --2- 172.21.15.107:0/2478496375 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5dfc109d20 0x7f5dfc11a3e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:50:59.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.580+0000 7f5dfb200700 1 -- 172.21.15.107:0/2478496375 >> 172.21.15.107:0/2478496375 conn(0x7f5dfc072b50 msgr2=0x7f5dfc075770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:50:59.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.580+0000 7f5dfb200700 1 -- 172.21.15.107:0/2478496375 shutdown_connections 2024-03-04T13:50:59.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.580+0000 7f5dfb200700 1 -- 172.21.15.107:0/2478496375 wait complete. 2024-03-04T13:50:59.583 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 16 2024-03-04T13:50:59.611 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:59 smithi169 ceph-mon[86183]: from='client.34314 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:50:59.611 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:59 smithi169 ceph-mon[86183]: from='client.34318 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:50:59.612 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:59 smithi169 ceph-mon[86183]: pgmap v127: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 255 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 59/264 objects degraded (22.348%) 2024-03-04T13:50:59.612 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:50:59 smithi169 ceph-mon[86183]: from='client.? 172.21.15.107:0/1193639395' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:50:59.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.726+0000 7f792ce00700 1 -- 172.21.15.107:0/236613837 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7928103dc0 msgr2=0x7f7928104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:50:59.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.726+0000 7f792ce00700 1 --2- 172.21.15.107:0/236613837 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7928103dc0 0x7f7928104210 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7914004940 tx=0x7f7914009e30 comp rx=0 tx=0).stop 2024-03-04T13:50:59.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.726+0000 7f792ce00700 1 -- 172.21.15.107:0/236613837 shutdown_connections 2024-03-04T13:50:59.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.726+0000 7f792ce00700 1 --2- 172.21.15.107:0/236613837 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7928103dc0 0x7f7928104210 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:50:59.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.726+0000 7f792ce00700 1 --2- 172.21.15.107:0/236613837 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79281095f0 0x7f79281099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:50:59.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.726+0000 7f792ce00700 1 -- 172.21.15.107:0/236613837 >> 172.21.15.107:0/236613837 conn(0x7f7928072d50 msgr2=0x7f7928075160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:50:59.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.726+0000 7f792ce00700 1 -- 172.21.15.107:0/236613837 shutdown_connections 2024-03-04T13:50:59.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.726+0000 7f792ce00700 1 -- 172.21.15.107:0/236613837 wait complete. 2024-03-04T13:50:59.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.727+0000 7f792ce00700 1 Processor -- start 2024-03-04T13:50:59.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.727+0000 7f792ce00700 1 -- start start 2024-03-04T13:50:59.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.727+0000 7f792ce00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7928103dc0 0x7f792811a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:50:59.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.727+0000 7f792ce00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79281095f0 0x7f792811a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:50:59.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.727+0000 7f792ce00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f792811abd0 con 0x7f7928103dc0 2024-03-04T13:50:59.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.727+0000 7f792ce00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7928113ee0 con 0x7f79281095f0 2024-03-04T13:50:59.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.728+0000 7f7925e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7928103dc0 0x7f792811a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:50:59.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.728+0000 7f7925e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7928103dc0 0x7f792811a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:35420/0 (socket says 172.21.15.107:35420) 2024-03-04T13:50:59.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.728+0000 7f7925e00700 1 -- 172.21.15.107:0/3740358554 learned_addr learned my addr 172.21.15.107:0/3740358554 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:50:59.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.728+0000 7f7925400700 1 --2- 172.21.15.107:0/3740358554 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79281095f0 0x7f792811a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:50:59.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.728+0000 7f7925e00700 1 -- 172.21.15.107:0/3740358554 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79281095f0 msgr2=0x7f792811a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:50:59.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.728+0000 7f7925e00700 1 --2- 172.21.15.107:0/3740358554 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79281095f0 0x7f792811a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:50:59.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.728+0000 7f7925e00700 1 -- 172.21.15.107:0/3740358554 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7914005040 con 0x7f7928103dc0 2024-03-04T13:50:59.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.728+0000 7f7925e00700 1 --2- 172.21.15.107:0/3740358554 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7928103dc0 0x7f792811a040 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7918002a50 tx=0x7f79180046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:50:59.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.729+0000 7f791e000700 1 -- 172.21.15.107:0/3740358554 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7918007cc0 con 0x7f7928103dc0 2024-03-04T13:50:59.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.729+0000 7f791e000700 1 -- 172.21.15.107:0/3740358554 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f791800f040 con 0x7f7928103dc0 2024-03-04T13:50:59.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.729+0000 7f791e000700 1 -- 172.21.15.107:0/3740358554 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f791800da30 con 0x7f7928103dc0 2024-03-04T13:50:59.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.729+0000 7f792ce00700 1 -- 172.21.15.107:0/3740358554 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79281141c0 con 0x7f7928103dc0 2024-03-04T13:50:59.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.729+0000 7f792ce00700 1 -- 172.21.15.107:0/3740358554 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7928114700 con 0x7f7928103dc0 2024-03-04T13:50:59.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.730+0000 7f792ce00700 1 -- 172.21.15.107:0/3740358554 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f792810b9c0 con 0x7f7928103dc0 2024-03-04T13:50:59.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.731+0000 7f791e000700 1 -- 172.21.15.107:0/3740358554 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f791800db90 con 0x7f7928103dc0 2024-03-04T13:50:59.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.731+0000 7f791e000700 1 --2- 172.21.15.107:0/3740358554 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f790c06d7f0 0x7f790c06fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:50:59.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.732+0000 7f791e000700 1 -- 172.21.15.107:0/3740358554 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f791808cc10 con 0x7f7928103dc0 2024-03-04T13:50:59.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.732+0000 7f7925400700 1 --2- 172.21.15.107:0/3740358554 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f790c06d7f0 0x7f790c06fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:50:59.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.732+0000 7f7925400700 1 --2- 172.21.15.107:0/3740358554 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f790c06d7f0 0x7f790c06fca0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f79140043d0 tx=0x7f7914009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:50:59.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.737+0000 7f791e000700 1 -- 172.21.15.107:0/3740358554 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7918059fe0 con 0x7f7928103dc0 2024-03-04T13:50:59.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.966+0000 7f792ce00700 1 -- 172.21.15.107:0/3740358554 --> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f792804bf40 con 0x7f790c06d7f0 2024-03-04T13:50:59.968 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.968+0000 7f791e000700 1 -- 172.21.15.107:0/3740358554 <== mgr.34102 v2:172.21.15.107:6800/1223039812 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f792804bf40 con 0x7f790c06d7f0 2024-03-04T13:50:59.969 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-03-04T13:50:59.969 INFO:teuthology.orchestra.run.smithi107.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377", 2024-03-04T13:50:59.969 INFO:teuthology.orchestra.run.smithi107.stdout: "in_progress": true, 2024-03-04T13:50:59.969 INFO:teuthology.orchestra.run.smithi107.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-04T13:50:59.969 INFO:teuthology.orchestra.run.smithi107.stdout: "services_complete": [ 2024-03-04T13:50:59.969 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr", 2024-03-04T13:50:59.969 INFO:teuthology.orchestra.run.smithi107.stdout: "mon", 2024-03-04T13:50:59.969 INFO:teuthology.orchestra.run.smithi107.stdout: "crash" 2024-03-04T13:50:59.969 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-04T13:50:59.969 INFO:teuthology.orchestra.run.smithi107.stdout: "progress": "9/21 daemons upgraded", 2024-03-04T13:50:59.970 INFO:teuthology.orchestra.run.smithi107.stdout: "message": "Currently upgrading osd daemons", 2024-03-04T13:50:59.970 INFO:teuthology.orchestra.run.smithi107.stdout: "is_paused": false 2024-03-04T13:50:59.970 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-03-04T13:50:59.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.971+0000 7f792ce00700 1 -- 172.21.15.107:0/3740358554 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f790c06d7f0 msgr2=0x7f790c06fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:50:59.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.971+0000 7f792ce00700 1 --2- 172.21.15.107:0/3740358554 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f790c06d7f0 0x7f790c06fca0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f79140043d0 tx=0x7f7914009e30 comp rx=0 tx=0).stop 2024-03-04T13:50:59.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.971+0000 7f792ce00700 1 -- 172.21.15.107:0/3740358554 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7928103dc0 msgr2=0x7f792811a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:50:59.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.971+0000 7f792ce00700 1 --2- 172.21.15.107:0/3740358554 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7928103dc0 0x7f792811a040 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7918002a50 tx=0x7f79180046b0 comp rx=0 tx=0).stop 2024-03-04T13:50:59.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.971+0000 7f792ce00700 1 -- 172.21.15.107:0/3740358554 shutdown_connections 2024-03-04T13:50:59.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.971+0000 7f792ce00700 1 --2- 172.21.15.107:0/3740358554 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f790c06d7f0 0x7f790c06fca0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:50:59.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.971+0000 7f792ce00700 1 --2- 172.21.15.107:0/3740358554 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7928103dc0 0x7f792811a040 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:50:59.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.971+0000 7f792ce00700 1 --2- 172.21.15.107:0/3740358554 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79281095f0 0x7f792811a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:50:59.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.971+0000 7f792ce00700 1 -- 172.21.15.107:0/3740358554 >> 172.21.15.107:0/3740358554 conn(0x7f7928072d50 msgr2=0x7f7928074850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:50:59.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.972+0000 7f792ce00700 1 -- 172.21.15.107:0/3740358554 shutdown_connections 2024-03-04T13:50:59.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:50:59.972+0000 7f792ce00700 1 -- 172.21.15.107:0/3740358554 wait complete. 2024-03-04T13:51:00.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:00.111+0000 7ff4ca600700 1 -- 172.21.15.107:0/2271268967 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff4c40702b0 msgr2=0x7ff4c4070680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:51:00.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:00.111+0000 7ff4ca600700 1 --2- 172.21.15.107:0/2271268967 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff4c40702b0 0x7ff4c4070680 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff4b4004940 tx=0x7ff4b4009e30 comp rx=0 tx=0).stop 2024-03-04T13:51:00.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:00.111+0000 7ff4ca600700 1 -- 172.21.15.107:0/2271268967 shutdown_connections 2024-03-04T13:51:00.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:00.111+0000 7ff4ca600700 1 --2- 172.21.15.107:0/2271268967 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4c4070bc0 0x7ff4c41092e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:51:00.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:00.111+0000 7ff4ca600700 1 --2- 172.21.15.107:0/2271268967 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff4c40702b0 0x7ff4c4070680 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:51:00.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:00.111+0000 7ff4ca600700 1 -- 172.21.15.107:0/2271268967 >> 172.21.15.107:0/2271268967 conn(0x7ff4c40f8ae0 msgr2=0x7ff4c40faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:51:00.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:00.112+0000 7ff4ca600700 1 -- 172.21.15.107:0/2271268967 shutdown_connections 2024-03-04T13:51:00.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:00.112+0000 7ff4ca600700 1 -- 172.21.15.107:0/2271268967 wait complete. 2024-03-04T13:51:00.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:00.112+0000 7ff4ca600700 1 Processor -- start 2024-03-04T13:51:00.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:00.112+0000 7ff4ca600700 1 -- start start 2024-03-04T13:51:00.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:00.113+0000 7ff4ca600700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4c40702b0 0x7ff4c4115bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:51:00.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:00.113+0000 7ff4ca600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff4c4070bc0 0x7ff4c4116130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:51:00.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:00.113+0000 7ff4ca600700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c4116780 con 0x7ff4c4070bc0 2024-03-04T13:51:00.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:00.113+0000 7ff4ca600700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c410fad0 con 0x7ff4c40702b0 2024-03-04T13:51:00.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:00.113+0000 7ff4c2a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff4c4070bc0 0x7ff4c4116130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:51:00.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:00.113+0000 7ff4c2a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff4c4070bc0 0x7ff4c4116130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:35438/0 (socket says 172.21.15.107:35438) 2024-03-04T13:51:00.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:00.113+0000 7ff4c2a00700 1 -- 172.21.15.107:0/3242356749 learned_addr learned my addr 172.21.15.107:0/3242356749 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:51:00.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:00.113+0000 7ff4c3400700 1 --2- 172.21.15.107:0/3242356749 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4c40702b0 0x7ff4c4115bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:51:00.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:00.113+0000 7ff4c2a00700 1 -- 172.21.15.107:0/3242356749 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4c40702b0 msgr2=0x7ff4c4115bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:51:00.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:00.113+0000 7ff4c2a00700 1 --2- 172.21.15.107:0/3242356749 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4c40702b0 0x7ff4c4115bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:51:00.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:00.113+0000 7ff4c2a00700 1 -- 172.21.15.107:0/3242356749 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4b4005040 con 0x7ff4c4070bc0 2024-03-04T13:51:00.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:00.114+0000 7ff4c2a00700 1 --2- 172.21.15.107:0/3242356749 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff4c4070bc0 0x7ff4c4116130 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff4b000eb40 tx=0x7ff4b000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:51:00.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:00.114+0000 7ff4bb400700 1 -- 172.21.15.107:0/3242356749 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff4b0007cc0 con 0x7ff4c4070bc0 2024-03-04T13:51:00.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:00.114+0000 7ff4ca600700 1 -- 172.21.15.107:0/3242356749 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4c410fd50 con 0x7ff4c4070bc0 2024-03-04T13:51:00.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:00.114+0000 7ff4bb400700 1 -- 172.21.15.107:0/3242356749 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff4b000f040 con 0x7ff4c4070bc0 2024-03-04T13:51:00.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:00.114+0000 7ff4bb400700 1 -- 172.21.15.107:0/3242356749 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff4b000ca10 con 0x7ff4c4070bc0 2024-03-04T13:51:00.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:00.114+0000 7ff4ca600700 1 -- 172.21.15.107:0/3242356749 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4c4110290 con 0x7ff4c4070bc0 2024-03-04T13:51:00.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:00.115+0000 7ff4ca600700 1 -- 172.21.15.107:0/3242356749 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4c41069a0 con 0x7ff4c4070bc0 2024-03-04T13:51:00.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:00.116+0000 7ff4bb400700 1 -- 172.21.15.107:0/3242356749 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7ff4b000cb70 con 0x7ff4c4070bc0 2024-03-04T13:51:00.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:00.117+0000 7ff4bb400700 1 --2- 172.21.15.107:0/3242356749 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7ff4a806d7e0 0x7ff4a806fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:51:00.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:00.117+0000 7ff4bb400700 1 -- 172.21.15.107:0/3242356749 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6324+0+0 (secure 0 0 0) 0x7ff4b008ca00 con 0x7ff4c4070bc0 2024-03-04T13:51:00.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:00.117+0000 7ff4c3400700 1 --2- 172.21.15.107:0/3242356749 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7ff4a806d7e0 0x7ff4a806fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:51:00.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:00.118+0000 7ff4c3400700 1 --2- 172.21.15.107:0/3242356749 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7ff4a806d7e0 0x7ff4a806fc90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff4b400b0d0 tx=0x7ff4b4009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:51:00.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:00.122+0000 7ff4bb400700 1 -- 172.21.15.107:0/3242356749 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff4b0056330 con 0x7ff4c4070bc0 2024-03-04T13:51:00.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:00.410+0000 7ff4ca600700 1 -- 172.21.15.107:0/3242356749 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7ff4c404bed0 con 0x7ff4c4070bc0 2024-03-04T13:51:00.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:00.410+0000 7ff4bb400700 1 -- 172.21.15.107:0/3242356749 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+1461 (secure 0 0 0) 0x7ff4b00599f0 con 0x7ff4c4070bc0 2024-03-04T13:51:00.411 INFO:teuthology.orchestra.run.smithi107.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data; 1 osds down; Degraded data redundancy: 59/264 objects degraded (22.348%), 18 pgs degraded 2024-03-04T13:51:00.411 INFO:teuthology.orchestra.run.smithi107.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-04T13:51:00.411 INFO:teuthology.orchestra.run.smithi107.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-04T13:51:00.411 INFO:teuthology.orchestra.run.smithi107.stdout:[WRN] OSD_DOWN: 1 osds down 2024-03-04T13:51:00.412 INFO:teuthology.orchestra.run.smithi107.stdout: osd.3 (root=default,host=smithi169) is down 2024-03-04T13:51:00.412 INFO:teuthology.orchestra.run.smithi107.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 59/264 objects degraded (22.348%), 18 pgs degraded 2024-03-04T13:51:00.412 INFO:teuthology.orchestra.run.smithi107.stdout: pg 1.0 is active+undersized+degraded, acting [0,1] 2024-03-04T13:51:00.412 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.0 is active+undersized+degraded, acting [1,0] 2024-03-04T13:51:00.412 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.5 is active+undersized+degraded, acting [0,4] 2024-03-04T13:51:00.412 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.6 is active+undersized+degraded, acting [1,4] 2024-03-04T13:51:00.412 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.7 is active+undersized+degraded, acting [4,2] 2024-03-04T13:51:00.412 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.8 is active+undersized+degraded, acting [5,0] 2024-03-04T13:51:00.412 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.a is active+undersized+degraded, acting [1,4] 2024-03-04T13:51:00.412 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.b is active+undersized+degraded, acting [4,5] 2024-03-04T13:51:00.412 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.c is active+undersized+degraded, acting [2,0] 2024-03-04T13:51:00.413 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.d is active+undersized+degraded, acting [1,2] 2024-03-04T13:51:00.413 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.e is active+undersized+degraded, acting [2,0] 2024-03-04T13:51:00.413 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.14 is active+undersized+degraded, acting [4,5] 2024-03-04T13:51:00.413 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.15 is active+undersized+degraded, acting [1,0] 2024-03-04T13:51:00.413 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.16 is active+undersized+degraded, acting [5,2] 2024-03-04T13:51:00.413 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.18 is active+undersized+degraded, acting [5,4] 2024-03-04T13:51:00.413 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.1a is active+undersized+degraded, acting [4,5] 2024-03-04T13:51:00.413 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.1d is active+undersized+degraded, acting [5,0] 2024-03-04T13:51:00.413 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.1f is active+undersized+degraded, acting [0,4] 2024-03-04T13:51:00.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:00.413+0000 7ff4ca600700 1 -- 172.21.15.107:0/3242356749 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7ff4a806d7e0 msgr2=0x7ff4a806fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:51:00.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:00.413+0000 7ff4ca600700 1 --2- 172.21.15.107:0/3242356749 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7ff4a806d7e0 0x7ff4a806fc90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff4b400b0d0 tx=0x7ff4b4009e30 comp rx=0 tx=0).stop 2024-03-04T13:51:00.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:00.413+0000 7ff4ca600700 1 -- 172.21.15.107:0/3242356749 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff4c4070bc0 msgr2=0x7ff4c4116130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:51:00.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:00.413+0000 7ff4ca600700 1 --2- 172.21.15.107:0/3242356749 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff4c4070bc0 0x7ff4c4116130 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff4b000eb40 tx=0x7ff4b000b040 comp rx=0 tx=0).stop 2024-03-04T13:51:00.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:00.413+0000 7ff4ca600700 1 -- 172.21.15.107:0/3242356749 shutdown_connections 2024-03-04T13:51:00.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:00.413+0000 7ff4ca600700 1 --2- 172.21.15.107:0/3242356749 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7ff4a806d7e0 0x7ff4a806fc90 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:51:00.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:00.413+0000 7ff4ca600700 1 --2- 172.21.15.107:0/3242356749 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff4c40702b0 0x7ff4c4115bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:51:00.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:00.413+0000 7ff4ca600700 1 --2- 172.21.15.107:0/3242356749 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff4c4070bc0 0x7ff4c4116130 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:51:00.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:00.413+0000 7ff4ca600700 1 -- 172.21.15.107:0/3242356749 >> 172.21.15.107:0/3242356749 conn(0x7ff4c40f8ae0 msgr2=0x7ff4c4103b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:51:00.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:00.414+0000 7ff4ca600700 1 -- 172.21.15.107:0/3242356749 shutdown_connections 2024-03-04T13:51:00.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:00.414+0000 7ff4ca600700 1 -- 172.21.15.107:0/3242356749 wait complete. 2024-03-04T13:51:00.594 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:00 smithi107 ceph-mon[112578]: Health check update: Degraded data redundancy: 59/264 objects degraded (22.348%), 18 pgs degraded (PG_DEGRADED) 2024-03-04T13:51:00.594 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:00 smithi107 ceph-mon[112578]: from='client.? 172.21.15.107:0/2478496375' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-04T13:51:00.610 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:00 smithi169 ceph-mon[86183]: Health check update: Degraded data redundancy: 59/264 objects degraded (22.348%), 18 pgs degraded (PG_DEGRADED) 2024-03-04T13:51:00.611 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:00 smithi169 ceph-mon[86183]: from='client.? 172.21.15.107:0/2478496375' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-04T13:51:01.594 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:01 smithi107 ceph-mon[112578]: from='client.34330 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:51:01.594 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:01 smithi107 ceph-mon[112578]: from='client.? 172.21.15.107:0/3242356749' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-04T13:51:01.594 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:01 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:51:01.594 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:01 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:51:01.594 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:01 smithi107 ceph-mon[112578]: pgmap v128: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 255 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 59/264 objects degraded (22.348%) 2024-03-04T13:51:01.611 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:01 smithi169 ceph-mon[86183]: from='client.34330 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:51:01.611 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:01 smithi169 ceph-mon[86183]: from='client.? 172.21.15.107:0/3242356749' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-04T13:51:01.611 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:01 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:51:01.611 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:01 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:51:01.611 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:01 smithi169 ceph-mon[86183]: pgmap v128: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 255 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 59/264 objects degraded (22.348%) 2024-03-04T13:51:02.111 INFO:journalctl@ceph.osd.3.smithi169.stdout:Mar 04 13:51:01 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-3[95252]: 2024-03-04T13:51:01.720+0000 7f384c83d7c0 -1 osd.3 0 read_superblock omap replica is missing. 2024-03-04T13:51:02.501 INFO:journalctl@ceph.osd.3.smithi169.stdout:Mar 04 13:51:02 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-3[95252]: 2024-03-04T13:51:02.178+0000 7f384c83d7c0 -1 osd.3 65 log_to_monitors true 2024-03-04T13:51:02.844 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:02 smithi107 ceph-mon[112578]: from='osd.3 [v2:172.21.15.169:6800/3021051994,v1:172.21.15.169:6801/3021051994]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-04T13:51:02.844 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:02 smithi107 ceph-mon[112578]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-04T13:51:02.844 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:02 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:51:02.845 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:02 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:51:02.861 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:02 smithi169 ceph-mon[86183]: from='osd.3 [v2:172.21.15.169:6800/3021051994,v1:172.21.15.169:6801/3021051994]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-04T13:51:02.861 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:02 smithi169 ceph-mon[86183]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-04T13:51:02.861 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:02 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:51:02.861 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:02 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:51:03.844 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:03 smithi107 ceph-mon[112578]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-04T13:51:03.844 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:03 smithi107 ceph-mon[112578]: osdmap e68: 6 total, 5 up, 6 in 2024-03-04T13:51:03.844 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:03 smithi107 ceph-mon[112578]: from='osd.3 [v2:172.21.15.169:6800/3021051994,v1:172.21.15.169:6801/3021051994]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-03-04T13:51:03.844 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:03 smithi107 ceph-mon[112578]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-03-04T13:51:03.844 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:03 smithi107 ceph-mon[112578]: pgmap v130: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 255 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s; 59/264 objects degraded (22.348%) 2024-03-04T13:51:03.861 INFO:journalctl@ceph.osd.3.smithi169.stdout:Mar 04 13:51:03 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-3[95252]: 2024-03-04T13:51:03.506+0000 7f3836c00700 -1 osd.3 65 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-04T13:51:03.861 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:03 smithi169 ceph-mon[86183]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-04T13:51:03.861 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:03 smithi169 ceph-mon[86183]: osdmap e68: 6 total, 5 up, 6 in 2024-03-04T13:51:03.862 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:03 smithi169 ceph-mon[86183]: from='osd.3 [v2:172.21.15.169:6800/3021051994,v1:172.21.15.169:6801/3021051994]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-03-04T13:51:03.862 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:03 smithi169 ceph-mon[86183]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-03-04T13:51:03.862 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:03 smithi169 ceph-mon[86183]: pgmap v130: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 255 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s; 59/264 objects degraded (22.348%) 2024-03-04T13:51:04.611 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:04 smithi169 ceph-mon[86183]: from='osd.3 ' entity='osd.3' 2024-03-04T13:51:04.611 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:04 smithi169 ceph-mon[86183]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-04T13:51:04.611 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:04 smithi169 ceph-mon[86183]: osd.3 [v2:172.21.15.169:6800/3021051994,v1:172.21.15.169:6801/3021051994] boot 2024-03-04T13:51:04.611 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:04 smithi169 ceph-mon[86183]: osdmap e69: 6 total, 6 up, 6 in 2024-03-04T13:51:04.611 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:04 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-04T13:51:04.844 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:04 smithi107 ceph-mon[112578]: from='osd.3 ' entity='osd.3' 2024-03-04T13:51:04.844 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:04 smithi107 ceph-mon[112578]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-04T13:51:04.844 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:04 smithi107 ceph-mon[112578]: osd.3 [v2:172.21.15.169:6800/3021051994,v1:172.21.15.169:6801/3021051994] boot 2024-03-04T13:51:04.844 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:04 smithi107 ceph-mon[112578]: osdmap e69: 6 total, 6 up, 6 in 2024-03-04T13:51:04.844 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:04 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-04T13:51:05.844 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:05 smithi107 ceph-mon[112578]: pgmap v132: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 255 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 59/264 objects degraded (22.348%) 2024-03-04T13:51:05.844 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:05 smithi107 ceph-mon[112578]: osdmap e70: 6 total, 6 up, 6 in 2024-03-04T13:51:05.861 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:05 smithi169 ceph-mon[86183]: pgmap v132: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 255 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 59/264 objects degraded (22.348%) 2024-03-04T13:51:05.861 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:05 smithi169 ceph-mon[86183]: osdmap e70: 6 total, 6 up, 6 in 2024-03-04T13:51:07.091 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:06 smithi169 ceph-mon[86183]: Health check update: Degraded data redundancy: 33/264 objects degraded (12.500%), 11 pgs degraded (PG_DEGRADED) 2024-03-04T13:51:07.097 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:06 smithi107 ceph-mon[112578]: Health check update: Degraded data redundancy: 33/264 objects degraded (12.500%), 11 pgs degraded (PG_DEGRADED) 2024-03-04T13:51:08.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:07 smithi107 ceph-mon[112578]: pgmap v134: 65 pgs: 11 active+undersized+degraded, 8 active+undersized, 46 active+clean; 255 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 33/264 objects degraded (12.500%) 2024-03-04T13:51:08.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:07 smithi169 ceph-mon[86183]: pgmap v134: 65 pgs: 11 active+undersized+degraded, 8 active+undersized, 46 active+clean; 255 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 33/264 objects degraded (12.500%) 2024-03-04T13:51:09.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:08 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:51:09.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:08 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:51:09.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:08 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:51:09.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:08 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:51:09.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:08 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:51:09.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:08 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:51:09.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:08 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:51:09.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:08 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:51:09.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:08 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:51:09.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:08 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-04T13:51:09.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:08 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:51:09.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:08 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:51:09.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:08 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:51:09.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:08 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:51:09.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:08 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:51:09.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:08 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:51:09.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:08 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:51:09.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:08 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:51:09.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:08 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:51:09.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:08 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-04T13:51:10.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:09 smithi169 ceph-mon[86183]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-04T13:51:10.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:09 smithi169 ceph-mon[86183]: Upgrade: unsafe to stop osd(s) at this time (10 PGs are or would become offline) 2024-03-04T13:51:10.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:09 smithi169 ceph-mon[86183]: pgmap v135: 65 pgs: 65 active+clean; 255 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 659 B/s rd, 1 op/s 2024-03-04T13:51:10.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:09 smithi169 ceph-mon[86183]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 33/264 objects degraded (12.500%), 11 pgs degraded) 2024-03-04T13:51:10.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:09 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:51:10.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:09 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:51:10.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:09 smithi107 ceph-mon[112578]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-04T13:51:10.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:09 smithi107 ceph-mon[112578]: Upgrade: unsafe to stop osd(s) at this time (10 PGs are or would become offline) 2024-03-04T13:51:10.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:09 smithi107 ceph-mon[112578]: pgmap v135: 65 pgs: 65 active+clean; 255 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 659 B/s rd, 1 op/s 2024-03-04T13:51:10.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:09 smithi107 ceph-mon[112578]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 33/264 objects degraded (12.500%), 11 pgs degraded) 2024-03-04T13:51:10.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:09 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:51:10.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:09 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:51:12.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:11 smithi169 ceph-mon[86183]: pgmap v136: 65 pgs: 65 active+clean; 255 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 511 B/s rd, 0 op/s 2024-03-04T13:51:12.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:11 smithi107 ceph-mon[112578]: pgmap v136: 65 pgs: 65 active+clean; 255 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 511 B/s rd, 0 op/s 2024-03-04T13:51:14.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:13 smithi107 ceph-mon[112578]: pgmap v137: 65 pgs: 65 active+clean; 255 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 2 op/s 2024-03-04T13:51:14.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:13 smithi169 ceph-mon[86183]: pgmap v137: 65 pgs: 65 active+clean; 255 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 2 op/s 2024-03-04T13:51:16.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:15 smithi107 ceph-mon[112578]: pgmap v138: 65 pgs: 65 active+clean; 255 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 921 B/s rd, 1 op/s 2024-03-04T13:51:16.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:15 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:51:16.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:15 smithi169 ceph-mon[86183]: pgmap v138: 65 pgs: 65 active+clean; 255 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 921 B/s rd, 1 op/s 2024-03-04T13:51:16.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:15 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:51:18.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:17 smithi107 ceph-mon[112578]: pgmap v139: 65 pgs: 65 active+clean; 255 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 823 B/s rd, 1 op/s 2024-03-04T13:51:18.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:17 smithi169 ceph-mon[86183]: pgmap v139: 65 pgs: 65 active+clean; 255 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 823 B/s rd, 1 op/s 2024-03-04T13:51:20.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:19 smithi107 ceph-mon[112578]: pgmap v140: 65 pgs: 65 active+clean; 255 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:51:20.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:19 smithi169 ceph-mon[86183]: pgmap v140: 65 pgs: 65 active+clean; 255 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:51:22.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:21 smithi107 ceph-mon[112578]: pgmap v141: 65 pgs: 65 active+clean; 255 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:51:22.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:21 smithi169 ceph-mon[86183]: pgmap v141: 65 pgs: 65 active+clean; 255 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:51:24.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:23 smithi107 ceph-mon[112578]: pgmap v142: 65 pgs: 65 active+clean; 255 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:51:24.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:23 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-04T13:51:24.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:23 smithi169 ceph-mon[86183]: pgmap v142: 65 pgs: 65 active+clean; 255 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:51:24.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:23 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-04T13:51:25.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:24 smithi107 ceph-mon[112578]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-04T13:51:25.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:24 smithi107 ceph-mon[112578]: Upgrade: osd.4 is safe to restart 2024-03-04T13:51:25.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:24 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:51:25.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:24 smithi169 ceph-mon[86183]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-04T13:51:25.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:24 smithi169 ceph-mon[86183]: Upgrade: osd.4 is safe to restart 2024-03-04T13:51:25.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:24 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:51:26.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:25 smithi107 ceph-mon[112578]: pgmap v143: 65 pgs: 65 active+clean; 255 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:51:26.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:25 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:51:26.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:25 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-04T13:51:26.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:25 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:51:26.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:25 smithi169 ceph-mon[86183]: pgmap v143: 65 pgs: 65 active+clean; 255 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:51:26.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:25 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:51:26.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:25 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-04T13:51:26.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:25 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:51:27.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:26 smithi169 ceph-mon[86183]: Upgrade: Updating osd.4 2024-03-04T13:51:27.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:26 smithi169 ceph-mon[86183]: Deploying daemon osd.4 on smithi169 2024-03-04T13:51:27.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:26 smithi107 ceph-mon[112578]: Upgrade: Updating osd.4 2024-03-04T13:51:27.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:26 smithi107 ceph-mon[112578]: Deploying daemon osd.4 on smithi169 2024-03-04T13:51:28.037 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:27 smithi169 ceph-mon[86183]: pgmap v144: 65 pgs: 65 active+clean; 255 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:51:28.333 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:27 smithi107 ceph-mon[112578]: pgmap v144: 65 pgs: 65 active+clean; 255 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:51:30.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:29 smithi169 ceph-mon[86183]: pgmap v145: 65 pgs: 65 active+clean; 255 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:51:30.111 INFO:journalctl@ceph.osd.4.smithi169.stdout:Mar 04 13:51:29 smithi169 systemd[1]: Stopping Ceph osd.4 for 1fc09a58-da2b-11ee-95c6-87774f69a715... 2024-03-04T13:51:30.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:29 smithi107 ceph-mon[112578]: pgmap v145: 65 pgs: 65 active+clean; 255 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:51:30.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:30.576+0000 7f767bc00700 1 -- 172.21.15.107:0/430007905 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7674105b00 msgr2=0x7f76740ff170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:51:30.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:30.576+0000 7f767bc00700 1 --2- 172.21.15.107:0/430007905 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7674105b00 0x7f76740ff170 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7660004940 tx=0x7f7660009e30 comp rx=0 tx=0).stop 2024-03-04T13:51:30.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:30.576+0000 7f767bc00700 1 -- 172.21.15.107:0/430007905 shutdown_connections 2024-03-04T13:51:30.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:30.576+0000 7f767bc00700 1 --2- 172.21.15.107:0/430007905 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7674105b00 0x7f76740ff170 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:51:30.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:30.576+0000 7f767bc00700 1 --2- 172.21.15.107:0/430007905 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7674105160 0x7f7674105530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:51:30.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:30.576+0000 7f767bc00700 1 -- 172.21.15.107:0/430007905 >> 172.21.15.107:0/430007905 conn(0x7f76740fad30 msgr2=0x7f76740fd140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:51:30.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:30.576+0000 7f767bc00700 1 -- 172.21.15.107:0/430007905 shutdown_connections 2024-03-04T13:51:30.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:30.577+0000 7f767bc00700 1 -- 172.21.15.107:0/430007905 wait complete. 2024-03-04T13:51:30.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:30.577+0000 7f767bc00700 1 Processor -- start 2024-03-04T13:51:30.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:30.578+0000 7f767bc00700 1 -- start start 2024-03-04T13:51:30.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:30.578+0000 7f767bc00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7674105160 0x7f7674115c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:51:30.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:30.578+0000 7f767bc00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7674105b00 0x7f76741161b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:51:30.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:30.578+0000 7f767bc00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7674116890 con 0x7f7674105b00 2024-03-04T13:51:30.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:30.578+0000 7f767bc00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f767410fb30 con 0x7f7674105160 2024-03-04T13:51:30.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:30.578+0000 7f7673e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7674105b00 0x7f76741161b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:51:30.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:30.578+0000 7f7673e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7674105b00 0x7f76741161b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40106/0 (socket says 172.21.15.107:40106) 2024-03-04T13:51:30.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:30.578+0000 7f7673e00700 1 -- 172.21.15.107:0/1773627427 learned_addr learned my addr 172.21.15.107:0/1773627427 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:51:30.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:30.579+0000 7f7679200700 1 --2- 172.21.15.107:0/1773627427 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7674105160 0x7f7674115c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:51:30.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:30.579+0000 7f7673e00700 1 -- 172.21.15.107:0/1773627427 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7674105160 msgr2=0x7f7674115c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:51:30.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:30.579+0000 7f7673e00700 1 --2- 172.21.15.107:0/1773627427 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7674105160 0x7f7674115c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:51:30.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:30.579+0000 7f7673e00700 1 -- 172.21.15.107:0/1773627427 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7660005040 con 0x7f7674105b00 2024-03-04T13:51:30.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:30.579+0000 7f7673e00700 1 --2- 172.21.15.107:0/1773627427 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7674105b00 0x7f76741161b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7660009030 tx=0x7f7660009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:51:30.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:30.579+0000 7f7670c00700 1 -- 172.21.15.107:0/1773627427 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f766000e040 con 0x7f7674105b00 2024-03-04T13:51:30.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:30.579+0000 7f7670c00700 1 -- 172.21.15.107:0/1773627427 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f766000bbf0 con 0x7f7674105b00 2024-03-04T13:51:30.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:30.579+0000 7f7670c00700 1 -- 172.21.15.107:0/1773627427 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f766001c630 con 0x7f7674105b00 2024-03-04T13:51:30.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:30.579+0000 7f767bc00700 1 -- 172.21.15.107:0/1773627427 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f767410fdb0 con 0x7f7674105b00 2024-03-04T13:51:30.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:30.580+0000 7f767bc00700 1 -- 172.21.15.107:0/1773627427 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76741102f0 con 0x7f7674105b00 2024-03-04T13:51:30.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:30.581+0000 7f767bc00700 1 -- 172.21.15.107:0/1773627427 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7674102470 con 0x7f7674105b00 2024-03-04T13:51:30.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:30.582+0000 7f7670c00700 1 -- 172.21.15.107:0/1773627427 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f766000bd60 con 0x7f7674105b00 2024-03-04T13:51:30.584 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:30.583+0000 7f7670c00700 1 --2- 172.21.15.107:0/1773627427 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f765c06d7f0 0x7f765c06fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:51:30.584 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:30.583+0000 7f7670c00700 1 -- 172.21.15.107:0/1773627427 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f766008e1d0 con 0x7f7674105b00 2024-03-04T13:51:30.584 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:30.583+0000 7f7679200700 1 --2- 172.21.15.107:0/1773627427 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f765c06d7f0 0x7f765c06fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:51:30.585 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:30.584+0000 7f7679200700 1 --2- 172.21.15.107:0/1773627427 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f765c06d7f0 0x7f765c06fca0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7668006fd0 tx=0x7f7668008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:51:30.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:30.588+0000 7f7670c00700 1 -- 172.21.15.107:0/1773627427 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f766005b620 con 0x7f7674105b00 2024-03-04T13:51:30.611 INFO:journalctl@ceph.osd.4.smithi169.stdout:Mar 04 13:51:30 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-4[48277]: 2024-03-04T13:51:30.272+0000 7fe808000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-04T13:51:30.612 INFO:journalctl@ceph.osd.4.smithi169.stdout:Mar 04 13:51:30 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-4[48277]: 2024-03-04T13:51:30.272+0000 7fe808000700 -1 osd.4 70 *** Got signal Terminated *** 2024-03-04T13:51:30.612 INFO:journalctl@ceph.osd.4.smithi169.stdout:Mar 04 13:51:30 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-4[48277]: 2024-03-04T13:51:30.272+0000 7fe808000700 -1 osd.4 70 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-04T13:51:30.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:30.829+0000 7f767bc00700 1 -- 172.21.15.107:0/1773627427 --> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f767404bf70 con 0x7f765c06d7f0 2024-03-04T13:51:30.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:30.831+0000 7f7670c00700 1 -- 172.21.15.107:0/1773627427 <== mgr.34102 v2:172.21.15.107:6800/1223039812 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f7668007dd0 con 0x7f765c06d7f0 2024-03-04T13:51:30.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:30.834+0000 7f767bc00700 1 -- 172.21.15.107:0/1773627427 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f765c06d7f0 msgr2=0x7f765c06fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:51:30.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:30.834+0000 7f767bc00700 1 --2- 172.21.15.107:0/1773627427 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f765c06d7f0 0x7f765c06fca0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7668006fd0 tx=0x7f7668008040 comp rx=0 tx=0).stop 2024-03-04T13:51:30.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:30.834+0000 7f767bc00700 1 -- 172.21.15.107:0/1773627427 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7674105b00 msgr2=0x7f76741161b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:51:30.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:30.834+0000 7f767bc00700 1 --2- 172.21.15.107:0/1773627427 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7674105b00 0x7f76741161b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7660009030 tx=0x7f7660009e30 comp rx=0 tx=0).stop 2024-03-04T13:51:30.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:30.834+0000 7f767bc00700 1 -- 172.21.15.107:0/1773627427 shutdown_connections 2024-03-04T13:51:30.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:30.834+0000 7f767bc00700 1 --2- 172.21.15.107:0/1773627427 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f765c06d7f0 0x7f765c06fca0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:51:30.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:30.834+0000 7f767bc00700 1 --2- 172.21.15.107:0/1773627427 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7674105160 0x7f7674115c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:51:30.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:30.834+0000 7f767bc00700 1 --2- 172.21.15.107:0/1773627427 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7674105b00 0x7f76741161b0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:51:30.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:30.834+0000 7f767bc00700 1 -- 172.21.15.107:0/1773627427 >> 172.21.15.107:0/1773627427 conn(0x7f76740fad30 msgr2=0x7f76740fc7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:51:30.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:30.834+0000 7f767bc00700 1 -- 172.21.15.107:0/1773627427 shutdown_connections 2024-03-04T13:51:30.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:30.834+0000 7f767bc00700 1 -- 172.21.15.107:0/1773627427 wait complete. 2024-03-04T13:51:30.851 INFO:teuthology.orchestra.run.smithi107.stdout:true 2024-03-04T13:51:30.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:30.991+0000 7f9f68000700 1 -- 172.21.15.107:0/34968562 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f60105b10 msgr2=0x7f9f600ff180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:51:30.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:30.991+0000 7f9f68000700 1 --2- 172.21.15.107:0/34968562 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f60105b10 0x7f9f600ff180 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c004940 tx=0x7f9f4c009e30 comp rx=0 tx=0).stop 2024-03-04T13:51:30.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:30.991+0000 7f9f68000700 1 -- 172.21.15.107:0/34968562 shutdown_connections 2024-03-04T13:51:30.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:30.991+0000 7f9f68000700 1 --2- 172.21.15.107:0/34968562 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f60105b10 0x7f9f600ff180 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:51:30.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:30.991+0000 7f9f68000700 1 --2- 172.21.15.107:0/34968562 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f60105170 0x7f9f60105540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:51:30.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:30.991+0000 7f9f68000700 1 -- 172.21.15.107:0/34968562 >> 172.21.15.107:0/34968562 conn(0x7f9f600face0 msgr2=0x7f9f600fd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:51:30.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:30.991+0000 7f9f68000700 1 -- 172.21.15.107:0/34968562 shutdown_connections 2024-03-04T13:51:30.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:30.991+0000 7f9f68000700 1 -- 172.21.15.107:0/34968562 wait complete. 2024-03-04T13:51:30.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:30.992+0000 7f9f68000700 1 Processor -- start 2024-03-04T13:51:30.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:30.992+0000 7f9f68000700 1 -- start start 2024-03-04T13:51:30.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:30.993+0000 7f9f68000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f60105170 0x7f9f60115ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:51:30.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:30.993+0000 7f9f68000700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f60105b10 0x7f9f60116220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:51:30.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:30.993+0000 7f9f68000700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f60116870 con 0x7f9f60105170 2024-03-04T13:51:30.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:30.993+0000 7f9f68000700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f6010fb80 con 0x7f9f60105b10 2024-03-04T13:51:30.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:30.993+0000 7f9f65600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f60105170 0x7f9f60115ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:51:30.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:30.993+0000 7f9f65600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f60105170 0x7f9f60115ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40122/0 (socket says 172.21.15.107:40122) 2024-03-04T13:51:30.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:30.993+0000 7f9f65600700 1 -- 172.21.15.107:0/3771537691 learned_addr learned my addr 172.21.15.107:0/3771537691 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:51:30.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:30.993+0000 7f9f64c00700 1 --2- 172.21.15.107:0/3771537691 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f60105b10 0x7f9f60116220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:51:30.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:30.994+0000 7f9f65600700 1 -- 172.21.15.107:0/3771537691 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f60105b10 msgr2=0x7f9f60116220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:51:30.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:30.994+0000 7f9f65600700 1 --2- 172.21.15.107:0/3771537691 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f60105b10 0x7f9f60116220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:51:30.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:30.994+0000 7f9f65600700 1 -- 172.21.15.107:0/3771537691 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f4c005040 con 0x7f9f60105170 2024-03-04T13:51:30.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:30.994+0000 7f9f65600700 1 --2- 172.21.15.107:0/3771537691 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f60105170 0x7f9f60115ce0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f9f5400c830 tx=0x7f9f54004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:51:30.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:30.994+0000 7f9f5d600700 1 -- 172.21.15.107:0/3771537691 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f5400a760 con 0x7f9f60105170 2024-03-04T13:51:30.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:30.994+0000 7f9f68000700 1 -- 172.21.15.107:0/3771537691 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f6010fe60 con 0x7f9f60105170 2024-03-04T13:51:30.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:30.994+0000 7f9f5d600700 1 -- 172.21.15.107:0/3771537691 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9f5400ad90 con 0x7f9f60105170 2024-03-04T13:51:30.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:30.994+0000 7f9f5d600700 1 -- 172.21.15.107:0/3771537691 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f540113a0 con 0x7f9f60105170 2024-03-04T13:51:30.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:30.994+0000 7f9f68000700 1 -- 172.21.15.107:0/3771537691 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f601103a0 con 0x7f9f60105170 2024-03-04T13:51:30.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:30.996+0000 7f9f68000700 1 -- 172.21.15.107:0/3771537691 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f60107640 con 0x7f9f60105170 2024-03-04T13:51:30.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:30.997+0000 7f9f5d600700 1 -- 172.21.15.107:0/3771537691 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f9f54012400 con 0x7f9f60105170 2024-03-04T13:51:30.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:30.997+0000 7f9f5d600700 1 --2- 172.21.15.107:0/3771537691 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f9f4806d7f0 0x7f9f4806fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:51:30.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:30.998+0000 7f9f5d600700 1 -- 172.21.15.107:0/3771537691 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f9f5408ceb0 con 0x7f9f60105170 2024-03-04T13:51:30.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:30.998+0000 7f9f64c00700 1 --2- 172.21.15.107:0/3771537691 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f9f4806d7f0 0x7f9f4806fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:51:30.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:30.998+0000 7f9f64c00700 1 --2- 172.21.15.107:0/3771537691 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f9f4806d7f0 0x7f9f4806fca0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c00b020 tx=0x7f9f4c009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:51:31.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:31.002+0000 7f9f5d600700 1 -- 172.21.15.107:0/3771537691 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9f5405a300 con 0x7f9f60105170 2024-03-04T13:51:31.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:30 smithi107 ceph-mon[112578]: osd.4 marked itself down and dead 2024-03-04T13:51:31.244 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:31.243+0000 7f9f68000700 1 -- 172.21.15.107:0/3771537691 --> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9f601112c0 con 0x7f9f4806d7f0 2024-03-04T13:51:31.247 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:31.245+0000 7f9f5d600700 1 -- 172.21.15.107:0/3771537691 <== mgr.34102 v2:172.21.15.107:6800/1223039812 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f9f601112c0 con 0x7f9f4806d7f0 2024-03-04T13:51:31.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:31.248+0000 7f9f68000700 1 -- 172.21.15.107:0/3771537691 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f9f4806d7f0 msgr2=0x7f9f4806fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:51:31.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:31.248+0000 7f9f68000700 1 --2- 172.21.15.107:0/3771537691 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f9f4806d7f0 0x7f9f4806fca0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c00b020 tx=0x7f9f4c009e30 comp rx=0 tx=0).stop 2024-03-04T13:51:31.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:31.248+0000 7f9f68000700 1 -- 172.21.15.107:0/3771537691 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f60105170 msgr2=0x7f9f60115ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:51:31.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:31.248+0000 7f9f68000700 1 --2- 172.21.15.107:0/3771537691 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f60105170 0x7f9f60115ce0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f9f5400c830 tx=0x7f9f54004470 comp rx=0 tx=0).stop 2024-03-04T13:51:31.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:31.248+0000 7f9f68000700 1 -- 172.21.15.107:0/3771537691 shutdown_connections 2024-03-04T13:51:31.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:31.248+0000 7f9f68000700 1 --2- 172.21.15.107:0/3771537691 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f9f4806d7f0 0x7f9f4806fca0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:51:31.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:31.248+0000 7f9f68000700 1 --2- 172.21.15.107:0/3771537691 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f60105170 0x7f9f60115ce0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:51:31.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:31.248+0000 7f9f68000700 1 --2- 172.21.15.107:0/3771537691 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f60105b10 0x7f9f60116220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:51:31.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:31.248+0000 7f9f68000700 1 -- 172.21.15.107:0/3771537691 >> 172.21.15.107:0/3771537691 conn(0x7f9f600face0 msgr2=0x7f9f600fc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:51:31.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:31.249+0000 7f9f68000700 1 -- 172.21.15.107:0/3771537691 shutdown_connections 2024-03-04T13:51:31.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:31.249+0000 7f9f68000700 1 -- 172.21.15.107:0/3771537691 wait complete. 2024-03-04T13:51:31.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:30 smithi169 ceph-mon[86183]: osd.4 marked itself down and dead 2024-03-04T13:51:31.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:31.386+0000 7f336aa00700 1 -- 172.21.15.107:0/3792875967 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33641095f0 msgr2=0x7f33641099c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:51:31.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:31.386+0000 7f336aa00700 1 --2- 172.21.15.107:0/3792875967 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33641095f0 0x7f33641099c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3354004940 tx=0x7f3354009e30 comp rx=0 tx=0).stop 2024-03-04T13:51:31.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:31.386+0000 7f336aa00700 1 -- 172.21.15.107:0/3792875967 shutdown_connections 2024-03-04T13:51:31.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:31.386+0000 7f336aa00700 1 --2- 172.21.15.107:0/3792875967 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3364103dc0 0x7f3364104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:51:31.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:31.386+0000 7f336aa00700 1 --2- 172.21.15.107:0/3792875967 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33641095f0 0x7f33641099c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:51:31.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:31.386+0000 7f336aa00700 1 -- 172.21.15.107:0/3792875967 >> 172.21.15.107:0/3792875967 conn(0x7f3364072d50 msgr2=0x7f3364075160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:51:31.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:31.387+0000 7f336aa00700 1 -- 172.21.15.107:0/3792875967 shutdown_connections 2024-03-04T13:51:31.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:31.387+0000 7f336aa00700 1 -- 172.21.15.107:0/3792875967 wait complete. 2024-03-04T13:51:31.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:31.387+0000 7f336aa00700 1 Processor -- start 2024-03-04T13:51:31.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:31.388+0000 7f336aa00700 1 -- start start 2024-03-04T13:51:31.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:31.388+0000 7f336aa00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3364103dc0 0x7f336411a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:51:31.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:31.388+0000 7f336aa00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33641095f0 0x7f336411a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:51:31.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:31.388+0000 7f336aa00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f336411ac70 con 0x7f3364103dc0 2024-03-04T13:51:31.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:31.388+0000 7f336aa00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3364113f80 con 0x7f33641095f0 2024-03-04T13:51:31.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:31.388+0000 7f3363e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3364103dc0 0x7f336411a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:51:31.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:31.388+0000 7f3363e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3364103dc0 0x7f336411a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40130/0 (socket says 172.21.15.107:40130) 2024-03-04T13:51:31.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:31.388+0000 7f3363e00700 1 -- 172.21.15.107:0/1721414215 learned_addr learned my addr 172.21.15.107:0/1721414215 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:51:31.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:31.389+0000 7f3363400700 1 --2- 172.21.15.107:0/1721414215 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33641095f0 0x7f336411a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:51:31.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:31.389+0000 7f3363e00700 1 -- 172.21.15.107:0/1721414215 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33641095f0 msgr2=0x7f336411a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:51:31.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:31.389+0000 7f3363e00700 1 --2- 172.21.15.107:0/1721414215 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33641095f0 0x7f336411a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:51:31.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:31.389+0000 7f3363e00700 1 -- 172.21.15.107:0/1721414215 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3354005040 con 0x7f3364103dc0 2024-03-04T13:51:31.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:31.389+0000 7f3363e00700 1 --2- 172.21.15.107:0/1721414215 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3364103dc0 0x7f336411a0e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3354009fd0 tx=0x7f3354009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:51:31.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:31.389+0000 7f335be00700 1 -- 172.21.15.107:0/1721414215 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f335400e040 con 0x7f3364103dc0 2024-03-04T13:51:31.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:31.389+0000 7f336aa00700 1 -- 172.21.15.107:0/1721414215 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3364114200 con 0x7f3364103dc0 2024-03-04T13:51:31.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:31.389+0000 7f335be00700 1 -- 172.21.15.107:0/1721414215 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f335400bbf0 con 0x7f3364103dc0 2024-03-04T13:51:31.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:31.389+0000 7f335be00700 1 -- 172.21.15.107:0/1721414215 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f335401c3f0 con 0x7f3364103dc0 2024-03-04T13:51:31.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:31.389+0000 7f336aa00700 1 -- 172.21.15.107:0/1721414215 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3364114740 con 0x7f3364103dc0 2024-03-04T13:51:31.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:31.390+0000 7f336aa00700 1 -- 172.21.15.107:0/1721414215 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3364106850 con 0x7f3364103dc0 2024-03-04T13:51:31.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:31.391+0000 7f335be00700 1 -- 172.21.15.107:0/1721414215 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f335400bd60 con 0x7f3364103dc0 2024-03-04T13:51:31.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:31.392+0000 7f335be00700 1 --2- 172.21.15.107:0/1721414215 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f334806d7f0 0x7f334806fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:51:31.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:31.392+0000 7f335be00700 1 -- 172.21.15.107:0/1721414215 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f335408e1c0 con 0x7f3364103dc0 2024-03-04T13:51:31.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:31.392+0000 7f3363400700 1 --2- 172.21.15.107:0/1721414215 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f334806d7f0 0x7f334806fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:51:31.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:31.393+0000 7f3363400700 1 --2- 172.21.15.107:0/1721414215 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f334806d7f0 0x7f334806fca0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3350007fd0 tx=0x7f3350009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:51:31.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:31.397+0000 7f335be00700 1 -- 172.21.15.107:0/1721414215 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f335405b610 con 0x7f3364103dc0 2024-03-04T13:51:31.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:31.623+0000 7f336aa00700 1 -- 172.21.15.107:0/1721414215 --> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f336404bed0 con 0x7f334806d7f0 2024-03-04T13:51:31.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:31.641+0000 7f335be00700 1 -- 172.21.15.107:0/1721414215 <== mgr.34102 v2:172.21.15.107:6800/1223039812 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f336404bed0 con 0x7f334806d7f0 2024-03-04T13:51:31.643 INFO:teuthology.orchestra.run.smithi107.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-04T13:51:31.643 INFO:teuthology.orchestra.run.smithi107.stdout:alertmanager.smithi107 smithi107 *:9093,9094 running (5m) 58s ago 19m 14.6M - 0.25.0 c8568f914cd2 e5ed7be32538 2024-03-04T13:51:31.643 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi107 smithi107 running (2m) 58s ago 20m 6660k - 19.0.0-1845-g717ce3cc d71b839abdf8 0558022b0d84 2024-03-04T13:51:31.643 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi169 smithi169 running (2m) 31s ago 18m 6665k - 19.0.0-1845-g717ce3cc d71b839abdf8 221ca53aa0d3 2024-03-04T13:51:31.643 INFO:teuthology.orchestra.run.smithi107.stdout:grafana.smithi107 smithi107 *:3000 running (5m) 58s ago 19m 74.5M - 9.4.12 a0ea5e3c46d9 c5e4d33fb599 2024-03-04T13:51:31.643 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.riexmm smithi107 running (12m) 58s ago 12m 14.9M - 17.2.7-545-g42608bb1 9eec3f22859d fc921f7d4328 2024-03-04T13:51:31.644 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.thycho smithi107 running (12m) 58s ago 12m 98.3M - 17.2.7-545-g42608bb1 9eec3f22859d 4069c6e0e063 2024-03-04T13:51:31.644 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi169.bxitzk smithi169 running (12m) 31s ago 12m 14.7M - 17.2.7-545-g42608bb1 9eec3f22859d 7d4115228c7c 2024-03-04T13:51:31.644 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi169.jusudl smithi169 running (12m) 31s ago 12m 145M - 17.2.7-545-g42608bb1 9eec3f22859d 2d2f2cc452e8 2024-03-04T13:51:31.644 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi107.wpzgwl smithi107 *:8443,9283,8765 running (8m) 58s ago 21m 504M - 19.0.0-1845-g717ce3cc d71b839abdf8 1c5fdee45eac 2024-03-04T13:51:31.644 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi169.eihczr smithi169 *:8443,9283,8765 running (7m) 31s ago 18m 451M - 19.0.0-1845-g717ce3cc d71b839abdf8 b5cac2b7d22a 2024-03-04T13:51:31.644 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi107 smithi107 running (4m) 58s ago 21m 46.5M 2048M 19.0.0-1845-g717ce3cc d71b839abdf8 32f07144274f 2024-03-04T13:51:31.644 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi169 smithi169 running (3m) 31s ago 18m 35.7M 2048M 19.0.0-1845-g717ce3cc d71b839abdf8 e3527449381a 2024-03-04T13:51:31.644 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi107 smithi107 *:9100 running (6m) 58s ago 20m 7680k - 1.5.0 0da6a335fe13 8a8d3d4f724f 2024-03-04T13:51:31.644 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi169 smithi169 *:9100 running (6m) 31s ago 18m 7680k - 1.5.0 0da6a335fe13 f240f5a975bc 2024-03-04T13:51:31.644 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0 smithi107 running (2m) 58s ago 17m 80.5M 1558M 19.0.0-1845-g717ce3cc d71b839abdf8 5801a5540dc6 2024-03-04T13:51:31.644 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1 smithi107 running (97s) 58s ago 17m 91.3M 1558M 19.0.0-1845-g717ce3cc d71b839abdf8 b614af658854 2024-03-04T13:51:31.644 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2 smithi107 running (66s) 58s ago 16m 11.1M 1558M 19.0.0-1845-g717ce3cc d71b839abdf8 6e4da4f6f146 2024-03-04T13:51:31.644 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3 smithi169 running (36s) 31s ago 15m 11.1M 2582M 19.0.0-1845-g717ce3cc d71b839abdf8 d31baf764d7a 2024-03-04T13:51:31.645 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4 smithi169 running (15m) 31s ago 15m 209M 2582M 17.2.7-545-g42608bb1 9eec3f22859d 08545e3d0659 2024-03-04T13:51:31.645 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5 smithi169 running (14m) 31s ago 14m 193M 2582M 17.2.7-545-g42608bb1 9eec3f22859d dcffe6cb3eac 2024-03-04T13:51:31.645 INFO:teuthology.orchestra.run.smithi107.stdout:prometheus.smithi107 smithi107 *:9095 running (6m) 58s ago 19m 51.0M - 2.43.0 a07b618ecd1d 1eeb1ad5cb75 2024-03-04T13:51:31.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:31.644+0000 7f336aa00700 1 -- 172.21.15.107:0/1721414215 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f334806d7f0 msgr2=0x7f334806fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:51:31.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:31.644+0000 7f336aa00700 1 --2- 172.21.15.107:0/1721414215 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f334806d7f0 0x7f334806fca0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3350007fd0 tx=0x7f3350009040 comp rx=0 tx=0).stop 2024-03-04T13:51:31.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:31.644+0000 7f336aa00700 1 -- 172.21.15.107:0/1721414215 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3364103dc0 msgr2=0x7f336411a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:51:31.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:31.644+0000 7f336aa00700 1 --2- 172.21.15.107:0/1721414215 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3364103dc0 0x7f336411a0e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3354009fd0 tx=0x7f3354009e30 comp rx=0 tx=0).stop 2024-03-04T13:51:31.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:31.644+0000 7f336aa00700 1 -- 172.21.15.107:0/1721414215 shutdown_connections 2024-03-04T13:51:31.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:31.644+0000 7f336aa00700 1 --2- 172.21.15.107:0/1721414215 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f334806d7f0 0x7f334806fca0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:51:31.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:31.645+0000 7f336aa00700 1 --2- 172.21.15.107:0/1721414215 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3364103dc0 0x7f336411a0e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:51:31.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:31.645+0000 7f336aa00700 1 --2- 172.21.15.107:0/1721414215 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f33641095f0 0x7f336411a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:51:31.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:31.645+0000 7f336aa00700 1 -- 172.21.15.107:0/1721414215 >> 172.21.15.107:0/1721414215 conn(0x7f3364072d50 msgr2=0x7f3364074790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:51:31.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:31.645+0000 7f336aa00700 1 -- 172.21.15.107:0/1721414215 shutdown_connections 2024-03-04T13:51:31.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:31.645+0000 7f336aa00700 1 -- 172.21.15.107:0/1721414215 wait complete. 2024-03-04T13:51:31.738 INFO:journalctl@ceph.osd.4.smithi169.stdout:Mar 04 13:51:31 smithi169 podman[99675]: 2024-03-04 13:51:31.374733331 +0000 UTC m=+1.227890592 container died 08545e3d065937e35962b59384d3efb6ac28eea647ac72c1bb296bdd9ecbf627 (image=quay.ceph.io/ceph-ci/ceph@sha256:ee85afea73f504aaae524393a00a3b140e2a1cfd81d76da4525eaa6e73286a7d, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-4, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=quincy-42608bb, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240222, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-03-04T13:51:31.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:31.796+0000 7fdac5e00700 1 -- 172.21.15.107:0/1062159620 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdac0070be0 msgr2=0x7fdac01092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:51:31.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:31.796+0000 7fdac5e00700 1 --2- 172.21.15.107:0/1062159620 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdac0070be0 0x7fdac01092c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fdaac004970 tx=0x7fdaac009e30 comp rx=0 tx=0).stop 2024-03-04T13:51:31.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:31.796+0000 7fdac5e00700 1 -- 172.21.15.107:0/1062159620 shutdown_connections 2024-03-04T13:51:31.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:31.796+0000 7fdac5e00700 1 --2- 172.21.15.107:0/1062159620 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdac0070be0 0x7fdac01092c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:51:31.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:31.796+0000 7fdac5e00700 1 --2- 172.21.15.107:0/1062159620 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdac00702d0 0x7fdac00706a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:51:31.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:31.796+0000 7fdac5e00700 1 -- 172.21.15.107:0/1062159620 >> 172.21.15.107:0/1062159620 conn(0x7fdac00f8ae0 msgr2=0x7fdac00faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:51:31.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:31.797+0000 7fdac5e00700 1 -- 172.21.15.107:0/1062159620 shutdown_connections 2024-03-04T13:51:31.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:31.797+0000 7fdac5e00700 1 -- 172.21.15.107:0/1062159620 wait complete. 2024-03-04T13:51:31.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:31.797+0000 7fdac5e00700 1 Processor -- start 2024-03-04T13:51:31.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:31.797+0000 7fdac5e00700 1 -- start start 2024-03-04T13:51:31.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:31.798+0000 7fdac5e00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdac00702d0 0x7fdac0115c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:51:31.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:31.798+0000 7fdac5e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdac0070be0 0x7fdac0116150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:51:31.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:31.798+0000 7fdac5e00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdac01167a0 con 0x7fdac0070be0 2024-03-04T13:51:31.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:31.798+0000 7fdac5e00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdac010faf0 con 0x7fdac00702d0 2024-03-04T13:51:31.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:31.798+0000 7fdabe000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdac0070be0 0x7fdac0116150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:51:31.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:31.798+0000 7fdabe000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdac0070be0 0x7fdac0116150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40138/0 (socket says 172.21.15.107:40138) 2024-03-04T13:51:31.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:31.798+0000 7fdabe000700 1 -- 172.21.15.107:0/1662846535 learned_addr learned my addr 172.21.15.107:0/1662846535 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:51:31.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:31.798+0000 7fdabea00700 1 --2- 172.21.15.107:0/1662846535 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdac00702d0 0x7fdac0115c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:51:31.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:31.799+0000 7fdabe000700 1 -- 172.21.15.107:0/1662846535 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdac00702d0 msgr2=0x7fdac0115c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:51:31.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:31.799+0000 7fdabe000700 1 --2- 172.21.15.107:0/1662846535 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdac00702d0 0x7fdac0115c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:51:31.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:31.799+0000 7fdabe000700 1 -- 172.21.15.107:0/1662846535 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaac005040 con 0x7fdac0070be0 2024-03-04T13:51:31.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:31.799+0000 7fdabe000700 1 --2- 172.21.15.107:0/1662846535 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdac0070be0 0x7fdac0116150 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fdaac000c00 tx=0x7fdaac009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:51:31.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:31.799+0000 7fdab6a00700 1 -- 172.21.15.107:0/1662846535 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdaac00e040 con 0x7fdac0070be0 2024-03-04T13:51:31.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:31.799+0000 7fdab6a00700 1 -- 172.21.15.107:0/1662846535 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdaac00bce0 con 0x7fdac0070be0 2024-03-04T13:51:31.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:31.799+0000 7fdab6a00700 1 -- 172.21.15.107:0/1662846535 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdaac01c630 con 0x7fdac0070be0 2024-03-04T13:51:31.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:31.799+0000 7fdac5e00700 1 -- 172.21.15.107:0/1662846535 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdac010fd70 con 0x7fdac0070be0 2024-03-04T13:51:31.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:31.799+0000 7fdac5e00700 1 -- 172.21.15.107:0/1662846535 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdac01102b0 con 0x7fdac0070be0 2024-03-04T13:51:31.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:31.801+0000 7fdac5e00700 1 -- 172.21.15.107:0/1662846535 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdac0106980 con 0x7fdac0070be0 2024-03-04T13:51:31.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:31.802+0000 7fdab6a00700 1 -- 172.21.15.107:0/1662846535 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fdaac0139f0 con 0x7fdac0070be0 2024-03-04T13:51:31.804 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:31.803+0000 7fdab6a00700 1 --2- 172.21.15.107:0/1662846535 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fdaa406d7e0 0x7fdaa406fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:51:31.804 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:31.803+0000 7fdab6a00700 1 -- 172.21.15.107:0/1662846535 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fdaac08e180 con 0x7fdac0070be0 2024-03-04T13:51:31.804 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:31.803+0000 7fdabea00700 1 --2- 172.21.15.107:0/1662846535 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fdaa406d7e0 0x7fdaa406fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:51:31.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:31.804+0000 7fdabea00700 1 --2- 172.21.15.107:0/1662846535 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fdaa406d7e0 0x7fdaa406fc90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fdab0004990 tx=0x7fdab0009ee0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:51:31.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:31.808+0000 7fdab6a00700 1 -- 172.21.15.107:0/1662846535 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fdaac057ac0 con 0x7fdac0070be0 2024-03-04T13:51:32.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:31 smithi107 ceph-mon[112578]: pgmap v146: 65 pgs: 65 active+clean; 255 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:51:32.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:31 smithi107 ceph-mon[112578]: from='client.34340 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:51:32.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:31 smithi107 ceph-mon[112578]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-04T13:51:32.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:31 smithi107 ceph-mon[112578]: osdmap e71: 6 total, 5 up, 6 in 2024-03-04T13:51:32.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:31 smithi169 ceph-mon[86183]: pgmap v146: 65 pgs: 65 active+clean; 255 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:51:32.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:31 smithi169 ceph-mon[86183]: from='client.34340 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:51:32.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:31 smithi169 ceph-mon[86183]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-04T13:51:32.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:31 smithi169 ceph-mon[86183]: osdmap e71: 6 total, 5 up, 6 in 2024-03-04T13:51:32.112 INFO:journalctl@ceph.osd.4.smithi169.stdout:Mar 04 13:51:31 smithi169 podman[99675]: 2024-03-04 13:51:31.882967947 +0000 UTC m=+1.736125260 container cleanup 08545e3d065937e35962b59384d3efb6ac28eea647ac72c1bb296bdd9ecbf627 (image=quay.ceph.io/ceph-ci/ceph@sha256:ee85afea73f504aaae524393a00a3b140e2a1cfd81d76da4525eaa6e73286a7d, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-4, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=quincy-42608bb, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-03-04T13:51:32.112 INFO:journalctl@ceph.osd.4.smithi169.stdout:Mar 04 13:51:31 smithi169 bash[99675]: ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-4 2024-03-04T13:51:32.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.118+0000 7fdac5e00700 1 -- 172.21.15.107:0/1662846535 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fdac004bed0 con 0x7fdac0070be0 2024-03-04T13:51:32.121 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.119+0000 7fdab6a00700 1 -- 172.21.15.107:0/1662846535 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+839 (secure 0 0 0) 0x7fdaac004de0 con 0x7fdac0070be0 2024-03-04T13:51:32.121 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-03-04T13:51:32.121 INFO:teuthology.orchestra.run.smithi107.stdout: "mon": { 2024-03-04T13:51:32.121 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-1845-g717ce3cc (717ce3cce8d0166e4e577faa8004e6f9cb4128c0) squid (dev)": 2 2024-03-04T13:51:32.121 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:51:32.121 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr": { 2024-03-04T13:51:32.121 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-1845-g717ce3cc (717ce3cce8d0166e4e577faa8004e6f9cb4128c0) squid (dev)": 2 2024-03-04T13:51:32.122 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:51:32.122 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": { 2024-03-04T13:51:32.122 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-545-g42608bb1 (42608bb191117be10dba720e6bf8a02f3171cd34) quincy (stable)": 1, 2024-03-04T13:51:32.122 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-1845-g717ce3cc (717ce3cce8d0166e4e577faa8004e6f9cb4128c0) squid (dev)": 4 2024-03-04T13:51:32.122 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:51:32.122 INFO:teuthology.orchestra.run.smithi107.stdout: "mds": { 2024-03-04T13:51:32.122 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-545-g42608bb1 (42608bb191117be10dba720e6bf8a02f3171cd34) quincy (stable)": 4 2024-03-04T13:51:32.122 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:51:32.122 INFO:teuthology.orchestra.run.smithi107.stdout: "overall": { 2024-03-04T13:51:32.122 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-545-g42608bb1 (42608bb191117be10dba720e6bf8a02f3171cd34) quincy (stable)": 5, 2024-03-04T13:51:32.122 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-1845-g717ce3cc (717ce3cce8d0166e4e577faa8004e6f9cb4128c0) squid (dev)": 8 2024-03-04T13:51:32.122 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-04T13:51:32.122 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-03-04T13:51:32.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.122+0000 7fdac5e00700 1 -- 172.21.15.107:0/1662846535 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fdaa406d7e0 msgr2=0x7fdaa406fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:51:32.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.122+0000 7fdac5e00700 1 --2- 172.21.15.107:0/1662846535 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fdaa406d7e0 0x7fdaa406fc90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fdab0004990 tx=0x7fdab0009ee0 comp rx=0 tx=0).stop 2024-03-04T13:51:32.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.122+0000 7fdac5e00700 1 -- 172.21.15.107:0/1662846535 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdac0070be0 msgr2=0x7fdac0116150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:51:32.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.122+0000 7fdac5e00700 1 --2- 172.21.15.107:0/1662846535 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdac0070be0 0x7fdac0116150 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fdaac000c00 tx=0x7fdaac009e30 comp rx=0 tx=0).stop 2024-03-04T13:51:32.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.122+0000 7fdac5e00700 1 -- 172.21.15.107:0/1662846535 shutdown_connections 2024-03-04T13:51:32.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.122+0000 7fdac5e00700 1 --2- 172.21.15.107:0/1662846535 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fdaa406d7e0 0x7fdaa406fc90 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:51:32.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.122+0000 7fdac5e00700 1 --2- 172.21.15.107:0/1662846535 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdac00702d0 0x7fdac0115c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:51:32.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.122+0000 7fdac5e00700 1 --2- 172.21.15.107:0/1662846535 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdac0070be0 0x7fdac0116150 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:51:32.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.122+0000 7fdac5e00700 1 -- 172.21.15.107:0/1662846535 >> 172.21.15.107:0/1662846535 conn(0x7fdac00f8ae0 msgr2=0x7fdac0103b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:51:32.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.123+0000 7fdac5e00700 1 -- 172.21.15.107:0/1662846535 shutdown_connections 2024-03-04T13:51:32.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.123+0000 7fdac5e00700 1 -- 172.21.15.107:0/1662846535 wait complete. 2024-03-04T13:51:32.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.272+0000 7f50c5c00700 1 -- 172.21.15.107:0/937403394 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f50c0105910 msgr2=0x7f50c00fef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:51:32.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.272+0000 7f50c5c00700 1 --2- 172.21.15.107:0/937403394 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f50c0105910 0x7f50c00fef80 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f50ac004940 tx=0x7f50ac009e30 comp rx=0 tx=0).stop 2024-03-04T13:51:32.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.272+0000 7f50c5c00700 1 -- 172.21.15.107:0/937403394 shutdown_connections 2024-03-04T13:51:32.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.272+0000 7f50c5c00700 1 --2- 172.21.15.107:0/937403394 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f50c0105910 0x7f50c00fef80 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:51:32.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.272+0000 7f50c5c00700 1 --2- 172.21.15.107:0/937403394 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f50c0104f70 0x7f50c0105340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:51:32.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.272+0000 7f50c5c00700 1 -- 172.21.15.107:0/937403394 >> 172.21.15.107:0/937403394 conn(0x7f50c00faae0 msgr2=0x7f50c00fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:51:32.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.272+0000 7f50c5c00700 1 -- 172.21.15.107:0/937403394 shutdown_connections 2024-03-04T13:51:32.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.272+0000 7f50c5c00700 1 -- 172.21.15.107:0/937403394 wait complete. 2024-03-04T13:51:32.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.273+0000 7f50c5c00700 1 Processor -- start 2024-03-04T13:51:32.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.273+0000 7f50c5c00700 1 -- start start 2024-03-04T13:51:32.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.273+0000 7f50c5c00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f50c0104f70 0x7f50c0115a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:51:32.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.273+0000 7f50c5c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f50c0105910 0x7f50c0115f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:51:32.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.273+0000 7f50c5c00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50c01165a0 con 0x7f50c0105910 2024-03-04T13:51:32.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.273+0000 7f50c5c00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50c010f8d0 con 0x7f50c0104f70 2024-03-04T13:51:32.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.274+0000 7f50bf400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f50c0105910 0x7f50c0115f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:51:32.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.274+0000 7f50bf400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f50c0105910 0x7f50c0115f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40164/0 (socket says 172.21.15.107:40164) 2024-03-04T13:51:32.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.274+0000 7f50bf400700 1 -- 172.21.15.107:0/1561689827 learned_addr learned my addr 172.21.15.107:0/1561689827 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:51:32.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.274+0000 7f50bfe00700 1 --2- 172.21.15.107:0/1561689827 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f50c0104f70 0x7f50c0115a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:51:32.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.274+0000 7f50bf400700 1 -- 172.21.15.107:0/1561689827 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f50c0104f70 msgr2=0x7f50c0115a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:51:32.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.274+0000 7f50bf400700 1 --2- 172.21.15.107:0/1561689827 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f50c0104f70 0x7f50c0115a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:51:32.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.274+0000 7f50bf400700 1 -- 172.21.15.107:0/1561689827 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50ac005040 con 0x7f50c0105910 2024-03-04T13:51:32.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.275+0000 7f50bf400700 1 --2- 172.21.15.107:0/1561689827 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f50c0105910 0x7f50c0115f50 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f50ac009030 tx=0x7f50ac009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:51:32.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.275+0000 7f50b7e00700 1 -- 172.21.15.107:0/1561689827 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f50ac00e040 con 0x7f50c0105910 2024-03-04T13:51:32.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.275+0000 7f50b7e00700 1 -- 172.21.15.107:0/1561689827 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f50ac00bbf0 con 0x7f50c0105910 2024-03-04T13:51:32.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.275+0000 7f50b7e00700 1 -- 172.21.15.107:0/1561689827 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f50ac01c630 con 0x7f50c0105910 2024-03-04T13:51:32.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.275+0000 7f50c5c00700 1 -- 172.21.15.107:0/1561689827 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50c010fb50 con 0x7f50c0105910 2024-03-04T13:51:32.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.275+0000 7f50c5c00700 1 -- 172.21.15.107:0/1561689827 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50c01100c0 con 0x7f50c0105910 2024-03-04T13:51:32.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.276+0000 7f50c5c00700 1 -- 172.21.15.107:0/1561689827 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50c005f5c0 con 0x7f50c0105910 2024-03-04T13:51:32.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.277+0000 7f50b7e00700 1 -- 172.21.15.107:0/1561689827 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f50ac00bd60 con 0x7f50c0105910 2024-03-04T13:51:32.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.278+0000 7f50b7e00700 1 --2- 172.21.15.107:0/1561689827 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f50a406d7f0 0x7f50a406fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:51:32.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.278+0000 7f50b7e00700 1 -- 172.21.15.107:0/1561689827 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f50ac08e270 con 0x7f50c0105910 2024-03-04T13:51:32.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.278+0000 7f50bfe00700 1 --2- 172.21.15.107:0/1561689827 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f50a406d7f0 0x7f50a406fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:51:32.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.279+0000 7f50bfe00700 1 --2- 172.21.15.107:0/1561689827 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f50a406d7f0 0x7f50a406fca0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f50b0007fd0 tx=0x7f50b0009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:51:32.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.283+0000 7f50b7e00700 1 -- 172.21.15.107:0/1561689827 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f50ac05b620 con 0x7f50c0105910 2024-03-04T13:51:32.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.548+0000 7f50c5c00700 1 -- 172.21.15.107:0/1561689827 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f50c004bed0 con 0x7f50c0105910 2024-03-04T13:51:32.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.550+0000 7f50b7e00700 1 -- 172.21.15.107:0/1561689827 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 16 v16) v1 ==== 76+0+1880 (secure 0 0 0) 0x7f50ac05b1b0 con 0x7f50c0105910 2024-03-04T13:51:32.552 INFO:teuthology.orchestra.run.smithi107.stdout:e16 2024-03-04T13:51:32.552 INFO:teuthology.orchestra.run.smithi107.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-04T13:51:32.552 INFO:teuthology.orchestra.run.smithi107.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-04T13:51:32.552 INFO:teuthology.orchestra.run.smithi107.stdout:legacy client fscid: 1 2024-03-04T13:51:32.552 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:51:32.552 INFO:teuthology.orchestra.run.smithi107.stdout:Filesystem 'cephfs' (1) 2024-03-04T13:51:32.552 INFO:teuthology.orchestra.run.smithi107.stdout:fs_name cephfs 2024-03-04T13:51:32.553 INFO:teuthology.orchestra.run.smithi107.stdout:epoch 15 2024-03-04T13:51:32.553 INFO:teuthology.orchestra.run.smithi107.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-04T13:51:32.553 INFO:teuthology.orchestra.run.smithi107.stdout:created 2024-03-04T13:38:30.854004+0000 2024-03-04T13:51:32.553 INFO:teuthology.orchestra.run.smithi107.stdout:modified 2024-03-04T13:38:50.015989+0000 2024-03-04T13:51:32.553 INFO:teuthology.orchestra.run.smithi107.stdout:tableserver 0 2024-03-04T13:51:32.553 INFO:teuthology.orchestra.run.smithi107.stdout:root 0 2024-03-04T13:51:32.553 INFO:teuthology.orchestra.run.smithi107.stdout:session_timeout 60 2024-03-04T13:51:32.553 INFO:teuthology.orchestra.run.smithi107.stdout:session_autoclose 300 2024-03-04T13:51:32.553 INFO:teuthology.orchestra.run.smithi107.stdout:max_file_size 1099511627776 2024-03-04T13:51:32.553 INFO:teuthology.orchestra.run.smithi107.stdout:max_xattr_size 65536 2024-03-04T13:51:32.553 INFO:teuthology.orchestra.run.smithi107.stdout:required_client_features {} 2024-03-04T13:51:32.553 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure 0 2024-03-04T13:51:32.553 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure_osd_epoch 47 2024-03-04T13:51:32.553 INFO:teuthology.orchestra.run.smithi107.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-04T13:51:32.553 INFO:teuthology.orchestra.run.smithi107.stdout:max_mds 1 2024-03-04T13:51:32.553 INFO:teuthology.orchestra.run.smithi107.stdout:in 0 2024-03-04T13:51:32.554 INFO:teuthology.orchestra.run.smithi107.stdout:up {0=24247} 2024-03-04T13:51:32.554 INFO:teuthology.orchestra.run.smithi107.stdout:failed 2024-03-04T13:51:32.554 INFO:teuthology.orchestra.run.smithi107.stdout:damaged 2024-03-04T13:51:32.554 INFO:teuthology.orchestra.run.smithi107.stdout:stopped 2024-03-04T13:51:32.554 INFO:teuthology.orchestra.run.smithi107.stdout:data_pools [3] 2024-03-04T13:51:32.554 INFO:teuthology.orchestra.run.smithi107.stdout:metadata_pool 2 2024-03-04T13:51:32.554 INFO:teuthology.orchestra.run.smithi107.stdout:inline_data enabled 2024-03-04T13:51:32.554 INFO:teuthology.orchestra.run.smithi107.stdout:balancer 2024-03-04T13:51:32.554 INFO:teuthology.orchestra.run.smithi107.stdout:bal_rank_mask -1 2024-03-04T13:51:32.554 INFO:teuthology.orchestra.run.smithi107.stdout:standby_count_wanted 1 2024-03-04T13:51:32.554 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi169.jusudl{0:24247} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.169:6824/2994866076,v1:172.21.15.169:6825/2994866076] compat {c=[1],r=[1],i=[7ff]}] 2024-03-04T13:51:32.554 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.thycho{0:14470} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.107:6828/1985577060,v1:172.21.15.107:6829/1985577060] compat {c=[1],r=[1],i=[7ff]}] 2024-03-04T13:51:32.554 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:51:32.554 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:51:32.554 INFO:teuthology.orchestra.run.smithi107.stdout:Standby daemons: 2024-03-04T13:51:32.554 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:51:32.555 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.riexmm{-1:14480} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.107:6826/3261622517,v1:172.21.15.107:6827/3261622517] compat {c=[1],r=[1],i=[7ff]}] 2024-03-04T13:51:32.555 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi169.bxitzk{-1:24259} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.169:6826/3696090493,v1:172.21.15.169:6827/3696090493] compat {c=[1],r=[1],i=[7ff]}] 2024-03-04T13:51:32.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.554+0000 7f50c5c00700 1 -- 172.21.15.107:0/1561689827 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f50a406d7f0 msgr2=0x7f50a406fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:51:32.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.554+0000 7f50c5c00700 1 --2- 172.21.15.107:0/1561689827 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f50a406d7f0 0x7f50a406fca0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f50b0007fd0 tx=0x7f50b0009040 comp rx=0 tx=0).stop 2024-03-04T13:51:32.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.554+0000 7f50c5c00700 1 -- 172.21.15.107:0/1561689827 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f50c0105910 msgr2=0x7f50c0115f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:51:32.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.554+0000 7f50c5c00700 1 --2- 172.21.15.107:0/1561689827 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f50c0105910 0x7f50c0115f50 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f50ac009030 tx=0x7f50ac009e30 comp rx=0 tx=0).stop 2024-03-04T13:51:32.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.555+0000 7f50c5c00700 1 -- 172.21.15.107:0/1561689827 shutdown_connections 2024-03-04T13:51:32.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.555+0000 7f50c5c00700 1 --2- 172.21.15.107:0/1561689827 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f50a406d7f0 0x7f50a406fca0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:51:32.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.555+0000 7f50c5c00700 1 --2- 172.21.15.107:0/1561689827 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f50c0104f70 0x7f50c0115a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:51:32.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.555+0000 7f50c5c00700 1 --2- 172.21.15.107:0/1561689827 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f50c0105910 0x7f50c0115f50 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:51:32.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.555+0000 7f50c5c00700 1 -- 172.21.15.107:0/1561689827 >> 172.21.15.107:0/1561689827 conn(0x7f50c00faae0 msgr2=0x7f50c00fc560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:51:32.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.555+0000 7f50c5c00700 1 -- 172.21.15.107:0/1561689827 shutdown_connections 2024-03-04T13:51:32.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.555+0000 7f50c5c00700 1 -- 172.21.15.107:0/1561689827 wait complete. 2024-03-04T13:51:32.557 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 16 2024-03-04T13:51:32.611 INFO:journalctl@ceph.osd.4.smithi169.stdout:Mar 04 13:51:32 smithi169 podman[99697]: 2024-03-04 13:51:32.168055565 +0000 UTC m=+0.774646047 container remove 08545e3d065937e35962b59384d3efb6ac28eea647ac72c1bb296bdd9ecbf627 (image=quay.ceph.io/ceph-ci/ceph@sha256:ee85afea73f504aaae524393a00a3b140e2a1cfd81d76da4525eaa6e73286a7d, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-4, GIT_BRANCH=HEAD, org.label-schema.build-date=20240222, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=quincy-42608bb, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-04T13:51:32.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.707+0000 7f2279200700 1 -- 172.21.15.107:0/1307597506 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2274105b10 msgr2=0x7f22740ff180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:51:32.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.707+0000 7f2279200700 1 --2- 172.21.15.107:0/1307597506 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2274105b10 0x7f22740ff180 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2260004940 tx=0x7f2260009e30 comp rx=0 tx=0).stop 2024-03-04T13:51:32.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.707+0000 7f2279200700 1 -- 172.21.15.107:0/1307597506 shutdown_connections 2024-03-04T13:51:32.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.707+0000 7f2279200700 1 --2- 172.21.15.107:0/1307597506 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2274105b10 0x7f22740ff180 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:51:32.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.707+0000 7f2279200700 1 --2- 172.21.15.107:0/1307597506 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2274105170 0x7f2274105540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:51:32.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.707+0000 7f2279200700 1 -- 172.21.15.107:0/1307597506 >> 172.21.15.107:0/1307597506 conn(0x7f22740face0 msgr2=0x7f22740fd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:51:32.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.707+0000 7f2279200700 1 -- 172.21.15.107:0/1307597506 shutdown_connections 2024-03-04T13:51:32.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.707+0000 7f2279200700 1 -- 172.21.15.107:0/1307597506 wait complete. 2024-03-04T13:51:32.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.708+0000 7f2279200700 1 Processor -- start 2024-03-04T13:51:32.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.708+0000 7f2279200700 1 -- start start 2024-03-04T13:51:32.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.708+0000 7f2279200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2274105170 0x7f2274115d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:51:32.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.708+0000 7f2279200700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2274105b10 0x7f22741162b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:51:32.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.708+0000 7f2279200700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2274116900 con 0x7f2274105170 2024-03-04T13:51:32.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.708+0000 7f2279200700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f227410fc10 con 0x7f2274105b10 2024-03-04T13:51:32.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.709+0000 7f2271e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2274105170 0x7f2274115d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:51:32.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.709+0000 7f2271e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2274105170 0x7f2274115d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40178/0 (socket says 172.21.15.107:40178) 2024-03-04T13:51:32.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.709+0000 7f2271e00700 1 -- 172.21.15.107:0/1145284474 learned_addr learned my addr 172.21.15.107:0/1145284474 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:51:32.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.709+0000 7f2271400700 1 --2- 172.21.15.107:0/1145284474 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2274105b10 0x7f22741162b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:51:32.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.709+0000 7f2271e00700 1 -- 172.21.15.107:0/1145284474 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2274105b10 msgr2=0x7f22741162b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:51:32.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.709+0000 7f2271e00700 1 --2- 172.21.15.107:0/1145284474 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2274105b10 0x7f22741162b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:51:32.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.709+0000 7f2271e00700 1 -- 172.21.15.107:0/1145284474 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2260005040 con 0x7f2274105170 2024-03-04T13:51:32.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.709+0000 7f2271e00700 1 --2- 172.21.15.107:0/1145284474 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2274105170 0x7f2274115d70 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f226400eb70 tx=0x7f226400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:51:32.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.710+0000 7f226a000700 1 -- 172.21.15.107:0/1145284474 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2264007cc0 con 0x7f2274105170 2024-03-04T13:51:32.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.710+0000 7f226a000700 1 -- 172.21.15.107:0/1145284474 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f226400f040 con 0x7f2274105170 2024-03-04T13:51:32.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.710+0000 7f2279200700 1 -- 172.21.15.107:0/1145284474 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f227410fe90 con 0x7f2274105170 2024-03-04T13:51:32.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.710+0000 7f226a000700 1 -- 172.21.15.107:0/1145284474 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2264009640 con 0x7f2274105170 2024-03-04T13:51:32.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.710+0000 7f2279200700 1 -- 172.21.15.107:0/1145284474 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22741103d0 con 0x7f2274105170 2024-03-04T13:51:32.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.711+0000 7f2279200700 1 -- 172.21.15.107:0/1145284474 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22740fe900 con 0x7f2274105170 2024-03-04T13:51:32.713 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.712+0000 7f226a000700 1 -- 172.21.15.107:0/1145284474 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f22640097a0 con 0x7f2274105170 2024-03-04T13:51:32.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.713+0000 7f226a000700 1 --2- 172.21.15.107:0/1145284474 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f225806d8c0 0x7f225806fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:51:32.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.713+0000 7f226a000700 1 -- 172.21.15.107:0/1145284474 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f226408ca80 con 0x7f2274105170 2024-03-04T13:51:32.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.713+0000 7f2271400700 1 --2- 172.21.15.107:0/1145284474 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f225806d8c0 0x7f225806fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:51:32.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.714+0000 7f2271400700 1 --2- 172.21.15.107:0/1145284474 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f225806d8c0 0x7f225806fd70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f226000b0d0 tx=0x7f2260009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:51:32.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.717+0000 7f226a000700 1 -- 172.21.15.107:0/1145284474 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2264059e30 con 0x7f2274105170 2024-03-04T13:51:32.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.941+0000 7f2279200700 1 -- 172.21.15.107:0/1145284474 --> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2274111350 con 0x7f225806d8c0 2024-03-04T13:51:32.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.942+0000 7f226a000700 1 -- 172.21.15.107:0/1145284474 <== mgr.34102 v2:172.21.15.107:6800/1223039812 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f2274111350 con 0x7f225806d8c0 2024-03-04T13:51:32.944 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-03-04T13:51:32.944 INFO:teuthology.orchestra.run.smithi107.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377", 2024-03-04T13:51:32.944 INFO:teuthology.orchestra.run.smithi107.stdout: "in_progress": true, 2024-03-04T13:51:32.944 INFO:teuthology.orchestra.run.smithi107.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-04T13:51:32.944 INFO:teuthology.orchestra.run.smithi107.stdout: "services_complete": [ 2024-03-04T13:51:32.944 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr", 2024-03-04T13:51:32.945 INFO:teuthology.orchestra.run.smithi107.stdout: "mon", 2024-03-04T13:51:32.945 INFO:teuthology.orchestra.run.smithi107.stdout: "crash" 2024-03-04T13:51:32.945 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-04T13:51:32.945 INFO:teuthology.orchestra.run.smithi107.stdout: "progress": "10/21 daemons upgraded", 2024-03-04T13:51:32.945 INFO:teuthology.orchestra.run.smithi107.stdout: "message": "Currently upgrading osd daemons", 2024-03-04T13:51:32.945 INFO:teuthology.orchestra.run.smithi107.stdout: "is_paused": false 2024-03-04T13:51:32.945 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-03-04T13:51:32.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.945+0000 7f2279200700 1 -- 172.21.15.107:0/1145284474 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f225806d8c0 msgr2=0x7f225806fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:51:32.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.945+0000 7f2279200700 1 --2- 172.21.15.107:0/1145284474 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f225806d8c0 0x7f225806fd70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f226000b0d0 tx=0x7f2260009e30 comp rx=0 tx=0).stop 2024-03-04T13:51:32.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.945+0000 7f2279200700 1 -- 172.21.15.107:0/1145284474 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2274105170 msgr2=0x7f2274115d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:51:32.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.945+0000 7f2279200700 1 --2- 172.21.15.107:0/1145284474 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2274105170 0x7f2274115d70 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f226400eb70 tx=0x7f226400b040 comp rx=0 tx=0).stop 2024-03-04T13:51:32.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.945+0000 7f2279200700 1 -- 172.21.15.107:0/1145284474 shutdown_connections 2024-03-04T13:51:32.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.945+0000 7f2279200700 1 --2- 172.21.15.107:0/1145284474 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f225806d8c0 0x7f225806fd70 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:51:32.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.945+0000 7f2279200700 1 --2- 172.21.15.107:0/1145284474 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2274105170 0x7f2274115d70 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:51:32.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.945+0000 7f2279200700 1 --2- 172.21.15.107:0/1145284474 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2274105b10 0x7f22741162b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:51:32.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.945+0000 7f2279200700 1 -- 172.21.15.107:0/1145284474 >> 172.21.15.107:0/1145284474 conn(0x7f22740face0 msgr2=0x7f22740fc710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:51:32.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.945+0000 7f2279200700 1 -- 172.21.15.107:0/1145284474 shutdown_connections 2024-03-04T13:51:32.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:32.945+0000 7f2279200700 1 -- 172.21.15.107:0/1145284474 wait complete. 2024-03-04T13:51:33.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:33.082+0000 7f4b82c00700 1 -- 172.21.15.107:0/2675116722 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b7c070be0 msgr2=0x7f4b7c1092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:51:33.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:33.082+0000 7f4b82c00700 1 --2- 172.21.15.107:0/2675116722 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b7c070be0 0x7f4b7c1092c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4b68004970 tx=0x7f4b68009e30 comp rx=0 tx=0).stop 2024-03-04T13:51:33.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:33.082+0000 7f4b82c00700 1 -- 172.21.15.107:0/2675116722 shutdown_connections 2024-03-04T13:51:33.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:33.082+0000 7f4b82c00700 1 --2- 172.21.15.107:0/2675116722 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b7c070be0 0x7f4b7c1092c0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:51:33.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:33.082+0000 7f4b82c00700 1 --2- 172.21.15.107:0/2675116722 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b7c0702d0 0x7f4b7c0706a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:51:33.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:33.082+0000 7f4b82c00700 1 -- 172.21.15.107:0/2675116722 >> 172.21.15.107:0/2675116722 conn(0x7f4b7c0f8ae0 msgr2=0x7f4b7c0faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:51:33.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:33.083+0000 7f4b82c00700 1 -- 172.21.15.107:0/2675116722 shutdown_connections 2024-03-04T13:51:33.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:33.083+0000 7f4b82c00700 1 -- 172.21.15.107:0/2675116722 wait complete. 2024-03-04T13:51:33.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:33.083+0000 7f4b82c00700 1 Processor -- start 2024-03-04T13:51:33.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:33.083+0000 7f4b82c00700 1 -- start start 2024-03-04T13:51:33.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:33.084+0000 7f4b82c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b7c0702d0 0x7f4b7c115c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:51:33.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:33.084+0000 7f4b82c00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b7c070be0 0x7f4b7c1161c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:51:33.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:33.084+0000 7f4b82c00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b7c1168a0 con 0x7f4b7c0702d0 2024-03-04T13:51:33.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:33.084+0000 7f4b82c00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b7c10fb40 con 0x7f4b7c070be0 2024-03-04T13:51:33.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:33.084+0000 7f4b7be00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b7c0702d0 0x7f4b7c115c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:51:33.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:33.084+0000 7f4b7be00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b7c0702d0 0x7f4b7c115c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40192/0 (socket says 172.21.15.107:40192) 2024-03-04T13:51:33.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:33.084+0000 7f4b7be00700 1 -- 172.21.15.107:0/2319449651 learned_addr learned my addr 172.21.15.107:0/2319449651 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:51:33.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:33.084+0000 7f4b7be00700 1 -- 172.21.15.107:0/2319449651 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b7c070be0 msgr2=0x7f4b7c1161c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:51:33.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:33.084+0000 7f4b7be00700 1 --2- 172.21.15.107:0/2319449651 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b7c070be0 0x7f4b7c1161c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:51:33.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:33.085+0000 7f4b7be00700 1 -- 172.21.15.107:0/2319449651 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b68005040 con 0x7f4b7c0702d0 2024-03-04T13:51:33.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:33.085+0000 7f4b7be00700 1 --2- 172.21.15.107:0/2319449651 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b7c0702d0 0x7f4b7c115c80 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c00b790 tx=0x7f4b6c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:51:33.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:33.085+0000 7f4b73e00700 1 -- 172.21.15.107:0/2319449651 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b6c00f770 con 0x7f4b7c0702d0 2024-03-04T13:51:33.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:33.085+0000 7f4b73e00700 1 -- 172.21.15.107:0/2319449651 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4b6c00fda0 con 0x7f4b7c0702d0 2024-03-04T13:51:33.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:33.085+0000 7f4b73e00700 1 -- 172.21.15.107:0/2319449651 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b6c00d310 con 0x7f4b7c0702d0 2024-03-04T13:51:33.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:33.085+0000 7f4b82c00700 1 -- 172.21.15.107:0/2319449651 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b7c10fe20 con 0x7f4b7c0702d0 2024-03-04T13:51:33.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:33.085+0000 7f4b82c00700 1 -- 172.21.15.107:0/2319449651 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b7c110310 con 0x7f4b7c0702d0 2024-03-04T13:51:33.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:33.086+0000 7f4b82c00700 1 -- 172.21.15.107:0/2319449651 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b7c106980 con 0x7f4b7c0702d0 2024-03-04T13:51:33.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:33.087+0000 7f4b73e00700 1 -- 172.21.15.107:0/2319449651 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f4b6c010420 con 0x7f4b7c0702d0 2024-03-04T13:51:33.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:33.088+0000 7f4b73e00700 1 --2- 172.21.15.107:0/2319449651 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f4b6006d7f0 0x7f4b6006fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:51:33.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:33.088+0000 7f4b73e00700 1 -- 172.21.15.107:0/2319449651 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f4b6c08cf60 con 0x7f4b7c0702d0 2024-03-04T13:51:33.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:33.088+0000 7f4b7b400700 1 --2- 172.21.15.107:0/2319449651 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f4b6006d7f0 0x7f4b6006fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:51:33.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:33.089+0000 7f4b7b400700 1 --2- 172.21.15.107:0/2319449651 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f4b6006d7f0 0x7f4b6006fca0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4b6800b370 tx=0x7f4b68009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:51:33.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:33.093+0000 7f4b73e00700 1 -- 172.21.15.107:0/2319449651 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4b6c05a310 con 0x7f4b7c0702d0 2024-03-04T13:51:33.162 INFO:journalctl@ceph.osd.4.smithi169.stdout:Mar 04 13:51:32 smithi169 podman[99782]: 2024-03-04 13:51:32.901784165 +0000 UTC m=+0.033986711 image pull d71b839abdf892550525dad59fdf20a6b925b73dbcc7526f0d04da8380b9fabf quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377 2024-03-04T13:51:33.162 INFO:journalctl@ceph.osd.4.smithi169.stdout:Mar 04 13:51:33 smithi169 podman[99782]: 2024-03-04 13:51:33.010721718 +0000 UTC m=+0.142924296 container create 69601ff346d8b2c1597810927522f89f859f9595854cfbded2a351c68c56b456 (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-4-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240222) 2024-03-04T13:51:33.162 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:32 smithi169 ceph-mon[86183]: from='client.34344 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:51:33.162 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:32 smithi169 ceph-mon[86183]: from='client.34348 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:51:33.162 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:32 smithi169 ceph-mon[86183]: osdmap e72: 6 total, 5 up, 6 in 2024-03-04T13:51:33.162 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:32 smithi169 ceph-mon[86183]: from='client.? 172.21.15.107:0/1662846535' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:51:33.162 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:32 smithi169 ceph-mon[86183]: from='client.? 172.21.15.107:0/1561689827' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-04T13:51:33.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:32 smithi107 ceph-mon[112578]: from='client.34344 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:51:33.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:32 smithi107 ceph-mon[112578]: from='client.34348 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:51:33.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:32 smithi107 ceph-mon[112578]: osdmap e72: 6 total, 5 up, 6 in 2024-03-04T13:51:33.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:32 smithi107 ceph-mon[112578]: from='client.? 172.21.15.107:0/1662846535' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:51:33.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:32 smithi107 ceph-mon[112578]: from='client.? 172.21.15.107:0/1561689827' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-04T13:51:33.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:33.393+0000 7f4b82c00700 1 -- 172.21.15.107:0/2319449651 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f4b7c04bf00 con 0x7f4b7c0702d0 2024-03-04T13:51:33.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:33.394+0000 7f4b73e00700 1 -- 172.21.15.107:0/2319449651 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+998 (secure 0 0 0) 0x7f4b6c059ea0 con 0x7f4b7c0702d0 2024-03-04T13:51:33.396 INFO:teuthology.orchestra.run.smithi107.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data; 1 osds down; Reduced data availability: 2 pgs peering; Degraded data redundancy: 22/264 objects degraded (8.333%), 5 pgs degraded 2024-03-04T13:51:33.396 INFO:teuthology.orchestra.run.smithi107.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-04T13:51:33.396 INFO:teuthology.orchestra.run.smithi107.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-04T13:51:33.396 INFO:teuthology.orchestra.run.smithi107.stdout:[WRN] OSD_DOWN: 1 osds down 2024-03-04T13:51:33.396 INFO:teuthology.orchestra.run.smithi107.stdout: osd.4 (root=default,host=smithi169) is down 2024-03-04T13:51:33.396 INFO:teuthology.orchestra.run.smithi107.stdout:[WRN] PG_AVAILABILITY: Reduced data availability: 2 pgs peering 2024-03-04T13:51:33.396 INFO:teuthology.orchestra.run.smithi107.stdout: pg 3.9 is stuck peering for 5m, current state peering, last acting [2,5] 2024-03-04T13:51:33.396 INFO:teuthology.orchestra.run.smithi107.stdout: pg 3.1a is stuck peering for 5m, current state peering, last acting [1,2] 2024-03-04T13:51:33.396 INFO:teuthology.orchestra.run.smithi107.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 22/264 objects degraded (8.333%), 5 pgs degraded 2024-03-04T13:51:33.396 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.5 is active+undersized+degraded, acting [3,0] 2024-03-04T13:51:33.396 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.7 is active+undersized+degraded, acting [3,2] 2024-03-04T13:51:33.396 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.b is active+undersized+degraded, acting [3,5] 2024-03-04T13:51:33.396 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.14 is active+undersized+degraded, acting [3,5] 2024-03-04T13:51:33.397 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.1a is active+undersized+degraded, acting [3,5] 2024-03-04T13:51:33.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:33.396+0000 7f4b82c00700 1 -- 172.21.15.107:0/2319449651 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f4b6006d7f0 msgr2=0x7f4b6006fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:51:33.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:33.396+0000 7f4b82c00700 1 --2- 172.21.15.107:0/2319449651 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f4b6006d7f0 0x7f4b6006fca0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4b6800b370 tx=0x7f4b68009e30 comp rx=0 tx=0).stop 2024-03-04T13:51:33.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:33.396+0000 7f4b82c00700 1 -- 172.21.15.107:0/2319449651 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b7c0702d0 msgr2=0x7f4b7c115c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:51:33.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:33.396+0000 7f4b82c00700 1 --2- 172.21.15.107:0/2319449651 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b7c0702d0 0x7f4b7c115c80 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c00b790 tx=0x7f4b6c009250 comp rx=0 tx=0).stop 2024-03-04T13:51:33.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:33.397+0000 7f4b82c00700 1 -- 172.21.15.107:0/2319449651 shutdown_connections 2024-03-04T13:51:33.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:33.397+0000 7f4b82c00700 1 --2- 172.21.15.107:0/2319449651 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f4b6006d7f0 0x7f4b6006fca0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:51:33.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:33.397+0000 7f4b82c00700 1 --2- 172.21.15.107:0/2319449651 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b7c0702d0 0x7f4b7c115c80 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:51:33.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:33.397+0000 7f4b82c00700 1 --2- 172.21.15.107:0/2319449651 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b7c070be0 0x7f4b7c1161c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:51:33.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:33.397+0000 7f4b82c00700 1 -- 172.21.15.107:0/2319449651 >> 172.21.15.107:0/2319449651 conn(0x7f4b7c0f8ae0 msgr2=0x7f4b7c103b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:51:33.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:33.397+0000 7f4b82c00700 1 -- 172.21.15.107:0/2319449651 shutdown_connections 2024-03-04T13:51:33.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:51:33.397+0000 7f4b82c00700 1 -- 172.21.15.107:0/2319449651 wait complete. 2024-03-04T13:51:33.612 INFO:journalctl@ceph.osd.4.smithi169.stdout:Mar 04 13:51:33 smithi169 podman[99782]: 2024-03-04 13:51:33.265698771 +0000 UTC m=+0.397901382 container init 69601ff346d8b2c1597810927522f89f859f9595854cfbded2a351c68c56b456 (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-4-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240222, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, CEPH_POINT_RELEASE=, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-03-04T13:51:33.612 INFO:journalctl@ceph.osd.4.smithi169.stdout:Mar 04 13:51:33 smithi169 podman[99782]: 2024-03-04 13:51:33.277796619 +0000 UTC m=+0.409999204 container start 69601ff346d8b2c1597810927522f89f859f9595854cfbded2a351c68c56b456 (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-4-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-04T13:51:33.612 INFO:journalctl@ceph.osd.4.smithi169.stdout:Mar 04 13:51:33 smithi169 podman[99782]: 2024-03-04 13:51:33.363241943 +0000 UTC m=+0.495444528 container attach 69601ff346d8b2c1597810927522f89f859f9595854cfbded2a351c68c56b456 (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-4-deactivate, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240222, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-03-04T13:51:34.041 INFO:journalctl@ceph.osd.4.smithi169.stdout:Mar 04 13:51:33 smithi169 podman[99782]: 2024-03-04 13:51:33.706577624 +0000 UTC m=+0.838780209 container died 69601ff346d8b2c1597810927522f89f859f9595854cfbded2a351c68c56b456 (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-4-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240222, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, maintainer=Guillaume Abrioux , GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.license=GPLv2) 2024-03-04T13:51:34.041 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:33 smithi169 ceph-mon[86183]: pgmap v149: 65 pgs: 5 active+undersized+degraded, 10 peering, 3 stale+active+clean, 6 active+undersized, 41 active+clean; 255 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s; 22/264 objects degraded (8.333%) 2024-03-04T13:51:34.041 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:33 smithi169 ceph-mon[86183]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-03-04T13:51:34.041 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:33 smithi169 ceph-mon[86183]: Health check failed: Degraded data redundancy: 22/264 objects degraded (8.333%), 5 pgs degraded (PG_DEGRADED) 2024-03-04T13:51:34.041 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:33 smithi169 ceph-mon[86183]: from='client.? 172.21.15.107:0/2319449651' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-04T13:51:34.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:33 smithi107 ceph-mon[112578]: pgmap v149: 65 pgs: 5 active+undersized+degraded, 10 peering, 3 stale+active+clean, 6 active+undersized, 41 active+clean; 255 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s; 22/264 objects degraded (8.333%) 2024-03-04T13:51:34.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:33 smithi107 ceph-mon[112578]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-03-04T13:51:34.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:33 smithi107 ceph-mon[112578]: Health check failed: Degraded data redundancy: 22/264 objects degraded (8.333%), 5 pgs degraded (PG_DEGRADED) 2024-03-04T13:51:34.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:33 smithi107 ceph-mon[112578]: from='client.? 172.21.15.107:0/2319449651' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-04T13:51:34.758 INFO:journalctl@ceph.osd.4.smithi169.stdout:Mar 04 13:51:34 smithi169 podman[99808]: 2024-03-04 13:51:34.375125724 +0000 UTC m=+0.652505643 container remove 69601ff346d8b2c1597810927522f89f859f9595854cfbded2a351c68c56b456 (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-4-deactivate, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240222, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-03-04T13:51:34.758 INFO:journalctl@ceph.osd.4.smithi169.stdout:Mar 04 13:51:34 smithi169 systemd[1]: ceph-1fc09a58-da2b-11ee-95c6-87774f69a715@osd.4.service: Deactivated successfully. 2024-03-04T13:51:34.758 INFO:journalctl@ceph.osd.4.smithi169.stdout:Mar 04 13:51:34 smithi169 systemd[1]: Stopped Ceph osd.4 for 1fc09a58-da2b-11ee-95c6-87774f69a715. 2024-03-04T13:51:34.758 INFO:journalctl@ceph.osd.4.smithi169.stdout:Mar 04 13:51:34 smithi169 systemd[1]: ceph-1fc09a58-da2b-11ee-95c6-87774f69a715@osd.4.service: Consumed 1min 27.882s CPU time. 2024-03-04T13:51:35.103 INFO:journalctl@ceph.osd.4.smithi169.stdout:Mar 04 13:51:34 smithi169 systemd[1]: Starting Ceph osd.4 for 1fc09a58-da2b-11ee-95c6-87774f69a715... 2024-03-04T13:51:35.103 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:34 smithi169 ceph-mon[86183]: from='client.34360 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:51:35.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:34 smithi107 ceph-mon[112578]: from='client.34360 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:51:35.361 INFO:journalctl@ceph.osd.4.smithi169.stdout:Mar 04 13:51:35 smithi169 podman[99909]: 2024-03-04 13:51:35.008193 +0000 UTC m=+0.033973169 image pull d71b839abdf892550525dad59fdf20a6b925b73dbcc7526f0d04da8380b9fabf quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377 2024-03-04T13:51:35.361 INFO:journalctl@ceph.osd.4.smithi169.stdout:Mar 04 13:51:35 smithi169 podman[99909]: 2024-03-04 13:51:35.134863096 +0000 UTC m=+0.160643302 container create b9575db8b0e5f2b53729e334b92eb8dd380c986d7d664470d5a9af71925967d4 (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-4-activate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240222, io.buildah.version=1.33.5, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-03-04T13:51:35.861 INFO:journalctl@ceph.osd.4.smithi169.stdout:Mar 04 13:51:35 smithi169 podman[99909]: 2024-03-04 13:51:35.438964256 +0000 UTC m=+0.464744535 container init b9575db8b0e5f2b53729e334b92eb8dd380c986d7d664470d5a9af71925967d4 (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-4-activate, io.buildah.version=1.33.5, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240222, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-04T13:51:35.861 INFO:journalctl@ceph.osd.4.smithi169.stdout:Mar 04 13:51:35 smithi169 podman[99909]: 2024-03-04 13:51:35.450638983 +0000 UTC m=+0.476419159 container start b9575db8b0e5f2b53729e334b92eb8dd380c986d7d664470d5a9af71925967d4 (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-4-activate, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-03-04T13:51:35.861 INFO:journalctl@ceph.osd.4.smithi169.stdout:Mar 04 13:51:35 smithi169 podman[99909]: 2024-03-04 13:51:35.529049551 +0000 UTC m=+0.554829834 container attach b9575db8b0e5f2b53729e334b92eb8dd380c986d7d664470d5a9af71925967d4 (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-4-activate, ceph=True, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c) 2024-03-04T13:51:36.291 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:35 smithi169 ceph-mon[86183]: pgmap v150: 65 pgs: 5 active+undersized+degraded, 10 peering, 3 stale+active+clean, 6 active+undersized, 41 active+clean; 255 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s; 22/264 objects degraded (8.333%) 2024-03-04T13:51:36.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:35 smithi107 ceph-mon[112578]: pgmap v150: 65 pgs: 5 active+undersized+degraded, 10 peering, 3 stale+active+clean, 6 active+undersized, 41 active+clean; 255 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s; 22/264 objects degraded (8.333%) 2024-03-04T13:51:36.611 INFO:journalctl@ceph.osd.4.smithi169.stdout:Mar 04 13:51:36 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-4-activate[99924]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-04T13:51:36.611 INFO:journalctl@ceph.osd.4.smithi169.stdout:Mar 04 13:51:36 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-4-activate[99924]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-03-04T13:51:36.612 INFO:journalctl@ceph.osd.4.smithi169.stdout:Mar 04 13:51:36 smithi169 bash[99909]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-04T13:51:36.612 INFO:journalctl@ceph.osd.4.smithi169.stdout:Mar 04 13:51:36 smithi169 bash[99909]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-03-04T13:51:36.612 INFO:journalctl@ceph.osd.4.smithi169.stdout:Mar 04 13:51:36 smithi169 bash[99909]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-03-04T13:51:36.612 INFO:journalctl@ceph.osd.4.smithi169.stdout:Mar 04 13:51:36 smithi169 bash[99909]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-03-04T13:51:36.612 INFO:journalctl@ceph.osd.4.smithi169.stdout:Mar 04 13:51:36 smithi169 bash[99909]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2024-03-04T13:51:36.612 INFO:journalctl@ceph.osd.4.smithi169.stdout:Mar 04 13:51:36 smithi169 bash[99909]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-04T13:51:36.612 INFO:journalctl@ceph.osd.4.smithi169.stdout:Mar 04 13:51:36 smithi169 bash[99909]: --> ceph-volume raw activate successful for osd ID: 4 2024-03-04T13:51:36.612 INFO:journalctl@ceph.osd.4.smithi169.stdout:Mar 04 13:51:36 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-4-activate[99924]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-03-04T13:51:36.612 INFO:journalctl@ceph.osd.4.smithi169.stdout:Mar 04 13:51:36 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-4-activate[99924]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-03-04T13:51:36.612 INFO:journalctl@ceph.osd.4.smithi169.stdout:Mar 04 13:51:36 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-4-activate[99924]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2024-03-04T13:51:36.613 INFO:journalctl@ceph.osd.4.smithi169.stdout:Mar 04 13:51:36 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-4-activate[99924]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-04T13:51:36.613 INFO:journalctl@ceph.osd.4.smithi169.stdout:Mar 04 13:51:36 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-4-activate[99924]: --> ceph-volume raw activate successful for osd ID: 4 2024-03-04T13:51:36.613 INFO:journalctl@ceph.osd.4.smithi169.stdout:Mar 04 13:51:36 smithi169 podman[99909]: 2024-03-04 13:51:36.352572936 +0000 UTC m=+1.378353112 container died b9575db8b0e5f2b53729e334b92eb8dd380c986d7d664470d5a9af71925967d4 (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-4-activate, maintainer=Guillaume Abrioux , GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-04T13:51:37.548 INFO:journalctl@ceph.osd.4.smithi169.stdout:Mar 04 13:51:37 smithi169 podman[100059]: 2024-03-04 13:51:37.174973432 +0000 UTC m=+0.801751164 container remove b9575db8b0e5f2b53729e334b92eb8dd380c986d7d664470d5a9af71925967d4 (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-4-activate, org.label-schema.build-date=20240222, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-03-04T13:51:37.848 INFO:journalctl@ceph.osd.4.smithi169.stdout:Mar 04 13:51:37 smithi169 podman[100119]: 2024-03-04 13:51:37.45461205 +0000 UTC m=+0.038593668 image pull d71b839abdf892550525dad59fdf20a6b925b73dbcc7526f0d04da8380b9fabf quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377 2024-03-04T13:51:37.848 INFO:journalctl@ceph.osd.4.smithi169.stdout:Mar 04 13:51:37 smithi169 podman[100119]: 2024-03-04 13:51:37.601320484 +0000 UTC m=+0.185302117 container create a70bacd97639a9d3a91412c5d681cae1aef2c396f16cc601f4a3a9d9ede8c4f0 (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-4, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-03-04T13:51:37.848 INFO:journalctl@ceph.osd.4.smithi169.stdout:Mar 04 13:51:37 smithi169 podman[100119]: 2024-03-04 13:51:37.846931153 +0000 UTC m=+0.430912796 container init a70bacd97639a9d3a91412c5d681cae1aef2c396f16cc601f4a3a9d9ede8c4f0 (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-4, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, GIT_CLEAN=True) 2024-03-04T13:51:38.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:37 smithi169 ceph-mon[86183]: pgmap v151: 65 pgs: 15 active+undersized+degraded, 2 peering, 17 active+undersized, 31 active+clean; 255 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 383 B/s rd, 0 op/s; 45/264 objects degraded (17.045%) 2024-03-04T13:51:38.111 INFO:journalctl@ceph.osd.4.smithi169.stdout:Mar 04 13:51:37 smithi169 podman[100119]: 2024-03-04 13:51:37.858541205 +0000 UTC m=+0.442522807 container start a70bacd97639a9d3a91412c5d681cae1aef2c396f16cc601f4a3a9d9ede8c4f0 (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-4, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-04T13:51:38.112 INFO:journalctl@ceph.osd.4.smithi169.stdout:Mar 04 13:51:37 smithi169 ceph-osd[100139]: -- 172.21.15.169:0/3713993401 <== mon.1 v2:172.21.15.169:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x555e5c0d1680 con 0x555e5c09b000 2024-03-04T13:51:38.112 INFO:journalctl@ceph.osd.4.smithi169.stdout:Mar 04 13:51:37 smithi169 bash[100119]: a70bacd97639a9d3a91412c5d681cae1aef2c396f16cc601f4a3a9d9ede8c4f0 2024-03-04T13:51:38.112 INFO:journalctl@ceph.osd.4.smithi169.stdout:Mar 04 13:51:37 smithi169 systemd[1]: Started Ceph osd.4 for 1fc09a58-da2b-11ee-95c6-87774f69a715. 2024-03-04T13:51:38.332 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:37 smithi107 ceph-mon[112578]: pgmap v151: 65 pgs: 15 active+undersized+degraded, 2 peering, 17 active+undersized, 31 active+clean; 255 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 383 B/s rd, 0 op/s; 45/264 objects degraded (17.045%) 2024-03-04T13:51:38.861 INFO:journalctl@ceph.osd.4.smithi169.stdout:Mar 04 13:51:38 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-4[100135]: 2024-03-04T13:51:38.423+0000 7f30aabd27c0 -1 Falling back to public interface 2024-03-04T13:51:39.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:39 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:51:39.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:39 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:51:39.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:39 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:51:39.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:39 smithi107 ceph-mon[112578]: pgmap v152: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 255 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s; 45/264 objects degraded (17.045%) 2024-03-04T13:51:39.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:39 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:51:39.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:39 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:51:39.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:39 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:51:39.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:39 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:51:39.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:39 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:51:39.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:39 smithi169 ceph-mon[86183]: pgmap v152: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 255 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s; 45/264 objects degraded (17.045%) 2024-03-04T13:51:39.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:39 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:51:39.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:39 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:51:40.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:40 smithi107 ceph-mon[112578]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-03-04T13:51:40.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:40 smithi107 ceph-mon[112578]: Health check update: Degraded data redundancy: 45/264 objects degraded (17.045%), 15 pgs degraded (PG_DEGRADED) 2024-03-04T13:51:40.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:40 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:51:40.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:40 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:51:40.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:40 smithi169 ceph-mon[86183]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-03-04T13:51:40.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:40 smithi169 ceph-mon[86183]: Health check update: Degraded data redundancy: 45/264 objects degraded (17.045%), 15 pgs degraded (PG_DEGRADED) 2024-03-04T13:51:40.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:40 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:51:40.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:40 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:51:41.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:41 smithi169 ceph-mon[86183]: pgmap v153: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 255 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 832 B/s rd, 1 op/s; 45/264 objects degraded (17.045%) 2024-03-04T13:51:41.594 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:41 smithi107 ceph-mon[112578]: pgmap v153: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 255 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 832 B/s rd, 1 op/s; 45/264 objects degraded (17.045%) 2024-03-04T13:51:43.861 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:43 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:51:43.861 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:43 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:51:43.861 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:43 smithi169 ceph-mon[86183]: pgmap v154: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 255 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s; 45/264 objects degraded (17.045%) 2024-03-04T13:51:43.861 INFO:journalctl@ceph.osd.4.smithi169.stdout:Mar 04 13:51:43 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-4[100135]: 2024-03-04T13:51:43.553+0000 7f30aabd27c0 -1 osd.4 0 read_superblock omap replica is missing. 2024-03-04T13:51:44.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:43 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:51:44.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:43 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:51:44.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:43 smithi107 ceph-mon[112578]: pgmap v154: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 255 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s; 45/264 objects degraded (17.045%) 2024-03-04T13:51:44.361 INFO:journalctl@ceph.osd.4.smithi169.stdout:Mar 04 13:51:43 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-4[100135]: 2024-03-04T13:51:43.960+0000 7f30aabd27c0 -1 osd.4 70 log_to_monitors true 2024-03-04T13:51:44.861 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:44 smithi169 ceph-mon[86183]: from='osd.4 [v2:172.21.15.169:6808/56453577,v1:172.21.15.169:6809/56453577]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-04T13:51:44.861 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:44 smithi169 ceph-mon[86183]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-04T13:51:44.861 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:44 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:51:44.862 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:44 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:51:44.862 INFO:journalctl@ceph.osd.4.smithi169.stdout:Mar 04 13:51:44 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-4[100135]: 2024-03-04T13:51:44.644+0000 7f309a000700 -1 osd.4 70 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-04T13:51:45.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:44 smithi107 ceph-mon[112578]: from='osd.4 [v2:172.21.15.169:6808/56453577,v1:172.21.15.169:6809/56453577]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-04T13:51:45.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:44 smithi107 ceph-mon[112578]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-04T13:51:45.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:44 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:51:45.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:44 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:51:45.773 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:45 smithi169 ceph-mon[86183]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-04T13:51:45.773 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:45 smithi169 ceph-mon[86183]: osdmap e73: 6 total, 5 up, 6 in 2024-03-04T13:51:45.773 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:45 smithi169 ceph-mon[86183]: from='osd.4 [v2:172.21.15.169:6808/56453577,v1:172.21.15.169:6809/56453577]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-03-04T13:51:45.773 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:45 smithi169 ceph-mon[86183]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-03-04T13:51:45.773 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:45 smithi169 ceph-mon[86183]: pgmap v156: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s; 45/264 objects degraded (17.045%) 2024-03-04T13:51:46.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:45 smithi107 ceph-mon[112578]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-04T13:51:46.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:45 smithi107 ceph-mon[112578]: osdmap e73: 6 total, 5 up, 6 in 2024-03-04T13:51:46.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:45 smithi107 ceph-mon[112578]: from='osd.4 [v2:172.21.15.169:6808/56453577,v1:172.21.15.169:6809/56453577]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-03-04T13:51:46.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:45 smithi107 ceph-mon[112578]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-03-04T13:51:46.095 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:45 smithi107 ceph-mon[112578]: pgmap v156: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s; 45/264 objects degraded (17.045%) 2024-03-04T13:51:46.861 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:46 smithi169 ceph-mon[86183]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-04T13:51:46.861 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:46 smithi169 ceph-mon[86183]: osd.4 [v2:172.21.15.169:6808/56453577,v1:172.21.15.169:6809/56453577] boot 2024-03-04T13:51:46.861 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:46 smithi169 ceph-mon[86183]: osdmap e74: 6 total, 6 up, 6 in 2024-03-04T13:51:46.861 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:46 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-04T13:51:47.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:46 smithi107 ceph-mon[112578]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-04T13:51:47.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:46 smithi107 ceph-mon[112578]: osd.4 [v2:172.21.15.169:6808/56453577,v1:172.21.15.169:6809/56453577] boot 2024-03-04T13:51:47.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:46 smithi107 ceph-mon[112578]: osdmap e74: 6 total, 6 up, 6 in 2024-03-04T13:51:47.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:46 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-04T13:51:47.861 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:47 smithi169 ceph-mon[86183]: osdmap e75: 6 total, 6 up, 6 in 2024-03-04T13:51:47.861 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:47 smithi169 ceph-mon[86183]: pgmap v159: 65 pgs: 17 peering, 7 active+undersized+degraded, 10 active+undersized, 31 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s; 27/264 objects degraded (10.227%) 2024-03-04T13:51:48.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:47 smithi107 ceph-mon[112578]: osdmap e75: 6 total, 6 up, 6 in 2024-03-04T13:51:48.095 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:47 smithi107 ceph-mon[112578]: pgmap v159: 65 pgs: 17 peering, 7 active+undersized+degraded, 10 active+undersized, 31 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s; 27/264 objects degraded (10.227%) 2024-03-04T13:51:49.025 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:48 smithi169 ceph-mon[86183]: Health check update: Degraded data redundancy: 27/264 objects degraded (10.227%), 7 pgs degraded (PG_DEGRADED) 2024-03-04T13:51:49.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:48 smithi107 ceph-mon[112578]: Health check update: Degraded data redundancy: 27/264 objects degraded (10.227%), 7 pgs degraded (PG_DEGRADED) 2024-03-04T13:51:50.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:49 smithi107 ceph-mon[112578]: pgmap v160: 65 pgs: 17 peering, 2 active+undersized+degraded, 4 active+undersized, 42 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 511 B/s rd, 0 op/s; 5/264 objects degraded (1.894%) 2024-03-04T13:51:50.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:49 smithi169 ceph-mon[86183]: pgmap v160: 65 pgs: 17 peering, 2 active+undersized+degraded, 4 active+undersized, 42 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 511 B/s rd, 0 op/s; 5/264 objects degraded (1.894%) 2024-03-04T13:51:51.323 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:51 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:51:51.324 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:51 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:51:51.324 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:51 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:51:51.324 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:51 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:51:51.324 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:51 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:51:51.324 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:51 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:51:51.324 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:51 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:51:51.324 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:51 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:51:51.324 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:51 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:51:51.324 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:51 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-04T13:51:51.324 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:51 smithi169 ceph-mon[86183]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-04T13:51:51.324 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:51 smithi169 ceph-mon[86183]: Upgrade: unsafe to stop osd(s) at this time (7 PGs are or would become offline) 2024-03-04T13:51:51.325 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:51 smithi169 ceph-mon[86183]: pgmap v161: 65 pgs: 13 peering, 52 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 504 B/s rd, 0 op/s 2024-03-04T13:51:51.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:51 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:51:51.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:51 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:51:51.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:51 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:51:51.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:51 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:51:51.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:51 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:51:51.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:51 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:51:51.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:51 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:51:51.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:51 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:51:51.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:51 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:51:51.346 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:51 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-04T13:51:51.346 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:51 smithi107 ceph-mon[112578]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-04T13:51:51.346 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:51 smithi107 ceph-mon[112578]: Upgrade: unsafe to stop osd(s) at this time (7 PGs are or would become offline) 2024-03-04T13:51:51.346 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:51 smithi107 ceph-mon[112578]: pgmap v161: 65 pgs: 13 peering, 52 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 504 B/s rd, 0 op/s 2024-03-04T13:51:52.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:52 smithi107 ceph-mon[112578]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 5/264 objects degraded (1.894%), 2 pgs degraded) 2024-03-04T13:51:52.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:52 smithi169 ceph-mon[86183]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 5/264 objects degraded (1.894%), 2 pgs degraded) 2024-03-04T13:51:53.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:53 smithi107 ceph-mon[112578]: pgmap v162: 65 pgs: 65 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s 2024-03-04T13:51:53.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:53 smithi169 ceph-mon[86183]: pgmap v162: 65 pgs: 65 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s 2024-03-04T13:51:56.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:55 smithi107 ceph-mon[112578]: pgmap v163: 65 pgs: 65 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 901 B/s rd, 1 op/s 2024-03-04T13:51:56.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:55 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:51:56.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:55 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:51:56.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:55 smithi169 ceph-mon[86183]: pgmap v163: 65 pgs: 65 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 901 B/s rd, 1 op/s 2024-03-04T13:51:56.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:55 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:51:56.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:55 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:51:58.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:57 smithi107 ceph-mon[112578]: pgmap v164: 65 pgs: 65 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 813 B/s rd, 1 op/s 2024-03-04T13:51:58.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:57 smithi169 ceph-mon[86183]: pgmap v164: 65 pgs: 65 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 813 B/s rd, 1 op/s 2024-03-04T13:52:00.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:51:59 smithi107 ceph-mon[112578]: pgmap v165: 65 pgs: 65 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 2 op/s 2024-03-04T13:52:00.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:51:59 smithi169 ceph-mon[86183]: pgmap v165: 65 pgs: 65 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 2 op/s 2024-03-04T13:52:01.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:00 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:52:01.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:00 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:52:02.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:01 smithi107 ceph-mon[112578]: pgmap v166: 65 pgs: 65 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:52:02.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:01 smithi169 ceph-mon[86183]: pgmap v166: 65 pgs: 65 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:52:03.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:03.561+0000 7f5d2e600700 1 -- 172.21.15.107:0/637810577 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d281038f0 msgr2=0x7f5d280fcf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:52:03.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:03.561+0000 7f5d2e600700 1 --2- 172.21.15.107:0/637810577 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d281038f0 0x7f5d280fcf60 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5d14004940 tx=0x7f5d14009e30 comp rx=0 tx=0).stop 2024-03-04T13:52:03.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:03.561+0000 7f5d2e600700 1 -- 172.21.15.107:0/637810577 shutdown_connections 2024-03-04T13:52:03.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:03.561+0000 7f5d2e600700 1 --2- 172.21.15.107:0/637810577 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d281038f0 0x7f5d280fcf60 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:52:03.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:03.561+0000 7f5d2e600700 1 --2- 172.21.15.107:0/637810577 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d28102f50 0x7f5d28103320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:52:03.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:03.561+0000 7f5d2e600700 1 -- 172.21.15.107:0/637810577 >> 172.21.15.107:0/637810577 conn(0x7f5d280f8ae0 msgr2=0x7f5d280faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:52:03.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:03.561+0000 7f5d2e600700 1 -- 172.21.15.107:0/637810577 shutdown_connections 2024-03-04T13:52:03.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:03.561+0000 7f5d2e600700 1 -- 172.21.15.107:0/637810577 wait complete. 2024-03-04T13:52:03.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:03.562+0000 7f5d2e600700 1 Processor -- start 2024-03-04T13:52:03.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:03.563+0000 7f5d2e600700 1 -- start start 2024-03-04T13:52:03.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:03.563+0000 7f5d2e600700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d28102f50 0x7f5d28115ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:52:03.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:03.563+0000 7f5d2e600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d281038f0 0x7f5d28116220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:52:03.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:03.563+0000 7f5d2e600700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d28116870 con 0x7f5d281038f0 2024-03-04T13:52:03.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:03.563+0000 7f5d2e600700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2810fb80 con 0x7f5d28102f50 2024-03-04T13:52:03.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:03.563+0000 7f5d26a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d281038f0 0x7f5d28116220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:52:03.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:03.563+0000 7f5d26a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d281038f0 0x7f5d28116220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38766/0 (socket says 172.21.15.107:38766) 2024-03-04T13:52:03.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:03.563+0000 7f5d26a00700 1 -- 172.21.15.107:0/4088697656 learned_addr learned my addr 172.21.15.107:0/4088697656 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:52:03.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:03.564+0000 7f5d27400700 1 --2- 172.21.15.107:0/4088697656 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d28102f50 0x7f5d28115ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:52:03.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:03.564+0000 7f5d26a00700 1 -- 172.21.15.107:0/4088697656 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d28102f50 msgr2=0x7f5d28115ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:52:03.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:03.564+0000 7f5d26a00700 1 --2- 172.21.15.107:0/4088697656 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d28102f50 0x7f5d28115ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:52:03.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:03.564+0000 7f5d26a00700 1 -- 172.21.15.107:0/4088697656 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d14005040 con 0x7f5d281038f0 2024-03-04T13:52:03.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:03.564+0000 7f5d26a00700 1 --2- 172.21.15.107:0/4088697656 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d281038f0 0x7f5d28116220 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5d140090d0 tx=0x7f5d14009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:52:03.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:03.564+0000 7f5d1f400700 1 -- 172.21.15.107:0/4088697656 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d1400e040 con 0x7f5d281038f0 2024-03-04T13:52:03.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:03.564+0000 7f5d1f400700 1 -- 172.21.15.107:0/4088697656 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5d1400bbf0 con 0x7f5d281038f0 2024-03-04T13:52:03.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:03.564+0000 7f5d2e600700 1 -- 172.21.15.107:0/4088697656 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d2810fe00 con 0x7f5d281038f0 2024-03-04T13:52:03.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:03.564+0000 7f5d1f400700 1 -- 172.21.15.107:0/4088697656 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d1401c630 con 0x7f5d281038f0 2024-03-04T13:52:03.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:03.565+0000 7f5d2e600700 1 -- 172.21.15.107:0/4088697656 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d28110340 con 0x7f5d281038f0 2024-03-04T13:52:03.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:03.566+0000 7f5d2e600700 1 -- 172.21.15.107:0/4088697656 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d280fc6e0 con 0x7f5d281038f0 2024-03-04T13:52:03.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:03.567+0000 7f5d1f400700 1 -- 172.21.15.107:0/4088697656 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f5d1400bd60 con 0x7f5d281038f0 2024-03-04T13:52:03.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:03.568+0000 7f5d1f400700 1 --2- 172.21.15.107:0/4088697656 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f5d0c06d7e0 0x7f5d0c06fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:52:03.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:03.568+0000 7f5d1f400700 1 -- 172.21.15.107:0/4088697656 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f5d1408e290 con 0x7f5d281038f0 2024-03-04T13:52:03.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:03.568+0000 7f5d27400700 1 --2- 172.21.15.107:0/4088697656 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f5d0c06d7e0 0x7f5d0c06fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:52:03.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:03.569+0000 7f5d27400700 1 --2- 172.21.15.107:0/4088697656 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f5d0c06d7e0 0x7f5d0c06fc90 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5d18009fd0 tx=0x7f5d18009e50 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:52:03.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:03.573+0000 7f5d1f400700 1 -- 172.21.15.107:0/4088697656 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5d1405b640 con 0x7f5d281038f0 2024-03-04T13:52:03.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:03.814+0000 7f5d2e600700 1 -- 172.21.15.107:0/4088697656 --> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5d28111330 con 0x7f5d0c06d7e0 2024-03-04T13:52:03.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:03.817+0000 7f5d1f400700 1 -- 172.21.15.107:0/4088697656 <== mgr.34102 v2:172.21.15.107:6800/1223039812 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f5d28111330 con 0x7f5d0c06d7e0 2024-03-04T13:52:03.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:03.820+0000 7f5d2e600700 1 -- 172.21.15.107:0/4088697656 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f5d0c06d7e0 msgr2=0x7f5d0c06fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:52:03.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:03.820+0000 7f5d2e600700 1 --2- 172.21.15.107:0/4088697656 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f5d0c06d7e0 0x7f5d0c06fc90 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5d18009fd0 tx=0x7f5d18009e50 comp rx=0 tx=0).stop 2024-03-04T13:52:03.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:03.820+0000 7f5d2e600700 1 -- 172.21.15.107:0/4088697656 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d281038f0 msgr2=0x7f5d28116220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:52:03.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:03.820+0000 7f5d2e600700 1 --2- 172.21.15.107:0/4088697656 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d281038f0 0x7f5d28116220 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5d140090d0 tx=0x7f5d14009e30 comp rx=0 tx=0).stop 2024-03-04T13:52:03.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:03.820+0000 7f5d2e600700 1 -- 172.21.15.107:0/4088697656 shutdown_connections 2024-03-04T13:52:03.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:03.820+0000 7f5d2e600700 1 --2- 172.21.15.107:0/4088697656 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f5d0c06d7e0 0x7f5d0c06fc90 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:52:03.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:03.820+0000 7f5d2e600700 1 --2- 172.21.15.107:0/4088697656 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d28102f50 0x7f5d28115ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:52:03.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:03.820+0000 7f5d2e600700 1 --2- 172.21.15.107:0/4088697656 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d281038f0 0x7f5d28116220 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:52:03.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:03.820+0000 7f5d2e600700 1 -- 172.21.15.107:0/4088697656 >> 172.21.15.107:0/4088697656 conn(0x7f5d280f8ae0 msgr2=0x7f5d280fa490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:52:03.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:03.820+0000 7f5d2e600700 1 -- 172.21.15.107:0/4088697656 shutdown_connections 2024-03-04T13:52:03.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:03.820+0000 7f5d2e600700 1 -- 172.21.15.107:0/4088697656 wait complete. 2024-03-04T13:52:03.837 INFO:teuthology.orchestra.run.smithi107.stdout:true 2024-03-04T13:52:03.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:03.969+0000 7f1ae0c00700 1 -- 172.21.15.107:0/2813775381 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1adc102f50 msgr2=0x7f1adc103320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:52:03.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:03.969+0000 7f1ae0c00700 1 --2- 172.21.15.107:0/2813775381 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1adc102f50 0x7f1adc103320 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1acc004940 tx=0x7f1acc009e30 comp rx=0 tx=0).stop 2024-03-04T13:52:03.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:03.969+0000 7f1ae0c00700 1 -- 172.21.15.107:0/2813775381 shutdown_connections 2024-03-04T13:52:03.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:03.969+0000 7f1ae0c00700 1 --2- 172.21.15.107:0/2813775381 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1adc1038f0 0x7f1adc0fcf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:52:03.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:03.969+0000 7f1ae0c00700 1 --2- 172.21.15.107:0/2813775381 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1adc102f50 0x7f1adc103320 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:52:03.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:03.969+0000 7f1ae0c00700 1 -- 172.21.15.107:0/2813775381 >> 172.21.15.107:0/2813775381 conn(0x7f1adc0f8ae0 msgr2=0x7f1adc0faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:52:03.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:03.969+0000 7f1ae0c00700 1 -- 172.21.15.107:0/2813775381 shutdown_connections 2024-03-04T13:52:03.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:03.969+0000 7f1ae0c00700 1 -- 172.21.15.107:0/2813775381 wait complete. 2024-03-04T13:52:03.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:03.970+0000 7f1ae0c00700 1 Processor -- start 2024-03-04T13:52:03.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:03.971+0000 7f1ae0c00700 1 -- start start 2024-03-04T13:52:03.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:03.971+0000 7f1ae0c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1adc102f50 0x7f1adc070290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:52:03.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:03.971+0000 7f1ae0c00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1adc1038f0 0x7f1adc0707d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:52:03.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:03.971+0000 7f1ae0c00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1adc070dd0 con 0x7f1adc102f50 2024-03-04T13:52:03.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:03.971+0000 7f1ae0c00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1adc070f40 con 0x7f1adc1038f0 2024-03-04T13:52:03.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:03.971+0000 7f1ad9e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1adc102f50 0x7f1adc070290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:52:03.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:03.971+0000 7f1ad9e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1adc102f50 0x7f1adc070290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38784/0 (socket says 172.21.15.107:38784) 2024-03-04T13:52:03.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:03.971+0000 7f1ad9e00700 1 -- 172.21.15.107:0/3889791781 learned_addr learned my addr 172.21.15.107:0/3889791781 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:52:03.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:03.972+0000 7f1ad9e00700 1 -- 172.21.15.107:0/3889791781 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1adc1038f0 msgr2=0x7f1adc0707d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:52:03.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:03.972+0000 7f1ad3e00700 1 --2- 172.21.15.107:0/3889791781 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1adc1038f0 0x7f1adc0707d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:52:03.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:03.972+0000 7f1ad9e00700 1 --2- 172.21.15.107:0/3889791781 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1adc1038f0 0x7f1adc0707d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:52:03.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:03.972+0000 7f1ad9e00700 1 -- 172.21.15.107:0/3889791781 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1acc005040 con 0x7f1adc102f50 2024-03-04T13:52:03.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:03.972+0000 7f1ad3e00700 1 --2- 172.21.15.107:0/3889791781 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1adc1038f0 0x7f1adc0707d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T13:52:03.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:03.972+0000 7f1ad9e00700 1 --2- 172.21.15.107:0/3889791781 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1adc102f50 0x7f1adc070290 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1acc009150 tx=0x7f1acc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:52:03.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:03.972+0000 7f1ad2000700 1 -- 172.21.15.107:0/3889791781 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1acc00e040 con 0x7f1adc102f50 2024-03-04T13:52:03.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:03.972+0000 7f1ad2000700 1 -- 172.21.15.107:0/3889791781 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1acc00bbf0 con 0x7f1adc102f50 2024-03-04T13:52:03.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:03.972+0000 7f1ad2000700 1 -- 172.21.15.107:0/3889791781 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1acc01c630 con 0x7f1adc102f50 2024-03-04T13:52:03.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:03.973+0000 7f1ae0c00700 1 -- 172.21.15.107:0/3889791781 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1adc119c90 con 0x7f1adc102f50 2024-03-04T13:52:03.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:03.973+0000 7f1ae0c00700 1 -- 172.21.15.107:0/3889791781 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1adc11a0f0 con 0x7f1adc102f50 2024-03-04T13:52:03.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:03.974+0000 7f1ae0c00700 1 -- 172.21.15.107:0/3889791781 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1adc0fc6e0 con 0x7f1adc102f50 2024-03-04T13:52:03.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:03.975+0000 7f1ad2000700 1 -- 172.21.15.107:0/3889791781 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f1acc00bd60 con 0x7f1adc102f50 2024-03-04T13:52:03.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:03.976+0000 7f1ad2000700 1 --2- 172.21.15.107:0/3889791781 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f1ac006d7f0 0x7f1ac006fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:52:03.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:03.976+0000 7f1ad2000700 1 -- 172.21.15.107:0/3889791781 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f1acc08e230 con 0x7f1adc102f50 2024-03-04T13:52:03.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:03.976+0000 7f1ad3e00700 1 --2- 172.21.15.107:0/3889791781 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f1ac006d7f0 0x7f1ac006fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:52:03.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:03.977+0000 7f1ad3e00700 1 --2- 172.21.15.107:0/3889791781 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f1ac006d7f0 0x7f1ac006fca0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1ac4007fd0 tx=0x7f1ac4009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:52:03.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:03.980+0000 7f1ad2000700 1 -- 172.21.15.107:0/3889791781 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1acc057b30 con 0x7f1adc102f50 2024-03-04T13:52:04.095 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:03 smithi107 ceph-mon[112578]: pgmap v167: 65 pgs: 65 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:52:04.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:03 smithi169 ceph-mon[86183]: pgmap v167: 65 pgs: 65 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:52:04.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:04.202+0000 7f1ae0c00700 1 -- 172.21.15.107:0/3889791781 --> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1adc0fac10 con 0x7f1ac006d7f0 2024-03-04T13:52:04.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:04.205+0000 7f1ad2000700 1 -- 172.21.15.107:0/3889791781 <== mgr.34102 v2:172.21.15.107:6800/1223039812 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f1adc0fac10 con 0x7f1ac006d7f0 2024-03-04T13:52:04.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:04.207+0000 7f1ae0c00700 1 -- 172.21.15.107:0/3889791781 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f1ac006d7f0 msgr2=0x7f1ac006fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:52:04.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:04.207+0000 7f1ae0c00700 1 --2- 172.21.15.107:0/3889791781 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f1ac006d7f0 0x7f1ac006fca0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1ac4007fd0 tx=0x7f1ac4009040 comp rx=0 tx=0).stop 2024-03-04T13:52:04.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:04.207+0000 7f1ae0c00700 1 -- 172.21.15.107:0/3889791781 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1adc102f50 msgr2=0x7f1adc070290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:52:04.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:04.207+0000 7f1ae0c00700 1 --2- 172.21.15.107:0/3889791781 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1adc102f50 0x7f1adc070290 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1acc009150 tx=0x7f1acc009e30 comp rx=0 tx=0).stop 2024-03-04T13:52:04.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:04.207+0000 7f1ae0c00700 1 -- 172.21.15.107:0/3889791781 shutdown_connections 2024-03-04T13:52:04.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:04.207+0000 7f1ae0c00700 1 --2- 172.21.15.107:0/3889791781 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f1ac006d7f0 0x7f1ac006fca0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:52:04.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:04.207+0000 7f1ae0c00700 1 --2- 172.21.15.107:0/3889791781 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1adc102f50 0x7f1adc070290 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:52:04.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:04.207+0000 7f1ae0c00700 1 --2- 172.21.15.107:0/3889791781 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1adc1038f0 0x7f1adc0707d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:52:04.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:04.207+0000 7f1ae0c00700 1 -- 172.21.15.107:0/3889791781 >> 172.21.15.107:0/3889791781 conn(0x7f1adc0f8ae0 msgr2=0x7f1adc0fa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:52:04.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:04.208+0000 7f1ae0c00700 1 -- 172.21.15.107:0/3889791781 shutdown_connections 2024-03-04T13:52:04.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:04.208+0000 7f1ae0c00700 1 -- 172.21.15.107:0/3889791781 wait complete. 2024-03-04T13:52:04.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:04.357+0000 7f1d62400700 1 -- 172.21.15.107:0/1812616204 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1d5c102f50 msgr2=0x7f1d5c103320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:52:04.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:04.357+0000 7f1d62400700 1 --2- 172.21.15.107:0/1812616204 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1d5c102f50 0x7f1d5c103320 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c004940 tx=0x7f1d4c009e30 comp rx=0 tx=0).stop 2024-03-04T13:52:04.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:04.357+0000 7f1d62400700 1 -- 172.21.15.107:0/1812616204 shutdown_connections 2024-03-04T13:52:04.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:04.357+0000 7f1d62400700 1 --2- 172.21.15.107:0/1812616204 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d5c1038f0 0x7f1d5c0fcf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:52:04.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:04.357+0000 7f1d62400700 1 --2- 172.21.15.107:0/1812616204 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1d5c102f50 0x7f1d5c103320 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:52:04.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:04.357+0000 7f1d62400700 1 -- 172.21.15.107:0/1812616204 >> 172.21.15.107:0/1812616204 conn(0x7f1d5c0f8ae0 msgr2=0x7f1d5c0faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:52:04.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:04.357+0000 7f1d62400700 1 -- 172.21.15.107:0/1812616204 shutdown_connections 2024-03-04T13:52:04.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:04.358+0000 7f1d62400700 1 -- 172.21.15.107:0/1812616204 wait complete. 2024-03-04T13:52:04.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:04.358+0000 7f1d62400700 1 Processor -- start 2024-03-04T13:52:04.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:04.358+0000 7f1d62400700 1 -- start start 2024-03-04T13:52:04.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:04.359+0000 7f1d62400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1d5c102f50 0x7f1d5c115b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:52:04.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:04.359+0000 7f1d62400700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d5c1038f0 0x7f1d5c1160c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:52:04.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:04.359+0000 7f1d62400700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d5c116710 con 0x7f1d5c102f50 2024-03-04T13:52:04.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:04.359+0000 7f1d62400700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d5c10fad0 con 0x7f1d5c1038f0 2024-03-04T13:52:04.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:04.359+0000 7f1d5b400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1d5c102f50 0x7f1d5c115b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:52:04.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:04.359+0000 7f1d5b400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1d5c102f50 0x7f1d5c115b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38800/0 (socket says 172.21.15.107:38800) 2024-03-04T13:52:04.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:04.359+0000 7f1d5b400700 1 -- 172.21.15.107:0/1085076778 learned_addr learned my addr 172.21.15.107:0/1085076778 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:52:04.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:04.360+0000 7f1d53e00700 1 --2- 172.21.15.107:0/1085076778 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d5c1038f0 0x7f1d5c1160c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:52:04.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:04.360+0000 7f1d5b400700 1 -- 172.21.15.107:0/1085076778 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d5c1038f0 msgr2=0x7f1d5c1160c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:52:04.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:04.360+0000 7f1d5b400700 1 --2- 172.21.15.107:0/1085076778 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d5c1038f0 0x7f1d5c1160c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:52:04.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:04.360+0000 7f1d5b400700 1 -- 172.21.15.107:0/1085076778 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d4c005040 con 0x7f1d5c102f50 2024-03-04T13:52:04.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:04.360+0000 7f1d5b400700 1 --2- 172.21.15.107:0/1085076778 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1d5c102f50 0x7f1d5c115b80 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c009fd0 tx=0x7f1d4c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:52:04.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:04.360+0000 7f1d53400700 1 -- 172.21.15.107:0/1085076778 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1d4c00e040 con 0x7f1d5c102f50 2024-03-04T13:52:04.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:04.360+0000 7f1d53400700 1 -- 172.21.15.107:0/1085076778 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1d4c00bbf0 con 0x7f1d5c102f50 2024-03-04T13:52:04.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:04.360+0000 7f1d53400700 1 -- 172.21.15.107:0/1085076778 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1d4c01c3f0 con 0x7f1d5c102f50 2024-03-04T13:52:04.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:04.360+0000 7f1d62400700 1 -- 172.21.15.107:0/1085076778 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d5c10fd50 con 0x7f1d5c102f50 2024-03-04T13:52:04.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:04.360+0000 7f1d62400700 1 -- 172.21.15.107:0/1085076778 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d5c110290 con 0x7f1d5c102f50 2024-03-04T13:52:04.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:04.361+0000 7f1d62400700 1 -- 172.21.15.107:0/1085076778 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d5c0fc6e0 con 0x7f1d5c102f50 2024-03-04T13:52:04.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:04.363+0000 7f1d53400700 1 -- 172.21.15.107:0/1085076778 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f1d4c00bd60 con 0x7f1d5c102f50 2024-03-04T13:52:04.364 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:04.363+0000 7f1d53400700 1 --2- 172.21.15.107:0/1085076778 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f1d4006d7f0 0x7f1d4006fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:52:04.364 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:04.364+0000 7f1d53400700 1 -- 172.21.15.107:0/1085076778 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f1d4c08e170 con 0x7f1d5c102f50 2024-03-04T13:52:04.364 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:04.364+0000 7f1d53e00700 1 --2- 172.21.15.107:0/1085076778 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f1d4006d7f0 0x7f1d4006fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:52:04.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:04.364+0000 7f1d53e00700 1 --2- 172.21.15.107:0/1085076778 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f1d4006d7f0 0x7f1d4006fca0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c111420 tx=0x7f1d4400b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:52:04.369 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:04.368+0000 7f1d53400700 1 -- 172.21.15.107:0/1085076778 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1d4c057a70 con 0x7f1d5c102f50 2024-03-04T13:52:04.585 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:04.584+0000 7f1d62400700 1 -- 172.21.15.107:0/1085076778 --> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f1d5c110500 con 0x7f1d4006d7f0 2024-03-04T13:52:04.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:04.599+0000 7f1d53400700 1 -- 172.21.15.107:0/1085076778 <== mgr.34102 v2:172.21.15.107:6800/1223039812 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f1d5c110500 con 0x7f1d4006d7f0 2024-03-04T13:52:04.600 INFO:teuthology.orchestra.run.smithi107.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-04T13:52:04.600 INFO:teuthology.orchestra.run.smithi107.stdout:alertmanager.smithi107 smithi107 *:9093,9094 running (6m) 91s ago 20m 14.6M - 0.25.0 c8568f914cd2 e5ed7be32538 2024-03-04T13:52:04.601 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi107 smithi107 running (3m) 91s ago 20m 6660k - 19.0.0-1845-g717ce3cc d71b839abdf8 0558022b0d84 2024-03-04T13:52:04.601 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi169 smithi169 running (3m) 22s ago 19m 6665k - 19.0.0-1845-g717ce3cc d71b839abdf8 221ca53aa0d3 2024-03-04T13:52:04.601 INFO:teuthology.orchestra.run.smithi107.stdout:grafana.smithi107 smithi107 *:3000 running (5m) 91s ago 20m 74.5M - 9.4.12 a0ea5e3c46d9 c5e4d33fb599 2024-03-04T13:52:04.601 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.riexmm smithi107 running (13m) 91s ago 13m 14.9M - 17.2.7-545-g42608bb1 9eec3f22859d fc921f7d4328 2024-03-04T13:52:04.601 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.thycho smithi107 running (13m) 91s ago 13m 98.3M - 17.2.7-545-g42608bb1 9eec3f22859d 4069c6e0e063 2024-03-04T13:52:04.601 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi169.bxitzk smithi169 running (13m) 22s ago 13m 14.9M - 17.2.7-545-g42608bb1 9eec3f22859d 7d4115228c7c 2024-03-04T13:52:04.601 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi169.jusudl smithi169 running (13m) 22s ago 13m 145M - 17.2.7-545-g42608bb1 9eec3f22859d 2d2f2cc452e8 2024-03-04T13:52:04.601 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi107.wpzgwl smithi107 *:8443,9283,8765 running (8m) 91s ago 22m 504M - 19.0.0-1845-g717ce3cc d71b839abdf8 1c5fdee45eac 2024-03-04T13:52:04.601 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi169.eihczr smithi169 *:8443,9283,8765 running (7m) 22s ago 19m 451M - 19.0.0-1845-g717ce3cc d71b839abdf8 b5cac2b7d22a 2024-03-04T13:52:04.601 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi107 smithi107 running (4m) 91s ago 22m 46.5M 2048M 19.0.0-1845-g717ce3cc d71b839abdf8 32f07144274f 2024-03-04T13:52:04.601 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi169 smithi169 running (4m) 22s ago 19m 36.7M 2048M 19.0.0-1845-g717ce3cc d71b839abdf8 e3527449381a 2024-03-04T13:52:04.601 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi107 smithi107 *:9100 running (7m) 91s ago 20m 7680k - 1.5.0 0da6a335fe13 8a8d3d4f724f 2024-03-04T13:52:04.602 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi169 smithi169 *:9100 running (7m) 22s ago 19m 7880k - 1.5.0 0da6a335fe13 f240f5a975bc 2024-03-04T13:52:04.602 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0 smithi107 running (2m) 91s ago 18m 80.5M 1558M 19.0.0-1845-g717ce3cc d71b839abdf8 5801a5540dc6 2024-03-04T13:52:04.602 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1 smithi107 running (2m) 91s ago 17m 91.3M 1558M 19.0.0-1845-g717ce3cc d71b839abdf8 b614af658854 2024-03-04T13:52:04.602 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2 smithi107 running (99s) 91s ago 16m 11.1M 1558M 19.0.0-1845-g717ce3cc d71b839abdf8 6e4da4f6f146 2024-03-04T13:52:04.602 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3 smithi169 running (69s) 22s ago 16m 93.5M 2582M 19.0.0-1845-g717ce3cc d71b839abdf8 d31baf764d7a 2024-03-04T13:52:04.602 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4 smithi169 running (27s) 22s ago 15m 11.2M 2582M 19.0.0-1845-g717ce3cc d71b839abdf8 a70bacd97639 2024-03-04T13:52:04.602 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5 smithi169 running (15m) 22s ago 15m 194M 2582M 17.2.7-545-g42608bb1 9eec3f22859d dcffe6cb3eac 2024-03-04T13:52:04.602 INFO:teuthology.orchestra.run.smithi107.stdout:prometheus.smithi107 smithi107 *:9095 running (6m) 91s ago 19m 51.0M - 2.43.0 a07b618ecd1d 1eeb1ad5cb75 2024-03-04T13:52:04.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:04.602+0000 7f1d62400700 1 -- 172.21.15.107:0/1085076778 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f1d4006d7f0 msgr2=0x7f1d4006fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:52:04.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:04.602+0000 7f1d62400700 1 --2- 172.21.15.107:0/1085076778 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f1d4006d7f0 0x7f1d4006fca0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c111420 tx=0x7f1d4400b040 comp rx=0 tx=0).stop 2024-03-04T13:52:04.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:04.602+0000 7f1d62400700 1 -- 172.21.15.107:0/1085076778 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1d5c102f50 msgr2=0x7f1d5c115b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:52:04.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:04.602+0000 7f1d62400700 1 --2- 172.21.15.107:0/1085076778 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1d5c102f50 0x7f1d5c115b80 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c009fd0 tx=0x7f1d4c009e30 comp rx=0 tx=0).stop 2024-03-04T13:52:04.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:04.602+0000 7f1d62400700 1 -- 172.21.15.107:0/1085076778 shutdown_connections 2024-03-04T13:52:04.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:04.602+0000 7f1d62400700 1 --2- 172.21.15.107:0/1085076778 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f1d4006d7f0 0x7f1d4006fca0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:52:04.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:04.602+0000 7f1d62400700 1 --2- 172.21.15.107:0/1085076778 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1d5c102f50 0x7f1d5c115b80 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:52:04.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:04.602+0000 7f1d62400700 1 --2- 172.21.15.107:0/1085076778 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d5c1038f0 0x7f1d5c1160c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:52:04.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:04.602+0000 7f1d62400700 1 -- 172.21.15.107:0/1085076778 >> 172.21.15.107:0/1085076778 conn(0x7f1d5c0f8ae0 msgr2=0x7f1d5c0fa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:52:04.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:04.602+0000 7f1d62400700 1 -- 172.21.15.107:0/1085076778 shutdown_connections 2024-03-04T13:52:04.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:04.602+0000 7f1d62400700 1 -- 172.21.15.107:0/1085076778 wait complete. 2024-03-04T13:52:04.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:04.753+0000 7f8ad8600700 1 -- 172.21.15.107:0/1375074000 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8ad0071f60 msgr2=0x7f8ad0070330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:52:04.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:04.753+0000 7f8ad8600700 1 --2- 172.21.15.107:0/1375074000 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8ad0071f60 0x7f8ad0070330 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8ac4004970 tx=0x7f8ac4009e30 comp rx=0 tx=0).stop 2024-03-04T13:52:04.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:04.753+0000 7f8ad8600700 1 -- 172.21.15.107:0/1375074000 shutdown_connections 2024-03-04T13:52:04.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:04.753+0000 7f8ad8600700 1 --2- 172.21.15.107:0/1375074000 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ad0070870 0x7f8ad0070ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:52:04.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:04.753+0000 7f8ad8600700 1 --2- 172.21.15.107:0/1375074000 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8ad0071f60 0x7f8ad0070330 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:52:04.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:04.754+0000 7f8ad8600700 1 -- 172.21.15.107:0/1375074000 >> 172.21.15.107:0/1375074000 conn(0x7f8ad00f8ae0 msgr2=0x7f8ad00faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:52:04.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:04.754+0000 7f8ad8600700 1 -- 172.21.15.107:0/1375074000 shutdown_connections 2024-03-04T13:52:04.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:04.754+0000 7f8ad8600700 1 -- 172.21.15.107:0/1375074000 wait complete. 2024-03-04T13:52:04.756 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:04.755+0000 7f8ad8600700 1 Processor -- start 2024-03-04T13:52:04.756 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:04.755+0000 7f8ad8600700 1 -- start start 2024-03-04T13:52:04.756 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:04.755+0000 7f8ad8600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8ad0070870 0x7f8ad011a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:52:04.756 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:04.755+0000 7f8ad8600700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ad0071f60 0x7f8ad011a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:52:04.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:04.755+0000 7f8ad8600700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad011ac00 con 0x7f8ad0070870 2024-03-04T13:52:04.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:04.755+0000 7f8ad8600700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad0113f10 con 0x7f8ad0071f60 2024-03-04T13:52:04.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:04.755+0000 7f8ad5c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8ad0070870 0x7f8ad011a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:52:04.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:04.755+0000 7f8ad5c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8ad0070870 0x7f8ad011a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38820/0 (socket says 172.21.15.107:38820) 2024-03-04T13:52:04.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:04.755+0000 7f8ad5c00700 1 -- 172.21.15.107:0/645968406 learned_addr learned my addr 172.21.15.107:0/645968406 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:52:04.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:04.756+0000 7f8ad5c00700 1 -- 172.21.15.107:0/645968406 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ad0071f60 msgr2=0x7f8ad011a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:52:04.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:04.756+0000 7f8ad5c00700 1 --2- 172.21.15.107:0/645968406 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ad0071f60 0x7f8ad011a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:52:04.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:04.756+0000 7f8ad5c00700 1 -- 172.21.15.107:0/645968406 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ac4005040 con 0x7f8ad0070870 2024-03-04T13:52:04.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:04.756+0000 7f8ad5c00700 1 --2- 172.21.15.107:0/645968406 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8ad0070870 0x7f8ad011a070 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8ac4004f50 tx=0x7f8ac4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:52:04.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:04.756+0000 7f8acd600700 1 -- 172.21.15.107:0/645968406 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8ac400e040 con 0x7f8ad0070870 2024-03-04T13:52:04.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:04.756+0000 7f8acd600700 1 -- 172.21.15.107:0/645968406 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8ac400bc10 con 0x7f8ad0070870 2024-03-04T13:52:04.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:04.756+0000 7f8ad8600700 1 -- 172.21.15.107:0/645968406 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ad0114190 con 0x7f8ad0070870 2024-03-04T13:52:04.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:04.756+0000 7f8acd600700 1 -- 172.21.15.107:0/645968406 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8ac401c630 con 0x7f8ad0070870 2024-03-04T13:52:04.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:04.757+0000 7f8ad8600700 1 -- 172.21.15.107:0/645968406 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ad01146d0 con 0x7f8ad0070870 2024-03-04T13:52:04.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:04.758+0000 7f8ad8600700 1 -- 172.21.15.107:0/645968406 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ad00fe400 con 0x7f8ad0070870 2024-03-04T13:52:04.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:04.758+0000 7f8acd600700 1 -- 172.21.15.107:0/645968406 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f8ac400bd80 con 0x7f8ad0070870 2024-03-04T13:52:04.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:04.759+0000 7f8acd600700 1 --2- 172.21.15.107:0/645968406 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f8ab806d7f0 0x7f8ab806fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:52:04.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:04.760+0000 7f8acd600700 1 -- 172.21.15.107:0/645968406 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f8ac408e250 con 0x7f8ad0070870 2024-03-04T13:52:04.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:04.760+0000 7f8ad5200700 1 --2- 172.21.15.107:0/645968406 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f8ab806d7f0 0x7f8ab806fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:52:04.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:04.760+0000 7f8ad5200700 1 --2- 172.21.15.107:0/645968406 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f8ab806d7f0 0x7f8ab806fca0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8ad0115910 tx=0x7f8abc007d40 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:52:04.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:04.764+0000 7f8acd600700 1 -- 172.21.15.107:0/645968406 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8ac405b680 con 0x7f8ad0070870 2024-03-04T13:52:05.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:05.068+0000 7f8ad8600700 1 -- 172.21.15.107:0/645968406 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f8ad004bed0 con 0x7f8ad0070870 2024-03-04T13:52:05.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:05.069+0000 7f8acd600700 1 -- 172.21.15.107:0/645968406 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+839 (secure 0 0 0) 0x7f8ac405b210 con 0x7f8ad0070870 2024-03-04T13:52:05.070 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-03-04T13:52:05.070 INFO:teuthology.orchestra.run.smithi107.stdout: "mon": { 2024-03-04T13:52:05.070 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-1845-g717ce3cc (717ce3cce8d0166e4e577faa8004e6f9cb4128c0) squid (dev)": 2 2024-03-04T13:52:05.070 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:52:05.070 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr": { 2024-03-04T13:52:05.071 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-1845-g717ce3cc (717ce3cce8d0166e4e577faa8004e6f9cb4128c0) squid (dev)": 2 2024-03-04T13:52:05.071 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:52:05.071 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": { 2024-03-04T13:52:05.071 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-545-g42608bb1 (42608bb191117be10dba720e6bf8a02f3171cd34) quincy (stable)": 1, 2024-03-04T13:52:05.071 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-1845-g717ce3cc (717ce3cce8d0166e4e577faa8004e6f9cb4128c0) squid (dev)": 5 2024-03-04T13:52:05.071 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:52:05.071 INFO:teuthology.orchestra.run.smithi107.stdout: "mds": { 2024-03-04T13:52:05.071 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-545-g42608bb1 (42608bb191117be10dba720e6bf8a02f3171cd34) quincy (stable)": 4 2024-03-04T13:52:05.071 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:52:05.071 INFO:teuthology.orchestra.run.smithi107.stdout: "overall": { 2024-03-04T13:52:05.072 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-545-g42608bb1 (42608bb191117be10dba720e6bf8a02f3171cd34) quincy (stable)": 5, 2024-03-04T13:52:05.072 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-1845-g717ce3cc (717ce3cce8d0166e4e577faa8004e6f9cb4128c0) squid (dev)": 9 2024-03-04T13:52:05.072 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-04T13:52:05.072 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-03-04T13:52:05.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:05.072+0000 7f8ad8600700 1 -- 172.21.15.107:0/645968406 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f8ab806d7f0 msgr2=0x7f8ab806fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:52:05.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:05.072+0000 7f8ad8600700 1 --2- 172.21.15.107:0/645968406 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f8ab806d7f0 0x7f8ab806fca0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8ad0115910 tx=0x7f8abc007d40 comp rx=0 tx=0).stop 2024-03-04T13:52:05.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:05.072+0000 7f8ad8600700 1 -- 172.21.15.107:0/645968406 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8ad0070870 msgr2=0x7f8ad011a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:52:05.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:05.072+0000 7f8ad8600700 1 --2- 172.21.15.107:0/645968406 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8ad0070870 0x7f8ad011a070 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8ac4004f50 tx=0x7f8ac4009e30 comp rx=0 tx=0).stop 2024-03-04T13:52:05.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:05.073+0000 7f8ad8600700 1 -- 172.21.15.107:0/645968406 shutdown_connections 2024-03-04T13:52:05.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:05.073+0000 7f8ad8600700 1 --2- 172.21.15.107:0/645968406 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f8ab806d7f0 0x7f8ab806fca0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:52:05.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:05.073+0000 7f8ad8600700 1 --2- 172.21.15.107:0/645968406 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8ad0070870 0x7f8ad011a070 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:52:05.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:05.073+0000 7f8ad8600700 1 --2- 172.21.15.107:0/645968406 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ad0071f60 0x7f8ad011a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:52:05.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:05.073+0000 7f8ad8600700 1 -- 172.21.15.107:0/645968406 >> 172.21.15.107:0/645968406 conn(0x7f8ad00f8ae0 msgr2=0x7f8ad00fa7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:52:05.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:05.073+0000 7f8ad8600700 1 -- 172.21.15.107:0/645968406 shutdown_connections 2024-03-04T13:52:05.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:05.073+0000 7f8ad8600700 1 -- 172.21.15.107:0/645968406 wait complete. 2024-03-04T13:52:05.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:04 smithi169 ceph-mon[86183]: from='client.34370 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:52:05.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:05.217+0000 7f5ba7e00700 1 -- 172.21.15.107:0/3494721594 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ba8109380 msgr2=0x7f5ba8109750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:52:05.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:05.217+0000 7f5ba7e00700 1 --2- 172.21.15.107:0/3494721594 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ba8109380 0x7f5ba8109750 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5b98004940 tx=0x7f5b98009e30 comp rx=0 tx=0).stop 2024-03-04T13:52:05.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:05.217+0000 7f5ba7e00700 1 -- 172.21.15.107:0/3494721594 shutdown_connections 2024-03-04T13:52:05.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:05.217+0000 7f5ba7e00700 1 --2- 172.21.15.107:0/3494721594 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5ba8109d20 0x7f5ba81033a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:52:05.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:05.217+0000 7f5ba7e00700 1 --2- 172.21.15.107:0/3494721594 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ba8109380 0x7f5ba8109750 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:52:05.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:05.217+0000 7f5ba7e00700 1 -- 172.21.15.107:0/3494721594 >> 172.21.15.107:0/3494721594 conn(0x7f5ba8072b50 msgr2=0x7f5ba8074f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:52:05.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:05.217+0000 7f5ba7e00700 1 -- 172.21.15.107:0/3494721594 shutdown_connections 2024-03-04T13:52:05.218 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:04 smithi107 ceph-mon[112578]: from='client.34370 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:52:05.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:05.217+0000 7f5ba7e00700 1 -- 172.21.15.107:0/3494721594 wait complete. 2024-03-04T13:52:05.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:05.218+0000 7f5ba7e00700 1 Processor -- start 2024-03-04T13:52:05.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:05.218+0000 7f5ba7e00700 1 -- start start 2024-03-04T13:52:05.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:05.218+0000 7f5ba7e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ba8109380 0x7f5ba8119ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:52:05.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:05.218+0000 7f5ba7e00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5ba8109d20 0x7f5ba811a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:52:05.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:05.218+0000 7f5ba7e00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba811aa30 con 0x7f5ba8109380 2024-03-04T13:52:05.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:05.218+0000 7f5ba7e00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba8113d40 con 0x7f5ba8109d20 2024-03-04T13:52:05.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:05.219+0000 7f5ba6a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ba8109380 0x7f5ba8119ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:52:05.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:05.219+0000 7f5ba6a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ba8109380 0x7f5ba8119ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38834/0 (socket says 172.21.15.107:38834) 2024-03-04T13:52:05.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:05.219+0000 7f5ba6a00700 1 -- 172.21.15.107:0/988997738 learned_addr learned my addr 172.21.15.107:0/988997738 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:52:05.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:05.219+0000 7f5ba6a00700 1 -- 172.21.15.107:0/988997738 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5ba8109d20 msgr2=0x7f5ba811a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:52:05.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:05.219+0000 7f5ba6a00700 1 --2- 172.21.15.107:0/988997738 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5ba8109d20 0x7f5ba811a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:52:05.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:05.219+0000 7f5ba6a00700 1 -- 172.21.15.107:0/988997738 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b98005040 con 0x7f5ba8109380 2024-03-04T13:52:05.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:05.219+0000 7f5ba6a00700 1 --2- 172.21.15.107:0/988997738 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ba8109380 0x7f5ba8119ea0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5b98009fd0 tx=0x7f5b98009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:52:05.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:05.220+0000 7f5b9ea00700 1 -- 172.21.15.107:0/988997738 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b9800e040 con 0x7f5ba8109380 2024-03-04T13:52:05.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:05.220+0000 7f5b9ea00700 1 -- 172.21.15.107:0/988997738 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5b9800bbf0 con 0x7f5ba8109380 2024-03-04T13:52:05.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:05.220+0000 7f5ba7e00700 1 -- 172.21.15.107:0/988997738 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ba8113fc0 con 0x7f5ba8109380 2024-03-04T13:52:05.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:05.220+0000 7f5b9ea00700 1 -- 172.21.15.107:0/988997738 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b9801c3f0 con 0x7f5ba8109380 2024-03-04T13:52:05.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:05.220+0000 7f5ba7e00700 1 -- 172.21.15.107:0/988997738 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ba8114500 con 0x7f5ba8109380 2024-03-04T13:52:05.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:05.221+0000 7f5ba7e00700 1 -- 172.21.15.107:0/988997738 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ba8106690 con 0x7f5ba8109380 2024-03-04T13:52:05.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:05.222+0000 7f5b9ea00700 1 -- 172.21.15.107:0/988997738 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f5b9800bd60 con 0x7f5ba8109380 2024-03-04T13:52:05.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:05.223+0000 7f5b9ea00700 1 --2- 172.21.15.107:0/988997738 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f5b8c06d7f0 0x7f5b8c06fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:52:05.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:05.223+0000 7f5b9ea00700 1 -- 172.21.15.107:0/988997738 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f5b9808e180 con 0x7f5ba8109380 2024-03-04T13:52:05.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:05.223+0000 7f5ba6000700 1 --2- 172.21.15.107:0/988997738 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f5b8c06d7f0 0x7f5b8c06fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:52:05.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:05.224+0000 7f5ba6000700 1 --2- 172.21.15.107:0/988997738 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f5b8c06d7f0 0x7f5b8c06fca0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5ba8115740 tx=0x7f5b94009ce0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:52:05.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:05.227+0000 7f5b9ea00700 1 -- 172.21.15.107:0/988997738 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5b9805b5b0 con 0x7f5ba8109380 2024-03-04T13:52:05.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:05.473+0000 7f5ba7e00700 1 -- 172.21.15.107:0/988997738 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f5ba8115480 con 0x7f5ba8109380 2024-03-04T13:52:05.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:05.474+0000 7f5b9ea00700 1 -- 172.21.15.107:0/988997738 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 16 v16) v1 ==== 76+0+1880 (secure 0 0 0) 0x7f5b9805b140 con 0x7f5ba8109380 2024-03-04T13:52:05.475 INFO:teuthology.orchestra.run.smithi107.stdout:e16 2024-03-04T13:52:05.475 INFO:teuthology.orchestra.run.smithi107.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-04T13:52:05.475 INFO:teuthology.orchestra.run.smithi107.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-04T13:52:05.475 INFO:teuthology.orchestra.run.smithi107.stdout:legacy client fscid: 1 2024-03-04T13:52:05.475 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:52:05.475 INFO:teuthology.orchestra.run.smithi107.stdout:Filesystem 'cephfs' (1) 2024-03-04T13:52:05.476 INFO:teuthology.orchestra.run.smithi107.stdout:fs_name cephfs 2024-03-04T13:52:05.476 INFO:teuthology.orchestra.run.smithi107.stdout:epoch 15 2024-03-04T13:52:05.476 INFO:teuthology.orchestra.run.smithi107.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-04T13:52:05.476 INFO:teuthology.orchestra.run.smithi107.stdout:created 2024-03-04T13:38:30.854004+0000 2024-03-04T13:52:05.476 INFO:teuthology.orchestra.run.smithi107.stdout:modified 2024-03-04T13:38:50.015989+0000 2024-03-04T13:52:05.476 INFO:teuthology.orchestra.run.smithi107.stdout:tableserver 0 2024-03-04T13:52:05.476 INFO:teuthology.orchestra.run.smithi107.stdout:root 0 2024-03-04T13:52:05.476 INFO:teuthology.orchestra.run.smithi107.stdout:session_timeout 60 2024-03-04T13:52:05.476 INFO:teuthology.orchestra.run.smithi107.stdout:session_autoclose 300 2024-03-04T13:52:05.476 INFO:teuthology.orchestra.run.smithi107.stdout:max_file_size 1099511627776 2024-03-04T13:52:05.476 INFO:teuthology.orchestra.run.smithi107.stdout:max_xattr_size 65536 2024-03-04T13:52:05.476 INFO:teuthology.orchestra.run.smithi107.stdout:required_client_features {} 2024-03-04T13:52:05.476 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure 0 2024-03-04T13:52:05.477 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure_osd_epoch 47 2024-03-04T13:52:05.477 INFO:teuthology.orchestra.run.smithi107.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-04T13:52:05.477 INFO:teuthology.orchestra.run.smithi107.stdout:max_mds 1 2024-03-04T13:52:05.477 INFO:teuthology.orchestra.run.smithi107.stdout:in 0 2024-03-04T13:52:05.477 INFO:teuthology.orchestra.run.smithi107.stdout:up {0=24247} 2024-03-04T13:52:05.477 INFO:teuthology.orchestra.run.smithi107.stdout:failed 2024-03-04T13:52:05.477 INFO:teuthology.orchestra.run.smithi107.stdout:damaged 2024-03-04T13:52:05.477 INFO:teuthology.orchestra.run.smithi107.stdout:stopped 2024-03-04T13:52:05.477 INFO:teuthology.orchestra.run.smithi107.stdout:data_pools [3] 2024-03-04T13:52:05.477 INFO:teuthology.orchestra.run.smithi107.stdout:metadata_pool 2 2024-03-04T13:52:05.477 INFO:teuthology.orchestra.run.smithi107.stdout:inline_data enabled 2024-03-04T13:52:05.477 INFO:teuthology.orchestra.run.smithi107.stdout:balancer 2024-03-04T13:52:05.477 INFO:teuthology.orchestra.run.smithi107.stdout:bal_rank_mask -1 2024-03-04T13:52:05.477 INFO:teuthology.orchestra.run.smithi107.stdout:standby_count_wanted 1 2024-03-04T13:52:05.478 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi169.jusudl{0:24247} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.169:6824/2994866076,v1:172.21.15.169:6825/2994866076] compat {c=[1],r=[1],i=[7ff]}] 2024-03-04T13:52:05.478 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.thycho{0:14470} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.107:6828/1985577060,v1:172.21.15.107:6829/1985577060] compat {c=[1],r=[1],i=[7ff]}] 2024-03-04T13:52:05.478 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:52:05.478 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:52:05.478 INFO:teuthology.orchestra.run.smithi107.stdout:Standby daemons: 2024-03-04T13:52:05.478 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:52:05.478 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.riexmm{-1:14480} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.107:6826/3261622517,v1:172.21.15.107:6827/3261622517] compat {c=[1],r=[1],i=[7ff]}] 2024-03-04T13:52:05.478 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi169.bxitzk{-1:24259} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.169:6826/3696090493,v1:172.21.15.169:6827/3696090493] compat {c=[1],r=[1],i=[7ff]}] 2024-03-04T13:52:05.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:05.477+0000 7f5ba7e00700 1 -- 172.21.15.107:0/988997738 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f5b8c06d7f0 msgr2=0x7f5b8c06fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:52:05.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:05.477+0000 7f5ba7e00700 1 --2- 172.21.15.107:0/988997738 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f5b8c06d7f0 0x7f5b8c06fca0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5ba8115740 tx=0x7f5b94009ce0 comp rx=0 tx=0).stop 2024-03-04T13:52:05.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:05.477+0000 7f5ba7e00700 1 -- 172.21.15.107:0/988997738 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ba8109380 msgr2=0x7f5ba8119ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:52:05.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:05.477+0000 7f5ba7e00700 1 --2- 172.21.15.107:0/988997738 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ba8109380 0x7f5ba8119ea0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5b98009fd0 tx=0x7f5b98009e30 comp rx=0 tx=0).stop 2024-03-04T13:52:05.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:05.477+0000 7f5ba7e00700 1 -- 172.21.15.107:0/988997738 shutdown_connections 2024-03-04T13:52:05.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:05.477+0000 7f5ba7e00700 1 --2- 172.21.15.107:0/988997738 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f5b8c06d7f0 0x7f5b8c06fca0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:52:05.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:05.477+0000 7f5ba7e00700 1 --2- 172.21.15.107:0/988997738 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ba8109380 0x7f5ba8119ea0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:52:05.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:05.477+0000 7f5ba7e00700 1 --2- 172.21.15.107:0/988997738 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5ba8109d20 0x7f5ba811a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:52:05.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:05.477+0000 7f5ba7e00700 1 -- 172.21.15.107:0/988997738 >> 172.21.15.107:0/988997738 conn(0x7f5ba8072b50 msgr2=0x7f5ba8075770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:52:05.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:05.477+0000 7f5ba7e00700 1 -- 172.21.15.107:0/988997738 shutdown_connections 2024-03-04T13:52:05.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:05.477+0000 7f5ba7e00700 1 -- 172.21.15.107:0/988997738 wait complete. 2024-03-04T13:52:05.480 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 16 2024-03-04T13:52:05.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:05.624+0000 7fc308a00700 1 -- 172.21.15.107:0/2978010523 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc304105b00 msgr2=0x7fc3040ff170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:52:05.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:05.624+0000 7fc308a00700 1 --2- 172.21.15.107:0/2978010523 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc304105b00 0x7fc3040ff170 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc2f0004940 tx=0x7fc2f0009e30 comp rx=0 tx=0).stop 2024-03-04T13:52:05.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:05.625+0000 7fc308a00700 1 -- 172.21.15.107:0/2978010523 shutdown_connections 2024-03-04T13:52:05.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:05.625+0000 7fc308a00700 1 --2- 172.21.15.107:0/2978010523 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc304105b00 0x7fc3040ff170 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:52:05.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:05.625+0000 7fc308a00700 1 --2- 172.21.15.107:0/2978010523 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc304105160 0x7fc304105530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:52:05.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:05.625+0000 7fc308a00700 1 -- 172.21.15.107:0/2978010523 >> 172.21.15.107:0/2978010523 conn(0x7fc3040facb0 msgr2=0x7fc3040fd0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:52:05.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:05.625+0000 7fc308a00700 1 -- 172.21.15.107:0/2978010523 shutdown_connections 2024-03-04T13:52:05.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:05.625+0000 7fc308a00700 1 -- 172.21.15.107:0/2978010523 wait complete. 2024-03-04T13:52:05.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:05.626+0000 7fc308a00700 1 Processor -- start 2024-03-04T13:52:05.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:05.626+0000 7fc308a00700 1 -- start start 2024-03-04T13:52:05.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:05.626+0000 7fc308a00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc304105160 0x7fc304115c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:52:05.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:05.626+0000 7fc308a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc304105b00 0x7fc304116160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:52:05.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:05.626+0000 7fc308a00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3041167b0 con 0x7fc304105b00 2024-03-04T13:52:05.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:05.626+0000 7fc308a00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc30410fae0 con 0x7fc304105160 2024-03-04T13:52:05.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:05.627+0000 7fc301400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc304105b00 0x7fc304116160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:52:05.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:05.627+0000 7fc301400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc304105b00 0x7fc304116160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38852/0 (socket says 172.21.15.107:38852) 2024-03-04T13:52:05.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:05.627+0000 7fc301400700 1 -- 172.21.15.107:0/901848355 learned_addr learned my addr 172.21.15.107:0/901848355 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:52:05.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:05.627+0000 7fc301e00700 1 --2- 172.21.15.107:0/901848355 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc304105160 0x7fc304115c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:52:05.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:05.627+0000 7fc301400700 1 -- 172.21.15.107:0/901848355 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc304105160 msgr2=0x7fc304115c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:52:05.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:05.627+0000 7fc301400700 1 --2- 172.21.15.107:0/901848355 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc304105160 0x7fc304115c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:52:05.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:05.627+0000 7fc301400700 1 -- 172.21.15.107:0/901848355 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2f0005040 con 0x7fc304105b00 2024-03-04T13:52:05.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:05.627+0000 7fc301400700 1 --2- 172.21.15.107:0/901848355 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc304105b00 0x7fc304116160 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc2f0000c00 tx=0x7fc2f0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:52:05.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:05.628+0000 7fc2fa000700 1 -- 172.21.15.107:0/901848355 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc2f0021070 con 0x7fc304105b00 2024-03-04T13:52:05.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:05.628+0000 7fc308a00700 1 -- 172.21.15.107:0/901848355 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc30410fd60 con 0x7fc304105b00 2024-03-04T13:52:05.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:05.628+0000 7fc2fa000700 1 -- 172.21.15.107:0/901848355 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc2f0013d00 con 0x7fc304105b00 2024-03-04T13:52:05.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:05.628+0000 7fc2fa000700 1 -- 172.21.15.107:0/901848355 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc2f000e070 con 0x7fc304105b00 2024-03-04T13:52:05.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:05.628+0000 7fc308a00700 1 -- 172.21.15.107:0/901848355 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc304110240 con 0x7fc304105b00 2024-03-04T13:52:05.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:05.629+0000 7fc308a00700 1 -- 172.21.15.107:0/901848355 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3040fe8f0 con 0x7fc304105b00 2024-03-04T13:52:05.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:05.630+0000 7fc2fa000700 1 -- 172.21.15.107:0/901848355 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fc2f001c400 con 0x7fc304105b00 2024-03-04T13:52:05.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:05.631+0000 7fc2fa000700 1 --2- 172.21.15.107:0/901848355 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fc2e806d7f0 0x7fc2e806fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:52:05.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:05.631+0000 7fc2fa000700 1 -- 172.21.15.107:0/901848355 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fc2f008e3c0 con 0x7fc304105b00 2024-03-04T13:52:05.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:05.631+0000 7fc301e00700 1 --2- 172.21.15.107:0/901848355 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fc2e806d7f0 0x7fc2e806fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:52:05.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:05.632+0000 7fc301e00700 1 --2- 172.21.15.107:0/901848355 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fc2e806d7f0 0x7fc2e806fca0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc2f4009b80 tx=0x7fc2f400b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:52:05.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:05.636+0000 7fc2fa000700 1 -- 172.21.15.107:0/901848355 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc2f005b770 con 0x7fc304105b00 2024-03-04T13:52:05.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:05.870+0000 7fc308a00700 1 -- 172.21.15.107:0/901848355 --> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc30404bed0 con 0x7fc2e806d7f0 2024-03-04T13:52:05.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:05.873+0000 7fc2fa000700 1 -- 172.21.15.107:0/901848355 <== mgr.34102 v2:172.21.15.107:6800/1223039812 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fc30404bed0 con 0x7fc2e806d7f0 2024-03-04T13:52:05.874 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-03-04T13:52:05.874 INFO:teuthology.orchestra.run.smithi107.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377", 2024-03-04T13:52:05.874 INFO:teuthology.orchestra.run.smithi107.stdout: "in_progress": true, 2024-03-04T13:52:05.874 INFO:teuthology.orchestra.run.smithi107.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-04T13:52:05.874 INFO:teuthology.orchestra.run.smithi107.stdout: "services_complete": [ 2024-03-04T13:52:05.874 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr", 2024-03-04T13:52:05.874 INFO:teuthology.orchestra.run.smithi107.stdout: "mon", 2024-03-04T13:52:05.874 INFO:teuthology.orchestra.run.smithi107.stdout: "crash" 2024-03-04T13:52:05.874 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-04T13:52:05.874 INFO:teuthology.orchestra.run.smithi107.stdout: "progress": "11/21 daemons upgraded", 2024-03-04T13:52:05.874 INFO:teuthology.orchestra.run.smithi107.stdout: "message": "Currently upgrading osd daemons", 2024-03-04T13:52:05.874 INFO:teuthology.orchestra.run.smithi107.stdout: "is_paused": false 2024-03-04T13:52:05.874 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-03-04T13:52:05.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:05.876+0000 7fc308a00700 1 -- 172.21.15.107:0/901848355 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fc2e806d7f0 msgr2=0x7fc2e806fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:52:05.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:05.876+0000 7fc308a00700 1 --2- 172.21.15.107:0/901848355 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fc2e806d7f0 0x7fc2e806fca0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc2f4009b80 tx=0x7fc2f400b040 comp rx=0 tx=0).stop 2024-03-04T13:52:05.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:05.876+0000 7fc308a00700 1 -- 172.21.15.107:0/901848355 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc304105b00 msgr2=0x7fc304116160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:52:05.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:05.876+0000 7fc308a00700 1 --2- 172.21.15.107:0/901848355 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc304105b00 0x7fc304116160 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc2f0000c00 tx=0x7fc2f0009e30 comp rx=0 tx=0).stop 2024-03-04T13:52:05.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:05.876+0000 7fc308a00700 1 -- 172.21.15.107:0/901848355 shutdown_connections 2024-03-04T13:52:05.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:05.876+0000 7fc308a00700 1 --2- 172.21.15.107:0/901848355 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fc2e806d7f0 0x7fc2e806fca0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:52:05.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:05.876+0000 7fc308a00700 1 --2- 172.21.15.107:0/901848355 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc304105160 0x7fc304115c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:52:05.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:05.876+0000 7fc308a00700 1 --2- 172.21.15.107:0/901848355 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc304105b00 0x7fc304116160 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:52:05.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:05.876+0000 7fc308a00700 1 -- 172.21.15.107:0/901848355 >> 172.21.15.107:0/901848355 conn(0x7fc3040facb0 msgr2=0x7fc3040fc720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:52:05.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:05.876+0000 7fc308a00700 1 -- 172.21.15.107:0/901848355 shutdown_connections 2024-03-04T13:52:05.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:05.876+0000 7fc308a00700 1 -- 172.21.15.107:0/901848355 wait complete. 2024-03-04T13:52:06.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:06.028+0000 7efbf8e00700 1 -- 172.21.15.107:0/2039052282 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efbf4105170 msgr2=0x7efbf4105540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:52:06.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:06.028+0000 7efbf8e00700 1 --2- 172.21.15.107:0/2039052282 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efbf4105170 0x7efbf4105540 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7efbe4004940 tx=0x7efbe4009e30 comp rx=0 tx=0).stop 2024-03-04T13:52:06.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:06.028+0000 7efbf8e00700 1 -- 172.21.15.107:0/2039052282 shutdown_connections 2024-03-04T13:52:06.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:06.028+0000 7efbf8e00700 1 --2- 172.21.15.107:0/2039052282 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efbf4105b10 0x7efbf40ff180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:52:06.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:06.028+0000 7efbf8e00700 1 --2- 172.21.15.107:0/2039052282 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efbf4105170 0x7efbf4105540 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:52:06.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:06.028+0000 7efbf8e00700 1 -- 172.21.15.107:0/2039052282 >> 172.21.15.107:0/2039052282 conn(0x7efbf40face0 msgr2=0x7efbf40fd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:52:06.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:06.028+0000 7efbf8e00700 1 -- 172.21.15.107:0/2039052282 shutdown_connections 2024-03-04T13:52:06.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:06.028+0000 7efbf8e00700 1 -- 172.21.15.107:0/2039052282 wait complete. 2024-03-04T13:52:06.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:06.029+0000 7efbf8e00700 1 Processor -- start 2024-03-04T13:52:06.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:06.029+0000 7efbf8e00700 1 -- start start 2024-03-04T13:52:06.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:06.029+0000 7efbf8e00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efbf4105170 0x7efbf4115bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:52:06.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:06.030+0000 7efbf8e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efbf4105b10 0x7efbf4116130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:52:06.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:06.030+0000 7efbf8e00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbf4116780 con 0x7efbf4105b10 2024-03-04T13:52:06.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:06.030+0000 7efbf8e00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbf410fad0 con 0x7efbf4105170 2024-03-04T13:52:06.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:06.030+0000 7efbf1400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efbf4105b10 0x7efbf4116130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:52:06.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:06.030+0000 7efbf1400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efbf4105b10 0x7efbf4116130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38874/0 (socket says 172.21.15.107:38874) 2024-03-04T13:52:06.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:06.030+0000 7efbf1400700 1 -- 172.21.15.107:0/2580135478 learned_addr learned my addr 172.21.15.107:0/2580135478 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:52:06.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:06.030+0000 7efbf1e00700 1 --2- 172.21.15.107:0/2580135478 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efbf4105170 0x7efbf4115bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:52:06.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:06.030+0000 7efbf1400700 1 -- 172.21.15.107:0/2580135478 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efbf4105170 msgr2=0x7efbf4115bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:52:06.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:06.030+0000 7efbf1400700 1 --2- 172.21.15.107:0/2580135478 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efbf4105170 0x7efbf4115bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:52:06.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:06.030+0000 7efbf1400700 1 -- 172.21.15.107:0/2580135478 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efbe4005040 con 0x7efbf4105b10 2024-03-04T13:52:06.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:06.031+0000 7efbf1400700 1 --2- 172.21.15.107:0/2580135478 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efbf4105b10 0x7efbf4116130 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7efbe0002a50 tx=0x7efbe00046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:52:06.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:06.031+0000 7efbea000700 1 -- 172.21.15.107:0/2580135478 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efbe0007cc0 con 0x7efbf4105b10 2024-03-04T13:52:06.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:06.031+0000 7efbf8e00700 1 -- 172.21.15.107:0/2580135478 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efbf410fdb0 con 0x7efbf4105b10 2024-03-04T13:52:06.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:06.031+0000 7efbea000700 1 -- 172.21.15.107:0/2580135478 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efbe000f040 con 0x7efbf4105b10 2024-03-04T13:52:06.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:06.031+0000 7efbf8e00700 1 -- 172.21.15.107:0/2580135478 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efbf41102f0 con 0x7efbf4105b10 2024-03-04T13:52:06.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:06.031+0000 7efbea000700 1 -- 172.21.15.107:0/2580135478 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efbe000daf0 con 0x7efbf4105b10 2024-03-04T13:52:06.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:06.032+0000 7efbf8e00700 1 -- 172.21.15.107:0/2580135478 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efbf40fe900 con 0x7efbf4105b10 2024-03-04T13:52:06.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:06.033+0000 7efbea000700 1 -- 172.21.15.107:0/2580135478 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7efbe000dc50 con 0x7efbf4105b10 2024-03-04T13:52:06.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:06.034+0000 7efbea000700 1 --2- 172.21.15.107:0/2580135478 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7efbd806d7f0 0x7efbd806fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:52:06.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:06.034+0000 7efbea000700 1 -- 172.21.15.107:0/2580135478 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6324+0+0 (secure 0 0 0) 0x7efbe008cc70 con 0x7efbf4105b10 2024-03-04T13:52:06.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:06.035+0000 7efbf1e00700 1 --2- 172.21.15.107:0/2580135478 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7efbd806d7f0 0x7efbd806fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:52:06.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:06.035+0000 7efbf1e00700 1 --2- 172.21.15.107:0/2580135478 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7efbd806d7f0 0x7efbd806fca0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7efbe400b3b0 tx=0x7efbe4009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:52:06.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:06.041+0000 7efbea000700 1 -- 172.21.15.107:0/2580135478 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7efbe0093080 con 0x7efbf4105b10 2024-03-04T13:52:06.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:05 smithi169 ceph-mon[86183]: from='client.34374 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:52:06.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:05 smithi169 ceph-mon[86183]: from='client.34378 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:52:06.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:05 smithi169 ceph-mon[86183]: pgmap v168: 65 pgs: 65 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:52:06.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:05 smithi169 ceph-mon[86183]: from='client.? 172.21.15.107:0/645968406' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:52:06.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:05 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-04T13:52:06.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:05 smithi169 ceph-mon[86183]: from='client.? 172.21.15.107:0/988997738' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-04T13:52:06.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:06.339+0000 7efbf8e00700 1 -- 172.21.15.107:0/2580135478 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7efbf41111d0 con 0x7efbf4105b10 2024-03-04T13:52:06.340 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:05 smithi107 ceph-mon[112578]: from='client.34374 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:52:06.340 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:05 smithi107 ceph-mon[112578]: from='client.34378 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:52:06.340 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:05 smithi107 ceph-mon[112578]: pgmap v168: 65 pgs: 65 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:52:06.340 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:05 smithi107 ceph-mon[112578]: from='client.? 172.21.15.107:0/645968406' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:52:06.340 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:05 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-04T13:52:06.341 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:05 smithi107 ceph-mon[112578]: from='client.? 172.21.15.107:0/988997738' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-04T13:52:06.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:06.339+0000 7efbea000700 1 -- 172.21.15.107:0/2580135478 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7efbe0056470 con 0x7efbf4105b10 2024-03-04T13:52:06.341 INFO:teuthology.orchestra.run.smithi107.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-03-04T13:52:06.341 INFO:teuthology.orchestra.run.smithi107.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-04T13:52:06.341 INFO:teuthology.orchestra.run.smithi107.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-04T13:52:06.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:06.342+0000 7efbf8e00700 1 -- 172.21.15.107:0/2580135478 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7efbd806d7f0 msgr2=0x7efbd806fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:52:06.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:06.342+0000 7efbf8e00700 1 --2- 172.21.15.107:0/2580135478 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7efbd806d7f0 0x7efbd806fca0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7efbe400b3b0 tx=0x7efbe4009e30 comp rx=0 tx=0).stop 2024-03-04T13:52:06.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:06.342+0000 7efbf8e00700 1 -- 172.21.15.107:0/2580135478 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efbf4105b10 msgr2=0x7efbf4116130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:52:06.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:06.342+0000 7efbf8e00700 1 --2- 172.21.15.107:0/2580135478 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efbf4105b10 0x7efbf4116130 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7efbe0002a50 tx=0x7efbe00046b0 comp rx=0 tx=0).stop 2024-03-04T13:52:06.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:06.342+0000 7efbf8e00700 1 -- 172.21.15.107:0/2580135478 shutdown_connections 2024-03-04T13:52:06.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:06.342+0000 7efbf8e00700 1 --2- 172.21.15.107:0/2580135478 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7efbd806d7f0 0x7efbd806fca0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:52:06.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:06.342+0000 7efbf8e00700 1 --2- 172.21.15.107:0/2580135478 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efbf4105170 0x7efbf4115bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:52:06.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:06.342+0000 7efbf8e00700 1 --2- 172.21.15.107:0/2580135478 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efbf4105b10 0x7efbf4116130 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:52:06.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:06.342+0000 7efbf8e00700 1 -- 172.21.15.107:0/2580135478 >> 172.21.15.107:0/2580135478 conn(0x7efbf40face0 msgr2=0x7efbf40fc640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:52:06.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:06.342+0000 7efbf8e00700 1 -- 172.21.15.107:0/2580135478 shutdown_connections 2024-03-04T13:52:06.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:06.342+0000 7efbf8e00700 1 -- 172.21.15.107:0/2580135478 wait complete. 2024-03-04T13:52:07.337 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:06 smithi169 ceph-mon[86183]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-04T13:52:07.337 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:06 smithi169 ceph-mon[86183]: Upgrade: osd.5 is safe to restart 2024-03-04T13:52:07.337 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:06 smithi169 ceph-mon[86183]: from='client.34390 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:52:07.337 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:06 smithi169 ceph-mon[86183]: from='client.? 172.21.15.107:0/2580135478' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-04T13:52:07.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:06 smithi107 ceph-mon[112578]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-04T13:52:07.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:06 smithi107 ceph-mon[112578]: Upgrade: osd.5 is safe to restart 2024-03-04T13:52:07.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:06 smithi107 ceph-mon[112578]: from='client.34390 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:52:07.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:06 smithi107 ceph-mon[112578]: from='client.? 172.21.15.107:0/2580135478' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-04T13:52:08.332 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:07 smithi107 ceph-mon[112578]: pgmap v169: 65 pgs: 65 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:52:08.332 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:07 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:52:08.333 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:07 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-04T13:52:08.333 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:07 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:52:08.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:07 smithi169 ceph-mon[86183]: pgmap v169: 65 pgs: 65 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:52:08.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:07 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:52:08.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:07 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-04T13:52:08.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:07 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:52:08.990 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:08 smithi169 ceph-mon[86183]: Upgrade: Updating osd.5 2024-03-04T13:52:08.990 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:08 smithi169 ceph-mon[86183]: Deploying daemon osd.5 on smithi169 2024-03-04T13:52:09.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:08 smithi107 ceph-mon[112578]: Upgrade: Updating osd.5 2024-03-04T13:52:09.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:08 smithi107 ceph-mon[112578]: Deploying daemon osd.5 on smithi169 2024-03-04T13:52:10.133 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:09 smithi169 ceph-mon[86183]: pgmap v170: 65 pgs: 65 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:52:10.133 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:09 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:52:10.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:09 smithi107 ceph-mon[112578]: pgmap v170: 65 pgs: 65 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:52:10.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:09 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:52:12.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:11 smithi169 ceph-mon[86183]: pgmap v171: 65 pgs: 65 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:52:12.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:11 smithi107 ceph-mon[112578]: pgmap v171: 65 pgs: 65 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:52:12.862 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 13:52:12 smithi169 systemd[1]: Stopping Ceph osd.5 for 1fc09a58-da2b-11ee-95c6-87774f69a715... 2024-03-04T13:52:13.362 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 13:52:12 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-5[54856]: 2024-03-04T13:52:12.904+0000 7f909c400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-04T13:52:13.362 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 13:52:12 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-5[54856]: 2024-03-04T13:52:12.904+0000 7f909c400700 -1 osd.5 75 *** Got signal Terminated *** 2024-03-04T13:52:13.362 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 13:52:12 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-5[54856]: 2024-03-04T13:52:12.904+0000 7f909c400700 -1 osd.5 75 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-04T13:52:14.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:13 smithi107 ceph-mon[112578]: pgmap v172: 65 pgs: 65 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:52:14.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:13 smithi107 ceph-mon[112578]: osd.5 marked itself down and dead 2024-03-04T13:52:14.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:13 smithi169 ceph-mon[86183]: pgmap v172: 65 pgs: 65 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:52:14.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:13 smithi169 ceph-mon[86183]: osd.5 marked itself down and dead 2024-03-04T13:52:14.830 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 13:52:14 smithi169 podman[105003]: 2024-03-04 13:52:14.517463266 +0000 UTC m=+1.730971029 container died dcffe6cb3eac70ae0dd33ecdd6e549b89f7f98dda149341181be8e386aae0ed6 (image=quay.ceph.io/ceph-ci/ceph@sha256:ee85afea73f504aaae524393a00a3b140e2a1cfd81d76da4525eaa6e73286a7d, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-5, RELEASE=quincy-42608bb, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-03-04T13:52:15.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:14 smithi169 ceph-mon[86183]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-04T13:52:15.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:14 smithi169 ceph-mon[86183]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-03-04T13:52:15.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:14 smithi169 ceph-mon[86183]: osdmap e76: 6 total, 5 up, 6 in 2024-03-04T13:52:15.112 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 13:52:14 smithi169 podman[105003]: 2024-03-04 13:52:14.974168431 +0000 UTC m=+2.187676240 container cleanup dcffe6cb3eac70ae0dd33ecdd6e549b89f7f98dda149341181be8e386aae0ed6 (image=quay.ceph.io/ceph-ci/ceph@sha256:ee85afea73f504aaae524393a00a3b140e2a1cfd81d76da4525eaa6e73286a7d, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=quincy-42608bb, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240222, io.buildah.version=1.33.5, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-03-04T13:52:15.112 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 13:52:14 smithi169 bash[105003]: ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-5 2024-03-04T13:52:15.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:14 smithi107 ceph-mon[112578]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-04T13:52:15.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:14 smithi107 ceph-mon[112578]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-03-04T13:52:15.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:14 smithi107 ceph-mon[112578]: osdmap e76: 6 total, 5 up, 6 in 2024-03-04T13:52:15.611 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 13:52:15 smithi169 podman[105022]: 2024-03-04 13:52:15.208544713 +0000 UTC m=+0.677761957 container remove dcffe6cb3eac70ae0dd33ecdd6e549b89f7f98dda149341181be8e386aae0ed6 (image=quay.ceph.io/ceph-ci/ceph@sha256:ee85afea73f504aaae524393a00a3b140e2a1cfd81d76da4525eaa6e73286a7d, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.license=GPLv2, RELEASE=quincy-42608bb, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240222) 2024-03-04T13:52:16.170 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 13:52:16 smithi169 podman[105112]: 2024-03-04 13:52:15.919870265 +0000 UTC m=+0.032635914 image pull d71b839abdf892550525dad59fdf20a6b925b73dbcc7526f0d04da8380b9fabf quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377 2024-03-04T13:52:16.170 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 13:52:16 smithi169 podman[105112]: 2024-03-04 13:52:16.02626139 +0000 UTC m=+0.139027068 container create e34fa179ae40359b0e0027f66d859c34fb7d0dd8e25ae82f37213b07ddb4a603 (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-5-deactivate, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.build-date=20240222, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, org.label-schema.schema-version=1.0) 2024-03-04T13:52:16.171 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:15 smithi169 ceph-mon[86183]: pgmap v174: 65 pgs: 7 peering, 9 stale+active+clean, 49 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s 2024-03-04T13:52:16.171 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:15 smithi169 ceph-mon[86183]: Health check failed: Reduced data availability: 4 pgs peering (PG_AVAILABILITY) 2024-03-04T13:52:16.171 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:15 smithi169 ceph-mon[86183]: osdmap e77: 6 total, 5 up, 6 in 2024-03-04T13:52:16.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:15 smithi107 ceph-mon[112578]: pgmap v174: 65 pgs: 7 peering, 9 stale+active+clean, 49 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s 2024-03-04T13:52:16.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:15 smithi107 ceph-mon[112578]: Health check failed: Reduced data availability: 4 pgs peering (PG_AVAILABILITY) 2024-03-04T13:52:16.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:15 smithi107 ceph-mon[112578]: osdmap e77: 6 total, 5 up, 6 in 2024-03-04T13:52:16.611 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 13:52:16 smithi169 podman[105112]: 2024-03-04 13:52:16.262909492 +0000 UTC m=+0.375675146 container init e34fa179ae40359b0e0027f66d859c34fb7d0dd8e25ae82f37213b07ddb4a603 (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, CEPH_POINT_RELEASE=) 2024-03-04T13:52:16.611 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 13:52:16 smithi169 podman[105112]: 2024-03-04 13:52:16.274589581 +0000 UTC m=+0.387355232 container start e34fa179ae40359b0e0027f66d859c34fb7d0dd8e25ae82f37213b07ddb4a603 (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-5-deactivate, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, io.buildah.version=1.33.5, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-03-04T13:52:16.611 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 13:52:16 smithi169 podman[105112]: 2024-03-04 13:52:16.353693607 +0000 UTC m=+0.466459331 container attach e34fa179ae40359b0e0027f66d859c34fb7d0dd8e25ae82f37213b07ddb4a603 (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-5-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.schema-version=1.0) 2024-03-04T13:52:16.955 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:16 smithi169 ceph-mon[86183]: pgmap v176: 65 pgs: 2 active+undersized+degraded, 7 peering, 6 stale+active+clean, 4 active+undersized, 46 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 3/264 objects degraded (1.136%) 2024-03-04T13:52:16.956 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 13:52:16 smithi169 podman[105112]: 2024-03-04 13:52:16.703910045 +0000 UTC m=+0.816675736 container died e34fa179ae40359b0e0027f66d859c34fb7d0dd8e25ae82f37213b07ddb4a603 (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-5-deactivate, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8) 2024-03-04T13:52:17.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:16 smithi107 ceph-mon[112578]: pgmap v176: 65 pgs: 2 active+undersized+degraded, 7 peering, 6 stale+active+clean, 4 active+undersized, 46 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 3/264 objects degraded (1.136%) 2024-03-04T13:52:17.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:16 smithi107 ceph-mon[112578]: Health check failed: Degraded data redundancy: 3/264 objects degraded (1.136%), 2 pgs degraded (PG_DEGRADED) 2024-03-04T13:52:17.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:16 smithi169 ceph-mon[86183]: Health check failed: Degraded data redundancy: 3/264 objects degraded (1.136%), 2 pgs degraded (PG_DEGRADED) 2024-03-04T13:52:17.795 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 13:52:17 smithi169 podman[105138]: 2024-03-04 13:52:17.40736843 +0000 UTC m=+0.686866738 container remove e34fa179ae40359b0e0027f66d859c34fb7d0dd8e25ae82f37213b07ddb4a603 (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-5-deactivate, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True) 2024-03-04T13:52:17.795 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 13:52:17 smithi169 systemd[1]: ceph-1fc09a58-da2b-11ee-95c6-87774f69a715@osd.5.service: Deactivated successfully. 2024-03-04T13:52:17.795 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 13:52:17 smithi169 systemd[1]: Stopped Ceph osd.5 for 1fc09a58-da2b-11ee-95c6-87774f69a715. 2024-03-04T13:52:17.795 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 13:52:17 smithi169 systemd[1]: ceph-1fc09a58-da2b-11ee-95c6-87774f69a715@osd.5.service: Consumed 1min 30.132s CPU time. 2024-03-04T13:52:17.795 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 13:52:17 smithi169 systemd[1]: Starting Ceph osd.5 for 1fc09a58-da2b-11ee-95c6-87774f69a715... 2024-03-04T13:52:18.416 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 13:52:18 smithi169 podman[105239]: 2024-03-04 13:52:18.047940872 +0000 UTC m=+0.034454754 image pull d71b839abdf892550525dad59fdf20a6b925b73dbcc7526f0d04da8380b9fabf quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377 2024-03-04T13:52:18.416 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 13:52:18 smithi169 podman[105239]: 2024-03-04 13:52:18.150080789 +0000 UTC m=+0.136594709 container create 796ca3117aea515c5cdf7d498cd3c808961d7e527f4405d7ea423b3a78d04246 (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-5-activate, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, io.buildah.version=1.33.5, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0) 2024-03-04T13:52:18.416 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 13:52:18 smithi169 podman[105239]: 2024-03-04 13:52:18.414299612 +0000 UTC m=+0.400813559 container init 796ca3117aea515c5cdf7d498cd3c808961d7e527f4405d7ea423b3a78d04246 (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-5-activate, org.label-schema.vendor=CentOS, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c) 2024-03-04T13:52:18.861 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 13:52:18 smithi169 podman[105239]: 2024-03-04 13:52:18.426602361 +0000 UTC m=+0.413116249 container start 796ca3117aea515c5cdf7d498cd3c808961d7e527f4405d7ea423b3a78d04246 (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-5-activate, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240222, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-03-04T13:52:18.861 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 13:52:18 smithi169 podman[105239]: 2024-03-04 13:52:18.535943291 +0000 UTC m=+0.522457275 container attach 796ca3117aea515c5cdf7d498cd3c808961d7e527f4405d7ea423b3a78d04246 (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-5-activate, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240222, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-04T13:52:19.613 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 13:52:19 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-5-activate[105253]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-03-04T13:52:19.613 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 13:52:19 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-5-activate[105253]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-03-04T13:52:19.613 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 13:52:19 smithi169 bash[105239]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-03-04T13:52:19.613 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 13:52:19 smithi169 bash[105239]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-03-04T13:52:19.614 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 13:52:19 smithi169 bash[105239]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-03-04T13:52:19.614 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 13:52:19 smithi169 bash[105239]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-03-04T13:52:19.614 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 13:52:19 smithi169 bash[105239]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2024-03-04T13:52:19.614 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 13:52:19 smithi169 bash[105239]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-03-04T13:52:19.614 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 13:52:19 smithi169 bash[105239]: --> ceph-volume raw activate successful for osd ID: 5 2024-03-04T13:52:19.614 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 13:52:19 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-5-activate[105253]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-03-04T13:52:19.614 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 13:52:19 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-5-activate[105253]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-03-04T13:52:19.614 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 13:52:19 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-5-activate[105253]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2024-03-04T13:52:19.614 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 13:52:19 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-5-activate[105253]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-03-04T13:52:19.614 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 13:52:19 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-5-activate[105253]: --> ceph-volume raw activate successful for osd ID: 5 2024-03-04T13:52:19.614 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 13:52:19 smithi169 podman[105239]: 2024-03-04 13:52:19.34495842 +0000 UTC m=+1.331472324 container died 796ca3117aea515c5cdf7d498cd3c808961d7e527f4405d7ea423b3a78d04246 (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-5-activate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240304.042522-717ce3c) 2024-03-04T13:52:20.013 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:19 smithi169 ceph-mon[86183]: pgmap v177: 65 pgs: 10 active+undersized+degraded, 7 peering, 11 active+undersized, 37 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 1 op/s; 31/264 objects degraded (11.742%) 2024-03-04T13:52:20.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:19 smithi107 ceph-mon[112578]: pgmap v177: 65 pgs: 10 active+undersized+degraded, 7 peering, 11 active+undersized, 37 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 1 op/s; 31/264 objects degraded (11.742%) 2024-03-04T13:52:20.328 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 13:52:20 smithi169 podman[105389]: 2024-03-04 13:52:20.012455442 +0000 UTC m=+0.648702587 container remove 796ca3117aea515c5cdf7d498cd3c808961d7e527f4405d7ea423b3a78d04246 (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-5-activate, ceph=True, org.label-schema.build-date=20240222, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8) 2024-03-04T13:52:20.589 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 13:52:20 smithi169 podman[105450]: 2024-03-04 13:52:20.340725622 +0000 UTC m=+0.113410806 container create 5f6e00f7993d0a8cfbc0e3313fdc656e9a80d5bee4cdc02ee9caa16ef569b1cd (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-5, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240222, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, org.label-schema.schema-version=1.0) 2024-03-04T13:52:20.589 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 13:52:20 smithi169 podman[105450]: 2024-03-04 13:52:20.258242143 +0000 UTC m=+0.030927308 image pull d71b839abdf892550525dad59fdf20a6b925b73dbcc7526f0d04da8380b9fabf quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377 2024-03-04T13:52:20.589 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 13:52:20 smithi169 podman[105450]: 2024-03-04 13:52:20.577629532 +0000 UTC m=+0.350314783 container init 5f6e00f7993d0a8cfbc0e3313fdc656e9a80d5bee4cdc02ee9caa16ef569b1cd (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-5, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240222, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-04T13:52:20.846 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 13:52:20 smithi169 podman[105450]: 2024-03-04 13:52:20.588616934 +0000 UTC m=+0.361302087 container start 5f6e00f7993d0a8cfbc0e3313fdc656e9a80d5bee4cdc02ee9caa16ef569b1cd (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-5, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240222) 2024-03-04T13:52:20.846 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 13:52:20 smithi169 ceph-osd[105469]: -- 172.21.15.169:0/4763058 <== mon.1 v2:172.21.15.169:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55c8ae1e3680 con 0x55c8ae1ad000 2024-03-04T13:52:20.846 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 13:52:20 smithi169 bash[105450]: 5f6e00f7993d0a8cfbc0e3313fdc656e9a80d5bee4cdc02ee9caa16ef569b1cd 2024-03-04T13:52:20.846 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 13:52:20 smithi169 systemd[1]: Started Ceph osd.5 for 1fc09a58-da2b-11ee-95c6-87774f69a715. 2024-03-04T13:52:21.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:20 smithi169 ceph-mon[86183]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 4 pgs peering) 2024-03-04T13:52:21.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:20 smithi107 ceph-mon[112578]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 4 pgs peering) 2024-03-04T13:52:21.611 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 13:52:21 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-5[105465]: 2024-03-04T13:52:21.156+0000 7f312b83a7c0 -1 Falling back to public interface 2024-03-04T13:52:22.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:21 smithi169 ceph-mon[86183]: pgmap v178: 65 pgs: 13 active+undersized+degraded, 15 active+undersized, 37 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 127 B/s rd, 0 op/s; 41/264 objects degraded (15.530%) 2024-03-04T13:52:22.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:21 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:52:22.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:21 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:52:22.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:21 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:52:22.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:21 smithi107 ceph-mon[112578]: pgmap v178: 65 pgs: 13 active+undersized+degraded, 15 active+undersized, 37 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 127 B/s rd, 0 op/s; 41/264 objects degraded (15.530%) 2024-03-04T13:52:22.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:21 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:52:22.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:21 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:52:22.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:21 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:52:24.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:23 smithi169 ceph-mon[86183]: pgmap v179: 65 pgs: 13 active+undersized+degraded, 15 active+undersized, 37 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 696 B/s rd, 1 op/s; 41/264 objects degraded (15.530%) 2024-03-04T13:52:24.147 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:23 smithi107 ceph-mon[112578]: pgmap v179: 65 pgs: 13 active+undersized+degraded, 15 active+undersized, 37 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 696 B/s rd, 1 op/s; 41/264 objects degraded (15.530%) 2024-03-04T13:52:25.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:24 smithi169 ceph-mon[86183]: Health check update: Degraded data redundancy: 41/264 objects degraded (15.530%), 13 pgs degraded (PG_DEGRADED) 2024-03-04T13:52:25.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:24 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:52:25.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:24 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:52:25.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:24 smithi107 ceph-mon[112578]: Health check update: Degraded data redundancy: 41/264 objects degraded (15.530%), 13 pgs degraded (PG_DEGRADED) 2024-03-04T13:52:25.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:24 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:52:25.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:24 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:52:26.041 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:25 smithi169 ceph-mon[86183]: pgmap v180: 65 pgs: 13 active+undersized+degraded, 15 active+undersized, 37 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 614 B/s rd, 1 op/s; 41/264 objects degraded (15.530%) 2024-03-04T13:52:26.041 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:25 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:52:26.041 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:25 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:52:26.041 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 13:52:25 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-5[105465]: 2024-03-04T13:52:25.664+0000 7f312b83a7c0 -1 osd.5 0 read_superblock omap replica is missing. 2024-03-04T13:52:26.041 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 13:52:26 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-5[105465]: 2024-03-04T13:52:26.039+0000 7f312b83a7c0 -1 osd.5 75 log_to_monitors true 2024-03-04T13:52:26.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:25 smithi107 ceph-mon[112578]: pgmap v180: 65 pgs: 13 active+undersized+degraded, 15 active+undersized, 37 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 614 B/s rd, 1 op/s; 41/264 objects degraded (15.530%) 2024-03-04T13:52:26.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:25 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:52:26.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:25 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:52:27.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:26 smithi107 ceph-mon[112578]: from='osd.5 [v2:172.21.15.169:6816/374254288,v1:172.21.15.169:6817/374254288]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-04T13:52:27.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:26 smithi107 ceph-mon[112578]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-04T13:52:27.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:26 smithi169 ceph-mon[86183]: from='osd.5 [v2:172.21.15.169:6816/374254288,v1:172.21.15.169:6817/374254288]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-04T13:52:27.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:26 smithi169 ceph-mon[86183]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-04T13:52:27.361 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 13:52:26 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-5[105465]: 2024-03-04T13:52:26.897+0000 7f311ac00700 -1 osd.5 75 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-04T13:52:28.154 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:27 smithi169 ceph-mon[86183]: pgmap v181: 65 pgs: 13 active+undersized+degraded, 15 active+undersized, 37 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 519 B/s rd, 1 op/s; 41/264 objects degraded (15.530%) 2024-03-04T13:52:28.154 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:27 smithi169 ceph-mon[86183]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-04T13:52:28.154 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:27 smithi169 ceph-mon[86183]: from='osd.5 [v2:172.21.15.169:6816/374254288,v1:172.21.15.169:6817/374254288]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-03-04T13:52:28.154 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:27 smithi169 ceph-mon[86183]: osdmap e78: 6 total, 5 up, 6 in 2024-03-04T13:52:28.155 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:27 smithi169 ceph-mon[86183]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-03-04T13:52:28.155 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:27 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:52:28.155 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:27 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:52:28.333 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:27 smithi107 ceph-mon[112578]: pgmap v181: 65 pgs: 13 active+undersized+degraded, 15 active+undersized, 37 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 519 B/s rd, 1 op/s; 41/264 objects degraded (15.530%) 2024-03-04T13:52:28.333 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:27 smithi107 ceph-mon[112578]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-04T13:52:28.333 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:27 smithi107 ceph-mon[112578]: from='osd.5 [v2:172.21.15.169:6816/374254288,v1:172.21.15.169:6817/374254288]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-03-04T13:52:28.333 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:27 smithi107 ceph-mon[112578]: osdmap e78: 6 total, 5 up, 6 in 2024-03-04T13:52:28.333 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:27 smithi107 ceph-mon[112578]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi169", "root=default"]}]: dispatch 2024-03-04T13:52:28.333 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:27 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:52:28.333 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:27 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:52:28.925 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:28 smithi107 ceph-mon[112578]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-04T13:52:28.925 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:28 smithi107 ceph-mon[112578]: osd.5 [v2:172.21.15.169:6816/374254288,v1:172.21.15.169:6817/374254288] boot 2024-03-04T13:52:28.925 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:28 smithi107 ceph-mon[112578]: osdmap e79: 6 total, 6 up, 6 in 2024-03-04T13:52:28.925 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:28 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-04T13:52:29.015 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:28 smithi169 ceph-mon[86183]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-04T13:52:29.016 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:28 smithi169 ceph-mon[86183]: osd.5 [v2:172.21.15.169:6816/374254288,v1:172.21.15.169:6817/374254288] boot 2024-03-04T13:52:29.016 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:28 smithi169 ceph-mon[86183]: osdmap e79: 6 total, 6 up, 6 in 2024-03-04T13:52:29.016 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:28 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-04T13:52:29.016 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:28 smithi169 ceph-mon[86183]: osdmap e80: 6 total, 6 up, 6 in 2024-03-04T13:52:29.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:28 smithi107 ceph-mon[112578]: osdmap e80: 6 total, 6 up, 6 in 2024-03-04T13:52:30.113 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:29 smithi169 ceph-mon[86183]: pgmap v184: 65 pgs: 13 active+undersized+degraded, 15 active+undersized, 37 active+clean; 255 MiB data, 963 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s; 41/264 objects degraded (15.530%) 2024-03-04T13:52:30.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:29 smithi107 ceph-mon[112578]: pgmap v184: 65 pgs: 13 active+undersized+degraded, 15 active+undersized, 37 active+clean; 255 MiB data, 963 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s; 41/264 objects degraded (15.530%) 2024-03-04T13:52:31.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:30 smithi107 ceph-mon[112578]: pgmap v186: 65 pgs: 11 active+undersized+degraded, 14 active+undersized, 40 active+clean; 255 MiB data, 963 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s; 36/264 objects degraded (13.636%) 2024-03-04T13:52:31.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:30 smithi107 ceph-mon[112578]: Health check update: Degraded data redundancy: 36/264 objects degraded (13.636%), 11 pgs degraded (PG_DEGRADED) 2024-03-04T13:52:31.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:30 smithi169 ceph-mon[86183]: pgmap v186: 65 pgs: 11 active+undersized+degraded, 14 active+undersized, 40 active+clean; 255 MiB data, 963 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s; 36/264 objects degraded (13.636%) 2024-03-04T13:52:31.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:30 smithi169 ceph-mon[86183]: Health check update: Degraded data redundancy: 36/264 objects degraded (13.636%), 11 pgs degraded (PG_DEGRADED) 2024-03-04T13:52:33.846 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:33 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:52:33.846 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:33 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:52:33.847 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:33 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:52:33.847 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:33 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:52:33.847 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:33 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:52:33.847 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:33 smithi107 ceph-mon[112578]: pgmap v187: 65 pgs: 65 active+clean; 255 MiB data, 963 MiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 2 op/s 2024-03-04T13:52:33.847 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:33 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:52:33.847 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:33 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:52:33.847 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:33 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:52:33.847 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:33 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:52:33.847 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:33 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:52:33.847 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:33 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:52:33.847 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:33 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-03-04T13:52:33.847 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:33 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-03-04T13:52:33.847 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:33 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-03-04T13:52:33.847 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:33 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-03-04T13:52:33.848 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:33 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-03-04T13:52:33.848 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:33 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-03-04T13:52:33.848 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:33 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-03-04T13:52:33.848 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:33 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-03-04T13:52:33.848 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:33 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-03-04T13:52:33.848 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:33 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-03-04T13:52:33.848 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:33 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-03-04T13:52:33.848 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:33 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-03-04T13:52:33.848 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:33 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd require-osd-release", "release": "squid"}]: dispatch 2024-03-04T13:52:33.862 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:33 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:52:33.862 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:33 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:52:33.862 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:33 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:52:33.862 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:33 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:52:33.862 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:33 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:52:33.863 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:33 smithi169 ceph-mon[86183]: pgmap v187: 65 pgs: 65 active+clean; 255 MiB data, 963 MiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 2 op/s 2024-03-04T13:52:33.863 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:33 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:52:33.863 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:33 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:52:33.863 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:33 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:52:33.863 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:33 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:52:33.863 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:33 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:52:33.863 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:33 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:52:33.863 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:33 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-03-04T13:52:33.863 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:33 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-03-04T13:52:33.863 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:33 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-03-04T13:52:33.863 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:33 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-03-04T13:52:33.863 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:33 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-03-04T13:52:33.864 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:33 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-03-04T13:52:33.864 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:33 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-03-04T13:52:33.864 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:33 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-03-04T13:52:33.864 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:33 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-03-04T13:52:33.864 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:33 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-03-04T13:52:33.864 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:33 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-03-04T13:52:33.864 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:33 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-03-04T13:52:33.864 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:33 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd require-osd-release", "release": "squid"}]: dispatch 2024-03-04T13:52:34.844 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:34 smithi107 ceph-mon[112578]: Upgrade: Setting container_image for all osd 2024-03-04T13:52:34.845 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:34 smithi107 ceph-mon[112578]: Upgrade: Setting require_osd_release to 19 squid 2024-03-04T13:52:34.845 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:34 smithi107 ceph-mon[112578]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 36/264 objects degraded (13.636%), 11 pgs degraded) 2024-03-04T13:52:34.845 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:34 smithi107 ceph-mon[112578]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-03-04T13:52:34.845 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:34 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-03-04T13:52:34.845 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:34 smithi107 ceph-mon[112578]: osdmap e81: 6 total, 6 up, 6 in 2024-03-04T13:52:34.845 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:34 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:52:34.845 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:34 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]: dispatch 2024-03-04T13:52:34.862 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:34 smithi169 ceph-mon[86183]: Upgrade: Setting container_image for all osd 2024-03-04T13:52:34.862 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:34 smithi169 ceph-mon[86183]: Upgrade: Setting require_osd_release to 19 squid 2024-03-04T13:52:34.862 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:34 smithi169 ceph-mon[86183]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 36/264 objects degraded (13.636%), 11 pgs degraded) 2024-03-04T13:52:34.862 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:34 smithi169 ceph-mon[86183]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-03-04T13:52:34.862 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:34 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-03-04T13:52:34.862 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:34 smithi169 ceph-mon[86183]: osdmap e81: 6 total, 6 up, 6 in 2024-03-04T13:52:34.862 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:34 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:52:34.862 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:34 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]: dispatch 2024-03-04T13:52:35.845 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:35 smithi107 ceph-mon[112578]: Upgrade: Disabling standby-replay for filesystem cephfs 2024-03-04T13:52:35.845 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:35 smithi107 ceph-mon[112578]: osdmap e82: 6 total, 6 up, 6 in 2024-03-04T13:52:35.845 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:35 smithi107 ceph-mon[112578]: pgmap v190: 65 pgs: 65 active+clean; 255 MiB data, 963 MiB used, 535 GiB / 536 GiB avail; 511 B/s rd, 0 op/s 2024-03-04T13:52:35.845 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:35 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]': finished 2024-03-04T13:52:35.845 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:35 smithi107 ceph-mon[112578]: fsmap cephfs:1 {0=cephfs.smithi169.jusudl=up:active} 2 up:standby 2024-03-04T13:52:35.845 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:35 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:52:35.846 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:35 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:52:35.846 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:35 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:52:35.846 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:35 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:52:35.846 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:35 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:52:35.846 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:35 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:52:35.846 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:35 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:52:35.846 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:35 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:52:35.846 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:35 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:52:35.862 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:35 smithi169 ceph-mon[86183]: Upgrade: Disabling standby-replay for filesystem cephfs 2024-03-04T13:52:35.862 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:35 smithi169 ceph-mon[86183]: osdmap e82: 6 total, 6 up, 6 in 2024-03-04T13:52:35.862 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:35 smithi169 ceph-mon[86183]: pgmap v190: 65 pgs: 65 active+clean; 255 MiB data, 963 MiB used, 535 GiB / 536 GiB avail; 511 B/s rd, 0 op/s 2024-03-04T13:52:35.862 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:35 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]': finished 2024-03-04T13:52:35.862 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:35 smithi169 ceph-mon[86183]: fsmap cephfs:1 {0=cephfs.smithi169.jusudl=up:active} 2 up:standby 2024-03-04T13:52:35.862 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:35 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:52:35.862 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:35 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:52:35.862 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:35 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:52:35.862 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:35 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:52:35.862 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:35 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:52:35.862 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:35 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:52:35.863 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:35 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:52:35.863 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:35 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:52:35.863 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:35 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:52:36.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:36.506+0000 7f1930400700 1 -- 172.21.15.107:0/647063645 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19281038f0 msgr2=0x7f19280fcf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:52:36.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:36.506+0000 7f1930400700 1 --2- 172.21.15.107:0/647063645 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19281038f0 0x7f19280fcf60 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1914004940 tx=0x7f1914009e30 comp rx=0 tx=0).stop 2024-03-04T13:52:36.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:36.506+0000 7f1930400700 1 -- 172.21.15.107:0/647063645 shutdown_connections 2024-03-04T13:52:36.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:36.506+0000 7f1930400700 1 --2- 172.21.15.107:0/647063645 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19281038f0 0x7f19280fcf60 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:52:36.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:36.506+0000 7f1930400700 1 --2- 172.21.15.107:0/647063645 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1928102f50 0x7f1928103320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:52:36.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:36.506+0000 7f1930400700 1 -- 172.21.15.107:0/647063645 >> 172.21.15.107:0/647063645 conn(0x7f19280f8ae0 msgr2=0x7f19280faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:52:36.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:36.507+0000 7f1930400700 1 -- 172.21.15.107:0/647063645 shutdown_connections 2024-03-04T13:52:36.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:36.507+0000 7f1930400700 1 -- 172.21.15.107:0/647063645 wait complete. 2024-03-04T13:52:36.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:36.508+0000 7f1930400700 1 Processor -- start 2024-03-04T13:52:36.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:36.508+0000 7f1930400700 1 -- start start 2024-03-04T13:52:36.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:36.509+0000 7f1930400700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1928102f50 0x7f1928113a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:52:36.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:36.509+0000 7f1930400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19281038f0 0x7f1928113fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:52:36.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:36.509+0000 7f1930400700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1928114600 con 0x7f19281038f0 2024-03-04T13:52:36.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:36.509+0000 7f1930400700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f192810d930 con 0x7f1928102f50 2024-03-04T13:52:36.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:36.509+0000 7f192d000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19281038f0 0x7f1928113fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:52:36.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:36.509+0000 7f192d000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19281038f0 0x7f1928113fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40398/0 (socket says 172.21.15.107:40398) 2024-03-04T13:52:36.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:36.509+0000 7f192d000700 1 -- 172.21.15.107:0/3589906207 learned_addr learned my addr 172.21.15.107:0/3589906207 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:52:36.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:36.510+0000 7f192da00700 1 --2- 172.21.15.107:0/3589906207 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1928102f50 0x7f1928113a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:52:36.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:36.510+0000 7f192d000700 1 -- 172.21.15.107:0/3589906207 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1928102f50 msgr2=0x7f1928113a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:52:36.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:36.510+0000 7f192d000700 1 --2- 172.21.15.107:0/3589906207 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1928102f50 0x7f1928113a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:52:36.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:36.510+0000 7f192d000700 1 -- 172.21.15.107:0/3589906207 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1914005040 con 0x7f19281038f0 2024-03-04T13:52:36.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:36.510+0000 7f192d000700 1 --2- 172.21.15.107:0/3589906207 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19281038f0 0x7f1928113fb0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1914009100 tx=0x7f1914009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:52:36.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:36.510+0000 7f1925600700 1 -- 172.21.15.107:0/3589906207 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f191400e040 con 0x7f19281038f0 2024-03-04T13:52:36.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:36.510+0000 7f1925600700 1 -- 172.21.15.107:0/3589906207 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1914004290 con 0x7f19281038f0 2024-03-04T13:52:36.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:36.510+0000 7f1925600700 1 -- 172.21.15.107:0/3589906207 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f191401c3f0 con 0x7f19281038f0 2024-03-04T13:52:36.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:36.510+0000 7f1930400700 1 -- 172.21.15.107:0/3589906207 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f192810dbb0 con 0x7f19281038f0 2024-03-04T13:52:36.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:36.511+0000 7f1930400700 1 -- 172.21.15.107:0/3589906207 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f192810e0f0 con 0x7f19281038f0 2024-03-04T13:52:36.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:36.512+0000 7f1930400700 1 -- 172.21.15.107:0/3589906207 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1928105420 con 0x7f19281038f0 2024-03-04T13:52:36.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:36.513+0000 7f1925600700 1 -- 172.21.15.107:0/3589906207 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f191400bbf0 con 0x7f19281038f0 2024-03-04T13:52:36.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:36.514+0000 7f1925600700 1 --2- 172.21.15.107:0/3589906207 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f191006d7f0 0x7f191006fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:52:36.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:36.514+0000 7f1925600700 1 -- 172.21.15.107:0/3589906207 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f191408e170 con 0x7f19281038f0 2024-03-04T13:52:36.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:36.514+0000 7f192da00700 1 --2- 172.21.15.107:0/3589906207 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f191006d7f0 0x7f191006fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:52:36.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:36.515+0000 7f192da00700 1 --2- 172.21.15.107:0/3589906207 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f191006d7f0 0x7f191006fca0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f191c004550 tx=0x7f191c009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:52:36.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:36.519+0000 7f1925600700 1 -- 172.21.15.107:0/3589906207 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f191405b5a0 con 0x7f19281038f0 2024-03-04T13:52:36.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:36.767+0000 7f1930400700 1 -- 172.21.15.107:0/3589906207 --> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1928113740 con 0x7f191006d7f0 2024-03-04T13:52:36.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:36.769+0000 7f1925600700 1 -- 172.21.15.107:0/3589906207 <== mgr.34102 v2:172.21.15.107:6800/1223039812 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f1928113740 con 0x7f191006d7f0 2024-03-04T13:52:36.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:36.772+0000 7f1930400700 1 -- 172.21.15.107:0/3589906207 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f191006d7f0 msgr2=0x7f191006fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:52:36.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:36.772+0000 7f1930400700 1 --2- 172.21.15.107:0/3589906207 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f191006d7f0 0x7f191006fca0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f191c004550 tx=0x7f191c009040 comp rx=0 tx=0).stop 2024-03-04T13:52:36.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:36.772+0000 7f1930400700 1 -- 172.21.15.107:0/3589906207 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19281038f0 msgr2=0x7f1928113fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:52:36.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:36.772+0000 7f1930400700 1 --2- 172.21.15.107:0/3589906207 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19281038f0 0x7f1928113fb0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1914009100 tx=0x7f1914009e30 comp rx=0 tx=0).stop 2024-03-04T13:52:36.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:36.772+0000 7f1930400700 1 -- 172.21.15.107:0/3589906207 shutdown_connections 2024-03-04T13:52:36.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:36.772+0000 7f1930400700 1 --2- 172.21.15.107:0/3589906207 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f191006d7f0 0x7f191006fca0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:52:36.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:36.772+0000 7f1930400700 1 --2- 172.21.15.107:0/3589906207 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1928102f50 0x7f1928113a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:52:36.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:36.772+0000 7f1930400700 1 --2- 172.21.15.107:0/3589906207 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19281038f0 0x7f1928113fb0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:52:36.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:36.772+0000 7f1930400700 1 -- 172.21.15.107:0/3589906207 >> 172.21.15.107:0/3589906207 conn(0x7f19280f8ae0 msgr2=0x7f19280fa570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:52:36.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:36.772+0000 7f1930400700 1 -- 172.21.15.107:0/3589906207 shutdown_connections 2024-03-04T13:52:36.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:36.773+0000 7f1930400700 1 -- 172.21.15.107:0/3589906207 wait complete. 2024-03-04T13:52:36.791 INFO:teuthology.orchestra.run.smithi107.stdout:true 2024-03-04T13:52:36.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:36.926+0000 7f2dffe00700 1 -- 172.21.15.107:0/4275253382 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2df8070be0 msgr2=0x7f2df81092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:52:36.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:36.926+0000 7f2dffe00700 1 --2- 172.21.15.107:0/4275253382 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2df8070be0 0x7f2df81092c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2de4004970 tx=0x7f2de4009e30 comp rx=0 tx=0).stop 2024-03-04T13:52:36.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:36.927+0000 7f2dffe00700 1 -- 172.21.15.107:0/4275253382 shutdown_connections 2024-03-04T13:52:36.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:36.927+0000 7f2dffe00700 1 --2- 172.21.15.107:0/4275253382 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2df8070be0 0x7f2df81092c0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:52:36.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:36.927+0000 7f2dffe00700 1 --2- 172.21.15.107:0/4275253382 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2df80702d0 0x7f2df80706a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:52:36.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:36.927+0000 7f2dffe00700 1 -- 172.21.15.107:0/4275253382 >> 172.21.15.107:0/4275253382 conn(0x7f2df80f8ae0 msgr2=0x7f2df80faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:52:36.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:36.927+0000 7f2dffe00700 1 -- 172.21.15.107:0/4275253382 shutdown_connections 2024-03-04T13:52:36.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:36.927+0000 7f2dffe00700 1 -- 172.21.15.107:0/4275253382 wait complete. 2024-03-04T13:52:36.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:36.928+0000 7f2dffe00700 1 Processor -- start 2024-03-04T13:52:36.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:36.928+0000 7f2dffe00700 1 -- start start 2024-03-04T13:52:36.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:36.928+0000 7f2dffe00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2df80702d0 0x7f2df8115c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:52:36.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:36.928+0000 7f2dffe00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2df8070be0 0x7f2df8116150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:52:36.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:36.928+0000 7f2dffe00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2df81167a0 con 0x7f2df8070be0 2024-03-04T13:52:36.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:36.928+0000 7f2dffe00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2df810faf0 con 0x7f2df80702d0 2024-03-04T13:52:36.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:36.929+0000 7f2dfca00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2df8070be0 0x7f2df8116150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:52:36.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:36.929+0000 7f2dfca00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2df8070be0 0x7f2df8116150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40404/0 (socket says 172.21.15.107:40404) 2024-03-04T13:52:36.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:36.929+0000 7f2dfca00700 1 -- 172.21.15.107:0/272027963 learned_addr learned my addr 172.21.15.107:0/272027963 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:52:36.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:36.929+0000 7f2dfd400700 1 --2- 172.21.15.107:0/272027963 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2df80702d0 0x7f2df8115c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:52:36.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:36.929+0000 7f2dfca00700 1 -- 172.21.15.107:0/272027963 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2df80702d0 msgr2=0x7f2df8115c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:52:36.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:36.929+0000 7f2dfca00700 1 --2- 172.21.15.107:0/272027963 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2df80702d0 0x7f2df8115c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:52:36.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:36.929+0000 7f2dfca00700 1 -- 172.21.15.107:0/272027963 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2de4005040 con 0x7f2df8070be0 2024-03-04T13:52:36.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:36.930+0000 7f2dfca00700 1 --2- 172.21.15.107:0/272027963 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2df8070be0 0x7f2df8116150 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f2de4000c00 tx=0x7f2de4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:52:36.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:36.930+0000 7f2df5600700 1 -- 172.21.15.107:0/272027963 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2de400e040 con 0x7f2df8070be0 2024-03-04T13:52:36.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:36.930+0000 7f2df5600700 1 -- 172.21.15.107:0/272027963 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2de400bce0 con 0x7f2df8070be0 2024-03-04T13:52:36.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:36.930+0000 7f2dffe00700 1 -- 172.21.15.107:0/272027963 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2df810fd70 con 0x7f2df8070be0 2024-03-04T13:52:36.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:36.930+0000 7f2df5600700 1 -- 172.21.15.107:0/272027963 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2de401c630 con 0x7f2df8070be0 2024-03-04T13:52:36.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:36.930+0000 7f2dffe00700 1 -- 172.21.15.107:0/272027963 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2df81102b0 con 0x7f2df8070be0 2024-03-04T13:52:36.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:36.931+0000 7f2dffe00700 1 -- 172.21.15.107:0/272027963 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2df8106980 con 0x7f2df8070be0 2024-03-04T13:52:36.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:36.933+0000 7f2df5600700 1 -- 172.21.15.107:0/272027963 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f2de40139f0 con 0x7f2df8070be0 2024-03-04T13:52:36.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:36.933+0000 7f2df5600700 1 --2- 172.21.15.107:0/272027963 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f2de006d7e0 0x7f2de006fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:52:36.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:36.934+0000 7f2df5600700 1 -- 172.21.15.107:0/272027963 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f2de408de70 con 0x7f2df8070be0 2024-03-04T13:52:36.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:36.934+0000 7f2dfd400700 1 --2- 172.21.15.107:0/272027963 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f2de006d7e0 0x7f2de006fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:52:36.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:36.934+0000 7f2dfd400700 1 --2- 172.21.15.107:0/272027963 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f2de006d7e0 0x7f2de006fc90 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2dec004990 tx=0x7f2dec009ee0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:52:36.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:36.938+0000 7f2df5600700 1 -- 172.21.15.107:0/272027963 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2de4057830 con 0x7f2df8070be0 2024-03-04T13:52:37.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:37.183+0000 7f2dffe00700 1 -- 172.21.15.107:0/272027963 --> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2df804bf40 con 0x7f2de006d7e0 2024-03-04T13:52:37.185 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:37 smithi107 ceph-mon[112578]: mds.? [v2:172.21.15.107:6828/2973025326,v1:172.21.15.107:6829/2973025326] up:boot 2024-03-04T13:52:37.185 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:37 smithi107 ceph-mon[112578]: fsmap cephfs:1 {0=cephfs.smithi169.jusudl=up:active} 3 up:standby 2024-03-04T13:52:37.186 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:37 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi107.thycho"}]: dispatch 2024-03-04T13:52:37.186 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:37 smithi107 ceph-mon[112578]: pgmap v191: 65 pgs: 65 active+clean; 255 MiB data, 963 MiB used, 535 GiB / 536 GiB avail; 391 B/s rd, 0 op/s 2024-03-04T13:52:37.186 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:37 smithi107 ceph-mon[112578]: from='client.34404 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:52:37.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:37.185+0000 7f2df5600700 1 -- 172.21.15.107:0/272027963 <== mgr.34102 v2:172.21.15.107:6800/1223039812 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f2df804bf40 con 0x7f2de006d7e0 2024-03-04T13:52:37.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:37.188+0000 7f2dffe00700 1 -- 172.21.15.107:0/272027963 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f2de006d7e0 msgr2=0x7f2de006fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:52:37.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:37.188+0000 7f2dffe00700 1 --2- 172.21.15.107:0/272027963 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f2de006d7e0 0x7f2de006fc90 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2dec004990 tx=0x7f2dec009ee0 comp rx=0 tx=0).stop 2024-03-04T13:52:37.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:37.188+0000 7f2dffe00700 1 -- 172.21.15.107:0/272027963 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2df8070be0 msgr2=0x7f2df8116150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:52:37.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:37.188+0000 7f2dffe00700 1 --2- 172.21.15.107:0/272027963 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2df8070be0 0x7f2df8116150 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f2de4000c00 tx=0x7f2de4009e30 comp rx=0 tx=0).stop 2024-03-04T13:52:37.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:37.189+0000 7f2dffe00700 1 -- 172.21.15.107:0/272027963 shutdown_connections 2024-03-04T13:52:37.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:37.189+0000 7f2dffe00700 1 --2- 172.21.15.107:0/272027963 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f2de006d7e0 0x7f2de006fc90 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:52:37.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:37.189+0000 7f2dffe00700 1 --2- 172.21.15.107:0/272027963 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2df80702d0 0x7f2df8115c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:52:37.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:37.189+0000 7f2dffe00700 1 --2- 172.21.15.107:0/272027963 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2df8070be0 0x7f2df8116150 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:52:37.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:37.189+0000 7f2dffe00700 1 -- 172.21.15.107:0/272027963 >> 172.21.15.107:0/272027963 conn(0x7f2df80f8ae0 msgr2=0x7f2df8103b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:52:37.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:37.189+0000 7f2dffe00700 1 -- 172.21.15.107:0/272027963 shutdown_connections 2024-03-04T13:52:37.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:37.189+0000 7f2dffe00700 1 -- 172.21.15.107:0/272027963 wait complete. 2024-03-04T13:52:37.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:37.345+0000 7fc25b200700 1 -- 172.21.15.107:0/3717336506 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc25c105910 msgr2=0x7fc25c0fef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:52:37.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:37.345+0000 7fc25b200700 1 --2- 172.21.15.107:0/3717336506 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc25c105910 0x7fc25c0fef80 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc244004940 tx=0x7fc244009e30 comp rx=0 tx=0).stop 2024-03-04T13:52:37.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:37.346+0000 7fc25b200700 1 -- 172.21.15.107:0/3717336506 shutdown_connections 2024-03-04T13:52:37.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:37.346+0000 7fc25b200700 1 --2- 172.21.15.107:0/3717336506 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc25c105910 0x7fc25c0fef80 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:52:37.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:37.346+0000 7fc25b200700 1 --2- 172.21.15.107:0/3717336506 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc25c104f70 0x7fc25c105340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:52:37.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:37.346+0000 7fc25b200700 1 -- 172.21.15.107:0/3717336506 >> 172.21.15.107:0/3717336506 conn(0x7fc25c0faae0 msgr2=0x7fc25c0fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:52:37.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:37.346+0000 7fc25b200700 1 -- 172.21.15.107:0/3717336506 shutdown_connections 2024-03-04T13:52:37.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:37.346+0000 7fc25b200700 1 -- 172.21.15.107:0/3717336506 wait complete. 2024-03-04T13:52:37.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:37.346+0000 7fc25b200700 1 Processor -- start 2024-03-04T13:52:37.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:37.347+0000 7fc25b200700 1 -- start start 2024-03-04T13:52:37.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:37.347+0000 7fc25b200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc25c104f70 0x7fc25c115a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:52:37.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:37.347+0000 7fc25b200700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc25c105910 0x7fc25c115f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:52:37.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:37.347+0000 7fc25b200700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc25c1165a0 con 0x7fc25c104f70 2024-03-04T13:52:37.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:37.347+0000 7fc25b200700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc25c10f8d0 con 0x7fc25c105910 2024-03-04T13:52:37.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:37.347+0000 7fc259e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc25c104f70 0x7fc25c115a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:52:37.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:37.347+0000 7fc259e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc25c104f70 0x7fc25c115a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40422/0 (socket says 172.21.15.107:40422) 2024-03-04T13:52:37.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:37.347+0000 7fc259e00700 1 -- 172.21.15.107:0/73555212 learned_addr learned my addr 172.21.15.107:0/73555212 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:52:37.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:37.348+0000 7fc259400700 1 --2- 172.21.15.107:0/73555212 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc25c105910 0x7fc25c115f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:52:37.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:37.348+0000 7fc259e00700 1 -- 172.21.15.107:0/73555212 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc25c105910 msgr2=0x7fc25c115f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:52:37.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:37.348+0000 7fc259e00700 1 --2- 172.21.15.107:0/73555212 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc25c105910 0x7fc25c115f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:52:37.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:37.348+0000 7fc259e00700 1 -- 172.21.15.107:0/73555212 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc244005040 con 0x7fc25c104f70 2024-03-04T13:52:37.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:37.348+0000 7fc259400700 1 --2- 172.21.15.107:0/73555212 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc25c105910 0x7fc25c115f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T13:52:37.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:37.348+0000 7fc259e00700 1 --2- 172.21.15.107:0/73555212 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc25c104f70 0x7fc25c115a10 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc24c00c830 tx=0x7fc24c004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:52:37.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:37.348+0000 7fc252000700 1 -- 172.21.15.107:0/73555212 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc24c00a760 con 0x7fc25c104f70 2024-03-04T13:52:37.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:37.348+0000 7fc252000700 1 -- 172.21.15.107:0/73555212 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc24c00ad90 con 0x7fc25c104f70 2024-03-04T13:52:37.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:37.348+0000 7fc252000700 1 -- 172.21.15.107:0/73555212 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc24c0113a0 con 0x7fc25c104f70 2024-03-04T13:52:37.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:37.348+0000 7fc25b200700 1 -- 172.21.15.107:0/73555212 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc25c10fbb0 con 0x7fc25c104f70 2024-03-04T13:52:37.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:37.349+0000 7fc25b200700 1 -- 172.21.15.107:0/73555212 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc25c110170 con 0x7fc25c104f70 2024-03-04T13:52:37.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:37.350+0000 7fc25b200700 1 -- 172.21.15.107:0/73555212 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc25c05f5c0 con 0x7fc25c104f70 2024-03-04T13:52:37.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:37.351+0000 7fc252000700 1 -- 172.21.15.107:0/73555212 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fc24c012400 con 0x7fc25c104f70 2024-03-04T13:52:37.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:37.351+0000 7fc252000700 1 --2- 172.21.15.107:0/73555212 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fc24006d7f0 0x7fc24006fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:52:37.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:37.352+0000 7fc252000700 1 -- 172.21.15.107:0/73555212 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fc24c08ceb0 con 0x7fc25c104f70 2024-03-04T13:52:37.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:37.352+0000 7fc259400700 1 --2- 172.21.15.107:0/73555212 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fc24006d7f0 0x7fc24006fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:52:37.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:37.353+0000 7fc259400700 1 --2- 172.21.15.107:0/73555212 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fc24006d7f0 0x7fc24006fca0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc24400b3b0 tx=0x7fc244009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:52:37.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:37.356+0000 7fc252000700 1 -- 172.21.15.107:0/73555212 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc24c05a2e0 con 0x7fc25c104f70 2024-03-04T13:52:37.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:37 smithi169 ceph-mon[86183]: mds.? [v2:172.21.15.107:6828/2973025326,v1:172.21.15.107:6829/2973025326] up:boot 2024-03-04T13:52:37.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:37 smithi169 ceph-mon[86183]: fsmap cephfs:1 {0=cephfs.smithi169.jusudl=up:active} 3 up:standby 2024-03-04T13:52:37.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:37 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi107.thycho"}]: dispatch 2024-03-04T13:52:37.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:37 smithi169 ceph-mon[86183]: pgmap v191: 65 pgs: 65 active+clean; 255 MiB data, 963 MiB used, 535 GiB / 536 GiB avail; 391 B/s rd, 0 op/s 2024-03-04T13:52:37.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:37 smithi169 ceph-mon[86183]: from='client.34404 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:52:37.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:37.607+0000 7fc25b200700 1 -- 172.21.15.107:0/73555212 --> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fc25c1104a0 con 0x7fc24006d7f0 2024-03-04T13:52:37.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:37.640+0000 7fc252000700 1 -- 172.21.15.107:0/73555212 <== mgr.34102 v2:172.21.15.107:6800/1223039812 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7fc25c1104a0 con 0x7fc24006d7f0 2024-03-04T13:52:37.642 INFO:teuthology.orchestra.run.smithi107.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-04T13:52:37.642 INFO:teuthology.orchestra.run.smithi107.stdout:alertmanager.smithi107 smithi107 *:9093,9094 running (6m) 2m ago 20m 14.6M - 0.25.0 c8568f914cd2 e5ed7be32538 2024-03-04T13:52:37.642 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi107 smithi107 running (3m) 2m ago 21m 6660k - 19.0.0-1845-g717ce3cc d71b839abdf8 0558022b0d84 2024-03-04T13:52:37.642 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi169 smithi169 running (3m) 12s ago 19m 6665k - 19.0.0-1845-g717ce3cc d71b839abdf8 221ca53aa0d3 2024-03-04T13:52:37.642 INFO:teuthology.orchestra.run.smithi107.stdout:grafana.smithi107 smithi107 *:3000 running (6m) 2m ago 20m 74.5M - 9.4.12 a0ea5e3c46d9 c5e4d33fb599 2024-03-04T13:52:37.643 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.riexmm smithi107 running (14m) 2m ago 14m 14.9M - 17.2.7-545-g42608bb1 9eec3f22859d fc921f7d4328 2024-03-04T13:52:37.643 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.thycho smithi107 running (13m) 2m ago 13m 98.3M - 17.2.7-545-g42608bb1 9eec3f22859d 4069c6e0e063 2024-03-04T13:52:37.643 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi169.bxitzk smithi169 running (13m) 12s ago 13m 15.1M - 17.2.7-545-g42608bb1 9eec3f22859d 7d4115228c7c 2024-03-04T13:52:37.643 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi169.jusudl smithi169 running (13m) 12s ago 13m 145M - 17.2.7-545-g42608bb1 9eec3f22859d 2d2f2cc452e8 2024-03-04T13:52:37.643 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi107.wpzgwl smithi107 *:8443,9283,8765 running (9m) 2m ago 22m 504M - 19.0.0-1845-g717ce3cc d71b839abdf8 1c5fdee45eac 2024-03-04T13:52:37.643 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi169.eihczr smithi169 *:8443,9283,8765 running (8m) 12s ago 19m 451M - 19.0.0-1845-g717ce3cc d71b839abdf8 b5cac2b7d22a 2024-03-04T13:52:37.643 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi107 smithi107 running (5m) 2m ago 22m 46.5M 2048M 19.0.0-1845-g717ce3cc d71b839abdf8 32f07144274f 2024-03-04T13:52:37.643 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi169 smithi169 running (4m) 12s ago 19m 37.6M 2048M 19.0.0-1845-g717ce3cc d71b839abdf8 e3527449381a 2024-03-04T13:52:37.643 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi107 smithi107 *:9100 running (8m) 2m ago 21m 7680k - 1.5.0 0da6a335fe13 8a8d3d4f724f 2024-03-04T13:52:37.643 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi169 smithi169 *:9100 running (7m) 12s ago 19m 7763k - 1.5.0 0da6a335fe13 f240f5a975bc 2024-03-04T13:52:37.643 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0 smithi107 running (3m) 2m ago 18m 80.5M 1558M 19.0.0-1845-g717ce3cc d71b839abdf8 5801a5540dc6 2024-03-04T13:52:37.643 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1 smithi107 running (2m) 2m ago 18m 91.3M 1558M 19.0.0-1845-g717ce3cc d71b839abdf8 b614af658854 2024-03-04T13:52:37.643 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2 smithi107 running (2m) 2m ago 17m 11.1M 1558M 19.0.0-1845-g717ce3cc d71b839abdf8 6e4da4f6f146 2024-03-04T13:52:37.643 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3 smithi169 running (102s) 12s ago 16m 94.2M 2582M 19.0.0-1845-g717ce3cc d71b839abdf8 d31baf764d7a 2024-03-04T13:52:37.644 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4 smithi169 running (60s) 12s ago 16m 84.1M 2582M 19.0.0-1845-g717ce3cc d71b839abdf8 a70bacd97639 2024-03-04T13:52:37.644 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5 smithi169 running (17s) 12s ago 15m 11.2M 2582M 19.0.0-1845-g717ce3cc d71b839abdf8 5f6e00f7993d 2024-03-04T13:52:37.644 INFO:teuthology.orchestra.run.smithi107.stdout:prometheus.smithi107 smithi107 *:9095 running (7m) 2m ago 20m 51.0M - 2.43.0 a07b618ecd1d 1eeb1ad5cb75 2024-03-04T13:52:37.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:37.643+0000 7fc25b200700 1 -- 172.21.15.107:0/73555212 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fc24006d7f0 msgr2=0x7fc24006fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:52:37.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:37.643+0000 7fc25b200700 1 --2- 172.21.15.107:0/73555212 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fc24006d7f0 0x7fc24006fca0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc24400b3b0 tx=0x7fc244009e30 comp rx=0 tx=0).stop 2024-03-04T13:52:37.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:37.643+0000 7fc25b200700 1 -- 172.21.15.107:0/73555212 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc25c104f70 msgr2=0x7fc25c115a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:52:37.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:37.643+0000 7fc25b200700 1 --2- 172.21.15.107:0/73555212 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc25c104f70 0x7fc25c115a10 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc24c00c830 tx=0x7fc24c004470 comp rx=0 tx=0).stop 2024-03-04T13:52:37.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:37.643+0000 7fc25b200700 1 -- 172.21.15.107:0/73555212 shutdown_connections 2024-03-04T13:52:37.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:37.643+0000 7fc25b200700 1 --2- 172.21.15.107:0/73555212 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fc24006d7f0 0x7fc24006fca0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:52:37.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:37.643+0000 7fc25b200700 1 --2- 172.21.15.107:0/73555212 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc25c104f70 0x7fc25c115a10 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:52:37.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:37.643+0000 7fc25b200700 1 --2- 172.21.15.107:0/73555212 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc25c105910 0x7fc25c115f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:52:37.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:37.643+0000 7fc25b200700 1 -- 172.21.15.107:0/73555212 >> 172.21.15.107:0/73555212 conn(0x7fc25c0faae0 msgr2=0x7fc25c0fc560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:52:37.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:37.643+0000 7fc25b200700 1 -- 172.21.15.107:0/73555212 shutdown_connections 2024-03-04T13:52:37.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:37.644+0000 7fc25b200700 1 -- 172.21.15.107:0/73555212 wait complete. 2024-03-04T13:52:37.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:37.797+0000 7fee33400700 1 -- 172.21.15.107:0/672501956 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee2c070bc0 msgr2=0x7fee2c1092e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:52:37.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:37.797+0000 7fee33400700 1 --2- 172.21.15.107:0/672501956 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee2c070bc0 0x7fee2c1092e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fee18004940 tx=0x7fee18009e30 comp rx=0 tx=0).stop 2024-03-04T13:52:37.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:37.797+0000 7fee33400700 1 -- 172.21.15.107:0/672501956 shutdown_connections 2024-03-04T13:52:37.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:37.797+0000 7fee33400700 1 --2- 172.21.15.107:0/672501956 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee2c070bc0 0x7fee2c1092e0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:52:37.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:37.797+0000 7fee33400700 1 --2- 172.21.15.107:0/672501956 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fee2c0702b0 0x7fee2c070680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:52:37.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:37.797+0000 7fee33400700 1 -- 172.21.15.107:0/672501956 >> 172.21.15.107:0/672501956 conn(0x7fee2c0f8ae0 msgr2=0x7fee2c0faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:52:37.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:37.797+0000 7fee33400700 1 -- 172.21.15.107:0/672501956 shutdown_connections 2024-03-04T13:52:37.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:37.798+0000 7fee33400700 1 -- 172.21.15.107:0/672501956 wait complete. 2024-03-04T13:52:37.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:37.799+0000 7fee33400700 1 Processor -- start 2024-03-04T13:52:37.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:37.799+0000 7fee33400700 1 -- start start 2024-03-04T13:52:37.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:37.799+0000 7fee33400700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fee2c0702b0 0x7fee2c115d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:52:37.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:37.799+0000 7fee33400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee2c070bc0 0x7fee2c116290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:52:37.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:37.799+0000 7fee33400700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee2c1168e0 con 0x7fee2c070bc0 2024-03-04T13:52:37.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:37.799+0000 7fee33400700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee2c10fbf0 con 0x7fee2c0702b0 2024-03-04T13:52:37.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:37.800+0000 7fee2be00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee2c070bc0 0x7fee2c116290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:52:37.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:37.800+0000 7fee2be00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee2c070bc0 0x7fee2c116290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40434/0 (socket says 172.21.15.107:40434) 2024-03-04T13:52:37.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:37.800+0000 7fee2be00700 1 -- 172.21.15.107:0/1206232010 learned_addr learned my addr 172.21.15.107:0/1206232010 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:52:37.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:37.800+0000 7fee30a00700 1 --2- 172.21.15.107:0/1206232010 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fee2c0702b0 0x7fee2c115d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:52:37.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:37.800+0000 7fee2be00700 1 -- 172.21.15.107:0/1206232010 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fee2c0702b0 msgr2=0x7fee2c115d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:52:37.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:37.800+0000 7fee2be00700 1 --2- 172.21.15.107:0/1206232010 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fee2c0702b0 0x7fee2c115d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:52:37.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:37.800+0000 7fee2be00700 1 -- 172.21.15.107:0/1206232010 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee18005040 con 0x7fee2c070bc0 2024-03-04T13:52:37.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:37.801+0000 7fee2be00700 1 --2- 172.21.15.107:0/1206232010 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee2c070bc0 0x7fee2c116290 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fee18009150 tx=0x7fee18009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:52:37.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:37.801+0000 7fee28c00700 1 -- 172.21.15.107:0/1206232010 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fee1800e040 con 0x7fee2c070bc0 2024-03-04T13:52:37.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:37.801+0000 7fee28c00700 1 -- 172.21.15.107:0/1206232010 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fee1800bbf0 con 0x7fee2c070bc0 2024-03-04T13:52:37.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:37.801+0000 7fee33400700 1 -- 172.21.15.107:0/1206232010 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee2c10fe70 con 0x7fee2c070bc0 2024-03-04T13:52:37.804 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:37.801+0000 7fee28c00700 1 -- 172.21.15.107:0/1206232010 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fee1801c630 con 0x7fee2c070bc0 2024-03-04T13:52:37.804 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:37.801+0000 7fee33400700 1 -- 172.21.15.107:0/1206232010 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee2c1103b0 con 0x7fee2c070bc0 2024-03-04T13:52:37.804 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:37.803+0000 7fee33400700 1 -- 172.21.15.107:0/1206232010 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee2c1069a0 con 0x7fee2c070bc0 2024-03-04T13:52:37.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:37.804+0000 7fee28c00700 1 -- 172.21.15.107:0/1206232010 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fee1800bd60 con 0x7fee2c070bc0 2024-03-04T13:52:37.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:37.805+0000 7fee28c00700 1 --2- 172.21.15.107:0/1206232010 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fee1406d7f0 0x7fee1406fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:52:37.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:37.805+0000 7fee28c00700 1 -- 172.21.15.107:0/1206232010 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fee1808e050 con 0x7fee2c070bc0 2024-03-04T13:52:37.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:37.806+0000 7fee30a00700 1 --2- 172.21.15.107:0/1206232010 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fee1406d7f0 0x7fee1406fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:52:37.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:37.806+0000 7fee30a00700 1 --2- 172.21.15.107:0/1206232010 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fee1406d7f0 0x7fee1406fca0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fee20007fd0 tx=0x7fee20009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:52:37.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:37.810+0000 7fee28c00700 1 -- 172.21.15.107:0/1206232010 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fee1805b4a0 con 0x7fee2c070bc0 2024-03-04T13:52:38.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:38.128+0000 7fee33400700 1 -- 172.21.15.107:0/1206232010 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fee2c0fc6e0 con 0x7fee2c070bc0 2024-03-04T13:52:38.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:38.128+0000 7fee28c00700 1 -- 172.21.15.107:0/1206232010 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+733 (secure 0 0 0) 0x7fee1805b030 con 0x7fee2c070bc0 2024-03-04T13:52:38.131 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-03-04T13:52:38.131 INFO:teuthology.orchestra.run.smithi107.stdout: "mon": { 2024-03-04T13:52:38.131 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-1845-g717ce3cc (717ce3cce8d0166e4e577faa8004e6f9cb4128c0) squid (dev)": 2 2024-03-04T13:52:38.131 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:52:38.131 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr": { 2024-03-04T13:52:38.131 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-1845-g717ce3cc (717ce3cce8d0166e4e577faa8004e6f9cb4128c0) squid (dev)": 2 2024-03-04T13:52:38.131 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:52:38.131 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": { 2024-03-04T13:52:38.131 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-1845-g717ce3cc (717ce3cce8d0166e4e577faa8004e6f9cb4128c0) squid (dev)": 6 2024-03-04T13:52:38.131 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:52:38.131 INFO:teuthology.orchestra.run.smithi107.stdout: "mds": { 2024-03-04T13:52:38.131 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-545-g42608bb1 (42608bb191117be10dba720e6bf8a02f3171cd34) quincy (stable)": 4 2024-03-04T13:52:38.131 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:52:38.131 INFO:teuthology.orchestra.run.smithi107.stdout: "overall": { 2024-03-04T13:52:38.131 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-545-g42608bb1 (42608bb191117be10dba720e6bf8a02f3171cd34) quincy (stable)": 4, 2024-03-04T13:52:38.132 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-1845-g717ce3cc (717ce3cce8d0166e4e577faa8004e6f9cb4128c0) squid (dev)": 10 2024-03-04T13:52:38.132 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-04T13:52:38.132 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-03-04T13:52:38.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:38.131+0000 7fee33400700 1 -- 172.21.15.107:0/1206232010 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fee1406d7f0 msgr2=0x7fee1406fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:52:38.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:38.131+0000 7fee33400700 1 --2- 172.21.15.107:0/1206232010 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fee1406d7f0 0x7fee1406fca0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fee20007fd0 tx=0x7fee20009040 comp rx=0 tx=0).stop 2024-03-04T13:52:38.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:38.131+0000 7fee33400700 1 -- 172.21.15.107:0/1206232010 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee2c070bc0 msgr2=0x7fee2c116290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:52:38.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:38.131+0000 7fee33400700 1 --2- 172.21.15.107:0/1206232010 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee2c070bc0 0x7fee2c116290 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fee18009150 tx=0x7fee18009e30 comp rx=0 tx=0).stop 2024-03-04T13:52:38.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:38.132+0000 7fee33400700 1 -- 172.21.15.107:0/1206232010 shutdown_connections 2024-03-04T13:52:38.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:38.132+0000 7fee33400700 1 --2- 172.21.15.107:0/1206232010 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fee1406d7f0 0x7fee1406fca0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:52:38.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:38.132+0000 7fee33400700 1 --2- 172.21.15.107:0/1206232010 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fee2c0702b0 0x7fee2c115d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:52:38.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:38.132+0000 7fee33400700 1 --2- 172.21.15.107:0/1206232010 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee2c070bc0 0x7fee2c116290 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:52:38.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:38.132+0000 7fee33400700 1 -- 172.21.15.107:0/1206232010 >> 172.21.15.107:0/1206232010 conn(0x7fee2c0f8ae0 msgr2=0x7fee2c103b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:52:38.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:38.132+0000 7fee33400700 1 -- 172.21.15.107:0/1206232010 shutdown_connections 2024-03-04T13:52:38.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:38.132+0000 7fee33400700 1 -- 172.21.15.107:0/1206232010 wait complete. 2024-03-04T13:52:38.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:38.281+0000 7fe50b200700 1 -- 172.21.15.107:0/3629830920 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe5041038f0 msgr2=0x7fe5040fcf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:52:38.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:38.281+0000 7fe50b200700 1 --2- 172.21.15.107:0/3629830920 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe5041038f0 0x7fe5040fcf60 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe4f0004940 tx=0x7fe4f0009e30 comp rx=0 tx=0).stop 2024-03-04T13:52:38.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:38.281+0000 7fe50b200700 1 -- 172.21.15.107:0/3629830920 shutdown_connections 2024-03-04T13:52:38.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:38.281+0000 7fe50b200700 1 --2- 172.21.15.107:0/3629830920 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe5041038f0 0x7fe5040fcf60 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:52:38.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:38.281+0000 7fe50b200700 1 --2- 172.21.15.107:0/3629830920 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe504102f50 0x7fe504103320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:52:38.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:38.281+0000 7fe50b200700 1 -- 172.21.15.107:0/3629830920 >> 172.21.15.107:0/3629830920 conn(0x7fe5040f8ae0 msgr2=0x7fe5040faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:52:38.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:38.281+0000 7fe50b200700 1 -- 172.21.15.107:0/3629830920 shutdown_connections 2024-03-04T13:52:38.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:38.281+0000 7fe50b200700 1 -- 172.21.15.107:0/3629830920 wait complete. 2024-03-04T13:52:38.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:38.282+0000 7fe50b200700 1 Processor -- start 2024-03-04T13:52:38.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:38.282+0000 7fe50b200700 1 -- start start 2024-03-04T13:52:38.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:38.283+0000 7fe50b200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe504102f50 0x7fe504115c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:52:38.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:38.283+0000 7fe50b200700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5041038f0 0x7fe5041161c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:52:38.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:38.283+0000 7fe50b200700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe504116810 con 0x7fe504102f50 2024-03-04T13:52:38.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:38.283+0000 7fe50b200700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe50410fb40 con 0x7fe5041038f0 2024-03-04T13:52:38.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:38.283+0000 7fe503e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe504102f50 0x7fe504115c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:52:38.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:38.283+0000 7fe503e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe504102f50 0x7fe504115c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51440/0 (socket says 172.21.15.107:51440) 2024-03-04T13:52:38.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:38.283+0000 7fe503e00700 1 -- 172.21.15.107:0/3191190878 learned_addr learned my addr 172.21.15.107:0/3191190878 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:52:38.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:38.283+0000 7fe503e00700 1 -- 172.21.15.107:0/3191190878 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5041038f0 msgr2=0x7fe5041161c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:52:38.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:38.283+0000 7fe503e00700 1 --2- 172.21.15.107:0/3191190878 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5041038f0 0x7fe5041161c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:52:38.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:38.283+0000 7fe503e00700 1 -- 172.21.15.107:0/3191190878 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4f0005040 con 0x7fe504102f50 2024-03-04T13:52:38.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:38.283+0000 7fe503e00700 1 --2- 172.21.15.107:0/3191190878 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe504102f50 0x7fe504115c80 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe4f4002a50 tx=0x7fe4f40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:52:38.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:38.284+0000 7fe4fbe00700 1 -- 172.21.15.107:0/3191190878 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe4f4007cc0 con 0x7fe504102f50 2024-03-04T13:52:38.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:38.284+0000 7fe4fbe00700 1 -- 172.21.15.107:0/3191190878 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe4f400f040 con 0x7fe504102f50 2024-03-04T13:52:38.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:38.284+0000 7fe50b200700 1 -- 172.21.15.107:0/3191190878 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe50410fe20 con 0x7fe504102f50 2024-03-04T13:52:38.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:38.284+0000 7fe4fbe00700 1 -- 172.21.15.107:0/3191190878 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe4f4009640 con 0x7fe504102f50 2024-03-04T13:52:38.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:38.284+0000 7fe50b200700 1 -- 172.21.15.107:0/3191190878 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe504110360 con 0x7fe504102f50 2024-03-04T13:52:38.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:38.285+0000 7fe50b200700 1 -- 172.21.15.107:0/3191190878 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5040fc6e0 con 0x7fe504102f50 2024-03-04T13:52:38.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:38.286+0000 7fe4fbe00700 1 -- 172.21.15.107:0/3191190878 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fe4f40097a0 con 0x7fe504102f50 2024-03-04T13:52:38.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:38.287+0000 7fe4fbe00700 1 --2- 172.21.15.107:0/3191190878 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fe4e806d7f0 0x7fe4e806fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:52:38.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:38.287+0000 7fe4fbe00700 1 -- 172.21.15.107:0/3191190878 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fe4f408ccd0 con 0x7fe504102f50 2024-03-04T13:52:38.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:38.287+0000 7fe503400700 1 --2- 172.21.15.107:0/3191190878 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fe4e806d7f0 0x7fe4e806fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:52:38.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:38.288+0000 7fe503400700 1 --2- 172.21.15.107:0/3191190878 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fe4e806d7f0 0x7fe4e806fca0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe4f0004f90 tx=0x7fe4f0009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:52:38.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:38.292+0000 7fe4fbe00700 1 -- 172.21.15.107:0/3191190878 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe4f405a000 con 0x7fe504102f50 2024-03-04T13:52:38.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:38.573+0000 7fe50b200700 1 -- 172.21.15.107:0/3191190878 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fe504111220 con 0x7fe504102f50 2024-03-04T13:52:38.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:38.574+0000 7fe4fbe00700 1 -- 172.21.15.107:0/3191190878 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 18 v18) v1 ==== 76+0+1852 (secure 0 0 0) 0x7fe4f4059b90 con 0x7fe504102f50 2024-03-04T13:52:38.576 INFO:teuthology.orchestra.run.smithi107.stdout:e18 2024-03-04T13:52:38.577 INFO:teuthology.orchestra.run.smithi107.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-04T13:52:38.577 INFO:teuthology.orchestra.run.smithi107.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-04T13:52:38.577 INFO:teuthology.orchestra.run.smithi107.stdout:legacy client fscid: 1 2024-03-04T13:52:38.577 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:52:38.577 INFO:teuthology.orchestra.run.smithi107.stdout:Filesystem 'cephfs' (1) 2024-03-04T13:52:38.577 INFO:teuthology.orchestra.run.smithi107.stdout:fs_name cephfs 2024-03-04T13:52:38.577 INFO:teuthology.orchestra.run.smithi107.stdout:epoch 17 2024-03-04T13:52:38.577 INFO:teuthology.orchestra.run.smithi107.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-04T13:52:38.577 INFO:teuthology.orchestra.run.smithi107.stdout:created 2024-03-04T13:38:30.854004+0000 2024-03-04T13:52:38.577 INFO:teuthology.orchestra.run.smithi107.stdout:modified 2024-03-04T13:52:34.079716+0000 2024-03-04T13:52:38.577 INFO:teuthology.orchestra.run.smithi107.stdout:tableserver 0 2024-03-04T13:52:38.577 INFO:teuthology.orchestra.run.smithi107.stdout:root 0 2024-03-04T13:52:38.577 INFO:teuthology.orchestra.run.smithi107.stdout:session_timeout 60 2024-03-04T13:52:38.577 INFO:teuthology.orchestra.run.smithi107.stdout:session_autoclose 300 2024-03-04T13:52:38.578 INFO:teuthology.orchestra.run.smithi107.stdout:max_file_size 1099511627776 2024-03-04T13:52:38.578 INFO:teuthology.orchestra.run.smithi107.stdout:max_xattr_size 65536 2024-03-04T13:52:38.578 INFO:teuthology.orchestra.run.smithi107.stdout:required_client_features {} 2024-03-04T13:52:38.578 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure 0 2024-03-04T13:52:38.578 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure_osd_epoch 0 2024-03-04T13:52:38.578 INFO:teuthology.orchestra.run.smithi107.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-04T13:52:38.578 INFO:teuthology.orchestra.run.smithi107.stdout:max_mds 1 2024-03-04T13:52:38.578 INFO:teuthology.orchestra.run.smithi107.stdout:in 0 2024-03-04T13:52:38.578 INFO:teuthology.orchestra.run.smithi107.stdout:up {0=24247} 2024-03-04T13:52:38.578 INFO:teuthology.orchestra.run.smithi107.stdout:failed 2024-03-04T13:52:38.578 INFO:teuthology.orchestra.run.smithi107.stdout:damaged 2024-03-04T13:52:38.578 INFO:teuthology.orchestra.run.smithi107.stdout:stopped 2024-03-04T13:52:38.578 INFO:teuthology.orchestra.run.smithi107.stdout:data_pools [3] 2024-03-04T13:52:38.578 INFO:teuthology.orchestra.run.smithi107.stdout:metadata_pool 2 2024-03-04T13:52:38.578 INFO:teuthology.orchestra.run.smithi107.stdout:inline_data enabled 2024-03-04T13:52:38.579 INFO:teuthology.orchestra.run.smithi107.stdout:balancer 2024-03-04T13:52:38.579 INFO:teuthology.orchestra.run.smithi107.stdout:bal_rank_mask -1 2024-03-04T13:52:38.579 INFO:teuthology.orchestra.run.smithi107.stdout:standby_count_wanted 1 2024-03-04T13:52:38.579 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi169.jusudl{0:24247} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.169:6824/2994866076,v1:172.21.15.169:6825/2994866076] compat {c=[1],r=[1],i=[7ff]}] 2024-03-04T13:52:38.579 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:52:38.579 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:52:38.579 INFO:teuthology.orchestra.run.smithi107.stdout:Standby daemons: 2024-03-04T13:52:38.579 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:52:38.579 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.riexmm{-1:14480} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.107:6826/3261622517,v1:172.21.15.107:6827/3261622517] compat {c=[1],r=[1],i=[7ff]}] 2024-03-04T13:52:38.579 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi169.bxitzk{-1:24259} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.169:6826/3696090493,v1:172.21.15.169:6827/3696090493] compat {c=[1],r=[1],i=[7ff]}] 2024-03-04T13:52:38.579 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.thycho{-1:34400} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.107:6828/2973025326,v1:172.21.15.107:6829/2973025326] compat {c=[1],r=[1],i=[7ff]}] 2024-03-04T13:52:38.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:38.576+0000 7fe50b200700 1 -- 172.21.15.107:0/3191190878 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fe4e806d7f0 msgr2=0x7fe4e806fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:52:38.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:38.576+0000 7fe50b200700 1 --2- 172.21.15.107:0/3191190878 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fe4e806d7f0 0x7fe4e806fca0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe4f0004f90 tx=0x7fe4f0009e30 comp rx=0 tx=0).stop 2024-03-04T13:52:38.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:38.576+0000 7fe50b200700 1 -- 172.21.15.107:0/3191190878 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe504102f50 msgr2=0x7fe504115c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:52:38.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:38.576+0000 7fe50b200700 1 --2- 172.21.15.107:0/3191190878 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe504102f50 0x7fe504115c80 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe4f4002a50 tx=0x7fe4f40046b0 comp rx=0 tx=0).stop 2024-03-04T13:52:38.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:38.577+0000 7fe50b200700 1 -- 172.21.15.107:0/3191190878 shutdown_connections 2024-03-04T13:52:38.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:38.577+0000 7fe50b200700 1 --2- 172.21.15.107:0/3191190878 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fe4e806d7f0 0x7fe4e806fca0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:52:38.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:38.577+0000 7fe50b200700 1 --2- 172.21.15.107:0/3191190878 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe504102f50 0x7fe504115c80 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:52:38.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:38.577+0000 7fe50b200700 1 --2- 172.21.15.107:0/3191190878 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5041038f0 0x7fe5041161c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:52:38.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:38.577+0000 7fe50b200700 1 -- 172.21.15.107:0/3191190878 >> 172.21.15.107:0/3191190878 conn(0x7fe5040f8ae0 msgr2=0x7fe5040fa470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:52:38.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:38.577+0000 7fe50b200700 1 -- 172.21.15.107:0/3191190878 shutdown_connections 2024-03-04T13:52:38.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:38.577+0000 7fe50b200700 1 -- 172.21.15.107:0/3191190878 wait complete. 2024-03-04T13:52:38.581 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 18 2024-03-04T13:52:38.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:38.734+0000 7fc0a9600700 1 -- 172.21.15.107:0/1472768345 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0a40fbdd0 msgr2=0x7fc0a40fc1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:52:38.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:38.734+0000 7fc0a9600700 1 --2- 172.21.15.107:0/1472768345 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0a40fbdd0 0x7fc0a40fc1a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc094004940 tx=0x7fc094009e30 comp rx=0 tx=0).stop 2024-03-04T13:52:38.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:38.734+0000 7fc0a9600700 1 -- 172.21.15.107:0/1472768345 shutdown_connections 2024-03-04T13:52:38.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:38.734+0000 7fc0a9600700 1 --2- 172.21.15.107:0/1472768345 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0a40fc6e0 0x7fc0a4109440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:52:38.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:38.734+0000 7fc0a9600700 1 --2- 172.21.15.107:0/1472768345 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0a40fbdd0 0x7fc0a40fc1a0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:52:38.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:38.734+0000 7fc0a9600700 1 -- 172.21.15.107:0/1472768345 >> 172.21.15.107:0/1472768345 conn(0x7fc0a4071d90 msgr2=0x7fc0a4072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:52:38.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:38.735+0000 7fc0a9600700 1 -- 172.21.15.107:0/1472768345 shutdown_connections 2024-03-04T13:52:38.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:38.735+0000 7fc0a9600700 1 -- 172.21.15.107:0/1472768345 wait complete. 2024-03-04T13:52:38.737 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:38 smithi107 ceph-mon[112578]: from='client.34408 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:52:38.737 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:38 smithi107 ceph-mon[112578]: Upgrade: Updating mds.cephfs.smithi107.riexmm 2024-03-04T13:52:38.738 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:38 smithi107 ceph-mon[112578]: from='client.34412 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:52:38.738 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:38 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:52:38.738 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:38 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi107.riexmm", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-04T13:52:38.738 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:38 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:52:38.738 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:38 smithi107 ceph-mon[112578]: Deploying daemon mds.cephfs.smithi107.riexmm on smithi107 2024-03-04T13:52:38.738 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:38 smithi107 ceph-mon[112578]: from='client.? 172.21.15.107:0/1206232010' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:52:38.738 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:38 smithi107 ceph-mon[112578]: from='client.? 172.21.15.107:0/3191190878' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-04T13:52:38.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:38.736+0000 7fc0a9600700 1 Processor -- start 2024-03-04T13:52:38.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:38.736+0000 7fc0a9600700 1 -- start start 2024-03-04T13:52:38.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:38.736+0000 7fc0a9600700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0a40fbdd0 0x7fc0a4115c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:52:38.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:38.736+0000 7fc0a9600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0a40fc6e0 0x7fc0a4116160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:52:38.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:38.736+0000 7fc0a9600700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a4116840 con 0x7fc0a40fc6e0 2024-03-04T13:52:38.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:38.736+0000 7fc0a9600700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a410fae0 con 0x7fc0a40fbdd0 2024-03-04T13:52:38.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:38.736+0000 7fc0a2000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0a40fc6e0 0x7fc0a4116160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:52:38.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:38.737+0000 7fc0a2000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0a40fc6e0 0x7fc0a4116160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51454/0 (socket says 172.21.15.107:51454) 2024-03-04T13:52:38.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:38.737+0000 7fc0a2000700 1 -- 172.21.15.107:0/2265597879 learned_addr learned my addr 172.21.15.107:0/2265597879 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:52:38.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:38.737+0000 7fc0a2a00700 1 --2- 172.21.15.107:0/2265597879 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0a40fbdd0 0x7fc0a4115c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:52:38.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:38.737+0000 7fc0a2000700 1 -- 172.21.15.107:0/2265597879 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0a40fbdd0 msgr2=0x7fc0a4115c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:52:38.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:38.737+0000 7fc0a2000700 1 --2- 172.21.15.107:0/2265597879 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0a40fbdd0 0x7fc0a4115c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:52:38.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:38.737+0000 7fc0a2000700 1 -- 172.21.15.107:0/2265597879 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc094005040 con 0x7fc0a40fc6e0 2024-03-04T13:52:38.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:38.737+0000 7fc0a2000700 1 --2- 172.21.15.107:0/2265597879 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0a40fc6e0 0x7fc0a4116160 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc09000b790 tx=0x7fc090009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:52:38.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:38.737+0000 7fc09aa00700 1 -- 172.21.15.107:0/2265597879 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc09000f770 con 0x7fc0a40fc6e0 2024-03-04T13:52:38.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:38.738+0000 7fc0a9600700 1 -- 172.21.15.107:0/2265597879 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0a410fdc0 con 0x7fc0a40fc6e0 2024-03-04T13:52:38.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:38.738+0000 7fc09aa00700 1 -- 172.21.15.107:0/2265597879 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc09000fda0 con 0x7fc0a40fc6e0 2024-03-04T13:52:38.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:38.738+0000 7fc09aa00700 1 -- 172.21.15.107:0/2265597879 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc09000d310 con 0x7fc0a40fc6e0 2024-03-04T13:52:38.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:38.738+0000 7fc0a9600700 1 -- 172.21.15.107:0/2265597879 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0a4110300 con 0x7fc0a40fc6e0 2024-03-04T13:52:38.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:38.739+0000 7fc0a9600700 1 -- 172.21.15.107:0/2265597879 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0a4106b70 con 0x7fc0a40fc6e0 2024-03-04T13:52:38.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:38.740+0000 7fc09aa00700 1 -- 172.21.15.107:0/2265597879 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fc090010420 con 0x7fc0a40fc6e0 2024-03-04T13:52:38.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:38.741+0000 7fc09aa00700 1 --2- 172.21.15.107:0/2265597879 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fc08806d7f0 0x7fc08806fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:52:38.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:38.741+0000 7fc09aa00700 1 -- 172.21.15.107:0/2265597879 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fc09008d5d0 con 0x7fc0a40fc6e0 2024-03-04T13:52:38.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:38.741+0000 7fc0a2a00700 1 --2- 172.21.15.107:0/2265597879 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fc08806d7f0 0x7fc08806fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:52:38.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:38.742+0000 7fc0a2a00700 1 --2- 172.21.15.107:0/2265597879 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fc08806d7f0 0x7fc08806fca0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc09400b3b0 tx=0x7fc094009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:52:38.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:38.749+0000 7fc09aa00700 1 -- 172.21.15.107:0/2265597879 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc09005aa00 con 0x7fc0a40fc6e0 2024-03-04T13:52:38.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:38.996+0000 7fc0a9600700 1 -- 172.21.15.107:0/2265597879 --> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc0a41158f0 con 0x7fc08806d7f0 2024-03-04T13:52:39.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:38.999+0000 7fc09aa00700 1 -- 172.21.15.107:0/2265597879 <== mgr.34102 v2:172.21.15.107:6800/1223039812 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fc0a41158f0 con 0x7fc08806d7f0 2024-03-04T13:52:39.001 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-03-04T13:52:39.001 INFO:teuthology.orchestra.run.smithi107.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377", 2024-03-04T13:52:39.001 INFO:teuthology.orchestra.run.smithi107.stdout: "in_progress": true, 2024-03-04T13:52:39.001 INFO:teuthology.orchestra.run.smithi107.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-04T13:52:39.001 INFO:teuthology.orchestra.run.smithi107.stdout: "services_complete": [ 2024-03-04T13:52:39.001 INFO:teuthology.orchestra.run.smithi107.stdout: "osd", 2024-03-04T13:52:39.001 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr", 2024-03-04T13:52:39.001 INFO:teuthology.orchestra.run.smithi107.stdout: "mon", 2024-03-04T13:52:39.002 INFO:teuthology.orchestra.run.smithi107.stdout: "crash" 2024-03-04T13:52:39.002 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-04T13:52:39.002 INFO:teuthology.orchestra.run.smithi107.stdout: "progress": "12/21 daemons upgraded", 2024-03-04T13:52:39.002 INFO:teuthology.orchestra.run.smithi107.stdout: "message": "Currently upgrading mds daemons", 2024-03-04T13:52:39.002 INFO:teuthology.orchestra.run.smithi107.stdout: "is_paused": false 2024-03-04T13:52:39.002 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-03-04T13:52:39.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:39.002+0000 7fc0a9600700 1 -- 172.21.15.107:0/2265597879 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fc08806d7f0 msgr2=0x7fc08806fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:52:39.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:39.002+0000 7fc0a9600700 1 --2- 172.21.15.107:0/2265597879 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fc08806d7f0 0x7fc08806fca0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc09400b3b0 tx=0x7fc094009e30 comp rx=0 tx=0).stop 2024-03-04T13:52:39.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:39.002+0000 7fc0a9600700 1 -- 172.21.15.107:0/2265597879 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0a40fc6e0 msgr2=0x7fc0a4116160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:52:39.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:39.002+0000 7fc0a9600700 1 --2- 172.21.15.107:0/2265597879 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0a40fc6e0 0x7fc0a4116160 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc09000b790 tx=0x7fc090009250 comp rx=0 tx=0).stop 2024-03-04T13:52:39.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:39.002+0000 7fc0a9600700 1 -- 172.21.15.107:0/2265597879 shutdown_connections 2024-03-04T13:52:39.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:39.002+0000 7fc0a9600700 1 --2- 172.21.15.107:0/2265597879 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fc08806d7f0 0x7fc08806fca0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:52:39.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:39.002+0000 7fc0a9600700 1 --2- 172.21.15.107:0/2265597879 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0a40fbdd0 0x7fc0a4115c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:52:39.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:39.002+0000 7fc0a9600700 1 --2- 172.21.15.107:0/2265597879 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0a40fc6e0 0x7fc0a4116160 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:52:39.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:39.002+0000 7fc0a9600700 1 -- 172.21.15.107:0/2265597879 >> 172.21.15.107:0/2265597879 conn(0x7fc0a4071d90 msgr2=0x7fc0a40fa920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:52:39.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:39.002+0000 7fc0a9600700 1 -- 172.21.15.107:0/2265597879 shutdown_connections 2024-03-04T13:52:39.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:39.002+0000 7fc0a9600700 1 -- 172.21.15.107:0/2265597879 wait complete. 2024-03-04T13:52:39.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:38 smithi169 ceph-mon[86183]: from='client.34408 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:52:39.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:38 smithi169 ceph-mon[86183]: Upgrade: Updating mds.cephfs.smithi107.riexmm 2024-03-04T13:52:39.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:38 smithi169 ceph-mon[86183]: from='client.34412 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:52:39.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:38 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:52:39.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:38 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi107.riexmm", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-04T13:52:39.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:38 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:52:39.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:38 smithi169 ceph-mon[86183]: Deploying daemon mds.cephfs.smithi107.riexmm on smithi107 2024-03-04T13:52:39.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:38 smithi169 ceph-mon[86183]: from='client.? 172.21.15.107:0/1206232010' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:52:39.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:38 smithi169 ceph-mon[86183]: from='client.? 172.21.15.107:0/3191190878' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-04T13:52:39.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:39.144+0000 7fd120c00700 1 -- 172.21.15.107:0/2689071939 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd11c0fe8f0 msgr2=0x7fd11c107050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:52:39.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:39.144+0000 7fd120c00700 1 --2- 172.21.15.107:0/2689071939 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd11c0fe8f0 0x7fd11c107050 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd108004970 tx=0x7fd108009e30 comp rx=0 tx=0).stop 2024-03-04T13:52:39.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:39.144+0000 7fd120c00700 1 -- 172.21.15.107:0/2689071939 shutdown_connections 2024-03-04T13:52:39.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:39.144+0000 7fd120c00700 1 --2- 172.21.15.107:0/2689071939 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd11c0fe8f0 0x7fd11c107050 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:52:39.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:39.144+0000 7fd120c00700 1 --2- 172.21.15.107:0/2689071939 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd11c0fdfe0 0x7fd11c0fe3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:52:39.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:39.144+0000 7fd120c00700 1 -- 172.21.15.107:0/2689071939 >> 172.21.15.107:0/2689071939 conn(0x7fd11c071d90 msgr2=0x7fd11c072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:52:39.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:39.144+0000 7fd120c00700 1 -- 172.21.15.107:0/2689071939 shutdown_connections 2024-03-04T13:52:39.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:39.144+0000 7fd120c00700 1 -- 172.21.15.107:0/2689071939 wait complete. 2024-03-04T13:52:39.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:39.145+0000 7fd120c00700 1 Processor -- start 2024-03-04T13:52:39.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:39.145+0000 7fd120c00700 1 -- start start 2024-03-04T13:52:39.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:39.146+0000 7fd120c00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd11c0fdfe0 0x7fd11c1139f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:52:39.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:39.146+0000 7fd120c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd11c0fe8f0 0x7fd11c113f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:52:39.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:39.146+0000 7fd120c00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd11c114580 con 0x7fd11c0fe8f0 2024-03-04T13:52:39.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:39.146+0000 7fd120c00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd11c10d8b0 con 0x7fd11c0fdfe0 2024-03-04T13:52:39.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:39.146+0000 7fd119400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd11c0fe8f0 0x7fd11c113f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:52:39.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:39.146+0000 7fd119400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd11c0fe8f0 0x7fd11c113f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51474/0 (socket says 172.21.15.107:51474) 2024-03-04T13:52:39.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:39.146+0000 7fd119400700 1 -- 172.21.15.107:0/1207012934 learned_addr learned my addr 172.21.15.107:0/1207012934 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:52:39.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:39.146+0000 7fd119e00700 1 --2- 172.21.15.107:0/1207012934 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd11c0fdfe0 0x7fd11c1139f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:52:39.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:39.146+0000 7fd119400700 1 -- 172.21.15.107:0/1207012934 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd11c0fdfe0 msgr2=0x7fd11c1139f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:52:39.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:39.146+0000 7fd119400700 1 --2- 172.21.15.107:0/1207012934 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd11c0fdfe0 0x7fd11c1139f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:52:39.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:39.146+0000 7fd119400700 1 -- 172.21.15.107:0/1207012934 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd108005040 con 0x7fd11c0fe8f0 2024-03-04T13:52:39.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:39.147+0000 7fd119400700 1 --2- 172.21.15.107:0/1207012934 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd11c0fe8f0 0x7fd11c113f30 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd108004f50 tx=0x7fd108009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:52:39.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:39.147+0000 7fd112000700 1 -- 172.21.15.107:0/1207012934 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd10800e040 con 0x7fd11c0fe8f0 2024-03-04T13:52:39.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:39.147+0000 7fd112000700 1 -- 172.21.15.107:0/1207012934 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd108004290 con 0x7fd11c0fe8f0 2024-03-04T13:52:39.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:39.147+0000 7fd112000700 1 -- 172.21.15.107:0/1207012934 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd10801c630 con 0x7fd11c0fe8f0 2024-03-04T13:52:39.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:39.147+0000 7fd120c00700 1 -- 172.21.15.107:0/1207012934 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd11c10db30 con 0x7fd11c0fe8f0 2024-03-04T13:52:39.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:39.147+0000 7fd120c00700 1 -- 172.21.15.107:0/1207012934 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd11c10e070 con 0x7fd11c0fe8f0 2024-03-04T13:52:39.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:39.148+0000 7fd120c00700 1 -- 172.21.15.107:0/1207012934 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd11c1046c0 con 0x7fd11c0fe8f0 2024-03-04T13:52:39.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:39.149+0000 7fd112000700 1 -- 172.21.15.107:0/1207012934 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fd10800bc10 con 0x7fd11c0fe8f0 2024-03-04T13:52:39.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:39.150+0000 7fd112000700 1 --2- 172.21.15.107:0/1207012934 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fd10006d7f0 0x7fd10006fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:52:39.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:39.150+0000 7fd112000700 1 -- 172.21.15.107:0/1207012934 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fd10808e230 con 0x7fd11c0fe8f0 2024-03-04T13:52:39.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:39.151+0000 7fd119e00700 1 --2- 172.21.15.107:0/1207012934 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fd10006d7f0 0x7fd10006fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:52:39.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:39.151+0000 7fd119e00700 1 --2- 172.21.15.107:0/1207012934 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fd10006d7f0 0x7fd10006fca0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd10c007fd0 tx=0x7fd10c009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:52:39.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:39.156+0000 7fd112000700 1 -- 172.21.15.107:0/1207012934 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd10805b660 con 0x7fd11c0fe8f0 2024-03-04T13:52:39.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:39.462+0000 7fd120c00700 1 -- 172.21.15.107:0/1207012934 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fd11c10efd0 con 0x7fd11c0fe8f0 2024-03-04T13:52:39.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:39.463+0000 7fd112000700 1 -- 172.21.15.107:0/1207012934 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7fd108013e60 con 0x7fd11c0fe8f0 2024-03-04T13:52:39.465 INFO:teuthology.orchestra.run.smithi107.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-03-04T13:52:39.465 INFO:teuthology.orchestra.run.smithi107.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-04T13:52:39.465 INFO:teuthology.orchestra.run.smithi107.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-04T13:52:39.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:39.466+0000 7fd120c00700 1 -- 172.21.15.107:0/1207012934 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fd10006d7f0 msgr2=0x7fd10006fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:52:39.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:39.466+0000 7fd120c00700 1 --2- 172.21.15.107:0/1207012934 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fd10006d7f0 0x7fd10006fca0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd10c007fd0 tx=0x7fd10c009040 comp rx=0 tx=0).stop 2024-03-04T13:52:39.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:39.466+0000 7fd120c00700 1 -- 172.21.15.107:0/1207012934 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd11c0fe8f0 msgr2=0x7fd11c113f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:52:39.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:39.466+0000 7fd120c00700 1 --2- 172.21.15.107:0/1207012934 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd11c0fe8f0 0x7fd11c113f30 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd108004f50 tx=0x7fd108009e30 comp rx=0 tx=0).stop 2024-03-04T13:52:39.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:39.466+0000 7fd120c00700 1 -- 172.21.15.107:0/1207012934 shutdown_connections 2024-03-04T13:52:39.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:39.466+0000 7fd120c00700 1 --2- 172.21.15.107:0/1207012934 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fd10006d7f0 0x7fd10006fca0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:52:39.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:39.466+0000 7fd120c00700 1 --2- 172.21.15.107:0/1207012934 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd11c0fdfe0 0x7fd11c1139f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:52:39.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:39.466+0000 7fd120c00700 1 --2- 172.21.15.107:0/1207012934 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd11c0fe8f0 0x7fd11c113f30 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:52:39.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:39.466+0000 7fd120c00700 1 -- 172.21.15.107:0/1207012934 >> 172.21.15.107:0/1207012934 conn(0x7fd11c071d90 msgr2=0x7fd11c101840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:52:39.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:39.466+0000 7fd120c00700 1 -- 172.21.15.107:0/1207012934 shutdown_connections 2024-03-04T13:52:39.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:52:39.466+0000 7fd120c00700 1 -- 172.21.15.107:0/1207012934 wait complete. 2024-03-04T13:52:40.095 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:39 smithi107 ceph-mon[112578]: pgmap v192: 65 pgs: 65 active+clean; 255 MiB data, 963 MiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-03-04T13:52:40.095 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:39 smithi107 ceph-mon[112578]: from='client.? 172.21.15.107:0/1207012934' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-04T13:52:40.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:39 smithi169 ceph-mon[86183]: pgmap v192: 65 pgs: 65 active+clean; 255 MiB data, 963 MiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-03-04T13:52:40.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:39 smithi169 ceph-mon[86183]: from='client.? 172.21.15.107:0/1207012934' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-04T13:52:40.844 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:40 smithi107 ceph-mon[112578]: from='client.34424 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:52:40.845 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:40 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:52:40.845 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:40 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:52:40.845 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:40 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:52:41.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:40 smithi169 ceph-mon[86183]: from='client.34424 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:52:41.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:40 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:52:41.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:40 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:52:41.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:40 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:52:42.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:41 smithi107 ceph-mon[112578]: pgmap v194: 65 pgs: 65 active+clean; 255 MiB data, 963 MiB used, 535 GiB / 536 GiB avail; 460 B/s rd, 0 op/s 2024-03-04T13:52:42.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:41 smithi107 ceph-mon[112578]: osdmap e83: 6 total, 6 up, 6 in 2024-03-04T13:52:42.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:41 smithi107 ceph-mon[112578]: fsmap cephfs:1 {0=cephfs.smithi169.jusudl=up:active} 2 up:standby 2024-03-04T13:52:42.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:41 smithi169 ceph-mon[86183]: pgmap v194: 65 pgs: 65 active+clean; 255 MiB data, 963 MiB used, 535 GiB / 536 GiB avail; 460 B/s rd, 0 op/s 2024-03-04T13:52:42.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:41 smithi169 ceph-mon[86183]: osdmap e83: 6 total, 6 up, 6 in 2024-03-04T13:52:42.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:41 smithi169 ceph-mon[86183]: fsmap cephfs:1 {0=cephfs.smithi169.jusudl=up:active} 2 up:standby 2024-03-04T13:52:44.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:43 smithi107 ceph-mon[112578]: pgmap v195: 65 pgs: 65 active+clean; 255 MiB data, 963 MiB used, 535 GiB / 536 GiB avail; 376 B/s rd, 0 op/s 2024-03-04T13:52:44.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:43 smithi169 ceph-mon[86183]: pgmap v195: 65 pgs: 65 active+clean; 255 MiB data, 963 MiB used, 535 GiB / 536 GiB avail; 376 B/s rd, 0 op/s 2024-03-04T13:52:46.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:45 smithi107 ceph-mon[112578]: pgmap v196: 65 pgs: 65 active+clean; 255 MiB data, 963 MiB used, 535 GiB / 536 GiB avail; 307 B/s rd, 0 op/s 2024-03-04T13:52:46.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:45 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:52:46.095 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:45 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:52:46.095 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:45 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:52:46.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:45 smithi169 ceph-mon[86183]: pgmap v196: 65 pgs: 65 active+clean; 255 MiB data, 963 MiB used, 535 GiB / 536 GiB avail; 307 B/s rd, 0 op/s 2024-03-04T13:52:46.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:45 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:52:46.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:45 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:52:46.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:45 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:52:47.020 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:46 smithi107 ceph-mon[112578]: mds.? [v2:172.21.15.107:6826/1144672731,v1:172.21.15.107:6827/1144672731] up:boot 2024-03-04T13:52:47.021 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:46 smithi107 ceph-mon[112578]: fsmap cephfs:1 {0=cephfs.smithi169.jusudl=up:active} 3 up:standby 2024-03-04T13:52:47.021 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:46 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi107.riexmm"}]: dispatch 2024-03-04T13:52:47.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:46 smithi169 ceph-mon[86183]: mds.? [v2:172.21.15.107:6826/1144672731,v1:172.21.15.107:6827/1144672731] up:boot 2024-03-04T13:52:47.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:46 smithi169 ceph-mon[86183]: fsmap cephfs:1 {0=cephfs.smithi169.jusudl=up:active} 3 up:standby 2024-03-04T13:52:47.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:46 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi107.riexmm"}]: dispatch 2024-03-04T13:52:48.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:47 smithi107 ceph-mon[112578]: pgmap v197: 65 pgs: 65 active+clean; 255 MiB data, 963 MiB used, 535 GiB / 536 GiB avail; 307 B/s rd, 0 op/s 2024-03-04T13:52:48.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:47 smithi169 ceph-mon[86183]: pgmap v197: 65 pgs: 65 active+clean; 255 MiB data, 963 MiB used, 535 GiB / 536 GiB avail; 307 B/s rd, 0 op/s 2024-03-04T13:52:49.971 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:49 smithi107 ceph-mon[112578]: pgmap v198: 65 pgs: 65 active+clean; 255 MiB data, 963 MiB used, 535 GiB / 536 GiB avail 2024-03-04T13:52:50.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:49 smithi169 ceph-mon[86183]: pgmap v198: 65 pgs: 65 active+clean; 255 MiB data, 963 MiB used, 535 GiB / 536 GiB avail 2024-03-04T13:52:51.861 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:51 smithi169 ceph-mon[86183]: pgmap v199: 65 pgs: 65 active+clean; 255 MiB data, 963 MiB used, 535 GiB / 536 GiB avail 2024-03-04T13:52:52.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:51 smithi107 ceph-mon[112578]: pgmap v199: 65 pgs: 65 active+clean; 255 MiB data, 963 MiB used, 535 GiB / 536 GiB avail 2024-03-04T13:52:54.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:53 smithi107 ceph-mon[112578]: pgmap v200: 65 pgs: 65 active+clean; 255 MiB data, 963 MiB used, 535 GiB / 536 GiB avail 2024-03-04T13:52:54.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:53 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:52:54.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:53 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:52:54.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:53 smithi169 ceph-mon[86183]: pgmap v200: 65 pgs: 65 active+clean; 255 MiB data, 963 MiB used, 535 GiB / 536 GiB avail 2024-03-04T13:52:54.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:53 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:52:54.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:53 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:52:55.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:54 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:52:55.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:54 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:52:55.095 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:54 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:52:55.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:54 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:52:55.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:54 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:52:55.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:54 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:52:55.951 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:55 smithi107 ceph-mon[112578]: pgmap v201: 65 pgs: 65 active+clean; 255 MiB data, 963 MiB used, 535 GiB / 536 GiB avail 2024-03-04T13:52:56.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:55 smithi169 ceph-mon[86183]: pgmap v201: 65 pgs: 65 active+clean; 255 MiB data, 963 MiB used, 535 GiB / 536 GiB avail 2024-03-04T13:52:58.095 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:57 smithi107 ceph-mon[112578]: pgmap v202: 65 pgs: 65 active+clean; 255 MiB data, 963 MiB used, 535 GiB / 536 GiB avail 2024-03-04T13:52:58.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:57 smithi169 ceph-mon[86183]: pgmap v202: 65 pgs: 65 active+clean; 255 MiB data, 963 MiB used, 535 GiB / 536 GiB avail 2024-03-04T13:53:00.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:52:59 smithi107 ceph-mon[112578]: pgmap v203: 65 pgs: 65 active+clean; 255 MiB data, 963 MiB used, 535 GiB / 536 GiB avail 2024-03-04T13:53:00.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:52:59 smithi169 ceph-mon[86183]: pgmap v203: 65 pgs: 65 active+clean; 255 MiB data, 963 MiB used, 535 GiB / 536 GiB avail 2024-03-04T13:53:01.701 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:01 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:53:01.702 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:01 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:53:01.702 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:01 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:53:01.702 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:01 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:53:01.702 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:01 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:53:01.702 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:01 smithi107 ceph-mon[112578]: pgmap v204: 65 pgs: 65 active+clean; 255 MiB data, 963 MiB used, 535 GiB / 536 GiB avail 2024-03-04T13:53:01.702 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:01 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:53:01.702 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:01 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:53:01.702 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:01 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:53:01.702 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:01 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:53:01.702 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:01 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:53:01.862 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:01 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:53:01.862 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:01 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:53:01.863 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:01 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:53:01.863 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:01 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:53:01.863 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:01 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:53:01.863 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:01 smithi169 ceph-mon[86183]: pgmap v204: 65 pgs: 65 active+clean; 255 MiB data, 963 MiB used, 535 GiB / 536 GiB avail 2024-03-04T13:53:01.863 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:01 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:53:01.863 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:01 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:53:01.863 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:01 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:53:01.863 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:01 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:53:01.863 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:01 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:53:04.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:03 smithi107 ceph-mon[112578]: pgmap v205: 65 pgs: 65 active+clean; 255 MiB data, 963 MiB used, 535 GiB / 536 GiB avail 2024-03-04T13:53:04.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:03 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:53:04.095 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:03 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi107.thycho", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-04T13:53:04.095 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:03 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:53:04.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:03 smithi169 ceph-mon[86183]: pgmap v205: 65 pgs: 65 active+clean; 255 MiB data, 963 MiB used, 535 GiB / 536 GiB avail 2024-03-04T13:53:04.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:03 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:53:04.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:03 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi107.thycho", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-04T13:53:04.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:03 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:53:05.095 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:04 smithi107 ceph-mon[112578]: Upgrade: Updating mds.cephfs.smithi107.thycho 2024-03-04T13:53:05.095 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:04 smithi107 ceph-mon[112578]: Deploying daemon mds.cephfs.smithi107.thycho on smithi107 2024-03-04T13:53:05.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:04 smithi169 ceph-mon[86183]: Upgrade: Updating mds.cephfs.smithi107.thycho 2024-03-04T13:53:05.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:04 smithi169 ceph-mon[86183]: Deploying daemon mds.cephfs.smithi107.thycho on smithi107 2024-03-04T13:53:06.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:05 smithi107 ceph-mon[112578]: pgmap v206: 65 pgs: 65 active+clean; 255 MiB data, 963 MiB used, 535 GiB / 536 GiB avail 2024-03-04T13:53:06.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:05 smithi169 ceph-mon[86183]: pgmap v206: 65 pgs: 65 active+clean; 255 MiB data, 963 MiB used, 535 GiB / 536 GiB avail 2024-03-04T13:53:08.095 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:07 smithi107 ceph-mon[112578]: pgmap v207: 65 pgs: 65 active+clean; 255 MiB data, 963 MiB used, 535 GiB / 536 GiB avail 2024-03-04T13:53:08.095 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:07 smithi107 ceph-mon[112578]: osdmap e84: 6 total, 6 up, 6 in 2024-03-04T13:53:08.095 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:07 smithi107 ceph-mon[112578]: fsmap cephfs:1 {0=cephfs.smithi169.jusudl=up:active} 2 up:standby 2024-03-04T13:53:08.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:07 smithi169 ceph-mon[86183]: pgmap v207: 65 pgs: 65 active+clean; 255 MiB data, 963 MiB used, 535 GiB / 536 GiB avail 2024-03-04T13:53:08.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:07 smithi169 ceph-mon[86183]: osdmap e84: 6 total, 6 up, 6 in 2024-03-04T13:53:08.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:07 smithi169 ceph-mon[86183]: fsmap cephfs:1 {0=cephfs.smithi169.jusudl=up:active} 2 up:standby 2024-03-04T13:53:09.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:09.634+0000 7f97b9a00700 1 -- 172.21.15.107:0/785610850 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97b4070870 msgr2=0x7f97b4070ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:53:09.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:09.634+0000 7f97b9a00700 1 --2- 172.21.15.107:0/785610850 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97b4070870 0x7f97b4070ce0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f97a0004970 tx=0x7f97a0009e30 comp rx=0 tx=0).stop 2024-03-04T13:53:09.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:09.634+0000 7f97b9a00700 1 -- 172.21.15.107:0/785610850 shutdown_connections 2024-03-04T13:53:09.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:09.634+0000 7f97b9a00700 1 --2- 172.21.15.107:0/785610850 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97b4070870 0x7f97b4070ce0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:53:09.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:09.634+0000 7f97b9a00700 1 --2- 172.21.15.107:0/785610850 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97b4071f60 0x7f97b4070330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:53:09.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:09.634+0000 7f97b9a00700 1 -- 172.21.15.107:0/785610850 >> 172.21.15.107:0/785610850 conn(0x7f97b40f8ae0 msgr2=0x7f97b40faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:53:09.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:09.634+0000 7f97b9a00700 1 -- 172.21.15.107:0/785610850 shutdown_connections 2024-03-04T13:53:09.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:09.634+0000 7f97b9a00700 1 -- 172.21.15.107:0/785610850 wait complete. 2024-03-04T13:53:09.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:09.635+0000 7f97b9a00700 1 Processor -- start 2024-03-04T13:53:09.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:09.635+0000 7f97b9a00700 1 -- start start 2024-03-04T13:53:09.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:09.635+0000 7f97b9a00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97b4070870 0x7f97b411a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:53:09.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:09.635+0000 7f97b9a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97b4071f60 0x7f97b411a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:53:09.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:09.635+0000 7f97b9a00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97b411aba0 con 0x7f97b4071f60 2024-03-04T13:53:09.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:09.635+0000 7f97b9a00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97b4113eb0 con 0x7f97b4070870 2024-03-04T13:53:09.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:09.636+0000 7f97b2000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97b4071f60 0x7f97b411a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:53:09.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:09.636+0000 7f97b2a00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97b4070870 0x7f97b411a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:53:09.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:09.636+0000 7f97b2000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97b4071f60 0x7f97b411a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39390/0 (socket says 172.21.15.107:39390) 2024-03-04T13:53:09.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:09.636+0000 7f97b2000700 1 -- 172.21.15.107:0/1982027075 learned_addr learned my addr 172.21.15.107:0/1982027075 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:53:09.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:09.637+0000 7f97b2000700 1 -- 172.21.15.107:0/1982027075 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97b4070870 msgr2=0x7f97b411a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:53:09.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:09.637+0000 7f97b2000700 1 --2- 172.21.15.107:0/1982027075 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97b4070870 0x7f97b411a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:53:09.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:09.637+0000 7f97b2000700 1 -- 172.21.15.107:0/1982027075 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97a0005040 con 0x7f97b4071f60 2024-03-04T13:53:09.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:09.637+0000 7f97b2000700 1 --2- 172.21.15.107:0/1982027075 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97b4071f60 0x7f97b411a550 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f97a0000c00 tx=0x7f97a0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:53:09.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:09.637+0000 7f97aaa00700 1 -- 172.21.15.107:0/1982027075 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f97a000e040 con 0x7f97b4071f60 2024-03-04T13:53:09.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:09.637+0000 7f97aaa00700 1 -- 172.21.15.107:0/1982027075 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f97a000bc10 con 0x7f97b4071f60 2024-03-04T13:53:09.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:09.637+0000 7f97aaa00700 1 -- 172.21.15.107:0/1982027075 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f97a001c630 con 0x7f97b4071f60 2024-03-04T13:53:09.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:09.638+0000 7f97b9a00700 1 -- 172.21.15.107:0/1982027075 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97b4114130 con 0x7f97b4071f60 2024-03-04T13:53:09.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:09.638+0000 7f97b9a00700 1 -- 172.21.15.107:0/1982027075 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97b4114670 con 0x7f97b4071f60 2024-03-04T13:53:09.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:09.639+0000 7f97b9a00700 1 -- 172.21.15.107:0/1982027075 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97b40fe400 con 0x7f97b4071f60 2024-03-04T13:53:09.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:09.641+0000 7f97aaa00700 1 -- 172.21.15.107:0/1982027075 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f97a000bd80 con 0x7f97b4071f60 2024-03-04T13:53:09.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:09.642+0000 7f97aaa00700 1 --2- 172.21.15.107:0/1982027075 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f979806d7e0 0x7f979806fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:53:09.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:09.642+0000 7f97aaa00700 1 -- 172.21.15.107:0/1982027075 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f97a008e1b0 con 0x7f97b4071f60 2024-03-04T13:53:09.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:09.643+0000 7f97b2a00700 1 --2- 172.21.15.107:0/1982027075 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f979806d7e0 0x7f979806fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:53:09.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:09.643+0000 7f97b2a00700 1 --2- 172.21.15.107:0/1982027075 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f979806d7e0 0x7f979806fc90 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f97a4007fd0 tx=0x7f97a4009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:53:09.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:09.652+0000 7f97aaa00700 1 -- 172.21.15.107:0/1982027075 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f97a005b5e0 con 0x7f97b4071f60 2024-03-04T13:53:09.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:09.902+0000 7f97b9a00700 1 -- 172.21.15.107:0/1982027075 --> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f97b404bed0 con 0x7f979806d7e0 2024-03-04T13:53:09.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:09.904+0000 7f97aaa00700 1 -- 172.21.15.107:0/1982027075 <== mgr.34102 v2:172.21.15.107:6800/1223039812 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f97b404bed0 con 0x7f979806d7e0 2024-03-04T13:53:09.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:09.907+0000 7f97b9a00700 1 -- 172.21.15.107:0/1982027075 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f979806d7e0 msgr2=0x7f979806fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:53:09.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:09.907+0000 7f97b9a00700 1 --2- 172.21.15.107:0/1982027075 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f979806d7e0 0x7f979806fc90 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f97a4007fd0 tx=0x7f97a4009040 comp rx=0 tx=0).stop 2024-03-04T13:53:09.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:09.907+0000 7f97b9a00700 1 -- 172.21.15.107:0/1982027075 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97b4071f60 msgr2=0x7f97b411a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:53:09.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:09.907+0000 7f97b9a00700 1 --2- 172.21.15.107:0/1982027075 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97b4071f60 0x7f97b411a550 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f97a0000c00 tx=0x7f97a0009e30 comp rx=0 tx=0).stop 2024-03-04T13:53:09.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:09.907+0000 7f97b9a00700 1 -- 172.21.15.107:0/1982027075 shutdown_connections 2024-03-04T13:53:09.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:09.907+0000 7f97b9a00700 1 --2- 172.21.15.107:0/1982027075 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f979806d7e0 0x7f979806fc90 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:53:09.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:09.907+0000 7f97b9a00700 1 --2- 172.21.15.107:0/1982027075 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f97b4070870 0x7f97b411a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:53:09.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:09.907+0000 7f97b9a00700 1 --2- 172.21.15.107:0/1982027075 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97b4071f60 0x7f97b411a550 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:53:09.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:09.907+0000 7f97b9a00700 1 -- 172.21.15.107:0/1982027075 >> 172.21.15.107:0/1982027075 conn(0x7f97b40f8ae0 msgr2=0x7f97b40fa750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:53:09.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:09.907+0000 7f97b9a00700 1 -- 172.21.15.107:0/1982027075 shutdown_connections 2024-03-04T13:53:09.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:09.908+0000 7f97b9a00700 1 -- 172.21.15.107:0/1982027075 wait complete. 2024-03-04T13:53:09.926 INFO:teuthology.orchestra.run.smithi107.stdout:true 2024-03-04T13:53:10.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.061+0000 7fa9d8200700 1 -- 172.21.15.107:0/273379986 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9d0101e40 msgr2=0x7fa9d0102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:53:10.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.062+0000 7fa9d8200700 1 --2- 172.21.15.107:0/273379986 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9d0101e40 0x7fa9d0102210 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa9c4004940 tx=0x7fa9c4009e30 comp rx=0 tx=0).stop 2024-03-04T13:53:10.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.062+0000 7fa9d8200700 1 -- 172.21.15.107:0/273379986 shutdown_connections 2024-03-04T13:53:10.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.062+0000 7fa9d8200700 1 --2- 172.21.15.107:0/273379986 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa9d01027e0 0x7fa9d00fbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:53:10.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.062+0000 7fa9d8200700 1 --2- 172.21.15.107:0/273379986 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9d0101e40 0x7fa9d0102210 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:53:10.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.062+0000 7fa9d8200700 1 -- 172.21.15.107:0/273379986 >> 172.21.15.107:0/273379986 conn(0x7fa9d0071d90 msgr2=0x7fa9d0072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:53:10.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.062+0000 7fa9d8200700 1 -- 172.21.15.107:0/273379986 shutdown_connections 2024-03-04T13:53:10.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.062+0000 7fa9d8200700 1 -- 172.21.15.107:0/273379986 wait complete. 2024-03-04T13:53:10.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.063+0000 7fa9d8200700 1 Processor -- start 2024-03-04T13:53:10.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.063+0000 7fa9d8200700 1 -- start start 2024-03-04T13:53:10.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.063+0000 7fa9d8200700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa9d0101e40 0x7fa9d0115ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:53:10.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.063+0000 7fa9d8200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9d01027e0 0x7fa9d0116220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:53:10.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.063+0000 7fa9d8200700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d0116870 con 0x7fa9d01027e0 2024-03-04T13:53:10.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.063+0000 7fa9d8200700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d010fb80 con 0x7fa9d0101e40 2024-03-04T13:53:10.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.064+0000 7fa9d4e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9d01027e0 0x7fa9d0116220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:53:10.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.064+0000 7fa9d4e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9d01027e0 0x7fa9d0116220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39414/0 (socket says 172.21.15.107:39414) 2024-03-04T13:53:10.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.064+0000 7fa9d4e00700 1 -- 172.21.15.107:0/3490041167 learned_addr learned my addr 172.21.15.107:0/3490041167 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:53:10.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.064+0000 7fa9d5800700 1 --2- 172.21.15.107:0/3490041167 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa9d0101e40 0x7fa9d0115ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:53:10.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.064+0000 7fa9d4e00700 1 -- 172.21.15.107:0/3490041167 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa9d0101e40 msgr2=0x7fa9d0115ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:53:10.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.064+0000 7fa9d4e00700 1 --2- 172.21.15.107:0/3490041167 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa9d0101e40 0x7fa9d0115ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:53:10.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.064+0000 7fa9d4e00700 1 -- 172.21.15.107:0/3490041167 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9c4005040 con 0x7fa9d01027e0 2024-03-04T13:53:10.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.065+0000 7fa9d5800700 1 --2- 172.21.15.107:0/3490041167 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa9d0101e40 0x7fa9d0115ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T13:53:10.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.065+0000 7fa9d4e00700 1 --2- 172.21.15.107:0/3490041167 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9d01027e0 0x7fa9d0116220 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc002a50 tx=0x7fa9bc004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:53:10.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.065+0000 7fa9cd600700 1 -- 172.21.15.107:0/3490041167 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa9bc007cc0 con 0x7fa9d01027e0 2024-03-04T13:53:10.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.065+0000 7fa9cd600700 1 -- 172.21.15.107:0/3490041167 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa9bc00f040 con 0x7fa9d01027e0 2024-03-04T13:53:10.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.065+0000 7fa9d8200700 1 -- 172.21.15.107:0/3490041167 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9d010fe60 con 0x7fa9d01027e0 2024-03-04T13:53:10.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.065+0000 7fa9cd600700 1 -- 172.21.15.107:0/3490041167 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa9bc00daf0 con 0x7fa9d01027e0 2024-03-04T13:53:10.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.066+0000 7fa9d8200700 1 -- 172.21.15.107:0/3490041167 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9d01103a0 con 0x7fa9d01027e0 2024-03-04T13:53:10.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.067+0000 7fa9d8200700 1 -- 172.21.15.107:0/3490041167 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9d00738b0 con 0x7fa9d01027e0 2024-03-04T13:53:10.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.068+0000 7fa9cd600700 1 -- 172.21.15.107:0/3490041167 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fa9bc009810 con 0x7fa9d01027e0 2024-03-04T13:53:10.070 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:09 smithi107 ceph-mon[112578]: pgmap v209: 65 pgs: 65 active+clean; 255 MiB data, 963 MiB used, 535 GiB / 536 GiB avail 2024-03-04T13:53:10.070 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:09 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:53:10.070 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:09 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:53:10.070 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:09 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:53:10.070 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:09 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:53:10.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.070+0000 7fa9cd600700 1 --2- 172.21.15.107:0/3490041167 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fa9b806d7e0 0x7fa9b806fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:53:10.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.070+0000 7fa9cd600700 1 -- 172.21.15.107:0/3490041167 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fa9bc08cb40 con 0x7fa9d01027e0 2024-03-04T13:53:10.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.071+0000 7fa9d5800700 1 --2- 172.21.15.107:0/3490041167 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fa9b806d7e0 0x7fa9b806fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:53:10.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.071+0000 7fa9d5800700 1 --2- 172.21.15.107:0/3490041167 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fa9b806d7e0 0x7fa9b806fc90 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa9c4004560 tx=0x7fa9c4009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:53:10.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.077+0000 7fa9cd600700 1 -- 172.21.15.107:0/3490041167 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa9bc059f90 con 0x7fa9d01027e0 2024-03-04T13:53:10.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:09 smithi169 ceph-mon[86183]: pgmap v209: 65 pgs: 65 active+clean; 255 MiB data, 963 MiB used, 535 GiB / 536 GiB avail 2024-03-04T13:53:10.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:09 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:53:10.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:09 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:53:10.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:09 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:53:10.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:09 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:53:10.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.322+0000 7fa9d8200700 1 -- 172.21.15.107:0/3490041167 --> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa9d01112c0 con 0x7fa9b806d7e0 2024-03-04T13:53:10.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.325+0000 7fa9cd600700 1 -- 172.21.15.107:0/3490041167 <== mgr.34102 v2:172.21.15.107:6800/1223039812 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fa9d01112c0 con 0x7fa9b806d7e0 2024-03-04T13:53:10.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.328+0000 7fa9d8200700 1 -- 172.21.15.107:0/3490041167 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fa9b806d7e0 msgr2=0x7fa9b806fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:53:10.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.328+0000 7fa9d8200700 1 --2- 172.21.15.107:0/3490041167 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fa9b806d7e0 0x7fa9b806fc90 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa9c4004560 tx=0x7fa9c4009e30 comp rx=0 tx=0).stop 2024-03-04T13:53:10.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.328+0000 7fa9d8200700 1 -- 172.21.15.107:0/3490041167 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9d01027e0 msgr2=0x7fa9d0116220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:53:10.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.328+0000 7fa9d8200700 1 --2- 172.21.15.107:0/3490041167 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9d01027e0 0x7fa9d0116220 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc002a50 tx=0x7fa9bc004600 comp rx=0 tx=0).stop 2024-03-04T13:53:10.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.328+0000 7fa9d8200700 1 -- 172.21.15.107:0/3490041167 shutdown_connections 2024-03-04T13:53:10.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.328+0000 7fa9d8200700 1 --2- 172.21.15.107:0/3490041167 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fa9b806d7e0 0x7fa9b806fc90 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:53:10.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.328+0000 7fa9d8200700 1 --2- 172.21.15.107:0/3490041167 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa9d0101e40 0x7fa9d0115ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:53:10.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.328+0000 7fa9d8200700 1 --2- 172.21.15.107:0/3490041167 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9d01027e0 0x7fa9d0116220 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:53:10.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.328+0000 7fa9d8200700 1 -- 172.21.15.107:0/3490041167 >> 172.21.15.107:0/3490041167 conn(0x7fa9d0071d90 msgr2=0x7fa9d0065740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:53:10.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.328+0000 7fa9d8200700 1 -- 172.21.15.107:0/3490041167 shutdown_connections 2024-03-04T13:53:10.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.328+0000 7fa9d8200700 1 -- 172.21.15.107:0/3490041167 wait complete. 2024-03-04T13:53:10.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.474+0000 7fa55aa00700 1 -- 172.21.15.107:0/472110500 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa554070870 msgr2=0x7fa554070ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:53:10.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.474+0000 7fa55aa00700 1 --2- 172.21.15.107:0/472110500 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa554070870 0x7fa554070ce0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa540004970 tx=0x7fa540009e30 comp rx=0 tx=0).stop 2024-03-04T13:53:10.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.474+0000 7fa55aa00700 1 -- 172.21.15.107:0/472110500 shutdown_connections 2024-03-04T13:53:10.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.474+0000 7fa55aa00700 1 --2- 172.21.15.107:0/472110500 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa554070870 0x7fa554070ce0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:53:10.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.474+0000 7fa55aa00700 1 --2- 172.21.15.107:0/472110500 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa554071f60 0x7fa554070330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:53:10.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.474+0000 7fa55aa00700 1 -- 172.21.15.107:0/472110500 >> 172.21.15.107:0/472110500 conn(0x7fa5540f8ae0 msgr2=0x7fa5540faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:53:10.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.474+0000 7fa55aa00700 1 -- 172.21.15.107:0/472110500 shutdown_connections 2024-03-04T13:53:10.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.474+0000 7fa55aa00700 1 -- 172.21.15.107:0/472110500 wait complete. 2024-03-04T13:53:10.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.475+0000 7fa55aa00700 1 Processor -- start 2024-03-04T13:53:10.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.475+0000 7fa55aa00700 1 -- start start 2024-03-04T13:53:10.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.475+0000 7fa55aa00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa554070870 0x7fa55411a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:53:10.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.475+0000 7fa55aa00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa554071f60 0x7fa55411a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:53:10.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.475+0000 7fa55aa00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa55411aba0 con 0x7fa554070870 2024-03-04T13:53:10.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.475+0000 7fa55aa00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa554113eb0 con 0x7fa554071f60 2024-03-04T13:53:10.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.475+0000 7fa553e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa554070870 0x7fa55411a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:53:10.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.476+0000 7fa553e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa554070870 0x7fa55411a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39430/0 (socket says 172.21.15.107:39430) 2024-03-04T13:53:10.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.476+0000 7fa553e00700 1 -- 172.21.15.107:0/374610763 learned_addr learned my addr 172.21.15.107:0/374610763 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:53:10.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.476+0000 7fa553400700 1 --2- 172.21.15.107:0/374610763 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa554071f60 0x7fa55411a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:53:10.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.476+0000 7fa553e00700 1 -- 172.21.15.107:0/374610763 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa554071f60 msgr2=0x7fa55411a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:53:10.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.476+0000 7fa553e00700 1 --2- 172.21.15.107:0/374610763 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa554071f60 0x7fa55411a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:53:10.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.476+0000 7fa553e00700 1 -- 172.21.15.107:0/374610763 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa540005040 con 0x7fa554070870 2024-03-04T13:53:10.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.476+0000 7fa553400700 1 --2- 172.21.15.107:0/374610763 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa554071f60 0x7fa55411a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T13:53:10.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.476+0000 7fa553e00700 1 --2- 172.21.15.107:0/374610763 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa554070870 0x7fa55411a010 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa54400c8c0 tx=0x7fa5440044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:53:10.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.477+0000 7fa54be00700 1 -- 172.21.15.107:0/374610763 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa54400a980 con 0x7fa554070870 2024-03-04T13:53:10.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.477+0000 7fa54be00700 1 -- 172.21.15.107:0/374610763 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa54400ce60 con 0x7fa554070870 2024-03-04T13:53:10.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.477+0000 7fa55aa00700 1 -- 172.21.15.107:0/374610763 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa554114190 con 0x7fa554070870 2024-03-04T13:53:10.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.477+0000 7fa54be00700 1 -- 172.21.15.107:0/374610763 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa5440115e0 con 0x7fa554070870 2024-03-04T13:53:10.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.477+0000 7fa55aa00700 1 -- 172.21.15.107:0/374610763 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5541146d0 con 0x7fa554070870 2024-03-04T13:53:10.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.478+0000 7fa55aa00700 1 -- 172.21.15.107:0/374610763 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5540fe400 con 0x7fa554070870 2024-03-04T13:53:10.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.479+0000 7fa54be00700 1 -- 172.21.15.107:0/374610763 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fa544012400 con 0x7fa554070870 2024-03-04T13:53:10.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.480+0000 7fa54be00700 1 --2- 172.21.15.107:0/374610763 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fa53806d7f0 0x7fa53806fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:53:10.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.481+0000 7fa54be00700 1 -- 172.21.15.107:0/374610763 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fa54408cff0 con 0x7fa554070870 2024-03-04T13:53:10.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.481+0000 7fa553400700 1 --2- 172.21.15.107:0/374610763 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fa53806d7f0 0x7fa53806fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:53:10.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.482+0000 7fa553400700 1 --2- 172.21.15.107:0/374610763 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fa53806d7f0 0x7fa53806fca0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa540000c00 tx=0x7fa540009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:53:10.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.485+0000 7fa54be00700 1 -- 172.21.15.107:0/374610763 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa5440c0d20 con 0x7fa554070870 2024-03-04T13:53:10.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.734+0000 7fa55aa00700 1 -- 172.21.15.107:0/374610763 --> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fa55404bf40 con 0x7fa53806d7f0 2024-03-04T13:53:10.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.746+0000 7fa54be00700 1 -- 172.21.15.107:0/374610763 <== mgr.34102 v2:172.21.15.107:6800/1223039812 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fa55404bf40 con 0x7fa53806d7f0 2024-03-04T13:53:10.748 INFO:teuthology.orchestra.run.smithi107.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-04T13:53:10.748 INFO:teuthology.orchestra.run.smithi107.stdout:alertmanager.smithi107 smithi107 *:9093,9094 running (7m) 17s ago 21m 14.4M - 0.25.0 c8568f914cd2 e5ed7be32538 2024-03-04T13:53:10.748 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi107 smithi107 running (4m) 17s ago 21m 6660k - 19.0.0-1845-g717ce3cc d71b839abdf8 0558022b0d84 2024-03-04T13:53:10.748 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi169 smithi169 running (4m) 45s ago 20m 6665k - 19.0.0-1845-g717ce3cc d71b839abdf8 221ca53aa0d3 2024-03-04T13:53:10.748 INFO:teuthology.orchestra.run.smithi107.stdout:grafana.smithi107 smithi107 *:3000 running (7m) 17s ago 21m 75.0M - 9.4.12 a0ea5e3c46d9 c5e4d33fb599 2024-03-04T13:53:10.748 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.riexmm smithi107 running (25s) 17s ago 14m 10.8M - 19.0.0-1845-g717ce3cc d71b839abdf8 8582388ad90a 2024-03-04T13:53:10.748 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.thycho smithi107 starting - - - - 2024-03-04T13:53:10.748 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi169.bxitzk smithi169 running (14m) 45s ago 14m 15.1M - 17.2.7-545-g42608bb1 9eec3f22859d 7d4115228c7c 2024-03-04T13:53:10.748 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi169.jusudl smithi169 running (14m) 45s ago 14m 145M - 17.2.7-545-g42608bb1 9eec3f22859d 2d2f2cc452e8 2024-03-04T13:53:10.748 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi107.wpzgwl smithi107 *:8443,9283,8765 running (10m) 17s ago 23m 508M - 19.0.0-1845-g717ce3cc d71b839abdf8 1c5fdee45eac 2024-03-04T13:53:10.748 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi169.eihczr smithi169 *:8443,9283,8765 running (8m) 45s ago 20m 451M - 19.0.0-1845-g717ce3cc d71b839abdf8 b5cac2b7d22a 2024-03-04T13:53:10.749 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi107 smithi107 running (5m) 17s ago 23m 50.8M 2048M 19.0.0-1845-g717ce3cc d71b839abdf8 32f07144274f 2024-03-04T13:53:10.749 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi169 smithi169 running (5m) 45s ago 20m 37.6M 2048M 19.0.0-1845-g717ce3cc d71b839abdf8 e3527449381a 2024-03-04T13:53:10.749 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi107 smithi107 *:9100 running (8m) 17s ago 21m 7994k - 1.5.0 0da6a335fe13 8a8d3d4f724f 2024-03-04T13:53:10.749 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi169 smithi169 *:9100 running (8m) 45s ago 20m 7763k - 1.5.0 0da6a335fe13 f240f5a975bc 2024-03-04T13:53:10.749 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0 smithi107 running (3m) 17s ago 19m 84.7M 1558M 19.0.0-1845-g717ce3cc d71b839abdf8 5801a5540dc6 2024-03-04T13:53:10.749 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1 smithi107 running (3m) 17s ago 18m 93.9M 1558M 19.0.0-1845-g717ce3cc d71b839abdf8 b614af658854 2024-03-04T13:53:10.749 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2 smithi107 running (2m) 17s ago 18m 84.8M 1558M 19.0.0-1845-g717ce3cc d71b839abdf8 6e4da4f6f146 2024-03-04T13:53:10.749 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3 smithi169 running (2m) 45s ago 17m 94.2M 2582M 19.0.0-1845-g717ce3cc d71b839abdf8 d31baf764d7a 2024-03-04T13:53:10.749 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4 smithi169 running (93s) 45s ago 16m 84.1M 2582M 19.0.0-1845-g717ce3cc d71b839abdf8 a70bacd97639 2024-03-04T13:53:10.749 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5 smithi169 running (50s) 45s ago 16m 11.2M 2582M 19.0.0-1845-g717ce3cc d71b839abdf8 5f6e00f7993d 2024-03-04T13:53:10.749 INFO:teuthology.orchestra.run.smithi107.stdout:prometheus.smithi107 smithi107 *:9095 running (7m) 17s ago 20m 54.6M - 2.43.0 a07b618ecd1d 1eeb1ad5cb75 2024-03-04T13:53:10.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.749+0000 7fa55aa00700 1 -- 172.21.15.107:0/374610763 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fa53806d7f0 msgr2=0x7fa53806fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:53:10.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.749+0000 7fa55aa00700 1 --2- 172.21.15.107:0/374610763 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fa53806d7f0 0x7fa53806fca0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa540000c00 tx=0x7fa540009e30 comp rx=0 tx=0).stop 2024-03-04T13:53:10.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.749+0000 7fa55aa00700 1 -- 172.21.15.107:0/374610763 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa554070870 msgr2=0x7fa55411a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:53:10.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.749+0000 7fa55aa00700 1 --2- 172.21.15.107:0/374610763 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa554070870 0x7fa55411a010 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa54400c8c0 tx=0x7fa5440044c0 comp rx=0 tx=0).stop 2024-03-04T13:53:10.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.749+0000 7fa55aa00700 1 -- 172.21.15.107:0/374610763 shutdown_connections 2024-03-04T13:53:10.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.749+0000 7fa55aa00700 1 --2- 172.21.15.107:0/374610763 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fa53806d7f0 0x7fa53806fca0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:53:10.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.749+0000 7fa55aa00700 1 --2- 172.21.15.107:0/374610763 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa554070870 0x7fa55411a010 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:53:10.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.749+0000 7fa55aa00700 1 --2- 172.21.15.107:0/374610763 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa554071f60 0x7fa55411a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:53:10.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.749+0000 7fa55aa00700 1 -- 172.21.15.107:0/374610763 >> 172.21.15.107:0/374610763 conn(0x7fa5540f8ae0 msgr2=0x7fa5540fa750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:53:10.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.749+0000 7fa55aa00700 1 -- 172.21.15.107:0/374610763 shutdown_connections 2024-03-04T13:53:10.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.749+0000 7fa55aa00700 1 -- 172.21.15.107:0/374610763 wait complete. 2024-03-04T13:53:10.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.891+0000 7f4d23200700 1 -- 172.21.15.107:0/1941565278 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d24102d40 msgr2=0x7f4d24103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:53:10.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.891+0000 7f4d23200700 1 --2- 172.21.15.107:0/1941565278 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d24102d40 0x7f4d24103110 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4d14004940 tx=0x7f4d14009e30 comp rx=0 tx=0).stop 2024-03-04T13:53:10.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.891+0000 7f4d23200700 1 -- 172.21.15.107:0/1941565278 shutdown_connections 2024-03-04T13:53:10.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.891+0000 7f4d23200700 1 --2- 172.21.15.107:0/1941565278 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d241036e0 0x7f4d240fcd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:53:10.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.891+0000 7f4d23200700 1 --2- 172.21.15.107:0/1941565278 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d24102d40 0x7f4d24103110 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:53:10.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.891+0000 7f4d23200700 1 -- 172.21.15.107:0/1941565278 >> 172.21.15.107:0/1941565278 conn(0x7f4d240f88b0 msgr2=0x7f4d240facc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:53:10.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.891+0000 7f4d23200700 1 -- 172.21.15.107:0/1941565278 shutdown_connections 2024-03-04T13:53:10.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.891+0000 7f4d23200700 1 -- 172.21.15.107:0/1941565278 wait complete. 2024-03-04T13:53:10.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.892+0000 7f4d23200700 1 Processor -- start 2024-03-04T13:53:10.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.893+0000 7f4d23200700 1 -- start start 2024-03-04T13:53:10.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.893+0000 7f4d23200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d24102d40 0x7f4d24070290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:53:10.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.893+0000 7f4d23200700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d241036e0 0x7f4d240707d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:53:10.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.893+0000 7f4d23200700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d24070dd0 con 0x7f4d24102d40 2024-03-04T13:53:10.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.893+0000 7f4d23200700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d24070f40 con 0x7f4d241036e0 2024-03-04T13:53:10.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.893+0000 7f4d21e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d24102d40 0x7f4d24070290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:53:10.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.894+0000 7f4d21e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d24102d40 0x7f4d24070290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39444/0 (socket says 172.21.15.107:39444) 2024-03-04T13:53:10.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.894+0000 7f4d21e00700 1 -- 172.21.15.107:0/2054516964 learned_addr learned my addr 172.21.15.107:0/2054516964 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:53:10.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.894+0000 7f4d21400700 1 --2- 172.21.15.107:0/2054516964 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d241036e0 0x7f4d240707d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:53:10.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.894+0000 7f4d21e00700 1 -- 172.21.15.107:0/2054516964 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d241036e0 msgr2=0x7f4d240707d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:53:10.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.894+0000 7f4d21e00700 1 --2- 172.21.15.107:0/2054516964 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d241036e0 0x7f4d240707d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:53:10.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.894+0000 7f4d21e00700 1 -- 172.21.15.107:0/2054516964 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d14005040 con 0x7f4d24102d40 2024-03-04T13:53:10.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.894+0000 7f4d21e00700 1 --2- 172.21.15.107:0/2054516964 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d24102d40 0x7f4d24070290 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4d14009150 tx=0x7f4d14009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:53:10.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.894+0000 7f4d1a000700 1 -- 172.21.15.107:0/2054516964 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d1400e040 con 0x7f4d24102d40 2024-03-04T13:53:10.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.895+0000 7f4d1a000700 1 -- 172.21.15.107:0/2054516964 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4d1400bbf0 con 0x7f4d24102d40 2024-03-04T13:53:10.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.895+0000 7f4d1a000700 1 -- 172.21.15.107:0/2054516964 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d1401c630 con 0x7f4d24102d40 2024-03-04T13:53:10.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.895+0000 7f4d23200700 1 -- 172.21.15.107:0/2054516964 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d24119a50 con 0x7f4d24102d40 2024-03-04T13:53:10.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.895+0000 7f4d23200700 1 -- 172.21.15.107:0/2054516964 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d24119f30 con 0x7f4d24102d40 2024-03-04T13:53:10.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.896+0000 7f4d23200700 1 -- 172.21.15.107:0/2054516964 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d2405f5c0 con 0x7f4d24102d40 2024-03-04T13:53:10.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.897+0000 7f4d1a000700 1 -- 172.21.15.107:0/2054516964 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f4d1400bd60 con 0x7f4d24102d40 2024-03-04T13:53:10.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.898+0000 7f4d1a000700 1 --2- 172.21.15.107:0/2054516964 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f4d0806d7f0 0x7f4d0806fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:53:10.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.898+0000 7f4d1a000700 1 -- 172.21.15.107:0/2054516964 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f4d1408e270 con 0x7f4d24102d40 2024-03-04T13:53:10.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.898+0000 7f4d21400700 1 --2- 172.21.15.107:0/2054516964 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f4d0806d7f0 0x7f4d0806fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:53:10.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.899+0000 7f4d21400700 1 --2- 172.21.15.107:0/2054516964 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f4d0806d7f0 0x7f4d0806fca0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4d24071c90 tx=0x7f4d10009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:53:10.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:10.902+0000 7f4d1a000700 1 -- 172.21.15.107:0/2054516964 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4d14057b70 con 0x7f4d24102d40 2024-03-04T13:53:11.073 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:10 smithi107 ceph-mon[112578]: mds.? [v2:172.21.15.107:6828/3010096376,v1:172.21.15.107:6829/3010096376] up:boot 2024-03-04T13:53:11.073 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:10 smithi107 ceph-mon[112578]: fsmap cephfs:1 {0=cephfs.smithi169.jusudl=up:active} 3 up:standby 2024-03-04T13:53:11.073 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:10 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi107.thycho"}]: dispatch 2024-03-04T13:53:11.074 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:10 smithi107 ceph-mon[112578]: from='client.34440 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:53:11.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:10 smithi169 ceph-mon[86183]: mds.? [v2:172.21.15.107:6828/3010096376,v1:172.21.15.107:6829/3010096376] up:boot 2024-03-04T13:53:11.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:10 smithi169 ceph-mon[86183]: fsmap cephfs:1 {0=cephfs.smithi169.jusudl=up:active} 3 up:standby 2024-03-04T13:53:11.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:10 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi107.thycho"}]: dispatch 2024-03-04T13:53:11.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:10 smithi169 ceph-mon[86183]: from='client.34440 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:53:11.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:11.213+0000 7f4d23200700 1 -- 172.21.15.107:0/2054516964 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f4d2411a1a0 con 0x7f4d24102d40 2024-03-04T13:53:11.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:11.214+0000 7f4d1a000700 1 -- 172.21.15.107:0/2054516964 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7f4d1405b230 con 0x7f4d24102d40 2024-03-04T13:53:11.216 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-03-04T13:53:11.217 INFO:teuthology.orchestra.run.smithi107.stdout: "mon": { 2024-03-04T13:53:11.217 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-1845-g717ce3cc (717ce3cce8d0166e4e577faa8004e6f9cb4128c0) squid (dev)": 2 2024-03-04T13:53:11.217 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:53:11.217 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr": { 2024-03-04T13:53:11.217 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-1845-g717ce3cc (717ce3cce8d0166e4e577faa8004e6f9cb4128c0) squid (dev)": 2 2024-03-04T13:53:11.217 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:53:11.217 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": { 2024-03-04T13:53:11.217 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-1845-g717ce3cc (717ce3cce8d0166e4e577faa8004e6f9cb4128c0) squid (dev)": 6 2024-03-04T13:53:11.217 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:53:11.217 INFO:teuthology.orchestra.run.smithi107.stdout: "mds": { 2024-03-04T13:53:11.217 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-545-g42608bb1 (42608bb191117be10dba720e6bf8a02f3171cd34) quincy (stable)": 2, 2024-03-04T13:53:11.217 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-1845-g717ce3cc (717ce3cce8d0166e4e577faa8004e6f9cb4128c0) squid (dev)": 2 2024-03-04T13:53:11.218 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:53:11.218 INFO:teuthology.orchestra.run.smithi107.stdout: "overall": { 2024-03-04T13:53:11.218 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-545-g42608bb1 (42608bb191117be10dba720e6bf8a02f3171cd34) quincy (stable)": 2, 2024-03-04T13:53:11.218 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-1845-g717ce3cc (717ce3cce8d0166e4e577faa8004e6f9cb4128c0) squid (dev)": 12 2024-03-04T13:53:11.218 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-04T13:53:11.218 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-03-04T13:53:11.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:11.217+0000 7f4d23200700 1 -- 172.21.15.107:0/2054516964 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f4d0806d7f0 msgr2=0x7f4d0806fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:53:11.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:11.217+0000 7f4d23200700 1 --2- 172.21.15.107:0/2054516964 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f4d0806d7f0 0x7f4d0806fca0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4d24071c90 tx=0x7f4d10009040 comp rx=0 tx=0).stop 2024-03-04T13:53:11.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:11.217+0000 7f4d23200700 1 -- 172.21.15.107:0/2054516964 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d24102d40 msgr2=0x7f4d24070290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:53:11.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:11.217+0000 7f4d23200700 1 --2- 172.21.15.107:0/2054516964 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d24102d40 0x7f4d24070290 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4d14009150 tx=0x7f4d14009e30 comp rx=0 tx=0).stop 2024-03-04T13:53:11.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:11.217+0000 7f4d23200700 1 -- 172.21.15.107:0/2054516964 shutdown_connections 2024-03-04T13:53:11.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:11.217+0000 7f4d23200700 1 --2- 172.21.15.107:0/2054516964 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f4d0806d7f0 0x7f4d0806fca0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:53:11.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:11.217+0000 7f4d23200700 1 --2- 172.21.15.107:0/2054516964 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d24102d40 0x7f4d24070290 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:53:11.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:11.217+0000 7f4d23200700 1 --2- 172.21.15.107:0/2054516964 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d241036e0 0x7f4d240707d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:53:11.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:11.217+0000 7f4d23200700 1 -- 172.21.15.107:0/2054516964 >> 172.21.15.107:0/2054516964 conn(0x7f4d240f88b0 msgr2=0x7f4d240fa2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:53:11.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:11.218+0000 7f4d23200700 1 -- 172.21.15.107:0/2054516964 shutdown_connections 2024-03-04T13:53:11.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:11.218+0000 7f4d23200700 1 -- 172.21.15.107:0/2054516964 wait complete. 2024-03-04T13:53:11.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:11.355+0000 7f4322a00700 1 -- 172.21.15.107:0/111798925 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f431c1038f0 msgr2=0x7f431c0fcf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:53:11.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:11.355+0000 7f4322a00700 1 --2- 172.21.15.107:0/111798925 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f431c1038f0 0x7f431c0fcf60 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f4308004940 tx=0x7f4308009e30 comp rx=0 tx=0).stop 2024-03-04T13:53:11.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:11.355+0000 7f4322a00700 1 -- 172.21.15.107:0/111798925 shutdown_connections 2024-03-04T13:53:11.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:11.355+0000 7f4322a00700 1 --2- 172.21.15.107:0/111798925 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f431c1038f0 0x7f431c0fcf60 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:53:11.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:11.355+0000 7f4322a00700 1 --2- 172.21.15.107:0/111798925 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f431c102f50 0x7f431c103320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:53:11.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:11.355+0000 7f4322a00700 1 -- 172.21.15.107:0/111798925 >> 172.21.15.107:0/111798925 conn(0x7f431c0f8ae0 msgr2=0x7f431c0faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:53:11.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:11.356+0000 7f4322a00700 1 -- 172.21.15.107:0/111798925 shutdown_connections 2024-03-04T13:53:11.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:11.356+0000 7f4322a00700 1 -- 172.21.15.107:0/111798925 wait complete. 2024-03-04T13:53:11.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:11.356+0000 7f4322a00700 1 Processor -- start 2024-03-04T13:53:11.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:11.356+0000 7f4322a00700 1 -- start start 2024-03-04T13:53:11.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:11.357+0000 7f4322a00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f431c102f50 0x7f431c115cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:53:11.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:11.357+0000 7f4322a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f431c1038f0 0x7f431c116210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:53:11.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:11.357+0000 7f4322a00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f431c1168f0 con 0x7f431c1038f0 2024-03-04T13:53:11.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:11.357+0000 7f4322a00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f431c10fb70 con 0x7f431c102f50 2024-03-04T13:53:11.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:11.357+0000 7f431b400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f431c1038f0 0x7f431c116210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:53:11.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:11.357+0000 7f431b400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f431c1038f0 0x7f431c116210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39452/0 (socket says 172.21.15.107:39452) 2024-03-04T13:53:11.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:11.357+0000 7f431b400700 1 -- 172.21.15.107:0/2651009974 learned_addr learned my addr 172.21.15.107:0/2651009974 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:53:11.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:11.357+0000 7f431b400700 1 -- 172.21.15.107:0/2651009974 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f431c102f50 msgr2=0x7f431c115cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:53:11.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:11.357+0000 7f431b400700 1 --2- 172.21.15.107:0/2651009974 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f431c102f50 0x7f431c115cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:53:11.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:11.357+0000 7f431b400700 1 -- 172.21.15.107:0/2651009974 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4308005040 con 0x7f431c1038f0 2024-03-04T13:53:11.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:11.358+0000 7f431b400700 1 --2- 172.21.15.107:0/2651009974 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f431c1038f0 0x7f431c116210 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4308009100 tx=0x7f4308009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:53:11.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:11.358+0000 7f4313e00700 1 -- 172.21.15.107:0/2651009974 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f430800e040 con 0x7f431c1038f0 2024-03-04T13:53:11.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:11.358+0000 7f4313e00700 1 -- 172.21.15.107:0/2651009974 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4308004290 con 0x7f431c1038f0 2024-03-04T13:53:11.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:11.358+0000 7f4313e00700 1 -- 172.21.15.107:0/2651009974 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f430801c3f0 con 0x7f431c1038f0 2024-03-04T13:53:11.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:11.358+0000 7f4322a00700 1 -- 172.21.15.107:0/2651009974 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f431c10fd90 con 0x7f431c1038f0 2024-03-04T13:53:11.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:11.358+0000 7f4322a00700 1 -- 172.21.15.107:0/2651009974 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f431c1102d0 con 0x7f431c1038f0 2024-03-04T13:53:11.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:11.359+0000 7f4322a00700 1 -- 172.21.15.107:0/2651009974 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f431c0fc6e0 con 0x7f431c1038f0 2024-03-04T13:53:11.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:11.360+0000 7f4313e00700 1 -- 172.21.15.107:0/2651009974 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f430800bbf0 con 0x7f431c1038f0 2024-03-04T13:53:11.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:11.361+0000 7f4313e00700 1 --2- 172.21.15.107:0/2651009974 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f430006d7f0 0x7f430006fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:53:11.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:11.361+0000 7f4313e00700 1 -- 172.21.15.107:0/2651009974 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f430808e240 con 0x7f431c1038f0 2024-03-04T13:53:11.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:11.361+0000 7f431be00700 1 --2- 172.21.15.107:0/2651009974 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f430006d7f0 0x7f430006fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:53:11.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:11.362+0000 7f431be00700 1 --2- 172.21.15.107:0/2651009974 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f430006d7f0 0x7f430006fca0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f430c00acf0 tx=0x7f430c009370 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:53:11.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:11.365+0000 7f4313e00700 1 -- 172.21.15.107:0/2651009974 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f430805b5f0 con 0x7f431c1038f0 2024-03-04T13:53:11.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:11.613+0000 7f4322a00700 1 -- 172.21.15.107:0/2651009974 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f431c1113a0 con 0x7f431c1038f0 2024-03-04T13:53:11.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:11.613+0000 7f4313e00700 1 -- 172.21.15.107:0/2651009974 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 22 v22) v1 ==== 76+0+1852 (secure 0 0 0) 0x7f430805b180 con 0x7f431c1038f0 2024-03-04T13:53:11.616 INFO:teuthology.orchestra.run.smithi107.stdout:e22 2024-03-04T13:53:11.616 INFO:teuthology.orchestra.run.smithi107.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-04T13:53:11.616 INFO:teuthology.orchestra.run.smithi107.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-04T13:53:11.616 INFO:teuthology.orchestra.run.smithi107.stdout:legacy client fscid: 1 2024-03-04T13:53:11.616 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:53:11.616 INFO:teuthology.orchestra.run.smithi107.stdout:Filesystem 'cephfs' (1) 2024-03-04T13:53:11.616 INFO:teuthology.orchestra.run.smithi107.stdout:fs_name cephfs 2024-03-04T13:53:11.617 INFO:teuthology.orchestra.run.smithi107.stdout:epoch 17 2024-03-04T13:53:11.617 INFO:teuthology.orchestra.run.smithi107.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-04T13:53:11.617 INFO:teuthology.orchestra.run.smithi107.stdout:created 2024-03-04T13:38:30.854004+0000 2024-03-04T13:53:11.617 INFO:teuthology.orchestra.run.smithi107.stdout:modified 2024-03-04T13:52:34.079716+0000 2024-03-04T13:53:11.617 INFO:teuthology.orchestra.run.smithi107.stdout:tableserver 0 2024-03-04T13:53:11.617 INFO:teuthology.orchestra.run.smithi107.stdout:root 0 2024-03-04T13:53:11.617 INFO:teuthology.orchestra.run.smithi107.stdout:session_timeout 60 2024-03-04T13:53:11.617 INFO:teuthology.orchestra.run.smithi107.stdout:session_autoclose 300 2024-03-04T13:53:11.617 INFO:teuthology.orchestra.run.smithi107.stdout:max_file_size 1099511627776 2024-03-04T13:53:11.617 INFO:teuthology.orchestra.run.smithi107.stdout:max_xattr_size 65536 2024-03-04T13:53:11.617 INFO:teuthology.orchestra.run.smithi107.stdout:required_client_features {} 2024-03-04T13:53:11.617 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure 0 2024-03-04T13:53:11.617 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure_osd_epoch 0 2024-03-04T13:53:11.618 INFO:teuthology.orchestra.run.smithi107.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-04T13:53:11.618 INFO:teuthology.orchestra.run.smithi107.stdout:max_mds 1 2024-03-04T13:53:11.618 INFO:teuthology.orchestra.run.smithi107.stdout:in 0 2024-03-04T13:53:11.618 INFO:teuthology.orchestra.run.smithi107.stdout:up {0=24247} 2024-03-04T13:53:11.618 INFO:teuthology.orchestra.run.smithi107.stdout:failed 2024-03-04T13:53:11.618 INFO:teuthology.orchestra.run.smithi107.stdout:damaged 2024-03-04T13:53:11.618 INFO:teuthology.orchestra.run.smithi107.stdout:stopped 2024-03-04T13:53:11.618 INFO:teuthology.orchestra.run.smithi107.stdout:data_pools [3] 2024-03-04T13:53:11.618 INFO:teuthology.orchestra.run.smithi107.stdout:metadata_pool 2 2024-03-04T13:53:11.618 INFO:teuthology.orchestra.run.smithi107.stdout:inline_data enabled 2024-03-04T13:53:11.618 INFO:teuthology.orchestra.run.smithi107.stdout:balancer 2024-03-04T13:53:11.618 INFO:teuthology.orchestra.run.smithi107.stdout:bal_rank_mask -1 2024-03-04T13:53:11.618 INFO:teuthology.orchestra.run.smithi107.stdout:standby_count_wanted 1 2024-03-04T13:53:11.618 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi169.jusudl{0:24247} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.169:6824/2994866076,v1:172.21.15.169:6825/2994866076] compat {c=[1],r=[1],i=[7ff]}] 2024-03-04T13:53:11.619 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:53:11.619 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:53:11.619 INFO:teuthology.orchestra.run.smithi107.stdout:Standby daemons: 2024-03-04T13:53:11.619 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:53:11.619 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi169.bxitzk{-1:24259} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.169:6826/3696090493,v1:172.21.15.169:6827/3696090493] compat {c=[1],r=[1],i=[7ff]}] 2024-03-04T13:53:11.619 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.riexmm{-1:34432} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.107:6826/1144672731,v1:172.21.15.107:6827/1144672731] compat {c=[1],r=[1],i=[fff]}] 2024-03-04T13:53:11.619 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.thycho{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.107:6828/3010096376,v1:172.21.15.107:6829/3010096376] compat {c=[1],r=[1],i=[fff]}] 2024-03-04T13:53:11.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:11.616+0000 7f4322a00700 1 -- 172.21.15.107:0/2651009974 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f430006d7f0 msgr2=0x7f430006fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:53:11.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:11.616+0000 7f4322a00700 1 --2- 172.21.15.107:0/2651009974 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f430006d7f0 0x7f430006fca0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f430c00acf0 tx=0x7f430c009370 comp rx=0 tx=0).stop 2024-03-04T13:53:11.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:11.616+0000 7f4322a00700 1 -- 172.21.15.107:0/2651009974 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f431c1038f0 msgr2=0x7f431c116210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:53:11.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:11.616+0000 7f4322a00700 1 --2- 172.21.15.107:0/2651009974 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f431c1038f0 0x7f431c116210 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4308009100 tx=0x7f4308009e30 comp rx=0 tx=0).stop 2024-03-04T13:53:11.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:11.616+0000 7f4322a00700 1 -- 172.21.15.107:0/2651009974 shutdown_connections 2024-03-04T13:53:11.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:11.616+0000 7f4322a00700 1 --2- 172.21.15.107:0/2651009974 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f430006d7f0 0x7f430006fca0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:53:11.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:11.616+0000 7f4322a00700 1 --2- 172.21.15.107:0/2651009974 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f431c102f50 0x7f431c115cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:53:11.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:11.616+0000 7f4322a00700 1 --2- 172.21.15.107:0/2651009974 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f431c1038f0 0x7f431c116210 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:53:11.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:11.616+0000 7f4322a00700 1 -- 172.21.15.107:0/2651009974 >> 172.21.15.107:0/2651009974 conn(0x7f431c0f8ae0 msgr2=0x7f431c0fa360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:53:11.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:11.616+0000 7f4322a00700 1 -- 172.21.15.107:0/2651009974 shutdown_connections 2024-03-04T13:53:11.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:11.617+0000 7f4322a00700 1 -- 172.21.15.107:0/2651009974 wait complete. 2024-03-04T13:53:11.621 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 22 2024-03-04T13:53:11.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:11.762+0000 7ff1b8600700 1 -- 172.21.15.107:0/2517057616 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff1b0070bc0 msgr2=0x7ff1b01092e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:53:11.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:11.762+0000 7ff1b8600700 1 --2- 172.21.15.107:0/2517057616 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff1b0070bc0 0x7ff1b01092e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff19c004940 tx=0x7ff19c009e30 comp rx=0 tx=0).stop 2024-03-04T13:53:11.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:11.762+0000 7ff1b8600700 1 -- 172.21.15.107:0/2517057616 shutdown_connections 2024-03-04T13:53:11.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:11.762+0000 7ff1b8600700 1 --2- 172.21.15.107:0/2517057616 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff1b0070bc0 0x7ff1b01092e0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:53:11.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:11.762+0000 7ff1b8600700 1 --2- 172.21.15.107:0/2517057616 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1b00702b0 0x7ff1b0070680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:53:11.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:11.762+0000 7ff1b8600700 1 -- 172.21.15.107:0/2517057616 >> 172.21.15.107:0/2517057616 conn(0x7ff1b00f8ae0 msgr2=0x7ff1b00faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:53:11.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:11.762+0000 7ff1b8600700 1 -- 172.21.15.107:0/2517057616 shutdown_connections 2024-03-04T13:53:11.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:11.762+0000 7ff1b8600700 1 -- 172.21.15.107:0/2517057616 wait complete. 2024-03-04T13:53:11.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:11.763+0000 7ff1b8600700 1 Processor -- start 2024-03-04T13:53:11.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:11.763+0000 7ff1b8600700 1 -- start start 2024-03-04T13:53:11.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:11.763+0000 7ff1b8600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff1b00702b0 0x7ff1b0115cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:53:11.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:11.763+0000 7ff1b8600700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1b0070bc0 0x7ff1b0116210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:53:11.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:11.763+0000 7ff1b8600700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b0116860 con 0x7ff1b00702b0 2024-03-04T13:53:11.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:11.763+0000 7ff1b8600700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b010fb70 con 0x7ff1b0070bc0 2024-03-04T13:53:11.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:11.764+0000 7ff1b5c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff1b00702b0 0x7ff1b0115cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:53:11.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:11.764+0000 7ff1b5c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff1b00702b0 0x7ff1b0115cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39478/0 (socket says 172.21.15.107:39478) 2024-03-04T13:53:11.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:11.764+0000 7ff1b5c00700 1 -- 172.21.15.107:0/3552638582 learned_addr learned my addr 172.21.15.107:0/3552638582 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:53:11.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:11.764+0000 7ff1b5200700 1 --2- 172.21.15.107:0/3552638582 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1b0070bc0 0x7ff1b0116210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:53:11.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:11.764+0000 7ff1b5c00700 1 -- 172.21.15.107:0/3552638582 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1b0070bc0 msgr2=0x7ff1b0116210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:53:11.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:11.764+0000 7ff1b5c00700 1 --2- 172.21.15.107:0/3552638582 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1b0070bc0 0x7ff1b0116210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:53:11.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:11.764+0000 7ff1b5c00700 1 -- 172.21.15.107:0/3552638582 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff19c005040 con 0x7ff1b00702b0 2024-03-04T13:53:11.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:11.764+0000 7ff1b5c00700 1 --2- 172.21.15.107:0/3552638582 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff1b00702b0 0x7ff1b0115cd0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff1a400c8c0 tx=0x7ff1a40044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:53:11.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:11.764+0000 7ff1ad600700 1 -- 172.21.15.107:0/3552638582 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff1a400a980 con 0x7ff1b00702b0 2024-03-04T13:53:11.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:11.765+0000 7ff1ad600700 1 -- 172.21.15.107:0/3552638582 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff1a400ce60 con 0x7ff1b00702b0 2024-03-04T13:53:11.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:11.765+0000 7ff1b8600700 1 -- 172.21.15.107:0/3552638582 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1b010fe50 con 0x7ff1b00702b0 2024-03-04T13:53:11.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:11.765+0000 7ff1ad600700 1 -- 172.21.15.107:0/3552638582 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff1a40115e0 con 0x7ff1b00702b0 2024-03-04T13:53:11.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:11.765+0000 7ff1b8600700 1 -- 172.21.15.107:0/3552638582 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1b0110390 con 0x7ff1b00702b0 2024-03-04T13:53:11.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:11.766+0000 7ff1b8600700 1 -- 172.21.15.107:0/3552638582 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1b01069a0 con 0x7ff1b00702b0 2024-03-04T13:53:11.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:11.767+0000 7ff1ad600700 1 -- 172.21.15.107:0/3552638582 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7ff1a4012400 con 0x7ff1b00702b0 2024-03-04T13:53:11.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:11.768+0000 7ff1ad600700 1 --2- 172.21.15.107:0/3552638582 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7ff19806d7f0 0x7ff19806fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:53:11.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:11.768+0000 7ff1ad600700 1 -- 172.21.15.107:0/3552638582 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6324+0+0 (secure 0 0 0) 0x7ff1a408cff0 con 0x7ff1b00702b0 2024-03-04T13:53:11.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:11.768+0000 7ff1b5200700 1 --2- 172.21.15.107:0/3552638582 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7ff19806d7f0 0x7ff19806fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:53:11.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:11.769+0000 7ff1b5200700 1 --2- 172.21.15.107:0/3552638582 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7ff19806d7f0 0x7ff19806fca0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff1b0111570 tx=0x7ff19c009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:53:11.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:11.777+0000 7ff1ad600700 1 -- 172.21.15.107:0/3552638582 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff1a40c0d20 con 0x7ff1b00702b0 2024-03-04T13:53:12.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:12.032+0000 7ff1b8600700 1 -- 172.21.15.107:0/3552638582 --> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff1b004bed0 con 0x7ff19806d7f0 2024-03-04T13:53:12.034 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:11 smithi107 ceph-mon[112578]: from='client.34444 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:53:12.034 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:11 smithi107 ceph-mon[112578]: from='client.34448 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:53:12.035 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:11 smithi107 ceph-mon[112578]: pgmap v210: 65 pgs: 65 active+clean; 255 MiB data, 963 MiB used, 535 GiB / 536 GiB avail 2024-03-04T13:53:12.035 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:11 smithi107 ceph-mon[112578]: from='client.? 172.21.15.107:0/2054516964' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:53:12.035 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:11 smithi107 ceph-mon[112578]: from='client.? 172.21.15.107:0/2651009974' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-04T13:53:12.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:12.034+0000 7ff1ad600700 1 -- 172.21.15.107:0/3552638582 <== mgr.34102 v2:172.21.15.107:6800/1223039812 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7ff1b004bed0 con 0x7ff19806d7f0 2024-03-04T13:53:12.036 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-03-04T13:53:12.036 INFO:teuthology.orchestra.run.smithi107.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377", 2024-03-04T13:53:12.036 INFO:teuthology.orchestra.run.smithi107.stdout: "in_progress": true, 2024-03-04T13:53:12.036 INFO:teuthology.orchestra.run.smithi107.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-04T13:53:12.036 INFO:teuthology.orchestra.run.smithi107.stdout: "services_complete": [ 2024-03-04T13:53:12.036 INFO:teuthology.orchestra.run.smithi107.stdout: "osd", 2024-03-04T13:53:12.036 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr", 2024-03-04T13:53:12.036 INFO:teuthology.orchestra.run.smithi107.stdout: "mon", 2024-03-04T13:53:12.036 INFO:teuthology.orchestra.run.smithi107.stdout: "crash" 2024-03-04T13:53:12.036 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-04T13:53:12.036 INFO:teuthology.orchestra.run.smithi107.stdout: "progress": "13/21 daemons upgraded", 2024-03-04T13:53:12.036 INFO:teuthology.orchestra.run.smithi107.stdout: "message": "Currently upgrading mds daemons", 2024-03-04T13:53:12.037 INFO:teuthology.orchestra.run.smithi107.stdout: "is_paused": false 2024-03-04T13:53:12.037 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-03-04T13:53:12.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:12.038+0000 7ff1b8600700 1 -- 172.21.15.107:0/3552638582 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7ff19806d7f0 msgr2=0x7ff19806fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:53:12.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:12.038+0000 7ff1b8600700 1 --2- 172.21.15.107:0/3552638582 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7ff19806d7f0 0x7ff19806fca0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff1b0111570 tx=0x7ff19c009e30 comp rx=0 tx=0).stop 2024-03-04T13:53:12.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:12.038+0000 7ff1b8600700 1 -- 172.21.15.107:0/3552638582 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff1b00702b0 msgr2=0x7ff1b0115cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:53:12.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:12.038+0000 7ff1b8600700 1 --2- 172.21.15.107:0/3552638582 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff1b00702b0 0x7ff1b0115cd0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff1a400c8c0 tx=0x7ff1a40044c0 comp rx=0 tx=0).stop 2024-03-04T13:53:12.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:12.038+0000 7ff1b8600700 1 -- 172.21.15.107:0/3552638582 shutdown_connections 2024-03-04T13:53:12.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:12.038+0000 7ff1b8600700 1 --2- 172.21.15.107:0/3552638582 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7ff19806d7f0 0x7ff19806fca0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:53:12.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:12.038+0000 7ff1b8600700 1 --2- 172.21.15.107:0/3552638582 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff1b00702b0 0x7ff1b0115cd0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:53:12.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:12.038+0000 7ff1b8600700 1 --2- 172.21.15.107:0/3552638582 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff1b0070bc0 0x7ff1b0116210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:53:12.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:12.038+0000 7ff1b8600700 1 -- 172.21.15.107:0/3552638582 >> 172.21.15.107:0/3552638582 conn(0x7ff1b00f8ae0 msgr2=0x7ff1b0103b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:53:12.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:12.038+0000 7ff1b8600700 1 -- 172.21.15.107:0/3552638582 shutdown_connections 2024-03-04T13:53:12.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:12.038+0000 7ff1b8600700 1 -- 172.21.15.107:0/3552638582 wait complete. 2024-03-04T13:53:12.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:11 smithi169 ceph-mon[86183]: from='client.34444 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:53:12.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:11 smithi169 ceph-mon[86183]: from='client.34448 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:53:12.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:11 smithi169 ceph-mon[86183]: pgmap v210: 65 pgs: 65 active+clean; 255 MiB data, 963 MiB used, 535 GiB / 536 GiB avail 2024-03-04T13:53:12.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:11 smithi169 ceph-mon[86183]: from='client.? 172.21.15.107:0/2054516964' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:53:12.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:11 smithi169 ceph-mon[86183]: from='client.? 172.21.15.107:0/2651009974' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-04T13:53:12.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:12.184+0000 7fc853400700 1 -- 172.21.15.107:0/3487145723 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc84c102f50 msgr2=0x7fc84c103320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:53:12.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:12.184+0000 7fc853400700 1 --2- 172.21.15.107:0/3487145723 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc84c102f50 0x7fc84c103320 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc840004940 tx=0x7fc840009e30 comp rx=0 tx=0).stop 2024-03-04T13:53:12.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:12.184+0000 7fc853400700 1 -- 172.21.15.107:0/3487145723 shutdown_connections 2024-03-04T13:53:12.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:12.184+0000 7fc853400700 1 --2- 172.21.15.107:0/3487145723 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc84c1038f0 0x7fc84c0fcf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:53:12.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:12.184+0000 7fc853400700 1 --2- 172.21.15.107:0/3487145723 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc84c102f50 0x7fc84c103320 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:53:12.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:12.184+0000 7fc853400700 1 -- 172.21.15.107:0/3487145723 >> 172.21.15.107:0/3487145723 conn(0x7fc84c0f8ae0 msgr2=0x7fc84c0faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:53:12.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:12.184+0000 7fc853400700 1 -- 172.21.15.107:0/3487145723 shutdown_connections 2024-03-04T13:53:12.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:12.184+0000 7fc853400700 1 -- 172.21.15.107:0/3487145723 wait complete. 2024-03-04T13:53:12.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:12.185+0000 7fc853400700 1 Processor -- start 2024-03-04T13:53:12.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:12.185+0000 7fc853400700 1 -- start start 2024-03-04T13:53:12.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:12.185+0000 7fc853400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc84c102f50 0x7fc84c115bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:53:12.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:12.185+0000 7fc853400700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc84c1038f0 0x7fc84c116130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:53:12.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:12.185+0000 7fc853400700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc84c116780 con 0x7fc84c102f50 2024-03-04T13:53:12.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:12.185+0000 7fc853400700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc84c10fad0 con 0x7fc84c1038f0 2024-03-04T13:53:12.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:12.185+0000 7fc850a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc84c102f50 0x7fc84c115bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:53:12.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:12.185+0000 7fc850a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc84c102f50 0x7fc84c115bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39498/0 (socket says 172.21.15.107:39498) 2024-03-04T13:53:12.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:12.185+0000 7fc850a00700 1 -- 172.21.15.107:0/73516073 learned_addr learned my addr 172.21.15.107:0/73516073 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:53:12.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:12.186+0000 7fc84be00700 1 --2- 172.21.15.107:0/73516073 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc84c1038f0 0x7fc84c116130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:53:12.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:12.186+0000 7fc850a00700 1 -- 172.21.15.107:0/73516073 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc84c1038f0 msgr2=0x7fc84c116130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:53:12.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:12.186+0000 7fc850a00700 1 --2- 172.21.15.107:0/73516073 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc84c1038f0 0x7fc84c116130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:53:12.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:12.186+0000 7fc850a00700 1 -- 172.21.15.107:0/73516073 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc840005040 con 0x7fc84c102f50 2024-03-04T13:53:12.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:12.186+0000 7fc850a00700 1 --2- 172.21.15.107:0/73516073 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc84c102f50 0x7fc84c115bf0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc840009150 tx=0x7fc840009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:53:12.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:12.186+0000 7fc848c00700 1 -- 172.21.15.107:0/73516073 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc84000e040 con 0x7fc84c102f50 2024-03-04T13:53:12.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:12.186+0000 7fc848c00700 1 -- 172.21.15.107:0/73516073 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc84000bbf0 con 0x7fc84c102f50 2024-03-04T13:53:12.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:12.186+0000 7fc848c00700 1 -- 172.21.15.107:0/73516073 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc84001c630 con 0x7fc84c102f50 2024-03-04T13:53:12.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:12.186+0000 7fc853400700 1 -- 172.21.15.107:0/73516073 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc84c10fd50 con 0x7fc84c102f50 2024-03-04T13:53:12.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:12.187+0000 7fc853400700 1 -- 172.21.15.107:0/73516073 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc84c110290 con 0x7fc84c102f50 2024-03-04T13:53:12.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:12.188+0000 7fc853400700 1 -- 172.21.15.107:0/73516073 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc84c0fc6e0 con 0x7fc84c102f50 2024-03-04T13:53:12.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:12.189+0000 7fc848c00700 1 -- 172.21.15.107:0/73516073 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fc84000bd60 con 0x7fc84c102f50 2024-03-04T13:53:12.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:12.189+0000 7fc848c00700 1 --2- 172.21.15.107:0/73516073 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fc83406d7f0 0x7fc83406fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:53:12.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:12.190+0000 7fc848c00700 1 -- 172.21.15.107:0/73516073 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fc84008e330 con 0x7fc84c102f50 2024-03-04T13:53:12.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:12.190+0000 7fc84be00700 1 --2- 172.21.15.107:0/73516073 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fc83406d7f0 0x7fc83406fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:53:12.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:12.190+0000 7fc84be00700 1 --2- 172.21.15.107:0/73516073 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fc83406d7f0 0x7fc83406fca0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc84c111490 tx=0x7fc838009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:53:12.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:12.194+0000 7fc848c00700 1 -- 172.21.15.107:0/73516073 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc840057bb0 con 0x7fc84c102f50 2024-03-04T13:53:12.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:12.506+0000 7fc853400700 1 -- 172.21.15.107:0/73516073 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fc84c111270 con 0x7fc84c102f50 2024-03-04T13:53:12.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:12.507+0000 7fc848c00700 1 -- 172.21.15.107:0/73516073 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7fc84005b270 con 0x7fc84c102f50 2024-03-04T13:53:12.509 INFO:teuthology.orchestra.run.smithi107.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-03-04T13:53:12.509 INFO:teuthology.orchestra.run.smithi107.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-04T13:53:12.509 INFO:teuthology.orchestra.run.smithi107.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-04T13:53:12.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:12.510+0000 7fc853400700 1 -- 172.21.15.107:0/73516073 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fc83406d7f0 msgr2=0x7fc83406fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:53:12.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:12.510+0000 7fc853400700 1 --2- 172.21.15.107:0/73516073 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fc83406d7f0 0x7fc83406fca0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc84c111490 tx=0x7fc838009040 comp rx=0 tx=0).stop 2024-03-04T13:53:12.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:12.510+0000 7fc853400700 1 -- 172.21.15.107:0/73516073 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc84c102f50 msgr2=0x7fc84c115bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:53:12.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:12.510+0000 7fc853400700 1 --2- 172.21.15.107:0/73516073 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc84c102f50 0x7fc84c115bf0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc840009150 tx=0x7fc840009e30 comp rx=0 tx=0).stop 2024-03-04T13:53:12.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:12.510+0000 7fc853400700 1 -- 172.21.15.107:0/73516073 shutdown_connections 2024-03-04T13:53:12.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:12.510+0000 7fc853400700 1 --2- 172.21.15.107:0/73516073 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fc83406d7f0 0x7fc83406fca0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:53:12.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:12.510+0000 7fc853400700 1 --2- 172.21.15.107:0/73516073 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc84c102f50 0x7fc84c115bf0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:53:12.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:12.510+0000 7fc853400700 1 --2- 172.21.15.107:0/73516073 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc84c1038f0 0x7fc84c116130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:53:12.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:12.510+0000 7fc853400700 1 -- 172.21.15.107:0/73516073 >> 172.21.15.107:0/73516073 conn(0x7fc84c0f8ae0 msgr2=0x7fc84c0fa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:53:12.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:12.510+0000 7fc853400700 1 -- 172.21.15.107:0/73516073 shutdown_connections 2024-03-04T13:53:12.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:12.510+0000 7fc853400700 1 -- 172.21.15.107:0/73516073 wait complete. 2024-03-04T13:53:13.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:12 smithi107 ceph-mon[112578]: from='client.? 172.21.15.107:0/73516073' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-04T13:53:13.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:12 smithi169 ceph-mon[86183]: from='client.? 172.21.15.107:0/73516073' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-04T13:53:14.135 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:13 smithi107 ceph-mon[112578]: from='client.34460 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:53:14.135 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:13 smithi107 ceph-mon[112578]: pgmap v211: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail 2024-03-04T13:53:14.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:13 smithi169 ceph-mon[86183]: from='client.34460 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:53:14.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:13 smithi169 ceph-mon[86183]: pgmap v211: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail 2024-03-04T13:53:16.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:15 smithi107 ceph-mon[112578]: pgmap v212: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail 2024-03-04T13:53:16.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:15 smithi169 ceph-mon[86183]: pgmap v212: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail 2024-03-04T13:53:17.594 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:17 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:53:17.594 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:17 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:53:17.594 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:17 smithi107 ceph-mon[112578]: pgmap v213: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail 2024-03-04T13:53:17.611 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:17 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:53:17.611 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:17 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:53:17.612 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:17 smithi169 ceph-mon[86183]: pgmap v213: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail 2024-03-04T13:53:19.507 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:19 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:53:19.508 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:19 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:53:19.508 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:19 smithi107 ceph-mon[112578]: pgmap v214: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail 2024-03-04T13:53:19.612 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:19 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:53:19.612 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:19 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:53:19.612 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:19 smithi169 ceph-mon[86183]: pgmap v214: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail 2024-03-04T13:53:22.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:21 smithi107 ceph-mon[112578]: pgmap v215: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail 2024-03-04T13:53:22.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:21 smithi169 ceph-mon[86183]: pgmap v215: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail 2024-03-04T13:53:24.095 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:23 smithi107 ceph-mon[112578]: pgmap v216: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail 2024-03-04T13:53:24.095 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:23 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:53:24.095 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:23 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:53:24.095 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:23 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:53:24.095 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:23 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:53:24.095 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:23 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:53:24.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:23 smithi169 ceph-mon[86183]: pgmap v216: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail 2024-03-04T13:53:24.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:23 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:53:24.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:23 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:53:24.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:23 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:53:24.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:23 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:53:24.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:23 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:53:25.075 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:24 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:53:25.075 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:24 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:53:25.075 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:24 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:53:25.076 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:24 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:53:25.076 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:24 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:53:25.076 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:24 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:53:25.095 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:24 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:53:25.095 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:24 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:53:25.095 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:24 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:53:25.095 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:24 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:53:25.095 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:24 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:53:25.095 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:24 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:53:26.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:25 smithi107 ceph-mon[112578]: pgmap v217: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail 2024-03-04T13:53:26.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:25 smithi169 ceph-mon[86183]: pgmap v217: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail 2024-03-04T13:53:27.561 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:27 smithi169 ceph-mon[86183]: Upgrade: Updating mds.cephfs.smithi169.jusudl 2024-03-04T13:53:27.561 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:27 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:53:27.561 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:27 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi169.jusudl", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-04T13:53:27.562 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:27 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:53:27.562 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:27 smithi169 ceph-mon[86183]: Deploying daemon mds.cephfs.smithi169.jusudl on smithi169 2024-03-04T13:53:27.562 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:27 smithi169 ceph-mon[86183]: pgmap v218: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail 2024-03-04T13:53:27.845 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:27 smithi107 ceph-mon[112578]: Upgrade: Updating mds.cephfs.smithi169.jusudl 2024-03-04T13:53:27.845 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:27 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:53:27.845 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:27 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi169.jusudl", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-04T13:53:27.845 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:27 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:53:27.845 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:27 smithi107 ceph-mon[112578]: Deploying daemon mds.cephfs.smithi169.jusudl on smithi169 2024-03-04T13:53:27.845 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:27 smithi107 ceph-mon[112578]: pgmap v218: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail 2024-03-04T13:53:29.595 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:29 smithi107 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-mon-smithi107[112574]: 2024-03-04T13:53:29.273+0000 7f6566400700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-04T13:53:30.095 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:29 smithi107 ceph-mon[112578]: pgmap v219: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail 2024-03-04T13:53:30.095 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:29 smithi107 ceph-mon[112578]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-04T13:53:30.095 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:29 smithi107 ceph-mon[112578]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-04T13:53:30.095 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:29 smithi107 ceph-mon[112578]: osdmap e85: 6 total, 6 up, 6 in 2024-03-04T13:53:30.095 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:29 smithi107 ceph-mon[112578]: Standby daemon mds.cephfs.smithi169.bxitzk assigned to filesystem cephfs as rank 0 2024-03-04T13:53:30.095 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:29 smithi107 ceph-mon[112578]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-04T13:53:30.095 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:29 smithi107 ceph-mon[112578]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-03-04T13:53:30.096 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:29 smithi107 ceph-mon[112578]: fsmap cephfs:1/1 {0=cephfs.smithi169.bxitzk=up:replay} 2 up:standby 2024-03-04T13:53:30.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:29 smithi169 ceph-mon[86183]: pgmap v219: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail 2024-03-04T13:53:30.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:29 smithi169 ceph-mon[86183]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-04T13:53:30.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:29 smithi169 ceph-mon[86183]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-04T13:53:30.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:29 smithi169 ceph-mon[86183]: osdmap e85: 6 total, 6 up, 6 in 2024-03-04T13:53:30.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:29 smithi169 ceph-mon[86183]: Standby daemon mds.cephfs.smithi169.bxitzk assigned to filesystem cephfs as rank 0 2024-03-04T13:53:30.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:29 smithi169 ceph-mon[86183]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-04T13:53:30.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:29 smithi169 ceph-mon[86183]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-03-04T13:53:30.113 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:29 smithi169 ceph-mon[86183]: fsmap cephfs:1/1 {0=cephfs.smithi169.bxitzk=up:replay} 2 up:standby 2024-03-04T13:53:31.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:30 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi169.jusudl"}]: dispatch 2024-03-04T13:53:31.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:30 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi169.jusudl"}]: dispatch 2024-03-04T13:53:32.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:31 smithi107 ceph-mon[112578]: pgmap v221: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 819 KiB/s rd, 0 op/s 2024-03-04T13:53:32.113 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:31 smithi169 ceph-mon[86183]: pgmap v221: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 819 KiB/s rd, 0 op/s 2024-03-04T13:53:34.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:33 smithi107 ceph-mon[112578]: pgmap v222: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 12 MiB/s rd, 4 op/s 2024-03-04T13:53:34.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:33 smithi169 ceph-mon[86183]: pgmap v222: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 12 MiB/s rd, 4 op/s 2024-03-04T13:53:36.095 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:35 smithi107 ceph-mon[112578]: pgmap v223: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 12 MiB/s rd, 4 op/s 2024-03-04T13:53:36.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:35 smithi169 ceph-mon[86183]: pgmap v223: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 12 MiB/s rd, 4 op/s 2024-03-04T13:53:38.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:37 smithi169 ceph-mon[86183]: pgmap v224: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 16 MiB/s rd, 5 op/s 2024-03-04T13:53:38.332 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:37 smithi107 ceph-mon[112578]: pgmap v224: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 16 MiB/s rd, 5 op/s 2024-03-04T13:53:39.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:38 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:53:39.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:38 smithi169 ceph-mon[86183]: mds.? [v2:172.21.15.169:6824/1336978861,v1:172.21.15.169:6825/1336978861] up:boot 2024-03-04T13:53:39.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:38 smithi169 ceph-mon[86183]: fsmap cephfs:1/1 {0=cephfs.smithi169.bxitzk=up:replay} 3 up:standby 2024-03-04T13:53:39.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:38 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi169.jusudl"}]: dispatch 2024-03-04T13:53:39.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:38 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:53:39.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:38 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:53:39.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:38 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:53:39.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:38 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:53:39.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:38 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:53:39.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:38 smithi107 ceph-mon[112578]: mds.? [v2:172.21.15.169:6824/1336978861,v1:172.21.15.169:6825/1336978861] up:boot 2024-03-04T13:53:39.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:38 smithi107 ceph-mon[112578]: fsmap cephfs:1/1 {0=cephfs.smithi169.bxitzk=up:replay} 3 up:standby 2024-03-04T13:53:39.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:38 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi169.jusudl"}]: dispatch 2024-03-04T13:53:39.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:38 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:53:39.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:38 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:53:39.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:38 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:53:39.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:38 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:53:40.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:39 smithi169 ceph-mon[86183]: pgmap v225: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 24 MiB/s rd, 7 op/s 2024-03-04T13:53:40.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:39 smithi169 ceph-mon[86183]: mds.? [v2:172.21.15.169:6826/3696090493,v1:172.21.15.169:6827/3696090493] up:reconnect 2024-03-04T13:53:40.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:39 smithi169 ceph-mon[86183]: fsmap cephfs:1/1 {0=cephfs.smithi169.bxitzk=up:reconnect} 3 up:standby 2024-03-04T13:53:40.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:39 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:53:40.113 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:39 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:53:40.348 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:39 smithi107 ceph-mon[112578]: pgmap v225: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 24 MiB/s rd, 7 op/s 2024-03-04T13:53:40.348 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:39 smithi107 ceph-mon[112578]: mds.? [v2:172.21.15.169:6826/3696090493,v1:172.21.15.169:6827/3696090493] up:reconnect 2024-03-04T13:53:40.348 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:39 smithi107 ceph-mon[112578]: fsmap cephfs:1/1 {0=cephfs.smithi169.bxitzk=up:reconnect} 3 up:standby 2024-03-04T13:53:40.348 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:39 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:53:40.348 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:39 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:53:41.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:40 smithi169 ceph-mon[86183]: reconnect by client.24293 192.168.0.1:0/2380524016 after 0.00299993 2024-03-04T13:53:41.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:40 smithi169 ceph-mon[86183]: reconnect by client.14524 192.168.0.1:0/317786685 after 0.00399991 2024-03-04T13:53:41.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:40 smithi169 ceph-mon[86183]: mds.? [v2:172.21.15.169:6826/3696090493,v1:172.21.15.169:6827/3696090493] up:rejoin 2024-03-04T13:53:41.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:40 smithi169 ceph-mon[86183]: fsmap cephfs:1/1 {0=cephfs.smithi169.bxitzk=up:rejoin} 3 up:standby 2024-03-04T13:53:41.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:40 smithi169 ceph-mon[86183]: daemon mds.cephfs.smithi169.bxitzk is now active in filesystem cephfs as rank 0 2024-03-04T13:53:41.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:40 smithi107 ceph-mon[112578]: reconnect by client.24293 192.168.0.1:0/2380524016 after 0.00299993 2024-03-04T13:53:41.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:40 smithi107 ceph-mon[112578]: reconnect by client.14524 192.168.0.1:0/317786685 after 0.00399991 2024-03-04T13:53:41.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:40 smithi107 ceph-mon[112578]: mds.? [v2:172.21.15.169:6826/3696090493,v1:172.21.15.169:6827/3696090493] up:rejoin 2024-03-04T13:53:41.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:40 smithi107 ceph-mon[112578]: fsmap cephfs:1/1 {0=cephfs.smithi169.bxitzk=up:rejoin} 3 up:standby 2024-03-04T13:53:41.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:40 smithi107 ceph-mon[112578]: daemon mds.cephfs.smithi169.bxitzk is now active in filesystem cephfs as rank 0 2024-03-04T13:53:42.216 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:41 smithi169 ceph-mon[86183]: pgmap v226: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 6 op/s 2024-03-04T13:53:42.216 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:41 smithi169 ceph-mon[86183]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-04T13:53:42.216 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:41 smithi169 ceph-mon[86183]: mds.? [v2:172.21.15.169:6826/3696090493,v1:172.21.15.169:6827/3696090493] up:active 2024-03-04T13:53:42.216 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:41 smithi169 ceph-mon[86183]: fsmap cephfs:1 {0=cephfs.smithi169.bxitzk=up:active} 3 up:standby 2024-03-04T13:53:42.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:41 smithi107 ceph-mon[112578]: pgmap v226: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 6 op/s 2024-03-04T13:53:42.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:41 smithi107 ceph-mon[112578]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-04T13:53:42.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:41 smithi107 ceph-mon[112578]: mds.? [v2:172.21.15.169:6826/3696090493,v1:172.21.15.169:6827/3696090493] up:active 2024-03-04T13:53:42.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:41 smithi107 ceph-mon[112578]: fsmap cephfs:1 {0=cephfs.smithi169.bxitzk=up:active} 3 up:standby 2024-03-04T13:53:42.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:42.678+0000 7f8946600700 1 -- 172.21.15.107:0/933645128 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89400fe800 msgr2=0x7f8940106f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:53:42.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:42.678+0000 7f8946600700 1 --2- 172.21.15.107:0/933645128 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89400fe800 0x7f8940106f60 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f892c004940 tx=0x7f892c009e30 comp rx=0 tx=0).stop 2024-03-04T13:53:42.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:42.678+0000 7f8946600700 1 -- 172.21.15.107:0/933645128 shutdown_connections 2024-03-04T13:53:42.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:42.678+0000 7f8946600700 1 --2- 172.21.15.107:0/933645128 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89400fe800 0x7f8940106f60 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:53:42.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:42.678+0000 7f8946600700 1 --2- 172.21.15.107:0/933645128 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f89400fdef0 0x7f89400fe2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:53:42.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:42.678+0000 7f8946600700 1 -- 172.21.15.107:0/933645128 >> 172.21.15.107:0/933645128 conn(0x7f89400f7970 msgr2=0x7f89400f9d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:53:42.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:42.678+0000 7f8946600700 1 -- 172.21.15.107:0/933645128 shutdown_connections 2024-03-04T13:53:42.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:42.678+0000 7f8946600700 1 -- 172.21.15.107:0/933645128 wait complete. 2024-03-04T13:53:42.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:42.679+0000 7f8946600700 1 Processor -- start 2024-03-04T13:53:42.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:42.679+0000 7f8946600700 1 -- start start 2024-03-04T13:53:42.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:42.679+0000 7f8946600700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f89400fdef0 0x7f89400fbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:53:42.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:42.679+0000 7f8946600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89400fe800 0x7f89400fc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:53:42.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:42.679+0000 7f8946600700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89400fc8f0 con 0x7f89400fe800 2024-03-04T13:53:42.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:42.679+0000 7f8946600700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89400fca60 con 0x7f89400fdef0 2024-03-04T13:53:42.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:42.680+0000 7f893ea00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89400fe800 0x7f89400fc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:53:42.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:42.680+0000 7f893ea00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89400fe800 0x7f89400fc2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46378/0 (socket says 172.21.15.107:46378) 2024-03-04T13:53:42.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:42.680+0000 7f893ea00700 1 -- 172.21.15.107:0/386092930 learned_addr learned my addr 172.21.15.107:0/386092930 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:53:42.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:42.680+0000 7f893f400700 1 --2- 172.21.15.107:0/386092930 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f89400fdef0 0x7f89400fbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:53:42.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:42.680+0000 7f893ea00700 1 -- 172.21.15.107:0/386092930 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f89400fdef0 msgr2=0x7f89400fbdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:53:42.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:42.680+0000 7f893ea00700 1 --2- 172.21.15.107:0/386092930 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f89400fdef0 0x7f89400fbdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:53:42.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:42.680+0000 7f893ea00700 1 -- 172.21.15.107:0/386092930 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f892c005040 con 0x7f89400fe800 2024-03-04T13:53:42.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:42.681+0000 7f893ea00700 1 --2- 172.21.15.107:0/386092930 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89400fe800 0x7f89400fc2f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f892c007f50 tx=0x7f892c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:53:42.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:42.681+0000 7f8937400700 1 -- 172.21.15.107:0/386092930 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f892c021070 con 0x7f89400fe800 2024-03-04T13:53:42.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:42.681+0000 7f8937400700 1 -- 172.21.15.107:0/386092930 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f892c013dd0 con 0x7f89400fe800 2024-03-04T13:53:42.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:42.681+0000 7f8946600700 1 -- 172.21.15.107:0/386092930 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8940119b30 con 0x7f89400fe800 2024-03-04T13:53:42.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:42.681+0000 7f8937400700 1 -- 172.21.15.107:0/386092930 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f892c00e070 con 0x7f89400fe800 2024-03-04T13:53:42.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:42.681+0000 7f8946600700 1 -- 172.21.15.107:0/386092930 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8940119f90 con 0x7f89400fe800 2024-03-04T13:53:42.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:42.683+0000 7f8946600700 1 -- 172.21.15.107:0/386092930 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8940104600 con 0x7f89400fe800 2024-03-04T13:53:42.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:42.684+0000 7f8937400700 1 -- 172.21.15.107:0/386092930 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f892c00b7b0 con 0x7f89400fe800 2024-03-04T13:53:42.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:42.685+0000 7f8937400700 1 --2- 172.21.15.107:0/386092930 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f892406d7e0 0x7f892406fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:53:42.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:42.685+0000 7f8937400700 1 -- 172.21.15.107:0/386092930 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f892c08e480 con 0x7f89400fe800 2024-03-04T13:53:42.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:42.685+0000 7f893f400700 1 --2- 172.21.15.107:0/386092930 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f892406d7e0 0x7f892406fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:53:42.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:42.686+0000 7f893f400700 1 --2- 172.21.15.107:0/386092930 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f892406d7e0 0x7f892406fc90 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8930004600 tx=0x7f893000b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:53:42.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:42.690+0000 7f8937400700 1 -- 172.21.15.107:0/386092930 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f892c01cb60 con 0x7f89400fe800 2024-03-04T13:53:42.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:42.932+0000 7f8946600700 1 -- 172.21.15.107:0/386092930 --> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f894004bf40 con 0x7f892406d7e0 2024-03-04T13:53:42.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:42.935+0000 7f8937400700 1 -- 172.21.15.107:0/386092930 <== mgr.34102 v2:172.21.15.107:6800/1223039812 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f894004bf40 con 0x7f892406d7e0 2024-03-04T13:53:42.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:42.938+0000 7f8946600700 1 -- 172.21.15.107:0/386092930 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f892406d7e0 msgr2=0x7f892406fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:53:42.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:42.938+0000 7f8946600700 1 --2- 172.21.15.107:0/386092930 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f892406d7e0 0x7f892406fc90 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8930004600 tx=0x7f893000b040 comp rx=0 tx=0).stop 2024-03-04T13:53:42.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:42.938+0000 7f8946600700 1 -- 172.21.15.107:0/386092930 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89400fe800 msgr2=0x7f89400fc2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:53:42.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:42.938+0000 7f8946600700 1 --2- 172.21.15.107:0/386092930 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89400fe800 0x7f89400fc2f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f892c007f50 tx=0x7f892c009e30 comp rx=0 tx=0).stop 2024-03-04T13:53:42.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:42.938+0000 7f8946600700 1 -- 172.21.15.107:0/386092930 shutdown_connections 2024-03-04T13:53:42.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:42.938+0000 7f8946600700 1 --2- 172.21.15.107:0/386092930 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f892406d7e0 0x7f892406fc90 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:53:42.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:42.938+0000 7f8946600700 1 --2- 172.21.15.107:0/386092930 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f89400fdef0 0x7f89400fbdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:53:42.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:42.938+0000 7f8946600700 1 --2- 172.21.15.107:0/386092930 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89400fe800 0x7f89400fc2f0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:53:42.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:42.938+0000 7f8946600700 1 -- 172.21.15.107:0/386092930 >> 172.21.15.107:0/386092930 conn(0x7f89400f7970 msgr2=0x7f8940101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:53:42.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:42.938+0000 7f8946600700 1 -- 172.21.15.107:0/386092930 shutdown_connections 2024-03-04T13:53:42.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:42.938+0000 7f8946600700 1 -- 172.21.15.107:0/386092930 wait complete. 2024-03-04T13:53:42.956 INFO:teuthology.orchestra.run.smithi107.stdout:true 2024-03-04T13:53:43.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.092+0000 7f63a5e00700 1 -- 172.21.15.107:0/466099630 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63a01038f0 msgr2=0x7f63a00fcf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:53:43.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.092+0000 7f63a5e00700 1 --2- 172.21.15.107:0/466099630 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63a01038f0 0x7f63a00fcf60 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f638c004940 tx=0x7f638c009e30 comp rx=0 tx=0).stop 2024-03-04T13:53:43.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.092+0000 7f63a5e00700 1 -- 172.21.15.107:0/466099630 shutdown_connections 2024-03-04T13:53:43.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.092+0000 7f63a5e00700 1 --2- 172.21.15.107:0/466099630 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63a01038f0 0x7f63a00fcf60 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:53:43.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.092+0000 7f63a5e00700 1 --2- 172.21.15.107:0/466099630 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63a0102f50 0x7f63a0103320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:53:43.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.092+0000 7f63a5e00700 1 -- 172.21.15.107:0/466099630 >> 172.21.15.107:0/466099630 conn(0x7f63a00f8ae0 msgr2=0x7f63a00faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:53:43.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.092+0000 7f63a5e00700 1 -- 172.21.15.107:0/466099630 shutdown_connections 2024-03-04T13:53:43.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.092+0000 7f63a5e00700 1 -- 172.21.15.107:0/466099630 wait complete. 2024-03-04T13:53:43.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.093+0000 7f63a5e00700 1 Processor -- start 2024-03-04T13:53:43.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.093+0000 7f63a5e00700 1 -- start start 2024-03-04T13:53:43.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.093+0000 7f63a5e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63a0102f50 0x7f63a0115ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:53:43.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.093+0000 7f63a5e00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63a01038f0 0x7f63a0116220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:53:43.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.093+0000 7f63a5e00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63a0116870 con 0x7f63a0102f50 2024-03-04T13:53:43.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.093+0000 7f63a5e00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63a010fb80 con 0x7f63a01038f0 2024-03-04T13:53:43.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.094+0000 7f639ea00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63a0102f50 0x7f63a0115ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:53:43.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.094+0000 7f639ea00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63a0102f50 0x7f63a0115ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46396/0 (socket says 172.21.15.107:46396) 2024-03-04T13:53:43.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.094+0000 7f639ea00700 1 -- 172.21.15.107:0/1170555870 learned_addr learned my addr 172.21.15.107:0/1170555870 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:53:43.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.094+0000 7f639e000700 1 --2- 172.21.15.107:0/1170555870 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63a01038f0 0x7f63a0116220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:53:43.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.094+0000 7f639ea00700 1 -- 172.21.15.107:0/1170555870 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63a01038f0 msgr2=0x7f63a0116220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:53:43.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.094+0000 7f639ea00700 1 --2- 172.21.15.107:0/1170555870 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63a01038f0 0x7f63a0116220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:53:43.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.094+0000 7f639ea00700 1 -- 172.21.15.107:0/1170555870 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f638c005040 con 0x7f63a0102f50 2024-03-04T13:53:43.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.094+0000 7f639ea00700 1 --2- 172.21.15.107:0/1170555870 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63a0102f50 0x7f63a0115ce0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6390002a50 tx=0x7f63900046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:53:43.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.095+0000 7f6396a00700 1 -- 172.21.15.107:0/1170555870 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6390007cc0 con 0x7f63a0102f50 2024-03-04T13:53:43.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.095+0000 7f6396a00700 1 -- 172.21.15.107:0/1170555870 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f639000f040 con 0x7f63a0102f50 2024-03-04T13:53:43.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.095+0000 7f6396a00700 1 -- 172.21.15.107:0/1170555870 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6390009640 con 0x7f63a0102f50 2024-03-04T13:53:43.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.095+0000 7f63a5e00700 1 -- 172.21.15.107:0/1170555870 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63a010fe60 con 0x7f63a0102f50 2024-03-04T13:53:43.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.095+0000 7f63a5e00700 1 -- 172.21.15.107:0/1170555870 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63a01103a0 con 0x7f63a0102f50 2024-03-04T13:53:43.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.096+0000 7f63a5e00700 1 -- 172.21.15.107:0/1170555870 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63a00fc6e0 con 0x7f63a0102f50 2024-03-04T13:53:43.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.097+0000 7f6396a00700 1 -- 172.21.15.107:0/1170555870 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f63900097a0 con 0x7f63a0102f50 2024-03-04T13:53:43.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.098+0000 7f6396a00700 1 --2- 172.21.15.107:0/1170555870 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f638406d7e0 0x7f638406fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:53:43.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.098+0000 7f6396a00700 1 -- 172.21.15.107:0/1170555870 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f639008cc80 con 0x7f63a0102f50 2024-03-04T13:53:43.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.098+0000 7f639e000700 1 --2- 172.21.15.107:0/1170555870 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f638406d7e0 0x7f638406fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:53:43.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.099+0000 7f639e000700 1 --2- 172.21.15.107:0/1170555870 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f638406d7e0 0x7f638406fc90 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f638c004f90 tx=0x7f638c009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:53:43.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.103+0000 7f6396a00700 1 -- 172.21.15.107:0/1170555870 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f639005a000 con 0x7f63a0102f50 2024-03-04T13:53:43.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.342+0000 7f63a5e00700 1 -- 172.21.15.107:0/1170555870 --> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f63a01113d0 con 0x7f638406d7e0 2024-03-04T13:53:43.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.345+0000 7f6396a00700 1 -- 172.21.15.107:0/1170555870 <== mgr.34102 v2:172.21.15.107:6800/1223039812 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f63a01113d0 con 0x7f638406d7e0 2024-03-04T13:53:43.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.348+0000 7f63a5e00700 1 -- 172.21.15.107:0/1170555870 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f638406d7e0 msgr2=0x7f638406fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:53:43.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.348+0000 7f63a5e00700 1 --2- 172.21.15.107:0/1170555870 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f638406d7e0 0x7f638406fc90 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f638c004f90 tx=0x7f638c009e30 comp rx=0 tx=0).stop 2024-03-04T13:53:43.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.348+0000 7f63a5e00700 1 -- 172.21.15.107:0/1170555870 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63a0102f50 msgr2=0x7f63a0115ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:53:43.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.348+0000 7f63a5e00700 1 --2- 172.21.15.107:0/1170555870 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63a0102f50 0x7f63a0115ce0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6390002a50 tx=0x7f63900046b0 comp rx=0 tx=0).stop 2024-03-04T13:53:43.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.348+0000 7f63a5e00700 1 -- 172.21.15.107:0/1170555870 shutdown_connections 2024-03-04T13:53:43.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.348+0000 7f63a5e00700 1 --2- 172.21.15.107:0/1170555870 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f638406d7e0 0x7f638406fc90 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:53:43.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.348+0000 7f63a5e00700 1 --2- 172.21.15.107:0/1170555870 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63a0102f50 0x7f63a0115ce0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:53:43.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.348+0000 7f63a5e00700 1 --2- 172.21.15.107:0/1170555870 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63a01038f0 0x7f63a0116220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:53:43.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.348+0000 7f63a5e00700 1 -- 172.21.15.107:0/1170555870 >> 172.21.15.107:0/1170555870 conn(0x7f63a00f8ae0 msgr2=0x7f63a00fa490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:53:43.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.348+0000 7f63a5e00700 1 -- 172.21.15.107:0/1170555870 shutdown_connections 2024-03-04T13:53:43.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.349+0000 7f63a5e00700 1 -- 172.21.15.107:0/1170555870 wait complete. 2024-03-04T13:53:43.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.501+0000 7f9ef5800700 1 -- 172.21.15.107:0/1784308063 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ef00fbdd0 msgr2=0x7f9ef00fc1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:53:43.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.501+0000 7f9ef5800700 1 --2- 172.21.15.107:0/1784308063 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ef00fbdd0 0x7f9ef00fc1a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9ee0004940 tx=0x7f9ee0009e30 comp rx=0 tx=0).stop 2024-03-04T13:53:43.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.501+0000 7f9ef5800700 1 -- 172.21.15.107:0/1784308063 shutdown_connections 2024-03-04T13:53:43.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.501+0000 7f9ef5800700 1 --2- 172.21.15.107:0/1784308063 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ef00fc6e0 0x7f9ef0109440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:53:43.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.501+0000 7f9ef5800700 1 --2- 172.21.15.107:0/1784308063 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ef00fbdd0 0x7f9ef00fc1a0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:53:43.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.501+0000 7f9ef5800700 1 -- 172.21.15.107:0/1784308063 >> 172.21.15.107:0/1784308063 conn(0x7f9ef0071d90 msgr2=0x7f9ef0072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:53:43.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.501+0000 7f9ef5800700 1 -- 172.21.15.107:0/1784308063 shutdown_connections 2024-03-04T13:53:43.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.501+0000 7f9ef5800700 1 -- 172.21.15.107:0/1784308063 wait complete. 2024-03-04T13:53:43.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.502+0000 7f9ef5800700 1 Processor -- start 2024-03-04T13:53:43.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.502+0000 7f9ef5800700 1 -- start start 2024-03-04T13:53:43.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.503+0000 7f9ef5800700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ef00fbdd0 0x7f9ef0115c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:53:43.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.503+0000 7f9ef5800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ef00fc6e0 0x7f9ef0116160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:53:43.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.503+0000 7f9ef5800700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ef0116840 con 0x7f9ef00fc6e0 2024-03-04T13:53:43.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.503+0000 7f9ef5800700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ef010fae0 con 0x7f9ef00fbdd0 2024-03-04T13:53:43.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.503+0000 7f9ee6000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ef00fc6e0 0x7f9ef0116160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:53:43.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.503+0000 7f9ee6000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ef00fc6e0 0x7f9ef0116160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46414/0 (socket says 172.21.15.107:46414) 2024-03-04T13:53:43.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.503+0000 7f9ee6000700 1 -- 172.21.15.107:0/557993862 learned_addr learned my addr 172.21.15.107:0/557993862 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:53:43.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.503+0000 7f9eeea00700 1 --2- 172.21.15.107:0/557993862 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ef00fbdd0 0x7f9ef0115c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:53:43.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.504+0000 7f9ee6000700 1 -- 172.21.15.107:0/557993862 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ef00fbdd0 msgr2=0x7f9ef0115c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:53:43.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.504+0000 7f9ee6000700 1 --2- 172.21.15.107:0/557993862 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ef00fbdd0 0x7f9ef0115c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:53:43.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.504+0000 7f9ee6000700 1 -- 172.21.15.107:0/557993862 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ee0005040 con 0x7f9ef00fc6e0 2024-03-04T13:53:43.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.504+0000 7f9ee6000700 1 --2- 172.21.15.107:0/557993862 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ef00fc6e0 0x7f9ef0116160 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9ed800b790 tx=0x7f9ed8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:53:43.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.504+0000 7f9ee7400700 1 -- 172.21.15.107:0/557993862 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9ed800f770 con 0x7f9ef00fc6e0 2024-03-04T13:53:43.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.504+0000 7f9ee7400700 1 -- 172.21.15.107:0/557993862 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9ed800fda0 con 0x7f9ef00fc6e0 2024-03-04T13:53:43.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.504+0000 7f9ef5800700 1 -- 172.21.15.107:0/557993862 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ef010fdc0 con 0x7f9ef00fc6e0 2024-03-04T13:53:43.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.504+0000 7f9ee7400700 1 -- 172.21.15.107:0/557993862 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9ed800d310 con 0x7f9ef00fc6e0 2024-03-04T13:53:43.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.504+0000 7f9ef5800700 1 -- 172.21.15.107:0/557993862 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ef0110300 con 0x7f9ef00fc6e0 2024-03-04T13:53:43.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.506+0000 7f9ef5800700 1 -- 172.21.15.107:0/557993862 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ef0106b70 con 0x7f9ef00fc6e0 2024-03-04T13:53:43.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.507+0000 7f9ee7400700 1 -- 172.21.15.107:0/557993862 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f9ed8010420 con 0x7f9ef00fc6e0 2024-03-04T13:53:43.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.508+0000 7f9ee7400700 1 --2- 172.21.15.107:0/557993862 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f9ed406d7f0 0x7f9ed406fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:53:43.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.509+0000 7f9ee7400700 1 -- 172.21.15.107:0/557993862 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f9ed808cef0 con 0x7f9ef00fc6e0 2024-03-04T13:53:43.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.509+0000 7f9eeea00700 1 --2- 172.21.15.107:0/557993862 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f9ed406d7f0 0x7f9ed406fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:53:43.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.510+0000 7f9eeea00700 1 --2- 172.21.15.107:0/557993862 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f9ed406d7f0 0x7f9ed406fca0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9ee000b3b0 tx=0x7f9ee0009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:53:43.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.514+0000 7f9ee7400700 1 -- 172.21.15.107:0/557993862 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9ed805a2d0 con 0x7f9ef00fc6e0 2024-03-04T13:53:43.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.744+0000 7f9ef5800700 1 -- 172.21.15.107:0/557993862 --> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f9ef010c2c0 con 0x7f9ed406d7f0 2024-03-04T13:53:43.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.773+0000 7f9ee7400700 1 -- 172.21.15.107:0/557993862 <== mgr.34102 v2:172.21.15.107:6800/1223039812 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f9ef010c2c0 con 0x7f9ed406d7f0 2024-03-04T13:53:43.775 INFO:teuthology.orchestra.run.smithi107.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-04T13:53:43.775 INFO:teuthology.orchestra.run.smithi107.stdout:alertmanager.smithi107 smithi107 *:9093,9094 running (8m) 27s ago 22m 14.4M - 0.25.0 c8568f914cd2 e5ed7be32538 2024-03-04T13:53:43.775 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi107 smithi107 running (4m) 27s ago 22m 6660k - 19.0.0-1845-g717ce3cc d71b839abdf8 0558022b0d84 2024-03-04T13:53:43.775 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi169 smithi169 running (4m) 1s ago 21m 6665k - 19.0.0-1845-g717ce3cc d71b839abdf8 221ca53aa0d3 2024-03-04T13:53:43.775 INFO:teuthology.orchestra.run.smithi107.stdout:grafana.smithi107 smithi107 *:3000 running (7m) 27s ago 21m 75.1M - 9.4.12 a0ea5e3c46d9 c5e4d33fb599 2024-03-04T13:53:43.775 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.riexmm smithi107 running (58s) 27s ago 15m 11.0M - 19.0.0-1845-g717ce3cc d71b839abdf8 8582388ad90a 2024-03-04T13:53:43.776 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.thycho smithi107 running (34s) 27s ago 15m 10.8M - 19.0.0-1845-g717ce3cc d71b839abdf8 f4b3cea14a6a 2024-03-04T13:53:43.776 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi169.bxitzk smithi169 running (14m) 1s ago 14m 143M - 17.2.7-545-g42608bb1 9eec3f22859d 7d4115228c7c 2024-03-04T13:53:43.776 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi169.jusudl smithi169 running (6s) 1s ago 15m 10.8M - 19.0.0-1845-g717ce3cc d71b839abdf8 e6d45009ccbb 2024-03-04T13:53:43.776 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi107.wpzgwl smithi107 *:8443,9283,8765 running (10m) 27s ago 23m 509M - 19.0.0-1845-g717ce3cc d71b839abdf8 1c5fdee45eac 2024-03-04T13:53:43.776 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi169.eihczr smithi169 *:8443,9283,8765 running (9m) 1s ago 20m 452M - 19.0.0-1845-g717ce3cc d71b839abdf8 b5cac2b7d22a 2024-03-04T13:53:43.776 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi107 smithi107 running (6m) 27s ago 24m 52.5M 2048M 19.0.0-1845-g717ce3cc d71b839abdf8 32f07144274f 2024-03-04T13:53:43.776 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi169 smithi169 running (5m) 1s ago 20m 40.5M 2048M 19.0.0-1845-g717ce3cc d71b839abdf8 e3527449381a 2024-03-04T13:53:43.776 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi107 smithi107 *:9100 running (9m) 27s ago 22m 8102k - 1.5.0 0da6a335fe13 8a8d3d4f724f 2024-03-04T13:53:43.776 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi169 smithi169 *:9100 running (8m) 1s ago 20m 8196k - 1.5.0 0da6a335fe13 f240f5a975bc 2024-03-04T13:53:43.776 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0 smithi107 running (4m) 27s ago 19m 85.3M 1558M 19.0.0-1845-g717ce3cc d71b839abdf8 5801a5540dc6 2024-03-04T13:53:43.776 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1 smithi107 running (3m) 27s ago 19m 94.0M 1558M 19.0.0-1845-g717ce3cc d71b839abdf8 b614af658854 2024-03-04T13:53:43.776 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2 smithi107 running (3m) 27s ago 18m 84.9M 1558M 19.0.0-1845-g717ce3cc d71b839abdf8 6e4da4f6f146 2024-03-04T13:53:43.776 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3 smithi169 running (2m) 1s ago 18m 104M 2582M 19.0.0-1845-g717ce3cc d71b839abdf8 d31baf764d7a 2024-03-04T13:53:43.776 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4 smithi169 running (2m) 1s ago 17m 88.6M 2582M 19.0.0-1845-g717ce3cc d71b839abdf8 a70bacd97639 2024-03-04T13:53:43.776 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5 smithi169 running (83s) 1s ago 16m 84.2M 2582M 19.0.0-1845-g717ce3cc d71b839abdf8 5f6e00f7993d 2024-03-04T13:53:43.776 INFO:teuthology.orchestra.run.smithi107.stdout:prometheus.smithi107 smithi107 *:9095 running (8m) 27s ago 21m 54.9M - 2.43.0 a07b618ecd1d 1eeb1ad5cb75 2024-03-04T13:53:43.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.775+0000 7f9ef5800700 1 -- 172.21.15.107:0/557993862 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f9ed406d7f0 msgr2=0x7f9ed406fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:53:43.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.775+0000 7f9ef5800700 1 --2- 172.21.15.107:0/557993862 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f9ed406d7f0 0x7f9ed406fca0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9ee000b3b0 tx=0x7f9ee0009e30 comp rx=0 tx=0).stop 2024-03-04T13:53:43.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.775+0000 7f9ef5800700 1 -- 172.21.15.107:0/557993862 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ef00fc6e0 msgr2=0x7f9ef0116160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:53:43.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.775+0000 7f9ef5800700 1 --2- 172.21.15.107:0/557993862 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ef00fc6e0 0x7f9ef0116160 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9ed800b790 tx=0x7f9ed8009250 comp rx=0 tx=0).stop 2024-03-04T13:53:43.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.775+0000 7f9ef5800700 1 -- 172.21.15.107:0/557993862 shutdown_connections 2024-03-04T13:53:43.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.775+0000 7f9ef5800700 1 --2- 172.21.15.107:0/557993862 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f9ed406d7f0 0x7f9ed406fca0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:53:43.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.775+0000 7f9ef5800700 1 --2- 172.21.15.107:0/557993862 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ef00fbdd0 0x7f9ef0115c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:53:43.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.775+0000 7f9ef5800700 1 --2- 172.21.15.107:0/557993862 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ef00fc6e0 0x7f9ef0116160 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:53:43.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.775+0000 7f9ef5800700 1 -- 172.21.15.107:0/557993862 >> 172.21.15.107:0/557993862 conn(0x7f9ef0071d90 msgr2=0x7f9ef00fa920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:53:43.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.776+0000 7f9ef5800700 1 -- 172.21.15.107:0/557993862 shutdown_connections 2024-03-04T13:53:43.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.776+0000 7f9ef5800700 1 -- 172.21.15.107:0/557993862 wait complete. 2024-03-04T13:53:43.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.921+0000 7f96a3c00700 1 -- 172.21.15.107:0/713336985 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f969c102f50 msgr2=0x7f969c103320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:53:43.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.921+0000 7f96a3c00700 1 --2- 172.21.15.107:0/713336985 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f969c102f50 0x7f969c103320 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9690004940 tx=0x7f9690009e30 comp rx=0 tx=0).stop 2024-03-04T13:53:43.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.922+0000 7f96a3c00700 1 -- 172.21.15.107:0/713336985 shutdown_connections 2024-03-04T13:53:43.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.922+0000 7f96a3c00700 1 --2- 172.21.15.107:0/713336985 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f969c1038f0 0x7f969c0fcf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:53:43.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.922+0000 7f96a3c00700 1 --2- 172.21.15.107:0/713336985 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f969c102f50 0x7f969c103320 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:53:43.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.922+0000 7f96a3c00700 1 -- 172.21.15.107:0/713336985 >> 172.21.15.107:0/713336985 conn(0x7f969c0f8ae0 msgr2=0x7f969c0faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:53:43.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.922+0000 7f96a3c00700 1 -- 172.21.15.107:0/713336985 shutdown_connections 2024-03-04T13:53:43.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.922+0000 7f96a3c00700 1 -- 172.21.15.107:0/713336985 wait complete. 2024-03-04T13:53:43.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.922+0000 7f96a3c00700 1 Processor -- start 2024-03-04T13:53:43.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.923+0000 7f96a3c00700 1 -- start start 2024-03-04T13:53:43.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.923+0000 7f96a3c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f969c102f50 0x7f969c070290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:53:43.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.923+0000 7f96a3c00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f969c1038f0 0x7f969c0707d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:53:43.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.923+0000 7f96a3c00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f969c070dd0 con 0x7f969c102f50 2024-03-04T13:53:43.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.923+0000 7f96a3c00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f969c070f40 con 0x7f969c1038f0 2024-03-04T13:53:43.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.923+0000 7f96a1200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f969c102f50 0x7f969c070290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:53:43.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.923+0000 7f96a1200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f969c102f50 0x7f969c070290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46432/0 (socket says 172.21.15.107:46432) 2024-03-04T13:53:43.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.923+0000 7f96a1200700 1 -- 172.21.15.107:0/4022984078 learned_addr learned my addr 172.21.15.107:0/4022984078 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:53:43.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.924+0000 7f96a1200700 1 -- 172.21.15.107:0/4022984078 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f969c1038f0 msgr2=0x7f969c0707d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:53:43.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.924+0000 7f969be00700 1 --2- 172.21.15.107:0/4022984078 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f969c1038f0 0x7f969c0707d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:53:43.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.924+0000 7f96a1200700 1 --2- 172.21.15.107:0/4022984078 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f969c1038f0 0x7f969c0707d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:53:43.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.924+0000 7f96a1200700 1 -- 172.21.15.107:0/4022984078 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9690005040 con 0x7f969c102f50 2024-03-04T13:53:43.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.924+0000 7f96a1200700 1 --2- 172.21.15.107:0/4022984078 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f969c102f50 0x7f969c070290 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9690009150 tx=0x7f9690009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:53:43.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.924+0000 7f9698c00700 1 -- 172.21.15.107:0/4022984078 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f969000e040 con 0x7f969c102f50 2024-03-04T13:53:43.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.924+0000 7f96a3c00700 1 -- 172.21.15.107:0/4022984078 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f969c119c90 con 0x7f969c102f50 2024-03-04T13:53:43.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.924+0000 7f9698c00700 1 -- 172.21.15.107:0/4022984078 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f969000bbf0 con 0x7f969c102f50 2024-03-04T13:53:43.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.924+0000 7f9698c00700 1 -- 172.21.15.107:0/4022984078 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f969001c630 con 0x7f969c102f50 2024-03-04T13:53:43.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.924+0000 7f96a3c00700 1 -- 172.21.15.107:0/4022984078 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f969c11a0f0 con 0x7f969c102f50 2024-03-04T13:53:43.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.926+0000 7f96a3c00700 1 -- 172.21.15.107:0/4022984078 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f969c0fc6e0 con 0x7f969c102f50 2024-03-04T13:53:43.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.927+0000 7f9698c00700 1 -- 172.21.15.107:0/4022984078 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f969000bd60 con 0x7f969c102f50 2024-03-04T13:53:43.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.927+0000 7f9698c00700 1 --2- 172.21.15.107:0/4022984078 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f968406d7f0 0x7f968406fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:53:43.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.928+0000 7f9698c00700 1 -- 172.21.15.107:0/4022984078 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f969008e1e0 con 0x7f969c102f50 2024-03-04T13:53:43.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.928+0000 7f969be00700 1 --2- 172.21.15.107:0/4022984078 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f968406d7f0 0x7f968406fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:53:43.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.929+0000 7f969be00700 1 --2- 172.21.15.107:0/4022984078 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f968406d7f0 0x7f968406fca0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f969c071c90 tx=0x7f9688009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:53:43.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:43.932+0000 7f9698c00700 1 -- 172.21.15.107:0/4022984078 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9690057b50 con 0x7f969c102f50 2024-03-04T13:53:44.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:43 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:53:44.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:43 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:53:44.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:43 smithi107 ceph-mon[112578]: pgmap v227: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 20 MiB/s rd, 255 B/s wr, 8 op/s 2024-03-04T13:53:44.095 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:43 smithi107 ceph-mon[112578]: from='client.34468 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:53:44.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:43 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:53:44.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:43 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:53:44.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:43 smithi169 ceph-mon[86183]: pgmap v227: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 20 MiB/s rd, 255 B/s wr, 8 op/s 2024-03-04T13:53:44.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:43 smithi169 ceph-mon[86183]: from='client.34468 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:53:44.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:44.229+0000 7f96a3c00700 1 -- 172.21.15.107:0/4022984078 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f969c071ad0 con 0x7f969c102f50 2024-03-04T13:53:44.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:44.230+0000 7f9698c00700 1 -- 172.21.15.107:0/4022984078 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7f969005b210 con 0x7f969c102f50 2024-03-04T13:53:44.232 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-03-04T13:53:44.232 INFO:teuthology.orchestra.run.smithi107.stdout: "mon": { 2024-03-04T13:53:44.232 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-1845-g717ce3cc (717ce3cce8d0166e4e577faa8004e6f9cb4128c0) squid (dev)": 2 2024-03-04T13:53:44.232 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:53:44.232 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr": { 2024-03-04T13:53:44.232 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-1845-g717ce3cc (717ce3cce8d0166e4e577faa8004e6f9cb4128c0) squid (dev)": 2 2024-03-04T13:53:44.233 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:53:44.233 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": { 2024-03-04T13:53:44.233 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-1845-g717ce3cc (717ce3cce8d0166e4e577faa8004e6f9cb4128c0) squid (dev)": 6 2024-03-04T13:53:44.233 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:53:44.233 INFO:teuthology.orchestra.run.smithi107.stdout: "mds": { 2024-03-04T13:53:44.233 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-545-g42608bb1 (42608bb191117be10dba720e6bf8a02f3171cd34) quincy (stable)": 1, 2024-03-04T13:53:44.233 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-1845-g717ce3cc (717ce3cce8d0166e4e577faa8004e6f9cb4128c0) squid (dev)": 3 2024-03-04T13:53:44.233 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:53:44.233 INFO:teuthology.orchestra.run.smithi107.stdout: "overall": { 2024-03-04T13:53:44.233 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-545-g42608bb1 (42608bb191117be10dba720e6bf8a02f3171cd34) quincy (stable)": 1, 2024-03-04T13:53:44.233 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-1845-g717ce3cc (717ce3cce8d0166e4e577faa8004e6f9cb4128c0) squid (dev)": 13 2024-03-04T13:53:44.233 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-04T13:53:44.233 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-03-04T13:53:44.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:44.233+0000 7f96a3c00700 1 -- 172.21.15.107:0/4022984078 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f968406d7f0 msgr2=0x7f968406fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:53:44.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:44.233+0000 7f96a3c00700 1 --2- 172.21.15.107:0/4022984078 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f968406d7f0 0x7f968406fca0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f969c071c90 tx=0x7f9688009040 comp rx=0 tx=0).stop 2024-03-04T13:53:44.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:44.233+0000 7f96a3c00700 1 -- 172.21.15.107:0/4022984078 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f969c102f50 msgr2=0x7f969c070290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:53:44.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:44.233+0000 7f96a3c00700 1 --2- 172.21.15.107:0/4022984078 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f969c102f50 0x7f969c070290 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9690009150 tx=0x7f9690009e30 comp rx=0 tx=0).stop 2024-03-04T13:53:44.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:44.233+0000 7f96a3c00700 1 -- 172.21.15.107:0/4022984078 shutdown_connections 2024-03-04T13:53:44.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:44.233+0000 7f96a3c00700 1 --2- 172.21.15.107:0/4022984078 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f968406d7f0 0x7f968406fca0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:53:44.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:44.233+0000 7f96a3c00700 1 --2- 172.21.15.107:0/4022984078 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f969c102f50 0x7f969c070290 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:53:44.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:44.233+0000 7f96a3c00700 1 --2- 172.21.15.107:0/4022984078 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f969c1038f0 0x7f969c0707d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:53:44.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:44.233+0000 7f96a3c00700 1 -- 172.21.15.107:0/4022984078 >> 172.21.15.107:0/4022984078 conn(0x7f969c0f8ae0 msgr2=0x7f969c0fa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:53:44.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:44.233+0000 7f96a3c00700 1 -- 172.21.15.107:0/4022984078 shutdown_connections 2024-03-04T13:53:44.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:44.233+0000 7f96a3c00700 1 -- 172.21.15.107:0/4022984078 wait complete. 2024-03-04T13:53:44.384 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:44.382+0000 7f9d74000700 1 -- 172.21.15.107:0/1950745185 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d6c1095f0 msgr2=0x7f9d6c1099c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:53:44.384 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:44.382+0000 7f9d74000700 1 --2- 172.21.15.107:0/1950745185 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d6c1095f0 0x7f9d6c1099c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9d60004940 tx=0x7f9d60009e30 comp rx=0 tx=0).stop 2024-03-04T13:53:44.384 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:44.382+0000 7f9d74000700 1 -- 172.21.15.107:0/1950745185 shutdown_connections 2024-03-04T13:53:44.384 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:44.382+0000 7f9d74000700 1 --2- 172.21.15.107:0/1950745185 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d6c103dc0 0x7f9d6c104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:53:44.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:44.382+0000 7f9d74000700 1 --2- 172.21.15.107:0/1950745185 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d6c1095f0 0x7f9d6c1099c0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:53:44.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:44.382+0000 7f9d74000700 1 -- 172.21.15.107:0/1950745185 >> 172.21.15.107:0/1950745185 conn(0x7f9d6c072d50 msgr2=0x7f9d6c075160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:53:44.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:44.382+0000 7f9d74000700 1 -- 172.21.15.107:0/1950745185 shutdown_connections 2024-03-04T13:53:44.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:44.382+0000 7f9d74000700 1 -- 172.21.15.107:0/1950745185 wait complete. 2024-03-04T13:53:44.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:44.383+0000 7f9d74000700 1 Processor -- start 2024-03-04T13:53:44.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:44.383+0000 7f9d74000700 1 -- start start 2024-03-04T13:53:44.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:44.384+0000 7f9d74000700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d6c103dc0 0x7f9d6c11a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:53:44.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:44.384+0000 7f9d74000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d6c1095f0 0x7f9d6c11a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:53:44.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:44.384+0000 7f9d74000700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d6c11ac70 con 0x7f9d6c1095f0 2024-03-04T13:53:44.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:44.384+0000 7f9d74000700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d6c113ef0 con 0x7f9d6c103dc0 2024-03-04T13:53:44.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:44.384+0000 7f9d70c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d6c1095f0 0x7f9d6c11a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:53:44.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:44.384+0000 7f9d70c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d6c1095f0 0x7f9d6c11a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46452/0 (socket says 172.21.15.107:46452) 2024-03-04T13:53:44.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:44.384+0000 7f9d70c00700 1 -- 172.21.15.107:0/3218439684 learned_addr learned my addr 172.21.15.107:0/3218439684 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:53:44.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:44.384+0000 7f9d71600700 1 --2- 172.21.15.107:0/3218439684 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d6c103dc0 0x7f9d6c11a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:53:44.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:44.385+0000 7f9d70c00700 1 -- 172.21.15.107:0/3218439684 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d6c103dc0 msgr2=0x7f9d6c11a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:53:44.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:44.385+0000 7f9d70c00700 1 --2- 172.21.15.107:0/3218439684 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d6c103dc0 0x7f9d6c11a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:53:44.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:44.385+0000 7f9d70c00700 1 -- 172.21.15.107:0/3218439684 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d60005040 con 0x7f9d6c1095f0 2024-03-04T13:53:44.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:44.385+0000 7f9d70c00700 1 --2- 172.21.15.107:0/3218439684 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d6c1095f0 0x7f9d6c11a590 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9d5800b790 tx=0x7f9d58009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:53:44.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:44.385+0000 7f9d69600700 1 -- 172.21.15.107:0/3218439684 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9d5800f770 con 0x7f9d6c1095f0 2024-03-04T13:53:44.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:44.385+0000 7f9d69600700 1 -- 172.21.15.107:0/3218439684 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9d5800fda0 con 0x7f9d6c1095f0 2024-03-04T13:53:44.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:44.385+0000 7f9d74000700 1 -- 172.21.15.107:0/3218439684 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d6c114170 con 0x7f9d6c1095f0 2024-03-04T13:53:44.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:44.385+0000 7f9d69600700 1 -- 172.21.15.107:0/3218439684 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9d5800d310 con 0x7f9d6c1095f0 2024-03-04T13:53:44.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:44.385+0000 7f9d74000700 1 -- 172.21.15.107:0/3218439684 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d6c1146b0 con 0x7f9d6c1095f0 2024-03-04T13:53:44.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:44.386+0000 7f9d74000700 1 -- 172.21.15.107:0/3218439684 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d6c075870 con 0x7f9d6c1095f0 2024-03-04T13:53:44.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:44.387+0000 7f9d69600700 1 -- 172.21.15.107:0/3218439684 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f9d58010420 con 0x7f9d6c1095f0 2024-03-04T13:53:44.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:44.389+0000 7f9d69600700 1 --2- 172.21.15.107:0/3218439684 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f9d5406d7f0 0x7f9d5406fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:53:44.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:44.389+0000 7f9d69600700 1 -- 172.21.15.107:0/3218439684 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f9d5808cfb0 con 0x7f9d6c1095f0 2024-03-04T13:53:44.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:44.389+0000 7f9d71600700 1 --2- 172.21.15.107:0/3218439684 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f9d5406d7f0 0x7f9d5406fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:53:44.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:44.390+0000 7f9d71600700 1 --2- 172.21.15.107:0/3218439684 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f9d5406d7f0 0x7f9d5406fca0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f9d6000b020 tx=0x7f9d60009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:53:44.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:44.394+0000 7f9d69600700 1 -- 172.21.15.107:0/3218439684 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9d580c0d20 con 0x7f9d6c1095f0 2024-03-04T13:53:44.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:44.656+0000 7f9d74000700 1 -- 172.21.15.107:0/3218439684 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f9d6c115630 con 0x7f9d6c1095f0 2024-03-04T13:53:44.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:44.657+0000 7f9d69600700 1 -- 172.21.15.107:0/3218439684 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+1855 (secure 0 0 0) 0x7f9d5805a310 con 0x7f9d6c1095f0 2024-03-04T13:53:44.660 INFO:teuthology.orchestra.run.smithi107.stdout:e28 2024-03-04T13:53:44.660 INFO:teuthology.orchestra.run.smithi107.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-04T13:53:44.660 INFO:teuthology.orchestra.run.smithi107.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-04T13:53:44.660 INFO:teuthology.orchestra.run.smithi107.stdout:legacy client fscid: 1 2024-03-04T13:53:44.660 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:53:44.660 INFO:teuthology.orchestra.run.smithi107.stdout:Filesystem 'cephfs' (1) 2024-03-04T13:53:44.660 INFO:teuthology.orchestra.run.smithi107.stdout:fs_name cephfs 2024-03-04T13:53:44.660 INFO:teuthology.orchestra.run.smithi107.stdout:epoch 28 2024-03-04T13:53:44.661 INFO:teuthology.orchestra.run.smithi107.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-04T13:53:44.661 INFO:teuthology.orchestra.run.smithi107.stdout:created 2024-03-04T13:38:30.854004+0000 2024-03-04T13:53:44.661 INFO:teuthology.orchestra.run.smithi107.stdout:modified 2024-03-04T13:53:40.783016+0000 2024-03-04T13:53:44.661 INFO:teuthology.orchestra.run.smithi107.stdout:tableserver 0 2024-03-04T13:53:44.661 INFO:teuthology.orchestra.run.smithi107.stdout:root 0 2024-03-04T13:53:44.661 INFO:teuthology.orchestra.run.smithi107.stdout:session_timeout 60 2024-03-04T13:53:44.661 INFO:teuthology.orchestra.run.smithi107.stdout:session_autoclose 300 2024-03-04T13:53:44.661 INFO:teuthology.orchestra.run.smithi107.stdout:max_file_size 1099511627776 2024-03-04T13:53:44.661 INFO:teuthology.orchestra.run.smithi107.stdout:max_xattr_size 65536 2024-03-04T13:53:44.661 INFO:teuthology.orchestra.run.smithi107.stdout:required_client_features {} 2024-03-04T13:53:44.661 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure 0 2024-03-04T13:53:44.661 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure_osd_epoch 85 2024-03-04T13:53:44.661 INFO:teuthology.orchestra.run.smithi107.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-04T13:53:44.661 INFO:teuthology.orchestra.run.smithi107.stdout:max_mds 1 2024-03-04T13:53:44.662 INFO:teuthology.orchestra.run.smithi107.stdout:in 0 2024-03-04T13:53:44.662 INFO:teuthology.orchestra.run.smithi107.stdout:up {0=24259} 2024-03-04T13:53:44.662 INFO:teuthology.orchestra.run.smithi107.stdout:failed 2024-03-04T13:53:44.662 INFO:teuthology.orchestra.run.smithi107.stdout:damaged 2024-03-04T13:53:44.662 INFO:teuthology.orchestra.run.smithi107.stdout:stopped 2024-03-04T13:53:44.662 INFO:teuthology.orchestra.run.smithi107.stdout:data_pools [3] 2024-03-04T13:53:44.662 INFO:teuthology.orchestra.run.smithi107.stdout:metadata_pool 2 2024-03-04T13:53:44.662 INFO:teuthology.orchestra.run.smithi107.stdout:inline_data enabled 2024-03-04T13:53:44.662 INFO:teuthology.orchestra.run.smithi107.stdout:balancer 2024-03-04T13:53:44.662 INFO:teuthology.orchestra.run.smithi107.stdout:bal_rank_mask -1 2024-03-04T13:53:44.662 INFO:teuthology.orchestra.run.smithi107.stdout:standby_count_wanted 1 2024-03-04T13:53:44.662 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi169.bxitzk{0:24259} state up:active seq 227 join_fscid=1 addr [v2:172.21.15.169:6826/3696090493,v1:172.21.15.169:6827/3696090493] compat {c=[1],r=[1],i=[7ff]}] 2024-03-04T13:53:44.662 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:53:44.663 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:53:44.663 INFO:teuthology.orchestra.run.smithi107.stdout:Standby daemons: 2024-03-04T13:53:44.663 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:53:44.663 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.riexmm{-1:34432} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.107:6826/1144672731,v1:172.21.15.107:6827/1144672731] compat {c=[1],r=[1],i=[fff]}] 2024-03-04T13:53:44.663 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.thycho{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.107:6828/3010096376,v1:172.21.15.107:6829/3010096376] compat {c=[1],r=[1],i=[fff]}] 2024-03-04T13:53:44.663 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi169.jusudl{-1:44243} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.169:6824/1336978861,v1:172.21.15.169:6825/1336978861] compat {c=[1],r=[1],i=[fff]}] 2024-03-04T13:53:44.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:44.659+0000 7f9d74000700 1 -- 172.21.15.107:0/3218439684 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f9d5406d7f0 msgr2=0x7f9d5406fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:53:44.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:44.659+0000 7f9d74000700 1 --2- 172.21.15.107:0/3218439684 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f9d5406d7f0 0x7f9d5406fca0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f9d6000b020 tx=0x7f9d60009e30 comp rx=0 tx=0).stop 2024-03-04T13:53:44.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:44.659+0000 7f9d74000700 1 -- 172.21.15.107:0/3218439684 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d6c1095f0 msgr2=0x7f9d6c11a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:53:44.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:44.660+0000 7f9d74000700 1 --2- 172.21.15.107:0/3218439684 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d6c1095f0 0x7f9d6c11a590 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9d5800b790 tx=0x7f9d58009250 comp rx=0 tx=0).stop 2024-03-04T13:53:44.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:44.660+0000 7f9d74000700 1 -- 172.21.15.107:0/3218439684 shutdown_connections 2024-03-04T13:53:44.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:44.660+0000 7f9d74000700 1 --2- 172.21.15.107:0/3218439684 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f9d5406d7f0 0x7f9d5406fca0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:53:44.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:44.660+0000 7f9d74000700 1 --2- 172.21.15.107:0/3218439684 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d6c103dc0 0x7f9d6c11a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:53:44.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:44.660+0000 7f9d74000700 1 --2- 172.21.15.107:0/3218439684 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d6c1095f0 0x7f9d6c11a590 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:53:44.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:44.660+0000 7f9d74000700 1 -- 172.21.15.107:0/3218439684 >> 172.21.15.107:0/3218439684 conn(0x7f9d6c072d50 msgr2=0x7f9d6c0746c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:53:44.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:44.660+0000 7f9d74000700 1 -- 172.21.15.107:0/3218439684 shutdown_connections 2024-03-04T13:53:44.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:44.660+0000 7f9d74000700 1 -- 172.21.15.107:0/3218439684 wait complete. 2024-03-04T13:53:44.665 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 28 2024-03-04T13:53:44.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:44.805+0000 7f2a88000700 1 -- 172.21.15.107:0/3224102290 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2a80101f10 msgr2=0x7f2a801022e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:53:44.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:44.805+0000 7f2a88000700 1 --2- 172.21.15.107:0/3224102290 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2a80101f10 0x7f2a801022e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2a74004940 tx=0x7f2a74009e30 comp rx=0 tx=0).stop 2024-03-04T13:53:44.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:44.805+0000 7f2a88000700 1 -- 172.21.15.107:0/3224102290 shutdown_connections 2024-03-04T13:53:44.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:44.805+0000 7f2a88000700 1 --2- 172.21.15.107:0/3224102290 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a801028b0 0x7f2a800fc6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:53:44.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:44.805+0000 7f2a88000700 1 --2- 172.21.15.107:0/3224102290 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2a80101f10 0x7f2a801022e0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:53:44.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:44.805+0000 7f2a88000700 1 -- 172.21.15.107:0/3224102290 >> 172.21.15.107:0/3224102290 conn(0x7f2a80071560 msgr2=0x7f2a80071960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:53:44.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:44.805+0000 7f2a88000700 1 -- 172.21.15.107:0/3224102290 shutdown_connections 2024-03-04T13:53:44.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:44.805+0000 7f2a88000700 1 -- 172.21.15.107:0/3224102290 wait complete. 2024-03-04T13:53:44.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:44.806+0000 7f2a88000700 1 Processor -- start 2024-03-04T13:53:44.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:44.806+0000 7f2a88000700 1 -- start start 2024-03-04T13:53:44.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:44.807+0000 7f2a88000700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a80101f10 0x7f2a8010c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:53:44.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:44.807+0000 7f2a88000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2a801028b0 0x7f2a8010cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:53:44.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:44.807+0000 7f2a88000700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a8010d350 con 0x7f2a801028b0 2024-03-04T13:53:44.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:44.807+0000 7f2a88000700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a8010d490 con 0x7f2a80101f10 2024-03-04T13:53:44.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:44.807+0000 7f2a84c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2a801028b0 0x7f2a8010cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:53:44.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:44.807+0000 7f2a84c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2a801028b0 0x7f2a8010cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46472/0 (socket says 172.21.15.107:46472) 2024-03-04T13:53:44.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:44.807+0000 7f2a84c00700 1 -- 172.21.15.107:0/3441929179 learned_addr learned my addr 172.21.15.107:0/3441929179 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:53:44.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:44.807+0000 7f2a85600700 1 --2- 172.21.15.107:0/3441929179 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a80101f10 0x7f2a8010c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:53:44.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:44.807+0000 7f2a84c00700 1 -- 172.21.15.107:0/3441929179 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a80101f10 msgr2=0x7f2a8010c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:53:44.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:44.807+0000 7f2a84c00700 1 --2- 172.21.15.107:0/3441929179 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a80101f10 0x7f2a8010c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:53:44.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:44.807+0000 7f2a84c00700 1 -- 172.21.15.107:0/3441929179 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a74005040 con 0x7f2a801028b0 2024-03-04T13:53:44.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:44.808+0000 7f2a84c00700 1 --2- 172.21.15.107:0/3441929179 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2a801028b0 0x7f2a8010cd80 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c00b790 tx=0x7f2a6c009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:53:44.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:44.808+0000 7f2a7d600700 1 -- 172.21.15.107:0/3441929179 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a6c00f6f0 con 0x7f2a801028b0 2024-03-04T13:53:44.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:44.808+0000 7f2a7d600700 1 -- 172.21.15.107:0/3441929179 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2a6c00fd20 con 0x7f2a801028b0 2024-03-04T13:53:44.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:44.808+0000 7f2a7d600700 1 -- 172.21.15.107:0/3441929179 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a6c0103a0 con 0x7f2a801028b0 2024-03-04T13:53:44.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:44.808+0000 7f2a88000700 1 -- 172.21.15.107:0/3441929179 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a80119eb0 con 0x7f2a801028b0 2024-03-04T13:53:44.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:44.808+0000 7f2a88000700 1 -- 172.21.15.107:0/3441929179 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a8011a310 con 0x7f2a801028b0 2024-03-04T13:53:44.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:44.809+0000 7f2a88000700 1 -- 172.21.15.107:0/3441929179 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a800ff230 con 0x7f2a801028b0 2024-03-04T13:53:44.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:44.810+0000 7f2a7d600700 1 -- 172.21.15.107:0/3441929179 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f2a6c011400 con 0x7f2a801028b0 2024-03-04T13:53:44.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:44.811+0000 7f2a7d600700 1 --2- 172.21.15.107:0/3441929179 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f2a6806d7f0 0x7f2a6806fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:53:44.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:44.811+0000 7f2a7d600700 1 -- 172.21.15.107:0/3441929179 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f2a6c08cde0 con 0x7f2a801028b0 2024-03-04T13:53:44.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:44.812+0000 7f2a85600700 1 --2- 172.21.15.107:0/3441929179 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f2a6806d7f0 0x7f2a6806fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:53:44.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:44.812+0000 7f2a85600700 1 --2- 172.21.15.107:0/3441929179 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f2a6806d7f0 0x7f2a6806fca0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2a74000c00 tx=0x7f2a74009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:53:44.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:44.816+0000 7f2a7d600700 1 -- 172.21.15.107:0/3441929179 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2a6c056590 con 0x7f2a801028b0 2024-03-04T13:53:45.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:45.061+0000 7f2a88000700 1 -- 172.21.15.107:0/3441929179 --> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2a8004bed0 con 0x7f2a6806d7f0 2024-03-04T13:53:45.063 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:44 smithi107 ceph-mon[112578]: from='client.34472 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:53:45.063 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:44 smithi107 ceph-mon[112578]: from='client.34476 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:53:45.064 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:44 smithi107 ceph-mon[112578]: from='client.? 172.21.15.107:0/4022984078' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:53:45.064 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:44 smithi107 ceph-mon[112578]: from='client.? 172.21.15.107:0/3218439684' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-04T13:53:45.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:45.065+0000 7f2a7d600700 1 -- 172.21.15.107:0/3441929179 <== mgr.34102 v2:172.21.15.107:6800/1223039812 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f2a8004bed0 con 0x7f2a6806d7f0 2024-03-04T13:53:45.066 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-03-04T13:53:45.067 INFO:teuthology.orchestra.run.smithi107.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377", 2024-03-04T13:53:45.067 INFO:teuthology.orchestra.run.smithi107.stdout: "in_progress": true, 2024-03-04T13:53:45.067 INFO:teuthology.orchestra.run.smithi107.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-04T13:53:45.067 INFO:teuthology.orchestra.run.smithi107.stdout: "services_complete": [ 2024-03-04T13:53:45.067 INFO:teuthology.orchestra.run.smithi107.stdout: "osd", 2024-03-04T13:53:45.067 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr", 2024-03-04T13:53:45.067 INFO:teuthology.orchestra.run.smithi107.stdout: "mon", 2024-03-04T13:53:45.067 INFO:teuthology.orchestra.run.smithi107.stdout: "crash" 2024-03-04T13:53:45.067 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-04T13:53:45.067 INFO:teuthology.orchestra.run.smithi107.stdout: "progress": "15/21 daemons upgraded", 2024-03-04T13:53:45.067 INFO:teuthology.orchestra.run.smithi107.stdout: "message": "Currently upgrading mds daemons", 2024-03-04T13:53:45.067 INFO:teuthology.orchestra.run.smithi107.stdout: "is_paused": false 2024-03-04T13:53:45.067 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-03-04T13:53:45.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:45.067+0000 7f2a88000700 1 -- 172.21.15.107:0/3441929179 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f2a6806d7f0 msgr2=0x7f2a6806fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:53:45.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:45.067+0000 7f2a88000700 1 --2- 172.21.15.107:0/3441929179 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f2a6806d7f0 0x7f2a6806fca0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2a74000c00 tx=0x7f2a74009e30 comp rx=0 tx=0).stop 2024-03-04T13:53:45.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:45.067+0000 7f2a88000700 1 -- 172.21.15.107:0/3441929179 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2a801028b0 msgr2=0x7f2a8010cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:53:45.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:45.067+0000 7f2a88000700 1 --2- 172.21.15.107:0/3441929179 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2a801028b0 0x7f2a8010cd80 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c00b790 tx=0x7f2a6c009220 comp rx=0 tx=0).stop 2024-03-04T13:53:45.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:45.067+0000 7f2a88000700 1 -- 172.21.15.107:0/3441929179 shutdown_connections 2024-03-04T13:53:45.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:45.067+0000 7f2a88000700 1 --2- 172.21.15.107:0/3441929179 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f2a6806d7f0 0x7f2a6806fca0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:53:45.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:45.067+0000 7f2a88000700 1 --2- 172.21.15.107:0/3441929179 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2a80101f10 0x7f2a8010c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:53:45.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:45.067+0000 7f2a88000700 1 --2- 172.21.15.107:0/3441929179 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2a801028b0 0x7f2a8010cd80 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:53:45.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:45.067+0000 7f2a88000700 1 -- 172.21.15.107:0/3441929179 >> 172.21.15.107:0/3441929179 conn(0x7f2a80071560 msgr2=0x7f2a800fa6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:53:45.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:45.068+0000 7f2a88000700 1 -- 172.21.15.107:0/3441929179 shutdown_connections 2024-03-04T13:53:45.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:45.068+0000 7f2a88000700 1 -- 172.21.15.107:0/3441929179 wait complete. 2024-03-04T13:53:45.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:44 smithi169 ceph-mon[86183]: from='client.34472 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:53:45.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:44 smithi169 ceph-mon[86183]: from='client.34476 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:53:45.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:44 smithi169 ceph-mon[86183]: from='client.? 172.21.15.107:0/4022984078' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:53:45.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:44 smithi169 ceph-mon[86183]: from='client.? 172.21.15.107:0/3218439684' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-04T13:53:45.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:45.219+0000 7f767f600700 1 -- 172.21.15.107:0/3835309290 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7678102f50 msgr2=0x7f7678103320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:53:45.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:45.219+0000 7f767f600700 1 --2- 172.21.15.107:0/3835309290 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7678102f50 0x7f7678103320 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f766c009ce0 tx=0x7f766c004d10 comp rx=0 tx=0).stop 2024-03-04T13:53:45.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:45.219+0000 7f767f600700 1 -- 172.21.15.107:0/3835309290 shutdown_connections 2024-03-04T13:53:45.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:45.219+0000 7f767f600700 1 --2- 172.21.15.107:0/3835309290 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f76781038f0 0x7f76780fcf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:53:45.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:45.219+0000 7f767f600700 1 --2- 172.21.15.107:0/3835309290 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7678102f50 0x7f7678103320 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:53:45.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:45.219+0000 7f767f600700 1 -- 172.21.15.107:0/3835309290 >> 172.21.15.107:0/3835309290 conn(0x7f76780f8ae0 msgr2=0x7f76780faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:53:45.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:45.219+0000 7f767f600700 1 -- 172.21.15.107:0/3835309290 shutdown_connections 2024-03-04T13:53:45.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:45.219+0000 7f767f600700 1 -- 172.21.15.107:0/3835309290 wait complete. 2024-03-04T13:53:45.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:45.220+0000 7f767f600700 1 Processor -- start 2024-03-04T13:53:45.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:45.220+0000 7f767f600700 1 -- start start 2024-03-04T13:53:45.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:45.220+0000 7f767f600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7678102f50 0x7f7678115e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:53:45.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:45.220+0000 7f767f600700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f76781038f0 0x7f7678116340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:53:45.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:45.220+0000 7f767f600700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7678116990 con 0x7f7678102f50 2024-03-04T13:53:45.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:45.220+0000 7f767f600700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f767810fce0 con 0x7f76781038f0 2024-03-04T13:53:45.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:45.221+0000 7f767cc00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7678102f50 0x7f7678115e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:53:45.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:45.221+0000 7f767cc00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7678102f50 0x7f7678115e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46490/0 (socket says 172.21.15.107:46490) 2024-03-04T13:53:45.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:45.221+0000 7f767cc00700 1 -- 172.21.15.107:0/974902385 learned_addr learned my addr 172.21.15.107:0/974902385 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:53:45.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:45.221+0000 7f7677e00700 1 --2- 172.21.15.107:0/974902385 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f76781038f0 0x7f7678116340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:53:45.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:45.221+0000 7f767cc00700 1 -- 172.21.15.107:0/974902385 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f76781038f0 msgr2=0x7f7678116340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:53:45.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:45.221+0000 7f767cc00700 1 --2- 172.21.15.107:0/974902385 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f76781038f0 0x7f7678116340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:53:45.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:45.221+0000 7f767cc00700 1 -- 172.21.15.107:0/974902385 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f766c009940 con 0x7f7678102f50 2024-03-04T13:53:45.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:45.221+0000 7f7677e00700 1 --2- 172.21.15.107:0/974902385 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f76781038f0 0x7f7678116340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T13:53:45.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:45.221+0000 7f767cc00700 1 --2- 172.21.15.107:0/974902385 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7678102f50 0x7f7678115e00 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f766c004b70 tx=0x7f766c004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:53:45.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:45.222+0000 7f7674c00700 1 -- 172.21.15.107:0/974902385 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f766c00e040 con 0x7f7678102f50 2024-03-04T13:53:45.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:45.222+0000 7f7674c00700 1 -- 172.21.15.107:0/974902385 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f766c004610 con 0x7f7678102f50 2024-03-04T13:53:45.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:45.222+0000 7f767f600700 1 -- 172.21.15.107:0/974902385 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f767810ff60 con 0x7f7678102f50 2024-03-04T13:53:45.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:45.222+0000 7f7674c00700 1 -- 172.21.15.107:0/974902385 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f766c01c810 con 0x7f7678102f50 2024-03-04T13:53:45.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:45.222+0000 7f767f600700 1 -- 172.21.15.107:0/974902385 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76781104a0 con 0x7f7678102f50 2024-03-04T13:53:45.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:45.223+0000 7f767f600700 1 -- 172.21.15.107:0/974902385 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76780fc6e0 con 0x7f7678102f50 2024-03-04T13:53:45.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:45.224+0000 7f7674c00700 1 -- 172.21.15.107:0/974902385 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f766c012c00 con 0x7f7678102f50 2024-03-04T13:53:45.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:45.225+0000 7f7674c00700 1 --2- 172.21.15.107:0/974902385 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f766006d7f0 0x7f766006fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:53:45.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:45.225+0000 7f7674c00700 1 -- 172.21.15.107:0/974902385 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f766c08eaf0 con 0x7f7678102f50 2024-03-04T13:53:45.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:45.225+0000 7f7677e00700 1 --2- 172.21.15.107:0/974902385 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f766006d7f0 0x7f766006fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:53:45.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:45.226+0000 7f7677e00700 1 --2- 172.21.15.107:0/974902385 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f766006d7f0 0x7f766006fca0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f76781116e0 tx=0x7f7664009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:53:45.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:45.229+0000 7f7674c00700 1 -- 172.21.15.107:0/974902385 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f766c058320 con 0x7f7678102f50 2024-03-04T13:53:45.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:45.529+0000 7f767f600700 1 -- 172.21.15.107:0/974902385 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f76781114c0 con 0x7f7678102f50 2024-03-04T13:53:45.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:45.530+0000 7f7674c00700 1 -- 172.21.15.107:0/974902385 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f766c05b9e0 con 0x7f7678102f50 2024-03-04T13:53:45.532 INFO:teuthology.orchestra.run.smithi107.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-03-04T13:53:45.532 INFO:teuthology.orchestra.run.smithi107.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-04T13:53:45.532 INFO:teuthology.orchestra.run.smithi107.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-04T13:53:45.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:45.532+0000 7f767f600700 1 -- 172.21.15.107:0/974902385 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f766006d7f0 msgr2=0x7f766006fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:53:45.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:45.532+0000 7f767f600700 1 --2- 172.21.15.107:0/974902385 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f766006d7f0 0x7f766006fca0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f76781116e0 tx=0x7f7664009040 comp rx=0 tx=0).stop 2024-03-04T13:53:45.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:45.532+0000 7f767f600700 1 -- 172.21.15.107:0/974902385 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7678102f50 msgr2=0x7f7678115e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:53:45.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:45.532+0000 7f767f600700 1 --2- 172.21.15.107:0/974902385 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7678102f50 0x7f7678115e00 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f766c004b70 tx=0x7f766c004d10 comp rx=0 tx=0).stop 2024-03-04T13:53:45.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:45.532+0000 7f767f600700 1 -- 172.21.15.107:0/974902385 shutdown_connections 2024-03-04T13:53:45.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:45.532+0000 7f767f600700 1 --2- 172.21.15.107:0/974902385 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f766006d7f0 0x7f766006fca0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:53:45.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:45.533+0000 7f767f600700 1 --2- 172.21.15.107:0/974902385 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7678102f50 0x7f7678115e00 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:53:45.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:45.533+0000 7f767f600700 1 --2- 172.21.15.107:0/974902385 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f76781038f0 0x7f7678116340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:53:45.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:45.533+0000 7f767f600700 1 -- 172.21.15.107:0/974902385 >> 172.21.15.107:0/974902385 conn(0x7f76780f8ae0 msgr2=0x7f76780fa420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:53:45.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:45.533+0000 7f767f600700 1 -- 172.21.15.107:0/974902385 shutdown_connections 2024-03-04T13:53:45.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:53:45.533+0000 7f767f600700 1 -- 172.21.15.107:0/974902385 wait complete. 2024-03-04T13:53:45.917 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:45 smithi169 ceph-mon[86183]: pgmap v228: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 11 MiB/s rd, 255 B/s wr, 5 op/s 2024-03-04T13:53:45.917 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:45 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:53:45.917 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:45 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:53:45.917 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:45 smithi169 ceph-mon[86183]: from='client.? 172.21.15.107:0/974902385' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-04T13:53:46.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:45 smithi107 ceph-mon[112578]: pgmap v228: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 11 MiB/s rd, 255 B/s wr, 5 op/s 2024-03-04T13:53:46.095 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:45 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:53:46.095 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:45 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:53:46.095 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:45 smithi107 ceph-mon[112578]: from='client.? 172.21.15.107:0/974902385' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-04T13:53:47.095 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:46 smithi107 ceph-mon[112578]: from='client.34488 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:53:47.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:46 smithi169 ceph-mon[86183]: from='client.34488 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:53:48.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:47 smithi107 ceph-mon[112578]: pgmap v229: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 11 MiB/s rd, 426 B/s wr, 6 op/s 2024-03-04T13:53:48.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:47 smithi169 ceph-mon[86183]: pgmap v229: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 11 MiB/s rd, 426 B/s wr, 6 op/s 2024-03-04T13:53:50.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:49 smithi107 ceph-mon[112578]: pgmap v230: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 8.0 MiB/s rd, 4.7 KiB/s wr, 7 op/s 2024-03-04T13:53:50.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:49 smithi169 ceph-mon[86183]: pgmap v230: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 8.0 MiB/s rd, 4.7 KiB/s wr, 7 op/s 2024-03-04T13:53:51.978 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:51 smithi169 ceph-mon[86183]: pgmap v231: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.0 MiB/s rd, 4.7 KiB/s wr, 5 op/s 2024-03-04T13:53:52.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:51 smithi107 ceph-mon[112578]: pgmap v231: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.0 MiB/s rd, 4.7 KiB/s wr, 5 op/s 2024-03-04T13:53:54.095 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:53 smithi107 ceph-mon[112578]: pgmap v232: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 686 KiB/s rd, 4.7 KiB/s wr, 5 op/s 2024-03-04T13:53:54.095 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:53 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:53:54.095 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:53 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:53:54.095 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:53 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:53:54.095 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:53 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:53:54.095 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:53 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:53:54.095 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:53 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:53:54.095 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:53 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:53:54.096 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:53 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:53:54.096 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:53 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:53:54.096 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:53 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:53:54.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:53 smithi169 ceph-mon[86183]: pgmap v232: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 686 KiB/s rd, 4.7 KiB/s wr, 5 op/s 2024-03-04T13:53:54.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:53 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:53:54.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:53 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:53:54.113 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:53 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:53:54.113 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:53 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:53:54.113 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:53 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:53:54.113 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:53 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:53:54.113 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:53 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:53:54.113 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:53 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:53:54.113 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:53 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:53:54.113 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:53 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:53:55.743 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:55 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:53:55.743 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:55 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:53:56.095 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:55 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:53:56.095 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:55 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:53:56.095 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:55 smithi107 ceph-mon[112578]: pgmap v233: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 4.4 KiB/s wr, 3 op/s 2024-03-04T13:53:56.111 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:55 smithi169 ceph-mon[86183]: pgmap v233: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 4.4 KiB/s wr, 3 op/s 2024-03-04T13:53:57.091 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:56 smithi169 ceph-mon[86183]: Upgrade: Updating mds.cephfs.smithi169.bxitzk 2024-03-04T13:53:57.091 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:56 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:53:57.091 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:56 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi169.bxitzk", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-04T13:53:57.092 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:56 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:53:57.092 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:56 smithi169 ceph-mon[86183]: Deploying daemon mds.cephfs.smithi169.bxitzk on smithi169 2024-03-04T13:53:57.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:56 smithi107 ceph-mon[112578]: Upgrade: Updating mds.cephfs.smithi169.bxitzk 2024-03-04T13:53:57.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:56 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:53:57.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:56 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi169.bxitzk", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-04T13:53:57.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:56 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:53:57.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:56 smithi107 ceph-mon[112578]: Deploying daemon mds.cephfs.smithi169.bxitzk on smithi169 2024-03-04T13:53:58.333 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:57 smithi107 ceph-mon[112578]: pgmap v234: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 4.4 KiB/s wr, 2 op/s 2024-03-04T13:53:58.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:57 smithi169 ceph-mon[86183]: pgmap v234: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 4.4 KiB/s wr, 2 op/s 2024-03-04T13:53:59.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:58 smithi107 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-mon-smithi107[112574]: 2024-03-04T13:53:58.927+0000 7f6566400700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-04T13:53:59.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:58 smithi107 ceph-mon[112578]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-04T13:53:59.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:58 smithi107 ceph-mon[112578]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-04T13:53:59.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:58 smithi107 ceph-mon[112578]: osdmap e86: 6 total, 6 up, 6 in 2024-03-04T13:53:59.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:58 smithi107 ceph-mon[112578]: Standby daemon mds.cephfs.smithi107.riexmm assigned to filesystem cephfs as rank 0 2024-03-04T13:53:59.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:58 smithi107 ceph-mon[112578]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-04T13:53:59.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:58 smithi107 ceph-mon[112578]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-03-04T13:53:59.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:58 smithi107 ceph-mon[112578]: fsmap cephfs:1/1 {0=cephfs.smithi107.riexmm=up:replay} 2 up:standby 2024-03-04T13:53:59.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:58 smithi169 ceph-mon[86183]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-04T13:53:59.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:58 smithi169 ceph-mon[86183]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-04T13:53:59.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:58 smithi169 ceph-mon[86183]: osdmap e86: 6 total, 6 up, 6 in 2024-03-04T13:53:59.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:58 smithi169 ceph-mon[86183]: Standby daemon mds.cephfs.smithi107.riexmm assigned to filesystem cephfs as rank 0 2024-03-04T13:53:59.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:58 smithi169 ceph-mon[86183]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-04T13:53:59.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:58 smithi169 ceph-mon[86183]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-03-04T13:53:59.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:58 smithi169 ceph-mon[86183]: fsmap cephfs:1/1 {0=cephfs.smithi107.riexmm=up:replay} 2 up:standby 2024-03-04T13:54:00.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:53:59 smithi107 ceph-mon[112578]: pgmap v235: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 4.2 KiB/s wr, 2 op/s 2024-03-04T13:54:00.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:53:59 smithi169 ceph-mon[86183]: pgmap v235: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 4.2 KiB/s wr, 2 op/s 2024-03-04T13:54:01.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:00 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi169.bxitzk"}]: dispatch 2024-03-04T13:54:01.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:00 smithi107 ceph-mon[112578]: pgmap v237: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 819 KiB/s rd, 0 B/s wr, 0 op/s 2024-03-04T13:54:01.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:00 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi169.bxitzk"}]: dispatch 2024-03-04T13:54:01.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:00 smithi169 ceph-mon[86183]: pgmap v237: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 819 KiB/s rd, 0 B/s wr, 0 op/s 2024-03-04T13:54:04.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:03 smithi107 ceph-mon[112578]: pgmap v238: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 13 MiB/s rd, 4 op/s 2024-03-04T13:54:04.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:03 smithi169 ceph-mon[86183]: pgmap v238: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 13 MiB/s rd, 4 op/s 2024-03-04T13:54:06.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:05 smithi107 ceph-mon[112578]: pgmap v239: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 13 MiB/s rd, 4 op/s 2024-03-04T13:54:06.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:05 smithi169 ceph-mon[86183]: pgmap v239: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 13 MiB/s rd, 4 op/s 2024-03-04T13:54:07.862 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:07 smithi169 ceph-mon[86183]: pgmap v240: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 17 MiB/s rd, 5 op/s 2024-03-04T13:54:08.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:07 smithi107 ceph-mon[112578]: pgmap v240: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 17 MiB/s rd, 5 op/s 2024-03-04T13:54:10.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:09 smithi107 ceph-mon[112578]: pgmap v241: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 24 MiB/s rd, 7 op/s 2024-03-04T13:54:10.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:09 smithi107 ceph-mon[112578]: mds.? [v2:172.21.15.107:6826/1144672731,v1:172.21.15.107:6827/1144672731] up:reconnect 2024-03-04T13:54:10.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:09 smithi107 ceph-mon[112578]: fsmap cephfs:1/1 {0=cephfs.smithi107.riexmm=up:reconnect} 2 up:standby 2024-03-04T13:54:10.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:09 smithi107 ceph-mon[112578]: reconnect by client.14524 192.168.0.1:0/317786685 after 0.00399993 2024-03-04T13:54:10.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:09 smithi107 ceph-mon[112578]: reconnect by client.24293 192.168.0.1:0/2380524016 after 0.00799986 2024-03-04T13:54:10.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:09 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:54:10.346 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:09 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:54:10.346 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:09 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:54:10.346 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:09 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:54:10.346 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:09 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:54:10.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:09 smithi169 ceph-mon[86183]: pgmap v241: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 24 MiB/s rd, 7 op/s 2024-03-04T13:54:10.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:09 smithi169 ceph-mon[86183]: mds.? [v2:172.21.15.107:6826/1144672731,v1:172.21.15.107:6827/1144672731] up:reconnect 2024-03-04T13:54:10.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:09 smithi169 ceph-mon[86183]: fsmap cephfs:1/1 {0=cephfs.smithi107.riexmm=up:reconnect} 2 up:standby 2024-03-04T13:54:10.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:09 smithi169 ceph-mon[86183]: reconnect by client.14524 192.168.0.1:0/317786685 after 0.00399993 2024-03-04T13:54:10.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:09 smithi169 ceph-mon[86183]: reconnect by client.24293 192.168.0.1:0/2380524016 after 0.00799986 2024-03-04T13:54:10.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:09 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:54:10.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:09 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:54:10.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:09 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:54:10.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:09 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:54:10.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:09 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:54:11.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:10 smithi169 ceph-mon[86183]: mds.? [v2:172.21.15.107:6826/1144672731,v1:172.21.15.107:6827/1144672731] up:rejoin 2024-03-04T13:54:11.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:10 smithi169 ceph-mon[86183]: mds.? [v2:172.21.15.169:6826/4015044684,v1:172.21.15.169:6827/4015044684] up:boot 2024-03-04T13:54:11.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:10 smithi169 ceph-mon[86183]: fsmap cephfs:1/1 {0=cephfs.smithi107.riexmm=up:rejoin} 3 up:standby 2024-03-04T13:54:11.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:10 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi169.bxitzk"}]: dispatch 2024-03-04T13:54:11.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:10 smithi169 ceph-mon[86183]: daemon mds.cephfs.smithi107.riexmm is now active in filesystem cephfs as rank 0 2024-03-04T13:54:11.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:10 smithi107 ceph-mon[112578]: mds.? [v2:172.21.15.107:6826/1144672731,v1:172.21.15.107:6827/1144672731] up:rejoin 2024-03-04T13:54:11.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:10 smithi107 ceph-mon[112578]: mds.? [v2:172.21.15.169:6826/4015044684,v1:172.21.15.169:6827/4015044684] up:boot 2024-03-04T13:54:11.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:10 smithi107 ceph-mon[112578]: fsmap cephfs:1/1 {0=cephfs.smithi107.riexmm=up:rejoin} 3 up:standby 2024-03-04T13:54:11.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:10 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi169.bxitzk"}]: dispatch 2024-03-04T13:54:11.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:10 smithi107 ceph-mon[112578]: daemon mds.cephfs.smithi107.riexmm is now active in filesystem cephfs as rank 0 2024-03-04T13:54:12.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:11 smithi169 ceph-mon[86183]: pgmap v242: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 6 op/s 2024-03-04T13:54:12.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:11 smithi169 ceph-mon[86183]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-04T13:54:12.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:11 smithi169 ceph-mon[86183]: mds.? [v2:172.21.15.107:6826/1144672731,v1:172.21.15.107:6827/1144672731] up:active 2024-03-04T13:54:12.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:11 smithi169 ceph-mon[86183]: fsmap cephfs:1 {0=cephfs.smithi107.riexmm=up:active} 3 up:standby 2024-03-04T13:54:12.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:11 smithi107 ceph-mon[112578]: pgmap v242: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 6 op/s 2024-03-04T13:54:12.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:11 smithi107 ceph-mon[112578]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-04T13:54:12.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:11 smithi107 ceph-mon[112578]: mds.? [v2:172.21.15.107:6826/1144672731,v1:172.21.15.107:6827/1144672731] up:active 2024-03-04T13:54:12.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:11 smithi107 ceph-mon[112578]: fsmap cephfs:1 {0=cephfs.smithi107.riexmm=up:active} 3 up:standby 2024-03-04T13:54:14.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:13 smithi107 ceph-mon[112578]: pgmap v243: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 20 MiB/s rd, 170 B/s wr, 8 op/s 2024-03-04T13:54:14.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:13 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:54:14.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:13 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:54:14.361 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:13 smithi169 ceph-mon[86183]: pgmap v243: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 20 MiB/s rd, 170 B/s wr, 8 op/s 2024-03-04T13:54:14.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:13 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:54:14.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:13 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:54:15.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:15.695+0000 7fe57b400700 1 -- 172.21.15.107:0/2816397581 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe574102f50 msgr2=0x7fe574103320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:54:15.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:15.695+0000 7fe57b400700 1 --2- 172.21.15.107:0/2816397581 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe574102f50 0x7fe574103320 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe568004940 tx=0x7fe568009e30 comp rx=0 tx=0).stop 2024-03-04T13:54:15.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:15.695+0000 7fe57b400700 1 -- 172.21.15.107:0/2816397581 shutdown_connections 2024-03-04T13:54:15.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:15.695+0000 7fe57b400700 1 --2- 172.21.15.107:0/2816397581 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5741038f0 0x7fe5740fcf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:54:15.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:15.695+0000 7fe57b400700 1 --2- 172.21.15.107:0/2816397581 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe574102f50 0x7fe574103320 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:54:15.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:15.695+0000 7fe57b400700 1 -- 172.21.15.107:0/2816397581 >> 172.21.15.107:0/2816397581 conn(0x7fe5740f8ae0 msgr2=0x7fe5740faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:54:15.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:15.695+0000 7fe57b400700 1 -- 172.21.15.107:0/2816397581 shutdown_connections 2024-03-04T13:54:15.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:15.695+0000 7fe57b400700 1 -- 172.21.15.107:0/2816397581 wait complete. 2024-03-04T13:54:15.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:15.696+0000 7fe57b400700 1 Processor -- start 2024-03-04T13:54:15.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:15.697+0000 7fe57b400700 1 -- start start 2024-03-04T13:54:15.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:15.697+0000 7fe57b400700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe574102f50 0x7fe574070290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:54:15.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:15.697+0000 7fe57b400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe5741038f0 0x7fe5740707d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:54:15.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:15.697+0000 7fe57b400700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe574070dd0 con 0x7fe5741038f0 2024-03-04T13:54:15.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:15.697+0000 7fe57b400700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe574070f40 con 0x7fe574102f50 2024-03-04T13:54:15.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:15.697+0000 7fe573e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe5741038f0 0x7fe5740707d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:54:15.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:15.697+0000 7fe573e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe5741038f0 0x7fe5740707d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51698/0 (socket says 172.21.15.107:51698) 2024-03-04T13:54:15.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:15.697+0000 7fe573e00700 1 -- 172.21.15.107:0/2382861925 learned_addr learned my addr 172.21.15.107:0/2382861925 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:54:15.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:15.698+0000 7fe578a00700 1 --2- 172.21.15.107:0/2382861925 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe574102f50 0x7fe574070290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:54:15.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:15.698+0000 7fe573e00700 1 -- 172.21.15.107:0/2382861925 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe574102f50 msgr2=0x7fe574070290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:54:15.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:15.698+0000 7fe573e00700 1 --2- 172.21.15.107:0/2382861925 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe574102f50 0x7fe574070290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:54:15.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:15.698+0000 7fe573e00700 1 -- 172.21.15.107:0/2382861925 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe568005040 con 0x7fe5741038f0 2024-03-04T13:54:15.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:15.698+0000 7fe573e00700 1 --2- 172.21.15.107:0/2382861925 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe5741038f0 0x7fe5740707d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe56000c890 tx=0x7fe5600044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:54:15.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:15.699+0000 7fe570c00700 1 -- 172.21.15.107:0/2382861925 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe56000a700 con 0x7fe5741038f0 2024-03-04T13:54:15.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:15.699+0000 7fe570c00700 1 -- 172.21.15.107:0/2382861925 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe56000ad30 con 0x7fe5741038f0 2024-03-04T13:54:15.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:15.699+0000 7fe570c00700 1 -- 172.21.15.107:0/2382861925 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe5600113a0 con 0x7fe5741038f0 2024-03-04T13:54:15.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:15.699+0000 7fe57b400700 1 -- 172.21.15.107:0/2382861925 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe574119c90 con 0x7fe5741038f0 2024-03-04T13:54:15.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:15.699+0000 7fe57b400700 1 -- 172.21.15.107:0/2382861925 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe57411a170 con 0x7fe5741038f0 2024-03-04T13:54:15.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:15.700+0000 7fe57b400700 1 -- 172.21.15.107:0/2382861925 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5740fc6e0 con 0x7fe5741038f0 2024-03-04T13:54:15.703 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:15.701+0000 7fe570c00700 1 -- 172.21.15.107:0/2382861925 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fe560012400 con 0x7fe5741038f0 2024-03-04T13:54:15.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:15.702+0000 7fe570c00700 1 --2- 172.21.15.107:0/2382861925 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fe55c06d7f0 0x7fe55c06fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:54:15.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:15.702+0000 7fe570c00700 1 -- 172.21.15.107:0/2382861925 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fe56008ce70 con 0x7fe5741038f0 2024-03-04T13:54:15.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:15.702+0000 7fe578a00700 1 --2- 172.21.15.107:0/2382861925 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fe55c06d7f0 0x7fe55c06fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:54:15.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:15.703+0000 7fe578a00700 1 --2- 172.21.15.107:0/2382861925 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fe55c06d7f0 0x7fe55c06fca0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe56800b3b0 tx=0x7fe568009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:54:15.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:15.707+0000 7fe570c00700 1 -- 172.21.15.107:0/2382861925 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe5600567b0 con 0x7fe5741038f0 2024-03-04T13:54:15.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:15.945+0000 7fe57b400700 1 -- 172.21.15.107:0/2382861925 --> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe574071ae0 con 0x7fe55c06d7f0 2024-03-04T13:54:15.948 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:15 smithi107 ceph-mon[112578]: pgmap v244: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 10 MiB/s rd, 170 B/s wr, 5 op/s 2024-03-04T13:54:15.948 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:15 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:54:15.948 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:15 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:54:15.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:15.948+0000 7fe570c00700 1 -- 172.21.15.107:0/2382861925 <== mgr.34102 v2:172.21.15.107:6800/1223039812 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fe574071ae0 con 0x7fe55c06d7f0 2024-03-04T13:54:15.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:15.951+0000 7fe57b400700 1 -- 172.21.15.107:0/2382861925 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fe55c06d7f0 msgr2=0x7fe55c06fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:54:15.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:15.951+0000 7fe57b400700 1 --2- 172.21.15.107:0/2382861925 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fe55c06d7f0 0x7fe55c06fca0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe56800b3b0 tx=0x7fe568009e30 comp rx=0 tx=0).stop 2024-03-04T13:54:15.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:15.951+0000 7fe57b400700 1 -- 172.21.15.107:0/2382861925 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe5741038f0 msgr2=0x7fe5740707d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:54:15.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:15.951+0000 7fe57b400700 1 --2- 172.21.15.107:0/2382861925 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe5741038f0 0x7fe5740707d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe56000c890 tx=0x7fe5600044c0 comp rx=0 tx=0).stop 2024-03-04T13:54:15.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:15.951+0000 7fe57b400700 1 -- 172.21.15.107:0/2382861925 shutdown_connections 2024-03-04T13:54:15.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:15.951+0000 7fe57b400700 1 --2- 172.21.15.107:0/2382861925 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fe55c06d7f0 0x7fe55c06fca0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:54:15.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:15.951+0000 7fe57b400700 1 --2- 172.21.15.107:0/2382861925 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe574102f50 0x7fe574070290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:54:15.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:15.951+0000 7fe57b400700 1 --2- 172.21.15.107:0/2382861925 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe5741038f0 0x7fe5740707d0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:54:15.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:15.951+0000 7fe57b400700 1 -- 172.21.15.107:0/2382861925 >> 172.21.15.107:0/2382861925 conn(0x7fe5740f8ae0 msgr2=0x7fe5740fa500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:54:15.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:15.951+0000 7fe57b400700 1 -- 172.21.15.107:0/2382861925 shutdown_connections 2024-03-04T13:54:15.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:15.951+0000 7fe57b400700 1 -- 172.21.15.107:0/2382861925 wait complete. 2024-03-04T13:54:15.969 INFO:teuthology.orchestra.run.smithi107.stdout:true 2024-03-04T13:54:16.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.107+0000 7f8a9fa00700 1 -- 172.21.15.107:0/649316864 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a98105170 msgr2=0x7f8a98105540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:54:16.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.107+0000 7f8a9fa00700 1 --2- 172.21.15.107:0/649316864 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a98105170 0x7f8a98105540 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c004940 tx=0x7f8a8c009e30 comp rx=0 tx=0).stop 2024-03-04T13:54:16.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.107+0000 7f8a9fa00700 1 -- 172.21.15.107:0/649316864 shutdown_connections 2024-03-04T13:54:16.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.107+0000 7f8a9fa00700 1 --2- 172.21.15.107:0/649316864 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a98105b10 0x7f8a980ff180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:54:16.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.107+0000 7f8a9fa00700 1 --2- 172.21.15.107:0/649316864 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a98105170 0x7f8a98105540 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:54:16.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.107+0000 7f8a9fa00700 1 -- 172.21.15.107:0/649316864 >> 172.21.15.107:0/649316864 conn(0x7f8a980face0 msgr2=0x7f8a980fd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:54:16.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.107+0000 7f8a9fa00700 1 -- 172.21.15.107:0/649316864 shutdown_connections 2024-03-04T13:54:16.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.107+0000 7f8a9fa00700 1 -- 172.21.15.107:0/649316864 wait complete. 2024-03-04T13:54:16.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.108+0000 7f8a9fa00700 1 Processor -- start 2024-03-04T13:54:16.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.108+0000 7f8a9fa00700 1 -- start start 2024-03-04T13:54:16.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.108+0000 7f8a9fa00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a98105170 0x7f8a98115c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:54:16.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.109+0000 7f8a9fa00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a98105b10 0x7f8a981161a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:54:16.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.109+0000 7f8a9fa00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a981167f0 con 0x7f8a98105170 2024-03-04T13:54:16.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.109+0000 7f8a9fa00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a9810fb40 con 0x7f8a98105b10 2024-03-04T13:54:16.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.109+0000 7f8a9d000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a98105170 0x7f8a98115c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:54:16.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.109+0000 7f8a9d000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a98105170 0x7f8a98115c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51714/0 (socket says 172.21.15.107:51714) 2024-03-04T13:54:16.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.109+0000 7f8a9d000700 1 -- 172.21.15.107:0/1397827303 learned_addr learned my addr 172.21.15.107:0/1397827303 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:54:16.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.109+0000 7f8a9d000700 1 -- 172.21.15.107:0/1397827303 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a98105b10 msgr2=0x7f8a981161a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:54:16.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.109+0000 7f8a9d000700 1 --2- 172.21.15.107:0/1397827303 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a98105b10 0x7f8a981161a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:54:16.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.109+0000 7f8a9d000700 1 -- 172.21.15.107:0/1397827303 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a8c005040 con 0x7f8a98105170 2024-03-04T13:54:16.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.110+0000 7f8a9d000700 1 --2- 172.21.15.107:0/1397827303 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a98105170 0x7f8a98115c60 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c009100 tx=0x7f8a8c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:54:16.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.110+0000 7f8a94c00700 1 -- 172.21.15.107:0/1397827303 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8a8c00e040 con 0x7f8a98105170 2024-03-04T13:54:16.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.110+0000 7f8a94c00700 1 -- 172.21.15.107:0/1397827303 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8a8c004290 con 0x7f8a98105170 2024-03-04T13:54:16.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.110+0000 7f8a94c00700 1 -- 172.21.15.107:0/1397827303 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8a8c01c630 con 0x7f8a98105170 2024-03-04T13:54:16.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.110+0000 7f8a9fa00700 1 -- 172.21.15.107:0/1397827303 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a9810fdc0 con 0x7f8a98105170 2024-03-04T13:54:16.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.110+0000 7f8a9fa00700 1 -- 172.21.15.107:0/1397827303 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a98110300 con 0x7f8a98105170 2024-03-04T13:54:16.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.111+0000 7f8a9fa00700 1 -- 172.21.15.107:0/1397827303 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a98102480 con 0x7f8a98105170 2024-03-04T13:54:16.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.112+0000 7f8a94c00700 1 -- 172.21.15.107:0/1397827303 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f8a8c00bbf0 con 0x7f8a98105170 2024-03-04T13:54:16.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.113+0000 7f8a94c00700 1 --2- 172.21.15.107:0/1397827303 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f8a8006d7e0 0x7f8a8006fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:54:16.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.114+0000 7f8a94c00700 1 -- 172.21.15.107:0/1397827303 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f8a8c08e140 con 0x7f8a98105170 2024-03-04T13:54:16.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.114+0000 7f8a97e00700 1 --2- 172.21.15.107:0/1397827303 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f8a8006d7e0 0x7f8a8006fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:54:16.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.114+0000 7f8a97e00700 1 --2- 172.21.15.107:0/1397827303 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f8a8006d7e0 0x7f8a8006fc90 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8a98111540 tx=0x7f8a84007c30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:54:16.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.118+0000 7f8a94c00700 1 -- 172.21.15.107:0/1397827303 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8a8c057a80 con 0x7f8a98105170 2024-03-04T13:54:16.234 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:15 smithi169 ceph-mon[86183]: pgmap v244: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 10 MiB/s rd, 170 B/s wr, 5 op/s 2024-03-04T13:54:16.234 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:15 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:54:16.234 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:15 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:54:16.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.364+0000 7f8a9fa00700 1 -- 172.21.15.107:0/1397827303 --> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8a9804bed0 con 0x7f8a8006d7e0 2024-03-04T13:54:16.368 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.366+0000 7f8a94c00700 1 -- 172.21.15.107:0/1397827303 <== mgr.34102 v2:172.21.15.107:6800/1223039812 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f8a9804bed0 con 0x7f8a8006d7e0 2024-03-04T13:54:16.371 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.369+0000 7f8a9fa00700 1 -- 172.21.15.107:0/1397827303 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f8a8006d7e0 msgr2=0x7f8a8006fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:54:16.371 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.369+0000 7f8a9fa00700 1 --2- 172.21.15.107:0/1397827303 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f8a8006d7e0 0x7f8a8006fc90 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8a98111540 tx=0x7f8a84007c30 comp rx=0 tx=0).stop 2024-03-04T13:54:16.371 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.369+0000 7f8a9fa00700 1 -- 172.21.15.107:0/1397827303 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a98105170 msgr2=0x7f8a98115c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:54:16.371 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.369+0000 7f8a9fa00700 1 --2- 172.21.15.107:0/1397827303 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a98105170 0x7f8a98115c60 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c009100 tx=0x7f8a8c009e30 comp rx=0 tx=0).stop 2024-03-04T13:54:16.371 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.369+0000 7f8a9fa00700 1 -- 172.21.15.107:0/1397827303 shutdown_connections 2024-03-04T13:54:16.372 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.369+0000 7f8a9fa00700 1 --2- 172.21.15.107:0/1397827303 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f8a8006d7e0 0x7f8a8006fc90 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:54:16.372 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.369+0000 7f8a9fa00700 1 --2- 172.21.15.107:0/1397827303 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a98105170 0x7f8a98115c60 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:54:16.372 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.369+0000 7f8a9fa00700 1 --2- 172.21.15.107:0/1397827303 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8a98105b10 0x7f8a981161a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:54:16.372 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.369+0000 7f8a9fa00700 1 -- 172.21.15.107:0/1397827303 >> 172.21.15.107:0/1397827303 conn(0x7f8a980face0 msgr2=0x7f8a980fc810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:54:16.372 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.369+0000 7f8a9fa00700 1 -- 172.21.15.107:0/1397827303 shutdown_connections 2024-03-04T13:54:16.372 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.370+0000 7f8a9fa00700 1 -- 172.21.15.107:0/1397827303 wait complete. 2024-03-04T13:54:16.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.520+0000 7ff7af800700 1 -- 172.21.15.107:0/1625207795 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7a80fe8f0 msgr2=0x7ff7a8106f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:54:16.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.520+0000 7ff7af800700 1 --2- 172.21.15.107:0/1625207795 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7a80fe8f0 0x7ff7a8106f30 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff790004940 tx=0x7ff790009e30 comp rx=0 tx=0).stop 2024-03-04T13:54:16.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.520+0000 7ff7af800700 1 -- 172.21.15.107:0/1625207795 shutdown_connections 2024-03-04T13:54:16.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.520+0000 7ff7af800700 1 --2- 172.21.15.107:0/1625207795 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7a80fe8f0 0x7ff7a8106f30 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:54:16.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.520+0000 7ff7af800700 1 --2- 172.21.15.107:0/1625207795 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7a80fdf50 0x7ff7a80fe320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:54:16.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.520+0000 7ff7af800700 1 -- 172.21.15.107:0/1625207795 >> 172.21.15.107:0/1625207795 conn(0x7ff7a80f79e0 msgr2=0x7ff7a80f9df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:54:16.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.521+0000 7ff7af800700 1 -- 172.21.15.107:0/1625207795 shutdown_connections 2024-03-04T13:54:16.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.521+0000 7ff7af800700 1 -- 172.21.15.107:0/1625207795 wait complete. 2024-03-04T13:54:16.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.522+0000 7ff7af800700 1 Processor -- start 2024-03-04T13:54:16.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.522+0000 7ff7af800700 1 -- start start 2024-03-04T13:54:16.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.522+0000 7ff7af800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7a80fdf50 0x7ff7a8115cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:54:16.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.522+0000 7ff7af800700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7a80fe8f0 0x7ff7a8116210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:54:16.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.522+0000 7ff7af800700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a8116860 con 0x7ff7a80fdf50 2024-03-04T13:54:16.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.522+0000 7ff7af800700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a810fb70 con 0x7ff7a80fe8f0 2024-03-04T13:54:16.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.522+0000 7ff7ace00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7a80fdf50 0x7ff7a8115cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:54:16.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.523+0000 7ff7ace00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7a80fdf50 0x7ff7a8115cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51730/0 (socket says 172.21.15.107:51730) 2024-03-04T13:54:16.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.523+0000 7ff7ace00700 1 -- 172.21.15.107:0/209032990 learned_addr learned my addr 172.21.15.107:0/209032990 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:54:16.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.523+0000 7ff79fe00700 1 --2- 172.21.15.107:0/209032990 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7a80fe8f0 0x7ff7a8116210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:54:16.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.523+0000 7ff7ace00700 1 -- 172.21.15.107:0/209032990 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7a80fe8f0 msgr2=0x7ff7a8116210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:54:16.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.523+0000 7ff7ace00700 1 --2- 172.21.15.107:0/209032990 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7a80fe8f0 0x7ff7a8116210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:54:16.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.523+0000 7ff7ace00700 1 -- 172.21.15.107:0/209032990 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff790005040 con 0x7ff7a80fdf50 2024-03-04T13:54:16.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.523+0000 7ff7ace00700 1 --2- 172.21.15.107:0/209032990 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7a80fdf50 0x7ff7a8115cd0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff79800c830 tx=0x7ff798004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:54:16.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.523+0000 7ff7a5600700 1 -- 172.21.15.107:0/209032990 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff79800a760 con 0x7ff7a80fdf50 2024-03-04T13:54:16.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.524+0000 7ff7a5600700 1 -- 172.21.15.107:0/209032990 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff79800ad90 con 0x7ff7a80fdf50 2024-03-04T13:54:16.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.524+0000 7ff7a5600700 1 -- 172.21.15.107:0/209032990 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff7980113a0 con 0x7ff7a80fdf50 2024-03-04T13:54:16.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.524+0000 7ff7af800700 1 -- 172.21.15.107:0/209032990 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7a810fe50 con 0x7ff7a80fdf50 2024-03-04T13:54:16.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.524+0000 7ff7af800700 1 -- 172.21.15.107:0/209032990 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7a8110390 con 0x7ff7a80fdf50 2024-03-04T13:54:16.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.525+0000 7ff7af800700 1 -- 172.21.15.107:0/209032990 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7a8106940 con 0x7ff7a80fdf50 2024-03-04T13:54:16.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.526+0000 7ff7a5600700 1 -- 172.21.15.107:0/209032990 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7ff798012400 con 0x7ff7a80fdf50 2024-03-04T13:54:16.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.527+0000 7ff7a5600700 1 --2- 172.21.15.107:0/209032990 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7ff78c06d7f0 0x7ff78c06fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:54:16.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.527+0000 7ff7a5600700 1 -- 172.21.15.107:0/209032990 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7ff79808cf30 con 0x7ff7a80fdf50 2024-03-04T13:54:16.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.527+0000 7ff79fe00700 1 --2- 172.21.15.107:0/209032990 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7ff78c06d7f0 0x7ff78c06fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:54:16.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.527+0000 7ff79fe00700 1 --2- 172.21.15.107:0/209032990 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7ff78c06d7f0 0x7ff78c06fca0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff790000c00 tx=0x7ff790009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:54:16.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.531+0000 7ff7a5600700 1 -- 172.21.15.107:0/209032990 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff79805a2e0 con 0x7ff7a80fdf50 2024-03-04T13:54:16.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.756+0000 7ff7af800700 1 -- 172.21.15.107:0/209032990 --> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7ff7a804bed0 con 0x7ff78c06d7f0 2024-03-04T13:54:16.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.783+0000 7ff7a5600700 1 -- 172.21.15.107:0/209032990 <== mgr.34102 v2:172.21.15.107:6800/1223039812 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7ff7a804bed0 con 0x7ff78c06d7f0 2024-03-04T13:54:16.785 INFO:teuthology.orchestra.run.smithi107.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-04T13:54:16.785 INFO:teuthology.orchestra.run.smithi107.stdout:alertmanager.smithi107 smithi107 *:9093,9094 running (8m) 60s ago 22m 14.4M - 0.25.0 c8568f914cd2 e5ed7be32538 2024-03-04T13:54:16.785 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi107 smithi107 running (5m) 60s ago 22m 6660k - 19.0.0-1845-g717ce3cc d71b839abdf8 0558022b0d84 2024-03-04T13:54:16.786 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi169 smithi169 running (5m) 3s ago 21m 6665k - 19.0.0-1845-g717ce3cc d71b839abdf8 221ca53aa0d3 2024-03-04T13:54:16.786 INFO:teuthology.orchestra.run.smithi107.stdout:grafana.smithi107 smithi107 *:3000 running (8m) 60s ago 22m 75.1M - 9.4.12 a0ea5e3c46d9 c5e4d33fb599 2024-03-04T13:54:16.786 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.riexmm smithi107 running (91s) 60s ago 15m 11.0M - 19.0.0-1845-g717ce3cc d71b839abdf8 8582388ad90a 2024-03-04T13:54:16.786 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.thycho smithi107 running (67s) 60s ago 15m 10.8M - 19.0.0-1845-g717ce3cc d71b839abdf8 f4b3cea14a6a 2024-03-04T13:54:16.786 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi169.bxitzk smithi169 running (8s) 3s ago 15m 10.9M - 19.0.0-1845-g717ce3cc d71b839abdf8 1ea541510c55 2024-03-04T13:54:16.786 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi169.jusudl smithi169 running (39s) 3s ago 15m 11.1M - 19.0.0-1845-g717ce3cc d71b839abdf8 e6d45009ccbb 2024-03-04T13:54:16.786 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi107.wpzgwl smithi107 *:8443,9283,8765 running (11m) 60s ago 24m 509M - 19.0.0-1845-g717ce3cc d71b839abdf8 1c5fdee45eac 2024-03-04T13:54:16.786 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi169.eihczr smithi169 *:8443,9283,8765 running (10m) 3s ago 21m 452M - 19.0.0-1845-g717ce3cc d71b839abdf8 b5cac2b7d22a 2024-03-04T13:54:16.786 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi107 smithi107 running (7m) 60s ago 24m 52.5M 2048M 19.0.0-1845-g717ce3cc d71b839abdf8 32f07144274f 2024-03-04T13:54:16.786 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi169 smithi169 running (6m) 3s ago 21m 41.6M 2048M 19.0.0-1845-g717ce3cc d71b839abdf8 e3527449381a 2024-03-04T13:54:16.786 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi107 smithi107 *:9100 running (9m) 60s ago 22m 8102k - 1.5.0 0da6a335fe13 8a8d3d4f724f 2024-03-04T13:54:16.786 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi169 smithi169 *:9100 running (9m) 3s ago 21m 8158k - 1.5.0 0da6a335fe13 f240f5a975bc 2024-03-04T13:54:16.786 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0 smithi107 running (4m) 60s ago 20m 85.3M 1558M 19.0.0-1845-g717ce3cc d71b839abdf8 5801a5540dc6 2024-03-04T13:54:16.786 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1 smithi107 running (4m) 60s ago 19m 94.0M 1558M 19.0.0-1845-g717ce3cc d71b839abdf8 b614af658854 2024-03-04T13:54:16.787 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2 smithi107 running (3m) 60s ago 19m 84.9M 1558M 19.0.0-1845-g717ce3cc d71b839abdf8 6e4da4f6f146 2024-03-04T13:54:16.787 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3 smithi169 running (3m) 3s ago 18m 107M 2582M 19.0.0-1845-g717ce3cc d71b839abdf8 d31baf764d7a 2024-03-04T13:54:16.787 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4 smithi169 running (2m) 3s ago 18m 90.7M 2582M 19.0.0-1845-g717ce3cc d71b839abdf8 a70bacd97639 2024-03-04T13:54:16.787 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5 smithi169 running (116s) 3s ago 17m 86.0M 2582M 19.0.0-1845-g717ce3cc d71b839abdf8 5f6e00f7993d 2024-03-04T13:54:16.787 INFO:teuthology.orchestra.run.smithi107.stdout:prometheus.smithi107 smithi107 *:9095 running (9m) 60s ago 21m 54.9M - 2.43.0 a07b618ecd1d 1eeb1ad5cb75 2024-03-04T13:54:16.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.786+0000 7ff7af800700 1 -- 172.21.15.107:0/209032990 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7ff78c06d7f0 msgr2=0x7ff78c06fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:54:16.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.786+0000 7ff7af800700 1 --2- 172.21.15.107:0/209032990 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7ff78c06d7f0 0x7ff78c06fca0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff790000c00 tx=0x7ff790009e30 comp rx=0 tx=0).stop 2024-03-04T13:54:16.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.786+0000 7ff7af800700 1 -- 172.21.15.107:0/209032990 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7a80fdf50 msgr2=0x7ff7a8115cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:54:16.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.786+0000 7ff7af800700 1 --2- 172.21.15.107:0/209032990 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7a80fdf50 0x7ff7a8115cd0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff79800c830 tx=0x7ff798004470 comp rx=0 tx=0).stop 2024-03-04T13:54:16.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.787+0000 7ff7af800700 1 -- 172.21.15.107:0/209032990 shutdown_connections 2024-03-04T13:54:16.790 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.787+0000 7ff7af800700 1 --2- 172.21.15.107:0/209032990 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7ff78c06d7f0 0x7ff78c06fca0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:54:16.790 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.787+0000 7ff7af800700 1 --2- 172.21.15.107:0/209032990 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7a80fdf50 0x7ff7a8115cd0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:54:16.790 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.787+0000 7ff7af800700 1 --2- 172.21.15.107:0/209032990 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7a80fe8f0 0x7ff7a8116210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:54:16.790 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.787+0000 7ff7af800700 1 -- 172.21.15.107:0/209032990 >> 172.21.15.107:0/209032990 conn(0x7ff7a80f79e0 msgr2=0x7ff7a8103ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:54:16.790 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.787+0000 7ff7af800700 1 -- 172.21.15.107:0/209032990 shutdown_connections 2024-03-04T13:54:16.790 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.787+0000 7ff7af800700 1 -- 172.21.15.107:0/209032990 wait complete. 2024-03-04T13:54:16.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.934+0000 7fbd82e00700 1 -- 172.21.15.107:0/696587315 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd7c102f50 msgr2=0x7fbd7c103320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:54:16.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.934+0000 7fbd82e00700 1 --2- 172.21.15.107:0/696587315 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd7c102f50 0x7fbd7c103320 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c009ce0 tx=0x7fbd6c004d10 comp rx=0 tx=0).stop 2024-03-04T13:54:16.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.934+0000 7fbd82e00700 1 -- 172.21.15.107:0/696587315 shutdown_connections 2024-03-04T13:54:16.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.934+0000 7fbd82e00700 1 --2- 172.21.15.107:0/696587315 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbd7c1038f0 0x7fbd7c0fcf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:54:16.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.934+0000 7fbd82e00700 1 --2- 172.21.15.107:0/696587315 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd7c102f50 0x7fbd7c103320 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:54:16.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.934+0000 7fbd82e00700 1 -- 172.21.15.107:0/696587315 >> 172.21.15.107:0/696587315 conn(0x7fbd7c0f8ae0 msgr2=0x7fbd7c0faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:54:16.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.934+0000 7fbd82e00700 1 -- 172.21.15.107:0/696587315 shutdown_connections 2024-03-04T13:54:16.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.935+0000 7fbd82e00700 1 -- 172.21.15.107:0/696587315 wait complete. 2024-03-04T13:54:16.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.935+0000 7fbd82e00700 1 Processor -- start 2024-03-04T13:54:16.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.936+0000 7fbd82e00700 1 -- start start 2024-03-04T13:54:16.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.936+0000 7fbd82e00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbd7c102f50 0x7fbd7c113b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:54:16.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.936+0000 7fbd82e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd7c1038f0 0x7fbd7c1140c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:54:16.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.936+0000 7fbd82e00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd7c114710 con 0x7fbd7c1038f0 2024-03-04T13:54:16.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.936+0000 7fbd82e00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd7c10dad0 con 0x7fbd7c102f50 2024-03-04T13:54:16.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.936+0000 7fbd73400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd7c1038f0 0x7fbd7c1140c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:54:16.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.936+0000 7fbd73400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd7c1038f0 0x7fbd7c1140c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51752/0 (socket says 172.21.15.107:51752) 2024-03-04T13:54:16.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.936+0000 7fbd73400700 1 -- 172.21.15.107:0/2927827742 learned_addr learned my addr 172.21.15.107:0/2927827742 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:54:16.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.937+0000 7fbd7be00700 1 --2- 172.21.15.107:0/2927827742 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbd7c102f50 0x7fbd7c113b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:54:16.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.937+0000 7fbd73400700 1 -- 172.21.15.107:0/2927827742 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbd7c102f50 msgr2=0x7fbd7c113b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:54:16.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.937+0000 7fbd73400700 1 --2- 172.21.15.107:0/2927827742 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbd7c102f50 0x7fbd7c113b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:54:16.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.937+0000 7fbd73400700 1 -- 172.21.15.107:0/2927827742 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd6c009940 con 0x7fbd7c1038f0 2024-03-04T13:54:16.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.937+0000 7fbd73400700 1 --2- 172.21.15.107:0/2927827742 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd7c1038f0 0x7fbd7c1140c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fbd6400c800 tx=0x7fbd64004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:54:16.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.937+0000 7fbd78c00700 1 -- 172.21.15.107:0/2927827742 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbd6400a760 con 0x7fbd7c1038f0 2024-03-04T13:54:16.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.937+0000 7fbd78c00700 1 -- 172.21.15.107:0/2927827742 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbd6400ad90 con 0x7fbd7c1038f0 2024-03-04T13:54:16.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.937+0000 7fbd78c00700 1 -- 172.21.15.107:0/2927827742 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbd640113a0 con 0x7fbd7c1038f0 2024-03-04T13:54:16.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.937+0000 7fbd82e00700 1 -- 172.21.15.107:0/2927827742 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd7c10ddb0 con 0x7fbd7c1038f0 2024-03-04T13:54:16.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.938+0000 7fbd82e00700 1 -- 172.21.15.107:0/2927827742 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd7c10e2f0 con 0x7fbd7c1038f0 2024-03-04T13:54:16.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.939+0000 7fbd82e00700 1 -- 172.21.15.107:0/2927827742 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd7c100260 con 0x7fbd7c1038f0 2024-03-04T13:54:16.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.940+0000 7fbd78c00700 1 -- 172.21.15.107:0/2927827742 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fbd64012400 con 0x7fbd7c1038f0 2024-03-04T13:54:16.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.940+0000 7fbd78c00700 1 --2- 172.21.15.107:0/2927827742 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fbd6006d7f0 0x7fbd6006fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:54:16.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.941+0000 7fbd78c00700 1 -- 172.21.15.107:0/2927827742 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fbd6408ceb0 con 0x7fbd7c1038f0 2024-03-04T13:54:16.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.941+0000 7fbd7be00700 1 --2- 172.21.15.107:0/2927827742 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fbd6006d7f0 0x7fbd6006fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:54:16.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.941+0000 7fbd7be00700 1 --2- 172.21.15.107:0/2927827742 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fbd6006d7f0 0x7fbd6006fca0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c00b740 tx=0x7fbd6c004d10 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:54:16.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:16.945+0000 7fbd78c00700 1 -- 172.21.15.107:0/2927827742 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbd64056730 con 0x7fbd7c1038f0 2024-03-04T13:54:17.093 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:16 smithi169 ceph-mon[86183]: from='client.34498 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:54:17.094 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:16 smithi107 ceph-mon[112578]: from='client.34498 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:54:17.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:17.234+0000 7fbd82e00700 1 -- 172.21.15.107:0/2927827742 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fbd7c10a080 con 0x7fbd7c1038f0 2024-03-04T13:54:17.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:17.235+0000 7fbd78c00700 1 -- 172.21.15.107:0/2927827742 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fbd64059df0 con 0x7fbd7c1038f0 2024-03-04T13:54:17.237 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-03-04T13:54:17.237 INFO:teuthology.orchestra.run.smithi107.stdout: "mon": { 2024-03-04T13:54:17.237 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-1845-g717ce3cc (717ce3cce8d0166e4e577faa8004e6f9cb4128c0) squid (dev)": 2 2024-03-04T13:54:17.237 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:54:17.237 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr": { 2024-03-04T13:54:17.237 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-1845-g717ce3cc (717ce3cce8d0166e4e577faa8004e6f9cb4128c0) squid (dev)": 2 2024-03-04T13:54:17.237 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:54:17.237 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": { 2024-03-04T13:54:17.238 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-1845-g717ce3cc (717ce3cce8d0166e4e577faa8004e6f9cb4128c0) squid (dev)": 6 2024-03-04T13:54:17.238 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:54:17.238 INFO:teuthology.orchestra.run.smithi107.stdout: "mds": { 2024-03-04T13:54:17.238 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-1845-g717ce3cc (717ce3cce8d0166e4e577faa8004e6f9cb4128c0) squid (dev)": 4 2024-03-04T13:54:17.238 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:54:17.238 INFO:teuthology.orchestra.run.smithi107.stdout: "overall": { 2024-03-04T13:54:17.238 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-1845-g717ce3cc (717ce3cce8d0166e4e577faa8004e6f9cb4128c0) squid (dev)": 14 2024-03-04T13:54:17.238 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-04T13:54:17.238 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-03-04T13:54:17.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:17.237+0000 7fbd82e00700 1 -- 172.21.15.107:0/2927827742 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fbd6006d7f0 msgr2=0x7fbd6006fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:54:17.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:17.237+0000 7fbd82e00700 1 --2- 172.21.15.107:0/2927827742 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fbd6006d7f0 0x7fbd6006fca0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c00b740 tx=0x7fbd6c004d10 comp rx=0 tx=0).stop 2024-03-04T13:54:17.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:17.237+0000 7fbd82e00700 1 -- 172.21.15.107:0/2927827742 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd7c1038f0 msgr2=0x7fbd7c1140c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:54:17.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:17.237+0000 7fbd82e00700 1 --2- 172.21.15.107:0/2927827742 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd7c1038f0 0x7fbd7c1140c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fbd6400c800 tx=0x7fbd64004470 comp rx=0 tx=0).stop 2024-03-04T13:54:17.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:17.238+0000 7fbd82e00700 1 -- 172.21.15.107:0/2927827742 shutdown_connections 2024-03-04T13:54:17.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:17.238+0000 7fbd82e00700 1 --2- 172.21.15.107:0/2927827742 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fbd6006d7f0 0x7fbd6006fca0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:54:17.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:17.238+0000 7fbd82e00700 1 --2- 172.21.15.107:0/2927827742 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbd7c102f50 0x7fbd7c113b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:54:17.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:17.238+0000 7fbd82e00700 1 --2- 172.21.15.107:0/2927827742 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd7c1038f0 0x7fbd7c1140c0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:54:17.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:17.238+0000 7fbd82e00700 1 -- 172.21.15.107:0/2927827742 >> 172.21.15.107:0/2927827742 conn(0x7fbd7c0f8ae0 msgr2=0x7fbd7c0fa5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:54:17.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:17.238+0000 7fbd82e00700 1 -- 172.21.15.107:0/2927827742 shutdown_connections 2024-03-04T13:54:17.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:17.238+0000 7fbd82e00700 1 -- 172.21.15.107:0/2927827742 wait complete. 2024-03-04T13:54:17.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:17.385+0000 7f1974000700 1 -- 172.21.15.107:0/4071311368 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f196c0fbdd0 msgr2=0x7f196c0fc1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:54:17.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:17.385+0000 7f1974000700 1 --2- 172.21.15.107:0/4071311368 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f196c0fbdd0 0x7f196c0fc1a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f1960004940 tx=0x7f1960009e30 comp rx=0 tx=0).stop 2024-03-04T13:54:17.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:17.385+0000 7f1974000700 1 -- 172.21.15.107:0/4071311368 shutdown_connections 2024-03-04T13:54:17.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:17.385+0000 7f1974000700 1 --2- 172.21.15.107:0/4071311368 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f196c0fc6e0 0x7f196c109440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:54:17.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:17.385+0000 7f1974000700 1 --2- 172.21.15.107:0/4071311368 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f196c0fbdd0 0x7f196c0fc1a0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:54:17.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:17.385+0000 7f1974000700 1 -- 172.21.15.107:0/4071311368 >> 172.21.15.107:0/4071311368 conn(0x7f196c071d90 msgr2=0x7f196c072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:54:17.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:17.385+0000 7f1974000700 1 -- 172.21.15.107:0/4071311368 shutdown_connections 2024-03-04T13:54:17.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:17.385+0000 7f1974000700 1 -- 172.21.15.107:0/4071311368 wait complete. 2024-03-04T13:54:17.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:17.386+0000 7f1974000700 1 Processor -- start 2024-03-04T13:54:17.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:17.386+0000 7f1974000700 1 -- start start 2024-03-04T13:54:17.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:17.387+0000 7f1974000700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f196c0fbdd0 0x7f196c115c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:54:17.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:17.387+0000 7f1974000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f196c0fc6e0 0x7f196c1161c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:54:17.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:17.387+0000 7f1974000700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f196c116810 con 0x7f196c0fc6e0 2024-03-04T13:54:17.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:17.387+0000 7f1974000700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f196c10fb40 con 0x7f196c0fbdd0 2024-03-04T13:54:17.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:17.387+0000 7f1970c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f196c0fc6e0 0x7f196c1161c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:54:17.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:17.387+0000 7f1970c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f196c0fc6e0 0x7f196c1161c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51782/0 (socket says 172.21.15.107:51782) 2024-03-04T13:54:17.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:17.387+0000 7f1970c00700 1 -- 172.21.15.107:0/4094559023 learned_addr learned my addr 172.21.15.107:0/4094559023 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:54:17.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:17.387+0000 7f1971600700 1 --2- 172.21.15.107:0/4094559023 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f196c0fbdd0 0x7f196c115c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:54:17.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:17.387+0000 7f1970c00700 1 -- 172.21.15.107:0/4094559023 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f196c0fbdd0 msgr2=0x7f196c115c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:54:17.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:17.387+0000 7f1970c00700 1 --2- 172.21.15.107:0/4094559023 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f196c0fbdd0 0x7f196c115c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:54:17.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:17.387+0000 7f1970c00700 1 -- 172.21.15.107:0/4094559023 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1960005040 con 0x7f196c0fc6e0 2024-03-04T13:54:17.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:17.387+0000 7f1971600700 1 --2- 172.21.15.107:0/4094559023 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f196c0fbdd0 0x7f196c115c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T13:54:17.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:17.388+0000 7f1970c00700 1 --2- 172.21.15.107:0/4094559023 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f196c0fc6e0 0x7f196c1161c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1958002a50 tx=0x7f19580046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:54:17.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:17.388+0000 7f1969600700 1 -- 172.21.15.107:0/4094559023 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1958007cc0 con 0x7f196c0fc6e0 2024-03-04T13:54:17.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:17.388+0000 7f1969600700 1 -- 172.21.15.107:0/4094559023 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f195800f040 con 0x7f196c0fc6e0 2024-03-04T13:54:17.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:17.388+0000 7f1969600700 1 -- 172.21.15.107:0/4094559023 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f195800db70 con 0x7f196c0fc6e0 2024-03-04T13:54:17.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:17.388+0000 7f1974000700 1 -- 172.21.15.107:0/4094559023 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f196c10fe20 con 0x7f196c0fc6e0 2024-03-04T13:54:17.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:17.388+0000 7f1974000700 1 -- 172.21.15.107:0/4094559023 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f196c110360 con 0x7f196c0fc6e0 2024-03-04T13:54:17.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:17.389+0000 7f1974000700 1 -- 172.21.15.107:0/4094559023 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f196c106b70 con 0x7f196c0fc6e0 2024-03-04T13:54:17.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:17.390+0000 7f1969600700 1 -- 172.21.15.107:0/4094559023 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f195800dd10 con 0x7f196c0fc6e0 2024-03-04T13:54:17.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:17.391+0000 7f1969600700 1 --2- 172.21.15.107:0/4094559023 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f195406d7f0 0x7f195406fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:54:17.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:17.391+0000 7f1969600700 1 -- 172.21.15.107:0/4094559023 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f195808c5c0 con 0x7f196c0fc6e0 2024-03-04T13:54:17.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:17.392+0000 7f1971600700 1 --2- 172.21.15.107:0/4094559023 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f195406d7f0 0x7f195406fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:54:17.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:17.392+0000 7f1971600700 1 --2- 172.21.15.107:0/4094559023 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f195406d7f0 0x7f195406fca0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f196000b3b0 tx=0x7f1960009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:54:17.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:17.396+0000 7f1969600700 1 -- 172.21.15.107:0/4094559023 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f19580598f0 con 0x7f196c0fc6e0 2024-03-04T13:54:17.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:17.664+0000 7f1974000700 1 -- 172.21.15.107:0/4094559023 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f196c04bed0 con 0x7f196c0fc6e0 2024-03-04T13:54:17.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:17.665+0000 7f1969600700 1 -- 172.21.15.107:0/4094559023 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 33 v33) v1 ==== 76+0+1876 (secure 0 0 0) 0x7f1958059480 con 0x7f196c0fc6e0 2024-03-04T13:54:17.667 INFO:teuthology.orchestra.run.smithi107.stdout:e33 2024-03-04T13:54:17.667 INFO:teuthology.orchestra.run.smithi107.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-04T13:54:17.667 INFO:teuthology.orchestra.run.smithi107.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-04T13:54:17.667 INFO:teuthology.orchestra.run.smithi107.stdout:legacy client fscid: 1 2024-03-04T13:54:17.668 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:54:17.668 INFO:teuthology.orchestra.run.smithi107.stdout:Filesystem 'cephfs' (1) 2024-03-04T13:54:17.668 INFO:teuthology.orchestra.run.smithi107.stdout:fs_name cephfs 2024-03-04T13:54:17.668 INFO:teuthology.orchestra.run.smithi107.stdout:epoch 33 2024-03-04T13:54:17.668 INFO:teuthology.orchestra.run.smithi107.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-04T13:54:17.668 INFO:teuthology.orchestra.run.smithi107.stdout:created 2024-03-04T13:38:30.854004+0000 2024-03-04T13:54:17.668 INFO:teuthology.orchestra.run.smithi107.stdout:modified 2024-03-04T13:54:09.979403+0000 2024-03-04T13:54:17.668 INFO:teuthology.orchestra.run.smithi107.stdout:tableserver 0 2024-03-04T13:54:17.668 INFO:teuthology.orchestra.run.smithi107.stdout:root 0 2024-03-04T13:54:17.668 INFO:teuthology.orchestra.run.smithi107.stdout:session_timeout 60 2024-03-04T13:54:17.668 INFO:teuthology.orchestra.run.smithi107.stdout:session_autoclose 300 2024-03-04T13:54:17.668 INFO:teuthology.orchestra.run.smithi107.stdout:max_file_size 1099511627776 2024-03-04T13:54:17.668 INFO:teuthology.orchestra.run.smithi107.stdout:max_xattr_size 65536 2024-03-04T13:54:17.669 INFO:teuthology.orchestra.run.smithi107.stdout:required_client_features {} 2024-03-04T13:54:17.669 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure 0 2024-03-04T13:54:17.669 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure_osd_epoch 86 2024-03-04T13:54:17.669 INFO:teuthology.orchestra.run.smithi107.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-04T13:54:17.669 INFO:teuthology.orchestra.run.smithi107.stdout:max_mds 1 2024-03-04T13:54:17.669 INFO:teuthology.orchestra.run.smithi107.stdout:in 0 2024-03-04T13:54:17.669 INFO:teuthology.orchestra.run.smithi107.stdout:up {0=34432} 2024-03-04T13:54:17.669 INFO:teuthology.orchestra.run.smithi107.stdout:failed 2024-03-04T13:54:17.669 INFO:teuthology.orchestra.run.smithi107.stdout:damaged 2024-03-04T13:54:17.669 INFO:teuthology.orchestra.run.smithi107.stdout:stopped 2024-03-04T13:54:17.669 INFO:teuthology.orchestra.run.smithi107.stdout:data_pools [3] 2024-03-04T13:54:17.669 INFO:teuthology.orchestra.run.smithi107.stdout:metadata_pool 2 2024-03-04T13:54:17.670 INFO:teuthology.orchestra.run.smithi107.stdout:inline_data enabled 2024-03-04T13:54:17.670 INFO:teuthology.orchestra.run.smithi107.stdout:balancer 2024-03-04T13:54:17.670 INFO:teuthology.orchestra.run.smithi107.stdout:bal_rank_mask -1 2024-03-04T13:54:17.670 INFO:teuthology.orchestra.run.smithi107.stdout:standby_count_wanted 1 2024-03-04T13:54:17.670 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.riexmm{0:34432} state up:active seq 24 join_fscid=1 addr [v2:172.21.15.107:6826/1144672731,v1:172.21.15.107:6827/1144672731] compat {c=[1],r=[1],i=[fff]}] 2024-03-04T13:54:17.670 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:54:17.670 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:54:17.670 INFO:teuthology.orchestra.run.smithi107.stdout:Standby daemons: 2024-03-04T13:54:17.670 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:54:17.670 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.thycho{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.107:6828/3010096376,v1:172.21.15.107:6829/3010096376] compat {c=[1],r=[1],i=[fff]}] 2024-03-04T13:54:17.670 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi169.jusudl{-1:44243} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.169:6824/1336978861,v1:172.21.15.169:6825/1336978861] compat {c=[1],r=[1],i=[fff]}] 2024-03-04T13:54:17.670 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi169.bxitzk{-1:44259} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.169:6826/4015044684,v1:172.21.15.169:6827/4015044684] compat {c=[1],r=[1],i=[fff]}] 2024-03-04T13:54:17.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:17.667+0000 7f1974000700 1 -- 172.21.15.107:0/4094559023 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f195406d7f0 msgr2=0x7f195406fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:54:17.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:17.667+0000 7f1974000700 1 --2- 172.21.15.107:0/4094559023 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f195406d7f0 0x7f195406fca0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f196000b3b0 tx=0x7f1960009e30 comp rx=0 tx=0).stop 2024-03-04T13:54:17.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:17.667+0000 7f1974000700 1 -- 172.21.15.107:0/4094559023 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f196c0fc6e0 msgr2=0x7f196c1161c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:54:17.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:17.667+0000 7f1974000700 1 --2- 172.21.15.107:0/4094559023 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f196c0fc6e0 0x7f196c1161c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1958002a50 tx=0x7f19580046b0 comp rx=0 tx=0).stop 2024-03-04T13:54:17.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:17.667+0000 7f1974000700 1 -- 172.21.15.107:0/4094559023 shutdown_connections 2024-03-04T13:54:17.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:17.667+0000 7f1974000700 1 --2- 172.21.15.107:0/4094559023 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f195406d7f0 0x7f195406fca0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:54:17.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:17.668+0000 7f1974000700 1 --2- 172.21.15.107:0/4094559023 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f196c0fbdd0 0x7f196c115c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:54:17.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:17.668+0000 7f1974000700 1 --2- 172.21.15.107:0/4094559023 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f196c0fc6e0 0x7f196c1161c0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:54:17.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:17.668+0000 7f1974000700 1 -- 172.21.15.107:0/4094559023 >> 172.21.15.107:0/4094559023 conn(0x7f196c071d90 msgr2=0x7f196c0fa9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:54:17.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:17.668+0000 7f1974000700 1 -- 172.21.15.107:0/4094559023 shutdown_connections 2024-03-04T13:54:17.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:17.668+0000 7f1974000700 1 -- 172.21.15.107:0/4094559023 wait complete. 2024-03-04T13:54:17.672 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 33 2024-03-04T13:54:17.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:17.816+0000 7f9205c00700 1 -- 172.21.15.107:0/2929718201 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92001095b0 msgr2=0x7f9200109980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:54:17.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:17.816+0000 7f9205c00700 1 --2- 172.21.15.107:0/2929718201 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92001095b0 0x7f9200109980 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f91f0004940 tx=0x7f91f0009e30 comp rx=0 tx=0).stop 2024-03-04T13:54:17.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:17.816+0000 7f9205c00700 1 -- 172.21.15.107:0/2929718201 shutdown_connections 2024-03-04T13:54:17.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:17.816+0000 7f9205c00700 1 --2- 172.21.15.107:0/2929718201 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9200103dc0 0x7f9200104230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:54:17.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:17.816+0000 7f9205c00700 1 --2- 172.21.15.107:0/2929718201 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92001095b0 0x7f9200109980 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:54:17.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:17.816+0000 7f9205c00700 1 -- 172.21.15.107:0/2929718201 >> 172.21.15.107:0/2929718201 conn(0x7f9200072d70 msgr2=0x7f9200075180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:54:17.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:17.816+0000 7f9205c00700 1 -- 172.21.15.107:0/2929718201 shutdown_connections 2024-03-04T13:54:17.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:17.816+0000 7f9205c00700 1 -- 172.21.15.107:0/2929718201 wait complete. 2024-03-04T13:54:17.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:17.817+0000 7f9205c00700 1 Processor -- start 2024-03-04T13:54:17.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:17.817+0000 7f9205c00700 1 -- start start 2024-03-04T13:54:17.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:17.817+0000 7f9205c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9200103dc0 0x7f920011a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:54:17.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:17.817+0000 7f9205c00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f92001095b0 0x7f920011a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:54:17.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:17.817+0000 7f9205c00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f920011ac30 con 0x7f9200103dc0 2024-03-04T13:54:17.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:17.817+0000 7f9205c00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9200113f40 con 0x7f92001095b0 2024-03-04T13:54:17.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:17.818+0000 7f91fea00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9200103dc0 0x7f920011a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:54:17.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:17.818+0000 7f91fea00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9200103dc0 0x7f920011a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51792/0 (socket says 172.21.15.107:51792) 2024-03-04T13:54:17.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:17.818+0000 7f91fea00700 1 -- 172.21.15.107:0/2461696758 learned_addr learned my addr 172.21.15.107:0/2461696758 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:54:17.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:17.818+0000 7f91fe000700 1 --2- 172.21.15.107:0/2461696758 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f92001095b0 0x7f920011a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:54:17.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:17.818+0000 7f91fea00700 1 -- 172.21.15.107:0/2461696758 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f92001095b0 msgr2=0x7f920011a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:54:17.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:17.818+0000 7f91fea00700 1 --2- 172.21.15.107:0/2461696758 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f92001095b0 0x7f920011a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:54:17.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:17.818+0000 7f91fea00700 1 -- 172.21.15.107:0/2461696758 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91f0005040 con 0x7f9200103dc0 2024-03-04T13:54:17.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:17.818+0000 7f91fea00700 1 --2- 172.21.15.107:0/2461696758 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9200103dc0 0x7f920011a0a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f91f0009100 tx=0x7f91f0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:54:17.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:17.819+0000 7f91f6a00700 1 -- 172.21.15.107:0/2461696758 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f91f000e040 con 0x7f9200103dc0 2024-03-04T13:54:17.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:17.819+0000 7f91f6a00700 1 -- 172.21.15.107:0/2461696758 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f91f0004290 con 0x7f9200103dc0 2024-03-04T13:54:17.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:17.819+0000 7f9205c00700 1 -- 172.21.15.107:0/2461696758 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92001141c0 con 0x7f9200103dc0 2024-03-04T13:54:17.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:17.819+0000 7f91f6a00700 1 -- 172.21.15.107:0/2461696758 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f91f001c3f0 con 0x7f9200103dc0 2024-03-04T13:54:17.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:17.819+0000 7f9205c00700 1 -- 172.21.15.107:0/2461696758 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9200114700 con 0x7f9200103dc0 2024-03-04T13:54:17.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:17.820+0000 7f9205c00700 1 -- 172.21.15.107:0/2461696758 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9200106850 con 0x7f9200103dc0 2024-03-04T13:54:17.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:17.821+0000 7f91f6a00700 1 -- 172.21.15.107:0/2461696758 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f91f000bbf0 con 0x7f9200103dc0 2024-03-04T13:54:17.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:17.822+0000 7f91f6a00700 1 --2- 172.21.15.107:0/2461696758 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f91e406d7f0 0x7f91e406fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:54:17.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:17.822+0000 7f91f6a00700 1 -- 172.21.15.107:0/2461696758 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f91f008e2c0 con 0x7f9200103dc0 2024-03-04T13:54:17.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:17.822+0000 7f91fe000700 1 --2- 172.21.15.107:0/2461696758 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f91e406d7f0 0x7f91e406fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:54:17.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:17.823+0000 7f91fe000700 1 --2- 172.21.15.107:0/2461696758 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f91e406d7f0 0x7f91e406fca0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9200115940 tx=0x7f91ec009ee0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:54:17.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:17.827+0000 7f91f6a00700 1 -- 172.21.15.107:0/2461696758 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f91f005b5f0 con 0x7f9200103dc0 2024-03-04T13:54:18.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:18.063+0000 7f9205c00700 1 -- 172.21.15.107:0/2461696758 --> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f920004bed0 con 0x7f91e406d7f0 2024-03-04T13:54:18.066 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:17 smithi107 ceph-mon[112578]: from='client.34502 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:54:18.067 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:17 smithi107 ceph-mon[112578]: from='client.34506 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:54:18.067 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:17 smithi107 ceph-mon[112578]: pgmap v245: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 10 MiB/s rd, 170 B/s wr, 5 op/s 2024-03-04T13:54:18.067 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:17 smithi107 ceph-mon[112578]: from='client.? 172.21.15.107:0/2927827742' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:54:18.067 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:17 smithi107 ceph-mon[112578]: from='client.? 172.21.15.107:0/4094559023' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-04T13:54:18.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:18.066+0000 7f91f6a00700 1 -- 172.21.15.107:0/2461696758 <== mgr.34102 v2:172.21.15.107:6800/1223039812 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f920004bed0 con 0x7f91e406d7f0 2024-03-04T13:54:18.069 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-03-04T13:54:18.069 INFO:teuthology.orchestra.run.smithi107.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377", 2024-03-04T13:54:18.069 INFO:teuthology.orchestra.run.smithi107.stdout: "in_progress": true, 2024-03-04T13:54:18.069 INFO:teuthology.orchestra.run.smithi107.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-04T13:54:18.069 INFO:teuthology.orchestra.run.smithi107.stdout: "services_complete": [ 2024-03-04T13:54:18.069 INFO:teuthology.orchestra.run.smithi107.stdout: "crash", 2024-03-04T13:54:18.070 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr", 2024-03-04T13:54:18.070 INFO:teuthology.orchestra.run.smithi107.stdout: "mds", 2024-03-04T13:54:18.070 INFO:teuthology.orchestra.run.smithi107.stdout: "mon", 2024-03-04T13:54:18.070 INFO:teuthology.orchestra.run.smithi107.stdout: "osd" 2024-03-04T13:54:18.070 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-04T13:54:18.070 INFO:teuthology.orchestra.run.smithi107.stdout: "progress": "16/21 daemons upgraded", 2024-03-04T13:54:18.070 INFO:teuthology.orchestra.run.smithi107.stdout: "message": "Currently upgrading mds daemons", 2024-03-04T13:54:18.070 INFO:teuthology.orchestra.run.smithi107.stdout: "is_paused": false 2024-03-04T13:54:18.070 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-03-04T13:54:18.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:18.069+0000 7f9205c00700 1 -- 172.21.15.107:0/2461696758 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f91e406d7f0 msgr2=0x7f91e406fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:54:18.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:18.069+0000 7f9205c00700 1 --2- 172.21.15.107:0/2461696758 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f91e406d7f0 0x7f91e406fca0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9200115940 tx=0x7f91ec009ee0 comp rx=0 tx=0).stop 2024-03-04T13:54:18.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:18.070+0000 7f9205c00700 1 -- 172.21.15.107:0/2461696758 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9200103dc0 msgr2=0x7f920011a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:54:18.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:18.070+0000 7f9205c00700 1 --2- 172.21.15.107:0/2461696758 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9200103dc0 0x7f920011a0a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f91f0009100 tx=0x7f91f0009e30 comp rx=0 tx=0).stop 2024-03-04T13:54:18.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:18.070+0000 7f9205c00700 1 -- 172.21.15.107:0/2461696758 shutdown_connections 2024-03-04T13:54:18.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:18.070+0000 7f9205c00700 1 --2- 172.21.15.107:0/2461696758 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f91e406d7f0 0x7f91e406fca0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:54:18.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:18.070+0000 7f9205c00700 1 --2- 172.21.15.107:0/2461696758 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9200103dc0 0x7f920011a0a0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:54:18.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:18.070+0000 7f9205c00700 1 --2- 172.21.15.107:0/2461696758 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f92001095b0 0x7f920011a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:54:18.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:18.070+0000 7f9205c00700 1 -- 172.21.15.107:0/2461696758 >> 172.21.15.107:0/2461696758 conn(0x7f9200072d70 msgr2=0x7f9200074890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:54:18.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:18.070+0000 7f9205c00700 1 -- 172.21.15.107:0/2461696758 shutdown_connections 2024-03-04T13:54:18.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:18.070+0000 7f9205c00700 1 -- 172.21.15.107:0/2461696758 wait complete. 2024-03-04T13:54:18.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:17 smithi169 ceph-mon[86183]: from='client.34502 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:54:18.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:17 smithi169 ceph-mon[86183]: from='client.34506 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:54:18.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:17 smithi169 ceph-mon[86183]: pgmap v245: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 10 MiB/s rd, 170 B/s wr, 5 op/s 2024-03-04T13:54:18.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:17 smithi169 ceph-mon[86183]: from='client.? 172.21.15.107:0/2927827742' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:54:18.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:17 smithi169 ceph-mon[86183]: from='client.? 172.21.15.107:0/4094559023' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-04T13:54:18.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:18.217+0000 7f143de00700 1 -- 172.21.15.107:0/649821110 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1438070870 msgr2=0x7f1438070ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:54:18.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:18.217+0000 7f143de00700 1 --2- 172.21.15.107:0/649821110 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1438070870 0x7f1438070ce0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1424004970 tx=0x7f1424009e30 comp rx=0 tx=0).stop 2024-03-04T13:54:18.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:18.217+0000 7f143de00700 1 -- 172.21.15.107:0/649821110 shutdown_connections 2024-03-04T13:54:18.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:18.217+0000 7f143de00700 1 --2- 172.21.15.107:0/649821110 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1438070870 0x7f1438070ce0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:54:18.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:18.217+0000 7f143de00700 1 --2- 172.21.15.107:0/649821110 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1438071f60 0x7f1438070330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:54:18.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:18.217+0000 7f143de00700 1 -- 172.21.15.107:0/649821110 >> 172.21.15.107:0/649821110 conn(0x7f14380f8ae0 msgr2=0x7f14380faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:54:18.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:18.217+0000 7f143de00700 1 -- 172.21.15.107:0/649821110 shutdown_connections 2024-03-04T13:54:18.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:18.217+0000 7f143de00700 1 -- 172.21.15.107:0/649821110 wait complete. 2024-03-04T13:54:18.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:18.218+0000 7f143de00700 1 Processor -- start 2024-03-04T13:54:18.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:18.218+0000 7f143de00700 1 -- start start 2024-03-04T13:54:18.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:18.218+0000 7f143de00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1438070870 0x7f143811a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:54:18.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:18.218+0000 7f143de00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1438071f60 0x7f143811a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:54:18.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:18.218+0000 7f143de00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f143811abc0 con 0x7f1438070870 2024-03-04T13:54:18.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:18.218+0000 7f143de00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1438113ed0 con 0x7f1438071f60 2024-03-04T13:54:18.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:18.219+0000 7f1436a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1438070870 0x7f143811a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:54:18.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:18.219+0000 7f1436a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1438070870 0x7f143811a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33952/0 (socket says 172.21.15.107:33952) 2024-03-04T13:54:18.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:18.219+0000 7f1436a00700 1 -- 172.21.15.107:0/3987027367 learned_addr learned my addr 172.21.15.107:0/3987027367 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:54:18.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:18.219+0000 7f1436000700 1 --2- 172.21.15.107:0/3987027367 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1438071f60 0x7f143811a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:54:18.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:18.219+0000 7f1436a00700 1 -- 172.21.15.107:0/3987027367 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1438071f60 msgr2=0x7f143811a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:54:18.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:18.219+0000 7f1436a00700 1 --2- 172.21.15.107:0/3987027367 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1438071f60 0x7f143811a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:54:18.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:18.219+0000 7f1436a00700 1 -- 172.21.15.107:0/3987027367 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1424005040 con 0x7f1438070870 2024-03-04T13:54:18.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:18.219+0000 7f1436a00700 1 --2- 172.21.15.107:0/3987027367 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1438070870 0x7f143811a030 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1428002a50 tx=0x7f14280046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:54:18.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:18.220+0000 7f142ea00700 1 -- 172.21.15.107:0/3987027367 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1428007cc0 con 0x7f1438070870 2024-03-04T13:54:18.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:18.220+0000 7f142ea00700 1 -- 172.21.15.107:0/3987027367 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f142800f040 con 0x7f1438070870 2024-03-04T13:54:18.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:18.220+0000 7f142ea00700 1 -- 172.21.15.107:0/3987027367 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1428009640 con 0x7f1438070870 2024-03-04T13:54:18.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:18.220+0000 7f143de00700 1 -- 172.21.15.107:0/3987027367 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14381141b0 con 0x7f1438070870 2024-03-04T13:54:18.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:18.220+0000 7f143de00700 1 -- 172.21.15.107:0/3987027367 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14381146f0 con 0x7f1438070870 2024-03-04T13:54:18.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:18.221+0000 7f143de00700 1 -- 172.21.15.107:0/3987027367 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14380fe400 con 0x7f1438070870 2024-03-04T13:54:18.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:18.222+0000 7f142ea00700 1 -- 172.21.15.107:0/3987027367 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f14280097a0 con 0x7f1438070870 2024-03-04T13:54:18.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:18.222+0000 7f142ea00700 1 --2- 172.21.15.107:0/3987027367 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f141c06d7f0 0x7f141c06fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:54:18.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:18.223+0000 7f142ea00700 1 -- 172.21.15.107:0/3987027367 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f142808cc90 con 0x7f1438070870 2024-03-04T13:54:18.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:18.223+0000 7f1436000700 1 --2- 172.21.15.107:0/3987027367 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f141c06d7f0 0x7f141c06fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:54:18.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:18.223+0000 7f1436000700 1 --2- 172.21.15.107:0/3987027367 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f141c06d7f0 0x7f141c06fca0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1424000c00 tx=0x7f1424009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:54:18.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:18.227+0000 7f142ea00700 1 -- 172.21.15.107:0/3987027367 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1428059fc0 con 0x7f1438070870 2024-03-04T13:54:18.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:18.513+0000 7f143de00700 1 -- 172.21.15.107:0/3987027367 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f143804bed0 con 0x7f1438070870 2024-03-04T13:54:18.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:18.514+0000 7f142ea00700 1 -- 172.21.15.107:0/3987027367 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f1428059b50 con 0x7f1438070870 2024-03-04T13:54:18.516 INFO:teuthology.orchestra.run.smithi107.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-03-04T13:54:18.516 INFO:teuthology.orchestra.run.smithi107.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-04T13:54:18.516 INFO:teuthology.orchestra.run.smithi107.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-04T13:54:18.519 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:18.517+0000 7f143de00700 1 -- 172.21.15.107:0/3987027367 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f141c06d7f0 msgr2=0x7f141c06fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:54:18.519 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:18.517+0000 7f143de00700 1 --2- 172.21.15.107:0/3987027367 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f141c06d7f0 0x7f141c06fca0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1424000c00 tx=0x7f1424009e30 comp rx=0 tx=0).stop 2024-03-04T13:54:18.519 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:18.517+0000 7f143de00700 1 -- 172.21.15.107:0/3987027367 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1438070870 msgr2=0x7f143811a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:54:18.519 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:18.517+0000 7f143de00700 1 --2- 172.21.15.107:0/3987027367 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1438070870 0x7f143811a030 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1428002a50 tx=0x7f14280046b0 comp rx=0 tx=0).stop 2024-03-04T13:54:18.519 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:18.517+0000 7f143de00700 1 -- 172.21.15.107:0/3987027367 shutdown_connections 2024-03-04T13:54:18.519 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:18.517+0000 7f143de00700 1 --2- 172.21.15.107:0/3987027367 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f141c06d7f0 0x7f141c06fca0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:54:18.519 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:18.517+0000 7f143de00700 1 --2- 172.21.15.107:0/3987027367 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1438070870 0x7f143811a030 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:54:18.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:18.517+0000 7f143de00700 1 --2- 172.21.15.107:0/3987027367 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1438071f60 0x7f143811a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:54:18.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:18.517+0000 7f143de00700 1 -- 172.21.15.107:0/3987027367 >> 172.21.15.107:0/3987027367 conn(0x7f14380f8ae0 msgr2=0x7f14380fa750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:54:18.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:18.517+0000 7f143de00700 1 -- 172.21.15.107:0/3987027367 shutdown_connections 2024-03-04T13:54:18.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:18.517+0000 7f143de00700 1 -- 172.21.15.107:0/3987027367 wait complete. 2024-03-04T13:54:19.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:18 smithi107 ceph-mon[112578]: from='client.? 172.21.15.107:0/3987027367' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-04T13:54:19.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:18 smithi169 ceph-mon[86183]: from='client.? 172.21.15.107:0/3987027367' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-04T13:54:20.022 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:19 smithi169 ceph-mon[86183]: from='client.34518 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:54:20.022 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:19 smithi169 ceph-mon[86183]: pgmap v246: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 7.0 MiB/s rd, 4.2 KiB/s wr, 6 op/s 2024-03-04T13:54:20.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:19 smithi107 ceph-mon[112578]: from='client.34518 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:54:20.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:19 smithi107 ceph-mon[112578]: pgmap v246: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 7.0 MiB/s rd, 4.2 KiB/s wr, 6 op/s 2024-03-04T13:54:21.846 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:21 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:54:21.846 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:21 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:54:21.846 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:21 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:54:21.846 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:21 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:54:21.846 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:21 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:54:21.846 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:21 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:54:21.846 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:21 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:54:21.846 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:21 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:54:21.846 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:21 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:54:21.846 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:21 smithi107 ceph-mon[112578]: pgmap v247: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 686 KiB/s rd, 4.2 KiB/s wr, 5 op/s 2024-03-04T13:54:21.846 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:21 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:54:21.847 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:21 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:54:21.847 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:21 smithi107 ceph-mon[112578]: Upgrade: Setting container_image for all mds 2024-03-04T13:54:21.847 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:21 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:54:21.847 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:21 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi107.riexmm"}]: dispatch 2024-03-04T13:54:21.847 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:21 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi107.riexmm"}]': finished 2024-03-04T13:54:21.847 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:21 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi107.thycho"}]: dispatch 2024-03-04T13:54:21.847 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:21 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi107.thycho"}]': finished 2024-03-04T13:54:21.847 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:21 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi169.bxitzk"}]: dispatch 2024-03-04T13:54:21.847 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:21 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi169.bxitzk"}]': finished 2024-03-04T13:54:21.847 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:21 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi169.jusudl"}]: dispatch 2024-03-04T13:54:21.847 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:21 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi169.jusudl"}]': finished 2024-03-04T13:54:21.847 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:21 smithi107 ceph-mon[112578]: Upgrade: Setting filesystem cephfs Joinable 2024-03-04T13:54:21.847 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:21 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-03-04T13:54:21.862 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:21 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:54:21.863 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:21 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:54:21.863 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:21 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:54:21.863 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:21 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:54:21.863 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:21 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:54:21.863 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:21 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:54:21.863 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:21 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:54:21.863 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:21 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:54:21.863 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:21 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:54:21.863 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:21 smithi169 ceph-mon[86183]: pgmap v247: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 686 KiB/s rd, 4.2 KiB/s wr, 5 op/s 2024-03-04T13:54:21.863 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:21 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:54:21.863 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:21 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:54:21.863 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:21 smithi169 ceph-mon[86183]: Upgrade: Setting container_image for all mds 2024-03-04T13:54:21.863 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:21 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:54:21.864 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:21 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi107.riexmm"}]: dispatch 2024-03-04T13:54:21.864 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:21 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi107.riexmm"}]': finished 2024-03-04T13:54:21.864 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:21 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi107.thycho"}]: dispatch 2024-03-04T13:54:21.864 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:21 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi107.thycho"}]': finished 2024-03-04T13:54:21.864 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:21 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi169.bxitzk"}]: dispatch 2024-03-04T13:54:21.864 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:21 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi169.bxitzk"}]': finished 2024-03-04T13:54:21.864 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:21 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi169.jusudl"}]: dispatch 2024-03-04T13:54:21.864 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:21 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi169.jusudl"}]': finished 2024-03-04T13:54:21.864 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:21 smithi169 ceph-mon[86183]: Upgrade: Setting filesystem cephfs Joinable 2024-03-04T13:54:21.864 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:21 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-03-04T13:54:23.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:22 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-03-04T13:54:23.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:22 smithi169 ceph-mon[86183]: fsmap cephfs:1 {0=cephfs.smithi107.riexmm=up:active} 3 up:standby 2024-03-04T13:54:23.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:22 smithi169 ceph-mon[86183]: Upgrade: Enabling allow_standby_replay on filesystem cephfs 2024-03-04T13:54:23.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:22 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]: dispatch 2024-03-04T13:54:23.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:22 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-03-04T13:54:23.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:22 smithi107 ceph-mon[112578]: fsmap cephfs:1 {0=cephfs.smithi107.riexmm=up:active} 3 up:standby 2024-03-04T13:54:23.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:22 smithi107 ceph-mon[112578]: Upgrade: Enabling allow_standby_replay on filesystem cephfs 2024-03-04T13:54:23.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:22 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]: dispatch 2024-03-04T13:54:24.113 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:23 smithi169 ceph-mon[86183]: pgmap v248: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 344 KiB/s rd, 4.2 KiB/s wr, 5 op/s 2024-03-04T13:54:24.113 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:23 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]': finished 2024-03-04T13:54:24.113 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:23 smithi169 ceph-mon[86183]: fsmap cephfs:1 {0=cephfs.smithi107.riexmm=up:active} 3 up:standby 2024-03-04T13:54:24.113 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:23 smithi169 ceph-mon[86183]: fsmap cephfs:1 {0=cephfs.smithi107.riexmm=up:active} 1 up:standby-replay 2 up:standby 2024-03-04T13:54:24.113 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:23 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:54:24.113 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:23 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:54:24.113 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:23 smithi169 ceph-mon[86183]: Upgrade: Setting container_image for all rgw 2024-03-04T13:54:24.114 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:23 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:54:24.114 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:23 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:54:24.114 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:23 smithi169 ceph-mon[86183]: Upgrade: Setting container_image for all rbd-mirror 2024-03-04T13:54:24.114 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:23 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:54:24.114 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:23 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:54:24.114 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:23 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:54:24.114 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:23 smithi169 ceph-mon[86183]: Upgrade: Setting container_image for all ceph-exporter 2024-03-04T13:54:24.114 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:23 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:54:24.114 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:23 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:54:24.114 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:23 smithi169 ceph-mon[86183]: Upgrade: Setting container_image for all iscsi 2024-03-04T13:54:24.114 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:23 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:54:24.114 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:23 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:54:24.114 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:23 smithi169 ceph-mon[86183]: Upgrade: Setting container_image for all nfs 2024-03-04T13:54:24.115 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:23 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:54:24.115 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:23 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:54:24.115 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:23 smithi169 ceph-mon[86183]: Upgrade: Setting container_image for all nvmeof 2024-03-04T13:54:24.115 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:23 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:54:24.115 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:23 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:54:24.115 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:23 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:54:24.115 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:23 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:54:24.115 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:23 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:54:24.115 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:23 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:54:24.115 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:23 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:54:24.116 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:23 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-03-04T13:54:24.116 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:23 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-03-04T13:54:24.116 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:23 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-04T13:54:24.116 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:23 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-03-04T13:54:24.116 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:23 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-03-04T13:54:24.116 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:23 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-03-04T13:54:24.116 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:23 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-03-04T13:54:24.116 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:23 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-03-04T13:54:24.116 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:23 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-03-04T13:54:24.116 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:23 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-03-04T13:54:24.116 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:23 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-03-04T13:54:24.116 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:23 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-03-04T13:54:24.117 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:23 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-03-04T13:54:24.117 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:23 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-03-04T13:54:24.117 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:23 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-04T13:54:24.117 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:23 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-03-04T13:54:24.117 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:23 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-03-04T13:54:24.117 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:23 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-03-04T13:54:24.117 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:23 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-03-04T13:54:24.117 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:23 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-03-04T13:54:24.117 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:23 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-03-04T13:54:24.117 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:23 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-03-04T13:54:24.117 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:23 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-03-04T13:54:24.117 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:23 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-04T13:54:24.117 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:23 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-04T13:54:24.117 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:23 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-04T13:54:24.118 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:23 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-04T13:54:24.118 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:23 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-04T13:54:24.118 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:23 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-04T13:54:24.118 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:23 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-03-04T13:54:24.118 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:23 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-03-04T13:54:24.118 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:23 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:54:24.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:23 smithi107 ceph-mon[112578]: pgmap v248: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 344 KiB/s rd, 4.2 KiB/s wr, 5 op/s 2024-03-04T13:54:24.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:23 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]': finished 2024-03-04T13:54:24.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:23 smithi107 ceph-mon[112578]: fsmap cephfs:1 {0=cephfs.smithi107.riexmm=up:active} 3 up:standby 2024-03-04T13:54:24.346 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:23 smithi107 ceph-mon[112578]: fsmap cephfs:1 {0=cephfs.smithi107.riexmm=up:active} 1 up:standby-replay 2 up:standby 2024-03-04T13:54:24.346 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:23 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:54:24.346 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:23 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:54:24.346 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:23 smithi107 ceph-mon[112578]: Upgrade: Setting container_image for all rgw 2024-03-04T13:54:24.346 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:23 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:54:24.346 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:23 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:54:24.346 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:23 smithi107 ceph-mon[112578]: Upgrade: Setting container_image for all rbd-mirror 2024-03-04T13:54:24.346 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:23 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:54:24.346 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:23 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:54:24.346 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:23 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:54:24.346 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:23 smithi107 ceph-mon[112578]: Upgrade: Setting container_image for all ceph-exporter 2024-03-04T13:54:24.346 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:23 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:54:24.346 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:23 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:54:24.346 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:23 smithi107 ceph-mon[112578]: Upgrade: Setting container_image for all iscsi 2024-03-04T13:54:24.347 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:23 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:54:24.347 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:23 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:54:24.347 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:23 smithi107 ceph-mon[112578]: Upgrade: Setting container_image for all nfs 2024-03-04T13:54:24.347 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:23 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:54:24.347 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:23 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:54:24.347 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:23 smithi107 ceph-mon[112578]: Upgrade: Setting container_image for all nvmeof 2024-03-04T13:54:24.347 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:23 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:54:24.347 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:23 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:54:24.347 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:23 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:54:24.347 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:23 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:54:24.347 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:23 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:54:24.347 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:23 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:54:24.347 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:23 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:54:24.348 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:23 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-03-04T13:54:24.348 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:23 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-03-04T13:54:24.348 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:23 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-04T13:54:24.348 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:23 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-03-04T13:54:24.348 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:23 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-03-04T13:54:24.348 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:23 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-03-04T13:54:24.348 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:23 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-03-04T13:54:24.348 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:23 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-03-04T13:54:24.348 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:23 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-03-04T13:54:24.349 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:23 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-03-04T13:54:24.349 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:23 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-03-04T13:54:24.349 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:23 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-03-04T13:54:24.349 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:23 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-03-04T13:54:24.349 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:23 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-03-04T13:54:24.349 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:23 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-04T13:54:24.349 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:23 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-03-04T13:54:24.349 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:23 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-03-04T13:54:24.349 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:23 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-03-04T13:54:24.349 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:23 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-03-04T13:54:24.349 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:23 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-03-04T13:54:24.349 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:23 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-03-04T13:54:24.349 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:23 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-03-04T13:54:24.349 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:23 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-03-04T13:54:24.349 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:23 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-04T13:54:24.350 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:23 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-04T13:54:24.350 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:23 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-04T13:54:24.350 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:23 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-04T13:54:24.350 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:23 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-04T13:54:24.350 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:23 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-04T13:54:24.350 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:23 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-03-04T13:54:24.350 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:23 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-03-04T13:54:24.350 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:23 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:54:25.246 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:24 smithi107 ceph-mon[112578]: Upgrade: Finalizing container_image settings 2024-03-04T13:54:25.246 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:24 smithi107 ceph-mon[112578]: Upgrade: Complete! 2024-03-04T13:54:25.246 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:24 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:54:25.246 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:24 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:54:25.246 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:24 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:54:25.246 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:24 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:54:25.246 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:24 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:54:25.246 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:24 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:54:25.246 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:24 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:54:25.246 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:24 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:54:25.246 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:24 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:54:25.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:24 smithi169 ceph-mon[86183]: Upgrade: Finalizing container_image settings 2024-03-04T13:54:25.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:24 smithi169 ceph-mon[86183]: Upgrade: Complete! 2024-03-04T13:54:25.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:24 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:54:25.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:24 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:54:25.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:24 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:54:25.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:24 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:54:25.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:24 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:54:25.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:24 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:54:25.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:24 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:54:25.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:24 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:54:25.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:24 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:54:26.275 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:25 smithi107 ceph-mon[112578]: pgmap v249: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 684 KiB/s rd, 4.1 KiB/s wr, 2 op/s 2024-03-04T13:54:26.276 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:25 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:54:26.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:25 smithi169 ceph-mon[86183]: pgmap v249: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 684 KiB/s rd, 4.1 KiB/s wr, 2 op/s 2024-03-04T13:54:26.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:25 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:54:28.095 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:27 smithi107 ceph-mon[112578]: pgmap v250: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 3.3 MiB/s rd, 4.1 KiB/s wr, 3 op/s 2024-03-04T13:54:28.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:27 smithi169 ceph-mon[86183]: pgmap v250: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 3.3 MiB/s rd, 4.1 KiB/s wr, 3 op/s 2024-03-04T13:54:30.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:29 smithi107 ceph-mon[112578]: pgmap v251: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 13 MiB/s rd, 4.1 KiB/s wr, 5 op/s 2024-03-04T13:54:30.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:29 smithi169 ceph-mon[86183]: pgmap v251: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 13 MiB/s rd, 4.1 KiB/s wr, 5 op/s 2024-03-04T13:54:32.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:31 smithi107 ceph-mon[112578]: pgmap v252: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 13 MiB/s rd, 3 op/s 2024-03-04T13:54:32.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:31 smithi169 ceph-mon[86183]: pgmap v252: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 13 MiB/s rd, 3 op/s 2024-03-04T13:54:34.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:33 smithi107 ceph-mon[112578]: pgmap v253: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 5 op/s 2024-03-04T13:54:34.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:33 smithi169 ceph-mon[86183]: pgmap v253: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 5 op/s 2024-03-04T13:54:36.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:35 smithi107 ceph-mon[112578]: pgmap v254: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 5 op/s 2024-03-04T13:54:36.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:35 smithi169 ceph-mon[86183]: pgmap v254: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 5 op/s 2024-03-04T13:54:38.332 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:37 smithi107 ceph-mon[112578]: pgmap v255: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 20 MiB/s rd, 5 op/s 2024-03-04T13:54:38.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:37 smithi169 ceph-mon[86183]: pgmap v255: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 20 MiB/s rd, 5 op/s 2024-03-04T13:54:40.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:39 smithi107 ceph-mon[112578]: pgmap v256: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 18 MiB/s rd, 5 op/s 2024-03-04T13:54:40.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:39 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:54:40.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:39 smithi169 ceph-mon[86183]: pgmap v256: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 18 MiB/s rd, 5 op/s 2024-03-04T13:54:40.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:39 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:54:42.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:41 smithi107 ceph-mon[112578]: pgmap v257: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 8.3 MiB/s rd, 2 op/s 2024-03-04T13:54:42.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:41 smithi169 ceph-mon[86183]: pgmap v257: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 8.3 MiB/s rd, 2 op/s 2024-03-04T13:54:44.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:43 smithi107 ceph-mon[112578]: pgmap v258: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 8.0 MiB/s rd, 3 op/s 2024-03-04T13:54:44.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:43 smithi169 ceph-mon[86183]: pgmap v258: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 8.0 MiB/s rd, 3 op/s 2024-03-04T13:54:46.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:45 smithi107 ceph-mon[112578]: pgmap v259: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:54:46.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:45 smithi169 ceph-mon[86183]: pgmap v259: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:54:48.333 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:47 smithi107 ceph-mon[112578]: pgmap v260: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:54:48.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:47 smithi169 ceph-mon[86183]: pgmap v260: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:54:48.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:48.685+0000 7f930d400700 1 -- 172.21.15.107:0/466454686 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93081095f0 msgr2=0x7f93081099c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:54:48.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:48.685+0000 7f930d400700 1 --2- 172.21.15.107:0/466454686 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93081095f0 0x7f93081099c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f92f8004940 tx=0x7f92f8009e30 comp rx=0 tx=0).stop 2024-03-04T13:54:48.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:48.685+0000 7f930d400700 1 -- 172.21.15.107:0/466454686 shutdown_connections 2024-03-04T13:54:48.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:48.685+0000 7f930d400700 1 --2- 172.21.15.107:0/466454686 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9308103dc0 0x7f9308104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:54:48.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:48.685+0000 7f930d400700 1 --2- 172.21.15.107:0/466454686 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93081095f0 0x7f93081099c0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:54:48.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:48.685+0000 7f930d400700 1 -- 172.21.15.107:0/466454686 >> 172.21.15.107:0/466454686 conn(0x7f9308072d50 msgr2=0x7f9308075160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:54:48.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:48.685+0000 7f930d400700 1 -- 172.21.15.107:0/466454686 shutdown_connections 2024-03-04T13:54:48.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:48.685+0000 7f930d400700 1 -- 172.21.15.107:0/466454686 wait complete. 2024-03-04T13:54:48.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:48.686+0000 7f930d400700 1 Processor -- start 2024-03-04T13:54:48.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:48.686+0000 7f930d400700 1 -- start start 2024-03-04T13:54:48.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:48.687+0000 7f930d400700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9308103dc0 0x7f930811a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:54:48.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:48.687+0000 7f930d400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93081095f0 0x7f930811a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:54:48.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:48.687+0000 7f930d400700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f930811ac70 con 0x7f93081095f0 2024-03-04T13:54:48.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:48.687+0000 7f930d400700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9308113f80 con 0x7f9308103dc0 2024-03-04T13:54:48.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:48.687+0000 7f9305e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93081095f0 0x7f930811a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:54:48.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:48.687+0000 7f9305e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93081095f0 0x7f930811a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46588/0 (socket says 172.21.15.107:46588) 2024-03-04T13:54:48.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:48.687+0000 7f9305e00700 1 -- 172.21.15.107:0/1703389212 learned_addr learned my addr 172.21.15.107:0/1703389212 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:54:48.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:48.687+0000 7f9306800700 1 --2- 172.21.15.107:0/1703389212 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9308103dc0 0x7f930811a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:54:48.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:48.687+0000 7f9305e00700 1 -- 172.21.15.107:0/1703389212 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9308103dc0 msgr2=0x7f930811a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:54:48.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:48.687+0000 7f9305e00700 1 --2- 172.21.15.107:0/1703389212 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9308103dc0 0x7f930811a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:54:48.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:48.687+0000 7f9305e00700 1 -- 172.21.15.107:0/1703389212 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92f8005040 con 0x7f93081095f0 2024-03-04T13:54:48.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:48.688+0000 7f9305e00700 1 --2- 172.21.15.107:0/1703389212 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93081095f0 0x7f930811a620 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f92f400c8c0 tx=0x7f92f40044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:54:48.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:48.688+0000 7f92fea00700 1 -- 172.21.15.107:0/1703389212 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f92f400a980 con 0x7f93081095f0 2024-03-04T13:54:48.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:48.688+0000 7f92fea00700 1 -- 172.21.15.107:0/1703389212 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f92f400ce60 con 0x7f93081095f0 2024-03-04T13:54:48.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:48.688+0000 7f92fea00700 1 -- 172.21.15.107:0/1703389212 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f92f40115e0 con 0x7f93081095f0 2024-03-04T13:54:48.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:48.688+0000 7f930d400700 1 -- 172.21.15.107:0/1703389212 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9308114260 con 0x7f93081095f0 2024-03-04T13:54:48.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:48.688+0000 7f930d400700 1 -- 172.21.15.107:0/1703389212 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93081147a0 con 0x7f93081095f0 2024-03-04T13:54:48.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:48.689+0000 7f930d400700 1 -- 172.21.15.107:0/1703389212 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9308106850 con 0x7f93081095f0 2024-03-04T13:54:48.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:48.690+0000 7f92fea00700 1 -- 172.21.15.107:0/1703389212 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f92f4012400 con 0x7f93081095f0 2024-03-04T13:54:48.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:48.691+0000 7f92fea00700 1 --2- 172.21.15.107:0/1703389212 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f92ec06d7e0 0x7f92ec06fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:54:48.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:48.691+0000 7f92fea00700 1 -- 172.21.15.107:0/1703389212 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f92f408cf90 con 0x7f93081095f0 2024-03-04T13:54:48.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:48.692+0000 7f9306800700 1 --2- 172.21.15.107:0/1703389212 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f92ec06d7e0 0x7f92ec06fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:54:48.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:48.692+0000 7f9306800700 1 --2- 172.21.15.107:0/1703389212 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f92ec06d7e0 0x7f92ec06fc90 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f92f8004f90 tx=0x7f92f8009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:54:48.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:48.696+0000 7f92fea00700 1 -- 172.21.15.107:0/1703389212 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f92f405a340 con 0x7f93081095f0 2024-03-04T13:54:48.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:48.933+0000 7f930d400700 1 -- 172.21.15.107:0/1703389212 --> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f930804bf40 con 0x7f92ec06d7e0 2024-03-04T13:54:48.938 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:48.937+0000 7f92fea00700 1 -- 172.21.15.107:0/1703389212 <== mgr.34102 v2:172.21.15.107:6800/1223039812 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7f930804bf40 con 0x7f92ec06d7e0 2024-03-04T13:54:48.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:48.940+0000 7f930d400700 1 -- 172.21.15.107:0/1703389212 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f92ec06d7e0 msgr2=0x7f92ec06fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:54:48.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:48.941+0000 7f930d400700 1 --2- 172.21.15.107:0/1703389212 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f92ec06d7e0 0x7f92ec06fc90 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f92f8004f90 tx=0x7f92f8009e30 comp rx=0 tx=0).stop 2024-03-04T13:54:48.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:48.941+0000 7f930d400700 1 -- 172.21.15.107:0/1703389212 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93081095f0 msgr2=0x7f930811a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:54:48.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:48.941+0000 7f930d400700 1 --2- 172.21.15.107:0/1703389212 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93081095f0 0x7f930811a620 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f92f400c8c0 tx=0x7f92f40044c0 comp rx=0 tx=0).stop 2024-03-04T13:54:48.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:48.941+0000 7f930d400700 1 -- 172.21.15.107:0/1703389212 shutdown_connections 2024-03-04T13:54:48.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:48.941+0000 7f930d400700 1 --2- 172.21.15.107:0/1703389212 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f92ec06d7e0 0x7f92ec06fc90 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:54:48.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:48.941+0000 7f930d400700 1 --2- 172.21.15.107:0/1703389212 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9308103dc0 0x7f930811a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:54:48.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:48.941+0000 7f930d400700 1 --2- 172.21.15.107:0/1703389212 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93081095f0 0x7f930811a620 secure :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f92f400c8c0 tx=0x7f92f40044c0 comp rx=0 tx=0).stop 2024-03-04T13:54:48.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:48.941+0000 7f930d400700 1 -- 172.21.15.107:0/1703389212 >> 172.21.15.107:0/1703389212 conn(0x7f9308072d50 msgr2=0x7f9308074790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:54:48.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:48.941+0000 7f930d400700 1 -- 172.21.15.107:0/1703389212 shutdown_connections 2024-03-04T13:54:48.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:48.941+0000 7f930d400700 1 -- 172.21.15.107:0/1703389212 wait complete. 2024-03-04T13:54:49.817 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 -e sha1=717ce3cce8d0166e4e577faa8004e6f9cb4128c0 -- bash -c 'ceph orch ps' 2024-03-04T13:54:50.341 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/mon.smithi107/config 2024-03-04T13:54:50.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:49 smithi107 ceph-mon[112578]: pgmap v261: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:54:50.344 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:49 smithi107 ceph-mon[112578]: from='client.34526 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:54:50.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:49 smithi169 ceph-mon[86183]: pgmap v261: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:54:50.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:49 smithi169 ceph-mon[86183]: from='client.34526 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:54:52.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:51 smithi169 ceph-mon[86183]: pgmap v262: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:54:52.206 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:51 smithi107 ceph-mon[112578]: pgmap v262: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:54:52.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:52.458+0000 7f09b7200700 1 -- 172.21.15.107:0/2195767019 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09b8109380 msgr2=0x7f09b8109750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:54:52.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:52.458+0000 7f09b7200700 1 --2- 172.21.15.107:0/2195767019 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09b8109380 0x7f09b8109750 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f09a8004940 tx=0x7f09a8009e30 comp rx=0 tx=0).stop 2024-03-04T13:54:52.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:52.458+0000 7f09b7200700 1 -- 172.21.15.107:0/2195767019 shutdown_connections 2024-03-04T13:54:52.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:52.458+0000 7f09b7200700 1 --2- 172.21.15.107:0/2195767019 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f09b8109d20 0x7f09b81033a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:54:52.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:52.458+0000 7f09b7200700 1 --2- 172.21.15.107:0/2195767019 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09b8109380 0x7f09b8109750 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:54:52.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:52.458+0000 7f09b7200700 1 -- 172.21.15.107:0/2195767019 >> 172.21.15.107:0/2195767019 conn(0x7f09b8072b50 msgr2=0x7f09b8074f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:54:52.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:52.459+0000 7f09b7200700 1 -- 172.21.15.107:0/2195767019 shutdown_connections 2024-03-04T13:54:52.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:52.459+0000 7f09b7200700 1 -- 172.21.15.107:0/2195767019 wait complete. 2024-03-04T13:54:52.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:52.459+0000 7f09b7200700 1 Processor -- start 2024-03-04T13:54:52.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:52.460+0000 7f09b7200700 1 -- start start 2024-03-04T13:54:52.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:52.460+0000 7f09b7200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09b8109d20 0x7f09b811a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:54:52.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:52.460+0000 7f09b7200700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f09b811a560 0x7f09b8113ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:54:52.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:52.460+0000 7f09b7200700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09b811aae0 con 0x7f09b8109d20 2024-03-04T13:54:52.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:52.460+0000 7f09b7200700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09b811ac50 con 0x7f09b811a560 2024-03-04T13:54:52.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:52.460+0000 7f09b5e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09b8109d20 0x7f09b811a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:54:52.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:52.460+0000 7f09b5e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09b8109d20 0x7f09b811a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46600/0 (socket says 172.21.15.107:46600) 2024-03-04T13:54:52.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:52.460+0000 7f09b5e00700 1 -- 172.21.15.107:0/2889465022 learned_addr learned my addr 172.21.15.107:0/2889465022 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:54:52.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:52.461+0000 7f09afe00700 1 --2- 172.21.15.107:0/2889465022 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f09b811a560 0x7f09b8113ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:54:52.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:52.461+0000 7f09b5e00700 1 -- 172.21.15.107:0/2889465022 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f09b811a560 msgr2=0x7f09b8113ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:54:52.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:52.461+0000 7f09b5e00700 1 --2- 172.21.15.107:0/2889465022 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f09b811a560 0x7f09b8113ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:54:52.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:52.461+0000 7f09b5e00700 1 -- 172.21.15.107:0/2889465022 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09a8005040 con 0x7f09b8109d20 2024-03-04T13:54:52.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:52.461+0000 7f09b5e00700 1 --2- 172.21.15.107:0/2889465022 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09b8109d20 0x7f09b811a020 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f09a8000c00 tx=0x7f09a8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:54:52.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:52.461+0000 7f09ae000700 1 -- 172.21.15.107:0/2889465022 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f09a800e040 con 0x7f09b8109d20 2024-03-04T13:54:52.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:52.461+0000 7f09b7200700 1 -- 172.21.15.107:0/2889465022 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09b81144c0 con 0x7f09b8109d20 2024-03-04T13:54:52.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:52.461+0000 7f09ae000700 1 -- 172.21.15.107:0/2889465022 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f09a8009430 con 0x7f09b8109d20 2024-03-04T13:54:52.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:52.462+0000 7f09ae000700 1 -- 172.21.15.107:0/2889465022 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f09a801c630 con 0x7f09b8109d20 2024-03-04T13:54:52.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:52.462+0000 7f09b7200700 1 -- 172.21.15.107:0/2889465022 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09b8114a80 con 0x7f09b8109d20 2024-03-04T13:54:52.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:52.463+0000 7f09b7200700 1 -- 172.21.15.107:0/2889465022 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09b8114cf0 con 0x7f09b8109d20 2024-03-04T13:54:52.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:52.464+0000 7f09ae000700 1 -- 172.21.15.107:0/2889465022 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f09a800bbf0 con 0x7f09b8109d20 2024-03-04T13:54:52.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:52.465+0000 7f09ae000700 1 --2- 172.21.15.107:0/2889465022 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f099c06d7f0 0x7f099c06fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:54:52.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:52.465+0000 7f09ae000700 1 -- 172.21.15.107:0/2889465022 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f09a808e200 con 0x7f09b8109d20 2024-03-04T13:54:52.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:52.465+0000 7f09afe00700 1 --2- 172.21.15.107:0/2889465022 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f099c06d7f0 0x7f099c06fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:54:52.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:52.466+0000 7f09afe00700 1 --2- 172.21.15.107:0/2889465022 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f099c06d7f0 0x7f099c06fca0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f09a0009fd0 tx=0x7f09a0009f20 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:54:52.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:52.470+0000 7f09ae000700 1 -- 172.21.15.107:0/2889465022 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f09a805b670 con 0x7f09b8109d20 2024-03-04T13:54:52.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:52.699+0000 7f09b7200700 1 -- 172.21.15.107:0/2889465022 --> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f09b8110490 con 0x7f099c06d7f0 2024-03-04T13:54:52.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:52.710+0000 7f09ae000700 1 -- 172.21.15.107:0/2889465022 <== mgr.34102 v2:172.21.15.107:6800/1223039812 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f09b8110490 con 0x7f099c06d7f0 2024-03-04T13:54:52.712 INFO:teuthology.orchestra.run.smithi107.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-04T13:54:52.712 INFO:teuthology.orchestra.run.smithi107.stdout:alertmanager.smithi107 smithi107 *:9093,9094 running (9m) 96s ago 23m 14.4M - 0.25.0 c8568f914cd2 e5ed7be32538 2024-03-04T13:54:52.712 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi107 smithi107 running (6m) 96s ago 23m 6660k - 19.0.0-1845-g717ce3cc d71b839abdf8 0558022b0d84 2024-03-04T13:54:52.712 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi169 smithi169 running (5m) 39s ago 22m 6665k - 19.0.0-1845-g717ce3cc d71b839abdf8 221ca53aa0d3 2024-03-04T13:54:52.713 INFO:teuthology.orchestra.run.smithi107.stdout:grafana.smithi107 smithi107 *:3000 running (8m) 96s ago 22m 75.1M - 9.4.12 a0ea5e3c46d9 c5e4d33fb599 2024-03-04T13:54:52.713 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.riexmm smithi107 running (2m) 96s ago 16m 11.0M - 19.0.0-1845-g717ce3cc d71b839abdf8 8582388ad90a 2024-03-04T13:54:52.713 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.thycho smithi107 running (103s) 96s ago 16m 10.8M - 19.0.0-1845-g717ce3cc d71b839abdf8 f4b3cea14a6a 2024-03-04T13:54:52.713 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi169.bxitzk smithi169 running (43s) 39s ago 16m 10.9M - 19.0.0-1845-g717ce3cc d71b839abdf8 1ea541510c55 2024-03-04T13:54:52.713 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi169.jusudl smithi169 running (75s) 39s ago 16m 11.1M - 19.0.0-1845-g717ce3cc d71b839abdf8 e6d45009ccbb 2024-03-04T13:54:52.713 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi107.wpzgwl smithi107 *:8443,9283,8765 running (11m) 96s ago 24m 509M - 19.0.0-1845-g717ce3cc d71b839abdf8 1c5fdee45eac 2024-03-04T13:54:52.713 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi169.eihczr smithi169 *:8443,9283,8765 running (10m) 39s ago 21m 452M - 19.0.0-1845-g717ce3cc d71b839abdf8 b5cac2b7d22a 2024-03-04T13:54:52.713 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi107 smithi107 running (7m) 96s ago 25m 52.5M 2048M 19.0.0-1845-g717ce3cc d71b839abdf8 32f07144274f 2024-03-04T13:54:52.713 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi169 smithi169 running (7m) 39s ago 21m 41.6M 2048M 19.0.0-1845-g717ce3cc d71b839abdf8 e3527449381a 2024-03-04T13:54:52.713 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi107 smithi107 *:9100 running (10m) 96s ago 23m 8102k - 1.5.0 0da6a335fe13 8a8d3d4f724f 2024-03-04T13:54:52.713 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi169 smithi169 *:9100 running (10m) 39s ago 22m 8158k - 1.5.0 0da6a335fe13 f240f5a975bc 2024-03-04T13:54:52.714 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0 smithi107 running (5m) 96s ago 20m 85.3M 1558M 19.0.0-1845-g717ce3cc d71b839abdf8 5801a5540dc6 2024-03-04T13:54:52.714 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1 smithi107 running (4m) 96s ago 20m 94.0M 1558M 19.0.0-1845-g717ce3cc d71b839abdf8 b614af658854 2024-03-04T13:54:52.714 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2 smithi107 running (4m) 96s ago 19m 84.9M 1558M 19.0.0-1845-g717ce3cc d71b839abdf8 6e4da4f6f146 2024-03-04T13:54:52.714 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3 smithi169 running (3m) 39s ago 19m 107M 2582M 19.0.0-1845-g717ce3cc d71b839abdf8 d31baf764d7a 2024-03-04T13:54:52.714 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4 smithi169 running (3m) 39s ago 18m 90.7M 2582M 19.0.0-1845-g717ce3cc d71b839abdf8 a70bacd97639 2024-03-04T13:54:52.714 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5 smithi169 running (2m) 39s ago 18m 86.0M 2582M 19.0.0-1845-g717ce3cc d71b839abdf8 5f6e00f7993d 2024-03-04T13:54:52.714 INFO:teuthology.orchestra.run.smithi107.stdout:prometheus.smithi107 smithi107 *:9095 running (9m) 96s ago 22m 54.9M - 2.43.0 a07b618ecd1d 1eeb1ad5cb75 2024-03-04T13:54:52.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:52.713+0000 7f09b7200700 1 -- 172.21.15.107:0/2889465022 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f099c06d7f0 msgr2=0x7f099c06fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:54:52.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:52.713+0000 7f09b7200700 1 --2- 172.21.15.107:0/2889465022 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f099c06d7f0 0x7f099c06fca0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f09a0009fd0 tx=0x7f09a0009f20 comp rx=0 tx=0).stop 2024-03-04T13:54:52.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:52.713+0000 7f09b7200700 1 -- 172.21.15.107:0/2889465022 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09b8109d20 msgr2=0x7f09b811a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:54:52.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:52.713+0000 7f09b7200700 1 --2- 172.21.15.107:0/2889465022 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09b8109d20 0x7f09b811a020 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f09a8000c00 tx=0x7f09a8009e30 comp rx=0 tx=0).stop 2024-03-04T13:54:52.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:52.713+0000 7f09b7200700 1 -- 172.21.15.107:0/2889465022 shutdown_connections 2024-03-04T13:54:52.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:52.713+0000 7f09b7200700 1 --2- 172.21.15.107:0/2889465022 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f099c06d7f0 0x7f099c06fca0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:54:52.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:52.713+0000 7f09b7200700 1 --2- 172.21.15.107:0/2889465022 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09b8109d20 0x7f09b811a020 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:54:52.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:52.713+0000 7f09b7200700 1 --2- 172.21.15.107:0/2889465022 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f09b811a560 0x7f09b8113ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:54:52.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:52.713+0000 7f09b7200700 1 -- 172.21.15.107:0/2889465022 >> 172.21.15.107:0/2889465022 conn(0x7f09b8072b50 msgr2=0x7f09b8075980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:54:52.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:52.714+0000 7f09b7200700 1 -- 172.21.15.107:0/2889465022 shutdown_connections 2024-03-04T13:54:52.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:52.714+0000 7f09b7200700 1 -- 172.21.15.107:0/2889465022 wait complete. 2024-03-04T13:54:53.586 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 -e sha1=717ce3cce8d0166e4e577faa8004e6f9cb4128c0 -- bash -c 'ceph orch upgrade status' 2024-03-04T13:54:54.063 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/mon.smithi107/config 2024-03-04T13:54:54.210 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:53 smithi107 ceph-mon[112578]: from='client.34530 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:54:54.210 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:53 smithi107 ceph-mon[112578]: pgmap v263: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:54:54.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:53 smithi169 ceph-mon[86183]: from='client.34530 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:54:54.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:53 smithi169 ceph-mon[86183]: pgmap v263: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:54:55.244 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:54 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:54:55.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:54 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:54:56.095 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:55 smithi107 ceph-mon[112578]: pgmap v264: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:54:56.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:55 smithi169 ceph-mon[86183]: pgmap v264: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:54:56.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:56.362+0000 7f10a0e00700 1 -- 172.21.15.107:0/3102467711 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f109c102f50 msgr2=0x7f109c103320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:54:56.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:56.362+0000 7f10a0e00700 1 --2- 172.21.15.107:0/3102467711 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f109c102f50 0x7f109c103320 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f108c004940 tx=0x7f108c009e30 comp rx=0 tx=0).stop 2024-03-04T13:54:56.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:56.362+0000 7f10a0e00700 1 -- 172.21.15.107:0/3102467711 shutdown_connections 2024-03-04T13:54:56.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:56.362+0000 7f10a0e00700 1 --2- 172.21.15.107:0/3102467711 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f109c1038f0 0x7f109c0fcf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:54:56.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:56.362+0000 7f10a0e00700 1 --2- 172.21.15.107:0/3102467711 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f109c102f50 0x7f109c103320 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:54:56.364 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:56.362+0000 7f10a0e00700 1 -- 172.21.15.107:0/3102467711 >> 172.21.15.107:0/3102467711 conn(0x7f109c0f8ae0 msgr2=0x7f109c0faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:54:56.364 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:56.362+0000 7f10a0e00700 1 -- 172.21.15.107:0/3102467711 shutdown_connections 2024-03-04T13:54:56.364 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:56.362+0000 7f10a0e00700 1 -- 172.21.15.107:0/3102467711 wait complete. 2024-03-04T13:54:56.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:56.363+0000 7f10a0e00700 1 Processor -- start 2024-03-04T13:54:56.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:56.363+0000 7f10a0e00700 1 -- start start 2024-03-04T13:54:56.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:56.363+0000 7f10a0e00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f109c102f50 0x7f109c10a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:54:56.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:56.363+0000 7f10a0e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f109c1038f0 0x7f109c10ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:54:56.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:56.363+0000 7f10a0e00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f109c10b370 con 0x7f109c1038f0 2024-03-04T13:54:56.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:56.363+0000 7f10a0e00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f109c10b4b0 con 0x7f109c102f50 2024-03-04T13:54:56.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:56.364+0000 7f1099400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f109c1038f0 0x7f109c10ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:54:56.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:56.364+0000 7f1099400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f109c1038f0 0x7f109c10ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46630/0 (socket says 172.21.15.107:46630) 2024-03-04T13:54:56.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:56.364+0000 7f1099400700 1 -- 172.21.15.107:0/697931448 learned_addr learned my addr 172.21.15.107:0/697931448 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:54:56.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:56.364+0000 7f1099e00700 1 --2- 172.21.15.107:0/697931448 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f109c102f50 0x7f109c10a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:54:56.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:56.364+0000 7f1099400700 1 -- 172.21.15.107:0/697931448 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f109c102f50 msgr2=0x7f109c10a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:54:56.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:56.364+0000 7f1099400700 1 --2- 172.21.15.107:0/697931448 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f109c102f50 0x7f109c10a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:54:56.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:56.364+0000 7f1099400700 1 -- 172.21.15.107:0/697931448 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f108c005040 con 0x7f109c1038f0 2024-03-04T13:54:56.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:56.364+0000 7f1099400700 1 --2- 172.21.15.107:0/697931448 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f109c1038f0 0x7f109c10ae30 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f1088002a50 tx=0x7f1088004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:54:56.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:56.365+0000 7f1092000700 1 -- 172.21.15.107:0/697931448 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1088007cc0 con 0x7f109c1038f0 2024-03-04T13:54:56.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:56.365+0000 7f1092000700 1 -- 172.21.15.107:0/697931448 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f108800f040 con 0x7f109c1038f0 2024-03-04T13:54:56.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:56.365+0000 7f1092000700 1 -- 172.21.15.107:0/697931448 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f108800daf0 con 0x7f109c1038f0 2024-03-04T13:54:56.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:56.365+0000 7f10a0e00700 1 -- 172.21.15.107:0/697931448 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f109c11e280 con 0x7f109c1038f0 2024-03-04T13:54:56.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:56.365+0000 7f10a0e00700 1 -- 172.21.15.107:0/697931448 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f109c11e5d0 con 0x7f109c1038f0 2024-03-04T13:54:56.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:56.366+0000 7f10a0e00700 1 -- 172.21.15.107:0/697931448 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f109c0fc6e0 con 0x7f109c1038f0 2024-03-04T13:54:56.368 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:56.367+0000 7f1092000700 1 -- 172.21.15.107:0/697931448 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f1088009810 con 0x7f109c1038f0 2024-03-04T13:54:56.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:56.369+0000 7f1092000700 1 --2- 172.21.15.107:0/697931448 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f108006d7f0 0x7f108006fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:54:56.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:56.369+0000 7f1092000700 1 -- 172.21.15.107:0/697931448 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f108808cc50 con 0x7f109c1038f0 2024-03-04T13:54:56.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:56.369+0000 7f1099e00700 1 --2- 172.21.15.107:0/697931448 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f108006d7f0 0x7f108006fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:54:56.371 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:56.370+0000 7f1099e00700 1 --2- 172.21.15.107:0/697931448 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f108006d7f0 0x7f108006fca0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f108c00b3b0 tx=0x7f108c009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:54:56.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:56.374+0000 7f1092000700 1 -- 172.21.15.107:0/697931448 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1088056450 con 0x7f109c1038f0 2024-03-04T13:54:56.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:56.615+0000 7f10a0e00700 1 -- 172.21.15.107:0/697931448 --> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f109c10c140 con 0x7f108006d7f0 2024-03-04T13:54:56.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:56.617+0000 7f1092000700 1 -- 172.21.15.107:0/697931448 <== mgr.34102 v2:172.21.15.107:6800/1223039812 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7f109c10c140 con 0x7f108006d7f0 2024-03-04T13:54:56.619 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-03-04T13:54:56.619 INFO:teuthology.orchestra.run.smithi107.stdout: "target_image": null, 2024-03-04T13:54:56.619 INFO:teuthology.orchestra.run.smithi107.stdout: "in_progress": false, 2024-03-04T13:54:56.619 INFO:teuthology.orchestra.run.smithi107.stdout: "which": "", 2024-03-04T13:54:56.619 INFO:teuthology.orchestra.run.smithi107.stdout: "services_complete": [], 2024-03-04T13:54:56.619 INFO:teuthology.orchestra.run.smithi107.stdout: "progress": null, 2024-03-04T13:54:56.619 INFO:teuthology.orchestra.run.smithi107.stdout: "message": "", 2024-03-04T13:54:56.619 INFO:teuthology.orchestra.run.smithi107.stdout: "is_paused": false 2024-03-04T13:54:56.619 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-03-04T13:54:56.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:56.620+0000 7f10a0e00700 1 -- 172.21.15.107:0/697931448 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f108006d7f0 msgr2=0x7f108006fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:54:56.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:56.620+0000 7f10a0e00700 1 --2- 172.21.15.107:0/697931448 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f108006d7f0 0x7f108006fca0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f108c00b3b0 tx=0x7f108c009e30 comp rx=0 tx=0).stop 2024-03-04T13:54:56.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:56.620+0000 7f10a0e00700 1 -- 172.21.15.107:0/697931448 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f109c1038f0 msgr2=0x7f109c10ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:54:56.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:56.620+0000 7f10a0e00700 1 --2- 172.21.15.107:0/697931448 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f109c1038f0 0x7f109c10ae30 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f1088002a50 tx=0x7f1088004600 comp rx=0 tx=0).stop 2024-03-04T13:54:56.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:56.620+0000 7f10a0e00700 1 -- 172.21.15.107:0/697931448 shutdown_connections 2024-03-04T13:54:56.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:56.620+0000 7f10a0e00700 1 --2- 172.21.15.107:0/697931448 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f108006d7f0 0x7f108006fca0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:54:56.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:56.620+0000 7f10a0e00700 1 --2- 172.21.15.107:0/697931448 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f109c102f50 0x7f109c10a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:54:56.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:56.620+0000 7f10a0e00700 1 --2- 172.21.15.107:0/697931448 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f109c1038f0 0x7f109c10ae30 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:54:56.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:56.620+0000 7f10a0e00700 1 -- 172.21.15.107:0/697931448 >> 172.21.15.107:0/697931448 conn(0x7f109c0f8ae0 msgr2=0x7f109c0fa3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:54:56.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:56.620+0000 7f10a0e00700 1 -- 172.21.15.107:0/697931448 shutdown_connections 2024-03-04T13:54:56.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:54:56.620+0000 7f10a0e00700 1 -- 172.21.15.107:0/697931448 wait complete. 2024-03-04T13:54:57.502 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 -e sha1=717ce3cce8d0166e4e577faa8004e6f9cb4128c0 -- bash -c 'ceph health detail' 2024-03-04T13:54:57.990 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/mon.smithi107/config 2024-03-04T13:54:58.334 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:57 smithi107 ceph-mon[112578]: from='client.34534 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:54:58.335 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:57 smithi107 ceph-mon[112578]: pgmap v265: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:54:58.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:57 smithi169 ceph-mon[86183]: from='client.34534 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:54:58.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:57 smithi169 ceph-mon[86183]: pgmap v265: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:54:59.330 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:54:59 smithi107 ceph-mon[112578]: pgmap v266: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:54:59.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:54:59 smithi169 ceph-mon[86183]: pgmap v266: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:55:00.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:55:00.130+0000 7f5c14000700 1 -- 172.21.15.107:0/2695023956 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c0c105170 msgr2=0x7f5c0c105540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:55:00.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:55:00.130+0000 7f5c14000700 1 --2- 172.21.15.107:0/2695023956 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c0c105170 0x7f5c0c105540 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5c00004940 tx=0x7f5c00009e30 comp rx=0 tx=0).stop 2024-03-04T13:55:00.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:55:00.130+0000 7f5c14000700 1 -- 172.21.15.107:0/2695023956 shutdown_connections 2024-03-04T13:55:00.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:55:00.130+0000 7f5c14000700 1 --2- 172.21.15.107:0/2695023956 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5c0c105b10 0x7f5c0c0ff180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:55:00.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:55:00.130+0000 7f5c14000700 1 --2- 172.21.15.107:0/2695023956 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c0c105170 0x7f5c0c105540 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:55:00.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:55:00.130+0000 7f5c14000700 1 -- 172.21.15.107:0/2695023956 >> 172.21.15.107:0/2695023956 conn(0x7f5c0c0face0 msgr2=0x7f5c0c0fd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:55:00.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:55:00.130+0000 7f5c14000700 1 -- 172.21.15.107:0/2695023956 shutdown_connections 2024-03-04T13:55:00.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:55:00.130+0000 7f5c14000700 1 -- 172.21.15.107:0/2695023956 wait complete. 2024-03-04T13:55:00.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:55:00.131+0000 7f5c14000700 1 Processor -- start 2024-03-04T13:55:00.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:55:00.131+0000 7f5c14000700 1 -- start start 2024-03-04T13:55:00.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:55:00.131+0000 7f5c14000700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5c0c105170 0x7f5c0c115c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:55:00.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:55:00.131+0000 7f5c14000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c0c105b10 0x7f5c0c1161c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:55:00.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:55:00.131+0000 7f5c14000700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c0c1168a0 con 0x7f5c0c105b10 2024-03-04T13:55:00.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:55:00.131+0000 7f5c14000700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c0c10fb40 con 0x7f5c0c105170 2024-03-04T13:55:00.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:55:00.132+0000 7f5c10c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c0c105b10 0x7f5c0c1161c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:55:00.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:55:00.132+0000 7f5c10c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c0c105b10 0x7f5c0c1161c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33758/0 (socket says 172.21.15.107:33758) 2024-03-04T13:55:00.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:55:00.132+0000 7f5c10c00700 1 -- 172.21.15.107:0/2331098674 learned_addr learned my addr 172.21.15.107:0/2331098674 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:55:00.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:55:00.132+0000 7f5c11600700 1 --2- 172.21.15.107:0/2331098674 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5c0c105170 0x7f5c0c115c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:55:00.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:55:00.132+0000 7f5c10c00700 1 -- 172.21.15.107:0/2331098674 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5c0c105170 msgr2=0x7f5c0c115c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:55:00.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:55:00.132+0000 7f5c10c00700 1 --2- 172.21.15.107:0/2331098674 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5c0c105170 0x7f5c0c115c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:55:00.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:55:00.132+0000 7f5c10c00700 1 -- 172.21.15.107:0/2331098674 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c00005040 con 0x7f5c0c105b10 2024-03-04T13:55:00.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:55:00.132+0000 7f5c10c00700 1 --2- 172.21.15.107:0/2331098674 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c0c105b10 0x7f5c0c1161c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5bf800b790 tx=0x7f5bf8009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:55:00.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:55:00.133+0000 7f5c09600700 1 -- 172.21.15.107:0/2331098674 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5bf800f6f0 con 0x7f5c0c105b10 2024-03-04T13:55:00.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:55:00.133+0000 7f5c09600700 1 -- 172.21.15.107:0/2331098674 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5bf800fd20 con 0x7f5c0c105b10 2024-03-04T13:55:00.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:55:00.133+0000 7f5c14000700 1 -- 172.21.15.107:0/2331098674 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c0c10fe20 con 0x7f5c0c105b10 2024-03-04T13:55:00.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:55:00.133+0000 7f5c09600700 1 -- 172.21.15.107:0/2331098674 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5bf80103a0 con 0x7f5c0c105b10 2024-03-04T13:55:00.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:55:00.133+0000 7f5c14000700 1 -- 172.21.15.107:0/2331098674 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c0c110360 con 0x7f5c0c105b10 2024-03-04T13:55:00.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:55:00.134+0000 7f5c14000700 1 -- 172.21.15.107:0/2331098674 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c0c0fe900 con 0x7f5c0c105b10 2024-03-04T13:55:00.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:55:00.135+0000 7f5c09600700 1 -- 172.21.15.107:0/2331098674 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f5bf8011400 con 0x7f5c0c105b10 2024-03-04T13:55:00.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:55:00.137+0000 7f5c09600700 1 --2- 172.21.15.107:0/2331098674 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f5bf406d7f0 0x7f5bf406fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:55:00.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:55:00.138+0000 7f5c09600700 1 -- 172.21.15.107:0/2331098674 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f5bf808cd60 con 0x7f5c0c105b10 2024-03-04T13:55:00.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:55:00.138+0000 7f5c11600700 1 --2- 172.21.15.107:0/2331098674 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f5bf406d7f0 0x7f5bf406fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:55:00.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:55:00.138+0000 7f5c11600700 1 --2- 172.21.15.107:0/2331098674 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f5bf406d7f0 0x7f5bf406fca0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5c0000b3b0 tx=0x7f5c00009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:55:00.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:55:00.142+0000 7f5c09600700 1 -- 172.21.15.107:0/2331098674 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5bf8091070 con 0x7f5c0c105b10 2024-03-04T13:55:00.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:55:00.447+0000 7f5c14000700 1 -- 172.21.15.107:0/2331098674 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f5c0c111290 con 0x7f5c0c105b10 2024-03-04T13:55:00.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:55:00.447+0000 7f5c09600700 1 -- 172.21.15.107:0/2331098674 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f5bf805a090 con 0x7f5c0c105b10 2024-03-04T13:55:00.450 INFO:teuthology.orchestra.run.smithi107.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-03-04T13:55:00.450 INFO:teuthology.orchestra.run.smithi107.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-04T13:55:00.450 INFO:teuthology.orchestra.run.smithi107.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-04T13:55:00.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:55:00.450+0000 7f5c14000700 1 -- 172.21.15.107:0/2331098674 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f5bf406d7f0 msgr2=0x7f5bf406fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:55:00.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:55:00.450+0000 7f5c14000700 1 --2- 172.21.15.107:0/2331098674 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f5bf406d7f0 0x7f5bf406fca0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5c0000b3b0 tx=0x7f5c00009e30 comp rx=0 tx=0).stop 2024-03-04T13:55:00.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:55:00.450+0000 7f5c14000700 1 -- 172.21.15.107:0/2331098674 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c0c105b10 msgr2=0x7f5c0c1161c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:55:00.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:55:00.450+0000 7f5c14000700 1 --2- 172.21.15.107:0/2331098674 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c0c105b10 0x7f5c0c1161c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5bf800b790 tx=0x7f5bf8009220 comp rx=0 tx=0).stop 2024-03-04T13:55:00.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:55:00.450+0000 7f5c14000700 1 -- 172.21.15.107:0/2331098674 shutdown_connections 2024-03-04T13:55:00.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:55:00.450+0000 7f5c14000700 1 --2- 172.21.15.107:0/2331098674 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f5bf406d7f0 0x7f5bf406fca0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:55:00.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:55:00.450+0000 7f5c14000700 1 --2- 172.21.15.107:0/2331098674 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5c0c105170 0x7f5c0c115c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:55:00.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:55:00.450+0000 7f5c14000700 1 --2- 172.21.15.107:0/2331098674 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c0c105b10 0x7f5c0c1161c0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:55:00.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:55:00.450+0000 7f5c14000700 1 -- 172.21.15.107:0/2331098674 >> 172.21.15.107:0/2331098674 conn(0x7f5c0c0face0 msgr2=0x7f5c0c0fc630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:55:00.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:55:00.450+0000 7f5c14000700 1 -- 172.21.15.107:0/2331098674 shutdown_connections 2024-03-04T13:55:00.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:55:00.450+0000 7f5c14000700 1 -- 172.21.15.107:0/2331098674 wait complete. 2024-03-04T13:55:00.845 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:55:00 smithi107 ceph-mon[112578]: from='client.? 172.21.15.107:0/2331098674' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-04T13:55:00.862 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:55:00 smithi169 ceph-mon[86183]: from='client.? 172.21.15.107:0/2331098674' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-04T13:55:01.279 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 -e sha1=717ce3cce8d0166e4e577faa8004e6f9cb4128c0 -- bash -c 'ceph versions' 2024-03-04T13:55:01.749 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/mon.smithi107/config 2024-03-04T13:55:01.848 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:55:01 smithi107 ceph-mon[112578]: pgmap v267: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:55:01.862 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:55:01 smithi169 ceph-mon[86183]: pgmap v267: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:55:03.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:55:03.830+0000 7fe36c800700 1 -- 172.21.15.107:0/2619815059 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3641095f0 msgr2=0x7fe3641099c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:55:03.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:55:03.830+0000 7fe36c800700 1 --2- 172.21.15.107:0/2619815059 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3641095f0 0x7fe3641099c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe358004940 tx=0x7fe358009e30 comp rx=0 tx=0).stop 2024-03-04T13:55:03.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:55:03.830+0000 7fe36c800700 1 -- 172.21.15.107:0/2619815059 shutdown_connections 2024-03-04T13:55:03.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:55:03.830+0000 7fe36c800700 1 --2- 172.21.15.107:0/2619815059 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe364103dc0 0x7fe364104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:55:03.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:55:03.830+0000 7fe36c800700 1 --2- 172.21.15.107:0/2619815059 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3641095f0 0x7fe3641099c0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:55:03.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:55:03.830+0000 7fe36c800700 1 -- 172.21.15.107:0/2619815059 >> 172.21.15.107:0/2619815059 conn(0x7fe364072d50 msgr2=0x7fe364075160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:55:03.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:55:03.831+0000 7fe36c800700 1 -- 172.21.15.107:0/2619815059 shutdown_connections 2024-03-04T13:55:03.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:55:03.831+0000 7fe36c800700 1 -- 172.21.15.107:0/2619815059 wait complete. 2024-03-04T13:55:03.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:55:03.831+0000 7fe36c800700 1 Processor -- start 2024-03-04T13:55:03.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:55:03.832+0000 7fe36c800700 1 -- start start 2024-03-04T13:55:03.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:55:03.832+0000 7fe36c800700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe364103dc0 0x7fe36411a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:55:03.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:55:03.832+0000 7fe36c800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3641095f0 0x7fe36411a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:55:03.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:55:03.832+0000 7fe36c800700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe36411ac70 con 0x7fe3641095f0 2024-03-04T13:55:03.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:55:03.832+0000 7fe36c800700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe364113f80 con 0x7fe364103dc0 2024-03-04T13:55:03.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:55:03.832+0000 7fe369400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3641095f0 0x7fe36411a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:55:03.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:55:03.832+0000 7fe369400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3641095f0 0x7fe36411a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33776/0 (socket says 172.21.15.107:33776) 2024-03-04T13:55:03.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:55:03.832+0000 7fe369400700 1 -- 172.21.15.107:0/3551485928 learned_addr learned my addr 172.21.15.107:0/3551485928 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:55:03.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:55:03.832+0000 7fe369e00700 1 --2- 172.21.15.107:0/3551485928 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe364103dc0 0x7fe36411a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:55:03.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:55:03.833+0000 7fe369400700 1 -- 172.21.15.107:0/3551485928 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe364103dc0 msgr2=0x7fe36411a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:55:03.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:55:03.833+0000 7fe369400700 1 --2- 172.21.15.107:0/3551485928 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe364103dc0 0x7fe36411a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:55:03.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:55:03.833+0000 7fe369400700 1 -- 172.21.15.107:0/3551485928 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe358005040 con 0x7fe3641095f0 2024-03-04T13:55:03.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:55:03.833+0000 7fe369400700 1 --2- 172.21.15.107:0/3551485928 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3641095f0 0x7fe36411a620 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe354002a50 tx=0x7fe3540046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:55:03.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:55:03.833+0000 7fe35e000700 1 -- 172.21.15.107:0/3551485928 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe354007cc0 con 0x7fe3641095f0 2024-03-04T13:55:03.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:55:03.833+0000 7fe35e000700 1 -- 172.21.15.107:0/3551485928 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe35400f040 con 0x7fe3641095f0 2024-03-04T13:55:03.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:55:03.833+0000 7fe36c800700 1 -- 172.21.15.107:0/3551485928 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe364114260 con 0x7fe3641095f0 2024-03-04T13:55:03.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:55:03.833+0000 7fe35e000700 1 -- 172.21.15.107:0/3551485928 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe354009640 con 0x7fe3641095f0 2024-03-04T13:55:03.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:55:03.833+0000 7fe36c800700 1 -- 172.21.15.107:0/3551485928 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3641147a0 con 0x7fe3641095f0 2024-03-04T13:55:03.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:55:03.835+0000 7fe36c800700 1 -- 172.21.15.107:0/3551485928 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe364106850 con 0x7fe3641095f0 2024-03-04T13:55:03.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:55:03.836+0000 7fe35e000700 1 -- 172.21.15.107:0/3551485928 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fe3540097a0 con 0x7fe3641095f0 2024-03-04T13:55:03.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:55:03.836+0000 7fe35e000700 1 --2- 172.21.15.107:0/3551485928 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fe34c06d7f0 0x7fe34c06fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:55:03.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:55:03.836+0000 7fe35e000700 1 -- 172.21.15.107:0/3551485928 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fe35408ccd0 con 0x7fe3641095f0 2024-03-04T13:55:03.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:55:03.837+0000 7fe369e00700 1 --2- 172.21.15.107:0/3551485928 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fe34c06d7f0 0x7fe34c06fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:55:03.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:55:03.837+0000 7fe369e00700 1 --2- 172.21.15.107:0/3551485928 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fe34c06d7f0 0x7fe34c06fca0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe35800b020 tx=0x7fe358009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:55:03.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:55:03.841+0000 7fe35e000700 1 -- 172.21.15.107:0/3551485928 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe35405a000 con 0x7fe3641095f0 2024-03-04T13:55:04.095 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:55:03 smithi107 ceph-mon[112578]: pgmap v268: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:55:04.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:55:03 smithi169 ceph-mon[86183]: pgmap v268: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:55:04.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:55:04.148+0000 7fe36c800700 1 -- 172.21.15.107:0/3551485928 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fe364075870 con 0x7fe3641095f0 2024-03-04T13:55:04.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:55:04.149+0000 7fe35e000700 1 -- 172.21.15.107:0/3551485928 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fe354059b90 con 0x7fe3641095f0 2024-03-04T13:55:04.151 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-03-04T13:55:04.151 INFO:teuthology.orchestra.run.smithi107.stdout: "mon": { 2024-03-04T13:55:04.151 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-1845-g717ce3cc (717ce3cce8d0166e4e577faa8004e6f9cb4128c0) squid (dev)": 2 2024-03-04T13:55:04.152 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:55:04.152 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr": { 2024-03-04T13:55:04.152 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-1845-g717ce3cc (717ce3cce8d0166e4e577faa8004e6f9cb4128c0) squid (dev)": 2 2024-03-04T13:55:04.152 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:55:04.152 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": { 2024-03-04T13:55:04.152 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-1845-g717ce3cc (717ce3cce8d0166e4e577faa8004e6f9cb4128c0) squid (dev)": 6 2024-03-04T13:55:04.152 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:55:04.152 INFO:teuthology.orchestra.run.smithi107.stdout: "mds": { 2024-03-04T13:55:04.152 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-1845-g717ce3cc (717ce3cce8d0166e4e577faa8004e6f9cb4128c0) squid (dev)": 4 2024-03-04T13:55:04.152 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:55:04.152 INFO:teuthology.orchestra.run.smithi107.stdout: "overall": { 2024-03-04T13:55:04.152 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-1845-g717ce3cc (717ce3cce8d0166e4e577faa8004e6f9cb4128c0) squid (dev)": 14 2024-03-04T13:55:04.152 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-04T13:55:04.152 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-03-04T13:55:04.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:55:04.151+0000 7fe36c800700 1 -- 172.21.15.107:0/3551485928 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fe34c06d7f0 msgr2=0x7fe34c06fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:55:04.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:55:04.151+0000 7fe36c800700 1 --2- 172.21.15.107:0/3551485928 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fe34c06d7f0 0x7fe34c06fca0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe35800b020 tx=0x7fe358009e30 comp rx=0 tx=0).stop 2024-03-04T13:55:04.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:55:04.151+0000 7fe36c800700 1 -- 172.21.15.107:0/3551485928 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3641095f0 msgr2=0x7fe36411a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:55:04.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:55:04.151+0000 7fe36c800700 1 --2- 172.21.15.107:0/3551485928 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3641095f0 0x7fe36411a620 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe354002a50 tx=0x7fe3540046b0 comp rx=0 tx=0).stop 2024-03-04T13:55:04.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:55:04.152+0000 7fe36c800700 1 -- 172.21.15.107:0/3551485928 shutdown_connections 2024-03-04T13:55:04.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:55:04.152+0000 7fe36c800700 1 --2- 172.21.15.107:0/3551485928 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fe34c06d7f0 0x7fe34c06fca0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:55:04.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:55:04.152+0000 7fe36c800700 1 --2- 172.21.15.107:0/3551485928 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe364103dc0 0x7fe36411a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:55:04.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:55:04.152+0000 7fe36c800700 1 --2- 172.21.15.107:0/3551485928 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3641095f0 0x7fe36411a620 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:55:04.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:55:04.152+0000 7fe36c800700 1 -- 172.21.15.107:0/3551485928 >> 172.21.15.107:0/3551485928 conn(0x7fe364072d50 msgr2=0x7fe364074790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:55:04.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:55:04.152+0000 7fe36c800700 1 -- 172.21.15.107:0/3551485928 shutdown_connections 2024-03-04T13:55:04.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:55:04.152+0000 7fe36c800700 1 -- 172.21.15.107:0/3551485928 wait complete. 2024-03-04T13:55:04.865 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 -e sha1=717ce3cce8d0166e4e577faa8004e6f9cb4128c0 -- bash -c 'echo "wait for servicemap items w/ changing names to refresh"' 2024-03-04T13:55:05.095 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:55:04 smithi107 ceph-mon[112578]: from='client.? 172.21.15.107:0/3551485928' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:55:05.112 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:55:04 smithi169 ceph-mon[86183]: from='client.? 172.21.15.107:0/3551485928' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:55:05.348 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/mon.smithi107/config 2024-03-04T13:55:06.095 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:55:05 smithi107 ceph-mon[112578]: pgmap v269: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:55:06.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:55:05 smithi169 ceph-mon[86183]: pgmap v269: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:55:07.497 INFO:teuthology.orchestra.run.smithi107.stdout:wait for servicemap items w/ changing names to refresh 2024-03-04T13:55:08.334 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:55:07 smithi107 ceph-mon[112578]: pgmap v270: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:55:08.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:55:07 smithi169 ceph-mon[86183]: pgmap v270: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:55:08.421 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 -e sha1=717ce3cce8d0166e4e577faa8004e6f9cb4128c0 -- bash -c 'sleep 60' 2024-03-04T13:55:08.900 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/mon.smithi107/config 2024-03-04T13:55:10.201 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:55:09 smithi107 ceph-mon[112578]: pgmap v271: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:55:10.201 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:55:09 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:55:10.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:55:09 smithi169 ceph-mon[86183]: pgmap v271: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:55:10.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:55:09 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:55:12.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:55:11 smithi107 ceph-mon[112578]: pgmap v272: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:55:12.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:55:11 smithi169 ceph-mon[86183]: pgmap v272: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:55:14.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:55:13 smithi107 ceph-mon[112578]: pgmap v273: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:55:14.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:55:13 smithi169 ceph-mon[86183]: pgmap v273: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:55:16.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:55:15 smithi107 ceph-mon[112578]: pgmap v274: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:55:16.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:55:15 smithi169 ceph-mon[86183]: pgmap v274: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:55:18.335 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:55:17 smithi107 ceph-mon[112578]: pgmap v275: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:55:18.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:55:17 smithi169 ceph-mon[86183]: pgmap v275: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:55:20.165 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:55:19 smithi169 ceph-mon[86183]: pgmap v276: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:55:20.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:55:19 smithi107 ceph-mon[112578]: pgmap v276: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:55:22.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:55:21 smithi107 ceph-mon[112578]: pgmap v277: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:55:22.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:55:21 smithi169 ceph-mon[86183]: pgmap v277: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:55:24.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:55:23 smithi107 ceph-mon[112578]: pgmap v278: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:55:24.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:55:23 smithi169 ceph-mon[86183]: pgmap v278: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:55:25.247 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:55:24 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:55:25.247 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:55:24 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:55:25.364 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:55:24 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:55:25.364 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:55:24 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:55:26.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:55:25 smithi107 ceph-mon[112578]: pgmap v279: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:55:26.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:55:25 smithi169 ceph-mon[86183]: pgmap v279: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:55:27.095 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:55:26 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:55:27.095 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:55:26 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:55:27.095 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:55:26 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:55:27.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:55:26 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:55:27.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:55:26 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:55:27.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:55:26 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:55:28.333 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:55:27 smithi107 ceph-mon[112578]: pgmap v280: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:55:28.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:55:27 smithi169 ceph-mon[86183]: pgmap v280: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:55:30.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:55:29 smithi107 ceph-mon[112578]: pgmap v281: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:55:30.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:55:29 smithi169 ceph-mon[86183]: pgmap v281: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:55:32.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:55:31 smithi107 ceph-mon[112578]: pgmap v282: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:55:32.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:55:31 smithi169 ceph-mon[86183]: pgmap v282: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:55:34.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:55:33 smithi107 ceph-mon[112578]: pgmap v283: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:55:34.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:55:33 smithi169 ceph-mon[86183]: pgmap v283: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:55:36.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:55:35 smithi107 ceph-mon[112578]: pgmap v284: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:55:36.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:55:35 smithi169 ceph-mon[86183]: pgmap v284: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:55:38.095 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:55:37 smithi107 ceph-mon[112578]: pgmap v285: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:55:38.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:55:37 smithi169 ceph-mon[86183]: pgmap v285: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:55:40.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:55:39 smithi107 ceph-mon[112578]: pgmap v286: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:55:40.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:55:39 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:55:40.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:55:39 smithi169 ceph-mon[86183]: pgmap v286: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:55:40.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:55:39 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:55:41.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:55:40 smithi107 ceph-mon[112578]: pgmap v287: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:55:41.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:55:40 smithi169 ceph-mon[86183]: pgmap v287: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:55:44.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:55:43 smithi107 ceph-mon[112578]: pgmap v288: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:55:44.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:55:43 smithi169 ceph-mon[86183]: pgmap v288: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:55:46.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:55:45 smithi107 ceph-mon[112578]: pgmap v289: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:55:46.372 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:55:45 smithi169 ceph-mon[86183]: pgmap v289: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:55:48.333 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:55:47 smithi107 ceph-mon[112578]: pgmap v290: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:55:48.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:55:47 smithi169 ceph-mon[86183]: pgmap v290: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:55:50.164 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:55:49 smithi169 ceph-mon[86183]: pgmap v291: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:55:50.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:55:49 smithi107 ceph-mon[112578]: pgmap v291: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:55:52.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:55:51 smithi107 ceph-mon[112578]: pgmap v292: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:55:52.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:55:51 smithi169 ceph-mon[86183]: pgmap v292: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:55:54.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:55:53 smithi107 ceph-mon[112578]: pgmap v293: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:55:54.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:55:53 smithi169 ceph-mon[86183]: pgmap v293: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:55:55.247 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:55:54 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:55:55.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:55:54 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:55:56.274 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:55:55 smithi107 ceph-mon[112578]: pgmap v294: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:55:56.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:55:55 smithi169 ceph-mon[86183]: pgmap v294: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:55:58.095 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:55:57 smithi107 ceph-mon[112578]: pgmap v295: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:55:58.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:55:57 smithi169 ceph-mon[86183]: pgmap v295: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:56:00.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:55:59 smithi107 ceph-mon[112578]: pgmap v296: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:56:00.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:55:59 smithi169 ceph-mon[86183]: pgmap v296: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:56:02.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:56:01 smithi107 ceph-mon[112578]: pgmap v297: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:56:02.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:56:01 smithi169 ceph-mon[86183]: pgmap v297: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:56:04.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:56:03 smithi107 ceph-mon[112578]: pgmap v298: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:56:04.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:56:03 smithi169 ceph-mon[86183]: pgmap v298: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:56:06.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:56:05 smithi107 ceph-mon[112578]: pgmap v299: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:56:06.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:56:05 smithi169 ceph-mon[86183]: pgmap v299: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:56:08.334 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:56:07 smithi107 ceph-mon[112578]: pgmap v300: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:56:08.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:56:07 smithi169 ceph-mon[86183]: pgmap v300: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:56:10.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:56:09 smithi107 ceph-mon[112578]: pgmap v301: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:56:10.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:56:09 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:56:10.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:56:09 smithi169 ceph-mon[86183]: pgmap v301: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:56:10.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:56:09 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:56:11.781 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 -e sha1=717ce3cce8d0166e4e577faa8004e6f9cb4128c0 -- bash -c 'ceph orch ps' 2024-03-04T13:56:12.275 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/mon.smithi107/config 2024-03-04T13:56:12.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:56:11 smithi107 ceph-mon[112578]: pgmap v302: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:56:12.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:56:11 smithi169 ceph-mon[86183]: pgmap v302: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:56:14.221 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:56:13 smithi107 ceph-mon[112578]: pgmap v303: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:56:14.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:56:13 smithi169 ceph-mon[86183]: pgmap v303: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:56:14.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:14.522+0000 7f19aae00700 1 -- 172.21.15.107:0/3137343019 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19a4105b10 msgr2=0x7f19a40ff180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:14.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:14.522+0000 7f19aae00700 1 --2- 172.21.15.107:0/3137343019 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19a4105b10 0x7f19a40ff180 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1990004940 tx=0x7f1990009e30 comp rx=0 tx=0).stop 2024-03-04T13:56:14.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:14.522+0000 7f19aae00700 1 -- 172.21.15.107:0/3137343019 shutdown_connections 2024-03-04T13:56:14.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:14.522+0000 7f19aae00700 1 --2- 172.21.15.107:0/3137343019 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19a4105b10 0x7f19a40ff180 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:14.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:14.522+0000 7f19aae00700 1 --2- 172.21.15.107:0/3137343019 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19a4105170 0x7f19a4105540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:14.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:14.522+0000 7f19aae00700 1 -- 172.21.15.107:0/3137343019 >> 172.21.15.107:0/3137343019 conn(0x7f19a40face0 msgr2=0x7f19a40fd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:56:14.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:14.522+0000 7f19aae00700 1 -- 172.21.15.107:0/3137343019 shutdown_connections 2024-03-04T13:56:14.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:14.522+0000 7f19aae00700 1 -- 172.21.15.107:0/3137343019 wait complete. 2024-03-04T13:56:14.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:14.523+0000 7f19aae00700 1 Processor -- start 2024-03-04T13:56:14.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:14.523+0000 7f19aae00700 1 -- start start 2024-03-04T13:56:14.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:14.523+0000 7f19aae00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19a4105170 0x7f19a4115de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:14.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:14.523+0000 7f19aae00700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19a4105b10 0x7f19a4116320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:14.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:14.523+0000 7f19aae00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19a4116a00 con 0x7f19a4105170 2024-03-04T13:56:14.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:14.523+0000 7f19aae00700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19a410fc80 con 0x7f19a4105b10 2024-03-04T13:56:14.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:14.523+0000 7f19a3e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19a4105170 0x7f19a4115de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:56:14.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:14.524+0000 7f19a3e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19a4105170 0x7f19a4115de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34284/0 (socket says 172.21.15.107:34284) 2024-03-04T13:56:14.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:14.524+0000 7f19a3e00700 1 -- 172.21.15.107:0/3046411715 learned_addr learned my addr 172.21.15.107:0/3046411715 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:56:14.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:14.524+0000 7f19a3400700 1 --2- 172.21.15.107:0/3046411715 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19a4105b10 0x7f19a4116320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:56:14.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:14.524+0000 7f19a3e00700 1 -- 172.21.15.107:0/3046411715 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19a4105b10 msgr2=0x7f19a4116320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:14.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:14.524+0000 7f19a3e00700 1 --2- 172.21.15.107:0/3046411715 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19a4105b10 0x7f19a4116320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:14.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:14.524+0000 7f19a3e00700 1 -- 172.21.15.107:0/3046411715 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1990005040 con 0x7f19a4105170 2024-03-04T13:56:14.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:14.524+0000 7f19a3400700 1 --2- 172.21.15.107:0/3046411715 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19a4105b10 0x7f19a4116320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T13:56:14.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:14.524+0000 7f19a3e00700 1 --2- 172.21.15.107:0/3046411715 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19a4105170 0x7f19a4115de0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f199400b790 tx=0x7f1994009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:56:14.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:14.525+0000 7f199be00700 1 -- 172.21.15.107:0/3046411715 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f199400f770 con 0x7f19a4105170 2024-03-04T13:56:14.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:14.525+0000 7f199be00700 1 -- 172.21.15.107:0/3046411715 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f199400fda0 con 0x7f19a4105170 2024-03-04T13:56:14.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:14.525+0000 7f19aae00700 1 -- 172.21.15.107:0/3046411715 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19a410ff00 con 0x7f19a4105170 2024-03-04T13:56:14.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:14.525+0000 7f199be00700 1 -- 172.21.15.107:0/3046411715 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f199400d310 con 0x7f19a4105170 2024-03-04T13:56:14.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:14.525+0000 7f19aae00700 1 -- 172.21.15.107:0/3046411715 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19a4110440 con 0x7f19a4105170 2024-03-04T13:56:14.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:14.526+0000 7f19aae00700 1 -- 172.21.15.107:0/3046411715 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19a40fe900 con 0x7f19a4105170 2024-03-04T13:56:14.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:14.527+0000 7f199be00700 1 -- 172.21.15.107:0/3046411715 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f1994010420 con 0x7f19a4105170 2024-03-04T13:56:14.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:14.528+0000 7f199be00700 1 --2- 172.21.15.107:0/3046411715 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f198806d7f0 0x7f198806fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:14.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:14.528+0000 7f199be00700 1 -- 172.21.15.107:0/3046411715 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f199408cf10 con 0x7f19a4105170 2024-03-04T13:56:14.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:14.528+0000 7f19a3400700 1 --2- 172.21.15.107:0/3046411715 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f198806d7f0 0x7f198806fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:56:14.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:14.529+0000 7f19a3400700 1 --2- 172.21.15.107:0/3046411715 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f198806d7f0 0x7f198806fca0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1990009fd0 tx=0x7f1990009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:56:14.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:14.533+0000 7f199be00700 1 -- 172.21.15.107:0/3046411715 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f199405a2c0 con 0x7f19a4105170 2024-03-04T13:56:14.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:14.762+0000 7f19aae00700 1 -- 172.21.15.107:0/3046411715 --> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f19a4111460 con 0x7f198806d7f0 2024-03-04T13:56:14.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:14.793+0000 7f199be00700 1 -- 172.21.15.107:0/3046411715 <== mgr.34102 v2:172.21.15.107:6800/1223039812 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f19a4111460 con 0x7f198806d7f0 2024-03-04T13:56:14.796 INFO:teuthology.orchestra.run.smithi107.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-04T13:56:14.796 INFO:teuthology.orchestra.run.smithi107.stdout:alertmanager.smithi107 smithi107 *:9093,9094 running (10m) 2m ago 24m 14.4M - 0.25.0 c8568f914cd2 e5ed7be32538 2024-03-04T13:56:14.796 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi107 smithi107 running (7m) 2m ago 24m 6660k - 19.0.0-1845-g717ce3cc d71b839abdf8 0558022b0d84 2024-03-04T13:56:14.796 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi169 smithi169 running (7m) 2m ago 23m 6665k - 19.0.0-1845-g717ce3cc d71b839abdf8 221ca53aa0d3 2024-03-04T13:56:14.796 INFO:teuthology.orchestra.run.smithi107.stdout:grafana.smithi107 smithi107 *:3000 running (10m) 2m ago 24m 75.1M - 9.4.12 a0ea5e3c46d9 c5e4d33fb599 2024-03-04T13:56:14.796 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.riexmm smithi107 running (3m) 2m ago 17m 11.0M - 19.0.0-1845-g717ce3cc d71b839abdf8 8582388ad90a 2024-03-04T13:56:14.797 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.thycho smithi107 running (3m) 2m ago 17m 10.8M - 19.0.0-1845-g717ce3cc d71b839abdf8 f4b3cea14a6a 2024-03-04T13:56:14.797 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi169.bxitzk smithi169 running (2m) 2m ago 17m 10.9M - 19.0.0-1845-g717ce3cc d71b839abdf8 1ea541510c55 2024-03-04T13:56:14.797 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi169.jusudl smithi169 running (2m) 2m ago 17m 11.1M - 19.0.0-1845-g717ce3cc d71b839abdf8 e6d45009ccbb 2024-03-04T13:56:14.797 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi107.wpzgwl smithi107 *:8443,9283,8765 running (13m) 2m ago 26m 509M - 19.0.0-1845-g717ce3cc d71b839abdf8 1c5fdee45eac 2024-03-04T13:56:14.797 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi169.eihczr smithi169 *:8443,9283,8765 running (12m) 2m ago 23m 452M - 19.0.0-1845-g717ce3cc d71b839abdf8 b5cac2b7d22a 2024-03-04T13:56:14.797 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi107 smithi107 running (9m) 2m ago 26m 52.5M 2048M 19.0.0-1845-g717ce3cc d71b839abdf8 32f07144274f 2024-03-04T13:56:14.797 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi169 smithi169 running (8m) 2m ago 23m 41.6M 2048M 19.0.0-1845-g717ce3cc d71b839abdf8 e3527449381a 2024-03-04T13:56:14.797 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi107 smithi107 *:9100 running (11m) 2m ago 24m 8102k - 1.5.0 0da6a335fe13 8a8d3d4f724f 2024-03-04T13:56:14.797 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi169 smithi169 *:9100 running (11m) 2m ago 23m 8158k - 1.5.0 0da6a335fe13 f240f5a975bc 2024-03-04T13:56:14.797 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0 smithi107 running (6m) 2m ago 22m 85.3M 1558M 19.0.0-1845-g717ce3cc d71b839abdf8 5801a5540dc6 2024-03-04T13:56:14.797 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1 smithi107 running (6m) 2m ago 21m 94.0M 1558M 19.0.0-1845-g717ce3cc d71b839abdf8 b614af658854 2024-03-04T13:56:14.797 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2 smithi107 running (5m) 2m ago 21m 84.9M 1558M 19.0.0-1845-g717ce3cc d71b839abdf8 6e4da4f6f146 2024-03-04T13:56:14.797 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3 smithi169 running (5m) 2m ago 20m 107M 2582M 19.0.0-1845-g717ce3cc d71b839abdf8 d31baf764d7a 2024-03-04T13:56:14.797 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4 smithi169 running (4m) 2m ago 20m 90.7M 2582M 19.0.0-1845-g717ce3cc d71b839abdf8 a70bacd97639 2024-03-04T13:56:14.798 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5 smithi169 running (3m) 2m ago 19m 86.0M 2582M 19.0.0-1845-g717ce3cc d71b839abdf8 5f6e00f7993d 2024-03-04T13:56:14.798 INFO:teuthology.orchestra.run.smithi107.stdout:prometheus.smithi107 smithi107 *:9095 running (10m) 2m ago 23m 54.9M - 2.43.0 a07b618ecd1d 1eeb1ad5cb75 2024-03-04T13:56:14.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:14.796+0000 7f19aae00700 1 -- 172.21.15.107:0/3046411715 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f198806d7f0 msgr2=0x7f198806fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:14.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:14.796+0000 7f19aae00700 1 --2- 172.21.15.107:0/3046411715 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f198806d7f0 0x7f198806fca0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1990009fd0 tx=0x7f1990009e30 comp rx=0 tx=0).stop 2024-03-04T13:56:14.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:14.796+0000 7f19aae00700 1 -- 172.21.15.107:0/3046411715 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19a4105170 msgr2=0x7f19a4115de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:14.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:14.796+0000 7f19aae00700 1 --2- 172.21.15.107:0/3046411715 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19a4105170 0x7f19a4115de0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f199400b790 tx=0x7f1994009250 comp rx=0 tx=0).stop 2024-03-04T13:56:14.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:14.796+0000 7f19aae00700 1 -- 172.21.15.107:0/3046411715 shutdown_connections 2024-03-04T13:56:14.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:14.796+0000 7f19aae00700 1 --2- 172.21.15.107:0/3046411715 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f198806d7f0 0x7f198806fca0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:14.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:14.796+0000 7f19aae00700 1 --2- 172.21.15.107:0/3046411715 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19a4105170 0x7f19a4115de0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:14.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:14.796+0000 7f19aae00700 1 --2- 172.21.15.107:0/3046411715 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f19a4105b10 0x7f19a4116320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:14.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:14.796+0000 7f19aae00700 1 -- 172.21.15.107:0/3046411715 >> 172.21.15.107:0/3046411715 conn(0x7f19a40face0 msgr2=0x7f19a40fc710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:56:14.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:14.797+0000 7f19aae00700 1 -- 172.21.15.107:0/3046411715 shutdown_connections 2024-03-04T13:56:14.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:14.797+0000 7f19aae00700 1 -- 172.21.15.107:0/3046411715 wait complete. 2024-03-04T13:56:15.650 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 -e sha1=717ce3cce8d0166e4e577faa8004e6f9cb4128c0 -- bash -c 'ceph versions' 2024-03-04T13:56:16.136 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/mon.smithi107/config 2024-03-04T13:56:16.284 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:56:15 smithi107 ceph-mon[112578]: from='client.34546 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:56:16.284 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:56:15 smithi107 ceph-mon[112578]: pgmap v304: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:56:16.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:56:15 smithi169 ceph-mon[86183]: from='client.34546 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T13:56:16.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:56:15 smithi169 ceph-mon[86183]: pgmap v304: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:56:18.095 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:56:17 smithi107 ceph-mon[112578]: pgmap v305: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:56:18.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:18.288+0000 7fe85e200700 1 -- 172.21.15.107:0/3662880084 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe858103dc0 msgr2=0x7fe858104230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:18.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:18.288+0000 7fe85e200700 1 --2- 172.21.15.107:0/3662880084 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe858103dc0 0x7fe858104230 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe844004940 tx=0x7fe844009e30 comp rx=0 tx=0).stop 2024-03-04T13:56:18.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:18.288+0000 7fe85e200700 1 -- 172.21.15.107:0/3662880084 shutdown_connections 2024-03-04T13:56:18.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:18.288+0000 7fe85e200700 1 --2- 172.21.15.107:0/3662880084 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe858103dc0 0x7fe858104230 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:18.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:18.288+0000 7fe85e200700 1 --2- 172.21.15.107:0/3662880084 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8581095b0 0x7fe858109980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:18.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:18.288+0000 7fe85e200700 1 -- 172.21.15.107:0/3662880084 >> 172.21.15.107:0/3662880084 conn(0x7fe858072d70 msgr2=0x7fe858075180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:56:18.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:18.288+0000 7fe85e200700 1 -- 172.21.15.107:0/3662880084 shutdown_connections 2024-03-04T13:56:18.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:18.289+0000 7fe85e200700 1 -- 172.21.15.107:0/3662880084 wait complete. 2024-03-04T13:56:18.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:18.289+0000 7fe85e200700 1 Processor -- start 2024-03-04T13:56:18.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:18.289+0000 7fe85e200700 1 -- start start 2024-03-04T13:56:18.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:18.290+0000 7fe85e200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe858103dc0 0x7fe85811a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:18.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:18.290+0000 7fe85e200700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8581095b0 0x7fe85811a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:18.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:18.290+0000 7fe85e200700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe85811ac00 con 0x7fe858103dc0 2024-03-04T13:56:18.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:18.290+0000 7fe85e200700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe858113f10 con 0x7fe8581095b0 2024-03-04T13:56:18.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:18.290+0000 7fe857400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe858103dc0 0x7fe85811a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:56:18.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:18.290+0000 7fe857400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe858103dc0 0x7fe85811a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41248/0 (socket says 172.21.15.107:41248) 2024-03-04T13:56:18.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:18.290+0000 7fe857400700 1 -- 172.21.15.107:0/3791971662 learned_addr learned my addr 172.21.15.107:0/3791971662 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:56:18.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:18.290+0000 7fe857400700 1 -- 172.21.15.107:0/3791971662 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8581095b0 msgr2=0x7fe85811a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:18.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:18.290+0000 7fe857400700 1 --2- 172.21.15.107:0/3791971662 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8581095b0 0x7fe85811a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:18.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:18.290+0000 7fe857400700 1 -- 172.21.15.107:0/3791971662 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe844005040 con 0x7fe858103dc0 2024-03-04T13:56:18.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:18.291+0000 7fe857400700 1 --2- 172.21.15.107:0/3791971662 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe858103dc0 0x7fe85811a070 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fe84800c8e0 tx=0x7fe8480044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:56:18.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:18.291+0000 7fe84f400700 1 -- 172.21.15.107:0/3791971662 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe84800a940 con 0x7fe858103dc0 2024-03-04T13:56:18.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:18.291+0000 7fe84f400700 1 -- 172.21.15.107:0/3791971662 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe84800ce80 con 0x7fe858103dc0 2024-03-04T13:56:18.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:18.291+0000 7fe84f400700 1 -- 172.21.15.107:0/3791971662 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe8480113a0 con 0x7fe858103dc0 2024-03-04T13:56:18.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:18.291+0000 7fe85e200700 1 -- 172.21.15.107:0/3791971662 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8581141f0 con 0x7fe858103dc0 2024-03-04T13:56:18.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:18.291+0000 7fe85e200700 1 -- 172.21.15.107:0/3791971662 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe858114730 con 0x7fe858103dc0 2024-03-04T13:56:18.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:18.292+0000 7fe85e200700 1 -- 172.21.15.107:0/3791971662 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe85810b990 con 0x7fe858103dc0 2024-03-04T13:56:18.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:18.293+0000 7fe84f400700 1 -- 172.21.15.107:0/3791971662 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fe848012400 con 0x7fe858103dc0 2024-03-04T13:56:18.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:18.294+0000 7fe84f400700 1 --2- 172.21.15.107:0/3791971662 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fe83c06d7f0 0x7fe83c06fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:18.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:18.294+0000 7fe84f400700 1 -- 172.21.15.107:0/3791971662 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fe84808cf50 con 0x7fe858103dc0 2024-03-04T13:56:18.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:18.295+0000 7fe856a00700 1 --2- 172.21.15.107:0/3791971662 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fe83c06d7f0 0x7fe83c06fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:56:18.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:18.295+0000 7fe856a00700 1 --2- 172.21.15.107:0/3791971662 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fe83c06d7f0 0x7fe83c06fca0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe844004f90 tx=0x7fe844009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:56:18.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:18.299+0000 7fe84f400700 1 -- 172.21.15.107:0/3791971662 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe84805a300 con 0x7fe858103dc0 2024-03-04T13:56:18.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:56:17 smithi169 ceph-mon[86183]: pgmap v305: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:56:18.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:18.608+0000 7fe85e200700 1 -- 172.21.15.107:0/3791971662 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fe85804bed0 con 0x7fe858103dc0 2024-03-04T13:56:18.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:18.609+0000 7fe84f400700 1 -- 172.21.15.107:0/3791971662 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fe848059e90 con 0x7fe858103dc0 2024-03-04T13:56:18.612 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-03-04T13:56:18.612 INFO:teuthology.orchestra.run.smithi107.stdout: "mon": { 2024-03-04T13:56:18.612 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-1845-g717ce3cc (717ce3cce8d0166e4e577faa8004e6f9cb4128c0) squid (dev)": 2 2024-03-04T13:56:18.612 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:56:18.612 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr": { 2024-03-04T13:56:18.613 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-1845-g717ce3cc (717ce3cce8d0166e4e577faa8004e6f9cb4128c0) squid (dev)": 2 2024-03-04T13:56:18.613 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:56:18.613 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": { 2024-03-04T13:56:18.613 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-1845-g717ce3cc (717ce3cce8d0166e4e577faa8004e6f9cb4128c0) squid (dev)": 6 2024-03-04T13:56:18.613 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:56:18.613 INFO:teuthology.orchestra.run.smithi107.stdout: "mds": { 2024-03-04T13:56:18.613 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-1845-g717ce3cc (717ce3cce8d0166e4e577faa8004e6f9cb4128c0) squid (dev)": 4 2024-03-04T13:56:18.613 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-04T13:56:18.613 INFO:teuthology.orchestra.run.smithi107.stdout: "overall": { 2024-03-04T13:56:18.613 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-1845-g717ce3cc (717ce3cce8d0166e4e577faa8004e6f9cb4128c0) squid (dev)": 14 2024-03-04T13:56:18.613 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-04T13:56:18.613 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-03-04T13:56:18.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:18.612+0000 7fe85e200700 1 -- 172.21.15.107:0/3791971662 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fe83c06d7f0 msgr2=0x7fe83c06fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:18.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:18.612+0000 7fe85e200700 1 --2- 172.21.15.107:0/3791971662 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fe83c06d7f0 0x7fe83c06fca0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe844004f90 tx=0x7fe844009e30 comp rx=0 tx=0).stop 2024-03-04T13:56:18.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:18.612+0000 7fe85e200700 1 -- 172.21.15.107:0/3791971662 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe858103dc0 msgr2=0x7fe85811a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:18.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:18.612+0000 7fe85e200700 1 --2- 172.21.15.107:0/3791971662 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe858103dc0 0x7fe85811a070 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fe84800c8e0 tx=0x7fe8480044c0 comp rx=0 tx=0).stop 2024-03-04T13:56:18.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:18.612+0000 7fe85e200700 1 -- 172.21.15.107:0/3791971662 shutdown_connections 2024-03-04T13:56:18.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:18.612+0000 7fe85e200700 1 --2- 172.21.15.107:0/3791971662 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fe83c06d7f0 0x7fe83c06fca0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:18.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:18.612+0000 7fe85e200700 1 --2- 172.21.15.107:0/3791971662 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe858103dc0 0x7fe85811a070 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:18.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:18.612+0000 7fe85e200700 1 --2- 172.21.15.107:0/3791971662 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe8581095b0 0x7fe85811a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:18.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:18.612+0000 7fe85e200700 1 -- 172.21.15.107:0/3791971662 >> 172.21.15.107:0/3791971662 conn(0x7fe858072d70 msgr2=0x7fe858074960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:56:18.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:18.612+0000 7fe85e200700 1 -- 172.21.15.107:0/3791971662 shutdown_connections 2024-03-04T13:56:18.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:18.612+0000 7fe85e200700 1 -- 172.21.15.107:0/3791971662 wait complete. 2024-03-04T13:56:19.332 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:56:18 smithi107 ceph-mon[112578]: from='client.? 172.21.15.107:0/3791971662' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:56:19.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:56:18 smithi169 ceph-mon[86183]: from='client.? 172.21.15.107:0/3791971662' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:56:19.408 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 -e sha1=717ce3cce8d0166e4e577faa8004e6f9cb4128c0 -- bash -c 'ceph versions | jq -e '"'"'.overall | length == 1'"'"'' 2024-03-04T13:56:19.909 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/mon.smithi107/config 2024-03-04T13:56:20.233 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:56:19 smithi107 ceph-mon[112578]: pgmap v306: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:56:20.264 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:56:19 smithi169 ceph-mon[86183]: pgmap v306: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:56:22.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:22.088+0000 7f84ea400700 1 -- 172.21.15.107:0/1370097938 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f84e40702d0 msgr2=0x7f84e40706a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:22.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:22.088+0000 7f84ea400700 1 --2- 172.21.15.107:0/1370097938 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f84e40702d0 0x7f84e40706a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f84d4004970 tx=0x7f84d4009e30 comp rx=0 tx=0).stop 2024-03-04T13:56:22.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:22.088+0000 7f84ea400700 1 -- 172.21.15.107:0/1370097938 shutdown_connections 2024-03-04T13:56:22.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:22.088+0000 7f84ea400700 1 --2- 172.21.15.107:0/1370097938 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84e4070be0 0x7f84e41092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:22.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:22.088+0000 7f84ea400700 1 --2- 172.21.15.107:0/1370097938 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f84e40702d0 0x7f84e40706a0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:22.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:22.088+0000 7f84ea400700 1 -- 172.21.15.107:0/1370097938 >> 172.21.15.107:0/1370097938 conn(0x7f84e40f8ae0 msgr2=0x7f84e40faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:56:22.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:22.088+0000 7f84ea400700 1 -- 172.21.15.107:0/1370097938 shutdown_connections 2024-03-04T13:56:22.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:22.088+0000 7f84ea400700 1 -- 172.21.15.107:0/1370097938 wait complete. 2024-03-04T13:56:22.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:22.089+0000 7f84ea400700 1 Processor -- start 2024-03-04T13:56:22.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:22.089+0000 7f84ea400700 1 -- start start 2024-03-04T13:56:22.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:22.090+0000 7f84ea400700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84e40702d0 0x7f84e4115c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:22.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:22.090+0000 7f84ea400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f84e4070be0 0x7f84e4116160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:22.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:22.090+0000 7f84ea400700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84e41167b0 con 0x7f84e4070be0 2024-03-04T13:56:22.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:22.090+0000 7f84ea400700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84e410fae0 con 0x7f84e40702d0 2024-03-04T13:56:22.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:22.090+0000 7f84daa00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f84e4070be0 0x7f84e4116160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:56:22.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:22.090+0000 7f84daa00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f84e4070be0 0x7f84e4116160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41272/0 (socket says 172.21.15.107:41272) 2024-03-04T13:56:22.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:22.090+0000 7f84daa00700 1 -- 172.21.15.107:0/2329369358 learned_addr learned my addr 172.21.15.107:0/2329369358 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:56:22.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:22.090+0000 7f84e3400700 1 --2- 172.21.15.107:0/2329369358 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84e40702d0 0x7f84e4115c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:56:22.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:22.091+0000 7f84daa00700 1 -- 172.21.15.107:0/2329369358 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84e40702d0 msgr2=0x7f84e4115c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:22.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:22.091+0000 7f84daa00700 1 --2- 172.21.15.107:0/2329369358 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84e40702d0 0x7f84e4115c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:22.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:22.091+0000 7f84daa00700 1 -- 172.21.15.107:0/2329369358 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84d4005040 con 0x7f84e4070be0 2024-03-04T13:56:22.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:22.091+0000 7f84daa00700 1 --2- 172.21.15.107:0/2329369358 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f84e4070be0 0x7f84e4116160 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f84cc00c8c0 tx=0x7f84cc0044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:56:22.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:22.091+0000 7f84dbe00700 1 -- 172.21.15.107:0/2329369358 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f84cc00a980 con 0x7f84e4070be0 2024-03-04T13:56:22.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:22.091+0000 7f84dbe00700 1 -- 172.21.15.107:0/2329369358 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f84cc00ce60 con 0x7f84e4070be0 2024-03-04T13:56:22.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:22.091+0000 7f84dbe00700 1 -- 172.21.15.107:0/2329369358 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f84cc0115e0 con 0x7f84e4070be0 2024-03-04T13:56:22.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:22.091+0000 7f84ea400700 1 -- 172.21.15.107:0/2329369358 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84e410fdc0 con 0x7f84e4070be0 2024-03-04T13:56:22.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:22.091+0000 7f84ea400700 1 -- 172.21.15.107:0/2329369358 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84e4110300 con 0x7f84e4070be0 2024-03-04T13:56:22.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:22.092+0000 7f84ea400700 1 -- 172.21.15.107:0/2329369358 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84e4106980 con 0x7f84e4070be0 2024-03-04T13:56:22.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:22.093+0000 7f84dbe00700 1 -- 172.21.15.107:0/2329369358 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f84cc012400 con 0x7f84e4070be0 2024-03-04T13:56:22.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:22.094+0000 7f84dbe00700 1 --2- 172.21.15.107:0/2329369358 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f84c806d7f0 0x7f84c806fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:22.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:22.094+0000 7f84dbe00700 1 -- 172.21.15.107:0/2329369358 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f84cc08cfa0 con 0x7f84e4070be0 2024-03-04T13:56:22.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:22.095+0000 7f84e3400700 1 --2- 172.21.15.107:0/2329369358 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f84c806d7f0 0x7f84c806fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:56:22.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:22.095+0000 7f84e3400700 1 --2- 172.21.15.107:0/2329369358 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f84c806d7f0 0x7f84c806fca0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f84d4004ee0 tx=0x7f84d4009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:56:22.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:22.100+0000 7f84dbe00700 1 -- 172.21.15.107:0/2329369358 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f84cc05a350 con 0x7f84e4070be0 2024-03-04T13:56:22.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:56:21 smithi107 ceph-mon[112578]: pgmap v307: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:56:22.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:56:21 smithi169 ceph-mon[86183]: pgmap v307: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:56:22.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:22.404+0000 7f84ea400700 1 -- 172.21.15.107:0/2329369358 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f84e410c240 con 0x7f84e4070be0 2024-03-04T13:56:22.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:22.405+0000 7f84dbe00700 1 -- 172.21.15.107:0/2329369358 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f84cc0126b0 con 0x7f84e4070be0 2024-03-04T13:56:22.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:22.408+0000 7f84ea400700 1 -- 172.21.15.107:0/2329369358 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f84c806d7f0 msgr2=0x7f84c806fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:22.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:22.408+0000 7f84ea400700 1 --2- 172.21.15.107:0/2329369358 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f84c806d7f0 0x7f84c806fca0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f84d4004ee0 tx=0x7f84d4009e30 comp rx=0 tx=0).stop 2024-03-04T13:56:22.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:22.408+0000 7f84ea400700 1 -- 172.21.15.107:0/2329369358 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f84e4070be0 msgr2=0x7f84e4116160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:22.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:22.408+0000 7f84ea400700 1 --2- 172.21.15.107:0/2329369358 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f84e4070be0 0x7f84e4116160 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f84cc00c8c0 tx=0x7f84cc0044c0 comp rx=0 tx=0).stop 2024-03-04T13:56:22.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:22.408+0000 7f84ea400700 1 -- 172.21.15.107:0/2329369358 shutdown_connections 2024-03-04T13:56:22.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:22.408+0000 7f84ea400700 1 --2- 172.21.15.107:0/2329369358 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f84c806d7f0 0x7f84c806fca0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:22.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:22.408+0000 7f84ea400700 1 --2- 172.21.15.107:0/2329369358 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84e40702d0 0x7f84e4115c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:22.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:22.408+0000 7f84ea400700 1 --2- 172.21.15.107:0/2329369358 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f84e4070be0 0x7f84e4116160 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:22.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:22.408+0000 7f84ea400700 1 -- 172.21.15.107:0/2329369358 >> 172.21.15.107:0/2329369358 conn(0x7f84e40f8ae0 msgr2=0x7f84e4103b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:56:22.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:22.408+0000 7f84ea400700 1 -- 172.21.15.107:0/2329369358 shutdown_connections 2024-03-04T13:56:22.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:22.408+0000 7f84ea400700 1 -- 172.21.15.107:0/2329369358 wait complete. 2024-03-04T13:56:22.426 INFO:teuthology.orchestra.run.smithi107.stdout:true 2024-03-04T13:56:23.095 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:56:22 smithi107 ceph-mon[112578]: from='client.? 172.21.15.107:0/2329369358' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:56:23.237 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 -e sha1=717ce3cce8d0166e4e577faa8004e6f9cb4128c0 -- bash -c 'ceph versions | jq -e '"'"'.overall | keys'"'"' | grep $sha1' 2024-03-04T13:56:23.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:56:22 smithi169 ceph-mon[86183]: from='client.? 172.21.15.107:0/2329369358' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:56:23.734 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/mon.smithi107/config 2024-03-04T13:56:24.159 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:56:23 smithi107 ceph-mon[112578]: pgmap v308: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:56:24.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:56:24 smithi169 ceph-mon[86183]: pgmap v308: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:56:25.257 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:56:25 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:56:25.257 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:56:25 smithi107 ceph-mon[112578]: pgmap v309: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-03-04T13:56:25.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:56:25 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:56:25.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:56:25 smithi169 ceph-mon[86183]: pgmap v309: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-03-04T13:56:25.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:25.845+0000 7f019f800700 1 -- 172.21.15.107:0/3583125751 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0198070c40 msgr2=0x7f0198109260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:25.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:25.845+0000 7f019f800700 1 --2- 172.21.15.107:0/3583125751 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0198070c40 0x7f0198109260 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0184004970 tx=0x7f0184009e30 comp rx=0 tx=0).stop 2024-03-04T13:56:25.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:25.845+0000 7f019f800700 1 -- 172.21.15.107:0/3583125751 shutdown_connections 2024-03-04T13:56:25.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:25.845+0000 7f019f800700 1 --2- 172.21.15.107:0/3583125751 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0198070c40 0x7f0198109260 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:25.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:25.845+0000 7f019f800700 1 --2- 172.21.15.107:0/3583125751 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0198070330 0x7f0198070700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:25.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:25.845+0000 7f019f800700 1 -- 172.21.15.107:0/3583125751 >> 172.21.15.107:0/3583125751 conn(0x7f01980f8ae0 msgr2=0x7f01980faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:56:25.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:25.846+0000 7f019f800700 1 -- 172.21.15.107:0/3583125751 shutdown_connections 2024-03-04T13:56:25.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:25.846+0000 7f019f800700 1 -- 172.21.15.107:0/3583125751 wait complete. 2024-03-04T13:56:25.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:25.846+0000 7f019f800700 1 Processor -- start 2024-03-04T13:56:25.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:25.847+0000 7f019f800700 1 -- start start 2024-03-04T13:56:25.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:25.847+0000 7f019f800700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0198070330 0x7f0198115c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:25.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:25.847+0000 7f019f800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0198070c40 0x7f0198116140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:25.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:25.847+0000 7f019f800700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0198116790 con 0x7f0198070c40 2024-03-04T13:56:25.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:25.847+0000 7f019f800700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f019810fae0 con 0x7f0198070330 2024-03-04T13:56:25.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:25.847+0000 7f018fe00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0198070c40 0x7f0198116140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:56:25.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:25.847+0000 7f018fe00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0198070c40 0x7f0198116140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41276/0 (socket says 172.21.15.107:41276) 2024-03-04T13:56:25.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:25.847+0000 7f018fe00700 1 -- 172.21.15.107:0/996584272 learned_addr learned my addr 172.21.15.107:0/996584272 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:56:25.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:25.848+0000 7f019ce00700 1 --2- 172.21.15.107:0/996584272 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0198070330 0x7f0198115c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:56:25.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:25.848+0000 7f018fe00700 1 -- 172.21.15.107:0/996584272 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0198070330 msgr2=0x7f0198115c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:25.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:25.848+0000 7f018fe00700 1 --2- 172.21.15.107:0/996584272 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0198070330 0x7f0198115c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:25.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:25.848+0000 7f018fe00700 1 -- 172.21.15.107:0/996584272 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0184005040 con 0x7f0198070c40 2024-03-04T13:56:25.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:25.848+0000 7f018fe00700 1 --2- 172.21.15.107:0/996584272 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0198070c40 0x7f0198116140 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0184009150 tx=0x7f0184009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:56:25.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:25.848+0000 7f0195600700 1 -- 172.21.15.107:0/996584272 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f018400e040 con 0x7f0198070c40 2024-03-04T13:56:25.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:25.848+0000 7f0195600700 1 -- 172.21.15.107:0/996584272 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0184004290 con 0x7f0198070c40 2024-03-04T13:56:25.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:25.848+0000 7f0195600700 1 -- 172.21.15.107:0/996584272 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f018401c630 con 0x7f0198070c40 2024-03-04T13:56:25.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:25.848+0000 7f019f800700 1 -- 172.21.15.107:0/996584272 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f019810fd60 con 0x7f0198070c40 2024-03-04T13:56:25.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:25.848+0000 7f019f800700 1 -- 172.21.15.107:0/996584272 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01981102a0 con 0x7f0198070c40 2024-03-04T13:56:25.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:25.850+0000 7f019f800700 1 -- 172.21.15.107:0/996584272 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0198106920 con 0x7f0198070c40 2024-03-04T13:56:25.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:25.851+0000 7f0195600700 1 -- 172.21.15.107:0/996584272 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f0184013980 con 0x7f0198070c40 2024-03-04T13:56:25.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:25.851+0000 7f0195600700 1 --2- 172.21.15.107:0/996584272 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f017406d7f0 0x7f017406fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:25.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:25.852+0000 7f0195600700 1 -- 172.21.15.107:0/996584272 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f018408e220 con 0x7f0198070c40 2024-03-04T13:56:25.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:25.852+0000 7f019ce00700 1 --2- 172.21.15.107:0/996584272 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f017406d7f0 0x7f017406fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:56:25.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:25.852+0000 7f019ce00700 1 --2- 172.21.15.107:0/996584272 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f017406d7f0 0x7f017406fca0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f017c004c40 tx=0x7f017c009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:56:25.859 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:25.857+0000 7f0195600700 1 -- 172.21.15.107:0/996584272 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0184057aa0 con 0x7f0198070c40 2024-03-04T13:56:26.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:26.169+0000 7f019f800700 1 -- 172.21.15.107:0/996584272 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f019804bed0 con 0x7f0198070c40 2024-03-04T13:56:26.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:26.170+0000 7f0195600700 1 -- 172.21.15.107:0/996584272 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f018405b160 con 0x7f0198070c40 2024-03-04T13:56:26.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:26.173+0000 7f019f800700 1 -- 172.21.15.107:0/996584272 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f017406d7f0 msgr2=0x7f017406fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:26.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:26.173+0000 7f019f800700 1 --2- 172.21.15.107:0/996584272 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f017406d7f0 0x7f017406fca0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f017c004c40 tx=0x7f017c009040 comp rx=0 tx=0).stop 2024-03-04T13:56:26.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:26.173+0000 7f019f800700 1 -- 172.21.15.107:0/996584272 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0198070c40 msgr2=0x7f0198116140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:26.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:26.173+0000 7f019f800700 1 --2- 172.21.15.107:0/996584272 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0198070c40 0x7f0198116140 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0184009150 tx=0x7f0184009e30 comp rx=0 tx=0).stop 2024-03-04T13:56:26.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:26.173+0000 7f019f800700 1 -- 172.21.15.107:0/996584272 shutdown_connections 2024-03-04T13:56:26.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:26.173+0000 7f019f800700 1 --2- 172.21.15.107:0/996584272 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f017406d7f0 0x7f017406fca0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:26.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:26.173+0000 7f019f800700 1 --2- 172.21.15.107:0/996584272 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0198070330 0x7f0198115c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:26.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:26.173+0000 7f019f800700 1 --2- 172.21.15.107:0/996584272 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0198070c40 0x7f0198116140 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:26.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:26.173+0000 7f019f800700 1 -- 172.21.15.107:0/996584272 >> 172.21.15.107:0/996584272 conn(0x7f01980f8ae0 msgr2=0x7f0198103aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:56:26.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:26.173+0000 7f019f800700 1 -- 172.21.15.107:0/996584272 shutdown_connections 2024-03-04T13:56:26.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:26.173+0000 7f019f800700 1 -- 172.21.15.107:0/996584272 wait complete. 2024-03-04T13:56:26.192 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-1845-g717ce3cc (717ce3cce8d0166e4e577faa8004e6f9cb4128c0) squid (dev)" 2024-03-04T13:56:26.556 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:56:26 smithi107 ceph-mon[112578]: from='client.? 172.21.15.107:0/996584272' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:56:26.612 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:56:26 smithi169 ceph-mon[86183]: from='client.? 172.21.15.107:0/996584272' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-04T13:56:27.037 DEBUG:teuthology.parallel:result is None 2024-03-04T13:56:27.038 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-04T13:56:27.050 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi107.front.sepia.ceph.com 2024-03-04T13:56:27.050 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 -- bash -c 'ceph fs dump' 2024-03-04T13:56:27.499 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/mon.smithi107/config 2024-03-04T13:56:27.595 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:56:27 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:56:27.595 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:56:27 smithi107 ceph-mon[112578]: pgmap v310: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-03-04T13:56:27.612 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:56:27 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:56:27.612 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:56:27 smithi169 ceph-mon[86183]: pgmap v310: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-03-04T13:56:28.595 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:56:28 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:56:28.595 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:56:28 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:56:28.595 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:56:28 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:56:28.612 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:56:28 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:56:28.612 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:56:28 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:56:28.612 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:56:28 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:56:29.505 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:56:29 smithi107 ceph-mon[112578]: pgmap v311: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:56:29.612 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:56:29 smithi169 ceph-mon[86183]: pgmap v311: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:56:29.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:29.668+0000 7f8ade000700 1 -- 172.21.15.107:0/3462696864 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8ad8103dc0 msgr2=0x7f8ad8104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:29.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:29.668+0000 7f8ade000700 1 --2- 172.21.15.107:0/3462696864 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8ad8103dc0 0x7f8ad8104210 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f8ac4004940 tx=0x7f8ac4009e30 comp rx=0 tx=0).stop 2024-03-04T13:56:29.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:29.668+0000 7f8ade000700 1 -- 172.21.15.107:0/3462696864 shutdown_connections 2024-03-04T13:56:29.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:29.668+0000 7f8ade000700 1 --2- 172.21.15.107:0/3462696864 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8ad8103dc0 0x7f8ad8104210 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:29.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:29.668+0000 7f8ade000700 1 --2- 172.21.15.107:0/3462696864 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ad81095f0 0x7f8ad81099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:29.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:29.668+0000 7f8ade000700 1 -- 172.21.15.107:0/3462696864 >> 172.21.15.107:0/3462696864 conn(0x7f8ad8072d50 msgr2=0x7f8ad8075160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:56:29.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:29.668+0000 7f8ade000700 1 -- 172.21.15.107:0/3462696864 shutdown_connections 2024-03-04T13:56:29.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:29.668+0000 7f8ade000700 1 -- 172.21.15.107:0/3462696864 wait complete. 2024-03-04T13:56:29.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:29.669+0000 7f8ade000700 1 Processor -- start 2024-03-04T13:56:29.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:29.669+0000 7f8ade000700 1 -- start start 2024-03-04T13:56:29.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:29.670+0000 7f8ade000700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ad8103dc0 0x7f8ad811a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:29.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:29.670+0000 7f8ade000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8ad81095f0 0x7f8ad811a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:29.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:29.670+0000 7f8ade000700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad811abd0 con 0x7f8ad81095f0 2024-03-04T13:56:29.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:29.670+0000 7f8ade000700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad8113ee0 con 0x7f8ad8103dc0 2024-03-04T13:56:29.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:29.670+0000 7f8ad6a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8ad81095f0 0x7f8ad811a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:56:29.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:29.670+0000 7f8ad6a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8ad81095f0 0x7f8ad811a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51724/0 (socket says 172.21.15.107:51724) 2024-03-04T13:56:29.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:29.670+0000 7f8ad6a00700 1 -- 172.21.15.107:0/3262429867 learned_addr learned my addr 172.21.15.107:0/3262429867 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:56:29.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:29.671+0000 7f8ad7400700 1 --2- 172.21.15.107:0/3262429867 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ad8103dc0 0x7f8ad811a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:56:29.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:29.671+0000 7f8ad6a00700 1 -- 172.21.15.107:0/3262429867 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ad8103dc0 msgr2=0x7f8ad811a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:29.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:29.671+0000 7f8ad6a00700 1 --2- 172.21.15.107:0/3262429867 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ad8103dc0 0x7f8ad811a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:29.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:29.671+0000 7f8ad6a00700 1 -- 172.21.15.107:0/3262429867 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ac4005040 con 0x7f8ad81095f0 2024-03-04T13:56:29.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:29.671+0000 7f8ad6a00700 1 --2- 172.21.15.107:0/3262429867 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8ad81095f0 0x7f8ad811a580 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f8ac4009100 tx=0x7f8ac4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:56:29.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:29.671+0000 7f8acf400700 1 -- 172.21.15.107:0/3262429867 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8ac400e040 con 0x7f8ad81095f0 2024-03-04T13:56:29.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:29.671+0000 7f8acf400700 1 -- 172.21.15.107:0/3262429867 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8ac4004290 con 0x7f8ad81095f0 2024-03-04T13:56:29.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:29.672+0000 7f8ade000700 1 -- 172.21.15.107:0/3262429867 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ad8114160 con 0x7f8ad81095f0 2024-03-04T13:56:29.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:29.672+0000 7f8acf400700 1 -- 172.21.15.107:0/3262429867 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8ac401c630 con 0x7f8ad81095f0 2024-03-04T13:56:29.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:29.672+0000 7f8ade000700 1 -- 172.21.15.107:0/3262429867 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ad81146a0 con 0x7f8ad81095f0 2024-03-04T13:56:29.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:29.673+0000 7f8ade000700 1 -- 172.21.15.107:0/3262429867 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ad810b9c0 con 0x7f8ad81095f0 2024-03-04T13:56:29.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:29.674+0000 7f8acf400700 1 -- 172.21.15.107:0/3262429867 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f8ac400bbf0 con 0x7f8ad81095f0 2024-03-04T13:56:29.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:29.675+0000 7f8acf400700 1 --2- 172.21.15.107:0/3262429867 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f8abc06d7e0 0x7f8abc06fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:29.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:29.675+0000 7f8acf400700 1 -- 172.21.15.107:0/3262429867 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f8ac408e250 con 0x7f8ad81095f0 2024-03-04T13:56:29.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:29.675+0000 7f8ad7400700 1 --2- 172.21.15.107:0/3262429867 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f8abc06d7e0 0x7f8abc06fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:56:29.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:29.676+0000 7f8ad7400700 1 --2- 172.21.15.107:0/3262429867 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f8abc06d7e0 0x7f8abc06fc90 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8ac80041d0 tx=0x7f8ac8009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:56:29.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:29.680+0000 7f8acf400700 1 -- 172.21.15.107:0/3262429867 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8ac405b600 con 0x7f8ad81095f0 2024-03-04T13:56:29.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:29.945+0000 7f8ade000700 1 -- 172.21.15.107:0/3262429867 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f8ad804bed0 con 0x7f8ad81095f0 2024-03-04T13:56:29.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:29.945+0000 7f8acf400700 1 -- 172.21.15.107:0/3262429867 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 36 v36) v1 ==== 76+0+1903 (secure 0 0 0) 0x7f8ac405b190 con 0x7f8ad81095f0 2024-03-04T13:56:29.948 INFO:teuthology.orchestra.run.smithi107.stdout:e36 2024-03-04T13:56:29.949 INFO:teuthology.orchestra.run.smithi107.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-04T13:56:29.949 INFO:teuthology.orchestra.run.smithi107.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-04T13:56:29.949 INFO:teuthology.orchestra.run.smithi107.stdout:legacy client fscid: 1 2024-03-04T13:56:29.949 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:56:29.949 INFO:teuthology.orchestra.run.smithi107.stdout:Filesystem 'cephfs' (1) 2024-03-04T13:56:29.949 INFO:teuthology.orchestra.run.smithi107.stdout:fs_name cephfs 2024-03-04T13:56:29.949 INFO:teuthology.orchestra.run.smithi107.stdout:epoch 36 2024-03-04T13:56:29.949 INFO:teuthology.orchestra.run.smithi107.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-04T13:56:29.949 INFO:teuthology.orchestra.run.smithi107.stdout:created 2024-03-04T13:38:30.854004+0000 2024-03-04T13:56:29.949 INFO:teuthology.orchestra.run.smithi107.stdout:modified 2024-03-04T13:54:22.860511+0000 2024-03-04T13:56:29.949 INFO:teuthology.orchestra.run.smithi107.stdout:tableserver 0 2024-03-04T13:56:29.949 INFO:teuthology.orchestra.run.smithi107.stdout:root 0 2024-03-04T13:56:29.949 INFO:teuthology.orchestra.run.smithi107.stdout:session_timeout 60 2024-03-04T13:56:29.950 INFO:teuthology.orchestra.run.smithi107.stdout:session_autoclose 300 2024-03-04T13:56:29.950 INFO:teuthology.orchestra.run.smithi107.stdout:max_file_size 1099511627776 2024-03-04T13:56:29.950 INFO:teuthology.orchestra.run.smithi107.stdout:max_xattr_size 65536 2024-03-04T13:56:29.950 INFO:teuthology.orchestra.run.smithi107.stdout:required_client_features {} 2024-03-04T13:56:29.950 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure 0 2024-03-04T13:56:29.950 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure_osd_epoch 86 2024-03-04T13:56:29.950 INFO:teuthology.orchestra.run.smithi107.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-04T13:56:29.950 INFO:teuthology.orchestra.run.smithi107.stdout:max_mds 1 2024-03-04T13:56:29.950 INFO:teuthology.orchestra.run.smithi107.stdout:in 0 2024-03-04T13:56:29.950 INFO:teuthology.orchestra.run.smithi107.stdout:up {0=34432} 2024-03-04T13:56:29.950 INFO:teuthology.orchestra.run.smithi107.stdout:failed 2024-03-04T13:56:29.950 INFO:teuthology.orchestra.run.smithi107.stdout:damaged 2024-03-04T13:56:29.950 INFO:teuthology.orchestra.run.smithi107.stdout:stopped 2024-03-04T13:56:29.950 INFO:teuthology.orchestra.run.smithi107.stdout:data_pools [3] 2024-03-04T13:56:29.950 INFO:teuthology.orchestra.run.smithi107.stdout:metadata_pool 2 2024-03-04T13:56:29.951 INFO:teuthology.orchestra.run.smithi107.stdout:inline_data enabled 2024-03-04T13:56:29.951 INFO:teuthology.orchestra.run.smithi107.stdout:balancer 2024-03-04T13:56:29.951 INFO:teuthology.orchestra.run.smithi107.stdout:bal_rank_mask -1 2024-03-04T13:56:29.951 INFO:teuthology.orchestra.run.smithi107.stdout:standby_count_wanted 1 2024-03-04T13:56:29.951 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.riexmm{0:34432} state up:active seq 24 join_fscid=1 addr [v2:172.21.15.107:6826/1144672731,v1:172.21.15.107:6827/1144672731] compat {c=[1],r=[1],i=[fff]}] 2024-03-04T13:56:29.951 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.thycho{0:34436} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.107:6828/3010096376,v1:172.21.15.107:6829/3010096376] compat {c=[1],r=[1],i=[fff]}] 2024-03-04T13:56:29.951 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:56:29.951 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:56:29.951 INFO:teuthology.orchestra.run.smithi107.stdout:Standby daemons: 2024-03-04T13:56:29.951 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:56:29.951 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi169.jusudl{-1:44243} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.169:6824/1336978861,v1:172.21.15.169:6825/1336978861] compat {c=[1],r=[1],i=[fff]}] 2024-03-04T13:56:29.951 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi169.bxitzk{-1:44259} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.169:6826/4015044684,v1:172.21.15.169:6827/4015044684] compat {c=[1],r=[1],i=[fff]}] 2024-03-04T13:56:29.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:29.948+0000 7f8ade000700 1 -- 172.21.15.107:0/3262429867 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f8abc06d7e0 msgr2=0x7f8abc06fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:29.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:29.948+0000 7f8ade000700 1 --2- 172.21.15.107:0/3262429867 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f8abc06d7e0 0x7f8abc06fc90 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8ac80041d0 tx=0x7f8ac8009040 comp rx=0 tx=0).stop 2024-03-04T13:56:29.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:29.948+0000 7f8ade000700 1 -- 172.21.15.107:0/3262429867 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8ad81095f0 msgr2=0x7f8ad811a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:29.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:29.948+0000 7f8ade000700 1 --2- 172.21.15.107:0/3262429867 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8ad81095f0 0x7f8ad811a580 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f8ac4009100 tx=0x7f8ac4009e30 comp rx=0 tx=0).stop 2024-03-04T13:56:29.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:29.949+0000 7f8ade000700 1 -- 172.21.15.107:0/3262429867 shutdown_connections 2024-03-04T13:56:29.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:29.949+0000 7f8ade000700 1 --2- 172.21.15.107:0/3262429867 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f8abc06d7e0 0x7f8abc06fc90 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:29.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:29.949+0000 7f8ade000700 1 --2- 172.21.15.107:0/3262429867 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8ad8103dc0 0x7f8ad811a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:29.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:29.949+0000 7f8ade000700 1 --2- 172.21.15.107:0/3262429867 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8ad81095f0 0x7f8ad811a580 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:29.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:29.949+0000 7f8ade000700 1 -- 172.21.15.107:0/3262429867 >> 172.21.15.107:0/3262429867 conn(0x7f8ad8072d50 msgr2=0x7f8ad8074850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:56:29.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:29.949+0000 7f8ade000700 1 -- 172.21.15.107:0/3262429867 shutdown_connections 2024-03-04T13:56:29.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:29.949+0000 7f8ade000700 1 -- 172.21.15.107:0/3262429867 wait complete. 2024-03-04T13:56:29.953 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 36 2024-03-04T13:56:30.595 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:56:30 smithi107 ceph-mon[112578]: from='client.? 172.21.15.107:0/3262429867' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-04T13:56:30.612 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:56:30 smithi169 ceph-mon[86183]: from='client.? 172.21.15.107:0/3262429867' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-04T13:56:30.798 INFO:teuthology.run_tasks:Running task fs.post_upgrade_checks... 2024-03-04T13:56:30.809 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-04T13:56:31.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.018+0000 7fbe32800640 1 -- 172.21.15.107:0/441071564 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe2c0719d0 msgr2=0x7fbe2c071da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:31.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.018+0000 7fbe32800640 1 --2- 172.21.15.107:0/441071564 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe2c0719d0 0x7fbe2c071da0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c0048d0 tx=0x7fbe1c009da0 comp rx=0 tx=0).stop 2024-03-04T13:56:31.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.018+0000 7fbe32800640 1 -- 172.21.15.107:0/441071564 shutdown_connections 2024-03-04T13:56:31.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.018+0000 7fbe32800640 1 --2- 172.21.15.107:0/441071564 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe2c0722e0 0x7fbe2c10cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:31.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.019+0000 7fbe32800640 1 --2- 172.21.15.107:0/441071564 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe2c0719d0 0x7fbe2c071da0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:31.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.019+0000 7fbe32800640 1 -- 172.21.15.107:0/441071564 >> 172.21.15.107:0/441071564 conn(0x7fbe2c0fa090 msgr2=0x7fbe2c0fc4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:56:31.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.019+0000 7fbe32800640 1 -- 172.21.15.107:0/441071564 shutdown_connections 2024-03-04T13:56:31.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.019+0000 7fbe32800640 1 -- 172.21.15.107:0/441071564 wait complete. 2024-03-04T13:56:31.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.020+0000 7fbe32800640 1 Processor -- start 2024-03-04T13:56:31.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.020+0000 7fbe32800640 1 -- start start 2024-03-04T13:56:31.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.020+0000 7fbe32800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe2c0719d0 0x7fbe2c1190a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:31.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.020+0000 7fbe32800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe2c0722e0 0x7fbe2c1195e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:31.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.020+0000 7fbe32800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe2c119be0 con 0x7fbe2c0719d0 2024-03-04T13:56:31.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.020+0000 7fbe32800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe2c113170 con 0x7fbe2c0722e0 2024-03-04T13:56:31.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.020+0000 7fbe2b400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe2c0719d0 0x7fbe2c1190a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:56:31.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.021+0000 7fbe2b400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe2c0719d0 0x7fbe2c1190a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51736/0 (socket says 172.21.15.107:51736) 2024-03-04T13:56:31.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.021+0000 7fbe2b400640 1 -- 172.21.15.107:0/916630467 learned_addr learned my addr 172.21.15.107:0/916630467 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:56:31.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.021+0000 7fbe2b400640 1 -- 172.21.15.107:0/916630467 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe2c0722e0 msgr2=0x7fbe2c1195e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:31.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.021+0000 7fbe2b400640 1 --2- 172.21.15.107:0/916630467 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe2c0722e0 0x7fbe2c1195e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:31.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.021+0000 7fbe2b400640 1 -- 172.21.15.107:0/916630467 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe1c005040 con 0x7fbe2c0719d0 2024-03-04T13:56:31.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.021+0000 7fbe2b400640 1 --2- 172.21.15.107:0/916630467 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe2c0719d0 0x7fbe2c1190a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c02bda0 tx=0x7fbe1c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:56:31.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.022+0000 7fbe23400640 1 -- 172.21.15.107:0/916630467 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe1c0094c0 con 0x7fbe2c0719d0 2024-03-04T13:56:31.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.022+0000 7fbe23400640 1 -- 172.21.15.107:0/916630467 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbe1c009620 con 0x7fbe2c0719d0 2024-03-04T13:56:31.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.022+0000 7fbe23400640 1 -- 172.21.15.107:0/916630467 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe1c0435f0 con 0x7fbe2c0719d0 2024-03-04T13:56:31.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.022+0000 7fbe32800640 1 -- 172.21.15.107:0/916630467 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe2c1133f0 con 0x7fbe2c0719d0 2024-03-04T13:56:31.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.022+0000 7fbe32800640 1 -- 172.21.15.107:0/916630467 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe2c1138b0 con 0x7fbe2c0719d0 2024-03-04T13:56:31.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.023+0000 7fbe32800640 1 -- 172.21.15.107:0/916630467 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe2c064030 con 0x7fbe2c0719d0 2024-03-04T13:56:31.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.024+0000 7fbe23400640 1 -- 172.21.15.107:0/916630467 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fbe1c02f070 con 0x7fbe2c0719d0 2024-03-04T13:56:31.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.025+0000 7fbe23400640 1 --2- 172.21.15.107:0/916630467 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fbe0c06d760 0x7fbe0c06fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:31.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.025+0000 7fbe23400640 1 -- 172.21.15.107:0/916630467 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fbe1c0aef00 con 0x7fbe2c0719d0 2024-03-04T13:56:31.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.026+0000 7fbe23e00640 1 --2- 172.21.15.107:0/916630467 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fbe0c06d760 0x7fbe0c06fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:56:31.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.026+0000 7fbe23e00640 1 --2- 172.21.15.107:0/916630467 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fbe0c06d760 0x7fbe0c06fc10 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c114960 tx=0x7fbe14007c30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:56:31.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.030+0000 7fbe23400640 1 -- 172.21.15.107:0/916630467 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbe1c078700 con 0x7fbe2c0719d0 2024-03-04T13:56:31.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.225+0000 7fbe32800640 1 -- 172.21.15.107:0/916630467 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbe2c1146a0 con 0x7fbe2c0719d0 2024-03-04T13:56:31.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.228+0000 7fbe23400640 1 -- 172.21.15.107:0/916630467 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+16693 (secure 0 0 0) 0x7fbe1c07bdc0 con 0x7fbe2c0719d0 2024-03-04T13:56:31.231 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:56:31.232 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":86,"fsid":"1fc09a58-da2b-11ee-95c6-87774f69a715","created":"2024-03-04T13:29:42.985888+0000","modified":"2024-03-04T13:53:58.927176+0000","last_up_change":"2024-03-04T13:52:27.869095+0000","last_in_change":"2024-03-04T13:36:31.424613+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T13:35:17.915858+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-03-04T13:38:29.362118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-03-04T13:38:30.373574+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c857e387-3d2b-478d-9807-24ea8b9fe02d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":50,"up_from":54,"up_thru":79,"down_at":51,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":3996509230},{"type":"v1","addr":"172.21.15.107:6803","nonce":3996509230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":3996509230},{"type":"v1","addr":"172.21.15.107:6805","nonce":3996509230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":3996509230},{"type":"v1","addr":"172.21.15.107:6809","nonce":3996509230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":3996509230},{"type":"v1","addr":"172.21.15.107:6807","nonce":3996509230}]},"public_addr":"172.21.15.107:6803/3996509230","cluster_addr":"172.21.15.107:6805/3996509230","heartbeat_back_addr":"172.21.15.107:6809/3996509230","heartbeat_front_addr":"172.21.15.107:6807/3996509230","state":["exists","up"]},{"osd":1,"uuid":"fa29dc8f-419b-4318-b43e-e13fffd87204","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":55,"up_from":59,"up_thru":79,"down_at":56,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1164427580},{"type":"v1","addr":"172.21.15.107:6811","nonce":1164427580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1164427580},{"type":"v1","addr":"172.21.15.107:6813","nonce":1164427580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1164427580},{"type":"v1","addr":"172.21.15.107:6817","nonce":1164427580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1164427580},{"type":"v1","addr":"172.21.15.107:6815","nonce":1164427580}]},"public_addr":"172.21.15.107:6811/1164427580","cluster_addr":"172.21.15.107:6813/1164427580","heartbeat_back_addr":"172.21.15.107:6817/1164427580","heartbeat_front_addr":"172.21.15.107:6815/1164427580","state":["exists","up"]},{"osd":2,"uuid":"580a07f0-7bd4-4341-9c1f-b3383d64c720","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":60,"up_from":64,"up_thru":79,"down_at":61,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":150828071},{"type":"v1","addr":"172.21.15.107:6819","nonce":150828071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":150828071},{"type":"v1","addr":"172.21.15.107:6821","nonce":150828071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":150828071},{"type":"v1","addr":"172.21.15.107:6825","nonce":150828071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":150828071},{"type":"v1","addr":"172.21.15.107:6823","nonce":150828071}]},"public_addr":"172.21.15.107:6819/150828071","cluster_addr":"172.21.15.107:6821/150828071","heartbeat_back_addr":"172.21.15.107:6825/150828071","heartbeat_front_addr":"172.21.15.107:6823/150828071","state":["exists","up"]},{"osd":3,"uuid":"7acac14d-998a-4651-b78c-e42584e13b43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":27,"last_clean_end":65,"up_from":69,"up_thru":79,"down_at":66,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3021051994},{"type":"v1","addr":"172.21.15.169:6801","nonce":3021051994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3021051994},{"type":"v1","addr":"172.21.15.169:6803","nonce":3021051994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3021051994},{"type":"v1","addr":"172.21.15.169:6807","nonce":3021051994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3021051994},{"type":"v1","addr":"172.21.15.169:6805","nonce":3021051994}]},"public_addr":"172.21.15.169:6801/3021051994","cluster_addr":"172.21.15.169:6803/3021051994","heartbeat_back_addr":"172.21.15.169:6807/3021051994","heartbeat_front_addr":"172.21.15.169:6805/3021051994","state":["exists","up"]},{"osd":4,"uuid":"01d2b77c-66ec-4b50-8170-43d95eeae471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":35,"last_clean_end":70,"up_from":74,"up_thru":79,"down_at":71,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":56453577},{"type":"v1","addr":"172.21.15.169:6809","nonce":56453577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":56453577},{"type":"v1","addr":"172.21.15.169:6811","nonce":56453577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":56453577},{"type":"v1","addr":"172.21.15.169:6815","nonce":56453577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":56453577},{"type":"v1","addr":"172.21.15.169:6813","nonce":56453577}]},"public_addr":"172.21.15.169:6809/56453577","cluster_addr":"172.21.15.169:6811/56453577","heartbeat_back_addr":"172.21.15.169:6815/56453577","heartbeat_front_addr":"172.21.15.169:6813/56453577","state":["exists","up"]},{"osd":5,"uuid":"0164ead7-c5d0-4541-bf22-f671558f8607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":40,"last_clean_end":75,"up_from":79,"up_thru":79,"down_at":76,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":374254288},{"type":"v1","addr":"172.21.15.169:6817","nonce":374254288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":374254288},{"type":"v1","addr":"172.21.15.169:6819","nonce":374254288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":374254288},{"type":"v1","addr":"172.21.15.169:6823","nonce":374254288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":374254288},{"type":"v1","addr":"172.21.15.169:6821","nonce":374254288}]},"public_addr":"172.21.15.169:6817/374254288","cluster_addr":"172.21.15.169:6819/374254288","heartbeat_back_addr":"172.21.15.169:6823/374254288","heartbeat_front_addr":"172.21.15.169:6821/374254288","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T13:33:59.729659+0000","dead_epoch":50},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T13:34:36.600326+0000","dead_epoch":55},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T13:35:14.068513+0000","dead_epoch":60},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T13:35:46.608007+0000","dead_epoch":65},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T13:36:19.595680+0000","dead_epoch":70},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T13:36:52.586933+0000","dead_epoch":75}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.169:6827/3696090493":"2024-03-05T13:53:58.927120+0000","172.21.15.169:6825/2994866076":"2024-03-05T13:53:29.273990+0000","172.21.15.107:0/395365799":"2024-03-05T13:41:21.904328+0000","172.21.15.107:0/2217624526":"2024-03-05T13:41:21.904328+0000","172.21.15.107:6801/2371648646":"2024-03-05T13:30:56.353578+0000","172.21.15.107:6800/3848495603":"2024-03-05T13:41:21.904328+0000","172.21.15.107:0/3451280196":"2024-03-05T13:30:16.230256+0000","172.21.15.107:6827/3814968386":"2024-03-05T13:38:46.993392+0000","172.21.15.107:0/3881311685":"2024-03-05T13:47:24.592361+0000","172.21.15.107:0/2723084639":"2024-03-05T13:32:27.229455+0000","172.21.15.107:6801/1723390229":"2024-03-05T13:30:16.230256+0000","172.21.15.107:0/248094819":"2024-03-05T13:47:24.592361+0000","172.21.15.107:6800/1723390229":"2024-03-05T13:30:16.230256+0000","172.21.15.169:6828/3524931243":"2024-03-05T13:43:38.105893+0000","172.21.15.107:0/601795289":"2024-03-05T13:32:27.229455+0000","172.21.15.169:0/1961312808":"2024-03-05T13:43:38.105893+0000","172.21.15.107:6801/2040330534":"2024-03-05T13:32:27.229455+0000","172.21.15.107:0/136342223":"2024-03-05T13:30:56.353578+0000","172.21.15.169:0/692246057":"2024-03-05T13:43:38.105893+0000","172.21.15.107:0/1780539770":"2024-03-05T13:30:16.230256+0000","172.21.15.107:0/3355652811":"2024-03-05T13:41:21.904328+0000","172.21.15.169:6829/3524931243":"2024-03-05T13:43:38.105893+0000","172.21.15.107:6800/2040330534":"2024-03-05T13:32:27.229455+0000","172.21.15.107:6801/1613321046":"2024-03-05T13:47:24.592361+0000","172.21.15.107:0/3503768751":"2024-03-05T13:32:27.229455+0000","172.21.15.169:0/2785350328":"2024-03-05T13:43:38.105893+0000","172.21.15.107:0/2928681569":"2024-03-05T13:30:56.353578+0000","172.21.15.107:0/2314831039":"2024-03-05T13:30:56.353578+0000","172.21.15.169:0/842094828":"2024-03-05T13:43:38.105893+0000","172.21.15.169:6824/2994866076":"2024-03-05T13:53:29.273990+0000","172.21.15.169:6826/3696090493":"2024-03-05T13:53:58.927120+0000","172.21.15.107:6801/3848495603":"2024-03-05T13:41:21.904328+0000","172.21.15.107:6826/3814968386":"2024-03-05T13:38:46.993392+0000","172.21.15.107:0/2605062903":"2024-03-05T13:41:21.904328+0000","172.21.15.107:6800/2371648646":"2024-03-05T13:30:56.353578+0000","172.21.15.107:6800/1613321046":"2024-03-05T13:47:24.592361+0000","172.21.15.107:0/2357638826":"2024-03-05T13:30:16.230256+0000","172.21.15.107:0/1849554796":"2024-03-05T13:47:24.592361+0000","172.21.15.107:0/2439430884":"2024-03-05T13:47:24.592361+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T13:56:31.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.231+0000 7fbe32800640 1 -- 172.21.15.107:0/916630467 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fbe0c06d760 msgr2=0x7fbe0c06fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:31.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.231+0000 7fbe32800640 1 --2- 172.21.15.107:0/916630467 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fbe0c06d760 0x7fbe0c06fc10 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c114960 tx=0x7fbe14007c30 comp rx=0 tx=0).stop 2024-03-04T13:56:31.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.231+0000 7fbe32800640 1 -- 172.21.15.107:0/916630467 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe2c0719d0 msgr2=0x7fbe2c1190a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:31.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.231+0000 7fbe32800640 1 --2- 172.21.15.107:0/916630467 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe2c0719d0 0x7fbe2c1190a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c02bda0 tx=0x7fbe1c009da0 comp rx=0 tx=0).stop 2024-03-04T13:56:31.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.231+0000 7fbe32800640 1 -- 172.21.15.107:0/916630467 shutdown_connections 2024-03-04T13:56:31.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.231+0000 7fbe32800640 1 --2- 172.21.15.107:0/916630467 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fbe0c06d760 0x7fbe0c06fc10 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:31.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.231+0000 7fbe32800640 1 --2- 172.21.15.107:0/916630467 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbe2c0722e0 0x7fbe2c1195e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:31.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.231+0000 7fbe32800640 1 --2- 172.21.15.107:0/916630467 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe2c0719d0 0x7fbe2c1190a0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:31.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.231+0000 7fbe32800640 1 -- 172.21.15.107:0/916630467 >> 172.21.15.107:0/916630467 conn(0x7fbe2c0fa090 msgr2=0x7fbe2c0facf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:56:31.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.231+0000 7fbe32800640 1 -- 172.21.15.107:0/916630467 shutdown_connections 2024-03-04T13:56:31.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.231+0000 7fbe32800640 1 -- 172.21.15.107:0/916630467 wait complete. 2024-03-04T13:56:31.252 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-04T13:35:17.915858+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-03-04T13:38:29.362118+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-03-04T13:38:30.373574+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-04T13:56:31.252 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-04T13:56:31.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.447+0000 7f9049c00640 1 -- 172.21.15.107:0/3127267481 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9044102d00 msgr2=0x7f90440fc390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:31.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.447+0000 7f9049c00640 1 --2- 172.21.15.107:0/3127267481 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9044102d00 0x7f90440fc390 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f902c0048d0 tx=0x7f902c009da0 comp rx=0 tx=0).stop 2024-03-04T13:56:31.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.447+0000 7f9049c00640 1 -- 172.21.15.107:0/3127267481 shutdown_connections 2024-03-04T13:56:31.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.447+0000 7f9049c00640 1 --2- 172.21.15.107:0/3127267481 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9044102d00 0x7f90440fc390 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:31.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.447+0000 7f9049c00640 1 --2- 172.21.15.107:0/3127267481 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9044102360 0x7f9044102730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:31.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.447+0000 7f9049c00640 1 -- 172.21.15.107:0/3127267481 >> 172.21.15.107:0/3127267481 conn(0x7f90440f7ef0 msgr2=0x7f90440fa300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:56:31.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.447+0000 7f9049c00640 1 -- 172.21.15.107:0/3127267481 shutdown_connections 2024-03-04T13:56:31.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.447+0000 7f9049c00640 1 -- 172.21.15.107:0/3127267481 wait complete. 2024-03-04T13:56:31.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.448+0000 7f9049c00640 1 Processor -- start 2024-03-04T13:56:31.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.448+0000 7f9049c00640 1 -- start start 2024-03-04T13:56:31.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.448+0000 7f9049c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9044102360 0x7f9044114de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:31.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.448+0000 7f9049c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9044102d00 0x7f9044115320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:31.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.448+0000 7f9049c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9044115920 con 0x7f9044102d00 2024-03-04T13:56:31.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.448+0000 7f9049c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f904410eeb0 con 0x7f9044102360 2024-03-04T13:56:31.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.448+0000 7f903a000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9044102d00 0x7f9044115320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:56:31.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.448+0000 7f903a000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9044102d00 0x7f9044115320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51754/0 (socket says 172.21.15.107:51754) 2024-03-04T13:56:31.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.448+0000 7f903a000640 1 -- 172.21.15.107:0/605198898 learned_addr learned my addr 172.21.15.107:0/605198898 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:56:31.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.449+0000 7f9042a00640 1 --2- 172.21.15.107:0/605198898 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9044102360 0x7f9044114de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:56:31.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.449+0000 7f903a000640 1 -- 172.21.15.107:0/605198898 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9044102360 msgr2=0x7f9044114de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:31.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.449+0000 7f903a000640 1 --2- 172.21.15.107:0/605198898 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9044102360 0x7f9044114de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:31.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.449+0000 7f903a000640 1 -- 172.21.15.107:0/605198898 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f902c005040 con 0x7f9044102d00 2024-03-04T13:56:31.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.449+0000 7f903a000640 1 --2- 172.21.15.107:0/605198898 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9044102d00 0x7f9044115320 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f902c02bda0 tx=0x7f902c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:56:31.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.449+0000 7f903b400640 1 -- 172.21.15.107:0/605198898 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f902c0094c0 con 0x7f9044102d00 2024-03-04T13:56:31.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.450+0000 7f9049c00640 1 -- 172.21.15.107:0/605198898 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f904410f130 con 0x7f9044102d00 2024-03-04T13:56:31.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.450+0000 7f903b400640 1 -- 172.21.15.107:0/605198898 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f902c009620 con 0x7f9044102d00 2024-03-04T13:56:31.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.450+0000 7f903b400640 1 -- 172.21.15.107:0/605198898 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f902c0435f0 con 0x7f9044102d00 2024-03-04T13:56:31.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.450+0000 7f9049c00640 1 -- 172.21.15.107:0/605198898 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f904410f5f0 con 0x7f9044102d00 2024-03-04T13:56:31.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.451+0000 7f9049c00640 1 -- 172.21.15.107:0/605198898 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9044064480 con 0x7f9044102d00 2024-03-04T13:56:31.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.452+0000 7f903b400640 1 -- 172.21.15.107:0/605198898 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f902c02f070 con 0x7f9044102d00 2024-03-04T13:56:31.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.452+0000 7f903b400640 1 --2- 172.21.15.107:0/605198898 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f902806d760 0x7f902806fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:31.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.453+0000 7f903b400640 1 -- 172.21.15.107:0/605198898 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f902c0aef60 con 0x7f9044102d00 2024-03-04T13:56:31.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.453+0000 7f9042a00640 1 --2- 172.21.15.107:0/605198898 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f902806d760 0x7f902806fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:56:31.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.454+0000 7f9042a00640 1 --2- 172.21.15.107:0/605198898 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f902806d760 0x7f902806fc10 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9034004550 tx=0x7f9034009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:56:31.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.458+0000 7f903b400640 1 -- 172.21.15.107:0/605198898 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f902c0e2d20 con 0x7f9044102d00 2024-03-04T13:56:31.595 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:56:31 smithi107 ceph-mon[112578]: pgmap v312: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-03-04T13:56:31.595 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:56:31 smithi107 ceph-mon[112578]: from='client.? 172.21.15.107:0/916630467' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T13:56:31.612 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:56:31 smithi169 ceph-mon[86183]: pgmap v312: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-03-04T13:56:31.612 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:56:31 smithi169 ceph-mon[86183]: from='client.? 172.21.15.107:0/916630467' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T13:56:31.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.654+0000 7f9049c00640 1 -- 172.21.15.107:0/605198898 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f904410f8d0 con 0x7f9044102d00 2024-03-04T13:56:31.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.655+0000 7f903b400640 1 -- 172.21.15.107:0/605198898 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) v1 ==== 93+0+10 (secure 0 0 0) 0x7f902c078760 con 0x7f9044102d00 2024-03-04T13:56:31.658 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 1 2024-03-04T13:56:31.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.657+0000 7f9049c00640 1 -- 172.21.15.107:0/605198898 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f902806d760 msgr2=0x7f902806fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:31.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.657+0000 7f9049c00640 1 --2- 172.21.15.107:0/605198898 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f902806d760 0x7f902806fc10 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9034004550 tx=0x7f9034009040 comp rx=0 tx=0).stop 2024-03-04T13:56:31.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.657+0000 7f9049c00640 1 -- 172.21.15.107:0/605198898 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9044102d00 msgr2=0x7f9044115320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:31.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.657+0000 7f9049c00640 1 --2- 172.21.15.107:0/605198898 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9044102d00 0x7f9044115320 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f902c02bda0 tx=0x7f902c009da0 comp rx=0 tx=0).stop 2024-03-04T13:56:31.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.658+0000 7f9049c00640 1 -- 172.21.15.107:0/605198898 shutdown_connections 2024-03-04T13:56:31.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.658+0000 7f9049c00640 1 --2- 172.21.15.107:0/605198898 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f902806d760 0x7f902806fc10 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:31.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.658+0000 7f9049c00640 1 --2- 172.21.15.107:0/605198898 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9044102d00 0x7f9044115320 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:31.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.658+0000 7f9049c00640 1 --2- 172.21.15.107:0/605198898 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9044102360 0x7f9044114de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:31.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.658+0000 7f9049c00640 1 -- 172.21.15.107:0/605198898 >> 172.21.15.107:0/605198898 conn(0x7f90440f7ef0 msgr2=0x7f90440f9820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:56:31.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.658+0000 7f9049c00640 1 -- 172.21.15.107:0/605198898 shutdown_connections 2024-03-04T13:56:31.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.658+0000 7f9049c00640 1 -- 172.21.15.107:0/605198898 wait complete. 2024-03-04T13:56:31.678 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-03-04T13:56:31.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.881+0000 7fb44ca00640 1 -- 172.21.15.107:0/777944558 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb448073e80 msgr2=0x7fb44810cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:31.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.881+0000 7fb44ca00640 1 --2- 172.21.15.107:0/777944558 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb448073e80 0x7fb44810cd40 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb4300048d0 tx=0x7fb430009da0 comp rx=0 tx=0).stop 2024-03-04T13:56:31.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.881+0000 7fb44ca00640 1 -- 172.21.15.107:0/777944558 shutdown_connections 2024-03-04T13:56:31.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.881+0000 7fb44ca00640 1 --2- 172.21.15.107:0/777944558 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb448073e80 0x7fb44810cd40 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:31.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.881+0000 7fb44ca00640 1 --2- 172.21.15.107:0/777944558 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb448073570 0x7fb448073940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:31.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.881+0000 7fb44ca00640 1 -- 172.21.15.107:0/777944558 >> 172.21.15.107:0/777944558 conn(0x7fb4480fa160 msgr2=0x7fb4480fc570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:56:31.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.882+0000 7fb44ca00640 1 -- 172.21.15.107:0/777944558 shutdown_connections 2024-03-04T13:56:31.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.882+0000 7fb44ca00640 1 -- 172.21.15.107:0/777944558 wait complete. 2024-03-04T13:56:31.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.882+0000 7fb44ca00640 1 Processor -- start 2024-03-04T13:56:31.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.882+0000 7fb44ca00640 1 -- start start 2024-03-04T13:56:31.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.883+0000 7fb44ca00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb448073570 0x7fb448119290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:31.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.883+0000 7fb44ca00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb448073e80 0x7fb4481197d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:31.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.883+0000 7fb44ca00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb448119dd0 con 0x7fb448073e80 2024-03-04T13:56:31.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.883+0000 7fb44ca00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb448113360 con 0x7fb448073570 2024-03-04T13:56:31.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.883+0000 7fb43fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb448073e80 0x7fb4481197d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:56:31.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.883+0000 7fb43fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb448073e80 0x7fb4481197d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51784/0 (socket says 172.21.15.107:51784) 2024-03-04T13:56:31.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.883+0000 7fb43fe00640 1 -- 172.21.15.107:0/3542848224 learned_addr learned my addr 172.21.15.107:0/3542848224 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:56:31.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.883+0000 7fb445e00640 1 --2- 172.21.15.107:0/3542848224 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb448073570 0x7fb448119290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:56:31.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.883+0000 7fb43fe00640 1 -- 172.21.15.107:0/3542848224 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb448073570 msgr2=0x7fb448119290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:31.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.883+0000 7fb43fe00640 1 --2- 172.21.15.107:0/3542848224 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb448073570 0x7fb448119290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:31.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.884+0000 7fb43fe00640 1 -- 172.21.15.107:0/3542848224 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb430005040 con 0x7fb448073e80 2024-03-04T13:56:31.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.884+0000 7fb43fe00640 1 --2- 172.21.15.107:0/3542848224 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb448073e80 0x7fb4481197d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb4300047f0 tx=0x7fb430009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:56:31.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.884+0000 7fb43e000640 1 -- 172.21.15.107:0/3542848224 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb4300094c0 con 0x7fb448073e80 2024-03-04T13:56:31.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.884+0000 7fb43e000640 1 -- 172.21.15.107:0/3542848224 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb430009620 con 0x7fb448073e80 2024-03-04T13:56:31.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.884+0000 7fb44ca00640 1 -- 172.21.15.107:0/3542848224 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4481135e0 con 0x7fb448073e80 2024-03-04T13:56:31.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.884+0000 7fb43e000640 1 -- 172.21.15.107:0/3542848224 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb4300435f0 con 0x7fb448073e80 2024-03-04T13:56:31.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.884+0000 7fb44ca00640 1 -- 172.21.15.107:0/3542848224 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb448113aa0 con 0x7fb448073e80 2024-03-04T13:56:31.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.885+0000 7fb44ca00640 1 -- 172.21.15.107:0/3542848224 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb448064030 con 0x7fb448073e80 2024-03-04T13:56:31.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.886+0000 7fb43e000640 1 -- 172.21.15.107:0/3542848224 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fb43002f070 con 0x7fb448073e80 2024-03-04T13:56:31.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.887+0000 7fb43e000640 1 --2- 172.21.15.107:0/3542848224 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fb42c06d760 0x7fb42c06fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:31.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.887+0000 7fb43e000640 1 -- 172.21.15.107:0/3542848224 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fb4300aef00 con 0x7fb448073e80 2024-03-04T13:56:31.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.888+0000 7fb445e00640 1 --2- 172.21.15.107:0/3542848224 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fb42c06d760 0x7fb42c06fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:56:31.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.888+0000 7fb445e00640 1 --2- 172.21.15.107:0/3542848224 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fb42c06d760 0x7fb42c06fc10 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb438004ce0 tx=0x7fb43800b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:56:31.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:31.892+0000 7fb43e000640 1 -- 172.21.15.107:0/3542848224 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb430078680 con 0x7fb448073e80 2024-03-04T13:56:32.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:32.087+0000 7fb44ca00640 1 -- 172.21.15.107:0/3542848224 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7fb448114890 con 0x7fb448073e80 2024-03-04T13:56:32.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:32.087+0000 7fb43e000640 1 -- 172.21.15.107:0/3542848224 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v86) v1 ==== 107+0+11 (secure 0 0 0) 0x7fb43007bd40 con 0x7fb448073e80 2024-03-04T13:56:32.090 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 32 2024-03-04T13:56:32.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:32.090+0000 7fb44ca00640 1 -- 172.21.15.107:0/3542848224 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fb42c06d760 msgr2=0x7fb42c06fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:32.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:32.090+0000 7fb44ca00640 1 --2- 172.21.15.107:0/3542848224 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fb42c06d760 0x7fb42c06fc10 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb438004ce0 tx=0x7fb43800b040 comp rx=0 tx=0).stop 2024-03-04T13:56:32.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:32.090+0000 7fb44ca00640 1 -- 172.21.15.107:0/3542848224 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb448073e80 msgr2=0x7fb4481197d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:32.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:32.090+0000 7fb44ca00640 1 --2- 172.21.15.107:0/3542848224 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb448073e80 0x7fb4481197d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb4300047f0 tx=0x7fb430009da0 comp rx=0 tx=0).stop 2024-03-04T13:56:32.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:32.090+0000 7fb44ca00640 1 -- 172.21.15.107:0/3542848224 shutdown_connections 2024-03-04T13:56:32.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:32.090+0000 7fb44ca00640 1 --2- 172.21.15.107:0/3542848224 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fb42c06d760 0x7fb42c06fc10 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:32.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:32.090+0000 7fb44ca00640 1 --2- 172.21.15.107:0/3542848224 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb448073e80 0x7fb4481197d0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:32.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:32.090+0000 7fb44ca00640 1 --2- 172.21.15.107:0/3542848224 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb448073570 0x7fb448119290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:32.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:32.090+0000 7fb44ca00640 1 -- 172.21.15.107:0/3542848224 >> 172.21.15.107:0/3542848224 conn(0x7fb4480fa160 msgr2=0x7fb4480fbe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:56:32.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:32.090+0000 7fb44ca00640 1 -- 172.21.15.107:0/3542848224 shutdown_connections 2024-03-04T13:56:32.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:32.090+0000 7fb44ca00640 1 -- 172.21.15.107:0/3542848224 wait complete. 2024-03-04T13:56:32.109 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-03-04T13:56:32.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:32.308+0000 7f20e1000640 1 -- 172.21.15.107:0/1505715810 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20dc101270 msgr2=0x7f20dc101640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:32.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:32.309+0000 7f20e1000640 1 --2- 172.21.15.107:0/1505715810 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20dc101270 0x7f20dc101640 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f20cc0048d0 tx=0x7f20cc009da0 comp rx=0 tx=0).stop 2024-03-04T13:56:32.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:32.309+0000 7f20e1000640 1 -- 172.21.15.107:0/1505715810 shutdown_connections 2024-03-04T13:56:32.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:32.309+0000 7f20e1000640 1 --2- 172.21.15.107:0/1505715810 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20dc101c10 0x7f20dc0fb2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:32.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:32.309+0000 7f20e1000640 1 --2- 172.21.15.107:0/1505715810 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20dc101270 0x7f20dc101640 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:32.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:32.309+0000 7f20e1000640 1 -- 172.21.15.107:0/1505715810 >> 172.21.15.107:0/1505715810 conn(0x7f20dc0f6de0 msgr2=0x7f20dc0f91f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:56:32.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:32.309+0000 7f20e1000640 1 -- 172.21.15.107:0/1505715810 shutdown_connections 2024-03-04T13:56:32.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:32.309+0000 7f20e1000640 1 -- 172.21.15.107:0/1505715810 wait complete. 2024-03-04T13:56:32.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:32.309+0000 7f20e1000640 1 Processor -- start 2024-03-04T13:56:32.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:32.310+0000 7f20e1000640 1 -- start start 2024-03-04T13:56:32.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:32.310+0000 7f20e1000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20dc101270 0x7f20dc114de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:32.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:32.310+0000 7f20e1000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20dc101c10 0x7f20dc115320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:32.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:32.310+0000 7f20e1000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20dc115920 con 0x7f20dc101270 2024-03-04T13:56:32.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:32.310+0000 7f20e1000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20dc115a90 con 0x7f20dc101c10 2024-03-04T13:56:32.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:32.310+0000 7f20d9e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20dc101270 0x7f20dc114de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:56:32.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:32.310+0000 7f20d9e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20dc101270 0x7f20dc114de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51804/0 (socket says 172.21.15.107:51804) 2024-03-04T13:56:32.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:32.310+0000 7f20d9e00640 1 -- 172.21.15.107:0/1627127958 learned_addr learned my addr 172.21.15.107:0/1627127958 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:56:32.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:32.310+0000 7f20d9e00640 1 -- 172.21.15.107:0/1627127958 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20dc101c10 msgr2=0x7f20dc115320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:32.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:32.310+0000 7f20d9e00640 1 --2- 172.21.15.107:0/1627127958 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20dc101c10 0x7f20dc115320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:32.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:32.311+0000 7f20d9e00640 1 -- 172.21.15.107:0/1627127958 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20cc005040 con 0x7f20dc101270 2024-03-04T13:56:32.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:32.311+0000 7f20d9e00640 1 --2- 172.21.15.107:0/1627127958 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20dc101270 0x7f20dc114de0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f20cc02bda0 tx=0x7f20cc009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:56:32.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:32.311+0000 7f20d2000640 1 -- 172.21.15.107:0/1627127958 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f20cc0094c0 con 0x7f20dc101270 2024-03-04T13:56:32.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:32.311+0000 7f20d2000640 1 -- 172.21.15.107:0/1627127958 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f20cc009620 con 0x7f20dc101270 2024-03-04T13:56:32.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:32.311+0000 7f20e1000640 1 -- 172.21.15.107:0/1627127958 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20dc10f000 con 0x7f20dc101270 2024-03-04T13:56:32.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:32.311+0000 7f20d2000640 1 -- 172.21.15.107:0/1627127958 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f20cc0435f0 con 0x7f20dc101270 2024-03-04T13:56:32.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:32.311+0000 7f20e1000640 1 -- 172.21.15.107:0/1627127958 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20dc10f4c0 con 0x7f20dc101270 2024-03-04T13:56:32.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:32.313+0000 7f20e1000640 1 -- 172.21.15.107:0/1627127958 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20dc064480 con 0x7f20dc101270 2024-03-04T13:56:32.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:32.313+0000 7f20d2000640 1 -- 172.21.15.107:0/1627127958 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f20cc02f070 con 0x7f20dc101270 2024-03-04T13:56:32.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:32.314+0000 7f20d2000640 1 --2- 172.21.15.107:0/1627127958 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f20c006d760 0x7f20c006fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:32.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:32.314+0000 7f20d2000640 1 -- 172.21.15.107:0/1627127958 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f20cc0aef80 con 0x7f20dc101270 2024-03-04T13:56:32.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:32.315+0000 7f20d9400640 1 --2- 172.21.15.107:0/1627127958 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f20c006d760 0x7f20c006fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:56:32.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:32.315+0000 7f20d9400640 1 --2- 172.21.15.107:0/1627127958 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f20c006d760 0x7f20c006fc10 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f20dc1106e0 tx=0x7f20c8007c30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:56:32.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:32.319+0000 7f20d2000640 1 -- 172.21.15.107:0/1627127958 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f20cc078700 con 0x7f20dc101270 2024-03-04T13:56:32.518 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:32.515+0000 7f20e1000640 1 -- 172.21.15.107:0/1627127958 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f20dc1083b0 con 0x7f20dc101270 2024-03-04T13:56:32.518 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:32.516+0000 7f20d2000640 1 -- 172.21.15.107:0/1627127958 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v86) v1 ==== 107+0+11 (secure 0 0 0) 0x7f20cc07bdc0 con 0x7f20dc101270 2024-03-04T13:56:32.518 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 32 2024-03-04T13:56:32.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:32.519+0000 7f20e1000640 1 -- 172.21.15.107:0/1627127958 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f20c006d760 msgr2=0x7f20c006fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:32.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:32.519+0000 7f20e1000640 1 --2- 172.21.15.107:0/1627127958 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f20c006d760 0x7f20c006fc10 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f20dc1106e0 tx=0x7f20c8007c30 comp rx=0 tx=0).stop 2024-03-04T13:56:32.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:32.519+0000 7f20e1000640 1 -- 172.21.15.107:0/1627127958 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20dc101270 msgr2=0x7f20dc114de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:32.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:32.519+0000 7f20e1000640 1 --2- 172.21.15.107:0/1627127958 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20dc101270 0x7f20dc114de0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f20cc02bda0 tx=0x7f20cc009da0 comp rx=0 tx=0).stop 2024-03-04T13:56:32.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:32.519+0000 7f20e1000640 1 -- 172.21.15.107:0/1627127958 shutdown_connections 2024-03-04T13:56:32.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:32.519+0000 7f20e1000640 1 --2- 172.21.15.107:0/1627127958 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f20c006d760 0x7f20c006fc10 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:32.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:32.519+0000 7f20e1000640 1 --2- 172.21.15.107:0/1627127958 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20dc101c10 0x7f20dc115320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:32.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:32.519+0000 7f20e1000640 1 --2- 172.21.15.107:0/1627127958 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20dc101270 0x7f20dc114de0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:32.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:32.519+0000 7f20e1000640 1 -- 172.21.15.107:0/1627127958 >> 172.21.15.107:0/1627127958 conn(0x7f20dc0f6de0 msgr2=0x7f20dc0656b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:56:32.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:32.519+0000 7f20e1000640 1 -- 172.21.15.107:0/1627127958 shutdown_connections 2024-03-04T13:56:32.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:32.519+0000 7f20e1000640 1 -- 172.21.15.107:0/1627127958 wait complete. 2024-03-04T13:56:32.536 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:56:32 smithi107 ceph-mon[112578]: from='client.? 172.21.15.107:0/605198898' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-04T13:56:32.537 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:56:32 smithi107 ceph-mon[112578]: from='client.? 172.21.15.107:0/3542848224' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-04T13:56:32.541 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T13:56:32.612 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:56:32 smithi169 ceph-mon[86183]: from='client.? 172.21.15.107:0/605198898' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-04T13:56:32.612 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:56:32 smithi169 ceph-mon[86183]: from='client.? 172.21.15.107:0/3542848224' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-04T13:56:32.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:32.743+0000 7f6b96400640 1 -- 172.21.15.107:0/2489642291 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b900fdb60 msgr2=0x7f6b901061a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:32.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:32.743+0000 7f6b96400640 1 --2- 172.21.15.107:0/2489642291 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b900fdb60 0x7f6b901061a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c0048d0 tx=0x7f6b7c009da0 comp rx=0 tx=0).stop 2024-03-04T13:56:32.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:32.743+0000 7f6b96400640 1 -- 172.21.15.107:0/2489642291 shutdown_connections 2024-03-04T13:56:32.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:32.743+0000 7f6b96400640 1 --2- 172.21.15.107:0/2489642291 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b900fdb60 0x7f6b901061a0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:32.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:32.743+0000 7f6b96400640 1 --2- 172.21.15.107:0/2489642291 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b900fd1c0 0x7f6b900fd590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:32.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:32.743+0000 7f6b96400640 1 -- 172.21.15.107:0/2489642291 >> 172.21.15.107:0/2489642291 conn(0x7f6b900f6c90 msgr2=0x7f6b900f90a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:56:32.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:32.743+0000 7f6b96400640 1 -- 172.21.15.107:0/2489642291 shutdown_connections 2024-03-04T13:56:32.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:32.743+0000 7f6b96400640 1 -- 172.21.15.107:0/2489642291 wait complete. 2024-03-04T13:56:32.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:32.744+0000 7f6b96400640 1 Processor -- start 2024-03-04T13:56:32.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:32.744+0000 7f6b96400640 1 -- start start 2024-03-04T13:56:32.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:32.744+0000 7f6b96400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b900fd1c0 0x7f6b90114cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:32.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:32.744+0000 7f6b96400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b900fdb60 0x7f6b90115210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:32.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:32.744+0000 7f6b96400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b90115810 con 0x7f6b900fd1c0 2024-03-04T13:56:32.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:32.744+0000 7f6b96400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b9010eda0 con 0x7f6b900fdb60 2024-03-04T13:56:32.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:32.744+0000 7f6b95000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b900fd1c0 0x7f6b90114cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:56:32.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:32.744+0000 7f6b95000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b900fd1c0 0x7f6b90114cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51812/0 (socket says 172.21.15.107:51812) 2024-03-04T13:56:32.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:32.744+0000 7f6b95000640 1 -- 172.21.15.107:0/2522476612 learned_addr learned my addr 172.21.15.107:0/2522476612 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:56:32.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:32.745+0000 7f6b95000640 1 -- 172.21.15.107:0/2522476612 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b900fdb60 msgr2=0x7f6b90115210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:32.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:32.745+0000 7f6b95000640 1 --2- 172.21.15.107:0/2522476612 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b900fdb60 0x7f6b90115210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:32.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:32.745+0000 7f6b95000640 1 -- 172.21.15.107:0/2522476612 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b7c005040 con 0x7f6b900fd1c0 2024-03-04T13:56:32.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:32.745+0000 7f6b95000640 1 --2- 172.21.15.107:0/2522476612 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b900fd1c0 0x7f6b90114cd0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6b8400c790 tx=0x7f6b840043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:56:32.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:32.745+0000 7f6b8cc00640 1 -- 172.21.15.107:0/2522476612 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b8400a820 con 0x7f6b900fd1c0 2024-03-04T13:56:32.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:32.745+0000 7f6b8cc00640 1 -- 172.21.15.107:0/2522476612 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6b8400ae50 con 0x7f6b900fd1c0 2024-03-04T13:56:32.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:32.745+0000 7f6b96400640 1 -- 172.21.15.107:0/2522476612 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b9010f080 con 0x7f6b900fd1c0 2024-03-04T13:56:32.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:32.745+0000 7f6b8cc00640 1 -- 172.21.15.107:0/2522476612 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b840115e0 con 0x7f6b900fd1c0 2024-03-04T13:56:32.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:32.745+0000 7f6b96400640 1 -- 172.21.15.107:0/2522476612 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b9010f5a0 con 0x7f6b900fd1c0 2024-03-04T13:56:32.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:32.747+0000 7f6b96400640 1 -- 172.21.15.107:0/2522476612 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b9004bcc0 con 0x7f6b900fd1c0 2024-03-04T13:56:32.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:32.748+0000 7f6b8cc00640 1 -- 172.21.15.107:0/2522476612 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f6b84012400 con 0x7f6b900fd1c0 2024-03-04T13:56:32.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:32.748+0000 7f6b8cc00640 1 --2- 172.21.15.107:0/2522476612 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f6b7806d750 0x7f6b7806fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:32.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:32.749+0000 7f6b8cc00640 1 -- 172.21.15.107:0/2522476612 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f6b8408d110 con 0x7f6b900fd1c0 2024-03-04T13:56:32.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:32.749+0000 7f6b8fe00640 1 --2- 172.21.15.107:0/2522476612 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f6b7806d750 0x7f6b7806fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:56:32.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:32.749+0000 7f6b8fe00640 1 --2- 172.21.15.107:0/2522476612 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f6b7806d750 0x7f6b7806fc00 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c034040 tx=0x7f6b7c009da0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:56:32.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:32.753+0000 7f6b8cc00640 1 -- 172.21.15.107:0/2522476612 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6b84056910 con 0x7f6b900fd1c0 2024-03-04T13:56:32.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:32.992+0000 7f6b96400640 1 -- 172.21.15.107:0/2522476612 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6b901102d0 con 0x7f6b900fd1c0 2024-03-04T13:56:32.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:32.992+0000 7f6b8cc00640 1 -- 172.21.15.107:0/2522476612 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+5016 (secure 0 0 0) 0x7f6b84059fd0 con 0x7f6b900fd1c0 2024-03-04T13:56:32.995 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:56:32.995 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44243,"name":"cephfs.smithi169.jusudl","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6825/1336978861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":1336978861},{"type":"v1","addr":"172.21.15.169:6825","nonce":1336978861}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25},{"gid":44259,"name":"cephfs.smithi169.bxitzk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6827/4015044684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":4015044684},{"type":"v1","addr":"172.21.15.169:6827","nonce":4015044684}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-04T13:38:30.854004+0000","modified":"2024-03-04T13:54:22.860511+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":86,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34432},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34432":{"gid":34432,"name":"cephfs.smithi107.riexmm","rank":0,"incarnation":30,"state":"up:active","state_seq":24,"addr":"172.21.15.107:6827/1144672731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1144672731},{"type":"v1","addr":"172.21.15.107:6827","nonce":1144672731}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34436":{"gid":34436,"name":"cephfs.smithi107.thycho","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6829/3010096376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3010096376},{"type":"v1","addr":"172.21.15.107:6829","nonce":3010096376}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-04T13:56:32.996 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 36 2024-03-04T13:56:32.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:32.996+0000 7f6b96400640 1 -- 172.21.15.107:0/2522476612 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f6b7806d750 msgr2=0x7f6b7806fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:32.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:32.996+0000 7f6b96400640 1 --2- 172.21.15.107:0/2522476612 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f6b7806d750 0x7f6b7806fc00 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c034040 tx=0x7f6b7c009da0 comp rx=0 tx=0).stop 2024-03-04T13:56:32.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:32.996+0000 7f6b96400640 1 -- 172.21.15.107:0/2522476612 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b900fd1c0 msgr2=0x7f6b90114cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:32.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:32.996+0000 7f6b96400640 1 --2- 172.21.15.107:0/2522476612 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b900fd1c0 0x7f6b90114cd0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6b8400c790 tx=0x7f6b840043c0 comp rx=0 tx=0).stop 2024-03-04T13:56:32.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:32.996+0000 7f6b96400640 1 -- 172.21.15.107:0/2522476612 shutdown_connections 2024-03-04T13:56:32.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:32.996+0000 7f6b96400640 1 --2- 172.21.15.107:0/2522476612 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f6b7806d750 0x7f6b7806fc00 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:32.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:32.996+0000 7f6b96400640 1 --2- 172.21.15.107:0/2522476612 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6b900fdb60 0x7f6b90115210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:32.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:32.996+0000 7f6b96400640 1 --2- 172.21.15.107:0/2522476612 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b900fd1c0 0x7f6b90114cd0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:32.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:32.996+0000 7f6b96400640 1 -- 172.21.15.107:0/2522476612 >> 172.21.15.107:0/2522476612 conn(0x7f6b900f6c90 msgr2=0x7f6b900fb790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:56:32.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:32.996+0000 7f6b96400640 1 -- 172.21.15.107:0/2522476612 shutdown_connections 2024-03-04T13:56:32.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:32.996+0000 7f6b96400640 1 -- 172.21.15.107:0/2522476612 wait complete. 2024-03-04T13:56:33.018 DEBUG:tasks.fs:checking fs fscid=1,name=cephfs state = {'epoch': 15, 'max_mds': 1, 'flags': 50} 2024-03-04T13:56:33.018 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 16 2024-03-04T13:56:33.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:33.213+0000 7f447cc00640 1 -- 172.21.15.107:0/3566699559 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4478104320 msgr2=0x7f44781046f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:33.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:33.213+0000 7f447cc00640 1 --2- 172.21.15.107:0/3566699559 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4478104320 0x7f44781046f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f44680048d0 tx=0x7f4468009da0 comp rx=0 tx=0).stop 2024-03-04T13:56:33.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:33.213+0000 7f447cc00640 1 -- 172.21.15.107:0/3566699559 shutdown_connections 2024-03-04T13:56:33.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:33.213+0000 7f447cc00640 1 --2- 172.21.15.107:0/3566699559 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4478104cc0 0x7f44780fe350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:33.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:33.213+0000 7f447cc00640 1 --2- 172.21.15.107:0/3566699559 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4478104320 0x7f44781046f0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:33.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:33.213+0000 7f447cc00640 1 -- 172.21.15.107:0/3566699559 >> 172.21.15.107:0/3566699559 conn(0x7f44780f9f90 msgr2=0x7f44780fc370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:56:33.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:33.213+0000 7f447cc00640 1 -- 172.21.15.107:0/3566699559 shutdown_connections 2024-03-04T13:56:33.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:33.213+0000 7f447cc00640 1 -- 172.21.15.107:0/3566699559 wait complete. 2024-03-04T13:56:33.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:33.214+0000 7f447cc00640 1 Processor -- start 2024-03-04T13:56:33.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:33.214+0000 7f447cc00640 1 -- start start 2024-03-04T13:56:33.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:33.214+0000 7f447cc00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4478104320 0x7f4478114b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:33.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:33.215+0000 7f447cc00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4478104cc0 0x7f44781150c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:33.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:33.215+0000 7f447cc00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44781156c0 con 0x7f4478104cc0 2024-03-04T13:56:33.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:33.215+0000 7f447cc00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f447810ec50 con 0x7f4478104320 2024-03-04T13:56:33.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:33.215+0000 7f4476a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4478104cc0 0x7f44781150c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:56:33.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:33.215+0000 7f4476a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4478104cc0 0x7f44781150c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51830/0 (socket says 172.21.15.107:51830) 2024-03-04T13:56:33.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:33.215+0000 7f4476a00640 1 -- 172.21.15.107:0/3019867416 learned_addr learned my addr 172.21.15.107:0/3019867416 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:56:33.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:33.215+0000 7f4476a00640 1 -- 172.21.15.107:0/3019867416 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4478104320 msgr2=0x7f4478114b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:33.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:33.215+0000 7f4476a00640 1 --2- 172.21.15.107:0/3019867416 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4478104320 0x7f4478114b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:33.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:33.215+0000 7f4476a00640 1 -- 172.21.15.107:0/3019867416 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4468005040 con 0x7f4478104cc0 2024-03-04T13:56:33.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:33.216+0000 7f4476a00640 1 --2- 172.21.15.107:0/3019867416 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4478104cc0 0x7f44781150c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f4464002a50 tx=0x7f4464004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:56:33.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:33.216+0000 7f446f400640 1 -- 172.21.15.107:0/3019867416 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4464007c30 con 0x7f4478104cc0 2024-03-04T13:56:33.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:33.216+0000 7f446f400640 1 -- 172.21.15.107:0/3019867416 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f446400f040 con 0x7f4478104cc0 2024-03-04T13:56:33.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:33.216+0000 7f447cc00640 1 -- 172.21.15.107:0/3019867416 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f447810ef30 con 0x7f4478104cc0 2024-03-04T13:56:33.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:33.216+0000 7f446f400640 1 -- 172.21.15.107:0/3019867416 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f446400db50 con 0x7f4478104cc0 2024-03-04T13:56:33.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:33.216+0000 7f447cc00640 1 -- 172.21.15.107:0/3019867416 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f447810f450 con 0x7f4478104cc0 2024-03-04T13:56:33.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:33.217+0000 7f447cc00640 1 -- 172.21.15.107:0/3019867416 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f447804bcc0 con 0x7f4478104cc0 2024-03-04T13:56:33.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:33.218+0000 7f446f400640 1 -- 172.21.15.107:0/3019867416 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f446401c400 con 0x7f4478104cc0 2024-03-04T13:56:33.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:33.220+0000 7f446f400640 1 --2- 172.21.15.107:0/3019867416 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f445c06d760 0x7f445c06fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:33.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:33.220+0000 7f446f400640 1 -- 172.21.15.107:0/3019867416 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f446408c7f0 con 0x7f4478104cc0 2024-03-04T13:56:33.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:33.220+0000 7f4477400640 1 --2- 172.21.15.107:0/3019867416 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f445c06d760 0x7f445c06fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:56:33.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:33.221+0000 7f4477400640 1 --2- 172.21.15.107:0/3019867416 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f445c06d760 0x7f445c06fc10 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4468034040 tx=0x7f4468009da0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:56:33.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:33.224+0000 7f446f400640 1 -- 172.21.15.107:0/3019867416 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4464055ff0 con 0x7f4478104cc0 2024-03-04T13:56:33.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:33.460+0000 7f447cc00640 1 -- 172.21.15.107:0/3019867416 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 16, "format": "json"} v 0) v1 -- 0x7f447810b480 con 0x7f4478104cc0 2024-03-04T13:56:33.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:33.461+0000 7f446f400640 1 -- 172.21.15.107:0/3019867416 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 16, "format": "json"}]=0 dumped fsmap epoch 16 v36) v1 ==== 107+0+4845 (secure 0 0 0) 0x7f44640596b0 con 0x7f4478104cc0 2024-03-04T13:56:33.464 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:56:33.464 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14480,"name":"cephfs.smithi107.riexmm","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6827/3261622517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3261622517},{"type":"v1","addr":"172.21.15.107:6827","nonce":3261622517}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24259,"name":"cephfs.smithi169.bxitzk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.169:6827/3696090493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3696090493},{"type":"v1","addr":"172.21.15.169:6827","nonce":3696090493}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16}],"filesystems":[{"mdsmap":{"epoch":15,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-04T13:38:30.854004+0000","modified":"2024-03-04T13:38:50.015989+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24247},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14470":{"gid":14470,"name":"cephfs.smithi107.thycho","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":2,"addr":"172.21.15.107:6829/1985577060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":1985577060},{"type":"v1","addr":"172.21.15.107:6829","nonce":1985577060}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24247":{"gid":24247,"name":"cephfs.smithi169.jusudl","rank":0,"incarnation":11,"state":"up:active","state_seq":6,"addr":"172.21.15.169:6825/2994866076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2994866076},{"type":"v1","addr":"172.21.15.169:6825","nonce":2994866076}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-04T13:56:33.464 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 16 2024-03-04T13:56:33.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:33.464+0000 7f447cc00640 1 -- 172.21.15.107:0/3019867416 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f445c06d760 msgr2=0x7f445c06fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:33.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:33.464+0000 7f447cc00640 1 --2- 172.21.15.107:0/3019867416 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f445c06d760 0x7f445c06fc10 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4468034040 tx=0x7f4468009da0 comp rx=0 tx=0).stop 2024-03-04T13:56:33.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:33.464+0000 7f447cc00640 1 -- 172.21.15.107:0/3019867416 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4478104cc0 msgr2=0x7f44781150c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:33.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:33.464+0000 7f447cc00640 1 --2- 172.21.15.107:0/3019867416 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4478104cc0 0x7f44781150c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f4464002a50 tx=0x7f4464004600 comp rx=0 tx=0).stop 2024-03-04T13:56:33.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:33.465+0000 7f447cc00640 1 -- 172.21.15.107:0/3019867416 shutdown_connections 2024-03-04T13:56:33.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:33.465+0000 7f447cc00640 1 --2- 172.21.15.107:0/3019867416 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f445c06d760 0x7f445c06fc10 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:33.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:33.465+0000 7f447cc00640 1 --2- 172.21.15.107:0/3019867416 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4478104cc0 0x7f44781150c0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:33.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:33.465+0000 7f447cc00640 1 --2- 172.21.15.107:0/3019867416 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4478104320 0x7f4478114b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:33.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:33.465+0000 7f447cc00640 1 -- 172.21.15.107:0/3019867416 >> 172.21.15.107:0/3019867416 conn(0x7f44780f9f90 msgr2=0x7f44780fb960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:56:33.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:33.465+0000 7f447cc00640 1 -- 172.21.15.107:0/3019867416 shutdown_connections 2024-03-04T13:56:33.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:33.465+0000 7f447cc00640 1 -- 172.21.15.107:0/3019867416 wait complete. 2024-03-04T13:56:33.484 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 17 2024-03-04T13:56:33.559 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:56:33 smithi107 ceph-mon[112578]: from='client.? 172.21.15.107:0/1627127958' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-04T13:56:33.560 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:56:33 smithi107 ceph-mon[112578]: pgmap v313: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:56:33.560 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:56:33 smithi107 ceph-mon[112578]: from='client.? 172.21.15.107:0/2522476612' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T13:56:33.612 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:56:33 smithi169 ceph-mon[86183]: from='client.? 172.21.15.107:0/1627127958' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-04T13:56:33.612 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:56:33 smithi169 ceph-mon[86183]: pgmap v313: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:56:33.612 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:56:33 smithi169 ceph-mon[86183]: from='client.? 172.21.15.107:0/2522476612' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T13:56:33.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:33.685+0000 7f7e32200640 1 -- 172.21.15.107:0/1092053582 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e2c102120 msgr2=0x7f7e2c1024f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:33.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:33.685+0000 7f7e32200640 1 --2- 172.21.15.107:0/1092053582 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e2c102120 0x7f7e2c1024f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f7e200048d0 tx=0x7f7e20009da0 comp rx=0 tx=0).stop 2024-03-04T13:56:33.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:33.685+0000 7f7e32200640 1 -- 172.21.15.107:0/1092053582 shutdown_connections 2024-03-04T13:56:33.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:33.685+0000 7f7e32200640 1 --2- 172.21.15.107:0/1092053582 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e2c102ac0 0x7f7e2c0fc150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:33.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:33.685+0000 7f7e32200640 1 --2- 172.21.15.107:0/1092053582 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e2c102120 0x7f7e2c1024f0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:33.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:33.685+0000 7f7e32200640 1 -- 172.21.15.107:0/1092053582 >> 172.21.15.107:0/1092053582 conn(0x7f7e2c0f7cf0 msgr2=0x7f7e2c0fa100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:56:33.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:33.685+0000 7f7e32200640 1 -- 172.21.15.107:0/1092053582 shutdown_connections 2024-03-04T13:56:33.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:33.685+0000 7f7e32200640 1 -- 172.21.15.107:0/1092053582 wait complete. 2024-03-04T13:56:33.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:33.686+0000 7f7e32200640 1 Processor -- start 2024-03-04T13:56:33.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:33.686+0000 7f7e32200640 1 -- start start 2024-03-04T13:56:33.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:33.686+0000 7f7e32200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e2c102120 0x7f7e2c114ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:33.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:33.686+0000 7f7e32200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e2c102ac0 0x7f7e2c1150e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:33.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:33.686+0000 7f7e32200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e2c1156e0 con 0x7f7e2c102120 2024-03-04T13:56:33.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:33.686+0000 7f7e32200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e2c10ec70 con 0x7f7e2c102ac0 2024-03-04T13:56:33.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:33.687+0000 7f7e30e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e2c102120 0x7f7e2c114ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:56:33.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:33.687+0000 7f7e30e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e2c102120 0x7f7e2c114ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51844/0 (socket says 172.21.15.107:51844) 2024-03-04T13:56:33.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:33.687+0000 7f7e30e00640 1 -- 172.21.15.107:0/2416556736 learned_addr learned my addr 172.21.15.107:0/2416556736 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:56:33.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:33.687+0000 7f7e2be00640 1 --2- 172.21.15.107:0/2416556736 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e2c102ac0 0x7f7e2c1150e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:56:33.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:33.687+0000 7f7e30e00640 1 -- 172.21.15.107:0/2416556736 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e2c102ac0 msgr2=0x7f7e2c1150e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:33.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:33.687+0000 7f7e30e00640 1 --2- 172.21.15.107:0/2416556736 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e2c102ac0 0x7f7e2c1150e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:33.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:33.687+0000 7f7e30e00640 1 -- 172.21.15.107:0/2416556736 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e20005040 con 0x7f7e2c102120 2024-03-04T13:56:33.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:33.687+0000 7f7e2be00640 1 --2- 172.21.15.107:0/2416556736 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e2c102ac0 0x7f7e2c1150e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T13:56:33.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:33.687+0000 7f7e30e00640 1 --2- 172.21.15.107:0/2416556736 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e2c102120 0x7f7e2c114ba0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7e2002bda0 tx=0x7f7e20009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:56:33.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:33.688+0000 7f7e28c00640 1 -- 172.21.15.107:0/2416556736 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7e200094c0 con 0x7f7e2c102120 2024-03-04T13:56:33.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:33.688+0000 7f7e28c00640 1 -- 172.21.15.107:0/2416556736 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7e20009620 con 0x7f7e2c102120 2024-03-04T13:56:33.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:33.688+0000 7f7e32200640 1 -- 172.21.15.107:0/2416556736 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e2c10eef0 con 0x7f7e2c102120 2024-03-04T13:56:33.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:33.688+0000 7f7e28c00640 1 -- 172.21.15.107:0/2416556736 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7e200435f0 con 0x7f7e2c102120 2024-03-04T13:56:33.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:33.688+0000 7f7e32200640 1 -- 172.21.15.107:0/2416556736 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e2c10f3b0 con 0x7f7e2c102120 2024-03-04T13:56:33.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:33.689+0000 7f7e32200640 1 -- 172.21.15.107:0/2416556736 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e2c04bcc0 con 0x7f7e2c102120 2024-03-04T13:56:33.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:33.690+0000 7f7e28c00640 1 -- 172.21.15.107:0/2416556736 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f7e2002f070 con 0x7f7e2c102120 2024-03-04T13:56:33.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:33.691+0000 7f7e28c00640 1 --2- 172.21.15.107:0/2416556736 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f7e1406d760 0x7f7e1406fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:33.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:33.691+0000 7f7e28c00640 1 -- 172.21.15.107:0/2416556736 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f7e200aef80 con 0x7f7e2c102120 2024-03-04T13:56:33.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:33.691+0000 7f7e2be00640 1 --2- 172.21.15.107:0/2416556736 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f7e1406d760 0x7f7e1406fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:56:33.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:33.692+0000 7f7e2be00640 1 --2- 172.21.15.107:0/2416556736 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f7e1406d760 0x7f7e1406fc10 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c110460 tx=0x7f7e18009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:56:33.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:33.696+0000 7f7e28c00640 1 -- 172.21.15.107:0/2416556736 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7e20078700 con 0x7f7e2c102120 2024-03-04T13:56:33.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:33.915+0000 7f7e32200640 1 -- 172.21.15.107:0/2416556736 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 17, "format": "json"} v 0) v1 -- 0x7f7e2c10f5d0 con 0x7f7e2c102120 2024-03-04T13:56:33.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:33.917+0000 7f7e28c00640 1 -- 172.21.15.107:0/2416556736 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 17, "format": "json"}]=0 dumped fsmap epoch 17 v36) v1 ==== 107+0+4056 (secure 0 0 0) 0x7f7e2007bdc0 con 0x7f7e2c102120 2024-03-04T13:56:33.920 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:56:33.920 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14480,"name":"cephfs.smithi107.riexmm","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6827/3261622517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3261622517},{"type":"v1","addr":"172.21.15.107:6827","nonce":3261622517}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24259,"name":"cephfs.smithi169.bxitzk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.169:6827/3696090493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3696090493},{"type":"v1","addr":"172.21.15.169:6827","nonce":3696090493}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-04T13:38:30.854004+0000","modified":"2024-03-04T13:52:34.079716+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24247},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24247":{"gid":24247,"name":"cephfs.smithi169.jusudl","rank":0,"incarnation":11,"state":"up:active","state_seq":6,"addr":"172.21.15.169:6825/2994866076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2994866076},{"type":"v1","addr":"172.21.15.169:6825","nonce":2994866076}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-04T13:56:33.920 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 17 2024-03-04T13:56:33.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:33.920+0000 7f7e32200640 1 -- 172.21.15.107:0/2416556736 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f7e1406d760 msgr2=0x7f7e1406fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:33.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:33.920+0000 7f7e32200640 1 --2- 172.21.15.107:0/2416556736 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f7e1406d760 0x7f7e1406fc10 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c110460 tx=0x7f7e18009040 comp rx=0 tx=0).stop 2024-03-04T13:56:33.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:33.920+0000 7f7e32200640 1 -- 172.21.15.107:0/2416556736 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e2c102120 msgr2=0x7f7e2c114ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:33.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:33.920+0000 7f7e32200640 1 --2- 172.21.15.107:0/2416556736 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e2c102120 0x7f7e2c114ba0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7e2002bda0 tx=0x7f7e20009da0 comp rx=0 tx=0).stop 2024-03-04T13:56:33.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:33.920+0000 7f7e32200640 1 -- 172.21.15.107:0/2416556736 shutdown_connections 2024-03-04T13:56:33.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:33.920+0000 7f7e32200640 1 --2- 172.21.15.107:0/2416556736 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f7e1406d760 0x7f7e1406fc10 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:33.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:33.920+0000 7f7e32200640 1 --2- 172.21.15.107:0/2416556736 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e2c102ac0 0x7f7e2c1150e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:33.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:33.920+0000 7f7e32200640 1 --2- 172.21.15.107:0/2416556736 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e2c102120 0x7f7e2c114ba0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:33.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:33.920+0000 7f7e32200640 1 -- 172.21.15.107:0/2416556736 >> 172.21.15.107:0/2416556736 conn(0x7f7e2c0f7cf0 msgr2=0x7f7e2c0f95a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:56:33.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:33.920+0000 7f7e32200640 1 -- 172.21.15.107:0/2416556736 shutdown_connections 2024-03-04T13:56:33.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:33.920+0000 7f7e32200640 1 -- 172.21.15.107:0/2416556736 wait complete. 2024-03-04T13:56:33.938 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 17 2024-03-04T13:56:33.938 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 18 2024-03-04T13:56:34.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:34.122+0000 7f7ca6a00640 1 -- 172.21.15.107:0/4005605322 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ca0104580 msgr2=0x7f7ca0104950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:34.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:34.122+0000 7f7ca6a00640 1 --2- 172.21.15.107:0/4005605322 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ca0104580 0x7f7ca0104950 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f7c900048d0 tx=0x7f7c90009da0 comp rx=0 tx=0).stop 2024-03-04T13:56:34.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:34.122+0000 7f7ca6a00640 1 -- 172.21.15.107:0/4005605322 shutdown_connections 2024-03-04T13:56:34.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:34.122+0000 7f7ca6a00640 1 --2- 172.21.15.107:0/4005605322 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ca0104e90 0x7f7ca00fe5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:34.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:34.122+0000 7f7ca6a00640 1 --2- 172.21.15.107:0/4005605322 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ca0104580 0x7f7ca0104950 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:34.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:34.122+0000 7f7ca6a00640 1 -- 172.21.15.107:0/4005605322 >> 172.21.15.107:0/4005605322 conn(0x7f7ca00fa160 msgr2=0x7f7ca00fc570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:56:34.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:34.122+0000 7f7ca6a00640 1 -- 172.21.15.107:0/4005605322 shutdown_connections 2024-03-04T13:56:34.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:34.122+0000 7f7ca6a00640 1 -- 172.21.15.107:0/4005605322 wait complete. 2024-03-04T13:56:34.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:34.123+0000 7f7ca6a00640 1 Processor -- start 2024-03-04T13:56:34.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:34.123+0000 7f7ca6a00640 1 -- start start 2024-03-04T13:56:34.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:34.123+0000 7f7ca6a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ca0104580 0x7f7ca0114e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:34.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:34.123+0000 7f7ca6a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ca0104e90 0x7f7ca0115360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:34.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:34.123+0000 7f7ca6a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca0115960 con 0x7f7ca0104580 2024-03-04T13:56:34.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:34.123+0000 7f7ca6a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca010eef0 con 0x7f7ca0104e90 2024-03-04T13:56:34.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:34.123+0000 7f7c9fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ca0104580 0x7f7ca0114e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:56:34.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:34.124+0000 7f7c9fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ca0104580 0x7f7ca0114e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51862/0 (socket says 172.21.15.107:51862) 2024-03-04T13:56:34.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:34.124+0000 7f7c9fe00640 1 -- 172.21.15.107:0/1394004342 learned_addr learned my addr 172.21.15.107:0/1394004342 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:56:34.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:34.124+0000 7f7c9fe00640 1 -- 172.21.15.107:0/1394004342 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ca0104e90 msgr2=0x7f7ca0115360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:34.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:34.124+0000 7f7c9f400640 1 --2- 172.21.15.107:0/1394004342 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ca0104e90 0x7f7ca0115360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:56:34.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:34.124+0000 7f7c9fe00640 1 --2- 172.21.15.107:0/1394004342 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ca0104e90 0x7f7ca0115360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:34.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:34.124+0000 7f7c9fe00640 1 -- 172.21.15.107:0/1394004342 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c90005040 con 0x7f7ca0104580 2024-03-04T13:56:34.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:34.124+0000 7f7c9f400640 1 --2- 172.21.15.107:0/1394004342 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ca0104e90 0x7f7ca0115360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T13:56:34.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:34.124+0000 7f7c9fe00640 1 --2- 172.21.15.107:0/1394004342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ca0104580 0x7f7ca0114e20 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f7c90031720 tx=0x7f7c90009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:56:34.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:34.125+0000 7f7c97e00640 1 -- 172.21.15.107:0/1394004342 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c900094c0 con 0x7f7ca0104580 2024-03-04T13:56:34.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:34.125+0000 7f7ca6a00640 1 -- 172.21.15.107:0/1394004342 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ca010f170 con 0x7f7ca0104580 2024-03-04T13:56:34.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:34.125+0000 7f7c97e00640 1 -- 172.21.15.107:0/1394004342 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7c90009620 con 0x7f7ca0104580 2024-03-04T13:56:34.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:34.125+0000 7f7c97e00640 1 -- 172.21.15.107:0/1394004342 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c9003b640 con 0x7f7ca0104580 2024-03-04T13:56:34.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:34.125+0000 7f7ca6a00640 1 -- 172.21.15.107:0/1394004342 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ca010f690 con 0x7f7ca0104580 2024-03-04T13:56:34.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:34.126+0000 7f7ca6a00640 1 -- 172.21.15.107:0/1394004342 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ca0064480 con 0x7f7ca0104580 2024-03-04T13:56:34.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:34.127+0000 7f7c97e00640 1 -- 172.21.15.107:0/1394004342 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f7c9002f070 con 0x7f7ca0104580 2024-03-04T13:56:34.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:34.127+0000 7f7c97e00640 1 --2- 172.21.15.107:0/1394004342 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f7c8406d750 0x7f7c8406fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:34.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:34.128+0000 7f7c97e00640 1 -- 172.21.15.107:0/1394004342 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f7c900af550 con 0x7f7ca0104580 2024-03-04T13:56:34.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:34.128+0000 7f7c9f400640 1 --2- 172.21.15.107:0/1394004342 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f7c8406d750 0x7f7c8406fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:56:34.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:34.128+0000 7f7c9f400640 1 --2- 172.21.15.107:0/1394004342 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f7c8406d750 0x7f7c8406fc00 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7ca01106e0 tx=0x7f7c8c009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:56:34.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:34.132+0000 7f7c97e00640 1 -- 172.21.15.107:0/1394004342 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7c90078d10 con 0x7f7ca0104580 2024-03-04T13:56:34.372 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:34.369+0000 7f7ca6a00640 1 -- 172.21.15.107:0/1394004342 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 18, "format": "json"} v 0) v1 -- 0x7f7ca010f820 con 0x7f7ca0104580 2024-03-04T13:56:34.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:34.371+0000 7f7c97e00640 1 -- 172.21.15.107:0/1394004342 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 18, "format": "json"}]=0 dumped fsmap epoch 18 v36) v1 ==== 107+0+4838 (secure 0 0 0) 0x7f7c9007c3d0 con 0x7f7ca0104580 2024-03-04T13:56:34.374 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:56:34.374 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14480,"name":"cephfs.smithi107.riexmm","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6827/3261622517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3261622517},{"type":"v1","addr":"172.21.15.107:6827","nonce":3261622517}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24259,"name":"cephfs.smithi169.bxitzk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.169:6827/3696090493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3696090493},{"type":"v1","addr":"172.21.15.169:6827","nonce":3696090493}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16},{"gid":34400,"name":"cephfs.smithi107.thycho","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6829/2973025326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":2973025326},{"type":"v1","addr":"172.21.15.107:6829","nonce":2973025326}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-04T13:38:30.854004+0000","modified":"2024-03-04T13:52:34.079716+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24247},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24247":{"gid":24247,"name":"cephfs.smithi169.jusudl","rank":0,"incarnation":11,"state":"up:active","state_seq":6,"addr":"172.21.15.169:6825/2994866076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2994866076},{"type":"v1","addr":"172.21.15.169:6825","nonce":2994866076}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-04T13:56:34.374 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 18 2024-03-04T13:56:34.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:34.373+0000 7f7ca6a00640 1 -- 172.21.15.107:0/1394004342 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f7c8406d750 msgr2=0x7f7c8406fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:34.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:34.373+0000 7f7ca6a00640 1 --2- 172.21.15.107:0/1394004342 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f7c8406d750 0x7f7c8406fc00 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7ca01106e0 tx=0x7f7c8c009040 comp rx=0 tx=0).stop 2024-03-04T13:56:34.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:34.373+0000 7f7ca6a00640 1 -- 172.21.15.107:0/1394004342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ca0104580 msgr2=0x7f7ca0114e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:34.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:34.373+0000 7f7ca6a00640 1 --2- 172.21.15.107:0/1394004342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ca0104580 0x7f7ca0114e20 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f7c90031720 tx=0x7f7c90009da0 comp rx=0 tx=0).stop 2024-03-04T13:56:34.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:34.373+0000 7f7ca6a00640 1 -- 172.21.15.107:0/1394004342 shutdown_connections 2024-03-04T13:56:34.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:34.373+0000 7f7ca6a00640 1 --2- 172.21.15.107:0/1394004342 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f7c8406d750 0x7f7c8406fc00 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:34.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:34.373+0000 7f7ca6a00640 1 --2- 172.21.15.107:0/1394004342 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ca0104e90 0x7f7ca0115360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:34.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:34.373+0000 7f7ca6a00640 1 --2- 172.21.15.107:0/1394004342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ca0104580 0x7f7ca0114e20 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:34.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:34.373+0000 7f7ca6a00640 1 -- 172.21.15.107:0/1394004342 >> 172.21.15.107:0/1394004342 conn(0x7f7ca00fa160 msgr2=0x7f7ca00fba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:56:34.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:34.374+0000 7f7ca6a00640 1 -- 172.21.15.107:0/1394004342 shutdown_connections 2024-03-04T13:56:34.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:34.374+0000 7f7ca6a00640 1 -- 172.21.15.107:0/1394004342 wait complete. 2024-03-04T13:56:34.395 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 18 2024-03-04T13:56:34.395 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 19 2024-03-04T13:56:34.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:34.592+0000 7f12e5000640 1 -- 172.21.15.107:0/3243218085 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12e0100110 msgr2=0x7f12e0100560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:34.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:34.592+0000 7f12e5000640 1 --2- 172.21.15.107:0/3243218085 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12e0100110 0x7f12e0100560 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f12c80048d0 tx=0x7f12c8009da0 comp rx=0 tx=0).stop 2024-03-04T13:56:34.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:34.592+0000 7f12e5000640 1 -- 172.21.15.107:0/3243218085 shutdown_connections 2024-03-04T13:56:34.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:34.592+0000 7f12e5000640 1 --2- 172.21.15.107:0/3243218085 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12e0100110 0x7f12e0100560 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:34.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:34.592+0000 7f12e5000640 1 --2- 172.21.15.107:0/3243218085 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12e00fe9c0 0x7f12e00fed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:34.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:34.592+0000 7f12e5000640 1 -- 172.21.15.107:0/3243218085 >> 172.21.15.107:0/3243218085 conn(0x7f12e00fa160 msgr2=0x7f12e00fc570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:56:34.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:34.592+0000 7f12e5000640 1 -- 172.21.15.107:0/3243218085 shutdown_connections 2024-03-04T13:56:34.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:34.592+0000 7f12e5000640 1 -- 172.21.15.107:0/3243218085 wait complete. 2024-03-04T13:56:34.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:34.593+0000 7f12e5000640 1 Processor -- start 2024-03-04T13:56:34.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:34.593+0000 7f12e5000640 1 -- start start 2024-03-04T13:56:34.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:34.593+0000 7f12e5000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12e00fe9c0 0x7f12e0114dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:34.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:34.593+0000 7f12e5000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12e0100110 0x7f12e0115300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:34.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:34.593+0000 7f12e5000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12e0115900 con 0x7f12e00fe9c0 2024-03-04T13:56:34.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:34.593+0000 7f12e5000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12e010ee90 con 0x7f12e0100110 2024-03-04T13:56:34.596 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:56:34 smithi107 ceph-mon[112578]: from='client.? 172.21.15.107:0/3019867416' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 16, "format": "json"}]: dispatch 2024-03-04T13:56:34.596 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:56:34 smithi107 ceph-mon[112578]: from='client.? 172.21.15.107:0/2416556736' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 17, "format": "json"}]: dispatch 2024-03-04T13:56:34.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:34.593+0000 7f12dde00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12e00fe9c0 0x7f12e0114dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:56:34.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:34.593+0000 7f12dde00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12e00fe9c0 0x7f12e0114dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51878/0 (socket says 172.21.15.107:51878) 2024-03-04T13:56:34.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:34.593+0000 7f12dde00640 1 -- 172.21.15.107:0/3176401593 learned_addr learned my addr 172.21.15.107:0/3176401593 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:56:34.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:34.594+0000 7f12d7e00640 1 --2- 172.21.15.107:0/3176401593 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12e0100110 0x7f12e0115300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:56:34.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:34.594+0000 7f12dde00640 1 -- 172.21.15.107:0/3176401593 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12e0100110 msgr2=0x7f12e0115300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:34.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:34.594+0000 7f12dde00640 1 --2- 172.21.15.107:0/3176401593 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12e0100110 0x7f12e0115300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:34.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:34.594+0000 7f12dde00640 1 -- 172.21.15.107:0/3176401593 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12c8005040 con 0x7f12e00fe9c0 2024-03-04T13:56:34.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:34.594+0000 7f12dde00640 1 --2- 172.21.15.107:0/3176401593 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12e00fe9c0 0x7f12e0114dc0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f12d000c810 tx=0x7f12d00043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:56:34.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:34.594+0000 7f12d6000640 1 -- 172.21.15.107:0/3176401593 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f12d000a820 con 0x7f12e00fe9c0 2024-03-04T13:56:34.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:34.594+0000 7f12d6000640 1 -- 172.21.15.107:0/3176401593 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f12d000ae50 con 0x7f12e00fe9c0 2024-03-04T13:56:34.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:34.594+0000 7f12e5000640 1 -- 172.21.15.107:0/3176401593 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12e010f170 con 0x7f12e00fe9c0 2024-03-04T13:56:34.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:34.594+0000 7f12d6000640 1 -- 172.21.15.107:0/3176401593 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f12d001b5f0 con 0x7f12e00fe9c0 2024-03-04T13:56:34.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:34.595+0000 7f12e5000640 1 -- 172.21.15.107:0/3176401593 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12e010f690 con 0x7f12e00fe9c0 2024-03-04T13:56:34.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:34.596+0000 7f12e5000640 1 -- 172.21.15.107:0/3176401593 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12e0064030 con 0x7f12e00fe9c0 2024-03-04T13:56:34.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:34.597+0000 7f12d6000640 1 -- 172.21.15.107:0/3176401593 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f12d000a980 con 0x7f12e00fe9c0 2024-03-04T13:56:34.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:34.598+0000 7f12d6000640 1 --2- 172.21.15.107:0/3176401593 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f12c406d760 0x7f12c406fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:34.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:34.598+0000 7f12d6000640 1 -- 172.21.15.107:0/3176401593 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f12d008d4e0 con 0x7f12e00fe9c0 2024-03-04T13:56:34.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:34.598+0000 7f12d7e00640 1 --2- 172.21.15.107:0/3176401593 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f12c406d760 0x7f12c406fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:56:34.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:34.599+0000 7f12d7e00640 1 --2- 172.21.15.107:0/3176401593 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f12c406d760 0x7f12c406fc10 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f12c8034040 tx=0x7f12c8009da0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:56:34.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:34.604+0000 7f12d6000640 1 -- 172.21.15.107:0/3176401593 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f12d0056c60 con 0x7f12e00fe9c0 2024-03-04T13:56:34.612 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:56:34 smithi169 ceph-mon[86183]: from='client.? 172.21.15.107:0/3019867416' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 16, "format": "json"}]: dispatch 2024-03-04T13:56:34.612 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:56:34 smithi169 ceph-mon[86183]: from='client.? 172.21.15.107:0/2416556736' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 17, "format": "json"}]: dispatch 2024-03-04T13:56:34.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:34.828+0000 7f12e5000640 1 -- 172.21.15.107:0/3176401593 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 19, "format": "json"} v 0) v1 -- 0x7f12e010f910 con 0x7f12e00fe9c0 2024-03-04T13:56:34.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:34.829+0000 7f12d6000640 1 -- 172.21.15.107:0/3176401593 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 19, "format": "json"}]=0 dumped fsmap epoch 19 v36) v1 ==== 107+0+4056 (secure 0 0 0) 0x7f12d005a320 con 0x7f12e00fe9c0 2024-03-04T13:56:34.831 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:56:34.831 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24259,"name":"cephfs.smithi169.bxitzk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.169:6827/3696090493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3696090493},{"type":"v1","addr":"172.21.15.169:6827","nonce":3696090493}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16},{"gid":34400,"name":"cephfs.smithi107.thycho","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6829/2973025326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":2973025326},{"type":"v1","addr":"172.21.15.107:6829","nonce":2973025326}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-04T13:38:30.854004+0000","modified":"2024-03-04T13:52:34.079716+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24247},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24247":{"gid":24247,"name":"cephfs.smithi169.jusudl","rank":0,"incarnation":11,"state":"up:active","state_seq":6,"addr":"172.21.15.169:6825/2994866076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2994866076},{"type":"v1","addr":"172.21.15.169:6825","nonce":2994866076}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-04T13:56:34.831 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 19 2024-03-04T13:56:34.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:34.832+0000 7f12e5000640 1 -- 172.21.15.107:0/3176401593 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f12c406d760 msgr2=0x7f12c406fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:34.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:34.832+0000 7f12e5000640 1 --2- 172.21.15.107:0/3176401593 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f12c406d760 0x7f12c406fc10 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f12c8034040 tx=0x7f12c8009da0 comp rx=0 tx=0).stop 2024-03-04T13:56:34.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:34.832+0000 7f12e5000640 1 -- 172.21.15.107:0/3176401593 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12e00fe9c0 msgr2=0x7f12e0114dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:34.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:34.832+0000 7f12e5000640 1 --2- 172.21.15.107:0/3176401593 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12e00fe9c0 0x7f12e0114dc0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f12d000c810 tx=0x7f12d00043c0 comp rx=0 tx=0).stop 2024-03-04T13:56:34.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:34.832+0000 7f12e5000640 1 -- 172.21.15.107:0/3176401593 shutdown_connections 2024-03-04T13:56:34.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:34.832+0000 7f12e5000640 1 --2- 172.21.15.107:0/3176401593 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f12c406d760 0x7f12c406fc10 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:34.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:34.832+0000 7f12e5000640 1 --2- 172.21.15.107:0/3176401593 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12e0100110 0x7f12e0115300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:34.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:34.832+0000 7f12e5000640 1 --2- 172.21.15.107:0/3176401593 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12e00fe9c0 0x7f12e0114dc0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:34.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:34.832+0000 7f12e5000640 1 -- 172.21.15.107:0/3176401593 >> 172.21.15.107:0/3176401593 conn(0x7f12e00fa160 msgr2=0x7f12e00fbc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:56:34.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:34.832+0000 7f12e5000640 1 -- 172.21.15.107:0/3176401593 shutdown_connections 2024-03-04T13:56:34.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:34.832+0000 7f12e5000640 1 -- 172.21.15.107:0/3176401593 wait complete. 2024-03-04T13:56:34.849 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 19 2024-03-04T13:56:34.849 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 20 2024-03-04T13:56:35.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.050+0000 7ff78a800640 1 -- 172.21.15.107:0/4229991248 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff784108760 msgr2=0x7ff784108b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:35.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.050+0000 7ff78a800640 1 --2- 172.21.15.107:0/4229991248 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff784108760 0x7ff784108b30 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ff7700048d0 tx=0x7ff770009da0 comp rx=0 tx=0).stop 2024-03-04T13:56:35.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.050+0000 7ff78a800640 1 -- 172.21.15.107:0/4229991248 shutdown_connections 2024-03-04T13:56:35.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.050+0000 7ff78a800640 1 --2- 172.21.15.107:0/4229991248 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff784102740 0x7ff784102bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:35.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.050+0000 7ff78a800640 1 --2- 172.21.15.107:0/4229991248 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff784108760 0x7ff784108b30 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:35.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.050+0000 7ff78a800640 1 -- 172.21.15.107:0/4229991248 >> 172.21.15.107:0/4229991248 conn(0x7ff784071fc0 msgr2=0x7ff7840743d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:56:35.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.050+0000 7ff78a800640 1 -- 172.21.15.107:0/4229991248 shutdown_connections 2024-03-04T13:56:35.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.050+0000 7ff78a800640 1 -- 172.21.15.107:0/4229991248 wait complete. 2024-03-04T13:56:35.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.051+0000 7ff78a800640 1 Processor -- start 2024-03-04T13:56:35.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.051+0000 7ff78a800640 1 -- start start 2024-03-04T13:56:35.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.052+0000 7ff78a800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff784102740 0x7ff784119160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:35.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.052+0000 7ff78a800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff784108760 0x7ff7841196a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:35.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.052+0000 7ff78a800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff784119ca0 con 0x7ff784102740 2024-03-04T13:56:35.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.052+0000 7ff78a800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff784113230 con 0x7ff784108760 2024-03-04T13:56:35.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.052+0000 7ff789400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff784102740 0x7ff784119160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:56:35.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.052+0000 7ff789400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff784102740 0x7ff784119160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51908/0 (socket says 172.21.15.107:51908) 2024-03-04T13:56:35.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.052+0000 7ff789400640 1 -- 172.21.15.107:0/1680122752 learned_addr learned my addr 172.21.15.107:0/1680122752 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:56:35.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.053+0000 7ff789400640 1 -- 172.21.15.107:0/1680122752 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff784108760 msgr2=0x7ff7841196a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:35.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.053+0000 7ff789400640 1 --2- 172.21.15.107:0/1680122752 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff784108760 0x7ff7841196a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:35.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.053+0000 7ff789400640 1 -- 172.21.15.107:0/1680122752 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff770005040 con 0x7ff784102740 2024-03-04T13:56:35.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.053+0000 7ff789400640 1 --2- 172.21.15.107:0/1680122752 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff784102740 0x7ff784119160 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ff770031720 tx=0x7ff770009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:56:35.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.053+0000 7ff781600640 1 -- 172.21.15.107:0/1680122752 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff7700094c0 con 0x7ff784102740 2024-03-04T13:56:35.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.053+0000 7ff781600640 1 -- 172.21.15.107:0/1680122752 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff770009620 con 0x7ff784102740 2024-03-04T13:56:35.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.053+0000 7ff781600640 1 -- 172.21.15.107:0/1680122752 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff77003b640 con 0x7ff784102740 2024-03-04T13:56:35.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.053+0000 7ff78a800640 1 -- 172.21.15.107:0/1680122752 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7841134b0 con 0x7ff784102740 2024-03-04T13:56:35.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.053+0000 7ff78a800640 1 -- 172.21.15.107:0/1680122752 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7841139d0 con 0x7ff784102740 2024-03-04T13:56:35.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.054+0000 7ff78a800640 1 -- 172.21.15.107:0/1680122752 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff78404bcc0 con 0x7ff784102740 2024-03-04T13:56:35.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.055+0000 7ff781600640 1 -- 172.21.15.107:0/1680122752 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7ff77002f070 con 0x7ff784102740 2024-03-04T13:56:35.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.056+0000 7ff781600640 1 --2- 172.21.15.107:0/1680122752 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7ff76c06d750 0x7ff76c06fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:35.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.056+0000 7ff781600640 1 -- 172.21.15.107:0/1680122752 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7ff7700af500 con 0x7ff784102740 2024-03-04T13:56:35.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.057+0000 7ff788a00640 1 --2- 172.21.15.107:0/1680122752 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7ff76c06d750 0x7ff76c06fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:56:35.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.057+0000 7ff788a00640 1 --2- 172.21.15.107:0/1680122752 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7ff76c06d750 0x7ff76c06fc00 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff784114a20 tx=0x7ff778007cb0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:56:35.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.061+0000 7ff781600640 1 -- 172.21.15.107:0/1680122752 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff770078d00 con 0x7ff784102740 2024-03-04T13:56:35.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.298+0000 7ff78a800640 1 -- 172.21.15.107:0/1680122752 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 20, "format": "json"} v 0) v1 -- 0x7ff7841079b0 con 0x7ff784102740 2024-03-04T13:56:35.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.300+0000 7ff781600640 1 -- 172.21.15.107:0/1680122752 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 20, "format": "json"}]=0 dumped fsmap epoch 20 v36) v1 ==== 107+0+4872 (secure 0 0 0) 0x7ff77007c3c0 con 0x7ff784102740 2024-03-04T13:56:35.301 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:56:35.302 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24259,"name":"cephfs.smithi169.bxitzk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.169:6827/3696090493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3696090493},{"type":"v1","addr":"172.21.15.169:6827","nonce":3696090493}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16},{"gid":34400,"name":"cephfs.smithi107.thycho","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6829/2973025326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":2973025326},{"type":"v1","addr":"172.21.15.107:6829","nonce":2973025326}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":34432,"name":"cephfs.smithi107.riexmm","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6827/1144672731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1144672731},{"type":"v1","addr":"172.21.15.107:6827","nonce":1144672731}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-04T13:38:30.854004+0000","modified":"2024-03-04T13:52:34.079716+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24247},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24247":{"gid":24247,"name":"cephfs.smithi169.jusudl","rank":0,"incarnation":11,"state":"up:active","state_seq":6,"addr":"172.21.15.169:6825/2994866076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2994866076},{"type":"v1","addr":"172.21.15.169:6825","nonce":2994866076}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-04T13:56:35.302 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 20 2024-03-04T13:56:35.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.303+0000 7ff78a800640 1 -- 172.21.15.107:0/1680122752 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7ff76c06d750 msgr2=0x7ff76c06fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:35.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.303+0000 7ff78a800640 1 --2- 172.21.15.107:0/1680122752 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7ff76c06d750 0x7ff76c06fc00 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff784114a20 tx=0x7ff778007cb0 comp rx=0 tx=0).stop 2024-03-04T13:56:35.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.303+0000 7ff78a800640 1 -- 172.21.15.107:0/1680122752 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff784102740 msgr2=0x7ff784119160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:35.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.303+0000 7ff78a800640 1 --2- 172.21.15.107:0/1680122752 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff784102740 0x7ff784119160 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ff770031720 tx=0x7ff770009da0 comp rx=0 tx=0).stop 2024-03-04T13:56:35.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.303+0000 7ff78a800640 1 -- 172.21.15.107:0/1680122752 shutdown_connections 2024-03-04T13:56:35.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.303+0000 7ff78a800640 1 --2- 172.21.15.107:0/1680122752 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7ff76c06d750 0x7ff76c06fc00 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:35.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.303+0000 7ff78a800640 1 --2- 172.21.15.107:0/1680122752 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff784108760 0x7ff7841196a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:35.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.303+0000 7ff78a800640 1 --2- 172.21.15.107:0/1680122752 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff784102740 0x7ff784119160 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:35.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.303+0000 7ff78a800640 1 -- 172.21.15.107:0/1680122752 >> 172.21.15.107:0/1680122752 conn(0x7ff784071fc0 msgr2=0x7ff784104560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:56:35.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.303+0000 7ff78a800640 1 -- 172.21.15.107:0/1680122752 shutdown_connections 2024-03-04T13:56:35.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.303+0000 7ff78a800640 1 -- 172.21.15.107:0/1680122752 wait complete. 2024-03-04T13:56:35.321 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 20 2024-03-04T13:56:35.321 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 21 2024-03-04T13:56:35.519 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.517+0000 7f3056e00640 1 -- 172.21.15.107:0/2118070789 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3050102120 msgr2=0x7f30501024f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:35.519 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.517+0000 7f3056e00640 1 --2- 172.21.15.107:0/2118070789 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3050102120 0x7f30501024f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f30440048d0 tx=0x7f3044009da0 comp rx=0 tx=0).stop 2024-03-04T13:56:35.519 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.517+0000 7f3056e00640 1 -- 172.21.15.107:0/2118070789 shutdown_connections 2024-03-04T13:56:35.519 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.517+0000 7f3056e00640 1 --2- 172.21.15.107:0/2118070789 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3050102ac0 0x7f30500fc150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:35.519 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.517+0000 7f3056e00640 1 --2- 172.21.15.107:0/2118070789 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3050102120 0x7f30501024f0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:35.519 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.517+0000 7f3056e00640 1 -- 172.21.15.107:0/2118070789 >> 172.21.15.107:0/2118070789 conn(0x7f30500f7cf0 msgr2=0x7f30500fa100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:56:35.519 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.517+0000 7f3056e00640 1 -- 172.21.15.107:0/2118070789 shutdown_connections 2024-03-04T13:56:35.519 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.517+0000 7f3056e00640 1 -- 172.21.15.107:0/2118070789 wait complete. 2024-03-04T13:56:35.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.518+0000 7f3056e00640 1 Processor -- start 2024-03-04T13:56:35.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.518+0000 7f3056e00640 1 -- start start 2024-03-04T13:56:35.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.518+0000 7f3056e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3050102120 0x7f305006f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:35.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.518+0000 7f3056e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3050102ac0 0x7f305006fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:35.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.518+0000 7f3056e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3050070220 con 0x7f3050102ac0 2024-03-04T13:56:35.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.518+0000 7f3056e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3050070390 con 0x7f3050102120 2024-03-04T13:56:35.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.519+0000 7f3055000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3050102ac0 0x7f305006fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:56:35.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.519+0000 7f3055000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3050102ac0 0x7f305006fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51930/0 (socket says 172.21.15.107:51930) 2024-03-04T13:56:35.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.519+0000 7f3055000640 1 -- 172.21.15.107:0/951383788 learned_addr learned my addr 172.21.15.107:0/951383788 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:56:35.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.519+0000 7f3055a00640 1 --2- 172.21.15.107:0/951383788 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3050102120 0x7f305006f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:56:35.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.519+0000 7f3055000640 1 -- 172.21.15.107:0/951383788 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3050102120 msgr2=0x7f305006f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:35.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.519+0000 7f3055000640 1 --2- 172.21.15.107:0/951383788 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3050102120 0x7f305006f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:35.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.519+0000 7f3055000640 1 -- 172.21.15.107:0/951383788 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3044005040 con 0x7f3050102ac0 2024-03-04T13:56:35.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.519+0000 7f3055000640 1 --2- 172.21.15.107:0/951383788 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3050102ac0 0x7f305006fce0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f303c00c790 tx=0x7f303c0043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:56:35.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.519+0000 7f304d600640 1 -- 172.21.15.107:0/951383788 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f303c00a820 con 0x7f3050102ac0 2024-03-04T13:56:35.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.520+0000 7f304d600640 1 -- 172.21.15.107:0/951383788 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f303c00ae50 con 0x7f3050102ac0 2024-03-04T13:56:35.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.520+0000 7f304d600640 1 -- 172.21.15.107:0/951383788 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f303c0115e0 con 0x7f3050102ac0 2024-03-04T13:56:35.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.520+0000 7f3056e00640 1 -- 172.21.15.107:0/951383788 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3050118ec0 con 0x7f3050102ac0 2024-03-04T13:56:35.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.520+0000 7f3056e00640 1 -- 172.21.15.107:0/951383788 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30501191f0 con 0x7f3050102ac0 2024-03-04T13:56:35.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.521+0000 7f3056e00640 1 -- 172.21.15.107:0/951383788 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f305004bcc0 con 0x7f3050102ac0 2024-03-04T13:56:35.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.522+0000 7f304d600640 1 -- 172.21.15.107:0/951383788 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f303c012400 con 0x7f3050102ac0 2024-03-04T13:56:35.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.523+0000 7f304d600640 1 --2- 172.21.15.107:0/951383788 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f303806d760 0x7f303806fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:35.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.523+0000 7f304d600640 1 -- 172.21.15.107:0/951383788 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f303c08cfa0 con 0x7f3050102ac0 2024-03-04T13:56:35.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.523+0000 7f3055a00640 1 --2- 172.21.15.107:0/951383788 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f303806d760 0x7f303806fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:56:35.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.524+0000 7f3055a00640 1 --2- 172.21.15.107:0/951383788 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f303806d760 0x7f303806fc10 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3044034040 tx=0x7f3044009da0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:56:35.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.528+0000 7f304d600640 1 -- 172.21.15.107:0/951383788 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f303c0c0d20 con 0x7f3050102ac0 2024-03-04T13:56:35.595 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:56:35 smithi107 ceph-mon[112578]: from='client.? 172.21.15.107:0/1394004342' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 18, "format": "json"}]: dispatch 2024-03-04T13:56:35.595 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:56:35 smithi107 ceph-mon[112578]: pgmap v314: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:56:35.595 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:56:35 smithi107 ceph-mon[112578]: from='client.? 172.21.15.107:0/3176401593' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 19, "format": "json"}]: dispatch 2024-03-04T13:56:35.612 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:56:35 smithi169 ceph-mon[86183]: from='client.? 172.21.15.107:0/1394004342' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 18, "format": "json"}]: dispatch 2024-03-04T13:56:35.612 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:56:35 smithi169 ceph-mon[86183]: pgmap v314: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:56:35.612 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:56:35 smithi169 ceph-mon[86183]: from='client.? 172.21.15.107:0/3176401593' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 19, "format": "json"}]: dispatch 2024-03-04T13:56:35.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.745+0000 7f3056e00640 1 -- 172.21.15.107:0/951383788 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 21, "format": "json"} v 0) v1 -- 0x7f3050070550 con 0x7f3050102ac0 2024-03-04T13:56:35.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.747+0000 7f304d600640 1 -- 172.21.15.107:0/951383788 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 21, "format": "json"}]=0 dumped fsmap epoch 21 v36) v1 ==== 107+0+4090 (secure 0 0 0) 0x7f303c0568e0 con 0x7f3050102ac0 2024-03-04T13:56:35.748 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:56:35.749 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24259,"name":"cephfs.smithi169.bxitzk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.169:6827/3696090493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3696090493},{"type":"v1","addr":"172.21.15.169:6827","nonce":3696090493}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16},{"gid":34432,"name":"cephfs.smithi107.riexmm","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6827/1144672731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1144672731},{"type":"v1","addr":"172.21.15.107:6827","nonce":1144672731}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-04T13:38:30.854004+0000","modified":"2024-03-04T13:52:34.079716+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24247},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24247":{"gid":24247,"name":"cephfs.smithi169.jusudl","rank":0,"incarnation":11,"state":"up:active","state_seq":6,"addr":"172.21.15.169:6825/2994866076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2994866076},{"type":"v1","addr":"172.21.15.169:6825","nonce":2994866076}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-04T13:56:35.749 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 21 2024-03-04T13:56:35.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.749+0000 7f3056e00640 1 -- 172.21.15.107:0/951383788 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f303806d760 msgr2=0x7f303806fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:35.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.749+0000 7f3056e00640 1 --2- 172.21.15.107:0/951383788 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f303806d760 0x7f303806fc10 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3044034040 tx=0x7f3044009da0 comp rx=0 tx=0).stop 2024-03-04T13:56:35.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.749+0000 7f3056e00640 1 -- 172.21.15.107:0/951383788 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3050102ac0 msgr2=0x7f305006fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:35.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.749+0000 7f3056e00640 1 --2- 172.21.15.107:0/951383788 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3050102ac0 0x7f305006fce0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f303c00c790 tx=0x7f303c0043c0 comp rx=0 tx=0).stop 2024-03-04T13:56:35.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.749+0000 7f3056e00640 1 -- 172.21.15.107:0/951383788 shutdown_connections 2024-03-04T13:56:35.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.749+0000 7f3056e00640 1 --2- 172.21.15.107:0/951383788 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f303806d760 0x7f303806fc10 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:35.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.749+0000 7f3056e00640 1 --2- 172.21.15.107:0/951383788 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3050102ac0 0x7f305006fce0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:35.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.749+0000 7f3056e00640 1 --2- 172.21.15.107:0/951383788 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3050102120 0x7f305006f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:35.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.749+0000 7f3056e00640 1 -- 172.21.15.107:0/951383788 >> 172.21.15.107:0/951383788 conn(0x7f30500f7cf0 msgr2=0x7f30500f9780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:56:35.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.749+0000 7f3056e00640 1 -- 172.21.15.107:0/951383788 shutdown_connections 2024-03-04T13:56:35.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.750+0000 7f3056e00640 1 -- 172.21.15.107:0/951383788 wait complete. 2024-03-04T13:56:35.769 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 21 2024-03-04T13:56:35.770 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 22 2024-03-04T13:56:35.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.970+0000 7fd8fba00640 1 -- 172.21.15.107:0/2467970195 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd8f4073570 msgr2=0x7fd8f4073940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:35.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.970+0000 7fd8fba00640 1 --2- 172.21.15.107:0/2467970195 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd8f4073570 0x7fd8f4073940 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd8e40048d0 tx=0x7fd8e4009da0 comp rx=0 tx=0).stop 2024-03-04T13:56:35.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.970+0000 7fd8fba00640 1 -- 172.21.15.107:0/2467970195 shutdown_connections 2024-03-04T13:56:35.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.970+0000 7fd8fba00640 1 --2- 172.21.15.107:0/2467970195 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd8f4073e80 0x7fd8f410cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:35.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.970+0000 7fd8fba00640 1 --2- 172.21.15.107:0/2467970195 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd8f4073570 0x7fd8f4073940 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:35.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.970+0000 7fd8fba00640 1 -- 172.21.15.107:0/2467970195 >> 172.21.15.107:0/2467970195 conn(0x7fd8f40fa160 msgr2=0x7fd8f40fc570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:56:35.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.970+0000 7fd8fba00640 1 -- 172.21.15.107:0/2467970195 shutdown_connections 2024-03-04T13:56:35.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.970+0000 7fd8fba00640 1 -- 172.21.15.107:0/2467970195 wait complete. 2024-03-04T13:56:35.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.971+0000 7fd8fba00640 1 Processor -- start 2024-03-04T13:56:35.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.971+0000 7fd8fba00640 1 -- start start 2024-03-04T13:56:35.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.972+0000 7fd8fba00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd8f4073570 0x7fd8f4119280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:35.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.972+0000 7fd8fba00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd8f4073e80 0x7fd8f41197c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:35.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.972+0000 7fd8fba00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f4119dc0 con 0x7fd8f4073e80 2024-03-04T13:56:35.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.972+0000 7fd8fba00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f4113350 con 0x7fd8f4073570 2024-03-04T13:56:35.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.972+0000 7fd8ebe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd8f4073e80 0x7fd8f41197c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:56:35.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.972+0000 7fd8ebe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd8f4073e80 0x7fd8f41197c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51938/0 (socket says 172.21.15.107:51938) 2024-03-04T13:56:35.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.972+0000 7fd8ebe00640 1 -- 172.21.15.107:0/1824294528 learned_addr learned my addr 172.21.15.107:0/1824294528 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:56:35.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.972+0000 7fd8ebe00640 1 -- 172.21.15.107:0/1824294528 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd8f4073570 msgr2=0x7fd8f4119280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:35.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.972+0000 7fd8ebe00640 1 --2- 172.21.15.107:0/1824294528 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd8f4073570 0x7fd8f4119280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:35.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.972+0000 7fd8ebe00640 1 -- 172.21.15.107:0/1824294528 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8e4005040 con 0x7fd8f4073e80 2024-03-04T13:56:35.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.973+0000 7fd8ebe00640 1 --2- 172.21.15.107:0/1824294528 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd8f4073e80 0x7fd8f41197c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc00c790 tx=0x7fd8dc0043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:56:35.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.973+0000 7fd8f1600640 1 -- 172.21.15.107:0/1824294528 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd8dc00a820 con 0x7fd8f4073e80 2024-03-04T13:56:35.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.973+0000 7fd8f1600640 1 -- 172.21.15.107:0/1824294528 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd8dc00ae50 con 0x7fd8f4073e80 2024-03-04T13:56:35.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.973+0000 7fd8f1600640 1 -- 172.21.15.107:0/1824294528 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd8dc0115e0 con 0x7fd8f4073e80 2024-03-04T13:56:35.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.973+0000 7fd8fba00640 1 -- 172.21.15.107:0/1824294528 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8f4113630 con 0x7fd8f4073e80 2024-03-04T13:56:35.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.973+0000 7fd8fba00640 1 -- 172.21.15.107:0/1824294528 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8f4113b50 con 0x7fd8f4073e80 2024-03-04T13:56:35.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.974+0000 7fd8fba00640 1 -- 172.21.15.107:0/1824294528 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8f4064030 con 0x7fd8f4073e80 2024-03-04T13:56:35.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.975+0000 7fd8f1600640 1 -- 172.21.15.107:0/1824294528 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fd8dc012400 con 0x7fd8f4073e80 2024-03-04T13:56:35.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.976+0000 7fd8f1600640 1 --2- 172.21.15.107:0/1824294528 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fd8d806d750 0x7fd8d806fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:35.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.976+0000 7fd8f1600640 1 -- 172.21.15.107:0/1824294528 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fd8dc08d060 con 0x7fd8f4073e80 2024-03-04T13:56:35.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.977+0000 7fd8f9000640 1 --2- 172.21.15.107:0/1824294528 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fd8d806d750 0x7fd8d806fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:56:35.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.977+0000 7fd8f9000640 1 --2- 172.21.15.107:0/1824294528 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fd8d806d750 0x7fd8d806fc00 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd8e4034040 tx=0x7fd8e4009da0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:56:35.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:35.981+0000 7fd8f1600640 1 -- 172.21.15.107:0/1824294528 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd8dc0568e0 con 0x7fd8f4073e80 2024-03-04T13:56:36.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:36.217+0000 7fd8fba00640 1 -- 172.21.15.107:0/1824294528 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 22, "format": "json"} v 0) v1 -- 0x7fd8f4114880 con 0x7fd8f4073e80 2024-03-04T13:56:36.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:36.219+0000 7fd8f1600640 1 -- 172.21.15.107:0/1824294528 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 22, "format": "json"}]=0 dumped fsmap epoch 22 v36) v1 ==== 107+0+4906 (secure 0 0 0) 0x7fd8dc059fa0 con 0x7fd8f4073e80 2024-03-04T13:56:36.220 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:56:36.221 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24259,"name":"cephfs.smithi169.bxitzk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.169:6827/3696090493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3696090493},{"type":"v1","addr":"172.21.15.169:6827","nonce":3696090493}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16},{"gid":34432,"name":"cephfs.smithi107.riexmm","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6827/1144672731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1144672731},{"type":"v1","addr":"172.21.15.107:6827","nonce":1144672731}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":34436,"name":"cephfs.smithi107.thycho","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6829/3010096376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3010096376},{"type":"v1","addr":"172.21.15.107:6829","nonce":3010096376}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-04T13:38:30.854004+0000","modified":"2024-03-04T13:52:34.079716+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24247},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24247":{"gid":24247,"name":"cephfs.smithi169.jusudl","rank":0,"incarnation":11,"state":"up:active","state_seq":6,"addr":"172.21.15.169:6825/2994866076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":2994866076},{"type":"v1","addr":"172.21.15.169:6825","nonce":2994866076}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-04T13:56:36.221 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 22 2024-03-04T13:56:36.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:36.221+0000 7fd8fba00640 1 -- 172.21.15.107:0/1824294528 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fd8d806d750 msgr2=0x7fd8d806fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:36.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:36.221+0000 7fd8fba00640 1 --2- 172.21.15.107:0/1824294528 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fd8d806d750 0x7fd8d806fc00 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd8e4034040 tx=0x7fd8e4009da0 comp rx=0 tx=0).stop 2024-03-04T13:56:36.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:36.222+0000 7fd8fba00640 1 -- 172.21.15.107:0/1824294528 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd8f4073e80 msgr2=0x7fd8f41197c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:36.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:36.222+0000 7fd8fba00640 1 --2- 172.21.15.107:0/1824294528 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd8f4073e80 0x7fd8f41197c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc00c790 tx=0x7fd8dc0043c0 comp rx=0 tx=0).stop 2024-03-04T13:56:36.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:36.222+0000 7fd8fba00640 1 -- 172.21.15.107:0/1824294528 shutdown_connections 2024-03-04T13:56:36.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:36.222+0000 7fd8fba00640 1 --2- 172.21.15.107:0/1824294528 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fd8d806d750 0x7fd8d806fc00 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:36.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:36.222+0000 7fd8fba00640 1 --2- 172.21.15.107:0/1824294528 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd8f4073e80 0x7fd8f41197c0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:36.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:36.222+0000 7fd8fba00640 1 --2- 172.21.15.107:0/1824294528 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd8f4073570 0x7fd8f4119280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:36.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:36.222+0000 7fd8fba00640 1 -- 172.21.15.107:0/1824294528 >> 172.21.15.107:0/1824294528 conn(0x7fd8f40fa160 msgr2=0x7fd8f40fbeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:56:36.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:36.222+0000 7fd8fba00640 1 -- 172.21.15.107:0/1824294528 shutdown_connections 2024-03-04T13:56:36.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:36.222+0000 7fd8fba00640 1 -- 172.21.15.107:0/1824294528 wait complete. 2024-03-04T13:56:36.241 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 22 2024-03-04T13:56:36.241 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 23 2024-03-04T13:56:36.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:36.444+0000 7f0635e00640 1 -- 172.21.15.107:0/2217099226 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06300fe9c0 msgr2=0x7f06300fed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:36.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:36.444+0000 7f0635e00640 1 --2- 172.21.15.107:0/2217099226 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06300fe9c0 0x7f06300fed90 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f06200048d0 tx=0x7f0620009da0 comp rx=0 tx=0).stop 2024-03-04T13:56:36.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:36.445+0000 7f0635e00640 1 -- 172.21.15.107:0/2217099226 shutdown_connections 2024-03-04T13:56:36.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:36.445+0000 7f0635e00640 1 --2- 172.21.15.107:0/2217099226 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0630100110 0x7f0630100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:36.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:36.445+0000 7f0635e00640 1 --2- 172.21.15.107:0/2217099226 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06300fe9c0 0x7f06300fed90 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:36.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:36.445+0000 7f0635e00640 1 -- 172.21.15.107:0/2217099226 >> 172.21.15.107:0/2217099226 conn(0x7f06300fa160 msgr2=0x7f06300fc570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:56:36.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:36.445+0000 7f0635e00640 1 -- 172.21.15.107:0/2217099226 shutdown_connections 2024-03-04T13:56:36.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:36.445+0000 7f0635e00640 1 -- 172.21.15.107:0/2217099226 wait complete. 2024-03-04T13:56:36.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:36.445+0000 7f0635e00640 1 Processor -- start 2024-03-04T13:56:36.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:36.446+0000 7f0635e00640 1 -- start start 2024-03-04T13:56:36.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:36.446+0000 7f0635e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06300fe9c0 0x7f0630114e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:36.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:36.446+0000 7f0635e00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0630100110 0x7f0630115380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:36.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:36.446+0000 7f0635e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0630115a10 con 0x7f06300fe9c0 2024-03-04T13:56:36.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:36.446+0000 7f0635e00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f063010ef10 con 0x7f0630100110 2024-03-04T13:56:36.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:36.446+0000 7f062ea00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06300fe9c0 0x7f0630114e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:56:36.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:36.446+0000 7f062ea00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06300fe9c0 0x7f0630114e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51954/0 (socket says 172.21.15.107:51954) 2024-03-04T13:56:36.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:36.446+0000 7f062ea00640 1 -- 172.21.15.107:0/1574691406 learned_addr learned my addr 172.21.15.107:0/1574691406 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:56:36.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:36.446+0000 7f062ea00640 1 -- 172.21.15.107:0/1574691406 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0630100110 msgr2=0x7f0630115380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:36.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:36.446+0000 7f062ea00640 1 --2- 172.21.15.107:0/1574691406 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0630100110 0x7f0630115380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:36.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:36.447+0000 7f062ea00640 1 -- 172.21.15.107:0/1574691406 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0620005040 con 0x7f06300fe9c0 2024-03-04T13:56:36.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:36.447+0000 7f062ea00640 1 --2- 172.21.15.107:0/1574691406 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06300fe9c0 0x7f0630114e40 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f062002bda0 tx=0x7f0620009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:56:36.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:36.447+0000 7f0626a00640 1 -- 172.21.15.107:0/1574691406 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f06200094c0 con 0x7f06300fe9c0 2024-03-04T13:56:36.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:36.447+0000 7f0626a00640 1 -- 172.21.15.107:0/1574691406 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0620009620 con 0x7f06300fe9c0 2024-03-04T13:56:36.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:36.447+0000 7f0626a00640 1 -- 172.21.15.107:0/1574691406 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f06200435f0 con 0x7f06300fe9c0 2024-03-04T13:56:36.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:36.447+0000 7f0635e00640 1 -- 172.21.15.107:0/1574691406 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f063010f190 con 0x7f06300fe9c0 2024-03-04T13:56:36.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:36.447+0000 7f0635e00640 1 -- 172.21.15.107:0/1574691406 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f063010f600 con 0x7f06300fe9c0 2024-03-04T13:56:36.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:36.449+0000 7f0635e00640 1 -- 172.21.15.107:0/1574691406 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0630064030 con 0x7f06300fe9c0 2024-03-04T13:56:36.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:36.449+0000 7f0626a00640 1 -- 172.21.15.107:0/1574691406 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f062002f070 con 0x7f06300fe9c0 2024-03-04T13:56:36.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:36.450+0000 7f0626a00640 1 --2- 172.21.15.107:0/1574691406 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f061406d760 0x7f061406fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:36.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:36.450+0000 7f0626a00640 1 -- 172.21.15.107:0/1574691406 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f06200aeec0 con 0x7f06300fe9c0 2024-03-04T13:56:36.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:36.451+0000 7f0627e00640 1 --2- 172.21.15.107:0/1574691406 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f061406d760 0x7f061406fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:56:36.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:36.451+0000 7f0627e00640 1 --2- 172.21.15.107:0/1574691406 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f061406d760 0x7f061406fc10 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0630110700 tx=0x7f0618006d20 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:56:36.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:36.455+0000 7f0626a00640 1 -- 172.21.15.107:0/1574691406 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f06200786c0 con 0x7f06300fe9c0 2024-03-04T13:56:36.595 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:56:36 smithi107 ceph-mon[112578]: from='client.? 172.21.15.107:0/1680122752' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 20, "format": "json"}]: dispatch 2024-03-04T13:56:36.595 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:56:36 smithi107 ceph-mon[112578]: from='client.? 172.21.15.107:0/951383788' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 21, "format": "json"}]: dispatch 2024-03-04T13:56:36.595 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:56:36 smithi107 ceph-mon[112578]: from='client.? 172.21.15.107:0/1824294528' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-03-04T13:56:36.614 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:56:36 smithi169 ceph-mon[86183]: from='client.? 172.21.15.107:0/1680122752' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 20, "format": "json"}]: dispatch 2024-03-04T13:56:36.614 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:56:36 smithi169 ceph-mon[86183]: from='client.? 172.21.15.107:0/951383788' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 21, "format": "json"}]: dispatch 2024-03-04T13:56:36.614 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:56:36 smithi169 ceph-mon[86183]: from='client.? 172.21.15.107:0/1824294528' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-03-04T13:56:36.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:36.691+0000 7f0635e00640 1 -- 172.21.15.107:0/1574691406 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 23, "format": "json"} v 0) v1 -- 0x7f0630110440 con 0x7f06300fe9c0 2024-03-04T13:56:36.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:36.693+0000 7f0626a00640 1 -- 172.21.15.107:0/1574691406 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 23, "format": "json"}]=0 dumped fsmap epoch 23 v36) v1 ==== 107+0+4114 (secure 0 0 0) 0x7f062007bd80 con 0x7f06300fe9c0 2024-03-04T13:56:36.694 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:56:36.694 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24259,"name":"cephfs.smithi169.bxitzk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.169:6827/3696090493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3696090493},{"type":"v1","addr":"172.21.15.169:6827","nonce":3696090493}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16},{"gid":34432,"name":"cephfs.smithi107.riexmm","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6827/1144672731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1144672731},{"type":"v1","addr":"172.21.15.107:6827","nonce":1144672731}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":34436,"name":"cephfs.smithi107.thycho","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6829/3010096376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3010096376},{"type":"v1","addr":"172.21.15.107:6829","nonce":3010096376}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-04T13:38:30.854004+0000","modified":"2024-03-04T13:53:29.274013+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-04T13:56:36.695 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 23 2024-03-04T13:56:36.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:36.695+0000 7f0635e00640 1 -- 172.21.15.107:0/1574691406 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f061406d760 msgr2=0x7f061406fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:36.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:36.695+0000 7f0635e00640 1 --2- 172.21.15.107:0/1574691406 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f061406d760 0x7f061406fc10 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0630110700 tx=0x7f0618006d20 comp rx=0 tx=0).stop 2024-03-04T13:56:36.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:36.695+0000 7f0635e00640 1 -- 172.21.15.107:0/1574691406 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06300fe9c0 msgr2=0x7f0630114e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:36.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:36.695+0000 7f0635e00640 1 --2- 172.21.15.107:0/1574691406 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06300fe9c0 0x7f0630114e40 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f062002bda0 tx=0x7f0620009da0 comp rx=0 tx=0).stop 2024-03-04T13:56:36.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:36.695+0000 7f0635e00640 1 -- 172.21.15.107:0/1574691406 shutdown_connections 2024-03-04T13:56:36.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:36.695+0000 7f0635e00640 1 --2- 172.21.15.107:0/1574691406 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f061406d760 0x7f061406fc10 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:36.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:36.695+0000 7f0635e00640 1 --2- 172.21.15.107:0/1574691406 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0630100110 0x7f0630115380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:36.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:36.695+0000 7f0635e00640 1 --2- 172.21.15.107:0/1574691406 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06300fe9c0 0x7f0630114e40 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:36.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:36.695+0000 7f0635e00640 1 -- 172.21.15.107:0/1574691406 >> 172.21.15.107:0/1574691406 conn(0x7f06300fa160 msgr2=0x7f06300fbcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:56:36.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:36.696+0000 7f0635e00640 1 -- 172.21.15.107:0/1574691406 shutdown_connections 2024-03-04T13:56:36.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:36.696+0000 7f0635e00640 1 -- 172.21.15.107:0/1574691406 wait complete. 2024-03-04T13:56:36.716 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 23 2024-03-04T13:56:36.716 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 24 2024-03-04T13:56:36.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:36.921+0000 7fd65c800640 1 -- 172.21.15.107:0/2259745040 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd6540fd330 msgr2=0x7fd6540fd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:36.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:36.921+0000 7fd65c800640 1 --2- 172.21.15.107:0/2259745040 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd6540fd330 0x7fd6540fd700 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd648004900 tx=0x7fd648009da0 comp rx=0 tx=0).stop 2024-03-04T13:56:36.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:36.921+0000 7fd65c800640 1 -- 172.21.15.107:0/2259745040 shutdown_connections 2024-03-04T13:56:36.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:36.921+0000 7fd65c800640 1 --2- 172.21.15.107:0/2259745040 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6540fdcd0 0x7fd6541063a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:36.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:36.921+0000 7fd65c800640 1 --2- 172.21.15.107:0/2259745040 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd6540fd330 0x7fd6540fd700 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:36.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:36.921+0000 7fd65c800640 1 -- 172.21.15.107:0/2259745040 >> 172.21.15.107:0/2259745040 conn(0x7fd6540f6e30 msgr2=0x7fd6540f9240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:56:36.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:36.921+0000 7fd65c800640 1 -- 172.21.15.107:0/2259745040 shutdown_connections 2024-03-04T13:56:36.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:36.921+0000 7fd65c800640 1 -- 172.21.15.107:0/2259745040 wait complete. 2024-03-04T13:56:36.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:36.922+0000 7fd65c800640 1 Processor -- start 2024-03-04T13:56:36.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:36.922+0000 7fd65c800640 1 -- start start 2024-03-04T13:56:36.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:36.922+0000 7fd65c800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6540fd330 0x7fd654114e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:36.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:36.922+0000 7fd65c800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd6540fdcd0 0x7fd6541153a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:36.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:36.922+0000 7fd65c800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6541159a0 con 0x7fd6540fdcd0 2024-03-04T13:56:36.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:36.922+0000 7fd65c800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd65410ef30 con 0x7fd6540fd330 2024-03-04T13:56:36.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:36.923+0000 7fd659400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd6540fdcd0 0x7fd6541153a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:56:36.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:36.923+0000 7fd659400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd6540fdcd0 0x7fd6541153a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51978/0 (socket says 172.21.15.107:51978) 2024-03-04T13:56:36.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:36.923+0000 7fd659400640 1 -- 172.21.15.107:0/3685446389 learned_addr learned my addr 172.21.15.107:0/3685446389 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:56:36.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:36.923+0000 7fd659400640 1 -- 172.21.15.107:0/3685446389 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6540fd330 msgr2=0x7fd654114e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:36.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:36.923+0000 7fd659400640 1 --2- 172.21.15.107:0/3685446389 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6540fd330 0x7fd654114e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:36.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:36.923+0000 7fd659400640 1 -- 172.21.15.107:0/3685446389 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd648005040 con 0x7fd6540fdcd0 2024-03-04T13:56:36.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:36.923+0000 7fd659400640 1 --2- 172.21.15.107:0/3685446389 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd6540fdcd0 0x7fd6541153a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd64400c790 tx=0x7fd6440043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:56:36.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:36.924+0000 7fd64e000640 1 -- 172.21.15.107:0/3685446389 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd64400a820 con 0x7fd6540fdcd0 2024-03-04T13:56:36.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:36.924+0000 7fd64e000640 1 -- 172.21.15.107:0/3685446389 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd64400ae50 con 0x7fd6540fdcd0 2024-03-04T13:56:36.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:36.924+0000 7fd65c800640 1 -- 172.21.15.107:0/3685446389 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd65410f210 con 0x7fd6540fdcd0 2024-03-04T13:56:36.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:36.924+0000 7fd64e000640 1 -- 172.21.15.107:0/3685446389 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd6440115e0 con 0x7fd6540fdcd0 2024-03-04T13:56:36.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:36.924+0000 7fd65c800640 1 -- 172.21.15.107:0/3685446389 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd65410f730 con 0x7fd6540fdcd0 2024-03-04T13:56:36.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:36.925+0000 7fd65c800640 1 -- 172.21.15.107:0/3685446389 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd654064480 con 0x7fd6540fdcd0 2024-03-04T13:56:36.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:36.926+0000 7fd64e000640 1 -- 172.21.15.107:0/3685446389 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fd644012400 con 0x7fd6540fdcd0 2024-03-04T13:56:36.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:36.927+0000 7fd64e000640 1 --2- 172.21.15.107:0/3685446389 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fd63c06d760 0x7fd63c06fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:36.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:36.927+0000 7fd64e000640 1 -- 172.21.15.107:0/3685446389 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fd64408d090 con 0x7fd6540fdcd0 2024-03-04T13:56:36.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:36.927+0000 7fd659e00640 1 --2- 172.21.15.107:0/3685446389 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fd63c06d760 0x7fd63c06fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:56:36.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:36.928+0000 7fd659e00640 1 --2- 172.21.15.107:0/3685446389 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fd63c06d760 0x7fd63c06fc10 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd648034040 tx=0x7fd648009da0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:56:36.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:36.933+0000 7fd64e000640 1 -- 172.21.15.107:0/3685446389 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd644056910 con 0x7fd6540fdcd0 2024-03-04T13:56:37.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:37.168+0000 7fd65c800640 1 -- 172.21.15.107:0/3685446389 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 24, "format": "json"} v 0) v1 -- 0x7fd654110460 con 0x7fd6540fdcd0 2024-03-04T13:56:37.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:37.169+0000 7fd64e000640 1 -- 172.21.15.107:0/3685446389 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 24, "format": "json"}]=0 dumped fsmap epoch 24 v36) v1 ==== 107+0+4125 (secure 0 0 0) 0x7fd644059fd0 con 0x7fd6540fdcd0 2024-03-04T13:56:37.171 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:56:37.171 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34432,"name":"cephfs.smithi107.riexmm","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6827/1144672731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1144672731},{"type":"v1","addr":"172.21.15.107:6827","nonce":1144672731}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":34436,"name":"cephfs.smithi107.thycho","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6829/3010096376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3010096376},{"type":"v1","addr":"172.21.15.107:6829","nonce":3010096376}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-04T13:38:30.854004+0000","modified":"2024-03-04T13:53:29.285057+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24259},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24259":{"gid":24259,"name":"cephfs.smithi169.bxitzk","rank":0,"incarnation":24,"state":"up:replay","state_seq":2,"addr":"172.21.15.169:6827/3696090493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3696090493},{"type":"v1","addr":"172.21.15.169:6827","nonce":3696090493}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-04T13:56:37.171 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 24 2024-03-04T13:56:37.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:37.171+0000 7fd65c800640 1 -- 172.21.15.107:0/3685446389 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fd63c06d760 msgr2=0x7fd63c06fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:37.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:37.171+0000 7fd65c800640 1 --2- 172.21.15.107:0/3685446389 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fd63c06d760 0x7fd63c06fc10 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd648034040 tx=0x7fd648009da0 comp rx=0 tx=0).stop 2024-03-04T13:56:37.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:37.171+0000 7fd65c800640 1 -- 172.21.15.107:0/3685446389 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd6540fdcd0 msgr2=0x7fd6541153a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:37.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:37.171+0000 7fd65c800640 1 --2- 172.21.15.107:0/3685446389 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd6540fdcd0 0x7fd6541153a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd64400c790 tx=0x7fd6440043c0 comp rx=0 tx=0).stop 2024-03-04T13:56:37.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:37.172+0000 7fd65c800640 1 -- 172.21.15.107:0/3685446389 shutdown_connections 2024-03-04T13:56:37.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:37.172+0000 7fd65c800640 1 --2- 172.21.15.107:0/3685446389 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fd63c06d760 0x7fd63c06fc10 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:37.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:37.172+0000 7fd65c800640 1 --2- 172.21.15.107:0/3685446389 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd6540fdcd0 0x7fd6541153a0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:37.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:37.172+0000 7fd65c800640 1 --2- 172.21.15.107:0/3685446389 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6540fd330 0x7fd654114e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:37.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:37.172+0000 7fd65c800640 1 -- 172.21.15.107:0/3685446389 >> 172.21.15.107:0/3685446389 conn(0x7fd6540f6e30 msgr2=0x7fd6540fb990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:56:37.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:37.172+0000 7fd65c800640 1 -- 172.21.15.107:0/3685446389 shutdown_connections 2024-03-04T13:56:37.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:37.172+0000 7fd65c800640 1 -- 172.21.15.107:0/3685446389 wait complete. 2024-03-04T13:56:37.191 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 24 2024-03-04T13:56:37.191 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 25 2024-03-04T13:56:37.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:37.388+0000 7f7b60200640 1 -- 172.21.15.107:0/3411893835 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b58073570 msgr2=0x7f7b58073940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:37.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:37.388+0000 7f7b60200640 1 --2- 172.21.15.107:0/3411893835 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b58073570 0x7f7b58073940 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c0048d0 tx=0x7f7b4c009da0 comp rx=0 tx=0).stop 2024-03-04T13:56:37.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:37.388+0000 7f7b60200640 1 -- 172.21.15.107:0/3411893835 shutdown_connections 2024-03-04T13:56:37.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:37.388+0000 7f7b60200640 1 --2- 172.21.15.107:0/3411893835 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b58073e80 0x7f7b5810cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:37.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:37.388+0000 7f7b60200640 1 --2- 172.21.15.107:0/3411893835 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b58073570 0x7f7b58073940 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:37.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:37.388+0000 7f7b60200640 1 -- 172.21.15.107:0/3411893835 >> 172.21.15.107:0/3411893835 conn(0x7f7b580fa160 msgr2=0x7f7b580fc570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:56:37.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:37.388+0000 7f7b60200640 1 -- 172.21.15.107:0/3411893835 shutdown_connections 2024-03-04T13:56:37.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:37.388+0000 7f7b60200640 1 -- 172.21.15.107:0/3411893835 wait complete. 2024-03-04T13:56:37.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:37.389+0000 7f7b60200640 1 Processor -- start 2024-03-04T13:56:37.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:37.389+0000 7f7b60200640 1 -- start start 2024-03-04T13:56:37.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:37.389+0000 7f7b60200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b58073570 0x7f7b581192e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:37.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:37.389+0000 7f7b60200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b58073e80 0x7f7b58119820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:37.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:37.389+0000 7f7b60200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b58119eb0 con 0x7f7b58073e80 2024-03-04T13:56:37.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:37.389+0000 7f7b60200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b581133b0 con 0x7f7b58073570 2024-03-04T13:56:37.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:37.390+0000 7f7b5ce00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b58073e80 0x7f7b58119820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:56:37.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:37.390+0000 7f7b5ce00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b58073e80 0x7f7b58119820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52000/0 (socket says 172.21.15.107:52000) 2024-03-04T13:56:37.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:37.390+0000 7f7b5ce00640 1 -- 172.21.15.107:0/187240943 learned_addr learned my addr 172.21.15.107:0/187240943 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:56:37.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:37.390+0000 7f7b5d800640 1 --2- 172.21.15.107:0/187240943 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b58073570 0x7f7b581192e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:56:37.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:37.390+0000 7f7b5ce00640 1 -- 172.21.15.107:0/187240943 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b58073570 msgr2=0x7f7b581192e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:37.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:37.390+0000 7f7b5ce00640 1 --2- 172.21.15.107:0/187240943 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b58073570 0x7f7b581192e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:37.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:37.390+0000 7f7b5ce00640 1 -- 172.21.15.107:0/187240943 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b4c005040 con 0x7f7b58073e80 2024-03-04T13:56:37.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:37.390+0000 7f7b5ce00640 1 --2- 172.21.15.107:0/187240943 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b58073e80 0x7f7b58119820 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7b4400b790 tx=0x7f7b440091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:56:37.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:37.391+0000 7f7b55600640 1 -- 172.21.15.107:0/187240943 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b4400c700 con 0x7f7b58073e80 2024-03-04T13:56:37.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:37.391+0000 7f7b55600640 1 -- 172.21.15.107:0/187240943 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7b4400cd30 con 0x7f7b58073e80 2024-03-04T13:56:37.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:37.391+0000 7f7b55600640 1 -- 172.21.15.107:0/187240943 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b44013370 con 0x7f7b58073e80 2024-03-04T13:56:37.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:37.391+0000 7f7b60200640 1 -- 172.21.15.107:0/187240943 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b58113690 con 0x7f7b58073e80 2024-03-04T13:56:37.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:37.391+0000 7f7b60200640 1 -- 172.21.15.107:0/187240943 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b58113bb0 con 0x7f7b58073e80 2024-03-04T13:56:37.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:37.392+0000 7f7b60200640 1 -- 172.21.15.107:0/187240943 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b58064030 con 0x7f7b58073e80 2024-03-04T13:56:37.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:37.393+0000 7f7b55600640 1 -- 172.21.15.107:0/187240943 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f7b44012070 con 0x7f7b58073e80 2024-03-04T13:56:37.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:37.394+0000 7f7b55600640 1 --2- 172.21.15.107:0/187240943 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f7b4006d760 0x7f7b4006fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:37.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:37.394+0000 7f7b55600640 1 -- 172.21.15.107:0/187240943 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f7b4408c810 con 0x7f7b58073e80 2024-03-04T13:56:37.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:37.395+0000 7f7b5d800640 1 --2- 172.21.15.107:0/187240943 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f7b4006d760 0x7f7b4006fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:56:37.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:37.395+0000 7f7b5d800640 1 --2- 172.21.15.107:0/187240943 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f7b4006d760 0x7f7b4006fc10 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c0047f0 tx=0x7f7b4c009da0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:56:37.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:37.399+0000 7f7b55600640 1 -- 172.21.15.107:0/187240943 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7b44056090 con 0x7f7b58073e80 2024-03-04T13:56:37.595 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:56:37 smithi107 ceph-mon[112578]: from='client.? 172.21.15.107:0/1574691406' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-03-04T13:56:37.595 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:56:37 smithi107 ceph-mon[112578]: pgmap v315: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:56:37.595 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:56:37 smithi107 ceph-mon[112578]: from='client.? 172.21.15.107:0/3685446389' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-03-04T13:56:37.612 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:56:37 smithi169 ceph-mon[86183]: from='client.? 172.21.15.107:0/1574691406' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-03-04T13:56:37.612 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:56:37 smithi169 ceph-mon[86183]: pgmap v315: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:56:37.612 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:56:37 smithi169 ceph-mon[86183]: from='client.? 172.21.15.107:0/3685446389' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-03-04T13:56:37.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:37.634+0000 7f7b60200640 1 -- 172.21.15.107:0/187240943 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 25, "format": "json"} v 0) v1 -- 0x7f7b581148e0 con 0x7f7b58073e80 2024-03-04T13:56:37.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:37.635+0000 7f7b55600640 1 -- 172.21.15.107:0/187240943 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 25, "format": "json"}]=0 dumped fsmap epoch 25 v36) v1 ==== 107+0+4941 (secure 0 0 0) 0x7f7b4405ee60 con 0x7f7b58073e80 2024-03-04T13:56:37.637 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:56:37.637 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34432,"name":"cephfs.smithi107.riexmm","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6827/1144672731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1144672731},{"type":"v1","addr":"172.21.15.107:6827","nonce":1144672731}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":34436,"name":"cephfs.smithi107.thycho","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6829/3010096376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3010096376},{"type":"v1","addr":"172.21.15.107:6829","nonce":3010096376}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":44243,"name":"cephfs.smithi169.jusudl","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6825/1336978861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":1336978861},{"type":"v1","addr":"172.21.15.169:6825","nonce":1336978861}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-04T13:38:30.854004+0000","modified":"2024-03-04T13:53:29.285057+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24259},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24259":{"gid":24259,"name":"cephfs.smithi169.bxitzk","rank":0,"incarnation":24,"state":"up:replay","state_seq":2,"addr":"172.21.15.169:6827/3696090493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3696090493},{"type":"v1","addr":"172.21.15.169:6827","nonce":3696090493}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-04T13:56:37.637 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 25 2024-03-04T13:56:37.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:37.638+0000 7f7b60200640 1 -- 172.21.15.107:0/187240943 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f7b4006d760 msgr2=0x7f7b4006fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:37.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:37.638+0000 7f7b60200640 1 --2- 172.21.15.107:0/187240943 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f7b4006d760 0x7f7b4006fc10 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c0047f0 tx=0x7f7b4c009da0 comp rx=0 tx=0).stop 2024-03-04T13:56:37.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:37.638+0000 7f7b60200640 1 -- 172.21.15.107:0/187240943 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b58073e80 msgr2=0x7f7b58119820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:37.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:37.638+0000 7f7b60200640 1 --2- 172.21.15.107:0/187240943 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b58073e80 0x7f7b58119820 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7b4400b790 tx=0x7f7b440091d0 comp rx=0 tx=0).stop 2024-03-04T13:56:37.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:37.638+0000 7f7b60200640 1 -- 172.21.15.107:0/187240943 shutdown_connections 2024-03-04T13:56:37.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:37.638+0000 7f7b60200640 1 --2- 172.21.15.107:0/187240943 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f7b4006d760 0x7f7b4006fc10 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:37.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:37.638+0000 7f7b60200640 1 --2- 172.21.15.107:0/187240943 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b58073e80 0x7f7b58119820 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:37.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:37.638+0000 7f7b60200640 1 --2- 172.21.15.107:0/187240943 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b58073570 0x7f7b581192e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:37.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:37.638+0000 7f7b60200640 1 -- 172.21.15.107:0/187240943 >> 172.21.15.107:0/187240943 conn(0x7f7b580fa160 msgr2=0x7f7b580fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:56:37.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:37.638+0000 7f7b60200640 1 -- 172.21.15.107:0/187240943 shutdown_connections 2024-03-04T13:56:37.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:37.638+0000 7f7b60200640 1 -- 172.21.15.107:0/187240943 wait complete. 2024-03-04T13:56:37.657 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 25 2024-03-04T13:56:37.657 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 26 2024-03-04T13:56:37.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:37.864+0000 7f99de800640 1 -- 172.21.15.107:0/3636584172 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99d8102d20 msgr2=0x7f99d80fc3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:37.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:37.864+0000 7f99de800640 1 --2- 172.21.15.107:0/3636584172 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99d8102d20 0x7f99d80fc3b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f99c00048d0 tx=0x7f99c0009da0 comp rx=0 tx=0).stop 2024-03-04T13:56:37.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:37.864+0000 7f99de800640 1 -- 172.21.15.107:0/3636584172 shutdown_connections 2024-03-04T13:56:37.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:37.864+0000 7f99de800640 1 --2- 172.21.15.107:0/3636584172 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99d8102d20 0x7f99d80fc3b0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:37.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:37.864+0000 7f99de800640 1 --2- 172.21.15.107:0/3636584172 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99d8102380 0x7f99d8102750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:37.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:37.864+0000 7f99de800640 1 -- 172.21.15.107:0/3636584172 >> 172.21.15.107:0/3636584172 conn(0x7f99d80f7f90 msgr2=0x7f99d80fa3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:56:37.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:37.864+0000 7f99de800640 1 -- 172.21.15.107:0/3636584172 shutdown_connections 2024-03-04T13:56:37.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:37.864+0000 7f99de800640 1 -- 172.21.15.107:0/3636584172 wait complete. 2024-03-04T13:56:37.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:37.865+0000 7f99de800640 1 Processor -- start 2024-03-04T13:56:37.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:37.865+0000 7f99de800640 1 -- start start 2024-03-04T13:56:37.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:37.865+0000 7f99de800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99d8102380 0x7f99d8114de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:37.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:37.865+0000 7f99de800640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99d8102d20 0x7f99d8115320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:37.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:37.865+0000 7f99de800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99d8115920 con 0x7f99d8102380 2024-03-04T13:56:37.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:37.865+0000 7f99de800640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99d810eeb0 con 0x7f99d8102d20 2024-03-04T13:56:37.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:37.866+0000 7f99d7400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99d8102380 0x7f99d8114de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:56:37.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:37.866+0000 7f99d7400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99d8102380 0x7f99d8114de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52022/0 (socket says 172.21.15.107:52022) 2024-03-04T13:56:37.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:37.866+0000 7f99d7400640 1 -- 172.21.15.107:0/3616750900 learned_addr learned my addr 172.21.15.107:0/3616750900 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:56:37.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:37.866+0000 7f99cfe00640 1 --2- 172.21.15.107:0/3616750900 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99d8102d20 0x7f99d8115320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:56:37.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:37.866+0000 7f99d7400640 1 -- 172.21.15.107:0/3616750900 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99d8102d20 msgr2=0x7f99d8115320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:37.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:37.866+0000 7f99d7400640 1 --2- 172.21.15.107:0/3616750900 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99d8102d20 0x7f99d8115320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:37.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:37.866+0000 7f99d7400640 1 -- 172.21.15.107:0/3616750900 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99c0005040 con 0x7f99d8102380 2024-03-04T13:56:37.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:37.866+0000 7f99cfe00640 1 --2- 172.21.15.107:0/3616750900 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99d8102d20 0x7f99d8115320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T13:56:37.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:37.866+0000 7f99d7400640 1 --2- 172.21.15.107:0/3616750900 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99d8102380 0x7f99d8114de0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f99c800c790 tx=0x7f99c80043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:56:37.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:37.867+0000 7f99cf400640 1 -- 172.21.15.107:0/3616750900 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f99c800a820 con 0x7f99d8102380 2024-03-04T13:56:37.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:37.867+0000 7f99cf400640 1 -- 172.21.15.107:0/3616750900 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f99c800ae50 con 0x7f99d8102380 2024-03-04T13:56:37.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:37.867+0000 7f99de800640 1 -- 172.21.15.107:0/3616750900 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99d810f190 con 0x7f99d8102380 2024-03-04T13:56:37.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:37.867+0000 7f99cf400640 1 -- 172.21.15.107:0/3616750900 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f99c80115e0 con 0x7f99d8102380 2024-03-04T13:56:37.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:37.867+0000 7f99de800640 1 -- 172.21.15.107:0/3616750900 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99d810f6b0 con 0x7f99d8102380 2024-03-04T13:56:37.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:37.868+0000 7f99de800640 1 -- 172.21.15.107:0/3616750900 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99d80637a0 con 0x7f99d8102380 2024-03-04T13:56:37.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:37.869+0000 7f99cf400640 1 -- 172.21.15.107:0/3616750900 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f99c8011740 con 0x7f99d8102380 2024-03-04T13:56:37.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:37.870+0000 7f99cf400640 1 --2- 172.21.15.107:0/3616750900 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f99bc06d760 0x7f99bc06fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:37.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:37.870+0000 7f99cf400640 1 -- 172.21.15.107:0/3616750900 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f99c808d160 con 0x7f99d8102380 2024-03-04T13:56:37.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:37.870+0000 7f99cfe00640 1 --2- 172.21.15.107:0/3616750900 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f99bc06d760 0x7f99bc06fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:56:37.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:37.871+0000 7f99cfe00640 1 --2- 172.21.15.107:0/3616750900 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f99bc06d760 0x7f99bc06fc10 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f99c002bda0 tx=0x7f99c0009da0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:56:37.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:37.875+0000 7f99cf400640 1 -- 172.21.15.107:0/3616750900 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f99c8056960 con 0x7f99d8102380 2024-03-04T13:56:38.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:38.110+0000 7f99de800640 1 -- 172.21.15.107:0/3616750900 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 26, "format": "json"} v 0) v1 -- 0x7f99d810f930 con 0x7f99d8102380 2024-03-04T13:56:38.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:38.112+0000 7f99cf400640 1 -- 172.21.15.107:0/3616750900 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 26, "format": "json"}]=0 dumped fsmap epoch 26 v36) v1 ==== 107+0+4946 (secure 0 0 0) 0x7f99c805a020 con 0x7f99d8102380 2024-03-04T13:56:38.113 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:56:38.114 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34432,"name":"cephfs.smithi107.riexmm","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6827/1144672731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1144672731},{"type":"v1","addr":"172.21.15.107:6827","nonce":1144672731}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":34436,"name":"cephfs.smithi107.thycho","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6829/3010096376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3010096376},{"type":"v1","addr":"172.21.15.107:6829","nonce":3010096376}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":44243,"name":"cephfs.smithi169.jusudl","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6825/1336978861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":1336978861},{"type":"v1","addr":"172.21.15.169:6825","nonce":1336978861}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-04T13:38:30.854004+0000","modified":"2024-03-04T13:53:38.706516+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24259},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24259":{"gid":24259,"name":"cephfs.smithi169.bxitzk","rank":0,"incarnation":24,"state":"up:reconnect","state_seq":225,"addr":"172.21.15.169:6827/3696090493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3696090493},{"type":"v1","addr":"172.21.15.169:6827","nonce":3696090493}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-04T13:56:38.114 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 26 2024-03-04T13:56:38.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:38.114+0000 7f99de800640 1 -- 172.21.15.107:0/3616750900 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f99bc06d760 msgr2=0x7f99bc06fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:38.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:38.114+0000 7f99de800640 1 --2- 172.21.15.107:0/3616750900 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f99bc06d760 0x7f99bc06fc10 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f99c002bda0 tx=0x7f99c0009da0 comp rx=0 tx=0).stop 2024-03-04T13:56:38.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:38.114+0000 7f99de800640 1 -- 172.21.15.107:0/3616750900 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99d8102380 msgr2=0x7f99d8114de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:38.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:38.114+0000 7f99de800640 1 --2- 172.21.15.107:0/3616750900 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99d8102380 0x7f99d8114de0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f99c800c790 tx=0x7f99c80043c0 comp rx=0 tx=0).stop 2024-03-04T13:56:38.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:38.115+0000 7f99de800640 1 -- 172.21.15.107:0/3616750900 shutdown_connections 2024-03-04T13:56:38.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:38.115+0000 7f99de800640 1 --2- 172.21.15.107:0/3616750900 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f99bc06d760 0x7f99bc06fc10 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:38.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:38.115+0000 7f99de800640 1 --2- 172.21.15.107:0/3616750900 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99d8102d20 0x7f99d8115320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:38.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:38.115+0000 7f99de800640 1 --2- 172.21.15.107:0/3616750900 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99d8102380 0x7f99d8114de0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:38.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:38.115+0000 7f99de800640 1 -- 172.21.15.107:0/3616750900 >> 172.21.15.107:0/3616750900 conn(0x7f99d80f7f90 msgr2=0x7f99d806fa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:56:38.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:38.115+0000 7f99de800640 1 -- 172.21.15.107:0/3616750900 shutdown_connections 2024-03-04T13:56:38.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:38.115+0000 7f99de800640 1 -- 172.21.15.107:0/3616750900 wait complete. 2024-03-04T13:56:38.133 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 26 2024-03-04T13:56:38.133 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 27 2024-03-04T13:56:38.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:38.333+0000 7ff1a4200640 1 -- 172.21.15.107:0/2390609794 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff19c104580 msgr2=0x7ff19c104950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:38.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:38.333+0000 7ff1a4200640 1 --2- 172.21.15.107:0/2390609794 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff19c104580 0x7ff19c104950 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff1900048d0 tx=0x7ff190009da0 comp rx=0 tx=0).stop 2024-03-04T13:56:38.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:38.333+0000 7ff1a4200640 1 -- 172.21.15.107:0/2390609794 shutdown_connections 2024-03-04T13:56:38.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:38.333+0000 7ff1a4200640 1 --2- 172.21.15.107:0/2390609794 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff19c104e90 0x7ff19c0fe5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:38.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:38.333+0000 7ff1a4200640 1 --2- 172.21.15.107:0/2390609794 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff19c104580 0x7ff19c104950 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:38.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:38.333+0000 7ff1a4200640 1 -- 172.21.15.107:0/2390609794 >> 172.21.15.107:0/2390609794 conn(0x7ff19c0fa160 msgr2=0x7ff19c0fc570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:56:38.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:38.333+0000 7ff1a4200640 1 -- 172.21.15.107:0/2390609794 shutdown_connections 2024-03-04T13:56:38.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:38.333+0000 7ff1a4200640 1 -- 172.21.15.107:0/2390609794 wait complete. 2024-03-04T13:56:38.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:38.334+0000 7ff1a4200640 1 Processor -- start 2024-03-04T13:56:38.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:38.334+0000 7ff1a4200640 1 -- start start 2024-03-04T13:56:38.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:38.335+0000 7ff1a4200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff19c104580 0x7ff19c114e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:38.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:38.335+0000 7ff1a4200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff19c104e90 0x7ff19c115360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:38.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:38.335+0000 7ff1a4200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff19c115960 con 0x7ff19c104580 2024-03-04T13:56:38.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:38.335+0000 7ff1a4200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff19c10eef0 con 0x7ff19c104e90 2024-03-04T13:56:38.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:38.335+0000 7ff1a1800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff19c104580 0x7ff19c114e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:56:38.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:38.335+0000 7ff1a1800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff19c104580 0x7ff19c114e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47106/0 (socket says 172.21.15.107:47106) 2024-03-04T13:56:38.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:38.335+0000 7ff1a1800640 1 -- 172.21.15.107:0/4188633124 learned_addr learned my addr 172.21.15.107:0/4188633124 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:56:38.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:38.335+0000 7ff1a1800640 1 -- 172.21.15.107:0/4188633124 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff19c104e90 msgr2=0x7ff19c115360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:38.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:38.335+0000 7ff1a1800640 1 --2- 172.21.15.107:0/4188633124 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff19c104e90 0x7ff19c115360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:38.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:38.335+0000 7ff1a1800640 1 -- 172.21.15.107:0/4188633124 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff190005040 con 0x7ff19c104580 2024-03-04T13:56:38.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:38.336+0000 7ff1a1800640 1 --2- 172.21.15.107:0/4188633124 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff19c104580 0x7ff19c114e20 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff190031720 tx=0x7ff190009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:56:38.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:38.336+0000 7ff199600640 1 -- 172.21.15.107:0/4188633124 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff1900094c0 con 0x7ff19c104580 2024-03-04T13:56:38.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:38.336+0000 7ff199600640 1 -- 172.21.15.107:0/4188633124 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff190009620 con 0x7ff19c104580 2024-03-04T13:56:38.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:38.336+0000 7ff199600640 1 -- 172.21.15.107:0/4188633124 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff19003b640 con 0x7ff19c104580 2024-03-04T13:56:38.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:38.336+0000 7ff1a4200640 1 -- 172.21.15.107:0/4188633124 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff19c10f170 con 0x7ff19c104580 2024-03-04T13:56:38.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:38.336+0000 7ff1a4200640 1 -- 172.21.15.107:0/4188633124 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff19c10f690 con 0x7ff19c104580 2024-03-04T13:56:38.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:38.337+0000 7ff1a4200640 1 -- 172.21.15.107:0/4188633124 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff19c064480 con 0x7ff19c104580 2024-03-04T13:56:38.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:38.338+0000 7ff199600640 1 -- 172.21.15.107:0/4188633124 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7ff19002f070 con 0x7ff19c104580 2024-03-04T13:56:38.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:38.339+0000 7ff199600640 1 --2- 172.21.15.107:0/4188633124 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7ff18406d750 0x7ff18406fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:38.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:38.339+0000 7ff199600640 1 -- 172.21.15.107:0/4188633124 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7ff1900af530 con 0x7ff19c104580 2024-03-04T13:56:38.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:38.339+0000 7ff1a0e00640 1 --2- 172.21.15.107:0/4188633124 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7ff18406d750 0x7ff18406fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:56:38.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:38.340+0000 7ff1a0e00640 1 --2- 172.21.15.107:0/4188633124 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7ff18406d750 0x7ff18406fc00 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff19c1106e0 tx=0x7ff188009c50 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:56:38.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:38.344+0000 7ff199600640 1 -- 172.21.15.107:0/4188633124 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff190078cb0 con 0x7ff19c104580 2024-03-04T13:56:38.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:38.560+0000 7ff1a4200640 1 -- 172.21.15.107:0/4188633124 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 27, "format": "json"} v 0) v1 -- 0x7ff19c10f820 con 0x7ff19c104580 2024-03-04T13:56:38.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:38.562+0000 7ff199600640 1 -- 172.21.15.107:0/4188633124 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 27, "format": "json"}]=0 dumped fsmap epoch 27 v36) v1 ==== 107+0+4943 (secure 0 0 0) 0x7ff19007c370 con 0x7ff19c104580 2024-03-04T13:56:38.564 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:56:38.564 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34432,"name":"cephfs.smithi107.riexmm","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6827/1144672731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1144672731},{"type":"v1","addr":"172.21.15.107:6827","nonce":1144672731}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":34436,"name":"cephfs.smithi107.thycho","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6829/3010096376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3010096376},{"type":"v1","addr":"172.21.15.107:6829","nonce":3010096376}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":44243,"name":"cephfs.smithi169.jusudl","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6825/1336978861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":1336978861},{"type":"v1","addr":"172.21.15.169:6825","nonce":1336978861}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-04T13:38:30.854004+0000","modified":"2024-03-04T13:53:39.700946+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24259},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24259":{"gid":24259,"name":"cephfs.smithi169.bxitzk","rank":0,"incarnation":24,"state":"up:rejoin","state_seq":226,"addr":"172.21.15.169:6827/3696090493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3696090493},{"type":"v1","addr":"172.21.15.169:6827","nonce":3696090493}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-04T13:56:38.564 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 27 2024-03-04T13:56:38.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:38.565+0000 7ff1a4200640 1 -- 172.21.15.107:0/4188633124 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7ff18406d750 msgr2=0x7ff18406fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:38.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:38.565+0000 7ff1a4200640 1 --2- 172.21.15.107:0/4188633124 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7ff18406d750 0x7ff18406fc00 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff19c1106e0 tx=0x7ff188009c50 comp rx=0 tx=0).stop 2024-03-04T13:56:38.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:38.565+0000 7ff1a4200640 1 -- 172.21.15.107:0/4188633124 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff19c104580 msgr2=0x7ff19c114e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:38.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:38.565+0000 7ff1a4200640 1 --2- 172.21.15.107:0/4188633124 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff19c104580 0x7ff19c114e20 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff190031720 tx=0x7ff190009da0 comp rx=0 tx=0).stop 2024-03-04T13:56:38.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:38.565+0000 7ff1a4200640 1 -- 172.21.15.107:0/4188633124 shutdown_connections 2024-03-04T13:56:38.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:38.565+0000 7ff1a4200640 1 --2- 172.21.15.107:0/4188633124 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7ff18406d750 0x7ff18406fc00 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:38.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:38.565+0000 7ff1a4200640 1 --2- 172.21.15.107:0/4188633124 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff19c104e90 0x7ff19c115360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:38.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:38.565+0000 7ff1a4200640 1 --2- 172.21.15.107:0/4188633124 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff19c104580 0x7ff19c114e20 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:38.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:38.565+0000 7ff1a4200640 1 -- 172.21.15.107:0/4188633124 >> 172.21.15.107:0/4188633124 conn(0x7ff19c0fa160 msgr2=0x7ff19c0fba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:56:38.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:38.565+0000 7ff1a4200640 1 -- 172.21.15.107:0/4188633124 shutdown_connections 2024-03-04T13:56:38.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:38.565+0000 7ff1a4200640 1 -- 172.21.15.107:0/4188633124 wait complete. 2024-03-04T13:56:38.580 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:56:38 smithi107 ceph-mon[112578]: from='client.? 172.21.15.107:0/187240943' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-03-04T13:56:38.580 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:56:38 smithi107 ceph-mon[112578]: from='client.? 172.21.15.107:0/3616750900' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-03-04T13:56:38.583 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 27 2024-03-04T13:56:38.583 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 28 2024-03-04T13:56:38.612 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:56:38 smithi169 ceph-mon[86183]: from='client.? 172.21.15.107:0/187240943' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-03-04T13:56:38.612 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:56:38 smithi169 ceph-mon[86183]: from='client.? 172.21.15.107:0/3616750900' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-03-04T13:56:38.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:38.781+0000 7fa4e3000640 1 -- 172.21.15.107:0/589698567 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4dc104560 msgr2=0x7fa4dc104930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:38.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:38.781+0000 7fa4e3000640 1 --2- 172.21.15.107:0/589698567 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4dc104560 0x7fa4dc104930 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc009c50 tx=0x7fa4cc004d10 comp rx=0 tx=0).stop 2024-03-04T13:56:38.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:38.781+0000 7fa4e3000640 1 -- 172.21.15.107:0/589698567 shutdown_connections 2024-03-04T13:56:38.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:38.781+0000 7fa4e3000640 1 --2- 172.21.15.107:0/589698567 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa4dc104f00 0x7fa4dc0fe5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:38.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:38.781+0000 7fa4e3000640 1 --2- 172.21.15.107:0/589698567 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4dc104560 0x7fa4dc104930 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:38.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:38.781+0000 7fa4e3000640 1 -- 172.21.15.107:0/589698567 >> 172.21.15.107:0/589698567 conn(0x7fa4dc0fa160 msgr2=0x7fa4dc0fc570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:56:38.784 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:38.782+0000 7fa4e3000640 1 -- 172.21.15.107:0/589698567 shutdown_connections 2024-03-04T13:56:38.784 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:38.782+0000 7fa4e3000640 1 -- 172.21.15.107:0/589698567 wait complete. 2024-03-04T13:56:38.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:38.782+0000 7fa4e3000640 1 Processor -- start 2024-03-04T13:56:38.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:38.782+0000 7fa4e3000640 1 -- start start 2024-03-04T13:56:38.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:38.783+0000 7fa4e3000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa4dc104560 0x7fa4dc114e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:38.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:38.783+0000 7fa4e3000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4dc104f00 0x7fa4dc115360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:38.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:38.783+0000 7fa4e3000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4dc115960 con 0x7fa4dc104f00 2024-03-04T13:56:38.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:38.783+0000 7fa4e3000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4dc10eef0 con 0x7fa4dc104560 2024-03-04T13:56:38.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:38.783+0000 7fa4db400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4dc104f00 0x7fa4dc115360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:56:38.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:38.783+0000 7fa4db400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4dc104f00 0x7fa4dc115360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47118/0 (socket says 172.21.15.107:47118) 2024-03-04T13:56:38.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:38.783+0000 7fa4db400640 1 -- 172.21.15.107:0/295300220 learned_addr learned my addr 172.21.15.107:0/295300220 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:56:38.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:38.783+0000 7fa4dbe00640 1 --2- 172.21.15.107:0/295300220 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa4dc104560 0x7fa4dc114e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:56:38.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:38.784+0000 7fa4db400640 1 -- 172.21.15.107:0/295300220 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa4dc104560 msgr2=0x7fa4dc114e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:38.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:38.784+0000 7fa4db400640 1 --2- 172.21.15.107:0/295300220 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa4dc104560 0x7fa4dc114e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:38.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:38.784+0000 7fa4db400640 1 -- 172.21.15.107:0/295300220 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4cc0098b0 con 0x7fa4dc104f00 2024-03-04T13:56:38.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:38.784+0000 7fa4db400640 1 --2- 172.21.15.107:0/295300220 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4dc104f00 0x7fa4dc115360 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fa4c4002840 tx=0x7fa4c4004570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:56:38.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:38.784+0000 7fa4d3e00640 1 -- 172.21.15.107:0/295300220 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa4c4007c30 con 0x7fa4dc104f00 2024-03-04T13:56:38.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:38.784+0000 7fa4e3000640 1 -- 172.21.15.107:0/295300220 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4dc10f1d0 con 0x7fa4dc104f00 2024-03-04T13:56:38.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:38.784+0000 7fa4d3e00640 1 -- 172.21.15.107:0/295300220 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa4c400f040 con 0x7fa4dc104f00 2024-03-04T13:56:38.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:38.784+0000 7fa4d3e00640 1 -- 172.21.15.107:0/295300220 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa4c4009590 con 0x7fa4dc104f00 2024-03-04T13:56:38.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:38.784+0000 7fa4e3000640 1 -- 172.21.15.107:0/295300220 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4dc10f770 con 0x7fa4dc104f00 2024-03-04T13:56:38.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:38.785+0000 7fa4e3000640 1 -- 172.21.15.107:0/295300220 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4dc064480 con 0x7fa4dc104f00 2024-03-04T13:56:38.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:38.786+0000 7fa4d3e00640 1 -- 172.21.15.107:0/295300220 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fa4c400d6b0 con 0x7fa4dc104f00 2024-03-04T13:56:38.790 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:38.788+0000 7fa4d3e00640 1 --2- 172.21.15.107:0/295300220 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fa4c006d760 0x7fa4c006fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:38.790 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:38.789+0000 7fa4d3e00640 1 -- 172.21.15.107:0/295300220 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fa4c408c7e0 con 0x7fa4dc104f00 2024-03-04T13:56:38.790 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:38.789+0000 7fa4dbe00640 1 --2- 172.21.15.107:0/295300220 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fa4c006d760 0x7fa4c006fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:56:38.791 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:38.789+0000 7fa4dbe00640 1 --2- 172.21.15.107:0/295300220 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fa4c006d760 0x7fa4c006fc10 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc009040 tx=0x7fa4cc02e040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:56:38.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:38.793+0000 7fa4d3e00640 1 -- 172.21.15.107:0/295300220 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa4c4055f60 con 0x7fa4dc104f00 2024-03-04T13:56:39.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.028+0000 7fa4e3000640 1 -- 172.21.15.107:0/295300220 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 28, "format": "json"} v 0) v1 -- 0x7fa4dc10b720 con 0x7fa4dc104f00 2024-03-04T13:56:39.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.030+0000 7fa4d3e00640 1 -- 172.21.15.107:0/295300220 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 28, "format": "json"}]=0 dumped fsmap epoch 28 v36) v1 ==== 107+0+4943 (secure 0 0 0) 0x7fa4c4059620 con 0x7fa4dc104f00 2024-03-04T13:56:39.031 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:56:39.031 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34432,"name":"cephfs.smithi107.riexmm","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6827/1144672731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1144672731},{"type":"v1","addr":"172.21.15.107:6827","nonce":1144672731}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":34436,"name":"cephfs.smithi107.thycho","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6829/3010096376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3010096376},{"type":"v1","addr":"172.21.15.107:6829","nonce":3010096376}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":44243,"name":"cephfs.smithi169.jusudl","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6825/1336978861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":1336978861},{"type":"v1","addr":"172.21.15.169:6825","nonce":1336978861}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-04T13:38:30.854004+0000","modified":"2024-03-04T13:53:40.783016+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24259},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24259":{"gid":24259,"name":"cephfs.smithi169.bxitzk","rank":0,"incarnation":24,"state":"up:active","state_seq":227,"addr":"172.21.15.169:6827/3696090493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":3696090493},{"type":"v1","addr":"172.21.15.169:6827","nonce":3696090493}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-04T13:56:39.032 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 28 2024-03-04T13:56:39.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.032+0000 7fa4e3000640 1 -- 172.21.15.107:0/295300220 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fa4c006d760 msgr2=0x7fa4c006fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:39.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.032+0000 7fa4e3000640 1 --2- 172.21.15.107:0/295300220 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fa4c006d760 0x7fa4c006fc10 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc009040 tx=0x7fa4cc02e040 comp rx=0 tx=0).stop 2024-03-04T13:56:39.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.032+0000 7fa4e3000640 1 -- 172.21.15.107:0/295300220 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4dc104f00 msgr2=0x7fa4dc115360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:39.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.032+0000 7fa4e3000640 1 --2- 172.21.15.107:0/295300220 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4dc104f00 0x7fa4dc115360 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fa4c4002840 tx=0x7fa4c4004570 comp rx=0 tx=0).stop 2024-03-04T13:56:39.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.032+0000 7fa4e3000640 1 -- 172.21.15.107:0/295300220 shutdown_connections 2024-03-04T13:56:39.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.032+0000 7fa4e3000640 1 --2- 172.21.15.107:0/295300220 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fa4c006d760 0x7fa4c006fc10 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:39.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.032+0000 7fa4e3000640 1 --2- 172.21.15.107:0/295300220 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4dc104f00 0x7fa4dc115360 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:39.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.032+0000 7fa4e3000640 1 --2- 172.21.15.107:0/295300220 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa4dc104560 0x7fa4dc114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:39.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.032+0000 7fa4e3000640 1 -- 172.21.15.107:0/295300220 >> 172.21.15.107:0/295300220 conn(0x7fa4dc0fa160 msgr2=0x7fa4dc0fbaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:56:39.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.032+0000 7fa4e3000640 1 -- 172.21.15.107:0/295300220 shutdown_connections 2024-03-04T13:56:39.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.032+0000 7fa4e3000640 1 -- 172.21.15.107:0/295300220 wait complete. 2024-03-04T13:56:39.053 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 28 2024-03-04T13:56:39.053 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 29 2024-03-04T13:56:39.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.264+0000 7fa510200640 1 -- 172.21.15.107:0/1532308008 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa5080719d0 msgr2=0x7fa508071da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:39.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.264+0000 7fa510200640 1 --2- 172.21.15.107:0/1532308008 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa5080719d0 0x7fa508071da0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fa4f80048d0 tx=0x7fa4f8009da0 comp rx=0 tx=0).stop 2024-03-04T13:56:39.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.264+0000 7fa510200640 1 -- 172.21.15.107:0/1532308008 shutdown_connections 2024-03-04T13:56:39.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.264+0000 7fa510200640 1 --2- 172.21.15.107:0/1532308008 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa5080722e0 0x7fa50810cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:39.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.264+0000 7fa510200640 1 --2- 172.21.15.107:0/1532308008 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa5080719d0 0x7fa508071da0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:39.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.264+0000 7fa510200640 1 -- 172.21.15.107:0/1532308008 >> 172.21.15.107:0/1532308008 conn(0x7fa5080fa160 msgr2=0x7fa5080fc570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:56:39.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.264+0000 7fa510200640 1 -- 172.21.15.107:0/1532308008 shutdown_connections 2024-03-04T13:56:39.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.264+0000 7fa510200640 1 -- 172.21.15.107:0/1532308008 wait complete. 2024-03-04T13:56:39.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.265+0000 7fa510200640 1 Processor -- start 2024-03-04T13:56:39.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.265+0000 7fa510200640 1 -- start start 2024-03-04T13:56:39.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.265+0000 7fa510200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa5080719d0 0x7fa508119190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:39.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.265+0000 7fa510200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa5080722e0 0x7fa5081196d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:39.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.265+0000 7fa510200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa508119cd0 con 0x7fa5080722e0 2024-03-04T13:56:39.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.265+0000 7fa510200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa508113260 con 0x7fa5080719d0 2024-03-04T13:56:39.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.265+0000 7fa50ce00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa5080722e0 0x7fa5081196d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:56:39.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.265+0000 7fa50ce00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa5080722e0 0x7fa5081196d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47130/0 (socket says 172.21.15.107:47130) 2024-03-04T13:56:39.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.265+0000 7fa50ce00640 1 -- 172.21.15.107:0/1236218103 learned_addr learned my addr 172.21.15.107:0/1236218103 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:56:39.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.266+0000 7fa50ce00640 1 -- 172.21.15.107:0/1236218103 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa5080719d0 msgr2=0x7fa508119190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:39.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.266+0000 7fa50d800640 1 --2- 172.21.15.107:0/1236218103 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa5080719d0 0x7fa508119190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:56:39.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.266+0000 7fa50ce00640 1 --2- 172.21.15.107:0/1236218103 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa5080719d0 0x7fa508119190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:39.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.266+0000 7fa50ce00640 1 -- 172.21.15.107:0/1236218103 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4f8005040 con 0x7fa5080722e0 2024-03-04T13:56:39.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.266+0000 7fa50ce00640 1 --2- 172.21.15.107:0/1236218103 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa5080722e0 0x7fa5081196d0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fa4f400c810 tx=0x7fa4f40043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:56:39.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.266+0000 7fa505600640 1 -- 172.21.15.107:0/1236218103 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa4f400a820 con 0x7fa5080722e0 2024-03-04T13:56:39.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.266+0000 7fa505600640 1 -- 172.21.15.107:0/1236218103 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa4f400ae50 con 0x7fa5080722e0 2024-03-04T13:56:39.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.266+0000 7fa510200640 1 -- 172.21.15.107:0/1236218103 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa508113540 con 0x7fa5080722e0 2024-03-04T13:56:39.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.266+0000 7fa505600640 1 -- 172.21.15.107:0/1236218103 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa4f40115c0 con 0x7fa5080722e0 2024-03-04T13:56:39.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.266+0000 7fa510200640 1 -- 172.21.15.107:0/1236218103 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa508113a60 con 0x7fa5080722e0 2024-03-04T13:56:39.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.268+0000 7fa510200640 1 -- 172.21.15.107:0/1236218103 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa508064480 con 0x7fa5080722e0 2024-03-04T13:56:39.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.269+0000 7fa505600640 1 -- 172.21.15.107:0/1236218103 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fa4f400a980 con 0x7fa5080722e0 2024-03-04T13:56:39.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.270+0000 7fa505600640 1 --2- 172.21.15.107:0/1236218103 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fa4ec06d760 0x7fa4ec06fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:39.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.270+0000 7fa505600640 1 -- 172.21.15.107:0/1236218103 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fa4f408d460 con 0x7fa5080722e0 2024-03-04T13:56:39.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.270+0000 7fa50d800640 1 --2- 172.21.15.107:0/1236218103 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fa4ec06d760 0x7fa4ec06fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:56:39.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.271+0000 7fa50d800640 1 --2- 172.21.15.107:0/1236218103 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fa4ec06d760 0x7fa4ec06fc10 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa4f8034040 tx=0x7fa4f8009da0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:56:39.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.275+0000 7fa505600640 1 -- 172.21.15.107:0/1236218103 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa4f405a790 con 0x7fa5080722e0 2024-03-04T13:56:39.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.511+0000 7fa510200640 1 -- 172.21.15.107:0/1236218103 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 29, "format": "json"} v 0) v1 -- 0x7fa508114790 con 0x7fa5080722e0 2024-03-04T13:56:39.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.512+0000 7fa505600640 1 -- 172.21.15.107:0/1236218103 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 29, "format": "json"}]=0 dumped fsmap epoch 29 v36) v1 ==== 107+0+4148 (secure 0 0 0) 0x7fa4f405a320 con 0x7fa5080722e0 2024-03-04T13:56:39.514 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:56:39.514 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34432,"name":"cephfs.smithi107.riexmm","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6827/1144672731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1144672731},{"type":"v1","addr":"172.21.15.107:6827","nonce":1144672731}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":34436,"name":"cephfs.smithi107.thycho","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6829/3010096376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3010096376},{"type":"v1","addr":"172.21.15.107:6829","nonce":3010096376}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":44243,"name":"cephfs.smithi169.jusudl","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6825/1336978861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":1336978861},{"type":"v1","addr":"172.21.15.169:6825","nonce":1336978861}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-04T13:38:30.854004+0000","modified":"2024-03-04T13:53:58.927149+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":86,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-04T13:56:39.514 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 29 2024-03-04T13:56:39.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.514+0000 7fa510200640 1 -- 172.21.15.107:0/1236218103 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fa4ec06d760 msgr2=0x7fa4ec06fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:39.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.514+0000 7fa510200640 1 --2- 172.21.15.107:0/1236218103 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fa4ec06d760 0x7fa4ec06fc10 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa4f8034040 tx=0x7fa4f8009da0 comp rx=0 tx=0).stop 2024-03-04T13:56:39.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.514+0000 7fa510200640 1 -- 172.21.15.107:0/1236218103 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa5080722e0 msgr2=0x7fa5081196d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:39.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.514+0000 7fa510200640 1 --2- 172.21.15.107:0/1236218103 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa5080722e0 0x7fa5081196d0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fa4f400c810 tx=0x7fa4f40043c0 comp rx=0 tx=0).stop 2024-03-04T13:56:39.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.515+0000 7fa510200640 1 -- 172.21.15.107:0/1236218103 shutdown_connections 2024-03-04T13:56:39.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.515+0000 7fa510200640 1 --2- 172.21.15.107:0/1236218103 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fa4ec06d760 0x7fa4ec06fc10 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:39.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.515+0000 7fa510200640 1 --2- 172.21.15.107:0/1236218103 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa5080722e0 0x7fa5081196d0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:39.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.515+0000 7fa510200640 1 --2- 172.21.15.107:0/1236218103 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa5080719d0 0x7fa508119190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:39.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.515+0000 7fa510200640 1 -- 172.21.15.107:0/1236218103 >> 172.21.15.107:0/1236218103 conn(0x7fa5080fa160 msgr2=0x7fa5080fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:56:39.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.515+0000 7fa510200640 1 -- 172.21.15.107:0/1236218103 shutdown_connections 2024-03-04T13:56:39.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.515+0000 7fa510200640 1 -- 172.21.15.107:0/1236218103 wait complete. 2024-03-04T13:56:39.535 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 29 2024-03-04T13:56:39.535 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 30 2024-03-04T13:56:39.595 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:56:39 smithi107 ceph-mon[112578]: from='client.? 172.21.15.107:0/4188633124' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-03-04T13:56:39.595 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:56:39 smithi107 ceph-mon[112578]: pgmap v316: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:56:39.595 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:56:39 smithi107 ceph-mon[112578]: from='client.? 172.21.15.107:0/295300220' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-03-04T13:56:39.612 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:56:39 smithi169 ceph-mon[86183]: from='client.? 172.21.15.107:0/4188633124' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-03-04T13:56:39.612 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:56:39 smithi169 ceph-mon[86183]: pgmap v316: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:56:39.612 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:56:39 smithi169 ceph-mon[86183]: from='client.? 172.21.15.107:0/295300220' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-03-04T13:56:39.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.732+0000 7fd21ec00640 1 -- 172.21.15.107:0/2589275716 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2180fd3b0 msgr2=0x7fd2180fd780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:39.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.732+0000 7fd21ec00640 1 --2- 172.21.15.107:0/2589275716 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2180fd3b0 0x7fd2180fd780 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd210009c50 tx=0x7fd210004d10 comp rx=0 tx=0).stop 2024-03-04T13:56:39.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.732+0000 7fd21ec00640 1 -- 172.21.15.107:0/2589275716 shutdown_connections 2024-03-04T13:56:39.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.732+0000 7fd21ec00640 1 --2- 172.21.15.107:0/2589275716 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd2180fdd50 0x7fd218106390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:39.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.733+0000 7fd21ec00640 1 --2- 172.21.15.107:0/2589275716 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2180fd3b0 0x7fd2180fd780 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:39.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.733+0000 7fd21ec00640 1 -- 172.21.15.107:0/2589275716 >> 172.21.15.107:0/2589275716 conn(0x7fd2180f6e80 msgr2=0x7fd2180f9290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:56:39.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.733+0000 7fd21ec00640 1 -- 172.21.15.107:0/2589275716 shutdown_connections 2024-03-04T13:56:39.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.733+0000 7fd21ec00640 1 -- 172.21.15.107:0/2589275716 wait complete. 2024-03-04T13:56:39.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.733+0000 7fd21ec00640 1 Processor -- start 2024-03-04T13:56:39.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.733+0000 7fd21ec00640 1 -- start start 2024-03-04T13:56:39.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.734+0000 7fd21ec00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2180fd3b0 0x7fd218114de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:39.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.734+0000 7fd21ec00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd2180fdd50 0x7fd218115320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:39.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.734+0000 7fd21ec00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd218115920 con 0x7fd2180fd3b0 2024-03-04T13:56:39.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.734+0000 7fd21ec00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd21810eeb0 con 0x7fd2180fdd50 2024-03-04T13:56:39.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.734+0000 7fd217e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2180fd3b0 0x7fd218114de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:56:39.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.734+0000 7fd217e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2180fd3b0 0x7fd218114de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47148/0 (socket says 172.21.15.107:47148) 2024-03-04T13:56:39.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.734+0000 7fd217e00640 1 -- 172.21.15.107:0/2852495673 learned_addr learned my addr 172.21.15.107:0/2852495673 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:56:39.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.734+0000 7fd217e00640 1 -- 172.21.15.107:0/2852495673 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd2180fdd50 msgr2=0x7fd218115320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:39.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.734+0000 7fd217400640 1 --2- 172.21.15.107:0/2852495673 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd2180fdd50 0x7fd218115320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:56:39.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.735+0000 7fd217e00640 1 --2- 172.21.15.107:0/2852495673 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd2180fdd50 0x7fd218115320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:39.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.735+0000 7fd217e00640 1 -- 172.21.15.107:0/2852495673 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2100098b0 con 0x7fd2180fd3b0 2024-03-04T13:56:39.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.735+0000 7fd217400640 1 --2- 172.21.15.107:0/2852495673 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd2180fdd50 0x7fd218115320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T13:56:39.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.735+0000 7fd217e00640 1 --2- 172.21.15.107:0/2852495673 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2180fd3b0 0x7fd218114de0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fd210003400 tx=0x7fd210004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:56:39.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.735+0000 7fd20fe00640 1 -- 172.21.15.107:0/2852495673 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd21003a070 con 0x7fd2180fd3b0 2024-03-04T13:56:39.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.735+0000 7fd20fe00640 1 -- 172.21.15.107:0/2852495673 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd210035b20 con 0x7fd2180fd3b0 2024-03-04T13:56:39.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.735+0000 7fd21ec00640 1 -- 172.21.15.107:0/2852495673 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd21810f130 con 0x7fd2180fd3b0 2024-03-04T13:56:39.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.735+0000 7fd20fe00640 1 -- 172.21.15.107:0/2852495673 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd21003eb60 con 0x7fd2180fd3b0 2024-03-04T13:56:39.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.736+0000 7fd21ec00640 1 -- 172.21.15.107:0/2852495673 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd21810f650 con 0x7fd2180fd3b0 2024-03-04T13:56:39.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.737+0000 7fd21ec00640 1 -- 172.21.15.107:0/2852495673 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2180637a0 con 0x7fd2180fd3b0 2024-03-04T13:56:39.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.738+0000 7fd20fe00640 1 -- 172.21.15.107:0/2852495673 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fd210032070 con 0x7fd2180fd3b0 2024-03-04T13:56:39.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.738+0000 7fd20fe00640 1 --2- 172.21.15.107:0/2852495673 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fd1fc06d750 0x7fd1fc06fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:39.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.739+0000 7fd20fe00640 1 -- 172.21.15.107:0/2852495673 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fd2100af260 con 0x7fd2180fd3b0 2024-03-04T13:56:39.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.739+0000 7fd217400640 1 --2- 172.21.15.107:0/2852495673 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fd1fc06d750 0x7fd1fc06fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:56:39.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.739+0000 7fd217400640 1 --2- 172.21.15.107:0/2852495673 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fd1fc06d750 0x7fd1fc06fc00 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd2181106a0 tx=0x7fd200009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:56:39.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.744+0000 7fd20fe00640 1 -- 172.21.15.107:0/2852495673 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd210078ae0 con 0x7fd2180fd3b0 2024-03-04T13:56:39.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.981+0000 7fd21ec00640 1 -- 172.21.15.107:0/2852495673 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 30, "format": "json"} v 0) v1 -- 0x7fd218105ed0 con 0x7fd2180fd3b0 2024-03-04T13:56:39.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.982+0000 7fd20fe00640 1 -- 172.21.15.107:0/2852495673 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 30, "format": "json"}]=0 dumped fsmap epoch 30 v36) v1 ==== 107+0+4193 (secure 0 0 0) 0x7fd218105ed0 con 0x7fd2180fd3b0 2024-03-04T13:56:39.983 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:56:39.983 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34436,"name":"cephfs.smithi107.thycho","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6829/3010096376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3010096376},{"type":"v1","addr":"172.21.15.107:6829","nonce":3010096376}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":44243,"name":"cephfs.smithi169.jusudl","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6825/1336978861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":1336978861},{"type":"v1","addr":"172.21.15.169:6825","nonce":1336978861}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-04T13:38:30.854004+0000","modified":"2024-03-04T13:53:58.940045+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":86,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34432},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34432":{"gid":34432,"name":"cephfs.smithi107.riexmm","rank":0,"incarnation":30,"state":"up:replay","state_seq":1,"addr":"172.21.15.107:6827/1144672731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1144672731},{"type":"v1","addr":"172.21.15.107:6827","nonce":1144672731}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-04T13:56:39.984 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 30 2024-03-04T13:56:39.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.984+0000 7fd21ec00640 1 -- 172.21.15.107:0/2852495673 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fd1fc06d750 msgr2=0x7fd1fc06fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:39.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.984+0000 7fd21ec00640 1 --2- 172.21.15.107:0/2852495673 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fd1fc06d750 0x7fd1fc06fc00 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd2181106a0 tx=0x7fd200009040 comp rx=0 tx=0).stop 2024-03-04T13:56:39.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.984+0000 7fd21ec00640 1 -- 172.21.15.107:0/2852495673 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2180fd3b0 msgr2=0x7fd218114de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:39.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.984+0000 7fd21ec00640 1 --2- 172.21.15.107:0/2852495673 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2180fd3b0 0x7fd218114de0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fd210003400 tx=0x7fd210004d10 comp rx=0 tx=0).stop 2024-03-04T13:56:39.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.984+0000 7fd21ec00640 1 -- 172.21.15.107:0/2852495673 shutdown_connections 2024-03-04T13:56:39.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.984+0000 7fd21ec00640 1 --2- 172.21.15.107:0/2852495673 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fd1fc06d750 0x7fd1fc06fc00 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:39.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.984+0000 7fd21ec00640 1 --2- 172.21.15.107:0/2852495673 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd2180fdd50 0x7fd218115320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:39.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.984+0000 7fd21ec00640 1 --2- 172.21.15.107:0/2852495673 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2180fd3b0 0x7fd218114de0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:39.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.984+0000 7fd21ec00640 1 -- 172.21.15.107:0/2852495673 >> 172.21.15.107:0/2852495673 conn(0x7fd2180f6e80 msgr2=0x7fd2180fb9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:56:39.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.984+0000 7fd21ec00640 1 -- 172.21.15.107:0/2852495673 shutdown_connections 2024-03-04T13:56:39.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:39.984+0000 7fd21ec00640 1 -- 172.21.15.107:0/2852495673 wait complete. 2024-03-04T13:56:40.004 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 30 2024-03-04T13:56:40.004 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 31 2024-03-04T13:56:40.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:40.198+0000 7fef3bc00640 1 -- 172.21.15.107:0/2012325536 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef34100110 msgr2=0x7fef34100560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:40.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:40.198+0000 7fef3bc00640 1 --2- 172.21.15.107:0/2012325536 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef34100110 0x7fef34100560 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fef1c0048d0 tx=0x7fef1c009da0 comp rx=0 tx=0).stop 2024-03-04T13:56:40.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:40.198+0000 7fef3bc00640 1 -- 172.21.15.107:0/2012325536 shutdown_connections 2024-03-04T13:56:40.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:40.198+0000 7fef3bc00640 1 --2- 172.21.15.107:0/2012325536 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef34100110 0x7fef34100560 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:40.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:40.198+0000 7fef3bc00640 1 --2- 172.21.15.107:0/2012325536 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef340fe9c0 0x7fef340fed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:40.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:40.198+0000 7fef3bc00640 1 -- 172.21.15.107:0/2012325536 >> 172.21.15.107:0/2012325536 conn(0x7fef340fa160 msgr2=0x7fef340fc570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:56:40.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:40.198+0000 7fef3bc00640 1 -- 172.21.15.107:0/2012325536 shutdown_connections 2024-03-04T13:56:40.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:40.198+0000 7fef3bc00640 1 -- 172.21.15.107:0/2012325536 wait complete. 2024-03-04T13:56:40.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:40.199+0000 7fef3bc00640 1 Processor -- start 2024-03-04T13:56:40.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:40.199+0000 7fef3bc00640 1 -- start start 2024-03-04T13:56:40.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:40.199+0000 7fef3bc00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef340fe9c0 0x7fef34114de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:40.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:40.199+0000 7fef3bc00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef34100110 0x7fef34115320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:40.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:40.199+0000 7fef3bc00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef34115920 con 0x7fef340fe9c0 2024-03-04T13:56:40.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:40.199+0000 7fef3bc00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef3410eeb0 con 0x7fef34100110 2024-03-04T13:56:40.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:40.199+0000 7fef39200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef340fe9c0 0x7fef34114de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:56:40.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:40.199+0000 7fef39200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef340fe9c0 0x7fef34114de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47174/0 (socket says 172.21.15.107:47174) 2024-03-04T13:56:40.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:40.199+0000 7fef39200640 1 -- 172.21.15.107:0/3270935878 learned_addr learned my addr 172.21.15.107:0/3270935878 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:56:40.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:40.200+0000 7fef39200640 1 -- 172.21.15.107:0/3270935878 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef34100110 msgr2=0x7fef34115320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:40.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:40.200+0000 7fef39200640 1 --2- 172.21.15.107:0/3270935878 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef34100110 0x7fef34115320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:40.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:40.200+0000 7fef39200640 1 -- 172.21.15.107:0/3270935878 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef1c005040 con 0x7fef340fe9c0 2024-03-04T13:56:40.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:40.200+0000 7fef39200640 1 --2- 172.21.15.107:0/3270935878 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef340fe9c0 0x7fef34114de0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fef24002a50 tx=0x7fef24004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:56:40.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:40.200+0000 7fef31600640 1 -- 172.21.15.107:0/3270935878 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fef24007c30 con 0x7fef340fe9c0 2024-03-04T13:56:40.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:40.200+0000 7fef31600640 1 -- 172.21.15.107:0/3270935878 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fef2400f040 con 0x7fef340fe9c0 2024-03-04T13:56:40.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:40.200+0000 7fef31600640 1 -- 172.21.15.107:0/3270935878 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fef2400db50 con 0x7fef340fe9c0 2024-03-04T13:56:40.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:40.200+0000 7fef3bc00640 1 -- 172.21.15.107:0/3270935878 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef3410f190 con 0x7fef340fe9c0 2024-03-04T13:56:40.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:40.200+0000 7fef3bc00640 1 -- 172.21.15.107:0/3270935878 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef3410f6b0 con 0x7fef340fe9c0 2024-03-04T13:56:40.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:40.201+0000 7fef3bc00640 1 -- 172.21.15.107:0/3270935878 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef34064030 con 0x7fef340fe9c0 2024-03-04T13:56:40.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:40.202+0000 7fef31600640 1 -- 172.21.15.107:0/3270935878 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fef2401c400 con 0x7fef340fe9c0 2024-03-04T13:56:40.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:40.204+0000 7fef31600640 1 --2- 172.21.15.107:0/3270935878 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fef1806d760 0x7fef1806fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:40.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:40.204+0000 7fef31600640 1 -- 172.21.15.107:0/3270935878 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fef2408c870 con 0x7fef340fe9c0 2024-03-04T13:56:40.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:40.204+0000 7fef2be00640 1 --2- 172.21.15.107:0/3270935878 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fef1806d760 0x7fef1806fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:56:40.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:40.205+0000 7fef2be00640 1 --2- 172.21.15.107:0/3270935878 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fef1806d760 0x7fef1806fc10 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fef1c0048a0 tx=0x7fef1c009da0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:56:40.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:40.208+0000 7fef31600640 1 -- 172.21.15.107:0/3270935878 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fef24055ff0 con 0x7fef340fe9c0 2024-03-04T13:56:40.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:40.431+0000 7fef3bc00640 1 -- 172.21.15.107:0/3270935878 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 31, "format": "json"} v 0) v1 -- 0x7fef3410f8d0 con 0x7fef340fe9c0 2024-03-04T13:56:40.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:40.432+0000 7fef31600640 1 -- 172.21.15.107:0/3270935878 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 31, "format": "json"}]=0 dumped fsmap epoch 31 v36) v1 ==== 107+0+4197 (secure 0 0 0) 0x7fef240596b0 con 0x7fef340fe9c0 2024-03-04T13:56:40.433 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:56:40.433 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34436,"name":"cephfs.smithi107.thycho","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6829/3010096376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3010096376},{"type":"v1","addr":"172.21.15.107:6829","nonce":3010096376}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":44243,"name":"cephfs.smithi169.jusudl","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6825/1336978861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":1336978861},{"type":"v1","addr":"172.21.15.169:6825","nonce":1336978861}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-04T13:38:30.854004+0000","modified":"2024-03-04T13:54:08.228482+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":86,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34432},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34432":{"gid":34432,"name":"cephfs.smithi107.riexmm","rank":0,"incarnation":30,"state":"up:reconnect","state_seq":22,"addr":"172.21.15.107:6827/1144672731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1144672731},{"type":"v1","addr":"172.21.15.107:6827","nonce":1144672731}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-04T13:56:40.434 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 31 2024-03-04T13:56:40.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:40.434+0000 7fef3bc00640 1 -- 172.21.15.107:0/3270935878 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fef1806d760 msgr2=0x7fef1806fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:40.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:40.434+0000 7fef3bc00640 1 --2- 172.21.15.107:0/3270935878 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fef1806d760 0x7fef1806fc10 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fef1c0048a0 tx=0x7fef1c009da0 comp rx=0 tx=0).stop 2024-03-04T13:56:40.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:40.434+0000 7fef3bc00640 1 -- 172.21.15.107:0/3270935878 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef340fe9c0 msgr2=0x7fef34114de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:40.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:40.434+0000 7fef3bc00640 1 --2- 172.21.15.107:0/3270935878 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef340fe9c0 0x7fef34114de0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fef24002a50 tx=0x7fef24004600 comp rx=0 tx=0).stop 2024-03-04T13:56:40.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:40.434+0000 7fef3bc00640 1 -- 172.21.15.107:0/3270935878 shutdown_connections 2024-03-04T13:56:40.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:40.434+0000 7fef3bc00640 1 --2- 172.21.15.107:0/3270935878 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fef1806d760 0x7fef1806fc10 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:40.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:40.434+0000 7fef3bc00640 1 --2- 172.21.15.107:0/3270935878 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef34100110 0x7fef34115320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:40.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:40.434+0000 7fef3bc00640 1 --2- 172.21.15.107:0/3270935878 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef340fe9c0 0x7fef34114de0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:40.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:40.434+0000 7fef3bc00640 1 -- 172.21.15.107:0/3270935878 >> 172.21.15.107:0/3270935878 conn(0x7fef340fa160 msgr2=0x7fef340fbc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:56:40.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:40.435+0000 7fef3bc00640 1 -- 172.21.15.107:0/3270935878 shutdown_connections 2024-03-04T13:56:40.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:40.435+0000 7fef3bc00640 1 -- 172.21.15.107:0/3270935878 wait complete. 2024-03-04T13:56:40.454 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 31 2024-03-04T13:56:40.454 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 32 2024-03-04T13:56:40.595 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:56:40 smithi107 ceph-mon[112578]: from='client.? 172.21.15.107:0/1236218103' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-03-04T13:56:40.595 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:56:40 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:56:40.595 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:56:40 smithi107 ceph-mon[112578]: from='client.? 172.21.15.107:0/2852495673' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 30, "format": "json"}]: dispatch 2024-03-04T13:56:40.612 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:56:40 smithi169 ceph-mon[86183]: from='client.? 172.21.15.107:0/1236218103' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-03-04T13:56:40.612 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:56:40 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:56:40.612 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:56:40 smithi169 ceph-mon[86183]: from='client.? 172.21.15.107:0/2852495673' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 30, "format": "json"}]: dispatch 2024-03-04T13:56:40.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:40.654+0000 7fa887a00640 1 -- 172.21.15.107:0/1105909198 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa880108990 msgr2=0x7fa880108d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:40.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:40.654+0000 7fa887a00640 1 --2- 172.21.15.107:0/1105909198 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa880108990 0x7fa880108d60 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa8740048d0 tx=0x7fa874009da0 comp rx=0 tx=0).stop 2024-03-04T13:56:40.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:40.654+0000 7fa887a00640 1 -- 172.21.15.107:0/1105909198 shutdown_connections 2024-03-04T13:56:40.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:40.654+0000 7fa887a00640 1 --2- 172.21.15.107:0/1105909198 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa880109330 0x7fa8801032d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:40.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:40.654+0000 7fa887a00640 1 --2- 172.21.15.107:0/1105909198 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa880108990 0x7fa880108d60 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:40.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:40.654+0000 7fa887a00640 1 -- 172.21.15.107:0/1105909198 >> 172.21.15.107:0/1105909198 conn(0x7fa880072210 msgr2=0x7fa880074620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:56:40.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:40.654+0000 7fa887a00640 1 -- 172.21.15.107:0/1105909198 shutdown_connections 2024-03-04T13:56:40.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:40.654+0000 7fa887a00640 1 -- 172.21.15.107:0/1105909198 wait complete. 2024-03-04T13:56:40.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:40.655+0000 7fa887a00640 1 Processor -- start 2024-03-04T13:56:40.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:40.655+0000 7fa887a00640 1 -- start start 2024-03-04T13:56:40.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:40.655+0000 7fa887a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa880108990 0x7fa8801192c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:40.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:40.655+0000 7fa887a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa880109330 0x7fa880119800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:40.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:40.655+0000 7fa887a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa880119e90 con 0x7fa880108990 2024-03-04T13:56:40.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:40.655+0000 7fa887a00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa880113390 con 0x7fa880109330 2024-03-04T13:56:40.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:40.656+0000 7fa885000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa880108990 0x7fa8801192c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:56:40.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:40.656+0000 7fa885000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa880108990 0x7fa8801192c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47192/0 (socket says 172.21.15.107:47192) 2024-03-04T13:56:40.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:40.656+0000 7fa885000640 1 -- 172.21.15.107:0/3353472365 learned_addr learned my addr 172.21.15.107:0/3353472365 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:56:40.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:40.656+0000 7fa885000640 1 -- 172.21.15.107:0/3353472365 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa880109330 msgr2=0x7fa880119800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:40.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:40.656+0000 7fa885000640 1 --2- 172.21.15.107:0/3353472365 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa880109330 0x7fa880119800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:40.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:40.656+0000 7fa885000640 1 -- 172.21.15.107:0/3353472365 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa874005040 con 0x7fa880108990 2024-03-04T13:56:40.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:40.656+0000 7fa885000640 1 --2- 172.21.15.107:0/3353472365 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa880108990 0x7fa8801192c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa87402bdc0 tx=0x7fa874009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:56:40.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:40.657+0000 7fa87cc00640 1 -- 172.21.15.107:0/3353472365 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa87403d450 con 0x7fa880108990 2024-03-04T13:56:40.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:40.657+0000 7fa87cc00640 1 -- 172.21.15.107:0/3353472365 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa87403da80 con 0x7fa880108990 2024-03-04T13:56:40.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:40.657+0000 7fa887a00640 1 -- 172.21.15.107:0/3353472365 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa880113610 con 0x7fa880108990 2024-03-04T13:56:40.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:40.657+0000 7fa87cc00640 1 -- 172.21.15.107:0/3353472365 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa87403bae0 con 0x7fa880108990 2024-03-04T13:56:40.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:40.657+0000 7fa887a00640 1 -- 172.21.15.107:0/3353472365 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa880113b30 con 0x7fa880108990 2024-03-04T13:56:40.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:40.658+0000 7fa887a00640 1 -- 172.21.15.107:0/3353472365 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa880064480 con 0x7fa880108990 2024-03-04T13:56:40.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:40.659+0000 7fa87cc00640 1 -- 172.21.15.107:0/3353472365 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fa87403dbf0 con 0x7fa880108990 2024-03-04T13:56:40.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:40.660+0000 7fa87cc00640 1 --2- 172.21.15.107:0/3353472365 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fa86806d750 0x7fa86806fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:40.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:40.660+0000 7fa87cc00640 1 -- 172.21.15.107:0/3353472365 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fa87402d040 con 0x7fa880108990 2024-03-04T13:56:40.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:40.660+0000 7fa87fe00640 1 --2- 172.21.15.107:0/3353472365 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fa86806d750 0x7fa86806fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:56:40.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:40.661+0000 7fa87fe00640 1 --2- 172.21.15.107:0/3353472365 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fa86806d750 0x7fa86806fc00 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa880114b80 tx=0x7fa870006d10 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:56:40.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:40.664+0000 7fa87cc00640 1 -- 172.21.15.107:0/3353472365 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa8740793a0 con 0x7fa880108990 2024-03-04T13:56:40.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:40.898+0000 7fa887a00640 1 -- 172.21.15.107:0/3353472365 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 32, "format": "json"} v 0) v1 -- 0x7fa8801148c0 con 0x7fa880108990 2024-03-04T13:56:40.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:40.900+0000 7fa87cc00640 1 -- 172.21.15.107:0/3353472365 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 32, "format": "json"}]=0 dumped fsmap epoch 32 v36) v1 ==== 107+0+5010 (secure 0 0 0) 0x7fa87407ca60 con 0x7fa880108990 2024-03-04T13:56:40.901 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:56:40.901 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34436,"name":"cephfs.smithi107.thycho","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6829/3010096376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3010096376},{"type":"v1","addr":"172.21.15.107:6829","nonce":3010096376}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":44243,"name":"cephfs.smithi169.jusudl","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6825/1336978861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":1336978861},{"type":"v1","addr":"172.21.15.169:6825","nonce":1336978861}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25},{"gid":44259,"name":"cephfs.smithi169.bxitzk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6827/4015044684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":4015044684},{"type":"v1","addr":"172.21.15.169:6827","nonce":4015044684}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-04T13:38:30.854004+0000","modified":"2024-03-04T13:54:08.855802+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":86,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34432},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34432":{"gid":34432,"name":"cephfs.smithi107.riexmm","rank":0,"incarnation":30,"state":"up:rejoin","state_seq":23,"addr":"172.21.15.107:6827/1144672731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1144672731},{"type":"v1","addr":"172.21.15.107:6827","nonce":1144672731}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-04T13:56:40.902 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 32 2024-03-04T13:56:40.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:40.902+0000 7fa887a00640 1 -- 172.21.15.107:0/3353472365 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fa86806d750 msgr2=0x7fa86806fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:40.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:40.902+0000 7fa887a00640 1 --2- 172.21.15.107:0/3353472365 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fa86806d750 0x7fa86806fc00 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa880114b80 tx=0x7fa870006d10 comp rx=0 tx=0).stop 2024-03-04T13:56:40.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:40.902+0000 7fa887a00640 1 -- 172.21.15.107:0/3353472365 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa880108990 msgr2=0x7fa8801192c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:40.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:40.902+0000 7fa887a00640 1 --2- 172.21.15.107:0/3353472365 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa880108990 0x7fa8801192c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa87402bdc0 tx=0x7fa874009da0 comp rx=0 tx=0).stop 2024-03-04T13:56:40.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:40.902+0000 7fa887a00640 1 -- 172.21.15.107:0/3353472365 shutdown_connections 2024-03-04T13:56:40.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:40.902+0000 7fa887a00640 1 --2- 172.21.15.107:0/3353472365 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fa86806d750 0x7fa86806fc00 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:40.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:40.902+0000 7fa887a00640 1 --2- 172.21.15.107:0/3353472365 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa880109330 0x7fa880119800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:40.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:40.902+0000 7fa887a00640 1 --2- 172.21.15.107:0/3353472365 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa880108990 0x7fa8801192c0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:40.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:40.902+0000 7fa887a00640 1 -- 172.21.15.107:0/3353472365 >> 172.21.15.107:0/3353472365 conn(0x7fa880072210 msgr2=0x7fa880073e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:56:40.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:40.903+0000 7fa887a00640 1 -- 172.21.15.107:0/3353472365 shutdown_connections 2024-03-04T13:56:40.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:40.903+0000 7fa887a00640 1 -- 172.21.15.107:0/3353472365 wait complete. 2024-03-04T13:56:40.920 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 32 2024-03-04T13:56:40.921 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 33 2024-03-04T13:56:41.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:41.106+0000 7f01bda00640 1 -- 172.21.15.107:0/3955479657 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01b806f7c0 msgr2=0x7f01b806fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:41.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:41.106+0000 7f01bda00640 1 --2- 172.21.15.107:0/3955479657 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01b806f7c0 0x7f01b806fb90 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f01a80048d0 tx=0x7f01a8009da0 comp rx=0 tx=0).stop 2024-03-04T13:56:41.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:41.107+0000 7f01bda00640 1 -- 172.21.15.107:0/3955479657 shutdown_connections 2024-03-04T13:56:41.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:41.107+0000 7f01bda00640 1 --2- 172.21.15.107:0/3955479657 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f01b80700d0 0x7f01b8108700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:41.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:41.107+0000 7f01bda00640 1 --2- 172.21.15.107:0/3955479657 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01b806f7c0 0x7f01b806fb90 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:41.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:41.107+0000 7f01bda00640 1 -- 172.21.15.107:0/3955479657 >> 172.21.15.107:0/3955479657 conn(0x7f01b80f7ee0 msgr2=0x7f01b80fa2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:56:41.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:41.107+0000 7f01bda00640 1 -- 172.21.15.107:0/3955479657 shutdown_connections 2024-03-04T13:56:41.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:41.107+0000 7f01bda00640 1 -- 172.21.15.107:0/3955479657 wait complete. 2024-03-04T13:56:41.110 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:41.107+0000 7f01bda00640 1 Processor -- start 2024-03-04T13:56:41.110 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:41.107+0000 7f01bda00640 1 -- start start 2024-03-04T13:56:41.110 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:41.108+0000 7f01bda00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f01b806f7c0 0x7f01b8114ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:41.110 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:41.108+0000 7f01bda00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01b80700d0 0x7f01b8115400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:41.110 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:41.108+0000 7f01bda00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01b8115a90 con 0x7f01b80700d0 2024-03-04T13:56:41.110 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:41.108+0000 7f01bda00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01b810ef90 con 0x7f01b806f7c0 2024-03-04T13:56:41.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:41.108+0000 7f01b6000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01b80700d0 0x7f01b8115400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:56:41.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:41.109+0000 7f01b6a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f01b806f7c0 0x7f01b8114ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:56:41.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:41.109+0000 7f01b6a00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f01b806f7c0 0x7f01b8114ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.107:58946/0 (socket says 172.21.15.107:58946) 2024-03-04T13:56:41.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:41.109+0000 7f01b6a00640 1 -- 172.21.15.107:0/2854660080 learned_addr learned my addr 172.21.15.107:0/2854660080 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:56:41.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:41.109+0000 7f01b6000640 1 -- 172.21.15.107:0/2854660080 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f01b806f7c0 msgr2=0x7f01b8114ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:41.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:41.109+0000 7f01b6000640 1 --2- 172.21.15.107:0/2854660080 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f01b806f7c0 0x7f01b8114ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:41.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:41.109+0000 7f01b6000640 1 -- 172.21.15.107:0/2854660080 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01a8005040 con 0x7f01b80700d0 2024-03-04T13:56:41.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:41.110+0000 7f01b6000640 1 --2- 172.21.15.107:0/2854660080 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01b80700d0 0x7f01b8115400 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f01a400b4e0 tx=0x7f01a4009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:56:41.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:41.110+0000 7f01aea00640 1 -- 172.21.15.107:0/2854660080 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f01a400f6f0 con 0x7f01b80700d0 2024-03-04T13:56:41.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:41.110+0000 7f01aea00640 1 -- 172.21.15.107:0/2854660080 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f01a400fd20 con 0x7f01b80700d0 2024-03-04T13:56:41.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:41.110+0000 7f01bda00640 1 -- 172.21.15.107:0/2854660080 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01b810f270 con 0x7f01b80700d0 2024-03-04T13:56:41.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:41.110+0000 7f01aea00640 1 -- 172.21.15.107:0/2854660080 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f01a40103a0 con 0x7f01b80700d0 2024-03-04T13:56:41.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:41.110+0000 7f01bda00640 1 -- 172.21.15.107:0/2854660080 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01b810f790 con 0x7f01b80700d0 2024-03-04T13:56:41.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:41.112+0000 7f01bda00640 1 -- 172.21.15.107:0/2854660080 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01b8064480 con 0x7f01b80700d0 2024-03-04T13:56:41.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:41.117+0000 7f01aea00640 1 -- 172.21.15.107:0/2854660080 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f01a4011400 con 0x7f01b80700d0 2024-03-04T13:56:41.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:41.118+0000 7f01aea00640 1 --2- 172.21.15.107:0/2854660080 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f019c06d750 0x7f019c06fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:41.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:41.118+0000 7f01aea00640 1 -- 172.21.15.107:0/2854660080 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f01a408cdb0 con 0x7f01b80700d0 2024-03-04T13:56:41.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:41.118+0000 7f01b6a00640 1 --2- 172.21.15.107:0/2854660080 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f019c06d750 0x7f019c06fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:56:41.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:41.119+0000 7f01b6a00640 1 --2- 172.21.15.107:0/2854660080 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f019c06d750 0x7f019c06fc00 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f01a8004540 tx=0x7f01a8009da0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:56:41.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:41.124+0000 7f01aea00640 1 -- 172.21.15.107:0/2854660080 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f01a4056630 con 0x7f01b80700d0 2024-03-04T13:56:41.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:41.359+0000 7f01bda00640 1 -- 172.21.15.107:0/2854660080 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 33, "format": "json"} v 0) v1 -- 0x7f01b810b720 con 0x7f01b80700d0 2024-03-04T13:56:41.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:41.361+0000 7f01aea00640 1 -- 172.21.15.107:0/2854660080 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 33, "format": "json"}]=0 dumped fsmap epoch 33 v36) v1 ==== 107+0+5010 (secure 0 0 0) 0x7f01a4059cf0 con 0x7f01b80700d0 2024-03-04T13:56:41.363 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:56:41.363 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34436,"name":"cephfs.smithi107.thycho","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6829/3010096376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3010096376},{"type":"v1","addr":"172.21.15.107:6829","nonce":3010096376}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":44243,"name":"cephfs.smithi169.jusudl","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6825/1336978861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":1336978861},{"type":"v1","addr":"172.21.15.169:6825","nonce":1336978861}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25},{"gid":44259,"name":"cephfs.smithi169.bxitzk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6827/4015044684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":4015044684},{"type":"v1","addr":"172.21.15.169:6827","nonce":4015044684}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-04T13:38:30.854004+0000","modified":"2024-03-04T13:54:09.979403+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":86,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34432},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34432":{"gid":34432,"name":"cephfs.smithi107.riexmm","rank":0,"incarnation":30,"state":"up:active","state_seq":24,"addr":"172.21.15.107:6827/1144672731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1144672731},{"type":"v1","addr":"172.21.15.107:6827","nonce":1144672731}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-04T13:56:41.363 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 33 2024-03-04T13:56:41.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:41.365+0000 7f01bda00640 1 -- 172.21.15.107:0/2854660080 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f019c06d750 msgr2=0x7f019c06fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:41.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:41.365+0000 7f01bda00640 1 --2- 172.21.15.107:0/2854660080 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f019c06d750 0x7f019c06fc00 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f01a8004540 tx=0x7f01a8009da0 comp rx=0 tx=0).stop 2024-03-04T13:56:41.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:41.365+0000 7f01bda00640 1 -- 172.21.15.107:0/2854660080 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01b80700d0 msgr2=0x7f01b8115400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:41.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:41.365+0000 7f01bda00640 1 --2- 172.21.15.107:0/2854660080 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01b80700d0 0x7f01b8115400 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f01a400b4e0 tx=0x7f01a4009120 comp rx=0 tx=0).stop 2024-03-04T13:56:41.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:41.365+0000 7f01bda00640 1 -- 172.21.15.107:0/2854660080 shutdown_connections 2024-03-04T13:56:41.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:41.365+0000 7f01bda00640 1 --2- 172.21.15.107:0/2854660080 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f019c06d750 0x7f019c06fc00 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:41.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:41.365+0000 7f01bda00640 1 --2- 172.21.15.107:0/2854660080 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01b80700d0 0x7f01b8115400 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:41.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:41.365+0000 7f01bda00640 1 --2- 172.21.15.107:0/2854660080 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f01b806f7c0 0x7f01b8114ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:41.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:41.365+0000 7f01bda00640 1 -- 172.21.15.107:0/2854660080 >> 172.21.15.107:0/2854660080 conn(0x7f01b80f7ee0 msgr2=0x7f01b8102f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:56:41.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:41.365+0000 7f01bda00640 1 -- 172.21.15.107:0/2854660080 shutdown_connections 2024-03-04T13:56:41.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:41.365+0000 7f01bda00640 1 -- 172.21.15.107:0/2854660080 wait complete. 2024-03-04T13:56:41.384 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 33 2024-03-04T13:56:41.384 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 34 2024-03-04T13:56:41.585 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:41.584+0000 7fcd85000640 1 -- 172.21.15.107:0/2841517604 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd80104320 msgr2=0x7fcd801046f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:41.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:41.584+0000 7fcd85000640 1 --2- 172.21.15.107:0/2841517604 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd80104320 0x7fcd801046f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fcd700048d0 tx=0x7fcd70009da0 comp rx=0 tx=0).stop 2024-03-04T13:56:41.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:41.584+0000 7fcd85000640 1 -- 172.21.15.107:0/2841517604 shutdown_connections 2024-03-04T13:56:41.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:41.584+0000 7fcd85000640 1 --2- 172.21.15.107:0/2841517604 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd80104cc0 0x7fcd800fe350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:41.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:41.584+0000 7fcd85000640 1 --2- 172.21.15.107:0/2841517604 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd80104320 0x7fcd801046f0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:41.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:41.584+0000 7fcd85000640 1 -- 172.21.15.107:0/2841517604 >> 172.21.15.107:0/2841517604 conn(0x7fcd800f9f90 msgr2=0x7fcd800fc370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:56:41.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:41.584+0000 7fcd85000640 1 -- 172.21.15.107:0/2841517604 shutdown_connections 2024-03-04T13:56:41.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:41.584+0000 7fcd85000640 1 -- 172.21.15.107:0/2841517604 wait complete. 2024-03-04T13:56:41.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:41.585+0000 7fcd85000640 1 Processor -- start 2024-03-04T13:56:41.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:41.585+0000 7fcd85000640 1 -- start start 2024-03-04T13:56:41.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:41.585+0000 7fcd85000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd80104320 0x7fcd80114c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:41.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:41.585+0000 7fcd85000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd80104cc0 0x7fcd80115160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:41.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:41.585+0000 7fcd85000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd80115760 con 0x7fcd80104cc0 2024-03-04T13:56:41.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:41.585+0000 7fcd85000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd8010ecf0 con 0x7fcd80104320 2024-03-04T13:56:41.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:41.586+0000 7fcd7ea00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd80104cc0 0x7fcd80115160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:56:41.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:41.586+0000 7fcd7ea00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd80104cc0 0x7fcd80115160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47230/0 (socket says 172.21.15.107:47230) 2024-03-04T13:56:41.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:41.586+0000 7fcd7ea00640 1 -- 172.21.15.107:0/2918857636 learned_addr learned my addr 172.21.15.107:0/2918857636 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:56:41.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:41.586+0000 7fcd7f400640 1 --2- 172.21.15.107:0/2918857636 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd80104320 0x7fcd80114c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:56:41.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:41.586+0000 7fcd7ea00640 1 -- 172.21.15.107:0/2918857636 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd80104320 msgr2=0x7fcd80114c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:41.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:41.586+0000 7fcd7ea00640 1 --2- 172.21.15.107:0/2918857636 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd80104320 0x7fcd80114c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:41.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:41.586+0000 7fcd7ea00640 1 -- 172.21.15.107:0/2918857636 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd70005040 con 0x7fcd80104cc0 2024-03-04T13:56:41.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:41.586+0000 7fcd7ea00640 1 --2- 172.21.15.107:0/2918857636 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd80104cc0 0x7fcd80115160 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c00c830 tx=0x7fcd6c004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:56:41.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:41.586+0000 7fcd77400640 1 -- 172.21.15.107:0/2918857636 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcd6c00a850 con 0x7fcd80104cc0 2024-03-04T13:56:41.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:41.586+0000 7fcd77400640 1 -- 172.21.15.107:0/2918857636 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcd6c00ae80 con 0x7fcd80104cc0 2024-03-04T13:56:41.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:41.587+0000 7fcd85000640 1 -- 172.21.15.107:0/2918857636 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd8010efd0 con 0x7fcd80104cc0 2024-03-04T13:56:41.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:41.587+0000 7fcd77400640 1 -- 172.21.15.107:0/2918857636 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcd6c012ea0 con 0x7fcd80104cc0 2024-03-04T13:56:41.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:41.587+0000 7fcd85000640 1 -- 172.21.15.107:0/2918857636 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd8010f4f0 con 0x7fcd80104cc0 2024-03-04T13:56:41.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:41.588+0000 7fcd85000640 1 -- 172.21.15.107:0/2918857636 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd8004bcc0 con 0x7fcd80104cc0 2024-03-04T13:56:41.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:41.589+0000 7fcd77400640 1 -- 172.21.15.107:0/2918857636 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fcd6c012400 con 0x7fcd80104cc0 2024-03-04T13:56:41.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:41.590+0000 7fcd77400640 1 --2- 172.21.15.107:0/2918857636 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fcd6406d760 0x7fcd6406fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:41.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:41.590+0000 7fcd77400640 1 -- 172.21.15.107:0/2918857636 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fcd6c08d070 con 0x7fcd80104cc0 2024-03-04T13:56:41.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:41.590+0000 7fcd7f400640 1 --2- 172.21.15.107:0/2918857636 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fcd6406d760 0x7fcd6406fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:56:41.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:41.591+0000 7fcd7f400640 1 --2- 172.21.15.107:0/2918857636 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fcd6406d760 0x7fcd6406fc10 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fcd70034040 tx=0x7fcd70009da0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:56:41.595 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:56:41 smithi107 ceph-mon[112578]: from='client.? 172.21.15.107:0/3270935878' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 31, "format": "json"}]: dispatch 2024-03-04T13:56:41.595 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:56:41 smithi107 ceph-mon[112578]: pgmap v317: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:56:41.595 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:56:41 smithi107 ceph-mon[112578]: from='client.? 172.21.15.107:0/3353472365' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 32, "format": "json"}]: dispatch 2024-03-04T13:56:41.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:41.595+0000 7fcd77400640 1 -- 172.21.15.107:0/2918857636 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fcd6c0569b0 con 0x7fcd80104cc0 2024-03-04T13:56:41.612 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:56:41 smithi169 ceph-mon[86183]: from='client.? 172.21.15.107:0/3270935878' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 31, "format": "json"}]: dispatch 2024-03-04T13:56:41.612 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:56:41 smithi169 ceph-mon[86183]: pgmap v317: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:56:41.612 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:56:41 smithi169 ceph-mon[86183]: from='client.? 172.21.15.107:0/3353472365' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 32, "format": "json"}]: dispatch 2024-03-04T13:56:41.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:41.837+0000 7fcd85000640 1 -- 172.21.15.107:0/2918857636 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 34, "format": "json"} v 0) v1 -- 0x7fcd8010f7c0 con 0x7fcd80104cc0 2024-03-04T13:56:41.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:41.838+0000 7fcd77400640 1 -- 172.21.15.107:0/2918857636 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 34, "format": "json"}]=0 dumped fsmap epoch 34 v36) v1 ==== 107+0+5010 (secure 0 0 0) 0x7fcd6c0126b0 con 0x7fcd80104cc0 2024-03-04T13:56:41.840 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:56:41.840 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34436,"name":"cephfs.smithi107.thycho","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6829/3010096376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3010096376},{"type":"v1","addr":"172.21.15.107:6829","nonce":3010096376}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":44243,"name":"cephfs.smithi169.jusudl","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6825/1336978861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":1336978861},{"type":"v1","addr":"172.21.15.169:6825","nonce":1336978861}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25},{"gid":44259,"name":"cephfs.smithi169.bxitzk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6827/4015044684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":4015044684},{"type":"v1","addr":"172.21.15.169:6827","nonce":4015044684}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-04T13:38:30.854004+0000","modified":"2024-03-04T13:54:20.846869+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":86,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34432},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34432":{"gid":34432,"name":"cephfs.smithi107.riexmm","rank":0,"incarnation":30,"state":"up:active","state_seq":24,"addr":"172.21.15.107:6827/1144672731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1144672731},{"type":"v1","addr":"172.21.15.107:6827","nonce":1144672731}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-04T13:56:41.840 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 34 2024-03-04T13:56:41.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:41.841+0000 7fcd85000640 1 -- 172.21.15.107:0/2918857636 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fcd6406d760 msgr2=0x7fcd6406fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:41.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:41.841+0000 7fcd85000640 1 --2- 172.21.15.107:0/2918857636 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fcd6406d760 0x7fcd6406fc10 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fcd70034040 tx=0x7fcd70009da0 comp rx=0 tx=0).stop 2024-03-04T13:56:41.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:41.841+0000 7fcd85000640 1 -- 172.21.15.107:0/2918857636 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd80104cc0 msgr2=0x7fcd80115160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:41.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:41.841+0000 7fcd85000640 1 --2- 172.21.15.107:0/2918857636 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd80104cc0 0x7fcd80115160 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c00c830 tx=0x7fcd6c004430 comp rx=0 tx=0).stop 2024-03-04T13:56:41.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:41.841+0000 7fcd85000640 1 -- 172.21.15.107:0/2918857636 shutdown_connections 2024-03-04T13:56:41.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:41.841+0000 7fcd85000640 1 --2- 172.21.15.107:0/2918857636 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fcd6406d760 0x7fcd6406fc10 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:41.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:41.841+0000 7fcd85000640 1 --2- 172.21.15.107:0/2918857636 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd80104cc0 0x7fcd80115160 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:41.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:41.841+0000 7fcd85000640 1 --2- 172.21.15.107:0/2918857636 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcd80104320 0x7fcd80114c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:41.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:41.841+0000 7fcd85000640 1 -- 172.21.15.107:0/2918857636 >> 172.21.15.107:0/2918857636 conn(0x7fcd800f9f90 msgr2=0x7fcd800fba40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:56:41.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:41.841+0000 7fcd85000640 1 -- 172.21.15.107:0/2918857636 shutdown_connections 2024-03-04T13:56:41.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:41.841+0000 7fcd85000640 1 -- 172.21.15.107:0/2918857636 wait complete. 2024-03-04T13:56:41.860 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 34 2024-03-04T13:56:41.860 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 35 2024-03-04T13:56:42.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:42.063+0000 7f4bd2400640 1 -- 172.21.15.107:0/3438349534 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4bcc070150 msgr2=0x7f4bcc108450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:42.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:42.063+0000 7f4bd2400640 1 --2- 172.21.15.107:0/3438349534 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4bcc070150 0x7f4bcc108450 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f4bb80048d0 tx=0x7f4bb8009da0 comp rx=0 tx=0).stop 2024-03-04T13:56:42.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:42.063+0000 7f4bd2400640 1 -- 172.21.15.107:0/3438349534 shutdown_connections 2024-03-04T13:56:42.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:42.063+0000 7f4bd2400640 1 --2- 172.21.15.107:0/3438349534 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4bcc070150 0x7f4bcc108450 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:42.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:42.063+0000 7f4bd2400640 1 --2- 172.21.15.107:0/3438349534 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4bcc06f840 0x7f4bcc06fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:42.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:42.063+0000 7f4bd2400640 1 -- 172.21.15.107:0/3438349534 >> 172.21.15.107:0/3438349534 conn(0x7f4bcc0f7cf0 msgr2=0x7f4bcc0fa100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:56:42.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:42.063+0000 7f4bd2400640 1 -- 172.21.15.107:0/3438349534 shutdown_connections 2024-03-04T13:56:42.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:42.063+0000 7f4bd2400640 1 -- 172.21.15.107:0/3438349534 wait complete. 2024-03-04T13:56:42.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:42.064+0000 7f4bd2400640 1 Processor -- start 2024-03-04T13:56:42.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:42.064+0000 7f4bd2400640 1 -- start start 2024-03-04T13:56:42.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:42.065+0000 7f4bd2400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4bcc06f840 0x7f4bcc114c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:42.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:42.065+0000 7f4bd2400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4bcc070150 0x7f4bcc115150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:42.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:42.065+0000 7f4bd2400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bcc115750 con 0x7f4bcc06f840 2024-03-04T13:56:42.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:42.065+0000 7f4bd2400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bcc10ece0 con 0x7f4bcc070150 2024-03-04T13:56:42.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:42.065+0000 7f4bd1000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4bcc06f840 0x7f4bcc114c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:56:42.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:42.065+0000 7f4bd1000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4bcc06f840 0x7f4bcc114c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47252/0 (socket says 172.21.15.107:47252) 2024-03-04T13:56:42.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:42.065+0000 7f4bd1000640 1 -- 172.21.15.107:0/789258664 learned_addr learned my addr 172.21.15.107:0/789258664 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:56:42.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:42.065+0000 7f4bd1000640 1 -- 172.21.15.107:0/789258664 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4bcc070150 msgr2=0x7f4bcc115150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:42.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:42.065+0000 7f4bd1000640 1 --2- 172.21.15.107:0/789258664 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4bcc070150 0x7f4bcc115150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:42.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:42.065+0000 7f4bd1000640 1 -- 172.21.15.107:0/789258664 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bb8005040 con 0x7f4bcc06f840 2024-03-04T13:56:42.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:42.065+0000 7f4bd1000640 1 --2- 172.21.15.107:0/789258664 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4bcc06f840 0x7f4bcc114c10 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f4bc000c830 tx=0x7f4bc0004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:56:42.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:42.066+0000 7f4bc8c00640 1 -- 172.21.15.107:0/789258664 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4bc000a850 con 0x7f4bcc06f840 2024-03-04T13:56:42.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:42.066+0000 7f4bc8c00640 1 -- 172.21.15.107:0/789258664 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4bc000ae80 con 0x7f4bcc06f840 2024-03-04T13:56:42.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:42.066+0000 7f4bd2400640 1 -- 172.21.15.107:0/789258664 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bcc10efc0 con 0x7f4bcc06f840 2024-03-04T13:56:42.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:42.066+0000 7f4bc8c00640 1 -- 172.21.15.107:0/789258664 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4bc0012ea0 con 0x7f4bcc06f840 2024-03-04T13:56:42.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:42.066+0000 7f4bd2400640 1 -- 172.21.15.107:0/789258664 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bcc10f4e0 con 0x7f4bcc06f840 2024-03-04T13:56:42.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:42.067+0000 7f4bd2400640 1 -- 172.21.15.107:0/789258664 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bcc04bcc0 con 0x7f4bcc06f840 2024-03-04T13:56:42.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:42.068+0000 7f4bc8c00640 1 -- 172.21.15.107:0/789258664 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f4bc0012400 con 0x7f4bcc06f840 2024-03-04T13:56:42.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:42.069+0000 7f4bc8c00640 1 --2- 172.21.15.107:0/789258664 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f4bb406d750 0x7f4bb406fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:42.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:42.069+0000 7f4bc8c00640 1 -- 172.21.15.107:0/789258664 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f4bc008d1a0 con 0x7f4bcc06f840 2024-03-04T13:56:42.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:42.069+0000 7f4bcbe00640 1 --2- 172.21.15.107:0/789258664 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f4bb406d750 0x7f4bb406fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:56:42.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:42.070+0000 7f4bcbe00640 1 --2- 172.21.15.107:0/789258664 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f4bb406d750 0x7f4bb406fc00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4bb802bda0 tx=0x7f4bb8009da0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:56:42.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:42.073+0000 7f4bc8c00640 1 -- 172.21.15.107:0/789258664 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4bc00569a0 con 0x7f4bcc06f840 2024-03-04T13:56:42.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:42.304+0000 7f4bd2400640 1 -- 172.21.15.107:0/789258664 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 35, "format": "json"} v 0) v1 -- 0x7f4bcc10b480 con 0x7f4bcc06f840 2024-03-04T13:56:42.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:42.306+0000 7f4bc8c00640 1 -- 172.21.15.107:0/789258664 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 35, "format": "json"}]=0 dumped fsmap epoch 35 v36) v1 ==== 107+0+5009 (secure 0 0 0) 0x7f4bcc10b480 con 0x7f4bcc06f840 2024-03-04T13:56:42.308 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:56:42.308 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34436,"name":"cephfs.smithi107.thycho","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6829/3010096376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3010096376},{"type":"v1","addr":"172.21.15.107:6829","nonce":3010096376}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":44243,"name":"cephfs.smithi169.jusudl","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6825/1336978861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":1336978861},{"type":"v1","addr":"172.21.15.169:6825","nonce":1336978861}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25},{"gid":44259,"name":"cephfs.smithi169.bxitzk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6827/4015044684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6826","nonce":4015044684},{"type":"v1","addr":"172.21.15.169:6827","nonce":4015044684}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":35,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-04T13:38:30.854004+0000","modified":"2024-03-04T13:54:21.850102+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":86,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34432},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34432":{"gid":34432,"name":"cephfs.smithi107.riexmm","rank":0,"incarnation":30,"state":"up:active","state_seq":24,"addr":"172.21.15.107:6827/1144672731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1144672731},{"type":"v1","addr":"172.21.15.107:6827","nonce":1144672731}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-04T13:56:42.308 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 35 2024-03-04T13:56:42.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:42.309+0000 7f4bd2400640 1 -- 172.21.15.107:0/789258664 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f4bb406d750 msgr2=0x7f4bb406fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:42.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:42.309+0000 7f4bd2400640 1 --2- 172.21.15.107:0/789258664 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f4bb406d750 0x7f4bb406fc00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4bb802bda0 tx=0x7f4bb8009da0 comp rx=0 tx=0).stop 2024-03-04T13:56:42.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:42.309+0000 7f4bd2400640 1 -- 172.21.15.107:0/789258664 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4bcc06f840 msgr2=0x7f4bcc114c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:42.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:42.309+0000 7f4bd2400640 1 --2- 172.21.15.107:0/789258664 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4bcc06f840 0x7f4bcc114c10 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f4bc000c830 tx=0x7f4bc0004430 comp rx=0 tx=0).stop 2024-03-04T13:56:42.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:42.309+0000 7f4bd2400640 1 -- 172.21.15.107:0/789258664 shutdown_connections 2024-03-04T13:56:42.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:42.309+0000 7f4bd2400640 1 --2- 172.21.15.107:0/789258664 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f4bb406d750 0x7f4bb406fc00 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:42.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:42.309+0000 7f4bd2400640 1 --2- 172.21.15.107:0/789258664 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4bcc070150 0x7f4bcc115150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:42.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:42.309+0000 7f4bd2400640 1 --2- 172.21.15.107:0/789258664 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4bcc06f840 0x7f4bcc114c10 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:42.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:42.309+0000 7f4bd2400640 1 -- 172.21.15.107:0/789258664 >> 172.21.15.107:0/789258664 conn(0x7f4bcc0f7cf0 msgr2=0x7f4bcc102c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:56:42.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:42.309+0000 7f4bd2400640 1 -- 172.21.15.107:0/789258664 shutdown_connections 2024-03-04T13:56:42.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:42.309+0000 7f4bd2400640 1 -- 172.21.15.107:0/789258664 wait complete. 2024-03-04T13:56:42.328 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-03-04T13:56:42.339 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-03-04T13:56:42.339 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-03-04T13:56:42.339 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-04T13:56:42.388 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-03-04T13:56:42.388 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-04T13:56:42.482 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-04T13:56:42.595 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:56:42 smithi107 ceph-mon[112578]: from='client.? 172.21.15.107:0/2854660080' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 33, "format": "json"}]: dispatch 2024-03-04T13:56:42.595 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:56:42 smithi107 ceph-mon[112578]: from='client.? 172.21.15.107:0/2918857636' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 34, "format": "json"}]: dispatch 2024-03-04T13:56:42.595 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:56:42 smithi107 ceph-mon[112578]: from='client.? 172.21.15.107:0/789258664' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 35, "format": "json"}]: dispatch 2024-03-04T13:56:42.612 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:56:42 smithi169 ceph-mon[86183]: from='client.? 172.21.15.107:0/2854660080' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 33, "format": "json"}]: dispatch 2024-03-04T13:56:42.612 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:56:42 smithi169 ceph-mon[86183]: from='client.? 172.21.15.107:0/2918857636' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 34, "format": "json"}]: dispatch 2024-03-04T13:56:42.612 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:56:42 smithi169 ceph-mon[86183]: from='client.? 172.21.15.107:0/789258664' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 35, "format": "json"}]: dispatch 2024-03-04T13:56:42.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:42.680+0000 7efc6be00640 1 -- 172.21.15.107:0/3271981799 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc640719d0 msgr2=0x7efc64071da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:42.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:42.680+0000 7efc6be00640 1 --2- 172.21.15.107:0/3271981799 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc640719d0 0x7efc64071da0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7efc580048d0 tx=0x7efc58009da0 comp rx=0 tx=0).stop 2024-03-04T13:56:42.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:42.680+0000 7efc6be00640 1 -- 172.21.15.107:0/3271981799 shutdown_connections 2024-03-04T13:56:42.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:42.680+0000 7efc6be00640 1 --2- 172.21.15.107:0/3271981799 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efc640722e0 0x7efc6410cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:42.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:42.680+0000 7efc6be00640 1 --2- 172.21.15.107:0/3271981799 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc640719d0 0x7efc64071da0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:42.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:42.680+0000 7efc6be00640 1 -- 172.21.15.107:0/3271981799 >> 172.21.15.107:0/3271981799 conn(0x7efc640fa090 msgr2=0x7efc640fc4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:56:42.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:42.681+0000 7efc6be00640 1 -- 172.21.15.107:0/3271981799 shutdown_connections 2024-03-04T13:56:42.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:42.681+0000 7efc6be00640 1 -- 172.21.15.107:0/3271981799 wait complete. 2024-03-04T13:56:42.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:42.681+0000 7efc6be00640 1 Processor -- start 2024-03-04T13:56:42.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:42.682+0000 7efc6be00640 1 -- start start 2024-03-04T13:56:42.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:42.682+0000 7efc6be00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efc640719d0 0x7efc641190a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:42.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:42.682+0000 7efc6be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc640722e0 0x7efc641195e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:42.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:42.682+0000 7efc6be00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc64119be0 con 0x7efc640722e0 2024-03-04T13:56:42.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:42.682+0000 7efc6be00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc64113170 con 0x7efc640719d0 2024-03-04T13:56:42.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:42.682+0000 7efc68a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc640722e0 0x7efc641195e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:56:42.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:42.682+0000 7efc68a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc640722e0 0x7efc641195e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47266/0 (socket says 172.21.15.107:47266) 2024-03-04T13:56:42.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:42.682+0000 7efc68a00640 1 -- 172.21.15.107:0/2395731933 learned_addr learned my addr 172.21.15.107:0/2395731933 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:56:42.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:42.683+0000 7efc68a00640 1 -- 172.21.15.107:0/2395731933 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efc640719d0 msgr2=0x7efc641190a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:42.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:42.683+0000 7efc68a00640 1 --2- 172.21.15.107:0/2395731933 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efc640719d0 0x7efc641190a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:42.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:42.683+0000 7efc68a00640 1 -- 172.21.15.107:0/2395731933 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc58005040 con 0x7efc640722e0 2024-03-04T13:56:42.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:42.683+0000 7efc68a00640 1 --2- 172.21.15.107:0/2395731933 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc640722e0 0x7efc641195e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7efc5000c830 tx=0x7efc50004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:56:42.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:42.684+0000 7efc61600640 1 -- 172.21.15.107:0/2395731933 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efc5000a850 con 0x7efc640722e0 2024-03-04T13:56:42.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:42.684+0000 7efc61600640 1 -- 172.21.15.107:0/2395731933 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efc5000ae80 con 0x7efc640722e0 2024-03-04T13:56:42.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:42.684+0000 7efc6be00640 1 -- 172.21.15.107:0/2395731933 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc64113450 con 0x7efc640722e0 2024-03-04T13:56:42.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:42.684+0000 7efc61600640 1 -- 172.21.15.107:0/2395731933 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efc50012ea0 con 0x7efc640722e0 2024-03-04T13:56:42.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:42.684+0000 7efc6be00640 1 -- 172.21.15.107:0/2395731933 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc64113970 con 0x7efc640722e0 2024-03-04T13:56:42.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:42.685+0000 7efc6be00640 1 -- 172.21.15.107:0/2395731933 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc64064030 con 0x7efc640722e0 2024-03-04T13:56:42.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:42.687+0000 7efc61600640 1 -- 172.21.15.107:0/2395731933 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7efc50012400 con 0x7efc640722e0 2024-03-04T13:56:42.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:42.687+0000 7efc61600640 1 --2- 172.21.15.107:0/2395731933 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7efc4c06d750 0x7efc4c06fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:42.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:42.688+0000 7efc61600640 1 -- 172.21.15.107:0/2395731933 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7efc5008d070 con 0x7efc640722e0 2024-03-04T13:56:42.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:42.688+0000 7efc69400640 1 --2- 172.21.15.107:0/2395731933 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7efc4c06d750 0x7efc4c06fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:56:42.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:42.689+0000 7efc69400640 1 --2- 172.21.15.107:0/2395731933 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7efc4c06d750 0x7efc4c06fc00 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7efc58034040 tx=0x7efc58009da0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:56:42.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:42.692+0000 7efc61600640 1 -- 172.21.15.107:0/2395731933 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7efc500569b0 con 0x7efc640722e0 2024-03-04T13:56:42.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:42.890+0000 7efc6be00640 1 -- 172.21.15.107:0/2395731933 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efc6410fa10 con 0x7efc640722e0 2024-03-04T13:56:42.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:42.894+0000 7efc61600640 1 -- 172.21.15.107:0/2395731933 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+16693 (secure 0 0 0) 0x7efc500126b0 con 0x7efc640722e0 2024-03-04T13:56:42.896 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:56:42.896 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":86,"fsid":"1fc09a58-da2b-11ee-95c6-87774f69a715","created":"2024-03-04T13:29:42.985888+0000","modified":"2024-03-04T13:53:58.927176+0000","last_up_change":"2024-03-04T13:52:27.869095+0000","last_in_change":"2024-03-04T13:36:31.424613+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T13:35:17.915858+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-03-04T13:38:29.362118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-03-04T13:38:30.373574+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c857e387-3d2b-478d-9807-24ea8b9fe02d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":50,"up_from":54,"up_thru":79,"down_at":51,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":3996509230},{"type":"v1","addr":"172.21.15.107:6803","nonce":3996509230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":3996509230},{"type":"v1","addr":"172.21.15.107:6805","nonce":3996509230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":3996509230},{"type":"v1","addr":"172.21.15.107:6809","nonce":3996509230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":3996509230},{"type":"v1","addr":"172.21.15.107:6807","nonce":3996509230}]},"public_addr":"172.21.15.107:6803/3996509230","cluster_addr":"172.21.15.107:6805/3996509230","heartbeat_back_addr":"172.21.15.107:6809/3996509230","heartbeat_front_addr":"172.21.15.107:6807/3996509230","state":["exists","up"]},{"osd":1,"uuid":"fa29dc8f-419b-4318-b43e-e13fffd87204","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":55,"up_from":59,"up_thru":79,"down_at":56,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1164427580},{"type":"v1","addr":"172.21.15.107:6811","nonce":1164427580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1164427580},{"type":"v1","addr":"172.21.15.107:6813","nonce":1164427580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1164427580},{"type":"v1","addr":"172.21.15.107:6817","nonce":1164427580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1164427580},{"type":"v1","addr":"172.21.15.107:6815","nonce":1164427580}]},"public_addr":"172.21.15.107:6811/1164427580","cluster_addr":"172.21.15.107:6813/1164427580","heartbeat_back_addr":"172.21.15.107:6817/1164427580","heartbeat_front_addr":"172.21.15.107:6815/1164427580","state":["exists","up"]},{"osd":2,"uuid":"580a07f0-7bd4-4341-9c1f-b3383d64c720","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":60,"up_from":64,"up_thru":79,"down_at":61,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":150828071},{"type":"v1","addr":"172.21.15.107:6819","nonce":150828071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":150828071},{"type":"v1","addr":"172.21.15.107:6821","nonce":150828071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":150828071},{"type":"v1","addr":"172.21.15.107:6825","nonce":150828071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":150828071},{"type":"v1","addr":"172.21.15.107:6823","nonce":150828071}]},"public_addr":"172.21.15.107:6819/150828071","cluster_addr":"172.21.15.107:6821/150828071","heartbeat_back_addr":"172.21.15.107:6825/150828071","heartbeat_front_addr":"172.21.15.107:6823/150828071","state":["exists","up"]},{"osd":3,"uuid":"7acac14d-998a-4651-b78c-e42584e13b43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":27,"last_clean_end":65,"up_from":69,"up_thru":79,"down_at":66,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3021051994},{"type":"v1","addr":"172.21.15.169:6801","nonce":3021051994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3021051994},{"type":"v1","addr":"172.21.15.169:6803","nonce":3021051994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3021051994},{"type":"v1","addr":"172.21.15.169:6807","nonce":3021051994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3021051994},{"type":"v1","addr":"172.21.15.169:6805","nonce":3021051994}]},"public_addr":"172.21.15.169:6801/3021051994","cluster_addr":"172.21.15.169:6803/3021051994","heartbeat_back_addr":"172.21.15.169:6807/3021051994","heartbeat_front_addr":"172.21.15.169:6805/3021051994","state":["exists","up"]},{"osd":4,"uuid":"01d2b77c-66ec-4b50-8170-43d95eeae471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":35,"last_clean_end":70,"up_from":74,"up_thru":79,"down_at":71,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":56453577},{"type":"v1","addr":"172.21.15.169:6809","nonce":56453577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":56453577},{"type":"v1","addr":"172.21.15.169:6811","nonce":56453577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":56453577},{"type":"v1","addr":"172.21.15.169:6815","nonce":56453577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":56453577},{"type":"v1","addr":"172.21.15.169:6813","nonce":56453577}]},"public_addr":"172.21.15.169:6809/56453577","cluster_addr":"172.21.15.169:6811/56453577","heartbeat_back_addr":"172.21.15.169:6815/56453577","heartbeat_front_addr":"172.21.15.169:6813/56453577","state":["exists","up"]},{"osd":5,"uuid":"0164ead7-c5d0-4541-bf22-f671558f8607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":40,"last_clean_end":75,"up_from":79,"up_thru":79,"down_at":76,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":374254288},{"type":"v1","addr":"172.21.15.169:6817","nonce":374254288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":374254288},{"type":"v1","addr":"172.21.15.169:6819","nonce":374254288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":374254288},{"type":"v1","addr":"172.21.15.169:6823","nonce":374254288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":374254288},{"type":"v1","addr":"172.21.15.169:6821","nonce":374254288}]},"public_addr":"172.21.15.169:6817/374254288","cluster_addr":"172.21.15.169:6819/374254288","heartbeat_back_addr":"172.21.15.169:6823/374254288","heartbeat_front_addr":"172.21.15.169:6821/374254288","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T13:33:59.729659+0000","dead_epoch":50},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T13:34:36.600326+0000","dead_epoch":55},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T13:35:14.068513+0000","dead_epoch":60},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T13:35:46.608007+0000","dead_epoch":65},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T13:36:19.595680+0000","dead_epoch":70},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T13:36:52.586933+0000","dead_epoch":75}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.169:6827/3696090493":"2024-03-05T13:53:58.927120+0000","172.21.15.169:6825/2994866076":"2024-03-05T13:53:29.273990+0000","172.21.15.107:0/395365799":"2024-03-05T13:41:21.904328+0000","172.21.15.107:0/2217624526":"2024-03-05T13:41:21.904328+0000","172.21.15.107:6801/2371648646":"2024-03-05T13:30:56.353578+0000","172.21.15.107:6800/3848495603":"2024-03-05T13:41:21.904328+0000","172.21.15.107:0/3451280196":"2024-03-05T13:30:16.230256+0000","172.21.15.107:6827/3814968386":"2024-03-05T13:38:46.993392+0000","172.21.15.107:0/3881311685":"2024-03-05T13:47:24.592361+0000","172.21.15.107:0/2723084639":"2024-03-05T13:32:27.229455+0000","172.21.15.107:6801/1723390229":"2024-03-05T13:30:16.230256+0000","172.21.15.107:0/248094819":"2024-03-05T13:47:24.592361+0000","172.21.15.107:6800/1723390229":"2024-03-05T13:30:16.230256+0000","172.21.15.169:6828/3524931243":"2024-03-05T13:43:38.105893+0000","172.21.15.107:0/601795289":"2024-03-05T13:32:27.229455+0000","172.21.15.169:0/1961312808":"2024-03-05T13:43:38.105893+0000","172.21.15.107:6801/2040330534":"2024-03-05T13:32:27.229455+0000","172.21.15.107:0/136342223":"2024-03-05T13:30:56.353578+0000","172.21.15.169:0/692246057":"2024-03-05T13:43:38.105893+0000","172.21.15.107:0/1780539770":"2024-03-05T13:30:16.230256+0000","172.21.15.107:0/3355652811":"2024-03-05T13:41:21.904328+0000","172.21.15.169:6829/3524931243":"2024-03-05T13:43:38.105893+0000","172.21.15.107:6800/2040330534":"2024-03-05T13:32:27.229455+0000","172.21.15.107:6801/1613321046":"2024-03-05T13:47:24.592361+0000","172.21.15.107:0/3503768751":"2024-03-05T13:32:27.229455+0000","172.21.15.169:0/2785350328":"2024-03-05T13:43:38.105893+0000","172.21.15.107:0/2928681569":"2024-03-05T13:30:56.353578+0000","172.21.15.107:0/2314831039":"2024-03-05T13:30:56.353578+0000","172.21.15.169:0/842094828":"2024-03-05T13:43:38.105893+0000","172.21.15.169:6824/2994866076":"2024-03-05T13:53:29.273990+0000","172.21.15.169:6826/3696090493":"2024-03-05T13:53:58.927120+0000","172.21.15.107:6801/3848495603":"2024-03-05T13:41:21.904328+0000","172.21.15.107:6826/3814968386":"2024-03-05T13:38:46.993392+0000","172.21.15.107:0/2605062903":"2024-03-05T13:41:21.904328+0000","172.21.15.107:6800/2371648646":"2024-03-05T13:30:56.353578+0000","172.21.15.107:6800/1613321046":"2024-03-05T13:47:24.592361+0000","172.21.15.107:0/2357638826":"2024-03-05T13:30:16.230256+0000","172.21.15.107:0/1849554796":"2024-03-05T13:47:24.592361+0000","172.21.15.107:0/2439430884":"2024-03-05T13:47:24.592361+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T13:56:42.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:42.897+0000 7efc6be00640 1 -- 172.21.15.107:0/2395731933 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7efc4c06d750 msgr2=0x7efc4c06fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:42.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:42.897+0000 7efc6be00640 1 --2- 172.21.15.107:0/2395731933 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7efc4c06d750 0x7efc4c06fc00 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7efc58034040 tx=0x7efc58009da0 comp rx=0 tx=0).stop 2024-03-04T13:56:42.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:42.897+0000 7efc6be00640 1 -- 172.21.15.107:0/2395731933 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc640722e0 msgr2=0x7efc641195e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:42.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:42.897+0000 7efc6be00640 1 --2- 172.21.15.107:0/2395731933 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc640722e0 0x7efc641195e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7efc5000c830 tx=0x7efc50004430 comp rx=0 tx=0).stop 2024-03-04T13:56:42.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:42.897+0000 7efc6be00640 1 -- 172.21.15.107:0/2395731933 shutdown_connections 2024-03-04T13:56:42.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:42.897+0000 7efc6be00640 1 --2- 172.21.15.107:0/2395731933 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7efc4c06d750 0x7efc4c06fc00 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:42.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:42.897+0000 7efc6be00640 1 --2- 172.21.15.107:0/2395731933 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc640722e0 0x7efc641195e0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:42.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:42.897+0000 7efc6be00640 1 --2- 172.21.15.107:0/2395731933 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efc640719d0 0x7efc641190a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:42.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:42.897+0000 7efc6be00640 1 -- 172.21.15.107:0/2395731933 >> 172.21.15.107:0/2395731933 conn(0x7efc640fa090 msgr2=0x7efc640facf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:56:42.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:42.897+0000 7efc6be00640 1 -- 172.21.15.107:0/2395731933 shutdown_connections 2024-03-04T13:56:42.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:42.897+0000 7efc6be00640 1 -- 172.21.15.107:0/2395731933 wait complete. 2024-03-04T13:56:42.916 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-04T13:35:17.915858+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-03-04T13:38:29.362118+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-03-04T13:38:30.373574+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-04T13:56:42.917 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-04T13:56:43.121 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.119+0000 7f2828400640 1 -- 172.21.15.107:0/1733092112 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2820100110 msgr2=0x7f2820100560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:43.121 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.119+0000 7f2828400640 1 --2- 172.21.15.107:0/1733092112 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2820100110 0x7f2820100560 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f280c0048d0 tx=0x7f280c009da0 comp rx=0 tx=0).stop 2024-03-04T13:56:43.121 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.119+0000 7f2828400640 1 -- 172.21.15.107:0/1733092112 shutdown_connections 2024-03-04T13:56:43.121 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.119+0000 7f2828400640 1 --2- 172.21.15.107:0/1733092112 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2820100110 0x7f2820100560 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:43.121 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.119+0000 7f2828400640 1 --2- 172.21.15.107:0/1733092112 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f28200fe9c0 0x7f28200fed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:43.121 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.119+0000 7f2828400640 1 -- 172.21.15.107:0/1733092112 >> 172.21.15.107:0/1733092112 conn(0x7f28200fa160 msgr2=0x7f28200fc570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:56:43.121 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.119+0000 7f2828400640 1 -- 172.21.15.107:0/1733092112 shutdown_connections 2024-03-04T13:56:43.121 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.119+0000 7f2828400640 1 -- 172.21.15.107:0/1733092112 wait complete. 2024-03-04T13:56:43.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.120+0000 7f2828400640 1 Processor -- start 2024-03-04T13:56:43.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.120+0000 7f2828400640 1 -- start start 2024-03-04T13:56:43.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.120+0000 7f2828400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28200fe9c0 0x7f2820114da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:43.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.120+0000 7f2828400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2820100110 0x7f28201152e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:43.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.120+0000 7f2828400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28201158e0 con 0x7f28200fe9c0 2024-03-04T13:56:43.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.120+0000 7f2828400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f282010ee70 con 0x7f2820100110 2024-03-04T13:56:43.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.120+0000 7f2825a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28200fe9c0 0x7f2820114da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:56:43.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.120+0000 7f2825a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28200fe9c0 0x7f2820114da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47284/0 (socket says 172.21.15.107:47284) 2024-03-04T13:56:43.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.121+0000 7f2825a00640 1 -- 172.21.15.107:0/2513028257 learned_addr learned my addr 172.21.15.107:0/2513028257 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:56:43.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.121+0000 7f2825000640 1 --2- 172.21.15.107:0/2513028257 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2820100110 0x7f28201152e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:56:43.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.121+0000 7f2825a00640 1 -- 172.21.15.107:0/2513028257 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2820100110 msgr2=0x7f28201152e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:43.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.121+0000 7f2825a00640 1 --2- 172.21.15.107:0/2513028257 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2820100110 0x7f28201152e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:43.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.121+0000 7f2825a00640 1 -- 172.21.15.107:0/2513028257 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f280c005040 con 0x7f28200fe9c0 2024-03-04T13:56:43.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.121+0000 7f2825000640 1 --2- 172.21.15.107:0/2513028257 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2820100110 0x7f28201152e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T13:56:43.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.121+0000 7f2825a00640 1 --2- 172.21.15.107:0/2513028257 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28200fe9c0 0x7f2820114da0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f281400c810 tx=0x7f28140043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:56:43.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.121+0000 7f281d600640 1 -- 172.21.15.107:0/2513028257 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f281400a820 con 0x7f28200fe9c0 2024-03-04T13:56:43.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.121+0000 7f2828400640 1 -- 172.21.15.107:0/2513028257 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f282010f150 con 0x7f28200fe9c0 2024-03-04T13:56:43.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.121+0000 7f281d600640 1 -- 172.21.15.107:0/2513028257 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f281400ae50 con 0x7f28200fe9c0 2024-03-04T13:56:43.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.122+0000 7f281d600640 1 -- 172.21.15.107:0/2513028257 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f28140115c0 con 0x7f28200fe9c0 2024-03-04T13:56:43.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.122+0000 7f2828400640 1 -- 172.21.15.107:0/2513028257 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f282010f670 con 0x7f28200fe9c0 2024-03-04T13:56:43.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.123+0000 7f2828400640 1 -- 172.21.15.107:0/2513028257 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2820064030 con 0x7f28200fe9c0 2024-03-04T13:56:43.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.124+0000 7f281d600640 1 -- 172.21.15.107:0/2513028257 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f281400a980 con 0x7f28200fe9c0 2024-03-04T13:56:43.126 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.125+0000 7f281d600640 1 --2- 172.21.15.107:0/2513028257 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f280806d750 0x7f280806fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:43.126 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.125+0000 7f281d600640 1 -- 172.21.15.107:0/2513028257 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f281408d4b0 con 0x7f28200fe9c0 2024-03-04T13:56:43.126 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.125+0000 7f2825000640 1 --2- 172.21.15.107:0/2513028257 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f280806d750 0x7f280806fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:56:43.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.126+0000 7f2825000640 1 --2- 172.21.15.107:0/2513028257 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f280806d750 0x7f280806fc00 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f280c02bda0 tx=0x7f280c009da0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:56:43.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.130+0000 7f281d600640 1 -- 172.21.15.107:0/2513028257 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2814056cb0 con 0x7f28200fe9c0 2024-03-04T13:56:43.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.323+0000 7f2828400640 1 -- 172.21.15.107:0/2513028257 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f282010f950 con 0x7f28200fe9c0 2024-03-04T13:56:43.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.324+0000 7f281d600640 1 -- 172.21.15.107:0/2513028257 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) v1 ==== 93+0+10 (secure 0 0 0) 0x7f281405a370 con 0x7f28200fe9c0 2024-03-04T13:56:43.325 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 1 2024-03-04T13:56:43.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.326+0000 7f2828400640 1 -- 172.21.15.107:0/2513028257 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f280806d750 msgr2=0x7f280806fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:43.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.326+0000 7f2828400640 1 --2- 172.21.15.107:0/2513028257 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f280806d750 0x7f280806fc00 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f280c02bda0 tx=0x7f280c009da0 comp rx=0 tx=0).stop 2024-03-04T13:56:43.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.326+0000 7f2828400640 1 -- 172.21.15.107:0/2513028257 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28200fe9c0 msgr2=0x7f2820114da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:43.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.326+0000 7f2828400640 1 --2- 172.21.15.107:0/2513028257 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28200fe9c0 0x7f2820114da0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f281400c810 tx=0x7f28140043c0 comp rx=0 tx=0).stop 2024-03-04T13:56:43.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.326+0000 7f2828400640 1 -- 172.21.15.107:0/2513028257 shutdown_connections 2024-03-04T13:56:43.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.326+0000 7f2828400640 1 --2- 172.21.15.107:0/2513028257 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f280806d750 0x7f280806fc00 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:43.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.326+0000 7f2828400640 1 --2- 172.21.15.107:0/2513028257 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2820100110 0x7f28201152e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:43.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.326+0000 7f2828400640 1 --2- 172.21.15.107:0/2513028257 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28200fe9c0 0x7f2820114da0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:43.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.326+0000 7f2828400640 1 -- 172.21.15.107:0/2513028257 >> 172.21.15.107:0/2513028257 conn(0x7f28200fa160 msgr2=0x7f28200fbc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:56:43.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.326+0000 7f2828400640 1 -- 172.21.15.107:0/2513028257 shutdown_connections 2024-03-04T13:56:43.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.326+0000 7f2828400640 1 -- 172.21.15.107:0/2513028257 wait complete. 2024-03-04T13:56:43.346 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-03-04T13:56:43.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.538+0000 7efe16000640 1 -- 172.21.15.107:0/2189551316 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe10108760 msgr2=0x7efe10108b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:43.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.538+0000 7efe16000640 1 --2- 172.21.15.107:0/2189551316 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe10108760 0x7efe10108b30 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7efe040048d0 tx=0x7efe04009da0 comp rx=0 tx=0).stop 2024-03-04T13:56:43.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.538+0000 7efe16000640 1 -- 172.21.15.107:0/2189551316 shutdown_connections 2024-03-04T13:56:43.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.538+0000 7efe16000640 1 --2- 172.21.15.107:0/2189551316 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe10102740 0x7efe10102bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:43.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.538+0000 7efe16000640 1 --2- 172.21.15.107:0/2189551316 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe10108760 0x7efe10108b30 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:43.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.538+0000 7efe16000640 1 -- 172.21.15.107:0/2189551316 >> 172.21.15.107:0/2189551316 conn(0x7efe10071fc0 msgr2=0x7efe100743d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:56:43.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.538+0000 7efe16000640 1 -- 172.21.15.107:0/2189551316 shutdown_connections 2024-03-04T13:56:43.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.538+0000 7efe16000640 1 -- 172.21.15.107:0/2189551316 wait complete. 2024-03-04T13:56:43.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.539+0000 7efe16000640 1 Processor -- start 2024-03-04T13:56:43.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.539+0000 7efe16000640 1 -- start start 2024-03-04T13:56:43.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.540+0000 7efe16000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe10102740 0x7efe10119160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:43.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.540+0000 7efe16000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe10108760 0x7efe101196a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:43.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.540+0000 7efe16000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe10119ca0 con 0x7efe10108760 2024-03-04T13:56:43.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.540+0000 7efe16000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe10113230 con 0x7efe10102740 2024-03-04T13:56:43.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.540+0000 7efe0fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe10108760 0x7efe101196a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:56:43.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.540+0000 7efe0fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe10108760 0x7efe101196a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47314/0 (socket says 172.21.15.107:47314) 2024-03-04T13:56:43.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.540+0000 7efe0fe00640 1 -- 172.21.15.107:0/4159804547 learned_addr learned my addr 172.21.15.107:0/4159804547 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:56:43.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.540+0000 7efe14c00640 1 --2- 172.21.15.107:0/4159804547 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe10102740 0x7efe10119160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:56:43.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.541+0000 7efe0fe00640 1 -- 172.21.15.107:0/4159804547 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe10102740 msgr2=0x7efe10119160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:43.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.541+0000 7efe0fe00640 1 --2- 172.21.15.107:0/4159804547 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe10102740 0x7efe10119160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:43.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.541+0000 7efe0fe00640 1 -- 172.21.15.107:0/4159804547 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe04005040 con 0x7efe10108760 2024-03-04T13:56:43.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.541+0000 7efe0fe00640 1 --2- 172.21.15.107:0/4159804547 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe10108760 0x7efe101196a0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7efdfc002a50 tx=0x7efdfc004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:56:43.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.541+0000 7efe0cc00640 1 -- 172.21.15.107:0/4159804547 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efdfc007c30 con 0x7efe10108760 2024-03-04T13:56:43.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.541+0000 7efe16000640 1 -- 172.21.15.107:0/4159804547 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe10113510 con 0x7efe10108760 2024-03-04T13:56:43.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.541+0000 7efe0cc00640 1 -- 172.21.15.107:0/4159804547 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efdfc00f040 con 0x7efe10108760 2024-03-04T13:56:43.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.541+0000 7efe0cc00640 1 -- 172.21.15.107:0/4159804547 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efdfc00dad0 con 0x7efe10108760 2024-03-04T13:56:43.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.541+0000 7efe16000640 1 -- 172.21.15.107:0/4159804547 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe10113a30 con 0x7efe10108760 2024-03-04T13:56:43.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.542+0000 7efe16000640 1 -- 172.21.15.107:0/4159804547 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe1004bcc0 con 0x7efe10108760 2024-03-04T13:56:43.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.543+0000 7efe0cc00640 1 -- 172.21.15.107:0/4159804547 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7efdfc00dd40 con 0x7efe10108760 2024-03-04T13:56:43.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.545+0000 7efe0cc00640 1 --2- 172.21.15.107:0/4159804547 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7efdf806d760 0x7efdf806fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:43.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.546+0000 7efe0cc00640 1 -- 172.21.15.107:0/4159804547 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7efdfc08c860 con 0x7efe10108760 2024-03-04T13:56:43.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.546+0000 7efe14c00640 1 --2- 172.21.15.107:0/4159804547 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7efdf806d760 0x7efdf806fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:56:43.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.546+0000 7efe14c00640 1 --2- 172.21.15.107:0/4159804547 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7efdf806d760 0x7efdf806fc10 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7efe04034040 tx=0x7efe04009da0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:56:43.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.550+0000 7efe0cc00640 1 -- 172.21.15.107:0/4159804547 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7efdfc055fe0 con 0x7efe10108760 2024-03-04T13:56:43.595 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:56:43 smithi107 ceph-mon[112578]: pgmap v318: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:56:43.595 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:56:43 smithi107 ceph-mon[112578]: from='client.? 172.21.15.107:0/2395731933' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T13:56:43.595 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:56:43 smithi107 ceph-mon[112578]: from='client.? 172.21.15.107:0/2513028257' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-04T13:56:43.612 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:56:43 smithi169 ceph-mon[86183]: pgmap v318: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:56:43.612 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:56:43 smithi169 ceph-mon[86183]: from='client.? 172.21.15.107:0/2395731933' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T13:56:43.612 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:56:43 smithi169 ceph-mon[86183]: from='client.? 172.21.15.107:0/2513028257' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-04T13:56:43.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.745+0000 7efe16000640 1 -- 172.21.15.107:0/4159804547 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7efe1010f860 con 0x7efe10108760 2024-03-04T13:56:43.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.746+0000 7efe0cc00640 1 -- 172.21.15.107:0/4159804547 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v86) v1 ==== 107+0+11 (secure 0 0 0) 0x7efdfc0596a0 con 0x7efe10108760 2024-03-04T13:56:43.747 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 32 2024-03-04T13:56:43.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.748+0000 7efe16000640 1 -- 172.21.15.107:0/4159804547 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7efdf806d760 msgr2=0x7efdf806fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:43.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.748+0000 7efe16000640 1 --2- 172.21.15.107:0/4159804547 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7efdf806d760 0x7efdf806fc10 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7efe04034040 tx=0x7efe04009da0 comp rx=0 tx=0).stop 2024-03-04T13:56:43.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.748+0000 7efe16000640 1 -- 172.21.15.107:0/4159804547 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe10108760 msgr2=0x7efe101196a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:43.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.748+0000 7efe16000640 1 --2- 172.21.15.107:0/4159804547 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe10108760 0x7efe101196a0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7efdfc002a50 tx=0x7efdfc004600 comp rx=0 tx=0).stop 2024-03-04T13:56:43.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.748+0000 7efe16000640 1 -- 172.21.15.107:0/4159804547 shutdown_connections 2024-03-04T13:56:43.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.748+0000 7efe16000640 1 --2- 172.21.15.107:0/4159804547 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7efdf806d760 0x7efdf806fc10 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:43.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.748+0000 7efe16000640 1 --2- 172.21.15.107:0/4159804547 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe10108760 0x7efe101196a0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:43.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.748+0000 7efe16000640 1 --2- 172.21.15.107:0/4159804547 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe10102740 0x7efe10119160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:43.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.748+0000 7efe16000640 1 -- 172.21.15.107:0/4159804547 >> 172.21.15.107:0/4159804547 conn(0x7efe10071fc0 msgr2=0x7efe10104560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:56:43.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.749+0000 7efe16000640 1 -- 172.21.15.107:0/4159804547 shutdown_connections 2024-03-04T13:56:43.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.749+0000 7efe16000640 1 -- 172.21.15.107:0/4159804547 wait complete. 2024-03-04T13:56:43.767 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-03-04T13:56:43.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.974+0000 7f4c62000640 1 -- 172.21.15.107:0/466724593 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c5c073e80 msgr2=0x7f4c5c10cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:43.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.974+0000 7f4c62000640 1 --2- 172.21.15.107:0/466724593 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c5c073e80 0x7f4c5c10cd40 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f4c440048d0 tx=0x7f4c44009da0 comp rx=0 tx=0).stop 2024-03-04T13:56:43.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.974+0000 7f4c62000640 1 -- 172.21.15.107:0/466724593 shutdown_connections 2024-03-04T13:56:43.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.974+0000 7f4c62000640 1 --2- 172.21.15.107:0/466724593 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c5c073e80 0x7f4c5c10cd40 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:43.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.974+0000 7f4c62000640 1 --2- 172.21.15.107:0/466724593 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c5c073570 0x7f4c5c073940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:43.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.974+0000 7f4c62000640 1 -- 172.21.15.107:0/466724593 >> 172.21.15.107:0/466724593 conn(0x7f4c5c0fa160 msgr2=0x7f4c5c0fc570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:56:43.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.974+0000 7f4c62000640 1 -- 172.21.15.107:0/466724593 shutdown_connections 2024-03-04T13:56:43.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.974+0000 7f4c62000640 1 -- 172.21.15.107:0/466724593 wait complete. 2024-03-04T13:56:43.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.975+0000 7f4c62000640 1 Processor -- start 2024-03-04T13:56:43.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.975+0000 7f4c62000640 1 -- start start 2024-03-04T13:56:43.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.975+0000 7f4c62000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c5c073570 0x7f4c5c119290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:43.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.975+0000 7f4c62000640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c5c073e80 0x7f4c5c1197d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:43.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.975+0000 7f4c62000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c5c119dd0 con 0x7f4c5c073570 2024-03-04T13:56:43.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.975+0000 7f4c62000640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c5c113360 con 0x7f4c5c073e80 2024-03-04T13:56:43.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.975+0000 7f4c5b400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c5c073570 0x7f4c5c119290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:56:43.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.975+0000 7f4c5b400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c5c073570 0x7f4c5c119290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47330/0 (socket says 172.21.15.107:47330) 2024-03-04T13:56:43.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.975+0000 7f4c5b400640 1 -- 172.21.15.107:0/1540296991 learned_addr learned my addr 172.21.15.107:0/1540296991 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:56:43.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.976+0000 7f4c5b400640 1 -- 172.21.15.107:0/1540296991 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c5c073e80 msgr2=0x7f4c5c1197d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:43.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.976+0000 7f4c5b400640 1 --2- 172.21.15.107:0/1540296991 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c5c073e80 0x7f4c5c1197d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:43.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.976+0000 7f4c5b400640 1 -- 172.21.15.107:0/1540296991 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c44005040 con 0x7f4c5c073570 2024-03-04T13:56:43.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.976+0000 7f4c5b400640 1 --2- 172.21.15.107:0/1540296991 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c5c073570 0x7f4c5c119290 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c002a50 tx=0x7f4c4c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:56:43.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.976+0000 7f4c53e00640 1 -- 172.21.15.107:0/1540296991 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c4c007c30 con 0x7f4c5c073570 2024-03-04T13:56:43.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.976+0000 7f4c53e00640 1 -- 172.21.15.107:0/1540296991 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4c4c00f040 con 0x7f4c5c073570 2024-03-04T13:56:43.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.976+0000 7f4c53e00640 1 -- 172.21.15.107:0/1540296991 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c4c00dbe0 con 0x7f4c5c073570 2024-03-04T13:56:43.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.976+0000 7f4c62000640 1 -- 172.21.15.107:0/1540296991 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c5c113640 con 0x7f4c5c073570 2024-03-04T13:56:43.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.976+0000 7f4c62000640 1 -- 172.21.15.107:0/1540296991 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c5c113b10 con 0x7f4c5c073570 2024-03-04T13:56:43.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.978+0000 7f4c62000640 1 -- 172.21.15.107:0/1540296991 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c5c064030 con 0x7f4c5c073570 2024-03-04T13:56:43.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.978+0000 7f4c53e00640 1 -- 172.21.15.107:0/1540296991 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f4c4c00dd40 con 0x7f4c5c073570 2024-03-04T13:56:43.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.979+0000 7f4c53e00640 1 --2- 172.21.15.107:0/1540296991 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f4c4006d760 0x7f4c4006fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:43.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.979+0000 7f4c53e00640 1 -- 172.21.15.107:0/1540296991 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f4c4c08cf90 con 0x7f4c5c073570 2024-03-04T13:56:43.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.980+0000 7f4c52a00640 1 --2- 172.21.15.107:0/1540296991 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f4c4006d760 0x7f4c4006fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:56:43.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.980+0000 7f4c52a00640 1 --2- 172.21.15.107:0/1540296991 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f4c4006d760 0x7f4c4006fc10 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f4c44000c00 tx=0x7f4c44009da0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:56:43.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:43.984+0000 7f4c53e00640 1 -- 172.21.15.107:0/1540296991 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4c4c056710 con 0x7f4c5c073570 2024-03-04T13:56:44.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:44.165+0000 7f4c62000640 1 -- 172.21.15.107:0/1540296991 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f4c5c10a3e0 con 0x7f4c5c073570 2024-03-04T13:56:44.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:44.166+0000 7f4c53e00640 1 -- 172.21.15.107:0/1540296991 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v86) v1 ==== 107+0+11 (secure 0 0 0) 0x7f4c4c059dd0 con 0x7f4c5c073570 2024-03-04T13:56:44.168 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 32 2024-03-04T13:56:44.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:44.169+0000 7f4c62000640 1 -- 172.21.15.107:0/1540296991 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f4c4006d760 msgr2=0x7f4c4006fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:44.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:44.169+0000 7f4c62000640 1 --2- 172.21.15.107:0/1540296991 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f4c4006d760 0x7f4c4006fc10 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f4c44000c00 tx=0x7f4c44009da0 comp rx=0 tx=0).stop 2024-03-04T13:56:44.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:44.169+0000 7f4c62000640 1 -- 172.21.15.107:0/1540296991 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c5c073570 msgr2=0x7f4c5c119290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:44.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:44.169+0000 7f4c62000640 1 --2- 172.21.15.107:0/1540296991 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c5c073570 0x7f4c5c119290 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c002a50 tx=0x7f4c4c004600 comp rx=0 tx=0).stop 2024-03-04T13:56:44.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:44.169+0000 7f4c62000640 1 -- 172.21.15.107:0/1540296991 shutdown_connections 2024-03-04T13:56:44.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:44.169+0000 7f4c62000640 1 --2- 172.21.15.107:0/1540296991 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f4c4006d760 0x7f4c4006fc10 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:44.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:44.169+0000 7f4c62000640 1 --2- 172.21.15.107:0/1540296991 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c5c073e80 0x7f4c5c1197d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:44.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:44.169+0000 7f4c62000640 1 --2- 172.21.15.107:0/1540296991 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c5c073570 0x7f4c5c119290 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:44.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:44.169+0000 7f4c62000640 1 -- 172.21.15.107:0/1540296991 >> 172.21.15.107:0/1540296991 conn(0x7f4c5c0fa160 msgr2=0x7f4c5c0fbe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:56:44.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:44.169+0000 7f4c62000640 1 -- 172.21.15.107:0/1540296991 shutdown_connections 2024-03-04T13:56:44.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:44.169+0000 7f4c62000640 1 -- 172.21.15.107:0/1540296991 wait complete. 2024-03-04T13:56:44.188 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-04T13:56:44.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:44.383+0000 7fec06400640 1 -- 172.21.15.107:0/2479378699 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec00100110 msgr2=0x7fec00100560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:44.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:44.383+0000 7fec06400640 1 --2- 172.21.15.107:0/2479378699 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec00100110 0x7fec00100560 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7febe80048d0 tx=0x7febe8009da0 comp rx=0 tx=0).stop 2024-03-04T13:56:44.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:44.383+0000 7fec06400640 1 -- 172.21.15.107:0/2479378699 shutdown_connections 2024-03-04T13:56:44.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:44.383+0000 7fec06400640 1 --2- 172.21.15.107:0/2479378699 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec00100110 0x7fec00100560 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:44.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:44.383+0000 7fec06400640 1 --2- 172.21.15.107:0/2479378699 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec000fe9c0 0x7fec000fed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:44.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:44.383+0000 7fec06400640 1 -- 172.21.15.107:0/2479378699 >> 172.21.15.107:0/2479378699 conn(0x7fec000fa160 msgr2=0x7fec000fc570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:56:44.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:44.383+0000 7fec06400640 1 -- 172.21.15.107:0/2479378699 shutdown_connections 2024-03-04T13:56:44.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:44.383+0000 7fec06400640 1 -- 172.21.15.107:0/2479378699 wait complete. 2024-03-04T13:56:44.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:44.384+0000 7fec06400640 1 Processor -- start 2024-03-04T13:56:44.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:44.384+0000 7fec06400640 1 -- start start 2024-03-04T13:56:44.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:44.384+0000 7fec06400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec000fe9c0 0x7fec00114de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:44.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:44.384+0000 7fec06400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec00100110 0x7fec00115320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:44.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:44.384+0000 7fec06400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec00115920 con 0x7fec000fe9c0 2024-03-04T13:56:44.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:44.384+0000 7fec06400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec0010eeb0 con 0x7fec00100110 2024-03-04T13:56:44.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:44.385+0000 7febff400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec000fe9c0 0x7fec00114de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:56:44.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:44.385+0000 7febff400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec000fe9c0 0x7fec00114de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47342/0 (socket says 172.21.15.107:47342) 2024-03-04T13:56:44.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:44.385+0000 7febff400640 1 -- 172.21.15.107:0/933256328 learned_addr learned my addr 172.21.15.107:0/933256328 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:56:44.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:44.385+0000 7febff400640 1 -- 172.21.15.107:0/933256328 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec00100110 msgr2=0x7fec00115320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:44.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:44.385+0000 7febff400640 1 --2- 172.21.15.107:0/933256328 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec00100110 0x7fec00115320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:44.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:44.385+0000 7febff400640 1 -- 172.21.15.107:0/933256328 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febe8005040 con 0x7fec000fe9c0 2024-03-04T13:56:44.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:44.385+0000 7febff400640 1 --2- 172.21.15.107:0/933256328 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec000fe9c0 0x7fec00114de0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7febf000eb70 tx=0x7febf000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:56:44.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:44.385+0000 7febf7400640 1 -- 172.21.15.107:0/933256328 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7febf0007c30 con 0x7fec000fe9c0 2024-03-04T13:56:44.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:44.386+0000 7febf7400640 1 -- 172.21.15.107:0/933256328 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7febf000f040 con 0x7fec000fe9c0 2024-03-04T13:56:44.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:44.386+0000 7febf7400640 1 -- 172.21.15.107:0/933256328 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7febf00095b0 con 0x7fec000fe9c0 2024-03-04T13:56:44.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:44.386+0000 7fec06400640 1 -- 172.21.15.107:0/933256328 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec0010f130 con 0x7fec000fe9c0 2024-03-04T13:56:44.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:44.386+0000 7fec06400640 1 -- 172.21.15.107:0/933256328 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec0010f650 con 0x7fec000fe9c0 2024-03-04T13:56:44.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:44.387+0000 7fec06400640 1 -- 172.21.15.107:0/933256328 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec00064030 con 0x7fec000fe9c0 2024-03-04T13:56:44.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:44.388+0000 7febf7400640 1 -- 172.21.15.107:0/933256328 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7febf0009710 con 0x7fec000fe9c0 2024-03-04T13:56:44.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:44.388+0000 7febf7400640 1 --2- 172.21.15.107:0/933256328 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7febe406d750 0x7febe406fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:44.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:44.389+0000 7febf7400640 1 -- 172.21.15.107:0/933256328 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7febf008cbd0 con 0x7fec000fe9c0 2024-03-04T13:56:44.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:44.389+0000 7febf7e00640 1 --2- 172.21.15.107:0/933256328 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7febe406d750 0x7febe406fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:56:44.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:44.389+0000 7febf7e00640 1 --2- 172.21.15.107:0/933256328 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7febe406d750 0x7febe406fc00 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7febe8034040 tx=0x7febe8009da0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:56:44.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:44.393+0000 7febf7400640 1 -- 172.21.15.107:0/933256328 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7febf0056450 con 0x7fec000fe9c0 2024-03-04T13:56:44.595 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:56:44 smithi107 ceph-mon[112578]: from='client.? 172.21.15.107:0/4159804547' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-04T13:56:44.595 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:56:44 smithi107 ceph-mon[112578]: from='client.? 172.21.15.107:0/1540296991' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-04T13:56:44.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:44.603+0000 7fec06400640 1 -- 172.21.15.107:0/933256328 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fec0010f8c0 con 0x7fec000fe9c0 2024-03-04T13:56:44.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:44.605+0000 7febf7400640 1 -- 172.21.15.107:0/933256328 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 39 entries v86) v1 ==== 81+0+2349 (secure 0 0 0) 0x7febf0019940 con 0x7fec000fe9c0 2024-03-04T13:56:44.606 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.169:6827/3696090493 2024-03-05T13:53:58.927120+0000 2024-03-04T13:56:44.606 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.169:6825/2994866076 2024-03-05T13:53:29.273990+0000 2024-03-04T13:56:44.607 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/395365799 2024-03-05T13:41:21.904328+0000 2024-03-04T13:56:44.607 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/2217624526 2024-03-05T13:41:21.904328+0000 2024-03-04T13:56:44.607 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6801/2371648646 2024-03-05T13:30:56.353578+0000 2024-03-04T13:56:44.607 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6800/3848495603 2024-03-05T13:41:21.904328+0000 2024-03-04T13:56:44.607 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/3451280196 2024-03-05T13:30:16.230256+0000 2024-03-04T13:56:44.607 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6827/3814968386 2024-03-05T13:38:46.993392+0000 2024-03-04T13:56:44.607 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/3881311685 2024-03-05T13:47:24.592361+0000 2024-03-04T13:56:44.607 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/2723084639 2024-03-05T13:32:27.229455+0000 2024-03-04T13:56:44.607 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6801/1723390229 2024-03-05T13:30:16.230256+0000 2024-03-04T13:56:44.607 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/248094819 2024-03-05T13:47:24.592361+0000 2024-03-04T13:56:44.607 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6800/1723390229 2024-03-05T13:30:16.230256+0000 2024-03-04T13:56:44.607 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.169:6828/3524931243 2024-03-05T13:43:38.105893+0000 2024-03-04T13:56:44.607 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/601795289 2024-03-05T13:32:27.229455+0000 2024-03-04T13:56:44.607 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.169:0/1961312808 2024-03-05T13:43:38.105893+0000 2024-03-04T13:56:44.608 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6801/2040330534 2024-03-05T13:32:27.229455+0000 2024-03-04T13:56:44.608 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/136342223 2024-03-05T13:30:56.353578+0000 2024-03-04T13:56:44.608 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.169:0/692246057 2024-03-05T13:43:38.105893+0000 2024-03-04T13:56:44.608 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/1780539770 2024-03-05T13:30:16.230256+0000 2024-03-04T13:56:44.608 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/3355652811 2024-03-05T13:41:21.904328+0000 2024-03-04T13:56:44.608 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.169:6829/3524931243 2024-03-05T13:43:38.105893+0000 2024-03-04T13:56:44.608 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6800/2040330534 2024-03-05T13:32:27.229455+0000 2024-03-04T13:56:44.608 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6801/1613321046 2024-03-05T13:47:24.592361+0000 2024-03-04T13:56:44.608 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/3503768751 2024-03-05T13:32:27.229455+0000 2024-03-04T13:56:44.608 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.169:0/2785350328 2024-03-05T13:43:38.105893+0000 2024-03-04T13:56:44.608 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/2928681569 2024-03-05T13:30:56.353578+0000 2024-03-04T13:56:44.608 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/2314831039 2024-03-05T13:30:56.353578+0000 2024-03-04T13:56:44.608 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.169:0/842094828 2024-03-05T13:43:38.105893+0000 2024-03-04T13:56:44.609 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.169:6824/2994866076 2024-03-05T13:53:29.273990+0000 2024-03-04T13:56:44.609 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.169:6826/3696090493 2024-03-05T13:53:58.927120+0000 2024-03-04T13:56:44.609 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6801/3848495603 2024-03-05T13:41:21.904328+0000 2024-03-04T13:56:44.609 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6826/3814968386 2024-03-05T13:38:46.993392+0000 2024-03-04T13:56:44.609 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/2605062903 2024-03-05T13:41:21.904328+0000 2024-03-04T13:56:44.609 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6800/2371648646 2024-03-05T13:30:56.353578+0000 2024-03-04T13:56:44.609 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6800/1613321046 2024-03-05T13:47:24.592361+0000 2024-03-04T13:56:44.609 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/2357638826 2024-03-05T13:30:16.230256+0000 2024-03-04T13:56:44.609 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/1849554796 2024-03-05T13:47:24.592361+0000 2024-03-04T13:56:44.609 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/2439430884 2024-03-05T13:47:24.592361+0000 2024-03-04T13:56:44.610 INFO:teuthology.orchestra.run.smithi107.stderr:listed 39 entries 2024-03-04T13:56:44.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:44.607+0000 7fec06400640 1 -- 172.21.15.107:0/933256328 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7febe406d750 msgr2=0x7febe406fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:44.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:44.607+0000 7fec06400640 1 --2- 172.21.15.107:0/933256328 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7febe406d750 0x7febe406fc00 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7febe8034040 tx=0x7febe8009da0 comp rx=0 tx=0).stop 2024-03-04T13:56:44.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:44.607+0000 7fec06400640 1 -- 172.21.15.107:0/933256328 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec000fe9c0 msgr2=0x7fec00114de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:44.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:44.607+0000 7fec06400640 1 --2- 172.21.15.107:0/933256328 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec000fe9c0 0x7fec00114de0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7febf000eb70 tx=0x7febf000b040 comp rx=0 tx=0).stop 2024-03-04T13:56:44.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:44.607+0000 7fec06400640 1 -- 172.21.15.107:0/933256328 shutdown_connections 2024-03-04T13:56:44.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:44.607+0000 7fec06400640 1 --2- 172.21.15.107:0/933256328 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7febe406d750 0x7febe406fc00 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:44.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:44.607+0000 7fec06400640 1 --2- 172.21.15.107:0/933256328 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec00100110 0x7fec00115320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:44.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:44.607+0000 7fec06400640 1 --2- 172.21.15.107:0/933256328 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec000fe9c0 0x7fec00114de0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:44.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:44.607+0000 7fec06400640 1 -- 172.21.15.107:0/933256328 >> 172.21.15.107:0/933256328 conn(0x7fec000fa160 msgr2=0x7fec000fbc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:56:44.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:44.608+0000 7fec06400640 1 -- 172.21.15.107:0/933256328 shutdown_connections 2024-03-04T13:56:44.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:44.608+0000 7fec06400640 1 -- 172.21.15.107:0/933256328 wait complete. 2024-03-04T13:56:44.612 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:56:44 smithi169 ceph-mon[86183]: from='client.? 172.21.15.107:0/4159804547' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-04T13:56:44.612 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:56:44 smithi169 ceph-mon[86183]: from='client.? 172.21.15.107:0/1540296991' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-04T13:56:44.627 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-03-04T13:56:44.627 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-04T13:56:44.677 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-04T13:56:44.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:44.923+0000 7fd835c00640 1 -- 172.21.15.107:0/3726874116 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd83006f7e0 msgr2=0x7fd83006fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:44.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:44.923+0000 7fd835c00640 1 --2- 172.21.15.107:0/3726874116 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd83006f7e0 0x7fd83006fbb0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fd820009c50 tx=0x7fd820004d10 comp rx=0 tx=0).stop 2024-03-04T13:56:44.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:44.923+0000 7fd835c00640 1 -- 172.21.15.107:0/3726874116 shutdown_connections 2024-03-04T13:56:44.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:44.923+0000 7fd835c00640 1 --2- 172.21.15.107:0/3726874116 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd8300700f0 0x7fd830108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:44.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:44.923+0000 7fd835c00640 1 --2- 172.21.15.107:0/3726874116 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd83006f7e0 0x7fd83006fbb0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:44.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:44.923+0000 7fd835c00640 1 -- 172.21.15.107:0/3726874116 >> 172.21.15.107:0/3726874116 conn(0x7fd8300f8030 msgr2=0x7fd8300fa440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:56:44.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:44.924+0000 7fd835c00640 1 -- 172.21.15.107:0/3726874116 shutdown_connections 2024-03-04T13:56:44.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:44.924+0000 7fd835c00640 1 -- 172.21.15.107:0/3726874116 wait complete. 2024-03-04T13:56:44.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:44.924+0000 7fd835c00640 1 Processor -- start 2024-03-04T13:56:44.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:44.925+0000 7fd835c00640 1 -- start start 2024-03-04T13:56:44.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:44.925+0000 7fd835c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd83006f7e0 0x7fd8301151d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:44.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:44.925+0000 7fd835c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd8300700f0 0x7fd830115710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:44.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:44.925+0000 7fd835c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd830115da0 con 0x7fd8300700f0 2024-03-04T13:56:44.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:44.925+0000 7fd835c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd83010f2e0 con 0x7fd83006f7e0 2024-03-04T13:56:44.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:44.925+0000 7fd827e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd8300700f0 0x7fd830115710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:56:44.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:44.925+0000 7fd827e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd8300700f0 0x7fd830115710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47362/0 (socket says 172.21.15.107:47362) 2024-03-04T13:56:44.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:44.925+0000 7fd827e00640 1 -- 172.21.15.107:0/3627741976 learned_addr learned my addr 172.21.15.107:0/3627741976 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:56:44.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:44.926+0000 7fd827e00640 1 -- 172.21.15.107:0/3627741976 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd83006f7e0 msgr2=0x7fd8301151d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:44.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:44.926+0000 7fd827e00640 1 --2- 172.21.15.107:0/3627741976 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd83006f7e0 0x7fd8301151d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:44.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:44.926+0000 7fd827e00640 1 -- 172.21.15.107:0/3627741976 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8200098b0 con 0x7fd8300700f0 2024-03-04T13:56:44.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:44.926+0000 7fd827e00640 1 --2- 172.21.15.107:0/3627741976 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd8300700f0 0x7fd830115710 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fd81800b790 tx=0x7fd8180091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:56:44.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:44.926+0000 7fd826a00640 1 -- 172.21.15.107:0/3627741976 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd81800c700 con 0x7fd8300700f0 2024-03-04T13:56:44.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:44.926+0000 7fd826a00640 1 -- 172.21.15.107:0/3627741976 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd81800cd30 con 0x7fd8300700f0 2024-03-04T13:56:44.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:44.926+0000 7fd826a00640 1 -- 172.21.15.107:0/3627741976 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd818013370 con 0x7fd8300700f0 2024-03-04T13:56:44.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:44.926+0000 7fd835c00640 1 -- 172.21.15.107:0/3627741976 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd83010f5c0 con 0x7fd8300700f0 2024-03-04T13:56:44.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:44.927+0000 7fd835c00640 1 -- 172.21.15.107:0/3627741976 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd83010fae0 con 0x7fd8300700f0 2024-03-04T13:56:44.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:44.928+0000 7fd835c00640 1 -- 172.21.15.107:0/3627741976 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd830064480 con 0x7fd8300700f0 2024-03-04T13:56:44.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:44.929+0000 7fd826a00640 1 -- 172.21.15.107:0/3627741976 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7fd818012070 con 0x7fd8300700f0 2024-03-04T13:56:44.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:44.930+0000 7fd826a00640 1 --2- 172.21.15.107:0/3627741976 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fd81406d750 0x7fd81406fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:44.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:44.930+0000 7fd826a00640 1 -- 172.21.15.107:0/3627741976 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fd81808ca80 con 0x7fd8300700f0 2024-03-04T13:56:44.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:44.930+0000 7fd82ea00640 1 --2- 172.21.15.107:0/3627741976 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fd81406d750 0x7fd81406fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:56:44.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:44.931+0000 7fd82ea00640 1 --2- 172.21.15.107:0/3627741976 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fd81406d750 0x7fd81406fc00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd820009c20 tx=0x7fd820004d10 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:56:44.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:44.935+0000 7fd826a00640 1 -- 172.21.15.107:0/3627741976 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd8180562c0 con 0x7fd8300700f0 2024-03-04T13:56:45.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:45.130+0000 7fd835c00640 1 -- 172.21.15.107:0/3627741976 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd83010b870 con 0x7fd8300700f0 2024-03-04T13:56:45.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:45.133+0000 7fd826a00640 1 -- 172.21.15.107:0/3627741976 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+16693 (secure 0 0 0) 0x7fd818059980 con 0x7fd8300700f0 2024-03-04T13:56:45.135 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-04T13:56:45.135 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":86,"fsid":"1fc09a58-da2b-11ee-95c6-87774f69a715","created":"2024-03-04T13:29:42.985888+0000","modified":"2024-03-04T13:53:58.927176+0000","last_up_change":"2024-03-04T13:52:27.869095+0000","last_in_change":"2024-03-04T13:36:31.424613+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T13:35:17.915858+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-03-04T13:38:29.362118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-03-04T13:38:30.373574+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c857e387-3d2b-478d-9807-24ea8b9fe02d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":50,"up_from":54,"up_thru":79,"down_at":51,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":3996509230},{"type":"v1","addr":"172.21.15.107:6803","nonce":3996509230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":3996509230},{"type":"v1","addr":"172.21.15.107:6805","nonce":3996509230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":3996509230},{"type":"v1","addr":"172.21.15.107:6809","nonce":3996509230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":3996509230},{"type":"v1","addr":"172.21.15.107:6807","nonce":3996509230}]},"public_addr":"172.21.15.107:6803/3996509230","cluster_addr":"172.21.15.107:6805/3996509230","heartbeat_back_addr":"172.21.15.107:6809/3996509230","heartbeat_front_addr":"172.21.15.107:6807/3996509230","state":["exists","up"]},{"osd":1,"uuid":"fa29dc8f-419b-4318-b43e-e13fffd87204","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":55,"up_from":59,"up_thru":79,"down_at":56,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1164427580},{"type":"v1","addr":"172.21.15.107:6811","nonce":1164427580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1164427580},{"type":"v1","addr":"172.21.15.107:6813","nonce":1164427580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1164427580},{"type":"v1","addr":"172.21.15.107:6817","nonce":1164427580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1164427580},{"type":"v1","addr":"172.21.15.107:6815","nonce":1164427580}]},"public_addr":"172.21.15.107:6811/1164427580","cluster_addr":"172.21.15.107:6813/1164427580","heartbeat_back_addr":"172.21.15.107:6817/1164427580","heartbeat_front_addr":"172.21.15.107:6815/1164427580","state":["exists","up"]},{"osd":2,"uuid":"580a07f0-7bd4-4341-9c1f-b3383d64c720","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":60,"up_from":64,"up_thru":79,"down_at":61,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":150828071},{"type":"v1","addr":"172.21.15.107:6819","nonce":150828071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":150828071},{"type":"v1","addr":"172.21.15.107:6821","nonce":150828071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":150828071},{"type":"v1","addr":"172.21.15.107:6825","nonce":150828071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":150828071},{"type":"v1","addr":"172.21.15.107:6823","nonce":150828071}]},"public_addr":"172.21.15.107:6819/150828071","cluster_addr":"172.21.15.107:6821/150828071","heartbeat_back_addr":"172.21.15.107:6825/150828071","heartbeat_front_addr":"172.21.15.107:6823/150828071","state":["exists","up"]},{"osd":3,"uuid":"7acac14d-998a-4651-b78c-e42584e13b43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":27,"last_clean_end":65,"up_from":69,"up_thru":79,"down_at":66,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":3021051994},{"type":"v1","addr":"172.21.15.169:6801","nonce":3021051994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":3021051994},{"type":"v1","addr":"172.21.15.169:6803","nonce":3021051994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":3021051994},{"type":"v1","addr":"172.21.15.169:6807","nonce":3021051994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":3021051994},{"type":"v1","addr":"172.21.15.169:6805","nonce":3021051994}]},"public_addr":"172.21.15.169:6801/3021051994","cluster_addr":"172.21.15.169:6803/3021051994","heartbeat_back_addr":"172.21.15.169:6807/3021051994","heartbeat_front_addr":"172.21.15.169:6805/3021051994","state":["exists","up"]},{"osd":4,"uuid":"01d2b77c-66ec-4b50-8170-43d95eeae471","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":35,"last_clean_end":70,"up_from":74,"up_thru":79,"down_at":71,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":56453577},{"type":"v1","addr":"172.21.15.169:6809","nonce":56453577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":56453577},{"type":"v1","addr":"172.21.15.169:6811","nonce":56453577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":56453577},{"type":"v1","addr":"172.21.15.169:6815","nonce":56453577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":56453577},{"type":"v1","addr":"172.21.15.169:6813","nonce":56453577}]},"public_addr":"172.21.15.169:6809/56453577","cluster_addr":"172.21.15.169:6811/56453577","heartbeat_back_addr":"172.21.15.169:6815/56453577","heartbeat_front_addr":"172.21.15.169:6813/56453577","state":["exists","up"]},{"osd":5,"uuid":"0164ead7-c5d0-4541-bf22-f671558f8607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":40,"last_clean_end":75,"up_from":79,"up_thru":79,"down_at":76,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":374254288},{"type":"v1","addr":"172.21.15.169:6817","nonce":374254288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":374254288},{"type":"v1","addr":"172.21.15.169:6819","nonce":374254288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6822","nonce":374254288},{"type":"v1","addr":"172.21.15.169:6823","nonce":374254288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":374254288},{"type":"v1","addr":"172.21.15.169:6821","nonce":374254288}]},"public_addr":"172.21.15.169:6817/374254288","cluster_addr":"172.21.15.169:6819/374254288","heartbeat_back_addr":"172.21.15.169:6823/374254288","heartbeat_front_addr":"172.21.15.169:6821/374254288","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T13:33:59.729659+0000","dead_epoch":50},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T13:34:36.600326+0000","dead_epoch":55},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T13:35:14.068513+0000","dead_epoch":60},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T13:35:46.608007+0000","dead_epoch":65},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T13:36:19.595680+0000","dead_epoch":70},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T13:36:52.586933+0000","dead_epoch":75}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.169:6827/3696090493":"2024-03-05T13:53:58.927120+0000","172.21.15.169:6825/2994866076":"2024-03-05T13:53:29.273990+0000","172.21.15.107:0/395365799":"2024-03-05T13:41:21.904328+0000","172.21.15.107:0/2217624526":"2024-03-05T13:41:21.904328+0000","172.21.15.107:6801/2371648646":"2024-03-05T13:30:56.353578+0000","172.21.15.107:6800/3848495603":"2024-03-05T13:41:21.904328+0000","172.21.15.107:0/3451280196":"2024-03-05T13:30:16.230256+0000","172.21.15.107:6827/3814968386":"2024-03-05T13:38:46.993392+0000","172.21.15.107:0/3881311685":"2024-03-05T13:47:24.592361+0000","172.21.15.107:0/2723084639":"2024-03-05T13:32:27.229455+0000","172.21.15.107:6801/1723390229":"2024-03-05T13:30:16.230256+0000","172.21.15.107:0/248094819":"2024-03-05T13:47:24.592361+0000","172.21.15.107:6800/1723390229":"2024-03-05T13:30:16.230256+0000","172.21.15.169:6828/3524931243":"2024-03-05T13:43:38.105893+0000","172.21.15.107:0/601795289":"2024-03-05T13:32:27.229455+0000","172.21.15.169:0/1961312808":"2024-03-05T13:43:38.105893+0000","172.21.15.107:6801/2040330534":"2024-03-05T13:32:27.229455+0000","172.21.15.107:0/136342223":"2024-03-05T13:30:56.353578+0000","172.21.15.169:0/692246057":"2024-03-05T13:43:38.105893+0000","172.21.15.107:0/1780539770":"2024-03-05T13:30:16.230256+0000","172.21.15.107:0/3355652811":"2024-03-05T13:41:21.904328+0000","172.21.15.169:6829/3524931243":"2024-03-05T13:43:38.105893+0000","172.21.15.107:6800/2040330534":"2024-03-05T13:32:27.229455+0000","172.21.15.107:6801/1613321046":"2024-03-05T13:47:24.592361+0000","172.21.15.107:0/3503768751":"2024-03-05T13:32:27.229455+0000","172.21.15.169:0/2785350328":"2024-03-05T13:43:38.105893+0000","172.21.15.107:0/2928681569":"2024-03-05T13:30:56.353578+0000","172.21.15.107:0/2314831039":"2024-03-05T13:30:56.353578+0000","172.21.15.169:0/842094828":"2024-03-05T13:43:38.105893+0000","172.21.15.169:6824/2994866076":"2024-03-05T13:53:29.273990+0000","172.21.15.169:6826/3696090493":"2024-03-05T13:53:58.927120+0000","172.21.15.107:6801/3848495603":"2024-03-05T13:41:21.904328+0000","172.21.15.107:6826/3814968386":"2024-03-05T13:38:46.993392+0000","172.21.15.107:0/2605062903":"2024-03-05T13:41:21.904328+0000","172.21.15.107:6800/2371648646":"2024-03-05T13:30:56.353578+0000","172.21.15.107:6800/1613321046":"2024-03-05T13:47:24.592361+0000","172.21.15.107:0/2357638826":"2024-03-05T13:30:16.230256+0000","172.21.15.107:0/1849554796":"2024-03-05T13:47:24.592361+0000","172.21.15.107:0/2439430884":"2024-03-05T13:47:24.592361+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T13:56:45.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:45.136+0000 7fd835c00640 1 -- 172.21.15.107:0/3627741976 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fd81406d750 msgr2=0x7fd81406fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:45.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:45.136+0000 7fd835c00640 1 --2- 172.21.15.107:0/3627741976 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fd81406d750 0x7fd81406fc00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd820009c20 tx=0x7fd820004d10 comp rx=0 tx=0).stop 2024-03-04T13:56:45.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:45.136+0000 7fd835c00640 1 -- 172.21.15.107:0/3627741976 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd8300700f0 msgr2=0x7fd830115710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:45.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:45.136+0000 7fd835c00640 1 --2- 172.21.15.107:0/3627741976 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd8300700f0 0x7fd830115710 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fd81800b790 tx=0x7fd8180091d0 comp rx=0 tx=0).stop 2024-03-04T13:56:45.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:45.136+0000 7fd835c00640 1 -- 172.21.15.107:0/3627741976 shutdown_connections 2024-03-04T13:56:45.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:45.136+0000 7fd835c00640 1 --2- 172.21.15.107:0/3627741976 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7fd81406d750 0x7fd81406fc00 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:45.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:45.136+0000 7fd835c00640 1 --2- 172.21.15.107:0/3627741976 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd8300700f0 0x7fd830115710 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:45.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:45.136+0000 7fd835c00640 1 --2- 172.21.15.107:0/3627741976 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd83006f7e0 0x7fd8301151d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:45.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:45.136+0000 7fd835c00640 1 -- 172.21.15.107:0/3627741976 >> 172.21.15.107:0/3627741976 conn(0x7fd8300f8030 msgr2=0x7fd830103130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:56:45.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:45.136+0000 7fd835c00640 1 -- 172.21.15.107:0/3627741976 shutdown_connections 2024-03-04T13:56:45.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:45.136+0000 7fd835c00640 1 -- 172.21.15.107:0/3627741976 wait complete. 2024-03-04T13:56:45.154 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-04T13:35:17.915858+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-03-04T13:38:29.362118+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-03-04T13:38:30.373574+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-04T13:56:45.155 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-04T13:56:45.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:45.360+0000 7f2400c00640 1 -- 172.21.15.107:0/946469846 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23fc071470 msgr2=0x7f23fc06f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:45.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:45.360+0000 7f2400c00640 1 --2- 172.21.15.107:0/946469846 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23fc071470 0x7f23fc06f840 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f23ec0048d0 tx=0x7f23ec009da0 comp rx=0 tx=0).stop 2024-03-04T13:56:45.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:45.361+0000 7f2400c00640 1 -- 172.21.15.107:0/946469846 shutdown_connections 2024-03-04T13:56:45.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:45.361+0000 7f2400c00640 1 --2- 172.21.15.107:0/946469846 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f23fc06fd80 0x7f23fc0701f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:45.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:45.361+0000 7f2400c00640 1 --2- 172.21.15.107:0/946469846 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23fc071470 0x7f23fc06f840 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:45.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:45.361+0000 7f2400c00640 1 -- 172.21.15.107:0/946469846 >> 172.21.15.107:0/946469846 conn(0x7f23fc0f7cf0 msgr2=0x7f23fc0fa100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:56:45.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:45.361+0000 7f2400c00640 1 -- 172.21.15.107:0/946469846 shutdown_connections 2024-03-04T13:56:45.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:45.361+0000 7f2400c00640 1 -- 172.21.15.107:0/946469846 wait complete. 2024-03-04T13:56:45.364 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:45.362+0000 7f2400c00640 1 Processor -- start 2024-03-04T13:56:45.364 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:45.362+0000 7f2400c00640 1 -- start start 2024-03-04T13:56:45.364 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:45.362+0000 7f2400c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23fc06fd80 0x7f23fc119180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:45.364 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:45.362+0000 7f2400c00640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f23fc071470 0x7f23fc1196c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:45.364 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:45.362+0000 7f2400c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23fc119d50 con 0x7f23fc06fd80 2024-03-04T13:56:45.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:45.362+0000 7f2400c00640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23fc113250 con 0x7f23fc071470 2024-03-04T13:56:45.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:45.363+0000 7f23fb400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23fc06fd80 0x7f23fc119180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:56:45.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:45.363+0000 7f23fb400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23fc06fd80 0x7f23fc119180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47378/0 (socket says 172.21.15.107:47378) 2024-03-04T13:56:45.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:45.363+0000 7f23fb400640 1 -- 172.21.15.107:0/909465418 learned_addr learned my addr 172.21.15.107:0/909465418 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:56:45.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:45.363+0000 7f23fb400640 1 -- 172.21.15.107:0/909465418 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f23fc071470 msgr2=0x7f23fc1196c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:45.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:45.363+0000 7f23fb400640 1 --2- 172.21.15.107:0/909465418 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f23fc071470 0x7f23fc1196c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:45.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:45.363+0000 7f23fb400640 1 -- 172.21.15.107:0/909465418 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23ec005040 con 0x7f23fc06fd80 2024-03-04T13:56:45.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:45.363+0000 7f23fb400640 1 --2- 172.21.15.107:0/909465418 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23fc06fd80 0x7f23fc119180 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f23ec00b020 tx=0x7f23ec009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:56:45.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:45.363+0000 7f23f3400640 1 -- 172.21.15.107:0/909465418 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f23ec0092e0 con 0x7f23fc06fd80 2024-03-04T13:56:45.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:45.363+0000 7f23f3400640 1 -- 172.21.15.107:0/909465418 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f23ec009440 con 0x7f23fc06fd80 2024-03-04T13:56:45.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:45.364+0000 7f2400c00640 1 -- 172.21.15.107:0/909465418 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23fc1134d0 con 0x7f23fc06fd80 2024-03-04T13:56:45.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:45.364+0000 7f23f3400640 1 -- 172.21.15.107:0/909465418 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f23ec03a5f0 con 0x7f23fc06fd80 2024-03-04T13:56:45.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:45.364+0000 7f2400c00640 1 -- 172.21.15.107:0/909465418 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23fc1139f0 con 0x7f23fc06fd80 2024-03-04T13:56:45.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:45.365+0000 7f2400c00640 1 -- 172.21.15.107:0/909465418 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23fc04bcc0 con 0x7f23fc06fd80 2024-03-04T13:56:45.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:45.366+0000 7f23f3400640 1 -- 172.21.15.107:0/909465418 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f23ec03b420 con 0x7f23fc06fd80 2024-03-04T13:56:45.368 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:45.367+0000 7f23f3400640 1 --2- 172.21.15.107:0/909465418 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f23e006d750 0x7f23e006fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:45.368 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:45.367+0000 7f23f3400640 1 -- 172.21.15.107:0/909465418 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f23ec0afbf0 con 0x7f23fc06fd80 2024-03-04T13:56:45.368 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:45.367+0000 7f23f3e00640 1 --2- 172.21.15.107:0/909465418 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f23e006d750 0x7f23e006fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:56:45.369 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:45.368+0000 7f23f3e00640 1 --2- 172.21.15.107:0/909465418 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f23e006d750 0x7f23e006fc00 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f23fc114a40 tx=0x7f23e4006d20 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:56:45.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:45.371+0000 7f23f3400640 1 -- 172.21.15.107:0/909465418 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f23ec079530 con 0x7f23fc06fd80 2024-03-04T13:56:45.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:45.568+0000 7f2400c00640 1 -- 172.21.15.107:0/909465418 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f23fc114780 con 0x7f23fc06fd80 2024-03-04T13:56:45.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:45.569+0000 7f23f3400640 1 -- 172.21.15.107:0/909465418 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) v1 ==== 93+0+10 (secure 0 0 0) 0x7f23ec07cbf0 con 0x7f23fc06fd80 2024-03-04T13:56:45.571 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 1 2024-03-04T13:56:45.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:45.572+0000 7f2400c00640 1 -- 172.21.15.107:0/909465418 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f23e006d750 msgr2=0x7f23e006fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:45.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:45.572+0000 7f2400c00640 1 --2- 172.21.15.107:0/909465418 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f23e006d750 0x7f23e006fc00 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f23fc114a40 tx=0x7f23e4006d20 comp rx=0 tx=0).stop 2024-03-04T13:56:45.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:45.572+0000 7f2400c00640 1 -- 172.21.15.107:0/909465418 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23fc06fd80 msgr2=0x7f23fc119180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:45.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:45.572+0000 7f2400c00640 1 --2- 172.21.15.107:0/909465418 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23fc06fd80 0x7f23fc119180 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f23ec00b020 tx=0x7f23ec009da0 comp rx=0 tx=0).stop 2024-03-04T13:56:45.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:45.572+0000 7f2400c00640 1 -- 172.21.15.107:0/909465418 shutdown_connections 2024-03-04T13:56:45.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:45.572+0000 7f2400c00640 1 --2- 172.21.15.107:0/909465418 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f23e006d750 0x7f23e006fc00 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:45.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:45.572+0000 7f2400c00640 1 --2- 172.21.15.107:0/909465418 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f23fc071470 0x7f23fc1196c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:45.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:45.572+0000 7f2400c00640 1 --2- 172.21.15.107:0/909465418 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23fc06fd80 0x7f23fc119180 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:45.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:45.572+0000 7f2400c00640 1 -- 172.21.15.107:0/909465418 >> 172.21.15.107:0/909465418 conn(0x7f23fc0f7cf0 msgr2=0x7f23fc10b4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:56:45.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:45.572+0000 7f2400c00640 1 -- 172.21.15.107:0/909465418 shutdown_connections 2024-03-04T13:56:45.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:45.572+0000 7f2400c00640 1 -- 172.21.15.107:0/909465418 wait complete. 2024-03-04T13:56:45.588 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:56:45 smithi107 ceph-mon[112578]: from='client.? 172.21.15.107:0/933256328' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-04T13:56:45.588 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:56:45 smithi107 ceph-mon[112578]: pgmap v319: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:56:45.588 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:56:45 smithi107 ceph-mon[112578]: from='client.? 172.21.15.107:0/3627741976' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T13:56:45.592 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-03-04T13:56:45.612 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:56:45 smithi169 ceph-mon[86183]: from='client.? 172.21.15.107:0/933256328' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-04T13:56:45.612 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:56:45 smithi169 ceph-mon[86183]: pgmap v319: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:56:45.612 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:56:45 smithi169 ceph-mon[86183]: from='client.? 172.21.15.107:0/3627741976' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T13:56:45.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:45.797+0000 7f7f67400640 1 -- 172.21.15.107:0/501173188 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f60073e80 msgr2=0x7f7f6010cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:45.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:45.797+0000 7f7f67400640 1 --2- 172.21.15.107:0/501173188 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f60073e80 0x7f7f6010cd40 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c0048d0 tx=0x7f7f4c009da0 comp rx=0 tx=0).stop 2024-03-04T13:56:45.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:45.797+0000 7f7f67400640 1 -- 172.21.15.107:0/501173188 shutdown_connections 2024-03-04T13:56:45.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:45.797+0000 7f7f67400640 1 --2- 172.21.15.107:0/501173188 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f60073e80 0x7f7f6010cd40 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:45.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:45.797+0000 7f7f67400640 1 --2- 172.21.15.107:0/501173188 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f60073570 0x7f7f60073940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:45.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:45.797+0000 7f7f67400640 1 -- 172.21.15.107:0/501173188 >> 172.21.15.107:0/501173188 conn(0x7f7f600fa160 msgr2=0x7f7f600fc570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:56:45.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:45.797+0000 7f7f67400640 1 -- 172.21.15.107:0/501173188 shutdown_connections 2024-03-04T13:56:45.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:45.797+0000 7f7f67400640 1 -- 172.21.15.107:0/501173188 wait complete. 2024-03-04T13:56:45.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:45.798+0000 7f7f67400640 1 Processor -- start 2024-03-04T13:56:45.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:45.798+0000 7f7f67400640 1 -- start start 2024-03-04T13:56:45.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:45.799+0000 7f7f67400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f60073570 0x7f7f601192b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:45.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:45.799+0000 7f7f67400640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f60073e80 0x7f7f601197f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:45.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:45.799+0000 7f7f67400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f60119df0 con 0x7f7f60073570 2024-03-04T13:56:45.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:45.799+0000 7f7f67400640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f60113380 con 0x7f7f60073e80 2024-03-04T13:56:45.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:45.799+0000 7f7f64a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f60073570 0x7f7f601192b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:56:45.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:45.799+0000 7f7f64a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f60073570 0x7f7f601192b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47392/0 (socket says 172.21.15.107:47392) 2024-03-04T13:56:45.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:45.799+0000 7f7f64a00640 1 -- 172.21.15.107:0/3089470132 learned_addr learned my addr 172.21.15.107:0/3089470132 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:56:45.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:45.799+0000 7f7f64a00640 1 -- 172.21.15.107:0/3089470132 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f60073e80 msgr2=0x7f7f601197f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:45.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:45.799+0000 7f7f64a00640 1 --2- 172.21.15.107:0/3089470132 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f60073e80 0x7f7f601197f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:45.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:45.799+0000 7f7f64a00640 1 -- 172.21.15.107:0/3089470132 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f4c005040 con 0x7f7f60073570 2024-03-04T13:56:45.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:45.800+0000 7f7f64a00640 1 --2- 172.21.15.107:0/3089470132 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f60073570 0x7f7f601192b0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f7f54002a50 tx=0x7f7f54004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:56:45.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:45.800+0000 7f7f5cc00640 1 -- 172.21.15.107:0/3089470132 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f54007c30 con 0x7f7f60073570 2024-03-04T13:56:45.804 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:45.800+0000 7f7f5cc00640 1 -- 172.21.15.107:0/3089470132 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7f5400f040 con 0x7f7f60073570 2024-03-04T13:56:45.804 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:45.800+0000 7f7f5cc00640 1 -- 172.21.15.107:0/3089470132 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f5400db50 con 0x7f7f60073570 2024-03-04T13:56:45.804 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:45.800+0000 7f7f67400640 1 -- 172.21.15.107:0/3089470132 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f60113660 con 0x7f7f60073570 2024-03-04T13:56:45.804 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:45.800+0000 7f7f67400640 1 -- 172.21.15.107:0/3089470132 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f60113b80 con 0x7f7f60073570 2024-03-04T13:56:45.804 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:45.801+0000 7f7f67400640 1 -- 172.21.15.107:0/3089470132 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f60064030 con 0x7f7f60073570 2024-03-04T13:56:45.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:45.802+0000 7f7f5cc00640 1 -- 172.21.15.107:0/3089470132 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f7f5400dcf0 con 0x7f7f60073570 2024-03-04T13:56:45.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:45.803+0000 7f7f5cc00640 1 --2- 172.21.15.107:0/3089470132 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f7f4806d760 0x7f7f4806fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:45.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:45.803+0000 7f7f5cc00640 1 -- 172.21.15.107:0/3089470132 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f7f5408c7f0 con 0x7f7f60073570 2024-03-04T13:56:45.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:45.803+0000 7f7f5fe00640 1 --2- 172.21.15.107:0/3089470132 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f7f4806d760 0x7f7f4806fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:56:45.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:45.804+0000 7f7f5fe00640 1 --2- 172.21.15.107:0/3089470132 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f7f4806d760 0x7f7f4806fc10 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c000c00 tx=0x7f7f4c009da0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:56:45.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:45.808+0000 7f7f5cc00640 1 -- 172.21.15.107:0/3089470132 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7f54055ff0 con 0x7f7f60073570 2024-03-04T13:56:46.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.003+0000 7f7f67400640 1 -- 172.21.15.107:0/3089470132 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f7f601148b0 con 0x7f7f60073570 2024-03-04T13:56:46.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.003+0000 7f7f5cc00640 1 -- 172.21.15.107:0/3089470132 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v86) v1 ==== 107+0+11 (secure 0 0 0) 0x7f7f540596b0 con 0x7f7f60073570 2024-03-04T13:56:46.005 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 32 2024-03-04T13:56:46.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.006+0000 7f7f67400640 1 -- 172.21.15.107:0/3089470132 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f7f4806d760 msgr2=0x7f7f4806fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:46.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.006+0000 7f7f67400640 1 --2- 172.21.15.107:0/3089470132 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f7f4806d760 0x7f7f4806fc10 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c000c00 tx=0x7f7f4c009da0 comp rx=0 tx=0).stop 2024-03-04T13:56:46.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.006+0000 7f7f67400640 1 -- 172.21.15.107:0/3089470132 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f60073570 msgr2=0x7f7f601192b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:46.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.006+0000 7f7f67400640 1 --2- 172.21.15.107:0/3089470132 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f60073570 0x7f7f601192b0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f7f54002a50 tx=0x7f7f54004600 comp rx=0 tx=0).stop 2024-03-04T13:56:46.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.006+0000 7f7f67400640 1 -- 172.21.15.107:0/3089470132 shutdown_connections 2024-03-04T13:56:46.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.006+0000 7f7f67400640 1 --2- 172.21.15.107:0/3089470132 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f7f4806d760 0x7f7f4806fc10 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:46.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.006+0000 7f7f67400640 1 --2- 172.21.15.107:0/3089470132 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7f60073e80 0x7f7f601197f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:46.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.006+0000 7f7f67400640 1 --2- 172.21.15.107:0/3089470132 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f60073570 0x7f7f601192b0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:46.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.006+0000 7f7f67400640 1 -- 172.21.15.107:0/3089470132 >> 172.21.15.107:0/3089470132 conn(0x7f7f600fa160 msgr2=0x7f7f600fbe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:56:46.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.006+0000 7f7f67400640 1 -- 172.21.15.107:0/3089470132 shutdown_connections 2024-03-04T13:56:46.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.006+0000 7f7f67400640 1 -- 172.21.15.107:0/3089470132 wait complete. 2024-03-04T13:56:46.025 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-03-04T13:56:46.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.226+0000 7f55ce200640 1 -- 172.21.15.107:0/2615292815 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55c80fdd30 msgr2=0x7f55c8106370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:46.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.226+0000 7f55ce200640 1 --2- 172.21.15.107:0/2615292815 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55c80fdd30 0x7f55c8106370 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f55b40048d0 tx=0x7f55b4009da0 comp rx=0 tx=0).stop 2024-03-04T13:56:46.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.226+0000 7f55ce200640 1 -- 172.21.15.107:0/2615292815 shutdown_connections 2024-03-04T13:56:46.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.226+0000 7f55ce200640 1 --2- 172.21.15.107:0/2615292815 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55c80fdd30 0x7f55c8106370 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:46.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.226+0000 7f55ce200640 1 --2- 172.21.15.107:0/2615292815 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55c80fd390 0x7f55c80fd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:46.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.226+0000 7f55ce200640 1 -- 172.21.15.107:0/2615292815 >> 172.21.15.107:0/2615292815 conn(0x7f55c80f6de0 msgr2=0x7f55c80f91f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:56:46.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.226+0000 7f55ce200640 1 -- 172.21.15.107:0/2615292815 shutdown_connections 2024-03-04T13:56:46.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.226+0000 7f55ce200640 1 -- 172.21.15.107:0/2615292815 wait complete. 2024-03-04T13:56:46.230 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.227+0000 7f55ce200640 1 Processor -- start 2024-03-04T13:56:46.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.227+0000 7f55ce200640 1 -- start start 2024-03-04T13:56:46.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.227+0000 7f55ce200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55c80fd390 0x7f55c8114e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:46.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.227+0000 7f55ce200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55c80fdd30 0x7f55c81153d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:46.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.227+0000 7f55ce200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55c81159d0 con 0x7f55c80fdd30 2024-03-04T13:56:46.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.227+0000 7f55ce200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55c810ef60 con 0x7f55c80fd390 2024-03-04T13:56:46.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.227+0000 7f55c6a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55c80fdd30 0x7f55c81153d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:56:46.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.227+0000 7f55c6a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55c80fdd30 0x7f55c81153d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47400/0 (socket says 172.21.15.107:47400) 2024-03-04T13:56:46.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.227+0000 7f55c6a00640 1 -- 172.21.15.107:0/3390443960 learned_addr learned my addr 172.21.15.107:0/3390443960 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:56:46.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.228+0000 7f55c7400640 1 --2- 172.21.15.107:0/3390443960 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55c80fd390 0x7f55c8114e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:56:46.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.228+0000 7f55c6a00640 1 -- 172.21.15.107:0/3390443960 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55c80fd390 msgr2=0x7f55c8114e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:46.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.228+0000 7f55c6a00640 1 --2- 172.21.15.107:0/3390443960 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55c80fd390 0x7f55c8114e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:46.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.228+0000 7f55c6a00640 1 -- 172.21.15.107:0/3390443960 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55b4005040 con 0x7f55c80fdd30 2024-03-04T13:56:46.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.228+0000 7f55c7400640 1 --2- 172.21.15.107:0/3390443960 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55c80fd390 0x7f55c8114e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T13:56:46.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.228+0000 7f55c6a00640 1 --2- 172.21.15.107:0/3390443960 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55c80fdd30 0x7f55c81153d0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f55b4031720 tx=0x7f55b4009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:56:46.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.228+0000 7f55bf400640 1 -- 172.21.15.107:0/3390443960 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f55b40094c0 con 0x7f55c80fdd30 2024-03-04T13:56:46.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.228+0000 7f55bf400640 1 -- 172.21.15.107:0/3390443960 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f55b4009620 con 0x7f55c80fdd30 2024-03-04T13:56:46.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.228+0000 7f55bf400640 1 -- 172.21.15.107:0/3390443960 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f55b403b640 con 0x7f55c80fdd30 2024-03-04T13:56:46.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.229+0000 7f55ce200640 1 -- 172.21.15.107:0/3390443960 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55c810f1e0 con 0x7f55c80fdd30 2024-03-04T13:56:46.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.229+0000 7f55ce200640 1 -- 172.21.15.107:0/3390443960 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55c810f700 con 0x7f55c80fdd30 2024-03-04T13:56:46.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.230+0000 7f55ce200640 1 -- 172.21.15.107:0/3390443960 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55c8064480 con 0x7f55c80fdd30 2024-03-04T13:56:46.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.231+0000 7f55bf400640 1 -- 172.21.15.107:0/3390443960 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f55b402f070 con 0x7f55c80fdd30 2024-03-04T13:56:46.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.231+0000 7f55bf400640 1 --2- 172.21.15.107:0/3390443960 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f55ac06d750 0x7f55ac06fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:46.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.232+0000 7f55bf400640 1 -- 172.21.15.107:0/3390443960 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f55b40af480 con 0x7f55c80fdd30 2024-03-04T13:56:46.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.232+0000 7f55c7400640 1 --2- 172.21.15.107:0/3390443960 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f55ac06d750 0x7f55ac06fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:56:46.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.232+0000 7f55c7400640 1 --2- 172.21.15.107:0/3390443960 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f55ac06d750 0x7f55ac06fc00 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f55c0007fc0 tx=0x7f55c0004d00 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:56:46.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.236+0000 7f55bf400640 1 -- 172.21.15.107:0/3390443960 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f55b4078c00 con 0x7f55c80fdd30 2024-03-04T13:56:46.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.432+0000 7f55ce200640 1 -- 172.21.15.107:0/3390443960 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f55c8110490 con 0x7f55c80fdd30 2024-03-04T13:56:46.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.434+0000 7f55bf400640 1 -- 172.21.15.107:0/3390443960 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v86) v1 ==== 107+0+11 (secure 0 0 0) 0x7f55b407c2c0 con 0x7f55c80fdd30 2024-03-04T13:56:46.435 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 32 2024-03-04T13:56:46.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.436+0000 7f55ce200640 1 -- 172.21.15.107:0/3390443960 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f55ac06d750 msgr2=0x7f55ac06fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:46.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.436+0000 7f55ce200640 1 --2- 172.21.15.107:0/3390443960 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f55ac06d750 0x7f55ac06fc00 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f55c0007fc0 tx=0x7f55c0004d00 comp rx=0 tx=0).stop 2024-03-04T13:56:46.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.436+0000 7f55ce200640 1 -- 172.21.15.107:0/3390443960 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55c80fdd30 msgr2=0x7f55c81153d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:46.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.436+0000 7f55ce200640 1 --2- 172.21.15.107:0/3390443960 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55c80fdd30 0x7f55c81153d0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f55b4031720 tx=0x7f55b4009da0 comp rx=0 tx=0).stop 2024-03-04T13:56:46.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.436+0000 7f55ce200640 1 -- 172.21.15.107:0/3390443960 shutdown_connections 2024-03-04T13:56:46.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.436+0000 7f55ce200640 1 --2- 172.21.15.107:0/3390443960 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f55ac06d750 0x7f55ac06fc00 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:46.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.436+0000 7f55ce200640 1 --2- 172.21.15.107:0/3390443960 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55c80fdd30 0x7f55c81153d0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:46.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.436+0000 7f55ce200640 1 --2- 172.21.15.107:0/3390443960 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55c80fd390 0x7f55c8114e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:46.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.436+0000 7f55ce200640 1 -- 172.21.15.107:0/3390443960 >> 172.21.15.107:0/3390443960 conn(0x7f55c80f6de0 msgr2=0x7f55c80fb960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:56:46.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.436+0000 7f55ce200640 1 -- 172.21.15.107:0/3390443960 shutdown_connections 2024-03-04T13:56:46.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.436+0000 7f55ce200640 1 -- 172.21.15.107:0/3390443960 wait complete. 2024-03-04T13:56:46.457 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-04T13:56:46.612 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:56:46 smithi169 ceph-mon[86183]: from='client.? 172.21.15.107:0/909465418' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-04T13:56:46.612 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:56:46 smithi169 ceph-mon[86183]: from='client.? 172.21.15.107:0/3089470132' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-04T13:56:46.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.661+0000 7f2847200640 1 -- 172.21.15.107:0/559091166 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2840102360 msgr2=0x7f2840102730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:46.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.661+0000 7f2847200640 1 --2- 172.21.15.107:0/559091166 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2840102360 0x7f2840102730 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f2830009c50 tx=0x7f2830004d10 comp rx=0 tx=0).stop 2024-03-04T13:56:46.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.662+0000 7f2847200640 1 -- 172.21.15.107:0/559091166 shutdown_connections 2024-03-04T13:56:46.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.662+0000 7f2847200640 1 --2- 172.21.15.107:0/559091166 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2840102d00 0x7f28400fc390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:46.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.662+0000 7f2847200640 1 --2- 172.21.15.107:0/559091166 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2840102360 0x7f2840102730 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:46.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.662+0000 7f2847200640 1 -- 172.21.15.107:0/559091166 >> 172.21.15.107:0/559091166 conn(0x7f28400f7ef0 msgr2=0x7f28400fa300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:56:46.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.662+0000 7f2847200640 1 -- 172.21.15.107:0/559091166 shutdown_connections 2024-03-04T13:56:46.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.662+0000 7f2847200640 1 -- 172.21.15.107:0/559091166 wait complete. 2024-03-04T13:56:46.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.662+0000 7f2847200640 1 Processor -- start 2024-03-04T13:56:46.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.663+0000 7f2847200640 1 -- start start 2024-03-04T13:56:46.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.663+0000 7f2847200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2840102360 0x7f2840114f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:46.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.663+0000 7f2847200640 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2840102d00 0x7f2840115450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:46.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.663+0000 7f2847200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2840115a50 con 0x7f2840102360 2024-03-04T13:56:46.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.663+0000 7f2847200640 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2840115bc0 con 0x7f2840102d00 2024-03-04T13:56:46.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.663+0000 7f283fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2840102360 0x7f2840114f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:56:46.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.663+0000 7f283fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2840102360 0x7f2840114f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47414/0 (socket says 172.21.15.107:47414) 2024-03-04T13:56:46.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.663+0000 7f283fe00640 1 -- 172.21.15.107:0/1763897784 learned_addr learned my addr 172.21.15.107:0/1763897784 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-04T13:56:46.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.664+0000 7f2837400640 1 --2- 172.21.15.107:0/1763897784 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2840102d00 0x7f2840115450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:56:46.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.664+0000 7f283fe00640 1 -- 172.21.15.107:0/1763897784 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2840102d00 msgr2=0x7f2840115450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:46.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.664+0000 7f283fe00640 1 --2- 172.21.15.107:0/1763897784 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2840102d00 0x7f2840115450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:46.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.664+0000 7f283fe00640 1 -- 172.21.15.107:0/1763897784 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28300098b0 con 0x7f2840102360 2024-03-04T13:56:46.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.664+0000 7f283fe00640 1 --2- 172.21.15.107:0/1763897784 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2840102360 0x7f2840114f10 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f2830000f80 tx=0x7f2830004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:56:46.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.664+0000 7f283cc00640 1 -- 172.21.15.107:0/1763897784 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f283003a070 con 0x7f2840102360 2024-03-04T13:56:46.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.664+0000 7f283cc00640 1 -- 172.21.15.107:0/1763897784 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2830009710 con 0x7f2840102360 2024-03-04T13:56:46.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.664+0000 7f283cc00640 1 -- 172.21.15.107:0/1763897784 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f283003e9f0 con 0x7f2840102360 2024-03-04T13:56:46.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.664+0000 7f2847200640 1 -- 172.21.15.107:0/1763897784 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f284010f130 con 0x7f2840102360 2024-03-04T13:56:46.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.665+0000 7f2847200640 1 -- 172.21.15.107:0/1763897784 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f284010f650 con 0x7f2840102360 2024-03-04T13:56:46.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.666+0000 7f2847200640 1 -- 172.21.15.107:0/1763897784 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2840064480 con 0x7f2840102360 2024-03-04T13:56:46.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.666+0000 7f283cc00640 1 -- 172.21.15.107:0/1763897784 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 40) v1 ==== 91485+0+0 (secure 0 0 0) 0x7f2830032070 con 0x7f2840102360 2024-03-04T13:56:46.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.667+0000 7f283cc00640 1 --2- 172.21.15.107:0/1763897784 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f282406d760 0x7f282406fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T13:56:46.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.668+0000 7f283cc00640 1 -- 172.21.15.107:0/1763897784 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f28300af140 con 0x7f2840102360 2024-03-04T13:56:46.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.668+0000 7f2837400640 1 --2- 172.21.15.107:0/1763897784 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f282406d760 0x7f282406fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T13:56:46.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.668+0000 7f2837400640 1 --2- 172.21.15.107:0/1763897784 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f282406d760 0x7f282406fc10 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2828007fd0 tx=0x7f2828009e90 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T13:56:46.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.672+0000 7f283cc00640 1 -- 172.21.15.107:0/1763897784 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2830078940 con 0x7f2840102360 2024-03-04T13:56:46.845 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:56:46 smithi107 ceph-mon[112578]: from='client.? 172.21.15.107:0/909465418' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-04T13:56:46.845 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:56:46 smithi107 ceph-mon[112578]: from='client.? 172.21.15.107:0/3089470132' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-04T13:56:46.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.867+0000 7f2847200640 1 -- 172.21.15.107:0/1763897784 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f28401094c0 con 0x7f2840102360 2024-03-04T13:56:46.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.868+0000 7f283cc00640 1 -- 172.21.15.107:0/1763897784 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 39 entries v86) v1 ==== 81+0+2349 (secure 0 0 0) 0x7f283007c000 con 0x7f2840102360 2024-03-04T13:56:46.869 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.169:6827/3696090493 2024-03-05T13:53:58.927120+0000 2024-03-04T13:56:46.870 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.169:6825/2994866076 2024-03-05T13:53:29.273990+0000 2024-03-04T13:56:46.870 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/395365799 2024-03-05T13:41:21.904328+0000 2024-03-04T13:56:46.870 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/2217624526 2024-03-05T13:41:21.904328+0000 2024-03-04T13:56:46.870 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6801/2371648646 2024-03-05T13:30:56.353578+0000 2024-03-04T13:56:46.870 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6800/3848495603 2024-03-05T13:41:21.904328+0000 2024-03-04T13:56:46.870 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/3451280196 2024-03-05T13:30:16.230256+0000 2024-03-04T13:56:46.870 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6827/3814968386 2024-03-05T13:38:46.993392+0000 2024-03-04T13:56:46.870 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/3881311685 2024-03-05T13:47:24.592361+0000 2024-03-04T13:56:46.870 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/2723084639 2024-03-05T13:32:27.229455+0000 2024-03-04T13:56:46.870 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6801/1723390229 2024-03-05T13:30:16.230256+0000 2024-03-04T13:56:46.870 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/248094819 2024-03-05T13:47:24.592361+0000 2024-03-04T13:56:46.870 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6800/1723390229 2024-03-05T13:30:16.230256+0000 2024-03-04T13:56:46.870 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.169:6828/3524931243 2024-03-05T13:43:38.105893+0000 2024-03-04T13:56:46.871 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/601795289 2024-03-05T13:32:27.229455+0000 2024-03-04T13:56:46.871 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.169:0/1961312808 2024-03-05T13:43:38.105893+0000 2024-03-04T13:56:46.871 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6801/2040330534 2024-03-05T13:32:27.229455+0000 2024-03-04T13:56:46.871 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/136342223 2024-03-05T13:30:56.353578+0000 2024-03-04T13:56:46.871 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.169:0/692246057 2024-03-05T13:43:38.105893+0000 2024-03-04T13:56:46.871 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/1780539770 2024-03-05T13:30:16.230256+0000 2024-03-04T13:56:46.871 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/3355652811 2024-03-05T13:41:21.904328+0000 2024-03-04T13:56:46.871 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.169:6829/3524931243 2024-03-05T13:43:38.105893+0000 2024-03-04T13:56:46.871 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6800/2040330534 2024-03-05T13:32:27.229455+0000 2024-03-04T13:56:46.871 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6801/1613321046 2024-03-05T13:47:24.592361+0000 2024-03-04T13:56:46.871 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/3503768751 2024-03-05T13:32:27.229455+0000 2024-03-04T13:56:46.871 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.169:0/2785350328 2024-03-05T13:43:38.105893+0000 2024-03-04T13:56:46.871 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/2928681569 2024-03-05T13:30:56.353578+0000 2024-03-04T13:56:46.872 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/2314831039 2024-03-05T13:30:56.353578+0000 2024-03-04T13:56:46.872 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.169:0/842094828 2024-03-05T13:43:38.105893+0000 2024-03-04T13:56:46.872 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.169:6824/2994866076 2024-03-05T13:53:29.273990+0000 2024-03-04T13:56:46.872 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.169:6826/3696090493 2024-03-05T13:53:58.927120+0000 2024-03-04T13:56:46.872 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6801/3848495603 2024-03-05T13:41:21.904328+0000 2024-03-04T13:56:46.872 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6826/3814968386 2024-03-05T13:38:46.993392+0000 2024-03-04T13:56:46.872 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/2605062903 2024-03-05T13:41:21.904328+0000 2024-03-04T13:56:46.872 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6800/2371648646 2024-03-05T13:30:56.353578+0000 2024-03-04T13:56:46.872 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6800/1613321046 2024-03-05T13:47:24.592361+0000 2024-03-04T13:56:46.872 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/2357638826 2024-03-05T13:30:16.230256+0000 2024-03-04T13:56:46.872 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/1849554796 2024-03-05T13:47:24.592361+0000 2024-03-04T13:56:46.872 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/2439430884 2024-03-05T13:47:24.592361+0000 2024-03-04T13:56:46.873 INFO:teuthology.orchestra.run.smithi107.stderr:listed 39 entries 2024-03-04T13:56:46.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.870+0000 7f2847200640 1 -- 172.21.15.107:0/1763897784 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f282406d760 msgr2=0x7f282406fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:46.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.870+0000 7f2847200640 1 --2- 172.21.15.107:0/1763897784 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f282406d760 0x7f282406fc10 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2828007fd0 tx=0x7f2828009e90 comp rx=0 tx=0).stop 2024-03-04T13:56:46.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.870+0000 7f2847200640 1 -- 172.21.15.107:0/1763897784 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2840102360 msgr2=0x7f2840114f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T13:56:46.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.870+0000 7f2847200640 1 --2- 172.21.15.107:0/1763897784 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2840102360 0x7f2840114f10 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f2830000f80 tx=0x7f2830004d10 comp rx=0 tx=0).stop 2024-03-04T13:56:46.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.870+0000 7f2847200640 1 -- 172.21.15.107:0/1763897784 shutdown_connections 2024-03-04T13:56:46.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.870+0000 7f2847200640 1 --2- 172.21.15.107:0/1763897784 >> [v2:172.21.15.107:6800/1223039812,v1:172.21.15.107:6801/1223039812] conn(0x7f282406d760 0x7f282406fc10 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:46.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.870+0000 7f2847200640 1 --2- 172.21.15.107:0/1763897784 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2840102d00 0x7f2840115450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:46.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.870+0000 7f2847200640 1 --2- 172.21.15.107:0/1763897784 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2840102360 0x7f2840114f10 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T13:56:46.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.870+0000 7f2847200640 1 -- 172.21.15.107:0/1763897784 >> 172.21.15.107:0/1763897784 conn(0x7f28400f7ef0 msgr2=0x7f28400f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T13:56:46.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.870+0000 7f2847200640 1 -- 172.21.15.107:0/1763897784 shutdown_connections 2024-03-04T13:56:46.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-04T13:56:46.870+0000 7f2847200640 1 -- 172.21.15.107:0/1763897784 wait complete. 2024-03-04T13:56:46.889 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi107.front.sepia.ceph.com... 2024-03-04T13:56:46.889 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T13:56:46.889 DEBUG:teuthology.orchestra.run.smithi107:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-03-04T13:56:46.972 INFO:teuthology.orchestra.run:waiting for 300 2024-03-04T13:56:47.845 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:56:47 smithi107 ceph-mon[112578]: from='client.? 172.21.15.107:0/3390443960' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-04T13:56:47.846 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:56:47 smithi107 ceph-mon[112578]: pgmap v320: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:56:47.846 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:56:47 smithi107 ceph-mon[112578]: from='client.? 172.21.15.107:0/1763897784' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-04T13:56:47.863 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:56:47 smithi169 ceph-mon[86183]: from='client.? 172.21.15.107:0/3390443960' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-04T13:56:47.863 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:56:47 smithi169 ceph-mon[86183]: pgmap v320: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:56:47.863 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:56:47 smithi169 ceph-mon[86183]: from='client.? 172.21.15.107:0/1763897784' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-04T13:56:50.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:56:49 smithi107 ceph-mon[112578]: pgmap v321: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:56:50.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:56:49 smithi169 ceph-mon[86183]: pgmap v321: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:56:52.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:56:51 smithi107 ceph-mon[112578]: pgmap v322: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:56:52.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:56:51 smithi169 ceph-mon[86183]: pgmap v322: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:56:54.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:56:53 smithi107 ceph-mon[112578]: pgmap v323: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:56:54.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:56:53 smithi169 ceph-mon[86183]: pgmap v323: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:56:55.845 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:56:55 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:56:55.846 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:56:55 smithi107 ceph-mon[112578]: pgmap v324: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:56:55.862 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:56:55 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:56:55.862 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:56:55 smithi169 ceph-mon[86183]: pgmap v324: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:56:58.334 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:56:57 smithi107 ceph-mon[112578]: pgmap v325: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:56:58.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:56:57 smithi169 ceph-mon[86183]: pgmap v325: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:57:00.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:56:59 smithi107 ceph-mon[112578]: pgmap v326: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:57:00.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:56:59 smithi169 ceph-mon[86183]: pgmap v326: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:57:02.346 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:57:01 smithi107 ceph-mon[112578]: pgmap v327: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:57:02.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:57:01 smithi169 ceph-mon[86183]: pgmap v327: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:57:04.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:57:03 smithi107 ceph-mon[112578]: pgmap v328: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:57:04.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:57:03 smithi169 ceph-mon[86183]: pgmap v328: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:57:06.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:57:05 smithi107 ceph-mon[112578]: pgmap v329: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:57:06.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:57:05 smithi169 ceph-mon[86183]: pgmap v329: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:57:08.095 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:57:07 smithi107 ceph-mon[112578]: pgmap v330: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:57:08.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:57:07 smithi169 ceph-mon[86183]: pgmap v330: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:57:10.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:57:09 smithi107 ceph-mon[112578]: pgmap v331: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:57:10.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:57:09 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:57:10.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:57:09 smithi169 ceph-mon[86183]: pgmap v331: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:57:10.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:57:09 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:57:12.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:57:11 smithi107 ceph-mon[112578]: pgmap v332: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:57:12.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:57:11 smithi169 ceph-mon[86183]: pgmap v332: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:57:14.347 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:57:13 smithi107 ceph-mon[112578]: pgmap v333: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:57:14.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:57:13 smithi169 ceph-mon[86183]: pgmap v333: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:57:16.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:57:15 smithi107 ceph-mon[112578]: pgmap v334: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:57:16.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:57:15 smithi169 ceph-mon[86183]: pgmap v334: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:57:18.333 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:57:17 smithi107 ceph-mon[112578]: pgmap v335: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:57:18.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:57:17 smithi169 ceph-mon[86183]: pgmap v335: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:57:20.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:57:19 smithi107 ceph-mon[112578]: pgmap v336: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:57:20.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:57:19 smithi169 ceph-mon[86183]: pgmap v336: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:57:22.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:57:21 smithi107 ceph-mon[112578]: pgmap v337: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:57:22.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:57:21 smithi169 ceph-mon[86183]: pgmap v337: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:57:24.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:57:23 smithi107 ceph-mon[112578]: pgmap v338: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:57:24.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:57:23 smithi169 ceph-mon[86183]: pgmap v338: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:57:25.247 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:57:24 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:57:25.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:57:24 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:57:26.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:57:25 smithi107 ceph-mon[112578]: pgmap v339: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:57:26.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:57:25 smithi169 ceph-mon[86183]: pgmap v339: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:57:28.333 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:57:27 smithi107 ceph-mon[112578]: pgmap v340: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:57:28.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:57:27 smithi169 ceph-mon[86183]: pgmap v340: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:57:29.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:57:29 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:57:29.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:57:29 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:57:30.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:57:30 smithi107 ceph-mon[112578]: pgmap v341: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:57:30.346 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:57:30 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:57:30.346 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:57:30 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:57:30.346 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:57:30 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:57:30.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:57:30 smithi169 ceph-mon[86183]: pgmap v341: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:57:30.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:57:30 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:57:30.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:57:30 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:57:30.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:57:30 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:57:31.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:57:31 smithi107 ceph-mon[112578]: pgmap v342: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:57:31.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:57:31 smithi169 ceph-mon[86183]: pgmap v342: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:57:34.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:57:33 smithi107 ceph-mon[112578]: pgmap v343: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:57:34.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:57:33 smithi169 ceph-mon[86183]: pgmap v343: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:57:36.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:57:35 smithi107 ceph-mon[112578]: pgmap v344: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:57:36.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:57:35 smithi169 ceph-mon[86183]: pgmap v344: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:57:38.335 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:57:37 smithi107 ceph-mon[112578]: pgmap v345: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:57:38.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:57:37 smithi169 ceph-mon[86183]: pgmap v345: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:57:40.346 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:57:39 smithi107 ceph-mon[112578]: pgmap v346: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:57:40.346 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:57:39 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:57:40.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:57:39 smithi169 ceph-mon[86183]: pgmap v346: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:57:40.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:57:39 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:57:42.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:57:41 smithi107 ceph-mon[112578]: pgmap v347: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:57:42.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:57:41 smithi169 ceph-mon[86183]: pgmap v347: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:57:44.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:57:43 smithi107 ceph-mon[112578]: pgmap v348: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:57:44.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:57:43 smithi169 ceph-mon[86183]: pgmap v348: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:57:46.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:57:45 smithi107 ceph-mon[112578]: pgmap v349: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:57:46.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:57:45 smithi169 ceph-mon[86183]: pgmap v349: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:57:48.335 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:57:47 smithi107 ceph-mon[112578]: pgmap v350: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:57:48.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:57:47 smithi169 ceph-mon[86183]: pgmap v350: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:57:50.346 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:57:49 smithi107 ceph-mon[112578]: pgmap v351: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:57:50.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:57:49 smithi169 ceph-mon[86183]: pgmap v351: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:57:52.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:57:51 smithi107 ceph-mon[112578]: pgmap v352: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:57:52.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:57:51 smithi169 ceph-mon[86183]: pgmap v352: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:57:54.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:57:53 smithi107 ceph-mon[112578]: pgmap v353: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:57:54.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:57:53 smithi169 ceph-mon[86183]: pgmap v353: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:57:55.247 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:57:54 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:57:55.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:57:54 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:57:56.346 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:57:55 smithi107 ceph-mon[112578]: pgmap v354: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:57:56.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:57:55 smithi169 ceph-mon[86183]: pgmap v354: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:57:58.334 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:57:57 smithi107 ceph-mon[112578]: pgmap v355: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:57:58.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:57:57 smithi169 ceph-mon[86183]: pgmap v355: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:58:00.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:57:59 smithi107 ceph-mon[112578]: pgmap v356: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:58:00.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:57:59 smithi169 ceph-mon[86183]: pgmap v356: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:58:02.346 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:58:02 smithi107 ceph-mon[112578]: pgmap v357: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:58:02.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:58:02 smithi169 ceph-mon[86183]: pgmap v357: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:58:04.346 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:58:04 smithi107 ceph-mon[112578]: pgmap v358: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:58:04.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:58:04 smithi169 ceph-mon[86183]: pgmap v358: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:58:05.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:58:05 smithi107 ceph-mon[112578]: pgmap v359: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:58:05.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:58:05 smithi169 ceph-mon[86183]: pgmap v359: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:58:08.334 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:58:07 smithi107 ceph-mon[112578]: pgmap v360: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:58:08.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:58:07 smithi169 ceph-mon[86183]: pgmap v360: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:58:10.346 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:58:09 smithi107 ceph-mon[112578]: pgmap v361: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:58:10.346 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:58:09 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:58:10.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:58:09 smithi169 ceph-mon[86183]: pgmap v361: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:58:10.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:58:09 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:58:12.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:58:11 smithi107 ceph-mon[112578]: pgmap v362: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:58:12.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:58:11 smithi169 ceph-mon[86183]: pgmap v362: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:58:14.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:58:13 smithi107 ceph-mon[112578]: pgmap v363: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:58:14.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:58:13 smithi169 ceph-mon[86183]: pgmap v363: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:58:16.346 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:58:15 smithi107 ceph-mon[112578]: pgmap v364: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:58:16.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:58:15 smithi169 ceph-mon[86183]: pgmap v364: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:58:18.334 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:58:17 smithi107 ceph-mon[112578]: pgmap v365: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:58:18.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:58:17 smithi169 ceph-mon[86183]: pgmap v365: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:58:20.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:58:19 smithi107 ceph-mon[112578]: pgmap v366: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:58:20.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:58:19 smithi169 ceph-mon[86183]: pgmap v366: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:58:22.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:58:21 smithi107 ceph-mon[112578]: pgmap v367: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:58:22.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:58:21 smithi169 ceph-mon[86183]: pgmap v367: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:58:24.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:58:23 smithi107 ceph-mon[112578]: pgmap v368: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:58:24.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:58:23 smithi169 ceph-mon[86183]: pgmap v368: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:58:25.247 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:58:24 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:58:25.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:58:24 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:58:26.346 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:58:25 smithi107 ceph-mon[112578]: pgmap v369: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:58:26.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:58:25 smithi169 ceph-mon[86183]: pgmap v369: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:58:28.334 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:58:27 smithi107 ceph-mon[112578]: pgmap v370: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:58:28.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:58:27 smithi169 ceph-mon[86183]: pgmap v370: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:58:30.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:58:29 smithi107 ceph-mon[112578]: pgmap v371: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:58:30.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:58:29 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:58:30.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:58:29 smithi169 ceph-mon[86183]: pgmap v371: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:58:30.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:58:29 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:58:32.346 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:58:32 smithi107 ceph-mon[112578]: pgmap v372: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:58:32.346 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:58:32 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:58:32.346 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:58:32 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:58:32.346 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:58:32 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:58:32.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:58:32 smithi169 ceph-mon[86183]: pgmap v372: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:58:32.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:58:32 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:58:32.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:58:32 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:58:32.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:58:32 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:58:34.346 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:58:34 smithi107 ceph-mon[112578]: pgmap v373: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:58:34.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:58:34 smithi169 ceph-mon[86183]: pgmap v373: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:58:36.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:58:36 smithi107 ceph-mon[112578]: pgmap v374: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:58:36.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:58:36 smithi169 ceph-mon[86183]: pgmap v374: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:58:37.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:58:37 smithi107 ceph-mon[112578]: pgmap v375: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:58:37.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:58:37 smithi169 ceph-mon[86183]: pgmap v375: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:58:40.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:58:39 smithi107 ceph-mon[112578]: pgmap v376: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:58:40.346 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:58:39 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:58:40.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:58:39 smithi169 ceph-mon[86183]: pgmap v376: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:58:40.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:58:39 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:58:42.346 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:58:41 smithi107 ceph-mon[112578]: pgmap v377: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:58:42.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:58:41 smithi169 ceph-mon[86183]: pgmap v377: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:58:44.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:58:43 smithi107 ceph-mon[112578]: pgmap v378: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:58:44.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:58:43 smithi169 ceph-mon[86183]: pgmap v378: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:58:46.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:58:45 smithi107 ceph-mon[112578]: pgmap v379: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:58:46.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:58:45 smithi169 ceph-mon[86183]: pgmap v379: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:58:48.334 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:58:47 smithi107 ceph-mon[112578]: pgmap v380: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:58:48.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:58:47 smithi169 ceph-mon[86183]: pgmap v380: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:58:50.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:58:49 smithi107 ceph-mon[112578]: pgmap v381: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:58:50.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:58:49 smithi169 ceph-mon[86183]: pgmap v381: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:58:52.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:58:51 smithi107 ceph-mon[112578]: pgmap v382: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:58:52.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:58:51 smithi169 ceph-mon[86183]: pgmap v382: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:58:54.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:58:53 smithi107 ceph-mon[112578]: pgmap v383: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:58:54.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:58:53 smithi169 ceph-mon[86183]: pgmap v383: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:58:55.247 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:58:54 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:58:55.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:58:54 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:58:56.346 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:58:55 smithi107 ceph-mon[112578]: pgmap v384: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:58:56.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:58:55 smithi169 ceph-mon[86183]: pgmap v384: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:58:58.333 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:58:57 smithi107 ceph-mon[112578]: pgmap v385: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:58:58.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:58:57 smithi169 ceph-mon[86183]: pgmap v385: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:59:00.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:59:00 smithi107 ceph-mon[112578]: pgmap v386: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:59:00.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:59:00 smithi169 ceph-mon[86183]: pgmap v386: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:59:02.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:59:02 smithi107 ceph-mon[112578]: pgmap v387: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:59:02.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:59:02 smithi169 ceph-mon[86183]: pgmap v387: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:59:04.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:59:04 smithi107 ceph-mon[112578]: pgmap v388: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:59:04.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:59:04 smithi169 ceph-mon[86183]: pgmap v388: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:59:06.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:59:06 smithi107 ceph-mon[112578]: pgmap v389: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:59:06.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:59:06 smithi169 ceph-mon[86183]: pgmap v389: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:59:07.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:59:07 smithi107 ceph-mon[112578]: pgmap v390: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:59:07.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:59:07 smithi169 ceph-mon[86183]: pgmap v390: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:59:10.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:59:09 smithi107 ceph-mon[112578]: pgmap v391: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:59:10.346 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:59:09 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:59:10.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:59:09 smithi169 ceph-mon[86183]: pgmap v391: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:59:10.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:59:09 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:59:12.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:59:11 smithi107 ceph-mon[112578]: pgmap v392: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:59:12.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:59:11 smithi169 ceph-mon[86183]: pgmap v392: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:59:14.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:59:13 smithi107 ceph-mon[112578]: pgmap v393: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:59:14.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:59:13 smithi169 ceph-mon[86183]: pgmap v393: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:59:16.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:59:15 smithi107 ceph-mon[112578]: pgmap v394: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:59:16.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:59:15 smithi169 ceph-mon[86183]: pgmap v394: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:59:18.335 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:59:17 smithi107 ceph-mon[112578]: pgmap v395: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:59:18.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:59:17 smithi169 ceph-mon[86183]: pgmap v395: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:59:20.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:59:19 smithi107 ceph-mon[112578]: pgmap v396: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:59:20.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:59:19 smithi169 ceph-mon[86183]: pgmap v396: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:59:22.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:59:21 smithi107 ceph-mon[112578]: pgmap v397: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:59:22.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:59:21 smithi169 ceph-mon[86183]: pgmap v397: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:59:24.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:59:23 smithi107 ceph-mon[112578]: pgmap v398: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:59:24.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:59:23 smithi169 ceph-mon[86183]: pgmap v398: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:59:25.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:59:25 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:59:25.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:59:25 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:59:26.346 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:59:26 smithi107 ceph-mon[112578]: pgmap v399: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:59:26.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:59:26 smithi169 ceph-mon[86183]: pgmap v399: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:59:28.334 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:59:28 smithi107 ceph-mon[112578]: pgmap v400: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:59:28.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:59:28 smithi169 ceph-mon[86183]: pgmap v400: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:59:30.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:59:30 smithi107 ceph-mon[112578]: pgmap v401: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:59:30.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:59:30 smithi169 ceph-mon[86183]: pgmap v401: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:59:32.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:59:32 smithi107 ceph-mon[112578]: pgmap v402: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:59:32.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:59:32 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:59:32.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:59:32 smithi169 ceph-mon[86183]: pgmap v402: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:59:32.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:59:32 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T13:59:33.346 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:59:33 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:59:33.346 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:59:33 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:59:33.346 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:59:33 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:59:33.346 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:59:33 smithi107 ceph-mon[112578]: pgmap v403: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:59:33.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:59:33 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T13:59:33.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:59:33 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T13:59:33.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:59:33 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T13:59:33.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:59:33 smithi169 ceph-mon[86183]: pgmap v403: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:59:36.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:59:35 smithi107 ceph-mon[112578]: pgmap v404: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:59:36.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:59:35 smithi169 ceph-mon[86183]: pgmap v404: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:59:38.334 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:59:37 smithi107 ceph-mon[112578]: pgmap v405: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:59:38.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:59:37 smithi169 ceph-mon[86183]: pgmap v405: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:59:40.346 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:59:39 smithi107 ceph-mon[112578]: pgmap v406: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:59:40.346 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:59:39 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:59:40.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:59:39 smithi169 ceph-mon[86183]: pgmap v406: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:59:40.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:59:39 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:59:42.346 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:59:41 smithi107 ceph-mon[112578]: pgmap v407: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:59:42.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:59:41 smithi169 ceph-mon[86183]: pgmap v407: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:59:44.346 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:59:43 smithi107 ceph-mon[112578]: pgmap v408: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:59:44.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:59:43 smithi169 ceph-mon[86183]: pgmap v408: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:59:46.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:59:45 smithi107 ceph-mon[112578]: pgmap v409: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:59:46.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:59:45 smithi169 ceph-mon[86183]: pgmap v409: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:59:48.334 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:59:47 smithi107 ceph-mon[112578]: pgmap v410: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:59:48.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:59:47 smithi169 ceph-mon[86183]: pgmap v410: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T13:59:50.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:59:50 smithi107 ceph-mon[112578]: pgmap v411: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:59:50.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:59:50 smithi169 ceph-mon[86183]: pgmap v411: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:59:52.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:59:52 smithi107 ceph-mon[112578]: pgmap v412: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:59:52.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:59:52 smithi169 ceph-mon[86183]: pgmap v412: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:59:54.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:59:54 smithi107 ceph-mon[112578]: pgmap v413: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:59:54.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:59:54 smithi169 ceph-mon[86183]: pgmap v413: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:59:55.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:59:55 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:59:55.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:59:55 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T13:59:56.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:59:56 smithi107 ceph-mon[112578]: pgmap v414: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:59:56.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:59:56 smithi169 ceph-mon[86183]: pgmap v414: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:59:58.333 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:59:58 smithi107 ceph-mon[112578]: pgmap v415: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:59:58.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:59:58 smithi169 ceph-mon[86183]: pgmap v415: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T13:59:59.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 13:59:59 smithi107 ceph-mon[112578]: pgmap v416: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T13:59:59.362 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 13:59:59 smithi169 ceph-mon[86183]: pgmap v416: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T14:00:00.346 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 14:00:00 smithi107 ceph-mon[112578]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-03-04T14:00:00.346 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 14:00:00 smithi107 ceph-mon[112578]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-04T14:00:00.346 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 14:00:00 smithi107 ceph-mon[112578]: fs cephfs has deprecated feature inline_data enabled. 2024-03-04T14:00:00.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 14:00:00 smithi169 ceph-mon[86183]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-03-04T14:00:00.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 14:00:00 smithi169 ceph-mon[86183]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-04T14:00:00.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 14:00:00 smithi169 ceph-mon[86183]: fs cephfs has deprecated feature inline_data enabled. 2024-03-04T14:00:01.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 14:00:01 smithi169 ceph-mon[86183]: pgmap v417: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T14:00:01.595 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 14:00:01 smithi107 ceph-mon[112578]: pgmap v417: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T14:00:04.346 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 14:00:03 smithi107 ceph-mon[112578]: pgmap v418: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T14:00:04.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 14:00:03 smithi169 ceph-mon[86183]: pgmap v418: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T14:00:06.346 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 14:00:05 smithi107 ceph-mon[112578]: pgmap v419: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T14:00:06.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 14:00:05 smithi169 ceph-mon[86183]: pgmap v419: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T14:00:08.333 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 14:00:07 smithi107 ceph-mon[112578]: pgmap v420: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T14:00:08.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 14:00:07 smithi169 ceph-mon[86183]: pgmap v420: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T14:00:10.346 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 14:00:09 smithi107 ceph-mon[112578]: pgmap v421: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T14:00:10.346 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 14:00:09 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T14:00:10.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 14:00:09 smithi169 ceph-mon[86183]: pgmap v421: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T14:00:10.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 14:00:09 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T14:00:12.346 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 14:00:11 smithi107 ceph-mon[112578]: pgmap v422: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T14:00:12.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 14:00:11 smithi169 ceph-mon[86183]: pgmap v422: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T14:00:14.346 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 14:00:14 smithi107 ceph-mon[112578]: pgmap v423: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T14:00:14.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 14:00:14 smithi169 ceph-mon[86183]: pgmap v423: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T14:00:16.346 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 14:00:16 smithi107 ceph-mon[112578]: pgmap v424: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T14:00:16.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 14:00:16 smithi169 ceph-mon[86183]: pgmap v424: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T14:00:18.333 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 14:00:18 smithi107 ceph-mon[112578]: pgmap v425: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T14:00:18.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 14:00:18 smithi169 ceph-mon[86183]: pgmap v425: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T14:00:20.346 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 14:00:20 smithi107 ceph-mon[112578]: pgmap v426: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T14:00:20.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 14:00:20 smithi169 ceph-mon[86183]: pgmap v426: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T14:00:22.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 14:00:22 smithi107 ceph-mon[112578]: pgmap v427: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T14:00:22.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 14:00:22 smithi169 ceph-mon[86183]: pgmap v427: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T14:00:24.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 14:00:24 smithi107 ceph-mon[112578]: pgmap v428: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T14:00:24.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 14:00:24 smithi169 ceph-mon[86183]: pgmap v428: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T14:00:25.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 14:00:25 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T14:00:25.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 14:00:25 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T14:00:26.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 14:00:26 smithi107 ceph-mon[112578]: pgmap v429: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T14:00:26.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 14:00:26 smithi169 ceph-mon[86183]: pgmap v429: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T14:00:27.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 14:00:27 smithi107 ceph-mon[112578]: pgmap v430: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T14:00:27.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 14:00:27 smithi169 ceph-mon[86183]: pgmap v430: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T14:00:30.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 14:00:29 smithi107 ceph-mon[112578]: pgmap v431: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T14:00:30.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 14:00:29 smithi169 ceph-mon[86183]: pgmap v431: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T14:00:32.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 14:00:31 smithi107 ceph-mon[112578]: pgmap v432: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T14:00:32.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 14:00:31 smithi169 ceph-mon[86183]: pgmap v432: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T14:00:34.346 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 14:00:33 smithi107 ceph-mon[112578]: pgmap v433: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T14:00:34.346 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 14:00:33 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T14:00:34.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 14:00:33 smithi169 ceph-mon[86183]: pgmap v433: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T14:00:34.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 14:00:33 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T14:00:35.346 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 14:00:34 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T14:00:35.346 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 14:00:34 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T14:00:35.346 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 14:00:34 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T14:00:35.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 14:00:34 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T14:00:35.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 14:00:34 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T14:00:35.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 14:00:34 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T14:00:36.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 14:00:35 smithi107 ceph-mon[112578]: pgmap v434: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T14:00:36.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 14:00:35 smithi169 ceph-mon[86183]: pgmap v434: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T14:00:38.334 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 14:00:38 smithi107 ceph-mon[112578]: pgmap v435: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T14:00:38.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 14:00:38 smithi169 ceph-mon[86183]: pgmap v435: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T14:00:40.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 14:00:40 smithi107 ceph-mon[112578]: pgmap v436: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T14:00:40.346 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 14:00:40 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T14:00:40.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 14:00:40 smithi169 ceph-mon[86183]: pgmap v436: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T14:00:40.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 14:00:40 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T14:00:42.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 14:00:42 smithi107 ceph-mon[112578]: pgmap v437: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T14:00:42.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 14:00:42 smithi169 ceph-mon[86183]: pgmap v437: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T14:00:44.346 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 14:00:44 smithi107 ceph-mon[112578]: pgmap v438: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T14:00:44.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 14:00:44 smithi169 ceph-mon[86183]: pgmap v438: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T14:00:46.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 14:00:46 smithi107 ceph-mon[112578]: pgmap v439: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T14:00:46.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 14:00:46 smithi169 ceph-mon[86183]: pgmap v439: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T14:00:48.334 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 14:00:48 smithi107 ceph-mon[112578]: pgmap v440: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T14:00:48.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 14:00:48 smithi169 ceph-mon[86183]: pgmap v440: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T14:00:50.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 14:00:50 smithi107 ceph-mon[112578]: pgmap v441: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T14:00:50.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 14:00:50 smithi169 ceph-mon[86183]: pgmap v441: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T14:00:52.346 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 14:00:52 smithi107 ceph-mon[112578]: pgmap v442: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T14:00:52.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 14:00:52 smithi169 ceph-mon[86183]: pgmap v442: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T14:00:53.346 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 14:00:53 smithi107 ceph-mon[112578]: pgmap v443: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T14:00:53.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 14:00:53 smithi169 ceph-mon[86183]: pgmap v443: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T14:00:55.096 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 14:00:54 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T14:00:55.113 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 14:00:54 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T14:00:56.096 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 14:00:55 smithi107 ceph-mon[112578]: pgmap v444: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T14:00:56.113 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 14:00:55 smithi169 ceph-mon[86183]: pgmap v444: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T14:00:58.333 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 14:00:57 smithi107 ceph-mon[112578]: pgmap v445: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T14:00:58.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 14:00:57 smithi169 ceph-mon[86183]: pgmap v445: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T14:01:00.346 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 14:00:59 smithi107 ceph-mon[112578]: pgmap v446: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T14:01:00.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 14:00:59 smithi169 ceph-mon[86183]: pgmap v446: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T14:01:02.113 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 14:01:02 smithi169 ceph-mon[86183]: pgmap v447: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T14:01:02.346 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 14:01:02 smithi107 ceph-mon[112578]: pgmap v447: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T14:01:04.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 14:01:04 smithi107 ceph-mon[112578]: pgmap v448: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T14:01:04.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 14:01:04 smithi169 ceph-mon[86183]: pgmap v448: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T14:01:06.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 14:01:06 smithi107 ceph-mon[112578]: pgmap v449: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T14:01:06.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 14:01:06 smithi169 ceph-mon[86183]: pgmap v449: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T14:01:08.334 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 14:01:08 smithi107 ceph-mon[112578]: pgmap v450: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T14:01:08.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 14:01:08 smithi169 ceph-mon[86183]: pgmap v450: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T14:01:10.346 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 14:01:10 smithi107 ceph-mon[112578]: pgmap v451: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T14:01:10.346 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 14:01:10 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T14:01:10.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 14:01:10 smithi169 ceph-mon[86183]: pgmap v451: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T14:01:10.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 14:01:10 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T14:01:12.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 14:01:12 smithi107 ceph-mon[112578]: pgmap v452: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T14:01:12.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 14:01:12 smithi169 ceph-mon[86183]: pgmap v452: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T14:01:14.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 14:01:14 smithi107 ceph-mon[112578]: pgmap v453: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T14:01:14.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 14:01:14 smithi169 ceph-mon[86183]: pgmap v453: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T14:01:16.346 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 14:01:16 smithi107 ceph-mon[112578]: pgmap v454: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T14:01:16.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 14:01:16 smithi169 ceph-mon[86183]: pgmap v454: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T14:01:18.333 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 14:01:18 smithi107 ceph-mon[112578]: pgmap v455: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T14:01:18.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 14:01:18 smithi169 ceph-mon[86183]: pgmap v455: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T14:01:19.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 14:01:19 smithi169 ceph-mon[86183]: pgmap v456: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T14:01:19.596 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 14:01:19 smithi107 ceph-mon[112578]: pgmap v456: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T14:01:22.346 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 14:01:22 smithi107 ceph-mon[112578]: pgmap v457: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T14:01:22.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 14:01:22 smithi169 ceph-mon[86183]: pgmap v457: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T14:01:24.346 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 14:01:24 smithi107 ceph-mon[112578]: pgmap v458: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T14:01:24.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 14:01:24 smithi169 ceph-mon[86183]: pgmap v458: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T14:01:25.277 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 14:01:25 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T14:01:25.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 14:01:25 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T14:01:26.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 14:01:26 smithi107 ceph-mon[112578]: pgmap v459: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T14:01:26.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 14:01:26 smithi169 ceph-mon[86183]: pgmap v459: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T14:01:28.335 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 14:01:28 smithi107 ceph-mon[112578]: pgmap v460: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T14:01:28.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 14:01:28 smithi169 ceph-mon[86183]: pgmap v460: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T14:01:30.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 14:01:30 smithi107 ceph-mon[112578]: pgmap v461: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T14:01:30.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 14:01:30 smithi169 ceph-mon[86183]: pgmap v461: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T14:01:32.346 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 14:01:32 smithi107 ceph-mon[112578]: pgmap v462: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T14:01:32.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 14:01:32 smithi169 ceph-mon[86183]: pgmap v462: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T14:01:34.346 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 14:01:34 smithi107 ceph-mon[112578]: pgmap v463: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T14:01:34.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 14:01:34 smithi169 ceph-mon[86183]: pgmap v463: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T14:01:35.346 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 14:01:35 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T14:01:35.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 14:01:35 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T14:01:36.346 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 14:01:36 smithi107 ceph-mon[112578]: pgmap v464: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T14:01:36.346 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 14:01:36 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T14:01:36.346 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 14:01:36 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T14:01:36.346 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 14:01:36 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T14:01:36.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 14:01:36 smithi169 ceph-mon[86183]: pgmap v464: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-04T14:01:36.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 14:01:36 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T14:01:36.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 14:01:36 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T14:01:36.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 14:01:36 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' 2024-03-04T14:01:37.346 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 14:01:37 smithi107 ceph-mon[112578]: pgmap v465: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T14:01:37.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 14:01:37 smithi169 ceph-mon[86183]: pgmap v465: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T14:01:40.347 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 14:01:40 smithi107 ceph-mon[112578]: pgmap v466: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T14:01:40.348 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 14:01:40 smithi107 ceph-mon[112578]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T14:01:40.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 14:01:40 smithi169 ceph-mon[86183]: pgmap v466: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T14:01:40.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 14:01:40 smithi169 ceph-mon[86183]: from='mgr.34102 172.21.15.107:0/1329748503' entity='mgr.smithi107.wpzgwl' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T14:01:42.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 14:01:42 smithi107 ceph-mon[112578]: pgmap v467: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T14:01:42.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 14:01:42 smithi169 ceph-mon[86183]: pgmap v467: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T14:01:44.346 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 14:01:44 smithi107 ceph-mon[112578]: pgmap v468: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T14:01:44.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 14:01:44 smithi169 ceph-mon[86183]: pgmap v468: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T14:01:46.041 ERROR:tasks.cephfs.fuse_mount:process failed to terminate after unmount. This probably indicates a bug within ceph-fuse. 2024-03-04T14:01:46.042 ERROR:teuthology.run_tasks:Manager failed: ceph-fuse Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_717ce3cce8d0166e4e577faa8004e6f9cb4128c0/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph-c_717ce3cce8d0166e4e577faa8004e6f9cb4128c0/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-03-04T14:01:46.043 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-03-04T14:01:46.055 INFO:tasks.cephadm:Teardown begin 2024-03-04T14:01:46.055 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_717ce3cce8d0166e4e577faa8004e6f9cb4128c0/qa/tasks/cephadm.py", line 1867, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_717ce3cce8d0166e4e577faa8004e6f9cb4128c0/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph-c_717ce3cce8d0166e4e577faa8004e6f9cb4128c0/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-03-04T14:01:46.056 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-04T14:01:46.162 DEBUG:teuthology.orchestra.run.smithi169:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-04T14:01:46.265 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-03-04T14:01:46.265 DEBUG:teuthology.orchestra.run.smithi107:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-04T14:01:46.322 DEBUG:teuthology.orchestra.run.smithi169:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-04T14:01:46.345 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 14:01:46 smithi107 ceph-mon[112578]: pgmap v469: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T14:01:46.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 14:01:46 smithi169 ceph-mon[86183]: pgmap v469: 65 pgs: 65 active+clean; 255 MiB data, 964 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-04T14:01:46.379 INFO:tasks.cephadm:Stopping all daemons... 2024-03-04T14:01:46.379 INFO:tasks.cephadm.mon.smithi107:Stopping mon.smithi107... 2024-03-04T14:01:46.379 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-1fc09a58-da2b-11ee-95c6-87774f69a715@mon.smithi107 2024-03-04T14:01:46.845 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 14:01:46 smithi107 systemd[1]: Stopping Ceph mon.smithi107 for 1fc09a58-da2b-11ee-95c6-87774f69a715... 2024-03-04T14:01:47.346 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 14:01:46 smithi107 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-mon-smithi107[112574]: 2024-03-04T14:01:46.923+0000 7f656d200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi107 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-04T14:01:47.346 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 14:01:46 smithi107 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-mon-smithi107[112574]: 2024-03-04T14:01:46.923+0000 7f656d200700 -1 mon.smithi107@0(leader) e3 *** Got Signal Terminated *** 2024-03-04T14:01:47.346 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 14:01:47 smithi107 podman[155334]: 2024-03-04 14:01:47.057447668 +0000 UTC m=+0.278612899 container died 32f07144274f6a2f547db4f34551424313f2e40c5d24141940b97c90ca4204eb (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-mon-smithi107, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240222, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-04T14:01:48.095 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 14:01:47 smithi107 podman[155334]: 2024-03-04 14:01:47.643690244 +0000 UTC m=+0.864855537 container cleanup 32f07144274f6a2f547db4f34551424313f2e40c5d24141940b97c90ca4204eb (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-mon-smithi107, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240222, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-03-04T14:01:48.096 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 14:01:47 smithi107 bash[155334]: ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-mon-smithi107 2024-03-04T14:01:48.096 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Mar 04 14:01:47 smithi107 podman[155386]: 2024-03-04 14:01:47.872708345 +0000 UTC m=+0.805355352 container remove 32f07144274f6a2f547db4f34551424313f2e40c5d24141940b97c90ca4204eb (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-mon-smithi107, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2) 2024-03-04T14:01:48.282 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1fc09a58-da2b-11ee-95c6-87774f69a715@mon.smithi107.service' 2024-03-04T14:01:48.421 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-04T14:01:48.421 INFO:tasks.cephadm.mon.smithi107:Stopped mon.smithi107 2024-03-04T14:01:48.421 INFO:tasks.cephadm.mon.smithi169:Stopping mon.smithi169... 2024-03-04T14:01:48.422 DEBUG:teuthology.orchestra.run.smithi169:> sudo systemctl stop ceph-1fc09a58-da2b-11ee-95c6-87774f69a715@mon.smithi169 2024-03-04T14:01:48.863 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 14:01:48 smithi169 systemd[1]: Stopping Ceph mon.smithi169 for 1fc09a58-da2b-11ee-95c6-87774f69a715... 2024-03-04T14:01:49.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 14:01:49 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-mon-smithi169[86179]: 2024-03-04T14:01:49.004+0000 7f0b7b600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi169 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-04T14:01:49.364 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 14:01:49 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-mon-smithi169[86179]: 2024-03-04T14:01:49.004+0000 7f0b7b600700 -1 mon.smithi169@1(peon) e3 *** Got Signal Terminated *** 2024-03-04T14:01:49.364 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 14:01:49 smithi169 podman[118762]: 2024-03-04 14:01:49.170286586 +0000 UTC m=+0.309341269 container died e3527449381aee95c0af0351abb7d1250f92f63206a092eabd894a5a84545a78 (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-mon-smithi169, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-04T14:01:49.863 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 14:01:49 smithi169 podman[118762]: 2024-03-04 14:01:49.688716239 +0000 UTC m=+0.827770929 container cleanup e3527449381aee95c0af0351abb7d1250f92f63206a092eabd894a5a84545a78 (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-mon-smithi169, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-03-04T14:01:49.863 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 14:01:49 smithi169 bash[118762]: ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-mon-smithi169 2024-03-04T14:01:50.363 INFO:journalctl@ceph.mon.smithi169.smithi169.stdout:Mar 04 14:01:49 smithi169 podman[118776]: 2024-03-04 14:01:49.973854741 +0000 UTC m=+0.789268042 container remove e3527449381aee95c0af0351abb7d1250f92f63206a092eabd894a5a84545a78 (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-mon-smithi169, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, ceph=True, org.label-schema.build-date=20240222, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-03-04T14:01:50.464 DEBUG:teuthology.orchestra.run.smithi169:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1fc09a58-da2b-11ee-95c6-87774f69a715@mon.smithi169.service' 2024-03-04T14:01:50.600 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-04T14:01:50.600 INFO:tasks.cephadm.mon.smithi169:Stopped mon.smithi169 2024-03-04T14:01:50.600 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-03-04T14:01:50.600 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-1fc09a58-da2b-11ee-95c6-87774f69a715@osd.0 2024-03-04T14:01:51.096 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 04 14:01:50 smithi107 systemd[1]: Stopping Ceph osd.0 for 1fc09a58-da2b-11ee-95c6-87774f69a715... 2024-03-04T14:01:51.596 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 04 14:01:51 smithi107 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-0[124261]: 2024-03-04T14:01:51.213+0000 7fa29fc00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-04T14:01:51.596 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 04 14:01:51 smithi107 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-0[124261]: 2024-03-04T14:01:51.213+0000 7fa29fc00700 -1 osd.0 86 *** Got signal Terminated *** 2024-03-04T14:01:51.596 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 04 14:01:51 smithi107 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-0[124261]: 2024-03-04T14:01:51.213+0000 7fa29fc00700 -1 osd.0 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-04T14:01:56.982 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 04 14:01:56 smithi107 podman[155551]: 2024-03-04 14:01:56.686399885 +0000 UTC m=+5.674366390 container died 5801a5540dc65ce77698d452b23e13632a4bf736fa4f5f30f57df1379a41f32a (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-0, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, CEPH_POINT_RELEASE=) 2024-03-04T14:01:57.345 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 04 14:01:57 smithi107 podman[155551]: 2024-03-04 14:01:57.114118282 +0000 UTC m=+6.102084818 container cleanup 5801a5540dc65ce77698d452b23e13632a4bf736fa4f5f30f57df1379a41f32a (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-0, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c) 2024-03-04T14:01:57.346 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 04 14:01:57 smithi107 bash[155551]: ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-0 2024-03-04T14:01:57.846 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 04 14:01:57 smithi107 podman[155570]: 2024-03-04 14:01:57.390504938 +0000 UTC m=+0.690984737 container remove 5801a5540dc65ce77698d452b23e13632a4bf736fa4f5f30f57df1379a41f32a (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8) 2024-03-04T14:01:58.522 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 04 14:01:58 smithi107 podman[155659]: 2024-03-04 14:01:58.144675802 +0000 UTC m=+0.036003795 image pull d71b839abdf892550525dad59fdf20a6b925b73dbcc7526f0d04da8380b9fabf quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377 2024-03-04T14:01:58.522 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 04 14:01:58 smithi107 podman[155659]: 2024-03-04 14:01:58.266499981 +0000 UTC m=+0.157828023 container create ec1cb8224f797dfd08bd933e4a25e933d4d5280e9a47a11677813f90bff42c37 (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-0-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, io.buildah.version=1.33.5, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True) 2024-03-04T14:01:58.522 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 04 14:01:58 smithi107 podman[155659]: 2024-03-04 14:01:58.519734183 +0000 UTC m=+0.411062225 container init ec1cb8224f797dfd08bd933e4a25e933d4d5280e9a47a11677813f90bff42c37 (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-0-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, org.label-schema.build-date=20240222, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-03-04T14:01:58.777 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 04 14:01:58 smithi107 podman[155659]: 2024-03-04 14:01:58.529843627 +0000 UTC m=+0.421171614 container start ec1cb8224f797dfd08bd933e4a25e933d4d5280e9a47a11677813f90bff42c37 (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-0-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240222, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-03-04T14:01:58.777 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 04 14:01:58 smithi107 podman[155659]: 2024-03-04 14:01:58.660247798 +0000 UTC m=+0.551575795 container attach ec1cb8224f797dfd08bd933e4a25e933d4d5280e9a47a11677813f90bff42c37 (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-0-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.license=GPLv2, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=) 2024-03-04T14:01:59.080 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 04 14:01:58 smithi107 podman[155659]: 2024-03-04 14:01:58.987861974 +0000 UTC m=+0.879189965 container died ec1cb8224f797dfd08bd933e4a25e933d4d5280e9a47a11677813f90bff42c37 (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8) 2024-03-04T14:01:59.751 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1fc09a58-da2b-11ee-95c6-87774f69a715@osd.0.service' 2024-03-04T14:01:59.862 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-04T14:01:59.862 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-03-04T14:01:59.862 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-03-04T14:01:59.862 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-1fc09a58-da2b-11ee-95c6-87774f69a715@osd.1 2024-03-04T14:02:00.346 INFO:journalctl@ceph.osd.1.smithi107.stdout:Mar 04 14:02:00 smithi107 systemd[1]: Stopping Ceph osd.1 for 1fc09a58-da2b-11ee-95c6-87774f69a715... 2024-03-04T14:02:00.846 INFO:journalctl@ceph.osd.1.smithi107.stdout:Mar 04 14:02:00 smithi107 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-1[130089]: 2024-03-04T14:02:00.431+0000 7f3647e00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-04T14:02:00.846 INFO:journalctl@ceph.osd.1.smithi107.stdout:Mar 04 14:02:00 smithi107 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-1[130089]: 2024-03-04T14:02:00.431+0000 7f3647e00700 -1 osd.1 86 *** Got signal Terminated *** 2024-03-04T14:02:00.846 INFO:journalctl@ceph.osd.1.smithi107.stdout:Mar 04 14:02:00 smithi107 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-1[130089]: 2024-03-04T14:02:00.431+0000 7f3647e00700 -1 osd.1 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-04T14:02:06.310 INFO:journalctl@ceph.osd.1.smithi107.stdout:Mar 04 14:02:06 smithi107 podman[155777]: 2024-03-04 14:02:06.013264712 +0000 UTC m=+5.739083369 container died b614af6588548b0b59096d1f9d8cc60aeb4e06c756b823e4092c9a2507fa20d6 (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-1, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c) 2024-03-04T14:02:06.596 INFO:journalctl@ceph.osd.1.smithi107.stdout:Mar 04 14:02:06 smithi107 podman[155777]: 2024-03-04 14:02:06.483500355 +0000 UTC m=+6.209319050 container cleanup b614af6588548b0b59096d1f9d8cc60aeb4e06c756b823e4092c9a2507fa20d6 (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-1, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-03-04T14:02:06.596 INFO:journalctl@ceph.osd.1.smithi107.stdout:Mar 04 14:02:06 smithi107 bash[155777]: ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-1 2024-03-04T14:02:07.345 INFO:journalctl@ceph.osd.1.smithi107.stdout:Mar 04 14:02:06 smithi107 podman[155798]: 2024-03-04 14:02:06.876489374 +0000 UTC m=+0.848294267 container remove b614af6588548b0b59096d1f9d8cc60aeb4e06c756b823e4092c9a2507fa20d6 (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-1, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-04T14:02:07.791 INFO:journalctl@ceph.osd.1.smithi107.stdout:Mar 04 14:02:07 smithi107 podman[155886]: 2024-03-04 14:02:07.387398877 +0000 UTC m=+0.033200516 image pull d71b839abdf892550525dad59fdf20a6b925b73dbcc7526f0d04da8380b9fabf quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377 2024-03-04T14:02:07.791 INFO:journalctl@ceph.osd.1.smithi107.stdout:Mar 04 14:02:07 smithi107 podman[155886]: 2024-03-04 14:02:07.502198833 +0000 UTC m=+0.148000509 container create 24e7685b7d8660c1022a2c746960f940e7eca3e699a3a528f151ef22728afa6e (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-1-deactivate, ceph=True, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-04T14:02:07.792 INFO:journalctl@ceph.osd.1.smithi107.stdout:Mar 04 14:02:07 smithi107 podman[155886]: 2024-03-04 14:02:07.7887895 +0000 UTC m=+0.434591230 container init 24e7685b7d8660c1022a2c746960f940e7eca3e699a3a528f151ef22728afa6e (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-1-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240222, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-03-04T14:02:08.096 INFO:journalctl@ceph.osd.1.smithi107.stdout:Mar 04 14:02:07 smithi107 podman[155886]: 2024-03-04 14:02:07.800090691 +0000 UTC m=+0.445892337 container start 24e7685b7d8660c1022a2c746960f940e7eca3e699a3a528f151ef22728afa6e (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-1-deactivate, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240222, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-03-04T14:02:08.096 INFO:journalctl@ceph.osd.1.smithi107.stdout:Mar 04 14:02:07 smithi107 podman[155886]: 2024-03-04 14:02:07.879921956 +0000 UTC m=+0.525723628 container attach 24e7685b7d8660c1022a2c746960f940e7eca3e699a3a528f151ef22728afa6e (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-1-deactivate, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240222, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-03-04T14:02:08.596 INFO:journalctl@ceph.osd.1.smithi107.stdout:Mar 04 14:02:08 smithi107 podman[155886]: 2024-03-04 14:02:08.237653493 +0000 UTC m=+0.883455132 container died 24e7685b7d8660c1022a2c746960f940e7eca3e699a3a528f151ef22728afa6e (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-1-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, ceph=True) 2024-03-04T14:02:09.059 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1fc09a58-da2b-11ee-95c6-87774f69a715@osd.1.service' 2024-03-04T14:02:09.183 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-04T14:02:09.183 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-03-04T14:02:09.184 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-03-04T14:02:09.184 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-1fc09a58-da2b-11ee-95c6-87774f69a715@osd.2 2024-03-04T14:02:09.597 INFO:journalctl@ceph.osd.2.smithi107.stdout:Mar 04 14:02:09 smithi107 systemd[1]: Stopping Ceph osd.2 for 1fc09a58-da2b-11ee-95c6-87774f69a715... 2024-03-04T14:02:10.096 INFO:journalctl@ceph.osd.2.smithi107.stdout:Mar 04 14:02:09 smithi107 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-2[135773]: 2024-03-04T14:02:09.758+0000 7f606e400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-04T14:02:10.096 INFO:journalctl@ceph.osd.2.smithi107.stdout:Mar 04 14:02:09 smithi107 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-2[135773]: 2024-03-04T14:02:09.758+0000 7f606e400700 -1 osd.2 86 *** Got signal Terminated *** 2024-03-04T14:02:10.096 INFO:journalctl@ceph.osd.2.smithi107.stdout:Mar 04 14:02:09 smithi107 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-2[135773]: 2024-03-04T14:02:09.758+0000 7f606e400700 -1 osd.2 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-04T14:02:15.502 INFO:journalctl@ceph.osd.2.smithi107.stdout:Mar 04 14:02:15 smithi107 podman[155991]: 2024-03-04 14:02:15.18287736 +0000 UTC m=+5.597668825 container died 6e4da4f6f146276914ee5dbafa3f0c124a08bea50684b074bf446e26f95314dd (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS) 2024-03-04T14:02:15.846 INFO:journalctl@ceph.osd.2.smithi107.stdout:Mar 04 14:02:15 smithi107 podman[155991]: 2024-03-04 14:02:15.610388609 +0000 UTC m=+6.025180094 container cleanup 6e4da4f6f146276914ee5dbafa3f0c124a08bea50684b074bf446e26f95314dd (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240222, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, CEPH_POINT_RELEASE=) 2024-03-04T14:02:15.846 INFO:journalctl@ceph.osd.2.smithi107.stdout:Mar 04 14:02:15 smithi107 bash[155991]: ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-2 2024-03-04T14:02:16.346 INFO:journalctl@ceph.osd.2.smithi107.stdout:Mar 04 14:02:15 smithi107 podman[156010]: 2024-03-04 14:02:15.889233543 +0000 UTC m=+0.697252485 container remove 6e4da4f6f146276914ee5dbafa3f0c124a08bea50684b074bf446e26f95314dd (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240222, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8) 2024-03-04T14:02:17.052 INFO:journalctl@ceph.osd.2.smithi107.stdout:Mar 04 14:02:16 smithi107 podman[156098]: 2024-03-04 14:02:16.621561401 +0000 UTC m=+0.034719828 image pull d71b839abdf892550525dad59fdf20a6b925b73dbcc7526f0d04da8380b9fabf quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377 2024-03-04T14:02:17.052 INFO:journalctl@ceph.osd.2.smithi107.stdout:Mar 04 14:02:16 smithi107 podman[156098]: 2024-03-04 14:02:16.795877332 +0000 UTC m=+0.209035783 container create 0c380c742b243820c3ef5b23e4f172498f66e0b147b4cc1b71b391b2377856f0 (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-2-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2) 2024-03-04T14:02:17.053 INFO:journalctl@ceph.osd.2.smithi107.stdout:Mar 04 14:02:17 smithi107 podman[156098]: 2024-03-04 14:02:17.049724985 +0000 UTC m=+0.462883439 container init 0c380c742b243820c3ef5b23e4f172498f66e0b147b4cc1b71b391b2377856f0 (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-2-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, org.label-schema.build-date=20240222, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-03-04T14:02:17.346 INFO:journalctl@ceph.osd.2.smithi107.stdout:Mar 04 14:02:17 smithi107 podman[156098]: 2024-03-04 14:02:17.061588892 +0000 UTC m=+0.474747312 container start 0c380c742b243820c3ef5b23e4f172498f66e0b147b4cc1b71b391b2377856f0 (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-2-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240222, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8) 2024-03-04T14:02:17.346 INFO:journalctl@ceph.osd.2.smithi107.stdout:Mar 04 14:02:17 smithi107 podman[156098]: 2024-03-04 14:02:17.206581667 +0000 UTC m=+0.619740121 container attach 0c380c742b243820c3ef5b23e4f172498f66e0b147b4cc1b71b391b2377856f0 (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-2-deactivate, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240222, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8) 2024-03-04T14:02:17.846 INFO:journalctl@ceph.osd.2.smithi107.stdout:Mar 04 14:02:17 smithi107 podman[156098]: 2024-03-04 14:02:17.488437338 +0000 UTC m=+0.901595749 container died 0c380c742b243820c3ef5b23e4f172498f66e0b147b4cc1b71b391b2377856f0 (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-2-deactivate, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8) 2024-03-04T14:02:18.310 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1fc09a58-da2b-11ee-95c6-87774f69a715@osd.2.service' 2024-03-04T14:02:18.434 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-04T14:02:18.434 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-03-04T14:02:18.435 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-03-04T14:02:18.435 DEBUG:teuthology.orchestra.run.smithi169:> sudo systemctl stop ceph-1fc09a58-da2b-11ee-95c6-87774f69a715@osd.3 2024-03-04T14:02:18.866 INFO:journalctl@ceph.osd.3.smithi169.stdout:Mar 04 14:02:18 smithi169 systemd[1]: Stopping Ceph osd.3 for 1fc09a58-da2b-11ee-95c6-87774f69a715... 2024-03-04T14:02:18.867 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 14:02:18 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-5[105465]: 2024-03-04T14:02:18.487+0000 7f311ca00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-04T14:01:52.377609+0000 front 2024-03-04T14:01:52.377686+0000 (oldest deadline 2024-03-04T14:02:18.277530+0000) 2024-03-04T14:02:19.363 INFO:journalctl@ceph.osd.3.smithi169.stdout:Mar 04 14:02:19 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-3[95252]: 2024-03-04T14:02:19.006+0000 7f3841800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-04T14:02:19.364 INFO:journalctl@ceph.osd.3.smithi169.stdout:Mar 04 14:02:19 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-3[95252]: 2024-03-04T14:02:19.006+0000 7f3841800700 -1 osd.3 86 *** Got signal Terminated *** 2024-03-04T14:02:19.364 INFO:journalctl@ceph.osd.3.smithi169.stdout:Mar 04 14:02:19 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-3[95252]: 2024-03-04T14:02:19.006+0000 7f3841800700 -1 osd.3 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-04T14:02:19.364 INFO:journalctl@ceph.osd.3.smithi169.stdout:Mar 04 14:02:19 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-3[95252]: 2024-03-04T14:02:19.077+0000 7f3838a00700 -1 osd.3 86 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-04T14:01:55.323914+0000 front 2024-03-04T14:01:55.323843+0000 (oldest deadline 2024-03-04T14:02:18.823526+0000) 2024-03-04T14:02:19.863 INFO:journalctl@ceph.osd.4.smithi169.stdout:Mar 04 14:02:19 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-4[100135]: 2024-03-04T14:02:19.745+0000 7f309be00700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-04T14:01:54.402425+0000 front 2024-03-04T14:01:54.402536+0000 (oldest deadline 2024-03-04T14:02:19.702404+0000) 2024-03-04T14:02:19.863 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 14:02:19 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-5[105465]: 2024-03-04T14:02:19.515+0000 7f311ca00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-04T14:01:52.377609+0000 front 2024-03-04T14:01:52.377686+0000 (oldest deadline 2024-03-04T14:02:18.277530+0000) 2024-03-04T14:02:20.469 INFO:journalctl@ceph.osd.3.smithi169.stdout:Mar 04 14:02:20 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-3[95252]: 2024-03-04T14:02:20.124+0000 7f3838a00700 -1 osd.3 86 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-04T14:01:55.323914+0000 front 2024-03-04T14:01:55.323843+0000 (oldest deadline 2024-03-04T14:02:18.823526+0000) 2024-03-04T14:02:20.752 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 14:02:20 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-5[105465]: 2024-03-04T14:02:20.466+0000 7f311ca00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-04T14:01:52.377609+0000 front 2024-03-04T14:01:52.377686+0000 (oldest deadline 2024-03-04T14:02:18.277530+0000) 2024-03-04T14:02:21.113 INFO:journalctl@ceph.osd.4.smithi169.stdout:Mar 04 14:02:20 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-4[100135]: 2024-03-04T14:02:20.749+0000 7f309be00700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-04T14:01:54.402425+0000 front 2024-03-04T14:01:54.402536+0000 (oldest deadline 2024-03-04T14:02:19.702404+0000) 2024-03-04T14:02:21.431 INFO:journalctl@ceph.osd.3.smithi169.stdout:Mar 04 14:02:21 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-3[95252]: 2024-03-04T14:02:21.169+0000 7f3838a00700 -1 osd.3 86 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-04T14:01:55.323914+0000 front 2024-03-04T14:01:55.323843+0000 (oldest deadline 2024-03-04T14:02:18.823526+0000) 2024-03-04T14:02:21.767 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 14:02:21 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-5[105465]: 2024-03-04T14:02:21.428+0000 7f311ca00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-04T14:01:52.377609+0000 front 2024-03-04T14:01:52.377686+0000 (oldest deadline 2024-03-04T14:02:18.277530+0000) 2024-03-04T14:02:22.113 INFO:journalctl@ceph.osd.4.smithi169.stdout:Mar 04 14:02:21 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-4[100135]: 2024-03-04T14:02:21.764+0000 7f309be00700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-04T14:01:54.402425+0000 front 2024-03-04T14:01:54.402536+0000 (oldest deadline 2024-03-04T14:02:19.702404+0000) 2024-03-04T14:02:22.455 INFO:journalctl@ceph.osd.3.smithi169.stdout:Mar 04 14:02:22 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-3[95252]: 2024-03-04T14:02:22.168+0000 7f3838a00700 -1 osd.3 86 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-04T14:01:55.323914+0000 front 2024-03-04T14:01:55.323843+0000 (oldest deadline 2024-03-04T14:02:18.823526+0000) 2024-03-04T14:02:22.759 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 14:02:22 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-5[105465]: 2024-03-04T14:02:22.451+0000 7f311ca00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-04T14:01:52.377609+0000 front 2024-03-04T14:01:52.377686+0000 (oldest deadline 2024-03-04T14:02:18.277530+0000) 2024-03-04T14:02:23.113 INFO:journalctl@ceph.osd.4.smithi169.stdout:Mar 04 14:02:22 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-4[100135]: 2024-03-04T14:02:22.756+0000 7f309be00700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-04T14:01:54.402425+0000 front 2024-03-04T14:01:54.402536+0000 (oldest deadline 2024-03-04T14:02:19.702404+0000) 2024-03-04T14:02:23.483 INFO:journalctl@ceph.osd.3.smithi169.stdout:Mar 04 14:02:23 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-3[95252]: 2024-03-04T14:02:23.123+0000 7f3838a00700 -1 osd.3 86 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-04T14:01:55.323914+0000 front 2024-03-04T14:01:55.323843+0000 (oldest deadline 2024-03-04T14:02:18.823526+0000) 2024-03-04T14:02:23.785 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 14:02:23 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-5[105465]: 2024-03-04T14:02:23.478+0000 7f311ca00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-04T14:01:52.377609+0000 front 2024-03-04T14:01:52.377686+0000 (oldest deadline 2024-03-04T14:02:18.277530+0000) 2024-03-04T14:02:24.113 INFO:journalctl@ceph.osd.4.smithi169.stdout:Mar 04 14:02:23 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-4[100135]: 2024-03-04T14:02:23.781+0000 7f309be00700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-04T14:01:54.402425+0000 front 2024-03-04T14:01:54.402536+0000 (oldest deadline 2024-03-04T14:02:19.702404+0000) 2024-03-04T14:02:24.789 INFO:journalctl@ceph.osd.3.smithi169.stdout:Mar 04 14:02:24 smithi169 podman[118894]: 2024-03-04 14:02:24.578390815 +0000 UTC m=+5.707288577 container died d31baf764d7ac1ac25131ad812b587e6790aa93e48d4e463bf412a4d705954e4 (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-3, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240222, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-03-04T14:02:24.789 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 14:02:24 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-5[105465]: 2024-03-04T14:02:24.471+0000 7f311ca00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-04T14:01:52.377609+0000 front 2024-03-04T14:01:52.377686+0000 (oldest deadline 2024-03-04T14:02:18.277530+0000) 2024-03-04T14:02:25.113 INFO:journalctl@ceph.osd.4.smithi169.stdout:Mar 04 14:02:24 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-4[100135]: 2024-03-04T14:02:24.783+0000 7f309be00700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-04T14:01:54.402425+0000 front 2024-03-04T14:01:54.402536+0000 (oldest deadline 2024-03-04T14:02:19.702404+0000) 2024-03-04T14:02:25.113 INFO:journalctl@ceph.osd.3.smithi169.stdout:Mar 04 14:02:24 smithi169 podman[118894]: 2024-03-04 14:02:24.991682264 +0000 UTC m=+6.120580073 container cleanup d31baf764d7ac1ac25131ad812b587e6790aa93e48d4e463bf412a4d705954e4 (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-3, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-03-04T14:02:25.114 INFO:journalctl@ceph.osd.3.smithi169.stdout:Mar 04 14:02:24 smithi169 bash[118894]: ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-3 2024-03-04T14:02:25.511 INFO:journalctl@ceph.osd.3.smithi169.stdout:Mar 04 14:02:25 smithi169 podman[118924]: 2024-03-04 14:02:25.276381447 +0000 UTC m=+0.688711779 container remove d31baf764d7ac1ac25131ad812b587e6790aa93e48d4e463bf412a4d705954e4 (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-3, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, CEPH_POINT_RELEASE=, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD) 2024-03-04T14:02:25.818 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 14:02:25 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-5[105465]: 2024-03-04T14:02:25.506+0000 7f311ca00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-04T14:01:52.377609+0000 front 2024-03-04T14:01:52.377686+0000 (oldest deadline 2024-03-04T14:02:18.277530+0000) 2024-03-04T14:02:26.113 INFO:journalctl@ceph.osd.4.smithi169.stdout:Mar 04 14:02:25 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-4[100135]: 2024-03-04T14:02:25.813+0000 7f309be00700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-04T14:01:54.402425+0000 front 2024-03-04T14:01:54.402536+0000 (oldest deadline 2024-03-04T14:02:19.702404+0000) 2024-03-04T14:02:26.489 INFO:journalctl@ceph.osd.3.smithi169.stdout:Mar 04 14:02:26 smithi169 podman[119013]: 2024-03-04 14:02:26.046350107 +0000 UTC m=+0.038188916 image pull d71b839abdf892550525dad59fdf20a6b925b73dbcc7526f0d04da8380b9fabf quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377 2024-03-04T14:02:26.490 INFO:journalctl@ceph.osd.3.smithi169.stdout:Mar 04 14:02:26 smithi169 podman[119013]: 2024-03-04 14:02:26.152397321 +0000 UTC m=+0.144236149 container create 169acd7949ab068a75a76e6c8bbff2815d08291a16ab66a82825a88cebe1f9c4 (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-3-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240222, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, ceph=True, maintainer=Guillaume Abrioux ) 2024-03-04T14:02:26.810 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 14:02:26 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-5[105465]: 2024-03-04T14:02:26.485+0000 7f311ca00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-04T14:01:52.377609+0000 front 2024-03-04T14:01:52.377686+0000 (oldest deadline 2024-03-04T14:02:18.277530+0000) 2024-03-04T14:02:26.810 INFO:journalctl@ceph.osd.3.smithi169.stdout:Mar 04 14:02:26 smithi169 podman[119013]: 2024-03-04 14:02:26.490619809 +0000 UTC m=+0.482458644 container init 169acd7949ab068a75a76e6c8bbff2815d08291a16ab66a82825a88cebe1f9c4 (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-3-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240222, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.schema-version=1.0) 2024-03-04T14:02:26.811 INFO:journalctl@ceph.osd.3.smithi169.stdout:Mar 04 14:02:26 smithi169 podman[119013]: 2024-03-04 14:02:26.502370867 +0000 UTC m=+0.494209665 container start 169acd7949ab068a75a76e6c8bbff2815d08291a16ab66a82825a88cebe1f9c4 (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-3-deactivate, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux ) 2024-03-04T14:02:26.811 INFO:journalctl@ceph.osd.3.smithi169.stdout:Mar 04 14:02:26 smithi169 podman[119013]: 2024-03-04 14:02:26.58173849 +0000 UTC m=+0.573577302 container attach 169acd7949ab068a75a76e6c8bbff2815d08291a16ab66a82825a88cebe1f9c4 (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240222, GIT_CLEAN=True, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-03-04T14:02:27.113 INFO:journalctl@ceph.osd.3.smithi169.stdout:Mar 04 14:02:26 smithi169 podman[119038]: 2024-03-04 14:02:26.991465185 +0000 UTC m=+0.035388228 container died 169acd7949ab068a75a76e6c8bbff2815d08291a16ab66a82825a88cebe1f9c4 (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-3-deactivate, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, GIT_BRANCH=HEAD, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-03-04T14:02:27.113 INFO:journalctl@ceph.osd.4.smithi169.stdout:Mar 04 14:02:26 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-4[100135]: 2024-03-04T14:02:26.805+0000 7f309be00700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-04T14:01:54.402425+0000 front 2024-03-04T14:01:54.402536+0000 (oldest deadline 2024-03-04T14:02:19.702404+0000) 2024-03-04T14:02:27.613 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 14:02:27 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-5[105465]: 2024-03-04T14:02:27.469+0000 7f311ca00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-04T14:01:52.377609+0000 front 2024-03-04T14:01:52.377686+0000 (oldest deadline 2024-03-04T14:02:18.277530+0000) 2024-03-04T14:02:27.723 DEBUG:teuthology.orchestra.run.smithi169:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1fc09a58-da2b-11ee-95c6-87774f69a715@osd.3.service' 2024-03-04T14:02:27.844 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-04T14:02:27.845 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-03-04T14:02:27.845 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-03-04T14:02:27.845 DEBUG:teuthology.orchestra.run.smithi169:> sudo systemctl stop ceph-1fc09a58-da2b-11ee-95c6-87774f69a715@osd.4 2024-03-04T14:02:27.962 INFO:journalctl@ceph.osd.4.smithi169.stdout:Mar 04 14:02:27 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-4[100135]: 2024-03-04T14:02:27.832+0000 7f309be00700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-04T14:01:54.402425+0000 front 2024-03-04T14:01:54.402536+0000 (oldest deadline 2024-03-04T14:02:19.702404+0000) 2024-03-04T14:02:27.962 INFO:journalctl@ceph.osd.4.smithi169.stdout:Mar 04 14:02:27 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-4[100135]: 2024-03-04T14:02:27.832+0000 7f309be00700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-03-04T14:02:01.403869+0000 front 2024-03-04T14:02:01.404335+0000 (oldest deadline 2024-03-04T14:02:27.303805+0000) 2024-03-04T14:02:28.363 INFO:journalctl@ceph.osd.4.smithi169.stdout:Mar 04 14:02:27 smithi169 systemd[1]: Stopping Ceph osd.4 for 1fc09a58-da2b-11ee-95c6-87774f69a715... 2024-03-04T14:02:28.825 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 14:02:28 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-5[105465]: 2024-03-04T14:02:28.456+0000 7f311ca00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-04T14:01:52.377609+0000 front 2024-03-04T14:01:52.377686+0000 (oldest deadline 2024-03-04T14:02:18.277530+0000) 2024-03-04T14:02:28.825 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 14:02:28 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-5[105465]: 2024-03-04T14:02:28.456+0000 7f311ca00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-03-04T14:02:02.979261+0000 front 2024-03-04T14:02:02.979462+0000 (oldest deadline 2024-03-04T14:02:28.278987+0000) 2024-03-04T14:02:28.825 INFO:journalctl@ceph.osd.4.smithi169.stdout:Mar 04 14:02:28 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-4[100135]: 2024-03-04T14:02:28.553+0000 7f309fc00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-04T14:02:28.825 INFO:journalctl@ceph.osd.4.smithi169.stdout:Mar 04 14:02:28 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-4[100135]: 2024-03-04T14:02:28.553+0000 7f309fc00700 -1 osd.4 86 *** Got signal Terminated *** 2024-03-04T14:02:28.825 INFO:journalctl@ceph.osd.4.smithi169.stdout:Mar 04 14:02:28 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-4[100135]: 2024-03-04T14:02:28.553+0000 7f309fc00700 -1 osd.4 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-04T14:02:28.825 INFO:journalctl@ceph.osd.4.smithi169.stdout:Mar 04 14:02:28 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-4[100135]: 2024-03-04T14:02:28.821+0000 7f309be00700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-04T14:01:54.402425+0000 front 2024-03-04T14:01:54.402536+0000 (oldest deadline 2024-03-04T14:02:19.702404+0000) 2024-03-04T14:02:29.780 INFO:journalctl@ceph.osd.4.smithi169.stdout:Mar 04 14:02:28 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-4[100135]: 2024-03-04T14:02:28.821+0000 7f309be00700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-03-04T14:02:01.403869+0000 front 2024-03-04T14:02:01.404335+0000 (oldest deadline 2024-03-04T14:02:27.303805+0000) 2024-03-04T14:02:29.781 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 14:02:29 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-5[105465]: 2024-03-04T14:02:29.431+0000 7f311ca00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-04T14:01:52.377609+0000 front 2024-03-04T14:01:52.377686+0000 (oldest deadline 2024-03-04T14:02:18.277530+0000) 2024-03-04T14:02:29.781 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 14:02:29 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-5[105465]: 2024-03-04T14:02:29.431+0000 7f311ca00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-03-04T14:02:02.979261+0000 front 2024-03-04T14:02:02.979462+0000 (oldest deadline 2024-03-04T14:02:28.278987+0000) 2024-03-04T14:02:30.113 INFO:journalctl@ceph.osd.4.smithi169.stdout:Mar 04 14:02:29 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-4[100135]: 2024-03-04T14:02:29.777+0000 7f309be00700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-04T14:01:54.402425+0000 front 2024-03-04T14:01:54.402536+0000 (oldest deadline 2024-03-04T14:02:19.702404+0000) 2024-03-04T14:02:30.113 INFO:journalctl@ceph.osd.4.smithi169.stdout:Mar 04 14:02:29 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-4[100135]: 2024-03-04T14:02:29.777+0000 7f309be00700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-03-04T14:02:01.403869+0000 front 2024-03-04T14:02:01.404335+0000 (oldest deadline 2024-03-04T14:02:27.303805+0000) 2024-03-04T14:02:30.808 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 14:02:30 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-5[105465]: 2024-03-04T14:02:30.461+0000 7f311ca00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-04T14:01:52.377609+0000 front 2024-03-04T14:01:52.377686+0000 (oldest deadline 2024-03-04T14:02:18.277530+0000) 2024-03-04T14:02:30.808 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 14:02:30 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-5[105465]: 2024-03-04T14:02:30.461+0000 7f311ca00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-03-04T14:02:02.979261+0000 front 2024-03-04T14:02:02.979462+0000 (oldest deadline 2024-03-04T14:02:28.278987+0000) 2024-03-04T14:02:31.113 INFO:journalctl@ceph.osd.4.smithi169.stdout:Mar 04 14:02:30 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-4[100135]: 2024-03-04T14:02:30.804+0000 7f309be00700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-04T14:01:54.402425+0000 front 2024-03-04T14:01:54.402536+0000 (oldest deadline 2024-03-04T14:02:19.702404+0000) 2024-03-04T14:02:31.113 INFO:journalctl@ceph.osd.4.smithi169.stdout:Mar 04 14:02:30 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-4[100135]: 2024-03-04T14:02:30.804+0000 7f309be00700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-03-04T14:02:01.403869+0000 front 2024-03-04T14:02:01.404335+0000 (oldest deadline 2024-03-04T14:02:27.303805+0000) 2024-03-04T14:02:31.853 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 14:02:31 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-5[105465]: 2024-03-04T14:02:31.485+0000 7f311ca00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-04T14:01:52.377609+0000 front 2024-03-04T14:01:52.377686+0000 (oldest deadline 2024-03-04T14:02:18.277530+0000) 2024-03-04T14:02:31.853 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 14:02:31 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-5[105465]: 2024-03-04T14:02:31.485+0000 7f311ca00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-03-04T14:02:02.979261+0000 front 2024-03-04T14:02:02.979462+0000 (oldest deadline 2024-03-04T14:02:28.278987+0000) 2024-03-04T14:02:32.113 INFO:journalctl@ceph.osd.4.smithi169.stdout:Mar 04 14:02:31 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-4[100135]: 2024-03-04T14:02:31.849+0000 7f309be00700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-04T14:01:54.402425+0000 front 2024-03-04T14:01:54.402536+0000 (oldest deadline 2024-03-04T14:02:19.702404+0000) 2024-03-04T14:02:32.114 INFO:journalctl@ceph.osd.4.smithi169.stdout:Mar 04 14:02:31 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-4[100135]: 2024-03-04T14:02:31.849+0000 7f309be00700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-03-04T14:02:01.403869+0000 front 2024-03-04T14:02:01.404335+0000 (oldest deadline 2024-03-04T14:02:27.303805+0000) 2024-03-04T14:02:32.864 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 14:02:32 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-5[105465]: 2024-03-04T14:02:32.523+0000 7f311ca00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-04T14:01:52.377609+0000 front 2024-03-04T14:01:52.377686+0000 (oldest deadline 2024-03-04T14:02:18.277530+0000) 2024-03-04T14:02:32.864 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 14:02:32 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-5[105465]: 2024-03-04T14:02:32.523+0000 7f311ca00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-03-04T14:02:02.979261+0000 front 2024-03-04T14:02:02.979462+0000 (oldest deadline 2024-03-04T14:02:28.278987+0000) 2024-03-04T14:02:33.363 INFO:journalctl@ceph.osd.4.smithi169.stdout:Mar 04 14:02:32 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-4[100135]: 2024-03-04T14:02:32.899+0000 7f309be00700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-04T14:01:54.402425+0000 front 2024-03-04T14:01:54.402536+0000 (oldest deadline 2024-03-04T14:02:19.702404+0000) 2024-03-04T14:02:33.363 INFO:journalctl@ceph.osd.4.smithi169.stdout:Mar 04 14:02:32 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-4[100135]: 2024-03-04T14:02:32.899+0000 7f309be00700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-03-04T14:02:01.403869+0000 front 2024-03-04T14:02:01.404335+0000 (oldest deadline 2024-03-04T14:02:27.303805+0000) 2024-03-04T14:02:33.779 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 14:02:33 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-5[105465]: 2024-03-04T14:02:33.482+0000 7f311ca00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-04T14:01:52.377609+0000 front 2024-03-04T14:01:52.377686+0000 (oldest deadline 2024-03-04T14:02:18.277530+0000) 2024-03-04T14:02:33.779 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 14:02:33 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-5[105465]: 2024-03-04T14:02:33.482+0000 7f311ca00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-03-04T14:02:02.979261+0000 front 2024-03-04T14:02:02.979462+0000 (oldest deadline 2024-03-04T14:02:28.278987+0000) 2024-03-04T14:02:34.439 INFO:journalctl@ceph.osd.4.smithi169.stdout:Mar 04 14:02:34 smithi169 podman[119118]: 2024-03-04 14:02:34.144113339 +0000 UTC m=+5.900469027 container died a70bacd97639a9d3a91412c5d681cae1aef2c396f16cc601f4a3a9d9ede8c4f0 (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-4, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, maintainer=Guillaume Abrioux , GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8) 2024-03-04T14:02:34.835 INFO:journalctl@ceph.osd.4.smithi169.stdout:Mar 04 14:02:34 smithi169 podman[119118]: 2024-03-04 14:02:34.546669113 +0000 UTC m=+6.303024882 container cleanup a70bacd97639a9d3a91412c5d681cae1aef2c396f16cc601f4a3a9d9ede8c4f0 (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-4, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8) 2024-03-04T14:02:34.835 INFO:journalctl@ceph.osd.4.smithi169.stdout:Mar 04 14:02:34 smithi169 bash[119118]: ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-4 2024-03-04T14:02:34.835 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 14:02:34 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-5[105465]: 2024-03-04T14:02:34.434+0000 7f311ca00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-04T14:01:52.377609+0000 front 2024-03-04T14:01:52.377686+0000 (oldest deadline 2024-03-04T14:02:18.277530+0000) 2024-03-04T14:02:34.835 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 14:02:34 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-5[105465]: 2024-03-04T14:02:34.434+0000 7f311ca00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-03-04T14:02:02.979261+0000 front 2024-03-04T14:02:02.979462+0000 (oldest deadline 2024-03-04T14:02:28.278987+0000) 2024-03-04T14:02:35.113 INFO:journalctl@ceph.osd.4.smithi169.stdout:Mar 04 14:02:34 smithi169 podman[119138]: 2024-03-04 14:02:34.831377092 +0000 UTC m=+0.678464580 container remove a70bacd97639a9d3a91412c5d681cae1aef2c396f16cc601f4a3a9d9ede8c4f0 (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-4, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True) 2024-03-04T14:02:35.768 INFO:journalctl@ceph.osd.4.smithi169.stdout:Mar 04 14:02:35 smithi169 podman[119229]: 2024-03-04 14:02:35.513691102 +0000 UTC m=+0.031712086 image pull d71b839abdf892550525dad59fdf20a6b925b73dbcc7526f0d04da8380b9fabf quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377 2024-03-04T14:02:35.768 INFO:journalctl@ceph.osd.4.smithi169.stdout:Mar 04 14:02:35 smithi169 podman[119229]: 2024-03-04 14:02:35.63246532 +0000 UTC m=+0.150486344 container create 0ce7009e0af2e1eb984ce7a0ea21bff6b29f71659169c7756dd9983e1f61e11c (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-4-deactivate, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-03-04T14:02:35.768 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 14:02:35 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-5[105465]: 2024-03-04T14:02:35.406+0000 7f311ca00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-04T14:01:52.377609+0000 front 2024-03-04T14:01:52.377686+0000 (oldest deadline 2024-03-04T14:02:18.277530+0000) 2024-03-04T14:02:35.768 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 14:02:35 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-5[105465]: 2024-03-04T14:02:35.406+0000 7f311ca00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-03-04T14:02:02.979261+0000 front 2024-03-04T14:02:02.979462+0000 (oldest deadline 2024-03-04T14:02:28.278987+0000) 2024-03-04T14:02:36.113 INFO:journalctl@ceph.osd.4.smithi169.stdout:Mar 04 14:02:35 smithi169 podman[119229]: 2024-03-04 14:02:35.861656553 +0000 UTC m=+0.379677626 container init 0ce7009e0af2e1eb984ce7a0ea21bff6b29f71659169c7756dd9983e1f61e11c (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-4-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240222, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8) 2024-03-04T14:02:36.113 INFO:journalctl@ceph.osd.4.smithi169.stdout:Mar 04 14:02:35 smithi169 podman[119229]: 2024-03-04 14:02:35.873649746 +0000 UTC m=+0.391670744 container start 0ce7009e0af2e1eb984ce7a0ea21bff6b29f71659169c7756dd9983e1f61e11c (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240222, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.license=GPLv2) 2024-03-04T14:02:36.113 INFO:journalctl@ceph.osd.4.smithi169.stdout:Mar 04 14:02:35 smithi169 podman[119229]: 2024-03-04 14:02:35.976526133 +0000 UTC m=+0.494547200 container attach 0ce7009e0af2e1eb984ce7a0ea21bff6b29f71659169c7756dd9983e1f61e11c (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-4-deactivate, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.build-date=20240222, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-04T14:02:36.614 INFO:journalctl@ceph.osd.4.smithi169.stdout:Mar 04 14:02:36 smithi169 podman[119229]: 2024-03-04 14:02:36.309185769 +0000 UTC m=+0.827206792 container died 0ce7009e0af2e1eb984ce7a0ea21bff6b29f71659169c7756dd9983e1f61e11c (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-4-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.build-date=20240222, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-04T14:02:36.909 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 14:02:36 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-5[105465]: 2024-03-04T14:02:36.362+0000 7f311ca00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-04T14:01:52.377609+0000 front 2024-03-04T14:01:52.377686+0000 (oldest deadline 2024-03-04T14:02:18.277530+0000) 2024-03-04T14:02:36.910 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 14:02:36 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-5[105465]: 2024-03-04T14:02:36.362+0000 7f311ca00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-03-04T14:02:02.979261+0000 front 2024-03-04T14:02:02.979462+0000 (oldest deadline 2024-03-04T14:02:28.278987+0000) 2024-03-04T14:02:36.910 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 14:02:36 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-5[105465]: 2024-03-04T14:02:36.362+0000 7f311ca00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.107:6822 osd.2 since back 2024-03-04T14:02:12.380254+0000 front 2024-03-04T14:02:12.380317+0000 (oldest deadline 2024-03-04T14:02:35.880099+0000) 2024-03-04T14:02:37.021 DEBUG:teuthology.orchestra.run.smithi169:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1fc09a58-da2b-11ee-95c6-87774f69a715@osd.4.service' 2024-03-04T14:02:37.134 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-04T14:02:37.135 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-03-04T14:02:37.135 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-03-04T14:02:37.135 DEBUG:teuthology.orchestra.run.smithi169:> sudo systemctl stop ceph-1fc09a58-da2b-11ee-95c6-87774f69a715@osd.5 2024-03-04T14:02:37.613 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 14:02:37 smithi169 systemd[1]: Stopping Ceph osd.5 for 1fc09a58-da2b-11ee-95c6-87774f69a715... 2024-03-04T14:02:37.613 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 14:02:37 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-5[105465]: 2024-03-04T14:02:37.371+0000 7f311ca00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-04T14:01:52.377609+0000 front 2024-03-04T14:01:52.377686+0000 (oldest deadline 2024-03-04T14:02:18.277530+0000) 2024-03-04T14:02:37.613 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 14:02:37 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-5[105465]: 2024-03-04T14:02:37.371+0000 7f311ca00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-03-04T14:02:02.979261+0000 front 2024-03-04T14:02:02.979462+0000 (oldest deadline 2024-03-04T14:02:28.278987+0000) 2024-03-04T14:02:37.613 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 14:02:37 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-5[105465]: 2024-03-04T14:02:37.371+0000 7f311ca00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.107:6822 osd.2 since back 2024-03-04T14:02:12.380254+0000 front 2024-03-04T14:02:12.380317+0000 (oldest deadline 2024-03-04T14:02:35.880099+0000) 2024-03-04T14:02:38.113 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 14:02:37 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-5[105465]: 2024-03-04T14:02:37.730+0000 7f3120800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-04T14:02:38.114 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 14:02:37 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-5[105465]: 2024-03-04T14:02:37.730+0000 7f3120800700 -1 osd.5 86 *** Got signal Terminated *** 2024-03-04T14:02:38.114 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 14:02:37 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-5[105465]: 2024-03-04T14:02:37.730+0000 7f3120800700 -1 osd.5 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-04T14:02:38.864 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 14:02:38 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-5[105465]: 2024-03-04T14:02:38.383+0000 7f311ca00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-04T14:01:52.377609+0000 front 2024-03-04T14:01:52.377686+0000 (oldest deadline 2024-03-04T14:02:18.277530+0000) 2024-03-04T14:02:38.864 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 14:02:38 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-5[105465]: 2024-03-04T14:02:38.383+0000 7f311ca00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-03-04T14:02:02.979261+0000 front 2024-03-04T14:02:02.979462+0000 (oldest deadline 2024-03-04T14:02:28.278987+0000) 2024-03-04T14:02:38.864 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 14:02:38 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-5[105465]: 2024-03-04T14:02:38.383+0000 7f311ca00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.107:6822 osd.2 since back 2024-03-04T14:02:12.380254+0000 front 2024-03-04T14:02:12.380317+0000 (oldest deadline 2024-03-04T14:02:35.880099+0000) 2024-03-04T14:02:39.863 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 14:02:39 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-5[105465]: 2024-03-04T14:02:39.382+0000 7f311ca00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-04T14:01:52.377609+0000 front 2024-03-04T14:01:52.377686+0000 (oldest deadline 2024-03-04T14:02:18.277530+0000) 2024-03-04T14:02:39.863 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 14:02:39 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-5[105465]: 2024-03-04T14:02:39.382+0000 7f311ca00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-03-04T14:02:02.979261+0000 front 2024-03-04T14:02:02.979462+0000 (oldest deadline 2024-03-04T14:02:28.278987+0000) 2024-03-04T14:02:39.863 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 14:02:39 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-5[105465]: 2024-03-04T14:02:39.382+0000 7f311ca00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.107:6822 osd.2 since back 2024-03-04T14:02:12.380254+0000 front 2024-03-04T14:02:12.380317+0000 (oldest deadline 2024-03-04T14:02:35.880099+0000) 2024-03-04T14:02:40.863 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 14:02:40 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-5[105465]: 2024-03-04T14:02:40.373+0000 7f311ca00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-04T14:01:52.377609+0000 front 2024-03-04T14:01:52.377686+0000 (oldest deadline 2024-03-04T14:02:18.277530+0000) 2024-03-04T14:02:40.863 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 14:02:40 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-5[105465]: 2024-03-04T14:02:40.373+0000 7f311ca00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-03-04T14:02:02.979261+0000 front 2024-03-04T14:02:02.979462+0000 (oldest deadline 2024-03-04T14:02:28.278987+0000) 2024-03-04T14:02:40.864 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 14:02:40 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-5[105465]: 2024-03-04T14:02:40.373+0000 7f311ca00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.107:6822 osd.2 since back 2024-03-04T14:02:12.380254+0000 front 2024-03-04T14:02:12.380317+0000 (oldest deadline 2024-03-04T14:02:35.880099+0000) 2024-03-04T14:02:41.613 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 14:02:41 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-5[105465]: 2024-03-04T14:02:41.341+0000 7f311ca00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-04T14:01:52.377609+0000 front 2024-03-04T14:01:52.377686+0000 (oldest deadline 2024-03-04T14:02:18.277530+0000) 2024-03-04T14:02:41.614 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 14:02:41 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-5[105465]: 2024-03-04T14:02:41.341+0000 7f311ca00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-03-04T14:02:02.979261+0000 front 2024-03-04T14:02:02.979462+0000 (oldest deadline 2024-03-04T14:02:28.278987+0000) 2024-03-04T14:02:41.614 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 14:02:41 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-5[105465]: 2024-03-04T14:02:41.341+0000 7f311ca00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.107:6822 osd.2 since back 2024-03-04T14:02:12.380254+0000 front 2024-03-04T14:02:12.380317+0000 (oldest deadline 2024-03-04T14:02:35.880099+0000) 2024-03-04T14:02:42.863 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 14:02:42 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-5[105465]: 2024-03-04T14:02:42.362+0000 7f311ca00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-04T14:01:52.377609+0000 front 2024-03-04T14:01:52.377686+0000 (oldest deadline 2024-03-04T14:02:18.277530+0000) 2024-03-04T14:02:42.863 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 14:02:42 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-5[105465]: 2024-03-04T14:02:42.362+0000 7f311ca00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-03-04T14:02:02.979261+0000 front 2024-03-04T14:02:02.979462+0000 (oldest deadline 2024-03-04T14:02:28.278987+0000) 2024-03-04T14:02:42.864 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 14:02:42 smithi169 ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-5[105465]: 2024-03-04T14:02:42.362+0000 7f311ca00700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.107:6822 osd.2 since back 2024-03-04T14:02:12.380254+0000 front 2024-03-04T14:02:12.380317+0000 (oldest deadline 2024-03-04T14:02:35.880099+0000) 2024-03-04T14:02:43.440 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 14:02:43 smithi169 podman[119334]: 2024-03-04 14:02:43.133718006 +0000 UTC m=+5.522619864 container died 5f6e00f7993d0a8cfbc0e3313fdc656e9a80d5bee4cdc02ee9caa16ef569b1cd (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, GIT_CLEAN=True, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-04T14:02:43.828 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 14:02:43 smithi169 podman[119334]: 2024-03-04 14:02:43.548991888 +0000 UTC m=+5.937893759 container cleanup 5f6e00f7993d0a8cfbc0e3313fdc656e9a80d5bee4cdc02ee9caa16ef569b1cd (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222) 2024-03-04T14:02:43.828 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 14:02:43 smithi169 bash[119334]: ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-5 2024-03-04T14:02:44.113 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 14:02:43 smithi169 podman[119353]: 2024-03-04 14:02:43.825035797 +0000 UTC m=+0.671679165 container remove 5f6e00f7993d0a8cfbc0e3313fdc656e9a80d5bee4cdc02ee9caa16ef569b1cd (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-5, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240222, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c) 2024-03-04T14:02:44.916 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 14:02:44 smithi169 podman[119442]: 2024-03-04 14:02:44.530787171 +0000 UTC m=+0.034924528 image pull d71b839abdf892550525dad59fdf20a6b925b73dbcc7526f0d04da8380b9fabf quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377 2024-03-04T14:02:44.916 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 14:02:44 smithi169 podman[119442]: 2024-03-04 14:02:44.642617177 +0000 UTC m=+0.146754552 container create 01d429670af06f542b14b8ee77e1f994b2b9b7eb84238b85105c336b0a0bb742 (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-5-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240222, maintainer=Guillaume Abrioux ) 2024-03-04T14:02:45.360 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 14:02:44 smithi169 podman[119442]: 2024-03-04 14:02:44.913589814 +0000 UTC m=+0.417727256 container init 01d429670af06f542b14b8ee77e1f994b2b9b7eb84238b85105c336b0a0bb742 (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-5-deactivate, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, GIT_CLEAN=True) 2024-03-04T14:02:45.360 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 14:02:44 smithi169 podman[119442]: 2024-03-04 14:02:44.923778482 +0000 UTC m=+0.427915884 container start 01d429670af06f542b14b8ee77e1f994b2b9b7eb84238b85105c336b0a0bb742 (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240222, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8) 2024-03-04T14:02:45.360 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 14:02:45 smithi169 podman[119442]: 2024-03-04 14:02:45.012220529 +0000 UTC m=+0.516357961 container attach 01d429670af06f542b14b8ee77e1f994b2b9b7eb84238b85105c336b0a0bb742 (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-5-deactivate, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=) 2024-03-04T14:02:45.613 INFO:journalctl@ceph.osd.5.smithi169.stdout:Mar 04 14:02:45 smithi169 podman[119468]: 2024-03-04 14:02:45.404770539 +0000 UTC m=+0.033683098 container died 01d429670af06f542b14b8ee77e1f994b2b9b7eb84238b85105c336b0a0bb742 (image=quay.ceph.io/ceph-ci/ceph@sha256:d881e7a4d830e0159c9b14ceabcad19438c4299e6d54a0d2518f6aa597d0b377, name=ceph-1fc09a58-da2b-11ee-95c6-87774f69a715-osd-5-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240222, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240304.042522-717ce3c, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-03-04T14:02:46.101 DEBUG:teuthology.orchestra.run.smithi169:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1fc09a58-da2b-11ee-95c6-87774f69a715@osd.5.service' 2024-03-04T14:02:46.220 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-04T14:02:46.220 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-03-04T14:02:46.220 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 --force --keep-logs 2024-03-04T14:02:51.692 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi107.stderr:ceph-fuse[75882]: fuse finished with error 0 and tester_r 0 2024-03-04T14:03:19.474 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 --force --keep-logs 2024-03-04T14:03:35.669 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-04T14:03:35.768 DEBUG:teuthology.orchestra.run.smithi169:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-04T14:03:35.870 INFO:tasks.cephadm:Archiving crash dumps... 2024-03-04T14:03:35.872 DEBUG:teuthology.misc:Transferring archived files from smithi107:/var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/crash to /home/teuthworker/archive/vshankar-2024-03-04_08:26:39-fs-wip-vshankar-testing-20240304.042522-testing-default-smithi/7581129/remote/smithi107/crash 2024-03-04T14:03:35.873 DEBUG:teuthology.orchestra.run.smithi107:> sudo tar c -f - -C /var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/crash -- . 2024-03-04T14:03:35.954 INFO:teuthology.orchestra.run.smithi107.stderr:tar: /var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/crash: Cannot open: No such file or directory 2024-03-04T14:03:35.955 INFO:teuthology.orchestra.run.smithi107.stderr:tar: Error is not recoverable: exiting now 2024-03-04T14:03:35.958 DEBUG:teuthology.misc:Transferring archived files from smithi169:/var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/crash to /home/teuthworker/archive/vshankar-2024-03-04_08:26:39-fs-wip-vshankar-testing-20240304.042522-testing-default-smithi/7581129/remote/smithi169/crash 2024-03-04T14:03:35.959 DEBUG:teuthology.orchestra.run.smithi169:> sudo tar c -f - -C /var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/crash -- . 2024-03-04T14:03:36.055 INFO:teuthology.orchestra.run.smithi169.stderr:tar: /var/lib/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/crash: Cannot open: No such file or directory 2024-03-04T14:03:36.055 INFO:teuthology.orchestra.run.smithi169.stderr:tar: Error is not recoverable: exiting now 2024-03-04T14:03:36.060 INFO:tasks.cephadm:Checking cluster log for badness... 2024-03-04T14:03:36.060 DEBUG:teuthology.orchestra.run.smithi107:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v OSD_DOWN | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | head -n 1 2024-03-04T14:03:36.157 INFO:teuthology.orchestra.run.smithi107.stdout:2024-03-04T13:50:00.000369+0000 mon.smithi107 (mon.0) 297 : cluster 3 [WRN] PG_DEGRADED: Degraded data redundancy: 35/264 objects degraded (13.258%), 13 pgs degraded 2024-03-04T14:03:36.158 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-03-04T14:03:36.158 INFO:tasks.cephadm:Compressing logs... 2024-03-04T14:03:36.158 DEBUG:teuthology.orchestra.run.smithi107:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-04T14:03:36.162 DEBUG:teuthology.orchestra.run.smithi169:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-04T14:03:36.223 INFO:teuthology.orchestra.run.smithi107.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-03-04T14:03:36.223 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-04T14:03:36.224 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.log 2024-03-04T14:03:36.225 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/ceph-mds.cephfs.smithi107.riexmm.log 2024-03-04T14:03:36.227 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.0.log: gzip -5 --verbose -- /var/log/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/ceph.audit.log 2024-03-04T14:03:36.228 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/ceph-mds.cephfs.smithi107.thycho.log 2024-03-04T14:03:36.229 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/ceph-mds.cephfs.smithi107.riexmm.log: /var/log/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/ceph-osd.1.log 2024-03-04T14:03:36.231 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/ceph-mds.cephfs.smithi107.thycho.log: gzip -5 --verbose -- /var/log/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/ceph-mgr.smithi107.wpzgwl.log 2024-03-04T14:03:36.235 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/ceph-osd.1.log: /var/log/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/ceph-mgr.smithi107.wpzgwl.log: gzip -5 --verbose -- /var/log/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/ceph-osd.0.log 2024-03-04T14:03:36.235 INFO:teuthology.orchestra.run.smithi107.stderr: 91.0% -- replaced with /var/log/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/ceph.audit.log.gz 2024-03-04T14:03:36.236 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/ceph.cephadm.log 2024-03-04T14:03:36.238 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/ceph-mon.smithi107.log 2024-03-04T14:03:36.243 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/ceph-mon.smithi107.log: /var/log/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/ceph.cephadm.log: 85.1% -- replaced with /var/log/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/ceph.cephadm.log.gz 2024-03-04T14:03:36.243 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/ceph.log 2024-03-04T14:03:36.245 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/ceph-osd.0.log: 90.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-04T14:03:36.247 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/ceph.log: gzip -5 --verbose -- /var/log/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/ceph-osd.2.log 2024-03-04T14:03:36.248 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/ceph-volume.log 2024-03-04T14:03:36.251 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/ceph-osd.2.log: 88.1% -- replaced with /var/log/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/ceph.log.gz 2024-03-04T14:03:36.255 INFO:teuthology.orchestra.run.smithi169.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-03-04T14:03:36.256 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-04T14:03:36.258 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.log 2024-03-04T14:03:36.259 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/ceph-mgr.smithi169.eihczr.log 2024-03-04T14:03:36.260 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/ceph-client.1.log: gzip -5 --verbose -- /var/log/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/ceph-mds.cephfs.smithi169.jusudl.log 2024-03-04T14:03:36.261 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/ceph-mgr.smithi169.eihczr.log: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-04T14:03:36.261 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/ceph.audit.log 2024-03-04T14:03:36.263 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/ceph-mds.cephfs.smithi169.jusudl.log: gzip -5 --verbose -- /var/log/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/ceph-mon.smithi169.log 2024-03-04T14:03:36.264 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/ceph-osd.4.log 2024-03-04T14:03:36.266 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/ceph-mon.smithi169.log: gzip -5 --verbose -- /var/log/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/ceph-mds.cephfs.smithi169.bxitzk.log 2024-03-04T14:03:36.268 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/ceph-osd.5.log 2024-03-04T14:03:36.271 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/ceph-mds.cephfs.smithi169.bxitzk.log: /var/log/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/ceph-osd.5.log: 91.2% -- replaced with /var/log/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/ceph.audit.log.gz 2024-03-04T14:03:36.272 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/ceph-osd.3.log 2024-03-04T14:03:36.274 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/ceph.cephadm.log 2024-03-04T14:03:36.276 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/ceph.log 2024-03-04T14:03:36.281 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/ceph.log: /var/log/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/ceph.cephadm.log: 84.9% -- replaced with /var/log/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/ceph.cephadm.log.gz 2024-03-04T14:03:36.282 INFO:teuthology.orchestra.run.smithi169.stderr:gzip -5 --verbose -- /var/log/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/ceph-volume.log 2024-03-04T14:03:36.292 INFO:teuthology.orchestra.run.smithi169.stderr:/var/log/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/ceph-volume.log: 88.2% -- replaced with /var/log/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/ceph.log.gz 2024-03-04T14:03:36.308 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/ceph-volume.log: 90.9% -- replaced with /var/log/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/ceph-volume.log.gz 2024-03-04T14:03:36.330 INFO:teuthology.orchestra.run.smithi169.stderr: 88.9% -- replaced with /var/log/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/ceph-mgr.smithi169.eihczr.log.gz 2024-03-04T14:03:36.333 INFO:teuthology.orchestra.run.smithi169.stderr: 90.9% -- replaced with /var/log/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/ceph-volume.log.gz 2024-03-04T14:03:36.687 INFO:teuthology.orchestra.run.smithi107.stderr: 88.5% -- replaced with /var/log/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/ceph-mgr.smithi107.wpzgwl.log.gz 2024-03-04T14:03:36.839 INFO:teuthology.orchestra.run.smithi169.stderr: 92.5% -- replaced with /var/log/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/ceph-mon.smithi169.log.gz 2024-03-04T14:03:37.853 INFO:teuthology.orchestra.run.smithi107.stderr: 90.4% -- replaced with /var/log/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/ceph-mon.smithi107.log.gz 2024-03-04T14:03:39.824 INFO:teuthology.orchestra.run.smithi169.stderr: 93.5% -- replaced with /var/log/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/ceph-osd.4.log.gz 2024-03-04T14:03:42.244 INFO:teuthology.orchestra.run.smithi169.stderr: 93.6% -- replaced with /var/log/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/ceph-osd.3.log.gz 2024-03-04T14:03:42.271 INFO:teuthology.orchestra.run.smithi107.stderr: 93.6% -- replaced with /var/log/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/ceph-osd.2.log.gz 2024-03-04T14:03:42.740 INFO:teuthology.orchestra.run.smithi107.stderr: 93.5% -- replaced with /var/log/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/ceph-osd.0.log.gz 2024-03-04T14:03:42.926 INFO:teuthology.orchestra.run.smithi169.stderr: 93.7% -- replaced with /var/log/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/ceph-osd.5.log.gz 2024-03-04T14:03:43.164 INFO:teuthology.orchestra.run.smithi107.stderr: 93.6% -- replaced with /var/log/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/ceph-osd.1.log.gz 2024-03-04T14:03:43.256 INFO:teuthology.orchestra.run.smithi107.stderr: 94.8% -- replaced with /var/log/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/ceph-mds.cephfs.smithi107.riexmm.log.gz 2024-03-04T14:03:43.977 INFO:teuthology.orchestra.run.smithi169.stderr: 94.8% -- replaced with /var/log/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/ceph-mds.cephfs.smithi169.bxitzk.log.gz 2024-03-04T14:03:51.765 INFO:teuthology.orchestra.run.smithi107.stderr: 92.9% -- replaced with /var/log/ceph/ceph-client.0.log.gz 2024-03-04T14:03:51.934 INFO:teuthology.orchestra.run.smithi169.stderr:gzip: /var/log/ceph/ceph-client.1.log: file size changed while zipping 2024-03-04T14:03:51.934 INFO:teuthology.orchestra.run.smithi169.stderr: 93.0% -- replaced with /var/log/ceph/ceph-client.1.log.gz 2024-03-04T14:03:52.126 INFO:teuthology.orchestra.run.smithi107.stderr: 94.8% -- replaced with /var/log/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/ceph-mds.cephfs.smithi107.thycho.log.gz 2024-03-04T14:03:52.129 INFO:teuthology.orchestra.run.smithi107.stderr: 2024-03-04T14:03:52.129 INFO:teuthology.orchestra.run.smithi107.stderr:real 0m15.936s 2024-03-04T14:03:52.129 INFO:teuthology.orchestra.run.smithi107.stderr:user 0m55.137s 2024-03-04T14:03:52.129 INFO:teuthology.orchestra.run.smithi107.stderr:sys 0m4.484s 2024-03-04T14:05:07.160 INFO:teuthology.orchestra.run.smithi169.stderr: 92.8% -- replaced with /var/log/ceph/1fc09a58-da2b-11ee-95c6-87774f69a715/ceph-mds.cephfs.smithi169.jusudl.log.gz 2024-03-04T14:05:07.163 INFO:teuthology.orchestra.run.smithi169.stderr: 2024-03-04T14:05:07.163 INFO:teuthology.orchestra.run.smithi169.stderr:real 1m30.947s 2024-03-04T14:05:07.163 INFO:teuthology.orchestra.run.smithi169.stderr:user 2m2.534s 2024-03-04T14:05:07.163 INFO:teuthology.orchestra.run.smithi169.stderr:sys 0m8.601s 2024-03-04T14:05:07.165 INFO:tasks.cephadm:Archiving logs... 2024-03-04T14:05:07.166 DEBUG:teuthology.misc:Transferring archived files from smithi107:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-03-04_08:26:39-fs-wip-vshankar-testing-20240304.042522-testing-default-smithi/7581129/remote/smithi107/log 2024-03-04T14:05:07.167 DEBUG:teuthology.orchestra.run.smithi107:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-04T14:05:14.147 DEBUG:teuthology.misc:Transferring archived files from smithi169:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-03-04_08:26:39-fs-wip-vshankar-testing-20240304.042522-testing-default-smithi/7581129/remote/smithi169/log 2024-03-04T14:05:14.148 DEBUG:teuthology.orchestra.run.smithi169:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-04T14:05:33.163 INFO:tasks.cephadm:Removing cluster... 2024-03-04T14:05:33.164 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 --force 2024-03-04T14:05:34.317 DEBUG:teuthology.orchestra.run.smithi169:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1fc09a58-da2b-11ee-95c6-87774f69a715 --force 2024-03-04T14:05:35.708 INFO:tasks.cephadm:Removing cephadm ... 2024-03-04T14:05:35.708 DEBUG:teuthology.orchestra.run.smithi107:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-04T14:05:35.765 DEBUG:teuthology.orchestra.run.smithi169:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-04T14:05:35.822 INFO:tasks.cephadm:Teardown complete 2024-03-04T14:05:35.822 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-03-04T14:05:35.835 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/task/install/__init__.py", line 626, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_717ce3cce8d0166e4e577faa8004e6f9cb4128c0/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph-c_717ce3cce8d0166e4e577faa8004e6f9cb4128c0/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-03-04T14:05:35.836 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-03-04T14:05:35.836 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-04T14:05:35.843 DEBUG:teuthology.orchestra.run.smithi169:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-04T14:05:35.995 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-03-04T14:05:36.006 INFO:teuthology.task.clock:Checking final clock skew... 2024-03-04T14:05:36.007 DEBUG:teuthology.orchestra.run.smithi107:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-04T14:05:36.014 DEBUG:teuthology.orchestra.run.smithi169:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-04T14:05:36.062 INFO:teuthology.orchestra.run.smithi107.stderr:bash: line 1: ntpq: command not found 2024-03-04T14:05:36.069 INFO:teuthology.orchestra.run.smithi169.stderr:bash: line 1: ntpq: command not found 2024-03-04T14:05:36.071 INFO:teuthology.orchestra.run.smithi107.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-04T14:05:36.071 INFO:teuthology.orchestra.run.smithi107.stdout:=============================================================================== 2024-03-04T14:05:36.071 INFO:teuthology.orchestra.run.smithi107.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 9 +2257us[+2257us] +/- 116ms 2024-03-04T14:05:36.071 INFO:teuthology.orchestra.run.smithi107.stdout:^* hv02.front.sepia.ceph.com 3 7 377 72 -3196us[-3114us] +/- 80ms 2024-03-04T14:05:36.071 INFO:teuthology.orchestra.run.smithi107.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 13 +1795us[+1795us] +/- 75ms 2024-03-04T14:05:36.071 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-04T14:05:36.078 INFO:teuthology.orchestra.run.smithi169.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-04T14:05:36.078 INFO:teuthology.orchestra.run.smithi169.stdout:=============================================================================== 2024-03-04T14:05:36.078 INFO:teuthology.orchestra.run.smithi169.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 11 +2232us[+2207us] +/- 116ms 2024-03-04T14:05:36.078 INFO:teuthology.orchestra.run.smithi169.stdout:^* hv02.front.sepia.ceph.com 3 6 377 9 -3345us[-3369us] +/- 81ms 2024-03-04T14:05:36.078 INFO:teuthology.orchestra.run.smithi169.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 16 +1669us[+1646us] +/- 75ms 2024-03-04T14:05:36.078 INFO:teuthology.orchestra.run.smithi169.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-04T14:05:36.079 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-03-04T14:05:36.090 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-03-04T14:05:36.091 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-03-04T14:05:36.102 DEBUG:teuthology.orchestra.run.smithi107:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-04T14:05:36.339 DEBUG:teuthology.orchestra.run.smithi169:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-04T14:05:36.574 DEBUG:teuthology.orchestra.run.smithi107:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-04T14:05:36.682 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-04T14:05:36.683 DEBUG:teuthology.orchestra.run.smithi169:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-04T14:05:36.784 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-04T14:05:36.785 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-03-04T14:05:36.797 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-03-04T14:05:36.831 INFO:teuthology.task.internal:Duration was 2901.644748 seconds 2024-03-04T14:05:36.831 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-03-04T14:05:36.842 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-03-04T14:05:36.842 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-04T14:05:36.850 DEBUG:teuthology.orchestra.run.smithi169:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-04T14:05:36.998 INFO:teuthology.orchestra.run.smithi169.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-04T14:05:37.011 INFO:teuthology.orchestra.run.smithi107.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-04T14:05:37.434 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-03-04T14:05:37.434 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi107.front.sepia.ceph.com 2024-03-04T14:05:37.434 DEBUG:teuthology.orchestra.run.smithi107:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-04T14:05:37.496 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi169.front.sepia.ceph.com 2024-03-04T14:05:37.496 DEBUG:teuthology.orchestra.run.smithi169:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-04T14:05:37.572 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-03-04T14:05:37.572 DEBUG:teuthology.orchestra.run.smithi107:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-04T14:05:37.580 DEBUG:teuthology.orchestra.run.smithi169:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-04T14:05:37.670 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-03-04T14:05:37.670 DEBUG:teuthology.orchestra.run.smithi107:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-04T14:05:37.675 DEBUG:teuthology.orchestra.run.smithi169:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-04T14:05:38.454 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-03-04T14:05:38.466 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-03-04T14:05:38.466 DEBUG:teuthology.orchestra.run.smithi107:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-04T14:05:38.555 DEBUG:teuthology.orchestra.run.smithi169:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-04T14:05:38.656 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-03-04T14:05:38.669 DEBUG:teuthology.orchestra.run.smithi107:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-04T14:05:38.677 DEBUG:teuthology.orchestra.run.smithi169:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-04T14:05:38.779 INFO:teuthology.orchestra.run.smithi107.stdout:kernel.core_pattern = core 2024-03-04T14:05:38.787 INFO:teuthology.orchestra.run.smithi169.stdout:kernel.core_pattern = core 2024-03-04T14:05:38.857 DEBUG:teuthology.orchestra.run.smithi107:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-04T14:05:38.911 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-04T14:05:38.911 DEBUG:teuthology.orchestra.run.smithi169:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-04T14:05:38.966 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-04T14:05:38.966 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-03-04T14:05:38.978 INFO:teuthology.task.internal:Transferring archived files... 2024-03-04T14:05:38.979 DEBUG:teuthology.misc:Transferring archived files from smithi107:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-03-04_08:26:39-fs-wip-vshankar-testing-20240304.042522-testing-default-smithi/7581129/remote/smithi107 2024-03-04T14:05:38.979 DEBUG:teuthology.orchestra.run.smithi107:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-04T14:05:39.099 DEBUG:teuthology.misc:Transferring archived files from smithi169:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-03-04_08:26:39-fs-wip-vshankar-testing-20240304.042522-testing-default-smithi/7581129/remote/smithi169 2024-03-04T14:05:39.100 DEBUG:teuthology.orchestra.run.smithi169:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-04T14:05:39.212 INFO:teuthology.task.internal:Removing archive directory... 2024-03-04T14:05:39.213 DEBUG:teuthology.orchestra.run.smithi107:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-04T14:05:39.220 DEBUG:teuthology.orchestra.run.smithi169:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-04T14:05:39.306 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-03-04T14:05:39.318 INFO:teuthology.task.internal:Not uploading archives. 2024-03-04T14:05:39.318 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-03-04T14:05:39.329 INFO:teuthology.task.internal:Tidying up after the test... 2024-03-04T14:05:39.330 DEBUG:teuthology.orchestra.run.smithi107:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-04T14:05:39.337 DEBUG:teuthology.orchestra.run.smithi169:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-04T14:05:39.387 INFO:teuthology.orchestra.run.smithi107.stdout: 395173 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Mar 4 14:05 /home/ubuntu/cephtest 2024-03-04T14:05:39.387 INFO:teuthology.orchestra.run.smithi107.stdout: 787470 4 d--------- 2 ubuntu ubuntu 4096 Mar 4 13:39 /home/ubuntu/cephtest/mnt.0 2024-03-04T14:05:39.388 INFO:teuthology.orchestra.run.smithi107.stderr:find: ‘/home/ubuntu/cephtest/mnt.0’: Permission denied 2024-03-04T14:05:39.390 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-04T14:05:39.390 INFO:teuthology.orchestra.run.smithi107.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2024-03-04T14:05:39.391 ERROR:teuthology.run_tasks:Manager failed: internal.base Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/task/internal/__init__.py", line 48, in base yield File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_717ce3cce8d0166e4e577faa8004e6f9cb4128c0/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph-c_717ce3cce8d0166e4e577faa8004e6f9cb4128c0/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/task/internal/__init__.py", line 53, in base run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi107 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2024-03-04T14:05:39.391 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-03-04T14:05:39.404 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-03-04T14:05:39.417 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-03-04T14:05:39.417 INFO:teuthology.nuke:Checking targets against current locks 2024-03-04T14:05:39.444 DEBUG:teuthology.nuke:shortname: smithi107 2024-03-04T14:05:39.445 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-04T14:05:39.471 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi107.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-03-04_08:26:39-fs-wip-vshankar-testing-20240304.042522-testing-default-smithi/7581129', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-04 13:05:02.532842', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLhjGpMlJpo1+6FPuaLmsZgZjsL41nHgbU1AOw2k0B5VtQJsWVD+BhhlM4IU1X4sUzrXnFX7jEjskdK7KCnCAVQ='} 2024-03-04T14:05:39.472 DEBUG:teuthology.nuke:shortname: smithi169 2024-03-04T14:05:39.472 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-04T14:05:39.493 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi169.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-03-04_08:26:39-fs-wip-vshankar-testing-20240304.042522-testing-default-smithi/7581129', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-04 13:05:02.531091', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBI/j2+/BeHyPDZdYKrYfQgkYQy9OghXD5ru6c5b5rpDDegrS8tuTyP3aqfG96F6vVqlUYb2frSFtDHdntTYU6rw='} 2024-03-04T14:05:39.524 INFO:teuthology.orchestra.console.smithi107:Power off 2024-03-04T14:05:39.524 DEBUG:teuthology.orchestra.console.smithi107:pexpect command: ipmitool -H smithi107.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-04T14:05:39.540 INFO:teuthology.orchestra.console.smithi169:Power off 2024-03-04T14:05:39.540 DEBUG:teuthology.orchestra.console.smithi169:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-04T14:05:39.549 DEBUG:teuthology.orchestra.console.smithi107:power off output: Chassis Power Control: Down/Off 2024-03-04T14:05:39.550 DEBUG:teuthology.orchestra.console.smithi107:pexpect command: ipmitool -H smithi107.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-04T14:05:39.562 DEBUG:teuthology.orchestra.console.smithi169:power off output: Chassis Power Control: Down/Off 2024-03-04T14:05:39.563 DEBUG:teuthology.orchestra.console.smithi169:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-04T14:05:39.573 DEBUG:teuthology.orchestra.console.smithi107:check power output: Chassis Power is on 2024-03-04T14:05:39.586 DEBUG:teuthology.orchestra.console.smithi169:check power output: Chassis Power is on 2024-03-04T14:05:43.574 DEBUG:teuthology.orchestra.console.smithi107:pexpect command: ipmitool -H smithi107.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-04T14:05:43.587 DEBUG:teuthology.orchestra.console.smithi169:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-04T14:05:43.685 DEBUG:teuthology.orchestra.console.smithi107:check power output: Chassis Power is on 2024-03-04T14:05:43.695 DEBUG:teuthology.orchestra.console.smithi169:check power output: Chassis Power is on 2024-03-04T14:05:47.690 DEBUG:teuthology.orchestra.console.smithi107:pexpect command: ipmitool -H smithi107.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-04T14:05:47.698 DEBUG:teuthology.orchestra.console.smithi169:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-04T14:05:47.799 DEBUG:teuthology.orchestra.console.smithi107:check power output: Chassis Power is on 2024-03-04T14:05:47.807 DEBUG:teuthology.orchestra.console.smithi169:check power output: Chassis Power is off 2024-03-04T14:05:47.908 INFO:teuthology.orchestra.console.smithi169:Power off completed 2024-03-04T14:05:51.805 DEBUG:teuthology.orchestra.console.smithi107:pexpect command: ipmitool -H smithi107.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-04T14:05:51.914 DEBUG:teuthology.orchestra.console.smithi107:check power output: Chassis Power is off 2024-03-04T14:05:52.014 INFO:teuthology.orchestra.console.smithi107:Power off completed 2024-03-04T14:05:52.120 INFO:teuthology.run:Summary data: description: fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} duration: 2901.6447484493256 failure_reason: reached maximum tries (51) after waiting for 300 seconds flavor: default host.a-kernel-sha1: 786939134d7eee4dc7a805245d3f595929af37d7 host.b-kernel-sha1: 786939134d7eee4dc7a805245d3f595929af37d7 owner: scheduled_vshankar@teuthology status: fail success: false 2024-03-04T14:05:52.120 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-04T14:05:52.202 INFO:teuthology.run:FAIL